1%|█▋ | 100/11670 [02:02<1:59:38, 1.61it/s] 2%|███▎ | 199/11670 [04:03<1:46:43, 1.79it/s] 3%|████▉ | 298/11670 [06:03<1:52:10, 1.69it/s] 3%|██████▋ | 400/11670 [08:06<1:53:58, 1.65it/s] 4%|████████▎ | 499/11670 [10:07<1:48:23, 1.72it/s] 4%|████████▎ | 500/11670 [10:08<1:59:55, 1.55it/s]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. ***** Running Evaluation ***** Num examples = 8301 Batch size = 32 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 260/260 [04:29<00:00, 1.58it/s] Configuration saved in ./checkpoint-500/config.json Model weights saved in ./checkpoint-500/pytorch_model.bin Configuration saved in ./checkpoint-500/preprocessor_config.json Configuration saved in ./preprocessor_config.json 5%|█████████▉ | 596/11670 [19:06<1:58:57, 1.55it/s] {'loss': 3.0583, 'learning_rate': 7.960000000000001e-05, 'epoch': 0.77} 6%|███████████▋ | 700/11670 [21:11<1:52:58, 1.62it/s] 7%|█████████████▎ | 799/11670 [23:21<3:56:14, 1.30s/it] 8%|██████████████▉ | 899/11670 [25:22<3:54:56, 1.31s/it] 9%|████████████████▌ | 1000/11670 [27:25<3:59:27, 1.35s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. ***** Running Evaluation ***** Num examples = 8301 Batch size = 32 {'loss': 1.7975, 'learning_rate': 0.00013293333333333333, 'epoch': 1.29} 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 260/260 [04:32<00:00, 1.60it/s] {'eval_loss': 0.7887413501739502, 'eval_wer': 0.5651080072872386, 'eval_runtime': 341.1915, 'eval_samples_per_second': 24.329, 'eval_steps_per_second': 0.762, 'epoch': 1.29} 9%|████████████████▌ | 1000/11670 [33:06<3:59:27, 1.35s/it]Saving model checkpoint to ./checkpoint-1000 Configuration saved in ./checkpoint-1000/config.json Model weights saved in ./checkpoint-1000/pytorch_model.bin Configuration saved in ./checkpoint-1000/preprocessor_config.json Configuration saved in ./preprocessor_config.json 9%|██████████████████▏ | 1100/11670 [36:33<3:58:49, 1.36s/it] 10%|███████████████████▊ | 1199/11670 [38:32<3:43:45, 1.28s/it] 11%|█████████████████████▍ | 1299/11670 [40:34<3:47:19, 1.32s/it] 12%|███████████████████████▏ | 1401/11670 [42:36<3:41:59, 1.30s/it] 13%|████████████████████████▊ | 1499/11670 [44:34<3:43:56, 1.32s/it] 13%|████████████████████████▊ | 1500/11670 [44:35<3:51:06, 1.36s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. ***** Running Evaluation ***** Num examples = 8301 Batch size = 32 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 260/260 [05:40<00:00, 1.54it/s] Configuration saved in ./checkpoint-1500/config.json Model weights saved in ./checkpoint-1500/pytorch_model.bin Configuration saved in ./checkpoint-1500/preprocessor_config.json Configuration saved in ./preprocessor_config.json 14%|██████████████████████████▍ | 1600/11670 [53:38<2:05:55, 1.33it/s] 15%|████████████████████████████ | 1699/11670 [55:39<2:00:54, 1.37it/s] 15%|█████████████████████████████▊ | 1800/11670 [57:41<2:06:33, 1.30it/s] 16%|███████████████████████████████▍ | 1902/11670 [59:44<1:55:25, 1.41it/s] 17%|████████████████████████████████▋ | 2000/11670 [1:01:43<1:59:04, 1.35it/s]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. ***** Running Evaluation ***** Num examples = 8301 Batch size = 32 {'loss': 1.344, 'learning_rate': 0.00019022615535889875, 'epoch': 2.57} 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 260/260 [04:33<00:00, 1.57it/s] Configuration saved in ./checkpoint-2000/config.json Model weights saved in ./checkpoint-2000/pytorch_model.bin Configuration saved in ./checkpoint-2000/preprocessor_config.json Configuration saved in ./preprocessor_config.json 02/01/2022 23:44:54 - WARNING - huggingface_hub.repository - Adding files tracked by Git LFS: ['wandb/run-20220201_223624-2b1hcyq3/run-2b1hcyq3.wandb']. This may take a bit of time if the files are large. Adding files tracked by Git LFS: ['wandb/run-20220201_223624-2b1hcyq3/run-2b1hcyq3.wandb']. This may take a bit of time if the files are large. Deleting older checkpoint [checkpoint-500] due to args.save_total_limit 18%|██████████████████████████████████▎ | 2099/11670 [1:10:47<1:50:20, 1.45it/s] 19%|███████████████████████████████████▉ | 2199/11670 [1:12:48<1:50:51, 1.42it/s] 20%|█████████████████████████████████████▋ | 2299/11670 [1:14:48<1:51:39, 1.40it/s] 21%|███████████████████████████████████████▎ | 2399/11670 [1:16:53<3:57:31, 1.54s/it] 21%|████████████████████████████████████████▉ | 2500/11670 [1:18:56<3:52:30, 1.52s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. ***** Running Evaluation ***** Num examples = 8301 Batch size = 32 0%| | 0/260 [00:00 main 13%|█████████████████████████▎ | 33/260 [00:40<04:42, 1.24s/it] 13%|█████████████████████████▎ | 33/260 [00:40<04:42, 1.24s/it] 13%|█████████████████████████▎ | 33/260 [00:40<04:42, 1.24s/it] 02/02/2022 05:11:09 - WARNING - huggingface_hub.repository - To https://huggingface.co/AlexN/xls-r-300m-pt 90fe400..1f92738 main -> main 13%|█████████████████████████▎ | 33/260 [00:40<04:42, 1.24s/it] 13%|█████████████████████████▎ | 33/260 [00:40<04:42, 1.24s/it] ***** train metrics ***** epoch = 15.0 train_loss = 1.3308 train_runtime = 6:34:00.52 train_samples = 24877 train_samples_per_second = 15.785 train_steps_per_second = 0.494 0%| | 0/260 [00:00 main 0%| | 0/260 [00:00 main 0%| | 0/260 [00:00