0%|▋ | 100/21520 [04:25<9:53:42, 1.66s/it] 1%|█▍ | 199/21520 [08:51<10:08:31, 1.71s/it] 1%|██▏ | 300/21520 [13:20<9:42:10, 1.65s/it] 2%|██▉ | 399/21520 [17:44<9:49:47, 1.68s/it] 2%|███▋ | 500/21520 [22:14<9:36:18, 1.65s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. ***** Running Evaluation ***** Num examples = 1839 Batch size = 64 {'loss': 5.4101, 'learning_rate': 2.3239999999999998e-05, 'epoch': 0.12} Configuration saved in ./checkpoint-500/config.json {'eval_loss': 5.053680896759033, 'eval_wer': 1.0, 'eval_runtime': 62.0508, 'eval_samples_per_second': 29.637, 'eval_steps_per_second': 0.467, 'epoch': 0.12} Model weights saved in ./checkpoint-500/pytorch_model.bin Configuration saved in ./checkpoint-500/preprocessor_config.json Configuration saved in ./preprocessor_config.json 3%|████▍ | 600/21520 [28:28<9:27:51, 1.63s/it] 3%|█████▏ | 700/21520 [32:54<9:54:50, 1.71s/it] 4%|█████▉ | 799/21520 [37:16<9:45:40, 1.70s/it] 4%|██████▋ | 899/21520 [41:40<9:40:54, 1.69s/it] 5%|███████▍ | 999/21520 [46:03<9:58:48, 1.75s/it] 5%|███████▎ | 1000/21520 [46:05<9:34:03, 1.68s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. ***** Running Evaluation ***** Num examples = 1839 Batch size = 64 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 29/29 [00:55<00:00, 1.82s/it] Configuration saved in ./checkpoint-1000/config.json Model weights saved in ./checkpoint-1000/pytorch_model.bin Configuration saved in ./checkpoint-1000/preprocessor_config.json Configuration saved in ./preprocessor_config.json 5%|████████ | 1099/21520 [53:19<9:33:07, 1.68s/it] 6%|████████▊ | 1199/21520 [57:45<9:59:52, 1.77s/it] 6%|█████████▍ | 1299/21520 [1:02:09<9:15:12, 1.65s/it] 7%|██████████▏ | 1399/21520 [1:06:35<9:28:45, 1.70s/it] 7%|██████████▊ | 1499/21520 [1:10:59<9:13:59, 1.66s/it] 7%|██████████▊ | 1500/21520 [1:11:00<8:58:38, 1.61s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. ***** Running Evaluation ***** Num examples = 1839 Batch size = 64 Configuration saved in ./checkpoint-1500/config.json {'eval_loss': 1.2048969268798828, 'eval_wer': 0.8062855432505238, 'eval_runtime': 64.3059, 'eval_samples_per_second': 28.598, 'eval_steps_per_second': 0.451, 'epoch': 0.35} Model weights saved in ./checkpoint-1500/pytorch_model.bin Configuration saved in ./checkpoint-1500/preprocessor_config.json Configuration saved in ./preprocessor_config.json Adding files tracked by Git LFS: ['wandb/run-20220129_131141-h6nhqm30/run-h6nhqm30.wandb']. This may take a bit of time if the files are large. 01/29/2022 14:25:13 - WARNING - huggingface_hub.repository - Adding files tracked by Git LFS: ['wandb/run-20220129_131141-h6nhqm30/run-h6nhqm30.wandb']. This may take a bit of time if the files are large. 7%|███████████▌ | 1599/21520 [1:18:18<9:12:21, 1.66s/it] 8%|████████████▎ | 1700/21520 [1:22:44<8:47:54, 1.60s/it] 8%|█████████████ | 1799/21520 [1:27:06<9:14:58, 1.69s/it] 9%|█████████████▊ | 1900/21520 [1:31:34<8:50:40, 1.62s/it] 9%|██████████████▍ | 2000/21520 [1:35:55<8:46:26, 1.62s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. ***** Running Evaluation ***** Num examples = 1839 Batch size = 64 {'loss': 1.5498, 'learning_rate': 6.826923076923076e-05, 'epoch': 0.46} 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 29/29 [00:55<00:00, 1.81s/it] Configuration saved in ./checkpoint-2000/config.json Model weights saved in ./checkpoint-2000/pytorch_model.bin Configuration saved in ./checkpoint-2000/preprocessor_config.json Configuration saved in ./preprocessor_config.json Deleting older checkpoint [checkpoint-500] due to args.save_total_limit 10%|███████████████▏ | 2100/21520 [1:43:15<8:45:13, 1.62s/it] 10%|███████████████▉ | 2200/21520 [1:47:39<8:48:39, 1.64s/it] 11%|████████████████▋ | 2299/21520 [1:52:02<8:59:39, 1.68s/it] 11%|█████████████████▍ | 2400/21520 [1:56:27<8:53:22, 1.67s/it] 12%|██████████████████ | 2499/21520 [2:00:52<9:23:31, 1.78s/it] 12%|██████████████████ | 2500/21520 [2:00:53<8:59:59, 1.70s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. ***** Running Evaluation ***** Num examples = 1839 Batch size = 64 Configuration saved in ./checkpoint-2500/config.json {'eval_loss': 0.4956221282482147, 'eval_wer': 0.5256510026938043, 'eval_runtime': 65.2452, 'eval_samples_per_second': 28.186, 'eval_steps_per_second': 0.444, 'epoch': 0.58} Model weights saved in ./checkpoint-2500/pytorch_model.bin Configuration saved in ./checkpoint-2500/preprocessor_config.json Configuration saved in ./preprocessor_config.json Deleting older checkpoint [checkpoint-1000] due to args.save_total_limit 12%|██████████████████▊ | 2600/21520 [2:08:12<8:38:33, 1.64s/it] 13%|███████████████████▌ | 2699/21520 [2:12:34<9:20:50, 1.79s/it] 13%|████████████████████▎ | 2799/21520 [2:16:58<8:44:37, 1.68s/it] 13%|█████████████████████ | 2900/21520 [2:21:21<8:26:00, 1.63s/it] 14%|█████████████████████▋ | 2998/21520 [2:25:39<9:03:14, 1.76s/it] 14%|█████████████████████▋ | 3000/21520 [2:25:42<8:12:20, 1.60s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. ***** Running Evaluation ***** Num examples = 1839 Batch size = 64 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 29/29 [00:55<00:00, 1.82s/it] Configuration saved in ./checkpoint-3000/config.json Model weights saved in ./checkpoint-3000/pytorch_model.bin Configuration saved in ./checkpoint-3000/preprocessor_config.json Configuration saved in ./preprocessor_config.json Deleting older checkpoint [checkpoint-1500] due to args.save_total_limit 14%|██████████████████████▍ | 3100/21520 [2:33:03<8:22:01, 1.64s/it] 15%|███████████████████████▏ | 3199/21520 [2:37:24<8:25:51, 1.66s/it] 15%|███████████████████████▉ | 3300/21520 [2:41:52<8:21:22, 1.65s/it] 16%|████████████████████████▋ | 3398/21520 [2:46:11<8:41:39, 1.73s/it] 16%|█████████████████████████▎ | 3499/21520 [2:50:38<8:32:27, 1.71s/it] 16%|█████████████████████████▎ | 3500/21520 [2:50:39<8:11:10, 1.64s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. ***** Running Evaluation ***** Num examples = 1839 Batch size = 64 Configuration saved in ./checkpoint-3500/config.json {'eval_loss': 0.38118353486061096, 'eval_wer': 0.4635139179886262, 'eval_runtime': 65.5254, 'eval_samples_per_second': 28.065, 'eval_steps_per_second': 0.443, 'epoch': 0.81} Model weights saved in ./checkpoint-3500/pytorch_model.bin Configuration saved in ./checkpoint-3500/preprocessor_config.json Configuration saved in ./preprocessor_config.json Deleting older checkpoint [checkpoint-2000] due to args.save_total_limit 17%|██████████████████████████ | 3600/21520 [2:58:00<8:11:02, 1.64s/it] 17%|██████████████████████████▊ | 3699/21520 [3:02:22<8:11:14, 1.65s/it] 18%|███████████████████████████▌ | 3800/21520 [3:06:46<7:57:07, 1.62s/it] 18%|████████████████████████████▎ | 3898/21520 [3:11:08<8:29:00, 1.73s/it] 19%|████████████████████████████▉ | 4000/21520 [3:15:36<7:50:45, 1.61s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. ***** Running Evaluation ***** Num examples = 1839 Batch size = 64 {'loss': 1.2001, 'learning_rate': 6.128321678321677e-05, 'epoch': 0.93} 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 29/29 [00:55<00:00, 1.79s/it] Configuration saved in ./checkpoint-4000/config.json Model weights saved in ./checkpoint-4000/pytorch_model.bin Configuration saved in ./checkpoint-4000/preprocessor_config.json Configuration saved in ./preprocessor_config.json Deleting older checkpoint [checkpoint-2500] due to args.save_total_limit 19%|█████████████████████████████▋ | 4100/21520 [3:22:57<7:45:28, 1.60s/it] 20%|██████████████████████████████▍ | 4199/21520 [3:27:18<7:58:53, 1.66s/it] 20%|███████████████████████████████▏ | 4299/21520 [3:31:42<8:00:50, 1.68s/it] 20%|███████████████████████████████▉ | 4400/21520 [3:36:10<9:34:20, 2.01s/it] 21%|████████████████████████████████▌ | 4500/21520 [3:40:32<9:18:48, 1.97s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. ***** Running Evaluation ***** Num examples = 1839 Batch size = 64 {'loss': 1.1671, 'learning_rate': 5.953496503496503e-05, 'epoch': 1.05} 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 29/29 [00:56<00:00, 1.81s/it] Configuration saved in ./checkpoint-4500/config.json Model weights saved in ./checkpoint-4500/pytorch_model.bin Configuration saved in ./checkpoint-4500/preprocessor_config.json Configuration saved in ./preprocessor_config.json Deleting older checkpoint [checkpoint-3000] due to args.save_total_limit 21%|█████████████████████████████████▎ | 4599/21520 [3:47:51<9:07:25, 1.94s/it] 22%|██████████████████████████████████ | 4700/21520 [3:52:17<9:25:30, 2.02s/it] 22%|██████████████████████████████████▊ | 4800/21520 [3:56:42<9:16:50, 2.00s/it] 23%|███████████████████████████████████▌ | 4900/21520 [4:01:06<9:15:05, 2.00s/it] 23%|████████████████████████████████████▏ | 5000/21520 [4:05:30<8:56:08, 1.95s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. ***** Running Evaluation ***** Num examples = 1839 Batch size = 64 {'loss': 1.1599, 'learning_rate': 5.779020979020979e-05, 'epoch': 1.16} Configuration saved in ./checkpoint-5000/config.json {'eval_loss': 0.3355213701725006, 'eval_wer': 0.4330439988027537, 'eval_runtime': 63.7849, 'eval_samples_per_second': 28.831, 'eval_steps_per_second': 0.455, 'epoch': 1.16} Model weights saved in ./checkpoint-5000/pytorch_model.bin Configuration saved in ./checkpoint-5000/preprocessor_config.json Configuration saved in ./preprocessor_config.json Deleting older checkpoint [checkpoint-3500] due to args.save_total_limit 24%|████████████████████████████████████▉ | 5100/21520 [4:12:48<8:56:31, 1.96s/it] 24%|█████████████████████████████████████▋ | 5200/21520 [4:17:12<9:28:46, 2.09s/it] 25%|██████████████████████████████████████▍ | 5299/21520 [4:21:32<8:51:36, 1.97s/it] 25%|███████████████████████████████████████▏ | 5399/21520 [4:25:55<9:07:16, 2.04s/it] 26%|███████████████████████████████████████▊ | 5500/21520 [4:30:20<8:58:27, 2.02s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. ***** Running Evaluation ***** Num examples = 1839 Batch size = 64 {'loss': 1.1568, 'learning_rate': 5.6041958041958036e-05, 'epoch': 1.28} Configuration saved in ./checkpoint-5500/config.json {'eval_loss': 0.327897846698761, 'eval_wer': 0.42490272373540855, 'eval_runtime': 64.3828, 'eval_samples_per_second': 28.564, 'eval_steps_per_second': 0.45, 'epoch': 1.28} Model weights saved in ./checkpoint-5500/pytorch_model.bin Configuration saved in ./checkpoint-5500/preprocessor_config.json Configuration saved in ./preprocessor_config.json Deleting older checkpoint [checkpoint-4000] due to args.save_total_limit 26%|████████████████████████████████████████▌ | 5599/21520 [4:37:38<8:44:34, 1.98s/it] 26%|█████████████████████████████████████████▎ | 5699/21520 [4:42:01<8:33:59, 1.95s/it] 27%|██████████████████████████████████████████ | 5799/21520 [4:46:25<8:30:31, 1.95s/it] 27%|██████████████████████████████████████████▊ | 5899/21520 [4:50:48<8:50:56, 2.04s/it] 28%|███████████████████████████████████████████▍ | 6000/21520 [4:55:17<8:35:28, 1.99s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. ***** Running Evaluation ***** Num examples = 1839 Batch size = 64 {'loss': 1.0994, 'learning_rate': 5.4297202797202796e-05, 'epoch': 1.39} Configuration saved in ./checkpoint-6000/config.json {'eval_loss': 0.318962037563324, 'eval_wer': 0.4240646513020054, 'eval_runtime': 65.8584, 'eval_samples_per_second': 27.924, 'eval_steps_per_second': 0.44, 'epoch': 1.39} Model weights saved in ./checkpoint-6000/pytorch_model.bin Configuration saved in ./checkpoint-6000/preprocessor_config.json Configuration saved in ./preprocessor_config.json Deleting older checkpoint [checkpoint-4500] due to args.save_total_limit 28%|████████████████████████████████████████████▏ | 6099/21520 [5:02:30<8:27:43, 1.98s/it] 29%|████████████████████████████████████████████▉ | 6199/21520 [5:06:54<8:34:39, 2.02s/it] 29%|█████████████████████████████████████████████▋ | 6299/21520 [5:11:18<8:31:19, 2.02s/it] 30%|██████████████████████████████████████████████▍ | 6400/21520 [5:15:44<8:25:03, 2.00s/it] 30%|███████████████████████████████████████████████ | 6500/21520 [5:20:08<8:19:08, 1.99s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. ***** Running Evaluation ***** Num examples = 1839 Batch size = 64 {'loss': 1.1201, 'learning_rate': 5.2548951048951044e-05, 'epoch': 1.51} 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 29/29 [00:57<00:00, 1.83s/it] Configuration saved in ./checkpoint-6500/config.json Model weights saved in ./checkpoint-6500/pytorch_model.bin Configuration saved in ./checkpoint-6500/preprocessor_config.json Configuration saved in ./preprocessor_config.json Deleting older checkpoint [checkpoint-5000] due to args.save_total_limit 31%|███████████████████████████████████████████████▊ | 6600/21520 [5:27:24<8:15:18, 1.99s/it] 31%|████████████████████████████████████████████████▌ | 6700/21520 [5:31:47<8:03:10, 1.96s/it] 32%|█████████████████████████████████████████████████▎ | 6799/21520 [5:36:08<7:56:35, 1.94s/it] 32%|██████████████████████████████████████████████████ | 6900/21520 [5:40:33<8:02:43, 1.98s/it] 33%|██████████████████████████████████████████████████▋ | 6999/21520 [5:44:52<7:54:39, 1.96s/it] 33%|██████████████████████████████████████████████████▋ | 7000/21520 [5:44:54<7:55:25, 1.96s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. ***** Running Evaluation ***** Num examples = 1839 Batch size = 64 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 29/29 [00:54<00:00, 1.80s/it] Configuration saved in ./checkpoint-7000/config.json Model weights saved in ./checkpoint-7000/pytorch_model.bin Configuration saved in ./checkpoint-7000/preprocessor_config.json Configuration saved in ./preprocessor_config.json Deleting older checkpoint [checkpoint-5500] due to args.save_total_limit 33%|███████████████████████████████████████████████████▍ | 7100/21520 [5:52:08<8:10:16, 2.04s/it] 33%|████████████████████████████████████████████████████▏ | 7199/21520 [5:56:30<8:01:29, 2.02s/it] 34%|████████████████████████████████████████████████████▉ | 7299/21520 [6:00:54<7:59:22, 2.02s/it] 34%|█████████████████████████████████████████████████████▋ | 7399/21520 [6:05:20<7:52:51, 2.01s/it] 35%|██████████████████████████████████████████████████████▎ | 7499/21520 [6:09:43<7:42:55, 1.98s/it] 35%|██████████████████████████████████████████████████████▎ | 7500/21520 [6:09:45<7:45:02, 1.99s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. ***** Running Evaluation ***** Num examples = 1839 Batch size = 64 Configuration saved in ./checkpoint-7500/config.json {'eval_loss': 0.3057877719402313, 'eval_wer': 0.4125112241843759, 'eval_runtime': 65.7311, 'eval_samples_per_second': 27.978, 'eval_steps_per_second': 0.441, 'epoch': 1.74} Model weights saved in ./checkpoint-7500/pytorch_model.bin Configuration saved in ./checkpoint-7500/preprocessor_config.json Configuration saved in ./preprocessor_config.json Deleting older checkpoint [checkpoint-6000] due to args.save_total_limit 35%|███████████████████████████████████████████████████████ | 7600/21520 [6:16:59<7:31:32, 1.95s/it] 36%|███████████████████████████████████████████████████████▊ | 7699/21520 [6:21:22<7:59:09, 2.08s/it] 36%|████████████████████████████████████████████████████████▌ | 7800/21520 [6:25:47<7:34:58, 1.99s/it] 37%|█████████████████████████████████████████████████████████▎ | 7899/21520 [6:30:08<7:34:20, 2.00s/it] 37%|█████████████████████████████████████████████████████████▉ | 8000/21520 [6:34:33<7:26:34, 1.98s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. ***** Running Evaluation ***** Num examples = 1839 Batch size = 64 {'loss': 1.1101, 'learning_rate': 4.7311188811188806e-05, 'epoch': 1.86} Configuration saved in ./checkpoint-8000/config.json {'eval_loss': 0.3026394248008728, 'eval_wer': 0.4099970068841664, 'eval_runtime': 64.4321, 'eval_samples_per_second': 28.542, 'eval_steps_per_second': 0.45, 'epoch': 1.86} Model weights saved in ./checkpoint-8000/pytorch_model.bin Configuration saved in ./checkpoint-8000/preprocessor_config.json Configuration saved in ./preprocessor_config.json Deleting older checkpoint [checkpoint-6500] due to args.save_total_limit 38%|██████████████████████████████████████████████████████████▋ | 8099/21520 [6:41:43<7:19:00, 1.96s/it] 38%|███████████████████████████████████████████████████████████▍ | 8200/21520 [6:46:08<7:12:30, 1.95s/it] 39%|████████████████████████████████████████████████████████████▏ | 8301/21520 [6:50:33<7:07:31, 1.94s/it] 39%|████████████████████████████████████████████████████████████▉ | 8400/21520 [6:54:55<7:36:42, 2.09s/it] 39%|█████████████████████████████████████████████████████████████▌ | 8499/21520 [6:59:15<7:12:41, 1.99s/it] 39%|█████████████████████████████████████████████████████████████▌ | 8500/21520 [6:59:17<7:12:02, 1.99s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length. ***** Running Evaluation ***** Num examples = 1839 Batch size = 64 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 29/29 [00:56<00:00, 1.81s/it] Configuration saved in ./checkpoint-8500/config.json Model weights saved in ./checkpoint-8500/pytorch_model.bin Configuration saved in ./checkpoint-8500/preprocessor_config.json