Training in progress, step 8000

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:668614f668fb91c4fef6a7f941dd4958fc73e871cf709e247da324057b2f3521
+oid sha256:c7d5c5665cb2f10cbee4073c92b5d77e0f1dafd3e633c65ef200d4ca4dca469e
 size 1262817457

wandb/run-20220129_131141-h6nhqm30/files/output.log

@@ -7941,3 +7941,532 @@ Configuration saved in ./checkpoint-7500/config.json
 {'eval_loss': 0.3057877719402313, 'eval_wer': 0.4125112241843759, 'eval_runtime': 65.7311, 'eval_samples_per_second': 27.978, 'eval_steps_per_second': 0.441, 'epoch': 1.74}
 Model weights saved in ./checkpoint-7500/pytorch_model.bin
 Configuration saved in ./checkpoint-7500/preprocessor_config.json
+Configuration saved in ./preprocessor_config.json
+Deleting older checkpoint [checkpoint-6000] due to args.save_total_limit
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 35%|███████████████████████████████████████████████████████                                                                                                     | 7600/21520 [6:16:59<7:31:32,  1.95s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 36%|███████████████████████████████████████████████████████▊                                                                                                    | 7699/21520 [6:21:22<7:59:09,  2.08s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 36%|████████████████████████████████████████████████████████▌                                                                                                   | 7800/21520 [6:25:47<7:34:58,  1.99s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 37%|█████████████████████████████████████████████████████████▎                                                                                                  | 7899/21520 [6:30:08<7:34:20,  2.00s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 37%|█████████████████████████████████████████████████████████▉                                                                                                  | 8000/21520 [6:34:33<7:26:34,  1.98s/it]The following columns in the evaluation set  don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length.
+***** Running Evaluation *****
+  Num examples = 1839
+  Batch size = 64
+{'loss': 1.1101, 'learning_rate': 4.7311188811188806e-05, 'epoch': 1.86}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Configuration saved in ./checkpoint-8000/config.json
+{'eval_loss': 0.3026394248008728, 'eval_wer': 0.4099970068841664, 'eval_runtime': 64.4321, 'eval_samples_per_second': 28.542, 'eval_steps_per_second': 0.45, 'epoch': 1.86}
+Model weights saved in ./checkpoint-8000/pytorch_model.bin
+Configuration saved in ./checkpoint-8000/preprocessor_config.json

wandb/run-20220129_131141-h6nhqm30/run-h6nhqm30.wandb

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:132db8bd550d292dd27ba83c905d22c9aae42965aec8be9243e54f66789bd4a7
-size 54300922
+oid sha256:65aa39669da36f28c6653cdb0f6c9fe3d2f0eba29768f21178ba8144dbee3edf
+size 58269618