Training in progress, step 7500

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:21e00f1fbdf5235912873c9780b97ccf2dd105a60845bacbc673aa73cbb3141e
+oid sha256:668614f668fb91c4fef6a7f941dd4958fc73e871cf709e247da324057b2f3521
 size 1262817457

wandb/run-20220129_131141-h6nhqm30/files/output.log

@@ -7412,3 +7412,532 @@ Deleting older checkpoint [checkpoint-5000] due to args.save_total_limit
 Configuration saved in ./checkpoint-7000/config.json
 Model weights saved in ./checkpoint-7000/pytorch_model.bin
 Configuration saved in ./checkpoint-7000/preprocessor_config.json
+Configuration saved in ./preprocessor_config.json
+Deleting older checkpoint [checkpoint-5500] due to args.save_total_limit
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 33%|███████████████████████████████████████████████████▍                                                                                                        | 7100/21520 [5:52:08<8:10:16,  2.04s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 33%|████████████████████████████████████████████████████▏                                                                                                       | 7199/21520 [5:56:30<8:01:29,  2.02s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 34%|████████████████████████████████████████████████████▉                                                                                                       | 7299/21520 [6:00:54<7:59:22,  2.02s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 34%|█████████████████████████████████████████████████████▋                                                                                                      | 7399/21520 [6:05:20<7:52:51,  2.01s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 35%|██████████████████████████████████████████████████████▎                                                                                                     | 7499/21520 [6:09:43<7:42:55,  1.98s/it]
+ 35%|██████████████████████████████████████████████████████▎                                                                                                     | 7500/21520 [6:09:45<7:45:02,  1.99s/it]The following columns in the evaluation set  don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length.
+***** Running Evaluation *****
+  Num examples = 1839
+  Batch size = 64
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Configuration saved in ./checkpoint-7500/config.json
+{'eval_loss': 0.3057877719402313, 'eval_wer': 0.4125112241843759, 'eval_runtime': 65.7311, 'eval_samples_per_second': 27.978, 'eval_steps_per_second': 0.441, 'epoch': 1.74}
+Model weights saved in ./checkpoint-7500/pytorch_model.bin
+Configuration saved in ./checkpoint-7500/preprocessor_config.json

wandb/run-20220129_131141-h6nhqm30/run-h6nhqm30.wandb

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5e966d830fccb0b02068e7ae0ce5ee22544c7244d3a83aa7a7fc679a0fdd1ab0
-size 50397433
+oid sha256:132db8bd550d292dd27ba83c905d22c9aae42965aec8be9243e54f66789bd4a7
+size 54300922