Training in progress, step 1000

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8e46f7437225e43f76d2ee2c7a3137b59f4088c38f3fa35c66ff2baafd8ae804
+oid sha256:ea8b6080f2824e9bfcaf111587189727b4ec250735d10c647a6a751e6ca08a2e
 size 1262083569

wandb/run-20220202_140930-21azk117/files/output.log

@@ -549,3 +549,553 @@ Model weights saved in ./checkpoint-500/pytorch_model.bin
 Configuration saved in ./checkpoint-500/preprocessor_config.json
 Configuration saved in ./preprocessor_config.json
 Adding files tracked by Git LFS: ['wandb/run-20220131_140404-gjg8nz5t/run-gjg8nz5t.wandb']. This may take a bit of time if the files are large.
+02/02/2022 15:34:32 - WARNING - huggingface_hub.repository - Adding files tracked by Git LFS: ['wandb/run-20220131_140404-gjg8nz5t/run-gjg8nz5t.wandb']. This may take a bit of time if the files are large.
+02/02/2022 15:34:47 - WARNING - huggingface_hub.repository - Several commits (2) will be pushed upstream.
+Several commits (2) will be pushed upstream.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 15%|████████████████████████▊                                                                                                                                             | 599/4000 [1:41:26<8:50:17,  9.36s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 18%|████████████████████████████▊                                                                                                                                        | 700/4000 [1:58:03<10:10:08, 11.09s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 20%|█████████████████████████████████▏                                                                                                                                    | 799/4000 [2:14:09<8:15:37,  9.29s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 22%|█████████████████████████████████████▎                                                                                                                                | 899/4000 [2:30:29<8:25:05,  9.77s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 25%|█████████████████████████████████████████▎                                                                                                                           | 1000/4000 [2:46:50<6:19:55,  7.60s/it]The following columns in the evaluation set  don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length.
+***** Running Evaluation *****
+  Num examples = 2609
+  Batch size = 72
+{'loss': 0.1592, 'learning_rate': 0.0009506789790182364, 'epoch': 25.0}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 37/37 [02:12<00:00,  2.76s/it]
+Configuration saved in ./checkpoint-1000/config.json
+Model weights saved in ./checkpoint-1000/pytorch_model.bin
+Configuration saved in ./checkpoint-1000/preprocessor_config.json