Training in progress, step 2000

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9e78b33a9909a4828e19fbd42966871353c13d8d63ea1386a60b23038fd15790
+oid sha256:8139be00a4aede982b5c73f2fc917d63e31d74564bb68e51f35677f1901cdf5a
 size 1262821553

wandb/run-20220130_191954-1mtmnz5y/files/output.log

@@ -1728,3 +1728,580 @@ Model weights saved in ./checkpoint-1500/pytorch_model.bin
 Configuration saved in ./checkpoint-1500/preprocessor_config.json
 Configuration saved in ./preprocessor_config.json
 Adding files tracked by Git LFS: ['wandb/run-20220130_191954-1mtmnz5y/run-1mtmnz5y.wandb']. This may take a bit of time if the files are large.
+01/30/2022 21:00:59 - WARNING - huggingface_hub.repository - Adding files tracked by Git LFS: ['wandb/run-20220130_191954-1mtmnz5y/run-1mtmnz5y.wandb']. This may take a bit of time if the files are large.
+Deleting older checkpoint [checkpoint-500] due to args.save_total_limit
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+  6%|█████████▎                                                                                                                                                        | 1599/27860 [1:47:00<13:37:20,  1.87s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+  6%|█████████▉                                                                                                                                                        | 1699/27860 [1:52:35<13:48:13,  1.90s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+  6%|██████████▍                                                                                                                                                       | 1800/27860 [1:58:15<12:32:51,  1.73s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+  7%|███████████                                                                                                                                                       | 1899/27860 [2:03:54<13:14:15,  1.84s/it]
+{'loss': 1.2874, 'learning_rate': 7.018518518518519e-05, 'epoch': 0.27}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+  7%|███████████▌                                                                                                                                                      | 1999/27860 [2:09:34<13:16:40,  1.85s/it]
+  7%|███████████▋                                                                                                                                                      | 2000/27860 [2:09:35<12:43:23,  1.77s/it]The following columns in the evaluation set  don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length.
+***** Running Evaluation *****
+  Num examples = 4469
+  Batch size = 64
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 70/70 [03:39<00:00,  3.08s/it]
+
+Configuration saved in ./checkpoint-2000/config.json
+Model weights saved in ./checkpoint-2000/pytorch_model.bin
+Configuration saved in ./checkpoint-2000/preprocessor_config.json

wandb/run-20220130_191954-1mtmnz5y/run-1mtmnz5y.wandb

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4ca123805487c11a01eabb3cfe00fd62287af559fc43c91e250ab52aa737a0a1
-size 10960201
+oid sha256:8a47923731c1c4b7262e6b72923204000deb0547409c7ad3994c6675e89c760c
+size 14576322