update model card README.md

README.md

@@ -17,8 +17,8 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model was trained from scratch on the xtreme_s dataset.
 It achieves the following results on the evaluation set:
+- Loss: 2.1356
 - Bleu: 0.0000
-- Loss: 3.0511
 
 ## Model description
 
@@ -37,12 +37,12 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 0.00022154311254852488
+- learning_rate: 3e-05
 - train_batch_size: 8
 - eval_batch_size: 8
 - seed: 42
-- gradient_accumulation_steps: 8
-- total_train_batch_size: 64
+- gradient_accumulation_steps: 16
+- total_train_batch_size: 128
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
 - lr_scheduler_warmup_steps: 500
@@ -51,27 +51,17 @@ The following hyperparameters were used during training:
 
 ### Training results
 
-| Training Loss | Epoch | Step | Bleu   | Validation Loss |
-|:-------------:|:-----:|:----:|:------:|:---------------:|
-| 3.1046        | 0.15  | 500  | 0.0    | 5.5272          |
-| 3.0735        | 0.31  | 1000 | 0.0000 | 5.5994          |
-| 2.9879        | 0.46  | 1500 | 0.0    | 5.2181          |
-| 2.9617        | 0.62  | 2000 | 0.0    | 4.9660          |
-| 2.8816        | 0.77  | 2500 | 0.0    | 4.6292          |
-| 2.5174        | 0.93  | 3000 | 0.0    | 4.6497          |
-| 4.1027        | 1.08  | 3500 | 0.0    | 4.3291          |
-| 3.8403        | 1.23  | 4000 | 0.0000 | 4.2548          |
-| 4.0396        | 1.39  | 4500 | 0.0    | 4.2812          |
-| 3.856         | 1.54  | 5000 | 0.0000 | 4.1702          |
-| 3.6918        | 1.7   | 5500 | 0.0000 | 4.1578          |
-| 3.4118        | 1.85  | 6000 | 0.0000 | 4.1312          |
-| 2.9148        | 2.01  | 6500 | 0.0000 | 4.0282          |
-| 2.6245        | 2.16  | 7000 | 0.0000 | 4.0361          |
-| 2.4924        | 2.31  | 7500 | 0.0000 | 3.8836          |
-| 2.6693        | 2.47  | 8000 | 0.0000 | 3.5585          |
-| 2.0685        | 2.62  | 8500 | 0.0000 | 3.2877          |
-| 2.1419        | 2.78  | 9000 | 0.0000 | 3.1468          |
-| 1.9175        | 2.93  | 9500 | 0.0000 | 3.0511          |
+| Training Loss | Epoch | Step | Validation Loss | Bleu   |
+|:-------------:|:-----:|:----:|:---------------:|:------:|
+| 2.6458        | 0.31  | 500  | 4.3754          | 0.0    |
+| 2.3505        | 0.62  | 1000 | 4.3071          | 0.0    |
+| 2.2152        | 0.93  | 1500 | 3.9444          | 0.0    |
+| 2.79          | 1.23  | 2000 | 3.2046          | 0.0000 |
+| 2.569         | 1.54  | 2500 | 2.6812          | 0.0000 |
+| 2.322         | 1.85  | 3000 | 2.4081          | 0.0000 |
+| 2.3435        | 2.16  | 3500 | 2.2696          | 0.0000 |
+| 2.2063        | 2.47  | 4000 | 2.2452          | 0.0000 |
+| 2.1087        | 2.78  | 4500 | 2.1356          | 0.0000 |
 
 
 ### Framework versions

wandb/run-20220505_173748-b097rk18/files/output.log

@@ -38672,3 +38672,42 @@ Could not estimate the number of tokens of the input, floating-point operations
 Saving model checkpoint to ./ble with gradient checkpointing. Setting `use_cache=False`...e computed50<00:31,  7.78s/it]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
 Saving model checkpoint to ./ble with gradient checkpointing. Setting `use_cache=False`...e computed50<00:31,  7.78s/it]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
 Saving model checkpoint to ./ble with gradient checkpointing. Setting `use_cache=False`...e computed50<00:31,  7.78s/it]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:   0%|              | 800k/996M [00:01<23:06, 753kB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:   0%|             | 1.25M/996M [00:03<48:50, 356kB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:   0%|             | 1.47M/996M [00:04<56:43, 306kB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:   0%|             | 2.09M/996M [00:06<55:23, 313kB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:   2%|▏           | 18.6M/996M [00:08<03:04, 5.55MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:   5%|▋           | 54.1M/996M [00:10<01:19, 12.4MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:   9%|█           | 90.0M/996M [00:12<01:00, 15.7MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  13%|█▋           | 125M/996M [00:14<00:53, 17.1MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  16%|██           | 162M/996M [00:16<00:48, 18.0MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  20%|██▌          | 199M/996M [00:18<00:44, 18.9MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  24%|███          | 235M/996M [00:20<00:42, 18.6MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  27%|███▌         | 269M/996M [00:22<00:41, 18.3MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  30%|███▉         | 297M/996M [00:24<00:44, 16.5MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  33%|████▎        | 329M/996M [00:26<00:42, 16.6MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  37%|████▊        | 364M/996M [00:28<00:37, 17.6MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  40%|█████▏       | 401M/996M [00:30<00:33, 18.7MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  44%|█████▊       | 441M/996M [00:32<00:29, 19.9MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  48%|██████▎      | 483M/996M [00:34<00:25, 20.8MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  52%|██████▊      | 523M/996M [00:36<00:23, 20.8MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  56%|███████▎     | 562M/996M [00:38<00:22, 20.6MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  60%|███████▊     | 601M/996M [00:40<00:20, 20.6MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  64%|████████▎    | 637M/996M [00:42<00:18, 19.8MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  68%|████████▊    | 672M/996M [00:44<00:17, 19.0MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  71%|█████████▏   | 706M/996M [00:46<00:16, 18.2MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  74%|█████████▋   | 739M/996M [00:48<00:15, 17.8MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  77%|██████████   | 771M/996M [00:50<00:13, 17.3MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  81%|██████████▌  | 805M/996M [00:52<00:11, 17.5MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  84%|██████████▉  | 838M/996M [00:54<00:09, 17.4MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  88%|███████████▍ | 874M/996M [00:56<00:07, 17.9MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  91%|███████████▉ | 910M/996M [00:58<00:04, 18.7MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  95%|████████████▎| 947M/996M [01:00<00:02, 18.9MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  99%|████████████▉| 986M/996M [01:02<00:00, 19.6MB/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  99%|████████████▉| 986M/996M [01:02<00:00, 19.6MB/s]remote: Enforcing permissions...        adient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb:  99%|████████████▉| 986M/996M [01:02<00:00, 19.6MB/s]remote: Enforcing permissions...        adient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+05/06/2022 12:36:36 - WARNING - huggingface_hub.repository - remote: Enforcing permissions...
+remote: Allowed refs: all
+To https://huggingface.co/sanchit-gandhi/xtreme_s_xlsr_2_bart_covost2_fr_en
+Upload file wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb: 100%|█████████████| 996M/996M [01:05<00:00, 16.0MB/s]remote: Enforcing permissions...        adient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.
+Dropping the following result as it does not have all the necessary fields:██████████| 996M/996M [01:05<00:00, 16.0MB/s]remote: Enforcing permissions...        adient checkpointing. Setting `use_cache=False`...hEncoderDecoderModel.forward` and have been ignored: length, lang. If length, lang are not expected by `SpeechEncoderDecoderModel.forward`,  you can safely ignore this message.

wandb/run-20220505_173748-b097rk18/logs/debug-internal.log

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:56ea050d8c90a4d620738ef1b283dddfb26eb25e6211951d6f420284ae413526
-size 12060078
+oid sha256:42751141239ac706b4a835b40129281b94443ab53ba2447e917934ee97a58715
+size 12069321

wandb/run-20220505_173748-b097rk18/run-b097rk18.wandb

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3c2e7df62fa15b939648454376afc271c256f791eb2f70a66c2d06c8b1626b54
-size 1043931187
+oid sha256:939ea830f8c246137259d33f9620ac9a5eeab390aa6ed3e6321f524870db8e2c
+size 1043963285