--- language: - es - maq license: apache-2.0 tags: - generated_from_trainer metrics: - bleu model-index: - name: byt5-base-es_maq results: [] --- # byt5-base-es_maq This model is a fine-tuned version of [google/byt5-base](https://huggingface.co/google/byt5-base) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 1.0862 - Bleu: 16.0295 - Gen Len: 98.8829 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 5e-05 - train_batch_size: 16 - eval_batch_size: 16 - seed: 65 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 100.0 ### Training results | Training Loss | Epoch | Step | Validation Loss | Bleu | Gen Len | |:-------------:|:-----:|:-----:|:---------------:|:------:|:-------:| | No log | 1.0 | 398 | 0.9955 | 0.0907 | 19.0 | | 1.4196 | 2.0 | 796 | 0.8670 | 0.5548 | 19.0 | | 0.9762 | 3.0 | 1194 | 0.8083 | 0.2508 | 19.0 | | 0.8703 | 4.0 | 1592 | 0.7638 | 0.5692 | 19.0 | | 0.8703 | 5.0 | 1990 | 0.7335 | 0.3461 | 19.0 | | 0.8098 | 6.0 | 2388 | 0.7079 | 0.399 | 19.0 | | 0.7592 | 7.0 | 2786 | 0.6846 | 0.3376 | 19.0 | | 0.7167 | 8.0 | 3184 | 0.6675 | 0.4617 | 19.0 | | 0.6881 | 9.0 | 3582 | 0.6496 | 0.438 | 19.0 | | 0.6881 | 10.0 | 3980 | 0.6297 | 0.4397 | 19.0 | | 0.6543 | 11.0 | 4378 | 0.6144 | 0.4078 | 19.0 | | 0.6245 | 12.0 | 4776 | 0.6091 | 0.3468 | 19.0 | | 0.5959 | 13.0 | 5174 | 0.6039 | 0.433 | 19.0 | | 0.5766 | 14.0 | 5572 | 0.5971 | 0.4332 | 19.0 | | 0.5766 | 15.0 | 5970 | 0.5931 | 0.4291 | 19.0 | | 0.5541 | 16.0 | 6368 | 0.5877 | 0.4504 | 19.0 | | 0.5331 | 17.0 | 6766 | 0.5873 | 0.4359 | 19.0 | | 0.5169 | 18.0 | 7164 | 0.5864 | 0.419 | 19.0 | | 0.4991 | 19.0 | 7562 | 0.5880 | 0.4191 | 19.0 | | 0.4991 | 20.0 | 7960 | 0.5845 | 0.4535 | 19.0 | | 0.4827 | 21.0 | 8358 | 0.5889 | 0.4614 | 19.0 | | 0.4646 | 22.0 | 8756 | 0.5894 | 0.4075 | 19.0 | | 0.4523 | 23.0 | 9154 | 0.5905 | 0.4399 | 19.0 | | 0.437 | 24.0 | 9552 | 0.5985 | 0.4369 | 19.0 | | 0.437 | 25.0 | 9950 | 0.5960 | 0.4056 | 19.0 | | 0.4229 | 26.0 | 10348 | 0.5962 | 0.4252 | 19.0 | | 0.4091 | 27.0 | 10746 | 0.6049 | 0.4713 | 19.0 | | 0.3965 | 28.0 | 11144 | 0.6118 | 0.4242 | 19.0 | | 0.3842 | 29.0 | 11542 | 0.6170 | 0.3924 | 19.0 | | 0.3842 | 30.0 | 11940 | 0.6114 | 0.3984 | 19.0 | | 0.3718 | 31.0 | 12338 | 0.6304 | 0.4186 | 19.0 | | 0.3585 | 32.0 | 12736 | 0.6364 | 0.3846 | 19.0 | | 0.3473 | 33.0 | 13134 | 0.6325 | 0.4058 | 19.0 | | 0.3377 | 34.0 | 13532 | 0.6434 | 0.3669 | 19.0 | | 0.3377 | 35.0 | 13930 | 0.6559 | 0.396 | 19.0 | | 0.3258 | 36.0 | 14328 | 0.6614 | 0.4449 | 19.0 | | 0.3144 | 37.0 | 14726 | 0.6619 | 0.3988 | 19.0 | | 0.3062 | 38.0 | 15124 | 0.6812 | 0.4133 | 19.0 | | 0.2976 | 39.0 | 15522 | 0.6795 | 0.4102 | 19.0 | | 0.2976 | 40.0 | 15920 | 0.6798 | 0.3953 | 19.0 | | 0.2883 | 41.0 | 16318 | 0.7088 | 0.3846 | 19.0 | | 0.2791 | 42.0 | 16716 | 0.7110 | 0.3701 | 19.0 | | 0.2701 | 43.0 | 17114 | 0.7160 | 0.3985 | 19.0 | | 0.2619 | 44.0 | 17512 | 0.7150 | 0.3654 | 19.0 | | 0.2619 | 45.0 | 17910 | 0.7197 | 0.394 | 19.0 | | 0.2527 | 46.0 | 18308 | 0.7387 | 0.4033 | 19.0 | | 0.2444 | 47.0 | 18706 | 0.7438 | 0.389 | 19.0 | | 0.239 | 48.0 | 19104 | 0.7597 | 0.3948 | 19.0 | | 0.2303 | 49.0 | 19502 | 0.7645 | 0.3976 | 19.0 | | 0.2303 | 50.0 | 19900 | 0.7786 | 0.385 | 19.0 | | 0.2212 | 51.0 | 20298 | 0.7699 | 0.3948 | 19.0 | | 0.2157 | 52.0 | 20696 | 0.7902 | 0.4265 | 19.0 | | 0.2108 | 53.0 | 21094 | 0.7906 | 0.3924 | 19.0 | | 0.2108 | 54.0 | 21492 | 0.8098 | 0.3849 | 19.0 | | 0.2041 | 55.0 | 21890 | 0.8167 | 0.3888 | 19.0 | | 0.1959 | 56.0 | 22288 | 0.8317 | 0.4139 | 19.0 | | 0.1899 | 57.0 | 22686 | 0.8345 | 0.4136 | 19.0 | | 0.1868 | 58.0 | 23084 | 0.8484 | 0.4093 | 19.0 | | 0.1868 | 59.0 | 23482 | 0.8663 | 0.4013 | 19.0 | | 0.1815 | 60.0 | 23880 | 0.8709 | 0.3858 | 19.0 | | 0.1744 | 61.0 | 24278 | 0.8845 | 0.3716 | 19.0 | | 0.1709 | 62.0 | 24676 | 0.8787 | 0.3781 | 19.0 | | 0.1659 | 63.0 | 25074 | 0.8844 | 0.3642 | 19.0 | | 0.1659 | 64.0 | 25472 | 0.9034 | 0.3818 | 19.0 | | 0.1625 | 65.0 | 25870 | 0.9117 | 0.3522 | 19.0 | | 0.1568 | 66.0 | 26268 | 0.9059 | 0.3892 | 19.0 | | 0.1539 | 67.0 | 26666 | 0.9160 | 0.398 | 19.0 | | 0.1501 | 68.0 | 27064 | 0.9333 | 0.3831 | 19.0 | | 0.1501 | 69.0 | 27462 | 0.9351 | 0.4036 | 19.0 | | 0.1461 | 70.0 | 27860 | 0.9484 | 0.3727 | 19.0 | | 0.1413 | 71.0 | 28258 | 0.9522 | 0.3638 | 19.0 | | 0.1405 | 72.0 | 28656 | 0.9725 | 0.3501 | 19.0 | | 0.1365 | 73.0 | 29054 | 0.9698 | 0.372 | 19.0 | | 0.1365 | 74.0 | 29452 | 0.9703 | 0.3727 | 19.0 | | 0.1328 | 75.0 | 29850 | 0.9798 | 0.3834 | 19.0 | | 0.1298 | 76.0 | 30248 | 0.9850 | 0.4008 | 19.0 | | 0.1283 | 77.0 | 30646 | 0.9988 | 0.3815 | 19.0 | | 0.1247 | 78.0 | 31044 | 0.9896 | 0.3621 | 19.0 | | 0.1247 | 79.0 | 31442 | 1.0035 | 0.3761 | 19.0 | | 0.1222 | 80.0 | 31840 | 1.0223 | 0.3729 | 19.0 | | 0.1195 | 81.0 | 32238 | 1.0171 | 0.3866 | 19.0 | | 0.1189 | 82.0 | 32636 | 1.0247 | 0.3698 | 19.0 | | 0.1175 | 83.0 | 33034 | 1.0151 | 0.3657 | 19.0 | | 0.1175 | 84.0 | 33432 | 1.0388 | 0.3786 | 19.0 | | 0.1146 | 85.0 | 33830 | 1.0413 | 0.3737 | 19.0 | | 0.1124 | 86.0 | 34228 | 1.0402 | 0.3803 | 19.0 | | 0.1125 | 87.0 | 34626 | 1.0519 | 0.3746 | 19.0 | | 0.1102 | 88.0 | 35024 | 1.0542 | 0.3863 | 19.0 | | 0.1102 | 89.0 | 35422 | 1.0626 | 0.3839 | 19.0 | | 0.1075 | 90.0 | 35820 | 1.0602 | 0.3615 | 19.0 | | 0.1069 | 91.0 | 36218 | 1.0701 | 0.3692 | 19.0 | | 0.1062 | 92.0 | 36616 | 1.0699 | 0.3719 | 19.0 | | 0.1051 | 93.0 | 37014 | 1.0732 | 0.3667 | 19.0 | | 0.1051 | 94.0 | 37412 | 1.0749 | 0.3701 | 19.0 | | 0.1041 | 95.0 | 37810 | 1.0796 | 0.3744 | 19.0 | | 0.1034 | 96.0 | 38208 | 1.0823 | 0.3771 | 19.0 | | 0.1031 | 97.0 | 38606 | 1.0797 | 0.3775 | 19.0 | | 0.1015 | 98.0 | 39004 | 1.0842 | 0.3822 | 19.0 | | 0.1015 | 99.0 | 39402 | 1.0859 | 0.3839 | 19.0 | | 0.1007 | 100.0 | 39800 | 1.0862 | 0.3829 | 19.0 | ### Framework versions - Transformers 4.29.2 - Pytorch 2.0.1+cu118 - Datasets 2.12.0 - Tokenizers 0.13.3