--- license: apache-2.0 tags: - generated_from_trainer metrics: - bleu - wer base_model: google/byt5-base model-index: - name: modernisa-v2-byt5-base-lr0.0001 results: [] --- # modernisa-v2-byt5-base-lr0.0001 This model is a fine-tuned version of [google/byt5-base](https://huggingface.co/google/byt5-base) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 0.4744 - Bleu: 30.8745 - Wer: 47.8194 - Cer: 34.4895 - Gen Len: 18.5499 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.0001 - train_batch_size: 16 - eval_batch_size: 16 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 5.0 ### Training results | Training Loss | Epoch | Step | Validation Loss | Bleu | Wer | Cer | Gen Len | |:-------------:|:-----:|:-----:|:---------------:|:-------:|:-------:|:-------:|:-------:| | 0.2696 | 0.09 | 1000 | 0.3027 | 27.8571 | 49.5134 | 34.4149 | 18.5 | | 0.2518 | 0.17 | 2000 | 0.2857 | 29.2213 | 49.1981 | 34.6336 | 18.5371 | | 0.2343 | 0.26 | 3000 | 0.2730 | 29.5067 | 49.117 | 34.9795 | 18.5537 | | 0.2292 | 0.35 | 4000 | 0.2690 | 29.884 | 48.7025 | 34.8015 | 18.5516 | | 0.2243 | 0.44 | 5000 | 0.2647 | 29.9577 | 48.8466 | 34.7218 | 18.5477 | | 0.2112 | 0.52 | 6000 | 0.2636 | 30.3115 | 48.3871 | 34.4895 | 18.5477 | | 0.2118 | 0.61 | 7000 | 0.2555 | 30.6364 | 48.3961 | 34.7455 | 18.5413 | | 0.205 | 0.7 | 8000 | 0.2508 | 31.0881 | 47.468 | 34.0759 | 18.5269 | | 0.2049 | 0.78 | 9000 | 0.2471 | 31.1481 | 47.5942 | 34.4133 | 18.5503 | | 0.2005 | 0.87 | 10000 | 0.2468 | 30.9375 | 47.6392 | 34.281 | 18.5405 | | 0.1999 | 0.96 | 11000 | 0.2431 | 30.9692 | 47.7023 | 34.4183 | 18.5405 | | 0.161 | 1.04 | 12000 | 0.2491 | 31.2337 | 47.3238 | 34.1878 | 18.5298 | | 0.1601 | 1.13 | 13000 | 0.2496 | 31.4422 | 47.3689 | 34.1657 | 18.5371 | | 0.1606 | 1.22 | 14000 | 0.2459 | 31.4582 | 47.3329 | 34.2386 | 18.5405 | | 0.1594 | 1.31 | 15000 | 0.2466 | 31.386 | 47.1166 | 34.2912 | 18.5375 | | 0.1617 | 1.39 | 16000 | 0.2412 | 31.6546 | 46.8373 | 34.0149 | 18.5294 | | 0.1582 | 1.48 | 17000 | 0.2461 | 31.2924 | 47.4139 | 34.2573 | 18.5503 | | 0.1572 | 1.57 | 18000 | 0.2425 | 31.1484 | 47.45 | 34.3675 | 18.5499 | | 0.1565 | 1.65 | 19000 | 0.2424 | 31.6967 | 46.9724 | 34.1047 | 18.5388 | | 0.1585 | 1.74 | 20000 | 0.2382 | 31.9026 | 47.0175 | 34.281 | 18.558 | | 0.1522 | 1.83 | 21000 | 0.2365 | 32.1619 | 46.5219 | 33.9369 | 18.5311 | | 0.156 | 1.92 | 22000 | 0.2381 | 31.7762 | 46.7922 | 33.9572 | 18.5401 | | 0.1538 | 2.0 | 23000 | 0.2402 | 31.8785 | 46.8012 | 34.2319 | 18.5516 | | 0.1083 | 2.09 | 24000 | 0.2654 | 31.9905 | 46.603 | 34.0098 | 18.5384 | | 0.1086 | 2.18 | 25000 | 0.2618 | 31.6257 | 46.9995 | 34.2607 | 18.5409 | | 0.1092 | 2.26 | 26000 | 0.2658 | 31.4886 | 47.1436 | 34.337 | 18.5422 | | 0.1086 | 2.35 | 27000 | 0.2666 | 31.8448 | 46.6751 | 34.1217 | 18.5375 | | 0.1098 | 2.44 | 28000 | 0.2659 | 31.709 | 46.8913 | 34.1946 | 18.5452 | | 0.1117 | 2.52 | 29000 | 0.2649 | 31.8114 | 46.8913 | 34.1708 | 18.5431 | | 0.1094 | 2.61 | 30000 | 0.2656 | 31.6955 | 46.8643 | 34.1606 | 18.5375 | | 0.1077 | 2.7 | 31000 | 0.2637 | 31.5495 | 46.8823 | 34.0064 | 18.5448 | | 0.1088 | 2.79 | 32000 | 0.2669 | 32.0837 | 46.612 | 33.9504 | 18.5413 | | 0.1087 | 2.87 | 33000 | 0.2646 | 31.5549 | 47.0806 | 34.2149 | 18.5286 | | 0.1077 | 2.96 | 34000 | 0.2630 | 32.1129 | 46.4318 | 33.9403 | 18.5452 | | 0.0652 | 3.05 | 35000 | 0.3360 | 31.3861 | 47.1977 | 34.1149 | 18.5396 | | 0.0662 | 3.13 | 36000 | 0.3401 | 31.2372 | 47.3869 | 34.203 | 18.552 | | 0.0666 | 3.22 | 37000 | 0.3389 | 31.3462 | 47.2968 | 34.1759 | 18.5469 | | 0.0648 | 3.31 | 38000 | 0.3339 | 30.835 | 47.6753 | 34.381 | 18.552 | | 0.0654 | 3.4 | 39000 | 0.3395 | 31.0958 | 47.7203 | 34.4692 | 18.5524 | | 0.0663 | 3.48 | 40000 | 0.3318 | 31.126 | 47.5942 | 34.4539 | 18.5499 | | 0.0648 | 3.57 | 41000 | 0.3397 | 31.0295 | 47.5852 | 34.3539 | 18.5477 | | 0.0635 | 3.66 | 42000 | 0.3414 | 31.1287 | 47.5491 | 34.4285 | 18.5494 | | 0.0656 | 3.74 | 43000 | 0.3394 | 30.9225 | 47.6392 | 34.4285 | 18.5563 | | 0.0625 | 3.83 | 44000 | 0.3420 | 31.2435 | 47.2968 | 34.1674 | 18.5439 | | 0.0636 | 3.92 | 45000 | 0.3448 | 31.0688 | 47.6843 | 34.3743 | 18.5439 | | 0.0586 | 4.0 | 46000 | 0.3675 | 31.2353 | 47.441 | 34.2963 | 18.549 | | 0.0298 | 4.09 | 47000 | 0.4566 | 30.698 | 47.8555 | 34.4319 | 18.5512 | | 0.0301 | 4.18 | 48000 | 0.4724 | 30.7773 | 47.8374 | 34.3861 | 18.5507 | | 0.0311 | 4.27 | 49000 | 0.4640 | 31.0878 | 47.6212 | 34.3861 | 18.5503 | | 0.03 | 4.35 | 50000 | 0.4654 | 30.8319 | 47.8915 | 34.459 | 18.5529 | | 0.0302 | 4.44 | 51000 | 0.4665 | 30.9236 | 47.9276 | 34.4997 | 18.552 | | 0.029 | 4.53 | 52000 | 0.4757 | 30.8307 | 47.9456 | 34.4997 | 18.5482 | | 0.0301 | 4.61 | 53000 | 0.4672 | 30.7983 | 47.9456 | 34.5218 | 18.5473 | | 0.0294 | 4.7 | 54000 | 0.4715 | 30.8924 | 47.7564 | 34.4353 | 18.5529 | | 0.0288 | 4.79 | 55000 | 0.4752 | 30.7372 | 47.7924 | 34.4675 | 18.5524 | | 0.0289 | 4.88 | 56000 | 0.4744 | 30.8554 | 47.8555 | 34.459 | 18.5516 | | 0.0288 | 4.96 | 57000 | 0.4744 | 30.8745 | 47.8194 | 34.4895 | 18.5499 | ### Framework versions - Transformers 4.30.0.dev0 - Pytorch 1.13.0+cu117 - Datasets 2.12.0 - Tokenizers 0.11.0