--- base_model: Karzan/walamakan-t5-base tags: - generated_from_trainer metrics: - bleu model-index: - name: walamakan-t5-base results: [] --- # walamakan-t5-base This model is a fine-tuned version of [Karzan/walamakan-t5-base](https://huggingface.co/Karzan/walamakan-t5-base) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 1.0693 - Bleu: 0.0 - Gen Len: 19.0 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 2e-05 - train_batch_size: 8 - eval_batch_size: 8 - seed: 42 - gradient_accumulation_steps: 4 - total_train_batch_size: 32 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 100 ### Training results | Training Loss | Epoch | Step | Validation Loss | Bleu | Gen Len | |:-------------:|:-----:|:----:|:---------------:|:----:|:-------:| | No log | 1.0 | 87 | 0.8774 | 0.0 | 19.0 | | No log | 1.99 | 174 | 0.8688 | 0.0 | 19.0 | | No log | 2.99 | 261 | 0.8646 | 0.0 | 19.0 | | No log | 4.0 | 349 | 0.8800 | 0.0 | 19.0 | | No log | 5.0 | 436 | 0.8954 | 0.0 | 19.0 | | 1.5379 | 5.99 | 523 | 0.8964 | 0.0 | 19.0 | | 1.5379 | 6.99 | 610 | 0.8949 | 0.0 | 19.0 | | 1.5379 | 8.0 | 698 | 0.9078 | 0.0 | 19.0 | | 1.5379 | 9.0 | 785 | 0.9080 | 0.0 | 19.0 | | 1.5379 | 9.99 | 872 | 0.9188 | 0.0 | 19.0 | | 1.5379 | 10.99 | 959 | 0.9262 | 0.0 | 19.0 | | 1.4663 | 12.0 | 1047 | 0.9254 | 0.0 | 19.0 | | 1.4663 | 13.0 | 1134 | 0.9270 | 0.0 | 19.0 | | 1.4663 | 13.99 | 1221 | 0.9348 | 0.0 | 19.0 | | 1.4663 | 14.99 | 1308 | 0.9354 | 0.0 | 19.0 | | 1.4663 | 16.0 | 1396 | 0.9399 | 0.0 | 19.0 | | 1.4663 | 17.0 | 1483 | 0.9453 | 0.0 | 19.0 | | 1.4062 | 17.99 | 1570 | 0.9441 | 0.0 | 19.0 | | 1.4062 | 18.99 | 1657 | 0.9510 | 0.0 | 19.0 | | 1.4062 | 20.0 | 1745 | 0.9555 | 0.0 | 19.0 | | 1.4062 | 21.0 | 1832 | 0.9549 | 0.0 | 19.0 | | 1.4062 | 21.99 | 1919 | 0.9629 | 0.0 | 19.0 | | 1.3566 | 22.99 | 2006 | 0.9607 | 0.0 | 19.0 | | 1.3566 | 24.0 | 2094 | 0.9686 | 0.0 | 19.0 | | 1.3566 | 25.0 | 2181 | 0.9687 | 0.0 | 19.0 | | 1.3566 | 25.99 | 2268 | 0.9741 | 0.0 | 19.0 | | 1.3566 | 26.99 | 2355 | 0.9719 | 0.0 | 19.0 | | 1.3566 | 28.0 | 2443 | 0.9711 | 0.0 | 19.0 | | 1.312 | 29.0 | 2530 | 0.9750 | 0.0 | 19.0 | | 1.312 | 29.99 | 2617 | 0.9815 | 0.0 | 19.0 | | 1.312 | 30.99 | 2704 | 0.9841 | 0.0 | 19.0 | | 1.312 | 32.0 | 2792 | 0.9865 | 0.0 | 19.0 | | 1.312 | 33.0 | 2879 | 0.9876 | 0.0 | 19.0 | | 1.312 | 33.99 | 2966 | 0.9914 | 0.0 | 19.0 | | 1.2731 | 34.99 | 3053 | 0.9942 | 0.0 | 19.0 | | 1.2731 | 36.0 | 3141 | 0.9938 | 0.0 | 19.0 | | 1.2731 | 37.0 | 3228 | 0.9994 | 0.0 | 19.0 | | 1.2731 | 37.99 | 3315 | 1.0028 | 0.0 | 19.0 | | 1.2731 | 38.99 | 3402 | 1.0039 | 0.0 | 19.0 | | 1.2731 | 40.0 | 3490 | 1.0050 | 0.0 | 19.0 | | 1.2366 | 41.0 | 3577 | 1.0093 | 0.0 | 19.0 | | 1.2366 | 41.99 | 3664 | 1.0072 | 0.0 | 19.0 | | 1.2366 | 42.99 | 3751 | 1.0118 | 0.0 | 19.0 | | 1.2366 | 44.0 | 3839 | 1.0137 | 0.0 | 19.0 | | 1.2366 | 45.0 | 3926 | 1.0156 | 0.0 | 19.0 | | 1.2054 | 45.99 | 4013 | 1.0157 | 0.0 | 19.0 | | 1.2054 | 46.99 | 4100 | 1.0179 | 0.0 | 19.0 | | 1.2054 | 48.0 | 4188 | 1.0212 | 0.0 | 19.0 | | 1.2054 | 49.0 | 4275 | 1.0237 | 0.0 | 19.0 | | 1.2054 | 49.99 | 4362 | 1.0230 | 0.0 | 19.0 | | 1.2054 | 50.99 | 4449 | 1.0262 | 0.0 | 19.0 | | 1.1797 | 52.0 | 4537 | 1.0269 | 0.0 | 19.0 | | 1.1797 | 53.0 | 4624 | 1.0293 | 0.0 | 19.0 | | 1.1797 | 53.99 | 4711 | 1.0311 | 0.0 | 19.0 | | 1.1797 | 54.99 | 4798 | 1.0323 | 0.0 | 19.0 | | 1.1797 | 56.0 | 4886 | 1.0342 | 0.0 | 19.0 | | 1.1797 | 57.0 | 4973 | 1.0371 | 0.0 | 19.0 | | 1.153 | 57.99 | 5060 | 1.0374 | 0.0 | 19.0 | | 1.153 | 58.99 | 5147 | 1.0397 | 0.0 | 19.0 | | 1.153 | 60.0 | 5235 | 1.0411 | 0.0 | 19.0 | | 1.153 | 61.0 | 5322 | 1.0410 | 0.0 | 19.0 | | 1.153 | 61.99 | 5409 | 1.0442 | 0.0 | 19.0 | | 1.153 | 62.99 | 5496 | 1.0438 | 0.0 | 19.0 | | 1.1322 | 64.0 | 5584 | 1.0466 | 0.0 | 19.0 | | 1.1322 | 65.0 | 5671 | 1.0471 | 0.0 | 19.0 | | 1.1322 | 65.99 | 5758 | 1.0501 | 0.0 | 19.0 | | 1.1322 | 66.99 | 5845 | 1.0501 | 0.0 | 19.0 | | 1.1322 | 68.0 | 5933 | 1.0505 | 0.0 | 19.0 | | 1.1102 | 69.0 | 6020 | 1.0519 | 0.0 | 19.0 | | 1.1102 | 69.99 | 6107 | 1.0535 | 0.0 | 19.0 | | 1.1102 | 70.99 | 6194 | 1.0548 | 0.0 | 19.0 | | 1.1102 | 72.0 | 6282 | 1.0546 | 0.0 | 19.0 | | 1.1102 | 73.0 | 6369 | 1.0558 | 0.0 | 19.0 | | 1.1102 | 73.99 | 6456 | 1.0569 | 0.0 | 19.0 | | 1.0962 | 74.99 | 6543 | 1.0590 | 0.0 | 19.0 | | 1.0962 | 76.0 | 6631 | 1.0585 | 0.0 | 19.0 | | 1.0962 | 77.0 | 6718 | 1.0598 | 0.0 | 19.0 | | 1.0962 | 77.99 | 6805 | 1.0605 | 0.0 | 19.0 | | 1.0962 | 78.99 | 6892 | 1.0616 | 0.0 | 19.0 | | 1.0962 | 80.0 | 6980 | 1.0627 | 0.0 | 19.0 | | 1.0821 | 81.0 | 7067 | 1.0633 | 0.0 | 19.0 | | 1.0821 | 81.99 | 7154 | 1.0633 | 0.0 | 19.0 | | 1.0821 | 82.99 | 7241 | 1.0634 | 0.0 | 19.0 | | 1.0821 | 84.0 | 7329 | 1.0648 | 0.0 | 19.0 | | 1.0821 | 85.0 | 7416 | 1.0651 | 0.0 | 19.0 | | 1.0739 | 85.99 | 7503 | 1.0657 | 0.0 | 19.0 | | 1.0739 | 86.99 | 7590 | 1.0657 | 0.0 | 19.0 | | 1.0739 | 88.0 | 7678 | 1.0671 | 0.0 | 19.0 | | 1.0739 | 89.0 | 7765 | 1.0680 | 0.0 | 19.0 | | 1.0739 | 89.99 | 7852 | 1.0680 | 0.0 | 19.0 | | 1.0739 | 90.99 | 7939 | 1.0684 | 0.0 | 19.0 | | 1.065 | 92.0 | 8027 | 1.0684 | 0.0 | 19.0 | | 1.065 | 93.0 | 8114 | 1.0687 | 0.0 | 19.0 | | 1.065 | 93.99 | 8201 | 1.0689 | 0.0 | 19.0 | | 1.065 | 94.99 | 8288 | 1.0690 | 0.0 | 19.0 | | 1.065 | 96.0 | 8376 | 1.0691 | 0.0 | 19.0 | | 1.065 | 97.0 | 8463 | 1.0692 | 0.0 | 19.0 | | 1.0606 | 97.99 | 8550 | 1.0692 | 0.0 | 19.0 | | 1.0606 | 98.99 | 8637 | 1.0692 | 0.0 | 19.0 | | 1.0606 | 99.71 | 8700 | 1.0693 | 0.0 | 19.0 | ### Framework versions - Transformers 4.33.0 - Pytorch 2.0.1+cu118 - Datasets 2.14.4 - Tokenizers 0.13.3