mt5-small-finetuned-indonesian-text-summarization-v3

This model is a fine-tuned version of google/mt5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7499
• Rouge1: 63.8193
• Rouge2: 58.2187
• Rougel: 63.1123
• Rougelsum: 63.2782

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
2.2426	1.0	1783	0.8618	62.2558	56.8185	61.5786	61.6898
0.9017	2.0	3566	0.7946	62.8862	57.2359	62.151	62.3178
0.7717	3.0	5349	0.7633	63.2898	57.6478	62.552	62.7381
0.7093	4.0	7132	0.7402	63.6899	58.0258	62.9883	63.137
0.6815	5.0	8915	0.7499	63.8193	58.2187	63.1123	63.2782

Framework versions

• Transformers 4.42.4
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1