vit5-base-vietnews-summarization-finetuned-VN

This model is a fine-tuned version of VietAI/vit5-base-vietnews-summarization on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.4435
• Rouge1: 52.8288
• Rouge2: 37.7126
• Rougel: 45.2296
• Rougelsum: 48.414
• Gen Len: 18.7023

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	399	1.6304	50.9347	33.9037	42.6387	45.4807	18.708
2.0723	2.0	799	1.5192	51.7414	35.6234	43.7745	46.7714	18.7089
1.5776	3.0	1198	1.4674	52.602	36.9274	44.6899	47.8724	18.7108
1.3842	4.0	1598	1.4452	52.6654	37.2948	44.8855	48.1186	18.7056
1.3842	4.99	1995	1.4435	52.8288	37.7126	45.2296	48.414	18.7023

Framework versions

• Transformers 4.30.0
• Pytorch 2.3.0+cu121
• Datasets 2.20.0
• Tokenizers 0.13.3