vit5-base-vietnews-summarization-finetuned-VN

This model is a fine-tuned version of VietAI/vit5-base-vietnews-summarization on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.8974
• Rouge1: 46.8897
• Rouge2: 21.2655
• Rougel: 33.489
• Rougelsum: 33.5578
• Gen Len: 18.9871

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
2.408	1.0	737	1.9329	46.6812	20.8232	33.1637	33.2377	18.9608
2.0444	2.0	1475	1.8841	46.7693	21.3121	33.37	33.4394	18.9623
1.7589	3.0	2212	1.8840	46.6303	21.0522	33.2678	33.3244	18.9777
1.6919	4.0	2950	1.8864	46.7928	21.395	33.3749	33.4439	18.9911
1.5844	5.0	3685	1.8974	46.8897	21.2655	33.489	33.5578	18.9871

Framework versions

• Transformers 4.30.0
• Pytorch 2.3.0+cu121
• Datasets 2.19.2
• Tokenizers 0.13.3