finetune-newwiki-summarization-ver-augmented

This model is a fine-tuned version of VietAI/vit5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4282
• Rouge1: 48.7749
• Rouge2: 26.3665
• Rougel: 35.7765
• Rougelsum: 38.0111

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 7
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
0.6784	1.0	2312	0.5136	46.7374	23.3000	33.5379	35.8923
0.6015	2.0	4624	0.4759	47.7112	24.5817	34.4939	36.9831
0.5587	3.0	6936	0.4543	48.4891	25.6592	35.2310	37.5477
0.5128	4.0	9248	0.4405	48.7777	26.0690	35.5187	37.7896
0.4899	5.0	11560	0.4338	48.6758	26.0670	35.5783	37.8850
0.4796	6.0	13872	0.4295	48.8914	26.5018	35.8671	38.1289
0.4671	7.0	16184	0.4282	48.7749	26.3665	35.7765	38.0111

Framework versions

• Transformers 4.17.0
• Pytorch 2.1.2
• Datasets 2.18.0
• Tokenizers 0.15.2