finetuning-summarization-model

This model is a fine-tuned version of google/mt5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.3028
• Rouge1: 29.1184
• Rouge2: 21.1309
• Rougel: 28.3412
• Rougelsum: 28.4871

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
6.821	1.0	450	1.7464	31.7328	21.1788	30.3949	30.5202
2.1307	2.0	900	1.4939	31.3208	22.0215	30.2589	30.3872
1.7915	3.0	1350	1.4322	28.7824	19.472	27.926	28.2177
1.6186	4.0	1800	1.3830	29.2568	20.6076	28.4825	28.6486
1.5148	5.0	2250	1.3504	29.308	21.0698	28.4755	28.6885
1.427	6.0	2700	1.3177	29.0294	20.706	28.271	28.3385
1.3793	7.0	3150	1.3172	28.9276	20.922	28.1795	28.3241
1.3536	8.0	3600	1.3028	29.1184	21.1309	28.3412	28.4871

Framework versions

• Transformers 4.28.1
• Pytorch 2.0.0+cu118
• Datasets 2.11.0
• Tokenizers 0.13.3