mBART-TextSimp-LT-BatchSize2-lr1e-4

This model is a fine-tuned version of facebook/mbart-large-50 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0962
• Rouge1: 0.76
• Rouge2: 0.6246
• Rougel: 0.7508
• Sacrebleu: 53.9078
• Gen Len: 32.9976

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 4
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Sacrebleu	Gen Len
0.0639	1.0	418	0.0779	0.7012	0.5432	0.6904	43.0798	32.9976
0.0653	2.0	836	0.0732	0.7197	0.5593	0.7091	44.8483	32.9976
0.0327	3.0	1254	0.0726	0.7319	0.5787	0.7206	47.842	32.9976
0.0168	4.0	1672	0.0782	0.7466	0.6031	0.7371	50.9225	32.9976
0.013	5.0	2090	0.0804	0.7507	0.6077	0.7409	51.8293	32.9976
0.0032	6.0	2508	0.0846	0.7606	0.6237	0.7507	53.5224	32.9976
0.0012	7.0	2926	0.0911	0.7597	0.6263	0.751	54.0182	32.9976
0.0012	8.0	3344	0.0962	0.76	0.6246	0.7508	53.9078	32.9976

Framework versions

• Transformers 4.33.0
• Pytorch 2.1.2+cu121
• Datasets 2.14.4
• Tokenizers 0.13.3