mt5-small-finetuned-icelandic-summary

This model is a fine-tuned version of google/mt5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.3019
• Rouge1: 24.5314
• Rouge2: 13.108
• Rougel: 21.8551
• Rougelsum: 22.5551

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
4.2489	1.0	1914	2.6014	17.9396	6.9767	15.0632	15.8916
3.0569	2.0	3828	2.4547	21.317	10.0433	18.6257	19.3813
2.8661	3.0	5742	2.3843	23.6521	12.2829	21.0046	21.7077
2.7543	4.0	7656	2.3642	23.8878	12.7041	21.3321	22.0867
2.682	5.0	9570	2.3379	24.2206	12.9703	21.6077	22.3493
2.6276	6.0	11484	2.2974	24.5156	13.1087	21.8464	22.5609
2.588	7.0	13398	2.3012	24.7653	13.3338	22.0548	22.762
2.5723	8.0	15312	2.3019	24.5314	13.108	21.8551	22.5551

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu118
• Datasets 2.15.0
• Tokenizers 0.15.0