Arabic_ATS_AraT5_AraSum__SFT_AraT5

This model is a fine-tuned version of UBC-NLP/AraT5v2-base-1024 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.4120
• Rouge-1: 30.4804
• Rouge-2: 13.6908
• Rouge-l: 24.7064

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 1400
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge-1	Rouge-2	Rouge-l
2.4578	1.0	5588	2.4719	29.8793	13.2857	24.1849
2.5548	2.0	11176	2.4583	30.0704	13.3697	24.3165
2.6228	3.0	16764	2.4103	30.3178	13.4762	24.5775
2.5009	4.0	22352	2.4131	30.492	13.6744	24.6939
2.5212	5.0	27940	2.4120	30.4804	13.6908	24.7064

Framework versions

• Transformers 4.40.1
• Pytorch 2.2.0+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1