results

This model is a fine-tuned version of riotu-lab/ArabianGPT-01B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.9084
• Bleu: 0.3172
• Rouge1: 0.5869
• Rouge2: 0.3505
• Rougel: 0.5504

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 20.0

Training results

Training Loss	Epoch	Step	Bleu	Validation Loss	Rouge1	Rouge2	Rougel
3.359	1.5674	500	0.1142	3.1283	0.3298	0.0843	0.2561
2.9208	3.1348	1000	0.1491	2.7298	0.4041	0.1430	0.3408
2.619	4.7022	1500	0.1607	2.6229	0.4264	0.1631	0.3675
2.4047	4.3384	2000	2.2002	0.2721	0.4976	0.2542	0.4506
2.19	5.4230	2500	2.0992	0.2854	0.5205	0.2788	0.4773
2.0473	6.5076	3000	2.0362	0.2929	0.5381	0.2965	0.4965
1.9397	7.5922	3500	1.9933	0.2996	0.5494	0.3103	0.5102
1.857	8.6768	4000	1.9647	0.3024	0.5598	0.3191	0.5203
1.784	9.7614	4500	1.9443	0.3062	0.5675	0.3269	0.5279
1.7239	10.8460	5000	1.9320	0.3099	0.5724	0.3339	0.5341
1.6713	11.9306	5500	1.9206	0.3116	0.5765	0.3383	0.5387
1.6263	13.0152	6000	1.9168	0.3127	0.5781	0.3416	0.5416
1.5869	14.0998	6500	1.9148	0.3137	0.5829	0.3448	0.5451
1.5544	15.1844	7000	1.9121	0.3158	0.5845	0.3476	0.5476
1.5307	16.2690	7500	1.9106	0.3165	0.5853	0.3488	0.5486
1.5087	17.3536	8000	1.9093	0.3169	0.5861	0.3504	0.5500
1.4937	18.4382	8500	1.9084	0.3172	0.5869	0.3505	0.5504
1.4824	19.5228	9000	1.9086	0.3178	0.5876	0.3513	0.5510

Framework versions

• Transformers 4.45.0.dev0
• Pytorch 2.3.1+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1