ArabDarija-V2

This model is a fine-tuned version of moussaKam/arabart on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.1950
• Bleu: 11.1256
• Gen Len: 10.23

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5.0

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
1.9806	0.22	200	2.8668	2.7224	10.2592
1.4782	0.43	400	2.6032	4.3628	10.0648
1.3116	0.65	600	2.5110	5.4138	10.5715
1.2438	0.86	800	2.4102	5.8161	10.2642
1.1213	1.08	1000	2.3553	6.8525	10.6232
0.982	1.3	1200	2.3270	7.1567	9.866
0.9242	1.51	1400	2.2670	7.9874	10.1892
0.918	1.73	1600	2.2573	8.1463	10.2868
0.9015	1.95	1800	2.2233	8.6499	10.4845
0.7779	2.16	2000	2.2362	8.5161	10.27
0.7352	2.38	2200	2.2090	9.5199	10.3112
0.7463	2.59	2400	2.2128	9.4998	10.2325
0.7317	2.81	2600	2.1775	10.1806	10.4035
0.71	3.03	2800	2.2113	9.9933	10.0615
0.6318	3.24	3000	2.1860	10.3507	10.22
0.6048	3.46	3200	2.1888	10.433	10.1788
0.6223	3.68	3400	2.1889	10.6846	10.122
0.595	3.89	3600	2.1748	10.581	10.1245
0.5627	4.11	3800	2.2142	10.8136	10.1905
0.5404	4.32	4000	2.2090	11.0132	10.2552
0.532	4.54	4200	2.2074	10.8431	10.1888
0.5533	4.76	4400	2.1968	11.0447	10.239
0.5357	4.97	4600	2.1950	11.1256	10.23

Framework versions

• Transformers 4.39.0.dev0
• Pytorch 2.1.0+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2