arabert_baseline_grammar_task7_fold1

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5050
• Qwk: 0.6844
• Mse: 0.4943

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.3333	2	1.0262	0.2825	1.0304
No log	0.6667	4	0.6994	0.6208	0.7254
No log	1.0	6	0.8144	0.4225	0.8374
No log	1.3333	8	0.6492	0.5810	0.6643
No log	1.6667	10	0.6577	0.4	0.6679
No log	2.0	12	0.5578	0.5957	0.5665
No log	2.3333	14	0.5481	0.6030	0.5532
No log	2.6667	16	0.4901	0.6290	0.4927
No log	3.0	18	0.4569	0.6780	0.4570
No log	3.3333	20	0.4974	0.6780	0.4927
No log	3.6667	22	0.5556	0.5326	0.5459
No log	4.0	24	0.5207	0.6370	0.5092
No log	4.3333	26	0.5155	0.6370	0.5024
No log	4.6667	28	0.5426	0.6370	0.5277
No log	5.0	30	0.5586	0.6416	0.5432
No log	5.3333	32	0.5285	0.6844	0.5150
No log	5.6667	34	0.4857	0.6844	0.4750
No log	6.0	36	0.4315	0.6780	0.4244
No log	6.3333	38	0.4483	0.6780	0.4406
No log	6.6667	40	0.5122	0.6844	0.5019
No log	7.0	42	0.5586	0.6283	0.5467
No log	7.3333	44	0.5142	0.6844	0.5032
No log	7.6667	46	0.4644	0.6844	0.4552
No log	8.0	48	0.4307	0.6780	0.4234
No log	8.3333	50	0.4377	0.6780	0.4300
No log	8.6667	52	0.4662	0.6780	0.4571
No log	9.0	54	0.4977	0.6844	0.4873
No log	9.3333	56	0.5069	0.6844	0.4961
No log	9.6667	58	0.5095	0.6844	0.4985
No log	10.0	60	0.5050	0.6844	0.4943

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1