README.md

metadata

base_model: aubmindlab/bert-base-arabertv02
tags:
  - generated_from_trainer
model-index:
  - name: arabert_cross_relevance_task1_fold4
    results: []

arabert_cross_relevance_task1_fold4

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2462
• Qwk: 0.2970
• Mse: 0.2462

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.125	2	1.1439	0.0152	1.1439
No log	0.25	4	0.4542	0.1058	0.4542
No log	0.375	6	0.3468	0.2290	0.3468
No log	0.5	8	0.3648	0.1816	0.3648
No log	0.625	10	0.4917	0.2054	0.4917
No log	0.75	12	0.3361	0.2720	0.3361
No log	0.875	14	0.2581	0.2606	0.2581
No log	1.0	16	0.2596	0.3452	0.2596
No log	1.125	18	0.2652	0.3776	0.2652
No log	1.25	20	0.2495	0.3605	0.2495
No log	1.375	22	0.2883	0.2801	0.2883
No log	1.5	24	0.3460	0.2241	0.3460
No log	1.625	26	0.3144	0.2483	0.3144
No log	1.75	28	0.2568	0.3617	0.2568
No log	1.875	30	0.2645	0.3732	0.2645
No log	2.0	32	0.2745	0.3354	0.2745
No log	2.125	34	0.2642	0.3416	0.2642
No log	2.25	36	0.2371	0.3323	0.2371
No log	2.375	38	0.2288	0.3278	0.2288
No log	2.5	40	0.2336	0.3517	0.2336
No log	2.625	42	0.2327	0.3836	0.2327
No log	2.75	44	0.2341	0.4092	0.2341
No log	2.875	46	0.2410	0.3449	0.2410
No log	3.0	48	0.2695	0.3349	0.2695
No log	3.125	50	0.2860	0.2593	0.2860
No log	3.25	52	0.2584	0.2899	0.2584
No log	3.375	54	0.2408	0.3216	0.2408
No log	3.5	56	0.2232	0.3190	0.2232
No log	3.625	58	0.2179	0.3172	0.2179
No log	3.75	60	0.2229	0.3029	0.2229
No log	3.875	62	0.2274	0.2855	0.2274
No log	4.0	64	0.2344	0.2787	0.2344
No log	4.125	66	0.2449	0.2616	0.2449
No log	4.25	68	0.2478	0.2753	0.2478
No log	4.375	70	0.2462	0.3017	0.2462
No log	4.5	72	0.2513	0.3343	0.2513
No log	4.625	74	0.2528	0.3607	0.2528
No log	4.75	76	0.2439	0.3638	0.2439
No log	4.875	78	0.2300	0.3536	0.2300
No log	5.0	80	0.2244	0.3127	0.2244
No log	5.125	82	0.2243	0.3034	0.2243
No log	5.25	84	0.2290	0.2891	0.2290
No log	5.375	86	0.2240	0.3203	0.2240
No log	5.5	88	0.2237	0.3522	0.2237
No log	5.625	90	0.2245	0.3408	0.2245
No log	5.75	92	0.2258	0.3271	0.2258
No log	5.875	94	0.2291	0.3021	0.2291
No log	6.0	96	0.2407	0.2882	0.2407
No log	6.125	98	0.2527	0.2783	0.2527
No log	6.25	100	0.2560	0.2686	0.2560
No log	6.375	102	0.2524	0.2882	0.2524
No log	6.5	104	0.2478	0.3136	0.2478
No log	6.625	106	0.2439	0.3199	0.2439
No log	6.75	108	0.2489	0.3136	0.2489
No log	6.875	110	0.2507	0.3033	0.2507
No log	7.0	112	0.2439	0.3109	0.2439
No log	7.125	114	0.2358	0.3238	0.2358
No log	7.25	116	0.2350	0.3238	0.2350
No log	7.375	118	0.2450	0.3069	0.2450
No log	7.5	120	0.2670	0.2709	0.2670
No log	7.625	122	0.2757	0.2529	0.2757
No log	7.75	124	0.2626	0.2709	0.2626
No log	7.875	126	0.2540	0.2774	0.2540
No log	8.0	128	0.2430	0.2904	0.2430
No log	8.125	130	0.2346	0.2878	0.2346
No log	8.25	132	0.2324	0.2981	0.2324
No log	8.375	134	0.2325	0.2981	0.2325
No log	8.5	136	0.2349	0.2878	0.2349
No log	8.625	138	0.2374	0.3005	0.2374
No log	8.75	140	0.2415	0.2942	0.2415
No log	8.875	142	0.2456	0.2942	0.2456
No log	9.0	144	0.2483	0.2942	0.2483
No log	9.125	146	0.2484	0.2942	0.2484
No log	9.25	148	0.2480	0.2970	0.2480
No log	9.375	150	0.2488	0.2970	0.2488
No log	9.5	152	0.2480	0.2970	0.2480
No log	9.625	154	0.2465	0.2970	0.2465
No log	9.75	156	0.2462	0.2942	0.2462
No log	9.875	158	0.2463	0.2970	0.2463
No log	10.0	160	0.2462	0.2970	0.2462

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1