arabert_cross_relevance_task5_fold2

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3102
• Qwk: 0.0
• Mse: 0.3106

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.125	2	0.8487	0.0102	0.8475
No log	0.25	4	0.3505	0.1091	0.3504
No log	0.375	6	0.4480	0.0599	0.4484
No log	0.5	8	0.3079	0.0534	0.3080
No log	0.625	10	0.3015	0.0245	0.3018
No log	0.75	12	0.3131	0.0	0.3134
No log	0.875	14	0.3140	0.0	0.3143
No log	1.0	16	0.3539	-0.0164	0.3543
No log	1.125	18	0.3562	-0.0164	0.3567
No log	1.25	20	0.3449	0.0	0.3454
No log	1.375	22	0.3408	0.0	0.3412
No log	1.5	24	0.3416	0.0	0.3421
No log	1.625	26	0.3321	0.0	0.3325
No log	1.75	28	0.3068	0.0	0.3072
No log	1.875	30	0.3135	0.0	0.3139
No log	2.0	32	0.3564	0.0122	0.3570
No log	2.125	34	0.3393	0.0	0.3399
No log	2.25	36	0.2992	0.0	0.2996
No log	2.375	38	0.3002	0.0	0.3005
No log	2.5	40	0.3396	0.0122	0.3402
No log	2.625	42	0.3703	0.0368	0.3710
No log	2.75	44	0.3599	0.0203	0.3605
No log	2.875	46	0.3243	0.0	0.3248
No log	3.0	48	0.3164	0.0	0.3169
No log	3.125	50	0.3455	-0.0208	0.3460
No log	3.25	52	0.3796	-0.0707	0.3802
No log	3.375	54	0.3560	-0.0495	0.3565
No log	3.5	56	0.3121	0.0	0.3125
No log	3.625	58	0.2932	0.0	0.2935
No log	3.75	60	0.2950	0.0	0.2953
No log	3.875	62	0.3159	0.0	0.3163
No log	4.0	64	0.3268	0.0	0.3273
No log	4.125	66	0.3199	0.0	0.3203
No log	4.25	68	0.3055	0.0	0.3059
No log	4.375	70	0.3033	0.0	0.3037
No log	4.5	72	0.3090	0.0	0.3094
No log	4.625	74	0.3329	0.0	0.3333
No log	4.75	76	0.3437	0.0	0.3442
No log	4.875	78	0.3238	0.0	0.3242
No log	5.0	80	0.2995	0.0	0.2998
No log	5.125	82	0.2952	0.0	0.2954
No log	5.25	84	0.3110	0.0122	0.3113
No log	5.375	86	0.3451	-0.0329	0.3455
No log	5.5	88	0.3521	-0.0373	0.3526
No log	5.625	90	0.3417	-0.0329	0.3422
No log	5.75	92	0.3246	0.0122	0.3250
No log	5.875	94	0.3260	0.0122	0.3264
No log	6.0	96	0.3312	0.0	0.3316
No log	6.125	98	0.3222	0.0	0.3226
No log	6.25	100	0.3187	0.0	0.3191
No log	6.375	102	0.3168	0.0122	0.3171
No log	6.5	104	0.3231	0.0122	0.3235
No log	6.625	106	0.3288	0.0122	0.3292
No log	6.75	108	0.3225	0.0122	0.3229
No log	6.875	110	0.3117	0.0122	0.3120
No log	7.0	112	0.3032	0.0	0.3034
No log	7.125	114	0.3022	0.0	0.3024
No log	7.25	116	0.3089	0.0122	0.3091
No log	7.375	118	0.3150	0.0122	0.3153
No log	7.5	120	0.3157	0.0122	0.3161
No log	7.625	122	0.3102	0.0122	0.3106
No log	7.75	124	0.3060	0.0	0.3063
No log	7.875	126	0.3061	0.0	0.3063
No log	8.0	128	0.3089	0.0122	0.3092
No log	8.125	130	0.3108	0.0122	0.3111
No log	8.25	132	0.3154	0.0122	0.3158
No log	8.375	134	0.3210	0.0122	0.3215
No log	8.5	136	0.3202	0.0122	0.3206
No log	8.625	138	0.3172	0.0122	0.3176
No log	8.75	140	0.3139	0.0	0.3143
No log	8.875	142	0.3121	0.0	0.3125
No log	9.0	144	0.3115	0.0	0.3118
No log	9.125	146	0.3118	0.0	0.3121
No log	9.25	148	0.3121	0.0	0.3124
No log	9.375	150	0.3120	0.0	0.3124
No log	9.5	152	0.3116	0.0	0.3119
No log	9.625	154	0.3110	0.0	0.3114
No log	9.75	156	0.3106	0.0	0.3109
No log	9.875	158	0.3103	0.0	0.3106
No log	10.0	160	0.3102	0.0	0.3106

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1