ArabicNewSplits4_FineTuningAraBERT_run2_AugV5_k10_task5_organization

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7588
• Qwk: 0.7225
• Mse: 0.7588
• Rmse: 0.8711

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse	Rmse
No log	0.0625	2	2.5357	-0.0254	2.5357	1.5924
No log	0.125	4	1.7293	0.1181	1.7293	1.3150
No log	0.1875	6	1.9231	-0.0830	1.9231	1.3868
No log	0.25	8	1.7912	-0.0059	1.7912	1.3383
No log	0.3125	10	1.7590	0.2047	1.7590	1.3263
No log	0.375	12	1.7392	0.2098	1.7392	1.3188
No log	0.4375	14	1.6662	0.1605	1.6662	1.2908
No log	0.5	16	1.6797	0.2150	1.6797	1.2960
No log	0.5625	18	1.5914	0.1734	1.5914	1.2615
No log	0.625	20	1.6793	0.3025	1.6793	1.2959
No log	0.6875	22	1.9655	0.2710	1.9655	1.4020
No log	0.75	24	2.3810	0.0868	2.3810	1.5430
No log	0.8125	26	2.3719	0.1592	2.3719	1.5401
No log	0.875	28	1.9483	0.2554	1.9483	1.3958
No log	0.9375	30	1.7168	0.2912	1.7168	1.3103
No log	1.0	32	1.6610	0.3092	1.6610	1.2888
No log	1.0625	34	1.8927	0.2877	1.8927	1.3758
No log	1.125	36	1.9944	0.2331	1.9944	1.4122
No log	1.1875	38	1.8289	0.2452	1.8289	1.3524
No log	1.25	40	1.5212	0.2542	1.5212	1.2334
No log	1.3125	42	1.3196	0.1711	1.3196	1.1487
No log	1.375	44	1.2710	0.1463	1.2710	1.1274
No log	1.4375	46	1.2653	0.1735	1.2653	1.1249
No log	1.5	48	1.3226	0.3388	1.3226	1.1500
No log	1.5625	50	1.3798	0.3933	1.3798	1.1746
No log	1.625	52	1.3424	0.3898	1.3424	1.1586
No log	1.6875	54	1.3321	0.3807	1.3321	1.1542
No log	1.75	56	1.2906	0.3904	1.2906	1.1361
No log	1.8125	58	1.2945	0.3904	1.2945	1.1378
No log	1.875	60	1.3487	0.3811	1.3487	1.1614
No log	1.9375	62	1.2989	0.4016	1.2989	1.1397
No log	2.0	64	1.2055	0.3972	1.2055	1.0980
No log	2.0625	66	1.2339	0.4170	1.2339	1.1108
No log	2.125	68	1.1816	0.4571	1.1816	1.0870
No log	2.1875	70	1.2425	0.4403	1.2425	1.1147
No log	2.25	72	1.2293	0.4594	1.2293	1.1087
No log	2.3125	74	1.1207	0.5294	1.1207	1.0586
No log	2.375	76	1.1419	0.5373	1.1419	1.0686
No log	2.4375	78	1.0901	0.5832	1.0901	1.0441
No log	2.5	80	0.9676	0.5893	0.9676	0.9837
No log	2.5625	82	0.8449	0.6099	0.8449	0.9192
No log	2.625	84	0.8454	0.6204	0.8454	0.9195
No log	2.6875	86	0.8830	0.6389	0.8830	0.9397
No log	2.75	88	1.0376	0.5969	1.0376	1.0186
No log	2.8125	90	1.0848	0.5951	1.0848	1.0415
No log	2.875	92	1.1660	0.5939	1.1660	1.0798
No log	2.9375	94	0.9577	0.6049	0.9577	0.9786
No log	3.0	96	0.8169	0.6368	0.8169	0.9038
No log	3.0625	98	0.8927	0.6194	0.8927	0.9448
No log	3.125	100	0.9282	0.5955	0.9282	0.9634
No log	3.1875	102	0.9463	0.6001	0.9463	0.9728
No log	3.25	104	1.1056	0.5813	1.1056	1.0515
No log	3.3125	106	1.4970	0.4779	1.4970	1.2235
No log	3.375	108	1.6234	0.4699	1.6234	1.2741
No log	3.4375	110	1.4349	0.5088	1.4349	1.1979
No log	3.5	112	1.1011	0.5737	1.1011	1.0493
No log	3.5625	114	0.8388	0.6991	0.8388	0.9159
No log	3.625	116	0.7753	0.7419	0.7753	0.8805
No log	3.6875	118	0.7992	0.7106	0.7992	0.8940
No log	3.75	120	0.8822	0.6315	0.8822	0.9392
No log	3.8125	122	0.9659	0.5608	0.9659	0.9828
No log	3.875	124	0.9292	0.6214	0.9292	0.9639
No log	3.9375	126	0.8328	0.6734	0.8328	0.9126
No log	4.0	128	0.7680	0.6970	0.7680	0.8764
No log	4.0625	130	0.7998	0.6861	0.7998	0.8943
No log	4.125	132	0.8435	0.6672	0.8435	0.9184
No log	4.1875	134	0.8521	0.6946	0.8521	0.9231
No log	4.25	136	0.8091	0.7065	0.8091	0.8995
No log	4.3125	138	0.7862	0.7216	0.7862	0.8867
No log	4.375	140	0.7394	0.7184	0.7394	0.8599
No log	4.4375	142	0.7762	0.7130	0.7762	0.8810
No log	4.5	144	0.8801	0.7050	0.8801	0.9381
No log	4.5625	146	0.8601	0.7160	0.8601	0.9274
No log	4.625	148	0.7410	0.7240	0.7410	0.8608
No log	4.6875	150	0.6922	0.6997	0.6922	0.8320
No log	4.75	152	0.6807	0.6995	0.6807	0.8250
No log	4.8125	154	0.6605	0.7047	0.6605	0.8127
No log	4.875	156	0.6577	0.7134	0.6577	0.8110
No log	4.9375	158	0.7246	0.6955	0.7246	0.8512
No log	5.0	160	0.8781	0.7056	0.8781	0.9371
No log	5.0625	162	0.9203	0.6823	0.9203	0.9593
No log	5.125	164	0.8994	0.6822	0.8994	0.9484
No log	5.1875	166	0.8389	0.7124	0.8389	0.9159
No log	5.25	168	0.7962	0.7077	0.7962	0.8923
No log	5.3125	170	0.7345	0.6980	0.7345	0.8570
No log	5.375	172	0.7595	0.7202	0.7595	0.8715
No log	5.4375	174	0.8421	0.7030	0.8421	0.9177
No log	5.5	176	0.9629	0.6721	0.9629	0.9813
No log	5.5625	178	1.0423	0.6430	1.0423	1.0209
No log	5.625	180	1.0827	0.6197	1.0827	1.0405
No log	5.6875	182	0.9818	0.6770	0.9818	0.9909
No log	5.75	184	0.8586	0.6911	0.8586	0.9266
No log	5.8125	186	0.8434	0.6948	0.8434	0.9184
No log	5.875	188	0.9379	0.6689	0.9379	0.9685
No log	5.9375	190	1.1064	0.6187	1.1064	1.0519
No log	6.0	192	1.1802	0.5888	1.1802	1.0864
No log	6.0625	194	1.1166	0.6074	1.1166	1.0567
No log	6.125	196	1.0113	0.6622	1.0113	1.0056
No log	6.1875	198	0.8615	0.6769	0.8615	0.9281
No log	6.25	200	0.8043	0.6962	0.8043	0.8968
No log	6.3125	202	0.8213	0.6864	0.8213	0.9063
No log	6.375	204	0.8022	0.7086	0.8022	0.8956
No log	6.4375	206	0.8165	0.7069	0.8165	0.9036
No log	6.5	208	0.8904	0.7030	0.8904	0.9436
No log	6.5625	210	1.0109	0.6413	1.0109	1.0054
No log	6.625	212	1.0596	0.6346	1.0596	1.0294
No log	6.6875	214	0.9962	0.6540	0.9961	0.9981
No log	6.75	216	0.8739	0.6974	0.8739	0.9348
No log	6.8125	218	0.7943	0.7134	0.7943	0.8912
No log	6.875	220	0.7889	0.7073	0.7889	0.8882
No log	6.9375	222	0.8277	0.6969	0.8277	0.9098
No log	7.0	224	0.8936	0.6850	0.8936	0.9453
No log	7.0625	226	0.9575	0.6648	0.9575	0.9785
No log	7.125	228	0.9880	0.6387	0.9880	0.9940
No log	7.1875	230	0.9775	0.6361	0.9775	0.9887
No log	7.25	232	0.9446	0.6441	0.9446	0.9719
No log	7.3125	234	0.8695	0.6837	0.8695	0.9325
No log	7.375	236	0.8186	0.6849	0.8186	0.9047
No log	7.4375	238	0.7506	0.7086	0.7506	0.8664
No log	7.5	240	0.7290	0.7252	0.7290	0.8538
No log	7.5625	242	0.7549	0.7066	0.7549	0.8688
No log	7.625	244	0.8199	0.6948	0.8199	0.9055
No log	7.6875	246	0.8839	0.6832	0.8839	0.9402
No log	7.75	248	0.8888	0.6731	0.8888	0.9428
No log	7.8125	250	0.8587	0.6776	0.8587	0.9266
No log	7.875	252	0.8131	0.6938	0.8131	0.9017
No log	7.9375	254	0.7631	0.6851	0.7631	0.8736
No log	8.0	256	0.7245	0.7078	0.7245	0.8511
No log	8.0625	258	0.6951	0.7147	0.6951	0.8338
No log	8.125	260	0.6945	0.7343	0.6945	0.8334
No log	8.1875	262	0.7129	0.7100	0.7129	0.8443
No log	8.25	264	0.7434	0.6984	0.7434	0.8622
No log	8.3125	266	0.7639	0.6851	0.7639	0.8740
No log	8.375	268	0.7820	0.6938	0.7820	0.8843
No log	8.4375	270	0.7944	0.6938	0.7944	0.8913
No log	8.5	272	0.8176	0.6864	0.8176	0.9042
No log	8.5625	274	0.8148	0.6948	0.8148	0.9027
No log	8.625	276	0.8070	0.7046	0.8070	0.8983
No log	8.6875	278	0.8077	0.7049	0.8077	0.8987
No log	8.75	280	0.8106	0.7053	0.8106	0.9003
No log	8.8125	282	0.7984	0.7053	0.7984	0.8935
No log	8.875	284	0.7794	0.7053	0.7794	0.8828
No log	8.9375	286	0.7711	0.7053	0.7711	0.8781
No log	9.0	288	0.7693	0.7149	0.7693	0.8771
No log	9.0625	290	0.7758	0.7132	0.7758	0.8808
No log	9.125	292	0.7952	0.7132	0.7952	0.8917
No log	9.1875	294	0.8044	0.7053	0.8044	0.8969
No log	9.25	296	0.8008	0.7053	0.8008	0.8949
No log	9.3125	298	0.7889	0.7053	0.7889	0.8882
No log	9.375	300	0.7722	0.7053	0.7722	0.8787
No log	9.4375	302	0.7526	0.7225	0.7526	0.8675
No log	9.5	304	0.7381	0.7225	0.7381	0.8592
No log	9.5625	306	0.7370	0.7225	0.7370	0.8585
No log	9.625	308	0.7371	0.7225	0.7371	0.8586
No log	9.6875	310	0.7368	0.7225	0.7368	0.8584
No log	9.75	312	0.7420	0.7225	0.7420	0.8614
No log	9.8125	314	0.7487	0.7225	0.7487	0.8652
No log	9.875	316	0.7540	0.7225	0.7540	0.8683
No log	9.9375	318	0.7578	0.7225	0.7578	0.8705
No log	10.0	320	0.7588	0.7225	0.7588	0.8711

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.0+cu118
• Datasets 2.21.0
• Tokenizers 0.19.1