xlm-roberta-base-finetuned-panx-en

This model is a fine-tuned version of xlm-roberta-base on the xtreme dataset. It achieves the following results on the evaluation set:

• Loss: 0.2487
• F1: 0.8237
• Precision: 0.8163
• Recall: 0.8311
• Accuracy: 0.8237

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 24
• eval_batch_size: 24
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	F1	Precision	Recall	Accuracy
1.2644	0.03	24	0.8175	0.4212	0.3696	0.4897	0.4212
0.7209	0.06	48	0.5633	0.4817	0.4190	0.5665	0.4817
0.5951	0.09	72	0.4670	0.6059	0.5588	0.6617	0.6059
0.4475	0.12	96	0.4425	0.6659	0.6336	0.7016	0.6659
0.4978	0.14	120	0.4469	0.6375	0.5930	0.6892	0.6375
0.4383	0.17	144	0.4093	0.7003	0.6668	0.7374	0.7003
0.4148	0.2	168	0.3688	0.7122	0.6877	0.7387	0.7122
0.4513	0.23	192	0.3700	0.7236	0.7081	0.7397	0.7236
0.3786	0.26	216	0.3666	0.7304	0.7125	0.7493	0.7304
0.425	0.29	240	0.3652	0.7046	0.6874	0.7227	0.7046
0.4014	0.32	264	0.3438	0.7246	0.6964	0.7552	0.7246
0.3789	0.35	288	0.3533	0.7208	0.6922	0.7519	0.7208
0.4032	0.37	312	0.3567	0.7252	0.7125	0.7383	0.7252
0.371	0.4	336	0.3282	0.7433	0.7255	0.7620	0.7433
0.3397	0.43	360	0.3304	0.7522	0.7312	0.7745	0.7522
0.3871	0.46	384	0.3244	0.7427	0.7160	0.7715	0.7427
0.3461	0.49	408	0.3284	0.7520	0.7298	0.7756	0.7520
0.3504	0.52	432	0.3049	0.7574	0.7418	0.7737	0.7574
0.3387	0.55	456	0.3178	0.7717	0.7537	0.7906	0.7717
0.3259	0.58	480	0.3026	0.7738	0.7636	0.7843	0.7738
0.3473	0.6	504	0.3254	0.7324	0.7090	0.7574	0.7324
0.2893	0.63	528	0.3102	0.7689	0.7571	0.7810	0.7689
0.3669	0.66	552	0.3119	0.7631	0.7528	0.7737	0.7631
0.312	0.69	576	0.2963	0.7818	0.7734	0.7905	0.7818
0.297	0.72	600	0.3217	0.7542	0.7332	0.7765	0.7542
0.3095	0.75	624	0.3038	0.7732	0.7580	0.7891	0.7732
0.3514	0.78	648	0.2913	0.7794	0.7669	0.7924	0.7794
0.2824	0.81	672	0.3008	0.7813	0.7752	0.7876	0.7813
0.3203	0.83	696	0.2915	0.7807	0.7641	0.7980	0.7807
0.3089	0.86	720	0.2941	0.7838	0.7755	0.7923	0.7838
0.3174	0.89	744	0.2986	0.7770	0.7609	0.7937	0.7770
0.3264	0.92	768	0.2783	0.7788	0.7630	0.7951	0.7788
0.2815	0.95	792	0.2861	0.7848	0.7704	0.7998	0.7848
0.2895	0.98	816	0.2799	0.7842	0.7702	0.7988	0.7842
0.3023	1.01	840	0.2818	0.7876	0.7722	0.8038	0.7876
0.2358	1.04	864	0.2924	0.7836	0.7750	0.7925	0.7836
0.2819	1.06	888	0.2861	0.7761	0.7696	0.7828	0.7761
0.2692	1.09	912	0.2924	0.7756	0.7680	0.7833	0.7756
0.2478	1.12	936	0.2963	0.7833	0.7599	0.8082	0.7833
0.2557	1.15	960	0.2960	0.7783	0.7814	0.7751	0.7783
0.3003	1.18	984	0.2656	0.7862	0.7727	0.8002	0.7862
0.2254	1.21	1008	0.2791	0.8007	0.7890	0.8129	0.8007
0.2496	1.24	1032	0.2702	0.7877	0.7701	0.8062	0.7877
0.2124	1.27	1056	0.2888	0.7952	0.7895	0.8011	0.7952
0.2841	1.29	1080	0.2761	0.7946	0.7870	0.8023	0.7946
0.2517	1.32	1104	0.2659	0.8026	0.7909	0.8146	0.8026
0.2355	1.35	1128	0.2681	0.8003	0.7876	0.8134	0.8003
0.2402	1.38	1152	0.2701	0.7991	0.7892	0.8093	0.7991
0.2296	1.41	1176	0.2753	0.7946	0.7819	0.8077	0.7946
0.2453	1.44	1200	0.2696	0.8029	0.7912	0.8149	0.8029
0.2689	1.47	1224	0.2700	0.7936	0.7819	0.8056	0.7936
0.2362	1.5	1248	0.2705	0.8028	0.8005	0.8051	0.8028
0.226	1.53	1272	0.2642	0.8042	0.7910	0.8180	0.8042
0.2139	1.55	1296	0.2690	0.8013	0.7942	0.8084	0.8013
0.2744	1.58	1320	0.2619	0.7999	0.7841	0.8163	0.7999
0.2015	1.61	1344	0.2640	0.8066	0.8035	0.8098	0.8066
0.1949	1.64	1368	0.2750	0.8075	0.8023	0.8129	0.8075
0.2259	1.67	1392	0.2669	0.8092	0.7997	0.8189	0.8092
0.1884	1.7	1416	0.2729	0.8061	0.7990	0.8133	0.8061
0.1868	1.73	1440	0.2679	0.8083	0.8007	0.8161	0.8083
0.2292	1.76	1464	0.2658	0.8055	0.7954	0.8158	0.8055
0.22	1.78	1488	0.2610	0.8066	0.8006	0.8126	0.8066
0.2335	1.81	1512	0.2613	0.7997	0.7816	0.8185	0.7997
0.2379	1.84	1536	0.2495	0.8081	0.7975	0.8190	0.8081
0.2394	1.87	1560	0.2619	0.8063	0.7951	0.8177	0.8063
0.2526	1.9	1584	0.2502	0.8116	0.8032	0.8202	0.8116
0.2167	1.93	1608	0.2528	0.8134	0.8000	0.8273	0.8134
0.2354	1.96	1632	0.2449	0.8099	0.8013	0.8188	0.8099
0.2808	1.99	1656	0.2469	0.8067	0.7938	0.8201	0.8067
0.1924	2.01	1680	0.2487	0.8077	0.7930	0.8229	0.8077
0.1498	2.04	1704	0.2619	0.8127	0.8015	0.8242	0.8127
0.2	2.07	1728	0.2590	0.8133	0.8044	0.8224	0.8133
0.151	2.1	1752	0.2623	0.8066	0.7949	0.8186	0.8066
0.1646	2.13	1776	0.2632	0.8186	0.8137	0.8236	0.8186
0.1659	2.16	1800	0.2561	0.8188	0.8096	0.8281	0.8188
0.1888	2.19	1824	0.2549	0.8136	0.8038	0.8237	0.8136
0.2084	2.22	1848	0.2557	0.8141	0.8087	0.8197	0.8141
0.1571	2.24	1872	0.2697	0.8150	0.8053	0.8249	0.8150
0.1541	2.27	1896	0.2605	0.8191	0.8121	0.8262	0.8191
0.1586	2.3	1920	0.2742	0.8109	0.8073	0.8144	0.8109
0.1641	2.33	1944	0.2679	0.8148	0.8104	0.8193	0.8148
0.1914	2.36	1968	0.2596	0.8159	0.8056	0.8265	0.8159
0.1441	2.39	1992	0.2644	0.8183	0.8139	0.8226	0.8183
0.1672	2.42	2016	0.2652	0.8180	0.8081	0.8281	0.8180
0.1852	2.45	2040	0.2576	0.8205	0.8101	0.8313	0.8205
0.192	2.47	2064	0.2459	0.8179	0.8063	0.8298	0.8179
0.1698	2.5	2088	0.2482	0.8213	0.8149	0.8277	0.8213
0.1802	2.53	2112	0.2519	0.8155	0.8066	0.8247	0.8155
0.1619	2.56	2136	0.2582	0.8175	0.8036	0.8319	0.8175
0.1974	2.59	2160	0.2535	0.8184	0.8108	0.8261	0.8184
0.1655	2.62	2184	0.2514	0.8229	0.8165	0.8295	0.8229
0.1844	2.65	2208	0.2536	0.8208	0.8152	0.8264	0.8208
0.1601	2.68	2232	0.2531	0.8194	0.8104	0.8286	0.8194
0.161	2.71	2256	0.2508	0.8226	0.8145	0.8310	0.8226
0.1672	2.73	2280	0.2527	0.8216	0.8137	0.8296	0.8216
0.2053	2.76	2304	0.2482	0.8208	0.8112	0.8306	0.8208
0.1776	2.79	2328	0.2486	0.8215	0.8143	0.8288	0.8215
0.1559	2.82	2352	0.2495	0.8233	0.8156	0.8312	0.8233
0.1509	2.85	2376	0.2472	0.8231	0.8142	0.8322	0.8231
0.1695	2.88	2400	0.2465	0.8229	0.8134	0.8326	0.8229
0.1523	2.91	2424	0.2466	0.8234	0.8154	0.8315	0.8234
0.1525	2.94	2448	0.2478	0.8241	0.8165	0.8319	0.8241
0.1386	2.96	2472	0.2486	0.8236	0.8164	0.8309	0.8236
0.1532	2.99	2496	0.2487	0.8237	0.8163	0.8311	0.8237

Framework versions

• Transformers 4.30.2
• Pytorch 2.0.1+cu118
• Datasets 2.13.0
• Tokenizers 0.13.3