sembr2023-distilbert-base-multilingual-cased

This model is a fine-tuned version of distilbert-base-multilingual-cased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2282
• Precision: 0.7986
• Recall: 0.8244
• F1: 0.8113
• Iou: 0.6825
• Accuracy: 0.9666
• Balanced Accuracy: 0.9023
• Overall Accuracy: 0.9521

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 64
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Iou	Accuracy	Balanced Accuracy	Overall Accuracy
0.4021	0.06	10	0.3642	0	0.0	0.0	0.0	0.9130	0.5	0.9130
0.2804	0.12	20	0.2550	0.7462	0.5589	0.6391	0.4696	0.9451	0.7704	0.9331
0.2173	0.18	30	0.2038	0.9157	0.4420	0.5962	0.4247	0.9479	0.7191	0.9406
0.1831	0.24	40	0.1824	0.8317	0.7186	0.7710	0.6274	0.9629	0.8524	0.9455
0.1816	0.29	50	0.1829	0.7540	0.7857	0.7695	0.6254	0.9591	0.8807	0.9405
0.1461	0.35	60	0.1619	0.8520	0.7210	0.7810	0.6407	0.9648	0.8545	0.9498
0.1424	0.41	70	0.1568	0.8091	0.7717	0.7900	0.6528	0.9643	0.8772	0.9483
0.11	0.47	80	0.1550	0.8189	0.7760	0.7969	0.6624	0.9656	0.8798	0.9504
0.1373	0.53	90	0.1584	0.8248	0.7640	0.7933	0.6573	0.9654	0.8743	0.9484
0.1202	0.59	100	0.1612	0.8292	0.7608	0.7935	0.6577	0.9656	0.8729	0.9496
0.0966	0.65	110	0.1390	0.8575	0.7597	0.8057	0.6746	0.9681	0.8738	0.9551
0.0832	0.71	120	0.1569	0.8292	0.7858	0.8069	0.6763	0.9673	0.8852	0.9508
0.0914	0.76	130	0.1399	0.8295	0.8009	0.8150	0.6877	0.9684	0.8926	0.9529
0.0793	0.82	140	0.1456	0.8161	0.7914	0.8035	0.6716	0.9663	0.8872	0.9512
0.0903	0.88	150	0.1461	0.8218	0.7882	0.8047	0.6732	0.9667	0.8860	0.9523
0.0976	0.94	160	0.1380	0.7999	0.8304	0.8148	0.6875	0.9672	0.9053	0.9533
0.0776	1.0	170	0.1633	0.8053	0.8144	0.8099	0.6805	0.9667	0.8978	0.9492
0.0722	1.06	180	0.1581	0.8569	0.7739	0.8133	0.6853	0.9691	0.8808	0.9528
0.0787	1.12	190	0.1563	0.7780	0.8350	0.8055	0.6743	0.9649	0.9062	0.9495
0.0642	1.18	200	0.1483	0.8007	0.8115	0.8060	0.6751	0.9660	0.8961	0.9527
0.0582	1.24	210	0.1559	0.8244	0.8067	0.8154	0.6884	0.9682	0.8952	0.9534
0.0617	1.29	220	0.1359	0.8158	0.8115	0.8136	0.6858	0.9677	0.8970	0.9560
0.0508	1.35	230	0.1638	0.8372	0.7872	0.8114	0.6827	0.9682	0.8863	0.9548
0.049	1.41	240	0.1699	0.8585	0.7432	0.7967	0.6620	0.9670	0.8657	0.9523
0.0604	1.47	250	0.1452	0.7935	0.8374	0.8148	0.6875	0.9669	0.9083	0.9523
0.0512	1.53	260	0.1551	0.8353	0.7820	0.8078	0.6776	0.9676	0.8837	0.9542
0.0496	1.59	270	0.1686	0.7757	0.8490	0.8107	0.6816	0.9655	0.9128	0.9496
0.0486	1.65	280	0.1474	0.8464	0.7834	0.8137	0.6859	0.9688	0.8849	0.9557
0.0529	1.71	290	0.1470	0.8114	0.8136	0.8125	0.6842	0.9673	0.8978	0.9535
0.0585	1.76	300	0.1625	0.8193	0.8057	0.8124	0.6841	0.9676	0.8944	0.9527
0.0373	1.82	310	0.1414	0.8038	0.8240	0.8138	0.6860	0.9672	0.9024	0.9555
0.0474	1.88	320	0.1600	0.8117	0.8073	0.8095	0.6800	0.9670	0.8947	0.9541
0.0475	1.94	330	0.1825	0.7709	0.8418	0.8048	0.6733	0.9645	0.9090	0.9498
0.0596	2.0	340	0.1688	0.8185	0.8078	0.8131	0.6851	0.9677	0.8954	0.9533
0.0391	2.06	350	0.1775	0.8006	0.8142	0.8073	0.6769	0.9662	0.8974	0.9514
0.0285	2.12	360	0.1653	0.8047	0.8165	0.8105	0.6814	0.9668	0.8988	0.9536
0.0288	2.18	370	0.1855	0.7697	0.8376	0.8022	0.6698	0.9641	0.9069	0.9509
0.0244	2.24	380	0.1813	0.8053	0.8171	0.8112	0.6823	0.9669	0.8991	0.9530
0.0324	2.29	390	0.1663	0.8029	0.8252	0.8139	0.6862	0.9672	0.9030	0.9538
0.0306	2.35	400	0.1692	0.7949	0.8291	0.8117	0.6830	0.9665	0.9044	0.9529
0.0279	2.41	410	0.1812	0.8171	0.8056	0.8113	0.6825	0.9674	0.8942	0.9526
0.0287	2.47	420	0.1768	0.8196	0.8106	0.8151	0.6879	0.9680	0.8968	0.9540
0.0287	2.53	430	0.1849	0.7927	0.8271	0.8095	0.6800	0.9662	0.9032	0.9532
0.0328	2.59	440	0.1765	0.8031	0.8197	0.8113	0.6825	0.9668	0.9003	0.9536
0.0226	2.65	450	0.1928	0.7879	0.8403	0.8133	0.6853	0.9664	0.9094	0.9515
0.0308	2.71	460	0.1905	0.7858	0.8354	0.8098	0.6804	0.9659	0.9069	0.9515
0.0277	2.76	470	0.1890	0.8083	0.8168	0.8125	0.6842	0.9672	0.8992	0.9537
0.0229	2.82	480	0.1802	0.8125	0.8177	0.8151	0.6879	0.9677	0.8999	0.9540
0.0218	2.88	490	0.1895	0.7805	0.8370	0.8078	0.6775	0.9654	0.9073	0.9506
0.0283	2.94	500	0.1926	0.7854	0.8337	0.8088	0.6790	0.9657	0.9060	0.9516
0.0231	3.0	510	0.2023	0.7766	0.8432	0.8085	0.6786	0.9653	0.9101	0.9509
0.0172	3.06	520	0.2051	0.7811	0.8305	0.8051	0.6737	0.9650	0.9042	0.9509
0.0206	3.12	530	0.1918	0.7894	0.8339	0.8110	0.6822	0.9662	0.9064	0.9523
0.0243	3.18	540	0.1982	0.7992	0.8198	0.8094	0.6798	0.9664	0.9001	0.9526
0.0193	3.24	550	0.2036	0.8024	0.8165	0.8094	0.6798	0.9666	0.8987	0.9517
0.0212	3.29	560	0.1967	0.8093	0.8175	0.8134	0.6854	0.9674	0.8996	0.9535
0.0188	3.35	570	0.1944	0.8056	0.8188	0.8122	0.6837	0.9671	0.9000	0.9527
0.0176	3.41	580	0.1975	0.8014	0.8239	0.8125	0.6842	0.9669	0.9022	0.9528
0.0197	3.47	590	0.2118	0.8058	0.8186	0.8122	0.6837	0.9671	0.8999	0.9529
0.0142	3.53	600	0.2000	0.8107	0.8187	0.8147	0.6873	0.9676	0.9003	0.9540
0.0145	3.59	610	0.2095	0.7950	0.8278	0.8111	0.6822	0.9665	0.9037	0.9522
0.023	3.65	620	0.2107	0.7881	0.8268	0.8070	0.6764	0.9656	0.9028	0.9511
0.0156	3.71	630	0.2191	0.7814	0.8366	0.8081	0.6780	0.9654	0.9072	0.9507
0.0175	3.76	640	0.2090	0.8081	0.8202	0.8141	0.6865	0.9674	0.9008	0.9528
0.02	3.82	650	0.2160	0.8068	0.8224	0.8145	0.6871	0.9674	0.9018	0.9524
0.0148	3.88	660	0.2049	0.7972	0.8306	0.8136	0.6857	0.9669	0.9052	0.9526
0.0184	3.94	670	0.2122	0.7908	0.8342	0.8119	0.6834	0.9664	0.9066	0.9513
0.0178	4.0	680	0.2090	0.7925	0.8271	0.8094	0.6799	0.9661	0.9032	0.9519
0.0159	4.06	690	0.2137	0.7963	0.8293	0.8125	0.6842	0.9667	0.9045	0.9528
0.0172	4.12	700	0.2150	0.7917	0.8317	0.8112	0.6824	0.9663	0.9054	0.9520
0.0163	4.18	710	0.2174	0.8010	0.8250	0.8128	0.6846	0.9670	0.9027	0.9524
0.0117	4.24	720	0.2175	0.8036	0.8229	0.8131	0.6851	0.9671	0.9019	0.9530
0.0149	4.29	730	0.2225	0.7990	0.8242	0.8114	0.6826	0.9667	0.9022	0.9515
0.014	4.35	740	0.2157	0.7916	0.8273	0.8090	0.6793	0.9660	0.9033	0.9520
0.0139	4.41	750	0.2217	0.7952	0.8231	0.8089	0.6792	0.9662	0.9015	0.9512
0.0143	4.47	760	0.2201	0.7914	0.8317	0.8111	0.6822	0.9663	0.9054	0.9518
0.0147	4.53	770	0.2218	0.7945	0.8264	0.8101	0.6809	0.9663	0.9030	0.9517
0.0124	4.59	780	0.2300	0.7915	0.8337	0.8120	0.6835	0.9664	0.9064	0.9515
0.0157	4.65	790	0.2244	0.8038	0.8191	0.8114	0.6826	0.9669	0.9000	0.9520
0.0126	4.71	800	0.2324	0.7983	0.8256	0.8117	0.6831	0.9667	0.9029	0.9514
0.0137	4.76	810	0.2263	0.8000	0.8226	0.8112	0.6823	0.9667	0.9015	0.9520
0.0133	4.82	820	0.2295	0.7932	0.8299	0.8111	0.6823	0.9664	0.9046	0.9512
0.0155	4.88	830	0.2266	0.8019	0.8226	0.8121	0.6837	0.9669	0.9016	0.9520
0.0135	4.94	840	0.2286	0.7932	0.8318	0.8121	0.6836	0.9665	0.9056	0.9518
0.0073	5.0	850	0.2283	0.8019	0.8235	0.8125	0.6843	0.9670	0.9021	0.9520
0.0147	5.06	860	0.2292	0.7965	0.8290	0.8124	0.6841	0.9667	0.9044	0.9518
0.0139	5.12	870	0.2293	0.8013	0.8251	0.8130	0.6850	0.9670	0.9028	0.9520
0.0162	5.18	880	0.2277	0.7977	0.8261	0.8116	0.6830	0.9667	0.9031	0.9518
0.0111	5.24	890	0.2285	0.7941	0.8288	0.8111	0.6822	0.9664	0.9042	0.9517
0.0128	5.29	900	0.2273	0.7987	0.8240	0.8112	0.6823	0.9666	0.9021	0.9521
0.0129	5.35	910	0.2282	0.7989	0.8244	0.8114	0.6827	0.9667	0.9023	0.9520
0.0115	5.41	920	0.2285	0.7991	0.8246	0.8117	0.6830	0.9667	0.9024	0.9520
0.011	5.47	930	0.2283	0.7999	0.8242	0.8119	0.6833	0.9668	0.9023	0.9522
0.013	5.53	940	0.2281	0.7995	0.8239	0.8115	0.6828	0.9667	0.9021	0.9522
0.0132	5.59	950	0.2281	0.7993	0.8239	0.8114	0.6826	0.9667	0.9021	0.9522
0.0104	5.65	960	0.2281	0.7992	0.8244	0.8116	0.6829	0.9667	0.9023	0.9522
0.0143	5.71	970	0.2279	0.7988	0.8245	0.8114	0.6827	0.9667	0.9024	0.9521
0.0133	5.76	980	0.2280	0.7988	0.8241	0.8113	0.6825	0.9667	0.9022	0.9521
0.0079	5.82	990	0.2282	0.7986	0.8244	0.8113	0.6825	0.9666	0.9023	0.9521
0.0108	5.88	1000	0.2282	0.7986	0.8244	0.8113	0.6825	0.9666	0.9023	0.9521

Framework versions

• Transformers 4.34.1
• Pytorch 2.0.1
• Datasets 2.14.6
• Tokenizers 0.14.1