sembr2023-bert-mini

This model is a fine-tuned version of prajjwal1/bert-mini on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1844
• Precision: 0.7925
• Recall: 0.7950
• F1: 0.7938
• Iou: 0.6581
• Accuracy: 0.9620
• Balanced Accuracy: 0.8870
• Overall Accuracy: 0.9443

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 64
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Iou	Accuracy	Balanced Accuracy	Overall Accuracy
0.5028	0.06	10	0.5036	0	0.0	0.0	0.0	0.9080	0.5	0.9080
0.417	0.12	20	0.4416	0	0.0	0.0	0.0	0.9080	0.5	0.9080
0.4292	0.18	30	0.4298	0	0.0	0.0	0.0	0.9080	0.5	0.9080
0.3964	0.24	40	0.4121	0	0.0	0.0	0.0	0.9080	0.5	0.9080
0.3611	0.3	50	0.3619	0.5833	0.0080	0.0158	0.0080	0.9082	0.5037	0.9081
0.3201	0.36	60	0.3103	0.6414	0.5085	0.5673	0.3960	0.9287	0.7399	0.9146
0.2983	0.42	70	0.2575	0.7866	0.6423	0.7072	0.5470	0.9511	0.8123	0.9333
0.2621	0.48	80	0.2402	0.7879	0.6824	0.7314	0.5765	0.9539	0.8319	0.9354
0.2464	0.55	90	0.2262	0.8304	0.6660	0.7391	0.5862	0.9568	0.8261	0.9384
0.21	0.61	100	0.2186	0.7990	0.7131	0.7536	0.6047	0.9571	0.8475	0.9382
0.2238	0.67	110	0.2042	0.8242	0.6860	0.7487	0.5984	0.9577	0.8356	0.9408
0.1849	0.73	120	0.1990	0.8799	0.6422	0.7425	0.5904	0.9590	0.8166	0.9432
0.1702	0.79	130	0.1959	0.7862	0.7436	0.7643	0.6185	0.9578	0.8615	0.9392
0.2033	0.85	140	0.1939	0.7996	0.7414	0.7694	0.6252	0.9591	0.8613	0.9404
0.1686	0.91	150	0.1916	0.7982	0.7437	0.7700	0.6260	0.9591	0.8623	0.9410
0.1484	0.97	160	0.1855	0.7898	0.7597	0.7745	0.6320	0.9593	0.8696	0.9408
0.1717	1.03	170	0.1864	0.8231	0.7301	0.7738	0.6310	0.9607	0.8571	0.9434
0.1584	1.09	180	0.1844	0.7919	0.7676	0.7796	0.6388	0.9601	0.8736	0.9414
0.1455	1.15	190	0.1807	0.8255	0.7496	0.7857	0.6470	0.9624	0.8668	0.9437
0.1521	1.21	200	0.1746	0.8277	0.7428	0.7829	0.6433	0.9621	0.8636	0.9450
0.1385	1.27	210	0.1833	0.8099	0.7573	0.7828	0.6431	0.9613	0.8697	0.9430
0.1246	1.33	220	0.1857	0.7400	0.8164	0.7763	0.6344	0.9567	0.8937	0.9366
0.1312	1.39	230	0.1738	0.8345	0.7441	0.7867	0.6484	0.9629	0.8646	0.9458
0.1139	1.45	240	0.1750	0.7964	0.7861	0.7912	0.6545	0.9618	0.8829	0.9428
0.1181	1.52	250	0.1728	0.8201	0.7578	0.7877	0.6498	0.9624	0.8705	0.9448
0.1248	1.58	260	0.1774	0.8148	0.7727	0.7932	0.6573	0.9629	0.8775	0.9441
0.1384	1.64	270	0.1748	0.7840	0.7925	0.7882	0.6505	0.9608	0.8852	0.9418
0.1068	1.7	280	0.1744	0.7943	0.7950	0.7947	0.6593	0.9622	0.8871	0.9430
0.114	1.76	290	0.1749	0.7916	0.7904	0.7910	0.6543	0.9616	0.8847	0.9428
0.1214	1.82	300	0.1778	0.7551	0.8233	0.7877	0.6498	0.9592	0.8981	0.9392
0.1139	1.88	310	0.1764	0.7897	0.7839	0.7868	0.6485	0.9609	0.8814	0.9425
0.1254	1.94	320	0.1771	0.7908	0.7925	0.7916	0.6551	0.9616	0.8856	0.9427
0.1001	2.0	330	0.1715	0.8057	0.7829	0.7942	0.6586	0.9627	0.8819	0.9445
0.0989	2.06	340	0.1705	0.8099	0.7803	0.7948	0.6595	0.9629	0.8809	0.9446
0.1222	2.12	350	0.1761	0.7843	0.7991	0.7916	0.6551	0.9613	0.8884	0.9422
0.1032	2.18	360	0.1754	0.7961	0.7864	0.7912	0.6545	0.9618	0.8830	0.9432
0.0799	2.24	370	0.1753	0.7867	0.7879	0.7873	0.6492	0.9608	0.8831	0.9432
0.099	2.3	380	0.1751	0.8101	0.7738	0.7915	0.6549	0.9625	0.8777	0.9451
0.0993	2.36	390	0.1699	0.8073	0.7791	0.7929	0.6569	0.9626	0.8801	0.9454
0.1025	2.42	400	0.1662	0.8203	0.7764	0.7978	0.6636	0.9638	0.8796	0.9465
0.1081	2.48	410	0.1762	0.8005	0.7893	0.7949	0.6596	0.9625	0.8847	0.9444
0.1118	2.55	420	0.1720	0.8130	0.7755	0.7938	0.6582	0.9630	0.8787	0.9458
0.0779	2.61	430	0.1712	0.8131	0.7797	0.7961	0.6612	0.9633	0.8808	0.9454
0.0944	2.67	440	0.1788	0.7754	0.8094	0.7921	0.6557	0.9609	0.8928	0.9419
0.1053	2.73	450	0.1696	0.7980	0.7901	0.7940	0.6584	0.9623	0.8849	0.9450
0.0889	2.79	460	0.1719	0.8215	0.7736	0.7968	0.6623	0.9637	0.8783	0.9465
0.0879	2.85	470	0.1712	0.8091	0.7828	0.7957	0.6608	0.9630	0.8820	0.9457
0.0867	2.91	480	0.1769	0.8021	0.78	0.7909	0.6541	0.9621	0.8803	0.9447
0.0787	2.97	490	0.1788	0.8044	0.7831	0.7936	0.6578	0.9625	0.8819	0.9447
0.0945	3.03	500	0.1736	0.8055	0.7820	0.7936	0.6578	0.9626	0.8815	0.9445
0.1011	3.09	510	0.1823	0.7881	0.7962	0.7921	0.6558	0.9616	0.8873	0.9432
0.0914	3.15	520	0.1819	0.7958	0.7939	0.7948	0.6595	0.9623	0.8866	0.9438
0.0837	3.21	530	0.1738	0.8129	0.7857	0.7991	0.6654	0.9637	0.8837	0.9460
0.0776	3.27	540	0.1828	0.7921	0.7961	0.7941	0.6585	0.9620	0.8874	0.9437
0.0916	3.33	550	0.1776	0.7835	0.7994	0.7913	0.6547	0.9612	0.8885	0.9433
0.081	3.39	560	0.1784	0.7784	0.8033	0.7907	0.6538	0.9609	0.8901	0.9428
0.0867	3.45	570	0.1793	0.7728	0.8074	0.7897	0.6525	0.9605	0.8917	0.9425
0.0816	3.52	580	0.1789	0.7829	0.8017	0.7922	0.6559	0.9613	0.8896	0.9433
0.0808	3.58	590	0.1791	0.7890	0.7941	0.7916	0.6550	0.9615	0.8863	0.9435
0.07	3.64	600	0.1844	0.7697	0.8071	0.7880	0.6501	0.9600	0.8913	0.9420
0.0775	3.7	610	0.1795	0.7849	0.7957	0.7902	0.6532	0.9612	0.8868	0.9433
0.0722	3.76	620	0.1772	0.7993	0.7814	0.7903	0.6532	0.9619	0.8808	0.9449
0.0786	3.82	630	0.1775	0.8159	0.7763	0.7956	0.6606	0.9633	0.8793	0.9457
0.0768	3.88	640	0.1823	0.8015	0.7848	0.7931	0.6571	0.9623	0.8826	0.9442
0.0728	3.94	650	0.1806	0.7918	0.7885	0.7901	0.6531	0.9615	0.8838	0.9438
0.0762	4.0	660	0.1831	0.7881	0.7935	0.7908	0.6540	0.9614	0.8859	0.9435
0.0776	4.06	670	0.1788	0.8015	0.7847	0.7930	0.6570	0.9623	0.8825	0.9453
0.0843	4.12	680	0.1824	0.8009	0.7876	0.7942	0.6587	0.9625	0.8839	0.9445
0.066	4.18	690	0.1843	0.7921	0.7918	0.7920	0.6556	0.9617	0.8854	0.9440
0.0832	4.24	700	0.1781	0.7957	0.7893	0.7925	0.6563	0.9620	0.8844	0.9447
0.0761	4.3	710	0.1871	0.7817	0.8017	0.7916	0.6550	0.9612	0.8895	0.9428
0.0696	4.36	720	0.1813	0.7957	0.7924	0.7940	0.6584	0.9622	0.8859	0.9446
0.0734	4.42	730	0.1827	0.7934	0.7938	0.7936	0.6578	0.9620	0.8864	0.9444
0.0823	4.48	740	0.1856	0.7956	0.7913	0.7935	0.6576	0.9621	0.8854	0.9443
0.0662	4.55	750	0.1790	0.7890	0.7952	0.7921	0.6557	0.9616	0.8868	0.9444
0.0775	4.61	760	0.1858	0.7899	0.7953	0.7926	0.6564	0.9617	0.8869	0.9439
0.0764	4.67	770	0.1853	0.7852	0.8011	0.7931	0.6572	0.9616	0.8895	0.9436
0.0689	4.73	780	0.1804	0.7964	0.7924	0.7944	0.6589	0.9623	0.8859	0.9445
0.0785	4.79	790	0.1817	0.7937	0.7921	0.7929	0.6569	0.9619	0.8856	0.9444
0.075	4.85	800	0.1856	0.7912	0.7929	0.7920	0.6556	0.9617	0.8858	0.9440
0.0691	4.91	810	0.1844	0.7805	0.8	0.7901	0.6531	0.9609	0.8886	0.9432
0.0835	4.97	820	0.1829	0.7984	0.7911	0.7947	0.6594	0.9624	0.8854	0.9448
0.0712	5.03	830	0.1820	0.7906	0.7927	0.7917	0.6552	0.9616	0.8857	0.9443
0.0594	5.09	840	0.1841	0.7902	0.7948	0.7925	0.6563	0.9617	0.8867	0.9441
0.0775	5.15	850	0.1834	0.7927	0.7936	0.7932	0.6572	0.9619	0.8863	0.9444
0.0755	5.21	860	0.1833	0.7924	0.7944	0.7934	0.6575	0.9619	0.8867	0.9444
0.0717	5.27	870	0.1838	0.7902	0.7958	0.7930	0.6570	0.9618	0.8872	0.9443
0.0694	5.33	880	0.1834	0.7918	0.7939	0.7928	0.6568	0.9618	0.8864	0.9444
0.0759	5.39	890	0.1826	0.7905	0.7954	0.7929	0.6569	0.9618	0.8870	0.9443
0.0666	5.45	900	0.1821	0.7945	0.7922	0.7933	0.6575	0.9620	0.8857	0.9446
0.08	5.52	910	0.1829	0.7924	0.7953	0.7938	0.6581	0.9620	0.8871	0.9444
0.0816	5.58	920	0.1837	0.7918	0.7954	0.7936	0.6578	0.9619	0.8871	0.9443
0.0762	5.64	930	0.1837	0.7922	0.7954	0.7938	0.6581	0.9620	0.8871	0.9443
0.0655	5.7	940	0.1843	0.7906	0.7962	0.7934	0.6575	0.9619	0.8874	0.9442
0.0737	5.76	950	0.1846	0.7904	0.7964	0.7934	0.6576	0.9619	0.8875	0.9441
0.0717	5.82	960	0.1846	0.7905	0.7961	0.7933	0.6574	0.9618	0.8873	0.9441
0.0829	5.88	970	0.1845	0.7917	0.7954	0.7935	0.6577	0.9619	0.8871	0.9443
0.0766	5.94	980	0.1844	0.7924	0.7952	0.7938	0.6581	0.9620	0.8870	0.9443
0.0704	6.0	990	0.1844	0.7925	0.7950	0.7938	0.6581	0.9620	0.8870	0.9443
0.0755	6.06	1000	0.1844	0.7925	0.7950	0.7938	0.6581	0.9620	0.8870	0.9443

Framework versions

• Transformers 4.34.1
• Pytorch 2.0.1
• Datasets 2.14.6
• Tokenizers 0.14.1