results

This model is a fine-tuned version of robzchhangte/MizBERT on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7248
• Accuracy: 0.5346
• F1: 0.5346
• Precision: 0.5346
• Recall: 0.5346

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 15
• eval_batch_size: 15
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1	Precision	Recall
1.6747	0.0585	10	1.2733	0.5	0.5	0.5	0.5
1.2979	0.1170	20	1.1629	0.5219	0.5219	0.5219	0.5219
1.0906	0.1754	30	1.1048	0.5234	0.5234	0.5234	0.5234
0.9134	0.2339	40	0.8426	0.5109	0.5109	0.5109	0.5109
0.7985	0.2924	50	0.7739	0.525	0.525	0.525	0.525
0.7278	0.3509	60	0.7949	0.4969	0.4969	0.4969	0.4969
0.7522	0.4094	70	0.7225	0.525	0.525	0.525	0.525
0.7134	0.4678	80	0.7187	0.5109	0.5109	0.5109	0.5109
0.6897	0.5263	90	0.7682	0.4781	0.4781	0.4781	0.4781
0.7369	0.5848	100	0.7019	0.5078	0.5078	0.5078	0.5078
0.6917	0.6433	110	0.6980	0.5109	0.5109	0.5109	0.5109
0.698	0.7018	120	0.7038	0.5297	0.5297	0.5297	0.5297
0.6974	0.7602	130	0.7039	0.5125	0.5125	0.5125	0.5125
0.7141	0.8187	140	0.6941	0.5047	0.5047	0.5047	0.5047
0.7127	0.8772	150	0.6937	0.5	0.5	0.5	0.5
0.7007	0.9357	160	0.7047	0.5266	0.5266	0.5266	0.5266
0.7483	0.9942	170	0.6975	0.4828	0.4828	0.4828	0.4828
0.7063	1.0526	180	0.6929	0.5266	0.5266	0.5266	0.5266
0.6848	1.1111	190	0.7107	0.4797	0.4797	0.4797	0.4797
0.7014	1.1696	200	0.6891	0.5422	0.5422	0.5422	0.5422
0.7113	1.2281	210	0.6950	0.5141	0.5141	0.5141	0.5141
0.6915	1.2865	220	0.6901	0.5391	0.5391	0.5391	0.5391
0.6834	1.3450	230	0.7117	0.5188	0.5188	0.5188	0.5188
0.7032	1.4035	240	0.7029	0.5031	0.5031	0.5031	0.5031
0.6962	1.4620	250	0.6952	0.5312	0.5312	0.5312	0.5312
0.7103	1.5205	260	0.7165	0.5297	0.5297	0.5297	0.5297
0.7405	1.5789	270	0.8608	0.475	0.4750	0.475	0.475
0.7633	1.6374	280	0.6994	0.5344	0.5344	0.5344	0.5344
0.7061	1.6959	290	0.6887	0.5531	0.5531	0.5531	0.5531
0.6975	1.7544	300	0.7105	0.475	0.4750	0.475	0.475
0.7098	1.8129	310	0.6959	0.5297	0.5297	0.5297	0.5297
0.7703	1.8713	320	0.6954	0.5281	0.5281	0.5281	0.5281
0.6948	1.9298	330	0.7116	0.475	0.4750	0.475	0.475
0.689	1.9883	340	0.7261	0.475	0.4750	0.475	0.475
0.7011	2.0468	350	0.7265	0.5234	0.5234	0.5234	0.5234
0.7026	2.1053	360	0.7217	0.4734	0.4734	0.4734	0.4734
0.6837	2.1637	370	0.7001	0.4984	0.4984	0.4984	0.4984
0.6579	2.2222	380	0.7106	0.525	0.525	0.525	0.525
0.6755	2.2807	390	0.7218	0.525	0.525	0.525	0.525
0.6739	2.3392	400	0.7054	0.5172	0.5172	0.5172	0.5172
0.6757	2.3977	410	0.7015	0.5406	0.5406	0.5406	0.5406
0.7135	2.4561	420	0.7396	0.4828	0.4828	0.4828	0.4828
0.6801	2.5146	430	0.7323	0.4906	0.4906	0.4906	0.4906
0.7349	2.5731	440	0.6939	0.5047	0.5047	0.5047	0.5047
0.6813	2.6316	450	0.6957	0.5234	0.5234	0.5234	0.5234
0.7054	2.6901	460	0.7156	0.5344	0.5344	0.5344	0.5344
0.7052	2.7485	470	0.7143	0.5437	0.5437	0.5437	0.5437
0.6915	2.8070	480	0.6947	0.5062	0.5062	0.5062	0.5062
0.679	2.8655	490	0.7109	0.5312	0.5312	0.5312	0.5312
0.6729	2.9240	500	0.7442	0.4938	0.4938	0.4938	0.4938
0.7035	2.9825	510	0.7041	0.5281	0.5281	0.5281	0.5281
0.7069	3.0409	520	0.7023	0.4766	0.4766	0.4766	0.4766
0.7089	3.0994	530	0.6936	0.5359	0.5359	0.5359	0.5359
0.6675	3.1579	540	0.6931	0.5188	0.5188	0.5188	0.5188
0.6202	3.2164	550	0.8091	0.4703	0.4703	0.4703	0.4703
0.6183	3.2749	560	0.7316	0.5406	0.5406	0.5406	0.5406
0.5781	3.3333	570	0.7620	0.5437	0.5437	0.5437	0.5437
0.6383	3.3918	580	0.7552	0.5219	0.5219	0.5219	0.5219
0.628	3.4503	590	0.7266	0.5437	0.5437	0.5437	0.5437
0.6198	3.5088	600	0.7217	0.5672	0.5672	0.5672	0.5672
0.6572	3.5673	610	0.7962	0.5047	0.5047	0.5047	0.5047
0.6119	3.6257	620	0.7258	0.5563	0.5563	0.5563	0.5563
0.6651	3.6842	630	0.7445	0.55	0.55	0.55	0.55
0.5399	3.7427	640	0.8115	0.5062	0.5062	0.5062	0.5062
0.6291	3.8012	650	0.8045	0.5312	0.5312	0.5312	0.5312

Framework versions

• Transformers 4.42.4
• Pytorch 2.3.1+cu121
• Tokenizers 0.19.1