multiberts-seed_2-step_2000k_crows_pairs_classifieronly

This model is a fine-tuned version of google/multiberts-seed_2-step_2000k on the crows_pairs dataset. It achieves the following results on the evaluation set:

• Loss: 0.6970
• Accuracy: 0.4603
• Tp: 0.2748
• Tn: 0.1854
• Fp: 0.3013
• Fn: 0.2384

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Tp	Tn	Fp	Fn
0.7105	1.05	20	0.6958	0.4934	0.4404	0.0530	0.4338	0.0728
0.7148	2.11	40	0.6962	0.4801	0.2020	0.2781	0.2086	0.3113
0.7057	3.16	60	0.6972	0.4536	0.0695	0.3841	0.1026	0.4437
0.6938	4.21	80	0.6966	0.5099	0.5	0.0099	0.4768	0.0132
0.7029	5.26	100	0.6993	0.4801	0.0166	0.4636	0.0232	0.4967
0.696	6.32	120	0.6961	0.4901	0.4536	0.0364	0.4503	0.0596
0.6999	7.37	140	0.6971	0.4305	0.0762	0.3543	0.1325	0.4371
0.7079	8.42	160	0.6963	0.4702	0.2252	0.2450	0.2417	0.2881
0.7038	9.47	180	0.6960	0.5033	0.4007	0.1026	0.3841	0.1126
0.6914	10.53	200	0.6966	0.4768	0.1954	0.2815	0.2053	0.3179
0.696	11.58	220	0.6962	0.5033	0.4404	0.0629	0.4238	0.0728
0.7009	12.63	240	0.6983	0.4669	0.0530	0.4139	0.0728	0.4603
0.7062	13.68	260	0.6965	0.4669	0.3013	0.1656	0.3212	0.2119
0.6966	14.74	280	0.6990	0.4868	0.0364	0.4503	0.0364	0.4768
0.7038	15.79	300	0.6975	0.5	0.4934	0.0066	0.4801	0.0199
0.7031	16.84	320	0.6964	0.5033	0.3974	0.1060	0.3808	0.1159
0.7032	17.89	340	0.6965	0.4801	0.3311	0.1490	0.3377	0.1821
0.7004	18.95	360	0.6990	0.4868	0.0364	0.4503	0.0364	0.4768
0.695	20.0	380	0.6966	0.4636	0.2715	0.1921	0.2947	0.2417
0.7052	21.05	400	0.6974	0.4338	0.1126	0.3212	0.1656	0.4007
0.6995	22.11	420	0.6965	0.4934	0.3642	0.1291	0.3576	0.1490
0.714	23.16	440	0.6971	0.4868	0.1821	0.3046	0.1821	0.3311
0.7004	24.21	460	0.6980	0.4536	0.0596	0.3940	0.0927	0.4536
0.7025	25.26	480	0.6966	0.4801	0.3344	0.1457	0.3411	0.1788
0.6987	26.32	500	0.6975	0.4404	0.1093	0.3311	0.1556	0.4040
0.6956	27.37	520	0.6975	0.4470	0.1291	0.3179	0.1689	0.3841
0.697	28.42	540	0.6974	0.4570	0.1424	0.3146	0.1722	0.3709
0.7051	29.47	560	0.6975	0.4536	0.1358	0.3179	0.1689	0.3775
0.7024	30.53	580	0.6979	0.4338	0.0828	0.3510	0.1358	0.4305
0.6908	31.58	600	0.6969	0.4636	0.2682	0.1954	0.2914	0.2450
0.6979	32.63	620	0.6970	0.4868	0.2583	0.2285	0.2583	0.2550
0.7026	33.68	640	0.6970	0.4834	0.2583	0.2252	0.2616	0.2550
0.6998	34.74	660	0.6970	0.4834	0.2583	0.2252	0.2616	0.2550
0.6964	35.79	680	0.6969	0.4669	0.2682	0.1987	0.2881	0.2450
0.709	36.84	700	0.6968	0.4868	0.3510	0.1358	0.3510	0.1623
0.6974	37.89	720	0.6969	0.4669	0.2881	0.1788	0.3079	0.2252
0.7039	38.95	740	0.6972	0.4934	0.2318	0.2616	0.2252	0.2815
0.6963	40.0	760	0.6970	0.4768	0.2715	0.2053	0.2815	0.2417
0.6891	41.05	780	0.6970	0.4801	0.2682	0.2119	0.2748	0.2450
0.7008	42.11	800	0.6969	0.4868	0.3245	0.1623	0.3245	0.1887
0.7026	43.16	820	0.6971	0.4934	0.2550	0.2384	0.2483	0.2583
0.6969	44.21	840	0.6974	0.4834	0.1821	0.3013	0.1854	0.3311
0.7057	45.26	860	0.6972	0.4967	0.2285	0.2682	0.2185	0.2848
0.6951	46.32	880	0.6971	0.4901	0.2550	0.2351	0.2517	0.2583
0.7041	47.37	900	0.6969	0.4934	0.3311	0.1623	0.3245	0.1821
0.7019	48.42	920	0.6969	0.4768	0.3046	0.1722	0.3146	0.2086
0.6998	49.47	940	0.6970	0.4603	0.2748	0.1854	0.3013	0.2384

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1
• Datasets 2.10.1
• Tokenizers 0.13.2