multiberts-seed_1_crows_pairs_classifieronly

This model is a fine-tuned version of google/multiberts-seed_1 on the crows_pairs dataset. It achieves the following results on the evaluation set:

• Loss: 0.6903
• Accuracy: 0.5331
• Tp: 0.3477
• Tn: 0.1854
• Fp: 0.3146
• Fn: 0.1523

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Tp	Tn	Fp	Fn
0.7016	1.05	20	0.6910	0.5364	0.4437	0.0927	0.4073	0.0563
0.7072	2.11	40	0.6914	0.5166	0.4834	0.0331	0.4669	0.0166
0.7003	3.16	60	0.6910	0.5497	0.0927	0.4570	0.0430	0.4073
0.7004	4.21	80	0.6908	0.5265	0.2583	0.2682	0.2318	0.2417
0.6985	5.26	100	0.6916	0.5033	0.0033	0.5	0.0	0.4967
0.7107	6.32	120	0.6909	0.5364	0.4603	0.0762	0.4238	0.0397
0.7135	7.37	140	0.6908	0.5364	0.4139	0.1225	0.3775	0.0861
0.7041	8.42	160	0.6908	0.5464	0.2848	0.2616	0.2384	0.2152
0.7022	9.47	180	0.6907	0.5298	0.2450	0.2848	0.2152	0.2550
0.6996	10.53	200	0.6908	0.5265	0.4371	0.0894	0.4106	0.0629
0.7049	11.58	220	0.6914	0.5066	0.0066	0.5	0.0	0.4934
0.6975	12.63	240	0.6912	0.5232	0.4834	0.0397	0.4603	0.0166
0.6967	13.68	260	0.6907	0.5497	0.2748	0.2748	0.2252	0.2252
0.705	14.74	280	0.6912	0.5132	0.0132	0.5	0.0	0.4868
0.6943	15.79	300	0.6909	0.5298	0.4768	0.0530	0.4470	0.0232
0.7057	16.84	320	0.6906	0.5364	0.4272	0.1093	0.3907	0.0728
0.6995	17.89	340	0.6905	0.5530	0.2252	0.3278	0.1722	0.2748
0.6989	18.95	360	0.6905	0.5464	0.1854	0.3609	0.1391	0.3146
0.7156	20.0	380	0.6912	0.5199	0.4834	0.0364	0.4636	0.0166
0.6957	21.05	400	0.6905	0.5497	0.2285	0.3212	0.1788	0.2715
0.6978	22.11	420	0.6906	0.5331	0.4371	0.0960	0.4040	0.0629
0.6988	23.16	440	0.6905	0.5331	0.3179	0.2152	0.2848	0.1821
0.7054	24.21	460	0.6905	0.5497	0.1623	0.3874	0.1126	0.3377
0.6997	25.26	480	0.6907	0.5166	0.4636	0.0530	0.4470	0.0364
0.7007	26.32	500	0.6904	0.5364	0.3940	0.1424	0.3576	0.1060
0.6973	27.37	520	0.6914	0.5	0.0	0.5	0.0	0.5
0.704	28.42	540	0.6905	0.5364	0.4338	0.1026	0.3974	0.0662
0.7028	29.47	560	0.6905	0.5530	0.1192	0.4338	0.0662	0.3808
0.6959	30.53	580	0.6905	0.5364	0.4404	0.0960	0.4040	0.0596
0.704	31.58	600	0.6904	0.5464	0.2583	0.2881	0.2119	0.2417
0.704	32.63	620	0.6903	0.5397	0.3146	0.2252	0.2748	0.1854
0.7003	33.68	640	0.6903	0.5430	0.3079	0.2351	0.2649	0.1921
0.7058	34.74	660	0.6903	0.5464	0.3079	0.2384	0.2616	0.1921
0.7095	35.79	680	0.6903	0.5430	0.2649	0.2781	0.2219	0.2351
0.7019	36.84	700	0.6905	0.5397	0.0695	0.4702	0.0298	0.4305
0.7167	37.89	720	0.6907	0.5099	0.0166	0.4934	0.0066	0.4834
0.6975	38.95	740	0.6903	0.5497	0.3113	0.2384	0.2616	0.1887
0.7036	40.0	760	0.6903	0.5397	0.3311	0.2086	0.2914	0.1689
0.7009	41.05	780	0.6903	0.5331	0.3940	0.1391	0.3609	0.1060
0.7004	42.11	800	0.6903	0.5430	0.2980	0.2450	0.2550	0.2020
0.699	43.16	820	0.6903	0.5430	0.2152	0.3278	0.1722	0.2848
0.6962	44.21	840	0.6903	0.5430	0.2980	0.2450	0.2550	0.2020
0.6981	45.26	860	0.6903	0.5397	0.3543	0.1854	0.3146	0.1457
0.7008	46.32	880	0.6903	0.5430	0.3808	0.1623	0.3377	0.1192
0.7048	47.37	900	0.6903	0.5530	0.3742	0.1788	0.3212	0.1258
0.707	48.42	920	0.6903	0.5497	0.3709	0.1788	0.3212	0.1291
0.7014	49.47	940	0.6903	0.5331	0.3477	0.1854	0.3146	0.1523

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1
• Datasets 2.10.1
• Tokenizers 0.13.2