bert-large-uncased_crows_pairs_finetuned

This model is a fine-tuned version of bert-large-uncased on the crows_pairs dataset. It achieves the following results on the evaluation set:

• Loss: 1.8687
• Accuracy: 0.7483

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
0.7241	0.53	10	0.7015	0.4868
0.6975	1.05	20	0.7124	0.4934
0.6992	1.58	30	0.7102	0.4901
0.6941	2.11	40	0.6890	0.5430
0.665	2.63	50	0.6981	0.5464
0.5656	3.16	60	0.5434	0.7119
0.3695	3.68	70	0.7431	0.7152
0.3913	4.21	80	0.7834	0.7185
0.2617	4.74	90	0.6394	0.7318
0.1225	5.26	100	0.8882	0.6921
0.1549	5.79	110	0.8629	0.7119
0.1094	6.32	120	1.0113	0.7185
0.0418	6.84	130	1.2568	0.7219
0.023	7.37	140	1.3223	0.7417
0.045	7.89	150	1.5015	0.7086
0.0216	8.42	160	1.1833	0.7550
0.0084	8.95	170	1.4125	0.7318
0.0198	9.47	180	1.5301	0.7152
0.0061	10.0	190	1.3163	0.7483
0.0041	10.53	200	1.3083	0.7517
0.0046	11.05	210	1.4028	0.7616
0.0034	11.58	220	1.5256	0.7583
0.0012	12.11	230	1.6067	0.7583
0.001	12.63	240	1.6199	0.7649
0.005	13.16	250	1.7140	0.7384
0.0031	13.68	260	1.7680	0.7318
0.0008	14.21	270	1.7353	0.7252
0.0026	14.74	280	1.7242	0.7450
0.0013	15.26	290	1.7290	0.7483
0.0001	15.79	300	1.7421	0.7450
0.0008	16.32	310	1.7536	0.7450
0.0013	16.84	320	1.7588	0.7483
0.0014	17.37	330	1.8153	0.7417
0.0025	17.89	340	1.8432	0.7450
0.0024	18.42	350	1.8597	0.7351
0.0022	18.95	360	1.8676	0.7384
0.0001	19.47	370	1.8602	0.7417
0.0032	20.0	380	1.8600	0.7450
0.0017	20.53	390	1.8576	0.7417
0.0022	21.05	400	1.8603	0.7417
0.0024	21.58	410	1.8649	0.7417
0.002	22.11	420	1.8704	0.7417
0.0008	22.63	430	1.8764	0.7417
0.0019	23.16	440	1.8914	0.7417
0.0015	23.68	450	1.9026	0.7384
0.0014	24.21	460	1.9146	0.7384
0.0017	24.74	470	1.9258	0.7384
0.0027	25.26	480	1.9280	0.7384
0.0027	25.79	490	1.9285	0.7384
0.001	26.32	500	1.9236	0.7384
0.0034	26.84	510	1.8905	0.7450
0.0013	27.37	520	1.8730	0.7417
0.0016	27.89	530	1.8687	0.7450
0.0007	28.42	540	1.8681	0.7483
0.0042	28.95	550	1.8683	0.7483
0.0009	29.47	560	1.8686	0.7483
0.0018	30.0	570	1.8687	0.7483

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1
• Datasets 2.10.1
• Tokenizers 0.13.2