bert-large-uncased_winobias_finetuned

This model is a fine-tuned version of bert-large-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4783
• Accuracy: 0.7986

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 128
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.38	5	0.7011	0.4994
No log	0.77	10	0.6942	0.4987
No log	1.15	15	0.6941	0.5063
No log	1.54	20	0.6936	0.4924
No log	1.92	25	0.6928	0.5114
No log	2.31	30	0.6925	0.5196
No log	2.69	35	0.6925	0.5215
No log	3.08	40	0.6923	0.5227
No log	3.46	45	0.6922	0.5259
No log	3.85	50	0.6922	0.5202
No log	4.23	55	0.6918	0.5316
No log	4.62	60	0.6912	0.5499
No log	5.0	65	0.6904	0.5574
No log	5.38	70	0.6899	0.5492
No log	5.77	75	0.6894	0.5417
No log	6.15	80	0.6890	0.5290
No log	6.54	85	0.6883	0.5366
No log	6.92	90	0.6863	0.5726
No log	7.31	95	0.6837	0.5909
No log	7.69	100	0.6812	0.5890
No log	8.08	105	0.6788	0.5915
No log	8.46	110	0.6738	0.6225
No log	8.85	115	0.6685	0.6503
No log	9.23	120	0.6616	0.6698
No log	9.62	125	0.6533	0.6799
No log	10.0	130	0.6403	0.7027
No log	10.38	135	0.6282	0.7077
No log	10.77	140	0.6142	0.7235
No log	11.15	145	0.5967	0.7355
No log	11.54	150	0.5814	0.7437
No log	11.92	155	0.5662	0.7513
No log	12.31	160	0.5454	0.7607
No log	12.69	165	0.5251	0.7771
No log	13.08	170	0.5091	0.7872
No log	13.46	175	0.4975	0.7942
No log	13.85	180	0.4892	0.7967
No log	14.23	185	0.4832	0.7992
No log	14.62	190	0.4797	0.8005
No log	15.0	195	0.4783	0.7986

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1
• Datasets 2.10.1
• Tokenizers 0.13.2