roberta-base_stereoset_finetuned

This model is a fine-tuned version of roberta-base on the stereoset dataset. It achieves the following results on the evaluation set:

• Loss: 0.8461
• Accuracy: 0.7904

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 128
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.21	5	0.6915	0.5149
No log	0.42	10	0.6945	0.4914
No log	0.62	15	0.6931	0.4945
No log	0.83	20	0.6814	0.5086
No log	1.04	25	0.6454	0.6978
No log	1.25	30	0.5807	0.7088
No log	1.46	35	0.5620	0.7284
No log	1.67	40	0.5410	0.7331
No log	1.88	45	0.4965	0.7630
No log	2.08	50	0.4924	0.7614
No log	2.29	55	0.4906	0.7661
No log	2.5	60	0.5141	0.7661
No log	2.71	65	0.4826	0.7700
No log	2.92	70	0.4977	0.7630
No log	3.12	75	0.4890	0.7802
No log	3.33	80	0.4819	0.7857
No log	3.54	85	0.4840	0.7834
No log	3.75	90	0.5189	0.7794
No log	3.96	95	0.5000	0.7912
No log	4.17	100	0.4958	0.7865
No log	4.38	105	0.5149	0.7896
No log	4.58	110	0.5515	0.7975
No log	4.79	115	0.5766	0.7873
No log	5.0	120	0.5867	0.7873
No log	5.21	125	0.6143	0.7936
No log	5.42	130	0.6226	0.7881
No log	5.62	135	0.6374	0.7865
No log	5.83	140	0.6405	0.7983
No log	6.04	145	0.6116	0.8006
No log	6.25	150	0.6372	0.7983
No log	6.46	155	0.6804	0.7881
No log	6.67	160	0.7237	0.7857
No log	6.88	165	0.7038	0.7904
No log	7.08	170	0.7100	0.7991
No log	7.29	175	0.6837	0.7920
No log	7.5	180	0.7203	0.8046
No log	7.71	185	0.7478	0.7959
No log	7.92	190	0.7667	0.7920
No log	8.12	195	0.7792	0.7959
No log	8.33	200	0.8014	0.7943
No log	8.54	205	0.8193	0.7959
No log	8.75	210	0.8316	0.7967
No log	8.96	215	0.8411	0.7896
No log	9.17	220	0.8652	0.7936
No log	9.38	225	0.8553	0.7841
No log	9.58	230	0.8458	0.7881
No log	9.79	235	0.8456	0.7912
No log	10.0	240	0.8461	0.7904

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1
• Datasets 2.9.0
• Tokenizers 0.13.2