roberta-large_stereoset_finetuned

This model is a fine-tuned version of roberta-large on the stereoset dataset. It achieves the following results on the evaluation set:

• Loss: 0.7989
• Accuracy: 0.8336

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 128
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.21	5	0.6920	0.5196
No log	0.42	10	0.6909	0.5290
No log	0.62	15	0.6899	0.5220
No log	0.83	20	0.6883	0.5408
No log	1.04	25	0.6573	0.6609
No log	1.25	30	0.5892	0.7088
No log	1.46	35	0.6633	0.5408
No log	1.67	40	0.6322	0.6852
No log	1.88	45	0.6393	0.7159
No log	2.08	50	0.5494	0.7410
No log	2.29	55	0.5498	0.7386
No log	2.5	60	0.5069	0.7692
No log	2.71	65	0.4930	0.7630
No log	2.92	70	0.4939	0.7614
No log	3.12	75	0.5379	0.7724
No log	3.33	80	0.5981	0.7732
No log	3.54	85	0.5842	0.7716
No log	3.75	90	0.4405	0.8030
No log	3.96	95	0.4970	0.7951
No log	4.17	100	0.5172	0.8093
No log	4.38	105	0.5052	0.8108
No log	4.58	110	0.4685	0.8085
No log	4.79	115	0.4663	0.8218
No log	5.0	120	0.5086	0.8218
No log	5.21	125	0.5096	0.8179
No log	5.42	130	0.5705	0.8203
No log	5.62	135	0.5294	0.8312
No log	5.83	140	0.4377	0.8375
No log	6.04	145	0.5699	0.8100
No log	6.25	150	0.6062	0.8265
No log	6.46	155	0.7237	0.8218
No log	6.67	160	0.6816	0.8210
No log	6.88	165	0.6413	0.8124
No log	7.08	170	0.5931	0.8359
No log	7.29	175	0.6149	0.8399
No log	7.5	180	0.7190	0.8195
No log	7.71	185	0.7339	0.8352
No log	7.92	190	0.7244	0.8352
No log	8.12	195	0.7722	0.8203
No log	8.33	200	0.6890	0.8344
No log	8.54	205	0.6938	0.8336
No log	8.75	210	0.7234	0.8320
No log	8.96	215	0.7517	0.8391
No log	9.17	220	0.7713	0.8383
No log	9.38	225	0.7745	0.8375
No log	9.58	230	0.8006	0.8375
No log	9.79	235	0.8003	0.8367
No log	10.0	240	0.7989	0.8336

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1
• Datasets 2.9.0
• Tokenizers 0.13.2