roberta-large-pp-5000-5e-07-8

This model is a fine-tuned version of roberta-large on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.6879

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-07
• train_batch_size: 32
• eval_batch_size: 1024
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	157	0.6980
0.7244	2.0	314	0.6961
0.7106	3.0	471	0.6921
0.7021	4.0	628	0.6923
0.7021	5.0	785	0.6917
0.7001	6.0	942	0.6916
0.6974	7.0	1099	0.6901
0.6986	8.0	1256	0.6904
0.6941	9.0	1413	0.6897
0.6941	10.0	1570	0.6894
0.696	11.0	1727	0.6895
0.6949	12.0	1884	0.6887
0.6941	13.0	2041	0.6884
0.6941	14.0	2198	0.6884
0.6891	15.0	2355	0.6881
0.6898	16.0	2512	0.6879
0.6897	17.0	2669	0.6886
0.6861	18.0	2826	0.6885
0.6861	19.0	2983	0.6878
0.6856	20.0	3140	0.6891
0.6852	21.0	3297	0.6886
0.6835	22.0	3454	0.6874
0.6806	23.0	3611	0.6876
0.6806	24.0	3768	0.6876
0.6799	25.0	3925	0.6875
0.6821	26.0	4082	0.6877
0.6801	27.0	4239	0.6879

Framework versions

• Transformers 4.40.2
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1