N_roberta_twitterfin_padding90model

This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.8650
• Accuracy: 0.9091

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
0.6054	1.0	597	0.3184	0.8769
0.3443	2.0	1194	0.2685	0.8974
0.2688	3.0	1791	0.3255	0.8974
0.1973	4.0	2388	0.4801	0.9020
0.1766	5.0	2985	0.4662	0.9024
0.1006	6.0	3582	0.5233	0.9020
0.0952	7.0	4179	0.6466	0.8991
0.059	8.0	4776	0.6995	0.8945
0.0509	9.0	5373	0.7444	0.8957
0.0455	10.0	5970	0.7315	0.9070
0.0373	11.0	6567	0.7778	0.8995
0.03	12.0	7164	0.7502	0.9012
0.0261	13.0	7761	0.7549	0.9100
0.0285	14.0	8358	0.8109	0.9028
0.0167	15.0	8955	0.8091	0.9054
0.009	16.0	9552	0.8166	0.9083
0.0116	17.0	10149	0.8317	0.9083
0.0059	18.0	10746	0.8309	0.9116
0.0031	19.0	11343	0.8648	0.9095
0.0026	20.0	11940	0.8650	0.9091

Framework versions

• Transformers 4.33.2
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.13.3