sentiment_output

This model is a fine-tuned version of mhdfadhil/mlm_output on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.3182
• Accuracy: 0.8625

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 256
• eval_batch_size: 256
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
0.6798	1.0	98	0.5304	0.8013
0.474	2.0	196	0.3774	0.8328
0.3806	3.0	294	0.3485	0.8456
0.3602	4.0	392	0.3465	0.8493
0.3606	5.0	490	0.3346	0.8522
0.3494	6.0	588	0.3317	0.8562
0.3509	7.0	686	0.3293	0.8572
0.3485	8.0	784	0.3298	0.8575
0.3486	9.0	882	0.3282	0.8583
0.3381	10.0	980	0.3240	0.8600
0.3351	11.0	1078	0.3263	0.8597
0.336	12.0	1176	0.3213	0.8615
0.335	13.0	1274	0.3220	0.8614
0.3423	14.0	1372	0.3197	0.8613
0.341	15.0	1470	0.3200	0.8626
0.3386	16.0	1568	0.3188	0.8622
0.3414	17.0	1666	0.3187	0.8613
0.3372	18.0	1764	0.3193	0.8628
0.3357	19.0	1862	0.3183	0.8628
0.3345	20.0	1960	0.3182	0.8625

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.1