distilbert_dair-ai_emotion_20240730_e20_cos

This model is a fine-tuned version of distilbert/distilbert-base-uncased on the dair-ai/emotion dataset. It achieves the following results on the evaluation set:

• Loss: 0.1601
• Accuracy: 0.9395

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 512
• eval_batch_size: 512
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	32	1.2155	0.576
No log	2.0	64	0.7162	0.7685
No log	3.0	96	0.3572	0.9045
No log	4.0	128	0.2206	0.9265
No log	5.0	160	0.1939	0.926
No log	6.0	192	0.1677	0.9335
No log	7.0	224	0.1637	0.9345
No log	8.0	256	0.1632	0.932
No log	9.0	288	0.1522	0.9385
No log	10.0	320	0.1505	0.937
No log	11.0	352	0.1539	0.9375
No log	12.0	384	0.1554	0.9375
No log	13.0	416	0.1564	0.9395
No log	14.0	448	0.1581	0.9405
No log	15.0	480	0.1597	0.9405
0.2897	16.0	512	0.1590	0.94
0.2897	17.0	544	0.1585	0.94
0.2897	18.0	576	0.1604	0.9385
0.2897	19.0	608	0.1602	0.9395
0.2897	20.0	640	0.1601	0.9395

Framework versions

• Transformers 4.38.2
• Pytorch 2.1.2+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2