Mistral_Sentiment_Classification_2024-08-19

This model is a fine-tuned version of mistralai/Mistral-7B-Instruct-v0.2 on the generator dataset. It achieves the following results on the evaluation set:

• Loss: 0.1144

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2.5e-05
• train_batch_size: 1
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 2
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss
0.1264	0.2048	500	0.1188
0.1096	0.4097	1000	0.1146
0.1289	0.6145	1500	0.1123
0.1089	0.8193	2000	0.1106
0.1218	1.0242	2500	0.1083
0.0987	1.2290	3000	0.1079
0.1074	1.4338	3500	0.1069
0.0906	1.6387	4000	0.1062
0.1094	1.8435	4500	0.1053
0.0951	2.0483	5000	0.1068
0.1	2.2532	5500	0.1068
0.0799	2.4580	6000	0.1062
0.0767	2.6628	6500	0.1058
0.1046	2.8677	7000	0.1054
0.0727	3.0725	7500	0.1091
0.0737	3.2773	8000	0.1092
0.0967	3.4822	8500	0.1092
0.0726	3.6870	9000	0.1091
0.0909	3.8918	9500	0.1085
0.074	4.0967	10000	0.1141
0.0777	4.3015	10500	0.1143
0.0695	4.5063	11000	0.1145
0.0604	4.7112	11500	0.1142
0.0612	4.9160	12000	0.1144

Framework versions

• PEFT 0.12.0
• Transformers 4.44.0
• Pytorch 2.4.0+cu121
• Datasets 2.21.0
• Tokenizers 0.19.1