finetuning-sentiment-model

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4498
• Accuracy: 0.9279
• F1: 0.9283

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 512
• eval_batch_size: 512
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	1.0	49	0.2333	0.9086	0.9090
No log	2.0	98	0.2126	0.9167	0.9193
No log	3.0	147	0.2129	0.9185	0.9213
No log	4.0	196	0.2369	0.9110	0.9155
No log	5.0	245	0.2155	0.9267	0.9270
No log	6.0	294	0.2311	0.9258	0.9259
No log	7.0	343	0.2463	0.926	0.9261
No log	8.0	392	0.2757	0.9237	0.9252
No log	9.0	441	0.2940	0.9224	0.9241
No log	10.0	490	0.3138	0.9232	0.9250
0.132	11.0	539	0.3189	0.9256	0.9267
0.132	12.0	588	0.3139	0.9264	0.9272
0.132	13.0	637	0.3534	0.9203	0.9225
0.132	14.0	686	0.3330	0.9263	0.9260
0.132	15.0	735	0.3483	0.9242	0.9228
0.132	16.0	784	0.3483	0.9257	0.9261
0.132	17.0	833	0.3528	0.9261	0.9261
0.132	18.0	882	0.3479	0.9274	0.9276
0.132	19.0	931	0.3592	0.9246	0.9262
0.132	20.0	980	0.3537	0.9272	0.9270
0.0211	21.0	1029	0.3574	0.9271	0.9268
0.0211	22.0	1078	0.3615	0.9273	0.9281
0.0211	23.0	1127	0.3684	0.9281	0.9276
0.0211	24.0	1176	0.3753	0.9270	0.9281
0.0211	25.0	1225	0.3774	0.9278	0.9282
0.0211	26.0	1274	0.3893	0.9284	0.9289
0.0211	27.0	1323	0.3882	0.9282	0.9275
0.0211	28.0	1372	0.3900	0.927	0.9280
0.0211	29.0	1421	0.3910	0.9272	0.9282
0.0211	30.0	1470	0.3970	0.9279	0.9289
0.0112	31.0	1519	0.3985	0.9295	0.9300
0.0112	32.0	1568	0.4030	0.9288	0.9286
0.0112	33.0	1617	0.4075	0.9284	0.9283
0.0112	34.0	1666	0.4183	0.9273	0.9277
0.0112	35.0	1715	0.4235	0.9261	0.9269
0.0112	36.0	1764	0.4316	0.9272	0.9268
0.0112	37.0	1813	0.4231	0.9286	0.9285
0.0112	38.0	1862	0.4222	0.9289	0.9290
0.0112	39.0	1911	0.4256	0.9294	0.9290
0.0112	40.0	1960	0.4314	0.9288	0.9291
0.0053	41.0	2009	0.4291	0.9286	0.9288
0.0053	42.0	2058	0.4483	0.9266	0.9277
0.0053	43.0	2107	0.4392	0.9282	0.9287
0.0053	44.0	2156	0.4453	0.9282	0.9286
0.0053	45.0	2205	0.4562	0.9265	0.9276
0.0053	46.0	2254	0.4564	0.9264	0.9275
0.0053	47.0	2303	0.4471	0.9278	0.9281
0.0053	48.0	2352	0.4473	0.928	0.9282
0.0053	49.0	2401	0.4506	0.9281	0.9285
0.0053	50.0	2450	0.4498	0.9279	0.9283

Framework versions

• Transformers 4.33.2
• Pytorch 1.13.1+cu117
• Datasets 2.19.1
• Tokenizers 0.13.3