distilbert-base-uncased-Distilbert-Model

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7383
• F1: 0.6823

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	F1
0.848	0.5015	500	0.7910	0.6663
0.7872	1.0030	1000	0.7383	0.6823
0.6766	1.5045	1500	0.7502	0.7054
0.6854	2.0060	2000	0.7424	0.7096
0.5239	2.5075	2500	0.9047	0.7219
0.525	3.0090	3000	0.8375	0.7221
0.3925	3.5105	3500	1.0093	0.7216
0.4061	4.0120	4000	1.1403	0.7245
0.2928	4.5135	4500	1.3150	0.6862
0.3055	5.0150	5000	1.3811	0.7101
0.2184	5.5165	5500	1.5753	0.6985
0.23	6.0181	6000	1.5571	0.7122
0.1705	6.5196	6500	1.6771	0.7155
0.1416	7.0211	7000	1.7773	0.7089
0.1085	7.5226	7500	1.9134	0.7124
0.1437	8.0241	8000	1.8510	0.7118
0.0967	8.5256	8500	2.0276	0.7074
0.0733	9.0271	9000	2.1793	0.7112
0.0671	9.5286	9500	2.1100	0.7118

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.1+cpu
• Datasets 2.20.0
• Tokenizers 0.19.1