distilbert-base-uncased-finetuned-clinc

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2759
• Accuracy: 0.9497

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 48
• eval_batch_size: 48
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
2.6757	1.0	318	1.9018	0.7606
1.4254	2.0	636	0.8887	0.8729
0.6809	3.0	954	0.4889	0.9190
0.3909	4.0	1272	0.3654	0.9423
0.2827	5.0	1590	0.3267	0.9471
0.2403	6.0	1908	0.3138	0.9465
0.2187	7.0	2226	0.3029	0.9477
0.2067	8.0	2544	0.2939	0.95
0.2001	9.0	2862	0.2933	0.9490
0.1951	10.0	3180	0.2909	0.9477
0.1912	11.0	3498	0.2861	0.9494
0.1884	12.0	3816	0.2853	0.9497
0.1861	13.0	4134	0.2872	0.9471
0.1845	14.0	4452	0.2815	0.9510
0.1833	15.0	4770	0.2829	0.9471
0.1821	16.0	5088	0.2836	0.9481
0.181	17.0	5406	0.2803	0.9474
0.1799	18.0	5724	0.2760	0.9510
0.1791	19.0	6042	0.2789	0.9506
0.1778	20.0	6360	0.2787	0.9506
0.1777	21.0	6678	0.2779	0.9506
0.177	22.0	6996	0.2768	0.9513
0.1765	23.0	7314	0.2762	0.9497
0.1763	24.0	7632	0.2769	0.9503
0.1758	25.0	7950	0.2779	0.9484
0.1754	26.0	8268	0.2754	0.95
0.1753	27.0	8586	0.2757	0.95
0.175	28.0	8904	0.2761	0.9503
0.1744	29.0	9222	0.2759	0.9494
0.1745	30.0	9540	0.2759	0.9497

Framework versions

• Transformers 4.16.2
• Pytorch 2.4.1+cu121
• Datasets 3.0.0
• Tokenizers 0.19.1