clinc-oos-distilbert-classifier

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.8628
• Accuracy: 0.8885
• F1 Macro: 0.7586
• F1 Out Of Scope: 0.5816
• F1 In Scope: 0.9357

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 64
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 200
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1 Macro	F1 Out Of Scope	F1 In Scope
0.1631	0.4982	275	0.1125	0.97	0.7760	0.5674	0.9845
0.0264	0.9964	550	0.1217	0.9787	0.8190	0.6489	0.9890
0.0167	1.4946	825	0.1408	0.9813	0.8266	0.6628	0.9904
0.0061	1.9928	1100	0.1509	0.9816	0.8460	0.7016	0.9905
0.0045	2.4909	1375	0.1817	0.9787	0.7803	0.5714	0.9891
0.0044	2.9891	1650	0.1642	0.9823	0.8346	0.6784	0.9909
0.0004	3.4873	1925	0.1997	0.9784	0.7783	0.5677	0.9889
0.0001	3.9855	2200	0.1902	0.9813	0.8457	0.7010	0.9903
0.0037	4.4837	2475	0.2113	0.9797	0.7942	0.5987	0.9896
0.0013	4.9819	2750	0.2232	0.9790	0.7794	0.5695	0.9893

Framework versions

• Transformers 5.6.2
• Pytorch 2.11.0+cu130
• Datasets 4.8.4
• Tokenizers 0.22.2