distilbert-base-uncased-airlines-news-multi-label

This model is a fine-tuned version of distilbert/distilbert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4164
• F1: 0.6705
• Roc Auc: 0.7913
• Accuracy: 0.6468

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 7e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 150
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	F1	Roc Auc	Accuracy
No log	1.0	59	0.3846	0.0	0.5	0.4468
No log	2.0	118	0.2943	0.2969	0.5876	0.5191
No log	3.0	177	0.2469	0.5548	0.7060	0.5745
No log	4.0	236	0.2451	0.575	0.7283	0.5787
No log	5.0	295	0.2360	0.6488	0.7739	0.6085
No log	6.0	354	0.2463	0.6190	0.7586	0.5915
No log	7.0	413	0.2724	0.6414	0.7741	0.6213
No log	8.0	472	0.2846	0.6435	0.7764	0.6085
0.1953	9.0	531	0.2961	0.6667	0.7942	0.6426
0.1953	10.0	590	0.3187	0.6627	0.7823	0.6298
0.1953	11.0	649	0.3204	0.6609	0.7874	0.6170
0.1953	12.0	708	0.3497	0.6529	0.7784	0.6298
0.1953	13.0	767	0.3465	0.6589	0.7833	0.6383
0.1953	14.0	826	0.3617	0.6494	0.7813	0.6298
0.1953	15.0	885	0.3759	0.6514	0.7836	0.6383
0.1953	16.0	944	0.3715	0.6512	0.7799	0.6213
0.008	17.0	1003	0.3808	0.6609	0.7856	0.6426
0.008	18.0	1062	0.3850	0.6629	0.7915	0.6383
0.008	19.0	1121	0.3958	0.6553	0.7862	0.6340
0.008	20.0	1180	0.3915	0.6610	0.7893	0.6340
0.008	21.0	1239	0.4016	0.6477	0.7827	0.6255
0.008	22.0	1298	0.4060	0.6496	0.7831	0.6255
0.008	23.0	1357	0.4058	0.6667	0.7923	0.6468
0.008	24.0	1416	0.4119	0.6667	0.7887	0.6468
0.008	25.0	1475	0.4094	0.6648	0.7901	0.6426
0.0021	26.0	1534	0.4151	0.6686	0.7891	0.6511
0.0021	27.0	1593	0.4146	0.6648	0.7901	0.6426
0.0021	28.0	1652	0.4164	0.6705	0.7913	0.6468
0.0021	29.0	1711	0.4174	0.6667	0.7905	0.6426
0.0021	30.0	1770	0.4171	0.6686	0.7928	0.6426

Framework versions

• Transformers 4.41.1
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1