distilbert-ner-qlorafinetune-runs

This model is a fine-tuned version of distilbert-base-uncased on the conll2002 dataset. It achieves the following results on the evaluation set:

• Loss: 0.2164
• Precision: 0.6299
• Recall: 0.6227
• F1: 0.6263
• Accuracy: 0.9372

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0004
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Use paged_adamw_8bit with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• training_steps: 640
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
1.0711	0.0766	20	0.7111	0.0	0.0	0.0	0.8570
0.5291	0.1533	40	0.5467	0.0	0.0	0.0	0.8570
0.4545	0.2299	60	0.4850	0.2172	0.1687	0.1899	0.8769
0.4113	0.3065	80	0.4470	0.3227	0.1765	0.2282	0.8816
0.3837	0.3831	100	0.4049	0.4187	0.3840	0.4006	0.8896
0.334	0.4598	120	0.3639	0.4695	0.4276	0.4476	0.8981
0.3342	0.5364	140	0.3499	0.5104	0.4520	0.4794	0.8997
0.322	0.6130	160	0.3281	0.4939	0.4920	0.4929	0.9091
0.2868	0.6897	180	0.3021	0.5207	0.4646	0.4911	0.9145
0.2788	0.7663	200	0.2878	0.5361	0.5064	0.5209	0.9185
0.2748	0.8429	220	0.2864	0.5419	0.5232	0.5324	0.9197
0.2435	0.9195	240	0.2750	0.5306	0.5294	0.5300	0.9205
0.238	0.9962	260	0.2636	0.5525	0.5623	0.5573	0.9239
0.2465	1.0728	280	0.2616	0.5574	0.5602	0.5588	0.9255
0.2296	1.1494	300	0.2607	0.5859	0.5409	0.5625	0.9252
0.2141	1.2261	320	0.2491	0.5728	0.5841	0.5784	0.9279
0.2229	1.3027	340	0.2483	0.5849	0.5767	0.5808	0.9289
0.2234	1.3793	360	0.2413	0.5906	0.5712	0.5808	0.9310
0.2217	1.4559	380	0.2416	0.5890	0.5944	0.5917	0.9321
0.208	1.5326	400	0.2337	0.6117	0.5889	0.6001	0.9326
0.1961	1.6092	420	0.2387	0.5950	0.6018	0.5984	0.9321
0.2237	1.6858	440	0.2263	0.6230	0.6094	0.6161	0.9353
0.2029	1.7625	460	0.2262	0.6377	0.6045	0.6207	0.9353
0.203	1.8391	480	0.2229	0.6246	0.6167	0.6206	0.9358
0.2098	1.9157	500	0.2221	0.6277	0.6264	0.6270	0.9363
0.1907	1.9923	520	0.2237	0.6197	0.6186	0.6191	0.9355
0.1774	2.0690	540	0.2214	0.6284	0.6170	0.6226	0.9365
0.1822	2.1456	560	0.2213	0.6267	0.6211	0.6239	0.9368
0.1783	2.2222	580	0.2180	0.6308	0.6266	0.6287	0.9371
0.1856	2.2989	600	0.2174	0.6289	0.6206	0.6247	0.9369
0.1773	2.3755	620	0.2172	0.6192	0.6278	0.6235	0.9362
0.1647	2.4521	640	0.2164	0.6299	0.6227	0.6263	0.9372

Framework versions

• PEFT 0.13.2
• Transformers 4.46.3
• Pytorch 2.5.1
• Datasets 3.1.0
• Tokenizers 0.20.3