bert-medical-ner

This model is a fine-tuned version of distilbert-base-cased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.1905
• Precision: 0.6552
• Recall: 0.6965
• F1: 0.6752
• Accuracy: 0.7449

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
No log	1.0	71	1.8255	0.3427	0.4460	0.3876	0.5555
No log	2.0	142	1.3139	0.4722	0.5703	0.5166	0.6442
No log	3.0	213	1.1147	0.5258	0.6029	0.5617	0.6886
No log	4.0	284	0.9873	0.5785	0.6151	0.5962	0.7048
No log	5.0	355	0.9282	0.6314	0.6558	0.6434	0.7312
No log	6.0	426	0.8760	0.642	0.6538	0.6478	0.7329
No log	7.0	497	0.8501	0.6608	0.6904	0.6753	0.7466
1.1706	8.0	568	0.8313	0.6791	0.7067	0.6926	0.7483
1.1706	9.0	639	0.8002	0.6616	0.7047	0.6824	0.7449
1.1706	10.0	710	0.8280	0.6640	0.6721	0.6680	0.7363
1.1706	11.0	781	0.8248	0.6594	0.6823	0.6707	0.7457
1.1706	12.0	852	0.7988	0.6610	0.7189	0.6888	0.7654
1.1706	13.0	923	0.8593	0.6587	0.6762	0.6673	0.7423
1.1706	14.0	994	0.8204	0.6719	0.6965	0.6840	0.7534
0.4317	15.0	1065	0.8478	0.6770	0.7128	0.6944	0.7526
0.4317	16.0	1136	0.8855	0.6610	0.7149	0.6869	0.7730
0.4317	17.0	1207	0.9091	0.6751	0.7067	0.6905	0.7560
0.4317	18.0	1278	0.9201	0.6555	0.7169	0.6848	0.7568
0.4317	19.0	1349	0.9840	0.6623	0.7189	0.6895	0.7483
0.4317	20.0	1420	0.9817	0.6833	0.7251	0.7036	0.7543
0.4317	21.0	1491	0.9958	0.6583	0.6945	0.6759	0.7509
0.2121	22.0	1562	0.9340	0.6647	0.7026	0.6832	0.7722
0.2121	23.0	1633	0.9906	0.6622	0.7108	0.6857	0.7619
0.2121	24.0	1704	1.0099	0.6692	0.7088	0.6884	0.7526
0.2121	25.0	1775	1.0627	0.6673	0.7189	0.6922	0.7662
0.2121	26.0	1846	1.0744	0.6584	0.7067	0.6817	0.7637
0.2121	27.0	1917	1.1328	0.6569	0.6864	0.6713	0.7389
0.2121	28.0	1988	1.0799	0.6641	0.7128	0.6876	0.7577
0.1201	29.0	2059	1.1156	0.6628	0.7047	0.6831	0.7568
0.1201	30.0	2130	1.0839	0.6628	0.6965	0.6792	0.75
0.1201	31.0	2201	1.1511	0.6526	0.6925	0.6719	0.7389
0.1201	32.0	2272	1.1140	0.6737	0.7149	0.6937	0.7543
0.1201	33.0	2343	1.1094	0.6609	0.6986	0.6792	0.7466
0.1201	34.0	2414	1.1332	0.6755	0.7251	0.6994	0.7534
0.1201	35.0	2485	1.1322	0.6841	0.7189	0.7011	0.7551
0.0776	36.0	2556	1.1603	0.6711	0.7189	0.6942	0.7551
0.0776	37.0	2627	1.1460	0.6504	0.7047	0.6764	0.7543
0.0776	38.0	2698	1.1387	0.6584	0.7067	0.6817	0.7577
0.0776	39.0	2769	1.1438	0.6641	0.7088	0.6857	0.7534
0.0776	40.0	2840	1.1791	0.6660	0.7149	0.6896	0.7577
0.0776	41.0	2911	1.1701	0.6641	0.7088	0.6857	0.75
0.0776	42.0	2982	1.1889	0.6615	0.6965	0.6786	0.7457
0.0571	43.0	3053	1.1810	0.6533	0.6945	0.6732	0.7449
0.0571	44.0	3124	1.1944	0.6577	0.6965	0.6766	0.7440
0.0571	45.0	3195	1.2032	0.6564	0.6925	0.6739	0.7432
0.0571	46.0	3266	1.2092	0.6609	0.6945	0.6773	0.7449
0.0571	47.0	3337	1.1864	0.6622	0.6986	0.6799	0.7466
0.0571	48.0	3408	1.1972	0.6538	0.6925	0.6726	0.7449
0.0571	49.0	3479	1.1899	0.6545	0.6945	0.6739	0.7449
0.0467	50.0	3550	1.1905	0.6552	0.6965	0.6752	0.7449

Framework versions

• Transformers 4.29.2
• Pytorch 2.0.1+cu117
• Datasets 2.12.0
• Tokenizers 0.13.3