bert-finetuned-ner-2

This model is a fine-tuned version of bert-base-cased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5673
• Precision: 0.6272
• Recall: 0.5072
• F1: 0.5608
• Accuracy: 0.9260
• By Entity: {'corporation': {'precision': 0.24390243902439024, 'recall': 0.29411764705882354, 'f1': 0.2666666666666666, 'number': 34}, 'creative-work': {'precision': 0.3888888888888889, 'recall': 0.2, 'f1': 0.2641509433962264, 'number': 105}, 'group': {'precision': 0.225, 'recall': 0.23076923076923078, 'f1': 0.22784810126582278, 'number': 39}, 'location': {'precision': 0.6571428571428571, 'recall': 0.6216216216216216, 'f1': 0.6388888888888888, 'number': 74}, 'person': {'precision': 0.7786259541984732, 'recall': 0.6510638297872341, 'f1': 0.709154113557358, 'number': 470}, 'product': {'precision': 0.41025641025641024, 'recall': 0.2807017543859649, 'f1': 0.3333333333333333, 'number': 114}}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy	By Entity
No log	1.0	425	0.3397	0.5156	0.3361	0.4070	0.9131	{'corporation': {'precision': 0.09523809523809523, 'recall': 0.11764705882352941, 'f1': 0.10526315789473684, 'number': 34}, 'creative-work': {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 105}, 'group': {'precision': 0.05970149253731343, 'recall': 0.10256410256410256, 'f1': 0.07547169811320753, 'number': 39}, 'location': {'precision': 0.5119047619047619, 'recall': 0.581081081081081, 'f1': 0.5443037974683543, 'number': 74}, 'person': {'precision': 0.7350993377483444, 'recall': 0.4723404255319149, 'f1': 0.5751295336787564, 'number': 470}, 'product': {'precision': 0.2962962962962963, 'recall': 0.07017543859649122, 'f1': 0.11347517730496454, 'number': 114}}
0.1992	2.0	850	0.3657	0.6123	0.4402	0.5122	0.9230	{'corporation': {'precision': 0.13846153846153847, 'recall': 0.2647058823529412, 'f1': 0.18181818181818182, 'number': 34}, 'creative-work': {'precision': 0.5185185185185185, 'recall': 0.13333333333333333, 'f1': 0.21212121212121213, 'number': 105}, 'group': {'precision': 0.25, 'recall': 0.15384615384615385, 'f1': 0.1904761904761905, 'number': 39}, 'location': {'precision': 0.6507936507936508, 'recall': 0.5540540540540541, 'f1': 0.5985401459854015, 'number': 74}, 'person': {'precision': 0.7548209366391184, 'recall': 0.5829787234042553, 'f1': 0.6578631452581032, 'number': 470}, 'product': {'precision': 0.4067796610169492, 'recall': 0.21052631578947367, 'f1': 0.27745664739884396, 'number': 114}}
0.0783	3.0	1275	0.3762	0.5803	0.5012	0.5379	0.9278	{'corporation': {'precision': 0.24, 'recall': 0.17647058823529413, 'f1': 0.20338983050847456, 'number': 34}, 'creative-work': {'precision': 0.28888888888888886, 'recall': 0.24761904761904763, 'f1': 0.26666666666666666, 'number': 105}, 'group': {'precision': 0.16279069767441862, 'recall': 0.1794871794871795, 'f1': 0.17073170731707318, 'number': 39}, 'location': {'precision': 0.7, 'recall': 0.5675675675675675, 'f1': 0.626865671641791, 'number': 74}, 'person': {'precision': 0.7383863080684596, 'recall': 0.6425531914893617, 'f1': 0.6871444823663253, 'number': 470}, 'product': {'precision': 0.37894736842105264, 'recall': 0.3157894736842105, 'f1': 0.3444976076555024, 'number': 114}}
0.0449	4.0	1700	0.4777	0.5914	0.4761	0.5275	0.9250	{'corporation': {'precision': 0.14035087719298245, 'recall': 0.23529411764705882, 'f1': 0.1758241758241758, 'number': 34}, 'creative-work': {'precision': 0.3559322033898305, 'recall': 0.2, 'f1': 0.25609756097560976, 'number': 105}, 'group': {'precision': 0.225, 'recall': 0.23076923076923078, 'f1': 0.22784810126582278, 'number': 39}, 'location': {'precision': 0.676923076923077, 'recall': 0.5945945945945946, 'f1': 0.6330935251798562, 'number': 74}, 'person': {'precision': 0.7553763440860215, 'recall': 0.597872340425532, 'f1': 0.667458432304038, 'number': 470}, 'product': {'precision': 0.4375, 'recall': 0.30701754385964913, 'f1': 0.36082474226804123, 'number': 114}}
0.0221	5.0	2125	0.4407	0.5666	0.5191	0.5418	0.9271	{'corporation': {'precision': 0.2, 'recall': 0.23529411764705882, 'f1': 0.2162162162162162, 'number': 34}, 'creative-work': {'precision': 0.3188405797101449, 'recall': 0.20952380952380953, 'f1': 0.2528735632183908, 'number': 105}, 'group': {'precision': 0.15873015873015872, 'recall': 0.2564102564102564, 'f1': 0.196078431372549, 'number': 39}, 'location': {'precision': 0.5802469135802469, 'recall': 0.6351351351351351, 'f1': 0.6064516129032258, 'number': 74}, 'person': {'precision': 0.7162471395881007, 'recall': 0.6659574468085107, 'f1': 0.6901874310915104, 'number': 470}, 'product': {'precision': 0.4473684210526316, 'recall': 0.2982456140350877, 'f1': 0.35789473684210527, 'number': 114}}
0.0136	6.0	2550	0.4757	0.5988	0.4964	0.5428	0.9266	{'corporation': {'precision': 0.15384615384615385, 'recall': 0.17647058823529413, 'f1': 0.1643835616438356, 'number': 34}, 'creative-work': {'precision': 0.3142857142857143, 'recall': 0.20952380952380953, 'f1': 0.25142857142857145, 'number': 105}, 'group': {'precision': 0.25, 'recall': 0.23076923076923078, 'f1': 0.24000000000000002, 'number': 39}, 'location': {'precision': 0.6619718309859155, 'recall': 0.6351351351351351, 'f1': 0.6482758620689655, 'number': 74}, 'person': {'precision': 0.7750677506775068, 'recall': 0.6085106382978723, 'f1': 0.6817640047675805, 'number': 470}, 'product': {'precision': 0.4166666666666667, 'recall': 0.39473684210526316, 'f1': 0.40540540540540543, 'number': 114}}
0.0136	7.0	2975	0.4970	0.5776	0.5120	0.5428	0.9273	{'corporation': {'precision': 0.16129032258064516, 'recall': 0.29411764705882354, 'f1': 0.20833333333333331, 'number': 34}, 'creative-work': {'precision': 0.3088235294117647, 'recall': 0.2, 'f1': 0.24277456647398846, 'number': 105}, 'group': {'precision': 0.24324324324324326, 'recall': 0.23076923076923078, 'f1': 0.23684210526315788, 'number': 39}, 'location': {'precision': 0.71875, 'recall': 0.6216216216216216, 'f1': 0.6666666666666667, 'number': 74}, 'person': {'precision': 0.7349397590361446, 'recall': 0.648936170212766, 'f1': 0.6892655367231639, 'number': 470}, 'product': {'precision': 0.3894736842105263, 'recall': 0.32456140350877194, 'f1': 0.35406698564593303, 'number': 114}}
0.0086	8.0	3400	0.5607	0.6195	0.5024	0.5548	0.9266	{'corporation': {'precision': 0.21428571428571427, 'recall': 0.2647058823529412, 'f1': 0.2368421052631579, 'number': 34}, 'creative-work': {'precision': 0.3389830508474576, 'recall': 0.19047619047619047, 'f1': 0.24390243902439024, 'number': 105}, 'group': {'precision': 0.23076923076923078, 'recall': 0.23076923076923078, 'f1': 0.23076923076923078, 'number': 39}, 'location': {'precision': 0.6428571428571429, 'recall': 0.6081081081081081, 'f1': 0.625, 'number': 74}, 'person': {'precision': 0.7791878172588832, 'recall': 0.6531914893617021, 'f1': 0.710648148148148, 'number': 470}, 'product': {'precision': 0.40540540540540543, 'recall': 0.2631578947368421, 'f1': 0.3191489361702128, 'number': 114}}
0.0047	9.0	3825	0.5584	0.6114	0.5024	0.5515	0.9260	{'corporation': {'precision': 0.20454545454545456, 'recall': 0.2647058823529412, 'f1': 0.23076923076923078, 'number': 34}, 'creative-work': {'precision': 0.3508771929824561, 'recall': 0.19047619047619047, 'f1': 0.24691358024691357, 'number': 105}, 'group': {'precision': 0.23684210526315788, 'recall': 0.23076923076923078, 'f1': 0.23376623376623376, 'number': 39}, 'location': {'precision': 0.6428571428571429, 'recall': 0.6081081081081081, 'f1': 0.625, 'number': 74}, 'person': {'precision': 0.7682619647355163, 'recall': 0.648936170212766, 'f1': 0.7035755478662054, 'number': 470}, 'product': {'precision': 0.3950617283950617, 'recall': 0.2807017543859649, 'f1': 0.3282051282051282, 'number': 114}}
0.0038	10.0	4250	0.5673	0.6272	0.5072	0.5608	0.9260	{'corporation': {'precision': 0.24390243902439024, 'recall': 0.29411764705882354, 'f1': 0.2666666666666666, 'number': 34}, 'creative-work': {'precision': 0.3888888888888889, 'recall': 0.2, 'f1': 0.2641509433962264, 'number': 105}, 'group': {'precision': 0.225, 'recall': 0.23076923076923078, 'f1': 0.22784810126582278, 'number': 39}, 'location': {'precision': 0.6571428571428571, 'recall': 0.6216216216216216, 'f1': 0.6388888888888888, 'number': 74}, 'person': {'precision': 0.7786259541984732, 'recall': 0.6510638297872341, 'f1': 0.709154113557358, 'number': 470}, 'product': {'precision': 0.41025641025641024, 'recall': 0.2807017543859649, 'f1': 0.3333333333333333, 'number': 114}}

Framework versions

• Transformers 4.35.0
• Pytorch 2.1.0+cu118
• Datasets 2.14.6
• Tokenizers 0.14.1