bert-finetuned-arc-ner

This model is a fine-tuned version of bert-base-cased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2353
• Precision: 0.4425
• Recall: 0.6553
• F1: 0.5283
• Accuracy: 0.9375
• Classification Report Details: {'B-ART': {'precision': 0.4732824427480916, 'recall': 0.7380952380952381, 'f1-score': 0.5767441860465117, 'support': 252.0}, 'B-CON': {'precision': 0.3142857142857143, 'recall': 0.6832298136645962, 'f1-score': 0.43052837573385516, 'support': 161.0}, 'B-LOC': {'precision': 0.7553956834532374, 'recall': 0.7046979865771812, 'f1-score': 0.7291666666666666, 'support': 149.0}, 'B-MAT': {'precision': 0.5, 'recall': 0.025, 'f1-score': 0.047619047619047616, 'support': 40.0}, 'B-PER': {'precision': 0.7688311688311689, 'recall': 0.8809523809523809, 'f1-score': 0.8210818307905686, 'support': 336.0}, 'B-SPE': {'precision': 0.46511627906976744, 'recall': 0.6451612903225806, 'f1-score': 0.5405405405405406, 'support': 31.0}, 'I-ART': {'precision': 0.5818181818181818, 'recall': 0.45325779036827196, 'f1-score': 0.5095541401273885, 'support': 353.0}, 'I-CON': {'precision': 0.38125, 'recall': 0.5169491525423728, 'f1-score': 0.43884892086330934, 'support': 118.0}, 'I-LOC': {'precision': 0.7925311203319502, 'recall': 0.7519685039370079, 'f1-score': 0.7717171717171717, 'support': 254.0}, 'I-MAT': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 37.0}, 'I-PER': {'precision': 0.8316831683168316, 'recall': 0.7516778523489933, 'f1-score': 0.7896592244418331, 'support': 447.0}, 'I-SPE': {'precision': 1.0, 'recall': 0.14285714285714285, 'f1-score': 0.25, 'support': 42.0}, 'O': {'precision': 0.9752960093553574, 'recall': 0.9669098111202358, 'f1-score': 0.9710848049679798, 'support': 20701.0}, 'accuracy': 0.937480912700144, 'macro avg': {'precision': 0.6030376744777155, 'recall': 0.558519766368154, 'f1-score': 0.5289649930396056, 'support': 22921.0}, 'weighted avg': {'precision': 0.9436850180373675, 'recall': 0.937480912700144, 'f1-score': 0.9385027973137299, 'support': 22921.0}}
• Classfication Report Seqeval: {'ART': {'precision': 0.36556603773584906, 'recall': 0.6150793650793651, 'f1-score': 0.45857988165680474, 'support': 252}, 'CON': {'precision': 0.2584856396866841, 'recall': 0.6149068322981367, 'f1-score': 0.36397058823529416, 'support': 161}, 'LOC': {'precision': 0.5508982035928144, 'recall': 0.6174496644295302, 'f1-score': 0.5822784810126582, 'support': 149}, 'MAT': {'precision': 0.5, 'recall': 0.025, 'f1-score': 0.047619047619047616, 'support': 40}, 'PER': {'precision': 0.6658595641646489, 'recall': 0.8184523809523809, 'f1-score': 0.7343124165554071, 'support': 336}, 'SPE': {'precision': 0.2826086956521739, 'recall': 0.41935483870967744, 'f1-score': 0.33766233766233766, 'support': 31}, 'micro avg': {'precision': 0.4425087108013937, 'recall': 0.6553147574819401, 'f1-score': 0.5282861896838602, 'support': 969}, 'macro avg': {'precision': 0.4372363568053617, 'recall': 0.518373846911515, 'f1-score': 0.42073712545692493, 'support': 969}, 'weighted avg': {'precision': 0.48329447364175315, 'recall': 0.6553147574819401, 'f1-score': 0.536658570577084, 'support': 969}}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 9.009263833878603e-06
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy	Classification Report Details	Classfication Report Seqeval
No log	1.0	249	0.2564	0.3916	0.5666	0.4631	0.9328	{'B-ART': {'precision': 0.43356643356643354, 'recall': 0.7380952380952381, 'f1-score': 0.5462555066079295, 'support': 252.0}, 'B-CON': {'precision': 0.2956521739130435, 'recall': 0.6335403726708074, 'f1-score': 0.4031620553359684, 'support': 161.0}, 'B-LOC': {'precision': 0.7058823529411765, 'recall': 0.1610738255033557, 'f1-score': 0.26229508196721313, 'support': 149.0}, 'B-MAT': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 40.0}, 'B-PER': {'precision': 0.7170263788968825, 'recall': 0.8898809523809523, 'f1-score': 0.7941567065073041, 'support': 336.0}, 'B-SPE': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 31.0}, 'I-ART': {'precision': 0.5747663551401869, 'recall': 0.34844192634560905, 'f1-score': 0.43386243386243384, 'support': 353.0}, 'I-CON': {'precision': 0.5797101449275363, 'recall': 0.3389830508474576, 'f1-score': 0.42780748663101603, 'support': 118.0}, 'I-LOC': {'precision': 0.6535714285714286, 'recall': 0.7204724409448819, 'f1-score': 0.6853932584269663, 'support': 254.0}, 'I-MAT': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 37.0}, 'I-PER': {'precision': 0.8485714285714285, 'recall': 0.6644295302013423, 'f1-score': 0.7452948557089084, 'support': 447.0}, 'I-SPE': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 42.0}, 'O': {'precision': 0.968387624500794, 'recall': 0.97222356407903, 'f1-score': 0.9703018031048115, 'support': 20701.0}, 'accuracy': 0.9327690763928276, 'macro avg': {'precision': 0.44439494777145466, 'recall': 0.4205493000822057, 'f1-score': 0.40527147601173474, 'support': 22921.0}, 'weighted avg': {'precision': 0.9321654893696512, 'recall': 0.9327690763928276, 'f1-score': 0.9295219726872646, 'support': 22921.0}}	{'ART': {'precision': 0.3006535947712418, 'recall': 0.5476190476190477, 'f1-score': 0.3881856540084388, 'support': 252}, 'CON': {'precision': 0.2219178082191781, 'recall': 0.5031055900621118, 'f1-score': 0.3079847908745247, 'support': 161}, 'LOC': {'precision': 0.4755244755244755, 'recall': 0.4563758389261745, 'f1-score': 0.4657534246575343, 'support': 149}, 'MAT': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 40}, 'PER': {'precision': 0.6022988505747127, 'recall': 0.7797619047619048, 'f1-score': 0.6796368352788588, 'support': 336}, 'SPE': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 31}, 'micro avg': {'precision': 0.39158345221112695, 'recall': 0.56656346749226, 'f1-score': 0.463095740194011, 'support': 969}, 'macro avg': {'precision': 0.266732454848268, 'recall': 0.3811437302282064, 'f1-score': 0.30692678413655944, 'support': 969}, 'weighted avg': {'precision': 0.3970268665138193, 'recall': 0.56656346749226, 'f1-score': 0.45940513216573187, 'support': 969}}
No log	2.0	498	0.2325	0.4422	0.6512	0.5267	0.9378	{'B-ART': {'precision': 0.4881889763779528, 'recall': 0.7380952380952381, 'f1-score': 0.5876777251184834, 'support': 252.0}, 'B-CON': {'precision': 0.3064516129032258, 'recall': 0.7080745341614907, 'f1-score': 0.4277673545966229, 'support': 161.0}, 'B-LOC': {'precision': 0.7481481481481481, 'recall': 0.6778523489932886, 'f1-score': 0.7112676056338029, 'support': 149.0}, 'B-MAT': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 40.0}, 'B-PER': {'precision': 0.742014742014742, 'recall': 0.8988095238095238, 'f1-score': 0.8129205921938089, 'support': 336.0}, 'B-SPE': {'precision': 0.65, 'recall': 0.41935483870967744, 'f1-score': 0.5098039215686274, 'support': 31.0}, 'I-ART': {'precision': 0.6261261261261262, 'recall': 0.3937677053824363, 'f1-score': 0.4834782608695652, 'support': 353.0}, 'I-CON': {'precision': 0.4357142857142857, 'recall': 0.5169491525423728, 'f1-score': 0.4728682170542636, 'support': 118.0}, 'I-LOC': {'precision': 0.7578125, 'recall': 0.7637795275590551, 'f1-score': 0.7607843137254902, 'support': 254.0}, 'I-MAT': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 37.0}, 'I-PER': {'precision': 0.8354755784061697, 'recall': 0.727069351230425, 'f1-score': 0.777511961722488, 'support': 447.0}, 'I-SPE': {'precision': 1.0, 'recall': 0.047619047619047616, 'f1-score': 0.09090909090909091, 'support': 42.0}, 'O': {'precision': 0.9738311404573482, 'recall': 0.9689386986135935, 'f1-score': 0.9713787592619497, 'support': 20701.0}, 'accuracy': 0.9377863094978404, 'macro avg': {'precision': 0.5818279315498461, 'recall': 0.5277161512858576, 'f1-score': 0.5081821386657072, 'support': 22921.0}, 'weighted avg': {'precision': 0.9420601441694352, 'recall': 0.9377863094978404, 'f1-score': 0.9376324183142309, 'support': 22921.0}}	{'ART': {'precision': 0.3602941176470588, 'recall': 0.5833333333333334, 'f1-score': 0.4454545454545455, 'support': 252}, 'CON': {'precision': 0.2594458438287154, 'recall': 0.639751552795031, 'f1-score': 0.3691756272401434, 'support': 161}, 'LOC': {'precision': 0.5402298850574713, 'recall': 0.6308724832214765, 'f1-score': 0.5820433436532508, 'support': 149}, 'MAT': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 40}, 'PER': {'precision': 0.647887323943662, 'recall': 0.8214285714285714, 'f1-score': 0.7244094488188976, 'support': 336}, 'SPE': {'precision': 0.5, 'recall': 0.3548387096774194, 'f1-score': 0.41509433962264153, 'support': 31}, 'micro avg': {'precision': 0.4421864050455501, 'recall': 0.651186790505676, 'f1-score': 0.5267111853088481, 'support': 969}, 'macro avg': {'precision': 0.38464286174615125, 'recall': 0.5050374417426386, 'f1-score': 0.42269621746491315, 'support': 969}, 'weighted avg': {'precision': 0.4605255853685404, 'recall': 0.651186790505676, 'f1-score': 0.5311514746914286, 'support': 969}}
0.2853	3.0	747	0.2353	0.4425	0.6553	0.5283	0.9375	{'B-ART': {'precision': 0.4732824427480916, 'recall': 0.7380952380952381, 'f1-score': 0.5767441860465117, 'support': 252.0}, 'B-CON': {'precision': 0.3142857142857143, 'recall': 0.6832298136645962, 'f1-score': 0.43052837573385516, 'support': 161.0}, 'B-LOC': {'precision': 0.7553956834532374, 'recall': 0.7046979865771812, 'f1-score': 0.7291666666666666, 'support': 149.0}, 'B-MAT': {'precision': 0.5, 'recall': 0.025, 'f1-score': 0.047619047619047616, 'support': 40.0}, 'B-PER': {'precision': 0.7688311688311689, 'recall': 0.8809523809523809, 'f1-score': 0.8210818307905686, 'support': 336.0}, 'B-SPE': {'precision': 0.46511627906976744, 'recall': 0.6451612903225806, 'f1-score': 0.5405405405405406, 'support': 31.0}, 'I-ART': {'precision': 0.5818181818181818, 'recall': 0.45325779036827196, 'f1-score': 0.5095541401273885, 'support': 353.0}, 'I-CON': {'precision': 0.38125, 'recall': 0.5169491525423728, 'f1-score': 0.43884892086330934, 'support': 118.0}, 'I-LOC': {'precision': 0.7925311203319502, 'recall': 0.7519685039370079, 'f1-score': 0.7717171717171717, 'support': 254.0}, 'I-MAT': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 37.0}, 'I-PER': {'precision': 0.8316831683168316, 'recall': 0.7516778523489933, 'f1-score': 0.7896592244418331, 'support': 447.0}, 'I-SPE': {'precision': 1.0, 'recall': 0.14285714285714285, 'f1-score': 0.25, 'support': 42.0}, 'O': {'precision': 0.9752960093553574, 'recall': 0.9669098111202358, 'f1-score': 0.9710848049679798, 'support': 20701.0}, 'accuracy': 0.937480912700144, 'macro avg': {'precision': 0.6030376744777155, 'recall': 0.558519766368154, 'f1-score': 0.5289649930396056, 'support': 22921.0}, 'weighted avg': {'precision': 0.9436850180373675, 'recall': 0.937480912700144, 'f1-score': 0.9385027973137299, 'support': 22921.0}}	{'ART': {'precision': 0.36556603773584906, 'recall': 0.6150793650793651, 'f1-score': 0.45857988165680474, 'support': 252}, 'CON': {'precision': 0.2584856396866841, 'recall': 0.6149068322981367, 'f1-score': 0.36397058823529416, 'support': 161}, 'LOC': {'precision': 0.5508982035928144, 'recall': 0.6174496644295302, 'f1-score': 0.5822784810126582, 'support': 149}, 'MAT': {'precision': 0.5, 'recall': 0.025, 'f1-score': 0.047619047619047616, 'support': 40}, 'PER': {'precision': 0.6658595641646489, 'recall': 0.8184523809523809, 'f1-score': 0.7343124165554071, 'support': 336}, 'SPE': {'precision': 0.2826086956521739, 'recall': 0.41935483870967744, 'f1-score': 0.33766233766233766, 'support': 31}, 'micro avg': {'precision': 0.4425087108013937, 'recall': 0.6553147574819401, 'f1-score': 0.5282861896838602, 'support': 969}, 'macro avg': {'precision': 0.4372363568053617, 'recall': 0.518373846911515, 'f1-score': 0.42073712545692493, 'support': 969}, 'weighted avg': {'precision': 0.48329447364175315, 'recall': 0.6553147574819401, 'f1-score': 0.536658570577084, 'support': 969}}

Framework versions

• Transformers 4.45.2
• Pytorch 2.4.1
• Datasets 3.0.1
• Tokenizers 0.20.1