bert-finetuned-arc-ner-hp

This model is a fine-tuned version of bert-base-cased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2393
• Precision: 0.5161
• Recall: 0.6801
• F1: 0.5868
• Accuracy: 0.9464
• Classification Report Details: {'B-ART': {'precision': 0.5029411764705882, 'recall': 0.6785714285714286, 'f1-score': 0.5777027027027027, 'support': 252.0}, 'B-CON': {'precision': 0.4345991561181435, 'recall': 0.639751552795031, 'f1-score': 0.5175879396984925, 'support': 161.0}, 'B-LOC': {'precision': 0.7590361445783133, 'recall': 0.8456375838926175, 'f1-score': 0.8, 'support': 149.0}, 'B-MAT': {'precision': 0.4358974358974359, 'recall': 0.425, 'f1-score': 0.43037974683544306, 'support': 40.0}, 'B-PER': {'precision': 0.8026666666666666, 'recall': 0.8958333333333334, 'f1-score': 0.8466947960618847, 'support': 336.0}, 'B-SPE': {'precision': 0.4444444444444444, 'recall': 0.7741935483870968, 'f1-score': 0.5647058823529412, 'support': 31.0}, 'I-ART': {'precision': 0.5704225352112676, 'recall': 0.45892351274787535, 'f1-score': 0.5086342229199372, 'support': 353.0}, 'I-CON': {'precision': 0.4090909090909091, 'recall': 0.4576271186440678, 'f1-score': 0.432, 'support': 118.0}, 'I-LOC': {'precision': 0.8913043478260869, 'recall': 0.8070866141732284, 'f1-score': 0.8471074380165289, 'support': 254.0}, 'I-MAT': {'precision': 0.2727272727272727, 'recall': 0.08108108108108109, 'f1-score': 0.125, 'support': 37.0}, 'I-PER': {'precision': 0.8743718592964824, 'recall': 0.7785234899328859, 'f1-score': 0.8236686390532545, 'support': 447.0}, 'I-SPE': {'precision': 0.8285714285714286, 'recall': 0.6904761904761905, 'f1-score': 0.7532467532467533, 'support': 42.0}, 'O': {'precision': 0.9771580989330747, 'recall': 0.9733346215158688, 'f1-score': 0.9752426127150844, 'support': 20701.0}, 'accuracy': 0.9463810479472973, 'macro avg': {'precision': 0.6310178058332395, 'recall': 0.654310775042362, 'f1-score': 0.6309208256617709, 'support': 22921.0}, 'weighted avg': {'precision': 0.9489387820828759, 'recall': 0.9463810479472973, 'f1-score': 0.9469897039453234, 'support': 22921.0}}
• Classfication Report Seqeval: {'ART': {'precision': 0.42032967032967034, 'recall': 0.6071428571428571, 'f1-score': 0.4967532467532467, 'support': 252}, 'CON': {'precision': 0.36328125, 'recall': 0.577639751552795, 'f1-score': 0.44604316546762585, 'support': 161}, 'LOC': {'precision': 0.6149425287356322, 'recall': 0.7181208053691275, 'f1-score': 0.6625386996904025, 'support': 149}, 'MAT': {'precision': 0.3170731707317073, 'recall': 0.325, 'f1-score': 0.3209876543209877, 'support': 40}, 'PER': {'precision': 0.7109375, 'recall': 0.8125, 'f1-score': 0.7583333333333333, 'support': 336}, 'SPE': {'precision': 0.3448275862068966, 'recall': 0.6451612903225806, 'f1-score': 0.44943820224719105, 'support': 31}, 'micro avg': {'precision': 0.5160532498042286, 'recall': 0.6800825593395253, 'f1-score': 0.5868210151380232, 'support': 969}, 'macro avg': {'precision': 0.4618986176673177, 'recall': 0.6142607840645601, 'f1-score': 0.5223490503021312, 'support': 969}, 'weighted avg': {'precision': 0.5348662300891313, 'recall': 0.6800825593395253, 'f1-score': 0.5957534824752098, 'support': 969}}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy	Classification Report Details	Classfication Report Seqeval
No log	1.0	249	0.2085	0.4926	0.6543	0.5621	0.9450	{'B-ART': {'precision': 0.47956403269754766, 'recall': 0.6984126984126984, 'f1-score': 0.568659127625202, 'support': 252.0}, 'B-CON': {'precision': 0.4345991561181435, 'recall': 0.639751552795031, 'f1-score': 0.5175879396984925, 'support': 161.0}, 'B-LOC': {'precision': 0.8048780487804879, 'recall': 0.6644295302013423, 'f1-score': 0.7279411764705882, 'support': 149.0}, 'B-MAT': {'precision': 0.4418604651162791, 'recall': 0.475, 'f1-score': 0.4578313253012048, 'support': 40.0}, 'B-PER': {'precision': 0.7972972972972973, 'recall': 0.8779761904761905, 'f1-score': 0.8356940509915014, 'support': 336.0}, 'B-SPE': {'precision': 0.4339622641509434, 'recall': 0.7419354838709677, 'f1-score': 0.5476190476190477, 'support': 31.0}, 'I-ART': {'precision': 0.58984375, 'recall': 0.42776203966005666, 'f1-score': 0.49589490968801314, 'support': 353.0}, 'I-CON': {'precision': 0.5051546391752577, 'recall': 0.4152542372881356, 'f1-score': 0.4558139534883721, 'support': 118.0}, 'I-LOC': {'precision': 0.8851674641148325, 'recall': 0.7283464566929134, 'f1-score': 0.7991360691144709, 'support': 254.0}, 'I-MAT': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 37.0}, 'I-PER': {'precision': 0.9272727272727272, 'recall': 0.6845637583892618, 'f1-score': 0.7876447876447876, 'support': 447.0}, 'I-SPE': {'precision': 0.8571428571428571, 'recall': 0.7142857142857143, 'f1-score': 0.7792207792207793, 'support': 42.0}, 'O': {'precision': 0.9723557692307693, 'recall': 0.9770059417419449, 'f1-score': 0.9746753090286981, 'support': 20701.0}, 'accuracy': 0.9450285764146416, 'macro avg': {'precision': 0.6253152670074725, 'recall': 0.6188248926010965, 'f1-score': 0.6113629596839352, 'support': 22921.0}, 'weighted avg': {'precision': 0.9459294919647149, 'recall': 0.9450285764146416, 'f1-score': 0.9443111214415887, 'support': 22921.0}}	{'ART': {'precision': 0.38303341902313626, 'recall': 0.5912698412698413, 'f1-score': 0.46489859594383776, 'support': 252}, 'CON': {'precision': 0.3568627450980392, 'recall': 0.5652173913043478, 'f1-score': 0.43750000000000006, 'support': 161}, 'LOC': {'precision': 0.6423841059602649, 'recall': 0.6510067114093959, 'f1-score': 0.6466666666666665, 'support': 149}, 'MAT': {'precision': 0.29545454545454547, 'recall': 0.325, 'f1-score': 0.30952380952380953, 'support': 40}, 'PER': {'precision': 0.6735218508997429, 'recall': 0.7797619047619048, 'f1-score': 0.7227586206896551, 'support': 336}, 'SPE': {'precision': 0.3728813559322034, 'recall': 0.7096774193548387, 'f1-score': 0.4888888888888889, 'support': 31}, 'micro avg': {'precision': 0.4926184926184926, 'recall': 0.6542827657378741, 'f1-score': 0.5620567375886526, 'support': 969}, 'macro avg': {'precision': 0.45402300372798865, 'recall': 0.6036555446833881, 'f1-score': 0.511706096952143, 'support': 969}, 'weighted avg': {'precision': 0.5153512911218657, 'recall': 0.6542827657378741, 'f1-score': 0.5720626253863906, 'support': 969}}
No log	2.0	498	0.2107	0.5069	0.6811	0.5812	0.9454	{'B-ART': {'precision': 0.5202492211838006, 'recall': 0.6626984126984127, 'f1-score': 0.5828970331588132, 'support': 252.0}, 'B-CON': {'precision': 0.38267148014440433, 'recall': 0.6583850931677019, 'f1-score': 0.4840182648401826, 'support': 161.0}, 'B-LOC': {'precision': 0.7924528301886793, 'recall': 0.8456375838926175, 'f1-score': 0.8181818181818182, 'support': 149.0}, 'B-MAT': {'precision': 0.5, 'recall': 0.375, 'f1-score': 0.42857142857142855, 'support': 40.0}, 'B-PER': {'precision': 0.7948051948051948, 'recall': 0.9107142857142857, 'f1-score': 0.8488210818307905, 'support': 336.0}, 'B-SPE': {'precision': 0.46296296296296297, 'recall': 0.8064516129032258, 'f1-score': 0.5882352941176471, 'support': 31.0}, 'I-ART': {'precision': 0.6188340807174888, 'recall': 0.3909348441926346, 'f1-score': 0.4791666666666667, 'support': 353.0}, 'I-CON': {'precision': 0.3821656050955414, 'recall': 0.5084745762711864, 'f1-score': 0.43636363636363634, 'support': 118.0}, 'I-LOC': {'precision': 0.8851063829787233, 'recall': 0.8188976377952756, 'f1-score': 0.8507157464212679, 'support': 254.0}, 'I-MAT': {'precision': 0.5, 'recall': 0.08108108108108109, 'f1-score': 0.13953488372093023, 'support': 37.0}, 'I-PER': {'precision': 0.8791773778920309, 'recall': 0.7651006711409396, 'f1-score': 0.8181818181818182, 'support': 447.0}, 'I-SPE': {'precision': 0.8235294117647058, 'recall': 0.6666666666666666, 'f1-score': 0.7368421052631579, 'support': 42.0}, 'O': {'precision': 0.9754975545978403, 'recall': 0.973141394135549, 'f1-score': 0.9743180499129426, 'support': 20701.0}, 'accuracy': 0.9453776013262947, 'macro avg': {'precision': 0.6551886232562593, 'recall': 0.6510141430507367, 'f1-score': 0.6296806020947, 'support': 22921.0}, 'weighted avg': {'precision': 0.9484931362139072, 'recall': 0.9453776013262947, 'f1-score': 0.9456490573608961, 'support': 22921.0}}	{'ART': {'precision': 0.39825581395348836, 'recall': 0.5436507936507936, 'f1-score': 0.4597315436241611, 'support': 252}, 'CON': {'precision': 0.32781456953642385, 'recall': 0.6149068322981367, 'f1-score': 0.42764578833693306, 'support': 161}, 'LOC': {'precision': 0.6449704142011834, 'recall': 0.7315436241610739, 'f1-score': 0.6855345911949686, 'support': 149}, 'MAT': {'precision': 0.3939393939393939, 'recall': 0.325, 'f1-score': 0.35616438356164376, 'support': 40}, 'PER': {'precision': 0.707808564231738, 'recall': 0.8363095238095238, 'f1-score': 0.7667121418826739, 'support': 336}, 'SPE': {'precision': 0.3684210526315789, 'recall': 0.6774193548387096, 'f1-score': 0.4772727272727273, 'support': 31}, 'micro avg': {'precision': 0.5069124423963134, 'recall': 0.6811145510835913, 'f1-score': 0.5812417437252311, 'support': 969}, 'macro avg': {'precision': 0.47353496808230106, 'recall': 0.621471688126373, 'f1-score': 0.5288435293121846, 'support': 969}, 'weighted avg': {'precision': 0.5306929912266649, 'recall': 0.6811145510835913, 'f1-score': 0.5918527188483838, 'support': 969}}
0.1401	3.0	747	0.2393	0.5161	0.6801	0.5868	0.9464	{'B-ART': {'precision': 0.5029411764705882, 'recall': 0.6785714285714286, 'f1-score': 0.5777027027027027, 'support': 252.0}, 'B-CON': {'precision': 0.4345991561181435, 'recall': 0.639751552795031, 'f1-score': 0.5175879396984925, 'support': 161.0}, 'B-LOC': {'precision': 0.7590361445783133, 'recall': 0.8456375838926175, 'f1-score': 0.8, 'support': 149.0}, 'B-MAT': {'precision': 0.4358974358974359, 'recall': 0.425, 'f1-score': 0.43037974683544306, 'support': 40.0}, 'B-PER': {'precision': 0.8026666666666666, 'recall': 0.8958333333333334, 'f1-score': 0.8466947960618847, 'support': 336.0}, 'B-SPE': {'precision': 0.4444444444444444, 'recall': 0.7741935483870968, 'f1-score': 0.5647058823529412, 'support': 31.0}, 'I-ART': {'precision': 0.5704225352112676, 'recall': 0.45892351274787535, 'f1-score': 0.5086342229199372, 'support': 353.0}, 'I-CON': {'precision': 0.4090909090909091, 'recall': 0.4576271186440678, 'f1-score': 0.432, 'support': 118.0}, 'I-LOC': {'precision': 0.8913043478260869, 'recall': 0.8070866141732284, 'f1-score': 0.8471074380165289, 'support': 254.0}, 'I-MAT': {'precision': 0.2727272727272727, 'recall': 0.08108108108108109, 'f1-score': 0.125, 'support': 37.0}, 'I-PER': {'precision': 0.8743718592964824, 'recall': 0.7785234899328859, 'f1-score': 0.8236686390532545, 'support': 447.0}, 'I-SPE': {'precision': 0.8285714285714286, 'recall': 0.6904761904761905, 'f1-score': 0.7532467532467533, 'support': 42.0}, 'O': {'precision': 0.9771580989330747, 'recall': 0.9733346215158688, 'f1-score': 0.9752426127150844, 'support': 20701.0}, 'accuracy': 0.9463810479472973, 'macro avg': {'precision': 0.6310178058332395, 'recall': 0.654310775042362, 'f1-score': 0.6309208256617709, 'support': 22921.0}, 'weighted avg': {'precision': 0.9489387820828759, 'recall': 0.9463810479472973, 'f1-score': 0.9469897039453234, 'support': 22921.0}}	{'ART': {'precision': 0.42032967032967034, 'recall': 0.6071428571428571, 'f1-score': 0.4967532467532467, 'support': 252}, 'CON': {'precision': 0.36328125, 'recall': 0.577639751552795, 'f1-score': 0.44604316546762585, 'support': 161}, 'LOC': {'precision': 0.6149425287356322, 'recall': 0.7181208053691275, 'f1-score': 0.6625386996904025, 'support': 149}, 'MAT': {'precision': 0.3170731707317073, 'recall': 0.325, 'f1-score': 0.3209876543209877, 'support': 40}, 'PER': {'precision': 0.7109375, 'recall': 0.8125, 'f1-score': 0.7583333333333333, 'support': 336}, 'SPE': {'precision': 0.3448275862068966, 'recall': 0.6451612903225806, 'f1-score': 0.44943820224719105, 'support': 31}, 'micro avg': {'precision': 0.5160532498042286, 'recall': 0.6800825593395253, 'f1-score': 0.5868210151380232, 'support': 969}, 'macro avg': {'precision': 0.4618986176673177, 'recall': 0.6142607840645601, 'f1-score': 0.5223490503021312, 'support': 969}, 'weighted avg': {'precision': 0.5348662300891313, 'recall': 0.6800825593395253, 'f1-score': 0.5957534824752098, 'support': 969}}

Framework versions

• Transformers 4.45.2
• Pytorch 2.4.1
• Datasets 3.0.1
• Tokenizers 0.20.1