bert-finetuned-arc-ner-default

This model is a fine-tuned version of bert-base-cased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2312
• Precision: 0.4879
• Recall: 0.6873
• F1: 0.5707
• Accuracy: 0.9441
• Classification Report Details: {'B-ART': {'precision': 0.5144508670520231, 'recall': 0.7063492063492064, 'f1-score': 0.5953177257525084, 'support': 252.0}, 'B-CON': {'precision': 0.3969465648854962, 'recall': 0.6459627329192547, 'f1-score': 0.491725768321513, 'support': 161.0}, 'B-LOC': {'precision': 0.7735849056603774, 'recall': 0.825503355704698, 'f1-score': 0.7987012987012987, 'support': 149.0}, 'B-MAT': {'precision': 0.5, 'recall': 0.425, 'f1-score': 0.4594594594594595, 'support': 40.0}, 'B-PER': {'precision': 0.7623762376237624, 'recall': 0.9166666666666666, 'f1-score': 0.8324324324324325, 'support': 336.0}, 'B-SPE': {'precision': 0.4426229508196721, 'recall': 0.8709677419354839, 'f1-score': 0.5869565217391305, 'support': 31.0}, 'I-ART': {'precision': 0.565359477124183, 'recall': 0.49008498583569404, 'f1-score': 0.5250379362670713, 'support': 353.0}, 'I-CON': {'precision': 0.35664335664335667, 'recall': 0.4322033898305085, 'f1-score': 0.39080459770114945, 'support': 118.0}, 'I-LOC': {'precision': 0.8446215139442231, 'recall': 0.8346456692913385, 'f1-score': 0.8396039603960396, 'support': 254.0}, 'I-MAT': {'precision': 1.0, 'recall': 0.05405405405405406, 'f1-score': 0.10256410256410256, 'support': 37.0}, 'I-PER': {'precision': 0.8132387706855791, 'recall': 0.7695749440715883, 'f1-score': 0.7908045977011494, 'support': 447.0}, 'I-SPE': {'precision': 0.8823529411764706, 'recall': 0.7142857142857143, 'f1-score': 0.7894736842105263, 'support': 42.0}, 'O': {'precision': 0.9792154566744731, 'recall': 0.9695183807545529, 'f1-score': 0.9743427919508703, 'support': 20701.0}, 'accuracy': 0.9440687579075957, 'macro avg': {'precision': 0.679339464791509, 'recall': 0.6657551416691354, 'f1-score': 0.6290172982459423, 'support': 22921.0}, 'weighted avg': {'precision': 0.9494873387897029, 'recall': 0.9440687579075957, 'f1-score': 0.945399024248339, 'support': 22921.0}}
• Classfication Report Seqeval: {'ART': {'precision': 0.391304347826087, 'recall': 0.6071428571428571, 'f1-score': 0.4758942457231726, 'support': 252}, 'CON': {'precision': 0.313588850174216, 'recall': 0.5590062111801242, 'f1-score': 0.40178571428571425, 'support': 161}, 'LOC': {'precision': 0.6566265060240963, 'recall': 0.7315436241610739, 'f1-score': 0.692063492063492, 'support': 149}, 'MAT': {'precision': 0.35294117647058826, 'recall': 0.3, 'f1-score': 0.3243243243243243, 'support': 40}, 'PER': {'precision': 0.6587677725118484, 'recall': 0.8273809523809523, 'f1-score': 0.7335092348284961, 'support': 336}, 'SPE': {'precision': 0.36923076923076925, 'recall': 0.7741935483870968, 'f1-score': 0.5, 'support': 31}, 'micro avg': {'precision': 0.4879120879120879, 'recall': 0.6873065015479877, 'f1-score': 0.570694087403599, 'support': 969}, 'macro avg': {'precision': 0.45707657037293425, 'recall': 0.6332111988753507, 'f1-score': 0.5212628352041999, 'support': 969}, 'weighted avg': {'precision': 0.5096425411731388, 'recall': 0.6873065015479877, 'f1-score': 0.5806629371672316, 'support': 969}}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy	Classification Report Details	Classfication Report Seqeval
No log	1.0	249	0.2141	0.4577	0.6305	0.5304	0.9420	{'B-ART': {'precision': 0.43448275862068964, 'recall': 0.75, 'f1-score': 0.5502183406113537, 'support': 252.0}, 'B-CON': {'precision': 0.4297520661157025, 'recall': 0.6459627329192547, 'f1-score': 0.5161290322580645, 'support': 161.0}, 'B-LOC': {'precision': 0.7564102564102564, 'recall': 0.7919463087248322, 'f1-score': 0.7737704918032787, 'support': 149.0}, 'B-MAT': {'precision': 0.7777777777777778, 'recall': 0.175, 'f1-score': 0.2857142857142857, 'support': 40.0}, 'B-PER': {'precision': 0.7746113989637305, 'recall': 0.8898809523809523, 'f1-score': 0.8282548476454293, 'support': 336.0}, 'B-SPE': {'precision': 0.5769230769230769, 'recall': 0.4838709677419355, 'f1-score': 0.5263157894736842, 'support': 31.0}, 'I-ART': {'precision': 0.5491803278688525, 'recall': 0.37960339943342775, 'f1-score': 0.4489112227805695, 'support': 353.0}, 'I-CON': {'precision': 0.5151515151515151, 'recall': 0.4322033898305085, 'f1-score': 0.4700460829493088, 'support': 118.0}, 'I-LOC': {'precision': 0.8304347826086956, 'recall': 0.7519685039370079, 'f1-score': 0.7892561983471075, 'support': 254.0}, 'I-MAT': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 37.0}, 'I-PER': {'precision': 0.8782608695652174, 'recall': 0.6778523489932886, 'f1-score': 0.7651515151515151, 'support': 447.0}, 'I-SPE': {'precision': 1.0, 'recall': 0.11904761904761904, 'f1-score': 0.2127659574468085, 'support': 42.0}, 'O': {'precision': 0.9725703817971462, 'recall': 0.9745905994879475, 'f1-score': 0.9735794426348172, 'support': 20701.0}, 'accuracy': 0.9419746084376772, 'macro avg': {'precision': 0.6535042470617431, 'recall': 0.5439943709612902, 'f1-score': 0.5492394774474019, 'support': 22921.0}, 'weighted avg': {'precision': 0.9438502279200854, 'recall': 0.9419746084376772, 'f1-score': 0.9407316741109021, 'support': 22921.0}}	{'ART': {'precision': 0.30148619957537154, 'recall': 0.5634920634920635, 'f1-score': 0.39280774550484093, 'support': 252}, 'CON': {'precision': 0.3425196850393701, 'recall': 0.5403726708074534, 'f1-score': 0.41927710843373495, 'support': 161}, 'LOC': {'precision': 0.5857988165680473, 'recall': 0.6644295302013423, 'f1-score': 0.6226415094339623, 'support': 149}, 'MAT': {'precision': 0.4444444444444444, 'recall': 0.1, 'f1-score': 0.163265306122449, 'support': 40}, 'PER': {'precision': 0.6616915422885572, 'recall': 0.7916666666666666, 'f1-score': 0.7208672086720868, 'support': 336}, 'SPE': {'precision': 0.43333333333333335, 'recall': 0.41935483870967744, 'f1-score': 0.4262295081967213, 'support': 31}, 'micro avg': {'precision': 0.45767790262172287, 'recall': 0.630546955624355, 'f1-score': 0.5303819444444445, 'support': 969}, 'macro avg': {'precision': 0.46154567020818726, 'recall': 0.5132192949795339, 'f1-score': 0.4575147310606325, 'support': 969}, 'weighted avg': {'precision': 0.48704198614348565, 'recall': 0.630546955624355, 'f1-score': 0.5378945927177803, 'support': 969}}
No log	2.0	498	0.2129	0.4670	0.6502	0.5436	0.9425	{'B-ART': {'precision': 0.5299684542586751, 'recall': 0.6666666666666666, 'f1-score': 0.5905096660808435, 'support': 252.0}, 'B-CON': {'precision': 0.375, 'recall': 0.6894409937888198, 'f1-score': 0.48577680525164113, 'support': 161.0}, 'B-LOC': {'precision': 0.7628205128205128, 'recall': 0.7986577181208053, 'f1-score': 0.780327868852459, 'support': 149.0}, 'B-MAT': {'precision': 0.6111111111111112, 'recall': 0.275, 'f1-score': 0.3793103448275862, 'support': 40.0}, 'B-PER': {'precision': 0.7560975609756098, 'recall': 0.9226190476190477, 'f1-score': 0.8310991957104558, 'support': 336.0}, 'B-SPE': {'precision': 0.47058823529411764, 'recall': 0.7741935483870968, 'f1-score': 0.5853658536585366, 'support': 31.0}, 'I-ART': {'precision': 0.6016597510373444, 'recall': 0.41076487252124644, 'f1-score': 0.4882154882154882, 'support': 353.0}, 'I-CON': {'precision': 0.38345864661654133, 'recall': 0.4322033898305085, 'f1-score': 0.4063745019920319, 'support': 118.0}, 'I-LOC': {'precision': 0.8156862745098039, 'recall': 0.8188976377952756, 'f1-score': 0.8172888015717092, 'support': 254.0}, 'I-MAT': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 37.0}, 'I-PER': {'precision': 0.8286445012787724, 'recall': 0.7248322147651006, 'f1-score': 0.7732696897374701, 'support': 447.0}, 'I-SPE': {'precision': 0.9259259259259259, 'recall': 0.5952380952380952, 'f1-score': 0.7246376811594203, 'support': 42.0}, 'O': {'precision': 0.9748375836323088, 'recall': 0.971305734022511, 'f1-score': 0.9730684540373121, 'support': 20701.0}, 'accuracy': 0.9424981458051569, 'macro avg': {'precision': 0.6181383505739018, 'recall': 0.6215246091350134, 'f1-score': 0.6027111039303811, 'support': 22921.0}, 'weighted avg': {'precision': 0.9447621348538526, 'recall': 0.9424981458051569, 'f1-score': 0.9425119084723276, 'support': 22921.0}}	{'ART': {'precision': 0.38055555555555554, 'recall': 0.5436507936507936, 'f1-score': 0.44771241830065356, 'support': 252}, 'CON': {'precision': 0.2875, 'recall': 0.5714285714285714, 'f1-score': 0.3825363825363825, 'support': 161}, 'LOC': {'precision': 0.6058823529411764, 'recall': 0.6912751677852349, 'f1-score': 0.64576802507837, 'support': 149}, 'MAT': {'precision': 0.5, 'recall': 0.225, 'f1-score': 0.3103448275862069, 'support': 40}, 'PER': {'precision': 0.6323185011709602, 'recall': 0.8035714285714286, 'f1-score': 0.7077326343381389, 'support': 336}, 'SPE': {'precision': 0.35185185185185186, 'recall': 0.6129032258064516, 'f1-score': 0.44705882352941173, 'support': 31}, 'micro avg': {'precision': 0.4670126019273536, 'recall': 0.6501547987616099, 'f1-score': 0.5435720448662639, 'support': 969}, 'macro avg': {'precision': 0.4596847102532573, 'recall': 0.5746381978737467, 'f1-score': 0.4901921852281939, 'support': 969}, 'weighted avg': {'precision': 0.49105303858522736, 'recall': 0.6501547987616099, 'f1-score': 0.5518081573862479, 'support': 969}}
0.1979	3.0	747	0.2312	0.4879	0.6873	0.5707	0.9441	{'B-ART': {'precision': 0.5144508670520231, 'recall': 0.7063492063492064, 'f1-score': 0.5953177257525084, 'support': 252.0}, 'B-CON': {'precision': 0.3969465648854962, 'recall': 0.6459627329192547, 'f1-score': 0.491725768321513, 'support': 161.0}, 'B-LOC': {'precision': 0.7735849056603774, 'recall': 0.825503355704698, 'f1-score': 0.7987012987012987, 'support': 149.0}, 'B-MAT': {'precision': 0.5, 'recall': 0.425, 'f1-score': 0.4594594594594595, 'support': 40.0}, 'B-PER': {'precision': 0.7623762376237624, 'recall': 0.9166666666666666, 'f1-score': 0.8324324324324325, 'support': 336.0}, 'B-SPE': {'precision': 0.4426229508196721, 'recall': 0.8709677419354839, 'f1-score': 0.5869565217391305, 'support': 31.0}, 'I-ART': {'precision': 0.565359477124183, 'recall': 0.49008498583569404, 'f1-score': 0.5250379362670713, 'support': 353.0}, 'I-CON': {'precision': 0.35664335664335667, 'recall': 0.4322033898305085, 'f1-score': 0.39080459770114945, 'support': 118.0}, 'I-LOC': {'precision': 0.8446215139442231, 'recall': 0.8346456692913385, 'f1-score': 0.8396039603960396, 'support': 254.0}, 'I-MAT': {'precision': 1.0, 'recall': 0.05405405405405406, 'f1-score': 0.10256410256410256, 'support': 37.0}, 'I-PER': {'precision': 0.8132387706855791, 'recall': 0.7695749440715883, 'f1-score': 0.7908045977011494, 'support': 447.0}, 'I-SPE': {'precision': 0.8823529411764706, 'recall': 0.7142857142857143, 'f1-score': 0.7894736842105263, 'support': 42.0}, 'O': {'precision': 0.9792154566744731, 'recall': 0.9695183807545529, 'f1-score': 0.9743427919508703, 'support': 20701.0}, 'accuracy': 0.9440687579075957, 'macro avg': {'precision': 0.679339464791509, 'recall': 0.6657551416691354, 'f1-score': 0.6290172982459423, 'support': 22921.0}, 'weighted avg': {'precision': 0.9494873387897029, 'recall': 0.9440687579075957, 'f1-score': 0.945399024248339, 'support': 22921.0}}	{'ART': {'precision': 0.391304347826087, 'recall': 0.6071428571428571, 'f1-score': 0.4758942457231726, 'support': 252}, 'CON': {'precision': 0.313588850174216, 'recall': 0.5590062111801242, 'f1-score': 0.40178571428571425, 'support': 161}, 'LOC': {'precision': 0.6566265060240963, 'recall': 0.7315436241610739, 'f1-score': 0.692063492063492, 'support': 149}, 'MAT': {'precision': 0.35294117647058826, 'recall': 0.3, 'f1-score': 0.3243243243243243, 'support': 40}, 'PER': {'precision': 0.6587677725118484, 'recall': 0.8273809523809523, 'f1-score': 0.7335092348284961, 'support': 336}, 'SPE': {'precision': 0.36923076923076925, 'recall': 0.7741935483870968, 'f1-score': 0.5, 'support': 31}, 'micro avg': {'precision': 0.4879120879120879, 'recall': 0.6873065015479877, 'f1-score': 0.570694087403599, 'support': 969}, 'macro avg': {'precision': 0.45707657037293425, 'recall': 0.6332111988753507, 'f1-score': 0.5212628352041999, 'support': 969}, 'weighted avg': {'precision': 0.5096425411731388, 'recall': 0.6873065015479877, 'f1-score': 0.5806629371672316, 'support': 969}}

Framework versions

• Transformers 4.45.2
• Pytorch 2.4.1
• Datasets 3.0.1
• Tokenizers 0.20.1