bert-finetuned-arc-ner-tunned

This model is a fine-tuned version of bert-base-cased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2274
• Precision: 0.4850
• Recall: 0.6852
• F1: 0.5680
• Accuracy: 0.9444
• Classification Report Details: {'B-ART': {'precision': 0.5236686390532544, 'recall': 0.7023809523809523, 'f1-score': 0.6, 'support': 252.0}, 'B-CON': {'precision': 0.3564356435643564, 'recall': 0.6708074534161491, 'f1-score': 0.46551724137931033, 'support': 161.0}, 'B-LOC': {'precision': 0.7531645569620253, 'recall': 0.7986577181208053, 'f1-score': 0.7752442996742671, 'support': 149.0}, 'B-MAT': {'precision': 0.48484848484848486, 'recall': 0.4, 'f1-score': 0.4383561643835616, 'support': 40.0}, 'B-PER': {'precision': 0.8101604278074866, 'recall': 0.9017857142857143, 'f1-score': 0.8535211267605634, 'support': 336.0}, 'B-SPE': {'precision': 0.45454545454545453, 'recall': 0.8064516129032258, 'f1-score': 0.5813953488372093, 'support': 31.0}, 'I-ART': {'precision': 0.5971731448763251, 'recall': 0.47875354107648727, 'f1-score': 0.5314465408805031, 'support': 353.0}, 'I-CON': {'precision': 0.3710691823899371, 'recall': 0.5, 'f1-score': 0.4259927797833935, 'support': 118.0}, 'I-LOC': {'precision': 0.881578947368421, 'recall': 0.7913385826771654, 'f1-score': 0.8340248962655602, 'support': 254.0}, 'I-MAT': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 37.0}, 'I-PER': {'precision': 0.8533007334963325, 'recall': 0.7807606263982103, 'f1-score': 0.8154205607476636, 'support': 447.0}, 'I-SPE': {'precision': 0.8787878787878788, 'recall': 0.6904761904761905, 'f1-score': 0.7733333333333333, 'support': 42.0}, 'O': {'precision': 0.9778080591785089, 'recall': 0.9705811313463117, 'f1-score': 0.9741811922713278, 'support': 20701.0}, 'accuracy': 0.9444177828192487, 'macro avg': {'precision': 0.6109647040675743, 'recall': 0.6532302710062472, 'f1-score': 0.6206487295628226, 'support': 22921.0}, 'weighted avg': {'precision': 0.9487247238341253, 'recall': 0.9444177828192487, 'f1-score': 0.9457357109181124, 'support': 22921.0}}
• Classfication Report Seqeval: {'ART': {'precision': 0.3974025974025974, 'recall': 0.6071428571428571, 'f1-score': 0.48037676609105184, 'support': 252}, 'CON': {'precision': 0.3076923076923077, 'recall': 0.6211180124223602, 'f1-score': 0.411522633744856, 'support': 161}, 'LOC': {'precision': 0.6227544910179641, 'recall': 0.697986577181208, 'f1-score': 0.6582278481012659, 'support': 149}, 'MAT': {'precision': 0.30303030303030304, 'recall': 0.25, 'f1-score': 0.27397260273972607, 'support': 40}, 'PER': {'precision': 0.6875, 'recall': 0.8184523809523809, 'f1-score': 0.7472826086956521, 'support': 336}, 'SPE': {'precision': 0.3728813559322034, 'recall': 0.7096774193548387, 'f1-score': 0.4888888888888889, 'support': 31}, 'micro avg': {'precision': 0.4850255661066472, 'recall': 0.6852425180598555, 'f1-score': 0.5680068434559453, 'support': 969}, 'macro avg': {'precision': 0.4485435091792292, 'recall': 0.6173962078422742, 'f1-score': 0.5100452247102402, 'support': 969}, 'weighted avg': {'precision': 0.5130597207437597, 'recall': 0.6852425180598555, 'f1-score': 0.5805856085055101, 'support': 969}}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy	Classification Report Details	Classfication Report Seqeval
No log	1.0	249	0.2125	0.4727	0.6264	0.5388	0.9439	{'B-ART': {'precision': 0.455470737913486, 'recall': 0.7103174603174603, 'f1-score': 0.5550387596899224, 'support': 252.0}, 'B-CON': {'precision': 0.3695652173913043, 'recall': 0.6335403726708074, 'f1-score': 0.4668192219679634, 'support': 161.0}, 'B-LOC': {'precision': 0.8409090909090909, 'recall': 0.7449664429530202, 'f1-score': 0.7900355871886121, 'support': 149.0}, 'B-MAT': {'precision': 0.5, 'recall': 0.025, 'f1-score': 0.047619047619047616, 'support': 40.0}, 'B-PER': {'precision': 0.8005390835579514, 'recall': 0.8839285714285714, 'f1-score': 0.8401697312588402, 'support': 336.0}, 'B-SPE': {'precision': 0.5416666666666666, 'recall': 0.41935483870967744, 'f1-score': 0.4727272727272727, 'support': 31.0}, 'I-ART': {'precision': 0.6376811594202898, 'recall': 0.37393767705382436, 'f1-score': 0.4714285714285714, 'support': 353.0}, 'I-CON': {'precision': 0.5092592592592593, 'recall': 0.4661016949152542, 'f1-score': 0.48672566371681414, 'support': 118.0}, 'I-LOC': {'precision': 0.8935185185185185, 'recall': 0.7598425196850394, 'f1-score': 0.8212765957446808, 'support': 254.0}, 'I-MAT': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 37.0}, 'I-PER': {'precision': 0.8907563025210085, 'recall': 0.7114093959731543, 'f1-score': 0.7910447761194029, 'support': 447.0}, 'I-SPE': {'precision': 1.0, 'recall': 0.16666666666666666, 'f1-score': 0.2857142857142857, 'support': 42.0}, 'O': {'precision': 0.9710966007297869, 'recall': 0.9770542485870248, 'f1-score': 0.9740663151051073, 'support': 20701.0}, 'accuracy': 0.9438506173378125, 'macro avg': {'precision': 0.6469586643759508, 'recall': 0.5286246068431155, 'f1-score': 0.5386666021754247, 'support': 22921.0}, 'weighted avg': {'precision': 0.9449994949499437, 'recall': 0.9438506173378125, 'f1-score': 0.9420965610139306, 'support': 22921.0}}	{'ART': {'precision': 0.33095238095238094, 'recall': 0.5515873015873016, 'f1-score': 0.4136904761904762, 'support': 252}, 'CON': {'precision': 0.3194444444444444, 'recall': 0.5714285714285714, 'f1-score': 0.4097995545657015, 'support': 161}, 'LOC': {'precision': 0.6530612244897959, 'recall': 0.6442953020134228, 'f1-score': 0.6486486486486487, 'support': 149}, 'MAT': {'precision': 0.5, 'recall': 0.025, 'f1-score': 0.047619047619047616, 'support': 40}, 'PER': {'precision': 0.6792929292929293, 'recall': 0.8005952380952381, 'f1-score': 0.7349726775956283, 'support': 336}, 'SPE': {'precision': 0.3225806451612903, 'recall': 0.3225806451612903, 'f1-score': 0.3225806451612903, 'support': 31}, 'micro avg': {'precision': 0.4727414330218069, 'recall': 0.6264189886480909, 'f1-score': 0.5388371060807813, 'support': 969}, 'macro avg': {'precision': 0.4675552707234734, 'recall': 0.48591450971430405, 'f1-score': 0.4295518416301321, 'support': 969}, 'weighted avg': {'precision': 0.5060671849813823, 'recall': 0.6264189886480909, 'f1-score': 0.5425510407746332, 'support': 969}}
No log	2.0	498	0.2123	0.4788	0.6883	0.5648	0.9436	{'B-ART': {'precision': 0.5238095238095238, 'recall': 0.6984126984126984, 'f1-score': 0.5986394557823129, 'support': 252.0}, 'B-CON': {'precision': 0.33630952380952384, 'recall': 0.7018633540372671, 'f1-score': 0.45472837022132795, 'support': 161.0}, 'B-LOC': {'precision': 0.7692307692307693, 'recall': 0.8053691275167785, 'f1-score': 0.7868852459016393, 'support': 149.0}, 'B-MAT': {'precision': 0.5714285714285714, 'recall': 0.2, 'f1-score': 0.2962962962962963, 'support': 40.0}, 'B-PER': {'precision': 0.7893401015228426, 'recall': 0.9255952380952381, 'f1-score': 0.852054794520548, 'support': 336.0}, 'B-SPE': {'precision': 0.45454545454545453, 'recall': 0.8064516129032258, 'f1-score': 0.5813953488372093, 'support': 31.0}, 'I-ART': {'precision': 0.642570281124498, 'recall': 0.45325779036827196, 'f1-score': 0.53156146179402, 'support': 353.0}, 'I-CON': {'precision': 0.40522875816993464, 'recall': 0.5254237288135594, 'f1-score': 0.4575645756457565, 'support': 118.0}, 'I-LOC': {'precision': 0.8433734939759037, 'recall': 0.8267716535433071, 'f1-score': 0.8349900596421471, 'support': 254.0}, 'I-MAT': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 37.0}, 'I-PER': {'precision': 0.8629441624365483, 'recall': 0.7606263982102909, 'f1-score': 0.8085612366230678, 'support': 447.0}, 'I-SPE': {'precision': 0.8888888888888888, 'recall': 0.5714285714285714, 'f1-score': 0.6956521739130435, 'support': 42.0}, 'O': {'precision': 0.9767487109641015, 'recall': 0.9700014492053524, 'f1-score': 0.9733633873821469, 'support': 20701.0}, 'accuracy': 0.9436324767680293, 'macro avg': {'precision': 0.6203398646081971, 'recall': 0.6342462786565046, 'f1-score': 0.6055148005045782, 'support': 22921.0}, 'weighted avg': {'precision': 0.9482366892277677, 'recall': 0.9436324767680293, 'f1-score': 0.944611529518263, 'support': 22921.0}}	{'ART': {'precision': 0.393048128342246, 'recall': 0.5833333333333334, 'f1-score': 0.46964856230031954, 'support': 252}, 'CON': {'precision': 0.28888888888888886, 'recall': 0.6459627329192547, 'f1-score': 0.39923224568138194, 'support': 161}, 'LOC': {'precision': 0.6432748538011696, 'recall': 0.738255033557047, 'f1-score': 0.6875, 'support': 149}, 'MAT': {'precision': 0.5, 'recall': 0.175, 'f1-score': 0.25925925925925924, 'support': 40}, 'PER': {'precision': 0.6714628297362111, 'recall': 0.8333333333333334, 'f1-score': 0.7436918990703852, 'support': 336}, 'SPE': {'precision': 0.3333333333333333, 'recall': 0.6129032258064516, 'f1-score': 0.43181818181818177, 'support': 31}, 'micro avg': {'precision': 0.47882268485283563, 'recall': 0.6883384932920537, 'f1-score': 0.5647756138865369, 'support': 969}, 'macro avg': {'precision': 0.47166800568364153, 'recall': 0.5981312764915699, 'f1-score': 0.4985250246882546, 'support': 969}, 'weighted avg': {'precision': 0.5132631958662864, 'recall': 0.6883384932920537, 'f1-score': 0.5765764100606465, 'support': 969}}
0.1964	3.0	747	0.2274	0.4850	0.6852	0.5680	0.9444	{'B-ART': {'precision': 0.5236686390532544, 'recall': 0.7023809523809523, 'f1-score': 0.6, 'support': 252.0}, 'B-CON': {'precision': 0.3564356435643564, 'recall': 0.6708074534161491, 'f1-score': 0.46551724137931033, 'support': 161.0}, 'B-LOC': {'precision': 0.7531645569620253, 'recall': 0.7986577181208053, 'f1-score': 0.7752442996742671, 'support': 149.0}, 'B-MAT': {'precision': 0.48484848484848486, 'recall': 0.4, 'f1-score': 0.4383561643835616, 'support': 40.0}, 'B-PER': {'precision': 0.8101604278074866, 'recall': 0.9017857142857143, 'f1-score': 0.8535211267605634, 'support': 336.0}, 'B-SPE': {'precision': 0.45454545454545453, 'recall': 0.8064516129032258, 'f1-score': 0.5813953488372093, 'support': 31.0}, 'I-ART': {'precision': 0.5971731448763251, 'recall': 0.47875354107648727, 'f1-score': 0.5314465408805031, 'support': 353.0}, 'I-CON': {'precision': 0.3710691823899371, 'recall': 0.5, 'f1-score': 0.4259927797833935, 'support': 118.0}, 'I-LOC': {'precision': 0.881578947368421, 'recall': 0.7913385826771654, 'f1-score': 0.8340248962655602, 'support': 254.0}, 'I-MAT': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 37.0}, 'I-PER': {'precision': 0.8533007334963325, 'recall': 0.7807606263982103, 'f1-score': 0.8154205607476636, 'support': 447.0}, 'I-SPE': {'precision': 0.8787878787878788, 'recall': 0.6904761904761905, 'f1-score': 0.7733333333333333, 'support': 42.0}, 'O': {'precision': 0.9778080591785089, 'recall': 0.9705811313463117, 'f1-score': 0.9741811922713278, 'support': 20701.0}, 'accuracy': 0.9444177828192487, 'macro avg': {'precision': 0.6109647040675743, 'recall': 0.6532302710062472, 'f1-score': 0.6206487295628226, 'support': 22921.0}, 'weighted avg': {'precision': 0.9487247238341253, 'recall': 0.9444177828192487, 'f1-score': 0.9457357109181124, 'support': 22921.0}}	{'ART': {'precision': 0.3974025974025974, 'recall': 0.6071428571428571, 'f1-score': 0.48037676609105184, 'support': 252}, 'CON': {'precision': 0.3076923076923077, 'recall': 0.6211180124223602, 'f1-score': 0.411522633744856, 'support': 161}, 'LOC': {'precision': 0.6227544910179641, 'recall': 0.697986577181208, 'f1-score': 0.6582278481012659, 'support': 149}, 'MAT': {'precision': 0.30303030303030304, 'recall': 0.25, 'f1-score': 0.27397260273972607, 'support': 40}, 'PER': {'precision': 0.6875, 'recall': 0.8184523809523809, 'f1-score': 0.7472826086956521, 'support': 336}, 'SPE': {'precision': 0.3728813559322034, 'recall': 0.7096774193548387, 'f1-score': 0.4888888888888889, 'support': 31}, 'micro avg': {'precision': 0.4850255661066472, 'recall': 0.6852425180598555, 'f1-score': 0.5680068434559453, 'support': 969}, 'macro avg': {'precision': 0.4485435091792292, 'recall': 0.6173962078422742, 'f1-score': 0.5100452247102402, 'support': 969}, 'weighted avg': {'precision': 0.5130597207437597, 'recall': 0.6852425180598555, 'f1-score': 0.5805856085055101, 'support': 969}}

Framework versions

• Transformers 4.45.2
• Pytorch 2.4.1
• Datasets 3.0.1
• Tokenizers 0.20.1