bert-finetuned-arcchialogy-ner-hp-tunned-hgf

This model is a fine-tuned version of bert-base-cased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2972
• Precision: 0.5083
• Recall: 0.6667
• F1: 0.5768
• F1 Macro: 0.5149
• F1 Micro: 0.5768
• Classification Report Details: {'B-ART': {'precision': 0.5060606060606061, 'recall': 0.6626984126984127, 'f1-score': 0.5738831615120275, 'support': 252.0}, 'B-CON': {'precision': 0.4375, 'recall': 0.6521739130434783, 'f1-score': 0.5236907730673317, 'support': 161.0}, 'B-LOC': {'precision': 0.8071428571428572, 'recall': 0.7583892617449665, 'f1-score': 0.7820069204152249, 'support': 149.0}, 'B-MAT': {'precision': 0.5357142857142857, 'recall': 0.375, 'f1-score': 0.4411764705882353, 'support': 40.0}, 'B-PER': {'precision': 0.7749360613810742, 'recall': 0.9017857142857143, 'f1-score': 0.8335625859697386, 'support': 336.0}, 'B-SPE': {'precision': 0.4067796610169492, 'recall': 0.7741935483870968, 'f1-score': 0.5333333333333333, 'support': 31.0}, 'I-ART': {'precision': 0.5416666666666666, 'recall': 0.40509915014164305, 'f1-score': 0.46353322528363045, 'support': 353.0}, 'I-CON': {'precision': 0.42857142857142855, 'recall': 0.4830508474576271, 'f1-score': 0.4541832669322709, 'support': 118.0}, 'I-LOC': {'precision': 0.8818565400843882, 'recall': 0.8228346456692913, 'f1-score': 0.8513238289205702, 'support': 254.0}, 'I-MAT': {'precision': 0.4166666666666667, 'recall': 0.13513513513513514, 'f1-score': 0.20408163265306123, 'support': 37.0}, 'I-PER': {'precision': 0.8345679012345679, 'recall': 0.756152125279642, 'f1-score': 0.7934272300469484, 'support': 447.0}, 'I-SPE': {'precision': 0.7666666666666667, 'recall': 0.5476190476190477, 'f1-score': 0.6388888888888888, 'support': 42.0}, 'O': {'precision': 0.9745303118342049, 'recall': 0.97222356407903, 'f1-score': 0.973375571300752, 'support': 20701.0}, 'accuracy': 0.9435888486540727, 'macro avg': {'precision': 0.6394353579261817, 'recall': 0.634335028118545, 'f1-score': 0.6204974529932318, 'support': 22921.0}, 'weighted avg': {'precision': 0.9455450522608214, 'recall': 0.9435888486540727, 'f1-score': 0.9437659943714384, 'support': 22921.0}}
• Classfication Report Seqeval: {'ART': {'precision': 0.4061624649859944, 'recall': 0.5753968253968254, 'f1-score': 0.47619047619047616, 'support': 252}, 'CON': {'precision': 0.3779527559055118, 'recall': 0.5962732919254659, 'f1-score': 0.4626506024096385, 'support': 161}, 'LOC': {'precision': 0.6234567901234568, 'recall': 0.6778523489932886, 'f1-score': 0.6495176848874598, 'support': 149}, 'MAT': {'precision': 0.3939393939393939, 'recall': 0.325, 'f1-score': 0.35616438356164376, 'support': 40}, 'PER': {'precision': 0.674937965260546, 'recall': 0.8095238095238095, 'f1-score': 0.7361299052774019, 'support': 336}, 'SPE': {'precision': 0.3064516129032258, 'recall': 0.6129032258064516, 'f1-score': 0.4086021505376344, 'support': 31}, 'micro avg': {'precision': 0.5082612116443745, 'recall': 0.6666666666666666, 'f1-score': 0.5767857142857143, 'support': 969}, 'macro avg': {'precision': 0.46381683051968814, 'recall': 0.5994915836076402, 'f1-score': 0.5148758671440424, 'support': 969}, 'weighted avg': {'precision': 0.5243912576788156, 'recall': 0.6666666666666666, 'f1-score': 0.5836096720521391, 'support': 969}}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 7.73381107021748e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 4

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	F1 Macro	F1 Micro	Classification Report Details	Classfication Report Seqeval
No log	1.0	249	0.2286	0.4996	0.5841	0.5385	0.4749	0.5385	{'B-ART': {'precision': 0.5092936802973977, 'recall': 0.5436507936507936, 'f1-score': 0.525911708253359, 'support': 252.0}, 'B-CON': {'precision': 0.4564102564102564, 'recall': 0.5527950310559007, 'f1-score': 0.5, 'support': 161.0}, 'B-LOC': {'precision': 0.8272727272727273, 'recall': 0.610738255033557, 'f1-score': 0.7027027027027027, 'support': 149.0}, 'B-MAT': {'precision': 0.36363636363636365, 'recall': 0.4, 'f1-score': 0.38095238095238093, 'support': 40.0}, 'B-PER': {'precision': 0.8184438040345822, 'recall': 0.8452380952380952, 'f1-score': 0.8316251830161054, 'support': 336.0}, 'B-SPE': {'precision': 0.358974358974359, 'recall': 0.9032258064516129, 'f1-score': 0.5137614678899083, 'support': 31.0}, 'I-ART': {'precision': 0.5942857142857143, 'recall': 0.29461756373937675, 'f1-score': 0.3939393939393939, 'support': 353.0}, 'I-CON': {'precision': 0.5584415584415584, 'recall': 0.3644067796610169, 'f1-score': 0.441025641025641, 'support': 118.0}, 'I-LOC': {'precision': 0.9136690647482014, 'recall': 0.5, 'f1-score': 0.6463104325699746, 'support': 254.0}, 'I-MAT': {'precision': 1.0, 'recall': 0.08108108108108109, 'f1-score': 0.15, 'support': 37.0}, 'I-PER': {'precision': 0.9193548387096774, 'recall': 0.6375838926174496, 'f1-score': 0.7529722589167768, 'support': 447.0}, 'I-SPE': {'precision': 0.6, 'recall': 0.7857142857142857, 'f1-score': 0.6804123711340206, 'support': 42.0}, 'O': {'precision': 0.9631611345234149, 'recall': 0.9826095357712188, 'f1-score': 0.9727881396461023, 'support': 20701.0}, 'accuracy': 0.9415383272981109, 'macro avg': {'precision': 0.6833033462564809, 'recall': 0.5770508553857221, 'f1-score': 0.5763385907727974, 'support': 22921.0}, 'weighted avg': {'precision': 0.9399703169611863, 'recall': 0.9415383272981109, 'f1-score': 0.9376545916465442, 'support': 22921.0}}	{'ART': {'precision': 0.40460526315789475, 'recall': 0.4880952380952381, 'f1-score': 0.4424460431654676, 'support': 252}, 'CON': {'precision': 0.3791469194312796, 'recall': 0.4968944099378882, 'f1-score': 0.4301075268817204, 'support': 161}, 'LOC': {'precision': 0.576, 'recall': 0.48322147651006714, 'f1-score': 0.5255474452554745, 'support': 149}, 'MAT': {'precision': 0.29545454545454547, 'recall': 0.325, 'f1-score': 0.30952380952380953, 'support': 40}, 'PER': {'precision': 0.6958904109589041, 'recall': 0.7559523809523809, 'f1-score': 0.724679029957204, 'support': 336}, 'SPE': {'precision': 0.2857142857142857, 'recall': 0.7741935483870968, 'f1-score': 0.417391304347826, 'support': 31}, 'micro avg': {'precision': 0.499558693733451, 'recall': 0.5841073271413829, 'f1-score': 0.538534728829686, 'support': 969}, 'macro avg': {'precision': 0.4394685707861516, 'recall': 0.5538928423137786, 'f1-score': 0.47494919318858364, 'support': 969}, 'weighted avg': {'precision': 0.5194238215704251, 'recall': 0.5841073271413829, 'f1-score': 0.5447497636017297, 'support': 969}}
No log	2.0	498	0.2315	0.5225	0.6347	0.5732	0.5046	0.5732	{'B-ART': {'precision': 0.5032679738562091, 'recall': 0.6111111111111112, 'f1-score': 0.5519713261648745, 'support': 252.0}, 'B-CON': {'precision': 0.5076142131979695, 'recall': 0.6211180124223602, 'f1-score': 0.5586592178770949, 'support': 161.0}, 'B-LOC': {'precision': 0.7913669064748201, 'recall': 0.738255033557047, 'f1-score': 0.7638888888888888, 'support': 149.0}, 'B-MAT': {'precision': 0.48148148148148145, 'recall': 0.325, 'f1-score': 0.3880597014925373, 'support': 40.0}, 'B-PER': {'precision': 0.8230337078651685, 'recall': 0.8720238095238095, 'f1-score': 0.846820809248555, 'support': 336.0}, 'B-SPE': {'precision': 0.43636363636363634, 'recall': 0.7741935483870968, 'f1-score': 0.5581395348837209, 'support': 31.0}, 'I-ART': {'precision': 0.5707762557077626, 'recall': 0.35410764872521244, 'f1-score': 0.4370629370629371, 'support': 353.0}, 'I-CON': {'precision': 0.44545454545454544, 'recall': 0.4152542372881356, 'f1-score': 0.4298245614035088, 'support': 118.0}, 'I-LOC': {'precision': 0.8625, 'recall': 0.8149606299212598, 'f1-score': 0.8380566801619433, 'support': 254.0}, 'I-MAT': {'precision': 0.3076923076923077, 'recall': 0.10810810810810811, 'f1-score': 0.16, 'support': 37.0}, 'I-PER': {'precision': 0.9085173501577287, 'recall': 0.6442953020134228, 'f1-score': 0.7539267015706806, 'support': 447.0}, 'I-SPE': {'precision': 0.8076923076923077, 'recall': 0.5, 'f1-score': 0.6176470588235294, 'support': 42.0}, 'O': {'precision': 0.968827691719258, 'recall': 0.9788899087000628, 'f1-score': 0.97383280870798, 'support': 20701.0}, 'accuracy': 0.9446359233890319, 'macro avg': {'precision': 0.6472760290510149, 'recall': 0.5967167192121251, 'f1-score': 0.6059915558681731, 'support': 22921.0}, 'weighted avg': {'precision': 0.9430665587612952, 'recall': 0.9446359233890319, 'f1-score': 0.9426405983679316, 'support': 22921.0}}	{'ART': {'precision': 0.4108761329305136, 'recall': 0.5396825396825397, 'f1-score': 0.46655231560891935, 'support': 252}, 'CON': {'precision': 0.4036697247706422, 'recall': 0.546583850931677, 'f1-score': 0.46437994722955145, 'support': 161}, 'LOC': {'precision': 0.5757575757575758, 'recall': 0.6375838926174496, 'f1-score': 0.6050955414012739, 'support': 149}, 'MAT': {'precision': 0.36363636363636365, 'recall': 0.3, 'f1-score': 0.32876712328767127, 'support': 40}, 'PER': {'precision': 0.7112299465240641, 'recall': 0.7916666666666666, 'f1-score': 0.7492957746478872, 'support': 336}, 'SPE': {'precision': 0.32142857142857145, 'recall': 0.5806451612903226, 'f1-score': 0.41379310344827586, 'support': 31}, 'micro avg': {'precision': 0.5225148683092609, 'recall': 0.6346749226006192, 'f1-score': 0.5731593662628146, 'support': 969}, 'macro avg': {'precision': 0.4644330525079552, 'recall': 0.5660270185314425, 'f1-score': 0.5046473009372632, 'support': 969}, 'weighted avg': {'precision': 0.5343678970756114, 'recall': 0.6346749226006192, 'f1-score': 0.5781602085926613, 'support': 969}}
0.1508	3.0	747	0.2536	0.4917	0.6760	0.5693	0.5163	0.5693	{'B-ART': {'precision': 0.478134110787172, 'recall': 0.6507936507936508, 'f1-score': 0.5512605042016807, 'support': 252.0}, 'B-CON': {'precision': 0.48372093023255813, 'recall': 0.6459627329192547, 'f1-score': 0.5531914893617021, 'support': 161.0}, 'B-LOC': {'precision': 0.7411764705882353, 'recall': 0.8456375838926175, 'f1-score': 0.7899686520376176, 'support': 149.0}, 'B-MAT': {'precision': 0.4107142857142857, 'recall': 0.575, 'f1-score': 0.4791666666666667, 'support': 40.0}, 'B-PER': {'precision': 0.7941952506596306, 'recall': 0.8958333333333334, 'f1-score': 0.8419580419580419, 'support': 336.0}, 'B-SPE': {'precision': 0.4107142857142857, 'recall': 0.7419354838709677, 'f1-score': 0.5287356321839081, 'support': 31.0}, 'I-ART': {'precision': 0.5204081632653061, 'recall': 0.43342776203966005, 'f1-score': 0.47295208655332305, 'support': 353.0}, 'I-CON': {'precision': 0.45255474452554745, 'recall': 0.5254237288135594, 'f1-score': 0.48627450980392156, 'support': 118.0}, 'I-LOC': {'precision': 0.84251968503937, 'recall': 0.84251968503937, 'f1-score': 0.84251968503937, 'support': 254.0}, 'I-MAT': {'precision': 0.225, 'recall': 0.24324324324324326, 'f1-score': 0.23376623376623376, 'support': 37.0}, 'I-PER': {'precision': 0.8463541666666666, 'recall': 0.727069351230425, 'f1-score': 0.7821901323706378, 'support': 447.0}, 'I-SPE': {'precision': 0.8148148148148148, 'recall': 0.5238095238095238, 'f1-score': 0.6376811594202898, 'support': 42.0}, 'O': {'precision': 0.9769036273461053, 'recall': 0.9705328245012318, 'f1-score': 0.9737078052681319, 'support': 20701.0}, 'accuracy': 0.9431089394005497, 'macro avg': {'precision': 0.6151700411810752, 'recall': 0.6631683771912952, 'f1-score': 0.6287209691255019, 'support': 22921.0}, 'weighted avg': {'precision': 0.9467156556961486, 'recall': 0.9431089394005497, 'f1-score': 0.9442987166110726, 'support': 22921.0}}	{'ART': {'precision': 0.36553524804177545, 'recall': 0.5555555555555556, 'f1-score': 0.4409448818897638, 'support': 252}, 'CON': {'precision': 0.40772532188841204, 'recall': 0.5900621118012422, 'f1-score': 0.48223350253807107, 'support': 161}, 'LOC': {'precision': 0.578125, 'recall': 0.7449664429530202, 'f1-score': 0.6510263929618768, 'support': 149}, 'MAT': {'precision': 0.2835820895522388, 'recall': 0.475, 'f1-score': 0.35514018691588783, 'support': 40}, 'PER': {'precision': 0.6775, 'recall': 0.8065476190476191, 'f1-score': 0.7364130434782609, 'support': 336}, 'SPE': {'precision': 0.3333333333333333, 'recall': 0.6129032258064516, 'f1-score': 0.43181818181818177, 'support': 31}, 'micro avg': {'precision': 0.49174174174174173, 'recall': 0.675954592363261, 'f1-score': 0.5693176879617557, 'support': 969}, 'macro avg': {'precision': 0.44096683213595994, 'recall': 0.6308391591939815, 'f1-score': 0.516262698267007, 'support': 969}, 'weighted avg': {'precision': 0.508994738127951, 'recall': 0.675954592363261, 'f1-score': 0.5787279570875793, 'support': 969}}
0.1508	4.0	996	0.2972	0.5083	0.6667	0.5768	0.5149	0.5768	{'B-ART': {'precision': 0.5060606060606061, 'recall': 0.6626984126984127, 'f1-score': 0.5738831615120275, 'support': 252.0}, 'B-CON': {'precision': 0.4375, 'recall': 0.6521739130434783, 'f1-score': 0.5236907730673317, 'support': 161.0}, 'B-LOC': {'precision': 0.8071428571428572, 'recall': 0.7583892617449665, 'f1-score': 0.7820069204152249, 'support': 149.0}, 'B-MAT': {'precision': 0.5357142857142857, 'recall': 0.375, 'f1-score': 0.4411764705882353, 'support': 40.0}, 'B-PER': {'precision': 0.7749360613810742, 'recall': 0.9017857142857143, 'f1-score': 0.8335625859697386, 'support': 336.0}, 'B-SPE': {'precision': 0.4067796610169492, 'recall': 0.7741935483870968, 'f1-score': 0.5333333333333333, 'support': 31.0}, 'I-ART': {'precision': 0.5416666666666666, 'recall': 0.40509915014164305, 'f1-score': 0.46353322528363045, 'support': 353.0}, 'I-CON': {'precision': 0.42857142857142855, 'recall': 0.4830508474576271, 'f1-score': 0.4541832669322709, 'support': 118.0}, 'I-LOC': {'precision': 0.8818565400843882, 'recall': 0.8228346456692913, 'f1-score': 0.8513238289205702, 'support': 254.0}, 'I-MAT': {'precision': 0.4166666666666667, 'recall': 0.13513513513513514, 'f1-score': 0.20408163265306123, 'support': 37.0}, 'I-PER': {'precision': 0.8345679012345679, 'recall': 0.756152125279642, 'f1-score': 0.7934272300469484, 'support': 447.0}, 'I-SPE': {'precision': 0.7666666666666667, 'recall': 0.5476190476190477, 'f1-score': 0.6388888888888888, 'support': 42.0}, 'O': {'precision': 0.9745303118342049, 'recall': 0.97222356407903, 'f1-score': 0.973375571300752, 'support': 20701.0}, 'accuracy': 0.9435888486540727, 'macro avg': {'precision': 0.6394353579261817, 'recall': 0.634335028118545, 'f1-score': 0.6204974529932318, 'support': 22921.0}, 'weighted avg': {'precision': 0.9455450522608214, 'recall': 0.9435888486540727, 'f1-score': 0.9437659943714384, 'support': 22921.0}}	{'ART': {'precision': 0.4061624649859944, 'recall': 0.5753968253968254, 'f1-score': 0.47619047619047616, 'support': 252}, 'CON': {'precision': 0.3779527559055118, 'recall': 0.5962732919254659, 'f1-score': 0.4626506024096385, 'support': 161}, 'LOC': {'precision': 0.6234567901234568, 'recall': 0.6778523489932886, 'f1-score': 0.6495176848874598, 'support': 149}, 'MAT': {'precision': 0.3939393939393939, 'recall': 0.325, 'f1-score': 0.35616438356164376, 'support': 40}, 'PER': {'precision': 0.674937965260546, 'recall': 0.8095238095238095, 'f1-score': 0.7361299052774019, 'support': 336}, 'SPE': {'precision': 0.3064516129032258, 'recall': 0.6129032258064516, 'f1-score': 0.4086021505376344, 'support': 31}, 'micro avg': {'precision': 0.5082612116443745, 'recall': 0.6666666666666666, 'f1-score': 0.5767857142857143, 'support': 969}, 'macro avg': {'precision': 0.46381683051968814, 'recall': 0.5994915836076402, 'f1-score': 0.5148758671440424, 'support': 969}, 'weighted avg': {'precision': 0.5243912576788156, 'recall': 0.6666666666666666, 'f1-score': 0.5836096720521391, 'support': 969}}

Framework versions

• Transformers 4.45.2
• Pytorch 2.4.1
• Datasets 3.0.1
• Tokenizers 0.20.1