bert-finetuned-ner-assignment2

This model is a fine-tuned version of bert-base-cased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2276
• Art: {'precision': 0.3963254593175853, 'recall': 0.5992063492063492, 'f1': 0.47709320695102686, 'number': 252}
• Con: {'precision': 0.3173076923076923, 'recall': 0.6073619631901841, 'f1': 0.4168421052631579, 'number': 163}
• Loc: {'precision': 0.6319018404907976, 'recall': 0.6912751677852349, 'f1': 0.6602564102564102, 'number': 149}
• Mat: {'precision': 0.3181818181818182, 'recall': 0.175, 'f1': 0.22580645161290322, 'number': 40}
• Per: {'precision': 0.6862745098039216, 'recall': 0.8308605341246291, 'f1': 0.7516778523489932, 'number': 337}
• Spe: {'precision': 0.3870967741935484, 'recall': 0.7741935483870968, 'f1': 0.5161290322580645, 'number': 31}
• Overall Precision: 0.4926
• Overall Recall: 0.6831
• Overall F1: 0.5724
• Overall Accuracy: 0.9458
• B-art: {'precision': 0.5297619047619048, 'recall': 0.7063492063492064, 'f1-score': 0.6054421768707483, 'support': 252.0}
• B-con: {'precision': 0.37755102040816324, 'recall': 0.6809815950920245, 'f1-score': 0.48577680525164113, 'support': 163.0}
• B-loc: {'precision': 0.7707006369426752, 'recall': 0.8120805369127517, 'f1-score': 0.7908496732026143, 'support': 149.0}
• B-mat: {'precision': 0.45454545454545453, 'recall': 0.25, 'f1-score': 0.3225806451612903, 'support': 40.0}
• B-per: {'precision': 0.7916666666666666, 'recall': 0.9020771513353115, 'f1-score': 0.8432732316227461, 'support': 337.0}
• B-spe: {'precision': 0.45, 'recall': 0.8709677419354839, 'f1-score': 0.5934065934065934, 'support': 31.0}
• I-art: {'precision': 0.6175438596491228, 'recall': 0.4985835694050991, 'f1-score': 0.5517241379310345, 'support': 353.0}
• I-con: {'precision': 0.4552238805970149, 'recall': 0.5041322314049587, 'f1-score': 0.47843137254901963, 'support': 121.0}
• I-loc: {'precision': 0.8613445378151261, 'recall': 0.8070866141732284, 'f1-score': 0.8333333333333334, 'support': 254.0}
• I-mat: {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 37.0}
• I-per: {'precision': 0.8251231527093597, 'recall': 0.7494407158836689, 'f1-score': 0.7854630715123095, 'support': 447.0}
• I-spe: {'precision': 0.8857142857142857, 'recall': 0.7380952380952381, 'f1-score': 0.8051948051948052, 'support': 42.0}
• O: {'precision': 0.9780855199222546, 'recall': 0.9721806326974161, 'f1-score': 0.9751241370957975, 'support': 20705.0}
• Accuracy: 0.9458
• Macro avg: {'precision': 0.6151739169024637, 'recall': 0.653228864098799, 'f1-score': 0.620815383317841, 'support': 22931.0}
• Weighted avg: {'precision': 0.9488441855978484, 'recall': 0.9457939034494789, 'f1-score': 0.9465026590720415, 'support': 22931.0}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Art	Con	Loc	Mat	Per	Spe	Overall Precision	Overall Recall	Overall F1	Overall Accuracy	B-art	B-con	B-loc	B-mat	B-per	B-spe	I-art	I-con	I-loc	I-mat	I-per	I-spe	O	Accuracy	Macro avg	Weighted avg
No log	1.0	249	0.2068	{'precision': 0.36046511627906974, 'recall': 0.6150793650793651, 'f1': 0.45454545454545453, 'number': 252}	{'precision': 0.30718954248366015, 'recall': 0.5766871165644172, 'f1': 0.40085287846481876, 'number': 163}	{'precision': 0.5, 'recall': 0.5771812080536913, 'f1': 0.5358255451713396, 'number': 149}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 40}	{'precision': 0.6437346437346437, 'recall': 0.7774480712166172, 'f1': 0.7043010752688174, 'number': 337}	{'precision': 0.41379310344827586, 'recall': 0.3870967741935484, 'f1': 0.39999999999999997, 'number': 31}	0.4531	0.6265	0.5259	0.9416	{'precision': 0.4626865671641791, 'recall': 0.7380952380952381, 'f1-score': 0.5688073394495413, 'support': 252.0}	{'precision': 0.3797909407665505, 'recall': 0.6687116564417178, 'f1-score': 0.48444444444444446, 'support': 163.0}	{'precision': 0.7012195121951219, 'recall': 0.7718120805369127, 'f1-score': 0.7348242811501597, 'support': 149.0}	{'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 40.0}	{'precision': 0.7557251908396947, 'recall': 0.8813056379821959, 'f1-score': 0.8136986301369863, 'support': 337.0}	{'precision': 0.5517241379310345, 'recall': 0.5161290322580645, 'f1-score': 0.5333333333333333, 'support': 31.0}	{'precision': 0.563573883161512, 'recall': 0.46458923512747874, 'f1-score': 0.5093167701863354, 'support': 353.0}	{'precision': 0.4811320754716981, 'recall': 0.4214876033057851, 'f1-score': 0.44933920704845814, 'support': 121.0}	{'precision': 0.8557692307692307, 'recall': 0.7007874015748031, 'f1-score': 0.7705627705627706, 'support': 254.0}	{'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 37.0}	{'precision': 0.851063829787234, 'recall': 0.7158836689038032, 'f1-score': 0.7776427703523694, 'support': 447.0}	{'precision': 1.0, 'recall': 0.047619047619047616, 'f1-score': 0.09090909090909091, 'support': 42.0}	{'precision': 0.9748947903061965, 'recall': 0.9733880705143685, 'f1-score': 0.9741408477935135, 'support': 20705.0}	0.9416	{'precision': 0.582890781414804, 'recall': 0.5307545132584165, 'f1-score': 0.5159245757974619, 'support': 22931.0}	{'precision': 0.943566071906345, 'recall': 0.941607430988618, 'f1-score': 0.9407978302275771, 'support': 22931.0}
No log	2.0	498	0.2062	{'precision': 0.38563829787234044, 'recall': 0.5753968253968254, 'f1': 0.4617834394904458, 'number': 252}	{'precision': 0.371900826446281, 'recall': 0.5521472392638037, 'f1': 0.4444444444444445, 'number': 163}	{'precision': 0.6047904191616766, 'recall': 0.6778523489932886, 'f1': 0.6392405063291139, 'number': 149}	{'precision': 0.2857142857142857, 'recall': 0.15, 'f1': 0.1967213114754098, 'number': 40}	{'precision': 0.6763990267639902, 'recall': 0.8249258160237388, 'f1': 0.7433155080213903, 'number': 337}	{'precision': 0.35714285714285715, 'recall': 0.6451612903225806, 'f1': 0.4597701149425288, 'number': 31}	0.5027	0.6584	0.5702	0.9465	{'precision': 0.48725212464589235, 'recall': 0.6825396825396826, 'f1-score': 0.5685950413223141, 'support': 252.0}	{'precision': 0.44206008583690987, 'recall': 0.6319018404907976, 'f1-score': 0.5202020202020202, 'support': 163.0}	{'precision': 0.7692307692307693, 'recall': 0.8053691275167785, 'f1-score': 0.7868852459016393, 'support': 149.0}	{'precision': 0.42857142857142855, 'recall': 0.225, 'f1-score': 0.29508196721311475, 'support': 40.0}	{'precision': 0.772609819121447, 'recall': 0.887240356083086, 'f1-score': 0.8259668508287292, 'support': 337.0}	{'precision': 0.47058823529411764, 'recall': 0.7741935483870968, 'f1-score': 0.5853658536585366, 'support': 31.0}	{'precision': 0.6403508771929824, 'recall': 0.41359773371104813, 'f1-score': 0.5025817555938038, 'support': 353.0}	{'precision': 0.5568181818181818, 'recall': 0.4049586776859504, 'f1-score': 0.4688995215311005, 'support': 121.0}	{'precision': 0.8565400843881856, 'recall': 0.7992125984251969, 'f1-score': 0.8268839103869654, 'support': 254.0}	{'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 37.0}	{'precision': 0.8203883495145631, 'recall': 0.756152125279642, 'f1-score': 0.7869615832363213, 'support': 447.0}	{'precision': 0.8928571428571429, 'recall': 0.5952380952380952, 'f1-score': 0.7142857142857143, 'support': 42.0}	{'precision': 0.9749240488016588, 'recall': 0.9764308138130886, 'f1-score': 0.975676849572897, 'support': 20705.0}	0.9465	{'precision': 0.6240147036364062, 'recall': 0.611679584551574, 'f1-score': 0.6044143318256274, 'support': 22931.0}	{'precision': 0.9464286502661701, 'recall': 0.9465352579477563, 'f1-score': 0.9454870238283699, 'support': 22931.0}
0.1922	3.0	747	0.2276	{'precision': 0.3963254593175853, 'recall': 0.5992063492063492, 'f1': 0.47709320695102686, 'number': 252}	{'precision': 0.3173076923076923, 'recall': 0.6073619631901841, 'f1': 0.4168421052631579, 'number': 163}	{'precision': 0.6319018404907976, 'recall': 0.6912751677852349, 'f1': 0.6602564102564102, 'number': 149}	{'precision': 0.3181818181818182, 'recall': 0.175, 'f1': 0.22580645161290322, 'number': 40}	{'precision': 0.6862745098039216, 'recall': 0.8308605341246291, 'f1': 0.7516778523489932, 'number': 337}	{'precision': 0.3870967741935484, 'recall': 0.7741935483870968, 'f1': 0.5161290322580645, 'number': 31}	0.4926	0.6831	0.5724	0.9458	{'precision': 0.5297619047619048, 'recall': 0.7063492063492064, 'f1-score': 0.6054421768707483, 'support': 252.0}	{'precision': 0.37755102040816324, 'recall': 0.6809815950920245, 'f1-score': 0.48577680525164113, 'support': 163.0}	{'precision': 0.7707006369426752, 'recall': 0.8120805369127517, 'f1-score': 0.7908496732026143, 'support': 149.0}	{'precision': 0.45454545454545453, 'recall': 0.25, 'f1-score': 0.3225806451612903, 'support': 40.0}	{'precision': 0.7916666666666666, 'recall': 0.9020771513353115, 'f1-score': 0.8432732316227461, 'support': 337.0}	{'precision': 0.45, 'recall': 0.8709677419354839, 'f1-score': 0.5934065934065934, 'support': 31.0}	{'precision': 0.6175438596491228, 'recall': 0.4985835694050991, 'f1-score': 0.5517241379310345, 'support': 353.0}	{'precision': 0.4552238805970149, 'recall': 0.5041322314049587, 'f1-score': 0.47843137254901963, 'support': 121.0}	{'precision': 0.8613445378151261, 'recall': 0.8070866141732284, 'f1-score': 0.8333333333333334, 'support': 254.0}	{'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 37.0}	{'precision': 0.8251231527093597, 'recall': 0.7494407158836689, 'f1-score': 0.7854630715123095, 'support': 447.0}	{'precision': 0.8857142857142857, 'recall': 0.7380952380952381, 'f1-score': 0.8051948051948052, 'support': 42.0}	{'precision': 0.9780855199222546, 'recall': 0.9721806326974161, 'f1-score': 0.9751241370957975, 'support': 20705.0}	0.9458	{'precision': 0.6151739169024637, 'recall': 0.653228864098799, 'f1-score': 0.620815383317841, 'support': 22931.0}	{'precision': 0.9488441855978484, 'recall': 0.9457939034494789, 'f1-score': 0.9465026590720415, 'support': 22931.0}

Framework versions

• Transformers 5.22.1
• Pytorch 2.14.1+cpu
• Datasets 4.8.5
• Tokenizers 0.23.3