bert-finetuned-ner-13-more-q-variations

This model is a fine-tuned version of FacebookAI/roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5523
• Precision: 0.6396
• Recall: 0.6004
• F1: 0.6194
• Accuracy: 0.8862

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 128
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
0.1814	0.2278	200	0.1767	0.6484	0.7109	0.6782	0.9195
0.1096	0.4556	400	0.2426	0.6404	0.5734	0.6050	0.8985
0.0783	0.6834	600	0.3193	0.5905	0.5639	0.5769	0.8911
0.0559	0.9112	800	0.3580	0.6168	0.5982	0.6074	0.8928
0.0357	1.1390	1000	0.5001	0.5797	0.5159	0.5460	0.8760
0.0308	1.3667	1200	0.4609	0.6015	0.5663	0.5834	0.8793
0.0279	1.5945	1400	0.4791	0.6488	0.6248	0.6366	0.8895
0.0276	1.8223	1600	0.5009	0.6211	0.5760	0.5977	0.8825
0.0248	2.0501	1800	0.4665	0.6462	0.6228	0.6343	0.8901
0.0173	2.2779	2000	0.5199	0.6712	0.6930	0.6819	0.9013
0.0147	2.5057	2200	0.5416	0.6328	0.6001	0.6160	0.8867
0.0146	2.7335	2400	0.5662	0.6233	0.5721	0.5966	0.8801
0.0146	2.9613	2600	0.5514	0.6403	0.6024	0.6208	0.8865
0.0146	3.0	2634	0.5523	0.6396	0.6004	0.6194	0.8862

Framework versions

• Transformers 5.7.0
• Pytorch 2.10.0+cu128
• Datasets 4.8.5
• Tokenizers 0.22.2