roberta-el-ner4

This model is a fine-tuned version of cvcio/roberta-el-news on the elNER dataset. It achieves the following results on the evaluation set:

• Loss: 0.0564
• Precision: 0.9116
• Recall: 0.9218
• F1: 0.9167
• Accuracy: 0.9883

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

More information needed

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 60.0

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
0.1723	1.87	250	0.0716	0.7937	0.8657	0.8281	0.9786
0.0372	3.73	500	0.0497	0.8587	0.9215	0.8890	0.9852
0.0243	5.6	750	0.0524	0.8746	0.9263	0.8997	0.9867
0.0114	7.46	1000	0.0564	0.9116	0.9218	0.9167	0.9883
0.0071	9.33	1250	0.0602	0.9019	0.9309	0.9161	0.9881
0.0037	11.19	1500	0.0667	0.9074	0.9306	0.9188	0.9885
0.003	13.06	1750	0.0688	0.9001	0.9306	0.9151	0.9883
0.0019	14.93	2000	0.0724	0.9082	0.9229	0.9155	0.9883
0.0022	16.79	2250	0.0745	0.9159	0.9169	0.9164	0.9878
0.0016	18.66	2500	0.0727	0.9068	0.9249	0.9158	0.9880
0.0018	20.52	2750	0.0732	0.9088	0.9272	0.9179	0.9887
0.0014	22.39	3000	0.0767	0.9017	0.9243	0.9129	0.9876
0.0012	24.25	3250	0.0745	0.9072	0.9206	0.9139	0.9882
0.0011	26.12	3500	0.0790	0.8995	0.9297	0.9144	0.9878
0.0008	27.99	3750	0.0786	0.9081	0.9275	0.9177	0.9883
0.0011	29.85	4000	0.0775	0.9091	0.9277	0.9183	0.9885
0.0011	31.72	4250	0.0851	0.9005	0.9269	0.9135	0.9879
0.0007	33.58	4500	0.0848	0.9041	0.9223	0.9131	0.9876
0.0006	35.45	4750	0.0842	0.9082	0.9263	0.9172	0.9881
0.0005	37.31	5000	0.0851	0.9085	0.9266	0.9175	0.9881
0.0004	39.18	5250	0.0878	0.9035	0.9272	0.9152	0.9879
0.0004	41.04	5500	0.0856	0.9091	0.9275	0.9182	0.9885
0.0004	42.91	5750	0.0870	0.9099	0.9255	0.9176	0.9884
0.0005	44.78	6000	0.0860	0.9010	0.9269	0.9138	0.9882
0.0004	46.64	6250	0.0851	0.9114	0.9246	0.9179	0.9884
0.0003	48.51	6500	0.0899	0.9058	0.9252	0.9154	0.9884
0.0002	50.37	6750	0.0898	0.9050	0.9294	0.9171	0.9882
0.0002	52.24	7000	0.0890	0.9104	0.9252	0.9177	0.9884
0.0002	54.1	7250	0.0898	0.9052	0.9260	0.9155	0.9879
0.0002	55.97	7500	0.0894	0.9080	0.9263	0.9171	0.9883
0.0001	57.84	7750	0.0910	0.9046	0.9277	0.9160	0.9883
0.0003	59.7	8000	0.0903	0.9041	0.9283	0.9161	0.9882

Eval results

	Precision	Recall	F1	Accuracy
eval	0.9116	0.9218	0.9167	0.9883
test	0.9022	0.9107	0.9064	0.9861

Framework versions

• Transformers 4.29.2
• Pytorch 1.13.1+cu117
• Datasets 2.9.0
• Tokenizers 0.13.2

Authors

Dimitris Papaevagelou - @andefined

About Us

Civic Information Office is a Non Profit Organization based in Athens, Greece focusing on creating technology and research products for the public interest.