metadata

base_model: FacebookAI/xlm-roberta-base
library_name: transformers
license: mit
metrics:
  - precision
  - recall
  - f1
  - accuracy
tags:
  - generated_from_trainer
model-index:
  - name: scenario-kd-pre-ner-full-xlmr_data-univner_full66
    results: []

scenario-kd-pre-ner-full-xlmr_data-univner_full66

This model is a fine-tuned version of FacebookAI/xlm-roberta-base on the None dataset. It achieves the following results on the evaluation set:

Loss: 48.2869
Precision: 0.8199
Recall: 0.8357
F1: 0.8277
Accuracy: 0.9820

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 3e-05
train_batch_size: 8
eval_batch_size: 32
seed: 66
gradient_accumulation_steps: 4
total_train_batch_size: 32
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
125.1691	0.2911	500	87.0081	0.7214	0.7142	0.7178	0.9733
79.398	0.5822	1000	74.2576	0.7483	0.7873	0.7673	0.9766
71.1356	0.8732	1500	69.3294	0.7983	0.7680	0.7829	0.9781
66.3695	1.1643	2000	65.7687	0.7888	0.8071	0.7978	0.9797
63.0397	1.4554	2500	63.3432	0.7887	0.8071	0.7978	0.9795
60.9741	1.7465	3000	61.3539	0.8072	0.8042	0.8057	0.9801
59.1711	2.0375	3500	59.9207	0.7996	0.8064	0.8030	0.9800
56.9342	2.3286	4000	58.2970	0.7933	0.8205	0.8067	0.9805
55.7631	2.6197	4500	57.1782	0.8095	0.8228	0.8161	0.9811
54.9048	2.9108	5000	56.2284	0.8047	0.8214	0.8129	0.9809
53.4488	3.2019	5500	55.2710	0.8025	0.8282	0.8151	0.9811
52.5197	3.4929	6000	54.6004	0.8184	0.8101	0.8142	0.9809
51.7352	3.7840	6500	53.8065	0.8014	0.8316	0.8163	0.9807
51.0298	4.0751	7000	53.1329	0.8129	0.8261	0.8195	0.9812
50.2097	4.3662	7500	52.4983	0.8155	0.8272	0.8213	0.9815
49.7445	4.6573	8000	52.1048	0.8134	0.8225	0.8179	0.9814
49.2896	4.9483	8500	51.5581	0.8169	0.8335	0.8251	0.9818
48.7639	5.2394	9000	51.2521	0.8145	0.8282	0.8213	0.9817
48.2298	5.5305	9500	50.7723	0.8134	0.8295	0.8213	0.9816
47.942	5.8216	10000	50.5179	0.8131	0.8323	0.8226	0.9817
47.4768	6.1126	10500	50.1601	0.8179	0.8322	0.8250	0.9818
47.176	6.4037	11000	49.9205	0.8154	0.8335	0.8243	0.9819
46.9979	6.6948	11500	49.6460	0.8201	0.8321	0.8260	0.9819
46.7367	6.9859	12000	49.4427	0.8145	0.8310	0.8227	0.9815
46.3668	7.2770	12500	49.3164	0.8139	0.8287	0.8213	0.9815
46.1975	7.5680	13000	49.0327	0.8137	0.8336	0.8235	0.9817
46.1108	7.8591	13500	48.9452	0.8249	0.8298	0.8273	0.9819
45.8657	8.1502	14000	48.7530	0.8234	0.8352	0.8293	0.9823
45.6956	8.4413	14500	48.5710	0.8210	0.8387	0.8298	0.9823
45.6259	8.7324	15000	48.5836	0.8200	0.8325	0.8262	0.9818
45.5612	9.0234	15500	48.4547	0.8214	0.8341	0.8277	0.9822
45.4012	9.3145	16000	48.3506	0.8224	0.8373	0.8298	0.9821
45.3943	9.6056	16500	48.3722	0.8207	0.8325	0.8265	0.9820
45.3452	9.8967	17000	48.2869	0.8199	0.8357	0.8277	0.9820

Framework versions

Transformers 4.44.2
Pytorch 2.1.1+cu121
Datasets 2.14.5
Tokenizers 0.19.1