CS505-NerCOQE-xlm-Predicate

This model is a fine-tuned version of xlm-roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0003
• F1: 0.9976

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	F1
No log	1.0	53	0.1876	0.5087
No log	2.0	106	0.1014	0.7119
No log	3.0	159	0.0564	0.8287
No log	4.0	212	0.0361	0.8835
No log	5.0	265	0.0282	0.8951
No log	6.0	318	0.0154	0.9392
No log	7.0	371	0.0231	0.8730
No log	8.0	424	0.0054	0.9763
No log	9.0	477	0.0031	0.9792
No log	10.0	530	0.0027	0.9828
No log	11.0	583	0.0015	0.9905
No log	12.0	636	0.0031	0.9929
No log	13.0	689	0.0023	0.9941
No log	14.0	742	0.0016	0.9923
No log	15.0	795	0.0011	0.9917
No log	16.0	848	0.0006	0.9964
No log	17.0	901	0.0003	0.9988
No log	18.0	954	0.0003	0.9976
No log	19.0	1007	0.0003	0.9976
No log	20.0	1060	0.0003	0.9976

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2