metadata

license: mit
base_model: roberta-large
tags:
  - generated_from_trainer
metrics:
  - accuracy
model-index:
  - name: roberta-large-fomc
    results: []

roberta-large-fomc

This model is a fine-tuned version of roberta-large on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.7874
Accuracy: 0.6660

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 16
eval_batch_size: 16
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 500
num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.0083	1	1.0582	0.4980
1.0574	0.2149	26	1.0428	0.4980
1.0702	0.4215	51	1.0500	0.4980
1.1065	0.6281	76	1.0374	0.4980
1.0241	0.8347	101	1.0391	0.4980
1.0324	1.0	121	1.0191	0.4980
1.0324	1.0413	126	1.0097	0.4980
0.9751	1.2479	151	1.0542	0.4737
1.0134	1.4545	176	0.9746	0.5931
0.9276	1.6612	201	0.8633	0.5648
0.8469	1.8678	226	0.7729	0.6538
0.7992	2.0	242	0.7874	0.6660
0.8853	2.0744	251	0.8597	0.6680
0.6466	2.2810	276	0.7767	0.6498
0.778	2.4876	301	1.0588	0.6498
0.7202	2.6942	326	0.7493	0.6721
0.7108	2.9008	351	0.8892	0.6397
0.6354	3.0	363	0.8265	0.6579
0.7704	3.1074	376	0.7833	0.6781
0.6867	3.3140	401	0.9702	0.6478
0.6973	3.5207	426	1.0300	0.6700
0.6682	3.7273	451	0.8206	0.6781
0.6605	3.9339	476	0.8862	0.6822
0.8521	4.0	484	0.8093	0.6316
0.6442	4.1405	501	0.9483	0.6437
0.577	4.3471	526	0.8860	0.6883
0.5252	4.5537	551	0.8797	0.7045
0.5274	4.7603	576	0.7289	0.7024
0.467	4.9669	601	0.8224	0.6903
0.467	5.0	605	0.8218	0.6903

Framework versions

Transformers 4.40.2
Pytorch 1.12.0
Datasets 2.19.1
Tokenizers 0.19.1