metadata

license: mit
base_model: roberta-base
tags:
  - generated_from_trainer
metrics:
  - accuracy
model-index:
  - name: roberta_multiple_choice
    results: []

roberta_multiple_choice

This model is a fine-tuned version of roberta-base on the None dataset. It achieves the following results on the evaluation set:

Loss: 0.1836
Accuracy: 0.955

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-06
train_batch_size: 16
eval_batch_size: 16
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.4783	1.0	1725	1.3350	0.405
1.3252	2.0	3450	1.1893	0.565
1.2436	3.0	5175	1.0734	0.57
1.1636	4.0	6900	0.9469	0.61
1.0818	5.0	8625	0.8554	0.675
0.9999	6.0	10350	0.7437	0.715
0.9227	7.0	12075	0.6424	0.76
0.8361	8.0	13800	0.5657	0.8
0.7636	9.0	15525	0.5241	0.815
0.696	10.0	17250	0.4742	0.83
0.6343	11.0	18975	0.4325	0.855
0.5758	12.0	20700	0.4263	0.86
0.5377	13.0	22425	0.3876	0.835
0.4889	14.0	24150	0.3675	0.87
0.4467	15.0	25875	0.3221	0.885
0.4172	16.0	27600	0.3175	0.89
0.3899	17.0	29325	0.3048	0.9
0.3649	18.0	31050	0.2795	0.91
0.3413	19.0	32775	0.2669	0.905
0.3219	20.0	34500	0.2764	0.92
0.3055	21.0	36225	0.2478	0.93
0.2826	22.0	37950	0.2069	0.94
0.2733	23.0	39675	0.2230	0.945
0.2537	24.0	41400	0.2138	0.945
0.2412	25.0	43125	0.1887	0.95
0.2281	26.0	44850	0.1923	0.955
0.2188	27.0	46575	0.1663	0.96
0.2031	28.0	48300	0.2389	0.945
0.2008	29.0	50025	0.1823	0.955
0.1858	30.0	51750	0.1994	0.95
0.1854	31.0	53475	0.2013	0.96
0.1721	32.0	55200	0.1836	0.955

Framework versions

Transformers 4.31.0
Pytorch 2.0.1+cu117
Datasets 2.14.4
Tokenizers 0.13.3