distilroberta-base-finetuned-marktextepoch_35

This model is a fine-tuned version of distilroberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.0029

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 35

Training results

Training Loss	Epoch	Step	Validation Loss
2.5158	1.0	1500	2.3385
2.4312	2.0	3000	2.2620
2.3563	3.0	4500	2.2279
2.3249	4.0	6000	2.2165
2.2515	5.0	7500	2.2246
2.2178	6.0	9000	2.1714
2.1822	7.0	10500	2.1461
2.1501	8.0	12000	2.1388
2.1342	9.0	13500	2.1085
2.1141	10.0	15000	2.1090
2.0833	11.0	16500	2.1130
2.0769	12.0	18000	2.0969
2.0474	13.0	19500	2.0823
2.0364	14.0	21000	2.0893
2.0269	15.0	22500	2.0501
1.9814	16.0	24000	2.0667
1.9716	17.0	25500	2.0570
1.9611	18.0	27000	2.0530
1.9557	19.0	28500	2.0590
1.9443	20.0	30000	2.0381
1.9229	21.0	31500	2.0433
1.9192	22.0	33000	2.0468
1.8865	23.0	34500	2.0361
1.914	24.0	36000	2.0412
1.867	25.0	37500	2.0165
1.8724	26.0	39000	2.0152
1.8644	27.0	40500	2.0129
1.8685	28.0	42000	2.0183
1.8458	29.0	43500	2.0082
1.8653	30.0	45000	1.9939
1.8584	31.0	46500	2.0015
1.8396	32.0	48000	1.9924
1.8399	33.0	49500	2.0102
1.8363	34.0	51000	1.9946
1.83	35.0	52500	1.9908

Framework versions

• Transformers 4.20.1
• Pytorch 1.12.0+cu113
• Datasets 2.4.0
• Tokenizers 0.12.1