distilroberta-base-finetuned-media-center

This model is a fine-tuned version of distilroberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.3850

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	38	1.6960
No log	2.0	76	1.5674
No log	3.0	114	1.5541
No log	4.0	152	1.5729
No log	5.0	190	1.5164
No log	6.0	228	1.5343
No log	7.0	266	1.5232
No log	8.0	304	1.4354
No log	9.0	342	1.4203
No log	10.0	380	1.3526
No log	11.0	418	1.3911
No log	12.0	456	1.4522
No log	13.0	494	1.4324
1.4666	14.0	532	1.3606
1.4666	15.0	570	1.3669
1.4666	16.0	608	1.3420
1.4666	17.0	646	1.3544
1.4666	18.0	684	1.3346
1.4666	19.0	722	1.3747
1.4666	20.0	760	1.3850

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.0