distilroberta-rbm231k-ep20-op40-news_api_all_5p1k

This model is a fine-tuned version of judy93536/distilroberta-rbm231k-ep20-op40 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2430
• Accuracy: 0.9207

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.53335054745316e-07
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.4
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	256	1.0947	0.3503
1.0947	2.0	512	1.0886	0.4100
1.0947	3.0	768	1.0783	0.4403
1.0814	4.0	1024	1.0618	0.4569
1.0814	5.0	1280	1.0324	0.4501
1.048	6.0	1536	0.9890	0.4795
1.048	7.0	1792	0.9498	0.5068
0.9796	8.0	2048	0.9066	0.5724
0.9796	9.0	2304	0.8510	0.6468
0.8954	10.0	2560	0.7850	0.6840
0.8954	11.0	2816	0.7160	0.7094
0.7824	12.0	3072	0.6495	0.7495
0.7824	13.0	3328	0.5894	0.7818
0.6548	14.0	3584	0.5305	0.8151
0.6548	15.0	3840	0.4613	0.8620
0.5446	16.0	4096	0.3921	0.8836
0.5446	17.0	4352	0.3424	0.8933
0.4323	18.0	4608	0.3111	0.9012
0.4323	19.0	4864	0.2838	0.9070
0.3524	20.0	5120	0.2739	0.9100
0.3524	21.0	5376	0.2674	0.9090
0.3148	22.0	5632	0.2583	0.9119
0.3148	23.0	5888	0.2517	0.9139
0.2931	24.0	6144	0.2496	0.9159
0.2931	25.0	6400	0.2451	0.9159
0.2703	26.0	6656	0.2443	0.9178
0.2703	27.0	6912	0.2435	0.9198
0.2649	28.0	7168	0.2424	0.9198
0.2649	29.0	7424	0.2431	0.9207
0.2607	30.0	7680	0.2430	0.9207

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu118
• Datasets 2.15.0
• Tokenizers 0.15.0