roberta_results

This model is a fine-tuned version of roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.1846
• Accuracy: 0.8767
• F1: 0.8904
• Precision: 0.8057
• Recall: 0.9950

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 0.1
• num_epochs: 5
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1	Precision	Recall
5.2894	0.0956	100	0.6470	0.7321	0.7549	0.6997	0.8196
2.1692	0.1912	200	0.2544	0.8494	0.8645	0.7900	0.9545
1.8798	0.2868	300	0.2252	0.8634	0.8444	0.9893	0.7366
1.8287	0.3824	400	0.2065	0.8692	0.8518	0.9909	0.7470
1.7022	0.4780	500	0.1999	0.8705	0.8852	0.7989	0.9925
1.7770	0.5736	600	0.2032	0.8729	0.8564	0.9926	0.7531
1.5242	0.6692	700	0.1919	0.8745	0.8888	0.8025	0.9958
1.5454	0.7648	800	0.2123	0.8728	0.8873	0.8011	0.9943
1.6533	0.8604	900	0.2035	0.8736	0.8581	0.9870	0.7589
1.5114	0.9560	1000	0.1991	0.8727	0.8557	0.9962	0.7499
1.6955	1.0516	1100	0.1909	0.8749	0.8889	0.8037	0.9944
1.5485	1.1472	1200	0.1876	0.8746	0.8886	0.8038	0.9934
1.4344	1.2428	1300	0.1849	0.8758	0.8898	0.8038	0.9964
1.5597	1.3384	1400	0.1902	0.8753	0.8591	0.9958	0.7555
1.4956	1.4340	1500	0.1937	0.8742	0.8887	0.8007	0.9985
1.6025	1.5296	1600	0.1856	0.8760	0.8897	0.8051	0.9942
1.2909	1.6252	1700	0.1846	0.8767	0.8904	0.8057	0.9950
1.5576	1.7208	1800	0.1857	0.8769	0.8612	0.9961	0.7584
1.5306	1.8164	1900	0.1898	0.8760	0.8901	0.8033	0.9979
1.4076	1.9120	2000	0.1956	0.8761	0.8608	0.9909	0.7609
1.3811	2.0076	2100	0.1851	0.8773	0.8616	0.9960	0.7593

Framework versions

• Transformers 5.12.1
• Pytorch 2.11.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2