STS-conventional-Fine-Tuning-Capstone-roberta-base-filtered-137

This model is a fine-tuned version of FacebookAI/roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.5730
• Accuracy: 0.7247

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	113	0.7973	0.6573
No log	2.0	226	0.6965	0.6948
No log	3.0	339	0.8914	0.6891
No log	4.0	452	0.8767	0.6948
0.5314	5.0	565	0.9786	0.6760
0.5314	6.0	678	1.1437	0.7079
0.5314	7.0	791	1.2355	0.6966
0.5314	8.0	904	1.5219	0.7022
0.1799	9.0	1017	1.4491	0.7041
0.1799	10.0	1130	1.6851	0.7060
0.1799	11.0	1243	1.9943	0.7060
0.1799	12.0	1356	2.0297	0.7060
0.1799	13.0	1469	2.0053	0.7247
0.0712	14.0	1582	1.9966	0.7266
0.0712	15.0	1695	2.1857	0.7097
0.0712	16.0	1808	2.2013	0.7228
0.0712	17.0	1921	2.2569	0.7172
0.0419	18.0	2034	2.2553	0.7172
0.0419	19.0	2147	2.3893	0.7022
0.0419	20.0	2260	2.4651	0.7116
0.0419	21.0	2373	2.4000	0.7135
0.0419	22.0	2486	2.5071	0.7135
0.0241	23.0	2599	2.4959	0.7285
0.0241	24.0	2712	2.5238	0.7191
0.0241	25.0	2825	2.5499	0.7285
0.0241	26.0	2938	2.5826	0.7247
0.0088	27.0	3051	2.6062	0.7228
0.0088	28.0	3164	2.5904	0.7154
0.0088	29.0	3277	2.5756	0.7228
0.0088	30.0	3390	2.5730	0.7247

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2