roberta-base_algorithm-dept

This model is a fine-tuned version of FacebookAI/roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3682

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 4

Training results

Training Loss	Epoch	Step	Validation Loss
0.8889	0.0588	50	0.6333
0.6473	0.1175	100	0.6382
0.6366	0.1763	150	0.5980
0.6096	0.2350	200	0.5917
0.5822	0.2938	250	0.5283
0.5294	0.3525	300	0.5007
0.5206	0.4113	350	0.4820
0.5279	0.4700	400	0.4569
0.5131	0.5288	450	0.4535
0.5239	0.5875	500	0.4615
0.4498	0.6463	550	0.4414
0.397	0.7051	600	0.4752
0.3985	0.7638	650	0.4178
0.4149	0.8226	700	0.4038
0.4588	0.8813	750	0.4175
0.3975	0.9401	800	0.4950
0.4089	0.9988	850	0.4144
0.3342	1.0576	900	0.4208
0.3696	1.1163	950	0.4182
0.3522	1.1751	1000	0.3990
0.3479	1.2338	1050	0.3977
0.3549	1.2926	1100	0.3849
0.3134	1.3514	1150	0.4000
0.3554	1.4101	1200	0.3850
0.3275	1.4689	1250	0.3698
0.3887	1.5276	1300	0.3659
0.314	1.5864	1350	0.3736
0.3408	1.6451	1400	0.3792
0.3259	1.7039	1450	0.3735
0.3127	1.7626	1500	0.3682
0.3493	1.8214	1550	0.3540
0.3224	1.8801	1600	0.3721
0.3145	1.9389	1650	0.3610
0.2975	1.9976	1700	0.3500
0.2481	2.0564	1750	0.3621
0.2421	2.1152	1800	0.3782
0.2614	2.1739	1850	0.3787
0.2476	2.2327	1900	0.3807

Framework versions

• Transformers 4.46.3
• Pytorch 2.5.1+cu121
• Datasets 3.2.0
• Tokenizers 0.20.3