tapt_helpfulness_base_pretraining_model_full_train

This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.2718

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 21
• eval_batch_size: 21
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-06
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss
1.6862	1.0	1068	1.5262
1.5864	2.0	2137	1.4706
1.5423	3.0	3205	1.4346
1.5058	4.0	4274	1.4223
1.4792	5.0	5342	1.4100
1.4549	6.0	6411	1.3948
1.435	7.0	7479	1.3851
1.4148	8.0	8548	1.3749
1.3968	9.0	9616	1.3567
1.3806	10.0	10685	1.3548
1.3662	11.0	11753	1.3347
1.3495	12.0	12822	1.3369
1.3382	13.0	13890	1.3232
1.3238	14.0	14959	1.3231
1.3138	15.0	16027	1.3157
1.2983	16.0	17096	1.3012
1.2916	17.0	18164	1.2871
1.282	18.0	19233	1.2919
1.2738	19.0	20301	1.2825
1.267	19.99	21360	1.2842

Framework versions

• Transformers 4.36.2
• Pytorch 2.2.1+cu121
• Datasets 2.19.0
• Tokenizers 0.15.2