DrinkIcedT/roberta-large_MBTI_pub_P

roberta-large_MBTI_P_MBTI_agg_balanced_50_1e-05_0.0

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.5241
• F1: 0.6533
• Threshold: 0.31
• F1 At 05: 0.6457

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 64
• total_eval_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 400
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	F1	Threshold	F1 At 05
2.7839	0.3175	200	2.7551	0.5531	0.53	0.4409
2.7422	0.6349	400	2.6916	0.5803	0.5	0.5803
2.5812	0.9524	600	2.6840	0.6210	0.63	0.5364
2.3646	1.2698	800	2.5167	0.6453	0.49	0.6435
2.2991	1.5873	1000	2.4703	0.6553	0.4500	0.6386
2.2415	1.9048	1200	2.4315	0.6560	0.47	0.6520
1.9308	2.2222	1400	2.5445	0.6533	0.5	0.6533
1.8541	2.5397	1600	2.7332	0.6563	0.4200	0.6482
1.8320	2.8571	1800	2.7127	0.6576	0.4500	0.6562
1.5062	3.1746	2000	2.9485	0.6556	0.4	0.6485
1.4682	3.4921	2200	3.1023	0.6575	0.4600	0.6525
1.4277	3.8095	2400	3.1742	0.6563	0.49	0.6550
1.1521	4.1270	2600	3.4051	0.6503	0.35	0.6447
1.1145	4.4444	2800	3.4738	0.6539	0.36	0.6501
1.1069	4.7619	3000	3.5241	0.6533	0.31	0.6457

Framework versions

• Transformers 5.3.0
• Pytorch 2.7.1
• Datasets 4.8.5
• Tokenizers 0.22.2