orkic21

This model is a fine-tuned version of EleutherAI/gpt-neo-125M on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.3258

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	29	2.6212
1.5241	2.0	58	1.5125
1.5241	3.0	87	1.3249
0.6325	4.0	116	1.3617
0.6325	5.0	145	1.3474
0.4563	6.0	174	1.2187
0.4082	7.0	203	1.1457
0.4082	8.0	232	1.3229
0.423	9.0	261	1.2785
0.423	10.0	290	1.3113
0.3466	11.0	319	1.2479
0.3466	12.0	348	1.2732
0.3367	13.0	377	1.3168
0.3236	14.0	406	1.3082
0.3236	15.0	435	1.3475
0.3098	16.0	464	1.2798
0.3098	17.0	493	1.2835
0.3107	18.0	522	1.2596
0.2952	19.0	551	1.3817
0.2952	20.0	580	1.3290
0.2865	21.0	609	1.3105
0.2865	22.0	638	1.3328
0.2829	23.0	667	1.3191
0.2829	24.0	696	1.3360
0.2872	25.0	725	1.3400
0.2768	26.0	754	1.3531
0.2768	27.0	783	1.3364
0.2708	28.0	812	1.3332
0.2708	29.0	841	1.3294
0.2698	30.0	870	1.3258

Framework versions

• Transformers 4.52.4
• Pytorch 2.6.0+cu124
• Datasets 3.6.0
• Tokenizers 0.21.2