LCIA_Test3_gpt2_small

This model is a fine-tuned version of uer/gpt2-chinese-cluecorpussmall on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1054

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 18
• eval_batch_size: 18
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 40
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	21	0.7690
No log	2.0	42	0.6252
No log	3.0	63	0.4116
No log	4.0	84	0.1728
No log	5.0	105	0.1211
No log	6.0	126	0.1188
No log	7.0	147	0.1166
No log	8.0	168	0.1113
No log	9.0	189	0.1090
No log	10.0	210	0.1100
No log	11.0	231	0.1029
No log	12.0	252	0.1016
No log	13.0	273	0.0963
No log	14.0	294	0.0998
No log	15.0	315	0.0935
No log	16.0	336	0.0956
No log	17.0	357	0.0925
No log	18.0	378	0.0913
No log	19.0	399	0.0993
No log	20.0	420	0.0981
No log	21.0	441	0.0946
No log	22.0	462	0.1039
No log	23.0	483	0.0984
0.1655	24.0	504	0.0977
0.1655	25.0	525	0.1018
0.1655	26.0	546	0.1040
0.1655	27.0	567	0.0988
0.1655	28.0	588	0.1047
0.1655	29.0	609	0.1059
0.1655	30.0	630	0.1061
0.1655	31.0	651	0.1064
0.1655	32.0	672	0.1049
0.1655	33.0	693	0.1038
0.1655	34.0	714	0.1054
0.1655	35.0	735	0.1016
0.1655	36.0	756	0.1076
0.1655	37.0	777	0.1047
0.1655	38.0	798	0.1055
0.1655	39.0	819	0.1056
0.1655	40.0	840	0.1054

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Tokenizers 0.19.1