poker-pretraining

This model is a fine-tuned version of nobody12321/poker-pretrain on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.7246

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 1000
• num_epochs: 1
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
2.651	0.0183	5000	2.3357
2.2946	0.0367	10000	2.2261
2.2212	0.0550	15000	2.1725
2.1702	0.0733	20000	2.1131
2.1332	0.0917	25000	2.0965
2.1062	0.1100	30000	2.0647
2.0854	0.1283	35000	2.0518
2.0688	0.1466	40000	2.0344
2.0556	0.1650	45000	2.0200
2.045	0.1833	50000	2.0177
2.0349	0.2016	55000	1.9994
2.0254	0.2200	60000	1.9950
2.0175	0.2383	65000	1.9861
2.0093	0.2566	70000	1.9792
2.0013	0.2750	75000	1.9761
1.9946	0.2933	80000	1.9676
1.988	0.3116	85000	1.9606
1.9817	0.3299	90000	1.9544
1.9729	0.3483	95000	1.9465
1.9662	0.3666	100000	1.9471
1.9597	0.3849	105000	1.9351
1.9532	0.4033	110000	1.9313
1.9475	0.4216	115000	1.9283
1.9407	0.4399	120000	1.9223
1.9356	0.4583	125000	1.9139
1.9308	0.4766	130000	1.9094
1.9244	0.4949	135000	1.9038
1.9194	0.5132	140000	1.8983
1.9134	0.5316	145000	1.8951
1.9093	0.5499	150000	1.8904
1.9038	0.5682	155000	1.8826
1.898	0.5866	160000	1.8776
1.8931	0.6049	165000	1.8738
1.8878	0.6232	170000	1.8685
1.882	0.6416	175000	1.8633
1.8755	0.6599	180000	1.8573
1.87	0.6782	185000	1.8517
1.8648	0.6965	190000	1.8465
1.8587	0.7149	195000	1.8407
1.8537	0.7332	200000	1.8346
1.8358	0.7515	205000	1.8094
1.8207	0.7699	210000	1.7976
1.8109	0.7882	215000	1.7890
1.8031	0.8065	220000	1.7788
1.7926	0.8249	225000	1.7671
1.7821	0.8432	230000	1.7571
1.7756	0.8615	235000	1.7497
1.7678	0.8799	240000	1.7423
1.7627	0.8982	245000	1.7366
1.7581	0.9165	250000	1.7317
1.7538	0.9348	255000	1.7286
1.7513	0.9532	260000	1.7262
1.75	0.9715	265000	1.7251
1.7492	0.9898	270000	1.7246

Framework versions

• Transformers 4.40.2
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1