---
tags:
- generated_from_trainer
base_model: austindavis/gpt2-lichess-uci-2016-01_11
model-index:
- name: gpt2-lichess-uci-202306
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# gpt2-lichess-uci-202306

This model is a fine-tuned version of [austindavis/gpt2-lichess-uci-2016-01_11](https://huggingface.co/austindavis/gpt2-lichess-uci-2016-01_11) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.8839

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.002
- train_batch_size: 20
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- num_epochs: 1

### Training results

| Training Loss | Epoch  | Step    | Validation Loss |
|:-------------:|:------:|:-------:|:---------------:|
| 1.022         | 0.1323 | 165000  | 1.0013          |
| 1.0204        | 0.1443 | 180000  | 1.0001          |
| 1.0186        | 0.1563 | 195000  | 0.9973          |
| 1.0155        | 0.1684 | 210000  | 0.9954          |
| 1.0133        | 0.1804 | 225000  | 0.9935          |
| 1.0118        | 0.1924 | 240000  | 0.9924          |
| 1.0092        | 0.2044 | 255000  | 0.9893          |
| 1.007         | 0.2165 | 270000  | 0.9881          |
| 1.0057        | 0.2285 | 285000  | 0.9868          |
| 1.0035        | 0.2405 | 300000  | 0.9879          |
| 1.004         | 0.2525 | 315000  | 0.9843          |
| 1.0005        | 0.2646 | 330000  | 0.9807          |
| 0.9986        | 0.2766 | 345000  | 0.9805          |
| 0.9983        | 0.2886 | 360000  | 0.9776          |
| 0.9965        | 0.3006 | 375000  | 0.9781          |
| 0.9935        | 0.3127 | 390000  | 0.9754          |
| 0.9935        | 0.3247 | 405000  | 0.9761          |
| 0.9916        | 0.3367 | 420000  | 0.9743          |
| 0.989         | 0.3487 | 435000  | 0.9712          |
| 0.988         | 0.3608 | 450000  | 0.9702          |
| 0.9862        | 0.3728 | 465000  | 0.9703          |
| 0.9837        | 0.3848 | 480000  | 0.9680          |
| 0.983         | 0.3968 | 495000  | 0.9643          |
| 0.9816        | 0.4089 | 510000  | 0.9634          |
| 0.9796        | 0.4209 | 525000  | 0.9628          |
| 0.9777        | 0.4329 | 540000  | 0.9612          |
| 0.9744        | 0.4449 | 555000  | 0.9587          |
| 0.9733        | 0.4570 | 570000  | 0.9590          |
| 0.97          | 0.4690 | 585000  | 0.9566          |
| 0.9693        | 0.4810 | 600000  | 0.9539          |
| 0.9684        | 0.4930 | 615000  | 0.9532          |
| 0.9652        | 0.5051 | 630000  | 0.9509          |
| 0.9644        | 0.5171 | 645000  | 0.9501          |
| 0.9614        | 0.5291 | 660000  | 0.9479          |
| 0.9606        | 0.5411 | 675000  | 0.9466          |
| 0.9597        | 0.5532 | 690000  | 0.9444          |
| 0.9556        | 0.5652 | 705000  | 0.9416          |
| 0.9541        | 0.5772 | 720000  | 0.9413          |
| 0.9522        | 0.5892 | 735000  | 0.9382          |
| 0.9491        | 0.6013 | 750000  | 0.9367          |
| 0.9471        | 0.6133 | 765000  | 0.9354          |
| 0.9459        | 0.6253 | 780000  | 0.9321          |
| 0.9416        | 0.6373 | 795000  | 0.9309          |
| 0.9401        | 0.6494 | 810000  | 0.9287          |
| 0.9383        | 0.6614 | 825000  | 0.9265          |
| 0.9375        | 0.6734 | 840000  | 0.9238          |
| 0.9354        | 0.6854 | 855000  | 0.9225          |
| 0.9323        | 0.6975 | 870000  | 0.9196          |
| 0.9291        | 0.7095 | 885000  | 0.9189          |
| 0.9276        | 0.7215 | 900000  | 0.9165          |
| 0.9266        | 0.7335 | 915000  | 0.9142          |
| 0.9221        | 0.7456 | 930000  | 0.9130          |
| 0.9216        | 0.7576 | 945000  | 0.9106          |
| 0.9191        | 0.7696 | 960000  | 0.9084          |
| 0.9152        | 0.7816 | 975000  | 0.9062          |
| 0.9127        | 0.7937 | 990000  | 0.9039          |
| 0.9133        | 0.8057 | 1005000 | 0.9014          |
| 0.9086        | 0.8177 | 1020000 | 0.8997          |
| 0.9078        | 0.8297 | 1035000 | 0.8978          |
| 0.9054        | 0.8418 | 1050000 | 0.8955          |
| 0.9037        | 0.8538 | 1065000 | 0.8943          |
| 0.9015        | 0.8658 | 1080000 | 0.8926          |
| 0.9006        | 0.8778 | 1095000 | 0.8912          |
| 0.8991        | 0.8899 | 1110000 | 0.8897          |
| 0.897         | 0.9019 | 1125000 | 0.8885          |
| 0.8971        | 0.9139 | 1140000 | 0.8873          |
| 0.894         | 0.9259 | 1155000 | 0.8864          |
| 0.8938        | 0.9380 | 1170000 | 0.8854          |
| 0.893         | 0.9500 | 1185000 | 0.8848          |
| 0.8922        | 0.9620 | 1200000 | 0.8844          |
| 0.8936        | 0.9740 | 1215000 | 0.8841          |
| 0.8923        | 0.9861 | 1230000 | 0.8840          |
| 0.8922        | 0.9981 | 1245000 | 0.8839          |


### Framework versions

- Transformers 4.40.1
- Pytorch 2.3.0
- Datasets 2.19.1
- Tokenizers 0.19.1