--- tags: - generated_from_trainer model-index: - name: gpt2-lichess-uci-202306-12x12 results: [] --- # gpt2-lichess-uci-202306-12x12 This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 0.8365 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.0002 - train_batch_size: 6 - eval_batch_size: 8 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: cosine - num_epochs: 1 ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:------:|:-------:|:---------------:| | 1.3438 | 0.0036 | 15000 | 1.3198 | | 1.2309 | 0.0072 | 30000 | 1.2102 | | 1.1738 | 0.0108 | 45000 | 1.1595 | | 1.1384 | 0.0144 | 60000 | 1.1267 | | 1.115 | 0.0180 | 75000 | 1.0988 | | 1.0937 | 0.0216 | 90000 | 1.0820 | | 1.0786 | 0.0253 | 105000 | 1.0670 | | 1.0649 | 0.0289 | 120000 | 1.0541 | | 1.0546 | 0.0325 | 135000 | 1.0429 | | 1.0446 | 0.0361 | 150000 | 1.0323 | | 1.0388 | 0.0397 | 165000 | 1.0247 | | 1.029 | 0.0433 | 180000 | 1.0184 | | 1.022 | 0.0469 | 195000 | 1.0127 | | 1.0144 | 0.0505 | 210000 | 1.0072 | | 1.01 | 0.0541 | 225000 | 1.0008 | | 1.0069 | 0.0577 | 240000 | 0.9964 | | 1.0033 | 0.0613 | 255000 | 0.9928 | | 0.9988 | 0.0649 | 270000 | 0.9867 | | 0.9948 | 0.0685 | 285000 | 0.9836 | | 0.9892 | 0.0722 | 300000 | 0.9803 | | 0.9864 | 0.0758 | 315000 | 0.9764 | | 0.9798 | 0.0794 | 330000 | 0.9733 | | 0.9804 | 0.0830 | 345000 | 0.9713 | | 0.9776 | 0.0866 | 360000 | 0.9684 | | 0.9754 | 0.0902 | 375000 | 0.9656 | | 0.9736 | 0.0938 | 390000 | 0.9639 | | 0.9687 | 0.0974 | 405000 | 0.9609 | | 0.9715 | 0.1010 | 420000 | 0.9592 | | 0.9644 | 0.1046 | 435000 | 0.9581 | | 0.9629 | 0.1082 | 450000 | 0.9550 | | 0.9609 | 0.1118 | 465000 | 0.9544 | | 0.9589 | 0.1154 | 480000 | 0.9511 | | 0.9585 | 0.1190 | 495000 | 0.9493 | | 0.9573 | 0.1227 | 510000 | 0.9491 | | 0.9533 | 0.1263 | 525000 | 0.9473 | | 0.9558 | 0.1299 | 540000 | 0.9453 | | 0.9498 | 0.1335 | 555000 | 0.9434 | | 0.9487 | 0.1371 | 570000 | 0.9415 | | 0.9492 | 0.1407 | 585000 | 0.9411 | | 0.9475 | 0.1443 | 600000 | 0.9390 | | 0.9443 | 0.1479 | 615000 | 0.9387 | | 0.9458 | 0.1515 | 630000 | 0.9372 | | 0.9441 | 0.1551 | 645000 | 0.9358 | | 0.9438 | 0.1587 | 660000 | 0.9350 | | 0.9423 | 0.1623 | 675000 | 0.9341 | | 0.9383 | 0.1659 | 690000 | 0.9321 | | 0.9395 | 0.1696 | 705000 | 0.9314 | | 0.9379 | 0.1732 | 720000 | 0.9301 | | 0.9382 | 0.1768 | 735000 | 0.9289 | | 0.9374 | 0.1804 | 750000 | 0.9284 | | 0.9353 | 0.1840 | 765000 | 0.9278 | | 0.9325 | 0.1876 | 780000 | 0.9270 | | 0.932 | 0.1912 | 795000 | 0.9255 | | 0.9289 | 0.1948 | 810000 | 0.9250 | | 0.9324 | 0.1984 | 825000 | 0.9233 | | 0.9303 | 0.2020 | 840000 | 0.9230 | | 0.9297 | 0.2056 | 855000 | 0.9224 | | 0.9279 | 0.2092 | 870000 | 0.9223 | | 0.9298 | 0.2128 | 885000 | 0.9202 | | 0.9252 | 0.2165 | 900000 | 0.9195 | | 0.9259 | 0.2201 | 915000 | 0.9194 | | 0.9255 | 0.2237 | 930000 | 0.9186 | | 0.9231 | 0.2273 | 945000 | 0.9181 | | 0.9258 | 0.2309 | 960000 | 0.9165 | | 0.9214 | 0.2345 | 975000 | 0.9162 | | 0.924 | 0.2381 | 990000 | 0.9161 | | 0.9245 | 0.2417 | 1005000 | 0.9146 | | 0.919 | 0.2453 | 1020000 | 0.9147 | | 0.9202 | 0.2489 | 1035000 | 0.9133 | | 0.92 | 0.2525 | 1050000 | 0.9125 | | 0.9161 | 0.2561 | 1065000 | 0.9128 | | 0.9174 | 0.2597 | 1080000 | 0.9123 | | 0.9176 | 0.2634 | 1095000 | 0.9107 | | 0.9177 | 0.2670 | 1110000 | 0.9097 | | 0.9178 | 0.2706 | 1125000 | 0.9098 | | 0.9144 | 0.2742 | 1140000 | 0.9085 | | 0.916 | 0.2778 | 1155000 | 0.9091 | | 0.913 | 0.2814 | 1170000 | 0.9088 | | 0.9135 | 0.2850 | 1185000 | 0.9075 | | 0.9145 | 0.2886 | 1200000 | 0.9059 | | 0.9113 | 0.2922 | 1215000 | 0.9064 | | 0.9121 | 0.2958 | 1230000 | 0.9060 | | 0.9108 | 0.2994 | 1245000 | 0.9052 | | 0.909 | 0.3030 | 1260000 | 0.9038 | | 0.9127 | 0.3066 | 1275000 | 0.9040 | | 0.9104 | 0.3103 | 1290000 | 0.9029 | | 0.9097 | 0.3139 | 1305000 | 0.9036 | | 0.9086 | 0.3175 | 1320000 | 0.9023 | | 0.9078 | 0.3211 | 1335000 | 0.9016 | | 0.9077 | 0.3247 | 1350000 | 0.9019 | | 0.9058 | 0.3283 | 1365000 | 0.9010 | | 0.9053 | 0.3319 | 1380000 | 0.9005 | | 0.9069 | 0.3355 | 1395000 | 0.8995 | | 0.9057 | 0.3391 | 1410000 | 0.8997 | | 0.9037 | 0.3427 | 1425000 | 0.8988 | | 0.9038 | 0.3463 | 1440000 | 0.8987 | | 0.9029 | 0.3499 | 1455000 | 0.8979 | | 0.9012 | 0.3535 | 1470000 | 0.8975 | | 0.9003 | 0.3571 | 1485000 | 0.8977 | | 0.9019 | 0.3608 | 1500000 | 0.8966 | | 0.9018 | 0.3644 | 1515000 | 0.8964 | | 0.9008 | 0.3680 | 1530000 | 0.8957 | | 0.8997 | 0.3716 | 1545000 | 0.8951 | | 0.8996 | 0.3752 | 1560000 | 0.8950 | | 0.9 | 0.3788 | 1575000 | 0.8941 | | 0.8989 | 0.3824 | 1590000 | 0.8934 | | 0.8962 | 0.3860 | 1605000 | 0.8938 | | 0.8962 | 0.3896 | 1620000 | 0.8936 | | 0.8972 | 0.3932 | 1635000 | 0.8926 | | 0.8985 | 0.3968 | 1650000 | 0.8915 | | 0.8959 | 0.4004 | 1665000 | 0.8919 | | 0.8966 | 0.4040 | 1680000 | 0.8911 | | 0.8952 | 0.4077 | 1695000 | 0.8908 | | 0.8949 | 0.4113 | 1710000 | 0.8897 | | 0.8943 | 0.4149 | 1725000 | 0.8895 | | 0.8921 | 0.4185 | 1740000 | 0.8892 | | 0.8924 | 0.4221 | 1755000 | 0.8887 | | 0.8926 | 0.4257 | 1770000 | 0.8880 | | 0.8902 | 0.4293 | 1785000 | 0.8876 | | 0.8931 | 0.4329 | 1800000 | 0.8871 | | 0.8909 | 0.4365 | 1815000 | 0.8871 | | 0.8904 | 0.4401 | 1830000 | 0.8863 | | 0.8892 | 0.4437 | 1845000 | 0.8861 | | 0.8886 | 0.4473 | 1860000 | 0.8854 | | 0.8894 | 0.4509 | 1875000 | 0.8855 | | 0.8908 | 0.4546 | 1890000 | 0.8852 | | 0.8898 | 0.4582 | 1905000 | 0.8848 | | 0.886 | 0.4618 | 1920000 | 0.8842 | | 0.8855 | 0.4654 | 1935000 | 0.8830 | | 0.8866 | 0.4690 | 1950000 | 0.8825 | | 0.8848 | 0.4726 | 1965000 | 0.8823 | | 0.8835 | 0.4762 | 1980000 | 0.8825 | | 0.8869 | 0.4798 | 1995000 | 0.8814 | | 0.885 | 0.4834 | 2010000 | 0.8814 | | 0.8869 | 0.4870 | 2025000 | 0.8812 | | 0.8842 | 0.4906 | 2040000 | 0.8804 | | 0.8865 | 0.4942 | 2055000 | 0.8804 | | 0.8837 | 0.4978 | 2070000 | 0.8799 | | 0.8799 | 0.5015 | 2085000 | 0.8798 | | 0.8824 | 0.5051 | 2100000 | 0.8784 | | 0.8821 | 0.5087 | 2115000 | 0.8786 | | 0.8802 | 0.5123 | 2130000 | 0.8779 | | 0.8811 | 0.5159 | 2145000 | 0.8778 | | 0.8805 | 0.5195 | 2160000 | 0.8774 | | 0.8837 | 0.5231 | 2175000 | 0.8765 | | 0.88 | 0.5267 | 2190000 | 0.8764 | | 0.8794 | 0.5303 | 2205000 | 0.8760 | | 0.8805 | 0.5339 | 2220000 | 0.8754 | | 0.8768 | 0.5375 | 2235000 | 0.8757 | | 0.8763 | 0.5411 | 2250000 | 0.8743 | | 0.8762 | 0.5447 | 2265000 | 0.8746 | | 0.8786 | 0.5484 | 2280000 | 0.8748 | | 0.8772 | 0.5520 | 2295000 | 0.8735 | | 0.8758 | 0.5556 | 2310000 | 0.8728 | | 0.874 | 0.5592 | 2325000 | 0.8724 | | 0.8752 | 0.5628 | 2340000 | 0.8722 | | 0.8748 | 0.5664 | 2355000 | 0.8712 | | 0.8749 | 0.5700 | 2370000 | 0.8714 | | 0.874 | 0.5736 | 2385000 | 0.8707 | | 0.8728 | 0.5772 | 2400000 | 0.8702 | | 0.8742 | 0.5808 | 2415000 | 0.8701 | | 0.8683 | 0.5844 | 2430000 | 0.8696 | | 0.8693 | 0.5880 | 2445000 | 0.8695 | | 0.8717 | 0.5916 | 2460000 | 0.8684 | | 0.8697 | 0.5952 | 2475000 | 0.8682 | | 0.8734 | 0.5989 | 2490000 | 0.8675 | | 0.87 | 0.6025 | 2505000 | 0.8672 | | 0.8714 | 0.6061 | 2520000 | 0.8669 | | 0.8671 | 0.6097 | 2535000 | 0.8661 | | 0.8681 | 0.6133 | 2550000 | 0.8661 | | 0.8689 | 0.6169 | 2565000 | 0.8658 | | 0.8655 | 0.6205 | 2580000 | 0.8652 | | 0.8667 | 0.6241 | 2595000 | 0.8646 | | 0.8676 | 0.6277 | 2610000 | 0.8647 | | 0.8671 | 0.6313 | 2625000 | 0.8638 | | 0.8665 | 0.6349 | 2640000 | 0.8632 | | 0.865 | 0.6385 | 2655000 | 0.8632 | | 0.8648 | 0.6421 | 2670000 | 0.8627 | | 0.8646 | 0.6458 | 2685000 | 0.8616 | | 0.8621 | 0.6494 | 2700000 | 0.8619 | | 0.8631 | 0.6530 | 2715000 | 0.8621 | | 0.8623 | 0.6566 | 2730000 | 0.8609 | | 0.8623 | 0.6602 | 2745000 | 0.8606 | | 0.8625 | 0.6638 | 2760000 | 0.8602 | | 0.8637 | 0.6674 | 2775000 | 0.8595 | | 0.8625 | 0.6710 | 2790000 | 0.8594 | | 0.8627 | 0.6746 | 2805000 | 0.8587 | | 0.8606 | 0.6782 | 2820000 | 0.8584 | | 0.8607 | 0.6818 | 2835000 | 0.8578 | | 0.8609 | 0.6854 | 2850000 | 0.8575 | | 0.8577 | 0.6890 | 2865000 | 0.8570 | | 0.8567 | 0.6927 | 2880000 | 0.8567 | | 0.8587 | 0.6963 | 2895000 | 0.8560 | | 0.8558 | 0.6999 | 2910000 | 0.8561 | | 0.8539 | 0.7035 | 2925000 | 0.8556 | | 0.8576 | 0.7071 | 2940000 | 0.8553 | | 0.8555 | 0.7107 | 2955000 | 0.8546 | | 0.8558 | 0.7143 | 2970000 | 0.8545 | | 0.8582 | 0.7179 | 2985000 | 0.8539 | | 0.8544 | 0.7215 | 3000000 | 0.8540 | | 0.8541 | 0.7251 | 3015000 | 0.8536 | | 0.8559 | 0.7287 | 3030000 | 0.8531 | | 0.8521 | 0.7323 | 3045000 | 0.8526 | | 0.8529 | 0.7359 | 3060000 | 0.8521 | | 0.8538 | 0.7396 | 3075000 | 0.8515 | | 0.8521 | 0.7432 | 3090000 | 0.8512 | | 0.8512 | 0.7468 | 3105000 | 0.8507 | | 0.8503 | 0.7504 | 3120000 | 0.8503 | | 0.8534 | 0.7540 | 3135000 | 0.8503 | | 0.8534 | 0.7576 | 3150000 | 0.8501 | | 0.8488 | 0.7612 | 3165000 | 0.8494 | | 0.85 | 0.7648 | 3180000 | 0.8490 | | 0.8483 | 0.7684 | 3195000 | 0.8488 | | 0.8481 | 0.7720 | 3210000 | 0.8484 | | 0.8516 | 0.7756 | 3225000 | 0.8480 | | 0.8483 | 0.7792 | 3240000 | 0.8476 | | 0.8504 | 0.7828 | 3255000 | 0.8471 | | 0.8485 | 0.7865 | 3270000 | 0.8469 | | 0.8461 | 0.7901 | 3285000 | 0.8465 | | 0.8458 | 0.7937 | 3300000 | 0.8460 | | 0.8467 | 0.7973 | 3315000 | 0.8460 | | 0.847 | 0.8009 | 3330000 | 0.8454 | | 0.8479 | 0.8045 | 3345000 | 0.8451 | | 0.8441 | 0.8081 | 3360000 | 0.8447 | | 0.8465 | 0.8117 | 3375000 | 0.8444 | | 0.8431 | 0.8153 | 3390000 | 0.8444 | | 0.8453 | 0.8189 | 3405000 | 0.8436 | | 0.8453 | 0.8225 | 3420000 | 0.8436 | | 0.8444 | 0.8261 | 3435000 | 0.8435 | | 0.8425 | 0.8297 | 3450000 | 0.8430 | | 0.8452 | 0.8333 | 3465000 | 0.8429 | | 0.8423 | 0.8370 | 3480000 | 0.8427 | | 0.8423 | 0.8406 | 3495000 | 0.8419 | | 0.8424 | 0.8442 | 3510000 | 0.8416 | | 0.8462 | 0.8478 | 3525000 | 0.8417 | | 0.8414 | 0.8514 | 3540000 | 0.8413 | | 0.8435 | 0.8550 | 3555000 | 0.8415 | | 0.8428 | 0.8586 | 3570000 | 0.8410 | | 0.8396 | 0.8622 | 3585000 | 0.8405 | | 0.8416 | 0.8658 | 3600000 | 0.8403 | | 0.8408 | 0.8694 | 3615000 | 0.8400 | | 0.8392 | 0.8730 | 3630000 | 0.8400 | | 0.8404 | 0.8766 | 3645000 | 0.8396 | | 0.838 | 0.8802 | 3660000 | 0.8396 | | 0.8401 | 0.8839 | 3675000 | 0.8392 | | 0.84 | 0.8875 | 3690000 | 0.8391 | | 0.8401 | 0.8911 | 3705000 | 0.8389 | | 0.841 | 0.8947 | 3720000 | 0.8389 | | 0.8389 | 0.8983 | 3735000 | 0.8386 | | 0.8384 | 0.9019 | 3750000 | 0.8385 | | 0.8375 | 0.9055 | 3765000 | 0.8384 | | 0.8391 | 0.9091 | 3780000 | 0.8381 | | 0.8371 | 0.9127 | 3795000 | 0.8381 | | 0.8387 | 0.9163 | 3810000 | 0.8379 | | 0.8375 | 0.9199 | 3825000 | 0.8377 | | 0.8376 | 0.9235 | 3840000 | 0.8375 | | 0.8377 | 0.9271 | 3855000 | 0.8374 | | 0.8383 | 0.9308 | 3870000 | 0.8373 | | 0.8368 | 0.9344 | 3885000 | 0.8373 | | 0.8385 | 0.9380 | 3900000 | 0.8372 | | 0.8369 | 0.9416 | 3915000 | 0.8372 | | 0.8364 | 0.9452 | 3930000 | 0.8370 | | 0.8381 | 0.9488 | 3945000 | 0.8369 | | 0.8369 | 0.9524 | 3960000 | 0.8368 | | 0.8378 | 0.9560 | 3975000 | 0.8367 | | 0.8341 | 0.9596 | 3990000 | 0.8367 | | 0.8365 | 0.9632 | 4005000 | 0.8367 | | 0.8396 | 0.9668 | 4020000 | 0.8367 | | 0.8345 | 0.9704 | 4035000 | 0.8366 | | 0.8375 | 0.9740 | 4050000 | 0.8366 | | 0.8368 | 0.9777 | 4065000 | 0.8366 | | 0.8372 | 0.9813 | 4080000 | 0.8366 | | 0.8393 | 0.9849 | 4095000 | 0.8365 | | 0.836 | 0.9885 | 4110000 | 0.8365 | | 0.8369 | 0.9921 | 4125000 | 0.8365 | | 0.8349 | 0.9957 | 4140000 | 0.8365 | | 0.8383 | 0.9993 | 4155000 | 0.8365 | ### Framework versions - Transformers 4.40.1 - Pytorch 2.3.0 - Datasets 2.19.1 - Tokenizers 0.19.1