trackmania-tmnf-gpt2

This model is a fine-tuned version of gpt2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.1693

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 6.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
1.6369	0.0279	200	1.5620
1.5510	0.0559	400	1.5119
1.4918	0.0838	600	1.4850
1.4874	0.1117	800	1.4480
1.4416	0.1397	1000	1.4274
1.4422	0.1676	1200	1.4114
1.3938	0.1955	1400	1.3911
1.4300	0.2235	1600	1.3820
1.3992	0.2514	1800	1.3702
1.3801	0.2793	2000	1.3655
1.3848	0.3073	2200	1.3538
1.4014	0.3352	2400	1.3492
1.3479	0.3631	2600	1.3457
1.3891	0.3911	2800	1.3392
1.3714	0.4190	3000	1.3300
1.3797	0.4469	3200	1.3279
1.3235	0.4749	3400	1.3253
1.3340	0.5028	3600	1.3166
1.3755	0.5307	3800	1.3107
1.3340	0.5587	4000	1.3069
1.3446	0.5866	4200	1.3065
1.3449	0.6145	4400	1.3000
1.3401	0.6425	4600	1.2957
1.3184	0.6704	4800	1.2941
1.3015	0.6983	5000	1.2910
1.2809	0.7263	5200	1.2852
1.2987	0.7542	5400	1.2822
1.3039	0.7821	5600	1.2822
1.3468	0.8101	5800	1.2784
1.2962	0.8380	6000	1.2749
1.3202	0.8659	6200	1.2730
1.3067	0.8939	6400	1.2689
1.3048	0.9218	6600	1.2662
1.2882	0.9497	6800	1.2660
1.3140	0.9777	7000	1.2631
1.2696	1.0056	7200	1.2622
1.2814	1.0335	7400	1.2598
1.2584	1.0615	7600	1.2576
1.2480	1.0894	7800	1.2558
1.2842	1.1173	8000	1.2523
1.2693	1.1453	8200	1.2498
1.2662	1.1732	8400	1.2502
1.2737	1.2011	8600	1.2464
1.2374	1.2291	8800	1.2464
1.2833	1.2570	9000	1.2437
1.2609	1.2849	9200	1.2436
1.2417	1.3128	9400	1.2405
1.2343	1.3408	9600	1.2407
1.2505	1.3687	9800	1.2382
1.2541	1.3966	10000	1.2350
1.2487	1.4246	10200	1.2345
1.2510	1.4525	10400	1.2336
1.2686	1.4804	10600	1.2322
1.2636	1.5084	10800	1.2306
1.2617	1.5363	11000	1.2285
1.2489	1.5642	11200	1.2287
1.2292	1.5922	11400	1.2269
1.2195	1.6201	11600	1.2255
1.2488	1.6480	11800	1.2248
1.2620	1.6760	12000	1.2245
1.2134	1.7039	12200	1.2234
1.2415	1.7318	12400	1.2209
1.2194	1.7598	12600	1.2227
1.2307	1.7877	12800	1.2182
1.2070	1.8156	13000	1.2184
1.2505	1.8436	13200	1.2172
1.2419	1.8715	13400	1.2178
1.2232	1.8994	13600	1.2150
1.2023	1.9274	13800	1.2147
1.2237	1.9553	14000	1.2127
1.2516	1.9832	14200	1.2127
1.2064	2.0112	14400	1.2140
1.2222	2.0391	14600	1.2114
1.1980	2.0670	14800	1.2096
1.2232	2.0950	15000	1.2099
1.2290	2.1229	15200	1.2100
1.2216	2.1508	15400	1.2081
1.2305	2.1788	15600	1.2066
1.2462	2.2067	15800	1.2053
1.2119	2.2346	16000	1.2072
1.1958	2.2626	16200	1.2062
1.1931	2.2905	16400	1.2047
1.2021	2.3184	16600	1.2043
1.2029	2.3464	16800	1.2029
1.2055	2.3743	17000	1.2035
1.2066	2.4022	17200	1.2039
1.1887	2.4302	17400	1.2033
1.2122	2.4581	17600	1.2011
1.2161	2.4860	17800	1.2001
1.2192	2.5140	18000	1.2003
1.2058	2.5419	18200	1.1991
1.2122	2.5698	18400	1.1976
1.2042	2.5978	18600	1.1997
1.2265	2.6257	18800	1.1970
1.2115	2.6536	19000	1.1976
1.2106	2.6816	19200	1.1958
1.2165	2.7095	19400	1.1961
1.2090	2.7374	19600	1.1944
1.1885	2.7654	19800	1.1938
1.2284	2.7933	20000	1.1943
1.2166	2.8212	20200	1.1926
1.2289	2.8492	20400	1.1927
1.1835	2.8771	20600	1.1916
1.2019	2.9050	20800	1.1916
1.2176	2.9330	21000	1.1915
1.1975	2.9609	21200	1.1905
1.1929	2.9888	21400	1.1913
1.1747	3.0168	21600	1.1918
1.2288	3.0447	21800	1.1886
1.1624	3.0726	22000	1.1898
1.1773	3.1006	22200	1.1896
1.2180	3.1285	22400	1.1877
1.1921	3.1564	22600	1.1882
1.1790	3.1844	22800	1.1877
1.1831	3.2123	23000	1.1909
1.1689	3.2402	23200	1.1896
1.1735	3.2682	23400	1.1875
1.1962	3.2961	23600	1.1866
1.1722	3.3240	23800	1.1863
1.1681	3.3520	24000	1.1854
1.1926	3.3799	24200	1.1831
1.1693	3.4078	24400	1.1849
1.1763	3.4358	24600	1.1834
1.1849	3.4637	24800	1.1831
1.1756	3.4916	25000	1.1839
1.1744	3.5196	25200	1.1835
1.1610	3.5475	25400	1.1822
1.1895	3.5754	25600	1.1822
1.1972	3.6034	25800	1.1826
1.1948	3.6313	26000	1.1808
1.1824	3.6592	26200	1.1805
1.2117	3.6872	26400	1.1802
1.1824	3.7151	26600	1.1806
1.1714	3.7430	26800	1.1802
1.1649	3.7709	27000	1.1799
1.1458	3.7989	27200	1.1793
1.1335	3.8268	27400	1.1795
1.1881	3.8547	27600	1.1787
1.1847	3.8827	27800	1.1784
1.1687	3.9106	28000	1.1779
1.1642	3.9385	28200	1.1792
1.1785	3.9665	28400	1.1774
1.1597	3.9944	28600	1.1789
1.2012	4.0223	28800	1.1781
1.1859	4.0503	29000	1.1781
1.2014	4.0782	29200	1.1774
1.1860	4.1061	29400	1.1770
1.1560	4.1341	29600	1.1767
1.1744	4.1620	29800	1.1763
1.1587	4.1899	30000	1.1765
1.1758	4.2179	30200	1.1778
1.1769	4.2458	30400	1.1760
1.1794	4.2737	30600	1.1767
1.1618	4.3017	30800	1.1756
1.1566	4.3296	31000	1.1769
1.1461	4.3575	31200	1.1761
1.1792	4.3855	31400	1.1753
1.1759	4.4134	31600	1.1734
1.1483	4.4413	31800	1.1747
1.2146	4.4693	32000	1.1743
1.1355	4.4972	32200	1.1749
1.1682	4.5251	32400	1.1735
1.1591	4.5531	32600	1.1737
1.1922	4.5810	32800	1.1729
1.1507	4.6089	33000	1.1724
1.1652	4.6369	33200	1.1744
1.1574	4.6648	33400	1.1730
1.1572	4.6927	33600	1.1723
1.1629	4.7207	33800	1.1717
1.1767	4.7486	34000	1.1717
1.1588	4.7765	34200	1.1727
1.1664	4.8045	34400	1.1713
1.1848	4.8324	34600	1.1714
1.1692	4.8603	34800	1.1715
1.1566	4.8883	35000	1.1715
1.1314	4.9162	35200	1.1713
1.1817	4.9441	35400	1.1714
1.1570	4.9721	35600	1.1713
1.1442	5.0	35800	1.1703
1.1454	5.0279	36000	1.1720
1.1830	5.0559	36200	1.1714
1.1713	5.0838	36400	1.1712
1.1788	5.1117	36600	1.1707
1.1463	5.1397	36800	1.1718
1.1553	5.1676	37000	1.1708
1.1635	5.1955	37200	1.1708
1.1593	5.2235	37400	1.1702
1.1525	5.2514	37600	1.1710
1.1351	5.2793	37800	1.1708
1.1545	5.3073	38000	1.1698
1.1688	5.3352	38200	1.1700
1.1738	5.3631	38400	1.1702
1.1702	5.3911	38600	1.1700
1.1495	5.4190	38800	1.1688
1.1391	5.4469	39000	1.1695
1.1812	5.4749	39200	1.1691
1.1363	5.5028	39400	1.1698
1.1420	5.5307	39600	1.1695
1.1360	5.5587	39800	1.1694
1.1755	5.5866	40000	1.1688
1.1438	5.6145	40200	1.1691
1.1406	5.6425	40400	1.1693

Framework versions

• Transformers 5.14.1
• Pytorch 2.13.0+cu130
• Datasets 5.0.1
• Tokenizers 0.22.2