Chess-Pretrain-Models

Pretrained (base) chess models from the pre-to-post study. One subfolder per model, named model_{size}_{pretraining_tokens} (tokens in billions, 2 s.f.). Each subfolder has its own card with a Pass@1 / Pass@16 figure (score vs pretraining tokens). The matching SFT-thinking models live in Chess-SFT-Models.

Loading

Models ship a custom tokenizer (tokenizer.py), so trust_remote_code=True is required. The remote-code loader ignores subfolder=, so snapshot the model folder locally first and load from that path:

from huggingface_hub import snapshot_download
from transformers import AutoModelForCausalLM, AutoTokenizer

name = "model_680m_16B"   # see the table below
path = snapshot_download("pavelslab-nyu/Chess-Pretrain-Models", allow_patterns=f"{name}/*") + f"/{name}"
model = AutoModelForCausalLM.from_pretrained(path, trust_remote_code=True)
tok   = AutoTokenizer.from_pretrained(path, trust_remote_code=True)

Continuing a game

generate.py continues a move sequence:

python generate.py --model model_680m_16B --prompt "1. d4 Nf6 2. c4 g6"

Models & evaluation

pass@k on the multi-turn chess benchmark (test_B0 family), pretraining base.

model	size	pretrain tokens	pass@1	pass@16
model_5m_2.1B	5m	2.1B	6.4%	29.3%
model_5m_6.3B	5m	6.3B	8.0%	33.4%
model_5m_11B	5m	11B	8.1%	32.8%
model_5m_21B	5m	21B	9.0%	35.5%
model_5m_42B	5m	42B	9.3%	35.4%
model_10m_0.92B	10m	0.92B	5.9%	29.0%
model_10m_2.7B	10m	2.7B	8.1%	33.0%
model_10m_4.6B	10m	4.6B	9.1%	34.5%
model_10m_9.2B	10m	9.2B	9.6%	34.7%
model_10m_18B	10m	18B	10.4%	36.1%
model_20m_0.53B	20m	0.53B	5.8%	30.3%
model_20m_1.6B	20m	1.6B	8.0%	33.2%
model_20m_2.6B	20m	2.6B	9.2%	35.5%
model_20m_5.3B	20m	5.3B	10.0%	35.6%
model_20m_11B	20m	11B	11.2%	36.6%
model_20m_16B	20m	16B	12.3%	38.2%
model_20m_32B	20m	32B	12.7%	38.5%
model_20m_53B	20m	53B	13.7%	39.0%
model_32m_0.34B	32m	0.34B	4.5%	25.9%
model_32m_1.0B	32m	1.0B	8.1%	33.2%
model_32m_1.7B	32m	1.7B	9.3%	36.9%
model_32m_3.4B	32m	3.4B	10.8%	36.2%
model_32m_6.9B	32m	6.9B	12.2%	38.1%
model_50m_0.23B	50m	0.23B	3.9%	23.0%
model_50m_0.69B	50m	0.69B	7.5%	32.2%
model_50m_1.1B	50m	1.1B	10.8%	36.8%
model_50m_2.3B	50m	2.3B	11.0%	36.7%
model_50m_4.6B	50m	4.6B	13.4%	39.2%
model_50m_17B	50m	17B	15.1%	40.8%
model_50m_23B	50m	23B	15.2%	39.9%
model_50m_41B	50m	41B	16.1%	41.5%
model_100m_0.53B	100m	0.53B	7.3%	32.8%
model_100m_1.1B	100m	1.1B	10.9%	37.0%
model_100m_2.1B	100m	2.1B	12.2%	38.0%
model_100m_4.3B	100m	4.3B	14.0%	39.9%
model_100m_8.0B	100m	8.0B	15.3%	40.8%
model_100m_11B	100m	11B	16.0%	41.0%
model_100m_15B	100m	15B	16.3%	40.6%
model_100m_20B	100m	20B	17.4%	41.8%
model_100m_40B	100m	40B	17.8%	41.7%
model_200m_0.53B	200m	0.53B	9.7%	36.1%
model_200m_5.3B	200m	5.3B	15.4%	40.6%
model_200m_11B	200m	11B	17.4%	41.7%
model_200m_21B	200m	21B	18.8%	42.7%
model_200m_40B	200m	40B	19.9%	42.9%
model_410m_0.13B	410m	0.13B	1.5%	12.2%
model_410m_0.26B	410m	0.26B	7.7%	33.0%
model_410m_0.53B	410m	0.53B	8.7%	33.5%
model_410m_1.1B	410m	1.1B	12.0%	37.1%
model_410m_2.0B	410m	2.0B	13.6%	39.7%
model_410m_2.6B	410m	2.6B	14.7%	39.5%
model_410m_5.3B	410m	5.3B	17.0%	40.5%
model_410m_11B	410m	11B	18.2%	42.4%
model_410m_20B	410m	20B	19.4%	42.7%
model_410m_26B	410m	26B	20.5%	43.5%
model_680m_0.080B	680m	0.080B	0.0%	0.8%
model_680m_0.16B	680m	0.16B	5.2%	26.9%
model_680m_0.32B	680m	0.32B	6.4%	31.2%
model_680m_0.64B	680m	0.64B	9.8%	35.1%
model_680m_1.2B	680m	1.2B	12.5%	38.5%
model_680m_1.6B	680m	1.6B	13.1%	38.6%
model_680m_3.2B	680m	3.2B	15.5%	40.2%
model_680m_6.4B	680m	6.4B	17.4%	41.2%
model_680m_12B	680m	12B	18.9%	42.5%
model_680m_16B	680m	16B	19.9%	42.5%
model_1000m_0.21B	1000m	0.21B	2.9%	20.7%
model_1000m_0.42B	1000m	0.42B	7.9%	33.6%
model_1000m_0.78B	1000m	0.78B	11.4%	37.6%
model_1000m_1.0B	1000m	1.0B	11.8%	37.7%
model_1000m_2.1B	1000m	2.1B	14.3%	40.0%
model_1000m_4.2B	1000m	4.2B	17.0%	41.7%
model_1000m_7.9B	1000m	7.9B	18.6%	42.9%
model_1000m_11B	1000m	11B	19.3%	43.0%

Citation

If you use the models, please cite it:

@article{pre2post-chess,
  title   = {Understanding Reasoning from Pretraining to Post-Training},
  author  = {Shen, Jingyan and Li, Ang and Rahman, Salman and Sun, Yifan and
             Goldblum, Micah and Telgarsky, Matus and Izmailov, Pavel},
  journal = {arXiv preprint arXiv:2607.16097},
  year    = {2026},
  url     = {https://arxiv.org/pdf/2607.16097}
}