avewright/chess-transformer-100m-squares64

Chess transformer 100M (squares64 recurrent)

~99M parameter recurrent chess policy/value net. Attention is only over the 64 board squares. Turn, castling, and en passant are FiLM on the square stream, not extra tokens.

This file is latest.pt at FT step 1131 (2026-09-06 15:09 UTC). Recent train loss ~3.3156.

Continued FT of the public 100M squares64 weights on policy-disagreement positions (teacher MultiPV, student greedy ≠ teacher best).

Architecture

Params	98.97M
Hidden / heads	736d / 8
Encoder dim	256
Trunk	prefix 4 + bank 7×3 unrolls + suffix 4
Effective depth	29 (15 unique layer modules)
Attention	64×64 squares only
Embeds	fused piece×color (13)
Side info	FiLM (turn / castling / EP), zero-init
Heads	spatial policy (vocab 1968 compact) + 3-way WDL
Recurrent grads	bank grads divided by 3 after backward

Config is in model_config.json (Squares64RecurrentConfig).

Training data

Soft labels from public Hugging Face packs (depth ≥ 12, phase-balanced open/mid/end):

• avewright/chess-soft-multipv-lichess — 8-wide MultiPV (~91M rows in the full pack)
• avewright/chess-soft-syzygy — tablebase WDL/policy (~498k rows)

This checkpoint (disagreement FT)

• Warm start from the prior public latest.pt (pretrain step 60995).
• Then 1131 Polar-NorMuon steps (30 min, Apple MPS) on avewright/chess-soft-100m-disagreements (~1.75M rows: teacher MultiPV where the 100M greedy argmax disagreed).
• Optimizer: Polar-NorMuon 97.7M + AdamW aux 1.3M. muon_lr=0.002, adam_lr=3e-5, warmup 100, cosine over 1200, batch 48.
• Soft objective unchanged (α=0.55, T=4). No Syzygy mix in this FT. No book / no search at eval.

Pretrain used the MultiPV + Syzygy packs above (see older commits for that recipe).

Inference

Requires this repo (chess_inference.py, chess_squares64.py) and compact vocab 1968.

pip install torch huggingface_hub python-chess
export MOVE_VOCAB_VERSION=compact

import os
os.environ["MOVE_VOCAB_VERSION"] = "compact"

import chess
from huggingface_hub import hf_hub_download
from chess_inference import load_checkpoint, get_model_move

path = hf_hub_download("avewright/chess-transformer-100m-squares64", "latest.pt")
device = "cpu"  # or "cuda"
model = load_checkpoint(path, device=device)

board = chess.Board()
move, info = get_model_move(model, board, device)
print(move, info["top_moves"], info["wdl"])

Local play GUI (policy argmax, no search):

export MOVE_VOCAB_VERSION=compact
python play_factory_gui.py -c latest.pt --policy-only --device cpu -p 8080

Elo (Stockfish UCI_Elo, not FIDE / Lichess)

Search-free greedy policy (legal-masked argmax). No book, no Syzygy, no MCTS. Opponent: Stockfish dev-20260803-762dd1da UCI_LimitStrength / UCI_Elo, movetime=0.05s, ply cap 160. This is not FIDE or Lichess Elo.

Estimate: 1788 UCI_Elo (bracket 1750–1900).

SF UCI_Elo	Score	W-D-L	Ply-cap (counted as draw)
1450	0.719 (32g)	18-10-4	3
1600	0.641 (32g)	14-13-5	4
1750	0.562 (32g)	12-12-8	3
1900	0.312 (32g)	0-20-12	8

bracketed by 1750 (score=0.562) and 1900 (score=0.312)

Files

• latest.pt — full train checkpoint (model_state_dict, config, steps, optimizer)
• model_config.json — architecture
• train.log — step/loss log