Lahusa/lanzenbrecher-state-value-net

Lanzenbrecher State Value Net

State value network for Lanzenbrecher, a simulator and AI agent sandbox for the rules of the board game CATAN. Given a 4-player base game state, it predicts each player's probability of winning, and the current model (htvpfhxw) also a policy over the legal moves. Lanzenbrecher uses it to evaluate leaves and, with the policy, to guide the search (PUCT) in its MCTS agents, and to score moves in its greedy agent.

Files

File	Input	SHA-256
state_value_net-htvpfhxw.onnx (current)	1965 features, policy head	12756846ec18fa4701e1fdfc3ff8c47ad19284cafce24c31eda4c73ead9c0bd9
state_value_net-n0e4owde.onnx	1965 features, policy head	0c965b757c623b450d3e419e7e79f6218c8935a2650e77fdb3af921c3911802d
state_value_net-kt5fwcz5.onnx	1965 features, policy head	1855e8b48ecf6f7f8621ce172bf900ecce72df51aad51e8c3d52fb6526f4991f
state_value_net-mhn8ihqb.onnx	1965 features, policy head	4134fe79db86ff2a63ee11b299d4602aad14c1118d8043247a0e7e84d9ee5244
state_value_net-02l97say.onnx	1965 features	3af3a0e58025e20b8a8653e6884170b489265a06c843d4924dbe7cb48fe61bd4
state_value_net-zjahe050.onnx	1764 features	b40ec4ed3ff53f87c8c54169b16bb57cb8dfd9f4e964eb38e2f63d547b78425c
state_value_net-xnebmrjf.onnx	1764 features	b2ac76e185beeddf0109f0db82c438702fa3c49bcd9a2ae75db3b4f354a08391
state_value_net-sl159q7y.onnx	1764 features	b35230b94965cb8ef67af07acc82e38188ac8c68cee0102f4ca87219052d05bd
state_value_net-j1fvwmta.onnx	1608 features	09f05dda7f1f9ec7423cb0d19b29201fc1d45f8699a741795970f6622ca46752
state_value_net-tvtaun4u.onnx	1609 features	4c8a73d26a1b6b3bd1a3e9aee0d3dd9653140d2b98b50aef222acac0874786f0

File names carry the training run ID. Each file only works with the feature layout it was trained on (see below).

Interface

• Input state: float32 [batch, 1965] for htvpfhxw, n0e4owde, kt5fwcz5, mhn8ihqb and 02l97say, [batch, 1764] for zjahe050, xnebmrjf and sl159q7y (1608 for j1fvwmta, 1609 for tvtaun4u), the unscaled state vector produced by StateVectorizer in Lanzenbrecher. The 1764 layout is the start of the 1965 layout, so those models take the first 1764 features. htvpfhxw, n0e4owde, kt5fwcz5, mhn8ihqb, 02l97say, zjahe050, xnebmrjf and sl159q7y standardize their inputs themselves (the normalization is part of the model).
• Output logits: float32 [batch, 4], one logit per player (by player index, not rotated to the acting player)
• Win probabilities: softmax over the 4 logits
• htvpfhxw, n0e4owde, kt5fwcz5, mhn8ihqb, 02l97say and zjahe050 only, output placements: float32 [batch, 4, 4], per player the logits of finishing 1st to 4th (softmax over the last axis)

import numpy as np
import onnxruntime as ort

session = ort.InferenceSession("state_value_net-02l97say.onnx")
logits = session.run(["logits"], {"state": features.astype(np.float32)})[0]  # features: [batch, 1965]
win_probabilities = np.exp(logits) / np.exp(logits).sum(axis=1, keepdims=True)

In Lanzenbrecher the model is loaded by Agents.Inference.StateValueNet, which applies the softmax.

htvpfhxw, n0e4owde, kt5fwcz5 and mhn8ihqb only, the policy head: a move is encoded as a move type token plus parameter tokens from a shared vocabulary of 213 tokens (Agents/Inference/ActionEncoding.cs in Lanzenbrecher from commit 3f0b807 onward: 19 move types, 54 intersections, 72 edges, 49 tiles, the robbed player relative to the mover, and given, received and discarded resources with the discarded count as weight).

• Output policy_query: float16 [batch, 19 * 64], one query of 64 numbers per move type
• Output policy_embeddings: float32 [213, 64], the token embeddings (the same for every batch)
• Logit of a move: the query of its type times the weighted sum of its token embeddings; the policy is the softmax over the logits of the legal moves. Moves with a random outcome (rolling, buying a development card, the robber with its steal) are one move each.

Feature layout

The features are defined by Agents/StateVectorizer.cs (C#) and DatasetProcessing/state_to_vector.py (Python) in Lanzenbrecher, which produce identical vectors. htvpfhxw, n0e4owde, kt5fwcz5, mhn8ihqb and 02l97say match the 1965 layout from commit 049bf0e onward; zjahe050, xnebmrjf and sl159q7y the 1764 layout from commit 677a9db onward (the start of the 1965 layout), j1fvwmta the 1608 layout from commit 486ecc6 up to that commit, tvtaun4u the 1609 layout from commit 2671919 up to 486ecc6.

Base features (the 1608 layout):

• Settings (2), turn state (16, including only the total of the last roll, 0 before anyone rolled), robber (12)
• 49 map tiles in storage order (type one-hot and normalized number token, 10 each)
• 54 intersections (building and owner one-hot, 8 each), 72 edges (road and owner one-hot, 7 each)
• Resource and development card bank (12)
• 4 player blocks of 35: victory points, played knights, longest road, resource cards, development cards, new development cards, building stock, harbor privileges

Derived features (156, after the base features in the 1764 layout), all per player:

• Production per resource (expected cards per roll, cities count twice) without the robber's tile, the production the robber blocks, and total production
• Trade rate per resource from the player's ports
• Per build (road, settlement, city, development card): whether it can be built now, missing cards, missing cards after trading surplus, rounds until affordable at the current production
• Race margins: victory points, lead over the best other player, points still needed, longest road and knights against the best other player, hand size, over the robber card limit

Port features (180, after the derived features in the 1965 layout): per coastal intersection (fewer than three land tiles, in index order, 30) the one-hot type of the port serving it (generic 3:1, lumber, brick, wool, grain, ore), all 0 without a port.

Endgame features (21, last in the 1965 layout): per player the settlement spots and road spots it could build on now, the victory points it can gain this turn at most with settlements and cities paid from its cards (trading surplus at its rates), whether that wins, and its knight lead counting the knight cards in its hand; then the share of victory point cards among the development cards left in the bank.

The 1609 layout additionally has the round counter as the third feature, and its last roll is a random placeholder during setup and in the first turn before the roll. Counts are unscaled in all layouts.

Training

htvpfhxw

• Generation 33 of a continuous self-play loop that started from n0e4owde (the checkpoint it was exported from), with the same architecture (about 13.4M parameters), so it evaluates positions as fast as n0e4owde
• Self-play: 8,000 games per generation with the newest net, exported to float16 for the collection: MCTS (max^n, PUCT with the net's policy, root noise), 800 iterations for the recorded searches (a quarter of the decisions), 150 otherwise
• Each generation trained on for 2M samples at a constant learning rate of 1e-4, continuing from the previous checkpoint. Training data per generation: the 4 newest generations (40%), the self-play data n0e4owde was fine-tuned on (20%: generations 6-11 and 1-6 of the two earlier loops, equal shares) and the v2 data (40%)
• Value target on the self-play positions: 0.5 times the game's result plus 0.5 times the search's root value; policy target: the root visit shares per legal move, softened with temperature 2. The same auxiliary heads and data augmentation as n0e4owde.
• Every third generation played n0e4owde and the previously evaluated generation at 800 iterations per decision. Generation 33 was the first whose test accepted a gain over n0e4owde (confirmed on new maps, see Playing strength). Later generations were not stronger than it: generations 51, 57, 58 and 60 against generation 33 at 800 iterations scored +5, -3, -8 and +1 Elo (three tests rejected a gain of 20 Elo, the fourth stayed open after 2,000 games), and the precision set (below) stopped improving after generation 30.

Precision set (as below): KL divergence 0.01045 against 0.01074 for n0e4owde (difference -0.0003 +/- 0.0001, 95% interval, clustered by game); best sibling chosen 57.2% against 56.9%; mean regret of the chosen sibling 0.46pp against 0.47pp.

n0e4owde

• Architecture: the same inputs and outputs as the MLP models, with encoders for the board's structure in front of their trunk (Agents/Training/structured_net.py in Lanzenbrecher). Shared small MLPs encode every land tile (with the buildings on its corners) into 32 numbers, every intersection (with the production of its tiles, what the robber blocks there, the buildings next to it and the owners of its roads) into 16, every edge (with the buildings on its ends and the owners of the roads next to it) into 8, and every player block into 128, to which the encodings of the intersections and edges the player owns are added. The trunk (2048 → 2048 → 1024, LayerNorm, GELU, dropout 0.2) takes these encodings with the global features; one head shared by all players gives each player's value logit, placements and victory point share from the trunk's features and that player's encoding. Players stay in absolute seat order as in the MLP models. About 13.4M parameters.
• Pretraining without a policy head (unpublished): the v2 data, split, auxiliary heads, input standardization and data augmentation as for 02l97say, 40 epochs
• Then a policy head (19 x 64 queries, 213 x 64 token embeddings) and fine-tuning on 50% the v2 data and 50% self-play data of two later loops that continued from kt5fwcz5 (generations 6-11 of one: 8,000 games each, 1,600 iterations for the recorded searches; generations 1-6 of the other: 4,000 games each, 3,200 iterations), each generation with an equal share; the same auxiliary heads and data augmentation (the board symmetries also move the move tokens)
• Value target on the self-play positions: 0.5 times the game's result plus 0.5 times the search's root value; policy target: the root visit shares per legal move, softened with temperature 2 (square root, renormalized). The v2 positions have no policy target.
• 16 epochs of 5M samples, AdamW (lr 3e-4, one-cycle schedule); kept the checkpoint with the lowest validation loss of all heads together (epoch 7). The policy ranks the most visited move first in 62.8% of the validation positions.

Precision set (as below): KL divergence 0.0107 against 0.0176 for kt5fwcz5 (difference -0.0068 +/- 0.0004, 95% interval, clustered by game) and 0.0106 for 02l97say; best sibling chosen 56.9% against 55.9% and 61.3%; calibration 5+ VP behind the leader 2.00% / 2.14%.

Speed: in the search the net evaluates about 0.65-0.85 times as many positions per second as kt5fwcz5 on GPUs (CUDA, DirectML) and 0.7-0.8 times on CPUs with up to 8 cores. On the CPU it gains nothing beyond about 8 threads while kt5fwcz5 keeps getting faster, so on a 16-core CPU it reaches only about a third of kt5fwcz5's rate.

kt5fwcz5

• Generation 6 of a continuous self-play loop (8,000 games per generation, each generation trained on for 2M samples at a constant learning rate of 1e-4, continuing from the previous checkpoint). The loop started from an unpublished second self-play iteration, which continued from mhn8ihqb on 40% new self-play data, 30% of mhn8ihqb's self-play data and 30% of the v2 data.
• Self-play: MCTS (max^n, PUCT with the current net's policy, root noise), 1,600 iterations for the recorded searches (a quarter of the decisions), 150 otherwise. The search evaluates a newly reached node itself and expands it only on its next visit (Lanzenbrecher from commit 8a4da3e); the earlier search evaluated one of its moves instead, picked with uniform priors, which made the labels of the previous models noisier.
• Training data per generation: the 4 newest generations (50%), 30% the v2 data (as for 02l97say), 20% the self-play data of the third iteration
• Value target on the self-play positions: 0.8 times the game's result plus 0.2 times the search's root value; policy target: the share of the root visits per legal move. Same auxiliary heads and data augmentation as mhn8ihqb.
• The generations improved in steps and fell back in between: generation 9 lost clearly to generation 6, which ended the loop.

mhn8ihqb

• Continued from the 02l97say checkpoint, with a policy head on its last hidden layer (19 x 64 queries, 213 x 64 token embeddings)
• Self-play data: 95,000 games of MCTS (max^n, 800 iterations for the recorded searches, 150 otherwise, a quarter of the decisions recorded) with 02l97say, 4.2M positions; 3.8M for training, mixed 50/50 with the training data of 02l97say
• Value target on the self-play positions: 0.5 times the game's result plus 0.5 times the search's root value; policy target: the share of the root visits per legal move
• Same auxiliary heads and data augmentation as 02l97say (the board symmetries also move the move tokens); 4 epochs, learning rate 3e-4
• The visit distributions of the value-only search are almost flat (the most visited move has 31% of the visits on average), so the policy learned mild preferences: it ranks the most visited move first in 45% of the positions

Precision set (as below): KL divergence 0.0133 against 0.0106 for 02l97say, best sibling chosen 58.3% against 61.3%, calibration 5+ VP behind the leader 2.03% / 2.14%. The value alone ranks siblings slightly worse on these weak-play labels; the playing strength comes from the policy (see below).

02l97say

• Data as for xnebmrjf (below), migrated to the 1965 layout: the ports come from the maps regenerated from the match seeds (checked against the tiles stored in every sample), the endgame features from the stored features
• Architecture, input standardization, auxiliary heads, optimizer and 40 epochs as for zjahe050
• Data augmentation by exact symmetries of the rules: player rotation (as for zjahe050) and the 12 symmetries of the board (6 rotations, each also mirrored), which move tiles, buildings, roads, ports and the robber together
• Result: best validation KL divergence 0.0147 (epoch 39), against 0.0170 for zjahe050

Precision set (as below):

Metric	02l97say	zjahe050
KL divergence (label noise floor 0.0007)	0.0106	0.0126
KL divergence with the leader at 9+ VP	0.0098	0.0157
Sibling pairs ordered like the labels: all / differing by 1pp / by 3pp	76.3% / 88.4% / 96.2%	75.5% / 87.2% / 95.4%
Best sibling chosen	61.3%	59.8%
Mean regret of the chosen sibling	0.38pp	0.41pp
Predicted / actual win probability 5+ VP behind the leader	2.05% / 2.14%	2.06% / 2.14%

The KL difference is -0.0020 +/- 0.0003 (95% interval, clustered by game). Most of it comes from the port and endgame features (a model with them but without the board symmetries reached 0.0108); the error no longer grows towards the end of the game.

zjahe050

• Data and split as for xnebmrjf (below), same architecture, input standardization, optimizer and 40 epochs
• Auxiliary heads on the shared trunk: placement distribution per player (KL, weight 0.5), victory point shares (weight 0.5) and rounds until the game ends (weight 0.2). Only the placement head is exported.
• Player rotation as data augmentation: every training state after the setup phase is relabeled with a random cyclic rotation of the players (player p becomes p + k), labels rotating along. Cyclic rotations keep the turn order, so the rotated state is the same game situation; setup states are never rotated (the snake order always starts with player 0). Validation is not rotated.
• Result: best validation KL divergence 0.0170 (epoch 39), against 0.0215 for xnebmrjf on the same split

Precision set (as below):

Metric	zjahe050	xnebmrjf
KL divergence (label noise floor 0.0007)	0.0126	0.0170
KL divergence with the leader at 9+ VP	0.0157	0.0243
Sibling pairs ordered like the labels: all / differing by 1pp / by 3pp	75.5% / 87.2% / 95.4%	73.2% / 84.5% / 93.3%
Best sibling chosen	59.8%	55.6%
Mean regret of the chosen sibling	0.41pp	0.52pp
Predicted / actual win probability 5+ VP behind the leader	2.06% / 2.14%	2.04% / 2.14%

The KL difference is -0.0044 +/- 0.0003 (95% interval, clustered by game).

xnebmrjf

• Data: 9,996,128 states from 1,000,000 games between mixed agents (30% random, 35% simple, 35% builder agents, each with 10-20% random moves). Per game 5 sampled states plus one sibling group: up to 6 states after different moves of the acting player at one of the sampled states.
• Labels: the win rates of 300 playouts from each state, played by the same agent mix. Siblings share their playouts' random numbers with their root state (common random numbers).
• Split by game: 70% train, 15% validation, 15% test (games that turned out identical despite different seeds share a split)
• Architecture, input standardization, loss and optimizer as for sl159q7y (below), 40 epochs
• Result: best validation KL divergence 0.0215 (epoch 37). Not comparable with sl159q7y's 0.0318: the labels come from other playout agents (flatter win rates) and fewer playouts (more label noise).

Precision set (6,000 separate games, 2,000 playouts per state, 59,906 states, 5,686 sibling groups):

Metric	xnebmrjf	sl159q7y
KL divergence (label noise floor 0.0007)	0.0170	0.1924
Sibling pairs ordered like the labels: all / differing by 1pp / by 3pp	73.2% / 84.5% / 93.3%	67.7% / 76.9% / 86.6%
Best sibling chosen	55.6%	47.9%
Mean regret of the chosen sibling	0.52pp	0.78pp
Predicted / actual win probability 5+ VP behind the leader	2.04% / 2.14%	0.77% / 2.14%

The KL divergence favors xnebmrjf by construction (the precision set has its label distribution); the sibling ranking compares the two fairly.

sl159q7y, j1fvwmta, tvtaun4u

• Data: 3,000,000 states from 600,000 random games (5 states per game). Each state is labeled with the win rates of 1,000 random playouts from that state. The later models were trained on the same states, migrated to their layout.
• Split by game: 70% train, 15% validation, 15% test
• Architecture: MLP 1764 → 2048 → 2048 → 1024 → 4 with LayerNorm, GELU and dropout 0.2 (about 9.9M parameters); sl159q7y first standardizes the non-binary inputs with statistics from the training set
• Loss: KL divergence between the softmax output and the playout win rates
• Optimizer: AdamW (lr 1e-3, weight decay 1e-4), one-cycle schedule, batch size 2048, bf16 mixed precision, up to 100 epochs
• Result: best validation KL divergence 0.0318 (epoch 92; j1fvwmta 0.0867, tvtaun4u 0.0862), exported to ONNX opset 17

Playing strength

htvpfhxw against n0e4owde, both in float32, every map played twice with the sides swapped, two agents per net in one game, sequential probability ratio test for a gain of at least 20 Elo:

Agents	Result	Elo (95% interval)
MCTS (max^n, PUCT), 800 iterations per decision	262 wins to 195 in 457 games	+48 (+16 to +80), gain accepted
MCTS (max^n, PUCT), 5,000 iterations per decision	638 wins to 562 in 1,200 games (test still open)	+22 (+4 to +40)
MCTS (max^n, PUCT), 50,000 iterations per decision	872 wins to 683 in 1,555 games	+43 (+26 to +59)

At 50,000 iterations the games ran in several parts on different maps; one part (460 games) accepted the gain on its own (+44 Elo, +14 to +75), and the row adds up the map pairs of all parts.

One information set MCTS agent (it sees only what its seat may know) with 5,000 iterations per decision against three agents of other engines, 400 games each; equal strength would win 25%. The n0e4owde runs used the same builds and settings, against Catanatron also the same maps:

Opponents	htvpfhxw	n0e4owde
3 StacSettlers agents, with trading in chat	83.3%	79.0%
3 Catanatron AlphaBeta (depth 2) agents	84.0%	78.5%
3 JSettlers 2.6.10 smart robots, with trading	82.8%	80.8%

On the same Catanatron maps htvpfhxw won 5.5 percentage points more often (95% interval +1.8 to +9.8).

n0e4owde against kt5fwcz5, every map played twice with the sides swapped, two agents per net in one game, sequential probability ratio test for a gain of at least 20 Elo:

Agents	Result	Elo (95% interval)
MCTS (max^n, PUCT), 0.1 s per decision, both nets in float16 on a CUDA GPU (8 CPU cores)	228 wins to 170 on 199 maps	+51 (+18 to +85)

At 5,000 iterations per decision, an unpublished version of the same fine-tuning with 4 epochs beat kt5fwcz5 by +55 Elo (+21 to +90, 221 wins to 162 on 191 maps); n0e4owde played even with it there (-3 Elo, -21 to +15, 613 maps) and stronger at 800 iterations per decision (+22 Elo, +4 to +39, 755 maps).

kt5fwcz5 against the unpublished net it started from (see Training), every map played twice with the sides swapped, two agents per net in one game, maps not used during training, sequential probability ratio test for a gain of at least 20 Elo:

Agents	Result	Elo (95% interval)
MCTS (max^n, PUCT), 5,000 iterations per decision	239 wins to 180 on 208 maps	+47 (+16 to +79)
Information set MCTS (each agent sees only what its seat may know), 5,000 iterations	199 wins to 137 on 168 maps	+65 (+27 to +104)
MCTS (max^n, PUCT), 0.1 s per decision	202 wins to 146 on 174 maps	+56 (+22 to +92)

That net in turn scored +31 Elo (-12 to +74, 102 maps) against mhn8ihqb at 5,000 iterations per decision; kt5fwcz5 was not compared with mhn8ihqb directly.

mhn8ihqb (MCTS max^n with PUCT and its policy) against 02l97say (UCT), 5,000 iterations per decision, two agents per net in one game, every map played twice with the sides swapped: +122 Elo (95% interval +78 to +169) on 104 maps, mhn8ihqb winning both games of a map 42 times and 02l97say 7 times; a repeat on the same maps gave +73 Elo (+33 to +115) on 125 maps. 02l97say with PUCT and uniform priors against 02l97say with UCT was even (+0 Elo, -47 to +47, 106 maps), so the gain comes from the learned policy. With its policy outputs mhn8ihqb searches at about 70% of 02l97say's iterations per second; a comparison at equal thinking time is still open.

02l97say against zjahe050, MCTS (max^n) with 60,000 iterations per decision, two agents per net in one game, every map played twice with the sides swapped: 154 wins to 120 in 274 games on 137 maps (56%, +43 Elo, 95% interval +8 to +80). 02l97say won both games of a map 35 times, zjahe050 18 times (p = 0.014 one-sided); on the other 84 maps the seats decided.

zjahe050 against xnebmrjf, MCTS (max^n) with 60,000 iterations per decision, two agents per net in one game with alternating seats: 50 wins to 36 in 86 games (58%, 95% interval 48-69%, p = 0.08 one-sided). The seats matter more than the nets at this distance: the side in seats 0 and 2 won 62% of the games.

xnebmrjf against sl159q7y, MCTS (max^n opponent model) with 60,000 iterations per decision, two agents per net in one game with alternating seats: 55 wins to 32 in 87 games (63%, 95% interval 53-73%, p = 0.018).

sl159q7y against j1fvwmta, MCTS at 0.5 s per decision:

Comparison	sl159q7y	j1fvwmta
Two agents per net in one game, 40 games	36 wins	4 wins
One MCTS agent against three BuilderAgents, same 30 maps and seats	17 wins (mean 8.9 VP)	8 wins (mean 7.5 VP)

On 9 of the 30 maps against BuilderAgents only sl159q7y won, on none only j1fvwmta.

Limitations

• Perfect information. The features include every player's hand, hidden victory point cards and the exact development card deck (the endgame features use them too). Lanzenbrecher's agents that play under the real game's hidden information evaluate worlds sampled from what their seat knows, not the true state.
• Weak-play labels. Win rates come from playouts between weak agents (02l97say, zjahe050, xnebmrjf, half of mhn8ihqb's and n0e4owde's data, 40% of htvpfhxw's and 30% of kt5fwcz5's: a mix of random, simple and builder agents; the older models: random agents with an older random policy that rarely ended its turn while trades were possible), not from strong play.
• Players far behind. With 300 playouts (02l97say, zjahe050, xnebmrjf) or 1,000 (the older models), win rates below about 0.3% or 0.1% cannot be resolved, and many labels of players far behind are exactly 0. The v2 models are calibrated there on average, but cannot rank such players against each other.
• Base game with 4 players on the classic board only. No player-to-player trading.

License

Copyright (c) 2026 Lasse Huber-Saffer. All rights reserved.

The files in this repository are published for viewing only. No license is granted to use, copy, modify or distribute them without prior written permission.