qleap/Eval_by_NAGISA_V4

Evaluation functions distilled from NAGISA_V4

Hand-crafted shogi evaluations fitted to NAGISA_V4's scores. There are three families, and each family comes in two forms:

folder	family	features	linear	nonlinear
material/	material (MT)	17	Material_Lv1.bin	Material_MLP.bin
king-safety/	king safety (KS)	50	KingSafety_Lv1.bin	KingSafety_MLP.bin
mobility/	mobility (MB)	210	Mobility_Lv1.bin	Mobility_MLP.bin

The families are never mixed. Each model reads only its own features, and no model takes another model's features or output as input. Each family is a complete evaluation on its own.

• Linear (*_Lv1): a weighted sum of the features, fitted by ridge regression.
• Nonlinear (*_MLP): a small multilayer perceptron over the same features, fitted by gradient descent. Not ridge regression.

Every folder also has the record of the fit: weights.json for the linear model and mlp.json for the nonlinear one. Each lists the features column by column, the training data, every setting, and the held-out error.

The data

All six models were trained on the same data with the same target.

qleap/Rescored_dataset_by_NAGISA_V4, revision 8075dbd8537e25980e695afed84350768e171c39: 500,000,000 positions in 256 parquet shards of 1,953,125 rows each.

split	shards	positions	rows used
training	data/teacher-00000.parquet to data/teacher-00254.parquet	498,046,875	455,090,813
held out	data/teacher-00255.parquet	1,953,125	1,784,668

Only two columns are read: packed (the position) and value_q (the score). weight is 1 on every row of this dataset, so it changes nothing. The positions come from another engine's corpus, and each one was searched once by NAGISA_V4 at depth 9 with MultiPV 5; the dataset's card says where they came from.

Target. value_q is the score from the side to move, squashed into (−1, 1). Turn it back into YaneuraOu internal units and onto Black's side:

y = atanh(value_q) × 1512.173 × 0.9      (× −1 when White is to move)

1512.173 is the eval_coef in each shard's footer. 0.9 converts centipawns to internal units, so the engine's own PawnValue is 90.

Rows left out and rows clamped. A row whose value_q is exactly ±1 is dropped, because atanh is undefined there. In float32 that is every mate, every position with no move, and every ordinary score from roughly 15,000 cp up. Of the rows kept, a target beyond ±3000 internal units is set to ±3000 (42,956,062 training rows).

The features

The exact definitions follow. hce_eval.py in this repository computes all three from an SFEN and reproduces the engine; see Reference implementation.

Sides. Every feature is counted from Black's side: a Black piece adds and a White piece subtracts, whichever side is to move.

Attacks. King safety and mobility both count attacks. A piece on the board attacks every square it could move to on an otherwise unchanged board: steps for the stepping pieces, and rays for the lance, bishop, rook, horse and dragon that stop at the first piece in the way. That square is attacked whichever side owns the piece on it. The king attacks too. Pins, checks and whether the move would be legal are not considered, and pieces in hand attack nothing.

Material (17)

column	counts
0–6	pawn, lance, knight, silver, gold, bishop, rook on the board
7	tokin, promoted lance, promoted knight and promoted silver on the board, together
8, 9	horse, dragon on the board
10–16	pawn, lance, knight, silver, gold, bishop, rook in hand, by the number held

King safety (50)

Each column is computed for Black's king, with "own" meaning Black's, and for White's king, with "own" meaning White's, and stored as Black's minus White's. Both kings must be on the board.

"Ring 1" is the squares at Chebyshev distance 1 from the king and "ring 2" those at distance 2, counting only squares on the board (up to 8 and up to 16). The king's own square is in neither.

Two counts are used for a square. Guard kinds is how many different kinds of own piece, the king excluded, attack it; the 14 kinds keep promoted pieces apart, so two golds count once and a gold and a tokin twice. Enemy attackers is how many opponent pieces attack it, the opponent's king included.

column	name	value
0	in_check	1 if the king's square has at least one enemy attacker, else 0
1	ring1_size	squares in ring 1
2	ring2_size	squares in ring 2
3, 16	ringN.own_pawns	own pawns in the ring
4, 17	ringN.own_golds	own golds and gold-moving promotions in the ring
5, 18	ringN.own_silvers	own unpromoted silvers in the ring
6, 19	ringN.enemy_occupied	ring squares holding an opponent piece
7–9, 20–22	ringN.own_guard_types_ge1..3	ring squares with at least 1, 2, 3 guard kinds
10–12, 23–25	ringN.enemy_attack_ge1..3	ring squares with at least 1, 2, 3 enemy attackers
13, 26	ringN.pressured_empty	empty ring squares with an enemy attacker
14, 27	ringN.unpressured_empty	empty ring squares with none
15, 28	ringN.attacked_without_nonking_guard	ring squares, empty or not, with an enemy attacker and no guard kind
29–42	ring1.enemy_attacks_by.KIND	ring 1 squares attacked by at least one opponent piece of that kind
43–49	enemy_hand_times_exposure.KIND	how many of that kind the opponent holds, × exposure

Columns 3–15 are ring 1 and 16–28 ring 2. The 14 kinds of columns 29–42 are, in order, pawn, lance, knight, silver, bishop, rook, gold, king, promoted pawn, promoted lance, promoted knight, promoted silver, horse, dragon. The 7 of columns 43–49 are pawn, lance, knight, silver, gold, bishop, rook. Exposure is the number of ring 1 squares that are empty and have no guard kind, divided by 8 (not by the ring's size, so a king on the edge never reaches 1). The names are also the columns list in king-safety/weights.json.

No position in the training rows has the side to move in check, so in_check is never non-zero there and its weight is 0 in both models.

Mobility (210)

Every attack by a piece on the board, classified by the attacking piece's kind and by what stands on the attacked square. The value of a column is the number of such attacks.

The 10 kinds are, in order, pawn, lance, knight, silver, bishop, rook, gold (with the four gold-moving promotions), king, horse, dragon. Column a × 21 + t, where a is the attacker's kind and t is

t	the attacked square holds
0–9	a piece of the attacker's side, of kind t
10–19	an opponent piece, of kind t − 10
20	nothing

The square, the direction and the distance of the attack are not counted. An attack on an own piece counts as a defence. The column names are in mobility/weights.json.

Linear models

Ridge regression with no intercept, w = (XᵀX + αI)⁻¹ Xᵀy, accumulated as the normal equations in float64 over all training rows. α is chosen from the grid 10^-4 to 10^9, 53 values evenly spaced in the exponent, as the one with the lowest RMSE on the held-out shard. No constraint is put on the weights.

model	α	train RMSE	held-out RMSE	held-out R²
Material_Lv1	100,000	633.9	633.5	0.568
KingSafety_Lv1	0.0001	776.6	776.3	0.352
Mobility_Lv1	31,622.8	799.8	799.9	0.312

RMSE is in internal units. The held-out RMSE of all-zero weights is 964.0, and R² is against that. King safety's α sits at the low edge of the grid: with 455 million rows the fit needs almost no regularisation.

Each *_Lv1.bin is the model's weights as little-endian float32, in the column order above. Material_Lv1.bin is 68 bytes, KingSafety_Lv1.bin 200, Mobility_Lv1.bin 840.

The material weights

From Black's side, in internal units.

piece	on the board	in hand	hand ÷ board
pawn	32.59	32.20	0.99
lance	82.39	126.97	1.54
knight	139.49	138.75	0.99
silver	241.59	267.73	1.11
gold	290.81	361.16	1.24
bishop	354.04	390.77	1.10
rook	393.57	489.40	1.24
tokin, promoted lance, knight and silver	185.88
horse	492.31
dragon	597.12

The weights are not on the engine's scale, and that is deliberate. A pawn comes out at 32.6, not 90. Multiplying all seventeen by 90 / 32.59 keeps every ratio and only stretches the scale, yet over 64 games that version won 1 and lost 63 against these weights as they are (−720 Elo). Use the numbers unscaled.

Nothing holds a piece in hand near its value on the board. The fit was left unconstrained on purpose.

Nonlinear models

One hidden layer of 32 ReLU units:

x  = features / scales
h  = W1 · x                          (32 values)
f  = w2 · relu(h + b1) − w2 · relu(−h + b1)
score = f / 2 × 3000                 (internal units, Black's side)

The output is antisymmetric by construction: swapping the colours of every piece negates every feature and so negates the score exactly. The output layer has no bias. scales is fixed before training, from the training rows, as the square root of each feature's mean square (a feature that is always 0 gets 1); no mean is subtracted.

setting	value
loss	mean squared error of score / 3000 against y / 3000
optimiser	Adam, learning rate 0.001, PyTorch defaults otherwise
batch	8,192 rows
order	shards 0–254 in order, rows in stored order, one pass
initialisation	PyTorch default for the hidden layer, seed 0; output layer 0
regularisation	none
PyTorch	2.8.0, CPU

The settings were not tuned. Each family was trained in its own process from its own features only.

model	parameters	held-out RMSE	linear model's	held-out R²
Material_MLP	608	621.7	633.5	0.584
KingSafety_MLP	1,664	715.5	776.3	0.449
Mobility_MLP	6,784	775.7	799.9	0.353

The *_MLP.bin format

Little-endian. A 32-byte header, then float32 values.

offset	type	content
0	8 bytes	IHCE0001
8	uint32	version, 1
12	uint32	kind: 1 material, 2 king safety, 3 mobility
16	uint32	input width n: 17, 50 or 210
20	uint32	hidden width, 32
24	float32	output scale, 3000
28	uint32	m: 17 for material, 0 otherwise
32	n floats	scales
	32 × n floats	W1, row by row (hidden unit by hidden unit)
	32 floats	b1
	32 floats	w2
	m floats	material only: the 17 Material_Lv1 values

The material file carries the linear piece values so that an engine can give the search's exchange evaluation and pruning margins the same piece values the linear model plays with. They are not part of the nonlinear evaluation.

mlp.json holds the same weights in the PyTorch layout, the normaliser, every training setting, and the row counts.

Measured in games

YaneuraOu, one thread, 500 ms a move, from 79,887 balanced ply-33 openings with colours swapped in pairs. SPRT with elo0 0 and elo1 10. The Elo figures are direct head-to-head differences between the two engines named, not ratings on a common scale.

match	games	result	Elo	verdict
Material_Lv1 vs the earlier V4 material fit	128	102–26	+237.5 ± 68.6	accepted
Material_Lv1 vs itself × 2.76 (pawn = 90)	64	63–1	+719.7 ± 340.4	accepted
KingSafety_Lv1 vs Material_Lv1	64	5–59	−428.8 ± 184.2	rejected
Mobility_Lv1 vs Material_Lv1	128	3–125	−647.9 ± 196.5	rejected
Mobility_Lv1 vs KingSafety_Lv1	128	36–92	−163.0 ± 66.8	no verdict at 128
Material_MLP vs Material_Lv1	128	54–74	−54.7 ± 58.2	no verdict at 128
KingSafety_MLP vs KingSafety_Lv1	128	73–55	+49.2 ± 59.2	no verdict at 128
Mobility_MLP vs Mobility_Lv1	128	44–84	−112.3 ± 55.0	no verdict at 128

Only the material and mobility nonlinear models are clearly no stronger than their linear forms at this length; king safety's nonlinear model is ahead but not yet decided. Each nonlinear model searches 11–16% fewer positions per second than its linear form, and that cost is inside these results. None of this was measured against an external anchor.

For Material_MLP vs Material_Lv1, both sides used one build and the same search piece values; only the static evaluation differed.

Evaluating a position

Each model gives a raw score from Black's side. The engine turned it into the integer it searched with as follows:

1. King safety and mobility: clamp the raw score to ±3000. Material is not clamped there; the linear model is not clamped at all, and the MLP only where the engine's mate scores begin.
2. Truncate towards zero.
3. Negate when White is to move.

The linear models are inner products of the features and the weights. For Material_Lv1 the engine adds each weight once per piece on the board, and a hand of n pieces as float32(n × weight), separately for each side. The MLPs are as in Nonlinear models. The engine computes KingSafety_MLP in float32, and the first layer of Material_MLP and Mobility_MLP in fixed point, W1 / scales rounded to multiples of 2⁻³²; so a raw score computed another way can land on the other side of an integer.

Search piece values

The search's exchange evaluation and pruning margins read three tables of integer piece values. The material models set them from the Material_Lv1 weights, each truncated to an integer first:

table	value for a piece of kind k
PieceValue	k on the board
CapturePieceValue	k on the board + k in hand; for a promoted piece, its own board value + the unpromoted kind in hand
ProDiffPieceValue	the promoted kind on the board − the unpromoted kind on the board

Reference implementation

hce_eval.py is a Python implementation of all three feature families and both kinds of model, written to be read. It needs only the standard library.

python hce_eval.py material/Material_MLP.bin "lnsgkgsnl/1r5b1/ppppppppp/9/9/9/PPPPPPPPP/1B5R1/LNSGKGSNL b - 1"
python hce_eval.py --check examples.jsonl

The first prints the features, the raw score, and the integer score from the side to move. hce_eval.from_packed reads the dataset's packed column instead of an SFEN. load() reads any of the six .bin files.

examples.jsonl holds 100 positions from data/teacher-00000.parquet, each with the features and the six integer scores as the engine produced them, not as hce_eval.py computes them. --check recomputes them all and reports any difference.

Before publishing, hce_eval.py was compared with the tools that made these files. On the first 20,000 kept rows of data/teacher-00000.parquet its features equal, in every column, the features the models were trained on. On the first 5,000 of those positions, its features equal the engine's, and its integer score equals the engine's for all six models.

import struct
material = struct.unpack("<17f", open("material/Material_Lv1.bin", "rb").read())
king_safety = struct.unpack("<50f", open("king-safety/KingSafety_Lv1.bin", "rb").read())
mobility = struct.unpack("<210f", open("mobility/Mobility_Lv1.bin", "rb").read())

The games were played with a fork of YaneuraOu that reads these files. That change is not published. Upstream YaneuraOu uses its own fixed piece values in the search.

The fits were made with tools from a private repository. Everything they do is described above.