Raifu Wars — RL Action Scorer (Self-Play)
A 57,730-parameter policy that plays a seat in Raifu Wars, a turn-based strategy game, through the Warrior protocol.
Same architecture and same features as RaifuWars-RL-ActionScorer. The only difference is the opponent it trained against: itself, rather than a scripted greedy bot. That one change is what made it generalise across boards.
Why this model exists
Its predecessor won 55.1% overall, and that number was an average across four maps that hid an enormous spread:
| map | cover tiles | predecessor | this model |
|---|---|---|---|
| Crossroads | 0 (0.0%) | 68% | 69% |
| Dustbowl | — | 80% | not evaluated |
| Glacier | — | 59% | not evaluated |
| Arboretum | 172 (37.2%) | 13% | 50% |
| Islands | 321 (33.4%) | not evaluated | 31% |
Crossroads is the only board in the pool with no cover at all. Every policy previously measured on this game — this net's predecessor and three frontier LLMs — wins there by standing still and shooting, because four knockouts is an instant win via the KO tier track and nothing blocks line of sight. Move to a board made of cover and that strategy evaporates:
| policy | Crossroads | Arboretum | Islands |
|---|---|---|---|
| this model | 69% | 50% | 31% |
predecessor (ppo-sim) |
68% | 13% | — |
| gpt-5.6-luna / deepseek-v4-flash / gemini-3.5-flash-lite | 71% | 8% | 0% |
Chance for one seat of four is 25%. The three LLMs went 0 for 48 on Islands.
On Islands this model averages 0.1 kills per match and still wins 31% — it is not shooting anything, it is contesting territory. That is a different strategy from the kill-rush every other policy defaults to, and it is the behaviour the KO-track exploit was masking.
Head-to-head
800 matches in the Hemlock sim, held-out seed, full seat rotation, all four seats drawn from the policies under test:
| policy | win rate | mean return | mean tier |
|---|---|---|---|
| this model | 67.4% | 15.51 | 3.65 |
predecessor (ppo-sim) |
25.1% | 11.15 | 3.20 |
| greedy heuristic | 5.2% | — | 2.69 |
| behaviour cloning | 2.2% | 7.72 | 2.69 |
It gave up nothing in the process: in the single-seat configuration (one learner against three greedy bots) it scores 80.0% against the predecessor's 75.0% on the same seed.
Training
- Initialised from the predecessor, then 7.5 hours of self-play in the Hemlock reimplementation of the game.
- 14.57M agent decisions, 7,112 updates, ~1,500 steps/sec.
- Seven boards, all four seats:
Arboretum, Crossroads, Dustbowl, Glacier, Cornfield, Trench Warfare, Twin Rivers. - PPO, lr 5e-5, 8 envs × 256 steps, batch 256.
Use last.pt, not best.pt. Checkpoint selection is by mean return, and under self-play return
is pinned: four copies of one policy always produce exactly one winner, so the terminal term is
fixed at 10/4 = 2.5 and what remains is noise. The best.pt of this run was written 13 minutes in,
never beaten, and loses to last.pt 26% to 56% head-to-head. This repo ships last.pt.
Architecture
Two towers and an interaction term. The state is embedded once, each candidate action is embedded, and the score is their product — so nothing in the network knows how many actions there are, which is the requirement: the legal set runs from 2 to ~670 between decisions.
state 33 -> 128 -> 64
action 26 -> 128 -> 64
head 192 -> 128 -> 1 softmax over exactly the N offered
value 64 -> 128 -> 1
Scoring rather than classifying makes an illegal action unrepresentable rather than merely penalised, and lets the same weights run on a 17×21 board and a 27×27 one.
Usage
python serve.py raifuwars-actionscorer-selfplay.pt --port 8901
# then point the game at http://127.0.0.1:8901 via the Warrior protocol
Limitations, stated plainly
- Islands is not established as above chance. 5/16 gives a 95% interval of 14–56%, which includes 25% (p=0.37). What is established is that it beats the LLMs' 0/48 there (p=0.0006) and the predecessor on Arboretum (8/16 vs 5/39, p=0.006).
- Per-map real-game evaluation is 16 matches per board — enough to separate 50% from 13%, not enough to rank two policies a few points apart.
- No terrain input. The 33 state and 26 action features contain nothing describing cover, water or line of sight; the policy infers position value indirectly. It reaches 50% on a 37%-cover board regardless, which suggests opponent strength mattered more than perception here — but a terrain feature set is untested at time of writing.
- Pure self-play against the current policy only, with no opponent pool. The arena showed no forgetting at 7.5 hours; that remains the risk over longer runs.
- Only three boards were evaluated in the real game. Dustbowl and Glacier numbers above are the predecessor's and should not be read as this model's.