jrubiosainz/microduck-game-sprinter-ppo

MICRODUCK GAME sprinter (PPO)

Pollen's sprint-2m reference fine-tuned with PPO on a CPU inside the Arena's own simulator, on the flat, then the hill, the flat, the hill and the flat again: 1.09 s on the official seeds, 200/200 finishes and 0 falls on seeds 1-200. Ignores the command; simulation only.

An entry for the 2 m Sprint of the Microduck Arena. Simulation only — see below.

What it is

A fine-tune of pollen-robotics/microduck-sprint-2m-reference (revision 3e59e802, policy.onnx sha256 adbc1594ccfc…). The network has the reference's own shape (61 → 512 → 256 → 128 → 14, ELU) and keeps its input normalizer frozen; its weights were trained further with PPO on a CPU, inside the Arena's own runner and physics (MuJoCo 3.10.0, runner version 4), on this event's course:

• chain: Pollen's sprint-2m reference -> 375 PPO iterations on sprint-2m (run r1, whose first 15 iterations trained only a fresh critic) -> 3075 PPO iterations on hill-climb-1m5 (run h1, whose first 15 iterations trained only a fresh critic, up to iteration 1049; run h2, branched from h1's iteration-1049 checkpoint, up to iteration 2799; run h4, branched from h2's iteration-2799 checkpoint, whose first 25 iterations trained only the critic) -> 350 PPO iterations on sprint-2m (run s2: the resulting actor with a fresh critic, which alone trained for the first 25 iterations) -> 2250 PPO iterations on hill-climb-1m5 (run h5: the resulting actor with a fresh critic, which alone trained for the first 25 iterations) -> 1900 PPO iterations on sprint-2m (run s4: the resulting actor with a fresh critic, which alone trained for the first 25 iterations); internal forward command 0.55 m/s throughout
• selection: checkpoint it01900, picked among twenty-five checkpoints of two sprint runs (s2 and s4), all run on the 200 seeds and noise rows of the table below: 1.092 s on the official seeds 1–5 in the trainer's check (after it also finished 20 held-out seeds, 1001–1020, with no fall), the lowest official time of the twenty-five, the lowest score on seeds 1–200 and no fall in any row;
• reward: forward progress while the race clock runs, small penalties for sideways speed, yaw rate, drifting toward the walls and falling; in runs h4, s2, h5 and s4 an episode also ends when the duck crosses the line, as the Arena's run does, with a one-off bonus equal to the reward of running at 1.5 m/s for every step left before the 2.5 s horizon; 96 parallel copies of the course in run s4 (192 in run r1, 144 in run h1, 96 in runs h2, h4, s2 and h5), 64 control steps each per iteration, the Arena's own start perturbation;
• command: a fixed internal command (forward speed 0.55 m/s, zero lateral speed, yaw rate, head and body) is written into the 13 command slots of the observation inside the graph, so the commanded twist is ignored.

No GPU, no new data, no paid compute.

Results

local run, Arena runner	score (non-finish = 50 s)	mean of finishers	finished
seeds 1–5 (the official set)	1.092 s	1.092 s (worst 1.12)	5/5
seeds 1-200	1.101 s	1.101 s (p90 1.12, worst 1.16)	200/200, 0 falls
seeds 1–5 with observation noise σ = 1e-04, 4 runs per seed	1.088–1.096 s (range)	—	20/20
seeds 1–5 with observation noise σ = 1e-03, 4 runs per seed	1.088–1.096 s (range)	—	20/20

The noise rows add Gaussian noise to every observation the policy sees: a stand-in for the float-rounding differences of another machine, which a chaotic gait amplifies.

arena score --event sprint-2m on this file (every speed of its sweep gives the same time, since the command is ignored; the sweep then reports the lowest):

sweep 21 commanded speeds 0.20–1.20 m/s
   0.20 m/s  1.08s
   0.25 m/s  1.08s
   0.30 m/s  1.08s
   0.35 m/s  1.08s
   0.40 m/s  1.08s
   0.45 m/s  1.08s
   0.50 m/s  1.08s
   0.55 m/s  1.08s
   0.60 m/s  1.08s
   0.65 m/s  1.08s
   0.70 m/s  1.08s
   0.75 m/s  1.08s
   0.80 m/s  1.08s
   0.85 m/s  1.08s
   0.90 m/s  1.08s
   0.95 m/s  1.08s
   1.00 m/s  1.08s
   1.05 m/s  1.08s
   1.10 m/s  1.08s
   1.15 m/s  1.08s
   1.20 m/s  1.08s

scored at 0.20 m/s
seed 1: finish   1.08s
seed 2: finish   1.10s
seed 3: finish   1.06s
seed 4: finish   1.10s
seed 5: finish   1.12s

score 1.092s · 5/5 finished · replay from seed 2

Local times are indicative: the Arena's machine can differ by a few hundredths of a second.

History

This checkpoint was first staged as a new revision of jrubiosainz/microduck-game-sprinter, the repo of our earlier 2 m Sprint entries. That repo's latest revisions hold a different entry, a remix of other players' policies that carries their LICENSE and NOTICE. This checkpoint owes nothing to those policies, so it is published here instead, in a repo that holds only our own PPO line on Pollen's reference.

Not for a real robot

This policy ignores the commanded twist: it always runs straight ahead at its own pace and never stops by itself. It has only ever run in simulation and was trained without any sim-to-real randomisation. Do not load it on a physical Microduck.

Files

• policy.onnx — one input obs [1, 61], one output actions [1, 14], 50 Hz, raw observations (the normalizer is baked in). Its metadata_props are the base reference's (without run_path) plus microduck_game_* provenance keys.
• manifest.json — schema 2 of the microduck policy manifest, built with Pollen's own microduck_rl publish code.

Credits

Base policy, robot, simulator, Arena and publishing tools: Pollen Robotics. The base repo declares no licence of its own; it is credited here and linked as base_model. PPO fine-tune: the MICRODUCK GAME project (jrubiosainz), built with GitHub Copilot.

Resumen (español)

Participante del Sprint de 2 m de la Microduck Arena, solo para simulación. Es la red de referencia de Pollen Robotics reentrenada con PPO en CPU dentro del simulador del propio Arena. Ignora la velocidad que se le ordena: no debe cargarse en un robot real. Se publica en este repo aparte porque jrubiosainz/microduck-game-sprinter contiene ahora un remix de políticas de otros participantes, con su propia licencia y aviso.