witcheer/microduck-walk-stairs

walk-stairs

walk-mixslope with short stairs added, with +-1 deg of gear play in every servo: walk-mixslope v1 plus 3,000 iterations (16,000 to 19,000) on a floor where each 4 m tile is a flight of 2 or 3 steps (risers 10 to 25 mm, 0.25 m treads, half of the tiles), a single step, a hill or flat floor, with foot friction from 0.2 to 1.3 and the training shove doubled to +-0.6 m/s every 3-6 s (our fork Mjlab-Velocity-Flat-Backlash-Stairs-MicroDuck), 4096 envs, 115 min on one RTX 5090. Trained in simulation only. Result: no measurable gain over walk-mixslope, so it does not replace it. On the test picked before training (flights of 2 or 3 steps with 25 mm risers, friction 0.2, doubled shoves, walking straight ahead at 0.3 m/s) it stayed on its feet for 20 s in 3 of 15 takes against 5 of 15 for walk-mixslope (one-sided Fisher p 0.89), though it walked at least 1 m from its start in 14 takes against 9. No better elsewhere: flat floor 13 of 15 against 14 of 15; walk-mixslope's own hill and step floor 6 of 15 against 11 of 15 (two-sided p 0.14, not significant, a drift down). Not yet tested on a real Microduck.

A perpetual policy for the microduck (61-D observation, 14 actions, 50 Hz). Runs until told otherwise — a gait for the walk slot.

Run it on a robot

sudo robotctl policy load walk witcheer/microduck-walk-stairs

The observation normalizer is baked into policy.onnx; feed raw observations. manifest.json follows schema 2 of the microduck policy manifest (docs/policy-manifest.md in the daemon repo).

Training

• repo: pollen-robotics/microduck_rl
• branch: develop
• commit: 53b8971b6
• exported from a checkout with uncommitted changes

Results in simulation (v1, 2026-10-26)

Simulation only. Not yet tested on a real Microduck.

Headline first, on the condition we picked in advance as the test of this run: on floors made only of flights of 2 or 3 steps (25 mm risers, 0.25 m treads), foot friction 0.2 and doubled shoves every 3 to 6 s, walk-stairs stayed on its feet for the whole 20 s in 3 of 15 takes against 5 of 15 for walk-mixslope, the policy it was trained from (one-sided Fisher exact p = 0.89). That is no measurable gain, so by the rule we set before training this run does not replace walk-mixslope. It did get at least 1.0 m from its spawn point (the edge of a 3-step platform, where the first riser is) in 14 takes against 9, so more takes reached the first step, and it still stayed up less often. We publish it anyway, with the numbers as they came out.

On the two other conditions it is no better, and on one it may be worse:

• the maker's flat task, default randomisation and the maker's play shoves: 13 of 15 against 14 of 15 for walk-mixslope (two-sided p = 1.0);
• walk-mixslope's own floor (0.15 hill, 25 mm step, friction 0.2, doubled shoves): 6 of 15 against 11 of 15 (two-sided p = 0.14). Not significant at 15 takes, but the direction is a drop, and by the loss rule we set (significant and lower) it is not called a loss.

Why it exists: a real staircase puts the second riser one stride after the first, which a single step never does. The question was whether walk-mixslope can learn that with 3,000 more iterations of the same conditions.

What changed against Pollen's Mjlab-Velocity-Flat-Backlash-MicroDuck, in our fork Mjlab-Velocity-Flat-Backlash-Stairs-MicroDuck (task file witcheer_velocity_stairs_env_cfg.py, which imports the step, slope, slip and push task files rather than copying them): only the tile mix of the generated floor. Each 4 m tile is drawn as a flight of 2 risers (25% of tiles), a flight of 3 risers (25%), the single step of walk-step (15%), a hill like walk-slope's (15%) or flat floor (20%). A riser is 10 to 25 mm, so a 3-step flight tops out at 30 to 75 mm; treads are 0.25 m, about one stride. Foot friction from 0.2 to 1.3 and the doubled shove (±0.6 m/s every 3 to 6 s) are walk-mixslope's. Rewards, terminations, the gear-play robot, the curricula and the observations are the maker's; the policy has no terrain scan and meets every riser blind. The checkout also carries our local patch that measures height terms from the ground under the feet instead of the tile origin (ground_height_patch.py). Those, the import line that registers the fork in tasks/__init__.py and the Hub card metadata patch are why the manifest says the checkout had uncommitted changes.

Training: warm start from walk-mixslope (model_16000), 3,000 more iterations (16,000 to 19,000), 4096 envs, 115 min on one RTX 5090 (2.31 s per iteration; the heightfield tiles cost about twice as much per iteration as the flat or step floor), one uncut block. Over 100-iteration windows, mean reward went 58.9, 61.4 and 60.7 at 17,000, 18,000 and 19,000 (49.8 at 16,100, the resume ramp), fell_over 2.20, 2.03 and 2.12, and the velocity tracking error 0.405, 0.404 and 0.406: flat over the last 2,000 iterations. These are under the stairs floor, so they do not compare with walk-mixslope's training numbers.

Proof takes with headless_play, 20 s each, standing start, walking command fixed straight ahead at 0.3 m/s, samples every 0.1 s, 15 takes per policy per condition, both policies of a pair under identical settings. A one-env take lands on one random tile, so the stairs condition forces the floor to flights only. Every take spawns on the platform at the centre of its tile, so it starts by walking off the platform and down the flight. A take counts as held when it has no tilt fall and no mid-episode reset. The takes drop mjlab's out-of-terrain-bounds timeout, which on a generated floor otherwise ends a take that starts in an edge tile with the duck still upright. Mid-episode resets over the 15 takes (a take can reset more than once): stairs condition 22 in 12 takes for walk-stairs against 11 in 10 takes for walk-mixslope; flat 2 in 2 against 1 in 1; walk-mixslope's floor 13 in 9 against 5 in 4.

Reading the take logs: trunk height is measured from the spawn point, so on a platform start it goes below zero once the duck steps down. That is the descent, not a fall.

Not tested: flights of more than 3 steps, risers above 25 mm, climbing a flight from the floor as a scored condition (the takes start on top and go down first), friction below 0.2, shoves above ±0.6 m/s, rough ground, any command other than straight ahead at 0.3 m/s, more than 20 s in the scored takes, a second simulator, and a real robot. Not yet tested on a real Microduck.

Run log: the microduck-skill-tree dataset carries the reward curves, queue items, fork files and take logs. Sibling policies: witcheer/microduck-*. walk-flat stays the default walk, walk-mix the mixed-floor walk and walk-mixslope the variant for hills; walk-stairs is published as a result, not a recommendation.

Simulation only.