mindthink/microduck-policies-repro

Microduck policies: independent reproduction

Locomotion policies for Pollen Robotics' Microduck, trained by reproducing the official recipes with microduck_rl at commit bae39f5. All the numbers below come from simulation only. None of these policies has run on hardware.

This is not an official Pollen Robotics release. The training code, the robot description and the teacher checkpoints (pollen-robotics/microduck-training) are theirs.

Files

Folder	What it is	Recommended
velstand-slowwalk-3750/	velstand (walk + stand + get-up in one policy) after the slow-walk fix, iteration 3750	yes
velstand-slowwalk-3998/	same run, last iteration: walks straighter, gets stuck more often after a fall	comparison
velstand-2999/	velstand reproduction before the fix (stands still at 0.15 m/s)	comparison
walk-teacher-3999/	Mjlab-Velocity-Flat-MicroDuck walking policy, 4000 iterations	comparison
patches/velstand-slowwalk.patch	the slow-walk fix, applies to microduck_rl at bae39f5
eval/	the evaluation scripts used for the tables below

Each folder has the ONNX policy, the rsl_rl checkpoint (.pt) and the env.yaml / agent.yaml the run used.

Run a policy in the official CPU simulator (BAM servo model):

uv run scripts/infer_policy.py --new-cmd-obs --walking velstand-slowwalk-3750/model_3750.onnx \
    --scene src/mjlab_microduck/robot/microduck/scene_backlash.xml

How they were trained

• Walk teacher: Mjlab-Velocity-Flat-MicroDuck, 4096 envs, 4000 iterations, official config.
• velstand (2999): Mjlab-VelStand-Rough-Backlash-MicroDuck, 3000 iterations, warm start (MICRODUCK_WARM_START=1) from pollen-robotics/microduck-training/bundles/velocity2/441tzs6d_3750. The standing teacher and the walk anchor are pulled from the same public bundles. The official v8 was trained in five staged runs; this is a single-run simplification.
• Slow-walk fix (3250 to 3998): 1000 more iterations from 2999 with patches/velstand-slowwalk.patch:
  • the walk anchor gets a remapped forward command for small commands (0.10 → 0.22, 0.15 → 0.25 m/s), mirroring the official anchor_yaw_remap for turning;
  • a narrow forward tracking reward track_lin_fine (std 0.12, weight 2.0) is added.

Why the fix was needed: with the stock forward tracking reward (2·exp(−err²/0.1)), standing still at a 0.15 m/s command scores 1.60 and walking at 0.08 m/s scores 1.90. The behaviour-cloning anchor also stands still at 0.15, so the student inherits that start threshold.

Results (simulation, scene_backlash.xml, BAM servo model)

Straight walking: 10 random starts per command, 20 s each. Speed is along the body heading; yaw is the mean drift.

Policy	0.15 m/s command	0.3 m/s: speed / yaw drift
official velstand v8	does not walk in 4 of 10 runs	0.117 m/s / +0.02 rad/s
velstand-2999	does not walk in 9 of 10 runs	0.124 m/s / −0.04 rad/s
velstand-slowwalk-3750	walks in 10 of 10, ~0.07 m/s	0.145 m/s / −0.26 rad/s
velstand-slowwalk-3998	walks in 10 of 10, ~0.07 m/s	0.116 m/s / +0.02 rad/s

Get-up and push battery: 9 families × 40 trials. Families: on the back, on the front and on the side, each with default or random joints; random orientation dropped from 12 cm; and a 0.8–1.2 m/s sideways kick while standing or walking. A trial fails when the trunk is not within 20° of upright at the end of a 10 s window.

Policy	Failures / 360	Get-up time, median
official velstand v8	4	0.62 s
velstand-2999	4	0.54 s
velstand-slowwalk-3750	0	0.74 s
velstand-slowwalk-3998	12 (all still stuck after 20 s)	0.78 s

Known issues

• 3750 drifts right at 0.2–0.3 m/s, by about 0.2 rad/s (±0.01 over 10 starts). In simulation, a counter-command of +0.25 rad/s at 0.3 m/s makes it walk straight with no speed loss. Plan for closed-loop heading control on a real robot.
• No policy walks at a 0.10 m/s command. The cause is not known yet.
• Achieved forward speed is 40–60 % of the command for every policy here, including the official ones.

License

Apache-2.0, the same as microduck_rl. The Microduck 3D meshes in microduck_rl are CC BY-NC-SA 4.0 and are not included here. Credit: Pollen Robotics / microduck_rl.