zhoumiaosen/microduck-running

MicroDuck Running

A PPO locomotion policy for the 14-actuator MicroDuck biped, trained in MuJoCo Warp through mjlab. The final policy achieves 1.50 m/s mean body-forward speed in simulation at a 2.0 m/s command. The sustained 2.0 m/s goal was not met.

This release includes the normalized ONNX policy, the final training checkpoint, a simulation video, and evaluation measurements. It is a simulation research checkpoint; hardware performance has not been evaluated for this release.

Watch the policy

Download the 10-second replay. The replay uses the final checkpoint, seed 123, one simulated robot, and a 2.0 m/s forward command. A single replay does not establish population-level reliability; see the evaluations below.

Measured results

All results below use a 2.0 m/s forward command with zero commanded lateral and yaw velocity. Each evaluation uses 512 parallel environments and excludes a one-second warm-up from speed measurements.

Metric	Three 10-second evaluations	30-second evaluation
Mean body-forward speed	1.499 m/s	1.499 m/s
Mean straight-progress speed	1.234 m/s	See raw evaluation
Survival	96.74%	91.21%

The short evaluations use seeds 123, 456, and 789; the long evaluation uses seed 456. These are simulation measurements, not hardware results or instantaneous peak speeds. Survival uses the evaluator's 70-degree fall-tilt threshold; it does not establish safe hardware operation.

The policy drifts in heading. Body-forward speed measures motion along the robot's changing forward axis; straight progress measures motion along its initial heading and penalizes failed trajectories. This difference matters when comparing running speed with useful straight-line travel. The evaluation uses VelocityCommandCommandOnly, without an external heading-hold controller.

The acceptance criterion was at least 2.0 m/s mean body-forward speed with at least 95% survival across the short evaluations, plus at least 2.0 m/s and 90% survival in the long evaluation. target_met is false. Raw measurements are in evaluation/, including the aggregate result.

Release contents

File	Purpose
policy.onnx	Inference policy exported with observation normalization baked in
model_10998.pt	Final PPO checkpoint; requires the matching training code
run.mp4	Ten-second simulation replay
evaluation/*.json	Per-seed, long-rollout, and aggregate measurements
SHA256SUMS.txt	Release file integrity hashes
LICENSE, NOTICE, LICENSE-HARDWARE	License and attribution notices

Inference interface

• Control frequency: 50 Hz.
• Actor observation: 61 values, comprising 48 proprioceptive values and a 13-value command block: twist (3), head pose (4), body pose (6).
• Action output: 14 values. These require the matching runtime's action scaling, joint order, and actuator interpretation; they are not direct motor commands.
• The ONNX export includes observation normalization. Do not normalize observations a second time.
• Preserve the matching observation definitions, command padding, joint layout, and actuator model. This policy was trained without action low-pass filtering.

With onnxruntime and NumPy installed, inspect the interface using:

import onnxruntime as ort

session = ort.InferenceSession("policy.onnx", providers=["CPUExecutionProvider"])
print([(x.name, x.shape, x.type) for x in session.get_inputs()])
print([(x.name, x.shape, x.type) for x in session.get_outputs()])
# obs must contain correctly constructed, unnormalized runtime observations.
# actions = session.run(None, {session.get_inputs()[0].name: obs})[0]

The checkpoint is not a standalone robot controller. The matching local training package is mjlab-microduck, task Mjlab-Running-Flat-MicroDuck. A public URL for this refactored code has not yet been supplied. Upstream code provides context but is not asserted to reproduce this release exactly.

Training

This policy was trained locally using PPO/rsl_rl, mjlab 1.3.0, MuJoCo Warp, Python 3.12, PyTorch 2.9.1, and an NVIDIA RTX 3070. The continuation used 512 environments, 24 steps per environment per update, and seed 42.

Training resumed the early model_999.pt policy and performed 10,000 additional PPO updates, producing model_10998.pt. A gradual curriculum raised the maximum forward command to 2.0 m/s; the reward speed cap was 2.2 m/s. The training source commit was 104b1d194b82320bcb4a5df42b6d2b88f2094306 in the local refactored repository. This release selects the final checkpoint, not a best checkpoint found through a full checkpoint sweep.

Limitations

• The achieved mean speed is below the 2.0 m/s training target.
• Heading drift reduces straight-line progress substantially.
• About 8.8% of environments failed the survival criterion over 30 seconds.
• Evaluation covers flat simulated terrain and the specified seeds and commands, not arbitrary terrain, disturbances, or hardware.
• Hardware transfer and matching-runtime validation remain outstanding.

Attribution and licensing

The training code is an independent refactor of Vottivott/microduck-playground, based on source commit 828d950134e29a8d04cbb51720a22c8729047fb7 plus local changes, and derives from pollen-robotics/microduck_rl. Original contributions and copyright remain with their authors, including Hannes von Essen / Vottivott and the MicroDuck contributors.

The software license is Apache-2.0; see LICENSE and NOTICE. Upstream 3D hardware designs have a separate CC BY-NC-SA 4.0 license documented in LICENSE-HARDWARE. No hardware design assets are included in this model release.