SSH111111/microduck-tug

tug

Mjlab-TugOfWar-MicroDuck

A perpetual policy for the microduck (61-D observation, 14 actions, 50 Hz). Runs until told otherwise — a gait, loaded into a policy slot.

Run it on a robot

sudo robotctl policy load <slot> SSH111111/microduck-tug

The observation normalizer is baked into policy.onnx; feed raw observations. manifest.json follows schema 2 of the microduck policy manifest (docs/policy-manifest.md in the daemon repo).

Training

• task_id: Mjlab-TugOfWar-MicroDuck
• repo: https://github.com/pollen-robotics/microduck-challenges.git
• branch: tug-league
• commit: 2473c9d84
• checkpoint: 400
• seed: 1
• base: mjlab-microduck 0.1.0 @ 1efd853be
• started: 2026-11-06T02:22:42Z
• exported from a checkout with uncommitted changes

Reproduce

Same code, same uv.lock, same command, same seed. Training it again yields a comparable policy, not the same weights: GPU reinforcement learning is not bit-reproducible across machines.

git clone https://github.com/pollen-robotics/microduck-challenges.git
cd microduck-challenges
git checkout 2473c9d84
uv sync
uv run train Mjlab-TugOfWar-MicroDuck --env.scene.num-envs 4096 --agent.max-iterations 600 --agent.save-interval 100 --agent.seed 1 --agent.logger tensorboard --agent.run-name leagueN3

How it was trained (and what it was not)

• One trained network, nothing else. policy.onnx is the exported actor MLP (61 -> 512 -> 256 -> 128 -> 14, ELU, normalizer baked in). No observation lookup tables, no seed-specific action replays, no hand-written rules.
• Built up in stages, each warm-started from our own previous stage. It starts from the challenge's default warm start (the reference duck).
  1. Reward shaping, against the reference duck alone. Velocity tracking made one-sided (no penalty for pulling faster than commanded), a larger marker_ground term for pulling the marker home, and that pull paid only while the trunk is within 45 deg of upright.
  2. Sparring. The opponent was drawn per episode from the reference duck (0.7), our own stage-1 policy (0.1) and the network of one other public Arena entry (0.2).
  3. Sparring, wider. The opponent was drawn from the reference duck (0.6), our own stage-2 policy (0.1) and the networks of two other public Arena entries (0.2 and 0.1). This policy is checkpoint 400 of this stage.
• Other players' policies were used as fixed sparring and evaluation opponents only. None of their weights were copied into, distilled into, or used to initialise this policy. Other public entries were used only to evaluate checkpoints.
• "Uncommitted changes" above refers to an untracked MUJOCO_LOG.TXT in the launching checkout; the training code itself ran from a clean worktree at commit 2473c9d84.
• Reproducing needs the earlier stages' checkpoints (the warm start) and the opponent networks, which are kept out of git, so the command above alone does not rebuild this policy.