weikaih/sonic-x2-smpl64

SONIC-X2 — NVIDIA SONIC ported to AgiBot X2 Ultra

Whole-body motion-tracking policy for the AgiBot X2 Ultra humanoid (29 DOF, head-fixed), finetuned from the released NVIDIA G1 SONIC checkpoint. The bet is data diversity: the full BONES-SEED motion corpus retargeted to X2.

This checkpoint

• model_step_011300.pt — training step 11,300 (in progress toward 20k).
• Warm-started from the official G1 SONIC release with a 6-joint X2→G1 permutation fix (waist roll/pitch, wrist yaw/roll) applied to the seed.
• Three encoders trained jointly (SONIC-X2 joint-trajectory / SMPL / VR-teleop), shared FSQ token space, shared decoder.

Metrics

Official-protocol offline eval (128 random clips, terminations on = fail-and-truncate like the official evaluator; root-relative MPJPE over pre-termination frames):

step	MPJPE (official protocol)	Success rate	notes
12.8k	38.3 mm	—
14k	38.3 mm (median 35)	35 % (45/128)

Official SONIC target: MPJPE < 30 mm, success > 0.98, reached at ~100k steps on 128 GPU. This run is at ~14k steps. The gap is dominated by global root-position tracking during locomotion: 78 % of failures are root drift (robot lags the reference velocity and exceeds the 0.25 m anchor threshold). Success rate by motion type at 14k: idle/gesture 75 %, but walk/run/turn ≈ 0 % — pose accuracy (MPJPE 38 mm) is already decent; holding global position across a full locomotion clip is the hardest part and converges last.

MPJPE-trend on the earlier 8-clip probe (biased toward dynamic clips, terminations off): 59.3 → 56.0 → 50.9 → 48.8 → 47.6 → 46.0 → 44.2 mm (1k → 12.7k), monotonic.

Training config

• Corpus: 122,572 clips (61,677 unique + mirrors), GMR-retargeted to X2.
• Physics: per-joint efforts transcribed from the X2 Ultra URDF; KP/KD from the AgiBot SOP §3.1 mimic profile, except wrist KD tuned 2.0→1.0 to match G1's tracking-lag coefficient (deploy must mirror kd=1.0 on the wrists).
• 32–64 GPU multi-node (Beaker), PPO with KL-adaptive LR, adaptive motion sampling, save every 100 steps.

Load

import torch
sd = torch.load("model_step_011300.pt", map_location="cpu")["policy_state_dict"]

config.yaml / meta.yaml carry the full Hydra config needed to instantiate the actor-critic.