USC-PSI-Lab/Psi0-SIMPLE-decoupled-wbc-Teleop

Psi0 · SIMPLE Decoupled-WBC Teleop (12 tasks)

A multi-task Psi0 policy for the Unitree G1 humanoid with a decoupled whole-body controller (WBC), finetuned on all 12 teleoperated tasks of SIMPLE-decoupled-wbc-Teleop. It is evaluated in the SIMPLE benchmark (MuJoCo physics + Isaac Sim rendering).

Repository layout

This repository is a Psi0 run directory, so it can be passed directly to the Psi0 server as --run-dir:

argv.txt                                  # exact training command-line arguments
run_config.json                           # resolved launch config (model, data transforms, normalization stats)
clip_pooled_cache_simple.pt               # cached CLIP pooled embeddings of the training instructions
checkpoints/ckpt_40000/model.safetensors  # final weights (bf16, ~11 GB), step 40k

Only the inference weights are included. The optimizer state and RNG states are not.

Training

Init	Psi0 pretrained VLM (Qwen3-VL backbone) pre.fast.1by1.2601091803.ckpt.ego200k.he30k + post-pretrained action header postpre.1by1.pad36.2601131206.ckpt.he30k (both in USC-PSI-Lab/psi-model under psi0/)
Data	SIMPLE-decoupled-wbc-Teleop: 12 tasks, 1128 episodes, 465k frames at 50 fps, with 2 episodes per task held out for validation
Action / state	37-D action (padded, target yaw encoded as sin/cos), 36-D proprio state; min/max ("bounds") normalization
Action head	12-block flow-matching DiT (1000 train steps), chunk 30, layer-wise VLM fusion, combined timestep + CLIP-pooled-instruction adaLN, state as action token with null-token dropout (p=0.1), RTC (max delay 8)
Optimization	40k steps, global batch 256 (32 × 8 GPUs, DDP, bf16), AdamW lr 1e-4 (VLM: language 1e-6, vision 1e-5, projector 1e-4), cosine schedule, 2k warmup, wd 0.01, grad clip 1.0
Augmentation	image + view augmentation (random crop, min scale 0.85), 180×320 input, state temporal jitter (±10 frames, p=0.5), Gaussian state noise (σ=0.05)

Reproduce from the Psi0 repo:

bash scripts/train/psi0/finetune-simple-decoupled-wbc.sh teleop12

Tasks

G1Wholebody{BendHandover, BendPickAndPlace, BendPick, CloseDoor, Handover, LocomotionPickBetweenTables, OpenFaucet, OpenOven, OpenTrashCan, PickAndPlaceAndHugContainer, PushOfficeChair, XMovePick}Teleop-v0

Usage

1. Download

hf download USC-PSI-Lab/Psi0-SIMPLE-decoupled-wbc-Teleop \
  --local-dir .runs/finetune/dwbc-teleop12.simple.flow1000.cosine.lr1.0e-04.b256.gpus8.2611091937

2. Serve the policy (Psi0 repo)

serve_psi0_simple --host 0.0.0.0 --port 22085 --policy=psi0 \
  --run-dir=.runs/finetune/dwbc-teleop12.simple.flow1000.cosine.lr1.0e-04.b256.gpus8.2611091937 \
  --ckpt-step=40000 --action-exec-horizon=24 --rtc

Or use bash scripts/deploy/serve_psi0_simple.sh <RUN_DIR> 40000.

3. Evaluate in SIMPLE

# pull the SIMPLE docker image and re-tag
docker pull ghcr.io/physical-superintelligence-lab/simple:260829
docker tag ghcr.io/physical-superintelligence-lab/simple:260829 simple:260829

# one task / one DR level against the server above
DATE=260829 docker compose run --rm eval-decoupled-wbc \
  simple/G1WholebodyCloseDoorTeleop-v0 psi0_decoupled_wbc dr-level-0 \
  --host=localhost --port=22085 --sim-mode=mujoco_isaac --headless \
  --data-format=lerobot --data-dir=data/evals/simple-eval/G1WholebodyCloseDoorTeleop-v0/dr-level-0 \
  --isaac-render=on-query

To run the parallel sweep over all 12 tasks × 4 DR levels, use a SIMPLE checkout with the Psi0 repo at PSI_DIR (it starts one server + client lane per GPU in tmux):

TASK_FILTER='Teleop-v0$' scripts/psi0_eval_sweep/launch.sh \
  .runs/finetune/dwbc-teleop12.simple.flow1000.cosine.lr1.0e-04.b256.gpus8.2611091937 40000

The first launch can take several extra minutes while Isaac Sim builds its caches.