You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Private real-robot deployment bundle. Access is granted manually by the owner.

Long-WAM on the Unitree G1

Same two-process layout as the pi0.5 deployment: the policy runs on the workstation GPU (RTX 5090) behind a WebSocket, the robot PC streams observations and applies joint targets.

 robot PC  (eval_longwam_g1.py)  --30 Hz obs (3 JPEG + 16-D state)-->  workstation (policy_server.py)
           <-- 32 x 16 joint targets every replan --                    LongWAMPolicy + accel

Files

file	runs on	purpose
`policy_server.py`	workstation	loads a checkpoint, keeps the 48-step observation history, serves WebSocket
`longwam_policy.py`	workstation	model wrapper: pinzi frame composition (identical to training), normalization, `infer_joint_ar`, acceleration switches
`bench_policy.py`	workstation	latency / memory / parity benchmark of the acceleration modes
`eval_longwam_g1.py`	robot PC	pi0.5-style control loop (unitree_lerobot cameras + arms + Dex1)
`contract.py`	both	the fixed I/O contract (16-D order, camera mapping, 384x320 layout, 48/32 timing)
`msgpack_numpy.py`, `websocket_client.py`	both	OpenPI-compatible wire protocol, no torch on the robot
`make_deploy_bundle.sh`	cluster	packs code + per-task model files into one directory to copy
`bench_g1_policy.sbatch`, `loopback_g1_policy.sbatch`	cluster	the tests that were run on H100

Contract (must match training)

The robot always sends all three cameras and the full 16-D state [left_arm q0..q6, right_arm q0..q6, left_dex1, right_dex1] (absolute joint positions, native units).
Each task defines which of them the model sees (contract.TASK_SPECS):

task	prompt	cameras -> pinzi tiles	action / state
`Dynamic_Cup` (both arms)	`pick up the blue cup and the green cup from the moving conveyor and stack the green cup into the blue cup.`	color_0 top, color_2 bottom-left, color_3 bottom-right	16-D
`Speed_Cup_20/30/40/50` (right arm; conveyor speed in the name)	`pick up the cup.`	color_0 top, color_3 bottom-right, black bottom-left	8-D = right arm 7 + right gripper

Actions are absolute joint targets (no deltas), 32 steps per chunk at 30 Hz. The server returns every chunk in the full 16-D robot order; dims a task does not control hold the current measured value, so the robot loop is the same for every task.
The server owns the 48-step history: send an observation every control step (mode: "observe"), ask for a chunk with mode: "infer", send reset: true at episode start.
Per task you need: config.yaml of the training run, one step_XXXXXX.pt, the task's fastwam_dataset_stats.json, and the text cache directory (1 MB; avoids the 11 GB T5). Start the server with --task <name>; it refuses a checkpoint whose action dim does not match.

1. Build the bundle (cluster)

bash FastWAM/deploy/unitree_g1/make_deploy_bundle.sh /path/to/longwam_g1_bundle \
  Dynamic_Cup step_020000.pt  Speed_Cup_20 step_020000.pt   # any list of <task> <weights>

The bundle also contains the Wan2.2 VAE under models/Wan-AI/Wan2.2-TI2V-5B/ (2.8 GB). Copy the whole bundle to the workstation and bundle/robot/ to the robot PC. The same layout is published at https://huggingface.co/AaronHuangWei/Long-WAM-G1-Dynamic-Task-Deploy (gated; request access).

2. Workstation (RTX 5090)

Requirements: Python 3.10/3.11, PyTorch 2.7.1+cu128 (the training runtime), then pip install -e code/FastWAM websockets msgpack (Pillow decodes JPEG; it comes with torchvision).

cd bundle/code/FastWAM
export DIFFSYNTH_MODEL_BASE_PATH=$PWD/../../models      # contains Wan-AI/Wan2.2-TI2V-5B/Wan2.2_VAE.pth
export PYTHONPATH=src:deploy/unitree_g1
export TORCHINDUCTOR_CACHE_DIR=$HOME/.cache/longwam_inductor    # compiled kernels survive restarts
python deploy/unitree_g1/policy_server.py \
  --run-dir ../../models/Dynamic_Cup \
  --checkpoint ../../models/Dynamic_Cup/step_020000.pt \
  --stats ../../models/Dynamic_Cup/fastwam_dataset_stats.json \
  --text-cache-dir ../../text_embeds_cache \
  --task Dynamic_Cup --accel lossless --port 8000

The server prints serving Long-WAM G1 policy at ws://... after warm-up. First start with a cold Inductor cache compiles for several minutes; afterwards start-up is about one minute.

--accel:

value	what	numerics	H100 p50 (step-2877 ckpt)
`eager`	plain BF16	reference	478 ms
`lossless` (default)	resident RoPE + `torch.compile` of VAE encode, video denoiser, video KV prefill, action denoiser; `--compile-mode max-autotune` = CUDA Graphs	max abs action diff vs eager 0.024 (BF16 noise)	173 ms
`nvfp4`	`lossless` + NVFP4 (W4A4) video expert via FourOverSix; action expert stays BF16, KV cache stays BF16	lossy, opt-in	needs sm_120 (5090 only)

Use --compile-mode max-autotune-no-cudagraphs to keep the compile speed-up without CUDA Graphs. --no-gpu-check allows GPUs other than the 5090 (compute capability 12.0). NVFP4 additionally needs FourOverSix built for sm_120 with the three patches in third_party/patches/; run scripts/build_nvfp4_sm120.sh on the workstation (see docs/nvfp4_environment.md).

Verify before driving the robot (prints p50/p95, peak memory, max-abs action difference vs eager):

python deploy/unitree_g1/bench_policy.py --run-dir ../../models/Dynamic_Cup \
  --checkpoint ../../models/Dynamic_Cup/step_020000.pt \
  --stats ../../models/Dynamic_Cup/fastwam_dataset_stats.json \
  --text-cache-dir ../../text_embeds_cache --task Dynamic_Cup --accel eager lossless nvfp4

3. Robot PC

Requirements: the unitree_lerobot environment (its eval_robot/make_robot.py) plus websockets msgpack numpy opencv-python. No torch.

cd unitree_lerobot
PYTHONPATH=/path/to/bundle/robot:$PYTHONPATH python /path/to/bundle/robot/eval_longwam_g1.py \
  --policy_server_host WORKSTATION_IP --policy_server_port 8000 \
  --arm G1_29 --ee dex1 --control_side both --replan_steps 8 \
  --send_real_robot true --motion true

--control_side right|left|both: which arm(s) follow the returned targets; use right for the Speed_Cup checkpoints (their non-controlled dims already equal the measured state, this is a second guard).
--replan_steps 8: execute 8 of the 32 predicted steps, then request a new chunk (every 0.27 s; keep it above server latency x 30 Hz).
--image_format jpeg (default, ~180 KB/step) or raw (2.7 MB/step).
Safety: a per-step joint jump above --max_arm_target_delta (rad) or --max_gripper_target_delta aborts the loop, as in the pi0.5 script.
The Dex1 access (ee_shared_mem["state"] = [left, right], ee_shared_mem["left"/"right"] targets of length 1) follows unitree_lerobot's gripper interface; check it against the installed make_robot.py before the first run.

What was tested

CPU: contract, protocol pack/unpack, JPEG decode, pinzi composition, config building (SMOKE OK); the ported acceleration unit tests (tests/test_nvfp4_inference_optimization.py, tests/test_rope_real_pair.py) and the pre-existing model tests pass.
H100 (bench_g1_policy.sbatch): eager vs lossless numbers above.
H100 (loopback_g1_policy.sbatch): policy_server.py (--accel lossless) driven by a synthetic JPEG client through the real WebSocket path: 64 steps, 8 chunks, client round-trip p50 184 ms of which server inference 172 ms.
Not tested here: NVFP4 (requires the 5090), the real Unitree hardware loop.

Downloads last month: -; Downloads are not tracked for this model. How to track

Video Preview

Robotics