zhicao/Zhi-Policy

Zhi/Policy - run learned policies on the yambox arm + Sharpa hand

Closed-loop rollout of LeRobot pi0.5 checkpoints (first model: lzy001Yuki/SteeringPolicyReal/left_pipette_correct/020000) on the LEFT yambox follower arm and the LEFT Sharpa hand. ../Zhi stays the teleop/record stack; this folder only consumes its interfaces (portal arm server, hand-driver UDP, camera roster in record_episode.py).

Layout

file	what
run.sh	launcher (./run.sh ... = rollout, ./run.sh eval ..., ./run.sh cams)
run_policy.py	live loop: obs -> pi0.5 chunk -> 30 Hz arm / 120 Hz hand commands
cams.py	full-rate latest-frame grabbers (head on the yambox, wrist_left ZED)
offline_eval.py	predicted chunk vs recorded action on a teleop take (no robot)
task_prompt.txt	language prompt fed to the model (must match training)
ckpt -> /data/zhicao/policy_ckpt/SteeringPolicyReal	weights (9.4 GB, NVMe; /home is full)

Python env: /data/zhicao/policy_env (uv, python 3.12, lerobot pinned to commit 71a11efe77f55e61f3ab2ce45b40da8cf626afa9 = the training env from the model repo's PI05_STEERING.md, plus portal, h5py, opencv). uv itself lives in /data/zhicao/uv.

Model I/O (verified against the checkpoint's normalizer stats)

• observation.images.head 480x640 RGB - D435 on the yambox (same mjpeg q4 path as the recorder's head.mp4)
• observation.images.wrist_left 720x1280 RGB - local ZED-M LEFT eye
• observation.state[28] = measured [arm_left q(6), hand_left(22)], rad (= h5 observation/state/arm_left + observation/hand_left)
• action[28] = absolute targets, same layout (= h5 action/arm_left + action/hand_left); chunk 50 @ 30 Hz, QUANTILES normalisation is done by the checkpoint's own pre/post-processors - feed RAW radians.

The stats match ../Zhi/episodes/12 (30 takes, 23.4 k engaged frames).

Before a rollout

1. Panel: yam followers up (:11334), LEFT hand driver up, Retarget tile stopped (two hand-target senders = shaking fingers), no recorder running (the policy needs udp :59250 for the measured hand joints and both cameras).
2. Zhi/tools/check_wrist_cams.sh - wrist_left must be a real UVC ZED (2b03:f682, 5000 Mbps). HID-only f681 = bad USB3 link; the runner refuses.
3. Arm at the demos' start pose (panel "Reset arms"), scene set like training.
4. Hand on the e-stop.

cd ~/Desktop/Zhi/Policy
./run.sh cams                       # 6 s camera check -> /tmp/policy_cam_*.jpg
./run.sh eval ../Zhi/episodes/12/episode_000030     # offline sanity check
./run.sh --dry-run --max-seconds 20 # full loop, nothing sent
./run.sh --max-seconds 120          # real rollout; Ctrl-C = stop + hold

Useful flags: --no-arm / --no-hand, --max-step-deg 4 (arm rate limit per 30 Hz tick), --replan 10, --task "...", --ckpt <pretrained_model dir>.

Each rollout logs to ../Zhi/episodes/block/rollout_<stamp>/ (override with --log-dir) (rollout.h5 with every sent action, every inferred chunk + latency, and *_policy_input.mp4 = the frames the model actually saw).

Safety behaviour

• arms never commanded faster than 30 Hz; per-tick step clamped (--max-step-deg), so the first chunk ramps from the measured pose instead of jumping
• arm targets clipped to the training action range + 0.10 rad
• stale camera / arm / hand data, an inference exception or a chunk that runs out (inference stalled) => stop sending, follower holds
• after any stop the follower is stiff at the last pose - "Reset arms" / "Loose"

Adding another checkpoint

/data/zhicao/policy_ckpt/pdl.sh      # edit U=/SIZE= for the new model.safetensors (8-way ranged curl;
                                     # `hf download` stalls here on the xet path)
./run.sh --ckpt ckpt/<task>/<step>/pretrained_model --task "<prompt>"

Right-hand / bimanual checkpoints need SIDE + the state layout in run_policy.py extended (currently left only).

Panel tile "Block" (2026-09-20)

teleop_panel.py has a compact block policy tile: Start spawns run.sh --status /tmp/zhi_block_policy_status.json --max-seconds 600 (tile shows LOADING MODEL -> waiting -> RUNNING), Pause = SIGUSR1 (nothing sent, follower holds, model stays loaded), Start again = SIGUSR2 resume from a fresh observation, Stop = SIGINT + save to Zhi/episodes/block/rollout_<stamp>/. Log: ~/teleop_logs/block_policy.log. Restart the panel with Zhi/tools/restart_panel.sh.

Hand twitching = chunk-switch jumps (2026-09-20)

Within a chunk the policy is smooth; at every switch (~2 Hz) targets jumped ~0.2 rad (hand) / ~19 deg (arm). Cause: flow-matching sampling variance - the SAME observation gives chunks ~9 deg / 0.2 rad apart (./run.sh var <episode>), the mean is accurate. Mitigation (defaults): ONE fixed initial noise for every inference + 8-tick cross-fade into each new chunk (--random-noise, --blend 0 restore the old behaviour); ./run.sh cons <episode> measures the switch jump offline. Real fix = retrain with RTC (policy.rtc_training_max_delay > measured delay ~18 ticks; this checkpoint has 0) or more data/steps. Prompt check: the long task-1 sentence scores best (2.3 deg) vs left_pipette_correct (3.0) - keep it.

eval_stack_cube.py - minimal eval loop (2026-09-24)

One file, no panel / status files / ssh bring-up. Assumes the two long-lived services are already up (left follower on :11334, left hand driver on :59201 + tee :59250) and does cameras + model in parallel: ready in ~11 s (model built directly on the GPU with torch.device("cuda"): 4 s instead of ~2 min; the yambox D435 first frame is the long pole).

cd ~/Desktop/Zhi/Policy
./eval_stack_cube.py --dry-run --max-seconds 15      # full loop + recording, nothing sent
./eval_stack_cube.py --max-seconds 120               # real run; Ctrl-C = stop + save, follower holds
./eval_stack_cube.py --task "stack the cubes"        # other prompt / --ckpt <pretrained_model dir>

Loop: 30 Hz ticks, 50-step chunk on its own clock (idx = (now - t_obs) * 30), new chunk requested after --replan 10 steps, 8-tick cross-fade, arm rate limit --max-step-deg 4, clip to the training action range, start guard --max-start-err 0.5 rad, stale obs / stalled inference => stop. Inference ~310 ms/chunk. The portal arm client lives in a child process: its socket thread busy-polls and in-process it cost +170 ms per inference.

Output Zhi/episodes/block/eval/episode_NNNN/: head.mp4 + wrist_left.mp4 (one frame per tick = exactly what the model saw), episode.h5 (recorder keys observation/state/arm_left, observation/hand_left, action/arm_left, action/hand_left, timestamps + policy/*: raw chunk target per tick, every inferred chunk with obs/latency), meta.json (prompt, ckpt, outcome asked at the end: s / f / Enter, stats). offline_eval.py reads these directly.

Safety guards in eval_stack_cube.py (the hand is the expensive part)

Every guard stops the loop; the follower then holds stiff and the hand is parked at its MEASURED pose (so it stops pushing). Numbers come from the block demos (episodes/12, 18.8 k engaged frames):

guard	default	demo reference
hand targets clipped to the training action range	+0.05 rad margin	checkpoint action.min/max
hand rate limit (--hand-max-step)	0.12 rad/tick	p99 0.08, p99.9 0.19
fingertip force (--max-finger-force, driver tactile tee :59300)	35 N, 3 packets	max 23 N while grasping
no / stale tactile stream	stop (--no-tactile-guard to run blind)
arm targets clipped to the training range	+0.1 rad	checkpoint action.min/max
arm rate limit (--max-step-deg)	4 deg/tick	max 3.44
start guard (--max-start-err)	0.5 rad
tracking error (--max-track-err, measured vs command, 0.3 s)	0.45 rad	max 0.29
workspace box on the commanded flange, FK (--workspace)	demo box + 6 cm, floor 3 cm below demo min	x 0.09..0.43 y -0.26..0.05 z 0.13..0.33
start pose outside the box	refuse to start
non-finite chunk, stale obs, inference stall, tick overrun > 0.2 s	stop

Recorded per tick in episode.h5: observation/tactile_force_left [N,5] (pinky..thumb) and observation/flange_xyz_left [N,3]. Before a real run check the startup line fingertip |F| [...] reacts when you press a fingertip - if it stays 0 the force guard is blind. The retarget bridge must be stopped (two hand-target senders = shaking fingers); the script refuses otherwise.

One-hand task, two-glove demos: what the policy actually sees (2026-09-24)

The checkpoint's state is 28-D = LEFT arm + LEFT hand only; the right side is not a model input, it is only visible in the head image (right follower parked at the panel reset pose in every demo, within-episode std < 0.4 deg; right hand open). eval_stack_cube.py reads the right follower once and warns when it is > 6 deg from that pose.

What WAS out of distribution: the left hand. The driver homes it to all-zeros, but in the demos the operator wore the glove during the lead-in, so the hand state was always the relaxed pose (thumb ~0.6 rad ...). Cross-test on a live observation: with the all-zero hand the chunk holds a wrong pose ~8 deg off and never starts the reach; swapping only the hand state for the demo pose gives a normal chunk (step 0 within 1.5 deg of the arm, reach starting by step 49). Swapping the images changes little. The script therefore ramps the left hand to HAND_READY (mean lead-in pose of the 30 demos) over 2 s before the first inference and refuses to start if it does not get there; --no-hand-ready disables it. Offline sanity on demo frames (episodes 12/20/28/30): arm MAE 2.3-3.1 deg vs hold-still 6.2-7.3, hand MAE 0.02-0.05 rad, no NaNs, 24 plans at one observation agree within 1 deg.

Run an eval WITHOUT the panel (2026-09-24): four scripts in order

cd ~/Desktop/Zhi/Policy
./01_start_followers.py            # yam followers on the yambox (left :11334 + right :11333); idempotent
./02_start_hand_driver.py          # left Sharpa hand driver (udp :59201, tees :59250/:59300); stops glove feeders; --restart if the eval says "no data from: tactile"
./03_reset_arms.py                 # ARMS MOVE: both arms -> home over 3 s, hands open; asks y/N (-y skips)
./04_eval_stack_cube.py --dry-run --max-seconds 15     # first time: nothing sent
./04_eval_stack_cube.py --max-seconds 120              # real session: many episodes, see keys below
./99_stop_all.py                   # end of session: hands home + release, followers stopped

04 is one long-lived process (model + cameras stay up, ~11 s once): Enter starts an episode after you set the scene, q stops + saves it, opens the hand, brings the arm back to home over 4 s and waits for the next Enter; Esc does the same and quits; Ctrl-C is the emergency stop (nothing more is sent, follower holds, hand parked, saved, no homing - re-home with 03). After each episode it asks s / f / Enter for the success label (--outcome s skips it). --max-seconds ends an episode automatically the same way q does; --episodes N quits after N episodes; without a terminal (stdin not a tty) it runs exactly one episode.

rig.py holds the rig facts they share (yambox address, CAN/port per side, driver paths, log names identical to the panel's so the panel still shows what is running). 01 and 02 do the same things as the panel tiles (motor ping sweeps, minimum_gello.py --mode follower ..., sharpa_hand_driver.py --listen ... --tee ...) and skip what is already up, so re-running them is harmless. --reset-can on 01 cycles the CAN buses when motors do not answer. The panel can stay closed; if it is open it just displays the same processes.

Grasp amplification (2026-09-24, user: the real hand slides off the object with the raw targets)

Two terms applied to the policy's hand target before the rate limiter, 04_eval_stack_cube.py (abduction joints untouched; the gain skips the INDEX finger - user 2026-09-24 - the boost acts on all 15 flexion joints):

• --hand-gain 1.5: sent = OPEN + gain * (policy - OPEN) with OPEN = the demos' open-phase median hand pose - deviations from it are scaled, so closing goes tighter and opening wider than the policy asks.
• --hand-boost 0.5 (cap --hand-boost-max 0.3 rad): sent += boost * (sent - measured) - a finger stopped by an object is pushed further, a finger that reached its target is not.

Amplified flexion targets are clipped to the driver's own limits (not the training range); the fingertip force guard (35 N) and the per-tick rate limit still apply. policy/action[:, 6:] in episode.h5 keeps the raw policy hand, action/hand_left what was sent. --hand-gain 1 --hand-boost 0 restores the plain policy.

Why (23 real episodes of 2026-09-24, all failures): the hand TRACKS its targets - sent minus measured flexion was 0.00-0.02 rad even on the cube, same as in the demos - so "target != actual angle" is not the hand. The policy commands a different closed shape than the demos: at its tightest, thumb 0.90 vs 1.05, ring PIP 1.03 vs 1.32, pinky 0.21 vs 0.68 rad (median over closing frames), while the index closes MORE (0.74 vs 0.40). Fingertip forces stayed < 2 N in most episodes (demos: 15-23 N while grasping). Offline on demo observations the policy predicts the demo grasp exactly, so this is distribution shift during the live approach, not a training gap. The gain around the open pose adds ~0.2-0.3 rad of closing on the under-closed joints.

Tactile stream dying (2026-09-24 23:29): the hand dropped its SDK session (Device ... disconnected in ~/teleop_logs/sharpa_driver.log); joint readback kept coming (UDP broadcast) but the tactile tee froze, so the eval refused to start (no data from: tactile). Zhi/sharpa_hand_driver.py now flags the session for reconnect when a tactile stream that was flowing stops for 10 s (backup: sharpa_hand_driver.py.bak-20260924-tactile-watchdog). Manual fix: ./02_start_hand_driver.py --restart.