pashuparthis/dex3-brainco-retargeted-policy

Dex3-1 → BrainCo Revo2 Touch: a retargeted and fine-tuned GR00T N1.7 policy

This is a GR00T N1.7 policy that picks up an apple and puts it on a plate, using a Unitree G1 that wears BrainCo Revo2 Touch hands. The interesting part is that the policy it started from had never seen those hands. It was trained on Dex3-1 hands, which have seven actuators and three fingers per hand; the Revo2 has six actuators and five fingers. Everything here is about closing that gap.

It was done in two stages, and neither involved a human demonstrator:

1. An analytic retargeting layer maps Dex3-1 finger commands onto the Revo2, leaving the policy's 50-D action space and its whole interface untouched. On its own this recovers the task from 0.00 to 0.06.
2. The retargeted policy's own successes were harvested, multiplied with Isaac Lab Mimic into 2,647 demonstrations, and used to fine-tune it. That is these weights.

What it scores

Every row is the same task and success term (galileo_g1_static_pick_and_place, apple to plate) at 50 Hz. The spawn jitter and the timeout are given per row because they are not comparable across values.

Policy	Hands	Spawn, timeout	Success
GN1x base, frozen	Dex3-1	fixed, 6 s	0.65 (13/20)
GN1x base, frozen	Dex3-1	±5 cm, 14 s	0.30 (6/20)
GN1x base + retargeting only	Revo2	fixed, 6 s	0.06 (6/100)
this checkpoint	Revo2	±5 cm, 14 s	0.50 (25/50)
this checkpoint	Revo2	±10 cm, 14 s	0.00

The comparison that matters is rows two and four, because they are the only pair measured under the same protocol: 0.30 on the hands the policy was built for, 0.50 on hands it had never seen. The 0.65 in the first row is the task as shipped and is an easier problem — a fixed apple position and less than half the time limit.

Success here is the environment's own termination term, which also fires if the apple is pushed into the plate region rather than carried there. Every rate quoted above was checked frame by frame before being believed.

How it was trained

Started from nvidia/GN1x-Tuned-Arena-G1-Static-PickNPlace and followed the IsaacLab-Arena recipe: adapt the vision tower, the projector and the flow-matching action head, and leave the language model frozen.

Data	2,647 episodes / 1,304,904 frames, from pashuparthis/mimic_apple_pick_and_place (r2r3)
Steps	10,000 at global batch 240, learning rate 1e-4
Trained	vision tower, projector, VLLN, diffusion action head. Language model frozen
Hardware	3 × RTX PRO 6000 Blackwell
Final loss	0.0103 flow-matching
Embodiment tag	new_embodiment
Action horizon	40 steps (0.8 s at 50 Hz)
Observations	480×640 head camera, 43-D joint state
Actions	50-D, in Dex3-1 coordinates

That last row is worth reading twice. The policy still speaks Dex3-1. It emits the same 50-D action it always did, and the retargeting layer converts finger commands to the Revo2's 41-DoF articulation underneath it. So you need that layer to use these weights on the Revo2 — the checkpoint alone is not enough.

Using it

The weights load through Gr00tPolicy from an Isaac-GR00T checkout that supports N1.7, and are served over ZeroMQ to an IsaacLab-Arena client:

from gr00t.model.policy import Gr00tPolicy

policy = Gr00tPolicy(
    model_path="pashuparthis/dex3-brainco-retargeted-policy",
    embodiment_tag="new_embodiment",
    denoising_steps=4,
)
action = policy.get_action(observation)   # 50-D, Dex3-1 coordinates

The full pipeline — the Revo2 asset, the retargeting layer, the data generation and the evaluation harness — is in dex3_brainco_retargeting. Serving and evaluation live in that repository's 5_gr00t_finetune/.

What it cannot do

This is the honest half, and it is more useful than the headline.

It does not generalise past its training spawn box. 0.50 inside ±5 cm, 0.00 at ±10 cm. Within the box the failures are positional rather than random: success runs 0.78 in the half nearer the plate against 0.41 in the far half, decaying monotonically with distance from the plate.

It never retries a failed grasp. The failure mode is almost always the same — the hand reaches the apple, closes, the apple rolls out during the lift, and the policy carries on to the plate and mimes a place with an empty hand until the clock runs out. Every failure runs the full 14 s while successes finish in 5 to 8.5 s. This is a property of the data, not a quirk of inference: the harvester that built the dataset filtered on success, so all 2,647 training episodes close the hand exactly once and none of them contains a recovery. Shortening the action chunk, temporal ensembling and real-time chunking were all tried on this checkpoint and none produced a single retry.

More data of the same kind does not help. 979 of the 2,647 episodes were generated specifically to target the weak cells of the spawn map, and they bought nothing measurable: 0.55 at step 8000 and 0.50 at step 10000, against the 0.50 the previous fine-tune had already reached. Demonstration coverage across the box was already uniform (165–199 per cell) while success ranged 0.17 to 1.00, so the weak cells were never short of data. What the data lacks is approach diversity — all 2,647 episodes are replays of 22 source trajectories, with one wrist orientation and one closure depth.

The loss will not tell you any of this. It stops being informative after about 3,000 steps. This checkpoint reaches a lower final loss than the earlier, smaller-dataset fine-tune while being no better at the task, and offline action error has no spatial structure at all where real success varies six-fold. Score it by rollout or not at all.

Also in this run

checkpoint-8000 of the same run measured 0.55 (22/40), which is within noise of the 0.50 here — every 95% interval across all six evaluated checkpoints of both fine-tunes overlaps every other. Only the final step is published.