MHBench decentralized baselines -- GR00T N1.7 and pi0.5
The two shipped MHBench decentralized policies, each one set of weights over
every task and both robot roles. Only what evaluation loads is here: GR00T's
merged standalone model dirs, and pi0.5's params/ and assets/. The
optimiser state that would let you resume training is not (it is ~2.4x the
size and eval never opens it).
GR00T_N17/checkpoints/mhbench-multitask-unitree_g1x2_decentralized-joint-0/merged-<step>Pi_05/checkpoints/mhbench-multitask-unitree_g1x2_decentralized-joint-0/<step>/
Layout
Paths mirror a checkout, so a download drops straight into place:
GR00T_N17/checkpoints/mhbench-multitask-unitree_g1x2_decentralized-joint-0/
merged-10000/ merged-20000/ merged-30000/ merged-40000/
Pi_05/checkpoints/mhbench-multitask-unitree_g1x2_decentralized-joint-0/
10000/ 20000/ 30000/ 39999/ # each: params/ assets/
python baselines/scripts/mhbench_release.py download checkpoints --namespace Jebeom
lands them under baselines/XPolicyLab/policy/. GR00T additionally needs
nvidia/Cosmos-Reason2-2B (~4.6 GB), which transformers fetches on first load -- the
merged weights carry the backbone but not its tokenizer or processor.
Evaluating
EVAL_CKPT_NUM=40000 baselines/scripts/eval_launch.sh GR00T_N17 <task>
The GR00T weights are published under the untagged run name, which is the one
the checkpoint resolver builds by default -- no CKPT_TAG needed. Upstream
they were trained as mhbench-multitask-unitree_g1x2_decentralized-joint-0-pi05match,
and that is the name they carry in wandb and in this benchmark's
eval_results/ (GR00T_N17-decentralized-pi05match-seed0-ckpt<step>).
Success rate over 50 episodes per cell, from this benchmark's own
eval_results/:
| run | cocarry | door_passage | frame_hang | handover |
|---|---|---|---|---|
GR00T_N17-decentralized-pi05match-seed0-ckpt10000 |
0.20 | 0.06 | 0.10 | 0.16 |
GR00T_N17-decentralized-pi05match-seed0-ckpt20000 |
0.18 | 0.00 | 0.16 | 0.14 |
GR00T_N17-decentralized-pi05match-seed0-ckpt30000 |
0.24 | 0.10 | 0.08 | 0.10 |
GR00T_N17-decentralized-pi05match-seed0-ckpt40000 |
0.16 | 0.02 | 0.24 | 0.14 |
Pi_05-decentralized-seed0-ckpt10000 |
0.56 | 0.10 | 0.06 | 0.02 |
Pi_05-decentralized-seed0-ckpt20000 |
0.56 | 0.14 | 0.30 | 0.04 |
Pi_05-decentralized-seed0-ckpt30000 |
0.50 | 0.16 | 0.26 | 0.00 |
Pi_05-decentralized-seed0-ckpt40000 |
0.56 | 0.12 | 0.34 | 0.00 |
Training
GR00T is a LoRA finetune of nvidia/GR00T-N1.7-3B (alpha 32, dropout 0,
adapters on the LLM's attention and MLP projections, the visual backbone and
the action head), merged into standalone dirs by
baselines/scripts/merge_lora_checkpoint.py. pi0.5 is an openpi finetune of
pi05_base under the pi05_mhbench_multitask_decentralized TrainConfig. Both
train on the flattened all-task dataset, where one row is one robot: 43 joint
angles, one ego camera, and that agent's own instruction.
License
These weights are derivatives of nvidia/GR00T-N1.7-3B and
nvidia/Cosmos-Reason2-2B (GR00T, merged) and of openpi's pi05_base (pi0.5), and
carry those models' terms. Check them before redistributing.