GR00T-N1.6 - Sim2Real Cabinet (task_0019, first half) (joint, 2-cam)

NVIDIA Isaac GR00T-N1.6-3B fine-tuned on the ManiGuard cabinet-task0019-firsthalf base task (sim Franka Panda). Part of the ManiGuard VLA benchmark - GR00T vs pi0.5 on the same task families with identical data, cameras, and controller.

Model

Base: nvidia/GR00T-N1.6-3B - Eagle (nvidia/Eagle-Block2A-2B-v2) VLM + flow-matching DiT action head
Embodiment: NEW_EMBODIMENT - Franka Panda, 8-D joint state/action (7 arm joints + 1 gripper)
Cameras (2): image_left (overview) + wrist (256x256)
Action: arm = state-relative chunks (reconstructed to absolute at inference), gripper = absolute; 16-step horizon at 30 fps (0.53 s); NON_EEF (joint space)
Tuning: GR00T-N1.6 default - VLM (LLM + visual) frozen, train projector + diffusion action head (no LoRA)

Training

single-card config, bf16, global batch 16, 50000 steps (~6 epochs over 126,982 frames), cosine LR (peak 5e-5, sqrt-scaled), warmup 0.05
Data: IDEAS-Lab-Northwestern/sim2real-cabinet-task0019-firsthalf-sim-joint-5cam; videos decoded as H.264 for GR00T's torchcodec loader

Usage

Load with Gr00tPolicy from Isaac-GR00T (n1d6), --embodiment-tag NEW_EMBODIMENT. The included processor/ carries the normalization stats + modality config.

WARNING - Convention (must match at eval): joint-space JointController (absolute joint targets, NON_EEF) + 2 cameras (image_left overview + wrist). A mismatched controller or camera set silently feeds an out-of-distribution input.