twanghcmut/GR00T-N1.7-ALOHA-RightArm-Multitask

GR00T N1.7 — ALOHA Right Arm, Multi-Task

GR00T N1.7 3B finetuned on two ALOHA right-arm teleoperation datasets in a single multi-task run. The two tasks are separated at inference time by the language instruction, not by separate weights.

Tasks

Task	Instruction	Episodes
Drawer	Use the right gripper to open the top drawer, pick up the red square block, place it inside the top drawer, and then close the drawer.	150
Kitchen	Use the right gripper to pick up the banana and place it into the pot. Then pick up the lid with the right gripper and place it on top of the pot to close it.	150

Total: 300 episodes / 79,857 frames @ 15 fps. The two datasets are mixed proportionally to their size (≈47% drawer / 53% kitchen).

Embodiment / modality

Single 6-DoF arm + 1-DoF gripper, controlled in joint space, registered under EmbodimentTag.NEW_EMBODIMENT.

• video: front (observation.images.color.high), wrist (observation.images.color.wrist_right), 480x640
• state: single_arm (6), gripper (1) — raw observation.state[0:7]
• action: single_arm (relative, non-EEF), gripper (absolute, non-EEF), 16-step chunk
• language: annotation.human.task_description

The source datasets carry a 13-dim state/action vector: [0:6] arm joints, [6:7] gripper, [7:13] an end-effector pose block. This model uses the joint block only. The EEF block was deliberately excluded because its roll/yaw wrap at ±π (~0.4-1% of frames jump by ~2π between consecutive samples), which the default min/max action normalization turns into spurious targets.

Training

Base model	nvidia/GR00T-N1.7-3B
Steps	8,000 (≈12.8 epochs)
Global batch size	128
Learning rate	1e-4, cosine, warmup ratio 0.05
Weight decay	1e-5
State dropout	0.2
Tuned	projector + diffusion action head (LLM and vision backbone frozen)
Hardware	1x NVIDIA H100 80GB, ~14 h
Final train loss	0.0535

Trained with the Isaac GR00T N1.7 finetune pipeline.

Open-loop evaluation

Predicted vs. ground-truth action chunks, 4 trajectories per task, 300 steps, action horizon 16:

Task	Avg MSE	Avg MAE	MSE range
Drawer	0.00345	0.0313 rad (≈1.8°)	0.0015 – 0.0069
Kitchen	0.00222	0.0272 rad (≈1.6°)	0.0009 – 0.0029

Caveat: neither dataset has a held-out split, so these trajectories were seen during training. The numbers show that the policy fits the demonstrations and that language conditioning selects the right behaviour per task — they are not a measure of generalization. Closed-loop evaluation on real hardware is required for that.

Usage

from gr00t.policy.gr00t_policy import Gr00tPolicy
from gr00t.data.embodiment_tags import EmbodimentTag

policy = Gr00tPolicy(
    model_path="twanghcmut/GR00T-N1.7-ALOHA-RightArm-Multitask",
    embodiment_tag=EmbodimentTag.NEW_EMBODIMENT,
    device="cuda",
)
action, _ = policy.get_action(observation)  # observation carries the task instruction

Serve it for a real robot client with:

python gr00t/eval/run_gr00t_server.py \
  --model-path twanghcmut/GR00T-N1.7-ALOHA-RightArm-Multitask \
  --embodiment-tag NEW_EMBODIMENT

Limitations

• Right arm only; single arm, joint-position control. Not a bimanual policy.
• Trained on two tasks in one fixed scene setup each — no evidence it transfers to new objects, new scenes, or new instruction phrasings.
• optimizer.pt and rng_state.pth are excluded from this repo, so it can be used for inference and further finetuning from the weights, but not to resume this exact run.