maskjp/groot-n17-relative-eef-all10-30k

GR00T N1.7 — relative actions, end-effector space

Fine-tune of nvidia/GR00T-N1.7-3B on all ten L5vel/* datasets, with actions expressed relative to the observation state at the start of each chunk. Step 30K of 30K — the full run.

Held-out loss

5% of episodes per task are held out (75 of 1499 episodes, all 11 tasks).

step	eval_loss
5K	0.0182
10K	0.0139
15K	0.0109
20K	0.0094
25K	0.0090
30K	0.0086

Still decreasing at 30K, and the run covers only 0.66 epochs of the mixture, so this checkpoint is undertrained rather than converged. Longer training is the obvious next step.

Do not compare this number to the pi05-relative-* repos. Both are MSE on the flow-matching velocity, but the normalisation differs: pi05 uses QUANTILES (q01/q99), GR00T N1.7 uses per-horizon min/max of the relative offsets. On this mixture the min–max span is 3.0x the q01–q99 span (median) on the arm dims and 5.0x (up to 49.7x) on the base dims, so GR00T's targets live in a compressed space and its loss is smaller by roughly the square of that factor for reasons that have nothing to do with fit quality.

Action representation

use_relative_actions=true with relative_exclude_joints=["gripper"]:

action[t+k] -= observation.state[anchor]     for all dims EXCEPT gripper
gripper                                       absolute (it is a command, not a pose)

Action dim 13: eef_x, eef_y, eef_z, eef_xx, eef_xy, eef_xz, eef_yx, eef_yy, eef_yz, gripper, base_x, base_y, base_yaw — 12 of 13 dims relative, gripper absolute.

Normalisation uses GR00T's native per-horizon relative statistics, computed from this mixture's own relative offsets (a separate min/max per chunk position, shape [40, d]) and baked into policy_preprocessor.json. The checkpoint is self-contained: it does not need the training data or the base model's statistics.json to normalise correctly.

The end-effector rotation is the stored continuous 6D form (first two rotation-matrix columns), not axis-angle. On this robot the gripper points down, so |rotvec| sits near pi where the sign flips arbitrarily; an axis-angle view carries frame-to-frame jumps of 2*pi.

Chunk size is 40, not 50

The run requested chunk_size=50 to match the pi05 arms, and GR00T N1.7 overrode it to 40: GrootConfig.__post_init__ remaps 50 as a legacy N1.5-era default, and the action head's delta_indices are clamped to the N1.7 horizon of 40 regardless. So the lookahead is 0.8 s at 50 fps rather than pi05's 1.0 s. n_action_steps=10 (0.2 s executed per replan) is unchanged, so the control cadence still matches.

Training data

11 language-conditioned tasks, 1499 episodes, 3,089,476 frames (17.2 h at 50 fps), one u850 arm on a mobile base, three cameras (left / right / wrist).

task	episodes	frames
grab a drink from the fridge	250	968,523
pick up the bag on the ground and place it on the table	200	477,058
open the door and move inside	200	506,473
clean the table with the green towel	199	534,608
pick up the blue cup from the table	102	75,870
pick up the green bottle from the table	100	69,748
open the fridge door	100	118,990
place the blue cup on the table	100	61,416
pick up the grocery bag from the ground	100	92,672
move the croissant to the empty plate	98	155,517
place the green cup on the table	50	28,601

The mixture is skewed: the four largest tasks are ~82% of frames. Sampling is uniform over frames, so it is not rebalanced.

Shipped training metadata (meta/)

meta/info.json and meta/stats.json are the training dataset's own files, byte for byte. They are here so a deployment-side fingerprint guard can bind this checkpoint to real column names and real statistics instead of inferring them. The datasets (all10-eefabs6d-v30) were merged locally from the ten L5vel/* sources and never pushed, so the repo_id in train_config.json 404s on the Hub.

file	sha256
meta/info.json	62ae3181d3a3f01593385486f0483728ddfadb1c2f7a55cf524307fd55042cd7
meta/stats.json	dd1065ec40d3b229cf730f7873d8f435ba28721ee83f3830f6d6ba28ea127199

The statistics bind exactly. The preprocessor step groot_n1_7_pack_inputs_v1 carries config.raw_stats with three modalities -- state, action, relative_action. The first two are this dataset's absolute statistics, copied verbatim. Concatenating the groups in modality order (single_arm, gripper, single_arm_2) and casting both sides to float32 reproduces stats.json exactly, for min, max, mean and std alike:

action            float32-cast exact = True
observation.state float32-cast exact = True

Compare in float32, not float64: that is the storage precision on the checkpoint side, and a float64 comparison leaves residuals up to 5.8e-08 relative on some columns.

relative_action is the modality with no on-disk counterpart. Those per-horizon [40, d] min/max are computed at training time from the relative offsets and never written as a dataset.

Two traps in these files

1. names is the grouped v3 form, {"motors": [...]}, not a flat list, so list(names) yields ["motors"] -- one name for 13 columns. Flatten it (lerobot.utils.feature_utils.flatten_feature_names). These files are shipped unmodified on purpose, so the flattening is the reader's job.
2. The 6D rotation columns are named twice, differently. action / observation.state use eef_xx ... eef_yz; action.eef / observation.eef_state use eef_xaxis_x ... eef_yaxis_z for the same quantity. This checkpoint's output_features carries both action and action.eef, both 13-dim, so shape alone will not disambiguate them: a name binding has to say which feature it binds. The trained action space is action.

Recipe

init	nvidia/GR00T-N1.7-3B
regime	projector + action head (DiT) + VL layernorms; LLM and vision tower frozen
batch	16/rank x 4 ranks = 64 effective
lr	1e-4, 5% warmup, cosine decay
chunk_size / n_action_steps	40 (see above) / 10
precision	bf16 compute, fp32 parameters
embodiment tag	new_embodiment
seed	1000
wall clock	7.3 h on 4x A100-class GPUs

Loading

from lerobot.policies.groot.modeling_groot import GrootPolicy
policy = GrootPolicy.from_pretrained("maskjp/groot-n17-relative-eef-all10-30k")

base_model_path in config.json points at nvidia/GR00T-N1.7-3B; the architecture is rebuilt from there and the fine-tuned weights come from model.safetensors in this repo.

License

NVIDIA License, inherited from the base model — research and evaluation use only (section 3.3). A complete copy ships as LICENSE in this repo, as section 3.1 requires.

Related

• maskjp/groot-n17-relative-joints-all10-30k — the same run in joint space.
• maskjp/pi05-relative-eef-all10-full-ft-100k — pi0.5 on the same mixture and the same relative-action scheme. See the loss caveat above before comparing numbers.