wcong521/k1-follow-gr00t

K1 person-following GR00T N1.7

GR00T N1.7 3B fine-tuned to make a Booster K1 humanoid follow a person named by shirt colour ("follow the person in the red shirt") in Isaac Sim, through low clutter and past other people.

GR00T modality	key	content
video	ego_view	K1 head camera, 488 × 448 (H × W), ≈95° × 99° FOV, 10 Hz
state	base_velocity	measured (vx, vy, wz) in the body frame
action	velocity_command	16-step chunks of (vx, vy, wz) at 10 Hz, absolute
language	annotation.human.task_description	the instruction

Embodiment tag: new_embodiment. The velocity commands are tracked by Booster's k1_walk locomotion policy.

Training

• Data: 512 simulated episodes (about 196k frames) from a privileged follow teacher. The teacher uses A* around clutter and people, keeps 1.3 m behind the leader with the leader in view, and, if the leader starts out of view, searches for it by spinning counter-clockwise. 1–4 animated people are in each episode, with shirt and trouser colours randomized. On 20% of steps a perturbed command was executed while the clean teacher command was recorded, so the data includes recoveries.
• Tuned: the projector, the diffusion action head and the VL layer norm (tune_projector, tune_diffusion_model, tune_vlln). The language model and vision encoder are frozen.
• Schedule: 20,000 steps at batch size 32, learning rate 1e-4 with 5% warmup, weight decay 1e-5.

Use

The simulation environment and closed-loop rollout scripts are in the companion repository k1-gr00t-isaac-sim. Its README covers setup; download this checkpoint into it with

hf download wcong521/k1-follow-gr00t --local-dir checkpoints/k1-follow-gr00t

The model loads its vision-language backbone from nvidia/Cosmos-Reason2-2B (gated: accept its license first). To run on a CPU, set use_flash_attention to false in config.json; the repository's policy server does this automatically for --device cpu.

License

Derived from GR00T N1.7 and distributed under the NVIDIA License included in LICENSE: non-commercial use only (research or evaluation purposes).