aabyaneh/lingbot-va-lift_new

LingBot-VA — lift_new (real-world Franka)

LingBot-VA (Wan2.2-TI2V-5B backbone + bolt-on action head) fine-tuned on George's real-world lift_new Franka dataset (single-arm teleop, 10 Hz, 2× 256×256 cameras, 7-dim action: delta-eef xyz + zero rotation + binary gripper).

This is the Phase-2 lift_new_base recipe: adapted from the robot-co-pretrained lingbot-va-base init.

Contents

Each subfolder is a diffusers-format transformer checkpoint (transformer/):

Subfolder	Training step
step_600/	600
step_800/	800 (latest)

What's included: only the trained transformer (shared Wan blocks + action head). The VAE and UMT5 text encoder are not included — load them from the base Wan-AI/Wan2.2-TI2V-5B model. These weights are the only trained component.

Notes / provenance

• Effective batch 80 (world_size × batch_size × grad_accum), lr 1e-5, constant schedule.
• The lift_new dataset presents one video-latent clip per episode (90 train / 10 val), so these steps correspond to a large number of epochs over a small set — treat val / sim-eval as the stopping signal rather than train loss.
• These are the surviving lift_new checkpoints (steps 200/400 were later overwritten on the training node by an unrelated run before archival).

Usage (sketch)

from wan_va.train import load_transformer
transformer = load_transformer("step_800/transformer", attn_mode="flex")
# pair with the Wan2.2-TI2V-5B VAE + UMT5 text encoder