Ngseo/basevla-libero-aug

BaseVLA — LIBERO 4-suite (Qwen2.5-VL-3B) with domain augmentation

Plain behaviour cloning baseline: no V-JEPA, no latent alignment, no decorrelation. Just L1(action) on the LIBERO 4-suite mix, with domain augmentation switched on. Trained for 100,000 steps.

This is the control condition for a study on viewpoint↔task spurious correlation — the "what does augmentation alone buy you?" cell.

Data

The four standard LIBERO suites, sampled with equal weight per suite (weight / n_frames, so suite size does not bias the mix):

suite	episodes	frames
libero-spatial	432	52,970
libero-object	454	66,984
libero-goal	428	52,042
libero-10	379	101,469
total	1,693	273,465

Front camera (observation.images.image) only; the wrist camera is dropped. 10 fps, 8-D state, 7-D action.

Architecture

Qwen2.5-VL-3B + LoRA ─ forward_full_hidden ─┬─ Head A → z_a ─┐
                                            └─ Head B → z_b ─┴─ concat → ResNetActionHead → 10×7

L = L1(action)        ← that is the whole objective

LoRA	r=32
Heads	dual AttentiveLatentHead, proj 4096, 8 queries, depth 2
Inputs	1 RGB frame @224 + task string + 8-D state
Output	10-step action chunk (1.0 s @ 10 fps)
Optimiser	AdamW, lr 5e-5, batch 32, bf16
Augmentation	ColorJitter/SharpnessJitter ×2 + DomainRandomization p=0.7

Augmentation detail — applied to the context frame only: lighting gain 0.3–2.0, sensor noise ISO 1–4, random crop, warping (corner shift 0.08–0.20), each drawn independently per sample.

Training curve

step	loss
10,000	0.0823
20,000	0.0731
30,000	0.0655
50,000	0.0537
70,000	0.0492
100,000	0.0479

17 h 28 min on one A100 80GB.

Not evaluated on a robot or in simulation. The number above is a training-set L1 on normalised actions. No LIBERO rollout success rate is reported here, and rollout is what actually matters for this benchmark.

Contents

checkpoint.pt holds policy (full VLM incl. LoRA), latent_head, free_latent_head, action_head, optimizer, agg_stats (the normalisation statistics used), and args (the exact CLI configuration).

Load with:

import torch
ck = torch.load("checkpoint.pt", map_location="cpu", weights_only=False)
ck["step"]        # 100000
ck["args"]        # full training config
ck["agg_stats"]   # action/state normalisation used at train time