Ngseo/microvla-seqgru-libero

MicroVLA — seqGRU (12.4M)

A vision-language-action policy for LIBERO at 12.37M deployed parameters, reaching 90.45% closed-loop success over the full protocol (4 suites x 10 tasks x 50 episodes, n=2000).

Roughly 1/560 the size of a 7B VLA at about 93% of its success rate.

Results

Closed-loop, 2000 episodes, chunk_exec=16, eval-time agentview centre crop 0.9.

suite	success
spatial	92.4%
object	95.0%
goal	89.2%
long-horizon (10)	85.2%
overall	90.45%

Across 4 training seeds this recipe averages 90.34 +- 0.69% at 400k steps. Retraining alone moves the score by sigma ~ 0.7-1.3pp, so differences under ~2.5pp should not be read as real.

Architecture

component	choice
vision	Theia-tiny ViT (5.5M), fine-tuned, shared across agentview + wrist
language	POTION-2M static embedding table, frozen, token sequence
text read-out	93K bidirectional GRU over the frozen token sequence
fusion	FiLM + Perceiver pooling, 16 latent queries, depth 3
head	MLP-ResNet (512 wide, 5 blocks) -> L1 regression
action	16-step chunk of 7-DoF delta end-effector commands
proprio	8-dim robot state

Trained with crop/jitter/warp/noise augmentation and a contrastive auxiliary loss (InfoNCE + VICReg terms) on the fused representation. The auxiliary projector is not included here — it is training-only, which is why the deployed model is 12.37M while training used 14.11M.

Why the GRU read-out

Mean-pooling a static embedding table is permutation-invariant by construction, so the policy cannot distinguish "put the red cup in the blue cup" from the reverse. Measured on identical observations, the mean-pooled variant emits bit-identical actions for an argument swap (cosine 1.0000000 +- 7e-8). The 93K GRU drops that to cosine 0.14 at no cost in success rate.

Usage

This checkpoint is for the microvla codebase, not transformers.

import torch
from microvla.config import MicroVLAConfig
from microvla.model import MicroVLA

ck = torch.load("model.pt", map_location="cpu", weights_only=False)
model = MicroVLA(MicroVLAConfig.from_dict(ck["config"]))
model.load_state_dict(ck["model"], strict=False)   # ssl_proj is absent by design
model.eval()

# images: ImageNet-normalised (B,3,224,224); proprio: (B,8)
pred = model(agentview, ["put the bowl on the plate"],
             wrist_image=wrist, proprio=state)      # (B,16,7), normalised
action = pred.cpu().numpy() * ck["action_std"] + ck["action_mean"]

action_mean / action_std are stored in the checkpoint and also in action_stats.npz. Predictions are normalised — denormalise before stepping the environment.

Evaluation protocol

The eval-time centre crop matters: switching it on or off moves totals by -2.2 to +4.4pp depending on the model, which is larger than the seed noise. The numbers above use agentview_crop_scale=0.9. Results from a different crop setting are not comparable.

results/ contains the raw evaluation JSONs, including instruction-manipulation diagnostics (correct / role-swapped instructions).

Limitations

• LIBERO simulation only; no real-robot validation.
• Long-horizon tasks lag the rest (85.2% vs 95.0% on object).
• The instruction vocabulary is LIBERO's noun-phrase style; this is not an open-vocabulary language model.
• Trained on ~1.7k demonstrations across 40 tasks; out-of-distribution scenes are untested here.