🐸 LEAP-VLA

Latent-Enhanced Action Prototyping via Continuous Residual Latent Spaces for Vision-Language-Action Models (ECCV 2026)

Bo-Yun Yu · Kuan-Chuan Peng · Jun-Wei Hsieh

💻 Code (GitHub) · 📄 Paper · ⚡ Quick start

Pretrained Stage-1 (MSRQ) and Stage-2 (Latent Action Aligner + LoRA on InternVL3-1B) checkpoints for every benchmark in the paper. Installation, training, evaluation and the inference API are in the GitHub repository.

Checkpoints

Folder	Evaluated on	Result
libero_spatial/	LIBERO-Spatial · LIBERO-plus (spatial)	98.4%
libero_object/	LIBERO-Object · LIBERO-plus (object)	99.4%
libero_goal/	LIBERO-Goal · LIBERO-plus (goal)	97.6%
libero_10/	LIBERO-Long · LIBERO-plus (10)	93.6%
calvin/	CALVIN ABC→D	4.28 avg len

LIBERO-plus total (all 10,030 perturbed tasks): 78.3%. Full tables and logs: Results.

<benchmark>/
├── bridge/                      # Stage-2 → --bridge_checkpoint
│   ├── bridge.pt                #   Latent Action Aligner
│   ├── vlm_lora/                #   LoRA adapter for OpenGVLab/InternVL3-1B-hf
│   ├── config.json
│   ├── dataset_statistics.json  #   action de-normalization
│   └── optimizer.pt · scheduler.pt · training_state.pt   # only needed to resume training
└── stage1/best.pt               # Stage-1 MSRQ → --stage1_checkpoint

Usage

git clone https://github.com/AndersonYu7/LEAP-VLA.git && cd LEAP-VLA
conda create -n leap_vla python=3.10 -y && conda activate leap_vla
bash install.sh base

hf download Boyun7/LEAP-VLA --include "libero_spatial/*" --exclude "*/optimizer.pt" --local-dir checkpoints

python scripts/inference.py \
    --bridge_checkpoint checkpoints/libero_spatial/bridge \
    --stage1_checkpoint checkpoints/libero_spatial/stage1/best.pt \
    --image assets/quickstart/libero_spatial_agentview.png \
    --wrist_image assets/quickstart/libero_spatial_wrist.png \
    --instruction "pick up the black bowl between the plate and the ramekin and place it on the plate"

from leap.policy import LEAPPolicy

policy = LEAPPolicy("checkpoints/libero_spatial/bridge", "checkpoints/libero_spatial/stage1/best.pt")
actions = policy.predict(["agentview.png", "wrist.png"], "put the bowl on the plate")  # (8, 7) action chunk

Benchmark evaluation (LIBERO, LIBERO-plus, CALVIN): docs/.

Citation

@inproceedings{yu2026leapvla,
  title     = {LEAP-VLA: Latent-Enhanced Action Prototyping via Continuous Residual Latent Spaces for Vision-Language-Action Models},
  author    = {Yu, Bo-Yun and Peng, Kuan-Chuan and Hsieh, Jun-Wei},
  booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
  year      = {2026},
  url       = {https://eccv.ecva.net/virtual/2026/poster/3297}
}