Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment
Paper β’ 2607.13429 β’ Published
Anchor-Align: Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment
π Project Page β’ π Paper (arXiv) β’ π» Code (GitHub) β’ π¬ Videos
TL;DR β Standard behavior-cloning finetuning of a vision-language model on robot demos silently erases the pretrained VLM's semantics and decouples its language output from its actions. Anchor-Align adds two lightweight losses on top of action prediction that (i) anchor the trainable VLA to a frozen copy of the pretrained VLM (preserving vision-language reasoning) and (ii) align the pre-action hidden state with a discrete motion-direction label derived from the executed motion. The result: 22.6% on LIBERO-PRO position swap (vs 2.3% for standard BC), 90.3% LIBERO-Plus average (vs 85.1%), and near-doubled real-world success (28.3% β 54.2%) on a UFactory xArm7 setup.
π¦ Companion GitHub Repository
This repo hosts the weights. The inference and evaluation code lives at github.com/dwipddalal/Anchor-Align.
HuggingFace (Dwipz/Anchor-Align) |
GitHub (dwipddalal/Anchor-Align) |
|
|---|---|---|
| Weights + tokenizer + configs | β | β |
| LIBERO / LIBERO-PRO / LIBERO-Plus eval scripts | β | β |
| Eval SLURM templates + 3-seed campaign runners | β | β |
| Paper results tables + figures | β | β |
Per-checkpoint MODEL_CARD.md |
β | β |
Reproduction recipe (REPRODUCE.md + verifier) |
β | β |
Note: the training code is not part of the initial release; we plan to release it in a follow-up update. Each checkpoint's full training configuration is documented in its
MODEL_CARD.md.
Available Checkpoints
| Folder | Suite | KL weight | Steps |
|---|---|---|---|
v7_kl01_spatial_10k/ |
LIBERO Spatial | 0.10 | 10k |
v7_kl015_object_2.5k/ |
LIBERO Object | 0.15 | 2.5k |
v7_kl01_goal_25k/ |
LIBERO Goal | 0.10 | 25k |
v7_kl015_l10_45k/ |
LIBERO-10 (Long) | 0.15 | 45k |
All four finetune Prismatic Qwen2.5-0.5B (DINOv2 + SigLIP, 24 layers) with LoRA rank 64 and an L1-regression MLPResNet action head. Per-checkpoint metrics live in each subfolder's MODEL_CARD.md; reproduction targets and tolerances are in the GitHub repo's REPRODUCE.md.
Results
Tables below are reproduced from the paper. Reproduction targets and tolerances for the released checkpoints are in REPRODUCE.md.
Standard LIBERO suites
Success rates on the four standard (unperturbed) LIBERO suites. Anchor-Align achieves the highest success rate on every suite, surpassing methods with substantially larger backbones and large-scale robot-action pretraining.
| Method | Spatial | Object | Goal | Long |
|---|---|---|---|---|
| Diffusion Policy | 78.3 | 92.5 | 68.3 | 50.5 |
| Οβ-FAST | 87.0 | 63.0 | 89.0 | 48.0 |
| SmolVLA-0.24B | 87.0 | 93.0 | 88.0 | 63.0 |
| SmolVLA-2.25B | 93.0 | 94.0 | 91.0 | 77.0 |
| OpenVLA-OFT | 94.3 | 95.2 | 91.7 | 86.5 |
| MolmoAct | 87.0 | 95.4 | 87.6 | 77.2 |
| Οβ.β -KI | 96.6 | 97.2 | 94.6 | 85.8 |
| VLA-0 | 93.6 | 96.0 | 95.6 | 87.6 |
| VLA-Adapter [Frozen] | 89.4 | 89.6 | 88.0 | 84.5 |
| VLA-Adapter (standard BC) | 96.0 | 99.8 | 96.0 | 89.0 |
| Anchor-Align VLA (ours) | 98.4 | 100.0 | 97.2 | 90.8 |
Robustness and generalization β LIBERO-PRO and LIBERO-Plus
Success rates under perturbation on the LIBERO-Spatial suite (paper Table 1). Bold = best, underline = second best.
| Method | LIBERO-PRO | LIBERO-Plus | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Lang. Reph. | Object Swap | Pos. Swap | Mean | Lang. Instr. | Bg. Text. | Robot Init | Cam. View | Obj. Layout | Light Cond. | Sensor Noise | Mean | |
| Co-training + KI* | 54.0 | 77.4 | 0.0 | 43.8 | 48.0 | 82.6 | 25.7 | 64.6 | 65.7 | 73.3 | 49.0 | 57.1 |
| MolmoAct | 77.8 | 82.4 | 0.0 | 53.4 | 79.5 | 84.1 | 47.4 | 10.1 | 76.5 | 77.4 | 53.4 | 60.8 |
| OpenVLA-OFT | 74.4 | 95.2 | 0.0 | 56.5 | 81.5 | 95.7 | 40.3 | 94.7 | 88.6 | 95.5 | 28.2 | 74.1 |
| VLA-Adapter [Frozen] | 56.0 | 73.4 | 0.0 | 43.1 | 41.5 | 70.9 | 35.1 | 94.4 | 62.3 | 84.9 | 36.2 | 59.9 |
| VLA-Adapter (standard BC) | 91.1 | 89.6 | 2.3 | 61.0 | 85.1 | 90.7 | 52.6 | 92.6 | 93.2 | 93.2 | 89.5 | 85.1 |
| Anchor-Align VLA (ours) | 97.0 | 96.2 | 22.6 | 71.9 | 87.2 | 99.6 | 59.1 | 96.3 | 97.4 | 99.0 | 96.9 | 90.3 |
*Our implementation of knowledge insulation adapted to VLA-Adapter. Position swap is the hardest axis: MolmoAct and OpenVLA-OFT score 0%, standard BC reaches 2.3%, while Anchor-Align reaches 22.6%.
Qualitative β generalization to semantic perturbations
Same task, different phrasing / different object identity / shuffled positions. Anchor-Align retains the VLM's semantic understanding of what the instruction refers to, while the baseline latches onto memorized appearance shortcuts.
Per-suite robustness β Object, Goal, and Long
The same gains carry over to the remaining three LIBERO suites. Each radar plot compares Anchor-Align (orange) against the standard BC VLA-Adapter baseline (gray) across nine evaluation axes: two from LIBERO-PRO (Language Rephrase, Object Swap) and seven from LIBERO-Plus.
Largest gains: Robot Init State +18.6 on Object; Language Instruction +11.9 on Goal; Lighting Condition +20.8, Object Layout +18.6, and Camera Viewpoint +17.7 on Long.
Long-horizon generalization β CALVIN ABCβD
Each rollout chains five language instructions; k/5 is the fraction of rollouts completing the first k, and Len is the average number of consecutively completed tasks.
| Method | 1/5 | 2/5 | 3/5 | 4/5 | 5/5 | Len |
|---|---|---|---|---|---|---|
| UniVLA | 95.5 | 85.8 | 75.4 | 66.9 | 56.5 | 3.8 |
| OpenVLA-OFT | 96.3 | 89.1 | 82.4 | 75.8 | 66.5 | 4.1 |
| OpenHelix | 97.1 | 91.4 | 82.8 | 72.6 | 64.1 | 4.1 |
| VLA-Adapter (standard BC) | 98.3 | 94.0 | 87.5 | 80.0 | 73.1 | 4.3 |
| Anchor-Align VLA (ours) | 99.1 | 95.8 | 90.6 | 84.7 | 77.9 | 4.5 |
Multi-seed significance
Mean Β± standard deviation over 5 training seeds on LIBERO-Spatial; the method gaps are far larger than seed-to-seed variability.
| Method | Spatial (Std) | Lang. Reph. | Object Swap | Pos. Swap | Plus |
|---|---|---|---|---|---|
| VLA-Adapter (standard BC) | 93.3 Β± 0.3 | 91.1 Β± 0.4 | 90.1 Β± 0.5 | 2.6 Β± 0.7 | 85.3 Β± 0.3 |
| Anchor-Align VLA (ours) | 97.9 Β± 0.3 | 97.1 Β± 0.5 | 96.1 Β± 0.4 | 23.5 Β± 0.2 | 90.5 Β± 0.6 |
Per-folder layout
Each subfolder is a self-contained inference bundle:
<benchmark>_<steps>/
MODEL_CARD.md
model.safetensors # merged base VLM weights (~2.5 GB)
lora_adapter/adapter_model.safetensors # separate LoRA weights (~208 MB)
action_head--<step>_checkpoint.pt # L1 regression head (~436 MB)
align_dir_proj--<step>_checkpoint.pt # alignment direction projector
proprio_projector--<step>_checkpoint.pt # proprioception projector
config.json, tokenizer.*, processor_config.json, dataset_statistics.json
Quickstart β download + run inference
1. Clone the code repo (contains all inference and eval scripts):
git clone https://github.com/dwipddalal/Anchor-Align.git
cd Anchor-Align
pip install -e . # + follow the installation section in the GitHub README
2. Download a checkpoint from this repo:
from huggingface_hub import snapshot_download
local_dir = snapshot_download(
repo_id="Dwipz/Anchor-Align",
allow_patterns="v7_kl01_spatial_10k/*", # or v7_kl015_object_2.5k/*, v7_kl01_goal_25k/*, v7_kl015_l10_45k/*
local_dir="./checkpoints",
)
3. Run inference using the eval scripts in the GitHub repo:
# LIBERO Standard
CUDA_VISIBLE_DEVICES=0 python experiments/robot/libero/run_libero_eval.py \
--pretrained_checkpoint ./checkpoints/v7_kl01_spatial_10k \
--task_suite_name libero_spatial \
--use_proprio True --num_images_in_input 2 --use_pro_version True
# LIBERO-PRO (paraphrased-language perturbation)
CUDA_VISIBLE_DEVICES=0 python experiments/robot/libero_pro/run_libero_pro_eval.py \
--pretrained_checkpoint ./checkpoints/v7_kl01_spatial_10k \
--base_suite_name libero_spatial --perturbation_type lan \
--use_proprio True --num_images_in_input 2 --use_pro_version True
Inference flags (--use_proprio True --num_images_in_input 2 --use_pro_version True --use_l1_regression True --center_crop True --num_open_loop_steps 8) match the paper. See the GitHub README's "Evaluation" section for LIBERO-Plus and batched variants.
License
MIT. See the GitHub repository LICENSE.
Citation
@article{dalal2026anchoralign,
title = {Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment},
author = {Dalal, Dwip and Patel, Shivansh and Jain, Chahit and Kim, Jeonghwan and Mishra, Utkarsh and Baratian, Alex and Ha, Hyeonjeong and Ji, Heng and Lazebnik, Svetlana and Jain, Unnat},
journal = {arXiv preprint arXiv:2607.13429},
year = {2026}
}