SmolVLA: Fine-Tuning and Visual Representation Alignment on LIBERO-10

This repository contains five 200k-step SmolVLA checkpoints from a comparative study of fine-tuning strategies for language-conditioned robot manipulation. The experiments test standard LoRA, action-expert-only fine-tuning, and auxiliary visual feature alignment with DINOv3 or RADIO teachers. The alignment experiments are inspired by Don't Blind Your VLA: Aligning Visual Representations for OOD Generalization; they adapt the paper's teacher–student alignment idea to SmolVLA and the LIBERO-10 setting.

This is an empirical evaluation and representation-analysis project, not a new VLA architecture. In particular, the results do not establish that greater teacher–student feature similarity improves task success.

Base policy: lerobot/smolvla_base
Training dataset: lerobot/libero_10
Framework: LeRobot, with custom alignment training code
Training objective: action prediction / imitation learning; alignment variants add a visual feature-matching auxiliary objective
Released weights: checkpoint 200000 for each experiment

Checkpoints

Directory	Strategy	Training steps
`All_Lora/200000/`	LoRA fine-tuning	200,000
`DINO_Alignment/200000/`	LoRA + DINOv3 visual alignment (layer 15, weight 0.1)	200,000
`RADIO_Alignment/200000/`	LoRA + RADIO visual alignment (layer 16, after normalization, weight 0.2)	200,000
`Expert_Only_FT/200000/`	Full fine-tuning of the action expert	200,000
`Expert_Only_Lora/200000/`	LoRA fine-tuning of the action expert	200,000

Checkpoint files are preserved in the format saved by the training code. A checkpoint may contain full weights or a PEFT adapter; inspect its files before loading. PEFT adapters require the compatible base weights. The custom alignment implementation may also be needed to reconstruct the training configuration.

LIBERO-10 evaluation (in-distribution)

Success rate (SR) is the fraction of successful evaluation episodes. The reported evaluation uses 10 tasks × 20 episodes per task = 200 episodes per checkpoint.

Model	LIBERO-10 SR	Successful episodes / total
All LoRA	78.5%	157 / 200
DINO Alignment	66.5%	133 / 200
RADIO Alignment	69.0%	138 / 200
Expert Only FT	53.0%	106 / 200
Expert Only LoRA	33.0%	66 / 200

These are results from the recorded 200k-step evaluations of the five runs. The exact evaluation software revision and configurations should be archived alongside the checkpoints for fully reproducible comparisons.

LIBERO-plus evaluation (out-of-distribution)

We evaluated the All_Lora, DINO_Alignment, and RADIO_Alignment checkpoints on two LIBERO-plus distribution-shift categories:

Background Textures — variations in background appearance.
Objects Layout — variations in object arrangement.

Each category contains 100 selected evaluation variants, with one rollout per variant, for a total of 200 episodes per model. Within each category, all three models were evaluated on the same 100 task IDs. Success rates are computed directly from the per-episode successes entries in the evaluation JSON files.

Model	Background Textures	Objects Layout	Combined OOD SR
All LoRA	42 / 100 (42%)	36 / 100 (36%)	78 / 200 (39.0%)
DINO Alignment	14 / 100 (14%)	21 / 100 (21%)	35 / 200 (17.5%)
RADIO Alignment	26 / 100 (26%)	26 / 100 (26%)	52 / 200 (26.0%)

Expert_Only_FT and Expert_Only_Lora are not included in the LIBERO-plus results above because corresponding OOD evaluations are not provided here.

Observations

Standard All LoRA performs best in both evaluated distribution-shift categories.
RADIO Alignment outperforms DINO Alignment by 8.5 percentage points in combined OOD SR, but trails standard All LoRA by 13 points.
DINO Alignment trails standard All LoRA by 21.5 points in combined OOD SR.
On LIBERO-10, All LoRA also has the highest SR among the five evaluated 200k-step checkpoints.

These outcomes indicate that auxiliary feature alignment, as implemented in these experiments, did not translate into better downstream success. They do not prove that feature alignment is ineffective in general: teacher choice, alignment layer, weighting, representation geometry, and optimization settings are confounding variables.

Evaluation caveats: OOD experiments cover only two shift categories. Each variant has one rollout, so the reported SR measures success over a particular sampled set of variants rather than repeated-trial reliability for an individual variant. LIBERO-10 and LIBERO-plus contain different evaluation conditions; percentages should not be interpreted as a controlled estimate of performance degradation attributable solely to a single visual shift.

Visual representation alignment

The alignment strategy is based on the approach proposed in Don't Blind Your VLA: Aligning Visual Representations for OOD Generalization by Kachaev et al. The original work introduces an auxiliary teacher–student visual representation alignment objective during VLA fine-tuning. Here, we investigate that idea in SmolVLA on LIBERO-10, with two different vision teachers and different alignment configurations. This is an adaptation and empirical evaluation of the published idea, not a claim of a new alignment method.

For the DINOv3 and RADIO experiments, spatial teacher embeddings were precomputed for the agent-view and wrist-view images. Teacher features were represented as 8 × 8 spatial tokens per view, and an auxiliary feature-alignment loss was added to standard action training.

Configuration	Teacher	Student VLM alignment point	Alignment loss weight (λ)
DINO Alignment	DINOv3	15th VLM layer	0.1
RADIO Alignment	RADIO	16th VLM layer, after normalization	0.2

Layer numbers above describe the experiments' human-readable layer numbering; consult the training code for the precise zero-based tensor/hidden-state indices. Apart from the teacher, the experiments also differ in alignment layer, normalization point, and loss weight, so their performance difference cannot be attributed solely to teacher selection.

A separate patch-wise representation analysis examines similarity between the teacher and student at corresponding spatial locations. High aggregate similarity does not guarantee agreement on task-relevant patches (e.g., objects or the robot gripper), nor does it directly measure policy quality.

Download a checkpoint

from pathlib import Path
from huggingface_hub import snapshot_download

repo_id = "NIckSokolov/SmolVLA-LIBERO10-Experiments"
experiment = "RADIO_Alignment"  # or All_Lora, DINO_Alignment, etc.

snapshot = snapshot_download(
    repo_id=repo_id,
    repo_type="model",
    allow_patterns=f"{experiment}/200000/*",
)
checkpoint_dir = Path(snapshot) / experiment / "200000"
print(checkpoint_dir)

Downloading is not the same as loading a policy: loading a PEFT adapter requires a compatible base model and PEFT support, while custom training configurations may require the corresponding LeRobot fork. Before reporting reproduced metrics, verify the model loads and runs inference with the same camera mapping, observation normalization, and environment configuration.

Limitations

All experiments were evaluated in simulation, not on a physical robot.
Results are specific to the training setup, dataset, selected checkpoints, and evaluation configurations.
No multiple-seed aggregate or uncertainty interval is reported here.
Visual alignment diagnostics are not a substitute for closed-loop manipulation evaluation.
The OOD evaluation does not establish robustness to all LIBERO-plus perturbations.