OpenDriveLab-org/ViRA

Do Better Visual Representations Always Lead to Better End-to-End Autonomous Driving?

Paper | Project Page | GitHub Repository

Visual foundation models (VFMs) are increasingly integrated into end-to-end autonomous driving, yet it remains unclear when these representations improve driving performance. ViRA is a planner-agnostic visual representation alignment framework: during training it aligns planner features with a frozen VFM, while the deployed planner architecture and inference cost stay unchanged.

Findings

The study is organized around three empirical findings:

1. Consistent driving gains: VFM-guided visual representations improve driving across regression, diffusion, and scoring planners. The gains extend to zero-shot closed-loop evaluation.
2. Target selection matters: Different VFM targets yield different planning gains for the same planner. Alignment to another VFM can further benefit planners that already use a pre-trained VFM encoder.
3. Supervision changes sensitivity: Auxiliary perception supervision narrows the EPDMS spread across five VFM targets from 2.7 to 0.5 points, and can compensate for a less effective target.

Performance

Guided by these findings, ViRA-Diffusion aligns a diffusion planner to DINOv3 and is trained without auxiliary perception supervision. It reaches 92.3 EPDMS on NAVSIM v2 navtest, exceeding the recent methods in our comparison by at least 1.9 points.

The same alignment improves three camera-only planners. On NAVSIM v2 navtest, EPDMS rises by +5.4 for TransFuser (83.6 → 89.0), +7.4 for DiffusionDrive (84.5 → 91.9), and +10.9 for Rap* (72.8 → 83.7). On the challenging navhard split, Rap*-ViRA gains +16.4 EPDMS. Checkpoints trained only on NAVSIM also improve zero-shot closed-loop HD-Score on HUGSIM.

Method	Decoder	NAVSIM v2 navtest EPDMS	NAVSIM v2 navhard EPDMS	HUGSIM HD-Score
TransFuser	Regression	83.6	27.6	22.1
TransFuser-ViRA	Regression	89.0 (+5.4)	31.7 (+4.1)	24.8 (+2.7)
DiffusionDrive	Diffusion	84.5	30.5	22.2
DiffusionDrive-ViRA	Diffusion	91.9 (+7.4)	35.8 (+5.3)	28.6 (+6.4)
Rap*	Scoring	72.8	32.8	7.3
Rap*-ViRA	Scoring	83.7 (+10.9)	49.2 (+16.4)	10.3 (+3.0)

Method	NAVSIM v2 navtest EPDMS
Epona	85.1
DiffusionDriveV2	87.5
Latent-WAM	89.3
DriveFuture	89.9
SparseDriveV2	90.1
Discrete-WAM	90.4
ViRA-Diffusion	92.3

Camera-only. Rap* is our reimplementation with a different backbone and without the original data augmentation. HUGSIM is zero-shot: planners are trained only on NAVSIM. ViRA-Diffusion is DiffusionDrive trained with DINOv3 alignment and without auxiliary perception supervision. DiffusionDrive-ViRA (91.9) and ViRA-Diffusion (92.3) are distinct configurations.

License

All content in this repository is licensed under the Apache 2.0 license.

HUGSIM Closed-loop Visualization

(a) Car following: the lead vehicle brakes suddenly

(b) Oncoming traffic: other vehicles enter the current driving area

Citation

If you find ViRA useful in your research, please consider citing:

@article{zhang2026vira,
  title={Do Better Visual Representations Always Lead to Better End-to-End Autonomous Driving?},
  author={Zhang, Zihao and Tian, Haochen and Li, Tianyu and Jing, Changhui and He, Jingliang and Ye, Naisheng and Pu, Ziyuan and Yang, Zhenjie},
  journal={arXiv preprint arXiv:2610.09695},
  year={2026},
  url={https://arxiv.org/abs/2610.09695}
}