VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planning
Authors: Bo Jiang
Shaoyu Chen
Hao Gao
Bencheng Liao
Qian Zhang
Wenyu Liu
Xinggang Wang
Venue: ICLR 2026 Poster (2026)
Keywords: Autonomous Driving
End-to-end
Probabilistic Planning
Closed-Loop
Planning Vocabulary
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Learning a human-like driving policy from large-scale driving demonstrations is promising, but the uncertainty and non-deterministic nature of planning make it challenging. Existing learning-based planning methods follow a deterministic paradigm to directly regress the action, failing to cope with the uncertainty problem. In this work, we propose a probabilistic planning model for end-to-end autonomous driving, termed VADv2. We resort to a probabilistic field function to model the mapping from the action space to the probabilistic distribution. Since the planning action space is a high-dimensional continuous spatiotemporal space and hard to tackle, we first discretize the planning action space to a large planning vocabulary and then tokenize the planning vocabulary into planning tokens. Planning tokens interact with scene tokens and output the probabilistic distribution of action. Mass driving demonstrations are leveraged to supervise the distribution. VADv2 achieves state-of-the-art closed-loop performance on the CARLA Town05 benchmark, significantly outperforming existing methods, and also leads the recent Bench2Drive benchmark. We further provide comprehensive evaluations on NAVSIM and a large-scale 3DGS-based benchmark, demonstrating its effectiveness in real-world applications. Code is available at https://github.com/hustvl/VAD.

Published as a conference paper at ICLR 2026
VADv2: END-TO-END VECTORIZED AUTONOMOUS DRIVING VIA PROBABILISTIC PLANNING
Bo Jiangh-> Shaoyu Chen?., Hao Gao!e Bencheng Liao
Qian Zhang',
Wenyu Liu', Xinggang Wang" 1.8
HHuzzhong University of Science and Technology Hoorizon Robotics (bjiang, ghac, beliao, liuwy, xgwang ) @hust. edu. cn (shaoyu. chen, glanC1. @horizon. auto
ABSTRACT
Learning a human-like driving policy from large-scale driving demonstrations is promising, but the uncertainty and non-deterministic nature of planning make it challenging. Existing learning-baste based planning methods follow a deterministic paradigm to directly regress the action, failing to cope with the uncertainty problem. In this work, we propose a probabilistic planning model for end-to-end autonomous driving, termed VADv2. We resort to a probabilistic field function model the mapping from the action space to the probabilistic distribution. Since the planning action space is a high--dmeensional continuous spatiotemporal space and hard tackle, we first discretize the planning action space to a large planning vocabulary and then tokenize the planning vocabulary into planning tokens. Planning tokens interact with scene tokens and output the probabilistic distribution of action. Mass driving demonstrations are leveraged to supervise the distribution. VADv2 achieves state-of-ther -htt-ffft closed-loop performance nificantly outperforming existing methods, and also leads the recent Bench2Drive benchmark. We further provide comprchensive evaluations NAVSIM and large-scale 3DGS- benchmark, demonstrating its effectiveness in rorl-world applications. Code is available at https: / htwp.ccom/ac hatmm/ussvl/VAA
the CARLA Town05 benchmark, sig-
INTRODUCTION
End-to-end autonomous driving is an important research topic currently. Large-scale human driving demonstrations in real-world scenarios are readily available. It very promising to derive human- like driving policy for vehicle control from these extensive demonstrations. However, the uncertainty and non-deterministic nature of planning make challenging to extract the driving knowledge from driving demonstrations. To illustrate such uncertainty, two scenarios presented in Figure D and explained as follows. 1) Following another vehicle: The human driver exhibits various reasonable driving maneuvers, such maintaining the current lane changing lanes to overtake. 2) Interaction with an oncoming vehicle: The human driver faces two potential driving maneuvers, i.e., yielding overtaking. From statistical perspective, the actions (including timing and speed) are highly stochastic, influenced by numerous latent factors that cannot be accurately modeled.
Existing learning-basee planning methods (Tiang al. 2023 Hu al. 2022c Jia al. 2023b
Prakash et al.] 20021b Hu et al. 2022a Zhang et al. 2021) follow a deterministic paradigm to directly regress the action. The regression target the future trajectory (Jinng et al. 2023 Hu al 2022c
Jia et al. 2023b Prakash et al. 202 and control signal (acceleration and steering) in (Hu et al.
2022a| Zhang al. (022). Such a paradigm assumes there exists deterministic relation between
the driving scene and action, which is not the case. The variance of human driving behavior causes the anbiguity of the regression target. Especially when the feasible solution space non-convex.
ie., there exist multiple feasible solutions (see Figure deterministic modeling may fail to properly handle such cases and produce an intermediate action, resulting in potential safety risks.
"Equal contribution. Work done during internship Horizon Roboties. R Corresponding author.

---

Published as a conference paper at ICLR 2026
In this work, we propose probabilistic plan- ning to cope with the uncertainty of plan- ning. We model the planning policy as scene-conditioned non-stationary stochas- tic process, formulated p(alo), where o is the historical and current observations of the driving scene, and is candidate plan- ning action. Compared with deterministic planning, probabilistic planning more ro- bust against uncertainty in planning and able to model non-convex feasible solution space, and thus achieves more accurate and safer planning.
8
We resort to probabilistic field fune- tion to model the mapping from the plan- ning action space to the probabilistic dis- tribution. Since the action space a high-dimensional continuous spatiotempo- ral space and hard to tackle, we first dis- cretize the planning action space a large planning vocabulary and then tokenize the planning vocabulary into planning tokens. Planning tokens interact with scene tokens and output the probabilistic distribution of action. Mass driving demonstrations are leveraged to supervise the distribution.
Driving Demonstrations
Deterministic Planning
Probabilistic Planning
Figure 1: Uncertainty exists in planning. There doesn't t exist deterministic relation between driving scene and action. The deterministic planning fails to model such uncertainty especially when the feasible solution space is non-convex. VADv2 is based on probabilistic plan- ning and learns scene-conditioned probabilistic distribu- tion of action from large-scale driving demonstrations.
Probabilistic planning has two other advantages. First, unlike deterministic planning which has to regress the optimal action based on scene information, probabilistic planning models the correlation between each action and the driving scene. It just ranks different actions and samples a scooiig one. Such modeling is much simpler. Besides, probabilistic planning is flexible in the inference stage. outputs multi-mode planning results and is easy to combine with rule- based and optimization-ban based planning methods. We flexibly add other candidate planning actions to the planning vocabulary and evaluate them because we model the distribution over the whole action space.
Based the probabilistic planning, we present VADv2, end-to-end driving model, which takes surround-view image sequence as input in a streaming manner, tokenizes sensor data and planning action space, outputs the probabilistic distribution of action, and samples one action control the vehicle. Using only camera sensors, VADv2 achieves state-of-the-al closed-loop performance on the CARLA Town05 benchmark, significantly outperforming existing methods, and also leads the recent Bench2Drive benchmark. It further delivers strong planning performance on NAVSIM and our 3DGS-based benchmark. VADv2 runs stably a fully end-to-end manner, even without rule-based wrapper as post-processing step to avoid infraction.
Our contributions are summarized as follows:
We propose probabilistic planning to cope with the uncertainty and non-deterministic nature of planning. We design probabilistic field map from the action space the probabilistic distribution and learn the distribution of action from large-scale driving demonstrations.
Based on the probabilistic planning, we present VADv2, an end-to-end driving model, which tok- enizes sensor data and planning action space for interaction, outputs the probabilistic distribution of action, and samples one action to control the vehicle.
VADv2 achieves state-of-the-art planning performance in both closed- and open-loop settings across multiple benchmarks. Abundant closed-loop simulations and real-world deployment results validate its effectiveness and stability in vehicle control.
2 RELATED WORK
Perception. Perception is the first step in achieving autonomous driving, and unified representation of driving scenes is beneficial for easy integration into downstream tasks. Bird's Eye View (BEV)
2

---

Published as a conference paper at ICLR 2026
representation has become a common strategy in recent years, enabling effective scene feature encoding and multimodal data fusion. LSS (Philion & Fidler, 2020) is a pioneering work that achieves the perspective view to BEV transformation by explicitly predicting depth for image pixels. BEVFormer (Li et a 2022c), on the other hand, avoids explicit depth prediction by designing spatial and temporal attention mechanisms. Subsequent works (Li et al.) 2022b; Wang et al.) 2023a) continuously optimize temporal modeling and BEV transformation strategies. In terms of vectorized mapping, HDMapNet (Li et al. 2022a) converts lane segmentation into vector maps through post-
processing. VectorMapNet (Liu et al. 2022) predicts vector map elements in an autoregressive manner. MapTR (Liao et al. [022]] 2023b) introduces permutation equivalence and hierarchical matching strategies, significantly improving mapping performance. LaneGAP (Liao et al. 2023a) introduces path-wise modeling for lane graphs.
Motion Prediction. Motion prediction aims to forecast future trajectories of other traffic participants, assisting the ego vehicle in making informed planning decisions. Traditional motion prediction methods utilizes input such as historical trajectories and high-definition maps to predict future
trajectories (Gao al.] 2020) [Lu al. (02I). However, recent end-to-end methods (Gu et al. 2022
Jiang et al. 2022) perform perception and motion prediction jointly. Some works (Hu et al., 2021
2022) represent future motion as dense occupancy and flow fields, while others (Gu et al. 2022 Jiang et al. 2022) predict agent-level multi-modal trajectories. Another line of work reformulate trajectory prediction as a classification problem rather than a regression task. Trajeglish (Philion et al. 2023) introduces K-disk sampling to construct compact one-step motion vocabulary, achieving lower discretization error compared to k-means. MotionLM (Seff et al. 2023) factorizes each single- step action into longitudinal and lateral components and applies axis-aligned uniform quantization. While such single-step modeling enables a compact representation and a small vocabulary, iterative rollout can lead error accumulation and may produce trajectories that violate physical constraints. In contrast, each action token in VADv2 represents a complete trajectory, ensuring physically feasible motion primitives and enabling one-shot planning without error accumulation.
Zhang et al.
Planning. Learning-baseed planning has shown great potential recently due to its data-driven nature and impressive performance with increasing amounts of data. Early attempts (Codevilla et al. 2019 Prakash et al. 2021a) use completely black-box spirit, where sensor data is directly used to predict control signals. However, this strategy lacks interpretability and is difficult to optimize. In addition. there are numerous studies combining reinforcement learning and planning (Zhang et al., 2021 Gao al. (025) by autonomously exploring driving behavior in closed- dlope-loop simulation environments. Imitation learning (Chekroun et al. 202T; Hu et al. 2022b] Ma et al. (2025) is another research direction, where models learn expert driving behavior to achieve good planning performance and
develop a driving style close to that of humans.
UniAD (Hu et al.) (022c) integrates multiple perception and prediction tasks to enhance planning performance. VAD (Jiang et al. 2023) explores the potential of vectorized scene representation for planning and getting rid of dense maps. Diffusion Planner ZZheng et al. 2025) jointly predicts ego and other agents' motions via iterative diffusion, but it relies on ground -ruth perception and HD maps. DiffusionDrive (Liao et al.) 2025) accelerates the denoising process through truncated diffusion; yet the limited and predefined trajectory anchors may constrain generation quality, and increasing the number of anchors introduces additional computational cost. GoalFlow (Xing et al. 2025) decomposes planning into goal-point selection followed by trajectory generation via flow matching, but relying on a single goal point may affect trajectory diversity, and its hand-crafted trajectory selection rules limit generality and scalability.
Large Language Model in Autonomous Driving, Recent researches explore the combination of LLMs and autonomous driving (Sha et al. 2023 Xu al. 2023). One line of work utilizes L.MSs
for driving scene understanding and evaluation through guestion-answering (Chen et al.] 2023 Sima
et al.
2024). Another approach goes step further by directly utilizing L.LMs for planning (Wang
et al. 2023b 2024). However, current LLM-based planning approaches inevitably suffer from limited inference speed, which constrains their practicality for real-time deployment in autonomous driving applications. VADv2 draws inspiration from GPT (Achiam et al. (2023) to cope with the uncertainty problem, which also exists in language modeling. Given a specific context, the next word is non- deterministic, LLM learns the context-conditioned probabilistic distribution of the next word from a large-scale corpus, and samples one word from the distribution. Inspired by LLM,, VADy2 models
3

---

Published as a conference paper at ICLR 2026
Action Space
Discretize
Planning Vocabulary
Encode
Large Scale Driving Demoastations
Planning Tokens
True Distribttion
Navigation Info.
Ego State
KL Divergence
Conflict Check
Encode
A
K.
Planning Transformer
Scene Constraint
Prob. Distribution
Mult-View
Scene Tokens
Image Sequence
Map Tokens
Agent Tokens
Image Tokens
Tralic Element Tokens
Figure 2: Overall architecture of VADv2. VADv2 takes multi-view image sequences as input in streaming manner, tokenizes sensor data and planning action space, outputs the probabilistic distribu- tion of action, and samples one action to control the vehicle. Large-scale driving demonstrations and scene constraints are used to supervise the predicted distribution.
the planning policy scene-conditioned nonstationary stochastic process. VADv2 discretizes the action space to generate a planning vocabulary, approximates the probabilistic distribution based on large-scale driving demonstrations, and samples one action from the distribution at each time step to control the vehicle.
VADv2
The overall framework of VADv2 is depicted in Figure VADv2 takes multi image sequences as input in a streaming manner, transforms sensor data into scene token embeddings, outputs the probabilistic distribution of action, and samples one action control the vehicle. Large-scale driving demonstrations and scene constraints are used to supervise the predicted distribution.
3.1 SCENE ENCODER
VADv2 uses a scene encoder to transform the sensor data into instance-level token embeddings Esrone E XD explicitly extract high-level information, where M is the number of scene tokens and D is the feature dimension. Concretely, Escene includes four kinds of scene tokens: map tokens, agent tokens, traffic element tokens, and image tokens.
BEV Encoder. A BEV encoder (Li et al. 2022c) first employed to transform the multi-viww image features from perspective view into Bird's Eye View, producing feature map in the BEV space. This feature map serves as the basis for learning instance-level map and agent features.
Map Tokens. A set of map tokens (Liao et al., 2023b) is introduced to learn vectorized map elements of the driving scene from the BEV feature map, including lane centerlines, lane dividers, road boundaries, and pedestrian crossings.
Agent Tokens. In addition, a group agent tokens (Jiang al. 2023) adopted predict motion information of traffic participants, including location, orientation, size, speed, and future trajectories.
Traffic Element Tokens. Traffic signals also play vital role in planning. In CARLA, consider two types of traffic signals: traffic lights and stop signs. Front-view image features extracted from the backbone further encoded using an MLP into traffic element tokens, which are then used to predict the states of the traffic signals.
Image Tokens. Apart from the above instance-level tokens, front-view image features also used as image tokens. These image tokens provide denser scene features for planning, capturing rich information that complements the instance-level tokens.
4

---

Published as a conference paper at ICLR 2026
Map tokens, agent tokens, and traffic element tokens supervised with corresponding supervision signals to ensure they explicitly encode corresponding high-level information. Besides, navigation information and ego state also encoded into embeddings (Enavi, Estate) with MLP. In summary, the Scene Encoder transforms the sparse sensor data into more compact high-level scene features (Eeceney Fnavi: ), which serve the foundation for the following planning module.
3.2 PROBABILISTIC PLANNING
We propose probabilisticc planning cope with the uncertainty of planning. We model the planning policy as a svene-conditioned nonstationary stochastic process, formulated as p(a oa) where o is the observed scene information and is the action, represented waypoint sequence of future planning trajectory.
o
Esceney
Fnavis
Fstaao); a (01, V1, 22, 12.-... 2T, yT)-
T is the number of waypoints. Each waypoint (æt, ys) corresponds a future timestamp 4
We approximate the probabilistic distribution of the planning action space based on large-scale driving demonstrations, and sample one action from the distribution each time step to control the vehicle.
Algorithm 1: Planning vocabulary sampling.
Input: Planning action set S, Planning vocabulary size N
Output: Planning vocabulary V
Initialization: V -- for i = 1 to N 1 do
The planning action space is high-dimensional continuous spatiotemporal space and hard to tackle. Thus, we discretize the planning ac- tion space to large planning vocabulary V {a') J where N is the vocabulary size. Specifi- cally, we collect all the planning actions in driv- ing demonstrations as the planning action set S and adopt the furthest trajectory sampling to select N representative actions to serve as the planning vocabulary. The vocabulary sampling algorithm is presented in Algorithm [ Each planning action in V is sampled from driving demonstrations and thus naturally satisfies the kinematic constraints of the ego vehicle, which means that when the action is converted into control signals (steer, throttle, and brake), the control signal values do not exceed the feasible range. By default, N is set to 4096.
if = ( then
random select action in S - V {a) add action to planning vocabulary
$ - S i {a} remove action from
planning action set
end mar mar_dis=
for trajectory in S do dis calculate distance V) if dis max dis then max dis = dis
à = a update the currently furthest trajectory
end end
(u
end Return: V
The probability p(a) is assumed to be continuous with respect to a and insen- sitive small perturbations in a, i.e., lim (p(a) p(a + Aa))= = 0. Inspired by NeRF (Mildenhall al. 2020,, which mod- els the continuous radiance field over the 5D space, we resort probabilistic field model the continuous mapping from the action space the probabilistic distribution.
calculate distance will calculate the distance be- tween the endpoint of action and the endpoints of all actions in set V. It will return the minimum value among these distances as the result.
calculate_distancee
Concretely, we first encode each action (trajectory waypoint) into a high-dimensional planning token embedding E(a),
E(a) = (T(x,), I())) r(pos) = (7(pos, j)) o rposs, j) = (oo(poss/1000025//1) sin(pos (ooppsss10000=3/1)))
(2)
an encoding function that maps each coordinate from into a high dimensional embedding space 2R and is applied separately each of the coordinate values of trajectory a. pos denotes the
5

---

Published as a conference paper at ICLR 2026
position (referring to the y coordinate of waypoint). We these functions to map continuous coordinates into a higher dimensional space to better approximate a higher frequency field function.
Then, cascaded Transformer decoder ó interacts with scene information Eseene and, combined with navigation Eavii and ego state Estates predicts the probability of each action,
p(a) (MLLL((EE(())) Esceno) + Enavi + Estate)).
a is the sigmoid function. In the Transformer decoder Φ. E((a) serves as query, and Escene serves as key and value. E(a), Fuavi, Estate, and the output of MLP are with the same dimension (1 D).
3.3 TRAINING
We train VADv2 with three kinds of supervision, distribution loss, conflict loss, and scene token loss.
Distribution Loss. We learn the probabilistic distribution from large-scale driving demonstra- tions. KL divergence is used to minimize the difference between the predicted distribution and the distribution of the data:
Prlata (a)
Edistribution DEL (Pdatal (Ppred)
Σ Pdata (a) log
Ppred (a)
Pdata (a) is estimated through occurrence frequency in demonstrations. Since (a)ta(a) is fixed, Pdata (a) log Pdata (a) is a constant and can be omitted. Therefore, minimizing KL divergence is
equivalent to optimizing the cross-entropy loss:
Σ (a) log Pprea (a). a
Ldistribution =
(5)
For each frame the demonstrations, we select the action from the planning vocabulary that has the lowest L2 distance to the ground-truth action. This best-matched action is assigned a label of 1, and all other actions are assigned Over all the frames, the occurrence frequency Paata (a) of one action is then estimated by counting how often each action is the best match, normalized by the total number of frames. This modeling is analogous to the standard formulation used in large language models, where the ground-truth token is labeled 1 and others O, and cross-entropy loss used to minimize the divergence between the predicted distribution and the empirical distribution.
Conflict Loss. Driving scene constraints are used to help model learn important driving priors and regularize the predicted action distribution. Specifically, if an action in the planning vocabulary conflicts with other agents' ground truth future motion road boundaries, it is treated negative sample, and a corresponding loss is applied to reduce its probability,
Leouflict
Iconiit (a) log Ppred (are((() (a). a
Σ
(6)
conftli (a) is the indicator function, whose value is if conflict happens to a, otherwise is 0.
Scene Token Loss. Map, agent, and traffic-element tokens are supervised with corresponding supervision signals to ensure they explicitly encode the relevant high-level information.
The loss of map tokens the same with Map (Liao et al. 2023b). loss adopted to calculate the regression loss between the predicted map points and the ground truth map points. Focal loss is used as the map classification loss.
The loss of agent tokens is composed of the detection loss and the motion prediction loss (Jiang et al., 2023). 1 loss is used as the regression loss to predict agent attributes (location, orientation, size, etc.), and focal loss to predict agent classes. For each agent who has matched with ground truth agent, we predict K future trajectories and use the trajectory that has the minimum final displacement error (minFDE) representative prediction. Then calculate / loss between this representative

---

Published as a conference paper at ICLR 2026
Table 1: Closed-loop evaluation on the Town05 Long benchmark.
Method
Reference
Modality
DS 1
RC 1
IS
Transfuser PPrakash et al. 2021b 2P
C+L
31.0
47.5
0.77
ThinkTwice (Jia et 2023b
CVPR'23
C+L
70.9
95.5
0.75
DrieAAdpter+ et al.
2023a
ICCV'23
C+L
71.9
97.3
0.74
DriveMLM (Wang
2023b
arXiv'23
C+L
76.1
98.1
0.78
Roach Zhang al.
2002
ICCV'21
C
41.6
96.4
0.43
ST-P3 Huu al.
2022b
ECCV'22
C
11.5
83.2
MILE (Hu et al.
2022a
NeurIPS'22
C
61.1
97.4
0.63
Interfuser (Shao
2023
CoRL'22
C
68.3
95.0
VAD (Jing et al
20233
ICCV'23
C
30.3
75.2
Rao et al. 2024
TIV'24
C
74.9
94.6
0.77
DrieCoo Wang al.
2024
arXiv'24
C
73.6
96.8
0.76
LcapVAD
Ma ct al. 2025
arXiv '25
C
73.7
95.7
0.78
VADv2
Ours
C
85.1
98.4
0.87
Table 2: End-to-end planning results on the NAVSIM navtest split with closed-loop metrics.
Method
Reference
Modality
NC
DAC
TTC f Comf
EP
PDMS
UniAD Hu al. al. 2022c
CVPR 23
C
97.8
91.9
92.9
100
78.8
83.4
Transfuser (Prakash et
2021b
PAM1'23
C+1.
97.7
92.8
92.8
100
79.2
84.0
PARA-Drive Weng et al.
2024
CVPR 24
C
97.9
92.4
93.0
99.8
79.3
84.0
GualFlow Xing ct al. 2025
CVPR'25
C+L
98.3
93.8
94.3
100
79.8
85.7
Hydra-MDP++
2025a
arXiv'25
97.6
96.0
93.1
100
80.4
86.6
DiffusionDrrvve [Laoo
2025
CVPR'25
C+L
98.2
96.2
94.7
100
82.2
88.1
WoTE et 20255
ICCV'25
C+I.
98.5
96.8
94.9
81.9
88.3
Hydra-NeXI 2025c
arXiv'25
C+L
98.1
94.6
81.8
VADv2
98.3
97.4
95.7
82.3
trajectory and the ground truth trajectory as the motion regression loss. Besides, focal loss adopted as the multi-modal motion classification loss.
Traffic element tokens consist of parts: the traffic light token and the stop sign token. On one hand, we send the traffic light token to an MLP to predict the state of the traffic light (yellow, red, and green) and whether the traffic light affects the ego vehicle. On the other hand, the stop sign token is also sent to an MLP to predict the overlap between the stop sign area and the ego vehicle. Focal loss used to supervise these predictions. The final loss can be denoted as:
Ldistribution Ceonttiet + Ltoken
3.4 INFERENCE
In closed-loop inference, it's flexible get the driving policy from the distribution. Intuitively, sample the action with the highest probability at each time step, and use the PID controller to convert the selected action control signals (steer, throttle, and brake).
In real-world applications, there are more robust strategies to make full use the probabilistic distribution. A good practice is, sampling top-K actions proposals, and adopting rule-based wrapper for filtering proposals and an optimization-base post-solver for fine-grained trajectory refinement. Besides, the probability of the action reflects how confident the end-to-end model is, and can be regarded as the judgment condition to switch between conventional rule-based planning and control and learning baannd planning and control.
EXPERIMENTS
4.1 EXPERIMENTAL SETTINGS
CARLA Benchmark. We first use CARLA (Dosovitskiy et al. 2017) simulator to evaluate the performance of VADv2. We conduct closed-loop evaluation the widely adopted Town05 and
7

---

Published as a conference paper at ICLR 2026
Table End-to-end planning results on the NAVSIMv2 benchmark with Extended Metrics.
Method
NC f DAC
DDC 1
TL t
EP1
TTC
LK
HC 1
EC t
EPDMS
Transfuser
96.9
89.9
97.8
99.7
87.1
95.4
92.7
98.3
87.2
76.7
HydraMDP++
97.2
97.5
99.4
99.6
83.1
96.5
94.4
98.2
70.9
81.4
PRIX
98.0
95.6
99.5
99.8
87.4
97.2
97.1
98.3
87.6
84.2
VADv2
98.0
98.3
99.4
99.8
87.1
96.8
95.2
98.3
88.1
85.8
Table 4: Closed-loop quantitative comparisons with other methods the 3DGS- benchmark.
Method
Reference
CR
DCRL
SCR|
DR1
PDR
HDR
ADD|
TransFuser
TPAMI 22
0.320
0.273
0.047
0.235
0.188
0.047
0.263
VAD
ICCV 23
0.335
0.273
0.062
0.314
0.255
0.059
0.304
GenAD
ECCV 24
0.341
0.299
0.042
0.291
0.160
0.131
0.265
VADv2
Ours
0.270
0.240
0.030
0.243
0.139
0.104
0.273
Bench2Drive Jia et al. (024) benchmarks. Specifically, each benchmark contains several pre-defined driving routes. The simulation and control frequency for dlosedllopp inference 10 Hz. VADv2 takes a multi-view image sequence as input in streaming manner and plans a 3-second future trajectory. The trajectory consists of waypoints (i.e., T 6). The time interval between two adjacent waypoints is 0.5s. The default feature dimension D for VADv2 is set to 256. All experiments are conducted based on 16 NVIDIA 4090 GPUs.
CARLA Data. As for generating the driving demonstration data for the Town05 benchmark, we use the official autonomous agent of CARLA collect training data by randomly generating driving routes in Town03, Town04, Town06, Town07, and Town10. We collect approximately 3 million clips for training. For each clip, we save 6-camera surround-view image sequences at 10Hz for past 1.6 seconds, along with information on traffic signals, traffic participants, and the state of the ego vehicle. Additionally, we obtain the vectorized maps for training the online mapping module by preprocessing the OpenStreetMap (Haklay & Weber 20088) format maps provided by CARLA. The maps are provided only as ground truth during training, and VADv2 does not use high-definition maps for evaluation.
NAVSIM and 3DGS-based Benchmarks. further validate generalization ability real-world scenarios, we also evaluate VADv2 on the NAVSIM, NAVSIMv2 (Dauner et al.) (024) and large-scale 3DGS-based (Kerbl et al., 2023) benchmarks. we collect 2000 hours of real-world human driving demonstrations for training, and utilizing 337 reconstructed 3D Gaussian Splatting (3DGS) environments for closed-loop evaluation. Each environment features an 8s scenario capturing interactions in dense traffic with potential collision risks, providing representative segment of real-world driving behaviors and multi-agent interactions.
The photorealistic 3DGS reconstruction enables accurate agent trajectory modeling and dynamic environment rendering, providing a testbed that closely mirrors real-world driving conditions. More details of the 3DGS-based benchmark can be found in Appendix [A]uue page limits. We also deploy VADv2 on real-world vehicles. The results are presented in the supplementary material.
4.2 METRICS
On the CARLA benchmark, we employ its official closed-loop metrics, Route Completion (RC), Infraction Score (IS) and Driving Score (DS). DS is the product between RC and IS, which serves as the main metric. In benchmark evaluation, most works adopt rule-based wrapper to reduce the infraction. For a fair comparison, we follow the common practice of adopting a rule-based wrapper over the learning-based policy, which is similar to Transfuser (Prakash al. 2021b), We also conduct open-loop evaluation using the L2 displacement error and collision rate. most ablations, we adopt open-loop metrics by default because they faster to evaluate and more stable. We use the official autonomous agent of CARLA to generate the validation set on the Town05 Long benchmark for open-loop evaluation, and the results averaged over all validation samples.
8

---

Published as a conference paper at ICLR 2026
Table Closed-loop results the Bench2Drive.
Table Ablation on the multi-modal outputs.
Method
DS
SR
Effi 1
Comf f
Traj.
NC
DAC 1
TTC 1
Coml
EP
PDMS
SpivseDrive
44.54
16.71
170.21
48.63
Top1
98.3
97.4
95.7
100
82.3
89.3
MomAD
47.91
18.11
174.91
51.20
Top2
98.2
97.3
94.8
100
82.2
89.1
DriveTransformer
63.46
35.01
100.64
20.78
Top3
98.0
96.7
94.6
99.8
82.0
83.3
RTA
74.33
48.33
186.04
25.77
Top4
97.8
96.2
93.2
81.7
87.6
VADv2
76.15
50.46
178.24
37.81
Top5
97.4
96.1
93.5
99.7
81.4
87.5
Table Ablations on design choices. "Dist.": Distribution; "Traf. Token": Traffic Element Token.
Dist. Loss
Conflict Loss
Agent Token
Map Token
Traf. Token
Image Token
L2 (m) 4
Collision (%)
4
ID
Is
2s
3s
1s
2s
3s
1
/
V
1.415
2.310
3.153
0.698
0.755
0.746
2
/
/
V
V
0,086
0.173
0.291
0.000
0.012
0.039
/
d
/
V
0.089
0.190
0.327
0.015
0.047
0.085
V
/
d
/
0.086
0.191
0.332
0.005
0.034
0.070
/
0.082
0.171
0.295
0.000
0.017
0.051
/
0.083
0.170
0.000
0.010
0.039
7
0.082
0.169
0.290
0.000
0.010
0.039
On the NAVSIM benchmark, its official closed-loop metrics such as PDMS are adopted. On the 3DGS-based benchmark, Collision Ratio (CR, sum of Dynamic and Static Collision Ratios) evaluates interaction safety in dense traffic, while Deviation Ratio (DR, combining Positional and Heading Deviation Ratios) and Average Deviation Distance (ADD) jointly assess trajectory consistency with expert human demonstrations.
safety-critical metrics grounded in real-world driving analytics:
4.3 COMPARISONS WITH STATE-OF-THE-AR METHODS
On the Town05 Long benchmark in Table 1 VADv2 achieves DS of 85.1, RC of 98.4, 48.4, and an IS of 0.87. Relative to the former ofate-of-the-arr method DriveMLMM (Wang et al.) 2023b). VADv2 achieves a higher RC while significantly improving DS by 9.0. It is worth noting that VADv2 only utilizes cameras as perception input, whereas DriveMLM uses both cameras and LiDAR. Furthermore, compared to the previous best camera-base based method (Rao et al. 2024), VADv2 demonstrates greater advantages, with a remarkable increase in DS of up to 10.2. On the Bench2Drive benchmark (Table 5) VADv2 also achieves the highest Drive Score of 76.15.
Besides the CARLA based benchmarks, VADv2 also achieves state-of of--o----hhh-----ee planning performance on the NAVSIM and NAVSIMv2 benchmarks, as shown in Table and Table Additionally, Table presents the results on the 3DGS-based benchmark. VADv2 reduces the Collision Ratio to 0.270, 15.6% improvement
TransFuser (0.320), while maintaining competitive Deviation Ratio of 0.243 that approaches the best reported performance. These results demonstrate robust safety in real-world dynamic interactions.
4.4 ABLATION STUDY
Multimodal Planning Performance. We further evaluate the multi-modal planning performance of VADv2. Beyond the as shown in T Results indicate that the top-1 trajectory achieves the best overall performance, and the performance of other candidates is basically comparable to the top-1, demonstrating that VADv2 can generate high-quality multi-modal outputs.
trajectory, consider other candidates within the top-5 set,
Key Modules. Table 7 shows the ablation experiments of the key modules in VADv2. 50k clips of driving demonstrations are used in training. The model performs poorly terms of planning accuracy without the supervision of expert driving behavior provided by the Distribution Loss (ID 1). The Conflict Loss provides critical prior information about driving; therefore, removing it (ID 2) also affects the model's planning aceuracy. Scene tokens encode important high-ding high-dimensional features, and the planning tokens interact with the scene tokens to learn both dynamic and static information about the driving scene. When any type of scene token is missing,
elements into

---

Published as a conference paper at ICLR 2026
Table 8: Ablation on the performance under different planning manners and traffic densities.
Planning Manner
Traffic Density
NC1 f
DAC1 1
TTC f
Comf f
EP
PDMS t
Deterministic
Low
98.1
97.4
95.2
99.9
83.2
89.4
Probabilistic
Low
98.3
99.0
95.5
100
83.5
90.6
Deterministic
Medium
97.8
96.3
94.8
100
81.9
87.5
Probabilistic
Medium
98.4
96.8
95.4
100
82.3
89.0
Deterministic
High
97.3
95.1
93.6
100
79.8
85.8
Probabilistic
High
98.0
97.4
94.3
100
82.0
87.7
GALRIHT
OMLMOOT
Figure 4: Qualitative results of VADv2 on the CARLA Town05 Long benchmark.
the model's planning performance will be affected (ID 3-ID 6). The best planning performance is achieved when the model incorporates all of the aforementioned designs (ID 7).
Planning Robustness. Based on the number of dynamic agents within 20 m the ego vehicle, we categorize traffic density Low (<5), Medium (5-10), and High (>10), and evaluate performance under different planning manners and density levels (Table 8). For deterministic planning, we modify MI.P that directly regress future trajectories, following common
the planning head of VADv2 practice (Jiang et al. 2023). Results show that probabilistic planning consistently outperforms its deterministic counterpart. Furthermore, while probabilistic planning maintains stable performance across varying density scenarios, deterministic planning exhibits noticeable degradation. These findings highlight the effectiveness and robustness of our probabilistic planning paradigm. Additional experiments and ablations are provided in Appendix A due to page limits.
4.5
VISUALIZATION
Figure presents qualitative results of VADv2. The top left image shows multi-modal planning trajectories at different driving speeds. The top right shows both forward creeping and left lane- changing. The bottom left shows a right-lane change scenario at an intersection, where both temporary straight and immediate lane-change trajectories are predicted. The bottom right depicts lane-change with vehicle in the target lane, and VADv2 generates multiple reasonable trajectories.
5 CONCLUSIONS AND LIMITATIONS
In this work, we introduce VADv2, an end-to-end driving model based on probabilistic planning. It runs stably the CARLA simulator and achieves state-of-the-art closed-loop performance, signifi- cantly outperforming existing methods. Comprchensive experiments
NAVSIM and the 3DGS- based benchmark further demonstrate its effectiveness and robustness in complex driving scenarios. The feasibility of this probabilistic planning paradigm is primarily validated.
Currently, both simulator and 3DGS-based closed-lood environments still present limitations, such as naive agent behaviors and insufficient scene realism, which may restrict the performance of VADv2. In future work, we plan to explore how large-scale expert driving data can be leveraged to further enhance planning performance and bridge the gap between simulation and real-world deployment.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported by the National Natural Science Foundation of China (NSFC No. 62276108).
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, lige Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et Gpt-4 technical report. arXiv preprint 1.Xi::2303.08774,, 2023.
Raphael Chekroun, Marin Toromanoff, Sascha Hornauer, and Fabien Moutarde. Gri: General reinforced imitation and its application to vision-based autonomous driving. arXiv preprint
arXiv:2111.08575. 2021.
Long Chen, Oleg Sinavski, Jan Hünermann, Alice Karnsund, Andrew James Willmott, Danny Birch, Daniel Maund, and Jamie Shotton. Driving with Ilms: Fusing object-level vector modality for explainable autonomous driving. arXiv preprint 2rx.0:2310.01997, 2023.
Felipe Codevilla, Eder Santana, Antonio M López, and Adrien Gaidon. Exploring the limitations of behavior cloning for autonomous driving. In ICCV, 2019.
Daniel Dauner, Marcel Hallgarten, Tianyu Li, Xinshuo Weng, Zhiyu Huang, Zetong Yang, Hongyang Li, Igor Gilitschenski, Boris Ivanovic, Marco Pavone, et al. Navsim: Data-driven non-reactive autonomous vehicle simulation and benchmarking. NeurlPS, 2024.
Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. Carla: An open urban driving simulator. In CoRL, 2017.
Hao Gao, Shaoyu Chen, Bo Jiang, Bencheng Liao, Yiang Shi, Xiaoyang Guo, Yuechuan Pu, Haoran Yin, Xiangyu Li, Xinbang Zhang, al. Rad: Training end-to-end driving policy via large-scale 3dgs-based reinforcement learning. arXiv preprint arXiv:2502.13144 131444 2025.
Jiyang Gao, Chen Sun, Hang Zhao, Yi Shen, Dragomir Anguelov, Congcong Li, and Cordelia Schmid. Vectornet: Encoding hd maps and agent dynamics from vectorized representation. In CVPR, 2020.
Junru Gu, Chenxu Hu, Tianyuan Zhang, Xuanyao Chen, Yilun Wang, Yue Wang, and Hang Zhao. Vip3d: End-to-end visual trajectory prediction via 3d agent queries. arXiv preprint arXiv:2208.01 588,, 2022.
Mordechai Haklay and Patrick Weber. Openstreetmap: User-generated street maps. IEEE Pervasive computing, 2008
Anthony Hu, Zak Murez, Nikhil Mohan, Soffa Dudas, Jeffrey Hawke, Vijay Badrinarayanan, Roberto Cipolla, and Alex Kendall. Fiery: Future instance prediction in bird's-eye view from surround monocular cameras. In ICCV, 2021.
Anthony Hu, Gianluca Corrado, Nicolas Griffiths, Zak Murez, Corina Gurau, Hudson Yeo, Alex Kendall, Roberto Cipolla, and Jamie Shotton. Model-based imitation learning for urban driving. In Advances Neural Information Processing Systems (NeurIPS), 2022a.
Shengchao Hu, Li Chen, Penghao Wu, Hongyang Li, Junchi Yan, and Dacheng Tao. St-p3: End-to- end vision-based autonomous driving via spatial-temporal feature learning. In ECCV, 2022b.
Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, et al. Planning-oriented autonomous driving. CVPR2023, 2022c.
Xiaosong Jia, Yulu Gao, Li Chen, Junchi Yan, Patrick Langechuan Liu, and Hongyang Li. Driveadapter: Breaking the coupling barrier of perception and planning in end-to-end autonomous driving. 2023a.
Xiaosong Jia, Penghao Wu, Li Chen, Jiangwei Xie, Conghui He, Junchi Yan, and Hongyang Li. Think twice before driving: Towards scalable decoders for end-to-end autonomous driving. In CVPR, 2023b.
11

---

Published as a conference paper at ICLR 2026
Xiaosong Jia, Zhenjie Yang, Qifeng Li, Zhiyuan Zhang, and Junchi Yan. Bench2drive: Towards multi-ability benchmarking of closed-loop end-to-end autonomous driving. In NeurIPS, 2024.
Bo Jiang, Shaoyu Chen, Xinggang Wang, Bencheng Liao, Tianheng Cheng, Jiajie Chen, Helong Zhou, Qian Zhang, Wenyu Liu, and Chang Huang. Perceive, interact, predict: Learning dynamic and static clues for end-to-end motion prediction. arXiv preprini arXiv:22 2022.
Bo Jiang, Shaoyu Chen, Qing Xu, Bencheng Liao, Jiajie Chen, Helong Zhou, Qian Zhang, Wenyu Liu, Chang Huang, and Xinggang Wang. Vad: Vectorized scene representation for efficient autonomous driving. ICCV, 2023.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Gruph., 42(4))139-1, 2023.
Kailin Li, Zhenxin Li, Shiyi Lan, Yuan Xie, Zhizhong Zhang, Jiayi Liu, Zuxuan Wu, Zhiding Yu, and Jose M Alvarez. Hydra-mdp++: Advancing end-to-end driving via expert-guided hydra-distillation. arxiv preprint arXiv:2503.12820, 2025a.
Qi Li, Yue Wang, Yilun Wang, and Hang Zhao. Hdmapnet: An online hd map construction and evaluation framework. In ICRA, 2022a.
Yingyan Li, Yuqi Wang, Yang Liu, Jiawei He, Lue Fan, and Zhaoxiang Zhang. End-to-end driving with online trajectory evaluation via bev world model. In ICCV, 2025b.
Yinhao Li, Zheng Ge, Guanyi Yu, Jinrong Yang, Zengran Wang. Yukang Shi, Jianjian Sun, and Zeming Li. Bevdepth: Acquisition of reliable depth for multi-view 3d object detection. arXiv preprint ArXi:22206. 10092, 2022b.
Zhenxin Li, Shihao Wang. Shiyi Lan, Zhiding Yu, Zuxuan Wu, and Jose M Alvarez. Hydra-next: Robust closed-loop driving with open-loop training. arXiv preprint arXiv:2503. /2030, 2025c.
Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chonghao Sima, Tong Lu, Qiao Yu, and Jifeng Dai. Bevformer: Learning bird's s-eye-view representation from multi-camera images via spatiotemporal transformers. arXiv preprint arXiv:2203. 2022c
Bencheng Liao, Shaoyu Chen, Xinggang Wang, Tianheng Cheng. Qian Zhang, Wenyu Liu, and Chang Huang. Maptr: Structured modeling and learning for online vectorized hd map construction. arXiv preprint 2208 /4437, 2022.
Bencheng Liao, Shaoyu Chen, Bo Jiang, Tianheng Cheng, Qian Zhang, Wenyu Liu, Chang Huang, and Xinggang Wang. Lane graph path: Continuity-preservnn path-wise modeling for online lane graph construction. arXiv preprint urXiv:2303.0881/ 2023a.
Bencheng Liao, Shaoyu Chen, Yunchi Zhang, Bo Jiang, Qian Zhang, Wenyu Liu, Chang Huang, and Xinggang Wang- Maptrv2: end-to-end framework for online vectorized hd map construction. arXiv preprint 230.v:2308.0577:8 2023b.
Bencheng Liao, Shaoyu Chen, Haoran Yin, Bo Jiang, Cheng Wang, Sixu Yan, Xinbang Zhang, Xiangyu Li, Ying Zhang, Qian Zhang, et al. Diffusiondrive: Truncated diffussin model for end-to-end autonomous driving. CVPR, 2025.
Yicheng Liu, Jinghuai Zhang, Liangji Fang, Qinhong Jiang, and Bolci Zhou. Multimodal motion prediction with stacked transformers. In CVPR, 2021.
Yicheng Liu, Yue Wang, Yilun Wang, and Hang Zhao. Vectormapnet: End-to-end vectorized hd map learning. arXiv preprint arXiv:2206.0800 2022.
Yukai Ma, Tiantian Wei, Naiting Zhong, Jianbiao Mei, Tao Hu, Licheng Wen, Xuemeng Yang, Botian Shi, and Yong Liu. Leapvad: A leap in autonomous driving via cognitive perception and dual-process thinking. arXiv preprint arXiv:2501.0816. 2025.
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes neural radiance fields for view synthesis. ECCV, 2020.
12

---

Published as a conference paper at ICLR 2026
Jonah Philion and Sanja Fidler. Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d. In ECCV, 2020.
Jonah Philion, Xue Bin Peng, and Sanja Fidler. Trajeglish: Traffic modeling as next-token prediction. arXiv preprint arXiv:23/2.04535, 2023.
Aditya Prakash, Kashyap Chitta, and Andreas Geiger. Multi-modal fusion transformer for end-to-end autonomous driving. In CVPR, 2021a.
Aditya Prakash, Kashyap Chitta, and Andreas Geiger. Multi-modal fusion transformer for end-to-end autonomous driving. 2021b.
Zhongyu Rao, Yingfeng Cai, Hai Wang, Yubo Lian, Yilin Zhong, Long Chen, and Yicheng Li.
Enhancing autonomous driving: A low-cost monocular end-to-end framework with multi-task integration and temporal fusion. IEEE Transactions on Intelligent Vehiclex, 2024.
Ari Seff, Brian Cera, Dian Chen, Mason Ng, Aurick Zhou, Nigamaa Nayakanti, Khaled Refaat, Rami Al-Rfou, and Benjamin Sapp. Motionlm: Multi-agent motion forecasting as language modeling. In ICCV, 2023.
Hao Sha, Yao Mu, Yuxuan Jiang, Li Chen, Chenfeng Xu, Ping Luo, Shengbo Eben Li, Masayoshi Tomizuka, Wei Zhan, and Mingyu Ding. Languagempe: Large language models as decision makers for autonomous driving. arXiv preprint arXiv:2310.03026, 2023.
Hao Shao, Letian Wang, Ruobing Chen, Hongsheng Li, and Yu Liu. Safety-enhanced autonomous driving using interpretable sensor fusion transformer. In Conference on Robot Learning, pp. 726-737. PMLR, 2023.
Chonghao Sima, Katrin Renz, Kashyap Chitta, Li Chen, Hanxue Zhang, Chengen Xie, Jens Beißwenger, Ping Luo, Andreas Geiger, and Hongyang Li. Drivelm: Driving with graph vi- sual question answering. In ECCV, 2024.
Shihao Wang, Yingfei Liu, Tiancai Wang, Ying Li, and Xiangyu Zhang. Exploring oeject-centric temporal modeling for efficient multi-view 3d object detection. arXiv preprint arXiv:2303.11926 2023a.
Tianqi Wang, Enze Xie, Ruihang Chu, Zhenguo Li, and Ping Luo. Drivecot: Integrating chain-of- thought reasoning with end-to-end driving. arXiv preprint arXiv:2403. 16996, 2024.
Wenhai Wang, Jiangwei Xie, YauanYang Yang Hu, Haoming Zou, Jianan Fan, Wenwen Tong, Yang Wen, Silei Wu, Hanming Deng, Zhiqi Li, al. Drivemlm: Aligning multi-modal large language models with behavioral planning states for autonomous driving, arXiv preprint arXiv:2312.091) 2023b.
Xinshuo Weng, Boris Ivanovic, Yan Wang, Yue Wang. and Marco Pavone. Para-drive: Parallelized architecture for real-time autonomous driving. In CVPR, 2024.
Zebin Xing, Xingyu Zhang, Yang Hu, Bo Jiang, Tong He, Qian Zhang, Xiaoxiao Long, and Wei Yin. Goalflow: Goal-driven flow matching for multimodal trajectories generation in end-to-end autonomous driving. In CVPR, 2025.
Zhenhua Xu, Yujia Zhang, Enze Xie, Zhen Zhao, Yong Guo, Kenneth KY Wong, Zhenguo Li, and Hengshuang Zhao. Drivegpt4: Interpretable end-to-end autonomous driving via large language model. arXiv preprint arXiv:2310.014: 2023.
Yunpeng Zhang, Zheng Zhu, Wenzhao Zheng, Junjie Huang, Guan Huang, Jie Zhou, and Jiwen Lu. Beverse: Unified perception and prediction in birds-eye-view for vision-centrie autonomous driving. arXiv preprint arXiv:2205.0974 2022.
Zhejun Zhang, Alexander Liniger, Dengxin Dai, Fisher Yu, and Lue Van Gool. End-to-end urban driving by imitating a reinforcement learning coach. 2021.
Yinan Zheng, Ruiming Liang, Kexin Zheng, Jinliang Zheng, Liyuan Mao, Jianxiong Li, Weihao Gu, Rui Ai, Shengbo Eben Li, Xianyuan Zhan, et al. Diffusion-bass based planning for autonomous driving with flexible guidance. arXiv preprint 2rXiv:2501.1555 15564, 2025.
13

---

Published as a conference paper at ICLR 2026
Table 9: Closed-loop results on Town05 Short.
Table 10: Ablation vocabulary size.
Method
Modality
DS
RC
(m)
Collision (5) |
Size
CILRS
C
1s
1s
2s
LBC
C
31.0
55.0
256
0.110
0.207
0.337
0.000
0.019
0.057
Transfuser
C+L
54.5
78.4
512
0.099
0.189
0.313
0.000
0.022
0.045
ST-P3
C
55.1
86.7
1024
0.093
0.175
0.293
0.000
0.020
0.044
VAD
C
64.3
2048
0.088
0.173
0.294
0.000
0.017
0.041
VAD
C.
88.2
0.082
0.290
0.000
0.010
0.039
VADv2
C
89.7
93.0
Table 11: Ablation on training clips.
Table 12: Ablation planning manners.
1.2 (m) 1
Collision (Si) |.
Amount
1s
2s
1s
2s
Planning
DS f
RC f
L2(m)
Collision(%)
@3s.
@3s.
0.121
0.264
0.461
0.015
0,061
0.107
3 10
0.082
0.169
0.290
0.000
0.010
0.039
Deterministic
74.6
95.1
0.223
0.006
x 10"
0.073
0.153
0.267
0.000
0.008
0.027
Probabilistic
85.1
98.4
0.225
0.007
x
109
0.072
0.133
0.225
0.000
0.000
0.007
A
APPENDIX
A.1 CLOSED-LOOPP RESULTS ON THE TOWNO5 SHORT BENCHMARK
We summarize the Town05 Short benchmark results of VADv2 in Table 9 This benchmark evaluates targeted driving behaviors, including lane changes in dense traffic and before intersections. VADv2 achieves the highest DS score, while LeapVAD exhibits higher RC but lower DS, suggesting more infractions. Overall, the results highlight the strong and reliable driving capability of VADv2 in challenging scenarios.
A.2 MORE ABLATION STUDY
Vocabulary Size. We ablate about the vocabulary size in Table 10 increasing, both L2 and collision metrics become better. larger vocabulary size can better represent the action space with less discretization error.
With the vocabulary size
Amount of Training Clips. Table 11 the ablation experiments about the amount of clips of driving demonstrations used for training the end-to-end model. As expected, the model achieves better 1.2 and collision metrics with the data amount increasing.
Probabilistic vs. Deterministic. Ablation results on the Town05 Long benchmark (Table 12) show that deterministic and probabilistic planning perform similarly in open-loop evaluation. However, in closed-loop settings, probabilistic planning achieves notably better stability and performance, while deterministic planning struggles with planning uncertainty.
we report discrete errors and planning performance for different vocabulary sampling strategies. For each training sample in NAVSIM, we select the vocabulary
Vocabulary Sampling. Table 13
Table 13: Ablation on the discretization error and performance of vocabulary sampling strategies.
Strategy
Avg. L2 |
Max L2 $
NC 1
DAC 1
TTC 1
Comf
EP 1
PDMS 1
k-means
0.132
0.217
97.9
97.2
94.5
100
81.9
89.0
K-disks
0.128
0.204
98.1
97.3
94.8
100
82.2
89.1
nuScenes
0.116
0.212
98.2
97.1
95.4
100
82.5
89.1
FTS
0.102
0.181
98.3
97.6
95.1
100
82.3
89.3
14

---

Published as a conference paper at ICLR 2026
Table 14: More detailed statistics of our real world 3DGS-bar baadeddddd validation dataset.
Figure Visualization of the planning vocabu- lary on the CARLA Town05 benchmark.
Scenario
Type
Percentage
Sunny
Weather
74.78%
Night & Rainy
Weather
25.22%
Crowded Road
Precise Behavior
6.23%
Narrow Road
Precise Behavior
6.82%
Interscction
Precise Behavior
38.58%
Cut-in
Interactive Scenario
9.79%
Ped. Crossing
Interactive Scenario
9.20%
Table 15: More evaluation details the 3DGS-based benchmark.
Method
Reference
Parameters
Latency
Collision (%)
Platform
TransFuser
TPAMI 22
0.36B
118ms
0.320
RTX 4090
VAD
ICCV 23
0.36B
1 118ms
0.335
RTX 4090
GenAD
ECCV 24
0.38B
121ms
0.341
RTX 4090
VADv2
Ours
0.40B
125ms
0.270
RTX 4090
trajectory that yields the minimum L2 distance and then measure the per-timestep L2 error. We derive both the Average and Max L2 Errors and average them across all samples.
The choice of strategy leads to only minor differences, with Furthest Trajectory Sampling (FTS) achieving the best action space coverage and strongest results. We also build the vocabulary by sampling trajectories from the nuScenes training set and evaluate on NAVSIM, where performance remains comparable, indicating that VADv2 can effectively generalize across scenarios.
A.3 MORE EVALUATION DETAILS OF THE 3DGS -BASED BENCHMARK
Table 14 showcases the diverse and challenging test scenarios in our dataset, enabling more robust evaluation. We also assess 3DGS-based reconstruction under different weather conditions, achieving PSNR (Peak Signis-to-Noiee Ratio) metrics of 29.5, 28.8, and 28.2 for sunny, rainy, and nighttime scenes, respectively, demonstrating leading level of performance.
Table reports the inference latency, model parameters, and hardware platforms of baseline methods like TransFuser and VAD. To fairly compare planning performance, we use the same perception back- bone across all methods. Thus, differences in latency mainly arise from the planning module design. While VADv2 adds some overhead with its planning vocabulary, latency remains comparable to other baselines and notably improves the primary collision rate metric.
We provide additional details of our real-world dataset and compare with popular benchmarks in Table 16 While nuScenes is dominated by straight-driving scenarios and NAVSIM features more turns but limited data, our dataset stands out both scale and scene diversity.
A.4 LLM USAGE
We only use LLMs check grammar and polish writing in this paper, and the authors take full responsibility for all content.
Table 16: Quantitative analysis of the real-world 3DGS-based dataset.
Dataset
Duration
Environment
Straight
Turning
nuScenes
5.55h
Urban
92.80%
7.20%
NAVSIM
120h
Urban
66.40%
33.60%
Ours
2000h
Urban, Highway, Rural
52.50%
47.50%
15


Reviews:
[{"review_id": "OeamxFywsE", "reviewer": "Reviewer_bZ8f", "summary": "The paper overall proposes a probabilistic-based end-to-end planner that achieves great results on several benchmarks, including a closed-loop one. The architecture is very similar to VAD, and the highlight is the use of a discrete action space. However, the author should perform a more systematic analysis on the choice of discretization and more informative ablation studies, given the high similarity to VAD. At this point, I tend to accept this paper for its comprehensive comparison with other methods and performance but I may change my rate based on the author's responses.", "questions": "see weakness", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This paper proposes a probabilistic-based end-to-end planner by using a discretized action space, which is neat.\n2. The writing is clear and easy to follow with illustrations.\n3. The performance is great and the comparison with other works is fair and solid with closed-loop validation.", "weaknesses": "1. I believe most of the performance gain comes from the use of discretized action vocabulary, the author uses furthest trajectory sampling to get the vocabulary. I'm wondering if the noise of the vocabulary can affect the result? If yes, then to what extent? I would like to see more experiments on that. Also, the pre-defined vocabulary can be seen as a prior, and it can be different across datasets. I want to know the zero-shot cross-dataset performance of your method.\n2. Since the vocabulary is very important, I would like to see more discussion on different ways of discretization, and its discretization error, a discussion on how this error would affect the performance would make your work more comprehensive.\n3. In terms of the loss function, since you have discretized the whole action space, why use cross-entropy instead of the KL-divergence?\n4. The ablation study on Tab. 4 is really hard to give useful information. For example, I want to know whether the conflict loss is useful; I can't find the exact setting that only w/ or w/o this loss. I strongly suggest the author make this up during the rebuttal. \n5. Visualize the discretized action vocabulary will make the reader better understand the ideology.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aAwCz7jdxI", "reviewer": "Reviewer_biv4", "summary": "This paper presents VADv2, an end-to-end vision-based autonomous driving model that introduces a probabilistic planning paradigm to handle uncertainty and non-deterministic behavior in human driving. Instead of regressing deterministic trajectories or control commands, VADv2 models the planning policy as a scene-conditioned stochastic process, predicting a probabilistic distribution over discretized action tokens (“planning vocabulary”). The system tokenizes both the scene (through BEV-based map, agent, traffic, and image tokens) and the action space, learning the probability field of feasible maneuvers via a KL-based distribution loss, conflict regularization, and scene supervision. Most probable trajectory are selected for control during inference, supporting both multi-modal planning and flexible rule-based refinement. Extensive experiments on CARLA Town05, NAVSIM, and a large-scale 3DGS testing showcasing strong performance, particularly improving safety (collision, deviation ratios) and stability in closed-loop tests.", "questions": "1. How sensitive is performance to the number and diversity of sampled trajectories (vocabulary size N)? Is there a principled way to balance coverage vs. redundancy?\n\n2. Have you evaluated whether the predicted probabilities are well-calibrated (e.g., reliability or ECE curves)? Does confidence correlate with actual driving success?\n\n3. How does stochastic sampling affect stability under closed-loop evaluation?\n\n4. The paper hints at integration between rule-based and learned planning based on probability confidence. How is this threshold determined and implemented in practice (e.g. HSD), and does it improve real-world safety?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. VADv2 introduces a well-motivated shift from deterministic to probabilistic planning, effectively addressing multi-modal and uncertain decision spaces, an enduring challenge in end-to-end driving.\n\n2. The model outperforms strong baselines on CARLA, NAVSIM, and 3DGS. The added 3DGS benchmark enhances credibility in real-world-like evaluations.\n\n3. The paper is well-organized and provides detailed architectural and experimental information, including ablations (vocabulary size, data scale, loss components) and insightful qualitative visualizations.", "weaknesses": "The study mainly varies vocabulary size and data scale, but omits analysis of probabilistic vs. deterministic stability across traffic densities, or uncertainty calibration (e.g., entropy of action distribution).\n\nWhile multi-modal outputs are demonstrated qualitatively, quantitative uncertainty analysis or calibration plots are missing. It is important for safety-critical evaluation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ePdaYBSQsL", "reviewer": "Reviewer_JaXb", "summary": "The paper proposes VADv2, a probabilistic planning model for end-to-end autonomous driving that discretizes the action space into planning tokens to better handle uncertainty. The method achieves state-of-the-art closed-loop results on CARLA Town05 and is further validated on the NAVSIM dataset and a large-scale 3DGS-based benchmark.", "questions": "N/A", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper is easy to read.\n\n2. Modeling the uncertainty of trajectory is important.\n\n3. The experiments are conducted on several benchmarks.", "weaknesses": "The main results on CARLA may be outdated; the paper should compare against state-of-the-art methods on more challenging benchmarks such as Bench2Drive [1].\n\nOn NAVSIM, the paper does not compare with the latest state-of-the-art algorithms.\n\nModeling trajectory uncertainty is important; however, there are potentially better approaches such as diffusion models [2,3] or flow matching models [4], which require further comparison and discussion.\n\n[1] Jia X, Yang Z, Li Q, et al. Bench2drive: Towards multi-ability benchmarking of closed-loop end-to-end autonomous driving[J]. Advances in Neural Information Processing Systems, 2024, 37: 819-844.\n\n[2] Liao B, Chen S, Yin H, et al. Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving[C]//Proceedings of the Computer Vision and Pattern Recognition Conference. 2025: 12037-12047.\n\n[3] Zheng Y, Liang R, Zheng K, et al. Diffusion-based planning for autonomous driving with flexible guidance[J]. arXiv preprint arXiv:2501.15564, 2025.\n\n[4] Xing Z, Zhang X, Hu Y, et al. Goalflow: Goal-driven flow matching for multimodal trajectories generation in end-to-end autonomous driving[C]//Proceedings of the Computer Vision and Pattern Recognition Conference. 2025: 1602-1611.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3UgrxYDU5r", "reviewer": "Reviewer_HtAi", "summary": "The paper introduces VADv2, a probabilistic planning model for end-to-end autonomous driving. The model tokenizes planning trajectories into discretize spaces with a vocabulary and output the probabilistic distribution of action.  Distribution loss, conflict loss and scene token loss are used for training. The model achieves state-of-the-art performance on CARLA Town05, NAVSIM and 3DGS-based benchmark.", "questions": "1. How about the performance of VADv2 on middle tasks, including detection, online mapping and motion prediction?\n2. Similar to ego planning, the future trajectories of other agents are also of uncertainty. However, the motion prediction of VADv2 is still in continues space with regression loss. Will conducting the task in discrete space improve the perfermance?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The motivation of probabilistic planning is clear.\n2. The model achieves state-of-the-art performance on both open-loop and closed-loop benchmarks.", "weaknesses": "1. The planning vocabulary plays a key role in proposed probabilistic planning. However, there lacks analysis on the quality of the vocabulary to support that the discrete action space formed by the vocubulary can well represent the continuous action space in real world, including qualitative results and statistical analysis.\n2. There are already several methods to construct vocubulary for trajectories in autonomous driving, such as K-disks sampling in Trajeglish [1] and uniform quantization in Motionlm [2]. However, there is not any comparation between the proposed planning vocabulary sampling method and existing methods.\n3. As mentioned in 2, the idea of predicting trajectories of ego or agents in discrete space with classification loss is not new in autonomous driving. The idea from motion tasks of [1] and [2] is easy to apply to end-to-end planning. Thus, the probabilistic planning lacks significant novelty as the major contribution of the paper.\n4. It is better to move the “LLM Usage” paragraph from “Experimental Settings” subsection to appendix.\n\n[1] Trajeglish: Traffic Modeling as Next-Token Prediction (ICLR 2024) \n\n[2] Motionlm: Multi-agent motion forecasting as language modeling (ICCV 2023)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)