ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving
Authors: Yongkang Li
Kaixin Xiong
Xiangyu Guo
Fang Li
Sixu Yan
Gangwei Xu
Lijun Zhou
Long Chen
Haiyang Sun
BING WANG
Kun Ma
Guang Chen
Hangjun Ye
Wenyu Liu
Xinggang Wang
Venue: ICLR 2026 Poster (2026)
Keywords: Autonomous Driving
Vision-Language Models
Reinforcement Learning
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Recent studies have explored leveraging the world knowledge and cognitive capabilities of Vision-Language Models (VLMs) to address the long-tail problem in end-to-end autonomous driving. However, existing methods typically formulate trajectory planning as a language modeling task, where physical actions are output in the language space, potentially leading to issues such as format-violating outputs, infeasible actions, and slow inference speeds. In this paper, we propose ReCogDrive, a novel **Re**inforced **Cog**nitive framework for end-to-end autonomous **Driv**ing, unifying driving understanding and planning by integrating an autoregressive model with a diffusion planner. First, to instill human driving cognition into the VLM, we introduce a hierarchical data pipeline that mimics the sequential cognitive process of human drivers through three stages: generation, refinement, and quality control. Building on this cognitive foundation, we then address the language-action mismatch by injecting the VLM's learned driving priors into a diffusion planner to efficiently generate continuous and stable trajectories. Furthermore, to enhance driving safety and reduce collisions, we introduce a Diffusion Group Relative Policy Optimization (DiffGRPO) stage, reinforcing the planner for enhanced safety and comfort. Extensive experiments on the NAVSIM and Bench2Drive benchmarks demonstrate that ReCogDrive achieves state-of-the-art performance. Additionally, qualitative results across diverse driving scenarios and DriveBench highlight the model's scene comprehension.

Published as a conference paper at ICLR 2026
RECOGDRIVE: A REINFORCED COGNITIVE FRAME- WORK FOR END-TO-END AUTONOMOUS DRIVING
Yongkang Li Kaixin Xiong²', Xiangyu Guo Fang Li, Sixu Yan', Gangwei Xu Lijun Zhou', Long Chen', Haiyang Sun't, i, Bing Wang", Kun Ma², Guang Chen', Hangjun Ye', Wenyu Liu', Xinggang Wang B
HHuzzonn University of Science and Technology Xiaami EV "[liyk, xgwang/ www.ehhuuuu....... edu (aunhaiyang]) (acang11aa exxi com
Model Data: ReCogDrive HuggingFace Collection
Code: ReCogDrive Github Repository
ABSTRACT
Recent studies have explored leveraging the world knowledge and cognitive capa- bilities of Vision-Language Models (VLMs) address the long-tail problem in end-to-end autonomous driving. However, existing methods typically formulate trajectory planning as a language modeling task, where physical actions out- put in the language space, potentially leading to issues such as format-violating outputs, infeasible actions, and slow inference speeds. In this paper, we propose ReCogDrive, a novel Reinforced Cognitive framework for end-to-end autonomous Driving, unifying driving understanding and planning by integrating autore- gressive model with a diffusion planner. First, to instill human driving cognition into the VLM, we introduce a hierarchical data pipeline that mimies the sequential cognitive process of human drivers through three stages: generation, refinement, and quality control. Building on this cognitive foundation, we then address the language-action mismatch by injecting the VLM's learned driving priors into a diffusion planner to efficiently generate continuous and stable trajectories. Further- more, to enhance driving safety and reduce collisions, we introduce Diffusion Group Relative Policy Optimization (DiffGRPO) stage, reinforcing the planner for enhanced safety and comfort. Extensive experiments Bench2Drive benchmarks demonstrate that ReCog Drive achieves state-of-the-art performance. Additionally, qualitative results across diverse driving scenarios and DriveBench highlight the model's 's scene comprehension.
the NAVSIM and
INTRODUCTION
Autonomous driving, which aims to predict a smooth, comfortable, and collision-free trajectory for vehicle, has seen significant advancements. Recent end-to-end autonomous driving systems (Jiang et al., 2023; Hu et al., 2023; Chen et al., 2024b; Hu al., 2022; Zhang et al., 2024a) unify the perception (Jiang et al., 2024b; Philion & Fidler, 2020; Zhang et al., 2023), prediction (Chai et al., 2019; Gu et al., 2023), and planning (Chitta et al., 2022; Liao al., 2024) modules into single pipeline for joint optimization, demonstrating impressive performance under open-loop evaluation. However, these systems often fail to generalize long-tail scenarios, where data is limited and the driving conditions deviate significantly from those encountered during training.
Recent research (Tian al., 2024; Jiang al., 2024a; Hwang al., 2024) addresses the long-tail challenge by introducing VLMs, which are pre-trained on large-scale internet datasets and exhibit strong generalization abilities along with rich world knowledge. Specifically, VLMs applications in autonomous driving can be categorized into two main approaches: (1) dual-system approaches (Jiang al., 2024a; Tian et al., 2024), where VLMs generate low-frequency trajectories or high-level commands to guide an end-to-end driving system; and (2) single-system approaches (Hwang et al.,
'This work was done when Yongkang Li was intern at Xiaomi EV. Equal contribution. Project leader. Corresponding author: xgwang@hust, edu. en.

---

Published as a conference paper at ICLR 2026
Large-scale Training Data
Model Architecture
Perception
"What is the next trajectory the ego car?
Weather: Cloudy
Traffic Gight:Green
Visual Instruction Data (COCO, Text-VOA
Road: Urban
Time: Daylight
Text Tokenizer
The driving scene is bustling urban intersction with multiple cars...
Encoder
Large-Language Model
Prediction
2.M Open- Sourced Driving Data
Ressoning
Action
Planning: Keep, Change Of Thought
turn right likely upcoming
The J.5. amo siing to the
the street.
Speed
Optionai
Planner
Planning
3.3H
Rw Coyyrrivee
The decision stop the red traffic light and braking cars.
0.911z
InlamMEL 3-25/--2111
752K Automatically Annotated G&As Data
0.31
Infemb 3.55/11)
FUMS
Figure 1: Overview of ReCogDrive. We present ReCogDrive, end-to-end autonomous driving system, which possesses rich driving priors and generates continuous, stable trajectories via a diffusion denoising process. ReCogDrive is capable of performing tasks spanning from low-level scene perception and motion prediction to high-level driving planning and decision making.
2024; Mao et al., 2023a; Xing et al., 2024; Mao et al., 2023b; Wang et al., 2024b; Bai et al., 2024; Zhang et al., 2024c; Zhao et al., 2025), where standalone VLMMs directly predict future trajectories and can be optimized in an end-to-end manner. Most of these approaches reformulate the motion planning task as language modeling problem, generating trajectories autoregressive prediction. These methods leverage the generalization of the models and chain-of- thought (Wei al., 2022) reasoning to enhance interpretability and reasoning in complex scenarios.
textual format through
Despite these advantages, directly applying pre-trained VLMs by reformulating trajectory planning as a text generation task for autonomous driving reveals three critical limitations: (1) Domain discrepancy of pre-trained knowledge. VI.Ms (Mao et al., 2023a;b) are trained on generic internet data that lacks nuanced knowledge required for driving, resulting in significant domain gap. (2) Modality mismatch in trajectory generation. The discrete language space of VLMs fundamentally conflicts with the continuous action space required for planning. This mismatch leads to practical issues, as the probabilistic nature of autoregressive decoding can lead not only physically infeasible trajectories but also to violatiii outputs that result in parsing errors. (3) Suboptimal policies from imitation learning. The heavy reliance behavior cloning causes models (Hwang al., 2024) to converge to a suboptimal policy that struggles in rare scenarios and can be unsafe.
To address these challenges, propose ReCogDrive, novel end-to-end autonomous driving system that integrates the cognitive reasoning of Vision-Language Models (VLMs) with a reinforcement learning-enhancee
diffusion planner. Our framework introduces three key innovations: First, to bridge the domain gap, we design hierarchical data pipeline that, through three stages of generation, refinement, and quality control, constructs a large-scale VQA dataset to instill human-like cognitive priors into the VLM. Second, to resolve the modality mismatch, we design a diffusion planner that effectively translates the VLM's latent cognitive representations into continuous and stable driving trajectories. Finally, to overcome the limitations of imitation learning, we introduce a reinforcement learning stage based on Diffusion Group Relative Policy Optimization (DiffGRPO), enabling the planner to explore safer, more optimal behaviors beyond the expert dataset.
extensively evaluate ReCogDrive on NAVSIM and Bench2Drive, achieving new state-of-the-an performance in both open-loop and closed-loop settings. Additionally, test ReCogDrive-VLM. on the DriveLM and DriveBench benchmarks, demonstrating its superiority over both closed-source generalist models and open-source expert models. Qualitative analyses further demonstrate that our reinforcement learning method enables the model to drive more safely and comfortably while significantly reducing collisions. The main contributions of this work follows:
2

---

Published as a conference paper at ICLR 2026
We propose ReCogDrive, novel end-to-end autonomous driving framework that integrates a cognitive VLM with a diffusion planner, leveraging the rich world knowledge of VLMs to provide cognitive guidance and enabling stable and precise trajectory generation.
We propose Diffusion Group Relative Policy Optimization (DiftGRPO) to fine-tune the planner, allowing the model to learn directly from experience and optimize for safety and comfort, moving beyond mere imitation.
We establish new state-of-the-ar performance on NAVSIM and Bench2Drive in both open-loop and closed-looy loop evaluations, demonstrating significant improvements prior approaches.
2 RELATED WORK
Vision- Lanoon--annugg-MMddeee in Autonomous Driving, Numerous studies have leveraged the world knowledge embedded in VLMs to explore their application in autonomous driving. Current ap- proaches for autonomous driving using VLMs can be categorized primarily into two types: dual- system (Tian et al., 2024; Jiang et al., 2024a) and single-syster (Hwang et al., 2024; Mao al., 2023a; Xing et al., 2024; Mao et al., 2023b; Wang et al., 2024b; Bai et al., 2024; Shao et al., 2024a; Zhang et al., 2024c; Zhao al., 2025; Fu et al., 2025; Zhou et al., 2025a). Dual-system methods, such as VLM and Senna, integrate VLMs with end-to-end driving systems, where VLMs generate low-frequency trajectories or high-level commands that guide the end-to-end model in producing the final trajectory. In addition, several recent approaches (Pan et al., 2024; Hegde et al., 2025) distill VLM knowledge into end-to-end planners. further improving generalization and robustness. In contrast, single-system methods, such GPT-Driver and EMMA, reformulate the trajectory prediction task as language modeling problem and leverage chain-of-thouu enhance explainability. However, these text-based approaches often suffer from high inference latency and malformed outputs, whereas our ReCogDrive framework combines a VLM with a diffusion planner to produce more stable, safer trajectories while achieving a 3.5x speedup.
(Wei et al., 2022) to
Diffusion Models for Policy Learning. The recent success of diffusion models in domains like image generation (Podell et al., 2023; Zheng et al., 2024), roboties (Chi et al., 2023; Liu al., 2024b; Black et al., 2024; Yan et al., 2025; Bjorck et al., 2025), and traffic simulation (Yang et al., 2024; Zhong et al., 2023; Zeng et al., 2025) has spurred their application autonomous driving. For instance, DiffusionDrivve (Liao et al., 2024) employs truncated diffusion with anchor priors for real-time multimodal planning. GoalFlow (Xing et al., 2025) applies goal-point guidance with flow matching to generate high-quality trajectories. Diffusion Planner (Zheng al., 2025) redefines planning as a future trajectory generation task, jointly producing plans and motion forecasts. Our approach uses world knowledge from VLMs to guide the diffusion process, thereby improving the model's 's understanding of driving scenarios.
Reinforcement Learning in Autonomous Driving, Reinforcement Learning has proven effective in games (Hafner et al., 2023), robotics (Ren et al., 2024), and LLMs (Guo et al., 2025). Recently, many studies have introduced reinforcement learning into autonomous driving to improve model generalization. RAD (Gao et al., 2025) proposes training an end-to-end AD agent using Rein- forcement Learning a photorealistic 3DGS environment. CarPlanner (Zhang al., 2025) learns an auto-regressive policy for consistent multi-modal trajectories, outperforming imitation learning. AlphaDrive (Jiang et al., 2025), R2SE (Liu et al., 2025), Drive-R1 Li et al. (2025c) and TrajHF (Li et al., 2025a) incorporate GRPO to enhance the generalization of driving policies. Gen-Drive (Huang et al., 2025) combines diffusion models with RL and reward modeling for better policies. Our method novelly employs Diffusion Group Relative Policy Optimization (DiffGRPO) scheme to fine-tune Vision-Language-Acii (VLA) models and enhance their planning capabilities.
3 PRELIMINARIES
Problem Definition. Autonomous driving aims to predict smooth and collision-free trajectory in future seconds, given the ego status Sego (c.g., ego speed E IR22 containing Ue, Uy and ego acceleration E 2 containing az a,)). the historical ego trajectory Inist. E R Thisa x3 (containing past waypoints and headings), sensor input Team E R XxVxXXxW x3 (containing current and historical K frames from V camera views), and navigation information Tray (High-level command such "go straight")..
3

---

Published as a conference paper at ICLR 2026
Conventional end-to-end driving algorithm d formulate this
V Vtraj = 2 (eams nnsts Lnav, Sego),
where Uuaa denotes the planned future trajectory, parameterized as {(2,, St. 0,))==1 R?*3 with (Fx, yt) the 2D waypoints and 0r the corresponding heading angles. Explicit headings are needed to construct oriented ego polygons for collision evaluation and to derive yaw-related comfort measures, which cannot be robustly obtained from sparsely sampled waypoints alone. While methods such as (Hu al., 2023; Jiang al., 2023; Liao et al., 2024; Chen et al., 2024b) have shown strong effectiveness, their black-box nature impedes interpretability and they often fail to generalize to rare corner cases in real-world driving.
Recent works (Hwang et al., 2024; Mao et al., 2023a; Zhang et al., 2024b) utilize the rich world knowledge and causal reasoning capabilities of Vision- Languuee Models for autonomous driving. VLMM output trajectories in textual form and generate explicit reasoning processes:
Tuaaj Treason VLM(
Juist:
Luav, Sego).
However, we observe an inherent mismatch between the language- formatted trajectory space and the continuous action space, and the autoregressive decoding process can suffer from output collapse, leading to erroneous trajectories.
Diffusion Policy. Denoising Diffusion Probabilistic Models (DDPMs) (Ho et al., 2020; Nichol Dhariwal, 2021) learn a generative model by reversing a fixed, Markovian noising process that gradually corrupts data with Gaussian noise. Given clean trajectory Xp, the forward process defines:
<xc x = N(x V1 of x Ii),, 1=
where df, is the noise standard deviation step t. At inference, trajectories generated by initializing XT ~ NN I) and iteratively denoising:
X
VI-of (x of
of ex( oz z~ N0..).
(4)
Denoting trajectory waypoint as Xo. this framework naturally extends to trajectory-level policy gener- ation, where the denoising network ep learns to refine noisy motion plans into smooth trajectories.
4 METHODOLOGY
In this section, we present the overall framework and training paradigm of ReCogDrive. First, we introduce scalable hierarchical data pipeline that curates multi-level cognitive dataset spanning from perception reasoning instill the VLM with nuanced, human-like driving priors. Second, we detail our cognition-guidde diffusion planner, a novel architecture that resolves the modality mismatch by translating the VLM's latent semantic representations into continuous and stable trajectories. Finally, we describe our policy refinement stage, where a Diffusion Group Relative Policy Optimization (DiffGRPO) algorithm is employed to fine-tune the planner with simulated rewards, optimizing for safety and comfort beyond the confines of the expert dataset.
4.1 SCALABLE HIERARCHICAL DATA PIPELINE FOR DRIVING PRETRAINING
To bridge the gap between general Dision-Langues Models (VLMs) and autonomous driving, we introduce a scalable, structured pipeline for the automated generation of high-fidelity driving data, as shown in Fig. 2. This pipeline comprises three stages: Generation, Refinement, and Quality Control.
In the Generation stage, we align with the sequential cognitive process of human drivers, organized into four levels: (1) Foundational Perception: constructing representations static and dynamic scene elements; (2) Dynamic Understanding: analyzing multi-agent dynamics and predicting near-term behaviors of surrounding entities; (3) Planning and Reasoning: formulating executable, safety- aware driving plans and providing concise rationales; and (4) Advanced Reasoning: conducting counterfactual analyses and fine- -raingr trade-off evaluations. To achieve this, we combine ground- truth data for objective tasks with a state-of-the-art VLM for subjective annotations.
4

---

Published as a conference paper at ICLR 2026
Stage-1 Generation
Stage-2 Refinement
Normalizing, rewriling expanding templates
Generste a description of... identifu anfertttaa... Explain the rasson for plan... Whit would occur you...
Stage-3 Quality Control
Automated quality scoring and filtering
Hierarchical Cognitive VQA
Level-1
Level-2
Level-3
Level-4
Foundational Perception
Dynamic Understanding
Planning and Reasoning
Advanced Reasoning
Holistic Scenes Description
Traffic State Estimation
High-Level Planning
Counterfactual Reasoning
3D Object Detection
Key Agent Identification
Explainable Planning
Spatial Reasoning
Vulnerable Road User
Motion Prediction
Behavior Description
Figure 2: Overview of our Scalable Hierarchical Data Pipeline. We employ three-stage process of Generation, Refinement, and Quality Control to produce a high-quality dataset that mimics the cognitive process of human driver.
The Refinement stage integrates open-source datasets, followed by normalization, rewriting, and crestion-templat augmentation to ensure linguistic and semantic consistency. Finally, the Quality Control stage applies automated scoring and rigorous filtering based on linguistic accuracy, visual clarity, etc., retaining only the high quality samples. This systematic design ensures our pre-training dataset is both reliable and scalable, enabling the adaptation of a general VLM into a domain- specialized cognitive model for autonomous driving.
4.2 COGNITION-GuiA DIFFUSION PLANNER
While autoregressive paradigms offer straightforward way to generate trajectories in textual form, they suffer from inherent limitations: (i) the mismatch between discrete language space and continu- action space often leads to infeasible jerky trajectories, and (ii) VLMS prone to hallucination and decoding errors, which compromise safety in autonomous driving. To address these issues, we propose a Ongnition-Guider Diffusion Planner, which integrates human cognitive driving priors from a VLM with a diffusion planner to generate smooth and safety-aware trajectories. Formally, given noisy trajectory sample x, IR x3 the denoising step is defined as:
(5) where Fy denotes the VLM hidden states, Sego represents the ego-vehiele status, and 2 is the fused latent combining the noisy-action latent from the action encoder Eact. the historical-trajcctrry latent of Jist encoded by the bistory-trajectory encoder Ebis+ and the semantic prior F), obtained by mean-pooling over Fn. And Dac is the action decoder that predicts the next step noisy trajectory. The diffusion model trained by minimizing:
X;-1 Duct (DiTo(zs; Fhi Segoi t)). 2t concat (Face (xt), Enio (Thist), Fi).
(Face
Ldit = Ez,, e ||e Ex Ex c)12
(6)
with ~ N/ I) and condition c= {Fhs Sago
Vision-Language Model Guidance. We adopt Intern (Zhu al., 2025; Chen al., 2024e) as the cognitive backbone for its strong multi-modal reasoning. The VLM encodes multi-view images and textual queries into hidden states F5 E RLXD which act as cognitive tokens carrying driving priors and serve aross-attention conditions in the diffusion transformer. We further obtain global semantic embedding Fi, via average pooling over Fh. providing stable contextual guidance during denoising. Additionally, the VLM can optionally output high-level driving instructions and chain-of-thoughh reasoning to enhance trajectory generation.
Diffusion Transformer Design. Our diffusion planner builds DiT (Peebles & Xie, 2023; Yao et al., 2025) blocks, enhanced with lightweight but effective design choices: SwiGLU FFN (Shazeer, 2020) for expressive non-linearity, RoPE (Su al., 2024) for relative positional encoding, and both
5

---

Published as a conference paper at ICLR 2026
Driving Pre-training
Imitation Learning Reinforcement Learning
"eerect everything 30"
"Dasrshee the sowne"
nafto lghr"
the rrjeclory"
Question:
Gove the Ingeclory'
Vision Encoder
Vision Encoder
Text Tokenizer
Text Tokenizer
Large Language Model
Large Language Model
TThe location
The trafti Ight
The tryectory is
Diffusion Planner
Noise Initialization
Historical
Lightning Diffusion Transformer
N
Action
Encoder
Nolsed Action
Encedar
Action
Projector
RMS Norm
Scale, Shift
Mulli-Haad Attention
Scale
RMS Norm
Scale, Shift
FFN
SWICL
alea
Action Decoder
c-
Future Trajectory
Cognitive
Tokens
Encoder
Condition Injection
Ego Status
Encoder
Figure 3: Training Pipeline and Model Architecture. ReCogDrive couples a VLM with diffusion planner: the VLM encodes inputs into cognitive tokens guiding trajectory denoising. Training follows three stages: Driving Pre-training, imitation learning, and reinforcement learning.
QK-Norm and RMSNorm
(Zhang & Semnrich, 2019) for stable optimization. Each block alternates between (i) self-attentinn, which models pairwise waypoint relations, and (ii) cross-attentin which injects semantic priors F3 into the trajectory space. This design enables structured fusion of scene-level cognition with low-level trajectory optimization.
To capture temporal consistency, historical trajectories Xhist are encoded and concatenated with the noisy trajectory embeddings. Hgo-vehicle status Sego (e.g., speed, acceleration) is injected via AdaLN modulation, ensuring that the planner conditioned on the vehicle's physical state. This dual conditioning mechanism enhances stability, safety, and adaptability in diverse driving scenarios.
Historical and Ego Information Fusion.
4.3 DIFFUSION GROUP RELATIVE POLICY OPTIMIZATION
Relying solely on imitation learning exhibits fundamental limitations (Chu et al., 2025; Osa et al., 2018; Ren al., 2024), since expert demonstrations may multi-modal, leading conflictual optimization results. As shown in Fig. 4(a), when trained in this rare intersection turn scenario with multiple expert trajectories, the model resorts to learning an average trajectory to achieve global optimality, which lead to incorrect unsafe maneuvers. Consequently, even though the diffusion planner trained through imitation learning closely matches expert trajectories terms displacement, it still frequently produces collisions drives outside the drivable area.
To enable exploration beyond imitation, we introduce DiffGRPO, a Group Relative Policy Opti- mization algorithm specifically designed for diffusion planners that tightly couples GRPO (Shao et al., 2024b) with the denoising process. We adopt GRPO since it naturally samples multiple candidate trajectories, which aligns with autonomous driving where multiple feasible ground truth trajectories can satisfy the same scene and high level intent. Following (Ren et al., 2024), we interpret the diffusion policy Te an internal Markov decision process: starting from Gaussian noise, the model gradually denoises to generate a full trajectory. Concretely, we sample G trajectories, each represented by a diffusion chain:
(TT, #T-1, zo),
where T' is the total number of denoising steps and transitions follow:
PT No0, I), Ft-1 no(x xx).
(8)

---

Published as a conference paper at ICLR 2026
Collisions Ego Progress
GT
Driving Simulator
Comfort Drivable Area
Optimize
-- Diruction
R.ward:0.8
Diffusion
L1/L2 Loss
Diffusion Planner
BC Loss
Diffusion
Policy
A
Planner
Planner
A₂
Group Computation
VLM
VLM
(8) Imitation Learning
(b) Reinforcement Learning (DiffGRPO)
Figure 4: Comparison of Training Paradigms. (a) Imitation Learning: the diffusion planner is trained offline to mimic ground truth trajectories using L./112 losses, but tends to learn averaged, suboptimal paths. (b) Reinforcement Learning: multiple trajectories are sampled and evaluated in the NAVSIM simulator, scored on collision avoidance, drivable area compliance and other metrics, and advantages are computed via group computation to update the diffusion planner.
Unlike prior approaches (Li et al., 2025a;c) that rely on simple f2 trajectory distances as surrogate rewards, we leverage the NAVSIM simulator to providing realistic feedback on safety and comfort. Each rollout is evaluated in terms of collisions, drivable-area compliance, and driving comfort, with results aggregated into a Predictive Driver Model Score (PDMS) that serves as the reward ry. We cast this as single-step decision, treating the whole trajectory composite action with its PDMS score as the reward. We then compute the group- souu-sttndardiiee advantage:
G E
A
ri P
p
1 G
Σ
Tj.
o²
G
(ry -)) i=
(9)
Each conditional step in the diffusion chain is a Gaussian policy:
To ( xx) . a1-1 (o( b), o?T).
(10)
where no (x,, t) is the model-predicted mean and of the (fixed) covariance. Thus, the probability density of the full chain Xy:7 under иe, where p(xr) is Gaussian prior independent of 8, is:
T
log To (x(:)))) = log ((x)(
+
Σ
log (2 [xz),
(11)
Finally, we compute the policy loss following (Williams, 1992; Shao et al., 2024b) while concurrently incorporating behavior cloning loss prevent collapse during exploration.
G
L
Σ t=
t- log T (2 1
A,
A
log n
(1) (
(12)
G
t=1
LRL
Lac
where yy is the discount coefficient mitigating instability in early denoising steps, is the weight for the behavior cloning loss, and ž,-1. 2, are values sampled from the reference policy Tref- We omit the PPO-style clipping and set the update iteration to 1, so Told Through DiffGRPO, the diffusion planner learns to generate safe and comfortable trajectories closed-loop settings, going beyond mere imitation and enhancing the robustness of framework.
7

---

Published as a conference paper at ICLR 2026
Table I: Performance comparison on NAVSIM navtest using closed-loop metrics. models fine-tuned on the NAVSIM trajectory dataset.
denotes
Method
Image
Lidar
NCT
DACT
TTC1
Comf.
EPT
PDMS1
Constant Velocity
68.0
57.8
50.0
100
19.4
20.6
Ego Status MI.P
93.0
77.3
83.6
100
62.8
65.6
VADv2-Vau2 (Chen et al., 2024b)
97.2
89.1
91.6
100
76.0
80.9
DPT (Chen al., 2024c)
98.9
90.7
94.9
95.6
79.7
82.4
UniAD (Hu et al., 2023)
97.8
92.9
78.8
83.4
TransFuser (Chitta et al., 2022)
97.7
92.8
92.8
100
79.2
84.0
PARA-Drive (Weng ct al., 2024)
97.9
92.4
93.0
99.8
79.3
84,0
DRAMA (Yuan al., 2024)
98.0
93.1
94.8
100
80.1
85.5
Hydde-MDDP-Vuug-WW-BB (Li al., 2024)
98.3
96.0
94.6
100
78.7
86.5
DiffusionDrivv (Liao et al., 2024)
98.2
94.7
WoTE (Li et al., 2025b)
98.5
96.8
94.9
99.9
81.9
88.3
VLss- Methods
QwenVL2.5-888 8 (Bai et al., 2025)
97.8
92.1
92.8
100
78.3
83.3
IntemVL3--8B' (Zhu et al., 2025)
97.0
92.4
91.8
100
78.9
83.3
AutoVLA Zhou et al. (2025b)
98.4
95.6
98,0
99.9
81.9
89.1
ReCogDrive(ours)
97.9
97.3
94.9
100
87.3
90.8
Table 2: Closed-loop and Multi-ability Testing Results in CARLA Bench2Drive Leaderboard.
Method
Closed-loop Metrie
Multi-Ability
Efficiency
Merging
Overtaking
Emerg.
GiveWay
Sign
TCP (Wa al., 2022)
54.26
47,80
15.00
60.70
16.18
20.00
20.00
10.00
6.99
14:63
ICP-ul (Wu et al,, 2022)
$3.97
51.51
7.27
30:47
10.29
4.44
10.00
10.00
6.45
5.23
IUP-traj (Wu et 2022)
70.54
18.08
30.00
59.90
8.89
24.29
51.67
40.00
46.28
54.22
TCP-raj weo dianil. (Wi et 2022)
78.76
22.96
30.05
49 30
17.14
6,67
40.00
40.00
28.72
28.51
ThinkTwice al., 20236)
76.93
16.22
3.10
62.41
27.38
18.42
35.82
50,00
54,23
37.17
DriveAdapter
56.41
AD-MPP (Zhai ala
UnIAD-T: 1.00-T100 (Hu al...2000 2023)
123
47.06
13.18
40.73
8,89
9:33
20.00
20.00
15.43
14.73
UniAD-B (Hu 2023)
129.2
43.58
16.36
45.81
14.10
17.78
21.67
10.00
14.21
15.55
VAD (lang et 2023)
157.94
16.01
15.00
42.35
8.11
24.41
18.64
20.00
19:15
18.07
TrnndoonennLL..... Ola 2025)
100.64
46.01
35.01
63.46
17.57
35.00
48.36
40.00
52,10
38.60
ReCogDrive
20.00
5
EXPERIMENTS
5.1
EXPERIMENTAL. SETUP.
Implementation Details. We choose Intern VL3 (Zhu et al., 2025), comprising 300M -aaammeeeee Intern VIT visual encoder (Chen al., 2024e) and a Qwen2.5 Large-Lagguaee-Mode (Bai et al., 2025), as our base model, which demonstrates strong performance on multiple benchmarks. Images processed through dynamic resolution preprocessing strategy. In the first stage, conduct supervised fine-tuning (SFT) on the VLM using the dataset constructed by our hicrarchical data pipeline for three epochs. In the second stage, with the VLM parameters frozen, train the diffusion model via DDPM for 200 epochs. In the third stage, we further optimize the diffusion model using reinforcement learning for 10 epochs, with one policy update per batch. The datasets and training hyperparameters for all three stages are described in detail in the supplementary material.
Dataset. We evaluate primarily on two challenging benchmarks: NAVSIM (Dauner et al., 2025) and Bench2Drive (Jia et al., 2024). NAVSIM is a planning- oriented autonomous driving dataset built on OpenScene (Contributors, 2023), a redistribution nuPlan (Caesar et al., 2021). The dataset split into navirain 11,192 training scenes) and naviest (136 evaluation scenes). Bench2Drive is CARLA-based benchmark composed of 220 short routes, each containing a distinct, safety-critical scenario. In addition, to build the VLM's cognitive foundation, ReCogDrive is trained collection of VQA datasets, including DriveLM (Sima et al., 2024) and LingoQA (Marcu et al., 2024), with further details provided in the supplementary material.
8

---

Published as a conference paper at ICLR 2026
Table Ablation study on the proposed components of ReCogDrive. We evaluate the effect of driving pre-training, diffusion planner, and reinforcement learning on NAVSIM evaluation.
ID Trajectory training Pre-training
Driving
Diffusion
Diff
NC
DAC
TTC
Conf.
EP
PDMS
Planner
GRPO
X
X
97.4
91.3
93.0
100
77.2
82.4
2
X
X
97.6
93.1
92.7
100
79.1
84.1
98.1
94.7
94.2
100
80.9
86.5
97.9
97.3
94.9
100
87.3
90.8
Table 4: Comparison of trajectory generation
Table 5: Model performance on Drive VQA
methods.f indicates inference with 1mdeplay oy.
Benchmark.
Mothod
Time
PDMS
Err. (%)
Method
DriveLM
DriveBench
VLM Plain Text!
1,0702
84.1
0.01
GPT-Scoree
Percep.
Predict.
Plan.
Behav
VLM + MLP
0.3045
75.5
0.00
LLaVA-1.5 1.5
61.91
23.22
22.02
29.15
13.60
22.00
InternV1.2
64.13
VLM + Query-based Decoder
0,3050
85.0
0.00
Qwen2-VI.
30.13
49.35
61.30
51.26
48.01
0.3060
DriveLM
65.25
16.85
44.33
68.71
42.78
43.17
SwiGLU FFN
0.3065
85.7
0.00
Dolphins
9.59
32.66
52.91
8.81
25.99
GPT-4o
67.27
35,37
51.30
75.75
45.40
51.96
RMS Narm
0.3070
85.8
0.00
RoPE QK Norm
0.3061
86.5
0.00
ReCogDrive
67.30
64.95
49.34
70.20
42.36
566.7
Table 6:
Comparison of different RL algo- rithms. We evaluate REINFORCE, DPPO, and DiffGRPO diffusion planner under the same training and evaluation setting.
Figure 5: Training curves on NAVSIM.
Algorithm
NC
DAC
TTC
Conf.
EP
PDMS
REINFORCE
REINFORCE
98.5
97.3
95.9
100
82.9
89.5
0.8
DPPO
DITIORPO
DPPO
98.6
97.6
96.0
100
82.4
89.5
DiftGRPO
97.9
97.3
94.9
100
87.3
90.8
163
0.7
Train
5.2 MAIN RESULTS AND ABLATION STUDY
Experiments on the NAVSIM Benchmark. Tab. reports results on NAVSIM. ReCogDrivv achieves PDMS of 90.8, establishing new state-of-the-art. Remarkably, outperforms Diffusion- Drive (Liao et al., 2024) and WoTE (Li et al., 2025b), both of which use camera and LiDAR inputs, by 2.7 and 2.5 PDMS, respectively, while using only camera input. Compared with our reproduced Intern VL3 (Zhu et al., 2025) and Qwen VL2.5 (Bai et al., 2025) trained directly on NAVSIM trajecto- ries, ReCogDrive improves PDMS by 7.5, validating the effectiveness of our training paradigm. also surpasses the prior vision-only sfate-of-thee-ar, PARA-Drive (Weng et al., 2024), by 6.8 PDMS.
Bench2Drive Closed-loop Performance. Tab. 2 reports closed-loop and multi-ability results on the CARLA Bench2Drive leaderboard. ReCogDrive achieves the highest scenario success rate of 45.45% and the top Driving Score of 71.36, surpassing prior end-to-end baselines. It also excels in safety-critical skills such as emergency braking 69.09% and traffic sign compliance 71.34, while maintaining strong efficiency and competitive multi-ability mean of 42.03. These results highlight the effectiveness and reliability of our framework in complex urban driving.
Ablation study on ReCogDrive. Tab. presents an ablation study on the proposed components of ReCogDrive. When training InternVL. solely on NAVSIM trajectory data, the model achieves a PDMS of 83.3. Adapting the VLM driving scenarios with our large-scale driving QA data increases PDMS by 1.7. Introducing the diffusion planner for continuous trajectory prediction further raises PDMS by 2.4. Finally, DiffGRPO achieves a PDMS 90.8 with a 4.3 improvement, demonstrating the effectiveness of our RL scheme in producing safer driving behavior.
Comparison of Trajectory Generation Methods. We compare trajectory generation methods in terms of inference speed, PDM Score, and error rate shown in Tab. 4. The error rate quantifies the

---

Published as a conference paper at ICLR 2026
Front Concat View
(a) Transfuser (b) ReCogDrive-IL (c) ReCogDrive-RL
Figure 6: Comparisons on the Navtest benchmark.
occurrence of malformed outputs that lack parsable trajectory, which constitutes a low-probability failure mode in direct trajectory regression using VLMs. All inference times in Tab. 4 are measured as end-to-end wall-clock latency for processing a single sample.
Generating trajectories plain text is slow due to the autoregressive process and may suffer from format errors that compromise safety. contrast, MLP, Query-based Decoder (Chitta et al., 2022), and diffusion planner are substantially faster, with diffusion-basee approaches further boosting PDMS. Building on this, we integrated lightweight architectural refinements including SwiGLU FFN, RMS Norm, RoPE, and QK Norm. Our final model ReCogDrive achieves 3.5x speedup and 2.4 PDMS gain over the text baseline, validating the effectiveness of our design.
Evaluation on Drive VQA Benchmarks. To provide a comprehensive assessment of our model's language-related capabilities driving scenarios, we evaluate its performance on the DriveLM and DriveBench benchmarks. The results, presented in Tab. 5, demonstrate the effectiveness of our approach. Our model, ReCogDrive, achieves state-of-the-ard performance, surpassing the powerful closed-source GPT-44o model on most key metrics, including the overall DriveBench average score. Furthermore, ReCogDrive significantly outperforms several specialized open-source driving models, validating its strong visual question answering and reasoning abilities in this domain.
Comparison of RL Algorithms for Diffusion Planning. To validate DiffGRPO, we compare it with DPPO (Ren et al., 2024) and REINFORCE (Williams, 1992) using the same diffusion planner and PDMS reward, with results in Tab. 6. Since DiffGRPO samples a group of candidate trajectories under the same scene condition, it naturally matches autonomous driving where a single scene often admits multiple safe ground-trus -trth trajectories, and the group-relative advantage encourages safer and more stable rollouts. As reflected by the training curves, DiftGRPO achieves the best PDMS while converging faster and more stably than DPPO and REINFORCE.
Qualitative Results. Fig. 6, we compare ReCogDrive (II. and RL. with Transfuser (Chitta et al., 2022), where RL yields safer and more reliable trajectories in challenging turning scenarios. More visualizations are in the supplementary material.
CONCLUSION
In this work, we proposed ReCogDrive, a novel reinforeed cognitive framework for end-to-end autonomous driving that integrates Vision-Language Model with a diffusion planner. We first introduced a scalable hierarchical data pipeline that mimics human driving cognition to construct foundation model with strong driving priors. Building on this foundation, we designed a cognition- guided diffusion planner that injects VLM-derived cognitive tokens into the denoising process, enabling the generation of continuous, stable, and human-like trajectories. To further refine planning behavior, we introduced DifiGRPO. a reinforcement learning scheme that explicitly optimizes safety and comfort in closed-loop loop driving. Extensive experiments multiple benchmarks demonstrate that ReCogDrive achieves state-of oftte-of-the-att performance, validating the effectiveness our approach. We hope work can advance the development of autonomous driving VL.ss.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS AND DISCLOSURE OF FUNDING
This work was partially supported by the National Natural Science Foundation of China under Grant U25B2067.
ETHICS STATEMENT
We acknowledge and adhere the ICLR Code of Ethics throughout this work. Our research does not involve human subjects or sensitive personal data, and all experiments were conducted in compliance with ethical research practices.
REPRODUCIBILITY STATEMENT
We have made extensive efforts to ensure reproducibility. In the supplementary materials, we provide details of our training datasets, including composition and preprocessing steps in Appx. D. We further include hyperparameters and training details in Appx. E. addition, we provide our model code as part of the supplementary materials to facilitate reproduction of our results.
REFERENCES
Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell Hewett, Mojan Javaheripi, Piero Kauffmann, et al. Phi-4 technical report. arXiv preprint 24Xiv:2412.000 02909 2024. 20
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang. Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang. Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. arXiv preprint arXiv:2502. 13923,, 2025. 8, 22
Yifan Bai, Dongming Wu, Yingfei Liu, Fan Jia, Weixin Mao, Ziheng Zhang, Yucheng Zhao, Jianbing Shen, Xing Wei, Tiancai Wang, et al. a 3d-tokenized IIm the key to reliable autonomous driving? arXiv preprint 24xx:2405./8366 2024. 2,
Johan Bjorek, Fernando Castañeda, Nikita Cherniadey, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang. Dieter Fox, Fengyuan Hu, Spencer Huang, et al. Gr0Ot nl: An open foundation model for generalist humanoid robots. arXiv preprint urXiv:2503. /4734, 2025. 3
Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. A lision-lagguageaction flow model for general robot control. arXiv preprint arXiv:2410.24164, 2(3):5, 2024.
Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishman, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: multimodal dataset for autonomous driving. In Proceedings the IEEE/CVF conference on computer vision and pattern recognition, pp. 1161--1663,, 2020. 22
Holger Caesar, Juraj Kabzan, Kok Seang Tan, Whye Kit Fong, Eric Wolff, Alex Lang, Luke Fletcher, Oscar Beijbom, and Sammy Omari. nuplan: A closed-loop ml-based planning benchmark for autonomous vehicles. arXiv preprini arXiv:2 106. 11100 2021. 8
Xu Cao, Tong Zhou, Yunsheng Ma, Wenqian Ye, Can Cui, Kun Tang, Zhipeng Cao, Kaizhao Liang, Ziran Wang, James M Rehg, et al. Maplm: A real-world large-scale vision-language benchmark map and traffic scene understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21819-21830, 2024. 22, 25
Yuning Chai, Benjamin Sapp, Mayank Bansal, and Dragomir Anguelov. Multipath: Multiple proba- bilistic anchor trajectory hypotheses for behavior prediction. arXiv preprint arXiv:1910.05.. 1910.05499 2019. 1
11

---

Published as a conference paper at ICLR 2026
Kai Chen, Yanze Li, Wenhua Zhang, Yanxin Liu, Pengxiang Li, Ruiyuan Gao, Lanqing Hong, Meng Tian, Xinhai Zhao, Zhenguo Li, et al. Automated evaluation of large vision-language models on self-driving comerr cases. arXiv preprint urXi:24404115595 2024a. 22, 25
Shaoyu Chen, Bo Jiang, Hao Gao, Bencheng Liao, Qing Xu, Qian Zhang, Chang Huang, Wenyu Liu, and Xinggang Wang. Vadv2: End-to-end vectorized autonomous driving via probabilistic planning. arXiv preprint arXiv:2402.13245 2024b. 1, 4, 8, 19
Yuntao Chen, Yuqi Wang, and Zhaoxiang Zhang. Drivinggpt: Unifying driving world modeling and planning with multi-modal autoregressive transformers. arXiv preprint arXiv:2412. 18607, 2024c.
Zhe Chen, Weiyun Wang. Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, al. How far are gpt-4v? closing the gap commercial multimodal models with open-source suites. Science China Information Sciences, 67(12):220101 2024d. 20
Zhe Chen, Jiannan Wu, Wenhai Wang. Weijie Su, Guo Chen, Sen Xing. Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 24185-218250 22198, 2024e. 5,
Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, pp. 02783649241273666. 2023. 3
Kashyap Chitta, Aditya Prakash, Bernhard Jaeger, Zehao Yu, Katrin Renz, and Andreas Geiger. Transfuser: Imitation with transformer-basee sensor fusion for autonomous driving. IEEE Trans- actions on Puttern Analysis and Machine Intelligence, 45(11):12878-1295, 2022. 8, 10, 18, 19
Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. Sft memorizes, generalizes: A comparative study of foundation model post-training. arXiv preprint arXiv:2501. 2025.
OpenScene Contributors. Openscene: The largest up-to-date 3d occupancy prediction benchmark in autonomous driving. https: /ggtttttbbbb//
wtt.eaaaaaaeeeeeeeeeeeeeeeaaaaee 2023. 8
Daniel Dauner, Marcel Hallgarten, Tianyu Li, Xinshuo Weng, Zhiyu Huang, Zetong Yang. Hongyang Li, Igor Gilitschenski, Boris Ivanovic, Marco Pavone, et al. Navsim: Data-driven nen-reactive autonomous vehicle simulation and benchmarking. Advances in Neural Information Processing Systems, 37:28706-2877 28719, 2025. 8, 22
Thierry Deruyttere, Simon Vandenhende, Dusan Grujicic, Lue Van Gool, and Marie-Francing Moens. Talk2car: Taking control of your self-driving car. arXiv preprint 1rXiv:I909.10:: 2019. 22, 25
Xinpeng Ding, Jianhua Han, Hang Xu, Xiaodan Liang. Wei Zhang, and Xiaomeng Li. Holistic autonomous driving understanding by bird' ued''--eye injected multi-modal large models. In Proceedings of the IEFF/CVE Conference on Computer Vision and Pattern Recognition, pp- 13668-13677, 2024. 22, 23,25
Renju Feng, Ning Xi, Duanfeng Chu, Rukang Wang, Zejian Deng, Anzheng Wang. Liping Lu, Jinxiang Wang, and Yanjun Huang. Artemis: Autoregressive end-to-end trajectory planning with mixture of experts for autonomous driving. arXiv preprint arXiv:2504. 19580, 2025. 19,20 20
Haoyu Fu. Diankun Zhang, Zongchuang Zhao, Jianfeng Cui, Dingkang Liang, Chong Zhang, Dingyuan Zhang, Hongwei Xie, Bing Wang, and Xiang Bai. Orion: A holistic end-to-ent autonomous driving framework by vision-language instructed action generation. arXiv preprint arXiv:2503. /9755, 2025.
Hao Gao, Shaoyu Chen, Bo Jiang, Bencheng Liao, Yiang Shi. Xiaoyang Guo, Yuechuan Pu, Haoran Yin, Xiangyu Li. Xinbang Zhang, et al. Rad: Training an end-to-end driving policy via large-scale 3dgs-based reinforcement learning. arXiv preprint arXiv:2502. / /3/44, 2025.
12

---

Published as a conference paper at ICLR 2026
Junru Gu, Chenxu Hu, Tianyuan Zhang. Xuanyao Chen, Yilun Wang, Yue Wang, and Hang Zhao. Vip3d: End-to-end visual trajectory prediction via 3d agent queries. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 5496-5506, 2023.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501. 2025. 3
Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models. arXiv preprint arXiv:2301.04 100444 2023.
Deepti Hegde, Rajeev Yasarla, Hong Cai, Shizhong Han, Apratim Bhattacharyya, Shweta Mahajan, Litian Liu, Risheek Garrepalli, Vishal M Patel, and Fatih Porikli. Distilling multi-modal large language models for autonomous driving. In Proceedings of the Computer Vision and Puttern Recognition Conference, pp. 27575-27585, 2025. 3,21
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851. 2020. 4, 24
Shengchao Hu, Li Chen, Penghao Wu, Hongyang Li, Junchi Yan, and Dacheng Tao. St-p3: End-to-ent end vision- based autonomous driving spatial-temporea feature learning. In European Conference on Computer Vision, pp. 533-549. Springer, 2022.
Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Planning-orientedd Tianwei Lin, Wenhai Wang, et al. Planning- oriented autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 77853-17862. 2023. 8,21
Zhiyu Huang, Xinshuo Weng, Maximilian Igl, Yuxiao Chen, Yulong Cao. Boris Ivanovic, Marco Pavone, and Chen Lv. Gen-drive: Enhancing diffusion generative driving policies with reward modeling and reinforcement learning fine-tuning. 2025 IEEE International Conference on Robotics and Automation (ICRA), pp. 3445-3451 IEEE, 2025. 3
Aaron Hurst, Adam Lerer, Adam Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21226 2024. 20
Jyh-Jing Hwang, Runsheng Xu, Hubert Lin, Wei-Chih Hung, Jingwei Ji, Kristy Choi, Di Huang, Tong He, Paul Covington, Benjamin Sapp, al. Emma: End-to-end multimodal model for autonomous driving, arXiv preprint TrXiv:2410.232666 2024. 1, 4, 18
Xiaosong Jia, Yulu Gao, Li Chen, Junchi Yan, Patrick Langechuan Liu, and Hongyang Li. Driveadapter: Breaking the coupling barrier of perception and planning in end-to-end autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7953-7963, 2023a. 8
Xiaosong Jia, Penghao Wu, Li Chen, Jiangwei Xie, Conghui He, Junchi Yan, and Hongyang Li. Think twice before driving: Towards scalable decoders for end-to-end autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patlern Recognition, pp.
11183-211994 2023b. 8
Xiaosong Jia, Zhenjie Yang, Qifeng Li, Zhiyuan Zhang, and Junchi Yan. Bench2drive: Towards multi-ability benchmarking of closed-loop end-to-end autonomous driving. Advances in Neural Information Processing Systemx,
844, 2024. 8
Xiaosong Jia, Junqi You, Zhiyuan Zhang, and Junchi Yan. Drivetransformer: Unified transformer for scalable end-to-end autonomous driving. arXiv preprint arXiv:2503.07656 2025. 8
Bo Jiang, Shaoyu Chen, Qing Xu, Bencheng Liao, Jiajie Chen, Helong Zhou, Qian Zhang, Wenyu Liu, Chang Huang, and Xinggang Wang. Vad: Vectorized scene representation for efficient autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8340-8350. 2023. 1, 1,4,8 8,
13

---

Published as a conference paper at ICLR 2026
Bo Jiang, Shaoyu Chen, Bencheng Liao, Xingyu Zhang, Wei Yin, Qian Zhang, Chang Huang, Wenyu Liu, and Xinggang Wang. Senna: Bridging large vision-language models and end-to-end autonomous driving. arXiv preprint arXiv:2410.22313, 2024a 1, 3, 22, 25
Bo Jiang, Shaoyu Chen, Qian Zhang, Wenyu Liu, and Xinggang Wang. Alphadrive: Unleashing the power of vlms in autonomous driving via reinforcement learning and reasoning, arXiv preprint arXiv:2503.07600 2025. 3
Xiaohui Jiang, Shuailin Li, Yingfei Liu, Shihao Wang, Fan Jia, Tiancai Wang, Lijin Han, and Xiangyu Zhang. Far3d: Expanding the horizon for surround-view 3d object detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 2561-2560 2024b.
Jinkyu Kim, Anna Rohrbach, Trevor Darrell, John Canny, and Zeynep Akata. Textual explanations for self-driving vehicles. In Proceedings of the European conference on computer vision (ECCV),
pp. 563-578, 2018. 22
Derun Li, Jianwei Ren, Yue Wang, Xin Wen, Pengxiang Li, Leimeng Xu, Kun Zhan, Zhongpu Xia, Peng Jia, Xianpeng Lang, et al. Finetuning generative trajectory model with reinforcement learning from human feedback. arXiv preprint arXiv:2503. 10434, 2025a. 3,7
Yingyan Li, Yuqi Wang, Yang Liu, Jiawei He, Lue Fan, and Zhaoxiang Zhang. aandtto-end driving with online trajectory evaluation via bev world model. arXiv preprint 21111122000.09941,,.. 2025b. 8,
Yue Li, Meng Tian, Dechang Zhu, Jiangtong Zhu, Zhenyu Lin, Zhiwei Xiong, and Xinhai Zhao. Drive-rl: Bridging reasoning and planning in vlms for autonomous driving with reinforcement learning. arXiv preprint arXiv:2506. 18234, 2025c. 3,7
Zhenxin Li, Kailin Li, Shihao Wang, Shiyi Lan, Zhiding Yu, Yishen Ji, Zhiqi Li, Ziyue Zhu, Jan Kautz, Zuxuan Wu, et al. Hydra-mdp End-to-end multimodal planning with multi multi-target hydra- dida-distilllatin. arXiv preprint arXiv:2406.06% 2024. 8, 19
Bencheng Liao, Shaoyu Chen, Haoran Yin, Bo Jiang, Cheng Wang, Sixu Yan, Xinbang Zhang, Xiangyu Li, Ying Zhang, Qian Zhang, et al. Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving, arXiv preprint arXiv:2411.15133 2024. 1, 8,
Haochen Liu, Tianyu Li, Haohan Yang, Li Chen, Caojun Wang, Ke Guo, Haochen Tian, Hongchen Li, Hongyang Li, and Chen Lv. Reinforced refinement with self-aware expansion for end-to-end autonomous driving. arXiv preprint arXiv:2506. 098000 2025.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023a. 25
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34916, 2023b. 20
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024a. URL https: 01 30- llavallaaanie -eett..ee/...ee 20
//llava-vll vi.iraalllghihhhhb.. io wwt.bo////oo//.///
Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. Rdt-Ib: diffusion foundation model for bimanual manipulation. arXiv preprint arXiv:2410.07864, 2024b. 3
Zuyan Liu, Yuhao Dong, Ziwei Liu, Winston Hu, Jiwen Lu, and Yongming Rao. Oryx mllm: On- demand spatial-temporal understanding at arbitrary resolution. arXiv preprint 129. 2024c. 20
Yingzi Ma, Yulong Cao, Jiachen Sun, Marco Pavone, and Chaowei Xiao. Dolphins: Multimodal language model for driving. In European Conference on Computer Vision, pp. 403-420. Springer, 2024. 20
Srikanth Malla, Chiho Choi, Isht Dwivedi, Joon Hee Choi, and Jiachen Li. Drama: Joint risk localization and captioning in driving. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 1043-1052, 2023. 22, 25
14

---

Published as a conference paper at ICLR 2026
Jiageng Mao, Yuxi Qian, Junjie Ye, Hang Zhao, and Yue Wang. Gpt-driver: Learning to drive with gpt. arXiv preprint arXiv:2310.01415, 2023a. 2,3,4
Jiageng Mao, Junjie Ye, Yuxi Qian, Marco Pavone, and Yue Wang. A language agent for autonomous driving. arXiv preprint (rXiv:2311.108111 2023b. 2, 3
Ana-Maria Marcu, Long Chen, Jan Hünermann, Alice Karnsund, Benoit Hanotte, Prajwal Chidananda, Saurabh Nair, Vijay Badrinarayanan, Alex Kendall, Jamie Shotton, et al. Lingoqa: Visual question answering for autonomous driving. In European Conference on Computer Vision, pp. 252-269. Springer, 2024, 8, 22, 25
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffuusio probabilistic models. In International conference machine learning, pp. 8162-8171, PMLR, 2021. 4, 20
Takayuki Osa, Joni Pajarinen, Gerhard Neumann, J Andrew Bagnell, Pieter Abbeel, Jan Peters, et al. An algorithmic perspective on imitation learning. Foundations and Trends@) in Robotics, 7(1-2): 1-179. 2018.
Chenbin Pan, Burhaneddin Yaman, Tommaso Nesti, Abhirup Mallik, Alessandro G Allievi, Senem Velipasalar, and Liu Ren. Vlp: Vision language planning for autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14760-14769, 2024. 3,
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference computer vision, pp. 4195-4205, 2023, 5, 24
Jonah Philion and Sanja Fidler. Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28 2020, Proceedings, Part XIV 16, pp. 194 210. Springer, 2020.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 3
Tianwen Qian, Jingjing Chen, Linhai Zhuo, Yang Jiao, and Yu-Gang Jiang. Nuscenes-qa: A multi- modal visual question answering benchmark for autonomous driving scenario. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pp. 4542-4550, 2024. 22, 25
Allen Z. Ren, Justin Lidard, Lars I. Ankile, Anthony Simeonov, Pulkit Agrawal, Anirudha Majumdar, Benjamin Burchfiel, Hongkai Dai, and Max Simchowitz. Diffusion policy policy optimization. arXiv preprint arXiv:2409. 00500, 2024. 6, 10, 20
Hao Shao, Yuxuan Hu, Letian Wang, Guanglu Song. Steven Waslander, Yu Liu, and Hongsheng
Li. Lmdrive: Closed-loop end-to-end driving with large language models. In Proceedings of the JEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 11120-1513,, 2024a.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.033000 2024b. 6,
Noam Shazeer. Glu variants improve transformer. arXiv preprint arXi::2002.05202 2020. 5
Chonghao Sima, Katrin Renz, Kashyap Chitta, Li Chen, Hanxue Zhang, Chengen Xie, Jens Beißwenger, Ping Luo, Andreas Geiger, and Hongyang Li. Drivelm: Driving with graph vi- sual question answering. In European Conference on Computer Vision, pp. 256-274. Springer, 2024. 8, 20, 22, 23, 25
Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568: 127063, 2024. 5
Xiaoyu Tian, Junru Gu, Bailin Li, Yicheng Liu, Yang Wang, Zhiyong Zhao, Kun Zhan, Peng Jia, Xianpeng Lang, and Hang Zhao. Drivevlm: The convergence of autonomous driving and large vision-language models. arXiv preprint arXiv:2402. /2289, 2024. 1, 21
15

---

Published as a conference paper at ICLR 2026
Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing risionllagguaee model's perception of the world at any resolution. arXiv preprint arXiv:2409. 12191, 2024a. 20
Shihao Wang, Zhiding Yu, Xiaohui Jiang, Shiyi Lan, Min Shi, Nadine Chang, Jan Kautz, Ying Li, and Jose M Alvarez. Omnidrive: A holistic Ilm-agent framework for autonomous driving with 3d perception, reasoning and planning. arXiv preprint arXiv:2455.0/333 2024b. 2, 3, 18, 22, 23, 25
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of- -f-thughhttt prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24837, 2022. 2,
Xinshuo Weng, Boris Ivanovic, Yan Wang. Yue Wang. and Marco Pavone. Para-drive: Parallelized architecture for real-timo autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15449-15458, 2024. 8,9
Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8:229-255 222,, 1992. 7, 10
Penghao Wu, Xiaosong Jia, Li Chen, Junchi Yan, Hongyang Li, and Yu Qiao. Trajectory-guided control prediction for end-to-end autonomous driving: A simple yet strong baseline. Advances in Neural Information Processing Systems, 35:61 19-6132, 2022.
Shuo Xing, Chengyuan Qian, Yuping Wang, Hongyuan Hua, Kexin Tian, Yang Zhou, and Zhengzhong Tu. Openemma: Open-source multimodal model for end-to-end autonomous driving. arXiv preprint arXiv:2412. 2024. 3,3
Zebin Xing, Xingyu Zhang. Yang Hu. Bo Jiang, Tong He, Qian Zhang, Xiaoxiao Long, and Wei Yin. Goalflow: Goal-driven flow matching for multimodal trajectories generation in end-to-end autonomous driving. Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 1602-1611, 1, 2025.
Li Xu, He Huang, and Jun Liu. Sutd-trafficqa: A question answering benchmark and an efficient network for video reasoning over traffic events. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9878 9888998888888 2021. 22, 25
Zhenhua Xu, Yujia Zhang, Enze Xie, Zhen Zhao, Yong Guo, Kwan- Yee K Wong, Zhenguo Li, and Hengshuang Zhao. Drivegpt4: Interpretable end-to-end autonomous driving via large language model. IEEE Robotics and Automation Letters, 2024. 22, 25
Sixu Yan, Zeyu Zhang, Muzhi Han, Zaijin Wang. Qi Xie, Zhitian Li, Zhehan Li, Hangxin Liu, Xinggang Wang, and Song-Chun Zhu. M 2 diffuser: Diffusion-based trajectory optimization for mobile manipulation in 3d scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025. 3
Brian Yang, Huangyuan Su, Nikolaos Gkanatsios, Tsung- Wei Ke, Ayush Jain, Jeff Schneider, and Katerina Fragkiadaki. Diffusion-es: Gradient-free planning with diffusion for autonomous and instruction-guided driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15342-15353, 2024. 3
Jingfeng Yao, Bin Yang, and Xinggang Wang. Reconstruction generation: Taming optimization dilemma in latent diffusion models. In Proceedings of the IEEE/CVE Conference on Computer Vision and Patlern Recognition, 2025.
Chengran Yuan, Zhanqi Zhang. Jiawei Sun, Shuo Sun, Zefan Huang, Christina Dao Wen Lee, Dongen Li, Yuhang Han, Anthony Wong, Keng Peng Tee, al. Drama: An efficient end-to-end motion planner for autonomous driving with mamba. arXiv preprint arXiv:2408.03601 2024. 8
Kai Zeng. Zhangian Wu, Kaixin Xiong, Xiaobao Wei, Xiangyu Guo, Zhenxin Zhu, Kalok Ho, Lijun Zhou, Bohan Zeng, Ming Lu, et al. Rethinking driving world model as synthetic data generator for perception tasks. arXiv preprint arXiv:2510. 19/95, 2025.
16

---

Published as a conference paper at ICLR 2026
Jiang-Tian Zhai, Ze Feng, Jinhao Du, Yongqiang Mao, Jiang- Jiing Liu, Zichang Tan, Yifu Zhang, Xiaoqing Ye, and Jingdong Wang. Rethinking the open-loop evaluation of end-to-end autonomous driving in nuscenes. arXiv preprint arXiv:2305..045:0 2023. 8
Biao Zhang and Rico Sennrich. Root mean square layer normalization. Advances in neural informa- lion processing systems, 32, 2019.
Diankun Zhang, Zhijie Zheng, Haoyu Niu, Xueqing Wang, and Xiaojun Liu. Fully sparse transformer 3-d detector for lidar point cloud. IEEE Transactions on Geoscience and Remote Sensing, 61:1-12, 2023. 1
Diankun Zhang, Guoan Wang, Runwen Zhu, Jianbo Zhao, Xiwu Chen, Siyu Zhang, Jiahao Gong, Qibin Zhou, Wenyuan Zhang, Ningzi Wang, et al. Sparsead: Sparse query-centric paradigm for efficient end-to-end autonomous driving. arXiv preprint arXiv:2404.06892, 2024a.
Dongkun Zhang, Jiaming Liang. Ke Guo, Sha Lu, Qi Wang, Rong Xiong. Zhenwei Miao, and Yue Wang. Carplanner: Consistent auto-regressive trajectory planning for large-scale reinforcement learning in autonomous driving. arXiv preprint arXiv:2502. 199088 2025.
Ruijun Zhang, Xianda Guo, Wenzhao Zheng, Chenming Zhang, Kurt Keutzer, and Long Chen. Instruct large language models to drive like humans. arXiv preprint arXiv:2406.07296 2024b. 4
Songyan Zhang, Wenhui Huang, Zihui Gao, Hao Chen, and Chen Lv. Wisead: Knowledge augmented end-to-end autonomous driving with vision-language model. arXiv preprint arXiv:2412.09951, 2024c. 3
Rui Zhao, Qirui Yuan, Jinyu Li, Haofeng Hu, Yun Li, Chengyuan Zheng, and Fei Gao. Sce2drivex: generalized mllm framework for scene-to-drive learning. arXiv preprint 2XX1: 2.3
Yinan Zheng, Ruiming Liang, Kexin Zheng, Jinliang Zheng, Liyuan Mao, Jianxiong Li, Weihao Gu, Rui Ai, Shengbo Eben Li, Xianyuan Zhan, al. Diffusion-basee planning for autonomous driving with flexible guidance. arXiv preprint arXiv:2501.1556 2025.
Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all. arXiv preprint ar 2rXiv:2412.20400 2024. 3
Ziyuan Zhong, Davis Rempe, Danfei Xu, Yuxiao Chen, Sushant Veer, Tong Che, Baishakhi Ray, and Marco Pavone. Guided conditional diffusion for controllable traffic simulation. 2023 IEEE international conference robotics and automation (ICRA), pp. 3560-3566. IEEE, 2023. 3
Xin Zhou, Dingkang Liang, Sifan Tu, Xiwu Chen, Yikang Ding, Dingyuan Zhang, Feiyang Tan, Hengshuang Zhao, and Xiang Bai. Hermes: A unified self-driving world model for simultaneous 3d scene understanding and generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 27817-27827, 2025a. 3
Zewei Zhou, Tianhui Cai, Yun Zhao, Seth Z.ndd Zhang, Zhiyu Huang, Bolei Zhou, and Jiaqi Ma. Autovla: vision-language-action model for end-to-end autonomous driving with adaptive reasoning and reinforcement fine-tuning. arXiv preprint arXiv:2506. 13757, 2025b. 8, 21
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Yuchen Duan, Hao Tian, Weijie Su, Jie Shao, et al. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint 2044499504.10479, 2025. 5, 8, 9, 23, 24
17

---

Published as a conference paper at ICLR 2026
A APPENDIX
We organize the supplementary material as follows. First, in Appx. B, we address potential questions that may arise from reading the main text. We then report ReCogDrive's performance the NAVSIM and DriveBench benchmarks, along with more detailed ablation studies in Appx. C. In Appx. D, we provide details of the training data collection and processing pipeline. Appx. E describes ReCogDrive's 's training and inference implementation, including all key hyperparameters, and also provides details of the evaluation metrics used on NAVSIM and Bench2Drive. Appx. and Appx. H discuss the limitations of the method and the broader impacts. We also provide our LLM Usage Statement in Appx. G. Finally, Appx. I presents additional qualitative results, including extensive visualizations on NAVSIM and Bench2Drive, comparisons between II. and RI. trajectories, driving dialogues, and analyses of failure cases.
B QUESTIONS
Q1. What is the technical novelty in this paper?
The key novelty of this work is a reinforced cognitive framework for autonomous driving that integrates a VLM with diffusion-based planner. Unlike prior driving VLA approaches (e.g.. EMMA (Hwang et al., 2024), OmniDrive (Wang et al., 2024b)) that generate trajectories purely in the text space, our framework injects VLM-derived driving priors into diffusion planner, avoiding slow inference, infeasible actions, and trajectory errors. Within this unified framework, we further contribute three innovations:
(i) a scalable hierarchical data pipeline that mimics the sequential cognitive process of human drivers through generation, refinement, and quality control, enabling flexible adaptation of VLMS to diverse driving scenarios; (ii) a cognition-guie guiiidd diffusion planner that leverages VLM-derived priors to efficiently generate continuous and stable trajectories, surpassing plain-text and Transfuser-like (Chitta et al., 2022) decoders; and (iii) Diffusion Group Relative Policy Optimization (DiffGRPO), reinforcement learning scheme tailored for diffusion- based planners in autonomous driving VLAs, enabling simulation-driin policy optimization for safer and more comfortable driving.
Q2. Does the VLM merely act as a.feature extractor?
ReCogDrive consists Vision- Laaiuuge Model and a diffusion planner. In our framework, the VLM can produce high-level instructions or chain-of-thouge reasoning to guide planning. However, as shown in Tab. 11, reinforcement learning alone already achieves state-of-the the NAVSIM benchmark. Incorporating CoT-based guidance provides no gain, mainly due to the limited diversity NAVSIM scenarios. Moreover, Bench2Drive already supplies explicit navigation instructions. For efficiency, we therefore keep VLM-based textual guidance optional.
performance on
Q3. Why was the model trained on datasets such as CODA-LM and OmniDrive, yet not evaluated on them?
Our model is trained on a mixture of 12 open-source driving datasets, which enables handling multiple scenes and tasks. We observed that most existing benchmarks, including CODA-LM and OmniDrive, employ traditional metrics such as BLEU and ROUGE. These metrics evaluate performance by calculating the textual similarity between a model's response and the ground-truth answer. Such metrics primarily assess dataset-specific fitting rather than general reasoning ability. Therefore, to better evaluate the model's true Visual Question Answering (VQA) capabilities, we utilize the GPT-Score from the DriveLM and DriveBench benchmarks, as presented in Tab. 5.
Q4. Does DriveVQA accurately reflect the capabilities of a VLM autonomous driving?
DriveBench has shown that many VLMs decision accuracy does not degrade with visual quality, indicating a reliance on priors rather than genuine visual understanding. Therefore, we primarily use the Planning task evaluate model's practical capabilities in dynamic driving scenarios. However, for completeness and to facilitate comparison with existing work, we also provide the VQA results on the Drivel.M M and Drive benchmarks.
18

---

Published as a conference paper at ICLR 2026
Q5. Does ReCoe rely on high-quality QA annotations to generalize other domains?
ReCogDrive achieves state-of-the art results by leveraging high-quality driving QA datasets. However, key contribution of work is the scalable hierarchical data generation pipeline. Built on advanced VLMs deployed with Sglang, this pipeline can automatically generate large-scale QA annotations, followed by normalization, scoring, and filtering ensure quality. This design ensures efficiency and adaptability, enabling transfer to new domains without costly manual annotation.
Q6. Why does applying the formula in Eq. 14 to the averaged sub-metrics in Tab. I not yield the reported PDM Score?
As documented in a NAVSIM issue ', the PDMS score is calculated per scene using the formula and is then averaged across all scenes. Applying the formula sub-metries that have already been averaged is different computation and will thus lead to different final value.
Q7. Why does the paper adopt such low discount factor (0.6) for long-horizon driving task?
In Eq. 12, the discount factor applied across denoising steps rather than over the temporal horizon of the driving task. In diffusion-based planning, early denoising steps amplify high-variance noise and often destabilize learning. A smaller discount factor places greater weight on later, more reliable denoising steps, which directly improves stability and feasibility of generated trajectories. This design choice is thus motivated by the dynamics of the denoising process rather than by a short-horizon assumption of the driving task itself.
Q8. Why does the paper employ BC loss instead of the standard Kl-divergence loss?
We employ a BC-style objective as a stable surrogate for KL regularization. Direct KL constraints of- lead unstable optimization diffusion-based RL, since variance amplification across denoising steps can cause noisy and unreliable gradients. In contrast, the BC formulation provides smoother updates and better training stability, which proved crucial for policy learning effectiveness.
Q9. Does the diffusion planner only provide performance gains?
As shown in Tab. 4, beyond the +2.4 PDMS improvement over plain-text VLMM outputs, the diffusion planner also achieves 3.5x faster inference by requiring only a forward pass rather than auto- regressive generation. More importantly, it eliminates the 0.01% template mismatch errors observed in plain-text decoding, an unacceptable rate in safety-critical driving, making it both more accurate and more reliable for practical autonomous driving.
C
MORE RESULTS
Table 7: Performance comparison on Navtest Benchmark with extended metrics.
Method
NCT
DAC
EP
TTC1
C1
TL1
DDCt
LK*
EC1
EPDMS
Transfuser (Chitta et al., 2022)
97.7
92.8
79.2
92.8
100
99.9
98.3
67.6
95.3
77.8
VADv2 (Chen et al., 2024b)
97.3
91.7
77.6
92.7
100
99.9
98.2
66.0
97.4
76.6
Hydra-MDP (Li ct al,, 2024)
97.5
96.3
80.1
93.0
100
99.9
98.3
97.4
79.8
Hydra-MDP++ (Li et al., 2024)
97.9
96.5
79.2
93.4
100
100
98.9
97.7
ARTEMIS (Feng al., 2025)
98.3
95.1
81.5
97.4
100
99.8
98.6
96.5
98.3
83.1
ReCogDrive
98.3
95.9
87.7
97.7
97.9
97.4
99.3
86.0
81.9
84.2
Experiments on NAVSIM with extended metries. Hydra MDP++ (Li et al., 2024) introduces
additional evaluation metrics: Traffic Light Compliance (TL), Lane Keeping Ability (LK), Driving Direction Compliance (DDC) and Extended Comfort (EC) to more comprehensively assess driving performance. We evaluate ReCogDrive NAVSIM using these extended metrics well. Tab. 7
httpps: :/gi thub. htt.://totoooooossssoooaoomm si i on/naonin////////// ssues/11
19

---

Published as a conference paper at ICLR 2026
Table 8: Evaluation of ReCogDrive-VLL VL on DriveBench. We report DooogDDivv--LMi perfor- mance across four key driving dimensions: perception, prediction, planning, and behavior. Results are shown under three input settings: Clean (unaltered images), Corr. (images with averaged corruptions), and T.O. (text-only inputs).
Method
Size
Pereeption
Prediction
Plauning Con.
T.O.
Clean
+ Behavior Coer.
TO.
Human
47.67
$8.32
09:51
51.00
GPF--4 (Huss et ct. 2024:
Commercial
W5.87
35.22
36,48
$1.40
40.94
40.05
70.75
76.36
T3.21
45. 411
64.33
50,013
LLaVA-1. al. 20236
78
Open
23.22
22.00
22.81
22. 02
17.54
14.554
29.15
31.51
32. 4b
13. 13.60
13.
14.91
LLaVA-1.3 (Lib e 20236)
13
Open
23.00
23.87
22.87
96.98
37.78
23.35
34.25
34.94
38.85
22:30 12.99
82. 434
32.79
LLNVANNNT (iinetal, 30346)
7R
Open
24.15
19.62
13.80
$5.07
35.89
28.5 96
45.27
04.36
27.58
65.
90- 46
11.02
Inee al, 20241)
8R
Open
12.30
32.68
53.60
45.52
37.93
18.59
53.27
65.25
$4.56
64. 58
40.7%
20.14
(Abiin 2024)
4.2 11
Open
22.56
23.95
26
40.11
35.22
22.51
60.03
61:31
45.88
05.
44.57
2.
(Ab-35 2022)
4.2 0
Open
27.52
27.51
28.25
45.15
21
4.92
31,01
25.30
46.30
37
49.13
50. 16
Oryt etan 30346)
1
Open
17:02
15.07
18.47
68.11
46.63
12.77
53.55
55.70
45.26
53:35 92
53.81
23.01
QwenZ VL (Waag et.l., 2014g)
7B
Open
28.99
27.85
37.89
54.75
43.06
49.07
47.08
Qwen2 VL (Wang etall, 2024a)
72
90.13
25.92
17.70
49.35
63:07
53.05
73
DriveLM (Sima 2024)
Specialist
16.85
16.00
8.75
44.33
67.68
65.22
42:35
40.07
27.83
Dolphins (Ma 2024)
7B
Specialist
9.50
10.84
11.01
32.91
58.77
92.08
5.81
11.92
R.CogDrvv-VLM
Specialist
64.95
72.5
71. 21.70
42.
76
44.00
Table 9: Impact of Different BC Loss Weights A. Table 10: Impact of Different Min Samplings o min
exp
BL
Wt.
NC
DAC
TTC
Conf.
EP
PDMS
Min Samp.
NC
DAC
TTC
Conf.
EP
PDMS
0.001
97.9
97.3
94.9
100
87.3
90.8
0.01
98.2
97.8
95.6
100
82,8
89.5
0.01
97.9
97.5
94.9
100
86.0
90.3
0.02
97.9
97.3
94.9
100
87.3
90.8
0.1
97.6
97.2
93.9
100
85.0
89.3
0.04
97.9
97.7
95.3
100
85.7
90.5
Table 11: Comparison between trajectory-only (Only Traj), adding high-level command (w/ High Com.), and chain-of-thouggh (w/ CoT).
Table 12: Impact of different discount factors y. We use discounting to reduce the influence of early-step noise during the diffusion process.
Mode
NC
DAC
TTC
Conf.
EP
PDMS
Discount
NC
DAC
TTC
Conf.
EP
PDMS
Only Traj
97.9
97.3
94.9
100
87.3
90.8
0.6
97.9
97.3
94.9
100
87.3
90.8
High Com.
97.9
97.3
94.9
100
87.2
90.8
0.8
98.0
97.6
95.3
100
85.9
90.5
w/ CoT
97.8
97.2
94.8
100
87.1
90.7
1.0
97.9
97.5
94.9
100
86.0
90.3
compares our approach against existing methods under this metries. ReCogDrive achieves state of the art scores in Driving Direction Compliance (DDC), No at-fault Collisions (NC), Ego Progress (EP) and Time to Collision (TTC), and delivers a 1.1 improvement in EPDMS over ARTEMIS (Feng et al., 2025), demonstrating the effectiveness of our method.
Evaluations of VLMs across driving tasks. Tab. 8 reports results on DriveBench across perception, prediction, planning, and behavior. ReCogDrive- VLM achieves the best overall performance and shows strong robustness under both corrupted and text-only settings, outperforming general-purpon VLMMs (e.g., Qwen2-VI., IntemVV..2) and specialist models (e.g.. DriveL.M, Dolphins). These results demonstrate the strong driving scene understanding capability of our ReCogDrive-VLM,
Impact of Different Behavior Clone Loss Weights. Tab. 9 examines the impact of the BC loss weight A. As X decreases, the policy learns more aggressively, and at À = 0.001 ego progress increases to 87.3 with slight declines in NC, DAC, and TTC, yielding the highest PDMS.
Impact of Different Min Sampling Values. Following (Ren al., 2024; Nichol & Dhariwal, 2021), we apply a cosine schedule to the diffusion noise variances σ and clip them to a minimum value o exp Clipping 0% to a nonzero minimum encourages the diffusion policy to sample more diverse trajectories, yet overly large o exp can destabilize training, Setting o cup 0.02 achieves a PDMS of 90.8, as shown Tab. 10.
Effect of VLM guidance modes. As shown in Tab. 11, outputting only trajectory achieves per- formance comparable adding high-level command guidance, with almost identical PDMS scores. Interestingly, incorporating chain-of of-thuughtt reasoning does not further improve the results; instead, it slightly decreases the PDMS score by (..1. This suggests that the current NAVSIM benchmark
20

---

Published as a conference paper at ICLR 2026
Table 13: Planning performance on the nuScenes val set. We train our model on Bench2Drive, and report both zero-shot transfer results on nuScenes and performance after fine-tuning on nuScenes.
1.2 (m)
Collision (%))
Method
Avg.
Avg.
UniAD (Hu et al., 2023)
0.20
0.42
0.75
0.46
0.02
0.25
0.84
0.37
VLP (Pan al., 2024)
0.30
0.53
0.84
0.55
0.07
0.15
VAD (Jiang et al., 2023)
0.17
0.34
0.60
0.37
0.07
0.10
0.24
0.14
DriVeVLM (Tian et al., 2024)
0.18
0.34
0.68
0.40
0.10
0.22
0.45
0.27
DiMA-Dual+ (Hegde et al., 2025)
0.14
0.27
0.46
0.29
0.05
0.07
0.15
0.09
AutoVLA (Zhou al., 2025b)
0.22
0.39
0.61
0.41
0.10
0.17
0.28
0.18
Revv (zero-shot)
1.34
1.91
2.69
1.98
0.07
0.57
0.82
0.82
ReCogDrive (fine-tune)
0.17
0.34
0.63
0.38
0.07
0.09
0.16
0.10
Table 14: Sensitivity of PDMS to the EP weight Table 15:
Sensitivity of PDMS to the TTC
WEP (fixed WrTC 5.0, wc 2.0).
weight wrrc (fixed wgP 10.0, we 2.0).
NC
DAC
TTC
Conf.
EP
PDMS
NC
DAC
TTC
Conf.
EP
PDMS
5.0
98.6
97.8
96.2
100
84.0
90.3
5.0
97.9
97.3
94.9
100
87.3
90.8
10.0
97.9
97.3
94.9
100
87.3
90.8
10.0
98.5
97.7
95.9
100
85.1
90.5
20.0
97.8
97.3
94.2
100
87.3
90.5
20.0
98.6
97.7
96.2
100
84.5
90.5
30.0
97.6
97.2
93.1
99.6
87.6
90.0
30.0
98.5
97.8
96.1
100
84.5
90.4
lacks sufficient diversity to demonstrate the benefits of high-level command and CoT reasoning, so we keep these guidance optional.
Impact of Different Discount Factors. Tab. 12 shows the impact of the discount factor on RL fine-tuning. When y 1.0, all timesteps. including the very noisy early steps, contribute equally to the policy gradient, which amplifies high variability noise and destabilizes learning. In contrast, setting ? 0.6 focuses the update on later, more reliable steps and achieves the best PDMS of 90.8.
Open-loop evaluation on nuScenes. We report planning results on the nuScenes val set in Tab. 13. Trained on Bench2Drive, our method achieves strong zero-shot performance on nuScenes, and fine-tuning further yields excellent results. In particular, fine tuning substantially lowers collision rates, outperforming several prior planners, while maintaining competitive L2 displacement errors across all horizons.
PDM Score Coefficient Sensitivity Analysis. Our PDMS reward is constructed from five compo- nents: EP, TTC, Comfort (C), NC, and DAC. Concretely, the PDMS is computed as:
WEP EP + WTTC TTC + we C) .).
PDMS NC x DAC
(13)
WEP + WTTC + we
Tabs. 14 and 15 present the sensitivity of PDMS to the coefficients WEP and wTrc respectively. Increasing the EP weight WEP makes the policy more aggressive, meaning that EP rises while NC, DAC, and TTC decrease slightly, and the overall PDMS correspondingly decreases when is set too large. In contrast, increasing the TTC weight wrrc results in safer tame-to-colliseee behavior with higher TTC but reduced EP, which leads to diminishing PDMS improvements as wrrc increases. Based on these results, we choose the configuration Wpp = 10.0, WTTC = 5.0, and wt = 2.0 as our PDMS reward setting, since it provides the best trade-off between progress and safety.
21

---

Published as a conference paper at ICLR 2026
D TRAINING DATASETS CONSTRUCTION
D.1 DATA COLLECTION
We collect 12 open-source driving QA datasets, including Talk2Car (Deruyttere al., 2019), SUTD (Xu et al., 2021), NuScenes-QA (Qian et al., 2024), OmniDrive (Wang et al., 2024b), and others, yielding over 3.1 million question-answer pairs that cover perception, prediction, and planning tasks across diverse real-world scenarios.
(Deruyttere et al., 2019) is built on the nuScenes (Caesar et al., 2020) dataset and contains 850 videos from the nuScenes (Caesar et al., 2020) training set. This dataset has 11,959 natural language commands.
Talk2car
SUTD (Xu et al., 2021) contains 10,080 in-the-wild videos and annotated 62,535 QA pairs. These videos are obtained through a combination of online collection and offline shooting, covering various weather conditions, times, and road conditions.
DRAMA (Malla et al., 2023) a dataset collected to investigate risk location and natural language description in driving scenarios. It contains 17,785 interactive driving scenarios.
NuScenes- QA (Qian al., 2024) encompasses 34,149 complex autonomous driving scenes and 459,941 question-ans answer pairs, with various types of questions. It aims to evaluate model's ability to understand and reason about complex visual data multi-modal, multi-frame, and outdoor scenarios.
DriveGPT4 (Xu et al., 2024) is built based on the BDD-X (Kim et al., 2018) dataset, which contains about 20,000 samples. By dividing them into 16,803 training segments and 2, 2.1 testing segments, question-answer pairs generated using the control signal data and text annotations.
LingoQA (Marcu et al., 2024) contains 28K unique short video scenarios and 419K annotations. The dataset covers various questions related to driving scenarios, including aspects such as behaviors, scenery, and perception.
DriveLM (Sima et al., 2024) consists of a training set of 4,072 frames and a validation set of 799 frames, with an average of 91.4 QA pairs per frame.
MAPLM (Cao et al., 2024) contains point-cloud BEV projections and surround view images of various traffic scenes, such as highways and urban roads, and is equipped with element-level, lane- level, and road-level scene descriptions.
NuInstruct (Ding et al., 2024) is a dataset constructed based on Nuscenes (Caesar et al., 2020), containing 91K multi-view video cetructioo-ressonse pairs in 17 subtasks.
CODA-LM (Chen et al., 2024a) comprises 9,768 real-world driving scenarios with 41,722 textual annotations for critical road entities and 21,537 annotations for road comee cases.
OminiDrive (Wang et al., 2024b) covers 3D perception, reasoning, and planning tasks, including offline and online question-response tasks.
Senna (Jiang et al., 2024a) design a series of planning-oriented QAs including scene description, traffic signal detection, vulnerable road user identification, motion intention prediction, meta-action planning and planning explanation. Since the senna dataset is not publicly available, we used Qwen2.5-VL-72B (Bai al., 2025) for question-answe -ansiirnn-nwwrreeeee data annotation.
D.2 SCALABLE HIERARCHICAL DATA PIPELINE
To systematically enhance VI.M performance for autonomous driving, we design scalable hier- archical data pipeline consisting of three stages: Generation, Refinement, and Quality Control, as shown in Fig. 2. This design allows us to construct diverse, semantically consistent, and high-quality annotations at scale.
Generation. We first leverage Qwen2.5-V2 (Bai et al., 2025) with crafted prompts to generate anno- tations across the full spectrum autonomous driving tasks on NAVSIM (Dauner et al., 2025). These tasks span perception (e.g., scene description, key object identification, road marking recognition, traffic light classification, vulnerable road user detection), prediction (e.g., motion prediction), plan- ning (e.g., high-level command prediction and decision explanation), and advanced reasoning (e.g..
22

---

Published as a conference paper at ICLR 2026
Holissic Scenes Descrigtonn
Objictt Detaction
Vulnerable Road User
OpPrwviding iccunioe indorsanniing of the drivingg unvimomentt
O-Denat every bicycle, pabisttion, and farthest 1.72 Ollyect 11 6.49 0.60. 1,36, pedestrias) Digect (16771 8.10, 9.5,, 0.77 0.50, 1,67, 1,62, polcerson)...
D. you vulneable red uners withn meters shoed
eyclists. motareydiiss, pecketriuns?
Ah scese set dusisg derlight with cloar sies and leight sunlight indicttig Iorocaale watha ortiitions for drivin....
A/Yoe. peckettian 10 34 sheed and
podesttan located meters aboad mee ngtt.
Traffic State Estimation
fligh Level Planning
QuPleese the current truffie congestion
O.Wile your for the secords?
A:TTw road aspecrs be wodenndy oregasied with nommal flane of trafiii....
the road.
A-STOP. STRAIGHT
Explainable Planning
Key A geat Identification
Q-Explaz the most likes meson for thi decidion in ate oc bwc condis sertence.
Qiidentryy the most influeetial object that affecss the curmet driving slbutio..
A: The dotaton to sup the car lidlly dae the prrecace of pedestnizs cmestng the radd abeed, as
A-Tha most untiantiaal object in the curvent caterag atbution is the pedestzan crossing the roca alocad...
obeerved in the drwwng encrrmment Adittonally See traffic light apgears nel recemstating stop.
Motion Prodiction
CounterCactual Resscaing
Rehavior Descritioon
Q-Plesse poodict thas tuture ativag betaroors.
O If you follow the trgeckory TPG. (1.53, 0.0. 0.01, 12.90, 0.0, 9.6). [4 31, f.i, 0.0y ..... wher would tuppent
Ordeugrise the current driving behenior of the vehidle Je is curnetly supped intersection likey waiting trafise ligtt change Its gradial deceteration over the past time staps...
<Obyett STRA HT
Objec STRA
A Becasse the tratic light is red the velude should remain stoped if cottinus forword, wriill onlidd with the load valticle
Objet STRAIG!T7
Figure 7: Example of data generated by our Scalable Hierarchical Data Pipeline. The pipeline produces structured QA annotations that cover four levels of autonomous-driving capability, from Foundational Perception and Dynamic Understanding to Planning and Reasoning and Advanced Reasoning, as illustrated by the multi-task QA set in this scene.
counterfactual scenarios inspired by OmniDrive (Wang et al., 2024b)). To strengthen supervision, we further incorporate NAVSIM's existing sensor and trajectory labels into the generation process. Applying this stage yields large pool of candidate QA pairs.
Refinement. Since open-source datasets employ heterogeneous formats, we unify them through normalization and linguistic enrichment. (i) Data Normalization: Different datasets adopt different bounding- box styles. For example, DriveLM (Sima et al., 2024) represents an object as "<c2, CAM BACK,864.2,4666. (ccar>(c2,584,4777 058,,,44777,0035116""" "<car><FRONT BINNNT___WW>wxxxxxrrrr V1, 22, ///<//ox>" following the Intern (Zhu et al., 2025) pre-training convention. Coordinates Fı, ,1, 42; 1/2 are rescaled to the integer range [0. 1000] relative to image resolution. (ii) Data Augmentation: To enrich linguistic diversity, especially for datasets with limited QA templates (c.g., CODA-LM with only three formats, DRAMA with terse answers), we employ a two-step process. First, an LLM paraphrases question templates to expand structural variation. Second, Qwen2.5-VL. is prompted with the original image, question, and answer generate more detailed, natural responses. This refinement ensures consistency in structure and greater variety in language.
while Nulnstruct (Ding et al., 2024) uses We convert all such variants into a standardized tag format:
Quality Control. Finally, we implement a rigorous filtering step. Qwen2.5-Vi assigns each QA pair a quality score based on predefined linguistic and semantic criteria. Any pair scoring below 60 is discarded. After this filtering process, we retain 2.3M high-quality QA pairs. Notably, applying the pipeline to NAVSIM produces 752K well-structured pairs, which we use fine-tune the VLM, substantially enhancing its planning and reasoning capabilities.
D.3 DATASET STATISTICS
The dataset consists of 41% perception samples, 11% prediction samples, and 24% planning samples, aimed at improving VLMs' understanding of driving scenarios. We also include 24% visual instruc- tion samples to maintain the model's instruction-followin ability. For NAVSIM training, ReCogDrive is trained using a mixture of NAVSIM trajectory data (5%), our newly generated 752K cognition- aligned QA pairs (22.9%), open-source driving QA datasets totaling 2.3M samples (68. 188.1%) and 665K LLaVA instruction data (3.9%). ). This composition balances driving-specific reasoning with gen- eral VLM instruction capability. For Bench2Drive adaptation, the NAVSIM -AVsIM-trained model is further fine-tuned on Bench2Drive execution trajectories (79.2%) together with Bench2Drive driving - annotations (20.8%), enabling the model to acquire Aimulationsppccifi driving priors required for closed-loop CARLA evaluation.
23

---

Published as a conference paper at ICLR 2026
E EXPERIMENT DETAILS
In this section, we first introduce the evaluation metrics of NAVSIM and Bench2Drive, and then detail our model configuration, the hyperparameters used in the three-stage training and evaluation, and our hardware setup.
Evaluation Metries. We evaluate our method two primary benchmarks. For the planning-oriented NAVSIM benchmark, we use the official Predictive Driver Model Score (PDMS). closed-loop metric that holistically assesses safety (NC, DAC, TTC), comfort (C), and progress (EP):
5. + TTC + + .
PDMS NC X D
(14)
12
For the CARLA-based Bench2Drive benchmark, which focuses on safety-critical scenarios, report the Success Rate (SR) and the average Driving Score (DS). The DS combines Route Completion (RC) with multiplicative penalties for infractions (IS):
Nucil
SR
success
DS
RC x
II IS
(15)
Nivial
Miotal
Model Architecture and Hyperparameter Details. Our model architecture consists of two main components: Vision-Language Models (VLMs) and diffusion-based trajectory planner. For the Vision-Language Models, we utilize the pre-trained Intern VL3-8B and Intern VL3-2B (Zhu et al., 2025) model. The model processes visual inputs by splitting images into patches: on NAVSIM, use a single front-view image divided into 9 patches, while on Bench2Drive we use multi-view inputs from six cameras, each cropped into 3 patches. The input images are cropped into 448x448 pixel patches. For the diffusion planner, we use the DiT (Peebles & Xie, 2023) architecture, employing DDPM/DDIM (Ho et al., 2020) denoising methods. During training, the denoising process involves 100 steps, while for inference, the diffusion process is reduced to 5 steps. The hidden layer size of the DiT architecture set to 384, with a head size of and 16 layers.
Table 16: Hyperparameters for ReCogDrive.
Stage
Parameter
Value
Number of epochs Batch size
1024
Learning rate Weight decay Warmup ratio Learning rate schedule
-e-5
I
0.05
0.10
Cosine
Number of epochs Batch size Leaming rate Learning rate schedule
200
512
II
1c
Cosine
Weight decay
1 x 10 4
Number of epochs
10
Batch size
128
Learning rate
1e-4
Learning rate schedule
Cosine le-2
BC loss weight
III
Denoised clip threshold
+1.0
Noise clip threshold
+3.0
Minimum denoising std
0.02
Minimum log-variance std
0.10
Discountt factor
0.6
Training Configuration. In the first stage, we fine-tune the Intern VLM for three epochs with all parameters unfrozen and batch size of 1,024 curated 3M real-world driving QA corpus
24

---

Published as a conference paper at ICLR 2026
that unifies Talk2Car (Deruyttere et al., 2019), SUTD (Xu et al., 2021), DRAMA (Malla al., 2023), NuScenes-Qa QA (Qian et al., 2024). DriveGPT4 (Xu et al., 2024), LingoQA (Marcu et al., 2024), DriveLM (Sima et al., 2024), MAPLM (Cao et al., 2024), Nulnstruct (Ding et al., 2024), CODA-LM (Chen et al., 2024a), Orniirive (Wang et al., 2024b), Senna (Jiang et al., 2024a), LLaVA Instruction-QA (Liu al., 2023a) and 752K additional QA pairs generated by our hierarchical data pipeline, providing a strong real-works foundation tailored for both DriveBench and NAVSIM. We use A with a base learning rate of 4e S weight decay of 0.05, and a cosine learning rate schedule with 10% linear warmup. For CARLA-basee simulation, we further adapt the real-world- model by fine-tuning it with Bench2Drive trajectory data and Bench2Drive-O0 using identical hyperparameters to ensure compatibility with the CARLA environment.
In the second stage, we train the diffusion-ban based planner with imitation learning on trajectory supervision from both NAVSIM and Bench2Drive for 200 epochs and a batch size of 512, using AdamW W with a learning rate that warms up to le over the first 1.5% of steps, then decays cosinely to le and weight decay of le=t.
In the third stage, we perform DifiGRPO reinforcement learning using the full NAVSIM and Bench2Drive training data for 10 epochs with a batch size of 128, using AdamW and a cosine schedule that decays the learning rate from 1e to O. Additionally, we stabilize diffusion training and inference by clipping the reconstructed clean estimate Zo +1.0, clipping Gaussian noise samples to +5.00, enforcing a nonzero floor on the denoising standard deviation o; of 0.02 for training, clamping oe to at least 0.10 when computing log pog,(11111 (24x, and applying discount factor & 0.6 during RL fine-tuning to downweight early, noisy timesteps. During RL fine-tuning, we optimize the planner using the PDMS reward with weighting coefficients
10.0 (EP), B 5.0 (TTC), and 2.0 (Comfort), which we identified as providing the best trade-off between progress and safety.
Hardware Configuration. We implement ReCogDrive on Debian with Py PyTorch, training across four noch equipped with an Intel@ Xcon@ Platinum 8457C CPU and eight NVIDIA H20 GPUs (32 GPUs in total). Inference is performed on single node with 8 GPUs.
F LIMITATIONS
Although ReCogDrive achieves state of the art performance on the NAVSIM benchmark, it still faces several limitations. These include the difficulty of processing multiple camera inputs, handling video frame sequences, and relatively high inference latency. Future work may address these issues by designing 3D vision encoder that aligns with textual features and developing more efficient model architectures. We also plan to deploy and evaluate our model on real vehicles.
G LLM USAGE STATEMENT
Large Language Models (LLMs) were employed solely writing assistants improve the clarity, grammar, and readability of the manuscript. They were not involved in research ideation, methodology design, data analysis, or result interpretation.
H BROADER IMPACTS
Our research promotes the application of Vision-Langueen Models (VLMs) in the autonomous driving domain. Through simulator-assisted reinforcement learning, our model more effectively mitigates collision risk and generalizes to rare, challenging scenarios. This advancement could lead to safer, more reliable autonomous systems capable of handling real-world driving scenarios.
I QUALITATIVE RESULTS
We first present extensive qualitative visualizations of ReCogDrive on NAVSIM and Bench2 highlighting its ability to produce safe and smooth trajectories. Next, we provide trajectory compar- isons between ReCogDrive-I IL and ReCogDrive-RR to demonstrate the effectiveness of our proposed DiftGRPO. We further showcase dialogue examples generated by ReCogDrive, illustrating its cog-
25

---

Published as a conference paper at ICLR 2026
nitive reasoning capabiliity Finally, analyze representative failure cases NAVSIM to provide insights into current limitations.
1.1
QUALITATIVE RESULTS ON NAVTEST SPLITS
We present representative cases from the Navtest splits, highlighting ReCogDrive's ability follow navigation instructions while ensuring safety and smoothness (see Fig. 8 and Fig. 11).
1.2 QUALITATIVE RESULTS ON BENCH2DRIVE TEST SCENARIOS
We evaluate ReCogDrive on challenging scenarios from Bench2Drive, demonstrating its robust- ness under diverse traffic conditions and rare corner cases, while consistently maintaining strong performance in closed-loop settings.
1.3 COMPARISON BEFORE AND AFTER REINFORCEMENT LEARNING
Representative qualitative comparisons are shown Fig. 12-16, where we contrast Transfuser, imitation learning (IL), and reinforcement learning (RL) variants of ReCogDrive, demonstrating the progressive improvements decision making, trajectory stability, and safety awareness.
1.4 FAILURE CASES
We also present representative failure cases of ReCogDrive the Navsim benchmark, shown in Fig. 17. These include instances of aggressive driving caused by the model's 's tendency to prioritize ego progress, leading assertive maneuvers. In addition, the relatively weak perception capability of current VLMMs means that directly feeding multi-view camera inputs sometimes results in suboptimal performance, particularly in turning scenarios. We also observe unsafe following distances due to insufficient modeling of surrounding vehicle behaviors, reflecting the limitations of VLMs in motion prediction. Finally, we identify cases where predicted trajectories are visually close to the ground truth but still scored as failures, suggesting that the evaluation metrics may suffer from false negatives.
1.5 MODEL-HCMAN DIALOGUE
As shown in Fig. 18-20, ReCogDrive engages in interactive dialogue by providing detailed scene descriptions, recognizing traffic signs, reasoning about driving intent, and generating safe turning trajectories.
26

---

Published as a conference paper at ICLR 2026
(a) Go straight
(b) Turning
(c) Intersection
Figure 8: Qualitative results the Navtest benchmark.
27

---

Published as a conference paper at ICLR 2026
(a) Go straight
(b) Turning
(c) Intersection
Figure 9: Qualitative results on the Navtest benchmark.
28

---

Published as a conference paper at ICLR 2026
(a) Turning Right
(b) Turning Left
Figure 10: Qualitative results on the Bench2drive benchmark.
29

---

Published as a conference paper at ICLR 2026
(c) Turning Left At Night
(d) Stop On Red, Proceed On Green.
Figure 11: Qualitative results on the Bench2drive benchmark.
30

---

Published as a conference paper at ICLR 2026
Front View
(a) Transfuser
(b) ReCogDrive-IL (c) ReCogDrive-RL
Figure 12: Qualitative comparisons on the Navtest benchmark.
31

---

Published as a conference paper at ICLR 2026
Front View
(a) Transfuser
(b) ReCogDrive-IL (c) ReCogDrive-RL
Figure 13: Qualitative comparisons on the Navtest benchmark.
32

---

Published as a conference paper at ICLR 2026
Front View
(a) Transfuser
(b) ReCogDrive-IL (c) ReCogDrive-RL
Figure 14: Qualitative comparisons on the Navtest benchmark.
33

---

Published as a conference paper at ICLR 2026
Front View
(a) Transfuser
(b) ReCogDrive-IL (c) ReCogDrive-RL
Figure 15: Qualitative comparisons on the Navtest benchmark.
34

---

Published as a conference paper at ICLR 2026
Front View
(a) Transfuser
(b) ReCogDrive-IL (c) ReCogDrive-RL
Figure 16: Qualitative comparisons on Navtest benchmark.
35

---

Published as a conference paper at ICLR 2026
(a) Aggreesive Driving
(b) Mult-View Perception Probiem
(G) Unsafe following Distance
Inexplicable fallrre
Representative failure cases of ReCogDrive on the Navsim benchmark. including aggres- sive driving, multi-view perception problem, unsafe following distances, and incxplicable failures. Zoom in for better view.
Figure 17:
36

---

Published as a conference paper at ICLR 2026
Please make description for the driving scene.
The scene is during daylight under clear sky and scattered clouds. The road is smooth and appears to be a multi-lane highway with no visible obstacles. Traffic lights ahead continued forward movement. A silver sedan is positioned in front on the right lane, moving steadily ahead, while white van is further up, also proceeding forward. On the far right, a black car is visible, maintaining its lane. truck is observed slightly ahead on the left, and bus further back, both maintaining their lanes and moving forward.
showing green, allowing for
Identify and describe the road markings and signs.
The image shows displays clear road scene where white dashed lines indicate lane separations, guiding drivers to stay within their respective lanes, while white arrow on the road suggests direction travel the lane it's in. A round blue sign with red slash over black right-turning arrow signals that right turns prohibited requiring drivers to either proceed straight or turn left if permitted. The presence of pedestrian crossing implied by the zebra lines the left, advising drivers to be cautious and ready to stop if pedestrians are present.
junction ahead,
Predict the vehicle's trajectory based on front- view image, historical trajectory and command.
Figure 18: An example shows ReCogDrive's capability in road sign recognition and intersection- turn planning: given a front-view image and a user query, ReCogDrive generates a detailed scene description, identifies key traffic signs at the junction, and predicts safe turning trajectory.
37

---

Published as a conference paper at ICLR 2026
Please make description for the driving scene.
The current driving scene is set during daylight hours under partly cloudy skies, providing ample natural lighting. The road surface appears smooth and clear of any obstacles, with few vehicles ahead maintaining a steady pace. To the left, there's line of cars, including dark sedan and white sedan approaching an intersection. advised as the cars ahead are preparing to move through the intersection. The presence of well-maantained roadside greenery and the archway adds to the urban aesthetic but does not impact the driving conditions.
right, taxi is visible. Caution is
Identify the most influential object that affects the current driving situation.
The most influential object in the current driving situation is the traffic light ahead, which currently red. traffic light is crucial as dictates the flow of vehicles and ensures safety at the intersection. The cars ahead, including silver vehicle directly front of the viewer, have come stop, indicating compliance with the traffic signal. Given the red light, the appropriate driving action to safely come to complete stop behind the silver vehicle, maintaining safe following distance. This will allow the driver to respond appropriately when the light turns green, ensuring smooth and safe transition through the intersection.
Predict the vehicle's trajectory based on the front- view image, historical trajectory and command.
An example shows ReCogDrive's capability in scenario understanding and trajectory planning: at a red-light intersection with braking vehicles ahead, ReCogDrive describes the traffic lights and road conditions, identifies both the traffic light and lead vehicles, and predicts accurate stop trajectory.
Figure 19:
38

---

Published as a conference paper at ICLR 2026
Please make description for the driving scene.
The scene is set during daylight with clear skies, suggesting favorable weather for driving. The road is a multi-lane highway with smooth surfaces. Traffic lights ahead green, indicating that it's safe proceed. Several vehicles are present, including a white car to the left and an orange convertible ahead, both moving forward at moderate speed. Pedestrians can be seen on the sidewalks the right. The overall setting is calm, with no immediate hazards or congestion. Based on this description, you can proceed with caution, maintaining safe distance from the vehicles ahead.
Please predict the orange car future driving behaviors
The red car is positioned the right lane, approaching an intersection with a green traffic light. Based on the visual context, the car will likely continue to keep its current speed and proceed straight through the intersection. This decision is supported by the clear visibility of the green light and the absence of any obstacles or pedestrians in the car's path.
What is your plan for the next three seconds?
KEEP, STRAIGHT
Predict the vehicle's trajectory based the frant- view image, historical trajectory and command.
Figure 20: An example shows ReCogDrive's capability in behavior prediction and planning: at green-light intersection, ReCogDrive describes the road and traffic conditions, identifies the green signal and the lead vehicle's behavior, generates a high-level "KEEP.STRAIGH"" command, and predicts the precise continuous trajectory for the ego vehicle.
39


Reviews:
[{"review_id": "mxZFU8NZoa", "reviewer": "Reviewer_RX3X", "summary": "This paper proposes ReCogDrive, an end-to-end autonomous driving framework that combines a Vision-Language Model (VLM) with a diffusion-based planner and an RL fine-tuning stage (DiffGRPO). The key idea is to use a cognitive data pipeline to teach the VLM human-like reasoning skills, then guide continuous trajectory generation through diffusion and reinforcement learning. The experiments on NAVSIM and Bench2Drive look solid — the model consistently outperforms both VLM- and diffusion-based baselines and achieves strong closed-loop results.", "questions": "1. Could the authors provide the weighting coefficients used for the PDMS reward and justify their selection?\n\n2. Is the VLM backbone frozen during diffusion and RL training, or does it receive gradient updates through the planner?\n\n3. What is the exact runtime efficiency (latency per frame) compared to baseline diffusion and text-based methods?\n\n4. Can the authors show qualitative examples of failure modes or collisions to highlight what DiffGRPO specifically improves?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is well-written and easy to follow, with clear figures and good motivation.\nI like the overall integration of cognitive reasoning (via VLM) and low-level control (via diffusion + RL).\nResults are strong and consistent across benchmarks, and the ablation studies help justify each component.\nTechnically, the approach is sound.", "weaknesses": "**(a) Novelty Overlap / Incremental Concerns**\n\nSeveral very recent works (e.g., Drive-R1 Li et al., 2025 and AlphaDrive Jiang et al., 2025) already explore reinforcement learning and reasoning within VLM-based driving.\n\nLikewise, Gen-Drive (Huang et al., 2025) combines diffusion with RL for driving policy optimization.\n\nGiven these, the claim of “first to apply reinforcement learning to VLA models” may be over-stated, and the conceptual contribution, though strong in integration, is not entirely novel.\n\n**(b) Ablation Limitations**\n\nNo comparison against other RL-enhanced planners such as under identical environments, limiting attribution of improvements to DiffGRPO specifically.\n\nThe reward design (PDMS) could be discussed in greater depth—its weighting and sensitivity are unspecified.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6PkGSnLE0s", "reviewer": "Reviewer_gJsD", "summary": "The paper proposes ReCogDrive, to address challenges in end-to-end autonomous driving framework that integrates a Vision-Language Model (VLM) with a diffusion-based planner and a reinforcement learning (RL) module. \nThe key contributions include:\nIntegrates the diffusion planner with VLM models that translates VLM outputs into continuous, stable driving trajectories, addressing the modality mismatch between language and action spaces in way to provide cognitive guidance and enabling precise trajectory generation.\nA Diffusion Group Relative Policy Optimization (DiffGRPO) algorithm that fine-tunes the planner using RL to improve safety and comfort.\n\nThe model is evaluated on NAVSIM and Bench2Drive benchmarks, achieving state-of-the-art performance in both open-loop and closed-loop settings. It also demonstrates superior performance on DriveLM and DriveBench for visual question answering (VQA) and reasoning tasks.", "questions": "Please refer weaknesses", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The integration of VLMs with diffusion models and reinforcement learning addresses key limitations in current end-to-end driving systems.\n- The structured approach to data generation and refinement is scalable, and enables the creation of high-quality VQA datasets for autonomous driving.\n- The use of a diffusion planner to bridge the gap between discrete language outputs and continuous control actions.\n- The introduction of DiffGRPO is a thoughtful addition that enhances the planner’s ability to generate safer and more comfortable trajectories.\n- The paper presents extensive experiments, ablation studies, and qualitative analyses across multiple benchmarks, demonstrating the robustness and generalization of the approach.", "weaknesses": "- Diffusion Planner: integration of diffusion planner with VLM is explored in ORION. Despite improvements using  the diffusion-based approach still incurs higher inference latency compared to VLP, DiMA which distills VLM knowledge to simpler planners. Comparisons with these relevant methods like ORION, VLP and DiMA are missing.\n- Zero shot testing: The model is evaluated in simulation environments (NAVSIM, CARLA), but lacks real-world deployment or testing, which is crucial for autonomous driving applications. It would be if authors perform zero-shot testing like training on Bench2Drive and testing on nuScenes.\n- Training Complexity: The system’s architecture is quite complex, involving multiple stages (VLM pretraining, diffusion planning, RL fine-tuning), which may pose challenges for reproducibility and deployment. Detail training procedures are missing.\n- Optional Use of VLM Reasoning: While the VLM can provide high-level reasoning, the paper notes that such guidance did not significantly improve performance in current benchmarks, raising questions about the practical use VLM high-level reasoning. This also raises a question  whether ReCogDrive can be applied to UniAD, VAD like architectures without VLM model, and UniAD-ReCogDrive gain improvements or not on base UniAD performance.\n\n- ORION, ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generation, ICCV 2025\n- VLP, VLP: Vision Language Planning for Autonomous Driving, CVPR 2024.\n- DiMA, Distilling Multi-modal Large Language Models for Autonomous Driving, CVPR 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0giFPevaKm", "reviewer": "Reviewer_diW3", "summary": "This paper proposes ReCogDrive, a novel end-to-end autonomous driving framework that integrates a Vision-Language Model (VLM) for cognitive reasoning with a diffusion planner to generate continuous trajectories. The framework is further refined using a reinforcement learning stage called DiffGRPO to improve safety and comfort, achieving state-of-the-art results on the NAVSIM and Bench2Drive benchmarks.", "questions": "1. Could you please provide a more precise definition of the problem setup? Specifically, what are the exact of the inputs $I_{cam}$, $L_{nav}$, and $S_{ego}$? Similarly, how is the output $V_{traj}$ defined? What is the rationale for predicting headings when waypoints (which imply direction) are already being generated?\n2. In Section 4.2, what is the architectural motivation for using both the full VLM hidden states $F_h$ and the pooled global embedding $\\bar{F}_h$? What distinct roles do these two forms of conditioning play?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. Well-designed Framework: The core architecture that integrates a cognitive VLM, a diffusion planner, and an RL refinement stage is well designed. It provides an effective solution to the modality mismatch problem for VLM-based agents.\n2. Strong Empirical Validation: The paper's claims are well-supported by strong empirical performance on two challenging benchmarks (NAVSIM and Bench2Drive), comprehensive ablation studies that isolate the gains from each component, and a methodical data curation pipeline.", "weaknesses": "**Major Weaknesses:**\n\n1. Lack of Motivation for GRPO: The paper provides insufficient insight into the choice of Group Relative Policy Optimization (GRPO). It is not clear why GRPO is better suited for optimizing a diffusion policy in this context compared to other well-established RL algorithms (e.g., PPO, or a simpler policy gradient method like REINFORCE).\n2. Unclear DiffGRPO Algorithm Design: The description of the DiffGRPO algorithm seems to imply a fundamental difference from typical GRPO. In GRPO, trajectories within one group are computed starting from the same initial state. This design choice is critical for the advantage calculation to be valid, and the paper needs to elaborate more on this.\n\n**Minor Weaknesses:**\n\n1. Unclear Methodology and Data Pipeline: The methodology, particularly regarding the specifics of the data pipeline (Section 4.1) and the diffusion planner architecture (Section 4.2), is explained at a high level and lacks detailed examples for full clarity.\n2. Messy Symbol Use: The paper suffers from unclear and at times inconsistent mathematical notation (e.g., $E_{act}$, $E_{hist}$, $D_{act}$, $x_{hist}$ are undefined, and $r_{1...G}$ in Eq. 9 is ambiguous).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ntyQLrbV7Z", "reviewer": "Reviewer_wtRP", "summary": "ReCogDrive introduces a reinforced cognitive framework that integrates high-level semantic reasoning from a Vision-Language Model (VLM) with continuous control via a Diffusion Planner for end-to-end autonomous driving. Through a three-stage training pipeline—driving Q&A pretraining to bridge the domain gap, imitation learning for language-to-action mapping, and reinforcement learning in simulation for safety and generalization—the model achieves robust, human-like decision-making in rare scenarios.", "questions": "see weakness", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. Integration of cognition and control: Combines semantic reasoning from a VLM with continuous control from a diffusion planner, forming a full cognitive-to-action pipeline.\n\n2. Three-stage training paradigm: Sequentially applies Q&A pretraining, imitation learning, and reinforcement learning to enhance understanding, controllability, and safety while reducing domain and generalization gaps.", "weaknesses": "1.The paper presents a \"cognitive framework\" that, upon inspection, is a modular integration of existing methods rather than a novel algorithmic solution. The \"hierarchical data pipeline\" (Sec 4.1) is a data curation strategy, not an algorithmic one. It offers no new mechanism for mitigating domain bias or improving multi-modal alignment. The use of a diffusion planner (Sec 4.2) to connect VLMs to continuous control is not new. This has been demonstrated by recent work (e.g., ORION [1]), making the contribution here incremental at best. The \"DiffGRPO\" (Sec 4.3) is not a new algorithm but a direct application of the existing GRPO framework to a diffusion planner. Framing this as a key innovation inflates the paper's novelty. Furthermore, this RL stage operates in a decoupled fashion, fine-tuning the action policy without any mechanism to enhance or feed back into the VLM's reasoning process.\n\n2.The central premise of using a VLM is for its reasoning, yet the paper's own results show the VLM is relegated to a simple feature extractor. The paper's own ablation study (Table 3) is damning: the \"Driving Pre-training\" (the cognitive part) yields a trivial 1.7 PDMS gain. In contrast, the non-cognitive RL fine-tuning provides the main performance boost (4.3 PDMS). Most critically, the appendix (Table 10) confirms that adding Chain-of-Thought (CoT) reasoning provides \"no gain\" and even hurts performance (a 0.1 PDMS drop). This finding fatally undermines the paper's core \"cognitive\" premise.\n\n3.The paper's strongest result (+4.3 PDMS) comes from an RL stage that is effectively \"training on the test.\" The model is fine-tuned within the NAVSIM simulator using the benchmark's own evaluation metric (PDMS) as the reward signal. This methodology is highly questionable. This approach raises significant concerns about the fairness of the comparison to other methods and suggests the results are likely overfit to the NAVSIM benchmark, severely limiting any claims of generalizability.\n\n4.The experimental comparison (Table 1) is weak. The paper fails to benchmark against key contemporary VLA-specific models (e.g., AutoVLA [2]), a notable omission in this fast-moving subfield. Instead, the comparisons are against general-purpose E2E models or VLM models. This makes it impossible to assess the paper's true contribution to VLA research.\n\n5.Finally, the experimental setup is ambiguous in several key areas, hindering reproducibility. The exact training data mix is unclear. The paper mentions multiple VQA datasets (Appx. D) plus 775K newly generated pairs, but the final composition and weighting for the reported models are not specified. The Bench2Drive protocol (Table 2) is unexplained. It's impossible to tell if these are zero-shot results or if the model was fine-tuned, making the scores difficult to interpret. The tables themselves are sloppy. Table 3 uses \"100\" and \"100.0\" inconsistently in the same column. Table 4 introduces an \"Err. (%)\" metric without a clear definition. The small differences are highlighted in red, creating unnecessary confusion.\n\n[1] ORION: A Holistic End-to-End Autonomous Driving Framework. ICCV 2025\n\n[2] AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning. NeurIPS 2025", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)