DriveRX: A Vision-Language Reasoning Model for Cross-Task Autonomous Driving

## Abstract

Abstract Autonomous driving requires real-time, robust reasoning across perception, prediction, planning, and behavior. However, conventional end-to-end models fail to generalize in complex scenarios due to the lack of structured reasoning. While recent vision-language models (VLMs) have been applied to driving tasks, they typically rely on isolated modules and static supervision, limiting their ability to support multi-stage decision-making.
We present AutoDriveRL , a unified training framework that formulates autonomous driving as a structured reasoning process over four core tasks. Each task is independently modeled as a vision-language QA problem and optimized using task-specific reward models, enabling fine-grained reinforcement signals at different reasoning stages.
Within this framework, we train DriveRX , a cross-task reasoning VLM designed for real-time decision-making. DriveRX achieves strong performance on the public benchmark, outperforming GPT-4o in behavior reasoning and demonstrating robustness under complex or corrupted driving conditions. Our analysis further highlights the impact of vision encoder design and reward-guided reasoning compression. We will release the AutoDriveRL framework and the DriveRX model to support future research.

## 1 Introduction

Autonomous driving systems operate in dynamic and uncertain traffic environments [1, 2, 3], requiring robust, real-time capabilities in perception, prediction, and decision-making [4, 5, 6]. However, in complex scenarios such as occlusion or abnormal behaviors, conventional end-to-end models exhibit limited generalization due to the lack of structured reasoning and semantic interpretability [7, 8, 9].

Recent progress in vision-language models (VLMs) has extended their applications to autonomous driving tasks, including object detection [10, 11, 12], trajectory prediction [13, 14, 15], and planning [16, 15]. Prior work [17, 15, 18] demonstrates VLMs’ potential through language prompts and QA mechanisms, yet primarily focuses on isolated task modules without a unified reasoning framework. This fragmentation hinders systematic understanding and cross-task generalization [19, 18].

However, applying VLMs to autonomous driving remains challenging due to:
(1) absence of intermediate reasoning chains [20, 16, 17];
(2) isolation across perception, prediction, planning, and behavior [21, 22, 23];
(3) reliance on static prompts and annotations without adaptive feedback [24, 25, 26].

Meanwhile, large language models (LLMs) have shown promising solutions to similar challenges [27, 28, 29, 30]. OpenAI’s o1 [30] leverages chain-of-thought (CoT) prompting to generate intermediate reasoning steps, while DeepSeek’s R1-Zero [29] uses reinforcement learning (RL) to improve model behavior through dynamic feedback—without relying on supervised fine-tuning. These approaches address key limitations in current VLMs by enabling structured reasoning, unifying task semantics, and supporting feedback-driven optimization, motivating their extension to autonomous driving.

Figure: Figure 1: Overview of the AutoDriveRL framework. To tackle the challenge of cross-task reasoning in autonomous driving, AutoDriveRL decomposes complex scenarios into four core tasks—Perception, Prediction, Planning and Behavior—each formulated in a VQA style. These tasks form a structured reasoning chain and are jointly optimized via RL. During training, we design task-specific reward models for each task to provide fine-grained feedback. The resulting model, DriveRX, demonstrates strong generalization and robustness under challenging driving conditions.
Refer to caption: x1.png

Several multimodal studies have explored integrating reasoning mechanisms to enhance model capabilities. For example, LlamaV-o1 [31] combines Monte Carlo Tree Search (MCTS) with CoT prompting to improve logical consistency in visual question answering (VQA); VLM-R1 [32] and MM-Eureka [33] apply RL and rule-based supervision to extend reasoning into math and science domains. While effective in their domains, these approaches are not designed for the real-time, decision-critical demands of autonomous driving. Long-form reasoning chains, common in CoT-based methods, are often too slow or verbose for driving scenarios that require concise, executable outputs. AlphaDrive [16] explores the application of RL in autonomous driving, but only within the planning task, and does not address multi-task coordination.

This motivates the development of a unified vision-language framework that supports structured reasoning, reinforcement learning, and multi-task coordination across core autonomous driving tasks.

To this end, we propose AutoDriveRL, a multi-task training framework for VLMs in autonomous driving. AutoDriveRL combines language-driven reasoning with task-specific RL strategies, aiming to systematically improve model performance and robustness across core autonomous driving tasks.

AutoDriveRL builds on prior work that decomposes autonomous driving into four core sub-tasks: perception, prediction, planning, and behavior. These tasks are organized into a sequential reasoning chain, but each task is independently modeled as a VQA problem without relying on outputs or contextual information from previous stages. Crucially, AutoDriveRL introduces task-specific reward models that provide fine-grained, stage-level reinforcement signals during training. From a global perspective, this structured supervision enables process-level feedback across reasoning steps, supports interpretable intermediate reasoning, facilitates cross-task optimization, and significantly enhances the model’s stability and overall performance in autonomous driving decision-making.

Within this framework, we train a unified cross-task vision-language model, DriveRX, where “RX” denotes its focus on reasoning and cross-task generalization in autonomous driving. DriveRX consistently outperforms existing models across perception, prediction, planning, and behavior tasks, in both in-distribution and out-of-distribution (OOD) benchmark settings. On the public benchmark DriveBench [34], DriveRX achieves a score of 62.02 in behavior, surpassing GPT-4o’s [27] 55.04 by 6.98 points. It maintains stable outputs under challenging conditions such as rain, occlusion, and lighting variation, demonstrating strong robustness and generalization. On the DriveBench-Corruption subset, DriveRX achieves 55.24, again outperforming GPT-4o (53.97).

Notably, driven by the reward design in AutoDriveRL, DriveRX produces more compact reasoning traces and faster responses than existing reasoning-based models—better aligning with the real-time requirements of autonomous driving.

Our main contributions are as follows:

- •
We propose AutoDriveRL, a unified training framework that leverages semantic reward modeling to enable structured reasoning across four core autonomous driving tasks: perception, prediction, planning, and behavior.
- •
We develop a cross-task vision-language model, DriveRX, which achieves strong performance across multiple driving tasks and complex traffic scenarios, demonstrating strong generalization and robustness, especially in behavior tasks.
- •
We conduct an in-depth analysis of the training dynamics, identifying cross-task interactions, vision encoder bottlenecks in perception, and the impact of reward-guided reasoning compression.

## 2 Related Work

### 2.1 Vision-Language Models for Autonomous Driving

Recent works have explored using vision-language models (VLMs) to enhance perception and reasoning in autonomous driving.
DriveLM [35] formulates multiple driving tasks as structured graph-based QA, constructing a multi-task dataset.
Dolphins [36] focuses on dialogue-based reasoning with historical control and counterfactuals.
DriveVLM [15] proposes a three-stage language-driven planning pipeline with templated prompts and task-specific instructions.
These methods largely rely on supervised fine-tuning (SFT) over static human-annotated data for task-specific outputs.
AlphaDrive [16] introduces reinforcement learning (RL) but is limited to the planning stage, lacking support for end-to-end reasoning and broader task-level optimization.
In contrast, we adopt an RL framework that decomposes driving into four semantically distinct subtasks, each equipped with a task-specific reward model to evaluate response quality and guide policy learning.
This enables the model to acquire task-aligned reasoning strategies at every stage without static supervision, improving robustness and consistency in complex real-world scenarios.

### 2.2 Vision-Language Reasoning Models

Recent years have seen notable progress in VLMs for visual reasoning. Representative works include R1-Onevision [37], which enhances visual-textual reasoning via structured intermediate representations and employs reinforcement learning to optimize reasoning trajectories; MM-Eureka [33], which introduces rule-based “aha” mechanisms for iterative refinement of visual answers; LlamaV-o1 [31], which adopts a step-by-step prompting strategy and achieves strong performance on complex benchmarks ; and VisuoThink [38], which leverages multimodal tree search to emulate human-like slow thinking, achieving strong performance in geometry and spatial reasoning tasks. These models are primarily developed for general-purpose vision-language reasoning tasks, typically under static image and single-turn QA settings. They seldom address complex task structures or action-oriented reasoning objectives, making them less applicable to domains like autonomous driving that require long-horizon, context-dependent reasoning across multiple subtasks.

## 3 AutoDriveRL

To enhance the reasoning capability and generalization of vision-language models in complex driving scenarios, we propose AutoDriveRL, a unified training framework. AutoDriveRL decomposes the autonomous driving task into four core sub-tasks—and adopts a consistent question-answering structure across all tasks. Each sub-task is equipped with a task-specific reward function, providing targeted feedback to guide the model toward learning stage-specific reasoning strategies. These components are jointly trained using reinforcement learning to develop a multi-task vision-language model, DriveRX. In this section, we detail the approach across four dimensions: task formulation, data construction, training framework, and reward design. An overview of the overall pipeline is illustrated in Figure [1](https://arxiv.org/html/2505.20665v1#S1.F1).

### 3.1 Task Formulation

Autonomous driving involves perceiving traffic elements, anticipating agent behaviors, and making timely planning and behavioral decisions in dynamic environments. Following prior work [35, 34], we decompose this process into four sub-tasks—perception, prediction, planning, and behavior—that span the full reasoning chain from low-level perception to high-level decision-making.

- •
Perception: Identify key elements in the scene, such as vehicles, pedestrians, and traffic signs.
- •
Prediction: Anticipate the behavior of dynamic agents.
- •
Planning: Determine near-term path or directional actions.
- •
Behavior: Conduct behavior prediction and strategy selection.

Each task is formulated as VQA problems, where the input consists of images and a task-specific natural language query, and the output is a grounded, interpretable response. The tasks share a unified input-output structure to facilitate consistent language-based reasoning, and are shuffled and jointly trained using reinforcement learning to encourage generalizable reasoning across task boundaries.

Although trained jointly with shared parameters, the tasks form an implicit reasoning chain in semantic space. We do not explicitly condition later tasks on earlier outputs; instead, all tasks attend to a shared image and question, allowing semantic coupling to emerge through language.

### 3.2 Data Construction

To ensure stable RL training and challenging evaluation, we construct two subsets from the DriveLM dataset: AutoDriveRL-Train for multi-task training and DriveLM-Hard for robustness evaluation.

We employ a unified scoring and filtering pipeline to control task difficulty, increase semantic diversity, and balance action distributions. Low-quality samples (e.g., with annotation errors or duplicate options) are first removed. For each visual question, we collect response scores $s_{ijkm}$ by sampling $K$ outputs from each of $M$ vision-language models. These scores are computed using task-specific reward models (see prompt in Section [I.1](https://arxiv.org/html/2505.20665v1#A9.SS1)), guided by structured prompts to ensure consistency and reproducibility without human labeling. Alignment with human judgments is shown in Appendix [F](https://arxiv.org/html/2505.20665v1#A6).

Each image frame typically contains multiple questions. We define a frame-level score $D_{i}$ as:

$$ $D_{i}=\frac{1}{T_{i}}\sum_{j=1}^{T_{i}}\left(\frac{1}{M}\sum_{k=1}^{M}\left( \frac{1}{K}\sum_{m=1}^{K}s_{ijkm}\right)\right)$ (1) $$

where $T_{i}$ is the number of questions in frame $i$, $M$ is the number of base models used for scoring, and $K$ is the number of sampled responses per model per question. Each score $s_{ijkm}\in[0,100]$ reflects the task-specific quality of a sampled response.

Based on $D_{i}$, we follow prior work [29, 39] on RL data filtering strategies and exclude both overly easy and overly hard samples from training. Specifically, we select mid-range frames to improve training stability by avoiding trivial examples and high-variance outliers. Low-scoring frames—typically involving occlusion, poor lighting, or rare behaviors—are used exclusively for evaluation to assess generalization under challenging conditions.

To address action imbalance—where common actions (e.g., going straight) dominate—we apply class balancing by oversampling rare action samples with explicit intent, improving coverage of low-frequency behaviors. Final distributions are shown in Fig. [6](https://arxiv.org/html/2505.20665v1#A3.F6).

This process yields a high-quality, difficulty-controlled, and semantically diverse dataset that supports robust multi-task RL and evaluation.

### 3.3 Training Framework

Autonomous driving comprises four semantically coupled sub-tasks—perception, prediction, planning, and behavior decision-making—each requiring intermediate reasoning that is hard to supervise directly. Instead of conventional supervised learning, we adopt a reinforcement-learning paradigm in which a single policy is trained across all tasks and updated from reward feedback.

Many tasks, especially planning and behavior prediction, admit multiple plausible outputs (one-to-many mapping). To cope with the resulting high-variance rewards, we optimize the policy with Group Relative Policy Optimization (GRPO) [40]. For every query $q$, the current policy $\pi_{\theta}$ samples $G$ candidate responses $\{o_{1},\dots,o_{G}\}$. The GRPO objective is

$$ $\mathcal{J}_{\text{GRPO}}(\theta)=\mathbb{E}_{q\sim\mathcal{D}}\!\left[\frac{1 }{G}\sum_{i=1}^{G}\frac{1}{|o_{i}|}\sum_{t=1}^{|o_{i}|}\min\bigl{(}r_{i,t}\, \hat{A}_{i,t},\,\operatorname{clip}(r_{i,t},1-\epsilon,1+\epsilon)\,\hat{A}_{i ,t}\bigr{)}\right]-\beta\,\mathrm{KL}\!\bigl{(}\pi_{\theta}\;\|\;\pi_{\text{ ref}}\bigr{)},$ (2) $$

where $r_{i,t}=\pi_{\theta}(o_{i,t}\mid q,o_{i,<t})\big{/}\pi_{\theta_{\text{old}}}(o
_{i,t}\mid q,o_{i,<t})$ is the token-level policy ratio, $t$ denotes the token position and $\hat{A}_{i,t}$ is the intra-group advantage defined in Section [3.4](https://arxiv.org/html/2505.20665v1#S3.SS4).

During training, samples from all four tasks are mixed, and the shared parameters are updated jointly. The framework relies solely on rule-based and model-based reward signals, requiring no extra human annotation and thus scaling easily to new tasks and datasets.

### 3.4 Reward Model Design

Autonomous driving demands low latency, clear expression, and accurate actions. To encourage concise and reliable outputs while avoiding verbosity or repetition, we design a two-stage reward mechanism that automatically evaluates response quality and guides GRPO training.

(i) Rule-based Reward Function. Outputs exceeding a predefined length or exhibiting repetitive sentence patterns are penalized with low rewards.

(ii) LLM-based Reward Model. For remaining candidates, we employ an open-source language model to assign a task-aware quality score $R_{i}\in[0,100]$ based on five criteria: correctness, behavioral understanding, contextual relevance, logic and reasoning and clarity of reasoning. We use distinct prompts tailored to each of the four tasks.

The normalized group-level advantage is computed as:

$$ $\hat{A}i=\frac{R_{i}-\mu_{R}}{\sigma_{R}},\quad\hat{A}_{i,t}=\hat{A}_{i},; \forall t,$ (3) $$

where $\mu_{R}$ and $\sigma_{R}$ are computed over the $G$ responses sampled for the same input. This scalar advantage is shared across tokens and used in the GRPO update in Eq. ([2](https://arxiv.org/html/2505.20665v1#S3.E2)).

## 4 Experiment

This section introduces the training setup of DriveRX with AutoDriveRL and evaluates the resulting model on public out-of-distribution (OOD) benchmarks.

### 4.1 Experiment Settings

Base Model: We adopt Align-DS-V[41] as our base model, which extends DeepSeek-R1-Distill-LLaMA-8B[29] with a CLIP-ViT-L/14-336 visual encoder [42], trained using the Align-Anything framework [43], and exhibits strong multi-modal reasoning performance.

Evaluated Models: We evaluate a diverse set of models, including the commercial GPT-4o (2024-08-06) [27], the general open-source model LLaVA-1.5 7B [44], LLaVA-NeXT 7B [45], InternVL3 8B/78B [46], Qwen2.5-VL 7B/72B [47], the reasoning MM-Eureka 7B [33], R1-Onevision 7B [37], and the domain-specific autonomous driving models DriveLM [35] and Dolphins [36].

Training Dataset: We use the DriveLM [35] dataset, which is constructed from nuScenes [48] and OpenLane-V2 [49] through a structured annotation process. Following the AutoDriveRL data selection strategy, we compute frame-level scores $D_{i}$ using Align-DS-V and retain samples within the range $[25,45]$. To ensure a non-overlapping evaluation, all samples appearing in DriveBench [35] are explicitly excluded. From the remaining subset, we randomly sample 6K instances for training. See Appendix [C](https://arxiv.org/html/2505.20665v1#A3) for the dataset composition.

Reward: We first apply a rule-based filter that assigns a reward of 0 to responses exceeding 4K tokens or exhibiting substantial repetition. The remaining responses are then evaluated using task-specific prompts by Qwen2.5-72B-Instruct [28] (see Appendix [I.1](https://arxiv.org/html/2505.20665v1#A9.SS1)). The consistency between human judgments and model scores is analyzed in Appendix [F](https://arxiv.org/html/2505.20665v1#A6).

Evaluation: We evaluate on DriveBench [35], which spans four driving tasks and consists of a clean set with re-sampled questions and a corruption set featuring 15 types of visual degradation (see Appendix [J](https://arxiv.org/html/2505.20665v1#A10)) for robustness testing. We additionally evaluate on DriveLM-Hard, a challenging subset selected using the AutoDriveRL data selection strategy, where frame-level scores $D_{i}$ (defined in Eq. [1](https://arxiv.org/html/2505.20665v1#S3.E1)) fall within the range $[10,31]$. The $D_{i}$ scores are computed by averaging model scores from Align-DS-V, Qwen2-VL-7B, and Qwen2-VL-72B.

Metrics: We follow DriveBench evaluation prompts and use GPT-3.5-Turbo as the judge model, with GPT score as the evaluation metric. Following prior work [35, 50, 51, 52, 53], we discard BLEU [54] and ROUGE [55], as they were designed for translation and are not suitable for evaluating decision quality in autonomous driving.

For more experimental setup details, please refer to Appendix [K](https://arxiv.org/html/2505.20665v1#A11).

### 4.2 Main Result

DriveRX shows strong overall performance and achieves the best result in behavior tasks: As shown in Figure [1](https://arxiv.org/html/2505.20665v1#S4.T1), DriveRX demonstrates competitive performance across all four tasks. On the clean test sets of Perception, Prediction, and Planning, it performs closely to GPT-4o and the larger 70B-scale models, despite its smaller size. Notably, it achieves the highest score ($\mathbf{62.02}$) on the critical Behavior task, surpassing all baselines including the closed-source GPT-4o ($55.04$), the open-source Qwen2.5-VL 72B ($55.57$), the reasoning model MM-Eureka ($50.50$), and the domain-specific DriveLM ($42.78$). These results highlight the effectiveness of the AutoDriveRL framework, as improvements in earlier tasks lead to significant gains in the final behavior task.

**Table 1: Evaluations of VLMs across different driving tasks on DriveBench (perception, prediction, planning, and behavior). “Clean” represents clean image inputs. “Corr.” represents corruption image inputs, averaged across fifteen corruptions. The evaluations are based on GPT scores ($\uparrow$), where we tailored detailed rubrics for each task and question type. We highlight the best-performing open-source model for each task, and use bold to mark the second-best performance.**
| Model | Size | Type | Perception | Prediction | Planning | Behavior |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Clean | Corr. | Clean | Corr. | Clean | Corr. | Clean | Corr. |  |  |  |
| GPT-4o [27] | - | Commercial | $41.51$ | $45.39$ | $52.28$ | $50.92$ | $84.82$ | $84.43$ | $55.04$ | $53.97$ |
| LLaVA-1.5 [44] | $7$ B | Open | $23.22$ | $22.95$ | $22.02$ | $17.54$ | $29.15$ | $31.51$ | $13.60$ | $13.62$ |
| LLaVA-NeXT [45] | $7$ B | Open | $24.15$ | $19.62$ | $35.07$ | $35.89$ | $45.27$ | $44.36$ | $48.16$ | $39.44$ |
| Qwen2.5-VL [28] | $7$ B | Open | $31.92$ | $27.16$ | $42.26$ | $47.13$ | $55.70$ | $57.97$ | $47.40$ | $42.82$ |
| InternVL3 [46] | $8$ B | Open | $33.21$ | $\mathbf{33.87}$ | $41.21$ | $38.93$ | $70.97$ | $56.71$ | $51.47$ | $50.32$ |
| Qwen2.5-VL [28] | $72$ B | Open | $\mathbf{37.56}$ | $32.25$ | 54.89 | 50.47 | $77.18$ | $74.01$ | $\mathbf{55.57}$ | $51.65$ |
| InternVL3 [46] | $78$ B | Open | 39.44 | 35.62 | $50.93$ | $\mathbf{49.71}$ | 82.24 | 78.41 | $50.73$ | $\mathbf{52.03}$ |
| MM-Eureka [33] | $7$ B | Reasoning | $34.44$ | $28.92$ | $40.31$ | $41.72$ | $65.38$ | $60.18$ | $49.4$ | $47.59$ |
| R1-Onevision [37] | $7$ B | Reasoning | $28.64$ | $25.40$ | $51.22$ | $46.91$ | $55.12$ | $55.54$ | $41.57$ | $39.88$ |
| DriveLM [35] | $7$ B | Specialist | $16.85$ | $16.00$ | $44.33$ | $39.71$ | $68.71$ | $67.60$ | $42.78$ | $40.37$ |
| Dolphins [36] | $7$ B | Specialist | $9.59$ | $10.84$ | $32.66$ | $29.88$ | $52.91$ | $53.77$ | $8.81$ | $8.25$ |
| Align-DS-V [41] | $8$ B | Reasoning | $31.41$ | $27.78$ | $41.51$ | $37.86$ | $51.16$ | $57.66$ | $50.53$ | $40.98$ |
| DriveRX | $8$ B | Reasoning | $32.88$ | $29.92$ | $\mathbf{52.20}$ | $48.29$ | $\mathbf{78.63}$ | $\mathbf{75.53}$ | 62.02 | 55.24 |

DriveRX maintains robust performance under visual corruptions: In Figure [1](https://arxiv.org/html/2505.20665v1#S4.T1), DriveRX exhibits consistently strong performance across all tasks under the corruption setting, demonstrating resilience to various types of visual degradation. Notably, it achieves the highest score of $\mathbf{55.24}$ on the Behavior task, outperforming all other models. This result highlights the robustness of DriveRX in handling distributional shifts, and further demonstrates the benefit of RL: the model receives step-wise feedback during training, enabling reliable behavior decision-making even under degraded visual conditions.

DriveRX achieves stable performance in complex traffic conditions:
As shown in Figure [2](https://arxiv.org/html/2505.20665v1#S4.T2), DriveRX performs strongly on DriveLM-Hard, a challenging test split we construct by selecting high-difficulty frames from the original DriveLM dataset. This benchmark features more congested traffic, diverse agents, and denser interactions across the same four tasks (see Appendix [H](https://arxiv.org/html/2505.20665v1#A8)). These results demonstrate that DriveRX retains reliable performance in complex and high-uncertainty scenarios, confirming its robustness and generalization. Such capabilities are critical for real-time autonomous driving applications that demand consistent reasoning under diverse and dynamic conditions.

**Table 2: Evaluations of VLMs across different driving tasks on DriveLM-Hard. Each column shows the GPT score ($\uparrow$) for the clean version of each task. We highlight the best-performing open-source model for each task, and use bold to mark the second-best performance.**
| Method | Size | Type | Perception | Prediction | Planning | Behavior |
| --- | --- | --- | --- | --- | --- | --- |
| GPT-4o [27] | - | Commercial | $36.12$ | $65.90$ | $51.42$ | $42.00$ |
| Qwen2.5-VL [47] | $7$B | Open | $26.10$ | $60.13$ | $32.41$ | $19.86$ |
| InternVL3 [46] | $8$B | Open | $31.29$ | $38.95$ | $31.78$ | $30.48$ |
| Qwen2.5-VL [47] | $72$B | Open | $32.49$ | $58.63$ | $\mathbf{50.55}$ | $40.02$ |
| InternVL3 [46] | $78$B | Open | $32.98$ | $58.67$ | $45.82$ | $35.69$ |
| MM-Eureka [33] | $7$B | Reasoning | $29.09$ | $\mathbf{67.41}$ | $29.67$ | $23.21$ |
| R1-Onevision [37] | $7$B | Reasoning | $25.51$ | $46.60$ | $34.45$ | $29.03$ |
| DriveLM [35] | $7$B | Specialist | $28.78$ | $19.93$ | $22.60$ | $30.75$ |
| Align-DS-V [41] | $8$B | Reasoning | $29.45$ | $52.90$ | $29.15$ | $32.48$ |
| DriveRX | $8$B | Reasoning | $34.83$ | $71.84$ | $51.42$ | $\mathbf{36.82}$ |

## 5 Analysis

### 5.1 Training Analysis

We analyze the training dynamics of DriveRX under AutoDriveRL to understand cross-task optimization.
In Figure [2](https://arxiv.org/html/2505.20665v1#S5.F2)(a), the overall training remains stable, with steadily increasing rewards. This suggests that our task-specific reward models are effective in guiding learning across heterogeneous tasks.
In Figure [2](https://arxiv.org/html/2505.20665v1#S5.F2)(b), the reward curves exhibit a clear staged trend: Perception and Prediction improve rapidly in early training and quickly saturate, while Planning and Behavior progress more slowly and gradually. This pattern reflects the structural hierarchy of the tasks—lower-level tasks are relatively simple, focusing on visual understanding and pattern recognition, whereas higher-level tasks require multi-step reasoning and rely on semantic representations generated by preceding tasks.

We also track the average response length during training (Figure [2](https://arxiv.org/html/2505.20665v1#S5.F2)(c)). In the early stages, response length increases, indicating that the model explores more expressive outputs. As training progresses and reward signals stabilize, responses gradually become shorter and more concise. Figure [2](https://arxiv.org/html/2505.20665v1#S5.F2)(d) shows the ratio of clipped responses exceeding the maximum length threshold, which closely follows the trend of response length—higher lengths lead to more clipping, while this ratio decreases as the model converges. These results confirm the effectiveness of our length-penalizing reward design, which encourages the model to reduce redundancy while maintaining output quality.

Figure: (a) Mean Training Reward
Refer to caption: extracted/6482279/img/critic_log.png

### 5.2 Impact of Vision Encoders on Perception Tasks

We conduct a statistical analysis on the impact of different vision encoders on the Perception task. In addition to performance scores, we also compare each model’s average response length and the number of image tokens generated by their visual encoders, as summarized in Table [3](https://arxiv.org/html/2505.20665v1#S5.T3).

We observe two main findings. First, the capacity and design of the vision encoder significantly affect perception performance. Larger and more advanced encoders consistently achieve higher scores. For example, InternVL3 78B (with InternViT-6B-448px-V2.5) achieves a DriveBench perception score of 39.44, substantially outperforming its smaller variant InternVL3 8B (with InternViT-300M-448px-V2.5), which only achieves 33.21. Furthermore, early ViT models like CLIP-ViT-L/14 underperform in perception tasks relative to newer large-scale ViT models such as InternViT-300M-448px-V2.5.

Second, the visual encoder plays a critical role in determining the upper bound of perception task performance during training. In our experiments, the improvement of DriveRX over its base model Align-DS-V is limited ($+1.47$), despite a substantial gain on the behavior task ($+11.49$). As shown in the training curves (Figure [2](https://arxiv.org/html/2505.20665v1#S5.F2)(b)), the perception reward for DriveRX saturates early (within 50 steps) and remains below 50, indicating a clear performance ceiling. We attribute this to the outdated architecture of CLIP-ViT-L/14, an early-generation ViT model whose limited perceptual capability falls behind modern ViT-based encoders.

However, high-performing models like Qwen2.5-VL [28] and InternVL3 [46] typically adopt dynamic resolution mechanisms, which allow them to process high-resolution inputs more effectively. While this improves visual understanding and perception scores, it also leads to significantly longer image token sequences. For instance, Qwen2.5-VL generates substantially more vision tokens per image compared to fixed-resolution VLMs like Align-DS-V. This increase in token length introduces notable inference latency, posing challenges for real-time decision-making in autonomous driving. In contrast to these models, DriveRX incorporates a fixed-length feature processing pipeline in its architectural design. While this design sacrifices some perceptual performance due to the output dimensionality constraints of the CLIP encoder, it achieves three critical advantages for practical deployment: deterministic inference efficiency, faster processing speed, and more predictable inference behavior.

**Table 3: Impact of different vision encoders on the perception task. We report DriveBench perception scores (higher is better), average response length (in tokens) per question, number of image tokens per image, and inference time per question (in seconds). For a fair comparison, inference time is reported only among models of the same size.**
| Model | Size | Vision Encoder | Response Length | Image Tokens | Perception ($\uparrow$) | Infer times |
| --- | --- | --- | --- | --- | --- | --- |
| MM-Eureka | $7\text{B}$ | Qwen-ViT-L/14 (Qwen2.5-VL) | $543.12$ | $1824$ | $34.44$ | $2.52$ |
| R1-Onevision | $7\text{B}$ | Qwen-ViT-L/14 (Qwen2.5-VL) | $447.55$ | $1824$ | $28.64$ | $2.59$ |
| Qwen2.5-VL | $7\text{B}$ | Qwen-ViT-L/14 (Qwen2.5-VL) | $357.85$ | $1824$ | $31.92$ | $2.76$ |
| Qwen2.5-VL | $72\text{B}$ | Qwen-ViT-L/14 (Qwen2.5-VL) | $343.27$ | $1824$ | $37.56$ | $-$ |
| InternVL3 | $8\text{B}$ | InternViT-300M-448px-V2.5 | $270.94$ | $2304$ | $33.21$ | $1.58$ |
| InternVL3 | $78\text{B}$ | InternViT-6B-448px-V2.5 | $275.69$ | $2304$ | $39.44$ | $-$ |
| DriveLM | $7\text{B}$ | CLIP ViT-L/14 | $113.28$ | $576$ | $16.85$ | $0.59$ |
| Align-DS-V | $8\text{B}$ | CLIP ViT-L/14 | $252.10$ | $576$ | $31.41$ | $0.68$ |
| DriveRX | $8\text{B}$ | CLIP ViT-L/14 | $363.44$ | $576$ | $32.88$ | $1.06$ |

### 5.3 Case Study

To better illustrate the advantages of our framework in terms of reasoning quality and interpretability, we present a representative example from the DriveLM-Hard set, featuring a challenging intersection scenario under rainy weather and blurred camera conditions. The task is to determine the correct behavior of the ego vehicle based solely on the visual input. We compare the outputs of the base model Align-DS-V and our RL-optimized model DriveRX.

As shown in Figure [3](https://arxiv.org/html/2505.20665v1#S5.F3), DriveRX decomposes the behavior reasoning process into four semantically distinct subtasks: Perception,
Prediction,
Planning,
and final Behavior, and answers them sequentially. This structured reasoning not only improves interpretability of the final decision, but also enables in-depth analysis of the intermediate steps.

Figure: Figure 3: Behavior task example comparing Align-DSV and DriveRX. The goal is to determine the ego vehicle’s behavior at an intersection. Align-DSV incorrectly outputs “going straight”, while DriveRX correctly concludes “steering to the right” by performing structured reasoning. The behavior task is decomposed into four subtasks: Perception, Prediction, Planning, and final Behavior decision, allowing DriveRX to make more accurate and interpretable decisions.
Refer to caption: extracted/6482279/img/prompt/case_cot.png

Perception: Align-DS-V focuses on describing the overall scene—e.g., “a rainy street scene with a vehicle on the left side, partially obscured by fog or mist”—which aligns more with a captioning-style response. In contrast, DriveRX pays closer attention to driving-relevant visual cues, such as the wet road surface and the current pose of the vehicle.

Prediction: Align-DS-V erroneously concludes that the vehicle is “aligned with the lane, indicating it may be driving straight”, overlooking the subtle orientation bias toward the right. This leads to a misprediction of the vehicle’s intent. DriveRX correctly identifies the vehicle as angled across the lane, indicating a right turn, and additionally predicts its motion status (e.g., speed and steering).

Planning: Align-DS-V adopts an elimination-based reasoning strategy—e.g., “lack of visible turning signals, the vehicle is not steering.” This VQA-style approach relies solely on explicit visual cues and therefore struggles to capture the latent intent embedded in dynamic context. In contrast, DriveRX integrates implicit cues from the surrounding context with prior outputs from the perception and prediction stages, enabling more robust planning decisions—particularly in cases where signals are incomplete or the environment is visually degraded.

Behavior: As a result of the flawed intermediate reasoning, Align-DS-V makes an incorrect decision (“going straight”), whereas DriveRX produces the correct and interpretable decision to steer right, grounded in structured multi-stage analysis.

## 6 Conclusion

We propose AutoDriveRL, a reinforcement learning framework that decomposes autonomous driving into four reasoning subtasks, each guided by a task-specific reward model. This enables structured, interpretable reasoning and supports cross-task optimization.

The resulting model, DriveRX, achieves strong performance across perception, prediction, planning, and behavior tasks, outperforming GPT-4o on DriveBench behavior tasks and maintaining robustness under occlusion and other challenging conditions. Thanks to reward-guided reasoning, DriveRX produces shorter outputs and more efficient inference, better suited for real-time decision-making.

## References

- Galceran et al. [2015]
Enric Galceran, Alexander G Cunningham, Ryan M Eustice, and Edwin Olson.
Multipolicy decision-making for autonomous driving via changepoint-based behavior prediction.
In *Robotics: Science and Systems*, volume 1, page 6, 2015.
- Xu et al. [2014]
Wenda Xu, Jia Pan, Junqing Wei, and John M. Dolan.
Motion planning under uncertainty for on-road autonomous driving.
In *2014 IEEE International Conference on Robotics and Automation (ICRA)*, pages 2507–2512, 2014.
doi: 10.1109/ICRA.2014.6907209.
- Marcu et al. [2024]
Ana-Maria Marcu, Long Chen, Jan Hünermann, Alice Karnsund, Benoit Hanotte, Prajwal Chidananda, Saurabh Nair, Vijay Badrinarayanan, Alex Kendall, Jamie Shotton, et al.
Lingoqa: Visual question answering for autonomous driving.
In *European Conference on Computer Vision*, pages 252–269. Springer, 2024.
- Grigorescu et al. [2020]
Sorin Grigorescu, Bogdan Trasnea, Tiberiu Cocias, and Gigel Macesanu.
A survey of deep learning techniques for autonomous driving.
*Journal of field robotics*, 37(3):362–386, 2020.
- Chen et al. [2015]
Chenyi Chen, Ari Seff, Alain Kornhauser, and Jianxiong Xiao.
Deepdriving: Learning affordance for direct perception in autonomous driving.
In *Proceedings of the IEEE international conference on computer vision*, pages 2722–2730, 2015.
- Janai et al. [2020]
Joel Janai, Fatma Güney, Aseem Behl, Andreas Geiger, et al.
Computer vision for autonomous vehicles: Problems, datasets and state of the art.
*Foundations and trends® in computer graphics and vision*, 12(1–3):1–308, 2020.
- Chen et al. [2024a]
Li Chen, Penghao Wu, Kashyap Chitta, Bernhard Jaeger, Andreas Geiger, and Hongyang Li.
End-to-end autonomous driving: Challenges and frontiers.
*IEEE Transactions on Pattern Analysis and Machine Intelligence*, 2024a.
- Xu et al. [2024a]
Zhenhua Xu, Yujia Zhang, Enze Xie, Zhen Zhao, Yong Guo, Kwan-Yee K Wong, Zhenguo Li, and Hengshuang Zhao.
Drivegpt4: Interpretable end-to-end autonomous driving via large language model.
*IEEE Robotics and Automation Letters*, 2024a.
- Shao et al. [2023]
Hao Shao, Letian Wang, Ruobing Chen, Steven L Waslander, Hongsheng Li, and Yu Liu.
Reasonnet: End-to-end driving with temporal and global reasoning.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 13723–13733, 2023.
- Liu et al. [2023]
Mengyin Liu, Jie Jiang, Chao Zhu, and Xu-Cheng Yin.
Vlpd: Context-aware pedestrian detection via vision-language semantic self-supervision.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 6662–6671, 2023.
- Choudhary et al. [2024]
Tushar Choudhary, Vikrant Dewangan, Shivam Chandhok, Shubham Priyadarshan, Anushka Jain, Arun K Singh, Siddharth Srivastava, Krishna Murthy Jatavallabhula, and K Madhava Krishna.
Talk2bev: Language-enhanced bird’s-eye view maps for autonomous driving.
In *2024 IEEE International Conference on Robotics and Automation (ICRA)*, pages 16345–16352. IEEE, 2024.
- Qian et al. [2024]
Tianwen Qian, Jingjing Chen, Linhai Zhuo, Yang Jiao, and Yu-Gang Jiang.
Nuscenes-qa: A multi-modal visual question answering benchmark for autonomous driving scenario.
In *Proceedings of the AAAI Conference on Artificial Intelligence*, volume 38, pages 4542–4550, 2024.
- Mao et al. [2023]
Jiageng Mao, Yuxi Qian, Junjie Ye, Hang Zhao, and Yue Wang.
Gpt-driver: Learning to drive with gpt.
*arXiv preprint arXiv:2310.01415*, 2023.
- Nguyen et al. [2024]
Phat Nguyen, Tsun-Hsuan Wang, Zhang-Wei Hong, Sertac Karaman, and Daniela Rus.
Text-to-drive: Diverse driving behavior synthesis via large language models.
In *2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)*, pages 10495–10502. IEEE, 2024.
- Tian et al. [2024]
Xiaoyu Tian, Junru Gu, Bailin Li, Yicheng Liu, Yang Wang, Zhiyong Zhao, Kun Zhan, Peng Jia, Xianpeng Lang, and Hang Zhao.
Drivevlm: The convergence of autonomous driving and large vision-language models.
*arXiv preprint arXiv:2402.12289*, 2024.
- Jiang et al. [2025]
Bo Jiang, Shaoyu Chen, Qian Zhang, Wenyu Liu, and Xinggang Wang.
Alphadrive: Unleashing the power of vlms in autonomous driving via reinforcement learning and reasoning.
*arXiv preprint arXiv:2503.07608*, 2025.
- Xu et al. [2024b]
Yi Xu, Yuxin Hu, Zaiwei Zhang, Gregory P Meyer, Siva Karthik Mustikovela, Siddhartha Srinivasa, Eric M Wolff, and Xin Huang.
Vlm-ad: End-to-end autonomous driving through vision-language model supervision.
*arXiv preprint arXiv:2412.14446*, 2024b.
- Wang et al. [2024]
Shihao Wang, Zhiding Yu, Xiaohui Jiang, Shiyi Lan, Min Shi, Nadine Chang, Jan Kautz, Ying Li, and Jose M Alvarez.
Omnidrive: A holistic llm-agent framework for autonomous driving with 3d perception, reasoning and planning.
*arXiv preprint arXiv:2405.01533*, 2024.
- Zhao et al. [2025]
Rui Zhao, Qirui Yuan, Jinyu Li, Haofeng Hu, Yun Li, Chengyuan Zheng, and Fei Gao.
Sce2drivex: A generalized mllm framework for scene-to-drive learning.
*arXiv preprint arXiv:2502.14917*, 2025.
- Nie et al. [2024]
Ming Nie, Renyuan Peng, Chunwei Wang, Xinyue Cai, Jianhua Han, Hang Xu, and Li Zhang.
Reason2drive: Towards interpretable and chain-based reasoning for autonomous driving.
In *European Conference on Computer Vision*, pages 292–308. Springer, 2024.
- Hu et al. [2023]
Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, et al.
Planning-oriented autonomous driving.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 17853–17862, 2023.
- Bansal et al. [2018]
Mayank Bansal, Alex Krizhevsky, and Abhijit Ogale.
Chauffeurnet: Learning to drive by imitating the best and synthesizing the worst.
*arXiv preprint arXiv:1812.03079*, 2018.
- Casas et al. [2021]
Sergio Casas, Abbas Sadat, and Raquel Urtasun.
Mp3: A unified model to map, perceive, predict and plan.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 14403–14412, 2021.
- Zhang et al. [2024]
Jimuyang Zhang, Zanming Huang, Arijit Ray, and Eshed Ohn-Bar.
Feedback-guided autonomous driving.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 15000–15011, 2024.
- Cui et al. [2024]
Can Cui, Yunsheng Ma, Xu Cao, Wenqian Ye, and Ziran Wang.
Receive, reason, and react: Drive as you say, with large language models in autonomous vehicles.
*IEEE Intelligent Transportation Systems Magazine*, 2024.
- Duan et al. [2024]
Yiqun Duan, Qiang Zhang, and Renjing Xu.
Prompting multi-modal tokens to enhance end-to-end autonomous driving imitation learning with llms.
In *2024 IEEE International Conference on Robotics and Automation (ICRA)*, pages 6798–6805. IEEE, 2024.
- OpenAI [2024a]
OpenAI.
Hello gpt-4o, 2024a.
URL [https://openai.com/index/hello-gpt-4o/](https://openai.com/index/hello-gpt-4o/).
- Team [2024]
Qwen Team.
Qwen2.5: A party of foundation models, September 2024.
URL [https://qwenlm.github.io/blog/qwen2.5/](https://qwenlm.github.io/blog/qwen2.5/).
- Guo et al. [2025]
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al.
Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.
*arXiv preprint arXiv:2501.12948*, 2025.
- OpenAI [2024b]
OpenAI.
Openai o1 hub | openai, 2024b.
URL [https://openai.com/o1/](https://openai.com/o1/).
- Thawakar et al. [2025]
Omkar Thawakar, Dinura Dissanayake, Ketan More, Ritesh Thawkar, Ahmed Heakl, Noor Ahsan, Yuhao Li, Mohammed Zumri, Jean Lahoud, Rao Muhammad Anwer, et al.
Llamav-o1: Rethinking step-by-step visual reasoning in llms.
*arXiv preprint arXiv:2501.06186*, 2025.
- Shen et al. [2025]
Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, et al.
Vlm-r1: A stable and generalizable r1-style large vision-language model.
*arXiv preprint arXiv:2504.07615*, 2025.
- Meng et al. [2025]
Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, et al.
Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning.
*arXiv preprint arXiv:2503.07365*, 2025.
- Xie et al. [2025]
Shaoyuan Xie, Lingdong Kong, Yuhao Dong, Chonghao Sima, Wenwei Zhang, Qi Alfred Chen, Ziwei Liu, and Liang Pan.
Are vlms ready for autonomous driving? an empirical study from the reliability, data, and metric perspectives.
*arXiv preprint arXiv:2501.04003*, 2025.
- Sima et al. [2024]
Chonghao Sima, Katrin Renz, Kashyap Chitta, Li Chen, Hanxue Zhang, Chengen Xie, Jens Beißwenger, Ping Luo, Andreas Geiger, and Hongyang Li.
Drivelm: Driving with graph visual question answering.
In *European Conference on Computer Vision*, pages 256–274. Springer, 2024.
- Ma et al. [2024]
Yingzi Ma, Yulong Cao, Jiachen Sun, Marco Pavone, and Chaowei Xiao.
Dolphins: Multimodal language model for driving.
In *European Conference on Computer Vision*, pages 403–420. Springer, 2024.
- Yang et al. [2025]
Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, et al.
R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization.
*arXiv preprint arXiv:2503.10615*, 2025.
- Wang et al. [2025]
Yikun Wang, Siyin Wang, Qinyuan Cheng, Zhaoye Fei, Liang Ding, Qipeng Guo, Dacheng Tao, and Xipeng Qiu.
Visuothink: Empowering lvlm reasoning with multimodal tree search.
*arXiv preprint arXiv:2504.09130*, 2025.
- Zeng et al. [2025]
Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He.
Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild.
*arXiv preprint arXiv:2503.18892*, 2025.
- Shao et al. [2024]
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al.
Deepseekmath: Pushing the limits of mathematical reasoning in open language models.
*arXiv preprint arXiv:2402.03300*, 2024.
- PKU-Alignment [2025]
PKU-Alignment.
Pku-alignment/align-ds-v · hugging face, 4 2025.
URL [https://huggingface.co/PKU-Alignment/Align-DS-V](https://huggingface.co/PKU-Alignment/Align-DS-V).
[Online; accessed 2025-05-12].
- Radford et al. [2021]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.
Learning transferable visual models from natural language supervision.
In *International conference on machine learning*, pages 8748–8763. PmLR, 2021.
- Ji et al. [2024]
Jiaming Ji, Jiayi Zhou, Hantao Lou, Boyuan Chen, Donghai Hong, Xuyao Wang, Wenqi Chen, Kaile Wang, Rui Pan, Jiahao Li, Mohan Wang, Josef Dai, Tianyi Qiu, Hua Xu, Dong Li, Weipeng Chen, Jun Song, Bo Zheng, and Yaodong Yang.
Align anything: Training all-modality models to follow instructions with language feedback.
2024.
URL [https://arxiv.org/abs/2412.15838](https://arxiv.org/abs/2412.15838).
- Liu et al. [2024a]
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee.
Improved baselines with visual instruction tuning.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 26296–26306, 2024a.
- Liu et al. [2024b]
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee.
Llava-next: Improved reasoning, ocr, and world knowledge, January 2024b.
URL [https://llava-vl.github.io/blog/2024-01-30-llava-next/](https://llava-vl.github.io/blog/2024-01-30-llava-next/).
- Zhu et al. [2025]
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Yuchen Duan, Hao Tian, Weijie Su, Jie Shao, et al.
Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models.
*arXiv preprint arXiv:2504.10479*, 2025.
- Bai et al. [2025]
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al.
Qwen2. 5-vl technical report.
*arXiv preprint arXiv:2502.13923*, 2025.
- Caesar et al. [2020]
Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom.
nuscenes: A multimodal dataset for autonomous driving.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 11621–11631, 2020.
- Wang et al. [2023]
Huijie Wang, Tianyu Li, Yang Li, Li Chen, Chonghao Sima, Zhenbo Liu, Bangjun Wang, Peijin Jia, Yuting Wang, Shengyin Jiang, et al.
Openlane-v2: A topology reasoning benchmark for unified 3d hd mapping.
*Advances in Neural Information Processing Systems*, 36:18873–18884, 2023.
- Tran et al. [2019]
Ngoc Tran, Hieu Tran, Son Nguyen, Hoan Nguyen, and Tien Nguyen.
Does bleu score work for code migration?
In *2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC)*, pages 165–176. IEEE, 2019.
- Anderson et al. [2016]
Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould.
Spice: Semantic propositional image caption evaluation.
In *Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part V 14*, pages 382–398. Springer, 2016.
- Akter et al. [2022]
Mousumi Akter, Naman Bansal, and Shubhra Kanti Karmaker.
Revisiting automatic evaluation of extractive summarization task: Can we do better than rouge?
In *Findings of the Association for Computational Linguistics: ACL 2022*, pages 1547–1560, 2022.
- Evtikhiev et al. [2023]
Mikhail Evtikhiev, Egor Bogomolov, Yaroslav Sokolov, and Timofey Bryksin.
Out of the bleu: how should we assess quality of the code generation models?
*Journal of Systems and Software*, 203:111741, 2023.
- Papineni et al. [2002]
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu.
Bleu: a method for automatic evaluation of machine translation.
In *Proceedings of the 40th annual meeting of the Association for Computational Linguistics*, pages 311–318, 2002.
- Lin [2004]
Chin-Yew Lin.
Rouge: A package for automatic evaluation of summaries.
In *Text summarization branches out*, pages 74–81, 2004.
- Zheng et al. [2023]
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al.
Judging llm-as-a-judge with mt-bench and chatbot arena.
*Advances in Neural Information Processing Systems*, 36:46595–46623, 2023.
- Kwon et al. [2023]
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica.
Efficient memory management for large language model serving with pagedattention.
In *Proceedings of the 29th Symposium on Operating Systems Principles*, pages 611–626, 2023.
- Ishaq et al. [2025]
Ayesha Ishaq, Jean Lahoud, Ketan More, Omkar Thawakar, Ritesh Thawkar, Dinura Dissanayake, Noor Ahsan, Yuhao Li, Fahad Shahbaz Khan, Hisham Cholakkal, et al.
Drivelmm-o1: A step-by-step reasoning dataset and large multimodal model for driving scenario understanding.
*arXiv preprint arXiv:2503.10621*, 2025.
- Chen et al. [2024b]
Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al.
Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.
*arXiv preprint arXiv:2412.05271*, 2024b.

## Appendix A Limitations and Broader Impact

In this paper, we introduce AutoDriveRL, a unified training framework for vision-language models in autonomous driving that enhances reasoning and generalization across four core tasks; however, the approach still faces practical limitations.

Limitations. AutoDriveRL relies on manually designed, task-specific reward functions, which may limit its scalability to new tasks or sensor types. Additionally, the multi-stage inference pipeline introduces latency and has not been tested in real-time or closed-loop settings.

Broader Impact. By promoting interpretable, modular reasoning, AutoDriveRL may benefit the development of safer and more transparent autonomous systems. However, improper deployment of partially trained models or misaligned reward signals could lead to unsafe behavior. We encourage responsible use and further validation in realistic driving scenarios.

In the future, we will explore the effectiveness of AutoDriveRL across different frameworks and model sizes.

## Appendix B Ethical Statement

This study involving vision - language models (VLMs) and large language models (LLMs) in the context of autonomous driving adheres to the following ethical principles:

- •
Data Privacy and Legal Compliance: All datasets used are sourced legally. Personal or sensitive information within the data is properly anonymized to protect individuals’ privacy.
- •
Research Transparency: We are transparent about the data sources, research methods, and model limitations. The capabilities and limitations of our models are fully disclosed without exaggeration or concealment.
- •
Scientific Integrity: We maintain the highest standards of scientific integrity. Our research findings are based on rigorous experiments and objective analysis. Data fabrication or falsification is strictly prohibited.
- •
Bias Mitigation: We strive to minimize bias and discrimination in our models. We are committed to developing models that are fair and equitable.
- •
Intellectual Property Respect: We respect intellectual property rights and comply with relevant laws and regulations. Unauthorized use of proprietary data or models is avoided.

Our research focuses on exploring the potential of VLMs and LLMs in autonomous driving primarily from a theoretical and technical perspective. The emphasis is on improving the models’ generalization capabilities and interpretability, which could provide valuable insights for future research and development in the field.

## Appendix C Data Composition

We conducted 8 inference runs for Align-DS-V, Qwen2.5-VL-7B-Instruct, and Qwen2.5-VL-72B-Instruct on the DriveLM train dataset. Following the method in Eq. [1](https://arxiv.org/html/2505.20665v1#S3.E1), we computed the difficulty of frames rather than individual questions. Figure. [4](https://arxiv.org/html/2505.20665v1#A3.F4) shows the resulting difficulty distributions for each model. Next, we filtered out DriveLM samples with mutually exclusive options using specific rules and cleaned and balanced the dataset with methods like Action balancing.

To meet different requirements, we used Aligs-DS-V model scores to get frame scores, selected samples with filtered frame scores of $[25,45]$, and randomly sampled 1,000 frames as AutoDriveRL-Train. For the 3 models, we took samples with filtered frame scores of $[10,31]$, removed overlaps with the training set, and obtained DriveLM-Hard, which represents a more challenging subset of the data.

The task distribution of AutoDriveRL-Train and DriveLM-Hard can be observed in Figure. [5](https://arxiv.org/html/2505.20665v1#A3.F5).

Figure: (a) Qwen2.5-VL-72B-Instruct Score Distribution
Refer to caption: extracted/6482279/img/data/qwen25_72b_question_scores_distribution.png

Figure: Figure 5: The proportion of each task in the dataset
Refer to caption: extracted/6482279/img/data/dataset.png

Figure: (a) DriveLM-Hard Action Distribution
Refer to caption: extracted/6482279/img/data/pie_hard_ac.png

## Appendix D Models

LLaVA-1.5 [44] is an open-source multimodal vision-language model developed by the LLaVA team, designed to understand and generate language in response to visual inputs. It integrates a CLIP-based vision encoder with a Vicuna [56] language model through a multi-layer perceptron (MLP) connector, enabling it to perform tasks such as visual question answering, image captioning, and optical character recognition.

LLaVA-NeXT [45] is an advanced multimodal model based on LLaVA-1.5, developed to enhance performance in various applications. It demonstrates improved capabilities in reasoning, OCR, and world knowledge. The model supports higher image resolutions, utilizes refined datasets, and is built on more powerful language model foundations.

Qwen2.5-VL [47] is a multimodal model family developed by Alibaba Cloud, capable of handling tasks such as visual question answering, image/video captioning, and document understanding. It demonstrates strong performance across diverse vision-language applications.

InternVL3 [46] is a multimodal large language model family developed by Shanghai Artificial Intelligence Laboratory. It features a native multimodal pre-training approach, integrating text, image, and video modalities within a unified framework.

MM-Eureka [33] is a multimodal reasoning model extends large-scale rule-based reinforcement learning (RL) to multimodal reasoning.

R1-Onevision [37] is a multimodal reasoning model designed to bridge the gap between visual perception and deep reasoning.

Align-DS-V [41] is an experimental vision-language model that extends the DeepSeek-R1-Distill-Llama-8B, focusing on enhancing reasoning capabilities through all-modality alignment. It demonstrates strong performance on visual question answering and mathematical reasoning tasks.

DriveLM [35] is a vision-language model designed for end-to-end driving systems, developed by OpenDriveLab. It integrates large-scale vision-language models (VLMs) into driving systems to enhance generalization and interaction with human users.

Dolphin [36] is a multimodal vision-language model tailored for autonomous driving, built upon OpenFlamingo, and enhanced via Grounded Chain-of-Thought and in-context instruction tuning to achieve human-like understanding, reasoning.

GPT-4o [27] is a unified multimodal model by OpenAI that processes text, image, and audio inputs natively, enabling fast and accurate cross-modal reasoning with low latency.

## Appendix E Benchmark

DriveBench [34] is a benchmark dataset designed to assess the reliability of Vision-Language Models (VLMs) for autonomous driving. It consists of 19,200 frames and 20,498 question-answer pairs, covering a wide range of settings and tasks. It’s constructed by extracting a subset of frames from DriveLM-Train. Its innovation lies in being specifically created for evaluating the reliability of VLMs in autonomous driving scenarios, with diverse settings offering a comprehensive evaluation of VLMs’ performance in practical applications.

DriveLM [35] is a method that uses a visual language model (VLM) to boost autonomous driving capabilities. It presents the Graph VQA task to replicate the multi-step reasoning of human drivers. Based on nuScenes and CARLA, the DriveLM dataset is created. The DriveLM-Agent, a VLM-based baseline approach, is proposed to perform Graph VQA and end-to-end driving jointly. Its innovation is introducing the Graph VQA task to integrate visual understanding, language interaction, and driving decisions, offering a new approach for autonomous driving development.

## Appendix F Alignment Between Human and Model Rewards

To reduce the cost of API-based evaluation during RL training and DriveLM-Hard benchmarking, we use Qwen2.5-72B-Instruct as the default reward model. To assess the validity of using open-source LLMs for reward estimation, we conduct a correlation analysis between model scores and human judgments.

We design an online annotation platform for human evaluation and randomly select 300 samples from the evaluation set. Each sample is independently rated by three annotators. To ensure consistency, all annotators follow the same rubric used by our reward models, including task-specific criteria such as correctness, reasoning, and action relevance (see Appendix [I](https://arxiv.org/html/2505.20665v1#A9)). The average of the three scores is treated as the human gold score.

We then compute the Pearson correlation coefficient between human scores and model scores across several judge models, including the GPT-4 family (GPT-4o, GPT-4-turbo, GPT-3.5) and the Qwen2.5 family (7B and 72B).

**Table 4: Pearson correlation between human scores and model scores.**
| Judge Model | GPT-4o | GPT-4t | GPT-4 | GPT-3.5 | Qwen2.5-72B | Qwen2.5-7B |
| --- | --- | --- | --- | --- | --- | --- |
| Pearson w/ Human | 0.8309 | 0.7473 | 0.7473 | 0.6563 | 0.8138 | 0.4956 |

Figure: Figure 7: The scoring website used by human evaluators. Case Perception
Refer to caption: extracted/6482279/img/website/perception.png

Figure: Figure 8: The scoring website used by human evaluators. Case Prediction
Refer to caption: extracted/6482279/img/website/prediction.png

Figure: Figure 9: The scoring website used by human evaluators. Case Planning
Refer to caption: extracted/6482279/img/website/planning.png

Figure: Figure 10: The scoring website used by human evaluators. Case Behavior
Refer to caption: extracted/6482279/img/website/behavior.png

## Appendix G Comparison with Different Training Strategies

To validate the effectiveness of joint rl in AutoDriveRL, we conduct comparative experiments under the same data scale. Specifically, we compare (i) supervised fine-tuning (SFT), (ii) task-wise separate training (on perception, prediction, planning, and behavior), and (iii) our joint RL training strategy. All three variants are based on the same base model, Align-DS-V, and are evaluated on DriveBench. The results are shown in Table [5](https://arxiv.org/html/2505.20665v1#A7.T5)

**Table 5: Performance Evaluation of Diverse Training Strategies on Align-DS-V Across Multiple Driving Tasks in DriveBench Each column shows the GPT score ($\uparrow$) for the clean version of each task. We highlight the best-performing open-source model for each task.**
| Method | Size | Perception | Prediction | Planning | Behavior |
| --- | --- | --- | --- | --- | --- |
| DriveRX-SFT | $8$B | $23.73$ | $43.07$ | $73.07$ | $15.34$ |
| DriveRX-Separate | $8$B | $24.73$ | $46.59$ | $76.11$ | $55.18$ |
| DriveRX-Joint | $8$B | $32.88$ | $52.20$ | $78.63$ | $62.02$ |

In Table [5](https://arxiv.org/html/2505.20665v1#A7.T5), we observe that the joint RL training scheme significantly improves performance across all four tasks under the same data scale. This result confirms that jointly optimizing multiple reasoning stages—rather than training them in isolation—enhances the model’s overall effectiveness in autonomous driving tasks.This highlights the benefit of jointly optimizing semantically distinct reasoning stages, allowing the model to learn shared representations and task interactions that improve generalization.

## Appendix H Case Study

Below is our Case Study, with Fig. [11](https://arxiv.org/html/2505.20665v1#A8.F11), Fig. [12](https://arxiv.org/html/2505.20665v1#A8.F12), Fig. [13](https://arxiv.org/html/2505.20665v1#A8.F13), and Fig. [14](https://arxiv.org/html/2505.20665v1#A8.F14) corresponding to the tasks of Perception, Prediction, Planning, and Behavior respectively.

Figure: Figure 11: Perception task example comparing DriveLM and DriveRX. The goal is to identify important objects in the current scene for future reasoning and driving decisions. DriveLM incorrectly includes irrelevant objects such as a fire hydrant and a bus (highlighted in red), while DriveRX correctly identifies the relevant vehicles in front of and behind the ego vehicle (highlighted in blue) by performing structured visual analysis. This perception task focuses on understanding key environmental elements that influence downstream planning and behavior.
Refer to caption: extracted/6482279/img/prompt/task1.png

Figure: Figure 12: Prediction task example comparing Qwen2.5-VL-72B-Instruct and DriveRX. The task is to determine if the object at the specified coordinates <c1,CAM_FRONT,394.2,496.7> is a traffic sign or a road barrier. Qwen2.5-VL-72B-Instruct incorrectly claims the object is a road barrier (highlighted in red), despite no such object being clearly visible at the coordinates, exhibiting object hallucination. In contrast, DriveRX correctly identifies that there is no visible traffic sign or road barrier at the specified coordinates (highlighted in blue) by performing a structured visual analysis of the indicated image region. This highlights DriveRX’s improved ability to accurately perceive the scene and avoid hallucinations, which is critical for safe and reliable downstream driving decisions.
Refer to caption: extracted/6482279/img/prompt/task2.png

Figure: Figure 13: Planning task example comparing Align-DS-V and DriveRX. The goal is to determine safe actions for the ego vehicle in the current scene. Align-DS-V, while considering potential future obstructions, advises unnecessary caution due to the presence of a construction barrier and pedestrian, which may not be immediately relevant to the current driving situation (highlighted in red). In contrast, DriveRX provides a more accurate and practical recommendation by emphasizing maintaining the current speed on a clear road with no immediate hazards, as there are no visible pedestrians, other vehicles, or obstacles in the immediate vicinity (highlighted in blue). This planning task highlights the importance of analyzing visible obstacles and preparing for potential changes in road conditions to ensure safe driving, with DriveRX demonstrating a more reliable and contextually appropriate approach.
Refer to caption: extracted/6482279/img/prompt/task3.png

Figure: Figure 14: Behavior prediction task example comparing GPT-4o and DriveRX. The goal is to predict the behavior of the ego vehicle in the current scene. GPT-4o incorrectly predicts that the ego vehicle is steering to the left, which could very likely result in a collision with the red vehicle on the left (highlighted in red). In contrast, DriveRX accurately concludes that the ego vehicle is going straight and driving with normal speed (highlighted in blue). DriveRX’s analysis focuses on the vehicle’s alignment within the lane markings and the absence of indicators for sharp turning or rapid movement, suggesting it is traveling straight. This task highlights the importance of accurate visual analysis and the ability to avoid incorrect assumptions in behavior prediction, with DriveRX demonstrating a more reliable and contextually appropriate approach.
Refer to caption: extracted/6482279/img/prompt/task4.png

## Appendix I Experiment Prompt

### I.1 Task-Specific Reward Prompt

Figure: Figure 15: The prompt guiding Qwen2.5-72B-Instruct to score the answers to visual question answering questions in perception task.
Refer to caption: extracted/6482279/img/prompt/rm_1_1.png

Figure: Figure 16: The prompt for guiding Qwen2.5-72B-Instruct to evaluate the answers to multiple-choice questions in perception task.
Refer to caption: extracted/6482279/img/prompt/rm_1_2.png

Figure: Figure 17: The prompt guiding Qwen2.5-72B-Instruct to score the answers to visual question answering questions in prediction task.
Refer to caption: extracted/6482279/img/prompt/rm_2_2.png

Figure: Figure 18: The prompt for guiding Qwen2.5-72B-Instruct to evaluate the answers to judgment-type questions in prediction task.
Refer to caption: extracted/6482279/img/prompt/rm_2_1.png

Figure: Figure 19: The prompt for guiding Qwen2.5-72B-Instruct to evaluate the answers to visual-related questions in planning task.
Refer to caption: extracted/6482279/img/prompt/rm_3.png

Figure: Figure 20: The prompt for guiding Qwen2.5-72B-Instruct to score the answers to multiple-choice questions in behavior task.
Refer to caption: extracted/6482279/img/prompt/rm_4.png

### I.2 Inference Prompt

The following prompt is used during our evaluation on the DriveLM-Hard benchmark.

Figure: Figure 21: The prompt guiding VLMs to answer DriveLM-Hard questions.
Refer to caption: extracted/6482279/img/prompt/infer.png

## Appendix J Corruption Details

We provide the detailed GPT scores, including corruption details, for different tasks in Tab. [6](https://arxiv.org/html/2505.20665v1#A10.T6), Tab. [7](https://arxiv.org/html/2505.20665v1#A10.T7), Tab. [8](https://arxiv.org/html/2505.20665v1#A10.T8), Tab. [9](https://arxiv.org/html/2505.20665v1#A10.T9). and Tab. [10](https://arxiv.org/html/2505.20665v1#A10.T10)

**Table 6: Detailed GPT score results of MCQs for the Perception task. “Clean” represents clean image inputs. The “Corrupt” settings range from weather conditions, external disturbances, sensor failures, motion blur, and transmission errors. The benchmarked VLMs include commercial, open-sourced, and driving specialist models, respectively.**
| Method | Size | $\bullet$ Clean | $\bullet$ Brightness | $\bullet$ Dark | $\bullet$ Snow | $\bullet$ Fog | $\bullet$ Rain | $\bullet$ Lens Obstacle | $\bullet$ Water Splash | $\bullet$ Camera Crash | $\bullet$ Frame Lost | $\bullet$ Saturate | $\bullet$ Motion Blur | $\bullet$ Zoom Blur | $\bullet$ Bit Error | $\bullet$ Color Quant | $\bullet$ H.265 Compression |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| LLaVA-1.5 | $7$B | $32.40$ | $32.48$ | $32.95$ | $31.95$ | $32.43$ | $32.30$ | $32.88$ | $32.18$ | $32.93$ | $31.63$ | $32.50$ | $32.43$ | $32.93$ | $32.48$ | $32.18$ | $32.63$ |
| LLaVA-NeXT | $7$B | $32.98$ | $33.85$ | $20.43$ | $11.62$ | $16.58$ | $27.33$ | $18.24$ | $32.30$ | $26.80$ | $20.83$ | $23.50$ | $34.00$ | $17.75$ | $24.50$ | $18.03$ | $26.24$ |
| Qwen2.5-VL | $7$B | $36.75$ | $37.26$ | $24.91$ | $30.21$ | $24.36$ | $32.41$ | $27.53$ | $26.71$ | $26.50$ | $19.03$ | $33.42$ | $34.50$ | $18.15$ | $22.41$ | $21.47$ | $38.41$ |
| InternVL3 | $8$B | $36.22$ | $39.97$ | $35.94$ | $42.59$ | $37.24$ | $41.41$ | $41.24$ | $39.39$ | $33.74$ | $26.38$ | $35.06$ | $39.82$ | $41.33$ | $31.00$ | $40.65$ | $42.12$ |
| Qwen2.5VL | $72$B | $47.68$ | $39.71$ | $26.35$ | $38.59$ | $42.56$ | $45.03$ | $38.41$ | $39.38$ | $28.21$ | $19.38$ | $44.18$ | $39.85$ | $36.76$ | $32.35$ | $29.55$ | $49.47$ |
| InternVL3 | $78$B | $47.33$ | $44.82$ | $38.35$ | $45.29$ | $43.50$ | $40.09$ | $46.09$ | $41.00$ | $32.09$ | $24.53$ | $40.68$ | $47.30$ | $39.44$ | $36.61$ | $41.30$ | $45.97$ |
| MM-Eureka | $7$B | $43.33$ | $29.21$ | $17.59$ | $31.97$ | $37.97$ | $41.44$ | $36.44$ | $32.21$ | $30.27$ | $22.71$ | $41.73$ | $34.68$ | $19.56$ | $26.39$ | $27.21$ | $41.21$ |
| R1-Onevision | $7$B | $36.89$ | $30.24$ | $23.71$ | $31.56$ | $37.36$ | $34.32$ | $36.12$ | $25.06$ | $30.18$ | $34.74$ | $26.82$ | $24.76$ | $34.03$ | $23.71$ | $29.56$ | $29.59$ |
| DriveLM | $7$B | $22.38$ | $20.78$ | $25.30$ | $18.98$ | $24.43$ | $25.95$ | $22.03$ | $21.03$ | $21.95$ | $16.28$ | $19.38$ | $22.98$ | $20.93$ | $19.90$ | $16.25$ | $26.48$ |
| Dolphin | $7$B | $6.50$ | $10.18$ | $11.08$ | $10.70$ | $9.53$ | $10.58$ | $9.93$ | $9.80$ | $10.08$ | $9.95$ | $11.20$ | $9.85$ | $10.10$ | $8.80$ | $10.00$ | $11.10$ |
| Align-DS-V | $8$B | $39.8$ | $39.65$ | $31.00$ | $41.50$ | $33.24$ | $42.50$ | $35.15$ | $36.85$ | $27.41$ | $38.32$ | $25.21$ | $28.68$ | $30.44$ | $35.91$ |  |  |
| DriveRX | $8$B | $42.79$ | $34.71$ | $36.09$ | $39.29$ | $43.74$ | $39.79$ | $27.56$ | $38.85$ | $33.68$ | $26.74$ | $32.18$ | $33.29$ | $30.53$ | $30.65$ | $25.59$ | $38.50$ |

**Table 7: Detailed GPT score results of open-ended questions for the Perception task. “Clean” represents clean image inputs. The “Corrupt” settings range from weather conditions, external disturbances, sensor failures, motion blur, and transmission errors. The benchmarked VLMs include commercial, open-sourced, and driving specialist models, respectively.**
| Method | Size | $\bullet$ Clean | $\bullet$ Brightness | $\bullet$ Dark | $\bullet$ Snow | $\bullet$ Fog | $\bullet$ Rain | $\bullet$ Lens Obstacle | $\bullet$ Water Splash | $\bullet$ Camera Crash | $\bullet$ Frame Lost | $\bullet$ Saturate | $\bullet$ Motion Blur | $\bullet$ Zoom Blur | $\bullet$ Bit Error | $\bullet$ Color Quant | $\bullet$ H.265 Compression |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| LLaVA-1.5 | $7$B | $14.03$ | $13.53$ | $13.31$ | $13.31$ | $13.61$ | $13.75$ | $13.91$ | $13.48$ | $13.95$ | $12.90$ | $12.98$ | $13.35$ | $13.05$ | $13.36$ | $12.83$ | $14.28$ |
| LLaVA-NeXT | $7$B | $15.33$ | $15.49$ | $14.95$ | $16.61$ | $15.62$ | $16.05$ | $15.66$ | $15.66$ | $15.19$ | $16.04$ | $15.16$ | $16.06$ | $17.92$ | $15.27$ | $15.10$ | $15.86$ |
| Qwen2.5-VL | $7$B | $27.09$ | $27.76$ | $23.14$ | $27.05$ | $28.05$ | $25.90$ | $28.24$ | $28.29$ | $28.48$ | $25.81$ | $28.62$ | $25.90$ | $25.86$ | $25.38$ | $23.62$ | $25.38$ |
| InternVL3 | $8$B | $30.21$ | $30.38$ | $31.29$ | $32.33$ | $28.52$ | $29.57$ | $28.95$ | $31.33$ | $29.95$ | $25.29$ | $30.76$ | $29.76$ | $30.67$ | $28.71$ | $29.90$ | $30.33$ |
| Qwen2.5VL | $72$B | $27.43$ | $29.29$ | $26.76$ | $23.86$ | $25.05$ | $29.38$ | $28.14$ | $28.43$ | $29.38$ | $26.14$ | $27.86$ | $28.71$ | $28.10$ | $27.52$ | $29.48$ | $29.71$ |
| InternVL3 | $78$B | $31.53$ | $32.33$ | $31.95$ | $32.05$ | $29.62$ | $30.38$ | $33.35$ | $28.76$ | $32.86$ | $28.76$ | $29.14$ | $28.90$ | $30.71$ | $30.76$ | $30.90$ | $31.10$ |
| MM-Eureka | $7$B | $24.46$ | $26.48$ | $25.38$ | $26.29$ | $26.33$ | $26.62$ | $27.71$ | $27.14$ | $26.10$ | $26.00$ | $26.19$ | $27.14$ | $26.52$ | $25.67$ | $26.81$ | $26.62$ |
| R1-Onevision | $7$B | $20.39$ | $21.39$ | $19.20$ | $19.52$ | $19.70$ | $20.60$ | $21.50$ | $21.89$ | $22.83$ | $20.95$ | $20.84$ | $21.68$ | $17.93$ | $18.67$ | $22.00$ | $21.67$ |
| DriveLM | $7$B | $11.32$ | $11.30$ | $10.03$ | $11.21$ | $9.71$ | $10.22$ | $10.71$ | $11.01$ | $11.14$ | $10.13$ | $9.38$ | $10.97$ | $9.03$ | $11.39$ | $10.50$ | $10.73$ |
| Dolphin | $7$B | $12.68$ | $12.07$ | $10.34$ | $12.37$ | $11.46$ | $11.73$ | $12.33$ | $11.04$ | $12.34$ | $11.39$ | $11.47$ | $11.34$ | $10.17$ | $11.40$ | $11.35$ | $11.65$ |
| Align-DS-V | $8$B | $23.03$ | $39.65$ | $31.00$ | $41.50$ | $33.24$ | $42.50$ | $35.15$ | $36.85$ | $27.41$ | $38.32$ | $25.21$ | $28.68$ | $30.44$ | $35.91$ | $36.32$ | $25.71$ |
| DriveRX | $8$B | $22.97$ | $24.19$ | $26.33$ | $26.33$ | $27.52$ | $25.86$ | $25.38$ | $25.14$ | $26.10$ | $26.81$ | $25.38$ | $27.05$ | $24.67$ | $23.71$ | $25.76$ | $25.62$ |

**Table 8: Detailed GPT score results of the open-ended questions for Prediction.**
| Method | Size | $\bullet$ Clean | $\bullet$ Brightness | $\bullet$ Dark | $\bullet$ Snow | $\bullet$ Fog | $\bullet$ Rain | $\bullet$ Lens Obstacle | $\bullet$ Water Splash | $\bullet$ Camera Crash | $\bullet$ Frame Lost | $\bullet$ Saturate | $\bullet$ Motion Blur | $\bullet$ Zoom Blur | $\bullet$ Bit Error | $\bullet$ Color Quant | $\bullet$ H.265 Compression |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| LLaVA-1.5 | $7$B | $22.02$ | $24.79$ | $20.95$ | $15.97$ | $15.30$ | $18.98$ | $16.28$ | $24.16$ | $6.11$ | $13.90$ | $20.30$ | $25.20$ | $10.56$ | $11.10$ | $15.61$ | $23.92$ |
| LLaVA-NeXT | $7$B | $35.07$ | $37.15$ | $35.31$ | $37.59$ | $37.62$ | $35.44$ | $37.00$ | $35.87$ | $36.25$ | $30.10$ | $40.56$ | $34.66$ | $39.36$ | $31.74$ | $34.07$ | $35.66$ |
| Qwen2.5-VL | $7$B | $42.26$ | $50.75$ | $36.58$ | $47.25$ | $46.00$ | $47.58$ | $54.50$ | $53.67$ | $45.00$ | $46.17$ | $44.83$ | $48.08$ | $43.00$ | $48.00$ | $43.25$ | $52.25$ |
| InternVL3 | $8$B | $41.21$ | $50.17$ | $43.08$ | $51.50$ | $54.50$ | $45.75$ | $47.67$ | $48.33$ | $39.17$ | $45.17$ | $46.92$ | $46.33$ | $48.33$ | $46.58$ | $50.42$ | $54.17$ |
| Qwen2.5VL | $72$B | $54.89$ | $56.00$ | $46.58$ | $38.83$ | $44.00$ | $51.50$ | $57.33$ | $54.42$ | $47.17$ | $52.83$ | $50.58$ | $50.50$ | $55.75$ | $50.42$ | $51.17$ | $49.92$ |
| InternVL3 | $78$B | $50.93$ | $55.50$ | $44.92$ | $51.50$ | $59.00$ | $53.92$ | $51.67$ | $50.33$ | $46.92$ | $42.83$ | $49.08$ | $47.75$ | $51.42$ | $45.08$ | $47.75$ | $48.00$ |
| MM-Eureka | $7$B | $40.31$ | $44.42$ | $37.58$ | $36.00$ | $36.75$ | $32.42$ | $42.91$ | $45.42$ | $45.17$ | $53.75$ | $45.33$ | $43.92$ | $39.75$ | $39.17$ | $38.75$ | $44.50$ |
| R1-Onevision | $7$B | $51.22$ | $44.40$ | $33.25$ | $47.92$ | $58.17$ | $47.75$ | $46.33$ | $47.42$ | $48.80$ | $46.25$ | $42.33$ | $46.27$ | $47.45$ | $52.08$ | $45.10$ | $50.18$ |
| DriveLM | $7$B | $44.33$ | $46.82$ | $43.90$ | $42.33$ | $35.84$ | $44.13$ | $44.00$ | $42.59$ | $46.25$ | $33.56$ | $29.69$ | $42.15$ | $19.00$ | $38.20$ | $44.33$ | $42.87$ |
| Dolphin | $7$B | $32.66$ | $29.85$ | $32.31$ | $24.64$ | $29.92$ | $31.38$ | $33.41$ | $31.79$ | $29.05$ | $30.93$ | $30.49$ | $31.59$ | $26.38$ | $30.13$ | $25.64$ | $30.62$ |
| Align-DS-V | $8$B | $41.51$ | $41.50$ | $35.83$ | $38.75$ | $32.67$ | $39.42$ | $42.50$ | $38.33$ | $41.92$ | $32.83$ | $42.92$ | $44.42$ | $35.25$ | $40.67$ | $35.42$ | $35.00$ |
| DriveRX | $8$B | $52.20$ | $51.42$ | $41.75$ | $48.83$ | $48.92$ | $47.17$ | $49.25$ | $51.92$ | $47.92$ | $38.67$ | $55.42$ | $48.17$ | $53.25$ | $48.50$ | $39.50$ | $53.75$ |

**Table 9: Detailed GPT score results of the open-ended questions for Planning.**
| Method | Size | $\bullet$ Clean | $\bullet$ Brightness | $\bullet$ Dark | $\bullet$ Snow | $\bullet$ Fog | $\bullet$ Rain | $\bullet$ Lens Obstacle | $\bullet$ Water Splash | $\bullet$ Camera Crash | $\bullet$ Frame Lost | $\bullet$ Saturate | $\bullet$ Motion Blur | $\bullet$ Zoom Blur | $\bullet$ Bit Error | $\bullet$ Color Quant | $\bullet$ H.265 Compression |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| LLaVA-1.5 | $7$B | $29.15$ | $31.52$ | $31.49$ | $32.58$ | $32.42$ | $31.00$ | $29.81$ | $31.72$ | $33.70$ | $35.95$ | $29.93$ | $31.20$ | $30.65$ | $30.05$ | $30.00$ | $30.61$ |
| LLaVA-NeXT | $7$B | $45.27$ | $45.64$ | $44.54$ | $43.55$ | $44.17$ | $45.08$ | $44.62$ | $44.21$ | $45.69$ | $44.51$ | $41.17$ | $45.30$ | $44.57$ | $43.67$ | $43.57$ | $45.13$ |
| Qwen2.5-VL | $7$B | $55.70$ | $62.34$ | $61.15$ | $61.76$ | $56.91$ | $54.27$ | $57.51$ | $57.48$ | $56.37$ | $48.50$ | $65.56$ | $60.16$ | $56.63$ | $58.27$ | $54.06$ | $58.60$ |
| InternVL3 | $8$B | $70.97$ | $54.07$ | $51.07$ | $50.00$ | $53.82$ | $55.96$ | $49.00$ | $49.57$ | $54.77$ | $52.09$ | $50.56$ | $49.07$ | $48.49$ | $51.88$ | $49.99$ | $52.94$ |
| Qwen2.5VL | $72$B | $77.18$ | $80.95$ | $69.67$ | $75.22$ | $76.24$ | $77.79$ | $78.12$ | $78.40$ | $69.98$ | $53.11$ | $74.74$ | $80.84$ | $75.13$ | $73.33$ | $68.98$ | $77.67$ |
| InternVL3 | $78$B | $82.24$ | $79.30$ | $73.96$ | $76.63$ | $79.51$ | $80.70$ | $82.28$ | $81.33$ | $77.95$ | $69.71$ | $77.17$ | $82.17$ | $79.67$ | $78.41$ | $77.48$ | $79.90$ |
| MM-Eureka | $7$B | $65.38$ | $63.35$ | $58.75$ | $60.85$ | $59.21$ | $66.11$ | $57.41$ | $62.58$ | $59.90$ | $49.80$ | $60.99$ | $62.35$ | $58.98$ | $57.99$ | $52.95$ | $71.53$ |
| R1-Onevision | $7$B | $55.12$ | $53.03$ | $55.92$ | $58.28$ | $58.74$ | $57.04$ | $57.44$ | $56.36$ | $57.13$ | $60.25$ | $53.11$ | $49.87$ | $55.22$ | $51.55$ | $50.57$ | $58.60$ |
| DriveLM | $7$B | $68.71$ | $67.25$ | $67.52$ | $65.72$ | $63.08$ | $69.60$ | $69.04$ | $67.97$ | $67.85$ | $66.47$ | $66.25$ | $67.93$ | $70.17$ | $68.46$ | $68.30$ | $68.59$ |
| Dolphin | $7$B | $52.91$ | $51.85$ | $55.39$ | $53.09$ | $54.78$ | $53.92$ | $51.79$ | $53.57$ | $55.73$ | $57.81$ | $55.78$ | $51.42$ | $50.95$ | $53.35$ | $54.89$ | $52.17$ |
| Align-DS-V | $8$B | $51.16$ | $61.88$ | $56.82$ | $53.65$ | $58.77$ | $59.20$ | $58.10$ | $60.98$ | $58.71$ | $57.55$ | $51.16$ | $59.95$ | $55.45$ | $60.63$ | $53.04$ | $60.82$ |
| DriveRX | $8$B | $78.63$ | $77.89$ | $74.99$ | $74.54$ | $74.33$ | $75.29$ | $75.44$ | $74.71$ | $75.96$ | $78.18$ | $74.87$ | $74.18$ | $76.48$ | $75.59$ | $73.67$ | $76.80$ |

**Table 10: Detailed GPT score results of the MCQs for Behavior.**
| Method | Size | $\bullet$ Clean | $\bullet$ Brightness | $\bullet$ Dark | $\bullet$ Snow | $\bullet$ Fog | $\bullet$ Rain | $\bullet$ Lens Obstacle | $\bullet$ Water Splash | $\bullet$ Camera Crash | $\bullet$ Frame Lost | $\bullet$ Saturate | $\bullet$ Motion Blur | $\bullet$ Zoom Blur | $\bullet$ Bit Error | $\bullet$ Color Quant | $\bullet$ H.265 Compression |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| LLaVA-1.5 | $7$B | $13.60$ | $12.79$ | $12.83$ | $15.57$ | $12.63$ | $14.06$ | $13.99$ | $12.79$ | $14.68$ | $13.65$ | $13.12$ | $13.55$ | $13.83$ | $13.98$ | $13.44$ | $13.48$ |
| LLaVA-NeXT | $7$B | $48.16$ | $48.84$ | $38.82$ | $15.90$ | $39.13$ | $47.07$ | $20.72$ | $47.02$ | $48.20$ | $36.67$ | $39.69$ | $47.36$ | $39.60$ | $46.99$ | $28.13$ | $47.55$ |
| Qwen2.5-VL | $7$B | $47.40$ | $50.59$ | $30.63$ | $46.11$ | $45.26$ | $46.30$ | $36.26$ | $37.85$ | $54.56$ | $49.41$ | $44.81$ | $39.56$ | $36.81$ | $33.59$ | $48.89$ | $41.67$ |
| InternVL3 | $8$B | $51.47$ | $40.22$ | $62.26$ | $54.07$ | $57.48$ | $48.59$ | $54.96$ | $53.93$ | $48.78$ | $49.70$ | $43.04$ | $51.89$ | $59.70$ | $41.70$ | $51.30$ | $60.15$ |
| Qwen2.5VL | $72$B | $55.57$ | $45.04$ | $58.67$ | $59.48$ | $57.56$ | $51.56$ | $49.67$ | $57.63$ | $57.93$ | $49.37$ | $53.52$ | $49.44$ | $35.19$ | $41.59$ | $56.37$ | $51.70$ |
| InternVL3 | $78$B | $50.73$ | $47.70$ | $56.04$ | $42.41$ | $55.78$ | $53.07$ | $50.67$ | $51.41$ | $59.30$ | $51.19$ | $68.19$ | $42.85$ | $37.04$ | $54.07$ | $58.59$ | $52.15$ |
| MM-Eureka | $7$B | $49.4$ | $47.58$ | $54.05$ | $36.73$ | $39.04$ | $59.79$ | $46.11$ | $51.45$ | $50.86$ | $61.00$ | $42.00$ | $56.93$ | $30.61$ | $46.48$ | $34.47$ | $56.80$ |
| R1-Onevision | $7$B | $41.57$ | $39.35$ | $57.17$ | $38.31$ | $31.74$ | $48.33$ | $45.15$ | $51.23$ | $44.00$ | $37.07$ | $37.92$ | $30.81$ | $32.04$ | $35.85$ | $32.26$ | $36.93$ |
| DriveLM | $7$B | $42.78$ | $47.18$ | $36.30$ | $40.70$ | $39.18$ | $40.93$ | $43.30$ | $40.98$ | $39.95$ | $38.23$ | $40.08$ | $45.68$ | $38.88$ | $41.10$ | $33.50$ | $39.65$ |
| Dolphin | $7$B | $8.81$ | $7.17$ | $9.54$ | $9.02$ | $6.48$ | $8.05$ | $7.95$ | $7.10$ | $9.29$ | $8.94$ | $8.02$ | $8.02$ | $9.42$ | $8.37$ | $10.07$ | $6.32$ |
| Align-DS-V | $8$B | $50.53$ | $50.26$ | $36.07$ | $38.04$ | $38.67$ | $44.78$ | $28.52$ | $54.41$ | $48.19$ | $34.30$ | $30.96$ | $36.22$ | $35.56$ | $52.33$ | $34.52$ | $50.63$ |
| DriveRX | $8$B | $62.02$ | $55.67$ | $61.04$ | $54.07$ | $57.15$ | $60.35$ | $56.22$ | $54.70$ | $58.63$ | $51.19$ | $46.85$ | $57.30$ | $41.15$ | $45.88$ | $69.04$ | $63.48$ |

## Appendix K Implementation Details

### K.1 Hardware Configuration

For all train and inference experiments, we utilized two computational cluster equipped with 8 NVIDIA A100-80GB GPUs. Our model DriveRX was trained across 16 NVIDIA A100-80GB GPUs for 5 epochs, a process that took 87 hours.

### K.2 Hyperparameter Settings

During GRPO [40] training, we adopt a prompt batch size of 128 is utilized, with 8 rollouts generated per prompt. The maximum rollout length is capped at 4,096 tokens. A default sampling temperature of 1.0 is set, and a KL loss coefficient of 1e-2 is employed. The training process consists of 5 epochs.

For Supervised Fine-tuning, we set the learining rate to 1e-6 trained for 3 epochs.

To ensure the reliability of the evaluation, each model is tested three times, with the average value taken as the final result. For evaluation purposes, all models are adjusted to a temperature of 0.2. The vLLM[57] framework is employed for inference, with the maximum length restricted to 4096 tokens.

## Appendix L Comparison with DriveLMM-o1 Using DriveBench Evaluation

DriveLMM-o1[58] is a recent work closely related to ours, also targeting multi-task visual-language reasoning in autonomous driving. It constructs a step-by-step reasoning dataset covering perception, prediction, and planning tasks, and fine-tunes an InternVL2.5-8B[59] model using LoRA to improve logical coherence and interpretability in decision making.

Compared to DriveLMM-o1, our method introduces several key extensions: (1) We incorporate the behavior decision-making task, thereby covering the full reasoning chain in autonomous driving. (2) Instead of relying on supervised fine-tuning, we adopt AutoDriveRL, a joint reinforcement learning framework that leverages task-specific reward models to optimize the four semantically distinct subtasks in a coordinated manner. (3) While interpretability remains important, our primary objective is to enhance robustness and generalization in complex driving scenarios through structured, reward-guided reasoning.

Since both models are trained on nuScenes-derived data, we conduct a comparison on the DriveBench evaluation set, which also builds upon nuScenes. As shown in Table [11](https://arxiv.org/html/2505.20665v1#A12.T11), DriveRX significantly outperforms DriveLMM-o1 on the prediction, planning, and behavior tasks, demonstrating the effectiveness of AutoDriveRL in learning structured, cross-task reasoning. Notably, this result also supports our earlier finding (Section [5.2](https://arxiv.org/html/2505.20665v1#S5.SS2)) that a stronger vision encoder contributes substantially to perception performance. While DriveRX adopts a CLIP-based encoder with efficient visual-language alignment, DriveLMM-o1 leverages the InternVL2.5 encoder, which offers higher capacity for fine-grained visual understanding. As a result, DriveLMM-o1 retains an advantage in the perception task, highlighting the importance of high-resolution visual representations for low-level scene comprehension.

**Table 11: Evaluations of VLMs across different driving tasks on DriveBench Each column shows the GPT score ($\uparrow$) for the clean version of each task. We highlight the best-performing open-source model for each task.**
| Method | Size | Perception | Prediction | Planning | Behavior |
| --- | --- | --- | --- | --- | --- |
| InternVL3 [46] | $8$B | $33.21$ | $41.21$ | $70.97$ | $51.47$ |
| DriveLMM-o1 [58] | $8$B | $34.31$ | $43.95$ | $73.7$ | $47.62$ |
| DriveRX | $8$B | $32.88$ | $52.20$ | $78.63$ | $62.02$ |

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: 1. How robust is DriveRX to distribution shifts, e.g., unseen weather or sensor noise, beyond the “corrupted” conditions mentioned?

2. Are there any failure cases observed during deployment or simulation that reveal limits in the cross-task reasoning design?
soundness: 3
strengths: 1. The formulation of each sub-task as a vision-language QA problem is interesting.

2. The experimental results are promising.
summary: This paper introduces AutoDriveRL, a unified reinforcement learning framework that models autonomous driving as a structured reasoning process across perception, prediction, planning, and behavior. It trains a cross-task reasoning model, DriveRX, which outperforms GPT-4o in behavior reasoning and shows strong robustness in complex driving scenarios.
weaknesses: 1. The introduction section feels somewhat wordy and difficult to follow. It’s not clearly structured around three key points: what common issues exist in current works, what direction and method are proposed, and how the proposed method is designed. These aspects should be laid out more clearly.

2. Additionally, the paper categorizes the autonomous driving pipeline into perception, prediction, planning, and behavior. Typically, autonomous driving is divided into three main tasks, perception, prediction, and planning, while behavior is often treated as part of prediction. What is the rationale for introducing behavior as a separate stage?

3. The description of the task-specific reward models lacks clarity. How the reinforcement signals are computed and whether they are consistent across tasks could be better elaborated.

4. It’s unclear how much each reasoning stage (perception, prediction, etc.) contributes to overall performance,  finer-grained ablation or visualization would improve interpretability.

---

confidence: 4
contribution: 2
initial_rating: 6
presentation: 3
questions: 1. Ambiguous Reinforcement Learning Details
Equation (2) defines GRPO, but the intra-group advantage term (Eq. 3) is shared across tokens, not time-dependent, this simplifies training but may degrade credit assignment.
No mention of reward normalization schedule, policy rollout horizon, or sampling temperature, which are essential for RL reproducibility.
The rule-based and LLM-based reward models are described qualitatively but lack explicit scoring functions, thresholds, or examples (Appendix M.1 is referenced but not shown in main text).

2. Missing Quantitative Ablations
The framework introduces multiple key design choices (reward composition, GRPO vs PPO, per-task weighting, data filtering), yet no ablation results are provided in the main paper.
For example, how much does AutoDriveRL contribute compared to supervised fine-tuning on the same data? The gain from GRPO remains unquantified.

3. Unclear Task Coupling
The paper claims cross-task reasoning, but tasks are trained independently with shared parameters, without conditioning on previous task outputs (explicitly stated in §3.1).
This design limits true reasoning chaining; the reasoning chain is only semantic, not computationally causal. Thus, DriveRX may not genuinely achieve process-level reasoning across tasks.

4. Evaluation Ambiguities
The GPT score metric is used exclusively, judged by GPT-3.5-Turbo. This raises reliability and circularity issues — the model is trained with LLM-based rewards and then evaluated by another LLM judge.
There is no mention of inter-rater agreement, or calibration with human labels (Appendix I claims alignment but gives no correlation coefficients in the main text).
Metrics like ADE/Col. in Table 3 and Driving Score in Table 4 are reported but lack variance or standard deviation, leaving statistical significance unclear.

5. Distillation Pipeline Unclear
In §6, reasoning data from DriveRX is distilled into DriveRX-VLA, but the exact mapping from language to action tokens and supervision loss are unspecified.In §6, reasoning data from DriveRX is distilled into DriveRX-VLA, but the exact mapping from language to action tokens and supervision loss are unspecified.
It’s not clear whether the VLA model receives image input, text prompts, or both during inference — this affects the claimed “unified backbone” property.

6. Interpretability and Example Limitation
Only one qualitative case (Figure 3) is provided. No multi-turn reasoning trace or counterfactual analysis is shown, weakening claims of interpretability.
soundness: 3
strengths: This study introduces a reinforcement-learning–driven, multi-task vision-language reasoning framework designed to enhance the coordination between perception, reasoning, and action in autonomous driving. By attaching task-specific reward models to each stage of the reasoning process, the approach provides finer-grained supervision, improving interpretability and stability.

It organizes the problem into four subtasks that reflect the cognitive pipeline of autonomous driving—from perception to behavior. A Visual Question Answering (VQA)–style data formulation is used to unify the data structure, supporting cross-task consistency and a cleaner, more integrated learning signal.

Empirical results show that DriveRX achieves 62.02 on the Behavior task, outpacing GPT-4o (55.04) and the strongest open-source vision-language models. The method also demonstrates robust performance under visual corruption, underscoring its resilience. Baseline comparisons cover a broad spectrum, including commercial models (e.g., GPT-4o), generalist models (LLaVA, Qwen2.5-VL), reasoning-oriented models (MM-Eureka, R1-OneVision), and domain-specific systems (DriveLM, Dolphins), all evaluated under consistent settings.

Beyond core reasoning, the approach extends to trajectory prediction and control through DriveRX-Agent and DriveRX-VLA. These extensions illustrate how high-level reasoning can be bridged with low-level action generation, pointing toward a unified driving agent that blends perception, reasoning, and control in a single framework.
summary: This paper proposes AutoDriveRL, a reinforcement learning (RL) framework for vision-language models (VLMs) applied to autonomous driving, decomposing driving into four interpretable sub-tasks: Perception, Prediction, Planning, and Behavior. Each sub-task is framed as a VQA problem with a task-specific reward model. Based on this framework, the authors train DriveRX, a cross-task reasoning VLM that aims to unify multi-stage decision-making. Experiments on DriveBench and DriveLM-Hard show that DriveRX outperforms GPT-4o and other baselines in the Behavior task and maintains robustness under visual corruptions. They further extend DriveRX to (1) DriveRX-Agent for trajectory generation and (2) DriveRX-VLA for closed-loop control via reasoning-based distillation.
weaknesses: 1. Ambiguous Reinforcement Learning Details
Equation (2) defines GRPO, but the intra-group advantage term (Eq. 3) is shared across tokens, not time-dependent, this simplifies training but may degrade credit assignment.
No mention of reward normalization schedule, policy rollout horizon, or sampling temperature, which are essential for RL reproducibility.
The rule-based and LLM-based reward models are described qualitatively but lack explicit scoring functions, thresholds, or examples (Appendix M.1 is referenced but not shown in main text).

2. Missing Quantitative Ablations
The framework introduces multiple key design choices (reward composition, GRPO vs PPO, per-task weighting, data filtering), yet no ablation results are provided in the main paper.
For example, how much does AutoDriveRL contribute compared to supervised fine-tuning on the same data? The gain from GRPO remains unquantified.

3. Unclear Task Coupling
The paper claims cross-task reasoning, but tasks are trained independently with shared parameters, without conditioning on previous task outputs (explicitly stated in §3.1).
This design limits true reasoning chaining; the reasoning chain is only semantic, not computationally causal. Thus, DriveRX may not genuinely achieve process-level reasoning across tasks.

4. Evaluation Ambiguities
The GPT score metric is used exclusively, judged by GPT-3.5-Turbo. This raises reliability and circularity issues — the model is trained with LLM-based rewards and then evaluated by another LLM judge.
There is no mention of inter-rater agreement, or calibration with human labels (Appendix I claims alignment but gives no correlation coefficients in the main text).
Metrics like ADE/Col. in Table 3 and Driving Score in Table 4 are reported but lack variance or standard deviation, leaving statistical significance unclear.

5. Distillation Pipeline Unclear
In §6, reasoning data from DriveRX is distilled into DriveRX-VLA, but the exact mapping from language to action tokens and supervision loss are unspecified.In §6, reasoning data from DriveRX is distilled into DriveRX-VLA, but the exact mapping from language to action tokens and supervision loss are unspecified.
It’s not clear whether the VLA model receives image input, text prompts, or both during inference — this affects the claimed “unified backbone” property.

6. Interpretability and Example Limitation
Only one qualitative case (Figure 3) is provided. No multi-turn reasoning trace or counterfactual analysis is shown, weakening claims of interpretability.

---

confidence: 5
contribution: 3
initial_rating: 6
presentation: 3
questions: + Could the authors clarify the time latency when using an LLM for scoring during training?

+ For the prediction and planning tasks, did the authors explore using a rule-based reward model? If so, how does it compare (advantages/disadvantages) to the LLM-based reward?

+ Could the authors provide results on the AlphaDrive benchmark for planning, so we can assess true out-of-distribution (OOD) performance?

+ Based on Table 8, I don’t understand why the gap between SFT (supervised fine-tuning) and RL in the behavior task is so large—this phenomenon isn’t observed in other related work. Could the authors explain?

+ **Most importantly, I’d like the authors to clearly articulate the intended application scenarios for DriveRX:**

(i) As a strong backbone for low-level planning tasks?

(ii) As a data annotator for long-tail driving scenarios?

(iii) As a reward model to improve other VLA/VLMs’ low-level planning performance?
soundness: 2
strengths: + Proposes a unified RL framework (AutoDriveRL) enabling interpretable, multi-stage reasoning across driving subtasks.

+ Demonstrates state-of-the-art performance and robustness, surpassing larger models under both clean and corrupted conditions.

+ Extends practical impact by showing reasoning-enhanced transferability to trajectory and control tasks.
summary: This paper presents DriveRX, a vision-language reasoning model for autonomous driving trained via the AutoDriveRL reinforcement learning framework.
AutoDriveRL decomposes driving into perception, prediction, planning, and behavior tasks, each optimized with task-specific reward models.
Using structured reasoning and joint RL training, DriveRX achieves strong generalization and robustness on DriveBench and DriveLM-Hard benchmarks, outperforming GPT-4o in behavior reasoning.
The model also supports downstream trajectory prediction and action generation through reasoning-based distillation.
weaknesses: **For the image input**: In the structured reasoning process you designed, including prediction and planning, you only feed the model multi-view images, which makes it very hard to ensure the model can capture temporal information like speed — this doesn’t really make sense. Although some other VLAs also only use images, they at least provide historical ego‐states to ensure the ego car’s temporal information.

**Application**: The authors design a very complex reasoning process; I am quite puzzled about what scenario the authors expect their work to be applied in. If it’s just high-level prediction, then they have not opened up a large gap compared with current open-sourced VLMs such as InternVL3-8B, Qwen3-VL-8B, etc. (And I must emphasize: the authors test on DriveBench but that is not truly OOD because both DriveRX and DriveBench data are constructed based on DriveLM.) I am not sure whether DriveRX has better performance than the open-sourced VLMs in other domains of driving scenarios (e.g., DriveAction).

**Latency**: Because I am confused about the application of DriveRX. If it is used for low-level planning, then latency is a big challenge; and based on Table 3 results, its performance gap with existing VLAs model is significant (e.g., Auto-VLA).

---

confidence: 3
contribution: 2
initial_rating: 2
presentation: 3
questions: Can the authors clarify the evaluation setup for Table 3? How are the metrics calculated? 

Why are the results not compared with other domain-specific models (e.g., UniAD, VAD, DriveVLM, OmniDrive, ORION)?

What's the main usecase for this VLM? Is it aiming for improving existing self driving system?
soundness: 3
strengths: The paper shows a clear pipeline for data construction and training framework of VLM training, and it shows some improvement over general-purpose models like Qwen and other large VLMs, which suggests that domain-specific fine-tuning helps the model better understand driving scenes.
summary: This paper presents DriveRX, a vision-language-action model designed to enhance reasoning and decision-making for autonomous driving. The authors aim to leverage language-based reasoning to handle complex or ambiguous driving cases and show some performance improvements over generalist vision-language models. The paper also includes an evaluation on a trajectory prediction benchmark to demonstrate the model’s potential for better planning.
weaknesses: I have some concerns about the motivation and experimental setup. Most recent works that use VLMs for driving focus on long-tail reasoning or trajectory prediction, and they evaluate on clear benchmarks with established baselines. This paper shows results in Table 3, but the setup isn’t well explained, so it’s hard to understand what the numbers actually mean or how fair the comparison is.

The model is not compared against any strong end-to-end driving systems, such as UniAD or VAD, which are the standard baselines for trajectory prediction.

It’s also missing comparisons with other VLM-based methods, such as DriveVLM, OmniDrive, or ORION. In fact, ORION is a very close baseline, and based on the results here, this proposed method doesn’t outperform it.

Overall, the improvement seems marginal, and the paper doesn’t convincingly show that this method leads to any meaningful gains in driving performance. The motivation would be stronger if the authors could clearly demonstrate where this approach provides a unique advantage over existing systems.


Decision: Reject