FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Driving
Authors: Shuang Zeng
Xinyuan Chang
Mengwei Xie
Xinran Liu
Yifan Bai
Zheng Pan
Mu Xu
Xing Wei
Venue: NeurIPS 2025 spotlight ()
Keywords: Autonomous driving
Visual language action models
Chain of thought
Autoregressive image generation
Visual reasoning
Unified generation understanding
Primary area: deep_learning

Abstract: Vision–Language–Action (VLA) models are increasingly used for end-to-end driving due to their world knowledge and reasoning ability. Most prior work, however, inserts textual chains-of-thought (CoT) as intermediate steps tailored to the current scene. Such symbolic compressions can blur spatio-temporal relations and discard fine visual cues, creating a cross-modal gap between perception and planning.
We propose FSDrive, a visual spatio-temporal CoT framework that enables VLAs to think in images. The model first acts as a world model to generate a unified future frame that overlays coarse but physically-plausible priors—future lane dividers and 3D boxes—on the predicted future image. This unified frame serves as the visual CoT, capturing both spatial structure and temporal evolution. The same VLA then functions as an inverse-dynamics model, planning trajectories from current observations and the visual CoT.
To equip VLAs with image generation while preserving understanding, we introduce a unified pre-training paradigm that expands the vocabulary to include visual tokens and jointly optimizes VQA (for semantics) and future-frame prediction (for dynamics). A progressive easy-to-hard scheme first predicts lane/box priors to enforce physical constraints, then completes full future frames for fine details.
On nuScenes and NAVSIM, FSDrive improves trajectory accuracy and reduces collisions under both ST-P3 and UniAD metrics, and attains competitive FID for future-frame generation despite using lightweight autoregression. It also advances scene understanding on DriveLM. Together, these results indicate that visual CoT narrows the cross-modal gap and yields safer, more anticipatory planning.
Code is available at https://github.com/MIV-XJTU/FSDrive.

FuiirvSightDrive: Thinking Visually with Spatio-Temport CoT for Autonomous Driving
Shuang Zengl.2 1.2" Xinyuan Chang", Mengwei Xie', Xinran Liu',
Yifan B Zheng Pan', Mu Xu², Xing Wei"f
Xi'an an Jiaotong University Amap,, Alibaba Group 'DAMO Academy, Alibaba Group Lengshuangdstu.xjie xxjuu.ed.... en, weixing@mail xjtu. edu. en, fchangxinyuan exy, xiemengwei... xmw, tom. lbbabaaaabbbb com, (baiyifan.be byf, panzheng. pan, xumu wum..mmeaalbbbba-innncccmmmmm
Abstract
LangunnLL Action (VLA) models are increasingly used for end-to-end driving due to their world knowledge and reasoning ability. Most prior work, however, inserts textual chains-of-thought (CoT) as intermediate steps tailored the current scene. Such symbolic compressions can blur spatio-temporal relations and discard fine visual cues, creating a cross-modal gap between perception and planning. We propose FSDrive, visual spatio-temporal CoT framework that enables VLAs to think images, The model first acts as a world model to generate unified future frame that overlays coarse but pyssically-plasibbe priors-future future lane dividers and 3D boxes- on the predicted future image. This unified frame serves the visual CoT, capturing both spatial structure and temporal evolution. The same VLA then functions as an mnverse-dnnamen trajectories from current observations and the visual COT. To equip VLAs with image generation while preserving understanding, we introduce a unified pre- training paradigm that expands the vocabulary include visual tokens and jointly optimizes VQA (for semantics) and future-frame prediction (for dynamics). progressive easy-to-hard scheme first predicts lane/box priors enforce physical constraints, then completes full future frames for fine details. On nuScenes and NAVSIM, FSDrive improves trajectory accuracy and reduces collisions under both ST-P3 and UniAD metrics, and attains competitive FID for future- freefframeeeeee generation despite using lightweight autoregression. It also advances scene understanding on DriveLM. Together, these results indicate that visual CoT narrows the cross- modal gap and yields safer, more anticipatory planning. Code is available https://githin com Www.//IVVVVUU//DDD......
model. planning
Introduction
Recently, given the superior capabilities of multimodal large language models (MLLMs) in world knowledge, reasoning ability, and interpretability. they have been widely applied autonomous driv- (VLA) model, which leverages pre-trained vision-language model (VLM) to directly extract scene features from visual observations and language instructions, subsequently generating vehicle control commands (e.g., speed and trajectory). This paradigm not only simplifies system architecture and minimizes information loss, but also enables the utilization of the model's world knowledge analyze driving environments and reason about safe decisions in complex scenarios.
ing 22
4
87 31.
One promising direction is the end-to-end vision-language-ace
Work done during the internship at Amap, Alibaba Group. Conresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Diserete Text CoT
Planning
Insight
Guides
Info Atteatien Plesical Future
right,
Go Straight
Modalll Gag
Current Observation
Willarl lanight
Collitoo
Discrate Text CoT
Image- Test Col
There are pedntriass aboed.
Shift Left
Thlak Vinualyy
Current Observation
Possible Callision
Medel Incanderrncy Cor
Image-Text CuT
Dontlo-Temport Co'
Planning
Bnter
Tläak Visaally
Observation
Centoo-eemporal CoT
Comparison
Figure I: Comparison of different COT. Textual CoT expression provides insufficient information. The modalities between the image-text CoT are inconsistent. The proposed spatio-temporal CoT captures the temporal and spatial relationships in the future.
In the field of language, Chain-of-Thoughh (CoT) 67 50 15 52)
improves reasoning capabilities
and interpretability by encouraging step-by-step thinking. However, existing autonomous driving [27] typically incorporate discrete text CoT (e.g.. language descriptions targeting current scenarios
44
[4]
and bounding box coordinates) intermediate reasoning steps. This method is essentially highly abstract and symbolized compression of visual information, which may lead to ambiguous temporal and spatial relationships, loss of fine-grained information, and modality conversion gaps [46] 55, (72), as shown in the top of Figure For autonomous vehicles requiring deep physical-world interaction, should their thinking process more closely resemble simulation and imagination of world, rather than merely relying on logical deduction of language?
Inspired by the human driver's cognitive mechanism of directly constructing visual representations of future scenarios in the mind, rather than converting them into language descriptions for reasoning, we propose a more intuitive spatio-temporal CoT method as shown the bottom part of Figure This method avoids information loss during text abstraction and enables the model to think visually about trajectory planning. First, the VLA serves as a world model to generate unified image frame for predicting future world states: Inspired by visual prompting engineering [53] 81 that draws red circles on images to guide model attention and by VLIPP (78) first predicts future bounding boxes to intro- duce physical priors when generating future frames, we represent future world spatial relationships through future red lane dividers and 3D detection boxes on the predicted unified frames [80]. These coarse- searse-grained visual cues direct the model's 's attention toward drivable areas and critical objects in future scenes while enforcing physically plausible constraints. Meanwhile, the temporal relationships are represented by the ordinary future frame, where dynamic evolution visual content intuitively characterizes temporal progression and the inherent laws of scene development. Subsequently, the apatio-temporal CoT acts as an intermediate reasoning step. enabling the VLA to function as an inverse dynamics model for trajectory planning based current observations and future predictions. Compared to traditional discrete text CoT, and even image-text CoT methods [27] 91 41 in the middle of the Figure I our method unifies both future scene representations and perception outputs in image format, which more effectively conveys the temporal and spatial relationships. This eliminates semantic gaps caused by cross cross-modal conversions (e.g.. converting visual perceptions into textual descriptions for reasoning), establishing an end-to-end visual reasoning pipeline that enables direct visual causal inference by the model.
as shown
To endow VLAAs with image generation capabilities, we propose a pre-training paradigm that si- multaneously preserves the semantic understanding of existing MLLM and activates their visual

---

generation capacity. Specifically, for the semantic understanding preservation part, we follow pre- vious approaches [64 27 25 by incorporating visual question answering (VQA) tasks for current scene comprehension. For the activation of visual generation capabilities, we investigate the shared vocabulary space between image and text, directly unleashing the visual generation potential of existing MILMMs in the field of autonomous driving through minimal data (approximately 0.3% of without requiring complex model architecture modifications or redesigns. However, directly generating complete detailed future scenes may fail to adhere to physical laws 78 88]. Thus, we propose a progressive, easy-to-hard generation method. We leverage the world knowledge of VLAs to first infer drivable regions and key object positions in future scenarios, generating coarse- goansee future perception images (e.g., lane dividers and 3D detection) to constrain physical laws. Subsequently, full future frames generated under this constraint to supplement fine-grained details, enabling the model to think visually about accurate future prediction.
previous methods [70 73 24 35
Extensive experiments on trajectory planning, future frames generation, and scene understanding tasks demonstrate the effectiveness of pre-training paradigm and spatio-temporal CoT in FSDrive. FSDrive achieves road scene comprehension by establishing pixel-level embodied associations with the environment, rather than relying on human-designed abstract linguistic symbols, advancing autonomous driving towards visual reasoning. In summary, our main contributions are as follows:
We propose a spatio-temporal CoT reasoning method that allows the model to enhance trajectory planning by thinking visually from future temporal and spatial dimensions.
We propose a unified pre- pre-training paradigm for visual generation and understanding. Mean- while, we introduce progressive generation approach that evolves from imposing physical constraints to supplementing details.
We conduct comprehensive evaluations across trajectory planning, future frames generation, and scene understanding tasks, demonstrating the effectiveness of our FSDrive.
2 Related work
2.1 Unified multimodal understanding and generation
49 (88) have increasingly focused on unifying multimodal under- standing and visual generation within a single LLM. On one front, methods like Show-o (74), and VILA-U [73] employ VQ-VAE [61] to transform images into discrete tokens while training LLMs to predict them. However, these methods suffer from insufficient semantic information preservation, often leading to performance degradation in downstream understanding tasks. Alternative meth- 82 utilize ViT [12] based vision encoders (e.g., CLIP (5I) to encode images into continuous feature maps. Nevertheless, such methods typically depend on external diffusion models for image generation or use different training objectives (i.e. diffusion and autoregression) for the two tasks, further complicates the infrastructure design with overall lower efficiency. Moreover, the afore- mentioned methods usually require massive billion-scale datasets for extensive training from scratch, which results in prohibitively high computational costs when disseminating explorations in this form. In this work, we demonstrate that the visual generative capabilities of existing MLLMs can be directly activated through minimal training costs (approximately 0.3% of previous methods (70 58 42 without requiring sophisticated architectural designs.
Recent research efforts [38 70
ods 57
2.2 Vision-language models for autonomous driving
Given the superior capabilities of large language models (LLLMs) in world knowledge, reasoning, and interpretability, recent researches 2 33 39 85) increasingly integrate Vision-Language Models LLLMs with autonomous driving systems to address limitations in end-to-end approaches. DriveGPT4 [76] employs LLMs through iterative question-answ answering interactions to explain vehicle behaviors and predict control signals. Drive VLM [60] synergizes LLMs with end-to-end architectures, where LLMs predict low ow-frequency trajectories that are subsequently refined by the end-to-end model for final planning. Doe-1 [95] reformulates autonomous driving as a next-token prediction task using Guni's (37] multimodal generation capabilities, executing diverse tasks through multimodal token processing. EMMA 27 leverages Gemini's multimodal foundation by encoding all non-sensor inputs (navigation instructions, vehicle status) and outputs (trajectories, 3D positions) as natural language text, fully exploiting pre-trained LLMs' world knowledge. In this work, propose
3

---

spatio-tempont chain of thought (CoT) reasoning method that unifies the form of images, allowing the model think visually about trajectory planning.
2.3 World models for autonomous driving
World models [66| 45 90, 89
aim to infer ego status and dynamic environments from past obser- vations to enable accurate future prediction and planning. Current applications of world models in autonomous driving primarily focus on driving scenario generation [47] 16 32), planning [66] [1], and representation learning 45
For driving scenario generation, most prior works are built upon diffusion models, with the exception of GAIA-1 [18] which incorporates progressive next-token predictor and an additional diffusion image decoder. Recent DrivingGPT [5] leverages existing vision generation LLM LlamaGen [56] while simultaneously outputting predictions for future states and actions. However, such VQ-VAE based visual tokens lack semantic information, often leading performance degradation in downstream visual understanding tasks [74] 40 [59]. In this work, we propose to directly activate the visual generation capabilities of existing multimodal large language models, enabling VLMs to act world models and predict future frames.
79 M
3 Proposed method: FSDrive
The proposed FSDrive is illustrated in Figure Section 3.1 describes the preliminaries. Section 3.2 presents a unified visual generation and understanding pre-training paradigm and progressive generation method. Section 3.3 proposes spatio-temporal toiin-of-thoughe methods. Section 3.4 details the training strategy.
3.1 Preliminary
End-to-end trajectory planning. End-to-end autonomous driving directly generates future trajec- tory from sensor data, convertible to vehicle control actions like acceleration and steering [27]. given N surround-vies images I, = {4 .f.................. 17 timestep t, model M outputs a BEV trajectory W. = (w]. wf ......,,u _,],, where each waypoint wt = (rt, vi)- The process is formulated as:
W₁ = M(,, (Te Tego)),
opt (Teom, Tego) denotes optional navigation commands and ego status (e.g., velocity, acceleration).
Unified visual generation and understanding. Recent works 70 unify multimodal under- standing and vision generation in single LLM. While understanding aligns with standard LLMs. generation methods [38] (33 typically use VQ-VAE [61] to encode images into discrete tokens. First, the image tokenizer quantizes image pixels
#x x3 into discrete tokens q Qh W/p. is the downsampling factor, and qi,, represents the index of the
where h H/p, image codebook. These w tokens are arranged in raster order to train a Transformer [62] autoregressive model. During image generation, general language modeling (LM) objective is adopted to autoregressively predict the next token, maximizing the likelihood of each image token:
L Σ log Po(ygless)
where q: denotes the visual token and represents the LLM parameters. Finally, the VQ- VAE's detokenizer converts these image tokens back into image pixels.
3.2 Unified pre-training paradigm for visual generation and understanding
To enable unified pre-training MLLMs require visual generation capabilities. As described in Sec- tion existing methods (e.g. Lumina-mGP" (37], the visual generation LLM used by Doe-1 (95) typically employ VQ-VAE encode images into discrete tokens when extracting visual information. However, these tokens lack semantic information, which hurts downstream understanding perfor-
mance 77] 197]. Moreover, current methods 77 96] demand expensive training from scratch on massive billion-scale datasets without leveraging existing LLM knowledge.

---

Pre-training
Fine-tuning
Spatlo Tersper
What are bonn Mike wikkl??
Brate giett
Hig Lord Command:: Taniff
Vinalt
Ege Statun
Predec
lasn diiides.
Pbe ingerinneee.....
Trnjscuuy Pla
Protie 30 Leneiias bous.
(00.3.3333) [-L48
FSDrive fu
FSDrive fo
Prodkt the frams.
Progreatcc
There warkig Sicee Understending
Opiina kepx Clock
Generallen
Figure 2: Overview of FSDrive. Taking the currently surround images and task instructions as input, MLLM is trained in the form of next token prediction. MLLM predicts the future spatio-tempont CoT, and then generates trajectory based on the current observation and predicted future.
Our method is directly built upon any existing MLLM that employs ViT-based encoders to convert images into continuous features. We preserve the original MLLM architecture without altering any structural components to maintain compatibility with pretrained weights. The sole modification involves expanding the MLLM's vocabulary by incorporating image tokens of the VQ-VAE into the text codebook, thereby extending the vocabulary's scope from language space to a multimodal space encompassing both visual and textual modalities. This enhancement enables the MLLM to predict image tokens, which can then be converted to image pixels through an VQ-VAE's detokenizer.
Pre-training for visual understanding. bilities of the native MILM during the pre-training stage, as shown in the left part of Figure 2 we follow previous methods [64, 27 by using a VQA task, which is crucial for autonomous vehicles to analyze complex driving scenarios. Given image-text question-answer pair (1, L), where represents the surround-viy -veww images the current scene and denotes the instructional question, model M generates corresponding answer A:
effectively preserve the semantic understanding capa-
A= MI,
Pre-training for visual generation. Inspired by the world models in autonomous driving [30 77 that generate future frames to learn physical laws, after activating the visual generation capability, we also enable the VLA to predict future frames, thereby capturing the dynamic evolution of the world. Specifically, given an image-instructio pair (I, L), the model predicts the next visual token of the future front-view frame through autoregressive generation:
P(q,, ... w
II lew Polas 9es).
The predicted visual tokens are then converted back into image pixels by VQ-VAE's detokenizer. Since future frames naturally exist in video datasets without requiring any labeled data, this approach unlocks the potential to harness abundant video data for improving generation quality.
Progressive image generation. However, directly generating complete detailed future scenes may fail to adhere physical laws (78). Therefore, during pre-training stage, we propose progressive, easy-to-hard generation method, incorporating annotated data containing lane divider and 3D detec- tion. Before generating visual tokens future frames Q,, we leverage the world knowledge of VLA to first reason about visual tokens of lane dividers Qi, which serve as the skeleton of the road scene and define drivable areas to enforce static physical constraints. Subsequently, we reason about visual tokens of 3D bounding boxes Qa. representing motion patterns of key objects to impose dynamic physical constraints. This progressive method sequence explicitly guides the model to infer structural layouts and geometric details of future scenes while enforcing physical laws. By leveraging these intermediate visual reasoning steps as context, the model learns to think visually about the dynamic evolution of scenes, ultimately enabling accurate future prediction:
5

---

p(2, Qr. Qa) -IIM- gen Qi Qa).
(5)
3.3 Think visually with spatio-temporal CoT
Autonomous driving planning requires not only understanding the current scene but also envisioning potential future developments to achieve forward-looking comprehension. This thinking process should resemble physical world simulation and imagination rather than purely text symbolic logical deduction. Since our model has already learned physical constraints through the progressive genera- tion during pre- wre-triinngg and considering efficiency, we no longer separately generate lane dividers, 3D detection, and future frames, but instead integrate all these results into a single unified frame. As shown in the right part of Figure 2 here, VLA serves frame predicting the future world state: Inspired by visual prompting engineering [53] that draws red circles on images to guide model attention and by VLIPP (78 first predicts future bounding boxes to introduce physical priors when generating future frames, we represent future world spatial relationships through future red lane dividers and 3D detection boxes on the predicted unified frames. These coarse-graine graseeeg visual cues direct the model's 's attention toward drivable areas and critical objects in future scenes while enforcing physically plausible constraints. Meanwhile, the temporal relationships are represented by the ordinary future frame, where the dynamic evolution of visual content intuitively characterizes temporal progression and the inherent laws of scene development. Subsequently, spatio-temporal CoT QCoT serves as an intermediate reasoning step, allowing the VLA to function an inverse dynamics model that plans trajectory based current observations and future predictions:
world model to generate a unified image
P(W. I, QCoT, opt Tego)) II Po(ws] Wei; Ie, opl.ooooooTTaaaooooooooooo opl T Togo))-
(6)
3.4 Training strategy
Our FSDrive can be initialized from any existing MLLM (c.g., Qwen2-VL, LLaVA), avoiding training from scratch and saving significant costs. During training, fully fine-tune the LLM parameters while freezing all encoders. The training process is divided into two stages:
Stage 1: Unified pre-training. Our objective is to preserve understanding capabilities of MLLMs through VQA tasks and activate their visual generation capabilities to predict future frames. VQA task data originates from Omniirive-nuSeeness [64] We incorporate large volume of unlabeled image data from nuScenes I for future frame prediction. To implement progressive easy-to-hard COT, we integrate nuScenes annotated data teach the model predicting image- formatto future lane dividers and 3D detection. Finally, we add future frame prediction with CoT datas containing intermediate reasoning steps. All the above understanding and generation tasks are trained together.
Stage 2: Supervised fine-tuning. We focus on autonomous driving scene understanding and trajectory planning. Following OmniDrive [64], scene understanding utilizes DriveL.'s GVQA [54] dataset. For trajectory planning, we follow VAD [29 21 using nuScenes, where our spatio-temporal CoT integrates the holistic future scene, explicit lane dividers, and 3D detection results into single future frame as intermediate reasoning steps. We train these tasks simultancously using a single model, enabling task-specific predictions during inference through different task prompts.
4 Experiments
4.1 Experimental settings
Datasets.
we evaluate trajectory planning and future frames generation on the nuScenes [1] The nuScenes contains 1,000 scenes of approximately 20 seconds each captured by 32-beam LiDAR and six cameras providing 360-degree field of view. Specifically, The dataset provides 28,130 (train), 6,019 (val), and 193,082 (unannotated) samples. Additionally, we conducted experiments on NAVSIM 10], a realistic scenario dataset designed for real-world planning. This dataset aims to highlight challenging driving scenarios involving dynamic changes in driving intent, while deliberately excluding simple situations such as static scenes or constant-speed driving.
Following the previous methods 29 L5

---

Table 1: End-to-end trajectory planning experiments on nuScenes [. We evaluated the L2 and collision metrics based on the distinct computational methodologies of ST-P3 [19 and UniAD pT respectively. * indicates that the ego status is additionally used. VAD 29 and UniAD 21 results are derived from BEV-Planner
34 while the remaining results are sourced from their respective papers.
ST-P3 metrics
UniAD metries
Method
L2 (m) |
Collision (%)
L2 (m)
Collision (%)
LLM
18
2s
3s
Avg.
1s
2/
35
Arg. 18
2s
3s Avg.
1s
2s
3s
Avg.
Non-Autoregressiv methods
ST-P3* [BCC)2)]
19
1.33 2.11
2.90
2.11
0.23 0.62 1.27 0.71
VAD
0.69
1.22
1.25
0.06
0.68 2.52
1.09
VAD* [CEV21)
29
0.17 0.34
0.60
0.37
0.04
0.27
0.67
0.33
UniAD [CVPR23]
21
0.59 1.01
1.03
0.16
0.51
1.64
0.77
UniAD* [CV95223]
21
0.20
0.42
0.75
11.46
0.02
0.25
0184
0.37
BEV-Planner [CVPR224
34
0.30
0.52
0.83
0.55
0.10
0.37
1,30
0.59
BEV-Planner* (CVPR24
34
0.16
0.32
0.57
0.35
0.00
0.29 0.73 0.34
PreWorld
32
0.49 1.22
2.32
34
0.57
2.65 1.14
Autoregressive methods
ELM [BCCV24
98
0.34 1.23
2.57
1.38
0.12
0..12.055
0.50
2.36
0.99
BLIP2-2,7B
FelD (сVAеа]]
66
1.27 0.53 0.94
0.58
0.00
0.04
0..5 0.19
I.L.VA-78
OceWorld (EC0V24)
0.39
118
0.67
0.52
1.27 2.41
1.40
0.12 0.40
2.08
0.87
GPT3-like
Doc-1 [uno22
95
0.37
0.67
1.07 0.70
0.02
0.14
0.47
0.21
0.50
1.18
2.11
1.26
0.04
0.37
1.19
0.53
Lumina-mGPT-7B
RDA- D VECC
26
0.17
0.37
0.69 0.40
0.01
0.05
0.26
0.10
0.23
0.73
1.54
0.80
0.00
0.13
0.83
0.32
LLaVA-7B
EMMA [anix24]
0.24 0.54 0.32
Gemini 1-1.8B
OmniDrive (CVPR25)
64
0.80 1.32 0184
0.04
0146
2.32 0.94
ILaVA-7B 78
OmniDrive (CVP925)
64
0.14 0.29 0.55 0.33 0.00
0.13
0.78 0.30
LLaVA 7B
FSDrive (ours)
0.28 0.52 0.80 0.53
0.06
0.13 0.32 0.17
0.40
0.89
1.60
0.96
0.07
0.12
1.02
0.40
Qwen2- Owen2-VL-2B VL-BB
FSDrive* (ours)
0.14 0.25 0,46 0.28 0.03
0.06
0.21 0.10
0.18
0.39
0.77
0.45
0,00
0.06
0.42
0.16
Qwen2-VL-22
FSDrive (ours)
0.29 0.57 0.94 0.60 0.04
0. 0.38
19
0.36
1001
1.90
1.09
0.08
0.34
1.11
0.51
LLaVA-7B -B
PSDrive (ours)
0.13 0.28 0.52 0.03
0.07
0.24
0.12
0.22
0.51
0.94
0.56
0.0
02
0.07
0.53
0.21
LLaVA-7B
Table 2: Performance comparison NAVSIM navtest using closed-loop metrics. All the methods only use images as input and do not use lidar.
Method
NC
DAC
TTC
Comf.
EP
PDMS
VADv2 [arXiv24]
97.2
89.1
91.6
100
76.0
80.9
UniAD [CVPR23]
21
97.8
91.9
92.9
100
78.8
83.4
DiffusionDrive- Cam [CVPR25]
36
97.8
92.2
92.6
99.9
78.9
83.6
LTF [TPAMI23] 0
97.4
92.8
92.4
100
79.0
83.8
PARA-Drive [CVPR24] 69
97.9
92.4
93.0
99.8
79.3
84.0
LAW [ICLR25]
33
96.4
95.4
88.7
99.9
81.7
84.6
FSDrive (ours)
98.2
93.8
93.3
99.9
80.1
85.1
Following the previous methods 7 64
we evaluate scene understanding DriveI.M [54]. This dataset features keyframe descriptions paired with QA annotations covering full-stack autonomous driving (perception, prediction, planning) offering comprehensive language support for development.
Metrics. We evaluate trajectory planning using L2 displacement error and collision rate following
previous methods 21
29
19
Notably, UniAD 21 computes 1.2 metrics and collision rate at
each timestep, whereas ST-P3
and VAD 29 considers the average of all previous time-steps. For a fair comparison, we adopted these two different calculation methods. Following existing methods [65 77 71). we report Fréchet Inception Distance (FID) [17] to measure the future frames generation quality. DriveLM GVQA (54] metrics include language metrics like BLEU, ROUGE and CIDEr for text generation, the ChatGPT Score for open-ended Q&A and accuracy for multiple- choice questions. For NAVSIM [10]. we adopt the official metrics for evaluation, especially PDMS.
19
Implementation details. We initialize our model with Qwen2- VL-2B 63 and pre-train it for 32 epochs to enable visual generation while preserving semantic understanding. During fine-tuning (12 epochs on 8 NVIDIA RTX A6000), we use 10-1 learning rate and batch size of 16. We expand the visual codebook of MoVQGAN [92] the vocabulary of the large language model and use its detokenizer convert the visual tokens predicted by the large language model the pixel space.
7

---

Table Future frames generation results on the nuScenes dataset.
DriveGAN
DriveDreamer
Drive-WM [CVPR24 (68)
GenAD
GEM
Doe-1 [anxiv24
Method
FSDrive
(CVPR2] k0]
[ECCV24 [65]
[CVPR24 M
[CVPR25 16
3S
Type
GAN 256x256
Diffusion
Diffusion
Diffusion
Diffusion
Autoregressive 384x672
Autoregressive
Resolution
128x 192
192x384
256x448
576x 1024
128x 192
FID
73.4
52.6
15.8
15.4
10.5
15.9
10.1
Table Results on DriveLM [54] GVQA benchmark.
Method
Accurayy ChatGPT BLEUL ROUGE_ CIDEr Match f FFinal Score
DriveLM baseline
[54
0.00
0.65
0,05
0.08
0.10
0.28
0.32
Cube-LLM 77
0.39
0.89
0.16
0.20
0.31
0.39
0,50
TrackingMectsl.NN
28
0.60
0.58
0.72
0.72
0.36
0.52
SimpleL.MMAA 2
0.66
0.57
0.76
0.73
0.15
0.35
0,53
OmniDrive [64]
0.70
0.65
0.52
0.73
0.13
0.37
0.56
FSDrive (ours)
0.72
0.63
0.76
0.74
0.17
0.39
0.57
Table 5: Ablation results of pre- pre-training.
Future frames
Future 3D detection
Future lane divider
L,2 (m) 1
Collision (%)
VQA
2s
Avg.
2s
Avg.
x
0.45
1.09
2.12
1.22
0.43
0.67
X
X
0.46
1.07
2,04
1.19
0.12
0.42
1.42
0.65
X
V
X
X
0.39
0.96
1.71
1.02
0.10
0.38
1.32
0.60
V
0.46
1.06
1.99
1.17
0.10
0.37
1.35
0.61
X
0.42
0.97
1.80
1.06
0.13
0.41
1.40
0.65
0.39
0.91
1.63
0.98
0.36
1.33
0.58
4.2 Main results
End-to-End trajectory planning. We present trajectory planning performance on nuScenes fol- lowing previous methods 29 21 in Table T When using ego status, FSDrive surpasses previous SOTA methods using ego status ST-P3 and UniAD metrics. However, following BEV-Planner [34] findings about ego-status's performance boost, prioritize non-ego-status evaluations. Compared to non-autoregressive (e.g., UniAD) and autoregressive methods (e.g., OmniDrive), FSDrive demon- strates superior effectiveness. Notably, FSDrive outperforms Doe-1 1 (95] which also enables vision generation (L2: 0.53 vs. 0.70 and 0.96 vs. 1.26; collision: 0.19 vs. 0.21 and 0.40 vs. 0.53), indicating limitations in their VQ-VAE-basee discrete visual features for understanding. For fair comparison, we also used LLaVA VA like methods [64] 26 86 75). Under the corresponding settings, FSDrive still has excellent competitiveness, indicating that FSDrive can be widely applied any existing MLLM.
Results NAVSIM. Table 2 shows the evaluation results for NAVSIM [10] All approaches rely exclusively on camera input, with no lidar data being used, Achieving a PDMS score of 85.1. FSDrive outperforms prior camera-only methods like LAW 33 and Diffusion (36], thus
showcasing its efficacy in the pseudo closed-loop setting.
Evaluation generation results. Although we generate future frames as CoT for trajectory planning, we still validate visual quality via FID in Table To enable rapid generation for real-time driving, we generate frames at 128 x 192 resolution. Our autoregressive FSDrive achieves competitive performance against specialized diffusion models. Compared Doe-1 [95] which employs the vision generation MLLM Lumina-mGPP 7B [37] FSDrive 2B maintains superior advantages, indicating that the visual generation capabilities of MILM can be effectively unlocked even with minimal data.
FSDrive's scene understanding was evaluated on DriveLM in Table achieving 0.57 and outperforming recent methods like Cube-I.LM 7 and OmniDrive (44. This highlights the effectivvnees of FSDrive pre-training paradigm for generation and understanding.
Results on DriveLM dataset.

---

Wrong Instruction
(
Stop
Go Straight
Turn Left
Turn Right
Trajectory
(WI)
CoT
(Correct
Instruction)
Trajectory (WI Our CoT)
Figure 3: Qualitative analysis of our COT. The red trajectory is the prediction and the green is the GT.
Table 6: Ablation results of different COT.
L.2 (m) 4
Collision (%) J
Type
2s
3s
Avg.
2s
3s
Avg.
None
0.39
0.91
1.63
0.98
0.09
0.36
1.33
0.58
Text CoT
0.39
0.92
1.61
0.97
0.10
0.29
1.21
0.53
Image-text CoT
0.38
0.90
1.65
0.98
0.09
0.25
1.15
0.50
CoT
0.40
0.89
1.60
0.96
0.07
0.12
1.02
0.40
4.3 Ablation study
In this section, unless otherwise specified, evaluate the computing metrics of UniAD on the Qwen2-VL-2BB model (63] and do not use the ego status.
21 based
Qualitative analysis. We evaluate CoT's effectiveness in Figure Without spatial-temporal CoT, erroneous navigation imputs caused significant trajectory deviations and potential collisions. Use correct instruction when reasoning COT. while still employing wrong instruction for planning. However, FSDrive mitigated instruction errors through observation-bast trajectory planning and future prediction, demonstrating its inverse dynamics modeling capability.
Table Ablation experiments of future frames generation.
(a) Ablations of pre-training data volume.
(b) Ablations of progressive method.
Data volume
FID.
Progressive Method
FIDJ
None
29.4
100k
16.2
12.7
200k
12.7
10.1
Pre-training ablation study. The impact of pre-training trajectory planning is summarized in Table 5 Pure VQA tasks show negligible effects. Future frame generation pre-training improves 1.2 by 16.4% and collisions by 15.8%, validating world-model--aasee prediction's ' effectiveness. in capturing physical dynamics. 3D detection and lane divider pre-training yield moderate gains in L2/collision metrics respectively. The combined understanding and generation pre- pre-training achieves better performance, demonstrating our unified paradigm's 's capacity to enhance scene representation and effectively learn physical laws, thereby strengthening spatial understanding capabilities.
( marginal changes but notable collision rate improvements. Pure text CoT (8.6% improvement) exhibits limited repre- sentation capability due to unimodal perception. Compared to text CoT, image-text CoT (combining future frames with textual perception) shows insignificant gains due to the inconsistent modalities
Results of different CОT. Ablation studies on CoT variants

---

between CoT's The proposed
CoT achieves 31% improvement, demonstrating that unified image-based reasoning effectively identifies future collision risks.
Ablation study generation results. We conduct ablation studies on future frames generation in Table 7 Table 77 shows that larger pre-training datasets improve MLLM's visual generation capability. Despite being much smaller (200K vs. 100M in previous work 770,, our data achieves more robust visual generation. Scaling datasets may further enhance performance. Table 7b confirms progressive method improves autoregressive image generation.
5 Conclusion
This paper proposes FSDrive, an autonomous driving framework based on spatio-temporal CoT that enables VLAs think visually. By unifying future scene generation and perception results through intermediate image-form reasoning steps, our FSDrive eliminates the semantic gap caused by cross-modal conversions and establishes an end-to-end visual reasoning pipeline. The VIA serves dual roles: world model that predicts future image frames with lane divider and 3D detection, and as an inverse dynamics model that plans trajectory based on both current observations and future predictions. To enable visual generation in VLAs, we present pretraining paradigm that unifies visual generation and understanding, along with a progressive easy-to-hard visual CoT to enhance autoregressive image generation. Extensive experimental results demonstrate the effectiveness of the proposed FSDrive method, advancing autonomous driving towards visual reasoning.
Limitations and broader impacts. Though autonomous driving requires surrounding environ- mental awareness, considering real-time efficiency, we currently only generate future frames for the front-view. Future work can attempt to generate Surround views achieve safer autonomous driving. Moreover, more robust visual quality can be achieved in future work through the use of larger training datasets and more advanced unified paradigm that integrates generation and understanding. In terms of impact, the ethical challenges posed by LLMs extend to autonomous driving. Advances in technology and regulation will drive development of safer, more efficient systems.
Acknowledgments. This work was support by the National Natural Science Foundation of China No. 62572385, the Fundamental Research Funds for the Central Universities No. xxj032023020, and CAAI-CANN Open Fund, developed on Openl Community.
References
[1] Н. Caesar, V. Bankiti, A. H. Lang, S. Vora, V. E. Liong, Q. Xu, A. Krishnan, Y. Pan, G. Baldan, and
O. Beijbom. nuscenes: A multimodal dataset for autonomous driving. CVPR, 2020.
[2] X. Chang, M. Xue, X. Liu, Z. Pan, and X. Wei. Driving by the rules: A benchmark for integrating traffic sign regulations into vectorized hd map. CVPR, 2025.
[3] S. Chen, B. Jiang, Gao, B. Liao, Q Xu, Q Zhang, C. Huang, W. Liu, and X. Wang. Vadv2: End-to-end vectorized autonomous driving via probabilistic planning. arXiv preprint arXiv:2402. 13243, 2024.
[4] Y. Chen and R. Greer. Technical report for argoverse2 scenario mining challenges on iterative error correction and spatially-aware prompting. arXiv preprint arXiv:2506.1 //24, 2025.
Y. Chen, Y.-Q. Wang, and Z. Zhang. Drivinggpt: Unifying driving world modeling and planning with multi-modal autoregressive transformers. ICCV, 2025.
[6] K. Chitta, A. Prakash, B. Jaeger, 7. Yu, K. Renz, and A. Geiger. Transfuser: Imitation with transformer-
based sensor fusion for autonomous driving. TPAMI, 2023.
[7] J. H. Cho, B. Ivanovic, Y. Cao, F. Schmerling, Y. Wang, X. Weng, B. Li, Y. You, P. Krähenbühl, Y. Wang, et al. Language-image models with 3d understanding. ICLR, 2025.
[8] M. Dai, S. Liu, Z. Zhao, J. Gao, H. Sun, and X. Li. Secure tug-of-war (sectow); Iterative defense-attack training with reinforcement learning for multimodal model security. arXiv preprint arXiv:2507.2203:: 2025.
[9] M. Dai, J. Sun, Z. Zhao, S. Liu, R. Li, J. Gao, and X. Li. From captions to rewards (carevl); Leveraging large language model experts for enhanced reward modeling in large vision- vision-language models. arXiv preprint arXi::2503.0 06260, 2025.
[10] D. Dauner, M. Hallgarten, T. Li, X. Weng. Z. Huang. Z. Yang. H. Li, Gilitschenski, B. Ivanovic, M. Pavone, A. Geiger, and K. Chitta. Navsim: Data driven non-reactive autonomous vehicle simulation and benchmarking. In NewrIPS, 2024.
10

---

[11] R. Dong, C. Han, Y. Peng. Z. Qi. Z. Ge, J. Yang. L. Zhao, J. Sun, H. Zhou, H. Wei, X. Kong, X. Zhang. K. Ma, and L. Yi. DM: Synergistic multimodal comprehension and creation. In ICLR. 2024. [12] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenbom, X. Zhai, T. Unterthiner, M. Dehghani, M. Min- derer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby. An image is worth 16x16 words: Transformers
for image recognition at scale. ICLR, 2021.
[13] S. Gao, J. Yang, L. Chen, K. Chitta, Y. Qiu, A. Geiger, J. Zhang, and H. Li. Vista: A generalizable driving world model with high fidelity and versatile controllability. In NeurlPS, 2024.
[14] Y. Gao, C. Li, Z. You, Liu, Z. Li, P. Chen, Q Chen, Z. Tang, L.. Wang, P. Yang, et al. Openfly: A comprehensive platform for acrial vision-language navigation. CoRR, 2025.
[15] J. Guo, Z. Li, Wu, Q. Wang. Y. Li, I.. Zhang. hai zhao, and Y. Yang. Tom: Leveraging trec-oriented
mapreduce for long-contexs reasoning in large language models. In EMNLP, 2025.
[16] M. Hassan, S. Stapf. A. Rahimi, P. M. B. Rezende. Y. Haghighi, D. Brüggemann, Katircioglu. L. Zhang,
X. Chen, S. Saha, M. Canmici, E. Aljalbout, B. Ye, X. Wang, A. Davtyan, M. Salzmann, D. Scaramuzza, M. Pollcfeys, P. Favaro, and A. Alahi. Gem: A generalizable ego-vision multimodal world model for fine-grained ego-motion, object dynamics, and scene composition control. CVPR, 2025.
[17] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. NeurlPS, 2017.
[18] A. Hu, L. Russell, H. Yeo, Z. Murez. G. Fedoseev, A. Kendall, J. Shotton, and G. Corrado. Gaia-1: A generative world model for autonomous driving. arXiv preprint arXiv:2309. / 70880, 2023.
[19] S. Hu, L. Chen, P. Wu, H. Li. J. Yan, and D. Tao. St-p3: End-to-end vision-based autonomous driving via spatial-temporal feature learning. In ECCV, 2022.
[20] Y. Hu. Q. Li, D. Zhang, J. Yan. and Y. Chen. Context-alignment: Activating and enhancing LLMs capabilities in time series. In ICLR, 2025.
[21] Y. Hu, J. Yang, I.. Chen, K. Li, C. Sima, X. Zhu, S. Chai, S. Du, T. Lin, W. Wang. I., Lu, X. Jia, Q. Liu,
J. Dai, Y. Qiao, and H. Li. Planning-orian oriented autonomous driving. In CVPR, 2023.
[22] J. Huang, M. Yan, S. Chen, Y. Huang. and S. Chen. Magicfight: Personalized martial arts combat video
generation. In Proceedings of the 32nd ACM International Conference on Multinedia, 2024.
[23] J. Huang. G. Zhang, 7. Jie, S. Jiao, Y. Qian, I. Chen, Y. Wei, and L., Ma. M4v: Multi-modal mamba for text-to-video generation. arXiv preprint arXiv:2506. 109 2025.
[24] Z. Huang, H. Qian. Z. Cai, A. Wang. J. Wang, and F. Xiong. Intelligent recognition method for urban road grid patterns by fusing mesh and road features. International Journal of Digital Earth, 2024.
[25] Z. Huang. H. Qian, Z. Cai, X. Wang. L. Xie, and X. Niu. An intelligent multilane roadway recognition
method based on tagging Cartography and Geographic Information Science, 2025.
[26] Z. Huang. Tang. S. Chen, S. Lin, and e. a. Jie. Making large language models better planners with
reasoning-decte decision alignment. ECCV, 2024.
[27] J.JJ. Hwang, R. Xu, H. Lin, W.-C. Hung, J. Ji, K. Choi, D. Huang, T. He, P. Covington, B. Sapp, J. Guo,
D. Anguelov, and M. Tan. Emma: End-to-end multimodal model for autonomous driving. TMLR, 2025.
[28] A. Ishaq. J. Lahoud, F. S. Khan, S. Khan, H. Cholakkal, and R. M. Anwer Tracking meets large multimodal models for driving scenario understanding. ArXiv preprint arXiv:2503. 14498, 2025.
[29] B. Jiang, S. Chen, Q. Xu, B. Liao, J. Chen, H. Zhou, Q Zhang, W. Liu, C. Huang, and X. Wang. Vad: Vectorized scene representation for efficient autonomous driving. ICCV, 2023.
[30] S. W. Kim, J. Philion, A. Torralba, and S. Fidler. Drivegan:
Towards a controllable high-quality neural
simulation. In CVPR, 2021.
[31] B. Li, Y. Wang, J. Mao, B. Ivanovic, S. Veer, K. Leung, and M. Pavone. Driving everywhere with large
language model policy adaptation. In CVPR, 2024.
[32] x. Li, P. Li, Y. Zheng, W. Sun, Y. Wang, and Y. Chen. Semi-supervised vision-centric 3d occupancy world model for autonomous driving. ICLR, 2025.
[33] Y. Li, L. Fan, J. He. Y. Wang, Y. Chen, Z. Zhang, and T. Tan. Enhancing end-to-end autonomous driving
with latent world model. ICLR, 2025.
[34] Z. Li, Z. Yu, S. Lan, J. Li. J. Kautz, T. Lu, and J. M. Álvarez. Is ego status all you need for open-loop
end-to-end autonomous driving? CVPR, 2024.
[35] S. Liang, X. Chang, C. Wu, H. Yan, Y. Bai, X. Liu, H. Zhang, Y. Yuan, S. Zeng, M. Xu, et al. Persistent autoregressive mapping with traffic rules for autonomous driving. arXiv preprint arXiv:2509.2276 2025. [36] B. Liao, S. Chen, H. Yin, B. Jiang, C. Wang, S. Yan, X. Zhang, X. Li, Y. Zhang, Q. Zhang, and X. Wang. Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving. CVPR, 2025.
[37] D. Liu, S. Zhao, I.. Zhuo, W. Lin, Y. Qiao, H. Li, and P. Gao. Lumina-mgpt: Illuminate flexible photoreal- istic text-to-image generation with multimodal generative pretraining. arXiv preprint arXiv:2408.00 026577, 2024.
[38] H. Liu, W. Yan, M. Zaharia, and P. Abbeel. World model on million-length video and language with ringattention. ICIR, 2025.
[39] J. Liu, F. Shang. Y. Liu, H. Liu, Y. Li, and Y. Gong. Fedbegd: Communication accelerated block
coordinate gradient descent for federated learning. ACM MM, 2024.
11

---

[40] W. Liu. J. Chen, K. Ji, L. Zhou, W. Chen, and B. Wang. Rag-instruct: Boosting Ilms with diverse retrieval-augmennter instructions. EMNLP, 2025.
[41] W. Liu, J. Xu, F. Yu, Y. Lin, K. Ji. W. Chen, Y. Xu, Y. Wang, L. Shang, and B. Wang. Qifi, question-free fine-tuning for adaptive reasoning. NeurlPS, 2025.
[42] Lu, Y. Tong. and Z. Ye. Dammfud: Domain-aware multimodal multi-view fake news detection. In Proceedings of the AAAI Conference on Artificial Intelligence, 2025.
[43] Y. Ma, Y. Cao,, Sun, M. Pavone, and C. Xiao. Dolphins: Multimodal language model for driving. ECCV,
2024.
[44] J. Mao, J. Ye, Y. Qian, M. Pavone, and Y. Wang. A language agent for autonomous driving. COLM, 2024.
[45] C. Min, D. Zhao, I.. Xiao, J. Zhao, X. Xu, Zhu, L., Jin, J. Li, Y. Guo, J. Xing. I.. Jing. Y. Nie, and B. Dai.
Driveworld: 4d pre-trained scene understanding via world models for autonomous driving. CVPR, 2024.
[46] S. Motamed, I., Culp, K. Swersky, P. Jaini, and R. Geirhos. Do generative video models understand
physical principles? arXiv preprint arXiv:2501.000 1.500,0938, 2025.
[47] J. Ni, Y. Guo, Y. Liu, R. Chen, L. Lu, and Z. Wu. Maskgwm: A generalizable driving world model with video mask reconstruction. CVPR, 2025.
[48] Y. Qian, X. Li. J. Zhang. X. Meng, Y. Li, H. Ding, and M. Wang. A diffusion--gaan -gan framework for spatio-temporal speed imputation and trajectory reconstruction. IEEE T-/TS, 2025.
[49] K. Qiu. Z. Gao, Z. Zhou, M. Sun, and Y. Guo. Noise-consistent diamese-diffusee for medical image synthesis and segmentation. In CVPR, 2025.
[50] D. Qu, H. Song. Q Chen, Yao, X. Ye, Y. Ding, Z. Wang, J. Gu. Zhao, D. Wang, et al. Spatialvia: model. RSS, 2025.
Exploring spatial representations for visual-language-actinn
[51] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, Clark, G. Krueger, and I. Sutskever. Learning transferable visual models from natural language supervision. In JCML, 2021.
[52] A. Sarkar, M. Y. L. Idris, and Z. Yu. Reasoning in computer vision: Taxonomy, models, tasks, and methodologies. arXiv preprint arXiv:2508. 10523, 2025.
[53] A. Shtedritski, C. Rupprecht, and A. Vedaldi. What does clip know about a red circle? visual prompt engineering for vims. ICCV, 2023.
[54] C. Sima, K. Renz, K. Chitta, L. Chen, H. Zhang, C. Xie, P. Luo, A. Geiger, and H. Li. Drivelm: Driving with graph visual question answering. ECCV, 2024.
[55] H. Song, D. Qu. Y. Yao. Q Chen, Q. Lv, Y. Tang, M. Shi, G. Ren, M. Yao, Zhao, et al. Hume: model. arXiv preprint arXiv:2505.21432, 2025. [56] P. Sun, Y. Jiang, S. Chen, S. Zhang, B. Peng, P. Luo, and Z. Yuan. Autoregressive model beats diffusion: Llama for scalable image generation, arXiv preprint arXiv:2406.000 06525, 2024.
Introducing system-2 thinking in lisual-language-actii
[57] Q. Sun, Q. Yu, Y. Cui, F. Zhang, X. Zhang, Y. Wang, HI Gao, J. Liu, T. Huang, and X. Wang. Generative pretraining in multimodality. ICLR, 2024.
[58] S. Sun, W. Yu, Y. Ren, W. Du, L.. Liu, X. Zhang, Y. Hu, and C. Ma. Gdiffretro: Retrosynthesis prediction
with dual graph enhanced molecular representation and diffusion generation. AAAI, 2025.
[59] W. Tan, D. Chen, J. Xue, Z. Wang, and Chen. teaching-inspires integrated prompting framework:
novel approach for enhancing reasoning in large language models. In COLING: Industry Track, 2025.
[60] X. Tian, J. Gu, B. Li, Y. Liu, Z. Zhao, Y. Wang, K. Zhan, P. Jia, X. Lang, and H. Zhao. Drivevlm: The
convergence of autonomous driving and large vision-language models. CoRL, 2024.
[61] A. van den Oord, O. Vinyals, and K. Kavukcuoglu. Neural discrete representation learning. In NeurlPS,
2017.
[62] A. Vaswani, N. M. Shazeer, N. Parmar, J. Uszkorcit, L. Jones, A. N. Gomez, L. Kaiser, and I, Polosukhin. Attention is all you need. In NeurlPS, 2017.
[63] P. Wang, S. Bai, S. Tan, S. Wang, Z. Fan, J. Bai, K. Chen, X. Liu, J. Wang, W. Ge, Y. Fan, K. Dang, M. Du, X. Ren, R. Men, D. Liu, C. Zhou, J. Zhou, and J. Lin. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprini arXiv:2409.122 12191, 2024.
[64] S. Wang, Z. Yu, X. Jiang, S. Lan, M. Shi, N. Chang, J. Kautz, Y. Li, and J. M. Álvarez. Omnidrive: A bolistic Imm-agent framework for autonomous driving with 3d perception, reasoning and planning. CVPR, 2025.
[65] x. Wang, 7. Thu, G. Huang, X. Chen, J. Zhu, and J. Lu. Drivedreamer: Towards real-world-driven world models for autonomous driving. ECCV, 2024.
[66] Y. Wang. J. Hc, I.. Fan, H. Li, Y. Chen, and Z. Zhang. Driving into the future: Multiview visual forecasting
and planning with world model for autonomous driving. CVPR, 2024.
[67] Wei, X. Wang, D. Schuurmans, M. Bosma, E. H. Chi, F. Xia, Q. Le, and D. Zhou. Chain thought
prompting clicits reasoning in large language models, NeurlPS, 2022.
[68] Q Wei, P. Dai, W. Li, B. Liu, and X. Wu. Copeft: Fast adaptation framework for multi-agent collaborative
perception with parameter-effiiient fine-tuning. In AAAI, 2025.
[69] X. Weng, B. Ivanovic, Y. Wang, Y. Wang, and M. Pavone. Para-drive: Parallelized architecture for real-time autonomous driving. In CVPR, 2024.
12

---

[70] C. Wu, X. Chen. Z. Wu, Y. Ma. X. Liu. Z. Pan, W. Liu. Z. Xie, X. Yu, C. Ruan, et al. Janus: Decoupling visual encoding for unified multimodal understanding and generation. CVPR, 2024.
[71] C. Wu, H. Huang. L. Zhang, J. Chen, Y. Tong. and M. Zhou. Towards automated 3d evaluation of water
leakage on tunnel face via improved gan and self-attention di model. Tunn Undergr Space Technol, 2023.
[72] Wu. H. Li. X. Zhang, X. Liu, Huang, Luo, Y. Zhang, Z. Li, R. Chu, Y. Yang. and S. Li. Teaching
your models to understand code via focal preference alignment. In EMNLP, 2025.
[73] Y. Wu, Z. Zhang. J. Chen, H. Tang, D. Li, Y. Fang. I.. Zhu, F. Xie, H. Yin, L.. Yi, al. Vila-u: a unified foundation model integrating visual understanding and generation. ICLR, 2025.
[74] J. Xie, W. Mao, Z. Bai, D. J. 7hang, W. Wang. K. Q. Lin, Y. Gu, 7. Chen, 7. Yang. and M. 7. Shou.
Show-o: One single transformer to unify multimodal understanding and generation. ICLR, 2025.
[75] M. Xie, S. 7eng, X. Chang, X. Liu, Z. Pan, M. Xu, and X. Wei. Seggrowgraph: Learning lane topology as
chain of graph expansions. In /CCV, 2025.
[76] Z. Xu, Zhang, F. Xie, Z. Zhan, Guo, K.-Y. K. Wong. 7. Li, and H. Zhao. Drivegpt4: Interpretable
end-to-end autonomous driving via large language model. IEFE Robotics and Automation Letters, 2024.
[77] J. Yang. Gao, Y. Qiu, I.. Chen, T. Li, B. Dai, K. Chitta, Wu, J. Zeng, P. Luo, Zhang, A. Geiger,
Y. Qian, and H. Li. Generalized predictive model for autonomous driving. In CVPR, 2024.
[78] X. Yang, B. Li, Y. Zhang, 7. Yin, L. Bai, I. Ma, Z. Wang, J. Cai, T.-T. Wong. H. Lu, and X. Jia. Vlipp: Towards physically plausible video generation with vision and language informed physical prior. ICCV, 2025.
[79] Z. Yang, I.. Chen, Y. Sun, and H. Li. Visual point cloud forecasting enables scalable autonomous driving.
In CVPR, 2024.
[80] 7. Yu, M. Y. T. Idris, H. Wang. P. Wang, J. Chen, and K. Wang. From physics to foundation models: A
review of ai-driven quantitative remote sensing inversion. arXiv preprint arXiv:2507.0000 2025.
[81] 7. Yu, M. I. Idris, P. Wang, Xia, and Y. Xiang. Forgetme: Benchmarking the selective forgetting
capabilities of generative models. Engineering Applications of Artificial Intelligence, 2025.
[82] Y. Yuan, C. Wu, X. Chang. S. Wang, H. Zhang, Liang. S. Zeng, and M. Xu. Unimapgen: A generative framework for large-scale map construction from multi-moda data. arXiv preprint arXiv:2509.22262. 2025.
[83] B. Yue, Guo, K. Hu, C. Wang, B. Wang, K. Jia, and G. Liu. Real-time verification of embodied reasoning
for generative skill acquisition. arXiv preprint arXiv:2505. 11155 2025.
[84] S. Zeng, X. Chang. X. Liu, Z. Pan, and X. Wei. Driving with prior maps: Unified vector prior encoding for autonomous vehicle mapping. arXiv preprint arXiv:2409.053: 25552, 2024.
[85] S. Zeng, D. Qi, X. Chang, F. Xiong, S. Xie, X. Wu, S. Liang. M. Xu, and X. Wei. Janusvin: Decoupling
semanties and spatiality with dual implicit memory for vision-language navigation. arXiv preprint
arXiv:2509.225 22548, 2025.
[86] J. Zhang, Z. Huang, A. Ray, and F. Ohn-Bar. Feedback-guided autonomous driving. In CVPR, 2024.
[87] J. Zhang, C. Xu, and B. Li. Chatscene: Knowledge-enabled safety-critical scenario generation for
autonomous vehicles. In CVPR, 2024.
[88] L. Zhang, B. Wang. X. Qiu, S. Reddy, and A. Agrawal. Rearank: Reasoning re re-ranking agent via reinforcement learning. EMNLP, 2025.
[89] Y. Zhang, X. Liu, R. Tao, Q Chen, H. Fei, W. Che, and L. Qin. Vitcot: Video-text interleaved chain-of- thought for boosting video understanding in large language models. ACM MM, 2025.
[90] Y. Zhang, X. Liu, R. Zhou, Q Chen, H. Fei, W. Lu, and L. Qin. Cchall: A novel benchmark for joint cross- moss-lingual and cross modal hallucinations detection in large language models. ACL, 2025.
[91] Q Zhao, Y. Lu, M. Kim, Z. Fu, Z. Zhang. Wu, Z. Li, Q Ma. S. Han, C. Finn, A. Handa, M.-Y. Liu,
D. Xiang, G. Wetzstein, and T.-Y. YY. Lin. Cot-vla: Visual chain-of of-thought reasoning for vision- language action models. In CVPR, 2025.
[92] C. Zheng, L. T. Vuong. J. Cai, and D. Q. Phung. Movq: Modulating quantized vectors for high-fidelity image generation. NeurIPS, 2022.
[93] P. Zheng, Y. Zhao, Z. Gong. H. Zhu, and S. Wu. Simplellm4ad: An end-to-end vision-language model with graph visual question answering for autonomous driving. ArXiv preprint arXiv:2407.21229, 2024. [94] W. Zheng, W. Chen, Y. Huang. B. Zhang, Y. Duan, and J. Lu. Occworld: Learning a 3d occupancy world
model for autonomous driving. ECCV, 2024.
[95] W. Zheng, Z. Xia, Y. Huang. S. Zuo, J. Zhou, and J. Lu. Doc-1: Closed-loop autonomous driving with large world model. arXiv preprint arXiv: 2412.09627, 2024.
[96] P. Zhou, W. Min, C. Fu, Y. Jin, M. Huang, X. Li, Mei, and S. Jiang. Foodsky: A food-oriented large language model that can pass the chef and dictctic examinations. Patterns, 2025.
[97] P. Zhou, X. Peng, J. Song, C. Li. Z. Xu, Y. Yang, Z. Guo, H. Zhang, Y. Lin, Y. He, et al. Opening: A comprehensive benchmark for judging open-ended interleaved image-text generation. In CVPR, 2025. [98] Y. Zhou, L. Huang, Q Bu, J. Zeng, T. Li, H. Qiu, H. Zhu, M. Guo, Y. Qiao, and H. Li. Embodicd understanding of driving scenarios. ECCV, 2024.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims in the abstract and introduction accurately reflect the contri- bution and scope of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of the work in Section 5
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

This paper focuses on experimental verification and does not involve strict mathematical proof.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We disclose information that reproduces the main experimental results of the paper in Section 4
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
15

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
We will release the code upon completion of the approval process.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
//nips //ipsscc//pill wwtpcccpppppppppeeeeeeeeeeeeeeee httpa///guddeee/eeeeabiiissssss /Cc////oddiiisssseeeeeee ssiiiiiiccyyyyyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
We illustrate all the training and testing details in the Section
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We report information about the statistical significance of the experiments in the Section
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
16

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide information on the computer resources needed to reproduce the experiments in the Section 4
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics htttps://eeurppe: http://wp...p.cpooo.ccccc/cacaaa
Answer: [Yes]
Justification: The research conducted in the paper complies in all respects with the NeurIPS ethical guidelines.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the impacts of the work in Section
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
17

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
The creators of the data and models used in the paper were appropriately credited and in compliance with the license and terms of use.
Justification:
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
18

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
batasets.papersstt
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Our assets (code and model) will be made publicly available upon completion of the approval process.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
19

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
We only use LLM to polish the paper.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https:/ /ttpa/:///eupp.... httpe//rrpsccc//ooffeereeee////p cc//oonferences//2255/LL. for what should or should be described.
20


Reviews:
[{"review_id": "41gmGRSUFP", "reviewer": "Reviewer_4rpa", "summary": "This paper proposes a spatio-temporal Chain-of-Thought (CoT) method to predict future trajectories in autonomous driving. In this framework, the Visual Language Model (VLM) functions as an inverse dynamics model, planning trajectories based on current observations and its own predicted future states. The training process is divided into pre-training and fine-tuning stages and is conducted without altering the existing MLLM architecture. During the pre-training phase, a \"progressive generation\" technique is employed to ensure the future predictions adhere to physical laws. The authors report that this method achieves state-of-the-art (SOTA) performance compared to existing approaches.", "questions": "1. I am curious about the necessity of the VQA task. Could the authors provide the results of an ablation study using only the three generation-related tasks (Future frames, Future 3D detection, and Future lane divider) without the VQA task?\n2. Would it be challenging to produce results on other datasets? For instance, evaluating on Argoverse 2, which is often used alongside nuScenes, would significantly strengthen the claims of robustness.\n3. Considering that other SOTA papers like OmniDrive report results specifically for intersections, why was this critical scenario analysis omitted? Would it be possible to conduct and share additional experiments for such cases?", "limitations": "1. The method's ability to operate in real-time is a significant concern, making its practical application in live driving scenarios challenging.\n2. There is no dedicated module to handle cases where the CoT contains errors, which could compromise safety.\n3. The results are limited to a single dataset, raising concerns about how well the model will generalize to diverse, unseen environments.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n1. The method is cost-effective as it does not modify the existing MLLM architecture, only requiring an expansion of the vocabulary during the pre-training stage.\n2. The progressive generation technique effectively incorporates physical laws, ensuring the model predicts more realistic future scenarios.\n\nWeaknesses:\n1. There is a significant concern regarding real-time performance. The paper provides no metrics on inference latency, and it seems challenging to apply the prediction results immediately in a live driving context.\n2. The paper does not present failure cases or incorrect examples of the generated CoT, which makes it difficult to assess the model's limitations from the qualitative results.\n3. The generated CoT is an output of the model's inference, not the fundamental reason for the decision itself. Therefore, it is difficult to consider it a true basis for the model's actions.", "was_revised": true, "final_justification": "The authors rebuttal has addressed my initial concerns. I am now convinced that most of their claims are well-supported by clear experimental evidence. Accordingly, I am raising my score from borderline accept to accept.", "rebuttal": "Thank you for your valuable time and insightful comments. Your feedback has been crucial for improving our manuscript. We have addressed your comments point-by-point as follows:\n\n---\n\n> **Q1: The paper did not provide the metrics of inference latency and real-time performance.** \n\n**A1:** \n| Method       | DriveVLM | Senna  | FSDrive |\n|--------------|----------|--------|---------|\n| Decode latency (s)↓ | 0.50    | 0.58   | 0.31    |\n\nFSDrive is efficient for real-time tasks. By integrating vLLM and quantization, it achieves a 0.31s decoding latency, surpassing models like DriveVLM and Senna. This performance establishes a solid foundation for real-time applications, poised to improve further with ongoing developments in large model acceleration.\n\n\n\n---\n\n> **Q2: The paper does not provide failure cases of the generated CoT, and there is no dedicated module to handle cases where the CoT contains errors.** \n\n**A2:** \n1. On CoT Failure Cases\n\nWe thank the reviewer for this insightful suggestion. We will incorporate a failure case analysis in the revision. Our key observations are twofold:\n\n* Our model can achieve correct planning even with minor imperfections in the visual CoT. It extracts the essential gist of the future scene, a finding that corroborates GenHancer [1], which states, \"imperfect visual generation can still be beneficial for understanding.\"\n* Due to the data distribution challenges in complex scenarios, while a degraded visual CoT can lead to trajectory deviations, the model's sometimes success under these conditions indicates it has learned a crucial fusion strategy. It effectively balances the generated future (CoT) with current observations, rather than relying solely on the former.\n\n2. On Handling Erroneous CoT\n\nWhile FSDrive does not include a dedicated module for CoT error correction, its design provides robustness. This resilience stems from two key aspects of our approach:\n\n* Dual-Source Fusion: The final planning policy is not solely dependent on the CoT. It learns to fuse and balance two information streams—current observations and the generated future—thereby mitigating the impact of an failed CoT.\n* Progressive Pre-training: Our design proactively reduces CoT errors from the outset. As detailed in Sec. 3.2, the pre-training phase employs a progressive, easy-to-hard settings. This strategy systematically improves the model's predictive accuracy, thus minimizing the generation of erroneous CoTs.\n* Future Work: We agree that an explicit safety module is a valuable future direction. For instance, the system could discard a low-quality CoT (e.g., below an FID threshold) and trigger a re-planning mechanism, adding another layer of safety.\n\n\n\n\n\n\n\n---\n\n> **Q3: The generated CoT is an output of the model's inference, not the fundamental reason for the decision itself.** \n\n**A3:** As argued in [2], a CoT is an essential mechanism for tackling complex tasks, rather than a mere output. To address your concerns, we elaborate on this from the following three aspects:\n* Sequential Generation Process: As shown in Equation 6, our model operates under an auto-regressive paradigm. The generation of our CoT and the subsequent trajectory planning is a sequential process. Critically, the prediction of trajectory planning tokens is conditioned on the previously generated CoT tokens, mirroring the standard CoT generation in LLMs[3].\n* Quantitative Impact: Our ablation study (Table 5) confirms the CoT's critical role. Its removal significantly degrades performance, increasing the collision rate from 0.40% to 0.58%, which directly proves its impact on decision-making.\n* Qualitative Analysis: As shown in Figure 3, when challenged with an incorrect command, our model leverages the CoT to mitigate the faulty instruction and execute a safe maneuver.  This demonstrates the CoT's function as a safety-critical reasoning layer, not a passive output.\n\n\n\n---\n\n> **Q4: Provide the results of an ablation study using only the three generation-related tasks without the VQA task.** \n\n**A4:** \n| Method                     | L2 (1s) | L2 (2s) | L2 (3s) | L2 Avg | Collision (1s) | Collision (2s) | Collision (3s) | Collision Avg |\n|----------------------------|---------|---------|---------|--------|-----------------|-----------------|-----------------|---------------|\n| w/o pre-training           | 0.45    | 1.09    | 2.12    | 1.22   | 0.12            | 0.43            | 1.45            | 0.67          |\n| w/ generation pre-training | 0.38    | 0.95    | 1.70    | 1.01   | 0.09            | 0.37            | 1.35            | 0.60          |\n| w/ generation & understanding pre-training | 0.39 | 0.91 | 1.63 | 0.98 | 0.09 | 0.36 | 1.33 | 0.58 |\n\nWe will address your concerns from the following two aspects:\n* VQA pre-training was specifically incorporated to boost performance on the DriveLM benchmark, successfully raising the score from 0.55 to 0.57. Notably, our model without VQA pre-training (0.55) still surpasses the OmniDrive (0.53).\n* Our ablation study reveals that VQA's impact on planning-centric tasks is modest.  Its removal leads to only a minor performance decrease of 0.02-0.03, suggesting it provides a slight enhancement but is not a dominant factor.  We maintain that exploring the synergy between understanding and generation remains a vital research direction for unified generation and understanding LLMs.\n\n\n\n---\n\n> **Q5: The evaluation results on other datasets, such as Argoverse 2.** \n\n**A5:** \n| Method           | NC↑  | DAC↑  | TTC↑  | Comf.↑ | EP↑   | PDMS↑ |\n|------------------|------|------|------|-------|------|------|\n| LAW [ICLR 25]    | 96.4 | 95.4 | 88.7 | 99.9  | 81.7 | 84.6 |\n| DiffusionDrive-Cam [CVPR 25] | 97.8 | 92.2 | 92.6 | 99.9  | 78.9 | 83.6 |\n| FSDrive          | 98.2 | 93.8 | 93.3 | 99.9  | 80.1 | 85.1 |\n\nThe Argoverse 2 dataset is not designed for planning tasks; consequently, prior works have generally not evaluated planning performance on it. To bridge this gap, we evaluated FSDrive on the NAVSIM dataset. Our method surpasses existing approaches, achieving a final PDMS score of 85.1, demonstrating excellent generalization capabilities.\n\n\n\n\n---\n\n> **Q6: Report results specifically for intersections.** \n\n**A6:** \n| Method           | Intersection (1s) | Intersection (2s) | Intersection (3s) | Intersection Avg |\n|------------------|--------------------|--------------------|--------------------|-------------------|\n| OmniDrive [CVPR25] | 0.93              | 3.65              | 8.28              | 4.29             |\n| FSDrive          | 0.85              | 3.24              | 6.75              | 3.61             |\n\nWe sincerely thank the reviewer for this insightful suggestion. To clarify, the 'Intersection' metric represents a fine-grained breakdown of the overall 'Collision'. This is the primary reason we did not report it individually in the initial submission. As requested, we now present the results for the Intersection metric. As shown above, our method outperforms OmniDrive by 16% on this metric as well, further demonstrating the robustness of our approach. We will add this result to the revised manuscript.\n\n## References\n[1] Ma, Shijie et al. “GenHancer: Imperfect Generative Models are Secretly Strong Vision-Centric Enhancers.” ICCV 2025.\n\n[2] Feng, Guhao et al. “Towards Revealing the Mystery behind Chain of Thought: a Theoretical Perspective.” NeurIPS 2023.\n\n[3] Wei, Jason, et al. \"Chain-of-thought prompting elicits reasoning in large language models.\" NeurIPS 2022."}, {"review_id": "7e3hrnyhsr", "reviewer": "Reviewer_TPKG", "summary": "This paper addresses the limitations of current visual language models (VLMs) in autonomous driving, particularly the reliance on discrete, text-based Chain-of-Thought (CoT) reasoning. The authors argue that such symbolic, language-based intermediate steps can lead to ambiguity in representing spatio-temporal relationships and result in the loss of fine-grained visual information, which is critical for safe and effective autonomous driving.\n\nTo overcome these issues, the paper introduces a spatio-temporal CoT reasoning method that enables VLMs to \"think visually\" rather than abstracting visual information into text. In this approach, the VLM acts as a world model, generating unified image frames that predict future world states. These frames include both perception results (like lane dividers and 3D object detections) to capture spatial relationships, and ordinary future frames to represent temporal evolution. This visual reasoning process serves as an intermediate step, allowing the VLM to function as an inverse dynamics model for trajectory planning.", "questions": "- The authors should explain why the improvement is only marginal under fair comparison, as this is critical for assessing the significance of the proposed methods.\n- A fair comparison should be provided in Table 2 to validate the effectiveness of the proposed methods.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The experimental evaluation is comprehensive, covering multiple relevant tasks (trajectory planning, future frame generation, scene understanding).\n2. The proposed spatio-temporal CoT method is clearly described.\n\nWeaknesses:\n1. Although the overall training process is intuitive and complete, the improvement in trajectory planning based on LLaVA-7B over other state-of-the-art methods is only marginal. Given the sophistication of the proposed methods, the performance gains do not seem commensurate. Furthermore, while the CoT process has been enhanced, the improvement over the DriveLM benchmark remains limited.\n2. As shown in Table 2, FID should be compared at the same resolution. Training with a lower resolution may yield a lower FID score, but this does not necessarily indicate better performance.", "was_revised": true, "final_justification": "After considering the rebuttal and the comments from other reviewers, I find that most of my concerns have been addressed. Accordingly, I am updating my score to the borderline accept.", "rebuttal": "Thank you for your thoughtful review and constructive feedback. Your comments were invaluable in helping us strengthen the manuscript. Please find our point-by-point responses below:\n\n---\n\n> **Q1: The improvement in trajectory planning based on LLaVA-7B over other SOTA methods is only marginal. And the improvement achieved on the DriveLM benchmark is still relatively limited.** \n\n**A1:** \n| Method           | L2 (1s) | L2 (2s) | L2 (3s) | L2 Avg | Collision (1s) | Collision (2s) | Collision (3s) | Collision Avg |\n|------------------|---------|---------|---------|--------|----------------|----------------|----------------|---------------|\n| OmniDrive[CVPR25] (LLaVA-7B) | 0.59    | 1.34    | 2.33    | 1.42   | 0.28           | 1.18           | 2.74           | 1.40          |\n| OmniDrive*[CVPR25] (LLaVA-7B)       | 0.19    | 0.56    | 1.19    | 0.65   | 0.10           | 0.40           | 1.43           | 0.64          |\n| FSDrive(LLaVA-7B)          | 0.36    | 1.01    | 1.90    | 1.09   | 0.08           | 0.34           | 1.11           | 0.51          |\n| FSDrive*(LLaVA-7B)         | 0.22    | 0.51    | 0.94    | 0.56   | 0.02           | 0.07           | 0.53           | 0.21          |\nWe apologize for the confusion and would like to offer the following clarification:\n* As shown in Table 1 of the paper under the ST-P3 metric, our method without ego status achieves a 29% reduction in L2 error (0.60 vs. 0.84) and an 80% reduction in collision rate (0.19 vs. 0.94) compared to OmniDrive (LLaVA-7B), which are very significant improvements. While it's true that all methods perform well when ego status is used, the ST-P3 metric is not stringent enough to reveal meaningful differences.\n* Therefore, for a clearer comparison, we evaluated OmniDrive using UniAD's metrics in the table above. Without ego status, our method achieves a 23% lower L2 error (1.09 vs. 1.42) and a 64% lower collision rate (0.51 vs. 1.40). With ego status, we still see a 14% L2 reduction (0.56 vs. 0.65) and a 67% collision rate reduction (0.21 vs. 0.64). This evidence confirms that our improvements are not \"marginal,\" but rather robust and substantial across different evaluation standards.\n* It is important to note that on the DriveLM benchmark, we compare our 2B model against the 7B models from competing methods. In VQA tasks like this, which typically benefit significantly from model size, our smaller model achieves superior performance. \n\n\n\n\n\n---\n\n> **Q2: Training with a lower resolution may yield a lower FID score, but this does not necessarily indicate better performance.** \n\n\n\n**A2:** We sincerely thank you for your insightful suggestions. We would like to address your concerns from the following perspectives: \n* Our choice of a lower resolution was a deliberate and carefully considered trade-off between inference efficiency and informational sufficiency. Adopting a higher resolution would significantly compromise the model's inference speed. \n* More importantly, we observed that increasing the generation resolution did not lead to a corresponding improvement in the final planning accuracy. Consequently, metrics like FID and fine-grained resolution details are not our primary focus. As detailed in Section 3.3 of our paper, our core objective is to ensure the physical plausibility of the generated scenes and to guide the model's attention, ultimately to support the downstream planning task.\n* It is also worth noting that when compared with DriveDreamer at the same resolution, our method's FID score is substantially better (10.1 vs. 52.6), marking an 81% improvement. This demonstrates that our autoregressive approach remains highly competitive against these diffusion-based methods. We will provide more extensive visual comparisons for a more thorough qualitative analysis in the revised manuscript."}, {"review_id": "4Q9mXUx8EP", "reviewer": "Reviewer_NHPc", "summary": "This paper proposes FSDrive, an autonomous driving framework based on Spatio-Temporal Chain-of-Thought (CoT), which addresses the issues of ambiguous spatio-temporal relationships and fine-grained information loss caused by discrete textual CoT in existing Vision-Language Models (VLMs). The key contributions include: Visual Reasoning: By generating future frames (with lane dividers and 3D detection boxes) as intermediate reasoning steps, the model performs visual simulation akin to human cognition rather than symbolic logic. Unified Pretraining: Joint optimization of visual generation (future frame prediction) and understanding (VQA), coupled with progressive generation (coarse physical constraints first, then fine details). Experimental Validation: On nuScenes and DriveLM datasets, FSDrive outperforms SOTA methods (e.g., UniAD, Doe-1) in trajectory planning, future frame generation, and scene understanding.", "questions": "How does the model handle noisy annotations (e.g., occlusions) for lane dividers and 3D boxes?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. Clearly identifies limitations of textual CoT in autonomous driving (loss of physical laws due to over-abstraction) and proposes visual reasoning as a principled solution.\n2. The core innovation lies in elevating text CoT to visual CoT, directly using images as intermediate reasoning steps, which better suits the spatio-temporal continuity requirements of autonomous driving.\n3. Experimental validation is comprehensive and robust. \n\nWeaknesses\n1. Inference speed for generating future frames is unreported. Compliance with real-time requirements (e.g., >10Hz) remains unclear.\n2. Only front-view future frames are generated, while autonomous driving requires 360° environmental awareness. This may limit real-world deployment safety.", "was_revised": true, "final_justification": "I have reviewed the author response. The rebuttal addressed the main concerns I raised, and the clarifications provided were reasonable. I have no further concerns and will maintain my original positive score.", "rebuttal": "We are deeply grateful for the time and effort you have dedicated to reviewing our manuscript. Your constructive suggestions are immensely valuable and have provided us with an excellent opportunity to substantially enhance our work. We have carefully considered each of your points and our detailed responses are provided below:\n\n---\n\n> **Q1: Inference speed for generating future frames is unreported.** \n\n**A1:** \n| Method       | DriveVLM | Senna  | FSDrive |\n|--------------|----------|--------|---------|\n| Decode latency (s)↓ | 0.50    | 0.58   | 0.31    |\n\nFSDrive is efficient for real-time tasks. By integrating vLLM and quantization, it achieves a 0.31s decoding latency, surpassing models like DriveVLM and Senna. This performance establishes a solid foundation for real-time applications, poised to improve further with ongoing developments in large model acceleration.\n\n\n---\n\n> **Q2: Only front-view future frames are generated, while autonomous driving requires 360° environmental awareness.** \n\n**A2:** We sincerely thank the reviewer for their insightful comments and constructive feedback. We offer the following clarifications: \n* In line with established previous methods [1,2], our approach generates only the front view, as this view contains the essential information.  This design choice allows us to achieve a practical trade-off between inference efficiency and informational sufficiency.\n* We would like to clarify that while the proposed method generates future frames exclusively for the front view, its input consists of images from six cameras, encompassing a full 360° perception of the environment, allowing  model to leverage information from all other views. Furthermore, we have proactively and explicitly acknowledged this point in the \"Limitations and broader impacts\" section on page 9 of our paper and have designated its extension as a direction for future work.\n\n\n\n---\n\n> **Q3: Noisy annotations (e.g., occlusions) for lane dividers and 3D boxes.** \n\n\n**A3:** We directly use the official annotations from the nuScenes dataset. The dataset creators have already addressed ambiguous cases during the collection process and provide reliable ground truth labels even under conditions of occlusion. And when the object is occluded in the front view, since our input is in the surround-view, the model can still obtain the information of the occluded object from other perspectives.\n\n## References\n[1] Hassan, Mariam et al. “GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Control.” CVPR 2025.\n\n[2] Yang, Jiazhi et al. “Generalized Predictive Model for Autonomous Driving.” CVPR 2024."}, {"review_id": "A3ju0N2SJS", "reviewer": "Reviewer_ETeY", "summary": "This paper presents FSDrive, a novel framework for autonomous driving that enables visual reasoning through spatio-temporal Chain-of-Thought (CoT), bridging the gap between perception and planning in a unified visual framework. The experiment part demonstrates that FSDrive achieves good results on the generation, understanding and planing tasks.", "questions": "1) Limited architectural novelty: The proposed architecture for generation and understanding tasks largely mirrors existing designs in general vision-language domains. While the application to driving is appreciated, the model structure itself lacks sufficient novelty.\n2) Dataset limitation: The claim of using only 0.3% of training data relies solely on experiments conducted on nuScenes. This raises concerns about generalizability. The authors are encouraged to validate their method on additional datasets to strengthen the robustness of their claims.\n3) Real-time applicability: The auto-regressive formulation in Equation 6 for action prediction appears computationally expensive and may not satisfy the real-time constraints required in real-world driving systems. Further clarification or optimization is needed here.\n4) Insufficient qualitative evaluation: The paper relies on FID to evaluate generative quality, but comparing FID scores across different output resolutions can be misleading. The authors should provide more visual comparisons to convincingly demonstrate the quality of generated future frames.\n5) Format of Tab6: A more compact and structured format would help improve the overall clarity of the paper.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "FSDrive presents an improved framework for driving policy prediction, guided by a novel Spatial-Temporal Chain-of-Thought (CoT) methodology. The authors conduct comprehensive experiments to validate the effectiveness of the proposed approach, covering generation, understanding, and policy prediction tasks. I recommend acceptance of this paper, primarily because extending the CoT paradigm into the physical world of autonomous driving is both novel and conceptually sound. The thorough experimental evaluation further supports its potential.\n\nHowever, there are several weaknesses that should be addressed:\n\n1) Limited architectural novelty: The proposed architecture for generation and understanding tasks largely mirrors existing designs in general vision-language domains. While the application to driving is appreciated, the model structure itself lacks sufficient novelty.\n2) Dataset limitation: The claim of using only 0.3% of training data relies solely on experiments conducted on nuScenes. This raises concerns about generalizability. The authors are encouraged to validate their method on additional datasets to strengthen the robustness of their claims.\n3) Real-time applicability: The auto-regressive formulation in Equation 6 for action prediction appears computationally expensive and may not satisfy the real-time constraints required in real-world driving systems. Further clarification or optimization is needed here.\n4) Insufficient qualitative evaluation: The paper relies on FID to evaluate generative quality, but comparing FID scores across different output resolutions can be misleading. The authors should provide more visual comparisons to convincingly demonstrate the quality of generated future frames.", "was_revised": true, "final_justification": "The authors have adequately addressed my concerns. The paper’s strengths:\n(1) identifying the limitations of textual CoT in autonomous driving,  (2) proposing visual CoT as a principled solution with images as reasoning steps, and (3) providing comprehensive experiments, demonstrating a solid contribution that meets the acceptance bar.", "rebuttal": "We would like to express our sincere appreciation for your perceptive comments and the rigorous review of our manuscript. Your guidance has been instrumental in refining the scientific rigor and clarity of our paper. We have revised the manuscript accordingly and offer our itemized responses to your queries below:\n\n---\n\n> **Q1: The model structure itself lacks sufficient novelty.** \n\n**A1:** We sincerely apologize for any confusion caused. We would like to clarify that the core innovation of our work is not to propose a new underlying model architecture. As stated by reviewers 4rpa, TPKG and NHPc, our primary contribution is twofold: firstly, we introduce a novel visual reasoning paradigm specifically for autonomous driving; secondly, we propose a unified and efficient pre-training method to activate the visual generation capabilities of existing LLMs. Furthermore, our proposed unified architecture for generation and understanding can be applied to any existing multi-modal large model, demonstrating its broad applicability and scalability.\n\n\n---\n\n> **Q2: The claim of using only 0.3% of training data relies solely on experiments conducted on nuScenes. And validating the results on other datasets.** \n\n**A2:** \n| Method           | NC↑  | DAC↑  | TTC↑  | Comf.↑ | EP↑   | PDMS↑ |\n|------------------|------|------|------|-------|------|------|\n| LAW [ICLR 25]    | 96.4 | 95.4 | 88.7 | 99.9  | 81.7 | 84.6 |\n| DiffusionDrive-Cam [CVPR 25] | 97.8 | 92.2 | 92.6 | 99.9  | 78.9 | 83.6 |\n| FSDrive          | 98.2 | 93.8 | 93.3 | 99.9  | 80.1 | 85.1 |\n\nWe thank the reviewer for their valuable feedback and sincerely apologize for the lack of clarity that led to this confusion.\n* We would like to clarify a key misunderstanding: our reference to \"using only about 0.3% of the data\" does not pertain to 0.3% of the nuScenes training set. Instead, this figure is relative to the billion-scale data required by large models that aim to unify visual understanding and generation (e.g., Chameleon, VILA-U). Compared to these models, our approach significantly reduces training costs.\n* Furthermore, we have now evaluated the planning performance of FSDrive on the NAVSIM dataset. FSDrive outperforms previous methods, achieving a PDMS of 85.1, demonstrating excellent generalization capabilities.\n\n\n\n---\n\n> **Q3: The auto-regressive formulation in Equation 6 may not satisfy the real-time constraints required in real-world driving systems.** \n\n**A3:** \n| Method       | DriveVLM | Senna  | FSDrive |\n|--------------|----------|--------|---------|\n| Decode latency (s)↓ | 0.50    | 0.58   | 0.31    |\n\nFSDrive is efficient for real-time tasks. By integrating vLLM and quantization, it achieves a 0.31s decoding latency, surpassing models like DriveVLM and Senna. This performance establishes a solid foundation for real-time applications, poised to improve further with ongoing developments in large model acceleration.\n\n\n---\n\n> **Q4: Comparing FID scores across different output resolutions can be misleading. The authors should provide more visual comparisons.** \n\n**A4:** We sincerely thank you for your insightful suggestions. We would like to address your concerns from the following perspectives: \n* Our choice of a lower resolution was a deliberate and carefully considered trade-off between inference efficiency and informational sufficiency. Adopting a higher resolution would significantly compromise the model's inference speed. \n* More importantly, we empirically observed that increasing the generation resolution did not lead to a corresponding improvement in the final planning accuracy. We attribute this to the fact that the number of visual tokens would vastly outnumber the planning tokens, causing the negative impacts to outweigh the potential benefits. Consequently, metrics like FID and fine-grained resolution details are not our primary focus. As detailed in Section 3.3 of our paper, our core objective is to ensure the physical plausibility of the generated scenes and to guide the model's attention, ultimately to support the downstream planning task.\n* It is also worth noting that when compared with DriveDreamer at the same resolution, our method's FID score is substantially better (10.1 vs. 52.6), marking an 81% improvement. This demonstrates that our autoregressive approach remains highly competitive against these diffusion-based methods. We will provide more extensive visual comparisons for a more thorough qualitative analysis in the revised manuscript.\n* Furthermore, as advised, we have included richer visual comparisons in the revised manuscript to better support the qualitative evaluation of our method.\n\n---\n\n> **Q5: Format of Tab6: A more compact and structured format would help improve the overall clarity of the paper.** \n\n**A5:** Thank you for your valuable suggestions. We have reformatted Table 6 for better conciseness."}]

Author rebuttal:

Meta-review: This paper introduces FutureSightDrive, which replaces text-based Chain-of-Thought reasoning in autonomous driving with visual reasoning through generated future image frames containing lane markers and 3D detection boxes as intermediate reasoning steps for trajectory planning. Four reviewers initially provided mixed feedback with concerns about real-time performance, front-view-only generation, and limited generalization, but all ultimately supported acceptance after the authors provided a comprehensive rebuttal with additional experiments demonstrating 0.31s decode latency, NAVSIM dataset results, and intersection-specific metrics. The paper's key strengths include a novel application of visual CoT that addresses limitations of abstract text reasoning, practical real-time viability, and cost-effective implementation requiring no architectural changes to existing MLLMs, while the main limitation remains the restriction to front-view generation. Despite modest architectural novelty, the visual reasoning paradigm represents meaningful progress in autonomous driving applications of vision-language models with adequate experimental validation.
Decision: Accept (spotlight)