ResWorld: Temporal Residual World Model for End-to-End Autonomous Driving
Authors: Jinqing Zhang
Zehua Fu
zelinxu
wenying.dai
Qingjie Liu
Yunhong Wang
Venue: ICLR 2026 Poster (2026)
Keywords: Autonomous Driving
End-to-End
World model
Primary area: applications to robotics, autonomy, planning

Abstract: The comprehensive understanding capabilities of world models for driving scenarios have significantly improved the planning accuracy of end-to-end autonomous driving frameworks. However, the redundant modeling of static regions and the lack of deep interaction with trajectories hinder world models from exerting their full effectiveness. In this paper, we propose Temporal Residual World Model (TR-World), which focuses on dynamic object modeling. By calculating the temporal residuals of scene representations, the information of dynamic objects can be extracted without relying on detection and tracking. TR-World takes only temporal residuals as input, thus predicting the future spatial distribution of dynamic objects more precisely. By combining the prediction with the static object information contained in the current BEV features, accurate future BEV features can be obtained. Furthermore, we propose Future-Guided Trajectory Refinement (FGTR) module, which conducts interaction between prior trajectories (predicted from the current scene representation) and the future BEV features. This module can not only utilize future road conditions to refine trajectories, but also provides sparse spatial-temporal supervision on future BEV features to prevent world model collapse. Comprehensive experiments conducted on the nuScenes and NAVSIM datasets demonstrate that our method, namely ResWorld, achieves state-of-the-art planning performance. The code is available at https://github.com/mengtan00/ResWorld.git.

Published as a conference paper at ICLR 2026
RESWORLD: TEMPORAL RESIDUAL WORLD MODEL FOR END-TO-END AUTONOMOUS DRIVING
Jinqing Zhang', Zehua Fu', Zelin Xu², Wenying Dai Qingjie Liu Yunhong Wang State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China *Zhongguaneun Laboratory, Beijing, China
BBeijing Jingwei Hirain Technologies Co., Inc. *Hangzhou Innovation Institute, Beihang University, Hangzhou, China
ABSTRACT
The comprehensive understanding capabilities of world models for driving scenar- ios have significantly improved the planning accuracy of end-to-end autonomous driving frameworks. However, the redundant modeling of static regions and the lack of deep interaction with trajectories hinder world models from exert- ing their full effectiveness. In this paper, we propose Temporal Residual World Model (TR-World), which focuses on dynamic object modeling. By calculat- ing the temporal residuals of scene representations, the information of dynamic objects can be extracted without relying on detection and tracking. TR-Wa World takes only temporal residuals as input, thus predicting the future spatial distribu- tion of dynamic objects more precisely. By combining the prediction with the static object information contained in the current BEV features, accurate future BEV features can be obtained. Furthermore, we propose Future-Guided Trajec- tory Refinement (FGTR) module, which conducts interaction between prior tra- jectories (predicted from the current scene representation) and the future BEV features. This module can not only utilize future road conditions to refine trajec- tories, but also provides sparse -patial-temporal supervision on future BEV fea- tures to prevent world model collapse. Comprehensive experiments conducted the nuScenes and NAVSIM datasets demonstrate that our method, namely ResWorld, achieves state-of-the-arr planning performance. The code is available https://gihubb.coomm/eennanttnne gorld.
INTRODUCTION
End-to-end autonomous driving framework has emerged as an important research direction recent years, presenting a cost-effective and highly scalable solution for autonomous driving applications. The traditional autonomous driving systems generally perform environmental perception at first,
including 3D object detection (Huang al. [202]: Wang al. [023 Zhang et al. 2023 2025), ) map
segmentation (Li et al. 2022a Liao et al. 2022 2024 Yuan et al.) (024b) and semantic occupancy prediction (Ma et al.) 2024 Yu et al. 2003] Zhang et al.] 2024), among others. Subsequently, the
multiple perception results are integrated through rule-based methods (Bouchard et al. 2022 Treiber
al. 2000) or independent DNN models (Huang et al. 2023 Cheng et al. 20024a] to generate the future trajectory of the ego vehicle. In contrast, end-lo-end autonomous driving Trameworks (Hu
al. 2023 Jiang al., 2023 Sima et al. 2023 Zheng al. 2024b Sun et al. 2024 Weng al.
2024
Hu et a 2022a Guo et al. 2024) integrate the multille tasks into single model. Such
design not only reduces information loss between raw data and the final planning results but also
enables collaborative optimization across various modules, thus exhibiting stronger adaptability to complex scenarios.
Recently, due to the high annotation cost required for training multiple perception and prediction modules, several end-to-end autonomous driving approaches (Li & Cuil 2025 Li et al. 2025 Zheng
al. 2024a Yang al. 2025 Zheng et al. 2025) have adopted world models to replace these
lliary task modules. As shown in Fig by treating future scene prediction as a proxy task, world
*Corresponding author: qingjie. liu @ huua.edu.cc

---

Published as a conference paper at ICLR 2026
Trajectory
World
Priec Trajectory
Curreat Secne
Curreat Scane
Predicted
Aligument
Frame T++1
Future Secne
Predicted Seene
Frame T-1
Presious Scane
Temporal Residual
Final Trajectery
(a) Normal World Model Framework
(b) Res World Framework
Figure I: Comparison Between Normal World Model Framework and ResWorld Framework. Different from the normal world models that model the entire scene and implicitly optimize trajecto- ries, Resworld uses the temporal residuals the scene representations to represent dynamic objects for precise modeling. Mcanwhile, the prior trajectories are corrected through explicit interaction with the predicted future BEV feature.
model frameworks effectively enhance the model's ability to understand and model driving scenes, thereby improving the planning accuracy. However, most information in the scene repre- sentations belongs to static objects such the ground and buildings, which be directly retained in future scenarios without the need for redundant modeling. In contrast, dynamic objects such as vehicles and pedestrians require more precise modeling, yet they are difficult to identify from the en- vironment without relying on perception tasks. Furthermore, current methods lack deep interaction between trajectories and the future scene representations predicted by the world model.
To address these issues, we propose Temporal Residual World Model (TR-World) as shown in Fig which can precisely model the dynamic objects and predict accurate future scene represen- tations. First, we shift BEV features at different timestamps into the current BEV coordinate system and use the same spatial attention mask to extract their sparse scene queries. Subsequently, we subtract the scene queries of adjacent timestamps to obtain the temporal residuals of scene queries. The temporal residuals represent the changes the same position across different timestamps, thus standing for the dynamic objects in the scene. When predicting future BEV features, the current BEV coordinate system is still adopted. This allows the current BEV features to depict the future distribution of static objects, thereby avoiding redundant modeling of static objects. TR- World only processes the temporal residuals and maps the predicted future spatial distribution of dynamic ob- jects onto the current BEV features, thereby obtaining accurate predictions of future BEV features.
Furthermore, to make full use of the predicted future BEV features, we propose Tuture-Guided Trajectory Refinement (FGTR) module. We first employ series of waypoint queries represent the ego vehicle's future trajectory, where each query corresponds to the ego vehicle's position at specific future timestamp. After decoding the prior trajectory from waypoint queries, this trajec- tory acts as a set of reference points to guide the interaction between waypoint queries and future BEV features. This operation can effectively verify whether the prior trajectory will collide with other objects or deviate from the drivable area, thereby correcting the prior trajectory and improving planning performance. FGTR additionally applies sparse spatial-temporal supervision on future BEV features, which can effectively alleviate world model collapse. It is worth noting that supervis- ing future BEV features with ground truth at any future timestamps will lose the spatial distribution of dynamic objects at other timestamps. Therefore, not applying supervision can instead allow the model to independently optimize the future BEV features and retain the most important information.
We integrate the proposed components into novel end-to-end autonomous driving model, namely Res World. Experiments conducted on nuScenes and NAVSIM benchmarks indicate that Res World achieves Mate-of-the-an planning accuracy. Our contributions can be summarized follows:
We use the current BEV coordinate system to represent the future BEV representations predicted by the world model, climinating the need for redundant modeling of static objects.
We utilize the temporal residuals of scene representations extract information about dy- namic objects without relying on auxiliary tasks. Temporal residuals are processed by Temporal Residual World Model predict dynamic objects' future spatial distribution.
2

---

Published as a conference paper at ICLR 2026
We propose anture-Guidel Trajectory Refinement module, which applies interaction be- tween prior trajectory and future BEV features to improve the planning accuracy and pre- vent world model collapse.
ResWorld achieves state-of-ther results on nuScenes and NAVSIM benchmarks, demon- strating the effectiveness of our proposed framework.
RELATED WORKS
2.1
END-TO-END AUTONOMOUS DRIVING
Nowadays, end-to-end autonomous driving approaches are gaining increasing attention for cost- effectiveness and high scalability. These methods generally adopt an integrated model that predicts trajectories from the input raw sensor data, achieving state-of-the-art trajectory prediction perfor- mance. ST-P3 (Hu et al.J 2022b) obtains future ego-vehicle movements by progressively utilizing map perception module, BEV occupancy module, and planning module. UniAD (Hu et al., 2023) enhances the robustness of the system by further adopting supplementary detection, tracking, and motion prediction modules. VAD (Jiang et al. 2023) represents object movements and lane lines using vectors, thereby reducing the total computation of the model. PARA-Drive (Weng et al. (024) comprehensively explores the design space of modular perception and prediction task stacks in tonomous driving. OceNet (Sima et al., 2023) introduces occupancy prediction to construct detailed 3D scene representations for planning. VADv2 (Chen et al. 2024) predicts multiple action candi- dates and samples one action as the planning result. GenAD (Zheng et al. (0224) adopts the gen- erative model for trajectory generation, jointly optimizing motion and planning heads. UAD (Guo) et al. 2024) utilizes the angular object mask as the scene representation to avoid collision. Diffu- Drive (Liao et al.) 2025) applies diffusion models boost trajectories' diversity and robustness in complex scenarios. However, these methods rely on fine-grained annotations to train auxiliary task modules, which restricts their ability to utilize large-scale raw data.
2.2 WORLD MODEL FOR END-TO-END AUTONOMOUS DRIVING
World models have demonstrated excellent spatial understanding and modeling capabilities, which are leveraged by some end-to-end autonomous driving models to replace auxiliary tasks. Occ- World (Zheng et al. 2024a) adopts unified occupancy-centre world modeling, enhancing spatial- temporal scene understanding for robust planning. Drive-WM (Wang et al.) (024) generates high- quality driving videos through joint spatial-temporal modeling. thereby improving the model's plan- ning accuracy. SSR (Li & Cui (025) converts the dense BEV feature into sparse scene queries and utilizes the world model to enhance the scene understanding. LAW (Li al.. (024a) adopts the latent world model framework and carries out experiments under perception free and perception- based settings. World4Drive (Zheng et al. 2025) generates multi-modal trajectories and utilizes the world model to select the most appropriate one. However, these world models tend to perform dundant modeling on static objects, while their modeling dynamic objects remains insufficient. Additionally, the absence of deep interaction between trajectories and future scene representations hinders world models from exerting their full effectiveness.
METHOD
3.1 PRIOR TRAJECTORY PREDICTION
To extract the temporal residuals of BEV features, it is necessary for BEV features to have high geometric quality, which facilitates the spatial alignment of BEV features across different times- tamps. Therefore, choose GeoBEV (Zhang et al., 2025) the base of the model, which can efficiently generate BEV features with high geometric quality. As shown in Fig the milti-view images for each timestamp are converted to BEV features, thus obtaining (B,, B....... R. B, E xC*H*W is the BEV feature of the current timestamp, where C.H.W are the channel, height and width dimensions, and is the number of past timestamps. Following BEVDet4D (Huang & Huang 2022), (B..1..... B,-b} are all transformed into the coordinate system of B, and fused by
B Fase Cony(Concat,, B,, B B....... B.+k))
3

---

Published as a conference paper at ICLR 2026
Multi-view Image
BEV Feuture
Prior Trajectory Prediction
Fused BEV Feature
Fused Query
Prior Trajectory
Temporal Residual World Model
Future BEV Feature
Reference Points
Deformable Attention
Final Trajectory
Future-Guided Trajectory Refinement
Scene Query
Temporal Residuul
Figure 2: Overall Framework ResWortd. Multi-view images at different timestamps are con- verted into BEV features, which are used to predict prior trajectories. On the other hand, BEV features are used to calculate temporal residuals, which are then processed by the Temporal Resid- ual World Model to predict the future distribution of dynamic objects. Future- Guided Trajectory Refinement module further utilizes the predicted future BEV features to refine the planning results.
We use the planning module of SSR (Li & Cui 2025 to perform perception-free planning. The dense B fuse is first processed by TokenLearner module (Ryoo et al. 2021 to obtain N, sparse scene queries S fuse E R N. which can be formulated by
S Juse = TokenLearner fuse) = AvgPool(SAA (B Fuse)
B Busee)
where SA denotes the generation of the spatial attention map and AvgPool denotes the global average pooling operation. See is operated by self-attention for further information extraction:
Suse SelfAttention (Suuse)
We use a set of waypoint queries W E N.xC to represent the ego vehicle's future status, where N₂ denotes the number of future timestamps to be predicted. After the cross attention operation between W and S Sase, the prior trajectories can be decoded by multi-layer perceptron (MLP) as:
T priio ACoosSCrssttttetiio W, Stusey S Sjuse))
where each row in T priior E N22 represents the ego vehicle's coordinates at future timestamp.
3.2 TEMPORAL RESIDUAL EXTRACTION
Since (B,, B........... B, } share the same coordinate system of B,, they represent the scene infor- mation of the same scene at different timestamps. By calculating their residuals, information about dynamic objects in the scene can be extracted.
Given that B fuse carries the spatial information across different timestamps, it can be utilized to pre- dict a spatial attention map that emphasizes the regions with dynamic objects. For each timestamp i, B; is weighted by this spatial attention map to extract the sparse scene queries formulated
S = Avglool((AABB Fase) O B:)
(5)
After obtaining (S S............ S.-k). set of temporal residuals (R,, R, R caleu- lated by subtracting scene queries of the previous timestamp as shown in Fig
3.3 TEMPORAL RESIDUAL WORLD MODEL
Previous world models used for end-to-end autonomous driving do not distinguish between dynamic objects and static objects in the scene and devote the same effort to predicting their future spatial
4

---

Published as a conference paper at ICLR 2026
Self-Attention
TokenFuser
Self-Attention
Self-Attention
Temporal Residual
Fused BEV Feature
Future Feature
Figure 3: Structure of Temporal Residual World Model
distribution. However, if the coordinate system of B, is still adopted when predicting future BEV features, the spatial distribution of static objects can be regarded as unchanged. As a result, B fuse serve as the appropriate future representation of static objects, eliminating the need for addi- tional modeling. In addition, the understanding of static objects already accomplished during the prediction of prior trajectories, and the world model is not required to participate in this process.
To avoid redundant modeling of static objects and make the world model focus more on dynamic objects, we propose the Temporal Residual World Model (TR- World) as shown in Fig IR-World takes only temporal residuals as input to predict the future spatial distribution of dynamic objects. Specifically, each temporal residual R; undergoes information extraction via self-attention opera- tions, followed by accumulation across timestamps to obtain a future representation of dynamic objects E R.NxC., This process can be formulated as
SelfAttention(R, (R;)
(6)
i=t-
needs to be presented on BEV features to restore the future spatial distribution of the dynamic objects accurately. We adopt TokenFuser (Ryoo et al. 2021), the inverse transformation of Token- Learner, to expand R on the base of B fuse by
fature TokenFuser( (R, fuse) + B fuse MLP ( fuse)
R B Buse
where MLP maps Juse to HX and @ denotes the combination of matrix transposition and multiplication, which outputs the prediction of future BEV features B Buture E RC*H*W
3.4 FUTURE-GUIDER TRAJECTORY REFINEMENT
Existing end-to-end autonomous driving methods generally utilize the world model optimize planning performance in an indirect manner. Specifically, by treating the prediction future scene representations as proxy task, the model's overall ability to understand autonomous driving sce- narios can be enhanced. However, the predicted future scene representations could serve as valuable references for trajectory planning, yet they have not been effectively utilized date. On the other hand, when future scene representations lack supervision from any auxiliary tasks, it is challenging to prevent the world model from collapsing. which means the model tends to map diverse driving scenes to identical scene representations.
To address the above issues, we have designed the Future-Guided Trajectory Refinement (FGTR) module. This module simply applies Deformable Attention Operation between waypoint queries W and future BEV features B fiture while T prior serves as the reference points on B as shown in Fig Subsequently, the final trajectory T fimall is decoded by MLP, which can be formulated by
w Deform AeffotmAttention((> W,
WWBBButuee futures T Tprior)
(8)
T final MLP(W)
Since each query in W represents the ego vehicle's status future timestamp, FGTR module can collect the future environmental information around the ego vehicle from B future based on T Tprior- This information can be used to check whether the ego vehicle will collide with other objects or drive out the drivable area, and thus correct T priir promptly. This not only makes full use B future but also provides sparse spatial-temporal supervision for it. While reference point coordinates provide
5

---

Published as a conference paper at ICLR 2026
Table 1: Comparison of state-of-the-art methods on the nuScenes dataset. denotes the metrics evaluated using the official models and code. ) denotes using ego status in the planning module
following BEVPlanner++ (Li al. 2024ch. & denotes the AVG metric calculated in the same way
as VAD @Jiang et al. 2023).
L2 (m) J
Collision Rate (%) 4
Method
Auxiliary Task
2s
Avg
1s
3s Avg
ST-P3 (Hu al. 2022b)
Map&
1.72 3.26
4.86
3.28
0.44
1.08
3.01 1.51
UniAD (uu ct al. 2023)
Det&Track&&M DendTTTnkkMMapeeeiindOO
0.48 0.96
1.65
1.03
0.05
0.17
0.71
0.31
OceNet Sima et al. 2023
Det& Mat&&Oe
1.29 2.13 2.99
2.14
0.21
0.59
1.37 0.72
PARA-Drive Weng et al.
2024
Det& Irack Ooci
0.40 0.77 1.31
0.83
0.07
0.25
0.60
0.30
GenAD Zheng et al. 2024b
Det&Map&Motion
0.36 0.83 1.55
0.91 0.06
0.23 1.00 0.43
SSR* (Li & Cm 2025
None
0.25 0.64
1.33 0.74 0.08 0.12 0.72 0.31
Res World
None
0.22 0.56
1.17
0.65
0.02 0.04
0.64 0.23
ResWorldg
None
0.19 0.50
1.08
0.59
0.02 0.06
0.43 0.17
ST-P3 Hu ct al.
2022b
Map
1.33 2.11 2.90 2.11 0.23 0.62 1.27 0.71
UniAD (Hu et a.
2023
Det& &aak&rack&Map& MrnteccokMaMMMooooookkkk Moe
0.44 0.67 0.96 0.69 0.04 0.08 0.23 0.12
VADÉ Jiang et al.
202.3
Det&Map&Motienn
0.41 0.70 1.05 0.72 0.07 0.17 0.41 0.22
BEV-Planner++01
[ et al.
2024c
None
0.16 0.32 0.57 0.35 0.00 0.29 0.73 0.34
PARA-Drivef (Weng et al.
2024
0.25 0.46 0.74 0.48 0.14 0.23 0.39 0.25
LAW; (Li et al.
2025
None
0.26 0.57 1.01 0.61 0.14 0.21 0.54 0.30
LAW: (Li et al.
2025
Det&Map&Motton
0.24 0.46 0.76 0.49 0.08
0.10 0.39 0.19
GenADī (Zheng
2024b
Mottioon
0.28 0.49 0.78 0.52 0.08
0.14 0.34 0.19
SparseDrivee (Sun et al. 2204
DeuKTraccs&Mppp&MMiii
0.29 0.58 0.96 0.61 0.01 0.05 0.18 0.08
Drive-OceWorldf; Yang ot al.
2025
Occ
0,25 0.44 0.72 0.47 0.03 0.08 0.22 0.11
SSR* (Li& Cui
2025)
None
0.19 0.36 0.62 0.39 0.10 0.10 0.24 0.15
MomAD (Song al. (2025)
Det&&racke& Mer&Track&Maa&&Moeooo Matin
0.31 0.57 0.91 0.60 0.01 0.05 0.22 0.09
DiffusionDDrvef (Lia0 2025
0.27 0.54 0.90 0.57 0.03 0.05 0.16 0.08
Res Worldf
None
0.17 0.32 0.55 0.35 0.01 0.02 0.16 0.07
Res ResWorld03
None
0.14 0.27 0.49 0.30
0.01
0.03 0.14 0.06
spatial supervision, the different timestamps represented by W offer temporal supervision. B future is encouraged to accurately represent cross-temporal spatial information such as the future positions of dynamic objects, thus preventing the world model from collapsing.
3.5 LOSS
During training, we only adopts the L11 loss for T prior and T fanal, which can be expressed as
C = pupporio,TTrr TGT) + LL(T final, TcT)
(10)
where TeT denotes the ground truth trajectory of ego vehicle. Unlike general world models, we do not utilize real future data to generate the label for supervising B future. This approach enables Buttrr to preserve the spatial distribution of dynamic objects across multiple future timestamps, rather than being limited to a specific timestamp. Experiments confirm that not supervising B future enables higher planning performance.
EXPERIMENT RESULS
4.1 DATASET AND METRIC
nuScenes We conduct the open-loop evaluation of WerWo nuScenes (Caesar et al. 2020). the commonly used autonomous driving dataset. Consistent with previous works (Li & Cui 2025 Li et al. 2025), we use displacement enror and collision rate (CR) metrics to evaluate the accuracy of trajectory prediction. The displacement eтror is represented by the L2 error between the pre- dicted trajectory and the ground truth trajectory, which indicates the degree of deviation between the planning model and the experts. The collision rate quantifies the percentage of cases involy- ing collisions with other objects when executing the predicted trajectory, reflecting the safety of the

---

Published as a conference paper at ICLR 2026
Table 2: Comparison of state-of-the-art methods on the NAVSIM navtest split. denotes the utilization of historical frame to obtain the temporal residual of the scene represenation.
Method
Auxiliary Task
NC
DAC f
TTC
Comf.
EP
PDMS 1
LAW (Li ct al.
2025
None
96.4
95.4
88.7
99.9
81.7
84.6
World-DDive Zheng al. 2025
None
97.4
94.3
92.8
100
79.9
85.1
RosWorld
None
98.1
95.6
94.3
100
81.8
87.3
UniAD (Hu et
2023
Mcp
97.8
91.9
92.9
100
78.8
83.4
PARA-Drvv (Weng ct al.
2024
Det&Map
97.9
92.4
93.0
99.8
79.3
84.0
Transfuser (Chitla el al. 2022
Det&Map
97.7
92.8
92.8
100
79.2
84.0
DRAMA Yuan al.
2024a)
Det&Map
98.0
93.1
94.8
100
80.1
85.5
VADv2 ( Chen
2024
Det&Map
97.2
89.1
91.6
100
76.0
80.9
Mydra-MDDP--EEI
2024b
Det&Map
98.3
96.0
94.6
100
78.7
86.5
DiffusinDrive (Liao et al
2025
Det&Map
98.2
96.2
94.7
100
82,2
88.1
ResWorld
Det&Map
98.2
96.4
98.9
100
82.5
88.3
ResWorld"
Det&Map
98.9
96.5
95.6
100
83.1
89.0
planning model. We adopt the evaluation methods of both UniAD (Hu et al. 2023 and VAD (Jiang
al., 2023) to compare with as many methods as possible, where VAD's metrics are the temporal averages of UniAD's metrics.
NAVSIM We further conduct the closed-loop evaluation of ResWorld on the NAVSIM bench- mark (Dauner et al. 2024). The data of NAVSIM benchmark are resampled from OpenScene (Con- tributors 2023), which contains 120 hours of driving logs selected from the nuPlan dataset (Caesar al. 2021). By removing simple scenarios from OpenScene, such as straight-driving scenarios, the evaluative capability NAVSIM benchmark for planning models is enhanced. NAVSIM bench- mark employs the Predietive Driver Model Score (PDMS) comprehensively evaluate the planning model, which is calculated using five key factors including No At-Fault Collision (NC), Drivable (TTC), Comfort (Comf.), and Ego Progress (EP).
Area Compliance (DAC),
4.2 IMPLEMENTATION DETAILS
nuScenes When conducting experiments on the nuScenes benchmark, we adopt model structure similar SSR (Li & Cui 2025). To extract the temporal residuals
residuuass of BEV features, we replaced the BEVFormer (Li et al. (0226) used in SSR with GeoBEV (Zhang et al. 2025), aiming to generate high geometric quality BEV features different timestamps separately. We adopt ResNet-50 (He) et al. 2016) the image backbone to process the multi-view images downsampled to 256 x 704. We set k = 2. which means data from the current frame and 2 previous frames are used. It is optional to use ego status in the planning module, which corresponds to the "in Planner" configuration in BEVPlanner (i et al. 2024c). Metrics for both configurations are reported. The model trained for 12 epochs
NVIDIA RTX 3090 GPUs with total batch size of 8. The AdamW (Loshchilov & Hutter 2019) optimizer with learning rate x 10 is utilized. We further conduct ablation studies the nuScenes benchmark evaluate the effectiveness of the proposed components.
NAVSIM For experiments conducted on NAVSIM benchmark, we adopt a model structure similar to TransFuser (Chitta al. 2022). which utilized two ResNet-34 backbones process concatenated images and LiDAR BEV maps. Since previous methods did not utilize historical frames, we used the agent queries employed in object detection replace temporal residuals as the input of the world model. We also implemented version without auxiliary tasks, which is used for comparison with perception free models. The model trained for 100 epochs on 8 NVIDIA RTX 3090 GPUs with total batch size of 512 and the learning rate is set to 6 x 10-4
4.3 MAIN RESULTS
nuScenes We conducted comprehensive comparisons between ResWorld and existing end-to-end autonomous driving methods on the nuScenes (Caesar et al. 2020) benchmark as shown in Tab [] It can be found that Res World achieved a new ofate-of-the-ar accuracy. When ego status is not used in the planning module, method outperforms methods such GenAD (Zheng et al. 2024b) and

---

Published as a conference paper at ICLR 2026
Table 3: Ablation study of each proposed component. "TR-World" and FGTR denote Temporal Residual World Model and the Future-Guided Trajectory Refinement, respectively.
Ego Status in Planner
1.2 (m)
Collision Rate (%) +
TR- TR-World
FGTR
Is
2s
38
Avg
1s
2s
3s
Avg
X
0.25
0.62
1.27
0.71
0.02
0.25
0.64
0.31
X
0.22
0.56
1.17
0.65
0.02
0.04
0.64
0.23
0.21
0,55
1.18
0.65
0.02
0.12
0.70
0.28
0.19
0,51
1.12
0.61
0.02
0.10
0.64
0.25
0.20
0.52
1.12
0.61
0.02
0.10
0.55
0.22
0.19
0.50
1.08
0.59
0.02
0.06
0.43
0.17
Table 4: Ablation study of Temporal Residual World Model. "Future Supervision" denotes the utilization of real future data supervise the future BEV features predicted by the world model.
Future
(m)
4
Collision Rate (%) +
World Model Type
Supervision
1s
2s
38
Avg
2s
3s
Avg
0.20
0.52
1.11
0.61
0.02
0.12
0.57
0.23
Normal World Model
0.20
0.53
1.11
0.61
0.02
0.12
0.49
0.21
0.19
0.51
1.12
0.61
0.02
0.08
0.53
0.21
TR-We Woold
0.19
0.50
1.08
0.59
0.02
0.06
0.43
0.17
2025), which rely on auxiliary perception/prediction tasks for driving scene understanding. Res World also demonstrates advantages over methods like SSR (Li & Cui 2025) and LAW (Li et al.] 2025), which employ none of the auxiliary tasks and fully rely on world models for scene understanding. When adopting ego status to help predict more accurate prior trajectories, the final planning accuracy of ResWorld is largely improved and outperforms BEV- Planner++ (Li et al. 2024c). This indicates that our framework boasts robust scene understanding capability, which prevents overfitting due to over-reliance on ego status.
DiffusionDrivee (iao et al.
NAVSIM We also evaluated the closed-loop planning accuracy of ResWorld on NAVSIM (Dauner et al., 2024) benchmark, and the experimental results presented in Tab 2 To conduct fair comparison with methods that do not utilize historical frame data, agent queries for object detection replace temporal residuals as the input of TR-Wor.d. Nevertheless, Res World still achieves the state- of-the-art planning accuracy of 88.3% PDMS, surpassing the performance of Hydra-MDP (Li et al., 2024b) and DiffusionDrive (Liao et al. 2025). When not using auxiliary tasks such as detection and BEV map segmentation, our proposed method also outperforms world model-based methods like LAW (Li et al. 2025) and 4rivd (Zheng et 2023). Furthermore, the complete World implemented with historical frame data achieves 89.0% PDMS, demonstrating the capacity of tem- poral residuals to represent dynamic information of the driving scene.
4.4 ABLATION STUDY
Efficiency of Components We conducted experiments to evaluate the effectiveness of Temporal Residual World Model (orll) and Guture Trajectory Refinement (FGTR) module, and the experimental results are shown in Tab 3 When only using TR- and implicitly optimizing trajectories in the manner of SSR, can significantly improve the model's s scene understanding capability and enhance planning accuracy. When only using the FGTR module and refining prior trajectories with the current BEV features, can also effectively improve the quality of trajectories. The combination of Forld and FGTR can further improve the planning performance. When not using ego status in the planning module, the two modules together reduce 8.4% of the baseline's average L2 error and 25.8% of the baseline's is average collision rate. When adopting ego status in the planning module, the two modules also reduce 9.2% of the baseline's 's average L2 error and 39.3% of the baseline's average collision rate.
8

---

Published as a conference paper at ICLR 2026
W/ FGTR
w/ FGIR
Figure 4: Effect of Future-Guided Trajectory Refinement Module alleviating world model collapse. The first row presents the future BEV features supervised using real future data, while those in the second features in the second row show more diversity in spatial distribution.
predicted by the world model equipped with FGTR module. The BEV
Temporal Residual World Model In Tab we compare the performance of TR-World and the normal world model. The impact of using real future data to supervise the prediction of the world model also evaluated. It can be found that TR- FRrWorld, which takes temporal residuals as input and focuses dynamic object modeling, normal world model, thereby achieving higher planning accuracy. Furthermore, the sparse spatial- temporal supervision effect of FGTR module enables TR-World to predict scene information for future time period, instead of being limited to the scene at timestamp t+1. Therefore, if the data time t+1 is used for future supervision, it will instead cause the future BEV representation to lose richer temporal information, leading to a decrease in planning performance. In contrast, the normal world model devotes most of its efforts redundant static object modeling. leading less accurate predictions of dynamic objects. This explains why future supervision has little impact on the performance of the normal world model.
predict more accurate futurre BEV features than the
Future-Guided Trajectory Refinement To verify the impact of FGTR module in alleviating world model collapse, we visualize the future BEV features predicted by the world model and present them in Fig 4 It can be observed that for the world model without FGTR module, the predicted future BEV features across different driving scenes show little difference and fail to exhibit complete spatial information. In contrast, through the interaction between prior trajectories and the predicted future BEV features at specific spatial points, FGTR module can urge the world model to predict accurate spatial information, thereby effecttively preventing the world model from collapsing.
Performance of Prior Trajectory We also evaluate the metric of the prior trajectory and show the results in Tab It be observed that although the model structure used for generating prior trajectories is the same as the baseline, the prior trajectories have achieved a significant accuracy improvement compared with the baseline. This is because BEV features of the scene are effectively optimized by TR-World and FGTR modules, thereby enhancing the planning capability of the base model. This also proposes new approach of utilizing larger-scale TR-World and FGTR modules during training to obtain the best BEV features, while taking prior trajectories as output for higher efficiency during inference.
Table 5: Performance of Prior Performance. The prior trajectory is predicted using the same model architecture as that of the baseline, while the prediction of the final trajectory requires the TR-World and FGTR models.
L2 (m) J
Collision Rate (5)
Trajectory
1s
2s
3s
Avg
1s
2s
Avg
Baseline
0.21
0.55
1.18
0.65
0.02
0.12
0.70
0.28
Prior Trajectory
0.20
0.53
1.12
0.61
0.02
0.10
0.41
0.18
Final Trajectory
0.19
0.50
1.08
0.59
0.02
0.06
0.43
0.17

---

Published as a conference paper at ICLR 2026
CAML FRONILEED LES
Gwn
GAM LEAK
FRONT RISET
BEV Oha Ing
BEY
GAML CAILBAGK LEFT
BAKLBACK
CANL BAI_BACKJJOOT RIGHT
Figure 5: Visualization of Planning Results. The object bounding boxes and lane lines on the BEV plane are rendered using the annotations. The green box denotes the ego vehicle. The areas enclosed by dashed circles indicate where collisions will occur.
4.5 VISUALIZATION
We compare the qualitative results of ResWorld with SSR (Li & Cui 2025) planning trajectories in Fig 5 It can be observed that the trajectories predicted by our method can effectively avoid collisions with other vehicles curbs, demonstrating stronger scene understanding ability.
CONCLUSION
Our proposed WesWorld adopts novel Temporal Residual World Model framework. It captures information about dynamic objects by calculating the temporal residuals of scene representations. This allows the world model driven by temporal residual focus explicitly on forecasting the future spatial distribution of dynamic objects, eliminating the redundant modeling of static objects. Fur- thermore. through the Future-Guided Trajectory Refinement module, the predicted future BEV fea- tures are utilized to correct prior trajectories, thereby reducing the probability of driving accidents. The spatial-tent temporal interaction between prior trajectories and future BEV features also serves as sparse supervision on the latent world model, effectively alleviating the model collapse. ResWorld achieves state-of-the- planning performance on both nuScenes and NAVSIM benchmarks.
Limitations and Future Work While TR- FR-World demonstrates greater sensitivity to subtle move- ments of dynamic objects than existing world models (e.g., SSR, LAW), cannot adequately capture potential dynamic objects (e.g. pedestrians and parked cars) through temporal residuals. As a result, such objects can only be processed alongside static objects by the prior trajectory prediction branch. perception to extract the information of poten-
Our future work will focus on how tial dynamic objects from the scene and perform preventive modeling for them. This will further enhance the safety of the planning results predicted by framework.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This research was supported by Zhejiang Provincial Natural Science Foundation of China under Grant No. LD24F020016 and National Natural Science Foundation of China under Grant No. 62576023.
REFERENCES
Frédéric Bouchard, Sean Sedwards, and Krzysztof Czamecki. A rule-based behaviour planner for autonomous driving. In International Joint Conference on Rules and Reasoning, pp. 263-279.
Springer, 2022.
Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. In CVPR, 2020.
Holger Caesar, Juraj Kabzan, Kok Seang Tan, Whye Kit Fong, Eric Wolff, Alex Lang, Luke Fletcher, Oscar Beijbom, and Sammy Omari. nuplan: closed-loop ml-based planning benchmark for autonomous vehicles. arXiv preprint arXiv:2106. 11110, 2021.
Shaoyu Chen, Bo Jiang, Hao Gao, Bencheng Liao, Qing Xu, Qian Zhang, Chang Huang. Wenyu Liu, and Xinggang Wang. Vadv2: End-to-end vectorized autonomous driving via probabilistic planning. arXiv preprint arXiv:240..13244 2024.
Jie Cheng, Yingbing Chen, and Qifeng Chen. Pluto: Pushing the limit of imitation learning-based planning for autonomous driving. arXiv preprint arXiv:2404. 212224444..4.... 2024a.
Jie Cheng, Yingbing Chen. Xiaodong Mei, Bowen Yang, Bo Li, and Ming Liu. Rethinking imitation- based planners for autonomous driving. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 14123-14130. IEEE, 2024b.
Kashyap Chitta, Aditya Prakash, Bernhard Jaeger, Zehao Yu, Katrin Renz, and Andreas Geiger. Transfuser: Imitation with transformer- based sensor fusion for autonomous driving. IEEE trans- actions on pattern analysis and machine intelligence, 45(1 01111288888299 2022.
OpenScene Contributors. Openscene: The largest up-to-date 3d occupancy prediction benchmark in autonomous driving. In Proceedings of the Conference on Computer Vision and Pattern Recogni- tion, Vancouver, Canada, pp. 18-22,20 2023.
Daniel Dauner, Marcel Hallgarten, Tianyu Li, Xinshuo Weng, Zhiyu Huang, Zetong Yang. Hongyang Li, Igor Gilitschenski, Boris Ivanovie, Marco Pavone, al. Navsim: Data-driven non-reactive autonomous vehicle simulation and benchmarking. Advances in Neural Information Processing Systems, 37:28706 220000222999..1 2024.
Mingzhe Guo, Zhipeng Zhang, Yuan He, Ke Wang, and Liping Jing. End-to-end autonomous driving without costly modularization and 3d manual annotation. arXiv preprint 47X1v:2406.17688 2024.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. CVPR, 2016.
Shengchao Hu, Li Chen, Penghao Wu, Hongyang Li, Junchi Yan, and Dacheng Tao. St-p3: End-to- end vision-based autonomous driving via teaporlltemporal feature learning. In European Confer- ence on Computer Vision, pp. 533-5443- Springer, 2022a.
Shengchao Hu, Li Chen, Penghao Wu, Hongyang Li, Junchi Yan, and Dacheng Tao. St-p3: End-to- end vision-based autonomous driving via spatial-temporal feature learning. In ECCV, 2022b.
Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tian- wei Lin, Wenhai Wang, Lewei Lu, Xiaosong Jia, Qiang Liu, Jifeng Dai, Yu Qiao, and Hongyang Li. Planning-orientee autonomous driving. In CVPR, 2023.
Junjie Huang and Guan Huang. Bevdet4d: Exploit temporal cues multi-camert 3d object detec- tion. arXiv preprint arXiv:2203.1 17054, 2022.
11

---

Published as a conference paper at ICLR 2026
Junjie Huang, Guan Huang, Zheng Zhu, and Dalong Du. Bevdet: High-performance multi-camen
3d object detection in bird-eye-views view. arXiv preprint arXi:2111..1190, 2021.
Zhiyu Huang, Haochen Liu, and Chen Lv. Gameformer: Game-theoretic modeling and learning of transformer-based interactive prediction and planning for autonomous driving, In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3903-3913, 2023.
Bo Jiang. Shaoyu Chen, Qing Xu, Bencheng Liao, Jiajie Chen, Helong Zhou, Qian Zhang, Wenyu Liu, Chang Huang, and Xinggang Wang. Vad: Vectorized scene representation for efficient au- tonomous driving. ICCV, 2023.
Peidong Li and Dixiao Cui. Navigation-guide sparse scene representation for end-to-end au- tonomous driving. In International Conference on Learning Representations (ICLR), 2025.
Qi Li, Yue Wang, Yilun Wang, and Hang Zhao. Hdmapnet: An online hd map construction and evaluation framework. In ICRA, 2022a.
Yingyan Li, Lue Fan, Jiawei He, Yuqi Wang, Yuntao Chen, Zhaoxiang Zhang, and Tieniu Tan. Enhancing end-to-end autonomous driving with latent world model. arXiv preprint arXiv:2406. 240.0888000.8881, 2024a.
Yingyan Li, Lue Fan, Jiawei He. Yugi Wang, Yuntao Chen, Zhaoxiang Zhang, and Tieniu Tan. Enhancing end-to-end autonomous driving with latent world model. In International Conference on Learning Representations (ICLR), 2025.
Zhenxin Li, Kailin Li, Shihao Wang. Shiyi Lan, Zhiding Yu, Yishen Ji, Zhiqi Li, Ziyue Zhu, Jan Kautz, Zuxuan Wu, et al. Hydra-mdp: End-to-end multimodal planning with multi-target hydra- distillation. arXiv preprini arXiv:2406.0060 06978, 2024b.
Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chonghao Sima, Tong Lu, Yu Qiao, and Jifeng Dai. Beyformer: Learning bird's-eye s-eee-view representation from multi-camera images via spa- tiotemporal transformers. In ECCV, 2022b.
Zhiqi Li, Zhiding Yu, Shiyi Lan, Jiahan Li, Jan Kautz, Tong Lu, and Jose M. Alvarez. Is ego status all you need for open-loop end-to-end autonomous driving? In CVPR, 2024c.
Bencheng Liao, Shaoyu Chen, Xinggang Wang, Tianheng Cheng, Qian Zhang, Wenyu Liu, and Chang Huang. Maptr: Structured modeling and learning for online vectorized hd map construc- tion. arXiv preprint arXiv:2208. 2022.
Bencheng Liao, Shaoyu Chen, Yunchi Zhang, Bo Jiang, Qian Zhang, Wenyu Liu, Chang Huang, and Xinggang Wang. Maptrv2: An end-to-end framework for online vectorized hd map construction. International Journal of Computer Vision, pp. 1-23, 2024.
Bencheng Liao, Shaoyu Chen, Haoran Yin, Bo Jiang, Cheng Wang, Sixu Yan, Xinbang Zhang, Xiangyu Li, Ying Zhang, Qian Zhang, et al. Diffusiondrive: Truncated diffusion model for end- to-end autonomous driving. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 12037-12047, 2025.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, In ICLR, 2019.
Qihang Ma, Xin Tan, Yanyun Qu, Lizhuang Ma, Zhizhong Zhang, and Yuan Xie. Cotr: Compact oc- cupancy transformer for based 3d occupancy prediction. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19936-19945 2024.
Michael Ryoo, AJ Piergiovanni, Anurag Arnab, Mostafa Dehghani, and Anelia Angelova. Token- learner: Adaptive space-time tokenization for videos. In NeurlPS, 2021.
Chonghao Sima, Wenwen Tong, Tai Wang, Li Chen, Silei Wu, Hanming Deng, Yi Gu, Lewei Lu, Ping Luo, Dahua Lin, and Hongyang Li. Scene as occupancy. In ICCV, 2023.
Ziying Song. Caiyan Jia, Lin Liu, Hongyu Pan, Yongchang Zhang, Jumming Wang, Xingyu Zhang, Shaoqing Xu, Lei Yang, and Yadan Luo. Don't shake the wheel: Momentum-awe aware planning in end-to-end autonomous driving. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 22432-22444 2025.
12

---

Published as a conference paper at ICLR 2026
Wenchao Sun, Xuewu Lin, Yining Shi, Chuang Zhang, Haoran Wu, and Sifa Zheng. Sparsedrive: End-to-end autonomous driving via sparse scene representation. arXiv preprint arXiv:2405.19620, 2024.
Martin Treiber, Ansgar Hennecke, and Dirk Helbing. Congested traffic states in empirical observa- tions and microscopic simulations. Physical review E, 62(2):1805, 2000.
Shihao Wang, Yingfei Liu, Tiancai Wang, Ying Li, and Xiangyu Zhang. Exploring object-centric temporal modeling for efficient multi-view 3d object detection. arXiv:2303. 1926, 2023.
Yuqi Wang, Jiawei He, Lue Fan, Hongxin Li, Yuntao Chen, and Zhaoxiang Zhang. Driving into the future: Multiview visual forecasting and planning with world model for autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 14749-14755 2024.
Xinshuo Weng, Boris Ivanovic, Yan Wang, Yue Wang, and Marco Pavone. Para-drive: Parallelized architecture for real-time autonomous driving. In CVPR, 2024.
Yu Yang. Jianbiao Mei, Yukai Ma, Siliang Du, Wenqing Chen, Yijie Qian, Yuxiang Feng, and Yong Liu. Driving in the occupancy world: Vision-centric 4d occupancy forecasting and planning via world models for autonomous driving. In Proceedings of the AAAI Conference Artificial Intelligence, volume 39, pp. 9327-9335, 2025.
Zichen Yu, Changyong Shu, Jiajun Deng, Kangjie Lu, Zongdai Liu, Jiangyong Yu, Dawei Yang, Hui Li, and Yan Chen. Flashocc: Fast and memory-effiiien occupancy prediction via channel- to-height plugin. arXiv preprint arXiv:2311.12058 2023.
Chengran Yuan, Zhanqi Zhang, Jiawei Sun, Shuo Sun, Zefan Huang, Christina Dao Wen Lee, Don- gen Li, Yuhang Han, Anthony Wong, Keng Peng Tee, al. Drama: An efficient end-to-end motion planner for autonomous driving with mamba. arXiv preprint arXiv:2408.0300 2024a.
Tianyuan Yuan, Yicheng Liu, Yue Wang, Yilun Wang, and Hang Zhao Streammapnet: Streaming mapping network for vectorized online hd map construction. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 7356-7365, 2024b.
Jinqing Zhang, Yanan Zhang, Qingjie Liu, and Yunhong Wang, Sa-bev: Generating semantic-aware bird's s-eee-view feature for multi-view 3d object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3348 03188-337,,00 2023.
Jinqing Zhang, Yanan Zhang, Qingjie Liu, and Yunhong Wang. Lightweight spatial embedding for vision-based 3d occupancy prediction. arXiv preprint M4Kiv:2412.059722 2024.
Jinqing Zhang, Yanan Zhang. Yunlong Qi, Zehua Fu, Qingjie Liu, and Yunhong Wang. Geobev: Learning geometric bev representation for multi-view 3d object detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 9960-9968. 2025.
Wenzhao Zheng, Weiliang Chen, Yuanhui Huang, Borui Zhang, Yueqi Duan, and Jiwen Lu. Occ- world: Learning 3d occupancy world model for autonomous driving. In European conference on computer vision, pp. 55-72. Springer, 2024a.
Wenzhao Zheng, Ruiqi Song, Xianda Guo, Chenming Zhang, and Long Chen. Genad: Generative end-to-end autonomous driving. In ECCV, 2024b.
Yupeng Zheng, Pengxuan Yang. Zebin Xing, Qichao Zhang, Yuhang Zheng, Yinfeng Gao, Pengfei Li, Teng Zhang, Zhongpu Xia, Peng Jia, et al. World4drive: End-to-end autonomous driving via intention-aware physical latent world model. arXiv preprint arXiv:2507.0060) 2025.
13


Reviews:
[{"review_id": "sFcMo51MBp", "reviewer": "Reviewer_xmSF", "summary": "This paper proposes ResWorld, a world model for end-to-end autonomous driving. It presents two main contributions: 1) The Temporal Residual World Model (TR-World), which separates dynamic and static information by calculating the \"temporal residual\" of BEV features. This allows the world model to focus solely on modeling dynamic objects, avoiding redundant computation on static scenes. 2) The Future-Guided Trajectory Refinement (FGTR) module, which uses the future feature maps predicted by TR-World to explicitly correct and optimize a \"prior trajectory,\" resulting in a safer final plan.", "questions": "1. While the authors state in Sec 4.2 that history frames were omitted for a fair comparison, I believe a closed-loop test of the core innovation (TR-World) is necessary. Can the authors provide closed-loop results for TR-World on NAVSIM to truly validate its contribution?\n2. Regarding Weakness 2, how do the authors ensure the learned BEV features are temporally stable? If the features are unstable, how does the model distinguish between residuals from \"feature jitter\" and residuals from \"true object motion\"?\n3. TR-World treats static-but-soon-to-move objects (like stopped cars and pedestrians) as static background. This is a critical safety failure. How do the authors plan to address this fundamental flaw? \n4. How does the model handle future static road structure? If the world model only predicts dynamic residuals and static information comes from the current BEV, how does the vehicle acquire information about the static road ahead when turning or entering an area not covered by the current field of view?\n5. The paper argues that the lack of future supervision is an advantage, but this may results in uninterpretability. Could the authors discuss the potential benefits of incorporating explicit future supervision, such as future point cloud prediction [1-2] or future BEV maps/occupancy, to provide a more interpretable signal to the world model?\n\n[1] ViDAR: Visual Point Cloud Forecasting. CVPR 24\n[2] HERMES: A Unified Self-Driving World Model for Simultaneous 3D Scene Understanding and Generation. ICCV 25", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The FGTR module establishes an explicit feedback loop between the planner and the world model, using predicted future information to correct the current plan, which is logically clear.\n2. The method achieves promising results on both the nuScenes (open-loop) and NAVSIM (closed-loop) benchmarks.", "weaknesses": "1. A major weakness lies in the experimental evaluation. In the NAVSIM closed-loop tests, the authors admit (Sec 4.2) to not using the core TR-World module, leaving its closed-loop effectiveness unverified.\n2. TR-World's residual calculation is highly sensitive to the stability of the BEV features themselves. If the underlying BEV encoder is unstable between frames, its \"feature noise\" will be conflated with \"true motion,\" leading to an unreliable residual signal.\n3. The core assumption of TR-World (residual = dynamic) has a critical safety flaw. As admitted in Appendix B.3, the method fails to model static-but-soon-to-move objects, such as vehicles waiting at a red light.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0OM1ioWtzF", "reviewer": "Reviewer_PiGQ", "summary": "This paper introduces ResWorld, an end-to-end autonomous driving system centering on a Temporal Residual World Model that selectively models dynamic objects via temporal residuals in BEV representations, reducing redundancy and improving dynamic prediction. The system further proposes a Future-Guided Trajectory Refinement mechanism that explicitly interacts predicted trajectories with future BEV features, aiming to improve planning accuracy and avoid “world model collapse.” Experiments on nuScenes and NAVSIM benchmarks show that ResWorld achieves superior performance.", "questions": "- Why do the collision rate results of SSR reported in this paper differ so significantly from those in the original publication?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- Temporal residuals in BEV separate dynamic from static content, alleviating redundant static region modeling and weak interaction between the world model and trajectories.\n- FGTR explicitly couples predicted trajectories with future BEV features under supervision, improving robustness.\n- Experiments on Nuscenes and NavSim demonstrate the superiority of the proposed methods.", "weaknesses": "- ResWorld utilizes GeoBEV to generate high-quality BEV features. Compared to other methods, especially the baseline SSR, this approach may be considered unfair. Moreover, the resolution of the BEV features and other hyperparameters, such as the number and dimensions of scene tokens, are not clearly defined, making it challenging to practically verify the effectiveness of the proposed method.\n- The analysis and conclusions presented in Table 4 are both confusing and overstated, lacking sufficient experimental evidence. For example, the results in Table 4 mainly show that the impact of future supervision on planning is minimal. This makes it difficult to conclude that TR-World focuses more on dynamic targets or produces more accurate BEV features. Furthermore, the claim that TR-World retains more detailed information about the spatial distribution of dynamic targets is not adequately supported by experimental data.\n- There is a lack of experiments on a closed-loop benchmark.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TrYsodUMJk", "reviewer": "Reviewer_podK", "summary": "This paper proposes a residual-based world model for end-to-end autonomous driving planning. Unlike previous works that predict future ego-centric image frames or BEV features, this approach centers on the current time step and leverages the differences among multiple BEV feature frames to make the model focus more on dynamic object motion while reducing redundant predictions of static elements. A deformable attention mechanism is then used to interact the prior trajectory with the predicted future BEV features to generate the final planning trajectory. The method demonstrates noticeable improvements on the nuScenes and NAVSIM datasets.", "questions": "In line 269, the paper states that “Experiments confirm that not supervising $B_{future}$ enables higher planning performance.” As I understand it, $B_{future}$ is obtained by fusing the current BEV features with the temporally redundant dynamic features from historical frames. It is therefore unclear why $B_{future}$ can “preserve the spatial distribution of dynamic objects across multiple future timestamps.” In my view, since end-to-end autonomous driving is a multi-task framework where different tasks can influence each other, using the ground truth BEV features as supervision provides a very dense supervisory signal, which may actually interfere with the planning performance. Therefore, I doubt the claimed reason may not be the true cause of the observed improvement.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The idea of utilizing the static information from the current frame to avoid redundant predictions of static objects in future frames, thereby focusing more on dynamic targets, is very insightful.\n2. On the recent popular NAVSIM benchmark, the proposed method achieves a noticeable improvement in planning performance compared to previous approaches.", "weaknesses": "1. The statements in the abstract, line 101, and the experimental tables suggesting that the proposed method does not rely on auxiliary tasks are somewhat misleading, since the BEV encoder still requires supervision from detection and mapping tasks during training.\n2. A major weakness of the paper is the lack of evaluation on inference speed. Given that the proposed method introduces multiple designs for feature interaction and adopts a two-stage trajectory generation process, it is unclear whether these additions incur significant computational overhead.\n3. Including ablation studies on the NAVSIM dataset would make the results more convincing. The nuScenes dataset contains a large proportion of straight-driving scenarios, and its metrics are already close to saturation, making it difficult to determine whether the reported improvements come from the proposed design or just statistical variation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)