DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Driving
Authors: Yang Zhou
Hao Shao
Letian Wang
Zhuofan Zong
Hongsheng Li
Steven L. Waslander
Venue: ICLR 2026 Poster (2026)
Keywords: Benchmark
Autonomous Driving
Generative World Model
Primary area: datasets and benchmarks

Abstract: Video generation models, as one form of world models, has emerged as one of the most exciting frontiers in AI, promising agents the ability to imagine the future by modeling the temporal evolution of complex scenes. 
In autonomous driving, this vision gives rise to driving world models—generative simulators that imagine ego and agent futures, enabling scalable simulation, safe testing of corner cases, and rich synthetic data generation.
Yet, despite fast-growing research activity, the field lacks a rigorous benchmark to measure progress and guide priorities. Existing evaluations remain limited: generic video metrics overlook safety-critical imaging factors; trajectory plausibility is rarely quantified; temporal and agent-level consistency is neglected; and controllability with respect to ego conditioning is ignored. Moreover, current datasets fail to cover the diversity of conditions required for real-world deployment.
To address these gaps, we present DrivingGen, the first comprehensive benchmark for generative driving world models. DrivingGen combines a diverse evaluation dataset—curated from both driving datasets and internet-scale video sources, spanning varied weather, time of day, geographic regions, and complex maneuvers—with a suite of new metrics that jointly assess visual realism, trajectory plausibility, temporal coherence, and controllability. Benchmarking 14 state-of-the-art models reveals clear trade-offs: general models look better but break physics, while driving-specific ones capture motion realistically but lag in visual quality.
DrivingGen offers a unified evaluation framework to foster reliable, controllable, and deployable driving world models, enabling scalable simulation, planning, and data-driven decision-making.

Published as a conference paper at ICLR 2026
DRIVINGGEN: A COMPREHENSIVE BENCHMARK FOR GENERATIVE VIDEO WORLD MODELS IN AU- TONOMOUS DRIVING
Yang Zhou 1"
Hao Shao2
Letian Wang!
Zhuofan Zong²
Hongsheng Li²
Steven L. Waslander
University of Toronto
CUHK MMLab
Project Website:
hutps://hivvnngen--bee hewp:/ddviiingeeeebbbbehcec www.h..iihu........
ABSTRACT
Video generation models, as one form of world models, have emerged as one of the most exciting frontiers in AI, promising agents the ability to imagine the future by modeling the temporal evolution of complex scenes. In autonomous driving, this vision gives rise driving world models: generative simulators that imagine ego and agent futures, enabling scalable simulation, safe testing of corner cases, and rich synthetic data generation. Yet, despite fast-growing research activity, the field lacks a rigorous benchmark to measure progress and guide priorities. Ex- isting evaluations remain limited: generic video metrics overlook safety-critical imaging factors; trajectory plausibility rarely quantified; temporal and agent- level consistency is neglected; and controllability with respect to ego conditioning is ignored. Morcover, current datasets fail to cover the diversity of conditions re- quired for real-world deployment. To address these gaps, we present DrivingGen, the first comprehensive benchmark for generative driving world models. Driv- ingGen combines diverse evaluation dataset curated from both driving datasets and internet-scale video sources, spanning varied weather, time of day, geographic regions, and complex maneuvers, with a suite of new metrics that jointly as- sess visual realism, trajectory plausibility, temporal coherence, and controllability. Benchmarking 14 state-of-the-an the-art art models reveals clear trade-offs: general models look better but break physics, while cally but lag in visual quality. DrivingGen offers unified evaluation framework to foster reliable, controllable, and deployable driving world models, enabling scalable simulation, planning, and data-driven decision-make -aaiing..
ones capture motion realisti-
INTRODUCTION
Driven by scalable learning techniques, generative video models have made remarkable progress in recent years, enabling the synthesis of high-fidelity videos across diverse scenes and motions. These models suggest promising path toward "world models" predictive simulators capable of imagin- ing the future, which can support planning, simulation, and decision-making in complex, dynamic environments. Inspired by this vision, there has been an accelerating surge in developing driving world models: generative models specialized for predicting future driving scenarios. Given an ini- tial scene and optional conditions (e.g., text prompts, driving actions), driving world model predicts both the ego-vehicle's 's future movements and the evolution of surrounding agents' trajectories. Such models enable closed-loop simulation and synthetic data generation, reducing reliance on real-world data and offering promising means to explore out-of-distribun
scenarios safely (Gao al. 2024
Hassan et al. 2024 Mousakhan et al. 2025 Li et al. 2025d Wang et al. 2025 Zhou et al. 2025 Driving world models are also tightly coupled with end-to-end autonomous driving systems, where
"qqual contribution.

---

Published as a conference paper at ICLR 2026
errors in predicted future scenes and trajectories directly lead to unsafe decisions Shao et al. 2023a 2024a 2023b
Wang et al. )023).
Evaluation Metries
Method / Benchmark
Distribution
Quality
Temporal Consistency
Aligument
Downstream Task
VBench (Hunn 2023
WorldMousTBeecl C 0 2025a
Instruction
WorldScore (Duan
Traj.
Vista Gao et al. 2024
Visual
Human eval Human
Traj- Traj.
GEM TTissan el 2024
Human eval, Agent
Doe-1 Zheng 3 202416
Visual
vQA, Planning
Drivingilao Wang
2124e
Visual
Traj- Traj.
Driverse (Li a. 20
Visual
UniFuture [L3mn)) 31 2025
Visual
Pereeption
VaViM Barlogeions et 1. 2025
Visual
Segmentation
GAIA-2 Kussel
Visual
Visual, Agent
ReSin Yang al 4043
Visual
Traj.
Plaaning
ACT-Bench Ami
Instruction, Traj.
DrivingGen (OBrs)
Visual, Traj.
Visual, Traj.
Visual, Agent, Traj.
Traj.
Table 1: Comparison of existing video benchmarks, driving world models, and driving video bench- marks. "x" indicates the missing metries, and "V" signifies that the evaluation is comprehensive. "Visual", "Agent" and "Traj." represent evaluation of images or videos, surrounding agents and ve- hicles' trajectories, respectively.
While a vibrant exploration of a wide range of approaches for driving world models is underway, a well-designed benchmark which not only measures progress but also guides research priorities and shapes the trajectory of the entire field- has not yet emerged. Current evaluations fail to fully capture the unique requirements of the driving domain, and are limited in several ways. 1) Visual Fidelity First, most benchmarks rely on distribution-lewe metrics such as Fréchet Video Distance (FVD) to models (e.g.. vision- rision-language mod-
assess video realism, and some adopt numan--reefrrecc-alignne els) to score visual quality or semantic consistency. However, driving imposes unique constraints on imaging: sensor artifacts, glare, or other coruptiinn can have critical safety implications that general video metrics fail to capture. 2) Trajectory Plausibility Second, the ego-motion trajectories underlying the generated videos are crucial. High-quality video generation in driving must pro- duce trajectories that are natural, dynamically feasible, interaction-aware and safe- prot-prreettees that go beyond mere visual realism. 3) Temporal and Agent-Level Consistency Third, temporal consis- tency is crucial for driving, where surrounding objects directly impact safety and mecision-making Prior benchmarks often focus on scene-level consistency but neglect agent-level consistency, such as abrupt appearance changes or abnormal disappearances of agents- inntessspppffeciiin that can severely compromise the realism and reliability of driving simulations. 4) Motion Controllability Finally, for ego-conditioned video generation, it is critical to assess whether the generated motion faithfully follows the conditioning trajectory. This aspect of controllability is largely overlooked in existing benchmarks, yet it is essential for safe planning and reliable closed-loop driving, where misalign- ment can lead catastrophic consequences.
Another major limitation in existing benchmarks for driving world models is the lack of diversity along crucial dimensions essential for real-world deployment. 1) First, Weather and Time of Day coverage is heavily skewed: datasets like nuScenes (Caesar et al. 2020) are dominated by clear- weather, daytime driving, leaving rare but safety-critical conditions (night, snow, fog) underrepre- sented. 2) Second, Geographic Coverage is limited, often confined to a few cities or countries, which restricts evaluation across varied scene appearance and with local traffic rules. 3) Third, Driving Ma- neuvers and Interactions rarely capture the full diversity of agent behaviors and complex multi-agent dynamics, such pedestrians waiting at crosswalks, aggressive driver cut-ins, dense traffic sce- narios (Wang et al. (002T). This lack of diversity makes it difficult to assess whether generative models can handle the wide range of scenarios encountered in real-world driving, undermining their reliability and safety for large-scale deployment.
To address the above gaps. this work proposes DrivingGen, comprehensive benchmark for gener- ative world models in the driving domain with a diverse data distribution and novel evaluation met- rics. DrivingGen evaluates models from both a visual perspective (the realism and overall quality of generated videos) and a robotics perspective (the physical plausibility, consistency and accuracy of generated trajectories). Our benchmark makes the following key contributions:
2

---

Published as a conference paper at ICLR 2026
World Condition
DrivingGen Metrics
Vision
Language
Action
Trajectones
Diser Distribution
Quality
the cameraa movis along
(16.72
(22
Temporal Consistency
Front-View RGB Image
Scone Description
Fig Trijectory
Videce
Trjectory Ai
Generative World Models
Generated Videos
World (sg., Wan)
Models le.s. Casmoo)
Driving World Models (h g., Vietal
Figure Overview of our DrivingGen benchmark. Video models take vision, and optional lan- guage/action as inputs to generate videos. The generated videos are then passed into our evaluation suite. Four comprehensive and novel sets of metries for both videos and trajectories (distribution, quality, temporal consistency, and trajectory alignment) are introduced to evaluate world models.
Diverse Driving Dataset. We present new evaluation dataset that captures diverse driving condi- tions and behaviors. Unlike prior datasets biased toward sunny, daytime urban scenes, ours includes varied weather (rain, snow, fog, floods, sandstorms), times of day (dawn, day, night), global regions (North America, Europe, Asia, Africa, etc.), and complex scenarios (dense traffic, sudden cut-ins, pedestrian crossings). This diversity enables more robust and unbiased evaluation of generative models under realistic driving distributions. Besides, considering that inference for video genera- tion is generally timecoonumiig, we carefully limit the number of samples to 400 to ensure efficient testing and iteration, achieving balance between efficiency and meaningful evaluation.
Driving-Speciffe Evaluation Metrics. We introduce a novel suite of multifaceted metrics specifi- cally designed for driving scenarios. These include distribution-level measures for both video and trajectory outputs, quality metrics that account for human perceptual quality, driving-specific imag- ing factors (such illumination flicker, motion blur, etc.). temporal consistency checking at both the scene level and individual agent level (e.g., appearance discrepancy or unnatural disappearances in videos), and trajectory realism metrics that evaluate kinematic feasibility and aligmment to intended paths (e.g., smoothness, physical plausibility, and accuracy in following given route). Together, these metrics provide a comprehensive 4-dimensional evaluation along distribution realism, visual quality, temporal coherence, and control/trajectrry fidelity covering aspects that generic metrics or single-number scores fail capture.
Extensive Benchmarking and Insights. We benchmark 14 generative world models on DrivingGen spanning three categories - general video world models, physics-based world models, and driving- specialized world models. This evaluation, the first of its kind in the driving domain, reveals impor- tant insights and open challenges. For example, we find that certain general world models produce visually appealing traffic scenes yet break physical consistency in vehicle motion, and some driving- specific models excel in trajectory accuracy but lag image fidelity. By analyzing performance across our metrics, we reveal the strengths and failure modes of each approach, offering insights for future research. All components of DrivingGen -anneetteeettttttttteeeeeeeeeeee to support reproducible research and advance realistic driving simulation.
dataset and evaluation code- are publicly released
RELATED WORKS
In this work, we focus on two primary research areas: generative world models applied to au- tonomous driving and benchmarks for evaluating these models. Due to space constraints, we provide a comprehensive review of the relevant literature, including recent advancements general video generation and specific driving- evaluations, in Appendix A
3

---

Published as a conference paper at ICLR 2026
19g
Day
Night
SunseiiSantii
Micdle NorthAhhic
12.6%
6.7%
13.3%
23.4%
PM
51.35
Porceriage
12.5%
Porconiage
26.75
53.0%
20.5%
Nuscomas tra0
Osand'v wa) Weather Distribution
Ous
NuSseries (Na)
OpenDV tvll
OuFS
Nu5ooric Ival)
OpendVv [va)
Dur
Time Distribution
Distribution
(a) Weather, time of day, and region distribution between existing datasets and ours.
(b) Specific driving locations inside each region of our dataset.
18
10 33
(c) Representative examples in benchmark, which diverse scenarios such dense city traffic night, unusual weather (e.g., fog, flood, sandstorm), and complex interactions (e.g.. waiting for pedestrians, agents cutting in).
Figure 2: Dataset distribution and gallery in our benchmark (top to bottom).
4

---

Published as a conference paper at ICLR 2026
3 DRIVINGGEN BENCHMARK
The goal of DrivingGen is to establish a comprehensive benchmark to evaluate generative world models under driving-specific constraints and criteria. To achieve this, the proposed benchmark includes several key components: 1) a carefully collected dataset that is diverse in weather, time of day, regions (and their driving styles), and driving maneuvers to support reasonable evaluation; 2) multifaceted metrics that not only evaluate the video quality from a general visual perspective (e.g., appearance), but also from a driving and roboties perspective (e.g., the physical feasibility of trajectories). To showcase the distinguishing capability of DrivingGen, we evaluate general world models, physics-based models, and driving-specifie models. An overview is given in Fig. T with Bec.3.1pm
dataset details in Sec. I and metrics in Sec. 3.2
3.1 BENCHMARK DATASET
Generative video models, as a form of world models, offer a promising way to anticipate future driv- ing scenarios, simulate rare or safety-critical events, and ultimately support planning and decision- making. However, real-world driving unfolds under highly variable conditions, encompassing dif- ferent weather, lighting, regions, and complex maneuvers. Therefore, evaluating generative models across diverse scenarios is crucial ensure their robustness and reliability. To this end, the majority of existing works (Gao et al. [024, Hassan et al. [2024] Liang et al.) [023] [Wang et al.) 2024e, Bar-
toccioni et al., 2025] Wang et al. 2004bb) in driving world models mainly utilize nuScenes (Caesar
al. (020)) and OpenDV (Zheng et al. 2024b) datasets for evaluation. However, the diversity of weather, region, time of day, and driving maneuvers in these datasets is limited and highly biases the data distribution. For example, as shown in Fig. 2a over 80% of the nuScenes validation data and 90% of the OVenDV validation data are collected during normal sunny daytime conditions. Addi- tionally, the data are collected from limited number of vehicles and locations, which further limits the comprehensiveness. Based on this observation, we curated a significantly more diverse dataset. An overview of our dataset is presented in Fig. 2a and Fig. 2b
Dataset Construction. We organize our dataset into two complementary tracks, offering distinct perspectives for evaluating driving videos.
Open-Domain Track is designed to evaluate models' generalization to open-domain, diverse, unseen driving scenarios. We construct this track using Internet-sourced data spanning multiple cities and regions worldwide, ensuring broad coverage beyond the training distribution.
Ego- Cop-CCodiiiooedd Track complements the open-domain track. While the open-domain setting evaluates generalization to diverse unseen scenarios. does not verify whether the generated trajectories follow a specified conditioning trajectory property that is critical for robotics and self-driving applications. The ego-conditioned track therefore focuses on trajectory controlla- bility, measuring how well the trajectories derived from generated vidcos align with the given ego-trajectory instructions. The ego trajectory is optional for model input and only provided in this track. To construct it, we aggregate data from five open-source driving datasets: Zod (Al- lieiii et al. 2023) (Europe), DrivingDojo (Wang et al., 2024e) (China), COVLA (Arai et al., 2025) (Japan), nuPlan (Karcchanachari et al. 20244) (US), and WOMD (Sun et al. 2020 (US).
Each data sample in the dataset consists of three components: front-view RGB image (vision), scene description (language), and an optional ego trajectory (action). For each scene, we employ Qwen (Bai et al. 2025) to capture descriptions of the future dynamics and camera movements within the scene. Given the time-consuming nature of video generation, we limit the number of samples for efficient testing and iteration, while ensuring quality and diversity. The dataset includes 400 samples-20 200 per track-striking a balance between efficiency and meaningful evaluation.
Balanced Data Dsitribution The overall distribution of our dataset, along with gallery of repre- sentative video examples, is shown in Fig. 2 To ensure meaningful evaluation, we explicitly control diversity across several dimensions:
Weather and Time of Day. Existing benchmarks are often dominated by, if not fully composed of, normal weather and daytime conditions. In contrast, our benchmark aims for a more bal- anced distribution. For the open-domain track, we limit normal weather and daytime clips to below 60% and increase the proportion of other conditions, such snow (13.1% %), fog (12.6%),
5

---

Published as a conference paper at ICLR 2026
Distribution
Quality
Temponal Consistency
Trajedoory Aligument
2. Frichet Trjectory Distance (FTD)
1. Frécher Video Distance (FVD)
2. Objecrive Image Quality
1. Subjective Image Quality
Ageni Disapeerrane Comistincc
Agent Consistency
Video Consisiency
2. Dynamic Time Warping (DTW)
1. Avenge Displacement Eroor (ADE)
3. Trajectocy Quality
Trijectory Consisencc
Table 2: Overview of metrics utilized in DrivingGen. Definition and details are in Sec. 3.2
and night/sunset/sunrise driving (50%), to ensure more comprehensive evaluation. Extreme events, including sandstorms, floods, and heavy snowfall at night, are also included. A similar strategy is applied to the ego-conditioned track, where normal weather/daytime clips make up 60% of the data, while the remainder covers diverse conditions to support trajectory controlla- bility evaluation across different scenarios.
Geographic Coverage. Prior benchmarks are often limited to a small number of cities or coun- tries, restricting the diversity of driving scenarios. For the open-domain track, we collect data from wide range of regions worldwide, including North America (20.7%), East Asia & Pa- cific (22.1 1)), Europe & Central Asia (26.6%), the Middle East & North Africa (12.1%), Latin America Caribbean (6.3%), South Asia (6.8%) and South-Saharan Africa (5.4%), to ensure broad geographic coverage. For the ego-conditioned track, data are drawn from existing datasets covering North America, Asia and Europe, providing diverse driving scenarios to evaluate ego- trajectory alignment and controllability.
Driving Maneuvers and Interactions. Capturing diverse driving behaviors and multi-agent inter- actions is critical for evaluating generative world models. For the open-domain track, scenarios include complex interactions such waiting pedestrians at crosswalks, other agents cutting in, and dense traffic, testing the model's understanding of the driving world. For the ego-conditioned track, scenarios are similarly diverse, emphasizing multi-agent interactions and challenging con- ditions to evaluate controllability and alignment with ego-trajectory instructions.
3.2 BENCHMARK METRICS
For all video models, DrivingGen metrics cover three key dimensions: distribution, quality, and temporal consistency, evaluated for both videos and trajectories. We extract trajectories using
standard PnP method within SIFT and RANSAC scheme (Lowe 2004 Fischler & Bolles 1981
Kneip et al., (011) and UniDepth V2 (Piccinelli et al., (025). We provide the details of our SLAM
pipeline (Mur-Artal et al. 2015 Schönberger & Frahm 2016 Teed & Deng 2022 Qu al.
2024
including guaranteeing that all videos reconstruct trajectories and a discussion to compare other benchmarks' trajectory reconstruction methods in Appendix [B.2] For models conditioned on ego trajectories, we include a fourth dimension: trajectory alignment, measuring adherence to the input. Table lists the metrics, grouped into four categories detailed below, each targeting different aspect of video fidelity.
3.2.1 DISTRIBUTION
How far is the generative distribution from the data distribution? A common practice is measure Fréchet Video Distance (FVD) (Unterthiner et al. 2019) on generated videos. However, our key in- sight is that video quality is not solely determined by visual realism-equally important, especially for self-driving and embodied agents, is the realism of the induced ego-motion. Focusing only on vi- sual fidelity gives an incomplete picture. Therefore, evaluate distributional closeness across both videos and trajectories, capturing complementary perspectives from visual perception and robotics.
For the video distribution, we utilize FVD quantify the similarity between generated videos and real videos. Specifically, we follow the standardized computation protocol from the original StyleGAN-V (Skorokhodov al.) 2022). For the trajectory distribution, we introduce novel metric, Fréchet Trajectory Distance (FTD), a distributional metric tailored for evaluating driving trajectories. The key requirement is trajectory encoder that maps trajectories into latent space suitable for measuring distributional distance. To this end, we draw from the motion prediction doeain-where models themselves are generative of future trajectories, and adopt the encoder of Motion Transformer (MTR) (Shi et al. 2023) as our encoding model. Details of FTD computation provided in Appendix B.3

---

Published as a conference paper at ICLR 2026
3.2.2 QUALITY
How good are the generated videos and trajectories? To evaluate the fidelity of generated videos and trajectories in driving scenarios, we propose a comprehensive quality suite covering three aspects: perceptual video quality, domain-specifie video quality, and trajectory quality.
Visual Quality. A common practice in generative video evaluation is to assess general perceptual quality with automatic, reference-free estimators aligned with human judgments. Specifically, we adopt CLIP-IQA+ (Wang et al. 2022), which leverages CLIP's vision-language representations to predict perceptual quality scores consistent with human subjective assessments. While effective, such subjective perceptual quality does not always align with what matters for driving, which un- folds outdoors, involves multiple agents, and occurs under real-world constraints. To additionally consider driving-specific imaging quality, we further adopt the Modulation Mitigation Probability (MMP) metric from the IEEE Automotive P2020 standard (Group et al. 2018: 996 2022). MMP targets Pulse- Width Modulation (PWM)-induced flicker that can disrupt perception and tracking, and reports the fraction of time windows where residual temporal luminance modulation falls below small threshold. Implementation details are in Appendix B.44
While prior evaluations often rely on video-based scores, they typically neglect whether the underlying motions are physically and kinematically plausible. To reduce the gap. Driv- ingGen introduces a composite, reference-free free metric assess the kinematic plausibility and ride comfort. Three individual submetrics are proposed and aggregated into single score: 1) comfort score penalizes extremes of longitudinal jerk, lateral acceleration, and yaw-rate, yielding a score to reward smoother, more confortable motion; 2) motion score that discourages under-mobility, as some trajectories barely move and stay static due to the model's weak ability: 3) a curvature score summarizes how much the path turns, discouraging zig-zags and unrealistically sharp bends. To- gether, these submetrics directly target properties that affect controllability, planning, and perceived comfort. Calculation details appear in Appendix B.5
Trajectory Quality.
3.2.3 TEMPORAL CONSISTENCY
How temporally consistent is the generated world? We assess the temporal consistency of both videos and trajectories. For videos, we evaluate scene-level consistency, agent-level consistency, and explicitly emphasize abnormal agent disappearance. For trajectories, we measure the consistency of speed and acceleration over time, independent of path shape and absolute mobility.
Video Consistency. Existing metrics directly calculate the consistency between consecutive frames (or each frame to the first) at a fixed rate. However, is easily hackable by generating near-static videos. To measure temporal consistency while accounting for the actual motion in the scene, we first pass the generated videos through an off-the-shelf optical flow model (Wang et al. 2024d) to compute the median optical flow magnitude per frame. We then adaptively downsample: videos with lower motion are sampled more sparsely so that the per-step displacement becomes comparable to hormal/high-speed driving. After this, the similarity of the DINOv3 (Siméoni et al., (025) features between consecutive frames of the downsampled videos is reported as the video consistency score. Unlike fixed-stride metrics, our approach fairly measures temporal consistency across videos with varying motion speeds, preventing static or near-static videos from obtaining artificially high scores.
Agent Appearance Consistency. Measuring only scene-level features can overlook small temporal changes in individual agents, such as shifts in color, texture, or shape, while these agents are often the key focus for driving, as they would more directly impact driving behavior and safety. To measure the agent's temporal consistency, we therefore detect agents in the first frame, track them across the video, crop their bounding boxes, and compute consistency purely at the agent level. We use YOLOv10 (Wang al. 2024a) as the detector and SAM2 (Ravi et al. 2024 Yang al. 2024a) for tracking. We measure DINOV3 feature similarity across consecutive frames and to the first frame.
Agent Abnormal Disappearance. In addition to appearance stability, agents in driving scenes must persist in a physically plausible manner. Sudden, non non-physical disappearances of surrounding agents commonly observed in generated videos, which can compromise realism and safety. DrivingGen quantifies this by diagnosing whether an agent's ' disappearance is normal (e.g., leaving the field of view or being occluded) abnormal. We consider three key frames for each disappearing agent:
the first and the last frames where the agent is visible, and the first frame after it vanishes. A vision
large language model (VLM) (Bai et al. 2025 Shao et al. 2024b Liu et al. 2024
Zong et al.,
7

---

Published as a conference paper at ICLR 2026
2025b a). Cosmos- Reasonl (NVIDIA et al. 2025 is prompted to judge disappearance based on visual and motion continuity, and the agent's 's local interactions with surrounding agents. We report the percentage of videos with no abnormal disappearances as the score. Implementation Details can be found in Appendix
2024b Li et al. 2024| Qu et al.
Trajectory Consistency. Realistic driving exhibits predictable kinematies: speed varies slowly around a cruise level and acceleration does not oscillate. To reveal this property, we compute how stable trajectory's 's velocity and acceleration are over time. The average of the two scores is taken as the overall trajectory consistency score. Trajectories that jitter, stop-go, or oscillate score low, while steady cruising with gradual changes scores high. Calculation details are provided in Appendix B.7
3.2.4 TRAJECTORY ALIGNMENT
In addition to trajectory consistency, the alignment of the trajectories underlying the generated videos with the conditioning (ego) trajectory is also critical, especially for rinjectory-groundee video generation. To assess this, we propose two complementary metrics.
Average Displacement Error (ADE). As a common practice, ADE measures the mean pointwise distance between the generated and input trajectories across the prediction horizon. emphasizes local, step-by-step fidelity and is standard in motion prediction and planning.
Dynamic Time Warping (DTW). In addition ADE, which compares trajectories each time step, we introduce a complementary metric that captures the overall contour and shape of the tra- jectory. Specifically, DTW (Keogh & Pazzani 2000) aligns predicted and reference trajectories via non-linear time warping and measures their path-shape discrepancy using Euclidean point-wise cost.
EXPERIMENTS
Evaluation Setup. We evaluate 14 competitive generative world models on DrivingGen, spanning three categories. 1) First, we include 7 general video world models, comprising two commercial closed-source models, Gen-3 [Runway] 2024.06) and Kling (Kuaishou) 2024.06). and five well- known open-source models: CogVideoX (Yang et al., 2024e). Wan (Wan et al.] 2025). Hunyuan- Video (Kong et al.) 2024), LTX- Video (HaCohen et al., 2024a). and SkyReels (Chen et al. 2025).
2) Second, we evaluate 2 physical world models that are developed specifically for the physical robotics domain, Cosmos-Predictl (Agarwal et al., 2025) and Cosmos-Predict2 (Cosmos) 2025). 3) Third, we assess 5 driving-specific world models: Vista (Gao et al.] 20024). DrivingDojo (Wang jet al. 2024e), GEM (Hassan et al.) 2022), VaMiM (Bartoccioni et al. 2025), and UniFuture (Liang jet al. 2025). All models are evaluated on prediction horizon of T00 frames. We report the time
and resource cost for our DrivingGen benchmark in Appendix B.8
4.1 OBSERVATIONS AND CHALLENGES
Table 3 presents the results. We provide the full table of metrics in a transparent way to evaluate the models comprehensively, and the average rank serves as a quick summary but not definitive score. We also show that our results align well with human judgement, by calculating the Spearman's 's correlation coefficient (see details in Appendix (.9)) In the following, we will discuss key findings from our results.
Closed-source models lead in visual quality and overall ranking. Across both tracks, closed- source models consistently occupy the top positions, achieving strong perceptual scores and main- taining stable agent behavior. They rarely exhibit abnormal object disappearance and generally preserve scene coherence over time, demonstrating robust overall world generation capabilities.
Top open-source general world models competitive on specific metries. Several open- source models approach or match the closed-source leaders on individual dimensions. For example, CogVideoX and Wan achieve strong video distributional realism (low FVD) across both tracks, sug- gesting that open-source models can excel in targeted aspects even if they do not lead overall.
No single model excels in both visual realism and trajectory fidelity. We observe distinct "per- sonas": some models achieve high visual quality but only moderate trajectory adherence and per- agent consistency, while driving-speecalizeed models accurately follow commanded paths with phys-
8

---

Published as a conference paper at ICLR 2026
Open- Domain Track
Distribution
Quality
Temporal Consistency
Avg. Rank
Subjective
Objective
Trajectory
Video
Agent
Agent Missing
Trajectory Consist
Models
Size
FVD
FTD
Quality
Quality
Quality
Consist
Consist
Kling 2.1
693.4
26.73
0.5838
0,8018
0.6438
0.8945
0.7981
0,9442
0.5377
Gen-3 Alpha Turbo
801.0
93.50
0.5456
0.8378
0.6535
0.8900
0.8170
0,9495
0.4788
2
LTX-Video
13B
648.2
31.29
0.5215
0.8288
0,5562
0.8851
0.7449
0.8977
0,4517
3
Wan2.2-12V
14B
609.0
63.86
0.5348
0.6396
0,5983
0.8883
0.7514
0,9128
0,4639
4
HlunyaanVidco-22
13B
957.5
30.95
0.4921
0.7207
0,4613
0.8821
0.8008
0.9306
0.4157
JkyReels-V2-122
14B
876.0
52.93
0,5134
0.7432
0,4799
0.8776
0.7329
0,9078
0,4326
7
Cog CigVid
5B
621.2
236.7
0,4932
0.6802
0,3856
0.8211
0.7581
0.7661
0.2949
12
Cosmos- Predice22
14B
524.1
83.20
0.4931
0.7568
0.5990
0.8597
0.5912
0.8657
0.3997
8
Cosmos- Predict
14B
821.1
81.22
0,5083
0.7207
0.2723
0.8429
0.6789
0.8796
0.2631
13
Vista
2.5B
675.7
54.66
0,4340
0.8468
0,6030
0.8565
0.6357
0,8211
0,4040
VaNiM
1.28
1446.6
449.2
0,4691
018468
0,3118
0.9159
0.7721
0.9752
0.0914
9
UniFuture
3.0B
774.3
50.66
0.4206
0,9054
0.4507
0.8799
0.5373
0.8310
0.3858
10
GFM
2.1B
770.1
147.1
0.5168
0.8423
0.5398
0.8176
0.6099
0.7788
0.3392
11
Drivingdojo
2.3B
810.4
126.74
0.4202
0.8333
0.4511
0.8480
0.6256
0.8303
0.2739
14
Ego- Conditioned Track
Distribution
Quality
Temporal Consistnncy
Trajectory Aligument
Models
Size
FVD
FTD
Subjective
Quality
Objective Quality
Trajectory Quality
Consist
Video
Consist
Agent
Agent Missing
Trajectory Consist
ADE
17TW
Aug. Rank
Kling 2,14
320.5
23.74
0.5468
0,7838
0,6860
0.8929
0,8186
0,9712
0,5430
29.97
2310
1
Gin-3 Alpha Turbo
555.9
24.72
0.5740
0.8604
0,8747
0.7986
(13466
0.4800
33.39
2749
Wan2. 2-12V
148
194,4
29.56
0.5084
0.6082
0.6419
018921
0.7561
0.9354
0.4849
27.39
1901
2
LTX. Video
138
378.1 1
61.09
0.4895
0.8604
0.5464
0.8705
0,7708
0.9020
0.4442
52.12
2505
6
Hunyuan V2V
13B
532.9
21.18
0.4741
0.6847
0,5542
0.8792
0.8240
0.4155
0.4771
33.80
2794
7
CogVideoX
307.
166.6
0,4884
0.6937
0.4252
0.8667
0.754]
0.881
0.3783
32.67
2413
V22-
148
428.2
57.02
0.4764
0.6622
0.5028
0.8661
0.7208
0.875
0.4322
31.54
2594
11
Cosmos-Prodist2
14B
260.5
56.26
0:1756
0,8198
0.6124
0.8-288
0.6707
0,8986
0:1108
22.38
1490
4
Predis-reddittl
345.2
34.96
014783
0,7505
0.3761
0,8229
0.7423
0.7961
0.3343
34.47
13
Vista
2.5B
392.8
27.33
0.4146
0.8198
0.6047
0.8741
0.6417
0.8676
0.4366
19.70
1216
5
UniPue
3.08
654.6
37.17
0.4000
0,9685
0.5353
0187599
0,5525
018759
0.4165
20.21
1352
8
VaVIM
1.28
1222
103.6
0,4910
0.8094
0.1936
0.9428
0.8290
0.9725
0.0984
41.92
3863
9
Drivingdojo
586.5
35.73
0,4264
0.4131
0.8419
0.6940
0.8439
0.2776
25.50
GEM
2.18
579.9
97.70
0.4484
0.8018
0.5085
0.7886
0.6180
0.7463
0.2983
25.73
1982
14
Table 3: Evaluation results of 14 generative world models on our benchmark. Best results are in red region, second best are in orange region, and third best are in blue region. "*" indicates commercial closed-source models. Models fall into four categories: closed-source, open-source general video models, physical-wond world models, and driving-specific models.
ically plausible motion (low ADE/DTW) yet underperform in visual fidelity, exhibiting noticeable artifacts. Currently, no model successfully combines strong photorealism with precise, physically consistent motion, highlighting a key frontier for driving world generation.
Trajectory alignment remains limited, revealing substantial gaps. Under ego-trajectory con- ditioning, models exhibit significant ADE/DTW errors, indicating poor adherence to commanded paths. This can stem from two main factors: 1) artifacts in the generated videos (e.g.. texture repeti- tion, blur, unstable geometry) that impair SLAM trajectory recovery, and 2) imperfect motion generation, where the model itself fails to follow the intended trajectory. These observations high- light that both video fidelity and trajectory modeling need further improvement.
DrivingGen exposes failure modes hidden from prior single metric. Existing benchmarks often rely solely on distribution-levt metrics such as FVD to evaluate generated driving videos. While useful for assessing overall distribution similarity, good FVD/FTD alone does not necessarily imply plausible driving-videon can appear distribution-close yet exhibit stop-go jitter, identity drift, or non-physical disappearances. Similarly, high objective quality (e.g.. low flicker) can coexist with poor subjective quality or unstable agent behavior. By jointly reporting distribution, perceptual quality, temporal consistency, and trajectory alignment, DrivingGen exposes these hidden failure modes and highlights precisely where each model falls short.
CONCLUSION
This work introduces DrivingGen, comprehensive benchmark designed to evaluate generative world models for autonomous driving. DrivingGen integrates a diverse dataset spanning varied weather, time of day, global regions, and complex driving maneuvers with multifaceted metric

---

Published as a conference paper at ICLR 2026
suite that jointly measures visual realism, trajectory plausibility, temporal coherence, and control- lability. By benchmarking a broad spectrum of state-of-thh-arr models, DrivingGen reveals critical trade-off's among visual fidelity, physical consistency, and controllability, providing clear insights into the strengths and limitations of current approaches. The benchmark establishes a unified and reproducible framework that can guide the development of reliable and deployment-ready driving world models, fostering progress toward safe and scalable simulation, planning, and decision mak- ing in autonomous driving.
6 FUTURE WORK AND LIMITATIONS
As DrivingGen is the first comprehensive benchmark for generative world models in autonomous driving, several intriguing ideas can be explored further in follow-up work.
Expanding More Meaningful Data. Currently, we collect 400 data samples (from the web and aggregated from existing driving datasets) balance efficiency and practicality, because generating and evaluating videos is resource-intensive. With this limited number, we may not fully cover the long tail of driving scenarios. In future expansions, scaling up the dataset an exciting future direction. As generative models become faster and datasets become more readily available, scaling up to thousands of clips is feasible and will further improve long-tail coverage.
Interactive and Closed-Loop Simulation. Ensuring reliable closed-loop performance (e.g., for safe planning) is crucial for Autonomous Driving, and DrivingGen marking open-loop predictive quality and realism. In the current work, all considered generative video world models are designed for open-loop video generation and no standardized closed-loop world generation framework exists yet. Performing a fair, unified closed-loop benchmark is infeasi- ble this stage. An exciting future direction is to consider closed-loop evaluation for driving world models (e.g., integrating generative models into an interactive simulator like CARLA or combining with closed-loop dataset simulation like Navsim).
step toward that by first bench-
Downstream Tasks Metrics and Enriching data modality. DrivingGen focuses on metrics that directly measure video realism. physical consistency, and controllability in the generated footage it- self. One complementary direction is to incorporate metrics from downstream tasks in Autonomous Driving (e.g.. how well an autonomous driving stack performs using synthetic videos). However, it may require collecting synchronized multi-camera footage and Map knowledge for a fair and meaningful benchmark. Our current dataset is limited poses challenges for more structural driving generation. A possible future direction is expanding the benchmark to multi-view video and sensor data (LiDAR, HD Map, etc.) to construct a more structured driving world generation and novel metrics (e.g., view consistency) can be proposed.
single front-view camera feed, which
Evaluation of Scene Controllability and State Transformation. Evaluating controllability over scene content (e.g.. controlling other agents, road layout in the scene) would be highly useful for autonomous applications. We did not include such metrics in our benchmark because implementing a unified evaluation for different models with scene-level control faces challenges both in model support and dataset complexity. Due to these challenges, we believe it is a great topic for driving world generation which controls scene content and map layout and assessing whether state transfor- mations of the world model are reasonable. One could imagine controlling the presence or behavior of a pedestrian or the configuration of lanes, and checking if the model can follow those constraints.
Counterfactual Reasoning Evaluation. In our current benchmark, we did not explicitly evaluate counterfactual reasoning. The main reason is that DrivingGin focuses on real driving videos. We limited evaluating the scenarios that actually happened. One novel future direction would be counterfactual reasoning evaluation. One can introduce hypothetical events or modifications (like an astronaut horse crossing the road, car jumping off the ground to overtake other agents, and other unrealistic edge cases) and propose new metrics to check whether the model follows this counterfactual generation.
Overall Score. We provide the full table of metrics transparently to evaluate the models, and the average rank serves as quick summary but not definitive score. Exploration of composite, single-index score is an interesting topic, which requires normalized distribution and alignment metrics (e.g., FVD and ADE).
10

---

Published as a conference paper at ICLR 2026
REFERENCES
leee draft standard for automotive system image quality. IEEE P2020/D3, June 2022, pp. 1-284, 2022.
Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chat- topadhyay, Yongxin Chen, Yin Cui, Yifan Ding, et al. Cosmos world foundation model platform for physical al. arXiv preprint arXiv:2501.088.83 03557, 2025.
Mina Alibeigi, William Ljungbergh, Adam Tonderski, Georg Hess, Adam Lilja. Carl Lindström, Daria Motorniuk, Junsheng Fu, Jenny Widahl, and Christoffer Petersson. Zenseact open dataset: A scage-scale and diverse multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 20178 277882200881. 2023.
Hidehisa Arai, Keishi Ishihara, Tsubasa Takahashi, and Yu Yamaguchi. Act-bench: Towards action controllable world models for autonomous driving, 2024. URL http:://aaxivvvoooo abs/
2412. 05337
Hidehisa Arai, Keita Miwa, Kento Sasaki, Kohei Watanabe, Yu Yamaguchi, Shunsuke Aoki, and Issei Yamamoto. Covla: Comprehensive asion-language-actionn dataset for autonomous driving- In Proceedings of the Winter Conference on Applications of Computer Vision (WACV). pp. 1933- 1943, February 2025.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang. Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding. Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang. Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. arXiv preprint arXiv:2502.1 13923, 2025.
Hritik Bansal, Zongyu Lin, Tianyi Xie, Zeshun Zong, Michal Yarom, Yonatan Bitton, Chenfanfu Jiang. Yizhou Sun, Kai-Wei Wei Chang, and Aditya Grover. Videophy: Evaluating physical common- sense for video generation, 2024. URL /ttas:///axiiii org/ htt///bss/2/00 2406. 03520
Florent Bartoccioni, Elias Ramzi, Victor Besnier, Shashanka Venkataramanan, Tuan-Hung Vu, Yihong Xu, Loick Chambon, Spyros Gidaris, Serkan Odabas, David Hurych, Renaud Marlet, Alexandre Boulch, Mickael Chen, Éloi Zablocki, Andrei Bursuc, Eduardo Valle, and Matthieu Cord. Vavim and vavam: Autonomous driving through video generative modeling, 2025. URL
https::/arrx http://aca...0r//////2222...../. 15672
org/ at////////222222
Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators. 2024. URL /
wwtpmaaa..ommeeeeeeeeeeeee www rete/rchs
siide--generaaionnnnne
as world- simulators
Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Osear Beijbom. nuscenes: A multimodal dataset for autonomous driving, 2020. URL htt.:httt://aaxtti
11027
Capcut. Dreamina. http:///wwemmin...ccouuucc..taat
2024.
Guibin Chen, Dixuan Lin, Jiangping Yang, Chunze Lin, Junchen Zhu, Mingyuan Fan, Hao Zhang, Sheng Chen, Zheng Chen, Chengcheng Ma. Weiming Xiong, Wei Wang, Nuo Pang, Kang Kang. Zhiheng Xu, Yuzhe Jin, Yupeng Liang, Yubing Song, Peng Zhao, Boyuan Xu, Di Qiu, Debang Li, Zhengcong Fei, Yang Li, and Yahui Zhou. Skyreels-v2: Infinite-length film generative model, 2025. URL huRllhhttss://axiitt wttp://wwwwgg/////......./ 1307 13074
NVIDIA Cosmos. crrmos--redict2, 2025. cosmos cosmos-predd -ratins-predic2222 Accessed 2025-12-30.
Haoyi Duan, Hong-Xing Yu, Sirui Chen, Li Fei-Fei, and Jiajun Wu. Worldscore: A unified eval- uation benchmark for world generation, 2025. URL fttpss:/aaxxivvoor//bb////
00983

---

Published as a conference paper at ICLR 2026
Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recogni- fion, pp. 12873-12883, 2021.
Fanda Fan, Chunjie Luo, Wanling Gao, and Jianfeng Zhan. Aigebench: Comprehensive evaluation of image-to-video content generated by ai, 2024. URL 01651
Martin A Fischler and Robert C Bolles. Random sample consensus. Commun, ACM, (4()):8113355, June 1981.
Shenyuan Gao, Jiazhi Yang, Li Chen, Kashyap Chitta, Yihang Qiu, Andreas Geiger, Jun Zhang, and Hongyang Li. Vista: generalizable driving world model with high fidelity and versatile controllability. Advances in Neural Information Processing Systems, 37:91560-91596, 2024.
lan J Goodfellow, Jean Pouget-Abadie Mehdi Mirza, Bing Xu, David Warde-Farley Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
IEEE P2020 Working Group et al. leee p2020 automotive imaging white paper. IEEE P2020 Automotive Imaging. IEFE-SA (IEEE, Piscataway, NJ, 2018),(Online.. Available: h ups://www. image-engineering de/content/li b rary/white paper//22200 white paper. pdf. 2018
Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Eitan Richardson, Eran Levin, Guy Shiran, Nir Zabari, Ori Gordon, Poriya Panet, Sapir Weissbuch, Victor Kulikov, Yaki Bitterman, Zeev Melumian, and Ofir Bibi. Lix-video: Realtime video latent diffusion. arXiv preprint arXiv:2501.00103 2024a.
Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Eitan Richardson, Eran Levin, Guy Shiran, Nir Zabari, Ori Gordon, et al. Ltx-video: Realtime video latent diffusion. arXiv preprint arXiv:2501.00/ 2024b.
Mariam Hassan, Sebastian Stapf, Ahmad Rahimi, Pedro M B Rezende, Yasaman Haghighi, David Brüggemann, Isinsu Katircioglu, Lin Zhang, Xiaoran Chen, Suman Saha, Marco Cannici, Elie Aljalbout, Botao Ye, Xi Wang, Aram Davtyan, Mathicu Salzmann, Davide Scaramuzza, Marc Pollefeys, Paolo Favaro, and Alexandre Alahi. Gem: A generalizable ego-vision multimodal world model for fine -rained ego-motion, object dynamics, and scene composition control, 2024. org/ /ttp//212..11199
URL httpt:ttt//aaaat
Dailan He, Xiahong Wang, Shulun Wang, Guanglu Song, Bingqi Ma, Hao Shao, Yu Liu, and Hong- sheng Li. figh-tiielity diffusion face swapping with id-constrained facial conditioning. arXiv preprint arXiv:2503.22 22..2999, 2025.
Jonathan Ho and Tim Salimans. arXiv:2207.122 12598, 2022.
Classifier-free
diffusion guidance.
arXiv preprint
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Anthony Hu, Lloyd Russell, Hudson Yeo, Zak Murez, George Fedoseev, Alex Kendall, Jamie Shot- ton, and Gianluca Corrado. Gaia-l: A generative world model for autonomous driving. arXiv preprint 2rXiv:2309.1 17080, 2023.
Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianx- ing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. Vbench: Comprehensive benchmark suite for video generative models, 2023. URL http:: //tp:://rxxvvvvv org/ 20.1.
Ziqi Huang, Fan Zhang, Xiaojie Xu, Yinan He, Jiashuo Yu, Ziyue Dong, Qianli Ma, Nattapol Chan- paisit, Chenyang Si, Yuming Jiang, Yaohui Wang. Xinyuan Chen, Ying-Cong Chen, Limin Wang. Dahua Lin, Yu Qiao, and Ziwei Liu. Vbench++: Comprehensive and versatile benchmark suite for video generative models, 2024.
2411 13503
12

---

Published as a conference paper at ICLR 2026
Napat Karnchanachari, Dimitris Geromichalos, Kok Seang Tan, Nanxiang Li, Christopher Eriksen, Shakiba Yaghoubi, Noushin Mchdipour, Gianmarco Bernasconi, Whye Kit Fong, Yiluan Guo, and Holger Caesar. Towards leamning-base planning:the nuplan benchmark for real-world au- tonomous driving, 2024. URL httpt:tpp//// /arp:vv///vvvv////aa org/ org/abs/ 2403. 04133
Eamonn J. Keogh and Michael J. Pazzani. Scaling up dynamic time warping for datamining ap- plications. In Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 285-289, 2000.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv:1312.6114 2013.
arXiv preprint
Laurent Kneip, Davide Scaramuzza, and Roland Siegwart. novel parametrization of the perspective-three-poe problem for direct computation of absolute camera position and ori- entation. In CVPR 20/ I, pp. 2969-29766 2976, 2011. doi: 10. 1019//CP..211..999990
Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.0336/1 2024.
Kuaishou. Kling ai. https://klingai..e http://linngiikuuusoo...ooo www.kkuu.iiiio...... 2024.06.
Soonwoo Kwon, Jin- Young Kim, Hyojun Go, and Kyungjune Baek. Toward stable world models: Measuring and addressing world instability in generative environments, 2025. URL https:
//axxiv.org/aba///.
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.0m 03326, 2024.
Dacheng Li, Yunhao Fang, Yukang Chen, Shuo Yang, Shiyi Cao, Justin Wong, Michael Luo, Xi- aolong Wang, Hongxu Yin, Joseph E. Gonzalez, Ion Stoica, Song Han, and Yao Lu. Worldmod- elbench: Judging video generation models world models, 2025a. URL https: //arxiv.
htc//bbs/2502. 20694
Junzhe Li, Yutao Cui, Tao Huang. Yinping Ma, Chun Fan, Miles Yang, and Zhao Zhong. Mixgrpo: Unlocking flow- bosed grpo efficiency with mixed ode-sde, 2025b. URL https://arxiv
akg/abbs25011 21802
Junzhe Li, Sifan Zhou, Liya Guo, Xuerui Qiu, Linrui Xu, Delin Qu. Tingting Long, Chun Fan, Ming Li, Hehe Fan, Jun Liu, and Shuicheng Yan. Unif2 A unified fine- ginener face understanding and generation model, 2025c. URLhttps: arxiv. org/ aabs/25033 08120
Xiaofan Li, Chenming Wu, Zhao Yang, Zhihao Xu, Dingkang Liang, Yumeng Zhang, Ji Wan, and Jun Wang. Driverse: Navigation world model for driving simulation via multimodal trajectory prompting and motion alignment, 2025d. URL httpt::t/axaiii...o..pp org/ abs/
Dingkang Liang, Dingyuan Zhang, Xin Zhou, Sifan Tu, Tianrui Feng. Xiaofan Li, Yumeng Zhang, Mingyang Du, Xiao Tan, and Xiang Bai. Seeing the future, perceiving the future: A unified driving world model for future generation and perception, 2025, URL
org/
abs/2503. 13587
Mingxiang Liao, Hannan Lu, Xinyu Zhang, Fang Wan, Tianyu Wang, Yuzhong Zhao, Wangmeng Zuo, Qixiang Ye, and Jingdong Wang. Evaluation of to generation models: A dynamics perspective, 2024. URL http:: //trvvv.a.iiv.a//aa www.arwwwww...aaaaa.aaaaaoeeeeee abs 2407.0400.00 01094
Dongyang Liu, Renrui Zhang, Longtian Qiu, Siyuan Huang, Weifeng Lin, Shitian Zhao, Shijie Geng, Ziyi Lin, Peng Jin, Kaipeng Zhang, et al. Sphinx-x: Scaling data and parameters for family of multi-modal large language models. arXiv preprint arXiv:2402.05935, 2024.
David G Lowe. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis., 60 (2):91-110, November 2004.
Lumal.abs Dream machine. http:://tumalabs.aaddaer-mmmeco 2024.06.
13

---

Published as a conference paper at ICLR 2026
MiniMax. Hailuo ai. lttps://haiiuucii.com http://hiilooci.ccmmm/ddeed. 2024.09.
Arian Mousakhan, Sudhanshu Mittal, Silvio Galesso, Karim Farid, and Thomas Brox. Orbis: Over- coming challenges of long-horizon prediction driving world models, 2025. URL https:
//arxiv. www.sii.orr//.oo abs/ 2507 130.1....2
Raúl Mur-Artal, J. M. M. Montiel, and Juan D. Tardós. Orb-slam: A versatile and aceu- rate monocular slam system. IEEE Transactions on Robotics, 31(5):1147-1163, 2015. doi:
10.1109/TR0.215..0 STIRR..201..244 2466677....
Munan Ning, Bin Zhu, Yujia Xie, Bin Lin, Jiaxi Cui, Lu Yuan, Dongdong Chen, and Li Yuan. Video-hench: A comprehensive benchmark and toolkit for evaluating video-based large language models, 2023. httpp://////axt
org/ abs 2 16103
NVIDIA, Alisson Azzolini, Junjie Bai, Hannah Brandon, Jiaxin Cao, Prithvijit Chattopadhyay, Huayu Chen, Jinju Chu, Yin Cui, Jenna Diamond, Yifan Ding, Liang Feng, Francesco Ferroni, Rama Govindaraju, Jinwei Gu, Siddharth Gururani, Imad El Hanafi, Zekun Hao, Jacob Huffman, Jingyi Jin, Brendan Johnson, Rizwan Khan, George Kurian, Flena Lantz, Nayeon Lee, Zhaoshuo Li, Xuan Li, Maosheng Liao, Tsung-Yi Yi Lin, Yen-Chen Lin, Ming-Yo Yu Liu, Xiangyu Lu, Alice Luo, Andrew Mathau, Yun Ni, Lindsey Pavao, Wei Ping, David W. Romero, Misha Smelyanskiy, Shuran Song, Lyne Tchapmi, Andrew Z. Wang, Boxin Wang, Haoxiang Wang, Fangyin Wei, Jiashu Xu, Yao Xu, Dinghao Yang, Xiaodong Yang, Zhuolin Yang, Jingxu Zhang, Xiaohui Zeng, and Zhe Zhang. Cosmos-reason From physical common sense to embodied reasoning, 2025. URL https: /ttp:://axxvvvv.
abs 220./2533..0 15558
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 4195-4205, 2023.
Luigi Piceinelli, Christos Sakaridis, Yung- Kung-Hsu Yang, Mattia Segu, Siyuan Li, Wim Abbeloos, and Luc Van Gool. Vni Universal monocular metric depth estimation made simpler, 2025.
abs/ wttp.//rorrrbbb//..2200000 20110
PikaLabs. Pika 1.5. https:/ppika.art// 2024.10.
VerVerse Pixverse. hmtp:://ixxersee.ai/ 2023.
Yiran Qin, Zhelun Shi, Jiwen Yu, Xijun Wang, Enshen Zhou, Lijun Li, Zhenfei Yin, Xihui Liu, Lu Sheng, Jing Shao, Lei Bai, Wanli Ouyang, and Ruimao Zhang. Worldsimbench: Towards video generation models as world simulators, 2024. URL. httpps//erxxvvorg//88 2410.18072
Delin Qu, Chi Yan, Dong Wang, Jie Yin, Qizhi Chen, Dan Xu, Yiting Zhang, Bin Zhao, and Xuelong Li. Implicit event-rghd neural slam. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 19584-19594, 2024.
Delin Qu Haoming Song, Qizhi Chen, Zhaoqing Chen, Xianqiang Gao, Xinyi Ye, Qi Lv, Modi Shi, Guanghui Ren, Cheng Ruan, et al. Eo-1: Interleaved vision-fext-action pretraining for general robot control. arXiv preprint arXiv:2508.211 2/12.2118.112.... 2025a.
Delin Qu, Haoming Song, Qizhi Chen, Yuanqi Yao, Xinyi Ye, Yan Ding. Zhigang Wang, Jia Yuan Gu, Bin Zhao, Dong Wang, et al. Spatialvla: Exploring spatial representations for visual- language-action model. arXiv preprint arXiv:2501.15830 2025b.
Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Va- sudev Alwala, Nicolas Carion, Chao- Yuan Wu, Ross Girshick, Piotr Dollár, and Christoph Fe- ichtenhofer. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714, 2024. URL htt.aaxvvvv... org abs 020./400000077 00714
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings the IEEE/CVF confer- on computer vision and pattern recognition, pp. 10684 10684-10695, 2022.
Runway. Gen-3. http://runwwymllcom// 2024.06.
14

---

Published as a conference paper at ICLR 2026
Lloyd Russell, Anthony Hu, Lorenzo Bertoni, George Fedoseev, Jamie Shotton, Elahe Arani, and Gianluca Corrado. Gaia-2: A controllable multi-view generative world model for autonomous driving, 2025.
http://w..wrrrb////..a333 20523
Johannes Lutz Schönberger and Jan-Michael Frahm. Structureffrommmotion revisited. In Confer- ence on Computer Vision and Pattern Recognition (CVPR), 2016.
Johannes Lutz Schönberger, Enliang Zheng, Mare Pollefeys, and Jan-Michael Frahm. Pixelwise view selection for unstructured mueti-view stereo. In European Conference on Computer Vision (ECCV), 2016.
Hao Shao, Letian Wang. Ruobing Chen, Hongsheng Li, and Yu Liu. Safety-enhanced autonomous driving using interpretable sensor fusion transformer. In Conference on Robot Leurning, pp. 726- 737. PMLR, 2023a.
Hao Shao, Letian Wang. Ruobing Chen, Steven L Waslander, Hongsheng Li, and Yu Liu. Reason- net: End-to-end driving with temporal and global reasoning. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 13723-13733, 2023b.
Hao Shao, Yuxuan Hu, Letian Wang, Guanglu Song, Steven I, Waslander, Yu Liu, and Hongsheng Li. Lmdrive: Closed-loop end-to-end driving with large language models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 15120-15133 2024a.
Hao Shao, Shengju Qian, Han Xiao, Guanglu Song. Zhuofan Zong, Letian Wang, Yu Liu, and Hong- sheng Li. Visual cot: Advancing multi-modal language models with comprehensive dataset and benchmark for chain-of-thouge reasoning. Advances Neural Information Processing Systems, 37:8612-8642, 2024b.
Hao Shao, Shulun Wang, Yang Zhou, Guanglu Song, Dailan He, Shuo Qin, Zhuofan Zong, Bingqi Ma, Yu Liu, and Hongsheng Li. Vividface: A feffusiin-baseed hybrid framework for high-fidelity video face swapping. arXiv preprint arXiv:2412. /279, 2024c.
Shaoshuai Shi, Li Jiang, Dengxin Dai, and Bernt Schiele. Motion transformer with global intention localization and local movement refinement, 2023. wttLitttp///ariv..oom abs 22099.
13508
Oriane Siméoni, Huy V. Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Michaël Ramamonjisoa, Francisco Massa, Daniel Haziza, Luca Wehrstedt, Jianyuan Wang, Timothée Darcet, Théo Moutakanni, Leonel Sentana, Claire Roberts, Andrea Vedaldi, Jamie Tolan, John Brandt, Camille Couprie, Julien Mairal, Hervé Jégou, Patrick Labatut, and Piotr Bojanowski. Dinov3, 2025. URL lattps://axxiv...
abs/ 10104
Ivan Skorokhodov, Sergey Tulyakov, and Mohamed Elhoseiny. Stylegan-v: A continuous video generator with the price, image quality and perks of stylegan2, 2022. URL https: 221//1122.11. 14683
Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofeev, Scott Ettinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Sheng Zhao, Shuyang Cheng. Yu Zhang, Jonathon Shlens, Zhifeng Chen, and Dragomir Anguelov. Scalability in perception for autonomous driving: Waymo open dataset, 2020. URL https::/arxiv. www.aaawww..rrgaos///222222
Zachary Teed and Jia Deng. Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras, 2022. URLAtttps///zxxvvoo2//////
10869
Ali Tongyi. Wanxiang video. http::/wwwr..all.......//wwwvaae 2024:44 2024.09.
Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. Towards accurate generative models of video: A new metric & challenges, 2019.
URL [tK
01777
15

---

Published as a conference paper at ICLR 2026
Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wer Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang. Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jingren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, Pandeng Li, Pingyu Wu, Ruihang Chu, Ruili Feng. Shiwei Zhang, Siyang Sun, Tao Fang, Tianxing Wang. Tianyi Gui, Tingyu Weng. Tong Shen, Wei Lin. Wei Wang. Wei Wang, Wemmeng Zhou, Wente Wang, Wenting Shen, Wenyuan Yu, Xianzhong Shi, Xiaoming Huang, Xin Xu, Yan Kou, Yangyu Lv, Yifei Li, Yijing Liu, Yiming Wang. Yingya Zhang, Yitong Huang, Yong Li, You Wu, Yu Liu, Yulin Pan, Yun Zheng, Yuntao Hong, Yupeng Shi, Yutong Feng, Zeyinzi Jiang. Zhen Han, Zhi-Fan Wu, and Ziyu Liu. Wan: Open and advanced large-scale video generative models, 2025. URL wttps::sss//.mmm / wrxiv. org. abs 2503. 20314
Ao Wang, Hui Chen, Lihao Liu, Kai Chen, Zijia Lin, Jungong Han, and Guiguang Ding, Yolov10. 10: Real-time end-to-end object detection. arXiv preprint arXiv:2405.1 14458, 2024a.
Jianyi Wang, Kelvin C. K. Chan, and Chen Change Loy. Exploring clip for assessing the look and feel of images, 2022. URL
abs 2207 12396
Letian Wang, Liting Sun, Masayoshi Tomizuka, and Wei Zhan. Socially-compatible behavior design of autonomous vehicles with verification on real human data. IEEE Robotics and Automation Letters, 6(2):3421-344 2021.
Letian Wang, Jie Liu, Hao Shao, Wenshuo Wang, Ruobing Chen, Yu Liu, and Steven L Waslander. Efficient reinforcement learning for autonomous driving with parameterized skills and priors. arXiv preprini ar arXiv:2305.044: 2023.
Letian Wang, Seung Wook Kim, Jiawei Yang, Cunjun Yu, Boris Ivanovie, Steven Waslander, Yue Wang, Sanja Fidler, Marco Pavone, and Peter Karkus. Distillnerf: Perceiving 3d scenes from single-glance images by distilling neural fields and foundation model features. Advances in Neu- ral Information Processing Systems, 37:62334-6223 2024b.
Letian Wang, Marc-Antoine Lavoie, Sandro Papais, Barza Nisar, Yuxiao Chen, Wenhao Ding, Boris Ivanovic, Hao Shao, Abulikemu Abuduweili, Evan Cook, et al. Deployable and gener- alizable motion prediction: Taxonomy, open challenges and future directions. arXiv preprint arXiv:2505.09074, 2025.
Xiaofeng Wang, Zheng Zhu, Guan Huang, Xinze Chen, Jiagang Zhu, and Jiwen Lu. Drivedreamer: Towards real-worid-drive world models for autonomous driving. In European conference on computer vision, pp. 55-72. Springer, 2024c.
Yihan Wang, Lahav Lipson, and Jia Deng. Sea-raft: Simple, efficient, accurate raft for optical flow, 2024d. URL htttp::/// org/ abtp//55s2445.... 1100005114999 14793
Yugi Wang, Ke Cheng, Jiawei Hc, Qitai Wang. Hengchen Dai, Yuntao Chen, Fei Xia, and Zhaoxiang Zhang. Drivingdojo dataset: Advancing interactive and knowledge-enricee driving world model, 2024c. Ut / aawiv. org/ abs/://b///// abss///11.. 2410. 10738
Yuqi Wang, Jiawei He, Lue Fan, Hongxin Li, Yuntao Chen, and Zhaoxiang Zhang. Driving into the future: Multiview visual forecasting and planning with world model for autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 14749-14759, 2024f.
Zihan Wang. Songlin Li. Lingyan Hao, Xinyu Hu, and Bowen Song. What you see is what matters: A novel visual and physics- based metric for evaluating video generation quality, 2024g. URL
https::/aara http://aaxvvv.vvm org. abs////////// 2411. 13609
Cheng-Yen Yang, Hsiang- Wei Huang, Wenhao Chai, Zhongyu Jiang, and Jenq-Neng Hwang. Samu- rai: Adapting segment anything model for zero-shot visual tracking with motion-aware memory, 2024a.
//tp///aavvv///a////a/eeaaeaava/ org/ abs 2212.411111 11922
Jiazhi Yang, Shenyuan Gao, Yihang Qiu, Li Chen, Tianyu Li, Bo Dai, Kashyap Chitta, Penghao Wu, Jia Zeng, Ping Luo, et al. Generalized predictive model for autonomous driving. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14662-14672, 2024b.
16

---

Published as a conference paper at ICLR 2026
Jiazhi Yang, Kashyap Chitta, Shenyuan Gao, Long Chen, Yuqian Shao, Xiaosong Jia, Hongyang Li, Andreas Geiger. Xiangyu Yue, and Li Chen. Resim: Reliable world simulation for autonomous driving, 2025. URL https: https:://a.i...orrttt//ooo/////e http:/aaxivvv.../a/////
09981
Lihe Yang, Bingyi Kang, Zilong Huang. Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2, 2024c.
09414
Zetong Yang, Li Chen, Yanan Sun, and Hongyang Li. Visual point cloud forecasting enables scal- able autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14673-14684, 2024d.
Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, al. Cogvideox: Texc-toviico diffusion models with an expert transformer. arXiv preprint Sttv:2408.06077 2024e.
Hu Yue, Siyuan Huang, Yue Liao, Shengcong Chen, Pengfei Zhou, Liliang Chen, Maoqing Yao, and Guanghui Ren. Ewmbench: Evaluating scene, motion, and semantic quality in embodied world models, 2025. Utthttts:://arkt
wwwp:/iwi.rrraaooo abs 2505. wtw. 09694
Xinjie Zhang. Jintao Guo, Shanshan Zhao, Minghao Fu, Lunhao Duan, Jiakui Hu, Yong Xien Chng. Guo-Hua Wang, Qing-Guo Chen, Zhao Xu, Weihua Luo, and Kaifu Zhang. Unified multimodal understanding and generation models: Advances, challenges, and opportunities, 2025. URL
Guosheng Zhao, Xiaofeng Wang, Zheng Zhu, Xinze Chen, Guan Huang, Xiaoyi Bao, and Xingang Wang. Drivedreamer-2: LIm-enhanced world models for diverse driving video generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 10412-10420, 2025.
Wenzhao Zheng, Weiliang Chen, Yuanhui Huang, Borui Zhang, Yueqi Duan, and Jiwen Lu. Oce- world: Learning 3d occupancy world model for autonomous driving. In European conference on computer vision, pp. 55-72. Springer, 2024a.
Wenzhao Zheng, Ruiqi Song, Xianda Guo, Chenming Zhang, and Long Chen. Genad: Generative end-to-end autonomous driving, 2024b. URL /RL/sttss://axxi org/ abs .110220022.112222
Wenzhao Zheng, Zetian Xia, Yuanhui Huang, Sicheng Zuo, Jie Zhou, and Jiwen Lu. Doe-1: Closed- loop autonomous driving with large world model, 2024c. URL wtt.ttts///xxiv..ottt /bs//
2412.0 09627
Yang Zhou, Hao Shao, Letian Wang, Steven L.. Waslander, Hongsheng Li, and Yu Liu. Smart- pretrain: Model-agnostic and dataset-agnostic representation learning for motion prediction. In The Thirteenth International Conference on Learning Representations, 2025. URL https:
/openrevieww form/ http://owmmmaaaameeeeeeeeeet d=
Zhuofan Zong, Dongzhi Jiang, Bingqi Ma, Guanglu Song, Hao Shao, Dazhong Shen, Yu Liu, and Hongsheng Li. Easyref: Omni fimi-generrllieed group image reference for diffusion models via mul- timodal 1lm. In Forty-second International Conference on Machine Learning, 2024a.
Zhuofan Zong, Bingqi Ma, Dazhong Shen, Guanglu Song, Hao Shao, Dongzhi Jiang, Hongsheng Li, and Yu Liu. Mova: Adapting mixture of vision experts to multimodal context. Advances in Neural Information Processing Systems, 37:103305-103333 2024b.
17

---

Published as a conference paper at ICLR 2026
A RELATED WORKS
A.1 GENERATIVE WORLD MODELS AND THEIR APPLICATION IN DRIVING
Driven by advances in image generative modeling (Kingma & Welling 2013 Goodfellow et al. 2014 [Esee et al. 2021] Ho et al. 2020; Peebles & Xie 2023 Zong et al. 20244a] He et al. 2025
the landscape of large-scale video models has evolved significantly, particularly
diffusion based
frameworks, Closed-source models (Brooks et al. 2024 Kuaishou 2024.06; LumaLabs 2024.06
Runway 2024.06 Pix Verse 2023 Capcut 2024 MiniMax 2024.09) Tongyi 2024.09 PikaLabs
2024.10 Shao et al. (202cc). mainly developed by major technology companies, aim at high-quality,
professional video generation with extensive resources invested. Sora (Brooks et al. 2024). intro-
duced by OpenAl, marked significant leap in Video Generation. Open-source models (Rombach
al., [202 Ho & Salimans) 2022] HaCohen et al., 2024b, Kong al., 2024, Wan al. 2025
Yang et al. 2024e] Agarwal et al. 2025). typically based on stable diffusion (Rombach et al. 2022
and flow matching (Li et al. (2225)). are quickly expanding and making real contributions to video generation as well. Wan (Wan et al. 2025), ), an open-source model, is widely used for video genera- tion and has achieved SOTA results on many benchmarks. Recent years have also seen remarkable
progress in both multimodal understanding and generation models (Li et al. 2025c Zhang et al., 2025
Besides general video generation, driving-focused generative models use sensor data such as lidar point clouds (Zheng et al. 20244a] Yang et al. (2024d) images (Gao al. 2024 Hassan al.) 2024 Hu et al. [023;] Wang et al. 2024cc Yang et al. 2024b Zhao et al.) (025). Since this work em- phasizes video generation, we focus on image-based methods. Early approaches before Vista (Gao
ct al. 2024) rely on multi-view RGB inputs and high-definition maps or 3D boxes. limiting gen-
eralization to new datasets and open-domain videos. Vista-based methods (Hassan al. 2024 Li
ctt al. 2025d Mousakhan et al. (2225) simplify inputs to single front-view image with optional ego trajectories, improving scalability YouTube videos and enabling broader open-domain evaluation.
A.2 BENCHMARKS FOR EVALUATING GENERATIVE WORLD MODELS
The rapid progress of open- and closed-source video generation has driven the creation of many benchmarks (Huang al.] 2023 2024; Bansal et al., 2024; Ning et al. 2023 Liao al., 2024] Fan et al.) 2024 Wang al. (002g), such as VBench, which evaluates models with multifaceted metrics based on human-collected prompts. Recently, evaluations have expanded to open, dynamic, and complex world-simulation scenarios (Yue et al., 2025; Duan et al. 20255 Li et al. 2025d Qin al.) 2024 Kwon et al. 2025). WorldScore (Duan et al. (025) measures generated videos using explicit camera trajectory layouts. However, a comprehensive drrving-world benchmark is still lacking due to limited test sample diversity, heterogeneous input modalities, and the absence of driving-specinee metrics. Recent works (Gao et al. 2024, Hassan et al. 2024h mainly adopt Frechet Video Distance (FVD) and Average Displacement Error (ADE) for trajectory alignment, while GEM (Hassan al.) 2024) adds human video evaluations that are subjective and hard to scale. The closest effort, ACT- Bench (Arai et al. 2024), focuses solely on trajectory alignment and overlooks key aspects such as video and trajectory distribution, quality, and temporal consistency.
18

---

Published as a conference paper at ICLR 2026
Dataset Source
Region
Ratio
10:5
10.5
Zod
Europe
26.5%
Drivingdojo
China
25.6%
Pertertag
140%
CoVLA
Japan 27.4%
Nuplan
U.S. 8.8%
WOMD
U.S. 11.6%
Das
The 4t7 jer
The of Des
(h) Data Ratio from existing open-sourced driving dataset our ego-condition track.
(a) Weather and time of day distribution in ego- condition track.
Figure 3: The statistics of our ego-condition track.
Figure 4: The gallery of our ego-condition track.
B APPENDIX
B. GALLERY OF THE CONDCIINDDO TRACK
We present the distribution and gallery of our ggodiondit track in Fig. data from five open-sourced driving datasets to diversify the distribution of weather, time of day, and locations (with various driving styles). The videos and ego-trajectories provided in these datasets are used as the target distribution for calculating metrics such as FVD and FTD.
Fig.Bland Bland Fig, We curated
B.2 DETAILS OF OUR SLAM PIPELINE AND COMPARIISION WITH OTHERS
Dealing with Unsuccessful Trajectory Reconstruction. Not every generated video will yield successful SLAMM reconstruction, especially if the video has tremendous artifacts very low tex- ture. Simply discarding those cases would bias the evaluation, because typically it's the worst videos (the most unrealistic ones) that cause SLAM fail. Dropping them would artificially inflate those noor-performing models' scores. We tackled this issue explicitly to ensure no video is left unevalu- ated. Our approach to build custom SLAM+depth estimation pipeline that is robust to failures. We ensure trajectory is obtained for every video by applying a failure-recoven, strategy: if at any frame the SLAM algorithm cannot estimate the next camera pose (e.g., fails in feature matching. solving PnP. etc.), we take the last known pose and extrapolate it forward. Specifically, we propa- gate the last pose with constant velocity model. To avoid giving unrealistic advantage, we add
19

---

Published as a conference paper at ICLR 2026
Pipeline
Success rate t
ADE 4
GEM: DROID-SLAM + Depth-Anything v2
17
14.61
DrivinDojo: COLMAP + scale to GT Ours w/o failure handling
17 16/ 20
14.99
15.18
Ours w/ failure handling
20 20
16.84
Table 4: Comparison of different SLAM pipelines on 20 nuPlan videos generated with Vista. "Suc- cess rate" counts how many videos yield valid reconstruction; ADE is the mean trajectory error over successfully reconstructed runs.
small random perturbations to the pose orientation during this extrapolation. This injects a bit of un- certainty to mimic the fact that the current estimation is noisy, preventing the extrapolated path from appearing "too perfect" in metrics. We chose not simply freeze the camera (no movement) upon failure, because completely static continuation could skew certain trajectory metrics. By using this continuous-an aawboouss-n--jiteee and-jitter method, obtain a complete trajectory from start to end for every video, no matter how poor its quality. This allows all videos to count toward the trajectory- based metrics, holding models accountable for cases where naive SLAM would have given up.
Comparison with Other SLAM Pipelines. We evaluated our reconstruction pipeline against those used in recent driving world-model systems. Concretely, compare the successful reconstruction rate and trajectory accuracy (ADE) on 20 nuPlan videos generated with Vista from our early ex- periments. A run is counted successful if the SLAM system returns valid camera trajectory without numerical failure. The results are summarized Table 4 Compared to the GEM pipeline (DROID-SLAM (Teed & Deng 2022 Depth-Anything v2 (Yang al. 2024ch) and the Drivin- Dojo pipeline (COLMAP (Schönberger et al. (2016) Schönberger Frahm 2016) with scale aligned to ground truth), our basic version (Ours w/o failure handling) achieves similar successful recon- struction rate (17/20 vs. 17/20 and 16/20) and a comparable ADE (15.18 vs. 14.61 and 14.99). When enable
hhandiin strategy (Ours w/ failure handling), the successful rate increases to 20/20, while the ADE remains in the same ballpark ( (16.84). This trade-off is important for Driv- all videos rather than dropping harder cases
ingGen: the benchmark needs robust reconstruction and evaluating on a subset of "easy" videos. Overall, our SLAM pipeline more robust than existing pipelines by handling reconstruction failure explicitly.
B.3 FRÉCHET TRAJECTORY DISTANCE (FTD)
Idea. FTD applies the FIDDstyle Gaussian Fréchet distance to trajectory embeddings, replacing image/video features with driving-domain encoder.
Representation model and input. We use MTR's agent_pollliineencode x(-) Shi et al,
2023). Crucially, MTR consumes fixed temporal horizon H.
Window embeddings & trajectory pooling. We slice the trajectory into windows to fit into the MTR encoder. Each window is encoded f <(window) E Rd A trajectory's embedding is the mean over its window embeddings, which stabilizes statistics and removes dependence on the number of windows.
Distributional distance. For generated embeddings {f(T;" D and reference embeddings
Y = ( with empirical means/covariances Px/r1, Ex/y, define
FTD(X,Y) 1 p 1 Tr (Ex Tr(Ex Ex 2 2(5.1/2 $1/3)/13 12)
We add sI (==10-6) 9) before the matrix square root and symmetrize products by (A+AT)/2 needed. Optional Ledoit-Wolf shrinkage can be used when or m < d.
Practical recipe (defaults).
* Encoder: MTR cotnnttpooylineeeccco.
Horizon & slicing: H=10 steps; stride s=Hl (non-overlapping): same slicing for gener- ated and reference.
20

---

Published as a conference paper at ICLR 2026
Normalization: agent-centric translation/rotation per window; MTR schema constants (E, (,,, h) = (4.5, 2.0, 1.8) m; type=vehicle; validity=1.
Aggregation: mean over a trajectory's window embeddings; FTD on the two sets of trajectory-level embeddings.
B.4 OBJECTIVE IMAGE QUALITY
Motivation and background. Pulse-width modulation (PWM) in vehicle lighting and roadside luminaires induces temporal luminance modulation that, when sampled by rolling-shutter cameras, can alias into low-frequency flicker and degrade detection and tracking- The IEFE Automotive P2020 standard formalizes Modulation Mitigation Probability (MMP) to quantify whether such modulation is sufficiently suppressed during operation (Group et al. 2018; 996] (022). We im- plement MMP on the frame-mean luminance provide a robust and efficient evaluation signal.
Definition. Given frames {4)m at sampling rate fps, form the luminance sequence L,
(geangrry((T)) and its periodogram P(J) (f) (real FFT). Let the dominant non-DC peak be
f arg max P(f).
130
If f" < H.2 set MMP =MP=1. 1.
Computation, With the band B(f*) = (f: 15 14| Af). define the band-power ratio
A = ΣsEB(S,) P(S)
= 10-8
E, P(f)+ +E
The metric is
MMP = 1/A r|
E {0.
Defaults. A band_h2 0.5 Hz, T thr 0.05, fps 10. The procedure single FFT per clip with complexity O(Tlog
B.5 TRAJECTORY QUALITY
Motivation. Video-only scores can miss whether motions are plausible and comfortable. We de- fine trajectory quality that aggregates three kinematic submetrics comfort, motion, and curva- ture-via a weighted geometric mean (equal weights by default). Each submetric lies in [0, 1] with larger being better, report per-trajectory scores and dataset means, skipping NaNs.
Preliminaries. trajectory r=((xe, yt) Velocities, accelerations, and jerks use centered fi- nite differences. Heading comes from velocity, and yaw rate uses wrapped heading differences. Path length the cumulative step distance. A trajectory marked moving if any speed exceeds Ustatic 0..1 U.tatic=0..1 m/s.
Comfort (Scont). We score comfort from three per-meter peaks: longitudinal jerk, lateral accel- eration, and yaw rate. Trajectories that are non-moving (speed < Vaatic) too short (S m) are set NaN. Each peak then mapped to a [o, 1] component score with an inverse transform S, 1/(1 + 9/8q) (higher is better), where 8g are scale factors (default 1.0). The final comfort score is the geometric mean of the three components.
Motion (Sypeea). We penalize under-mobility using trajectory's 's mean speed. A monotone log map- ping compresses high speeds and scales by Umax thet (defaults: 6..= m/s, k=2.5) to obtain Sopeed E [0. 1). Never-moving trajectories receive 0.
Curvature (Scurv). Discrete curvature is formed from first/second derivatives of (Ft, y1). We then compute an RMS curvature Kymss then map
Seury
E (0, 1].
+ Krms
Non-moving trajectories return Na
21

---

Published as a conference paper at ICLR 2026
Component
Example
Approx. Time
Wan2.2-14B
Days
Video Generation
Vista
About One Day
Trajectory Reconstruction
SLAM + Depth model
Hours
FVD
Hours
Distribution Metrics
FTD
Minutes
Subjective Image Quality Objective Image Quality
Hours
Quality Metrics
Minutes
Trajectory Quality
Minutes
Video Consistency
Hours
Agent Consistency
More Hours
Consistency Metrics
Agent Disappearance Consistency
Hours
Trajectory Consistency
Minutes
ADE DTW
Minutes
Trajectory Alignment Metries
Minutes
All Metrics (Total)
All Above Metric Groups
1-2 Days on Single GPU
Table 5: Approximate runtime of different components in DrivingGen on 400 videos with 100 frames each, evaluated on single modern GPU. Times are coarse estimates and may vary with hardware.
B.6 AGENT ABNORMAL DISAPPEARANCE
Motivation. Agents should not vanish without plausible cause (e.g., occlusion leaving the view). We detect such cases directly from video with a minimal vision-language check.
Method. For each agent that disappears, we prepare three frames: (1) the first frame where the agent visible, (2) the last frame where visible (both with the agent box drawn green), and (3) the first frame after it disappears (no box). We ask a VLM to classify the disappearance with the following prompt:
Given three frames around the moment a green-boxed object disappears, classify the disappearance occlusion or leaving the field of view) Unnatural (abrupt or non-physical). Base your decision on visual and motion continuity and interactions with nearby objects. Output one word: Natural or Unnatural.
Natural (e.g.,
Scoring. A tracklet is abnormal if the VLM outputs Unnatural; otherwise it is not abnormal. A video clean only if all evaluated tracklets are not abnormal. The final score is the percentage of clean videos (higher is better).
B.7 TRAJECTORY CONSISTENCY
Definition, From positions sampled at step At, form the speed series v, and the acceleration series a by finite differences. Measure each signal's dispersion relative to its typical level using simple ratio, then squash with an exponential:
std(r) R, me)
std(a)
R.
S, exp(-Ra), Ro), S. exp(pxRR Ra).
mean()a))'
The trajectory consistency score is the average
Soons = (S, + S.) E (0,
where higher indicates smoother, more realistic kinematics.
22

---

Published as a conference paper at ICLR 2026
Human
Video quality:
Video consistency. Human
10
0.0
p
0.8899
0.
p- 0.8344
0.8
P 0.8757
Human Vin Rate
0.8-
Human win nate
04-
0.6
Jaryaan
03-
Human
34
Kolepn capiiee
04
96
344
02
044
055
0.8
Metric Win Rate
Metric Rate
Metric Win Rate
Trajectory realism: Metric Human
Trajectory quality: Metric vs Human
Tejectory consistency, Metric vs Human
p 0.7682
p= 0.6261
0 0.7467
Human Win hate
Humnn Win Reieer
sprricer bea
with
0.6
0.-
0.-
Jatnor 143
84-
2331
84-
0.3
Metric Win Rate
04
0.9
67 0/1
Metric Win Rate
0.8
1.0
Metric
Figure 5: Human Validation of Our benchmark. Our metrics closely match human preferences. Trajectory-related metrics are less accurate in comparison humans, likely due to noisy monocular SLAM and metric-depth recovery from generated videos with artifacts.
B.8 TIME AND RESOURCE FOR DRIVINGGEN
In our experiments, the bottleneck is primarily the video generation itself: many of the state-of-the- art generative models we benchmark are slow and memory hungry (e.g., Wan2.2-14B takes about 20-30 minutes to generate one 100-frame video on a single GPU with at least 40 GB memory). In contrast, the evaluation suite is comparatively manageable. The approximate wall-clock time for each metric group on 400 videos is summarized in Table 5 On a single modern GPU, running all metrics for 400 videos with 100 frames takes roughly 1-2 days.
Within this budget, the main cost on the evaluation side comes from image quality and video con- sistency metrics, which require running heavy visual backbones over every frame. The most time- consuming metrics would be agent consistency and disappearance consistency, which run models for each agent in the first frame of the video. Trajectory measures (FTD, quality, consistency and align- ment) are much cheaper (minutes), since they operate on compact embeddings or low-dimensional trajectories. These numbers are indicative and may vary with hardware and implementation, but they show that: (i) video generation dominates the overall runtime, and (ii) among the metrics, the image, video and agent quality and consistency components are the main contributors, while the rest of the metrics are comparatively fast.
B.9 HUMAN ALIGNMENT OF DRIVINGGEN
We employ similar method in VBench to determine whether each category aligns with human preferences. Given the human labels, we calculate the win ratio of each model. During pairwise comparisons, if a model's video is selected as better, then the model scores and the other model scores O. If there is a tie, then both models score 0.5. For each model, the win ratio is calculated as the total score divided by the total number of pairwise comparisons in which it participated.
For fast and reasonable evaluation, we select three categories: distribution, quality and consistency. We evaluate with both videos and trajectories and use the primary metric in each category. Metries are FVD and FTD, Subjective image quality and trajectory quality, video consistency and trajectory consistency. The results are shown in Fig.
23


Reviews:
[{"review_id": "9RToozh3aA", "reviewer": "Reviewer_2jMw", "summary": "This paper introduces DrivingGen, a comprehensive benchmark designed to evaluate generative world models for autonomous driving. The authors argue that existing benchmarks are limited in scope, failing to capture the full requirements of driving-specific simulation—such as visual realism, trajectory plausibility, temporal consistency, and controllability—while also lacking diversity in weather, geography, and driving maneuvers.", "questions": "- Why was the dataset limited to 400 clips? Are there plans to expand it in the future to better cover rare and safety-critical scenarios?\n- How do you mitigate the impact of video artifacts on SLAM and depth estimation, especially for models with low visual fidelity?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- DrivingGen is the first benchmark to jointly evaluate visual, kinematic, and interactive aspects of generative driving world models, addressing critical gaps in prior works.\n- Introduces FTD for trajectory distribution, kinematic quality scores, and agent disappearance detection using VLMs—all tailored for driving safety and realism.\n- The dataset includes under-represented conditions (e.g., night, snow, sandstorms) and global geographic variety, enabling more robust and realistic evaluation.\n- The authors validate metric alignment with human preferences, increasing confidence in the benchmark’s practical relevance.", "weaknesses": "- With only 400 clips, the dataset may not fully represent the long-tail of real-world driving scenarios, despite its diversity.\n- The benchmark focuses on open-loop video generation and does not assess models in interactive, closed-loop simulation settings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xuzvJB5xs7", "reviewer": "Reviewer_cB5B", "summary": "The DrivingGen benchmark is introduced as the first comprehensive evaluation framework for generative world models in autonomous driving, targeting key gaps where existing methods overlook safety-critical factors, trajectory plausibility, temporal consistency, and motion controllability. DrivingGen utilizes a highly diverse dataset spanning varied weather, time of day, global regions, and complex maneuvers essential for robust deployment. It introduces a suite of novel, specialized metrics that jointly assess performance across four dimensions: visual realism, trajectory plausibility, temporal coherence, and control fidelity. Benchmarking 14 state-of-the-art models revealed a key finding: a trade-off exists where general models may appear visually superior but \"break physics,\" while driving-specific models prioritize realistic motion accuracy but often lag in visual quality, thus establishing a unified framework to guide future development.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The dataset distributions are balanced and diverse across all conditions. \n\n2. DrivingGen has created a new, specialized set of various measurements to evaluate generated driving videos; these are designed for the complexities of driving and are therefore more effective than standard video evaluation tools.\n\n3. The experiments are comprehensive.", "weaknesses": "1. It would be more convincing to incorporate downstream task's performance (detection, mapping, planning) into the evaluation system. But this seems difficult since the data contains only front-view data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TMHOp5SI1C", "reviewer": "Reviewer_XepP", "summary": "This paper focuses on evaluating driving video generation methods. Existing benchmarks emphasize image quality metrics (e.g., FID, FVD) and often fail to assess aspects critical to autonomous driving, such as the controllability of the ego vehicle's trajectory, the reasonableness of agent trajectories, and the consistency of 3D contents. Current benchmarks also lack data diversity in terms of weather, lighting, road conditions, and geographic locations. This work samples data from multiple datasets to enhance data diversity. proposes a set of metrics for evaluating video output, and benchmarks the performance of several state-of-the-art models.", "questions": "see weakness", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "Establishing a more practical benchmark for driving video generation is valuable. The contributions of providing more diverse test data and comprehensive evaluation metrics are clear. To maximize its contribution, the benchmark and evaluation code should be made open-source.", "weaknesses": "No major flaws were identified in the current work. However, the authors could further improve the benchmark by including evaluations for scene content controllability. While the paper addresses video quality, temporal consistency, and ego trajectory controllability, the controllability of generated scene contents( such as agents controlled with bounding boxes or roads via maps/lane) is also important for autonomous driving applications. These control signals can be extracted from videos using detection and segmentation methods if not available in the original datasets.\n\nTypo: 'Qulity' in Table 2.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QD9feSWH08", "reviewer": "Reviewer_7qoX", "summary": "This work focuses on advancing world models for autonomous driving by introducing a new benchmark, DrivingGen. Unlike previous evaluations that primarily assess visual quality, DrivingGen offers a fine-grained assessment of physical realism, safety, and trajectory consistency, establishing a unified evaluation framework to promote the development of reliable, controllable, and deployable driving world models. Through benchmarking 14 state-of-the-art models, our study reveals clear trade-offs: general-purpose models produce visually appealing results but often violate physical constraints, whereas driving-specific models capture realistic motion dynamics yet fall short in visual quality.", "questions": "1. How to deal with cases where the generated video fails to reconstruct trajectories? SLAM-based methods, do not successfully produce a trajectory for every video sequence.\n2. DrivingDojo proposes the AIF metric to measure trajectory alignment. It might be worth discussing the advantages and limitations of the trajectory metrics proposed in this paper.\n3. What is the approximate time and resource cost for evaluating 400 videos? This is also an important consideration when assessing the practicality of a benchmark.\n4. AVG Ranking may not be an appropriate metric. Could a composite, weighted total score be designed to more comprehensively evaluate world model performance", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The motivation is clear and valuable for advancing future driving world models.\n2. The authors present a thorough empirical study: they evaluate a wide range of methods.\n3. Comprehensive evaluation framework: by assessing methods along multiple dimensions — distribution, quality, temporal alignment, and trajectory alignment.", "weaknesses": "1. Although the metrics are intended for evaluating world models, most of them still focus on visual quality. There are relatively few metrics that assess whether the state transformations of the world model are reasonable and accurate.\n2. Counterfactual reasoning is also an important capability of a world model. It could be interesting to consider evaluating this aspect in future work.\n3. The evaluation of physical reasonable is somewhat lacking. It might be useful to draw inspiration from VBench 2.0 and incorporate metrics.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)