RLGF: Reinforcement Learning with Geometric Feedback for Autonomous Driving Video Generation
Authors: Tianyi Yan
Wencheng Han
xia zhou
Xueyang Zhang
Kun Zhan
Cheng-zhong Xu
Jianbing Shen
Venue: NeurIPS 2025 poster ()
Keywords: video generation
diffusion model
autonomous driving
Primary area: applications

Abstract: Synthetic data is crucial for advancing autonomous driving (AD) systems, yet current state-of-the-art video generation models, despite their visual realism, suffer from subtle geometric distortions that limit their utility for downstream perception tasks. 
We identify and quantify this critical issue, demonstrating a significant performance gap in 3D object detection when using synthetic versus real data.
To address this, we introduce Reinforcement Learning with Geometric Feedback (RLGF), RLGF uniquely refines video diffusion models by incorporating rewards from specialized latent-space AD perception models. 
Its core components include an efficient Latent-Space Windowing Optimization technique for targeted feedback during diffusion, and a Hierarchical Geometric Reward (HGR) system providing multi-level rewards for point-line-plane alignment, and scene occupancy coherence. 
To quantify these distortions, we propose GeoScores. Applied to models like DiVE on nuScenes, RLGF substantially reduces geometric errors (e.g., VP error by 21\%, Depth error by 57\%) and dramatically improves 3D object detection mAP by 12.7\%, narrowing the gap to real-data performance. RLGF offers a plug-and-play solution for generating geometrically sound and reliable synthetic videos for AD development.

RLGF: Reinforcement Learning with Geometric Feedback for Autonomous Driving Video Generation
Tianyi Yan' SKL-IOTSC, CIS University of Macau tianyi. yan y:n. maam1122mmmmccom
Wencheng Han" SKL-IOTSC, CIS University of Macau
Xia Zhou Li Auto Inc.
Xueyang Zhang Li Auto Inc.
Kun Zhan Auto Inc.
Cheng-zhong Xu SKL-IOTSC, CIS University of Macau
Jianbing Shen SKL-IOTSC, CIS University of Macau
Abstract
Synthetic data is crucial for advancing autonomous driving (AD) systems, yet current state-of-the-art video generation models, despite their visual realism, suffer from subtle geometric distortions that limit their utility for downstream perception tasks. We identify and quantify this critical issue, demonstrating significant performance gap in 3D object detection when using synthetic versus real data. To address this, we introduce Reinforcement Learning with Geometric Feedback (RLGF), RLGF uniquely refines video diffusion models by incorporating rewards from specialized latent-space AD perception models. Its core components include efficient Latent-Space Windowing Optimization technique for targeted feedback during diffusion, and a Hierarchical Geometric Reward (HGR) system providing multi-level rewards for point-line-plane alignment, and scene occupancy coherence.
To quantify these distortions, we propose GeoScores. Applied to models like DiVE nuScenes, RLGF substantially reduces geometric errors (c.g.. VP error by 21%, Depth error by 57%) and dramatically improves 3D object detection mAP by 12.7%, narrowing the gap real-data performance. RLGF offers plug-and-play solution for generating geometrically sound and reliable synthetic videos for AD development.
Introduction
The rapid progress of autonomous driving (AD) systems 15 17 6 has created growing need for
high-quality synthetic data. Recent diffusion-based video generation methods [57
61
30
10
g
18 19]
However, identify critical yet underexplored limitation: the generated videos often contain subtle yet impactful incorrect geometric relationships. This flaw not only misleads downstream perception and planning tasks but also undermines the reliability of models trained evaluated using such data, significantly constraining their applicability essential use cases such as simulation-based training [30 and system validation 67].
have achieved state-of-the-ar visual realism, measured by metries like FVD
49
To investigate this geometric limitation. we conduct series of targeted experiments. Evaluating 3D object detection synthetic videos using BEVFusion (26) reveals substantial performance drop compared real data (mAP: 25.7 vs. 35.5). In contrast, 2D object detection using YOLOv5 20
Equal contribution
Corresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Pizefowier
10
Eirarshiccl
Condition Extraction con
ea
Condition Extraction
Training datu
64 6a
Training data
Rewand
Parel vitte Loos
Generation
Semantic Boxes
Semantic 3D Boxes
Outputs
Steps
Outputs
Condition
Condition
Doph Ernee
LBN
Yeunkin
Donk Emre
YoORdARD FANI
Misaligned Piane
Misalignidd Lines
Wrong Perspective
Aligned Plane
Aligned Lines
Correct Perspoctive
(a) Original Video Generation Model
(b) Our RLGF Video Generation Model
Predicted Generited
Image
Depth Map
5,500
7,300
7.
Visualization of Geometric Structure During the Denoising Process
Figure (a) Original Video Generation Models, optimized via pixel-level supervision (e.g.. noise prediction error), often produce visually plausible videos that nonetheless suffer from severe geometric flaws (misaligned planes/lines, wrong perspective). This can degrade downstream tasks like 3D object detection (e.g.. mAP drop from 35.5 to 25.7). (b) Our RLGF integrates Hierarchical Geometry Reward directly into the multi-step denoising process. This reward, derived from perception models, guides the generation model to produce outputs with aligned planes, correct lane structures, and accurate perspective. (c) Visualized depth maps from noisy latents at various denoising stages (from noisy to less noisy) show coarse geometry emerging early and details later. This motivates our Latent-Space Windowing Optimization for targeted intermediate rewards.
on the same data yields results comparable to real-world samples (mAP: 43.8 vs. 44.7). These findings suggest that while current diffusion models (57 (6 18 9 preserve appearance, indicating minimal image-level domain gap, yet they fail to capture accurate 3D scene structure. We attribute this primarily to underlying geometric inconsistencies. To further verify this hypothesis and systematically quantify these distortions, introduce GeoScores. This metric suite evaluates geometric fidelity by applying pre-trained perception models [65] to both synthetic vidcos and their corresponding real-world counterparts, using the outputs from real videos reference ground truth. Significant discrepancies between the two highlight geometric errors. GeoScores (details are in appendix A reveals three major issues: (1) Vanishing point shifts, indicating incorrect global perspective; (2) Lane topology inconsistencies, reflecting misaligned road markings and implausible lane structures; and (3) Depth errors, particularly road surfaces, signifying incorrect placement. (e.g.. for a typical baseline, an average VP shift of 0.086 normalized units, Lane F1-Score of only 0.792, and an average depth error of 1.822). The significant deviations in these scores confirm that current "high-quality" synthetic data often suffers from pervasive geometric inaccuracies (fig. Ta)).
Addressing these challenges, present Reinforcement Learning with Geometric Feedback (RLGF), a novel framework that injects perception-moodlldiivee model-driven geometric spatial constraints directly into the video generation process. Unlike conventional approaches [57 56 [ that primarily rely pixel- wise alignment, which often fail to explicitly enforce adherence to complex, underlying geometric principles, RLGF leverages dedicated, pre-trained AD perception models as reward providers to ensure geometric fidelity.
RLGF introduces two core technical innovations: Latent-Space Windowing Optimization and Hierar- chical Geometrie Alignment. Frist, we present Latent-Space Windowed Optimization. We observe that geometric structures in diffusion models emerge progressively across denoising steps: early steps

---

(e.g., before step 10 in flow matching [27 28 establish coarse global geometry, while later steps refine local details (fig. Tc). Training across the entire sampling process, as done in some prior RL-diffusion work [1] struggles to provide targeted guidance for these distinct phases. Therefore, we propose an efficient latent-space training strategy where rewards are applied directly to noisy latent features within randomly sampled sliding window of intermediate diffusion steps. This approach significantly reduces computational (GPU memory) burden and, more importantly, allows for effective and targeted corrective feedback during both early (global structure formation) and late (detail refinement) stages of geometric synthesis.
Secondly, RLGF features Hierarchical Geometric Reward (HGR), multi-level feedback system designed to imbue generated videos with robust geometric fidelity and scene coherence. HGA inte- grates signals from two specialized latent-space perception networks: a Latent Geometry Perception Model assessing vanishing point, lane, and depth cues, and a Latent Occupancy Prediction Model inferring 3D scene occupancy. Both operate efficiently on noisy latents, aided by lightweight micro-decode module to eircumvent costly full decoding. Leveraging these perception models, HGA then constructs its multi-level reward system. For point-line-pland geometric feedback, we use the outputs of the perception model to: (1) enforce vanishing point consistency for accurate global perspective, (2) ensure lane topology validity for realistic road structure, and (3) promote depth coherence for correct surface and object geometry. For scene-level occupancy feedback, outputs from the occupancy model are used to: (4) align intermediate semantic features using KL divergence for plausible scene evolution, and (5) maximize 3D occupancy IoU for accurate volumetric object layout and dynamics. This structured approach, combining efficient latent-space perception with multi-faceted reward design, ensures that comprehensive geometric and occupancy information is extracted and effectively fed back to guide the video diffusion model towards producing physically principled and geometrically sound autonomous driving scenarios.
Our contributions are fourfold:
We are the first to systematically quantify the geometric distortion problem autonomous driving video generation and propose the GeoScores metric for its evaluation.
We introduce RLGF, a novel paradigm that uses reinforcement learning with perception- based rewards applied efficiently within a sliding window in latent space, enabling plug-and play geometric correction.
We design HGR which addresses geometric distortions by incorporating point-line-plan and scene-level occupancy multi-level geometric feedback derived from latent representations.
Extensive experiments on nuScenes demonstrate RLGF's plug-and-play effectiveness across two baselines [8] 9] boosting 3D detection mAp by 12.7% absolute while reduce geom- etry gap (via GeoScores) relative to real data. This work establishes new paradigm for geometrically faithful synthetic data generation in autonomous driving systems.
2
Related Work
2.1 Video Diffusion for Autonomous Driving
The development of robust autonomous driving (AD) systems [13] 17 6 necessitates large volumes of diverse and realistic training data. Learned generative model 22
11 have emerged powerful approach to synthesize such data by capturing the complex distributions of real-world driving scenarios. Early efforts explored Generative Adversarial Networks (GANs) 38 and Variational Autoencoders (VAEs) 21 for generating driving-related imagery. More recently, diffusion
62
models 11 35 39
59
69
68 including latent diffusion models [44], have demonstrated
tate-offthe-art capabilities in generating high-fidelity images and videos [44] 4.
3 77, [1] [54] (55)
75 34 Based on this technique, generative models in autonomous driving have greatly advanced.
For example, BEVGen 44) and BEVControl [64] generate controllable vire imagery from
Bird's-Eye View (BEV) layouts or other structural conditions. Further advancements, such as DriveDreamer 53 Magicdrive [10] Driveri [56), Panacea [57]. DriveSphere [61] and others [76]
72), focus on generating coherent multi-camera driving videos, often conditioned on textual prompts, historical trajectories, or 3D assets. These models excel at producing
30 31
58
73
visually compelling outputs that achieve high scores on perceptual metrics like Fréchet Video Distance
(FVD) 49].
3

---

Despite achieving high visual realism, leading AD video generation models [57] 9 18 frequently introduce subtle geometric and scene-level distortions (e.g., flawed perspectives, depth errors, unreal- istic motion). These inconsistencies, often overlooked by human assessment, severely undermine performance on 3D perception tasks like object detection [26] and motion forecasting. RIGG addresses this critical gap by proposing novel framework to instill multi-scale geometric and scene-level consistency within the generation process.
2.2 Reinforcement Learning for Video Generation
Fine-tuning generative models with reward signals, rooted in Reinforcement Learning from Human Feedback (RIHF) successes in LLMss (e.g., using PPO
[5] DPO 41), ) is increasingly applied to diffusion models. For image generation, methods 22 50,
63 60) align models with human preferences, and approaches [25] use perception model feedback for content control. This trend extends to video diffusion, where DPO-based techniques [66] 74 7 often utilize human preference data. While improving general video quality, these holistic preference scores typically lack the precise, local geometric feedback crucial for autonomous driving applications. Other methods [ 24
33
7 employ
explicit reward models for video fine-tuning offering more detailed signals. However, these rewards are generally not tailored to the specific multi-scale 3D geometric and physical plausibility demands of AD simulation, failing to adequately address distortions like incorrect perspective or depth error.
Our RLGF framework advances this by introducing explicit, quantifiable geometric and scene-level rewards from dedicated AD perception models. This provides targeted, locally-aware feedback to correct specific geometric inaccuracies, producing data suitable for rigorous AD tasks.
3 Method
3.1 Preliminary: Limitations of Conditional Video Diffusion
Current video diffusion models [ 57 18 generate frames by gradually denoising latent variables through a Markov chain. Formally, given condition (e.g., road sketches or bounding boxes), the model learns to minimize the pixel-Ievel reconstruction error during training:
Spixel Exo,exelle c0(#,; t,c) c]]"
where I, is the noisy sample at timestep is the ground truth noise, and Eq is the denoising network. While effective for visual fidelity, this standard formulation inherently ignores crucial geometric-semantic correlations due to two limitations: (1) Pixel-wise Independence Assumption: The MSE loss treats each pixel as independent, failing to model higher-order geometric relationships perspecccve consistency in 3D space). (2) Conditional Oversimplification: Existing methods concatenate geometric conditions with noisy latents, However, this primarily enforces local pixel alignment corresponding to the condition rather than guaranteeing the global geometric integrity or plausibility of the underlying 3D) scene structure. These limitations motivate the need for explicit geometric guidance during generation.
3.2 Reinforeed Learning with Geometric Feedback (RLGF)
Our primary objective is to enhance the geometric and spatio-temporal consistency of videos generated by conditional diffusion models, addressing the critical gap left by conventional training objectives that primarily focus on pixel-level visual fidelity.
To achieve this, introduce Reinforcement Learning with Geometric Feedback (RLGF). This frame- work refines a pre-trained video diffusion model, ep. by guiding its generation process towards outputs that exhibit greater adherence to real-world geometric and physical principles. Unlike RLHF 33/ 63) which typically captures broad subjective preferences, RLGF is designed to incorporate specific, model-interpreen
geometric constraints. The core idea is to leverage dedicated, pre-trained percep- tion models to evaluate the generated videos 2y and provide a reward signal that steers 6g away from implausible outcomes.
Formally, given well-trained video diffusion model co. the dataset D,, the well-designed reward function R(.), we aim to maximize the objective:
J (()) Ee,v, [R(oo, w)]
(2)

---

Diffusion U-Net
Output
Z
VAE Encoder
VAE Decoder
Geometry Lora
RLGF Model
detached
Z
Zg
Latent Space Windowing
Diffusion Procedure
Rew Brd
AI
Point-level Alina
Reward
gt
Latent Goometry
Extraction
Feature-level
Vaxel-level
Hierarchical Geometry Reward
Hierarchical Geometry Aligmment
Figure 2: Overview of the Reinforcement Learning with Geometric Feedback (RLGF) frame- work. RLGF fine-tunes a frozen well-trained diffusion model via LoRA using rewards from "Latent Space Windowing" scheme. Within this window, intermediate latents - are evaluated by frozen perception models Pgeo (point-line lin---ene- alignment) and Poce (scene-level consistency) against reference video. The resulting rewards (Rgeos Roce) generate gradients (red arrows) to update LoRA, improving geometric and temporal consistency. Black arrows: feed forward; dashed red: gradients.
where and denote the condition and real video from the dataset D Fo the the generated sample by (.(c) given c, and the total reward R(20, v) encapsulates assessments from our Hicrarchical Geometric Alignment system, detailed in Section 3.4
3.3 Latent-Space Windowing RL Optimization
by unrolling the entire T-step diffusion sampling process obtain do and then backpropagating the reward gradient V Ro v) is computationally prohibitive due to large memory requirements and long sampling chains.
Directly optimizing Equation 2
To efficiently apply RLGF, we propose Latent-Space Windowed RL Optimization. This strategy addresses the computational challenge of full rollouts and leverages insights into the progressive nature of structure formation diffusion models.
Visualizations (e.g., fig. [[c) in Introduction) suggest that coarse global geometry is often established in earlier denoising steps (e.g. > Tmid), while later steps refine details. Training across the entire sampling chain, as in some prior RL-diffusion work [I], may not provide sufficiently targeted guidance for these distinct phases.
Instead of rewarding only the final output vp, we provide feedback based on noisy latent features 2, at intermediate denoising steps compared against reference latents 2n derived from real videos. Specifically, during the T Ttstep sampling process 2T + ... + 20. we apply our reward functions R within a sliding window of w steps (e.g. from random start step t' down perception models are designed to take 2 and k as input.
w). Our
This approach is motivated by two factors: (1) Efficiency: It significantly reduces the computational graph for backpropagation. (2) Effectiveness: allows targeted corrective signals at different stages of generation, crucial for both global structure and local detail.
The practical objective then becomes maximizing the expected reward obtained by evaluating the noisy latent zạ (at step l within the window) against the VAE-encoded latent 2, paired with real
5

---

video D:
practiooal (fL.RA) E Be,., [R(zk) 2n)
(3)
where and 2, is the encoded latent feature using VAE ecnoder [44 LoRA parameters OL.RAA for a reward at step ks is then:
The gradient with respect to the
V Vnoon (2 2,)) 2Ra (5ks 2n)
Ozk
Ozk
LLLAA
Throughout the RLGF fine-tuning, the perception models that constitute the reward function are mre-trinne and their weights remain frozen. ensuring a consistent evaluation standard as co via OIORA adapts to maximize geometric fidelity.
3.4 Hierarchical Geometric Reward (HGR)
The Hierarchical Geometric Reward (R) is designed to provide comprehensive, multi-level feedback on the geometric integrity and scene coherence of generated video latents. It evaluates consistency across point, line, plane, perceptual feature, and voxel-level representations, derived by applying specialized perception models to both the generated latent and the reference real-video latent ž,.
3.4.1 Efficient Latent-Space Perception
To avoid computationally expensive full VAE decoding at each step & for perception, our perception models operate directly on latent features. Given noisy video latent 2ą- R xw' (where I, frames, C' channels, H' W" latent dimensions) at diffusion step k, we first employ a lightweight Micro-Decode Module, F micero- This module, constructed using shallow layers from the VAE decoder, processes zμ and & to produce enhanced per-frame features:
fl Fmieral ( k)
(5)
where k is processed by the Fourier Embedding (77 48). f. is suitable for downstream perception tasks, significantly reducing memory and computation. The same Fmiero is applied to 2, (with k = 0) to obtain reference features f.
3.4.2 Perception Model Architectures
We train two models that take these micro-decoded features {fg or {ff) and timestep k as input.
Latent Geometry Perception Model (P (Pgea): This multi-task model processes per-frame features f to assess static 2.5D geometry. It uses DINOv2 (36) backbone followed by task-specific heads:
(1) For vanishing point detection, we reformulate the task as heatmap regression better capture positional uncertainty, following [13]. The network head takes g as input and predicts probability heatmap E R The ground truth heatmap Hot is constructed as 2D Gaussian centered the annotated VP location, balancing localization precision and learning difficulty. We optimize using MSE:
Cup
|| 1
(6)
(2) The lane parsing branch complements this by detecting road markings through a topology- segmentation head, following 42
(3) for the depth estimation task, we fine-tune the weight of 655 mic (SiLog) loss:
using a scale-invariant logarith-
Ldepth = -
A
2
d₂2²
n
where d; logy: logy," Yi is the predicted results and X E [0, 1]
Finally, we optimize Pgeo using the total loss Lgeo = Lop + Ltane + Edepti--
Latent Occupancy Prediction Model (Poc):: This model processes per-frame features p using an image backbone followed by prediction head to infer the intermediate backbone features feat... and the 3D occupancy grid O E xxXxxxx representing the occupied space around the ego vehicle for that frame. Poce is trained based on [16].

---

3.4.3 Multi-Granularity Reward Signals
The hierarchical geometric reward R = Rgeo t Roce quantifies geometric fidelity and scene coherence through a multi-scale decomposition.
For Rgeo. it addresses vanishing point consistency, lane topology validity, and depth coherence. Given the outputs (Pops Lepred; Dpredj from P Paea (Zt) and reference conditions (Urefa Loef, Dres) from P (ze) we define:
Rgeo Rzk zu) )
Uves) + Atane (Ptane: Lres) + AdeythTdeppe (pdepth Dres)
(8)
Point
where PopsPop-Pop are the predictions from P geo for vanashing point, lanes and depth, respectively. Ampddup and App weights balancing the contribution of each task. The individual reward functions
r(.) are designed to be high for geometrically accurate samples and low (or negative) for inconsistent ones:
Top (Pup. Uref) PPpup Uref 112
where Uref is the ground-truie vanishing point in conditions.
Plane
FI FI-Score(Lppead Liee)
(10)
where F1-Score indicates the FI-Scores.
deppth - (Dpred ① Mroad Dref ? Mroad)2 2 +
/ (D (Dpred @ Moehiete Dres O Muchicle )
(11)
where pixel masks Mroad and Moehiele are generated from real data and identify road and vehicle regions, respectively.
After that, we define Roce(Et, ze) = Talign + Tiou. Talign encourages similarity in high-level scene interpretation by aligning distributions of intermediate occupancy features. For each frame f:
Talign, - D (p( (ffffttttttttt one, Ilp( feat gen oce,,)); ));
(12)
where fcat f are backbone features from Poee- fit over a batch).
Distributions p(-) can be estimated (e.g., Gaussian
Toce promotes accurate 3D structure and object layout. For each frame f:
10gem nOgeal 10gmm U real
Tiony IoU .ppaa)=
(13)
where O are the 3D occupancy grids from Poec-
4
Experiments
4.1 Experimental Setup
We conduct comprehensive experiments to validate the efficacy of our proposed Reinforcement Learning with Geometric Feedback (RLGF) framework. This section details the datasets, baseline models, evaluation metrics, and implementation specifics used in our evaluation.
Datasets. Our experiments are primarily conducted on nuScenes [5] using the official validation split. Its multi-camera setup and comprehensive annotations, including 3D object labels and HD maps, provide challenging benchmark. For this dataset, conditions e for the diffusion models are extracted from ground truth annotations, simulating realistic inputs for controllable generation.
Baselines. We demonstrate the plug-and and- nature of RLGF by integrating it with two representative video diffusion models, referred to as MagicDrive- V2 9 and DiVE [88
Evaluation Metries. GeoScores: We use proposed GeoScores suite: Vanishing Point Error: NormDist between the predicted VP and the pseudo-ground truth VP derived from real data. Lower better. Lane Topology Score: Fl-score for semantic segmentation of lane markings against ground
7

---

Table 1: Performance of our Latent Geometry Perception Model gee the nuScenes
validation split. Pgeo operates directly on micro-decoded latent features and is compared against representative pixel-space baselines.
Task
Metric
Model Method
Input Space
Performance
URVP 29
Pixel
0.045
VP Detection
NormDist
VPD
14
Pixel
0.032
geo
Latent
0.024
LaneATT 47
Pixel
0.822
Lane Parsing
Fl-Score
PriorLane 440
Pixel
0.879
P
Latent
0.865
DnythAny 66
Pixel
1.798
Depth Estimation
RMSE 4
Pgo
Latent
2.596
Table 2: Comparison with state-of-the-art video generation methods on the nuScemes validation set. We evaluate visual quality (FVD), 3D Object Detection (3DOD) performance, and geometric fidelity (GeoScores).
Quality FVD
3DOD
GeoScore Lane
Methods
MAP
NDS
VP
Depth
Real Data
35.53
41.20
Panacea
[57
139.0
11.38
22.31
Drive-W W
[57
122.7
20.66
MagicDrive-v2
18
101.2
18.95
21.10
0.092
0.787
1.732
DiVE 18
68.4
25.75
33.61
0.086
0.792
1.822
MagicDrive-12HOurs
99.8
23.21
27.80
0,079
0.854
0.983
DiVE+Ours
67.6
31.42
36.07
0,068
0.879
0.772
truth lane masks. Higher is better. Depth Error: We use RMSE between predicted depth for road surface regions and pseudo-ground truth depth. Lower is better.
Downstream Task Performance: 3D Object Detection: We employ a strong BEV-based detector, BEVFusion (26), trained solely synthetic data generated by different methods real data. We report the standard nuScenes detection score (NDS) and mean Average Precision (mAP).
Visual Realism: We report Fréchet Video Distance (FVD) 449 to ensure RLGF does not degrade the visual quality achieved by the baseline diffusion models.
Implementation Details are included the supplemental materials.
4.2
Performance of Pre-trained Perception Models
We first verify the capabilities of perception models (Pgeo and Poce which form the basis of our reward functions. These models operate directly on latent features 2; and timestep t. Our goal here to demonstrate their competence in extracting meaningful geometric and scene information from the latent domain.
Latent Geometry Perception Model (Paea) is multi-task model responsible for assessing fine- grained geometric properties from latent features. validation split for vanishing point (VP) detection, lane parsing, and depth estimation, compared against established pixel-space methods. The results indicate that Pyeo effectively captures these geometric cues from latent features. For instance, it achieves normalized distance error of 0.024 for detection and an Fl-score of 0.865 for lane parsing. Although its depth estimation RMSE (2.596) higher than the specialized pixel space model DepthAnything-v2 it still provides consistent measure of depth relationships.
its performance on the nuScenes
Latent Occupancy Model (Poce) is tasked with understanding 3D scene layout from sequences of
latent features. performance on the Oce3D-nuSeenes is shown Table 3 Poee achieves an overall mloU of 29.96 when operating from latent representations. This level of performance, compared to pixel-based method like FlashOcc (32.08 mloU), demonstrates solid capability infer volumetric scene structure from the latent domain.

---

Baseline
Ours
Real Data
Figure 3: Qualitative comparison of 3D object bounding box alignment. RLGF-enhanced video exhibits much- much-improved 3D box alignment, closely matching the geometry implied by the scene.
SyageSttie / M Figure 4: Left: Detecfoö Yelifss real nuScenes image. Right. Beeeenof Yesults on corre- sponding synthetic image generated by the DiVE baseline. Bounding boxes indicate detected objects (primarily vehicles).
4.3 Main Results: Improving Geometric Fidelity and Downstream Tasks
Table[[shows RLGGF's impact on the nuScenes validation set. RI.GF consistently enhances geometric integrity (GeoScores) across baselines while maintaining or improving visual quality (FVD), For instance, DiVE RLGF significantly improves all GeoScore components (VP error: 0.086 - 0.068; Lane Fl: 0.792 +0,87; Depth RMSE: 1,822 - 0.772).
Crucially, this geometric enhancement translates to substantial 3DOD gains: DIVE RLGF boosts mAP from 25.75 31.42 and NDS from 33.61 to 36.07, markedly closing the gap real data performance. Similar improvements are seen for MagicDrive v2 + RLGF (mAP: 18.95 - 23.21). These results underscore RLGF's effectiveness as a plug-and-play module for improving both geometric soundness and downstream utility of synthetic videos. We also present the qualitative result fig. 3] about the detection results on our synthetic data and baseline [T8] and the real data. (More qualitative visualization results and further analysis are included in the supplementary materials).
4.4 2D Object Detection Results
To illustrate that current diffusion models like DiVE can generate visually realistic data with minimal 2D domain gap for certain tasks, we present qualitative 2D object detection results. 4 shows outputs from a YOLOv5 [20] detector applied to (a) real nuScenes data and (b) synthetic data generated by the DiVE baseline. The detector is pre-trained on large-scale dataset (c.g.. COCO) and then fine-tuned on real nuScenes training data.
As observed in Figure the 2D detection performance on DiVE-generated synthetic data is qualita- tively very similar that on real data. Objects are generally detected with comparable confidence and bounding box accuracy. This visual consistency aligns with our quantitative findings (mAP: 43.8 on synthetic vs. 44.7 on real, as mentioned in the Introduction), suggesting that the semantic content and 2D appearance features necessary for 2D detection are well-preserved in the synthetic videos. This further reinforces our hypothesis that the primary limitation of such synthetic data lies its 3D geometric fidelity, which is specifically addressed by RLGF framework.

---

Table 3: Performance of our Latent Occupancy Prediction Model (Poce) the Oce3D-nuScenes. Pace predicts 3D occupancy grids from sequences of micro-decoded latent features. Vehicle and Dri. Sur indicate loU of vehicles and driving surface.
Table 4: Ablation study of HGA reward components within RLGF on nuScenes.
HGA Rewards
3DOD
ID
Tup
Flane
deepth
Falign
Fiou
mAPt
NDS1
DIVE
18
25.75
33.61
1
V
26.31
33.66
2
V
26.93
33,98
Method
Vehicle
Dri. Sur mloU
3
27.12
34.82
FlashOce
70
43.2
72.2
32.08
28.06
35.11
Pooc
37.9
65.7
29.96
Full
31.42
36.07
Table 5: Ablation studies on key RLGF hyperparameters.
Hyperparameter
Value
3D Detection mAP
3
30.89
Window Size (w)
5 (Ours)
31.42
8
31,25
Equal Weights (all 0.2)
30.76
Reward Weights (A)
Balanced (Ours)
31.42
Early (Random in [20, 30))
30.55
Window Range Position (t))
Mid (Random in [8, 30] (Ours)
31.42
Late (Random in [1, 15])
29.91
4.5
Ablation Study
4.5.1
Ablations about HGA Reward Components
We conduct ablation studies on nuScenes using DiVE as the baseline to understand the contributions of different HGA reward components. Results shown in Table Incrementally adding reward components (Fap-"lanee-"depth and occupancy rewards (Tatign + Tion) progressively improves 3DOD performance (Table 4). For example, adding point-line-plee
rewards (ID: 3) boosts DiVE's mAP to 27.12. Incorporating only occupancy rewards (ID; 4) yields 28 30666 mAP. The full HGA system, combining all five components, achieves the highest mAP (31.42) and NDS (36.07), highlighting the synergistic benefits of our comprehensive multi-level feedback. (More qualitative visualization results and further analysis are included in the supplementary materials).
4.5.2 A Ablations about Hyerparameters.
We provide the detailed ablation results in table clarity, including the RL window size, reward weights, and the starting step of the sliding window. All ablations are performed on the DiVE baseline and evaluated with BEVFusion.
5 Conclusion
In this work, we addressed the critical issue of geometric distortions in diffusion-based video generation for autonomous driving. We introduced GeoScores for quantitative evaluation and proposed Reinforcement Learning with Geometric Feedback (RLGF), a novel framework to enhance the geometric integrity of synthetic videos. RLGF, through its Hierarchical Geometric Alignment (HGA) module which incorporates multi-level geometric and scene occupancy feedback, effectively injects ceception-driver constraints into the generation process by fine-tuni fine-tuning pre-trale pre-trained diffusion models. Our experiments demonstrate that RLGF significantly improves geometric fidelity across multiple baselines and, crucially, boosts downstream 3D object detection performance by up substantially closing the gap with real-data performance. This work establishes a new direction for generating more reliable and task-aware synthetic data for autonomous systems.
10

---

Acknowledgments
This work was supported by the Science and Technology Development Fund of Macau SAR (FDCT) under grants 0102/2023/RIA2 and 0154/2022/A3 and 001/2024/SKL, and the Jiangyin Hi-tech Industrial Development Zone under the Taihu Innovation Scheme (EF2023-00033KKL-10T3)))
References
[1] Alexander Black, Simon Jenni, Tu Bui, Md Mehrab Tanjim, Stefano Petrangeli, Ritwik Sinha, Viswanathan Swaminathan, and John Collomosse. Vader: video alignment differencing and retrieval. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 22357-22367, 2023.
[2] Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov. and Sergey Levine. Training diffusion models with reinforcement learning. arXiv preprint arXiv:2305. 2023.
[3] Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Do- minik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:231 1.111.1,,,,,,. 2023.
[4] Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22563-22575 2023.
[5] Holger Caesar, Varun Bankiti, Alex Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11621-11633 2020.
[6] Shaoyu Chen, Bo Jiang, Hao Gao, Bencheng Liao, Qing Xu, Qian Zhang, Chang Huang, Wenyu Liu, and Xinggang Wang. Vadv2: End-to-end vectorized autonomous driving via probabilistic planning. arXiv preprint arXiv:2402.1322 2024.
[7] Zhangquan Chen, Xufang Luo, and Dongsheng Li. Visrl: reinforced reasoning. arXiv preprint arXiv:2503.07523, 2025.
visual perception via
[8] Zeyu Dong, Yuyang Yin, Yuqi Li, Eric Li, Hao-Xiang Guo, and Yikai Wang. Panolora: Bridging perspective and panoramic video generation with lora adaptation. arXiv preprint arXiv:2509.11099 1092, 2025.
[9] Ruiyuan Gao, Kai Chen, Bo Xiao, Lanqing Hong, Zhenguo Li, and Qiang Xu. Magicdrivedit: High-resolution long video generation for autonomous driving with adaptive control. arXiv preprint arXiv:2411. 2024.
[10] Ruiyuan Gao, Kai Chen, Enze Xie, Lanqing Hong, Zhenguo Li, Dit- Ra-Yan Yeung, and Qiang Xu. Magicdrive: Street view generation with diverse 3d geometry control. arXiv preprint arXiv:2310002011, 2023.
[11] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63(1 13:19--144, 2020.
[12] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
[13] Hiroto Honda, Motoki Kimura, Takumi Karasawa, and Yusuke Uchida. End-to-end monocular vanishing point detection exploiting lane annotations. arXiv preprint arXiv:2 /3699, 2021.
[14] Hiroto Honda, Motoki Kimura, Takumi Karasawa, and Yusuke Uchida. End-to-end monocular vanishing point detection exploiting lane annotations. arXiv preprint arXiv:2108.13699 2021.
11

---

[15] Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, et al. Planning-oriented autonomous driving. In Proceedings of the IEEF/CVE conference computer vision and pattern recognition, pages 17853-17862, 2023.
[16] Yuanhui Huang, Amonnut Thammatadatrakoon, Wenzhao Zheng, Yunpeng Zhang, Dalong Du, and Jiwen Lu. Probabilistic gaussian superposition for efficient 3d occupancy prediction. arXiv preprint axivv2442..44384 2024.
[17] Bo Jiang, Shaoyu Chen, Qing Xu, Bencheng Liao, Jiajie Chen, Helong Zhou, Qian Zhang, Wenyu Liu, Chang Huang, and Xinggang Wang. Vad: Vectorized scene representation for efficient autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8340-8350, 2023.
[18] Junpeng Jiang, Gangyi Hong. Miao Zhang, Hengtong Hu, Kun Zhan, Rui Shao, and Liqiang Nie. Dive: Efficient multi-view driving scenes generation based on video diffusion transformer. arXiv preprint arXiv:2504.19611 2025.
[19] Junpeng Jiang, Gangyi Hong. Miao Zhang, Hengtong Hu, Kun Zhan, Rui Shao, and Liqiang Nie. Dive: Efficient multi-view driving scenes generation based on video diffusion transformer. arXiv preprini arXiv:2504. 19614, 2025.
[20] Glenn Jocher, Alex Stoken, Jirka Borovec, Liu Changyu, Adam Hogan, Laurentiu Diaconu, Jake Poznanski, Lijun Yu, Prashant Rai, Russ Ferriday, al. ultralytics/yolovS: v3. 0. Zenodo, 2020.
[21] Diederik Kingma, Max Welling, et al. Auto-encoding variational bayes, 2013.
[22] Lingdong Kong, Wesley Yang, Jianbiao Mei, Youquan Liu, Ao Liang, Dekai Zhu, Dongyue Lu, Wei Yin, Xiaotao Hu, Mingkai Jia, et al. 3d and 4d world modeling: A survey, arXiv preprint vrXiv:2509.07996, 2025.
[23] Bohan Li, Jiazhe Guo, Hongsi Liu, Yingshuang Zou, Yikang Ding, Xiwu Chen, Hu Zhu, Feiyang Tan, Chi Zhang, Tiancai Wang, et al. Uniscene: Unified occupancy-centric driving scene generation. arXiv preprini arXiv:2412.05445, 2024.
[24] Jiachen Li. Weixi Feng, Tsu-Jui Fu, Xinyi Wang. Sugato Basu, Wenhu Chen, and William Yang Wang. T2v-turbo: Breaking the quality bottleneck of video consistency model with mixed reward feedback. arXiv preprint arXiv:2405.18755 2024.
[25] Ming Li, Taojiannan Yang, Huafeng Kuang, Jie Wu, Zhaoning Wang, Xuefeng Xiao, and Chen Chen. Controlnet++: Improving conditional controls with efficient consistency feedback: Project page: liming-ai. github. diccottronnet_pls_pppl In European Conference on Computer Vision, pages 129-147. Springer, 2024.
[26] Tingting Liang, Hongwei Xie, Kaicheng Yu, Zhongyu Xia, Zhiwei Lin, Yongtao Wang, Tao Tang, Bing Wang, and Zhi Tang. Bevfusion: A simple and robust lidar-camera fusion framework. Advances in Neural Information Processing Systems, 15:10021-10433 2022.
[27] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling, arXiv preprint arXiv:2210.02 2022.
[28] Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03000 2022.
[29] Yin-Bo Liu, Ming Zeng, and Qing- Meng. Unstructured road vanishing point detection using convolutional neural networks and heatmap regression. IEEE Transactions on Instrumentation and Measurement, 70:1-8, 2020.
[30] Enhui Ma, Lijun Zhou, Tao Tang. Zhan Zhang, Dong Han, Junpeng Jiang, Kun Zhan, Peng Jia, Xiampeng Lang, Haiyang Sun, et al. Unleashing generalization of end-to-end autonomous driving with controllable long video generation. arXiv preprint 24Xiv2240..03344 2024.
[31] Jianbiao Mei, Tao Hu, Xuemeng Yang, Licheng Wen, Yu Yang, Tiantian Wei, Yukai Ma, Min Dou, Botian Shi, and Yong Liu. Dreamforge: Motion-aware autoregressive video generation for multi-view driving scenes. arXiv preprint arXiv:24409044003 2024.
12

---

[32] Chaojun Ni, Xiaofeng Wang, Zheng Zhu, Weijie Wang, Haoyun Li, Guosheng Zhao, Jie Li, Wenkang Qin, Guan Huang, and Wenjun Mei. Wonderturbo: Generating interactive 3d world in 0.72 seconds. arXiv preprint arXiv:2504.02661, 2025.
[33] Chaojun Ni, Guosheng Zhao, Xiaofeng Wang, Zheng Zhu, Wenkang Qin, Xinze Chen, Guanghong Jia, Guan Huang, and Wenjun Mei. Recondreamer-ll: Enhancing reinforcement learning via diffusion-basee scene reconstruction. arXiv preprint arXiv:2508.08811 081700 2025.
[34] Chaojun Ni, Guosheng Zhao, Xiaofeng Wang, Zheng Zhu, Wenkang Qin, Guan Huang, Chen Liu, Yuyin Chen, Yida Wang, Xueyang Zhang, et al. Recondreamer: Crafting world models for driving scene reconstruction via online restoration. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 1559-1569, 2025.
[35] Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference machine learning, pages 8162-8171. PMLR, 2021.
[36] Maxime Oquab, Timothée Darcet, Theo Moutakanni, Huy V. Vo. Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaacldin El-Nouby, Russell Howes, Po- Yao Huang, Hu Xu, Vasu Sharma, Shang- Wen Li, Wojeiech Galuba, Mike Rabbat, Mido Assran, Nicolas Ballas, Gabriel Synnaeve, Ishan Misra, Herve Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. Dinov2: Learning robust visual features without supervision, 2023.
[37] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730-27744, 2022.
[38] Yu Qian, Xunhao Li, Jian Zhang, Xiaolin Meng. Yongfu Li, Heng Ding, and Maoze Wang. diffusion-tgan framework for spatio-temporal speed imputation and trajectory reconstruction. IEEE Transactions on Intelligent Transportation Systems, pages 1-15, 2025.
[39] Yu Qian, Jian Zhang, Zhanyu Feng, Xunhao Li, Zhiyuan Liu, and Hua Wang. Multi-task itransformer: A saturation- based model for short-term highway traffic congestion prediction considering event-induces capacity variability. IEEE Transactions on Vehicular Technology. pages 1-15, 2025.
[40] Qibo Qiu, Haiming Gao, Wei Hua, Gang Huang, and Xiaofei He. Priorlane: A prior knowledge enhanced lane detection approach based on transformer. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 5618-5622 IEEE, 2023.
[41] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728-53741, 2023.
[42] Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao- Wu, Ross Girshick, Piotr Dollár, and Christoph Feichtenhofer. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.007/1 2024.
[43] Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang. Grounded sam: Assembling open-world models for diverse visual tasks, 2024.
[44] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björ Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 10684-10695, 2022.
[45] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint 1rX1v:1707.06347. 2017.
13

---

[46] Alexander Swerdlow, Runsheng Xu, and Bolei Zhou. Street-view image generation from bird's eeye view layout. IEEE Robotics and Automation Letters, 2024.
[47] Lucas Tabelini, Rodrigo Berriel, Thiago M. Paix ao, Claudine Badue, Alberto Ferreira De Souza, and Thiago Oliveira-Santos. Keep your Eyes on the Lane: Real-time Attention-guided Lane Detection. In Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
[48] Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Kein Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains. Advances in neural information processing systems, 33:7537-7547, 2020.
[49]
Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphaël Marinier, Marcin Michalski, and Sylvain Gelly. Fvd: A new metric for video generation. 2019.
[50] Bram Wallace, Meihua Dang, Rafael Rafailov, Lingi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong. Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 9228-8238. 2024.
[51] Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wa Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang. Jingren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, Pandeng Li, Pingyu Wu, Ruihang Chu, Ruili Feng, Shiwei Zhang, Siyang Sun, Tao Fang, Tianxing Wang. Tianyi Gui, Tingyu Weng. Tong Shen, Wei Lin, Wei Wang. Wei Wang. Wenmeng Zhou, Wente Wang, Wenting Shen, Wenyuan Yu, Xianzhong Shi, Xiaoming Huang, Xin Xu, Yan Kou, Yangyu Lv, Yifei Li, Yijing Liu, Yiming Wang, Yingya Zhang, Yitong Huang. Yong Li, You Wu, Yu Liu, Yulin Pan, Yun Zheng, Yuntao Hong, Yupeng Shi, Yutong Feng, Zeyinzi Jiang, Zhen Han, Zhi-Fan Wu, and Ziyu Liu. Wan: Open and advanced large-scale scale video generative models. arXiv preprint arXiv: arXi:25033.20344 2025.
[52] Shihao Wang, Yingfei Liu, Tiancai Wang. Ying Li, and Xiangyu Zhang. Exploring object-centric temporal modeling for efficient multi-view 3d object detection. arXiv preprint arXiv:2303.11926 2023.
[53] Xiaofeng Wang, Zheng Zhu, Guan Huang, Xinze Chen, Jiagang Zhu, and Jiwen Lu. Drive- dreamer: Towards ence on Computer Vision, pages 55-72. Springer, 2024.
world models for autonomous driving. In European Confer-
[54] Yujia Wang, Fang-Lue Zhang, and Neil A Dodgson. Scantd: 360 scanpath prediction based on time-series diffusion. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 7764-7773, 2024.
[55] Yujia Wang, Fang-Lue Zhang, and Neil A Dodgson. Target scanpath-guided 360-degree image enhancement. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 8169-8177, 2025.
[56] Yuqi Wang, Jiawei He, Lue Fan, Hongxin Li, Yuntao Chen, and Zhaoxiang Zhang. Driving into the future: Multiview visual forecasting and planning with world model for autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 14749-14759, 2024.
[57] Yuqing Wen, Yucheng Zhao, Yingfei Liu, Fan Jia, Yanhui Wang, Chong Luo, Chi Zhang, Tiancai Wang, Xiaoyan Sun, and Xiangyu Zhang. Panacea: Panoramic and controllable video generation for autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6902-6912, 2024.
[58] Yichen Xie, Chenfeng Xu, Chensheng Peng, Shuqi Zhao, Nhat Ho, Alexander T Pham, Mingyu Ding, Masayoshi Tomizuka, and Wei Zhan. X-drive: Cross-modality consistent multi-sensor data synthesis for driving scenarios. arXiv preprint arXiv:2411.01 123, 2024.
[59] Yi Xin, Qi Qin, Siqi Luo, Kaiwen Zhu, Juncheng Yan, Yan Tai, Jiayi Lei, Yuewen Cao, Keqi Wang, Yibin Wang, et al. Lumina-dimoo: An omni diffusion large language model for multi-modal generation and understanding, arXiv preprint arXiv:2510.06308, 2025.
14

---

[60] Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36: 15903-1555,3, 2023.
[61] Tianyi Yan, Dongming Wu, Wencheng Han, Junpeng Jiang, Xia Zhou, Kun Zhan, Cheng-zhong Xu, and Jianbing Shen. Drivingsphere: Building a high-fidelity 4d world for closed-loop simulation. arXiv preprini 1r 1252, 2024.
[62] Tianyi Yan, Junbo Yin, Xianpeng Lang, Ruigang Yang, Sheng-Zhong Xu, and Jianbing Shen. Olidm: Object-aware lidar diffusion models for autonomous driving. In Proceedings of the AAAI Conference on Artificial Intelligncee volume 39, pages 9121-9129, 2025.
[63] Kai Yang, Jian Tao, Jiafei Lyu, Chunjiang Ge, Jiaxin Chen, Weihan Shen, Xiaolong Zhu, and Xiu Li. Using human feedback to fine-tune diffusion models without any reward model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8941-8951, 2024.
[64] Kairui Yang. Enhui Ma, Jibin Peng, Qing Guo, Di Lin, and Kaicheng Yu. Bevcontrol: Accurately controlling street-view elements with multi-perspective consistency via bev sketch layout. arXiv preprint arXiv:2308.0166) 2023.
[65] Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. arXiv:2406.094: 2024.
[66] Xiaomeng Yang, Zhiyu Tan, and Hao Li. Ipo: Iterative preference optimization for text-to-video generation. arXiv preprint wrxiv:2502.02088 2025.
[67] Xuemeng Yang, Licheng Wen, Yukai Ma, Jianbiao Mei, Xin Li, Tiantian Wei, Wenjie Lei, Daocheng Fu, Pinlong Cai, Min Dou, et al. Drivearena: A closed-loop generative simulation platform for autonomous driving, arXiv preprint arXiv:2408 00415, 2024.
[68] Zhongqi Yang, Wenhang Ge, Yuqi Li, Jiaqi Chen, Haoyuan Li, Mengyin An, Fei Kang, Hua Xue, Baixin Xu, Yuyang Yin, et al. Matrix-3d: Omnidirectional explorable 3d world generation. arXiv preprint arXiv:2508.008 08086, 2025.
[69] Mingyang Yi, Aoxue Li. Yi Xin, and Zhenguo Li. Towards understanding the working mecha- nism of text-to-image diffusion model. Advances in Neural Information Processing Systems, 37:55342 553222,5336611 2024.
[70] Zichen Yu, Changyong Shu, Jiajun Deng, Kangjie Lu, Zongdai Liu, Jiangyong Yu, Dawei Yang, Hui Li, and Yan Chen. Flashocc: Fast and memory-efficient occupancy prediction via channel-to-height plugin. 2023.
[71] Hangjie Yuan, Shiwei Zhang, Xiang Wang, Yujie Wei, Tao Feng. Yining Pan, Yingya Zhang, Ziwei Liu. Samuel Albanic, and Dong Ni. Instructvideo: Instructing video diffusion models with human feedback. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6463-6474, 2024.
[72] Shuang Zeng, Xinyuan Chang, Xinran Liu, Zheng Pan, and Xing Wei. Driving with prior maps: Unified vector prior encoding for autonomous vehicle mapping. arXiv preprins arXiv:2409.0S: 05352, 2024.
[73] Shuang Zeng, Xinyuan Chang, Mengwei Xie, Xinran Liu, Yifan Bai, Zheng Pan, Mu Xu, and Xing Wei. Futuresightdrive Thinking visually with spatio-temporal cot for autonomous driving. arXiv preprint arXiv:2505. /7685, 2025.
[74] Jiacheng Zhang, Jie Wu, Weifeng Chen, Yatai Ji, Xuefeng Xiao, Weilin Huang, and Kai Han. Onlinevpo: Align video diffusion model with online video-centric preference optimization. arXiv preprint ArXiv: 15159, 2024.
[75] Rongchao Zhang, Yu Huang, Yiwei Lou, Weiping Ding, Yongzhi Cao, and Hanpin Wang. Synergistic attention- guided cascaded graph diffusion model for complementarity determining region synthesis. IEEE Trans. Neural Networks Learn. 3167) 2025.
15

---

[76] Guosheng Zhao, Xiaofeng Wang, Zheng Zhu, Xinze Chen, Guan Huang, Xiaoyi Bao, and Xingang Wang. Drivedreamer-2: Lim-enhanced world models for diverse driving video gen- eration. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 10412-10420, 2025.
[77] Zangwei Zheng, Xiangyu Peng. Tianji Yang. Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all. arXiv preprint arXiv:2412.20404 2024.
16

---

Appendix
This supplementary material provides additional details to support the findings presented in our main paper. We include: (1) comprehensive implementation specifics for our RLGF framework and the pre-trained perception models; further details on the GeoScores metric computation; (2) additional experimental results (3) discussion on the limitations of our current work and potential future directions.
A Detailed Implementation Details
This section elaborates on the implementation details of our proposed RLGF framework, the pre- trained perception models (P (Pgeo geo and Pocc) and the experimental setup.
A.1 Dataset Preparation
All experiments, including the pre-training of our perception models (Pgeo and P Pace) and the fine- tuning of diffusion models with RLGF, are conducted using the nuScenes dataset [5]. We primarily utilize the official training and validation splits. While nuScenes provides rich annotations like 3D bounding boxes and HD maps, it does not directly offer ground truth labels for vanishing points (VP), dense segmentation masks for all relevant classes (like fine-graing fine-grained lanes beyond HD map polylines), or per-pixel depth maps required by our Pgea- Therefore, we generate high-quality pseudo-labels for these tasks using strong, pre-existing perception models, detailed below. These pseudo-labels labels serve as the training targets for our latent-space perception models.
Depth Pseudo-Labels: To obtain dense depth information for training the depth estimation com- ponent Pgeo. we utilize Depth Anything V2 (vit-l version) [65]. This state-of-the-an monocular depth estimation model is applied to all images in the nuScenes training set to generate per-pixel depth maps. These output depth maps serve as the pseudo-grooe truth for our latent depth estimation task.
Semantic Segmentation Pseudo-Labels (Lanes, Road Surface, Vehicles): For precise segmentation masks of various scene elements, we employ Crounded-SAM-21 42 For the lanes, the model prompted to accurately segment visible lane markings. The resulting binary segmentation masks are used as pscudo-ground truth for training the lane parsing head of Pgeo. For the road surface and vehicle masks, SAM-2 is also utilized to generate segmentation masks for road surfaces and vehicles.
With accurate lane segmentation masks obtained via SAM-2 2 (as described above). we derive vamishing point pseudo-labels through a geometric procedure. For each detected lane marking in a frame: The center point of the lanc marking calculated from its left and right edges (derived from the SAM-2 segmentation mask) for every horizontal image line at 5-pixel intervals. These extracted center points are grouped to represent the centerline of each individual lane marking. Robust curve fitting (e.g., RANSAC with line model) applied to these centerlines. The intersection point of multiple fitted lane centerlines is then computed to determine the scene's vanishing point. This computed VP serves as the greudo-ground truth for the VP detection task.
Vanishing Point Pseudo-Label from Lane Masks, following [13]:
The use of these high-quality pseudo-labels enables us to train effective latent-space perception models tailored to the nuScenes domain, which subsequently provide the nuanced reward signals for our RLGF framework. The conditions for the main diffusion models (e.g., semantic 3D boxes for some baselines) are derived from the original nuScenes ground truth annotations.
A.2 Perception Model Architectures and Pre-training
Micro-Decode Module(Fmmioro): The Fmicro module is constructed using the first upper block of the official VAE decoder from the OpenSora [77 used by our bascline video diffusion models. The
input Fmicro is the noisy latent feature 2f for a frame f with the timestep k. The same Fmiero architecture used when processing the reference real-video latent 2, (with & typically set 0).
Latent Geometry Perception Model(P,goo:: We use a pre-trained DINOV2-VIT-S/14 (6) as the backbone feature extractor and the pre-trained weight from DepthAnything- V2 (65). P geo is trained
17

---

for 50 epochs on the nuScenes training split using the Adam W optimizer with learning rate of 5 x 10-5 and batchsize of 16. We use 8x NVIDIA A100 GPUs cover the experiment.
Latent Occupancy Prediction Model(Poed oec): Pece is trained on ocuceD-ssseeess dataset for 24 epochs using A optimizer following 770 16
A.3
RLGF Fine-tuning Details
Baselines: We used publicly available checkpoints for MagicDrive-Vi V2 9 and DiVE [8]
LoRA Configuration: For LoRA, we applied it to the attention layers (Q, K. V projections) of the DiT backbone in the diffusion models. We used rank
16 following [
Latent-Space Windowed Optimization: The window size w set to The starting step 1' for the window was randomly sampled from the range [8, 30), with T = 30 is the total number of diffusion steps.
Reward Weights: We set Aup 0.1, Atone = 0.1, Adepth 0.5. These weights were determined empirically based on early experiments on a small validation subset, aiming to balance the scale of individual reward components and their perceived impact on generation quality.
We Adam W with learning rate of 1 x 10 and batchsize of 1 with 8 frames per video clip.
A.4 GeoScores Metric Details
This section provides further clarification the computation of our GeoScores components. For all GeoScores, the "reference ground truth" is derived by applying the corresponding pre-trained perception model to the real video data, following appendix A.1 The score then measures the deviation of the synthetic video's perception output from this real real-data-derived reference.
Vanishing Point Error (VP.):: Calculated the L2 Normalized Distance (NormDist) between the calculated VP on synthetic frame and the VP calculated on a real frame. Lane Topology Score (Lanet): Calculated as the Fl score for semantic segmentation of lane markings. The predictions is from Grounded-S AAM2 43 42 on the synthetic frame, and the target is applied to the real frame. Depth Error (Depth){]: Calculated the Root Mean Squared Error (RMSE) between the depth map predicted by Depth Anything V2 (65) for road surface regions on a synthetic frame and the depth map for the same regions the real frame. Road surface masks obtained from SAM-2 42
B Additional Experiment Results
B.1 Extended 3D Object Detection Results on Multiple Detecters
To further demonstrate the generalizability of the improvements conferred by RLGF, we evaluated the generated synthetic data using an additional slateoof-the-ars 3D object detector, StreamPETR (52), alongside the BEVFusion results presented in the main paper. Table 6 presents the performance (mAP and NDS on nuScenes validation) for StreamPETR and the average performance across both BEVFusion and StreamPETR. Both detectors were trained from scratch solely on the respective synthetic data real data.
B.2 Robustness to Different Downstream Detectors:
To demonstrate generality, we evaluated our method on the stronger StreamPETT detector. RLGF achieved a substantial +4.75% mAP gain, proving geometric improvements are robust and benefit diverse downstream architectures.
C Limitations and Future Work
This section discusses the current limitations of our RLGF framework and GeoScores metric, along- side potential avenues for future research.
Dependence on Perception Models: RLGGF's performance is inherently tied to the accuracy and robustness of the pre-trained perception models (P geoc Poce). Biases errors in these models could
18

---

Table 6: Detailed 3D Object Detection (3DOD) performance on nuScenes validation using Stream-
PETR [22
RLGF applied to MagicDrive-v2 and DiVE.
Quality FVD
BevFusion
StreamPETR
Methods
mAP
NDS
mAP
NDS
Real Data
35.53
41.20
38.01
49.02
Panacea [57
139.0
11.58
22.31
Drive- WM (5
122.7
20.66
MagicDrive-v2 [18
101.2
18.95
21.10
22.77
28.93
DiVE [8]
68.4
25.75
33.61
29.19
36.23
MagicDrive-v2+Ourss
99.8
23.21
27.80
26.01
35.64
DiVE+Ours
67.6
31.42
36.07
33.94
39.68
Method
mÁP (BEVFusion)
DiVE [16] (Baseline)
25.75
Detector Reward
26.51 (+0.76)
Ours (RLGF)
31.42 (+5.67)
Table 7: Comparison of reward signal effectiveness on the DiVE baseline. Our Hierarchical Geometric Reward (HGR) is significantly more effective than a high-level detector-based reward.
propagate into the reward signal and mislead the generation process. Future work could explore jointly training or adapting perception models during RLGF, or using ensembles.
Computational Cost: While Latent-Space Windowed Optimization significantly reduces costs compared to full rollouts, RL--based fine-tuning remains more computationally intensive than standard diffusion model training. Exploring more sample-efficient RL algorithms or distillation techniques could be beneficial.
The current HGA reward combines five components with manually tuned weights. Optimizing these weights automatically or learning more adaptive reward function is promising direction. Furthermore, incorporating even more diverse geometric or physical constraints (e.g., collision avoidance, traffic rule adherence) could further enhance realism.
Reward Design and Balancing:
While demonstrated on nuScenes, further investigation needed to assess RIGFFs generalization capabilities across diverse datasets, environmental conditions (e.g., adverse weather, night scenes not well-represented training), and different diffusion model architectures.
Generalization:
GeoScores Scope: Current GeoScores focus on camera-based geometric aspects. Expanding them to include LiDAR consistency or multi-modal geometric agreement could provide more holistic evaluation.
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
In the supplementary materials
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
20

---

Justification:
not include theoretical results
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Detailed implementation details are included in the supplementary materials.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
21

---

Answer:
[No]
Justification:
We will open source the code once we are ready.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm
wwwoooooolllyy il for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ ccpppp.ccppulli//guiiii wwwiicccguie//eeeeeeeeeis CoodeSubeiiieeee ssi wsionnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Details will be included in the included in the supplementary materials. Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
22

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: httpp:://neuripees http///uris.cppuuupppp/eee http:///.ccccuccccttttccttttteea
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
23

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwppppswpptseeee www.ree.wthhhe....c.. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
24

---

Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
[nsw
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [No]
25

---

Justification:
Only for editing.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy httees////earrppcccc////f/rreees for what should or should not be described.
26


Reviews:
[{"review_id": "pS6ZWa4630", "reviewer": "Reviewer_NpjA", "summary": "The paper presents a novel framework, Reinforcement Learning with Geometric Feedback (RLGF), aimed at improving the geometric fidelity of synthetic video generation for autonomous driving. The authors identify a critical yet underexplored issue in current state-of-the-art video generation models: subtle geometric distortions that limit their utility for downstream perception tasks. RLGF addresses this by incorporating specialized perception models as reward providers to refine video diffusion models. The results demonstrate significant improvements in geometric accuracy and downstream task performance. This is a well-executed and innovative contribution to the field.", "questions": "1. Why choose only these two frozen perception models Pgeo (point-line-plane alignment) and Pocc (scene-level consistency) instead of others?\n2. How to constrain the temporal consistency in this framework?\n3. How are the comparative experiments of different RL Windowsize? It is recommended to add corresponding experimental results.\n4. How robust is the method to different detectors as rewards? It is recommended to add corresponding experimental results.\n5. How to balance the selection of hyperparameters between different geometric constraint signals? It is recommended to add corresponding experimental results.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. Propose the GeoScores metric for systematically evaluating and quantifing the geometric distortion problem in autonomous driving video generation.\n2. Offers a plug-and-play solution for generating geometrically sound and reliable synthetic videos for AD development.\n3. Allow for effective and targeted corrective feedback during both early (global structure formation) and late (detail refinement) stages of geometric synthesis.\n4. The article logic and writing expression are smooth.\n\nWeaknesses：\n1. The experiments are not sufficient. For example, there are no control experiments on the robustness of different GT detectors and the balance between different rewards.\n2. Why VP, Lane, and Depth losses can cover geometric constraints is not fully explained. For example, occlusion relationships and temporal continuity are not included in the above losses.\n3. There is a lack of discussion on the shortcomings of the method and the direction of further improvement.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive and constructive review. We are delighted you found our contributions valuable and address your questions below.\n\n**Regarding Q1 & Q4: On the choice and robustness of reward models.**\nThank you for these excellent and related questions. Our choice of reward model is a cornerstone of our work. It was a deliberate design decision based on providing a reward signal that is dense, stable, and directly addresses the core problem. We validate this approach with the following three key points.\n\n\n*   **a) Why Geometric Rewards:** Our core hypothesis is that flawed 3D geometry is the *root cause* of poor downstream performance. Instead of using a complex, indirect signal, we directly target this issue with our HGR. This multi-level system proposes **point-line-plane alignment** and **scene-level coherence**, providing a dense and comprehensive reward signal. This structure enables fine-grained feedback on specific geometric aspects, which is crucial for effective and stable RL training.\n*   **b) Why Not a Detector Reward:** We experimentally tested this intuitive alternative. As shown in the new Table 1 below, using a 3D detector as a reward is far less effective. This is because detector-based rewards are **sparse** (only rewarding on detected objects) and suffer from **unstable gradients** (due to non-differentiable operations like NMS), providing a poor learning signal for the diffusion model.\n*   **c) Robustness to Downstream Detectors:** To demonstrate generality, we evaluated our method on the stronger **StreamPETR** detector (appendix). RLGF achieved a substantial **+4.75% mAP** gain, proving our geometric improvements are robust and benefit diverse downstream architectures.\n\n**Table 1: Effectiveness of reward signals.**\n| Method | mAP (BEVFusion) |\n| :--- | :---: |\n| DiVE [16] (Baseline) | 25.75% |\n| + Detector Reward | 26.51% (+0.76%) |\n| **+ Ours (RLGF)** | **31.42% (+5.67%)** |\n\n\n**Regarding Q2: Temporal Consistency.**\nTemporal consistency is primarily enforced by our Latent Occupancy Prediction Model ($P_\\text{occ}$), which explicitly operates on video sequences. We will clarify this in Section 3.4.\n\n**Regarding Q3 & Q5: Ablation Study on Hyperparameters.**\nWe provide detailed ablation results in the new Table 2 below, which validate our choices for key hyperparameters. The results show that a window size of `w=5` and our balanced reward weights are optimal. We will add this to the appendix.\n\n**Table 2: Ablation Study about Hyperparameters**\n| Hyperparameter | Value | 3D Detection mAP |\n| :--- | :--- | :---: |\n| **Window Size (w)** | 3 | 30.89 |\n| | **5 (Ours)** | **31.42** |\n| | 8 | 31.25 |\n| **Reward Weights ($\\lambda$)** | Equal Weights | 30.76 |\n| | **Balanced (Ours)**| **31.42** |\n| **Window Range Position ($t'$)** | Early (Random in [20,30]) | 30.55 |\n| | **Mid (Random in [8, 30]) (Ours)**| **31.42** |\n| | Late (Random in [1, 15])| 29.91 |\n\n\n**Regarding W3: Limitation \\& Future Work.** We have expanded the \"Limitations and Future Work\" section in the appendix (Section C)."}, {"review_id": "36XNKuBoAA", "reviewer": "Reviewer_UXxn", "summary": "This manuscript aims to address geometric inaccuracies prevalent in synthetic videos generated for AD. The authors present RLGF (Reinforcement Learning with Geometric Feedback), a reinforcement learning framework tailored to enhance video diffusion models with geometrically coherent outputs. \n\nRLGF introduces two components: \n- Latent-Space Windowing Optimization, providing efficient and targeted reinforcement during intermediate diffusion stages.\n- Hierarchical Geometric Reward (HGR), leveraging latent-space perception models to provide detailed geometric constraints (vanishing point consistency, lane structure coherence, and accurate depth). \n\nThe manuscript further introduces \"GeoScores”, a suite of metrics designed explicitly to quantify geometric fidelity in synthetic videos. \n\nEvaluations on the nuScenes dataset demonstrate good geometric improvements and substantial performance gains in downstream tasks, notably 3D object detection.", "questions": "1. Based on Weakness #1: Can you provide detailed computational complexity analyses or runtime benchmarks for the introduced latent-space windowing optimization and hierarchical rewards?\n\n---\n\n2. Based on Weakness #2: How robust is RLGF to inaccuracies or biases inherent within the perception models used for generating geometric rewards?\n\n---\n\n3. Based on Weakness #3: Have you tested or considered the generalization capability of your method across datasets beyond nuScenes, or across varying sensor configurations and environmental conditions?\n\n---\n\n4. Based on Weakness #4: Can you provide additional insights or experimental results regarding how the choice of window size and position impacts model performance?\n\n---\n\nAdditional questions (minor):\n\n5. It is suggested to provide additional qualitative visualizations to demonstrate intermediate latent states before and after RLGF optimization.\n\n---\n\n6. It is suggested to explicitly clarify hyperparameter choices for RL window size and reward weights.", "limitations": "- The model might require extensive tuning due to multiple interdependent reward components and RL optimization hyperparameters, possibly limiting straightforward applicability.\n\n- The computational complexity introduced by the latent-space optimization and multiple perception-based rewards may limit deployment to high-resource environments, challenging real-time or resource-constrained applications.\n\n- The geometric rewards depend on the performance and reliability of perception models, which might propagate errors or biases into the final synthetic data.\n\n- Without broader evaluations across diverse AD scenarios, the generalizability of this method to other datasets, sensors, or environmental conditions remains uncertain.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "### Strengths\n(+) The manuscript identifies and clearly motivates an important yet overlooked limitation in existing AD synthetic data—geometric distortion affecting downstream perception tasks. The introduction of latent-space windowed RL optimization and hierarchical geometric rewards advances video diffusion training methods. \n\n(+) The GeoScores metric suite provides a way of quantifying geometric inaccuracies. This strengthens the manuscript by defining the problem and providing a reproducible method for evaluation. Experiments demonstrate the efficacy of RLGF. Particularly is the considerable reduction in geometric errors and good improvements in 3D object detection.\n\n(+) The manuscript is overall coherent, structured, and with supporting visualizations and tables that enhance readability and understanding.\n\n---\n\n### Weaknesses\n\n(-) RLGF introduces significant complexity via reinforcement learning in the latent space and multiple perception-based rewards. A deeper exploration into computational demands (runtime, memory usage) and scalability is needed for practical considerations.\n\n(-) The effectiveness of RLGF heavily relies on pre-trained perception models. A thorough analysis or discussion of how inaccuracies or limitations within these perception models may affect the geometric rewards would strengthen robustness claims.\n\n(-) Evaluations are conducted solely on the nuScenes dataset. The generalization of RLGF's performance to other autonomous driving datasets or sensor setups remains unclear and is not sufficiently discussed.\n\n(-) The latent-space windowing optimization seems to have a good motivation, yet critical parameters (e.g., window size and position) are not extensively ablated, potentially obscuring the sensitivity and robustness of the proposed method.", "was_revised": true, "final_justification": "I have read the response, as well as the responses for review comments from other reviewers. \n\nBased on what I learn from the review comments and responses, I believe part of the concerns have been addressed. Therefore, I am leaning towards maintaining the 4 - borderline accept rating. \n\nI suggest that the authors incorporate all clarifications, modifications, and new experiments into the revised manuscript, to ensure that the quality meets what NeurIPS always looks for.", "rebuttal": "Thank you for your thoughtful feedback on the practical aspects of our work.\n\n**Regarding Q1: Computational Complexity.**\nOur Latent-Space Windowing Optimization is the key to making our approach computationally feasible. As the new Table 3 shown below, full rollouts in latent or image space lead to Out-of-Memory (OOM) errors. Our method requires 28 hours of fine-tuning on 8 A100 GPUs, which represents only about 15% additional training time over the baseline. We will add this analysis to the appendix.\n\n\n**Table 3: Computational cost analysis.**\n| Method | GPU Memory | Training Time |\n| :--- | :---: | :---: |\n| DiVE (Baseline) | ~ 49 GB | ~180 hours |\n| Image-Space RL | OOM | N/A |\n| Latent-Space RL (No Windowing) | OOM | N/A |\n| **Ours (Latent-Space + Windowing)** | **+23 GB** | **+28 hours** |\n\n\n**Regarding Q2: Robustness to inaccurate perception models.**\nThis is a critical point. To directly test robustness, we intentionally trained a \"weaker\" perception model (using only 30% of training data, resulting in a ~25% performance drop). Even with this degraded reward signal, RLGF still boosted the downstream mAP by a significant +4.2%. This demonstrates our framework's robustness. We will add this new study to the appendix.\n\n**Regarding Q3: Generalization beyond nuScenes.**\nTo demonstrate generalization, we built a new pipeline on the **CARLA** with Dive. On a baseline of 57.6% mAP, applying RLGF provided a **+5.4% mAP** improvement. This confirms that our method's principles are not dataset-specific.\n\n**Table 4: Generalization results on the CARLA**\n| Method on CARLA Data | 3D Detection mAP |\n| :--- | :---: |\n| Baseline Synthetic Data | 57.6% |\n| **+ Ours (RLGF)** | **63.0% (+5.4%)** |\n\n**Regarding Q4 & Q6:  Ablations and clarifications.**\nWe have noted in the appendix that our parameters were chosen based on empirical experiments, and we provide the detailed ablation results here for clarity, including the RL window size, reward weights, and the starting step of the sliding window. All ablations are performed on the DiVE baseline and evaluated with BEVFusion.\n\n| Hyperparameter | Value | 3D Detection mAP |\n| :--- | :--- | :---: |\n| **Window Size (w)** | 3 | 30.89 |\n| | **5 (Ours)** | **31.42** |\n| | 8 | 31.25 |\n| **Reward Weights ($\\lambda$)** | Equal Weights | 30.76 |\n| | **Balanced (Ours)**| **31.42** |\n| **Window Range Position ($t'$)** | Early (Random in [20,30]) | 30.55 |\n| | **Mid (Random in [8, 30]) (Ours)**| **31.42** |\n| | Late (Random in [1, 15])| 29.91 |"}, {"review_id": "9E0jgy7974", "reviewer": "Reviewer_vAYk", "summary": "This paper proposes reinforcement learning with geometric feedback, to make the generative model learn to capture 3D scene structure rather than only 2D appearance.  The main innovation lies in Latent-Space Windowing RL Optimization to be both effective and efficient, and Hierarchical Geometric Reward to provide multi-level feedback. The results show that the geometric consistency of generated images is greatly improved.", "questions": "* What is reason to choose these perception tasks,  without using detection as a perception task, which aligns better with evaluation?\n* As I am not an expert of RL, could you explain the relationship between your model learning with RL?", "limitations": "No major limitations, see Weaknesses.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n* This work identities the problem of geometric distortions in current generative models, which is important for driving task.\n* Latent-Space Windowing Optimization is introduced to enable learning on noisy latent, without the need for complete sampling chain.\n* Multi-Granularity Reward Signals are proposed to enhance the geometric consistency of generative model on multi-levels.\n* Evaluation results on detection task show great improvement compared to baseline without RLGF.\n\nWeaknesses:\n* There are some typos in paper writing,  e.g. Line 162, fig.2(c), Line 304, up to.\n* The generative model is video diffusion model, while the four geometric perception tasks take image as input, which seems to have the potential for further improvement.", "was_revised": true, "final_justification": "Considering the reviews from other reviewers and the author responses, this paper provides a new RL finetuning method from geometric feedbacks to address geometric inaccuracies prevalent in synthetic videos. The effectiveness of the method is demonstrated by extensive experiments including newly added ones in rebuttal. As my concerns have been solved, I will maintain my rating as \"accept\".", "rebuttal": "We thank you for your positive evaluation and for questions that help clarify our work.\n\n**Regarding Q1: Why not use a detection-based reward?**\nWe tested this experimentally and found it to be ineffective. \nOur choice of reward model is a cornerstone of our work—a deliberate design decision based on principles of signal density, stability, and directness, which we validate with the following three key points.\n\n*   **a) Why Geometric Rewards:** Our core hypothesis is that flawed 3D geometry is the *root cause* of poor downstream performance. Instead of using a complex, indirect signal, we directly target this issue with our HGR. This multi-level system evaluates **point-line-plane alignment** and **scene-level coherence**, providing a dense and stable reward signal. This structure enables fine-grained feedback on specific geometric aspects, which is crucial for effective and stable RL training.\n*   **b) Why Not a Detector Reward:** We experimentally tested this intuitive alternative. As shown in the Table 1 below, using a 3D detector as a reward is far less effective. This is because detector-based rewards are **sparse** (only rewarding on detected objects) and suffer from **unstable gradients** (due to non-differentiable operations like NMS), providing a poor learning signal for the diffusion model.\n*   **c) Robustness to Downstream Detectors:** To demonstrate generality, we evaluated our method on the stronger **StreamPETR** detector (appendix). RLGF achieved a substantial **+4.75% mAP** gain, proving our geometric improvements are robust and benefit diverse downstream architectures.\n\n**Table 1: Effectiveness of reward signals.**\n| Method | mAP (BEVFusion) |\n| :--- | :---: |\n| DiVE [16] (Baseline) | 25.75% |\n| + Detector Reward | 26.51% (+0.76%) |\n| **+ Ours (RLGF)** | **31.42% (+5.67%)** |\n\n**Regarding Q2: Relationship with RL.**\nThank you for the suggestion. We will add a more intuitive explanation: the diffusion model is an **Agent** that learns to take noise-prediction **Actions** to maximize a geometric correctness **Reward** from our perception-model **Environment**."}, {"review_id": "Hga8zQau9h", "reviewer": "Reviewer_hKwg", "summary": "The authors presents RLGF, a method for fine-tuning diffusion-based driving video generators using reinforcement learning with geometric feedback. The method employs a lightweight LoRA modules and optimizing them with a PPO-style RL algorithm. RLGF leverages a hierarchical reward that captures vanishing point alignment, lane structure, depth accuracy, and 3D occupancy. The method significantly reduces geometric distortions of the generated videos, and improves downstream 3D object detection performance on nuScenes trained with synthetic data.", "questions": "1. How would the model perform under a synthetic+real dataset setting, similar to Table 2 of the DiVE paper and Table 3 of the GLAD paper (Glad: A Streaming Scene Generator for Autonomous Driving)? This experiment is crucial in evaluating the practical effectiveness of the proposed method. \n2. A more comprehensive experiment is needed on the StreamPETR detectors instead of BEVFusion, aligning with prior works. \n3. How would the model perform if the video diffusion model is fine-tuned with RL and less capable perception models? It seems that the RL pipeline is heavily dependent on a capable latent perception model. Would the latent perception model hinder the upper bound of the usefulness of the synthetic data generator?", "limitations": "The authors discussed the limitations in the supplementary materials.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength\n1. The paper addresses an important issue in driving video generation, geometric distortion, which can significantly impact downstream 3D perception. The authors provide an effective solution by leveraging RL with geometric feedback and demonstrate solid improvements in both geometric consistency and task performance.\n2. The design of the Hierarchical Geometric Reward (HGR), which incorporates perception models in latent space and uses multi-level geometric cues, is novel and insightful. It could inspire further research in both generative modeling and autonomous driving. \n3. Table 1 and Table 4 provide interesting insights into the effectiveness of latent-space perception and the contribution of each reward component, offering valuable results.\n\nWeaknesses\n1. All experiments are conducted using only synthetic data. Neither the main paper nor the supplementary material includes results that combine real and synthetic data during training, which is essential for evaluating practical effectiveness. For comparison, Table 2 of the DiVE paper includes such experiments. \n2. The evaluation is also limited to 3D object detection. Including more downstream tasks, such as BEV segmentation, object tracking, etc, would make the paper more comprehensive. \n3. The choice to use BEVFusion as the main detector in Table 1 is questionable, as it underperforms compared to stronger models like StreamPETR, which are commonly used in related work. This results in lower baseline results compared with prior works. Although I see a table of using StreamPETR in the supplementary material, the performance of some baseline methods is not complete. \n4. The RL pipeline seems heavily dependent on a capable latent perception model. The performance of the latent perception model might be hinder larger performance growth on larger models / dataset.", "was_revised": true, "final_justification": "From the authors' rebuttal and updated results in Table 5, RLGF improves a strong baseline (DiVE, real+synthetic data setting, StreamPETR model) by 1.4 on mAP and 1.1 on NDS, which is a smaller but valid improvement. The updated Table 6 also shows consistent improvement of RLGF on StreamPETR. I will increase my rating to “borderline accept”.", "rebuttal": "We are very grateful for your expert review, which has pushed us to conduct crucial new experiments that we believe significantly strengthen our paper. We are happy to share the new results below.\n\n**Regarding Q1: `Synthetic+real` dataset setting.**\nFollowing your suggestion, we conducted this crucial experiment, aligning with DiVE and Glad. Table 5 below shows that augmenting real data with our RLGF-enhanced synthetic data provides a significantly larger boost to downstream performance than using baseline synthetic data.\n\n**Table 5: Impact of augmenting real data with synthetic data.**\n\n| Training Data | mAP (StreamPETR) | NDS (StreamPETR)|\n| :--- | :---: | :---: |\n| Real Only (256 × 512)| 34.6 | 47.0\n| Real + Panacea | 36.3 | 48.9\n| Real + Glad | 37.1 | 49.2\n| Real Only (480 × 854)| 38.0 | 49.0\n| Real + DiVE [16] | 40.9 | 52.0\n| **Real + DiVE+Ours** | **42.3 (+1.4)** | **53.1 (+1.1)**\n\n\n**Regarding Q2: On the Completeness of StreamPETR Results.**\nWe apologize that the original table was incomplete and have run the full evaluation on StreamPETR (results below), including other SOTA baselines for a comprehensive comparison. Our key finding remains: RLGF provides a +4.75% mAP gain over the strong DiVE baseline.\nWe agree that models like GLAD show strong performance. We wish to clarify that RLGF is a **plug-and-play refinement technique** designed to improve any diffusion-based generator. We see our work as complementary, not competitive with SOTA generators.  We will apply RLGF to Glad as soon as it releases.\n\n\n**Table 6: Detailed 3D Object Detection (3DOD) performance on nuScenes validation using BevFusion and StreamPETR.**\n\n| Methods | Quality FVD | BevFusion mAP | BevFusion NDS | StreamPETR mAP | StreamPETR NDS |\n| :--- | :---: | :---: | :---: | :---: | :---: |\n| Real Data | - | 35.53 | 41.20 | 34.50 | 46.90 |\n| Panacea | 139.0 | 11.58 | 22.31 | 18.80 | 32.10 |\n| Drive-WM | 122.7 | 20.66 | - | 20.70 | - |\n| MagicDrive-v2 | 101.2 | 18.95 | 21.10 | 22.77 | 28.93 |\n| Glad | 207.0 | - | - | 26.30 | 39.60 |\n| DiVE | 68.4 | 25.75 | 33.61 | 29.19 | 36.23 |\n| MagicDrive-v2+Ours | 99.8 | 23.21 | 27.80 | 26.01 | 35.64 |\n| **DiVE+Ours** | **67.6** | **31.42** | **36.07** | **33.94** | **39.68** |\n\n**Regarding Q3: Dependence on perception models.**\nThis is a fair point. We tested robustness by using a \"weaker\" perception model (with 30% training data and a 25% performance drop). RLGF still improved downstream mAP by **+4.2%**. This demonstrates that our framework is robust, though we agree a better perception model yields better results. We will add this analysis to the appendix."}]

Author rebuttal:

Meta-review: The paper introduces RLGF, a reinforcement learning framework with geometric feedback for improving the geometric fidelity of diffusion-based driving video generation. Reviewers highlighted the novelty of the latent-space optimization and hierarchical geometric reward design, as well as the introduction of GeoScores for systematic evaluation.

While some concerns were raised about evaluation scope, reliance on perception models, and ablations, the rebuttal successfully clarified several issues and presented new experiments. The updated results with stronger baselines show consistent and meaningful improvements. 

Overall, despite some remaining limitations, the paper proposes an original and effective method with demonstrated improvements and clear potential impact for both generative modeling and autonomous driving. Therefore, all reviewers agree on its acceptance.
Decision: Accept (poster)