ReconDreamer-RL: Enhancing Reinforcement Learning via Diffusion-based Scene Reconstruction

## Abstract

Abstract Reinforcement learning for training end-to-end autonomous driving models in closed-loop simulations is gaining growing attention. However, most simulation environments differ significantly from real-world conditions, creating a substantial simulation-to-reality (sim2real) gap. To bridge this gap, some approaches utilize scene reconstruction techniques to create photorealistic environments as a simulator. While this improves realistic sensor simulation, these methods are inherently constrained by the distribution of the training data, making it difficult to render high-quality sensor data for novel trajectories or corner case scenarios. Therefore, we propose ReconDreamer-RL , a framework designed to integrate video diffusion priors into scene reconstruction to aid reinforcement learning, thereby enhancing end-to-end autonomous driving training. Specifically, in ReconDreamer-RL , we introduce ReconSimulator, which combines the video diffusion prior for appearance modeling and incorporates a kinematic model for physical modeling, thereby reconstructing driving scenarios from real-world data. This narrows the sim2real gap for closed-loop evaluation and reinforcement learning. To cover more corner-case scenarios, we introduce the Dynamic Adversary Agent (DAA), which adjusts the trajectories of surrounding vehicles relative to the ego vehicle, autonomously generating corner-case traffic scenarios (e.g., cut-in). Finally, the Cousin Trajectory Generator (CTG) is proposed to address the issue of training data distribution, which is often biased toward simple straight-line movements. Experiments show that ReconDreamer-RL improves end-to-end autonomous driving training, outperforming imitation learning methods with a 5 × \times reduction in the Collision Ratio.

## Introduction

End-to-end training of autonomous driving models (Zhang et al. 2023; Liao et al. 2024; Li et al. 2024b; Sun et al. 2024; Weng et al. 2024; Huang et al. 2024b; Jia et al. 2024) through reinforcement learning in closed-loop simulation environments attracts growing interest. Compared to imitation learning, which relies solely on expert-collected demonstrations, closed-loop reinforcement learning enables the model to interact with the environment, enhancing its robustness and adaptability across diverse scenarios.

Despite these advantages, existing methods still face significant challenges. One key issue is creating realistic driving environments capable of effective interactions with autonomous driving policies. Game-engine-based simulators (Zhou et al. 2023; Automotive 2023) lack sensor-level realism, whereas real-world closed-loop training is costly and risky. To overcome these limitations, recent approaches (Gao et al. 2025; Wu et al. 2023; Yang et al. 2023c) employ scene reconstruction methods to build photorealistic digital twins of real-world scenarios. However, these reconstruction-based methods are constrained by their training data, typically generating high-quality sensor outputs only within regions covered by recorded camera trajectories. Additionally, these methods fail to adequately account for corner cases such as sudden braking, as such rare behaviors are typically absent from the reconstruction data.

In this paper, we introduce ReconDreamer-RL, a framework that integrates the video diffusion priors (Wang et al. 2023; Zhao et al. 2024b) into scene reconstruction to enhance end-to-end autonomous driving training by reinforcement learning. The framework consists of three components: the ReconSimulator, the Dynamic Adversary Agent (DAA), and the Cousin Trajectory Generator (CTG). ReconDreamer-RL enhances the training process in two key stages: (1) The imitation learning stage uses behavior cloning for plan initialization. (2) The reinforcement learning stage optimizes the policy through closed-loop trial-and-error interactions with the environment. Specifically, ReconSimulator employs 3D Gaussian Splatting (3DGS) to reconstruct the driving scene and incorporates the video diffusion prior for appearance modeling. Meanwhile, a kinematic model is used to ensure the validity of vehicles’ trajectories during ego-vehicle movement and scene editing. Then, in the first stage, we propose that DAA enrich corner case scenarios by controlling surrounding vehicles to generate challenging situations such as sudden lane changes. Meanwhile, the CTG enhances the diversity of sensor-collected data by synthesizing new trajectories from expert trajectories, addressing the data bias toward straight-line driving. In the second stage, the autonomous driving policy is trained within ReconSimulator via reinforcement learning, while the DAA continues to dynamically alter surrounding vehicle trajectories to create corner cases.

Through evaluation in the closed-loop 3DGS environment, we demonstrate that the end-to-end driving policy trained with the ReconDreamer-RL framework performs better, especially in corner cases, with a 5$\times$ reduction in Collision Ratio compared to imitation learning methods.

The main contributions of this paper are summarized as follows:

- •
We introduce ReconDreamer-RL, the framework that enhances end-to-end autonomous driving training by scene reconstruction with diffusion prior. It uses ReconSimulator to create realistic and freely explorable environments for reinforcement learning by enhancing appearance and physical modeling, reducing the sim2real gap.
- •
We propose DAA to auto-generate corner case instances to improve scenario coverage. Additionally, the CTG is introduced to address data distribution challenges by enriching sensor-collected expert trajectories, ensuring more balanced and comprehensive training data.
- •
We validate the effectiveness of ReconDreamer-RL in a closed-loop 3DGS environment. ReconDreamer-RL performs stronger in challenging closed-loop evaluations, achieving a 5$\times$ reduction in collision rate.

## Related Work

### Driving Scene Reconstruction with Diffusion Prior

3DGS has become a prominent technique in 3D scene reconstruction. Various works (Yan et al. 2024; Chen et al. 2023; Huang et al. 2024a; Chen et al. 2024b) have applied 3DGS to autonomous driving scenarios. However, these methods exhibit notable performance degradation when rendering trajectories that deviate from the training distribution. To address this issue, some studies leverage video diffusion models (Rombach et al. 2022; Blattmann et al. 2023) to enhance the representation of driving scenes. DriveDreamer4D (Zhao et al. 2024a) is the first to leverage the video diffusion prior to generate novel trajectory videos for training reconstruction models. ReconDreamer (Ni et al. 2024) further improves reconstruction by integrating the video diffusion model with online artifact correction, enabling better rendering of large maneuvers. ReconDreamer++ (Zhao et al. 2025) enhances rendering quality by reducing domain gaps and refining the ground surface. However, these works focus solely on improving scene representation. In contrast, we leverage the video diffusion prior to improve the simulator’s appearance modeling and introduce physical modeling, enhancing end-to-end autonomous driving training by reinforcement learning.

### End-to-End Autonomous Driving

Recent advancements in end-to-end autonomous driving algorithms showcase the immense potential of learning-based planning, where raw sensor inputs are directly mapped to control outputs. UniAD (Hu et al. 2023) integrates various perception tasks (Delavari, Khalil, and Kwon 2025; Zare et al. 2024; Mahmoudi et al. 2024; Di Palo and Johns 2024; Cheng, Chen, and Chen 2024) to enhance planning performance. VAD (Jiang et al. 2023) focuses on improving trajectory planning by utilizing compact vectorized scene representations. Furthermore, VADv2 (Chen et al. 2024a) extends this by introducing a framework that models probability distributions over planning vocabularies. However, methods based on imitation learning rely heavily on expert demonstrations and suffer from poor generalization. To address this issue, some approaches (Li et al. 2024a; Zhang et al. 2025; Caesar et al. 2021; Shalev-Shwartz, Shammah, and Shashua 2016; Gao et al. 2024; Jiang et al. 2025) utilize deep reinforcement learning (Schrittwieser et al. 2020; Silver et al. 2016; Jumper et al. 2021; Mnih et al. 2013; Schulman et al. 2017; Shao et al. 2024; Wang et al. 2025) for training end-to-end autonomous driving models. Nevertheless, these methods often encounter limitations, either due to simulation environments lacking photorealistic fidelity (Dosovitskiy et al. 2017) or reliance on open-loop data. Recent work, RAD (Gao et al. 2025), introduces a reinforcement learning framework specifically designed for training autonomous driving agents in photorealistic 3DGS environments. However, RAD (Gao et al. 2025) still suffers from inadequate supervision for policy learning due to limitations in rendering novel views and a lack of corner-case scenarios in reconstructed data, leading to limited generalization and a persistent sim-to-real gap.

Figure: Figure 1: In ReconDreamer-RL, ReconSimulator improves appearance modeling by ReconDreamer and incorporates physical modeling to reconstruct driving scenes. In the imitation learning stage, DAA generates corner-case scenario trajectories, while CTG diversifies the ego vehicle’s actions and uses ReconSimulator to render sensor data for training the policy. In the reinforcement learning stage, the policy is trained in a closed-loop environment, interacting with DAA-controlled surrounding vehicles.
Refer to caption: x1.png

Figure: Figure 2: The process of integrating the diffusion prior for appearance modeling. During the reconstruction of driving scenes, we first render novel trajectory view videos. These rendered videos are then processed by the DriveRestorer to enhance their visual quality, and the restored results are used to further optimize the reconstruction model. This iterative process continues until the reconstruction model converges.
Refer to caption: x2.png

## Method

### Preliminary

ReconDreamer (Ni et al. 2024) integrates the video diffusion prior to improve the performance of reconstruction methods in handling large maneuvers. The core of ReconDreamer is DriveRestorer, designed to restore artifacts in rendered videos that arise in novel trajectory views. It is fine-tuned based on the video diffusion models (Wang et al. 2023; Zhao et al. 2024b) and uses a diffusion loss as follows:

$$ $\mathcal{L}_{\mathcal{R}}=\mathbb{E}_{\boldsymbol{z},\epsilon\sim\mathcal{N}(0,1),t}\left[\left\|\epsilon_{t}-\epsilon_{\theta}\left(\boldsymbol{z}_{t},t,\boldsymbol{c}\right)\right\|_{2}^{2}\right],$ (1) $$

where $\epsilon_{t}$ is the random noise at time step $t$, $\epsilon_{\theta}$ is the denoising network, $\boldsymbol{z}_{t}$ is the noisy latent variable, and $\boldsymbol{c}$ represents the control conditions, including the degraded video $\hat{V}_{\text{novel}}$, 3D bounding boxes, and HDMaps.

During the inference stage, DriveRestorer freezes its network parameters and restores novel trajectory renderings. The inference process is expressed as:

$$ $V_{\text{novel}}=\mathcal{R}(\hat{V}_{\text{novel}},s),$ (2) $$

where $\mathcal{R}$ represents the DriveRestorer model, and $s$ denotes the structural conditions corresponding to the degraded video $\hat{V}_{\text{novel}}$ (such as 3D bounding boxes and HDMaps). In ReconDreamer-RL, we leverage DriveRestorer to enhance scene rendering quality, enabling free ego car navigation and scene editing, as detailed in the ReconSimulator Section.

Figure: Figure 3: Examples of Dynamic Adversary Agent (DAA) controlling surrounding vehicles to simulate cut-in scenarios.
Refer to caption: x3.png

Figure: Figure 4: The pipeline of the DAA. DAA identifies the target vehicles based on their distances to the ego car from the BEV view, where the blue line represents the ego car’s trajectory and the red line represents the target vehicle. Then, DAA generates novel trajectories based on the specified interactive behavior. The generated trajectories are checked, and feasible ones are rendered using ReconSimulator.
Refer to caption: x4.png

### Overview of the ReconDreamer-RL framework

Existing end-to-end autonomous driving algorithms are typically trained based on imitation learning, leading to poor generalization. To address this, RAD (Gao et al. 2025) trains the policy in 3DGS-based simulations, which provide photorealistic sensor data. However, it still faces challenges in rendering novel views and covering diverse corner cases.

We introduce the ReconDreamer-RL pipeline in Fig. [1](https://arxiv.org/html/2508.08170v2#Sx2.F1). ReconSimulator first reconstructs the driving scene, providing high-fidelity sensor data for ego vehicle navigation by integrating video diffusion priors and physical modeling. In the imitation learning stage, DAA generates corner cases for imitation learning, while CTG improves driving behavior through diverse actions. In the reinforcement learning stage, the policy is trained in closed-loop environments, with DAA generating new corner case trajectories to increase difficulty.

### ReconSimulator

The ideal reinforcement learning environment for autonomous driving should minimize the sim2real gap. However, existing 3DGS-based methods merely reconstruct data, failing to ensure realistic appearances along novel trajectories. Meanwhile, it is essential to ensure that the trajectories of all vehicles adhere to physical constraints. Therefore, we propose ReconSimulator to address appearance and physical modeling challenges. In terms of appearance, we integrate the diffusion prior to enhance scene representation for high-quality sensor data rendering on novel trajectories. For physical modeling, kinematic modeling is used to ensure the physical feasibility of vehicle trajectories. Next, we delve into the details of the appearance and physical modeling.

Appearance Modeling. To ensure high-quality rendering and scene editing, we first use 3DGS to reconstruct the scene and render novel trajectories. Then, DriveRestorer (Ni et al. 2024) mitigates artifacts in the rendered videos, and the results are used to fine-tune the reconstruction model. This enables the final model to produce high-quality rendering from diverse viewpoints. The overall process is illustrated in Fig. [2](https://arxiv.org/html/2508.08170v2#Sx2.F2). Meanwhile, we model the static background and moving vehicles separately, which enables the modification of trajectories and appearances for all vehicles. The background model is represented by $G_{\text{Background},w}$ in the world coordinate system. For each moving object $v$, represented by Gaussians $G_{\text{Rigid},l}^{v}$ in the local coordinate system $l$, these Gaussians must be transformed into the background coordinate system during the rendering process:

$$ $G_{\text{Rigid},w}^{v}(t)=M_{t}^{v}\cdot G_{\text{Rigid},l}^{v}+S_{t}^{v},$ (3) $$

where $M_{t}^{v}$ and $S_{t}^{v}$ represent the rotation matrix and translation vector that describe the pose of object $v$ at time $t$.
Physical Modeling. To maintain the authenticity of vehicle trajectories, a kinematic model is used to govern the vehicle’s motion. Specifically, the vehicle’s pose at time $t$ in the world coordinate system is $W_{t}=[R_{t}\mid P_{t}]\in SE(3)$, where $R_{t}$ is the rotation matrix and $P_{t}$ is the position. At each time step, the pose is updated using linear velocity $v_{t}$ and steering angle $\delta_{t}$ in a kinematic bicycle model:

$$ $P_{t+1}=P_{t}+v_{t}\cdot\Delta t\cdot\hat{d}_{t},$ (4) $$

where $\hat{d}_{t}$ is the forward direction vector derived from the rotation matrix $R_{t}$. The vehicle orientation is updated by applying a rotation about the vertical axis:

$$ $R_{t+1}=\text{Rot}_{z}(\Delta\theta_{t})\cdot R_{t},$ (5) $$

where the incremental rotation angle $\Delta\theta_{t}$ is calculated based on the bicycle model as:

$$ $\Delta\theta_{t}=\frac{v_{t}}{L}\tan(\delta_{t})\Delta t.$ (6) $$

The $L$ denotes the wheelbase length of the vehicle. The rotation matrix $\text{Rot}_{z}(\Delta\theta_{t})$ is explicitly expressed as:

$$ $\text{Rot}_{z}(\Delta\theta_{t})=\begin{bmatrix}\cos(\Delta\theta_{t})&\sin(\Delta\theta_{t})&0\\ -\sin(\Delta\theta_{t})&\cos(\Delta\theta_{t})&0\\ 0&0&1\end{bmatrix}.$ (7) $$

We define different kinematic parameters for each vehicle category and perform checks when updating the trajectories to ensure the trajectory updates remain physically plausible and within the vehicle’s operational constraints, including the maximum steering angle and velocity.

Figure: Figure 5: Cousin Trajectory Generator (CTG) generates cousin trajectories and performs trajectory checks to eliminate unreasonable trajectories (e.g., the pink cross marks), and finally renders the corresponding sensor data in the ReconSimulator.
Refer to caption: x5.png

Figure: Figure 6: The Cousin-nuScenes dataset created by CTG has an increased variety of actions compared to the nuScenes.
Refer to caption: x6.png

### Dynamic Adversary Agent

Existing end-to-end autonomous driving models are trained with limited exposure to corner cases. To tackle this, we propose the Dynamic Adversary Agent (DAA), which generates realistic and diverse corner case interactions. The overview of DAA is provided in Fig. [4](https://arxiv.org/html/2508.08170v2#Sx3.F4), and as shown in Fig. [3](https://arxiv.org/html/2508.08170v2#Sx3.F3), it can generate complex cut-in scenarios.

DAA first identifies suitable interactive target vehicles from the Bird’s-Eye View (BEV) perspective, based on the distances to the ego vehicle and the specified interactive behavior $\mathcal{B}$. Different interactive behaviors correspond to different distance settings, which refer to traditional traffic flow methods (Kesting and Treiber 2013; Treiber and Kesting 2013). Then, a new trajectory is generated for the selected target vehicle by modifying its original trajectory $T_{\text{target}}$ based on both the ego vehicle’s trajectory $T_{\text{ego}}$ and $\mathcal{B}$. The new trajectory $T^{{}^{\prime}}_{\text{target}}$ can be represented as:

$$ $T^{{}^{\prime}}_{\text{target}}=f(T_{\text{ego}},T_{\text{target}},\mathcal{B}),$ (8) $$

where $f(\cdot)$ represents the trajectory generation function, which can be implemented using methods such as text-to-trajectory (Zhao et al. 2024b) and customized based on specific requirements. Then, the generated trajectory is checked for feasibility. First, the vehicle trajectory $T^{\prime}_{\text{target}}$ should remain within the drivable region, and avoid collisions with other vehicles (collisions with the ego-vehicle are allowed):

$$ $\displaystyle\|T^{\prime}_{\text{target}}-o_{j}\|\geq d_{\text{min}},\quad\forall j\in\{1,\dots,M\},$ (9) $$

where $d_{\text{min}}$ is the minimum distance between different agents $\{o_{j}\}_{j=1}^{M}$. Then, the trajectory is further checked to meet the constraints of the kinematic model and converted into the BEV perspective to verify whether it satisfies the interactive behavior $\mathcal{B}$.

DAA can be used in both imitation learning and reinforcement learning stages. In the first stage, DAA generates corner case scenarios by controlling surrounding vehicles and generating the ego vehicle’s trajectory to avoid collisions, using a similar method mentioned above. The ReconSimulator then renders offline autonomous driving data based on the trajectories for imitation learning. In the second stage, the reinforcement learning policy controls the actions of the ego vehicle, while DAA controls the trajectory of the target vehicle in ReconSimulator to create corner cases. To enhance robustness, DAA has a certain probability of fine-tuning the trajectories (e.g., adjusting the target vehicle’s speed) instead of directly reusing them from the first stage.

Figure: Figure 7: Comparison of different methods in challenging corner cases, with collisions highlighted by orange boxes.
Refer to caption: x7.png

### Cousin Trajectory Generator

To address cold-start issues in reinforcement learning (Hua et al. 2021; Rashidinejad et al. 2021), behavior cloning for pretraining is effective but requires large-scale real-world data, which is often biased toward simple scenarios (Codevilla et al. 2019; Zheng et al. 2024b). We propose the Cousin Trajectory Generator (CTG) to enhance action diversity via trajectory extension and interpolation. It is used to create the Cousin-nuScenes dataset with more diverse actions.

For trajectory extension, we generate new trajectories for the ego vehicle, such as lane changes and sharp turns, following a process similar to that described in Equation. [8](https://arxiv.org/html/2508.08170v2#Sx3.E8). Subsequently, a trajectory check is conducted to evaluate the validity of the generated trajectories, ensuring they remain physically plausible and adhere to the vehicle’s operational limitations, including steering angle and velocity, and verifying collision avoidance from the BEV perspective. Meanwhile, to better leverage expert trajectories from rare scenarios like U-turns, we interpolate the expert data for more detailed driving information. Given expert trajectories $\mathbf{X}_{\text{ego}}=\{X_{\text{ego}}^{t_{1}},X_{\text{ego}}^{t_{2}},\dots,X_{\text{ego}}^{t_{n}}\}$, where each $X_{\text{ego}}^{t_{i}}$ is the ego vehicle’s position at time $t_{i}$, we perform linear interpolation between consecutive time steps $t_{i}$ and $t_{i+1}$. For an interpolated point $X_{\text{ego}}^{t}$ at time $t$, where $t_{i}\leq t\leq t_{i+1}$, the formula is:

$$ $X_{\text{ego}}^{t}=X_{\text{ego}}^{t_{i}}+\frac{t-t_{i}}{t_{i+1}-t_{i}}\left(X_{\text{ego}}^{t_{i+1}}-X_{\text{ego}}^{t_{i}}\right).$ (10) $$

The time $t$ is expressed as:

$$ $t=t_{i}+k\cdot\Delta t,\quad k\in\{1,2,\dots,m\},$ (11) $$

where $\Delta t=\frac{t_{i+1}-t_{i}}{m+1}$ and $m$ is the number of interpolation points between two time steps. For each interpolated trajectory point $X_{\text{ego}}^{t}$, we accordingly adjust the positions of surrounding vehicles to ensure the interpolated trajectories maintain a realistic spatial relationship and interactions.

After generating the interpolation and extension trajectories, we use the ReconSimulator to obtain the corresponding sensor data, which is then used for imitation training. We provide examples of trajectory extension in Fig. [5](https://arxiv.org/html/2508.08170v2#Sx3.F5), illustrating two cases of ego-vehicle lane-changing maneuvers. Moreover, as shown in Fig. [6](https://arxiv.org/html/2508.08170v2#Sx3.F6), Cousin-nuScenes created by CTG contains 4$\times$ more non-straight-line driving maneuvers than the nuScenes (Caesar et al. 2020) dataset.

## Experiments

**Table 1: Comparison of different methods on various metrics.**
| Method | CR $\downarrow$ | DCR$\downarrow$ | SCR$\downarrow$ | DR$\downarrow$ | PDR$\downarrow$ | HDR$\downarrow$ |
| --- | --- | --- | --- | --- | --- | --- |
| VAD (Jiang et al. 2023) | 0.386 | 0.234 | 0.152 | 0.163 | 0.103 | 0.060 |
| GenAD (Zheng et al. 2024a) | 0.333 | 0.190 | 0.143 | 0.146 | 0.093 | 0.053 |
| VADv2 (Chen et al. 2024a) | 0.290 | 0.162 | 0.128 | 0.154 | 0.107 | 0.047 |
| RAD (Gao et al. 2025) | 0.238 | 0.143 | 0.095 | 0.084 | 0.057 | 0.027 |
| ReconDreamer-RL | 0.077 | 0.048 | 0.029 | 0.040 | 0.027 | 0.013 |

**Table 2: Ablation study of ReconDreamer-RL, evaluating the effectiveness of each module on various metrics.**
| ReconSimulator | DAA | CTG | CR$\downarrow$ | DCR$\downarrow$ | SCR$\downarrow$ | DR$\downarrow$ | PDR$\downarrow$ | HDR$\downarrow$ |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  |  |  | 0.238 | 0.143 | 0.095 | 0.084 | 0.057 | 0.027 |
| $\checkmark$ |  |  | 0.172 | 0.103 | 0.069 | 0.073 | 0.053 | 0.020 |
| $\checkmark$ | $\checkmark$ |  | 0.117 | 0.069 | 0.048 | 0.067 | 0.050 | 0.017 |
| $\checkmark$ |  | $\checkmark$ | 0.143 | 0.086 | 0.057 | 0.053 | 0.040 | 0.013 |
|  | $\checkmark$ | $\checkmark$ | 0.122 | 0.082 | 0.040 | 0.063 | 0.043 | 0.020 |
| $\checkmark$ | $\checkmark$ | $\checkmark$ | 0.077 | 0.048 | 0.029 | 0.040 | 0.027 | 0.013 |

**Table 3: Impact of diffusion prior within ReconSimulator on novel view rendering quality across lane shift and lane change.**
| Method | Lane Shift @ 3m | Lane Shift @ 6m | Lane Change |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| NTA-IoU $\uparrow$ | NTL-IoU $\uparrow$ | FID $\downarrow$ | NTA-IoU $\uparrow$ | NTL-IoU $\uparrow$ | FID $\downarrow$ | NTA-IoU $\uparrow$ | NTL-IoU $\uparrow$ | FID $\downarrow$ |  |
| w/o Video Diffusion Prior | 0.224 | 49.67 | 160.23 | 0.148 | 44.89 | 256.42 | 0.215 | 46.23 | 178.34 |
| w/ Video Diffusion Prior | 0.338 | 50.14 | 121.56 | 0.302 | 48.94 | 140.24 | 0.326 | 50.08 | 124.31 |

**Table 4: Comparison of collision metrics in cut-in scenarios**
| Method | CR$\downarrow$ | DCR$\downarrow$ | SCR$\downarrow$ |
| --- | --- | --- | --- |
| VAD (Jiang et al. 2023) | 0.449 | 0.293 | 0.156 |
| GenAD (Zheng et al. 2024a) | 0.379 | 0.234 | 0.145 |
| VADv2 (Chen et al. 2024a) | 0.436 | 0.276 | 0.160 |
| RAD (Gao et al. 2025) | 0.317 | 0.210 | 0.107 |
| ReconDreamer-RL | 0.089 | 0.053 | 0.036 |

### Experimental Setup

#### Implementation Details.

To demonstrate the capability of the ReconDreamer-RL framework in enhancing training, we use it to train the end-to-end autonomous driving policy in RAD (Gao et al. 2025). The training process has two stages: the imitation learning stage and the reinforcement learning stage. In the first stage, the perception capabilities are trained, followed by planning pre-training through behavior cloning. In the second stage, reinforcement learning is employed within closed-loop 3DGS environments to optimize planning further. Finally, we build an evaluation benchmark including edited 3DGS environments and original 3DGS environments to test the performance of the end-to-end autonomous driving method. All 3DGS environments are reconstructed from nuScenes (Caesar et al. 2020), and additional details on the edited scenes are in the supplement.

#### Baseline.

In comparative analysis, we select representative end-to-end autonomous driving methods. For imitation learning, we choose VAD (Jiang et al. 2023), GenAD (Zheng et al. 2024a), and VADv2 (Chen et al. 2024a). For reinforcement learning, we use RAD (Gao et al. 2025), which is trained in closed-loop 3DGS environments.

#### Evaluation Metrics.

Following RAD (Gao et al. 2025), we evaluate policy performance using six metrics. The Dynamic Collision Ratio (DCR) and Static Collision Ratio (SCR) quantify collisions with dynamic and static obstacles, and their sum is referred to as the Collision Ratio (CR). The Positional Deviation Ratio (PDR) and Heading Deviation Ratio (HDR) measure the ego vehicle’s deviation from the expert trajectory for position and heading, summed up and referred to as the Deviation Ratio (DR). To assess the realism of the reinforcement learning environment, we use three metrics from DriveDreamer4D (Zhao et al. 2024a): Novel Trajectory Agent IoU (NTA-IoU) for foreground quality, Novel Trajectory Lane IoU (NTL-IoU) for evaluating lane markings, and Fréchet Inception Distance (FID) (Heusel et al. 2017) to quantify overall rendering fidelity.

**Table 5: Comparison of rendering speeds among reinforcement learning simulators, including EmerNeRF (Yang et al. 2023b), RAD (Gao et al. 2025), and ReconSimulator.**
| Simulator | EmerNeRF | RAD-3DGS | ReconSimulator |
| --- | --- | --- | --- |
| Speed (FPS) | 0.21 | 135 | 125 |

### Main Results

#### Quantitative Results.

ReconDreamer-RL enhances the performance of RAD (Gao et al. 2025) across all evaluation metrics, as shown in Tab. [1](https://arxiv.org/html/2508.08170v2#Sx4.T1). Traditional imitation learning-based methods, such as VAD (Jiang et al. 2023) and VADv2 (Chen et al. 2024a), struggle with closed-loop evaluation tasks due to the discrepancy between training and closed-loop inference. RAD (Gao et al. 2025) enhances performance by incorporating reinforcement learning and training within a 3DGS-based simulator. However, due to limited corner case coverage and poor novel view rendering ability of 3DGS, RAD exhibits a higher collision rate in our challenging evaluation benchmark. However, ReconDreamer-RL leverages diffusion prior and data augmentation to enhance end-to-end autonomous driving, achieving a 3$\times$ reduction in Collision Ratio compared to RAD (Gao et al. 2025).

#### Corner Case Results.

As shown in Tab. [4](https://arxiv.org/html/2508.08170v2#Sx4.T4), we compare the collision metrics of different methods in corner cases, focusing on cut-in situations. Imitation learning-based methods exhibit a high Dynamic Collision Ratio in cut-in corner cases, as such scenarios are completely absent from the training data. RAD (Gao et al. 2025) improves the model’s ability to handle corner cases through reinforcement learning, but still lacks the necessary data to train on such situations. In contrast, ReconDreamer-RL improves the policy’s performance through a two-stage process, achieving 404.5% improvement in Collision Ratio over imitation methods.

#### Rendering Speed.

Rendering speed is crucial for reinforcement learning environments. As shown in Tab. [5](https://arxiv.org/html/2508.08170v2#Sx4.T5), ReconSimulator achieves efficient rendering speed, thus satisfying the requirements of reinforcement learning tasks.

#### Qualitative Results.

As shown in Fig. [7](https://arxiv.org/html/2508.08170v2#Sx3.F7), we compare ReconDreamer-RL and RAD (Gao et al. 2025) under two corner cases. The first scenario involves a vehicle on the right of the ego car quickly cutting in. The second scenario features a vehicle on the right performing a cut-in, followed by a sudden brake. In the first case, ReconDreamer-RL successfully avoids the collision and maintains a safe distance, whereas RAD (Gao et al. 2025) fails to react. In the second case, although RAD (Gao et al. 2025) attempts to avoid the collision by shifting lanes, it fails to control its speed and steering angle properly. In contrast, ReconDreamer-RL safely changes lanes and decelerates to avoid the collision.

### Ablation Studies

#### ReconSimulator.

As shown in Tab. [3](https://arxiv.org/html/2508.08170v2#Sx4.T3), we conduct ablation experiments on ReconSimulator to demonstrate the impact of the diffusion prior on novel view rendering quality. Specifically, without ReconSimulator, the NTA-IoU and NTL-IoU drop by 54.46% and 9.71%, and the FID metrics increase by 104.43% for the lane shift of 6m, indicating that the appearance of surrounding vehicles and lane markings has degraded substantially. Meanwhile, Tab. [2](https://arxiv.org/html/2508.08170v2#Sx4.T2) shows that removing ReconSimulator leads to worse end-to-end autonomous driving training performance.

#### Impact of the DAA.

The DAA contributes to reducing collision occurrences by generating corner case scenarios to aid training. As demonstrated in Tab. [2](https://arxiv.org/html/2508.08170v2#Sx4.T2), introducing DAA decreases the Collision Ratio from 0.172 to 0.117.

#### Impact of the CTG.

In Tab. [2](https://arxiv.org/html/2508.08170v2#Sx4.T2), the benefit of CTG is validated by reductions in CR (from 0.172 to 0.143) and DR (from 0.073 to 0.052), indicating improved trajectory adherence. By addressing the data bias, CTG enhances the action distribution learned during the imitation learning phase.

## Conclusion

This paper introduces ReconDreamer-RL, a framework designed to enhance reinforcement learning for end-to-end autonomous driving through the integration of video diffusion priors. It focuses on constructing a more realistic simulator, enriching corner cases, and addressing the issue of training data distribution. Specifically, we introduce ReconSimulator, which combines the video diffusion prior for appearance modeling and a kinematic model for physical dynamics, providing realistic environments for reinforcement learning. Additionally, we develop the DAA, which generates complex traffic scenarios, such as sudden lane changes. To address the issue of training data distribution being mostly on simple straight-line movements, CTG is proposed to enhance trajectory diversity through extension and interpolation. Experimental evaluations indicate that ReconDreamer-RL significantly improves the performance of end-to-end autonomous driving models, reducing the Collision Ratio by 5$\times$ compared to imitation-based learning methods.

## References

- Automotive (2023)
Automotive, I. 2023.
CarMaker.
[https://www.ipg-automotive.com](https://www.ipg-automotive.com).
Accessed: 2025-06-22.
- Blattmann et al. (2023)
Blattmann, A.; Dockhorn, T.; Kulal, S.; Mendelevitch, D.; Kilian, M.; Lorenz, D.; Levi, Y.; English, Z.; Voleti, V.; Letts, A.; et al. 2023.
Stable video diffusion: Scaling latent video diffusion models to large datasets.
*arXiv preprint arXiv:2311.15127*.
- Caesar et al. (2020)
Caesar, H.; Bankiti, V.; Lang, A. H.; Vora, S.; Liong, V. E.; Xu, Q.; Krishnan, A.; Pan, Y.; Baldan, G.; and Beijbom, O. 2020.
nuscenes: A multimodal dataset for autonomous driving.
In *CVPR*.
- Caesar et al. (2021)
Caesar, H.; Kabzan, J.; Tan, K. S.; Fong, W. K.; Wolff, E.; Lang, A.; Fletcher, L.; Beijbom, O.; and Omari, S. 2021.
nuplan: A closed-loop ml-based planning benchmark for autonomous vehicles.
*arXiv preprint arXiv:2106.11810*.
- Che et al. (2023)
Che, Q.-H.; Nguyen, D.-P.; Pham, M.-Q.; and Lam, D.-K. 2023.
TwinLiteNet: An efficient and lightweight model for driveable area and lane segmentation in self-driving cars.
In *MAPR*.
- Chen et al. (2024a)
Chen, S.; Jiang, B.; Gao, H.; Liao, B.; Xu, Q.; Zhang, Q.; Huang, C.; Liu, W.; and Wang, X. 2024a.
Vadv2: End-to-end vectorized autonomous driving via probabilistic planning.
*arXiv preprint arXiv:2402.13243*.
- Chen et al. (2023)
Chen, Y.; Gu, C.; Jiang, J.; Zhu, X.; and Zhang, L. 2023.
Periodic vibration gaussian: Dynamic urban scene reconstruction and real-time rendering.
*arXiv preprint arXiv:2311.18561*.
- Chen et al. (2024b)
Chen, Z.; Yang, J.; Huang, J.; Lutio, R. d.; Esturo, J. M.; Ivanovic, B.; Litany, O.; Gojcic, Z.; Fidler, S.; Pavone, M.; Song, L.; and Wang, Y. 2024b.
OmniRe: Omni Urban Scene Reconstruction.
*arXiv preprint arXiv:2408.16760*.
- Cheng, Chen, and Chen (2024)
Cheng, J.; Chen, Y.; and Chen, Q. 2024.
Pluto: Pushing the limit of imitation learning-based planning for autonomous driving.
*arXiv preprint arXiv:2404.14327*.
- Codevilla et al. (2019)
Codevilla, F.; Santana, E.; López, A. M.; and Gaidon, A. 2019.
Exploring the limitations of behavior cloning for autonomous driving.
In *Proceedings of the IEEE/CVF international conference on computer vision*, 9329–9338.
- Delavari, Khalil, and Kwon (2025)
Delavari, E.; Khalil, A.; and Kwon, J. 2025.
CARIL: Confidence-Aware Regression in Imitation Learning for Autonomous Driving.
*arXiv preprint arXiv:2503.00783*.
- Di Palo and Johns (2024)
Di Palo, N.; and Johns, E. 2024.
Keypoint action tokens enable in-context imitation learning in robotics.
*arXiv preprint arXiv:2403.19578*.
- Dosovitskiy et al. (2017)
Dosovitskiy, A.; Ros, G.; Codevilla, F.; Lopez, A.; and Koltun, V. 2017.
CARLA: An open urban driving simulator.
In *Conference on robot learning*, 1–16. PMLR.
- Gao et al. (2025)
Gao, H.; Chen, S.; Jiang, B.; Liao, B.; Shi, Y.; Guo, X.; Pu, Y.; Yin, H.; Li, X.; Zhang, X.; et al. 2025.
Rad: Training an end-to-end driving policy via large-scale 3dgs-based reinforcement learning.
*arXiv preprint arXiv:2502.13144*.
- Gao et al. (2024)
Gao, H.; Zhao, M.; Zheng, X.; Wang, C.; Zhou, L.; Wang, Y.; Ma, L.; Cheng, B.; Wu, Z.; and Li, Y. 2024.
An improved hierarchical deep reinforcement learning algorithm for multi-intelligent vehicle lane change.
*Neurocomputing*, 609: 128482.
- Heusel et al. (2017)
Heusel, M.; Ramsauer, H.; Unterthiner, T.; Nessler, B.; and Hochreiter, S. 2017.
Gans trained by a two time-scale update rule converge to a local nash equilibrium.
*NeurIPS*.
- Hu et al. (2023)
Hu, Y.; Yang, J.; Chen, L.; Li, K.; Sima, C.; Zhu, X.; Chai, S.; Du, S.; Lin, T.; Wang, W.; et al. 2023.
Planning-oriented autonomous driving.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, 17853–17862.
- Hua et al. (2021)
Hua, J.; Zeng, L.; Li, G.; and Ju, Z. 2021.
Learning for a robot: Deep reinforcement learning, imitation learning, transfer learning.
*Sensors*, 21(4): 1278.
- Huang et al. (2024a)
Huang, N.; Wei, X.; Zheng, W.; An, P.; Lu, M.; Zhan, W.; Tomizuka, M.; Keutzer, K.; and Zhang, S. 2024a.
$S^{3}$Gaussian: Self-Supervised Street Gaussians for Autonomous Driving.
*arXiv preprint arXiv:2405.20323*.
- Huang et al. (2024b)
Huang, Z.; Weng, X.; Igl, M.; Chen, Y.; Cao, Y.; Ivanovic, B.; Pavone, M.; and Lv, C. 2024b.
Gen-Drive: Enhancing Diffusion Generative Driving Policies with Reward Modeling and Reinforcement Learning Fine-tuning.
*arXiv preprint arXiv:2410.05582*.
- Jia et al. (2024)
Jia, X.; Yang, Z.; Li, Q.; Zhang, Z.; and Yan, J. 2024.
Bench2drive: Towards multi-ability benchmarking of closed-loop end-to-end autonomous driving.
*Advances in Neural Information Processing Systems*, 37: 819–844.
- Jiang et al. (2023)
Jiang, B.; Chen, S.; Xu, Q.; Liao, B.; Chen, J.; Zhou, H.; Zhang, Q.; Liu, W.; Huang, C.; and Wang, X. 2023.
Vad: Vectorized scene representation for efficient autonomous driving.
In *Proceedings of the IEEE/CVF International Conference on Computer Vision*, 8340–8350.
- Jiang et al. (2025)
Jiang, B.; Chen, S.; Zhang, Q.; Liu, W.; and Wang, X. 2025.
Alphadrive: Unleashing the power of vlms in autonomous driving via reinforcement learning and reasoning.
*arXiv preprint arXiv:2503.07608*.
- Jumper et al. (2021)
Jumper, J.; Evans, R.; Pritzel, A.; Green, T.; Figurnov, M.; Ronneberger, O.; Tunyasuvunakool, K.; Bates, R.; Žídek, A.; Potapenko, A.; et al. 2021.
Highly accurate protein structure prediction with AlphaFold.
*nature*, 596(7873): 583–589.
- Kerbl et al. (2023)
Kerbl, B.; Kopanas, G.; Leimkühler, T.; and Drettakis, G. 2023.
3D Gaussian splatting for real-time radiance field rendering.
*ACM Trans. Graph.*, 42(4): 139–1.
- Kesting and Treiber (2013)
Kesting, A.; and Treiber, M. 2013.
Traffic flow dynamics: data, models and simulation.
*No. Book, Whole)(Springer Berlin Heidelberg, Berlin, Heidelberg*.
- Khanam and Hussain (2024)
Khanam, R.; and Hussain, M. 2024.
Yolov11: An overview of the key architectural enhancements.
*arXiv preprint arXiv:2410.17725*.
- Li et al. (2024a)
Li, Q.; Jia, X.; Wang, S.; and Yan, J. 2024a.
Think2Drive: Efficient Reinforcement Learning by Thinking with Latent World Model for Autonomous Driving (in CARLA-V2).
In *European Conference on Computer Vision*, 142–158. Springer.
- Li et al. (2024b)
Li, Z.; Li, K.; Wang, S.; Lan, S.; Yu, Z.; Ji, Y.; Li, Z.; Zhu, Z.; Kautz, J.; Wu, Z.; et al. 2024b.
Hydra-mdp: End-to-end multimodal planning with multi-target hydra-distillation.
*arXiv preprint arXiv:2406.06978*.
- Li et al. (2024c)
Li, Z.; Wang, W.; Li, H.; Xie, E.; Sima, C.; Lu, T.; Yu, Q.; and Dai, J. 2024c.
Bevformer: learning bird’s-eye-view representation from lidar-camera via spatiotemporal transformers.
*IEEE Transactions on Pattern Analysis and Machine Intelligence*.
- Liao et al. (2024)
Liao, B.; Chen, S.; Yin, H.; Jiang, B.; Wang, C.; Yan, S.; Zhang, X.; Li, X.; Zhang, Y.; Zhang, Q.; et al. 2024.
DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving.
*arXiv preprint arXiv:2411.15139*.
- Loshchilov and Hutter (2017)
Loshchilov, I.; and Hutter, F. 2017.
Decoupled weight decay regularization.
*arXiv preprint arXiv:1711.05101*.
- Mahmoudi et al. (2024)
Mahmoudi, S.; Davar, A.; Sohrabipour, P.; Bist, R. B.; Tao, Y.; and Wang, D. 2024.
Leveraging imitation learning in agricultural robotics: a comprehensive survey and comparative analysis.
*Frontiers in Robotics and AI*, 11: 1441312.
- Mnih et al. (2013)
Mnih, V.; Kavukcuoglu, K.; Silver, D.; Graves, A.; Antonoglou, I.; Wierstra, D.; and Riedmiller, M. 2013.
Playing atari with deep reinforcement learning.
*arXiv preprint arXiv:1312.5602*.
- Ni et al. (2024)
Ni, C.; Zhao, G.; Wang, X.; Zhu, Z.; Qin, W.; Huang, G.; Liu, C.; Chen, Y.; Wang, Y.; Zhang, X.; et al. 2024.
ReconDreamer: Crafting World Models for Driving Scene Reconstruction via Online Restoration.
*arXiv preprint arXiv:2411.19548*.
- Rashidinejad et al. (2021)
Rashidinejad, P.; Zhu, B.; Ma, C.; Jiao, J.; and Russell, S. 2021.
Bridging offline reinforcement learning and imitation learning: A tale of pessimism.
*Advances in Neural Information Processing Systems*, 34: 11702–11716.
- Rombach et al. (2022)
Rombach, R.; Blattmann, A.; Lorenz, D.; Esser, P.; and Ommer, B. 2022.
High-resolution image synthesis with latent diffusion models.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, 10684–10695.
- Schrittwieser et al. (2020)
Schrittwieser, J.; Antonoglou, I.; Hubert, T.; Simonyan, K.; Sifre, L.; Schmitt, S.; Guez, A.; Lockhart, E.; Hassabis, D.; Graepel, T.; et al. 2020.
Mastering atari, go, chess and shogi by planning with a learned model.
*Nature*, 588(7839): 604–609.
- Schulman et al. (2017)
Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; and Klimov, O. 2017.
Proximal policy optimization algorithms.
*arXiv preprint arXiv:1707.06347*.
- Shalev-Shwartz, Shammah, and Shashua (2016)
Shalev-Shwartz, S.; Shammah, S.; and Shashua, A. 2016.
Safe, multi-agent, reinforcement learning for autonomous driving.
*arXiv preprint arXiv:1610.03295*.
- Shao et al. (2024)
Shao, Z.; Wang, P.; Zhu, Q.; Xu, R.; Song, J.; Bi, X.; Zhang, H.; Zhang, M.; Li, Y.; Wu, Y.; et al. 2024.
Deepseekmath: Pushing the limits of mathematical reasoning in open language models.
*arXiv preprint arXiv:2402.03300*.
- Silver et al. (2016)
Silver, D.; Huang, A.; Maddison, C. J.; Guez, A.; Sifre, L.; Van Den Driessche, G.; Schrittwieser, J.; Antonoglou, I.; Panneershelvam, V.; Lanctot, M.; et al. 2016.
Mastering the game of Go with deep neural networks and tree search.
*nature*, 529(7587): 484–489.
- Sun et al. (2020)
Sun, P.; Kretzschmar, H.; Dotiwalla, X.; Chouard, A.; Patnaik, V.; Tsui, P.; Guo, J.; Zhou, Y.; Chai, Y.; Caine, B.; Vasudevan, V.; Han, W.; Ngiam, J.; Zhao, H.; Timofeev, A.; Ettinger, S.; Krivokon, M.; Gao, A.; Joshi, A.; Zhang, Y.; Shlens, J.; Chen, Z.; and Anguelov, D. 2020.
Scalability in Perception for Autonomous Driving: Waymo Open Dataset.
In *CVPR*.
- Sun et al. (2024)
Sun, W.; Lin, X.; Shi, Y.; Zhang, C.; Wu, H.; and Zheng, S. 2024.
Sparsedrive: End-to-end autonomous driving via sparse scene representation.
*arXiv preprint arXiv:2405.19620*.
- Treiber and Kesting (2013)
Treiber, M.; and Kesting, A. 2013.
*Traffic flow dynamics*, volume 1.
Springer.
- Wang et al. (2025)
Wang, B.; Meng, X.; Wang, X.; Zhu, Z.; Ye, A.; Wang, Y.; Yang, Z.; Ni, C.; Huang, G.; and Wang, X. 2025.
EmbodieDreamer: Advancing Real2Sim2Real Transfer for Policy Training via Embodied World Modeling.
*arXiv preprint arXiv:2507.05198*.
- Wang et al. (2023)
Wang, X.; Zhu, Z.; Huang, G.; Chen, X.; Zhu, J.; and Lu, J. 2023.
Drivedreamer: Towards real-world-driven world models for autonomous driving.
*arXiv preprint arXiv:2309.09777*.
- Weng et al. (2024)
Weng, X.; Ivanovic, B.; Wang, Y.; Wang, Y.; and Pavone, M. 2024.
Para-drive: Parallelized architecture for real-time autonomous driving.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 15449–15458.
- Wu et al. (2023)
Wu, Z.; Liu, T.; Luo, L.; Zhong, Z.; Chen, J.; Xiao, H.; Hou, C.; Lou, H.; Chen, Y.; Yang, R.; et al. 2023.
Mars: An instance-aware, modular and realistic simulator for autonomous driving.
In *CAAI International Conference on Artificial Intelligence*, 3–15. Springer.
- Xiao et al. (2021)
Xiao, P.; Shao, Z.; Hao, S.; Zhang, Z.; Chai, X.; Jiao, J.; Li, Z.; Wu, J.; Sun, K.; Jiang, K.; et al. 2021.
Pandaset: Advanced sensor suite dataset for autonomous driving.
In *2021 IEEE international intelligent transportation systems conference (ITSC)*, 3095–3101. IEEE.
- Yan et al. (2024)
Yan, Y.; Lin, H.; Zhou, C.; Wang, W.; Sun, H.; Zhan, K.; Lang, X.; Zhou, X.; and Peng, S. 2024.
Street gaussians for modeling dynamic urban scenes.
*arXiv preprint arXiv:2401.01339*.
- Yang et al. (2023a)
Yang, C.; Chen, Y.; Tian, H.; Tao, C.; Zhu, X.; Zhang, Z.; Huang, G.; Li, H.; Qiao, Y.; Lu, L.; et al. 2023a.
Bevformer v2: Adapting modern image backbones to bird’s-eye-view recognition via perspective supervision.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, 17830–17839.
- Yang et al. (2023b)
Yang, J.; Ivanovic, B.; Litany, O.; Weng, X.; Kim, S. W.; Li, B.; Che, T.; Xu, D.; Fidler, S.; Pavone, M.; et al. 2023b.
Emernerf: Emergent spatial-temporal scene decomposition via self-supervision.
*arXiv preprint arXiv:2311.02077*.
- Yang et al. (2023c)
Yang, Z.; Chen, Y.; Wang, J.; Manivasagam, S.; Ma, W.-C.; Yang, A. J.; and Urtasun, R. 2023c.
Unisim: A neural closed-loop sensor simulator.
In *CVPR*.
- Zare et al. (2024)
Zare, M.; Kebria, P. M.; Khosravi, A.; and Nahavandi, S. 2024.
A survey of imitation learning: Algorithms, recent developments, and challenges.
*IEEE Transactions on Cybernetics*.
- Zhang et al. (2025)
Zhang, D.; Liang, J.; Guo, K.; Lu, S.; Wang, Q.; Xiong, R.; Miao, Z.; and Wang, Y. 2025.
Carplanner: Consistent auto-regressive trajectory planning for large-scale reinforcement learning in autonomous driving.
In *Proceedings of the Computer Vision and Pattern Recognition Conference*, 17239–17248.
- Zhang et al. (2023)
Zhang, H.; Wang, Z.; Wu, Z.; and Jiang, Y.-G. 2023.
DiffusionAD: Norm-guided one-step denoising diffusion for anomaly detection.
*arXiv preprint arXiv:2303.08730*.
- Zhao et al. (2024a)
Zhao, G.; Ni, C.; Wang, X.; Zhu, Z.; Huang, G.; Chen, X.; Wang, B.; Zhang, Y.; Mei, W.; and Wang, X. 2024a.
Drivedreamer4d: World models are effective data machines for 4d driving scene representation.
*arXiv preprint arXiv:2410.13571*.
- Zhao et al. (2025)
Zhao, G.; Wang, X.; Ni, C.; Zhu, Z.; Qin, W.; Huang, G.; and Wang, X. 2025.
Recondreamer++: Harmonizing generative and reconstructive models for driving scene representation.
*arXiv preprint arXiv:2503.18438*.
- Zhao et al. (2024b)
Zhao, G.; Wang, X.; Zhu, Z.; Chen, X.; Huang, G.; Bao, X.; and Wang, X. 2024b.
Drivedreamer-2: Llm-enhanced world models for diverse driving video generation.
*arXiv preprint arXiv:2403.06845*.
- Zheng et al. (2024a)
Zheng, W.; Song, R.; Guo, X.; Zhang, C.; and Chen, L. 2024a.
Genad: Generative end-to-end autonomous driving.
In *European Conference on Computer Vision*, 87–104. Springer.
- Zheng et al. (2024b)
Zheng, Y.; Xia, Z.; Zhang, Q.; Zhang, T.; Lu, B.; Huo, X.; Han, C.; Li, Y.; Yu, M.; Jin, B.; et al. 2024b.
Preliminary Investigation into Data Scaling Laws for Imitation Learning-Based End-to-End Autonomous Driving.
*arXiv preprint arXiv:2412.02689*.
- Zhou et al. (2023)
Zhou, L.; Song, Y.; Gao, Y.; Yu, Z.; Sodamin, M.; Liu, H.; Ma, L.; Liu, L.; Liu, H.; Liu, Y.; et al. 2023.
GarchingSim: An Autonomous Driving Simulator with Photorealistic Scenes and Minimalist Workflow.
In *2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC)*, 4227–4232. IEEE.

## Implementation Details

### Datasets

We reconstruct the entire nuScenes (Caesar et al. 2020) and Waymo (Sun et al. 2020) datasets into 3D Gaussian Splatting (3DGS) (Kerbl et al. 2023) environments. These reconstructed environments will be made publicly available to support future research.

#### nuScenes.

The nuScenes (Caesar et al. 2020) dataset comprises a large collection of diverse urban driving scenarios. It provides about 1.4 million annotated 3D bounding boxes across 23 object categories. Scene imagery is captured using six cameras offering a full 360-degree horizontal field of view, and keyframes are annotated at 2 Hz.

#### Waymo.

To further evaluate the generalization capability of , we also conduct experiments on the Waymo (Sun et al. 2020) dataset, a comprehensive benchmark featuring complex urban driving scenes under varied environmental conditions. It provides high-quality sensor data, including LiDAR, radar, and high-resolution imagery, enabling robust assessment across diverse scenarios.

### Metric

To evaluate the performance of the autonomous driving policy, we use six key metrics following RAD (Gao et al. 2025).

The Dynamic Collision Ratio (DCR) represents the frequency of collisions with dynamic obstacles and is calculated as:

$$ $DCR=\frac{N_{\text{dc}}}{N_{\text{total}}},$ (12) $$

where $N_{\text{dc}}$ denotes the number of clips where collisions with dynamic obstacles occur, and $N_{\text{total}}$ represents the total number of clips.

The Static Collision Ratio (SCR) quantifies the occurrence of collisions with static obstacles and is defined as:

$$ $SCR=\frac{N_{\text{sc}}}{N_{\text{total}}},$ (13) $$

where $N_{\text{sc}}$ is the number of clips with static obstacle collisions.

The Collision Ratio (CR) represents the overall collision frequency and is given by:

$$ $CR=DCR+SCR.$ (14) $$

The Positional Deviation Ratio (PDR) measures how closely the ego vehicle follows the expert trajectory in terms of position. It is defined as:

$$ $PDR=\frac{N_{\text{pd}}}{N_{\text{total}}},$ (15) $$

where $N_{\text{pd}}$ denotes the number of trajectory clips in which the positional deviation exceeds a predefined threshold.

The Heading Deviation Ratio (HDR) evaluates orientation accuracy by calculating the proportion of clips in which the heading deviation exceeds a predefined threshold.

$$ $HDR=\frac{N_{\text{hd}}}{N_{\text{total}}},$ (16) $$

where $N_{\text{hd}}$ is the number of clips where the heading deviation exceeds the threshold.

The Deviation Ratio (DR) provides a combined metric of positional and orientation deviations from the expert trajectory, formulated as:

$$ $DR=PDR+HDR.$ (17) $$

Meanwhile, since the edited scenes lack ground-truth expert trajectories, we only compute the collision-related metrics for these scenarios. And, we adopt Novel Trajectory Agent IoU (NTA-IoU) and Novel Trajectory Lane IoU (NTL-IoU) from DriveDreamer4D (Zhao et al. 2024a) to evaluate the realism of the environment in reinforcement learning, as they are specifically designed to measure the spatiotemporal consistency of dynamic agents (foreground) and static lane structures (background) in rendered videos.

NTA-IoU applies the YOLO11 detector (Khanam and Hussain 2024) to obtain 2D bounding boxes of vehicles from rendered images generated along novel trajectories. In parallel, the original 3D bounding boxes are transformed into the coordinate frame of the novel trajectory, allowing accurate projection into 2D bounding boxes under the new viewpoint. For each projected box, NTA-IoU identifies the closest detected box and computes the Intersection over Union (IoU).

In an analogous manner, the NTL-IoU is calculated by first applying geometric transformations to project lane structures from the original trajectories into the viewpoint of novel trajectories. Concurrently, lane lines in the newly rendered images are extracted using the TwinLiteNet model (Che et al. 2023). Ultimately, the mean Intersection over Union (IoU) between these projected lane lines and those detected by TwinLiteNet is computed.

### Training Details

To demonstrate the capability of the framework in enhancing training, we apply it to train RAD (Gao et al. 2025). Next, we introduce the network architecture of RAD (Gao et al. 2025) and its training stages. And, we summarize the detailed hyperparameter settings for both the imitation learning and reinforcement learning stages in Tab. [6](https://arxiv.org/html/2508.08170v2#Sx6.T6) and Tab. [7](https://arxiv.org/html/2508.08170v2#Sx6.T7), respectively.

#### Network Architecture of RAD.

RAD (Gao et al. 2025) takes multi-view images as input, transforms sensor data into scene token embeddings, and outputs the probabilistic distribution of actions. The framework consists of a BEV encoder, map head, agent head, image encoder, and planning head. Specifically, the BEV encoder transforms multi-view features from the perspective view into bird’s-eye-view (BEV) representations (Li et al. 2024c; Yang et al. 2023a). The map head extracts vectorized road elements. The agent head predicts motion states and trajectories of surrounding dynamic agents. Finally, the planning head outputs the probabilistic distribution of driving actions. Meanwhile, RAD (Gao et al. 2025) employs a two-stage training approach, including the imitation learning stage and the reinforcement learning stage.

#### Imitation Learning Stage.

In this stage, perception training is conducted, where the map head and agent head explicitly predict map elements and agent motion information, supervised by ground-truth labels. Subsequently, imitation learning is employed to initialize the action distribution based on large-scale driving demonstrations from expert drivers (Caesar et al. 2020; Sun et al. 2020; Xiao et al. 2021).

#### Reinforcement Learning Stage.

In the reinforcement learning stage, independent 3DGS models are trained for various driving scenario video clips, serving as digital driving environments. The autonomous driving policy continuously interacts with these 3DGS environments, calculates rewards based on actions, and optimizes the driving policy using the PPO (Schulman et al. 2017) algorithm.

**Table 6: Hyperparameters for the Imitation Learning Stage.**
| Config | Imitation Learning |
| --- | --- |
| Learning rate | $1\times 10^{-4}$ |
| Learning rate schedule | Cosine decay |
| Optimizer | AdamW (Loshchilov and Hutter 2017) |
| Optimizer hyper-parameters | $\beta_{1}=0.9$, $\beta_{2}=0.999$, $\epsilon=10^{-8}$ |
| Weight decay | $1\times 10^{-4}$ |
| Planning head dimension | 256 |

**Table 7: Hyperparameters for the Reinforcement Learning Stage.**
| Config | Reinforcement Learning |
| --- | --- |
| Learning rate | $5\times 10^{-6}$ |
| Learning rate schedule | Cosine decay |
| Optimizer | AdamW (Loshchilov and Hutter 2017) |
| Optimizer hyper-parameters | $\beta_{1}=0.9$, $\beta_{2}=0.999$, $\epsilon=10^{-8}$ |
| Weight decay | $1\times 10^{-4}$ |
| GAE parameters | $\gamma=0.9$, $\lambda=0.95$ |
| Clipping thresholds | $\epsilon_{x}=0.1$, $\epsilon_{y}=0.2$ |
| Planning head dimension | 256 |

Figure: Figure 8: Various trajectories generated by the Dynamic Adversary Agent (DAA) for the vehicle in the red box at the same timestep.
Refer to caption: x8.png

**Table 8: Impact of diffusion prior within ReconSimulator on novel view rendering quality across lane shift and lane change on the Waymo (Sun et al. 2020).**
| Method | Lane Shift @ 3m | Lane Shift @ 6m | Lane Change |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| NTA-IoU $\uparrow$ | NTL-IoU $\uparrow$ | FID $\downarrow$ | NTA-IoU $\uparrow$ | NTL-IoU $\uparrow$ | FID $\downarrow$ | NTA-IoU $\uparrow$ | NTL-IoU $\uparrow$ | FID $\downarrow$ |  |
| w/o Video Diffusion Prior | 0.477 | 53.23 | 128.90 | 0.353 | 47.14 | 224.62 | 0.486 | 55.34 | 89.83 |
| w/ Video Diffusion Prior | 0.543 | 54.84 | 90.12 | 0.454 | 52.92 | 148.23 | 0.552 | 56.12 | 72.86 |

### Details of Edited Scenes

We design a series of corner cases inspired by (Dosovitskiy et al. 2017; Jia et al. 2024). As shown in Fig. [10](https://arxiv.org/html/2508.08170v2#Sx7.F10) to [15](https://arxiv.org/html/2508.08170v2#Sx7.F15), we present a diverse set of corner case scenarios, including BlockedIntersection, DynamicCutIn, OppositeLaneIntrusion, ParkingCutIn, HardBrake, and HazardAtSideLane. Then, we detail several corner cases from our edited scenes.

#### BlockedIntersection.

While performing a maneuver, the ego vehicle encounters a stationary vehicle blocking the road and must take evasive action or apply the brakes to avoid a collision.

#### DynamicCutIn.

The ego vehicle must slow down or brake to allow a vehicle from traffic in an adjacent lane to cut in front.

#### OppositeLaneIntrusion.

While driving straight, a vehicle from the opposite lane encroaches into the ego vehicle’s lane, forcing the ego vehicle to brake or steer to the right to avoid a potential collision.

#### ParkingCutIn.

The ego vehicle must reduce its speed or apply the brakes to allow a parked vehicle exiting a parallel parking space to merge into the lane ahead.

#### HardBrake.

The leading vehicle decelerates abruptly, and the ego vehicle is required to perform an emergency stop.

#### HazardAtSideLane.

The ego vehicle encounters a slow-moving obstacle partially blocking the lane and needs to maneuver into a lane of traffic moving in the same direction to bypass it. Unlike ParkedObstacle, this scenario places greater emphasis on merging into another lane.

#### ParkedObstacle.

The ego vehicle encounters a parked car blocking part of the lane and must change lanes into moving traffic in the same direction to bypass it.

#### MergeIntoSlowTraffic.

The ego vehicle is required to merge into a slow-moving traffic flow.

#### Construction.

The ego vehicle encounters a construction zone blocking part of the lane and must shift into traffic moving in the same direction to navigate around it. In comparison to ParkedObstacle, the construction area occupies more of the lane’s width, and the ego vehicle must temporarily divert from its planned route to bypass it.

#### OppositeLaneRightTurn.

While preparing to make a right turn, a vehicle from the opposite lane enters the ego vehicle’s lane, forcing it to move right to avoid a potential collision.

#### BlindIntersectionCrossing.

The ego vehicle approaches an intersection with its view obstructed by stationary objects or vehicles, requiring it to slow down or stop and proceed only after confirming safety.

#### WrongWayVehicle.

A vehicle traveling in the wrong direction approaches the ego vehicle’s lane head-on, compelling it to immediately brake or steer away to avoid collision.

#### LaneChangeConflict.

Both the ego vehicle and a vehicle in an adjacent lane simultaneously attempt to merge into the same lane, requiring the ego vehicle to yield or abort the lane change.

#### ParkedObstacleTwoWays.

The ”TwoWays” version of ParkedObstacle. The ego vehicle faces a parked car blocking the lane and must change lanes into traffic moving in the opposite direction to avoid it.

#### ConstructionTwoWays.

The ”TwoWays” version of Construction. The ego vehicle encounters a construction area blocking the lane in both directions and must change lanes into traffic flowing in the opposite direction to avoid it. Compared to Construction, this obstruction occupies a wider lane area, requiring the ego vehicle to navigate through traffic in both directions.

#### HazardAtSideLaneTwoWays.

The ego vehicle faces a slow-moving obstacle partially blocking the lane and must either brake or maneuver into traffic moving in the opposite direction to avoid it.

**Table 9: Comparison of collision metrics on the Waymo (Sun et al. 2020) dataset**
| Method | CR$\downarrow$ | DCR$\downarrow$ | SCR$\downarrow$ |
| --- | --- | --- | --- |
| RAD (Gao et al. 2025) | 0.140 | 0.093 | 0.046 |
|  | 0.093 | 0.060 | 0.033 |

Figure: Figure 9: Comparison of different methods in challenging corner cases, with collisions highlighted by red boxes.
Refer to caption: x9.png

## Additional Experimental Results.

#### ReconSimulator.

We also perform ablation experiments on ReconSimulator to assess the impact of the diffusion prior (Ni et al. 2024; Zhao et al. 2024b; Wang et al. 2023) on novel view rendering quality, using the Waymo dataset (Sun et al. 2020), as shown in Tab. [8](https://arxiv.org/html/2508.08170v2#Sx6.T8). Specifically, for the lane change, integrating the video diffusion prior significantly improves the NTA-IoU by 13.58%, NTL-IoU by 1.41%, and reduces the FID score by 18.89%. These improvements clearly demonstrate that incorporating the video diffusion prior allows the rendered environment to provide better support for reinforcement learning tasks by enhancing the visual consistency and realism. Moreover, these results underline the strong generalization capability of ReconSimulator’s appearance modeling across diverse datasets, highlighting its robustness and effectiveness across datasets.

#### Quantitative Results on Waymo.

As shown in Tab. [9](https://arxiv.org/html/2508.08170v2#Sx6.T9), we quantitatively compare the performance of and RAD (Gao et al. 2025) on the Waymo dataset (Sun et al. 2020). Specifically, achieves a reduction in the Collision Ratio from 0.140 to 0.093, demonstrating superior safety performance and enhanced robustness in complex driving scenarios.

#### Qualitative Results on Waymo.

As shown in Fig. [9](https://arxiv.org/html/2508.08170v2#Sx6.F9), we compare and RAD (Gao et al. 2025) under two corner cases: HardBrake and DynamicCutIn. successfully avoids collisions by properly adjusting its speed, benefiting from a more realistic simulator enabled by diffusion-based scene reconstruction and further enhanced through data augmentation.

#### Visualization of DAA.

As mentioned in the main text, in the reinforcement learning stage, the Dynamic Adversary Agent (DAA) has a certain probability of fine-tuning the trajectories in the edited scenes (e.g., adjusting the target vehicle’s speed) instead of directly reusing those generated during the imitation stage. In Fig. [8](https://arxiv.org/html/2508.08170v2#Sx6.F8), we present different trajectories generated by DAA at the same timestep. Since the method directly modifies the speed based on the original trajectory, it does not affect the overall timing of the pipeline.

Figure: Figure 10: Example scenario of BlockedIntersection. The vehicle in the red box is stationary and blocks the intersection when the ego vehicle intends to turn. The ego vehicle must brake or change lanes to the left to avoid a potential collision.
Refer to caption: x10.png

Figure: Figure 11: Example scenario of DynamicCutIn. The vehicle in the red box is attempting to cut in from an adjacent lane while the ego vehicle is moving straight. The ego vehicle needs to slow down or brake to yield.
Refer to caption: x11.png

Figure: Figure 12: Example scenario of OppositeLaneIntrusion. The vehicle in the red box is attempting to encroach into the ego vehicle’s lane from the opposite direction. The ego vehicle must slow down or steer away to avoid a collision.
Refer to caption: x12.png

Figure: Figure 13: Example scenario of ParkingCutIn. The vehicle in the red box is attempting to exit a parallel parking space and merge into the lane ahead. The ego vehicle must slow down or stop to avoid a collision.
Refer to caption: x13.png

Figure: Figure 14: Example scenario of HardBrake. The vehicle in the red box is decelerating abruptly. The ego vehicle needs to change lanes to avoid a rear-end collision.
Refer to caption: x14.png

Figure: Figure 15: Example scenario of HazardAtSideLane. The vehicle in the red box is slowly moving and partially occupying the ego vehicle’s lane. The ego vehicle needs to change lanes to the left and merge into the traffic moving in the same direction.
Refer to caption: x15.png

## Peer Reviews

confidence: 5
contribution: 3
initial_rating: 4
presentation: 2
questions: As mentioned above, I believe there are lots of merits in the paper, while I also have concerns about the key technical contribution, since they are not well-presented in the current manuscript. I will consider adjusting my rating later if the weaknesses and questions are properly addressed later. 

- Do the authors empirically observe any forgetting issues when training on the synthetic data, if tested back into the real-domain with nominal driving cases?
- In line 265, what is the $f(\cdot)$ specifically defined in the DAA module? What is the sample scale needed for DAA to get interesting adversaries. If we conduct random samples, would most of the samples remain boring without challenging behaviors? 
- In Figure 6, the author mentions cousin-nuScenes created by CTG from the original nuScenes dataset. This is primarily used to cold-start the Rl algorithm. In the online rollout in the PPO, what is the data diversity, sample scale, as well as their impact to the final perfomance? An ablation study on the final performance vs sample complexity will be interesting. 
- Is DAA and CTG used to only cold-start the training data for RL, or also incorporated during the RL process? If they are both used during online learning, how are they used?
soundness: 3
strengths: - **Significance and Scope.** The paper addresses an important and timely problem in autonomous driving—closed-loop, end-to-end learning—by coupling photorealistic simulation with reinforcement learning in a way that directly targets deployment-relevant failures.
- **Comprehensive Qualitative Evaluation.** The study presents rich qualitative evidence across multiple benchmarks and driving scenarios, helping to substantiate generality and offering interpretable insights through scenario visualizations and case studies.
- **Strong Empirical Gains with Targeted Guidance.** Leveraging the Dynamic Adversary Agent (DAA) and Cousin Trajectory Generation (CTG), ReconDreamer-RL achieves consistent improvements over RAD and other E2E baselines on key safety and efficiency metrics in closed-loop settings.
- **Well-Designed Ablations.** Ablation studies are carefully constructed to isolate the contributions of ReconSimulator, DAA, and CTG, demonstrating each component’s necessity and the synergy of the full system.
summary: This work proposes ReconDreamer-RL, a two-stage framework that strengthens end-to-end autonomous-driving training by bridging imitation learning with reinforcement learning inside a photorealistic, editable simulator. The framework starts with ReconSimulato to fuse video-diffusion priors for high-quality novel-view rendering with a kinematic model for physically valid agent motion. On top of sensor simulator, a Dynamic Adversary Agent (DAA) auto-generates corner-case interactions (e.g., cut-ins) and a Cousin Trajectory Generator (CTG) diversifies ego behaviors to counter dataset bias. Together these components reduce the sim-to-real gap and improve closed-loop robustness, yielding large drops in collision metrics over IL/RL baselines while maintaining fast rendering suitable for RL training.
weaknesses: Despite the strengths mentioned above, I have some significant concerns on the technical contribution claimed by the paper, listed in this and the question section below:

- **Unclear presentation of the ReconSimulator**. The core technical contribution of this work comes from this ReconSimulator, however, the details of this simulator is unclear. For instance, How is the diffusion prior incorporated? From section 3.3, it seems that the ReconSimulator use a training checkpoint rendered by 3DGS and refined by DriveRestorer. What if we directly utilize 3DGS with decomposed foreground and background, just like what is done in HUGISM [1]? 
- **Limited contribution in the RL algorithm**. The title of this paper is ReconDreamer-RL, yet the novelty and analysis on the RL algorithm is missing in the main text. The authors mention in the appendix that they use PPO algorithm following RAD, which seems to be based on RAD with some incremental modification in synthetic data on the sensor simulation side. See more in the question part. 

> [1] Zhou, Hongyu, et al. "Hugsim: A real-time, photo-realistic and closed-loop simulator for autonomous driving." *arXiv 2024.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: See Weaknesses.
soundness: 3
strengths: 1. The paper introduces a novel method of integrating video diffusion priors directly into the reinforcement learning loop, creating a dynamic simulator that effectively addresses the sim2real gap.

2. It proposes a well-structured solution with dedicated modules (DAA and CTG) to systematically generate adversarial corner cases and mitigate common training data biases.

3. The framework demonstrates a significant and quantifiable performance improvement, achieving a 5x reduction in collision ratio over strong imitation learning baselines, providing clear evidence of its effectiveness.
summary: This paper presents ReconDreamer-RL, a framework designed to improve the training of end-to-end autonomous driving agents through reinforcement learning by addressing the critical simulation-to-reality (sim2real) gap. The authors identify a key limitation in existing scene reconstruction-based simulators: their inability to render high-quality sensor data for novel trajectories or rare corner cases not present in the original training data. To overcome this, the proposed method integrates video diffusion priors into the simulation process. The core of the framework is the ReconSimulator, which combines a diffusion-based appearance model with a kinematic physical model to reconstruct realistic driving scenarios. To further enhance the training process and cover a wider range of challenging situations, the paper introduces two additional components: the Dynamic Adversary Agent (DAA), which autonomously generates corner-case traffic scenarios by adjusting the trajectories of surrounding vehicles, and the Cousin Trajectory Generator (CTG), designed to mitigate the common bias in training data towards simple, straight-line movements.
weaknesses: 1. Lack of originality. Most techniques proposed in the paper have already been well studied in other works. Using video models to boost novel-trajcectory reconstruction performance is studied in many works such as Drivedreamer4d[1]; Decoupled static and dynamic scene representation is proposed in MTGS[2], OmniRe[3]; Adversary agent interaction is studied in a line of research featuring safty-critical interaction such as DiffScene[4].

2. Missing details in the dynamic adversary agent section. How exactly is the trajectory generated and how to perform sanity check for the generated trajectory?

[1] Drivedreamer4d: World models are effective data machines for 4d driving scene representation; Zhao, et.al.

[2] MTGS: Multi-Traversal Gaussian Splatting; Li, et.al

[3] OmniRe: Omni Urban Scene Reconstruction; Chen, et.al.

[4] DiffScene: Guided Diffusion Models for Safety-Critical Scenario Generation

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: Can you report RAD trained with the same DAA‑generated scenarios and/or CTG‑augmented data (keeping rendering identical), and vice‑versa, to isolate where the gains come from?
soundness: 3
strengths: 1. The paper shows strong empirical gains on collision metrics across standard and corner‑case evaluations; large gap over both IL baselines and RAD.
2. RL‑friendly simulator: 3DGS‑based with diffusion priors while retaining real‑time speed (Table 2). This addresses a common bottleneck in closed‑loop training.
summary: This paper proposes ReconDreamer‑RL, a framework for end‑to‑end autonomous‑driving policy training that couples (i) a reconstruction‑based simulator (ReconSimulator) enhanced by a video‑diffusion prior for appearance modeling with (ii) kinematic physical modeling of dynamic agents, and augments training with two data‑generation modules: a Dynamic Adversary Agent (DAA) that synthesizes corner‑case interactions and a Cousin Trajectory Generator (CTG) that diversifies ego trajectories. Training proceeds in two stages: imitation learning (IL) for initialization and reinforcement learning (RL) in closed loop. Experiments (nuScenes reconstruction as the main setting; Waymo for generalization) report large reductions in collision rate relative to IL baselines and sizable gains over the best prior RL baseline (RAD), while maintaining real‑time rendering speed.
weaknesses: 1. The policy is evaluated in the same simulator family that was fine‑tuned using diffusion priors, and many corner cases are generated by the paper’s own procedures (DAA, CTG). The real sim‑to‑real implications remain untested; no closed‑loop evaluation outside the authors’ reconstructions (e.g., Bench2Drive/nuPlan/CARLA‑v2 evaluations or any real‑vehicle trials).
2. Metrics are limited for driving quality. Collisions are critical, but comfort (jerk/accel), traffic‑rule compliance, and route completion are absent. The paper relies on PDR/HDR (Table 1) for trajectory adherence on unedited clips and on collision‑only metrics for edited scenes (Appx. A.2). This may overemphasize safety at the expense of smoothness and legality.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: * How does the collision avoidance in the CTG work and how does CTG interact with the more adversarial behaviors created from the DAA?
soundness: 3
strengths: * This paper focuses on a highly relevant research direction for AVs.
* The problem of reducing the sim2real gap and training e2e models with RL is addressed holistically with improvements in the simulator’s fidelity (3DGS with video diffusion prior), focusing on long-tail problems (DAA), and leverage diverse ego trajectories (CTG).
* Overall, the results demonstrate the superior performance of ReconDreamer-RL, e.g. a 3x reduction in collisions rates over baselines. And the method is validated on both nuScenes and the Waymo Open Dataset.
* The ablations also clearly demonstrate the value that each component brings.
summary: * This paper addresses the sim2real gap in training e2e autonomous driving models with RL.
* It proposes ReconDreamer-RL, a framework that integrates video diffusion priors to create more realistic and robust simulation environments for RL training.
* ReconSimulator: A simulator that uses 3DGS for reconstruction, a video diffusion prior for realistic appearance modeling of novel views, and a kinematic model.
* Dynamic Adversary Agent (DAA): Generates challenging traffic scenarios by controlling surrounding vehicles.
* Cousin Trajectory Generator (CTG): Enriches training data by synthesizing diverse ego trajectories to reduce simple straight driving.
* Training happens in two stages: 1) IL for policy initialization using dataset from CTG / DAA. 2) Closed-loop RL stage where policy interacts with ReconSimulator and adversarial DAA.
* Experiments demonstrate that this framework significantly outperforms baseline IL methods and achieves a new SOTA in RL + 3DGS.
weaknesses: * Lack of trajectory generation function implementation details. The authors write e.g. text-to-trajectory can be used but it seems that this paper lacks any additional details about the implementation used here. Given the importance of the DAA in the ablations, more details should be provided here.
* Unclear computational cost of the iterative scene reconstruction. You provide information about fast rendering speeds (125 fps) but not about the scene reconstruction, which seems particularly expensive. Can you provide more details here, e.g. how many steps are usually needed?


Decision: Reject