Rethinking Driving World Model as Synthetic Data Generator for Perception Tasks

## Abstract

Abstract Recent advancements in driving world models enable controllable generation of high-quality RGB videos or multimodal videos.
Existing methods primarily focus on metrics related to generation quality and controllability.
However, they often overlook the evaluation of downstream perception tasks, which are really crucial for the performance of autonomous driving.
Existing methods usually leverage a training strategy that first pretrains on synthetic data and finetunes on real data, resulting in twice the epochs compared to the baseline (real data only).
When we double the epochs in the baseline, the benefit of synthetic data becomes negligible.
To thoroughly demonstrate the benefit of synthetic data, we introduce Dream4Drive, a novel synthetic data generation framework designed for enhancing the downstream perception tasks.
Dream4Drive first decomposes the input video into several 3D-aware guidance maps and subsequently renders the 3D assets onto these guidance maps.
Finally, the driving world model is fine-tuned to produce the edited, multi-view photorealistic videos, which can be used to train the downstream perception models.
Dream4Drive enables unprecedented flexibility in generating multi-view corner cases at scale, significantly boosting corner case perception in autonomous driving.
To facilitate future research, we also contribute a large-scale 3D asset dataset named DriveObj3D, covering the typical categories in driving scenarios and enabling diverse 3D-aware video editing.
We conduct comprehensive experiments to show that Dream4Drive can effectively boost the performance of downstream perception models under various training epochs.

## 1 Introduction

Perception tasks such as 3D object detection (Li et al., 2022, 2024c; Wang et al., 2023a, 2025) and 3D tracking (Wang et al., 2023a; Zhang et al., 2023c; Han et al., 2025), which support planning and decision-making (Jiang et al., 2023; Hu et al., 2023), are extremely important in autonomous driving.
The performance of perception models, however, is highly dependent on large-scale annotated training datasets (Caesar et al., 2020; Wang et al., 2023b).
To ensure reliability in rare but critical safety scenarios, it is essential to gather adequate long-tail data.
Although the autonomous driving community has developed a thorough 3D annotation pipeline to facilitate data acquisition (Zhao et al., 2025b), collecting long-tail data remains highly time-consuming and labor-intensive.

Driving world models based on diffusion model (Rombach et al., 2022) and ControlNet (Zhang et al., 2023b) have been developed in autonomous driving to generate synthetic data from scene layouts (e.g., BEV maps, 3D bounding boxes) and text (Gao et al., 2023; Wen et al., 2024; Guo et al., 2025).
However, they provide limited control over the poses and appearances of individual objects, which restricts the ability to generate diverse synthetic data (Li et al., 2025).
Editing methods (Singh et al., 2024; Liang et al., 2025b; Yu et al., 2025) instead of generation extend driving world models by using reference images and 3D bounding boxes to modify object appearance and pose, allowing for diverse corner cases.
However, single-view insertion limits their application in multi-view BEV perception. Reconstruction-based methods using NeRF or 3DGS enable precise control over geometric structures (Chen et al., 2021; Zanjani et al., 2025). While geometry is preserved, sparse training views often lead to artifacts and incomplete renderings. Additionally, the lack of illumination modeling results in inconsistencies between inserted objects and the background.

More importantly, we argue that the data augmentation experiments of previous methods (Wen et al., 2024; Li et al., 2024a) are unfair, as they usually leverage a training strategy that first pretrains on synthetic data and finetunes on real data, resulting in twice the epochs compared to the baseline (real data only). We find that, under the same number of training epochs, large amounts of synthetic datasets offer little to no advantage and can even perform worse than using real data alone. As shown in Fig. [1](https://arxiv.org/html/2510.19195v2#S0.F1), under the 2× epoch setting, models trained exclusively on real data achieve higher mAP and NDS compared to those trained on real and synthetic data. Given the importance of downstream perception tasks for autonomous driving, we believe it is essential to rethink the effectiveness of the driving world model as a synthetic data generator for these tasks.

To reevaluate the value of synthetic data, we introduce Dream4Drive, a novel 3D-aware synthetic data generation framework designed for downstream perception tasks.
The core idea of Dream4Drive is to first decompose the input
video into several 3D-aware guidance maps and subsequently render the 3D assets onto these guidance maps.
Finally, the driving world model is fine-tuned to produce the edited, multi-view photorealistic videos, which can be used to train the downstream perception models.
Consequently, we can incorporate various assets with different trajectories(e.g., views, poses, and distance) into the same scene, significantly improving the diversity of the synthetic data.
As shown in Fig. [1](https://arxiv.org/html/2510.19195v2#S0.F1), under identical training epochs (1×, 2×, or 3×), our method requires only 420 synthetic samples—less than 2% of real samples—to surpass prior augmentation methods.
To be best of our knowledge, we are the first to demonstrate under fair comparisons that synthetic data can provide real benefits beyond training solely on real data.

Specifically, Dream4Drive leverages a multi-view video inpainting model finetuned from the Diffusion Transformer (Peebles and Xie, 2023). Unlike prior methods that rely on sparse spatial controls (e.g., BEV maps and 3D bounding boxes), Dream4Drive uses dense 3D-aware guidance maps (Yu and Smith, 2019; Liang et al., 2025a) (e.g., depth, normal, edge, cutout, and mask) to preserve the geometry and appearance of the original video, while editing them by rendering 3D assets into these maps. This design enables instance-level, cross-view consistent video editing, ensuring both visual realism and geometric fidelity. The generated videos not only achieve superior quality but can also be directly used to train state-of-the-art perception models.

To facilitate diverse 3D-aware video editing, we design a pipeline that automatically acquires high-quality 3D assets based on a target scene image or video of a desired asset category.
We first apply the image segmentation model (Ren et al., 2024; Lin et al., 2024) to localize and crop objects of the specified category, then employ the image generation model (Wu et al., 2025) to generate multi-view consistent images of the target object. These images are fed into a mesh generation model (Hunyuan3D et al., 2025) to generate high-quality 3D assets.
We present DriveObj3D, a large-scale 3D asset dataset that encompasses typical categories found in driving scenarios, to support future research.
Our main contributions are:

- •
We find that previous data augmentation methods are evaluated unfairly: under the same training epochs, hybrid datasets do not show any advantage over real data alone.
- •
We propose Dream4Drive, a 3D-aware synthetic data generation framework that edits the video with dense guidance maps, producing synthetic data with diverse appearances and geometric consistency.
- •
We contribute a large-scale dataset named DriveObj3D, covering the typical categories in driving scenarios for 3D-aware video editing.
- •
Extensive experiments across different training epochs show that adding less than 2% synthetic data can significantly improve perception performance, highlighting the effectiveness of Dream4Drive.

Figure: Figure 2: The illustration of Dream4Drive, which provides precise annotations, geometric variety, and appearance diversity to improve downstream perception tasks.
Refer to caption: x3.png

## 2 Related Work

Video Generation in Autonomous Driving.
High-quality data is crucial for training perception models in autonomous driving, motivating growing interest in driving video generation.
Early approaches (Yang et al., 2023; Gao et al., 2023; Luo et al., 2025; Li et al., 2024b) employ diffusion models with ControlNet, conditioned on BEV maps (Ma et al., 2024b) and 3D bounding boxes, to generate paired image data (An et al., 2024, 2025b; Luo et al., 2024c, a, b).
Recent works (Gao et al., 2024a; Jiang et al., 2024; Li et al., 2024a; Ji et al., 2025; Guo et al., 2025) adopt powerful Diffusion Transformers (DiTs) to further enhance generation quality.
SubjectDrive (Huang et al., 2024a) uses an external subject bank to enhance vehicle appearance diversity. However, these methods depend on original scene layouts, which limit geometric diversity and struggle to generate high-quality long-tail corner cases, thereby restricting their effectiveness for downstream perception.

Video Editing in Autonomous Driving.
To enrich scene diversity, object-level editing methods insert new objects into existing videos using reference images and 3D bounding boxes (Singh et al., 2024; Liang et al., 2025b; Buburuzan et al., 2025; An et al., 2025a). More recent NeRF- (Mildenhall et al., 2021) and 3DGS-based (Kerbl et al., 2023) approaches (Chen et al., 2021; Huang et al., 2024b; Chen et al., 2024; Wei et al., 2024) improve geometric fidelity, yet remain limited by sparse views, inconsistent lighting, and restricted background diversity. In contrast, our approach builds on generative models and introduces a novel 3D-aware video editing mechanism, enabling seamless insertion of diverse 3D assets and producing geometrically and visually diverse data that effectively boosts downstream performance.

## 3 Dream4Drive

Our goal is to generate high-quality synthetic videos from real videos with 3D box annotations and target 3D assets for training perception models.
We first introduce the preliminaries in Sec. [3.1](https://arxiv.org/html/2510.19195v2#S3.SS1), then explain how to conduct 3D-aware scene editing in Sec. [3.2](https://arxiv.org/html/2510.19195v2#S3.SS2), and finally describe how to render the edited video from the guidance maps in Sec. [3.3](https://arxiv.org/html/2510.19195v2#S3.SS3). The overall framework is shown in Fig. [2](https://arxiv.org/html/2510.19195v2#S1.F2).

### 3.1 Preliminaries

Latent Diffusion Models (LDMs) (Rombach et al., 2022) address the high computational cost of diffusion models by operating in a lower-dimensional latent space. Given an image $\mathbf{x}$, an encoder $\mathcal{E}$ is used to obtain the corresponding latent representation $\mathbf{z}=\mathcal{E}(\mathbf{x})$. The forward diffusion process in the latent space is defined as a gradual noising process:

$$ $q(\mathbf{z}_{t}|\mathbf{z}_{0})=\mathcal{N}(\mathbf{z}_{t};\sqrt{\bar{\alpha}_{t}}\mathbf{z}_{0},(1-\bar{\alpha}_{t})\mathbf{I}),$ (1) $$

where $\bar{\alpha}_{t}=\prod_{i=1}^{t}(1-\beta_{i})$ with $\beta_{i}$ being a variance schedule. The reverse process is modeled by a neural network $\epsilon_{\theta}$ and parameterized as:

$$ $p_{\theta}(\mathbf{z}_{t-1}|\mathbf{z}_{t})=\mathcal{N}(\mathbf{z}_{t-1};\mu_{\theta}(\mathbf{z}_{t},t),\Sigma_{\theta}(\mathbf{z}_{t},t)),$ (2) $$

Finally, a decoder $\mathcal{D}$ maps the denoised latent variable back to the image space, i.e., $\mathbf{x}^{\prime}=\mathcal{D}(\mathbf{z}_{0})$.

ControlNets (Zhang et al., 2023a)
extend LDMs by incorporating additional conditioning signals $\mathbf{c}$ to provide finer control over the generation process. In particular, the reverse diffusion process is modified to condition on $\mathbf{c}$:

$$ $p_{\theta}(\mathbf{z}_{t-1}|\mathbf{z}_{t},\mathbf{c})=\mathcal{N}(\mathbf{z}_{t-1};\mu_{\theta}(\mathbf{z}_{t},t,\mathbf{c}),\Sigma_{\theta}(\mathbf{z}_{t},t,\mathbf{c})),$ (3) $$

The conditioning variable $\mathbf{c}$ can incorporate various forms of guidance, including spatial maps, semantic layouts, and other task-specific signals. By integrating $\mathbf{c}$, ControlNets allow for more precise manipulation of the generation process and improving the quality of synthesized images.

### 3.2 3D-aware scene editing

Figure: Figure 3: The illustration of 3D-aware scene editing. Given the input images, we first obtain the depth map, normal map, and edge map for the background and then render the object image and object mask for the target 3D asset.
Refer to caption: x4.png

For an input RGB image $I\in\mathbb{R}^{H\times W\times 3}$, we use Depth Anything (Yang et al., 2024) to obtain the depth map $D\in\mathbb{R}^{H\times W\times 1}$. The normal map $N\in\mathbb{R}^{H\times W\times 3}$ is then derived from the depth map. We also use OpenCV’s Canny edge detector to get the edge map $E\in\mathbb{R}^{H\times W\times 1}$. It is important to note that the depth, normal, and edge information within the foreground object regions are masked since our model needs to learn to generate an edited video based on the target 3D asset.

For a target 3D asset in our DriveObj3D, we position it within the 3D space of the original video based on the provided 3D bounding boxes ${\{B_{i}}\}_{i=1}^{T}$. For each frame and each view, we then use calibrated camera intrinsics $K_{v}$ and extrinsics $E_{v}$ to render the target 3D asset. Therefore, we can obtain the object image $O\in\mathbb{R}^{H\times W\times 3}$ and object mask $M\in\mathbb{R}^{H\times W\times 1}$.

Once we have obtained the depth map, normal map, and edge map for the background, as well as the rendered object image and object mask for the target 3D asset, we utilize a fine-tuned driving world model to render the edited video based on these 3D-aware guidance maps.
This 3D-aware scene editing pipeline effectively utilizes the accurate pose, geometry, and texture information provided by 3D assets, ensuring geometric consistency in the results generated. Notably, our method does not depend on 3D bounding box embeddings for controlling object placement. Instead, we directly edit in 3D space, offering a more intuitive and reliable way to manage control.

### 3.3 3D-aware video rendering

Figure: Figure 4: The illustration of 3D-aware video rendering. Given the 3D-aware guidance maps, we employ a multi-condition fusion adapter to control the video generation of a diffusion transformer, rendering the edited video.
Refer to caption: x5.png

The video generation process in recent methods (Wen et al., 2024; Li et al., 2024a; Guo et al., 2025) relies on sparse spatial controls, such as bird’s eye view (BEV) maps and projected 3D bounding boxes. While this approach allows for some control over the synthesized content, it often falls short in achieving high-quality generation.
Instead of relying on sparse spatial controls, we fine-tune a driving world model to generate edited videos from the dense 3D-aware guidance maps, including depth map ($D$), normal map ($N$), and edge map ($E$) for the background and the image ($O$) and mask ($M$) for the foreground object.

The architecture of the multi-condition fusion adapter is shown in Fig. [4](https://arxiv.org/html/2510.19195v2#S3.F4).
We first encode the five conditions using a VAE, and then apply different 3D embedders to patchify the latents.
A FusionNet module then combines these five sets of features, as described by the following equation:

$$ $\displaystyle F_{\text{fusion}}=\text{FusionNet}\!\left(\bigoplus_{k=1}^{5}\text{3DEmbedder}_{k}(\text{VAE}(C_{k}))\;\Big|\;C_{k}\in\{D,N,E,O,M\}\right),$ (4) $$

where $D,N,E,O,M$ denote the depth, normal, edge, object, and mask, respectively, and $\oplus$ indicates concatenation along the channel dimension. The fused features are incorporated into the control blocks of the DiT architecture, enriching semantic information and thereby facilitating instance-level spatial alignment, temporal consistency, and semantic fidelity. The outputs of the control blocks are further integrated into the base block. Additionally, a spatial view attention mechanism is introduced to enhance cross-view coherence, which is especially beneficial in driving scenes.

We adopt rectified flow (Liu et al., 2022) for stable sampling and classifier-free guidance (Ho and Salimans, 2022) to balance text with multiple 3D geometric conditions, thereby improving controllability. The main training objective is a simplified diffusion loss that predicts the noise component at each timestep:

$$ $\mathcal{L}_{\text{diffusion}}=\mathbb{E}_{t,\mathbf{z}_{0},\epsilon\sim\mathcal{N}(0,\mathbf{I})}\!\left[||\epsilon-\epsilon_{\theta}(\mathbf{z}_{t},t,\mathbf{c})||^{2}\right],$ (5) $$

where $\epsilon$ is Gaussian noise, $\epsilon_{\theta}$ the predicted noise, $\mathbf{z}_{t}$ the noisy latent at timestep $t$, and $\mathbf{c}$ all conditioning signals. To achieve precise instance-level control, we introduce a Foreground Mask Loss as Ji et al. (2025) and an LPIPS loss (Zhang et al., 2018). The final training objective is:

$$ $\mathcal{L}_{\text{total}}=\lambda_{\text{diffusion}}\mathcal{L}_{\text{diffusion}}+\lambda_{\text{mask}}\mathcal{L}_{\text{mask}}+\lambda_{\text{lpips}}\mathcal{L}_{\text{LPIPS}},$ (6) $$

In our experiments, the weights are empirically set as $\lambda_{\text{diffusion}}=1.0$, $\lambda_{\text{mask}}=0.1$, and $\lambda_{\text{lpips}}=0.1$.
Notably, our training framework requires no expensive 3D annotations, relying solely on RGB videos and their 3D-aware guidance maps, which can be generated in real time using off-the-shelf tools. This approach significantly reduces training costs. More details are included in the Appx. [A](https://arxiv.org/html/2510.19195v2#A1).

## 4 DriveObj3D

Figure: Figure 5: The illustration of creating a 3D asset in DriveObj3D. We first apply a segmentation model to segment the target object, then generate multi-view images, and finally create a 3D mesh from those images.
Refer to caption: x6.png

To build a large-scale 3D assets for diverse 3D-aware video editing, we design a simple yet effective pipeline.
The core idea is to decompose the asset generation process into three steps: (i) 2D instance segmentation; (ii) multi-view image generation; (iii) 3D mesh generation. As shown in Fig. [5](https://arxiv.org/html/2510.19195v2#S4.F5), the pipeline inputs a video or image and the target asset category, and generates a 3D mesh for downstream applications.

Concretely, we first localize and segment the target object. Given an input image $I$ and category label *class*, Grounded-SAM (Ren et al., 2024) is applied to segment the object $I_{\text{target}}$. To overcome the occlusion of target object, we employ a multi-view image generation model Qwen-Image (Wu et al., 2025). Conditioned on $I_{\text{target}}$, Qwen-Image synthesizes a set of novel views $\{I_{v}\}_{v=1}^{N}$, which are subsequently fed into a multi-view reconstruction model Hunyuan3D (Hunyuan3D et al., 2025) to recover the final 3D mesh.

Figure: Figure 6: Comparison of 3D asset generation across different methods. Our simple yet effective method produces better 3D assets across diverse categories in autonomous driving, outperforming existing baselines. Con_vehicle is construction vehicle; Pdes is Pedestrian; T_cone is traffic cone.
Refer to caption: x7.png

As shown in Fig. [6](https://arxiv.org/html/2510.19195v2#S4.F6), assets generated by Text-to-3D methods (Xiang et al., 2025) often exhibit style inconsistencies with the original data, while single-view approaches (Hunyuan3D et al., 2025) tend to produce incomplete assets. In contrast, our simple yet effective pipeline leverages multi-view synthesis to generate complete and high-fidelity assets, even under severe occlusions. To support large-scale downstream driving tasks, we construct a diverse asset dataset, DriveObj3D, covering a wide range of categories in driving scenarios in Fig. [7](https://arxiv.org/html/2510.19195v2#S4.F7). All assets will be released publicly.

Figure: Figure 7: DriveObj3D dataset. A large-scale collection of diverse 3D assets across typical driving categories, supporting scalable video editing and synthetic data generation.
Refer to caption: x8.png

## 5 Experiments

### 5.1 Setup

#### Datasets.

We utilize the nuScenes dataset (Caesar et al., 2020) for building 3D assets and finetuning our video generation model.
It comprises 1000 scenes in total, with 700 designated for training, 150 for validation, and 150 for testing. Each scene contains a 20-second multi-view video captured by 6 cameras.
More details on inserted scene and asset selection are given in the Appx. [A](https://arxiv.org/html/2510.19195v2#A1).

#### Metrics.

Following Panacea (Wen et al., 2024) and SubjectDrive (Huang et al., 2024a), we primarily evaluate how the generated data improves perceptual model performance on detection and tracking tasks. Detection metrics include nuScenes Detection Score (NDS), mean Average Precision (mAP), mean Average Orientation Error (mAOE), and mean Average Velocity Error (mAVE). Tracking metrics include Average MultiObject Tracking Accuracy (AMOTA), Precision (AMOTP), Recall, and Accuracy (MOTA).

### 5.2 Main Results

#### Effectiveness for Downstream Tasks.

We evaluate the effectiveness of Dream4Drive against prior driving world models, with detection and tracking results reported in Tab. [1](https://arxiv.org/html/2510.19195v2#S5.T1) and Tab. [2](https://arxiv.org/html/2510.19195v2#S5.T2).
While Panacea and SubjectDrive outperform the real dataset baseline at double training epochs, aligning the training epochs shows minimal gains over using real data alone.
In contrast, our method explicitly edits objects at specified 3D positions and leverages 3D-aware guidance maps to guide foreground-background synthesis, generating accurately annotated videos that consistently improve downstream perception models.
Remarkably, with only 420 inserted samples, our approach outperforms prior methods that used the full set of synthetic data.
Moreover, for the first time, synthetic data achieves performance that surpasses real data when training epochs are equal.

**Table 1: Comparison of detection under different training epochs. * indicates the evaluation of WoVoGen is only on the vehicle classes of cars, trucks, and buses. Bold and underlines indicate the best and second best results respectively.**
|  | 1x Epochs | 2x Epochs |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Real | Dream4Drive | Real | DriveDreamer | WoVoGen^∗ | MagicDrive | Panacea | SubjectDrive | Dream4Drive |
|  | (28130) | (+420, <2%) | (28130) | (Wang et al., 2024a) | (Lu et al., 2024) | (Gao et al., 2023) | (Wen et al., 2024) | (Huang et al., 2024a) | (Ours) |
| mAP $\uparrow$ | 34.5 | 36.1 | 38.4 | 35.8 | 36.2 | 35.4 | 37.1 | 38.1 | 38.7 |
| mAVE $\downarrow$ | 29.1 | 28.9 | 27.7 | – | 123.4 | – | 27.3 | 26.4 | 26.8 |
| NDS $\uparrow$ | 46.9 | 47.8 | 50.4 | 39.5 | 18.1 | 39.8 | 49.2 | 50.2 | 50.6 |

**Table 2: Comparison of tracking under different training epochs. Bold and underlines indicate the best and second best.**
|  | 1x Epochs | 2x Epochs |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | Real | Dream4Drive | Real | Panacea | SubjectDrive | Dream4Drive |
|  | (28130) | (+420, <2%) | (28130) | (Wen et al., 2024) | (Huang et al., 2024a) | (Ours) |
| AMOTA $\uparrow$ | 30.1 | 31.2 | 34.1 | 33.7 | 33.7 | 34.4 |
| AMOTP $\downarrow$ | 137.9 | 135.4 | 134.1 | 135.3 | 135.3 | 133.5 |

#### Effectiveness for Various Resolutions.

As generative models continue to advance, the ability to synthesize high-resolution videos has become achievable (Gao et al., 2024b, a). To investigate the effect of high-resolution synthetic data on downstream perception models, we further conduct experiments for detection and tracking tasks at a resolution of $512\times 768$, as reported in Tab. [3](https://arxiv.org/html/2510.19195v2#S5.T3) and [4](https://arxiv.org/html/2510.19195v2#S5.T4).

**Table 3: Detection performance under different training epochs (1x, 2x, 3x). “Naive Insert" denotes the direct projection of 3D assets into the original scene. Results are reported at 512×768 resolution.**
|  | 1x Epochs | 2x Epochs | 3x Epochs |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Real | Naive Insert | Dream4Drive | Real | Naive Insert | Dream4Drive | Real | Naive Insert | Dream4Drive |
| mAP $\uparrow$ | 36.1 | 40.1 | 40.7 | 42.2 | 42.9 | 43.6 | 43.1 | 43.1 | 44.5 |
| mATE $\downarrow$ | 69.2 | 64.7 | 64.2 | 61.6 | 62.4 | 61.6 | 60.5 | 61.5 | 59.8 |
| mAOE $\downarrow$ | 56.7 | 49.0 | 48.0 | 43.2 | 37.5 | 39.4 | 45.7 | 38.9 | 40.1 |
| mAVE $\downarrow$ | 28.5 | 28.4 | 27.1 | 27.5 | 27.3 | 27.4 | 27.4 | 27.4 | 27.2 |
| NDS $\uparrow$ | 47.9 | 51.3 | 52.0 | 53.2 | 54.0 | 54.3 | 53.6 | 54.2 | 55.0 |

Under both 1x and 2x epochs, real data and Dream4Drive significantly outperform their low-resolution counterparts ($256\times 512$, Tab. [1](https://arxiv.org/html/2510.19195v2#S5.T1) and [2](https://arxiv.org/html/2510.19195v2#S5.T2)). Remarkably, with high-resolution augmentation, Dream4Drive requires only 420 samples to achieve a 4.6 point (12.7%) mAP increase and a 4.1 point (8.6%) NDS improvement. Most of the gains come from large vehicle categories, including bus, construction_vehicle, and truck; detailed AP for each category is provided in the Appx. [B](https://arxiv.org/html/2510.19195v2#A2). Unlike prior augmentation paradigms, Dream4Drive consistently outperforms training on real data alone, regardless of the number of epochs, highlighting the value of high-quality synthetic data.

#### Quantitative and Qualitative Comparison with Naive Insertion.

After extracting 3D assets, we can directly generate edited videos with projection.
To assess the impact of 3D-aware video rendering versus direct insertion on downstream tasks, we conduct comprehensive evaluations, as reported in Tab. [3](https://arxiv.org/html/2510.19195v2#S5.T3) and [4](https://arxiv.org/html/2510.19195v2#S5.T4). While direct insertion improves performance over real data alone, its results remain inferior to our generative method due to missing realism, such as shadows and reflections.
Interestingly, direct insertion achieves the highest mAOE, likely because the inserted assets perfectly align with the original bounding box orientations. Fig. [8](https://arxiv.org/html/2510.19195v2#S5.F8) presents visual comparisons between naive insertion and our generative approach across multiple scenes.

Figure: Figure 8: Comparison with naive insertion. As can be seen, Dream4Drive generates more realistic edited videos than naive insertion.
Refer to caption: x9.png

**Table 4: Tracking performance under different training epochs (1x, 2x, 3x). “Naive Insert" denotes the direct projection of 3D assets into the original scene. Results are reported at 512×768 resolution.**
|  | 1x Epochs | 2x Epochs | 3x Epochs |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Real | Naive Insert | Ours | Real | Naive Insert | Ours | Real | Naive Insert | Ours |
| AMOTA $\uparrow$ | 32.8 | 36.5 | 37.9 | 39.7 | 42.2 | 42.6 | 41.3 | 42.2 | 43.5 |
| AMOTP $\downarrow$ | 134.0 | 128.7 | 128.0 | 125.1 | 124.0 | 123.3 | 124.1 | 123.7 | 121.3 |
| MOTA $\uparrow$ | 28.1 | 31.7 | 33.1 | 35.6 | 37.3 | 37.4 | 36.8 | 37.5 | 38.5 |
| RECALL $\uparrow$ | 44.0 | 45.4 | 46.9 | 50.7 | 51.1 | 51.8 | 52.4 | 51.8 | 52.5 |

#### Comprehensive visualization of asset insertion results.

More asset categories, insertion locations, and corner case scenarios are comprehensively presented in the Appx. [D](https://arxiv.org/html/2510.19195v2#A4).

### 5.3 Ablation Studies

#### Effect of Insertion Position.

Dream4Drive can edit videos by projecting 3D assets anywhere in a scene. To systematically evaluate the impact of insertion position on downstream model performance, we categorize positions as front, back, left, and right, as shown in Tab. [5](https://arxiv.org/html/2510.19195v2#S5.T5).

Results indicate that inserting assets in the front or back yields similar performance, whereas left-side insertions outperform right-side ones, with a 0.4 point increase in mAP, 0.9 point increase in NDS, and a 5.7 point reduction in mAOE.
This likely indicates dataset bias: most vehicles appear on the ego vehicle’s left side, so enhancing such corner cases improves model predictions, while right-side corner cases yield limited gains on the validation set.

We further examine the effect of insertion distance on performance in Tab. [5](https://arxiv.org/html/2510.19195v2#S5.T5). Close insertions tend to perform poorly, likely due to the asset blocking the camera view, which interferes with the training of other instances.
Distant insertions offer more effective augmentation, as detectors often struggle with distant objects. Increasing their prevalence enhances detection performance.

#### Effect of 3D Asset Source.

We observe that the source of inserted assets affects the quality of synthetic data, consistent with (Ljungbergh et al., 2025). We therefore investigate how different asset sources influence downstream perception performance.

**Table 5: Ablation Studies. We report detection performance across insertion positions and asset, with best results per block (Views, Distances, 3D Methods) in bold, at 512×768 resolution.**
|  | Views | Distances | 3D Asset Generation Methods |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Front | Back | Left | Right | Close | Mid | Far | Trellis | Hunyuan3D | Ours |
| mAP $\uparrow$ | 40.2 | 40.2 | 40.2 | 39.8 | 39.7 | 40.3 | 40.5 | 39.8 | 40.2 | 40.7 |
| mATE $\downarrow$ | 66.2 | 66.0 | 64.6 | 66.2 | 65.7 | 65.4 | 65.1 | 65.6 | 65.1 | 64.2 |
| mAOE $\downarrow$ | 51.2 | 55.1 | 45.7 | 51.4 | 52.2 | 51.7 | 49.7 | 51.8 | 50.8 | 48.0 |
| mAVE $\downarrow$ | 27.7 | 27.5 | 28.5 | 27.9 | 28.1 | 28.0 | 27.9 | 27.8 | 28.0 | 27.1 |
| NDS $\uparrow$ | 51.0 | 50.6 | 51.6 | 50.7 | 50.5 | 50.9 | 51.3 | 50.8 | 50.9 | 52.0 |

As shown in Tab. [5](https://arxiv.org/html/2510.19195v2#S5.T5), although Trellis (Xiang et al., 2025) can generate high-quality assets, its style does not fully match autonomous driving scenarios, which can lead to artifacts and degraded quality when assets are inserted, negatively affecting downstream tasks.
Hunyuan3D (Hunyuan3D et al., 2025)’s single-view generation also underperforms our multiview approach, since single-image assets may be incomplete, whereas our method produces complete, high-quality 3D assets.

### 5.4 Takeaways

We summarize the main observations from our experiments with perception model augmentation as follows:

- •
Duplying original layouts to create synthetic data does not improve performance; instead, enhancing scenes by inserting new 3D assets is an effective strategy for augmentation.
- •
High-resolution synthetic data offers greater benefits for data augmentation.
- •
The placement of inserted assets influences the effectiveness of augmentation, highlighting biases present in the dataset.
- •
Insertions at farther distances generally improve performance, while close-range insertions may introduce strong occlusions that hinder training.
- •
Using assets from the same dataset reduces the domain gap between synthetic and real data, benefiting downstream model training.

## 6 Conclusion

In this paper, we find that previous driving world models inaccurately assess the effectiveness of synthetic data for downstream tasks. To address this, we present Dream4Drive, a 3D-aware synthetic data generation pipeline that synthesizes high-quality multi-view corner cases. To facilitate future research, we also contribute a large-scale 3D asset dataset named DriveObj3D, covering the typical categories in driving scenarios. Extensive experiments demonstrate that with less than 2% additional synthetic data, Dream4Drive consistently improves downstream perception, validating the effectiveness of synthetic data for autonomous driving.

## References

- An et al. (2024)
Ruichuan An, Sihan Yang, Ming Lu, Renrui Zhang, Kai Zeng, Yulin Luo, Jiajun Cao, Hao Liang, Ying Chen, Qi She, et al.
Mc-llava: Multi-concept personalized vision-language model.
*arXiv preprint arXiv:2411.11706*, 2024.
- An et al. (2025a)
Ruichuan An, Sihan Yang, Renrui Zhang, Zijun Shen, Ming Lu, Gaole Dai, Hao Liang, Ziyu Guo, Shilin Yan, Yulin Luo, et al.
Unictokens: Boosting personalized understanding and generation via unified concept tokens.
*arXiv preprint arXiv:2505.14671*, 2025a.
- An et al. (2025b)
Ruichuan An, Kai Zeng, Ming Lu, Sihan Yang, Renrui Zhang, Huitong Ji, Qizhe Zhang, Yulin Luo, Hao Liang, and Wentao Zhang.
Concept-as-tree: Synthetic data is all you need for vlm personalization.
*arXiv preprint arXiv:2503.12999*, 2025b.
- Buburuzan et al. (2025)
Alexandru Buburuzan, Anuj Sharma, John Redford, Puneet K Dokania, and Romain Mueller.
Mobi: Multimodal object inpainting using diffusion models.
In *Proceedings of the Computer Vision and Pattern Recognition Conference*, pages 1974–1984, 2025.
- Caesar et al. (2020)
Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom.
nuscenes: A multimodal dataset for autonomous driving.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 11621–11631, 2020.
- Chen et al. (2021)
Yun Chen, Frieda Rong, Shivam Duggal, Shenlong Wang, Xinchen Yan, Sivabalan Manivasagam, Shangjie Xue, Ersin Yumer, and Raquel Urtasun.
Geosim: Realistic video simulation via geometry-aware composition for self-driving.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 7230–7240, 2021.
- Chen et al. (2024)
Ziyu Chen, Jiawei Yang, Jiahui Huang, Riccardo de Lutio, Janick Martinez Esturo, Boris Ivanovic, Or Litany, Zan Gojcic, Sanja Fidler, Marco Pavone, et al.
Omnire: Omni urban scene reconstruction.
*arXiv preprint arXiv:2408.16760*, 2024.
- Gao et al. (2023)
Ruiyuan Gao, Kai Chen, Enze Xie, Lanqing Hong, Zhenguo Li, Dit-Yan Yeung, and Qiang Xu.
Magicdrive: Street view generation with diverse 3d geometry control.
*arXiv preprint arXiv:2310.02601*, 2023.
- Gao et al. (2024a)
Ruiyuan Gao, Kai Chen, Bo Xiao, Lanqing Hong, Zhenguo Li, and Qiang Xu.
Magicdrivedit: High-resolution long video generation for autonomous driving with adaptive control.
*arXiv preprint arXiv:2411.13807*, 2024a.
- Gao et al. (2024b)
Shenyuan Gao, Jiazhi Yang, Li Chen, Kashyap Chitta, Yihang Qiu, Andreas Geiger, Jun Zhang, and Hongyang Li.
Vista: A generalizable driving world model with high fidelity and versatile controllability.
*arXiv preprint arXiv:2405.17398*, 2024b.
- Guo et al. (2025)
Xiangyu Guo, Zhanqian Wu, Kaixin Xiong, Ziyang Xu, Lijun Zhou, Gangwei Xu, Shaoqing Xu, Haiyang Sun, Bing Wang, Guang Chen, et al.
Genesis: Multimodal driving scene generation with spatio-temporal and cross-modal consistency.
*arXiv preprint arXiv:2506.07497*, 2025.
- Han et al. (2025)
Liuyang Han, Jun Wang, Chen Li, Fazhan Tao, and Zhumu Fu.
A novel multi-object tracking framework based on multi-sensor data fusion for autonomous driving in adverse weather environments.
*IEEE Sensors Journal*, 2025.
- Ho and Salimans (2022)
Jonathan Ho and Tim Salimans.
Classifier-free diffusion guidance.
*arXiv preprint arXiv:2207.12598*, 2022.
- Hu et al. (2023)
Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, et al.
Planning-oriented autonomous driving.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 17853–17862, 2023.
- Huang et al. (2024a)
Binyuan Huang, Yuqing Wen, Yucheng Zhao, Yaosi Hu, Yingfei Liu, Fan Jia, Weixin Mao, Tiancai Wang, Chi Zhang, Chang Wen Chen, et al.
Subjectdrive: Scaling generative data in autonomous driving via subject control.
*arXiv preprint arXiv:2403.19438*, 2024a.
- Huang et al. (2024b)
Nan Huang, Xiaobao Wei, Wenzhao Zheng, Pengju An, Ming Lu, Wei Zhan, Masayoshi Tomizuka, Kurt Keutzer, and Shanghang Zhang.
S3gaussian: Self-supervised street gaussians for autonomous driving.
*arXiv preprint arXiv:2405.20323*, 2024b.
- Hunyuan3D et al. (2025)
Team Hunyuan3D, Shuhui Yang, Mingxin Yang, Yifei Feng, Xin Huang, Sheng Zhang, Zebin He, Di Luo, Haolin Liu, Yunfei Zhao, et al.
Hunyuan3d 2.1: From images to high-fidelity 3d assets with production-ready pbr material.
*arXiv preprint arXiv:2506.15442*, 2025.
- Ji et al. (2025)
Yishen Ji, Ziyue Zhu, Zhenxin Zhu, Kaixin Xiong, Ming Lu, Zhiqi Li, Lijun Zhou, Haiyang Sun, Bing Wang, and Tong Lu.
Cogen: 3d consistent video generation via adaptive conditioning for autonomous driving.
*arXiv preprint arXiv:2503.22231*, 2025.
- Jiang et al. (2023)
Bo Jiang, Shaoyu Chen, Qing Xu, Bencheng Liao, Jiajie Chen, Helong Zhou, Qian Zhang, Wenyu Liu, Chang Huang, and Xinggang Wang.
Vad: Vectorized scene representation for efficient autonomous driving.
In *Proceedings of the IEEE/CVF International Conference on Computer Vision*, pages 8340–8350, 2023.
- Jiang et al. (2024)
Junpeng Jiang, Gangyi Hong, Lijun Zhou, Enhui Ma, Hengtong Hu, Xia Zhou, Jie Xiang, Fan Liu, Kaicheng Yu, Haiyang Sun, et al.
Dive: Dit-based video generation with enhanced control.
*arXiv preprint arXiv:2409.01595*, 2024.
- Kerbl et al. (2023)
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis.
3d gaussian splatting for real-time radiance field rendering.
*ACM Trans. Graph.*, 42(4):139–1, 2023.
- Li et al. (2024a)
Bohan Li, Jiazhe Guo, Hongsi Liu, Yingshuang Zou, Yikang Ding, Xiwu Chen, Hu Zhu, Feiyang Tan, Chi Zhang, Tiancai Wang, et al.
Uniscene: Unified occupancy-centric driving scene generation.
*arXiv preprint arXiv:2412.05435*, 2024a.
- Li et al. (2024b)
Jian Li, Weiheng Lu, Hao Fei, Meng Luo, Ming Dai, Min Xia, Yizhang Jin, Zhenye Gan, Ding Qi, Chaoyou Fu, et al.
A survey on benchmarks of multimodal large language models.
*arXiv preprint arXiv:2408.08632*, 2024b.
- Li et al. (2025)
Jiusi Li, Jackson Jiang, Jinyu Miao, Miao Long, Tuopu Wen, Peijin Jia, Shengxiang Liu, Chunlei Yu, Maolin Liu, Yuzhan Cai, et al.
Realistic and controllable 3d gaussian-guided object editing for driving video generation.
*arXiv preprint arXiv:2508.20471*, 2025.
- Li et al. (2022)
Kaican Li, Kai Chen, Haoyu Wang, Lanqing Hong, Chaoqiang Ye, Jianhua Han, Yukuai Chen, Wei Zhang, Chunjing Xu, Dit-Yan Yeung, et al.
Coda: A real-world road corner case dataset for object detection in autonomous driving.
In *European Conference on Computer Vision*, pages 406–423. Springer, 2022.
- Li et al. (2024c)
Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chonghao Sima, Tong Lu, Qiao Yu, and Jifeng Dai.
Bevformer: learning bird’s-eye-view representation from lidar-camera via spatiotemporal transformers.
*IEEE Transactions on Pattern Analysis and Machine Intelligence*, 2024c.
- Liang et al. (2025a)
Ruofan Liang, Zan Gojcic, Huan Ling, Jacob Munkberg, Jon Hasselgren, Chih-Hao Lin, Jun Gao, Alexander Keller, Nandita Vijaykumar, Sanja Fidler, et al.
Diffusion renderer: Neural inverse and forward rendering with video diffusion models.
In *Proceedings of the Computer Vision and Pattern Recognition Conference*, pages 26069–26080, 2025a.
- Liang et al. (2025b)
Yiyuan Liang, Zhiying Yan, Liqun Chen, Jiahuan Zhou, Luxin Yan, Sheng Zhong, and Xu Zou.
Driveeditor: A unified 3d information-guided framework for controllable object editing in driving scenes.
In *Proceedings of the AAAI Conference on Artificial Intelligence*, pages 5164–5172, 2025b.
- Lin et al. (2024)
Weifeng Lin, Xinyu Wei, Ruichuan An, Peng Gao, Bocheng Zou, Yulin Luo, Siyuan Huang, Shanghang Zhang, and Hongsheng Li.
Draw-and-understand: Leveraging visual prompts to enable mllms to comprehend what you want.
*arXiv preprint arXiv:2403.20271*, 2024.
- Liu et al. (2022)
Xingchao Liu, Chengyue Gong, and Qiang Liu.
Flow straight and fast: Learning to generate and transfer data with rectified flow.
*arXiv preprint arXiv:2209.03003*, 2022.
- Ljungbergh et al. (2025)
William Ljungbergh, Bernardo Taveira, Wenzhao Zheng, Adam Tonderski, Chensheng Peng, Fredrik Kahl, Christoffer Petersson, Michael Felsberg, Kurt Keutzer, Masayoshi Tomizuka, et al.
R3d2: Realistic 3d asset insertion via diffusion for autonomous driving simulation.
*arXiv preprint arXiv:2506.07826*, 2025.
- Lu et al. (2024)
Jiachen Lu, Ze Huang, Zeyu Yang, Jiahui Zhang, and Li Zhang.
Wovogen: World volume-aware diffusion for controllable multi-camera driving scene generation.
In *European Conference on Computer Vision*, pages 329–345. Springer, 2024.
- Luo et al. (2024a)
Meng Luo, Hao Fei, Bobo Li, Shengqiong Wu, Qian Liu, Soujanya Poria, Erik Cambria, Mong-Li Lee, and Wynne Hsu.
Panosent: A panoptic sextuple extraction benchmark for multimodal conversational aspect-based sentiment analysis.
In *Proceedings of the 32nd ACM International Conference on Multimedia*, pages 7667–7676, 2024a.
- Luo et al. (2024b)
Meng Luo, Han Zhang, Shengqiong Wu, Bobo Li, Hong Han, and Hao Fei.
Nus-emo at semeval-2024 task 3: Instruction-tuning llm for multimodal emotion-cause analysis in conversations.
*arXiv preprint arXiv:2501.17261*, 2024b.
- Luo et al. (2025)
Meng Luo, Shengqiong Wu, Liqiang Jing, Tianjie Ju, Li Zheng, Jinxiang Lai, Tianlong Wu, Xinya Du, Jian Li, Siyuan Yan, Jiebo Luo, William Yang Wang, Hao Fei, Mong-Li Lee, and Wynne Hsu.
Dr.v: A hierarchical perception-temporal-cognition framework to diagnose video hallucination by fine-grained spatial-temporal grounding, 2025.
- Luo et al. (2024c)
Yulin Luo, Ruichuan An, Bocheng Zou, Yiming Tang, Jiaming Liu, and Shanghang Zhang.
Llm as dataset analyst: Subpopulation structure discovery with large language model.
In *European Conference on Computer Vision*, pages 235–252. Springer, 2024c.
- Ma et al. (2024a)
Enhui Ma, Lijun Zhou, Tao Tang, Zhan Zhang, Dong Han, Junpeng Jiang, Kun Zhan, Peng Jia, Xianpeng Lang, Haiyang Sun, et al.
Unleashing generalization of end-to-end autonomous driving with controllable long video generation.
*arXiv preprint arXiv:2406.01349*, 2024a.
- Ma et al. (2024b)
Yuexin Ma, Tai Wang, Xuyang Bai, Huitong Yang, Yuenan Hou, Yaming Wang, Yu Qiao, Ruigang Yang, and Xinge Zhu.
Vision-centric bev perception: A survey.
*IEEE Transactions on Pattern Analysis and Machine Intelligence*, 2024b.
- Mildenhall et al. (2021)
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng.
Nerf: Representing scenes as neural radiance fields for view synthesis.
*Communications of the ACM*, 65(1):99–106, 2021.
- Peebles and Xie (2023)
William Peebles and Saining Xie.
Scalable diffusion models with transformers.
In *Proceedings of the IEEE/CVF international conference on computer vision*, pages 4195–4205, 2023.
- Ren et al. (2024)
Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, et al.
Grounded sam: Assembling open-world models for diverse visual tasks.
*arXiv preprint arXiv:2401.14159*, 2024.
- Rombach et al. (2022)
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.
High-resolution image synthesis with latent diffusion models.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 10684–10695, 2022.
- Singh et al. (2024)
Bharat Singh, Viveka Kulharia, Luyu Yang, Avinash Ravichandran, Ambrish Tyagi, and Ashish Shrivastava.
Genmm: Geometrically and temporally consistent multimodal data generation for video and lidar.
*arXiv preprint arXiv:2406.10722*, 2024.
- Wang et al. (2025)
Hai Wang, Junhao Liu, Haoran Dong, and Zheng Shao.
A survey of the multi-sensor fusion object detection task in autonomous driving.
*Sensors*, 25(9):2794, 2025.
- Wang et al. (2023a)
Shihao Wang, Yingfei Liu, Tiancai Wang, Ying Li, and Xiangyu Zhang.
Exploring object-centric temporal modeling for efficient multi-view 3d object detection.
In *Proceedings of the IEEE/CVF international conference on computer vision*, pages 3621–3631, 2023a.
- Wang et al. (2023b)
Xiaofeng Wang, Zheng Zhu, Yunpeng Zhang, Guan Huang, Yun Ye, Wenbo Xu, Ziwei Chen, and Xingang Wang.
Are we ready for vision-centric driving streaming perception? the asap benchmark.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 9600–9610, 2023b.
- Wang et al. (2024a)
Xiaofeng Wang, Zheng Zhu, Guan Huang, Xinze Chen, Jiagang Zhu, and Jiwen Lu.
Drivedreamer: Towards real-world-drive world models for autonomous driving.
In *European Conference on Computer Vision*, pages 55–72. Springer, 2024a.
- Wang et al. (2024b)
Yuqi Wang, Jiawei He, Lue Fan, Hongxin Li, Yuntao Chen, and Zhaoxiang Zhang.
Driving into the future: Multiview visual forecasting and planning with world model for autonomous driving.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 14749–14759, 2024b.
- Wei et al. (2024)
Xiaobao Wei, Qingpo Wuwu, Zhongyu Zhao, Zhuangzhe Wu, Nan Huang, Ming Lu, Ningning Ma, and Shanghang Zhang.
Emd: Explicit motion modeling for high-quality street gaussian splatting.
*arXiv preprint arXiv:2411.15582*, 2024.
- Wen et al. (2024)
Yuqing Wen, Yucheng Zhao, Yingfei Liu, Fan Jia, Yanhui Wang, Chong Luo, Chi Zhang, Tiancai Wang, Xiaoyan Sun, and Xiangyu Zhang.
Panacea: Panoramic and controllable video generation for autonomous driving.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 6902–6912, 2024.
- Wu et al. (2025)
Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, Yuxiang Chen, Zecheng Tang, Zekai Zhang, Zhengyi Wang, An Yang, Bowen Yu, Chen Cheng, Dayiheng Liu, Deqing Li, Hang Zhang, Hao Meng, Hu Wei, Jingyuan Ni, Kai Chen, Kuan Cao, Liang Peng, Lin Qu, Minggang Wu, Peng Wang, Shuting Yu, Tingkun Wen, Wensen Feng, Xiaoxiao Xu, Yi Wang, Yichang Zhang, Yongqiang Zhu, Yujia Wu, Yuxuan Cai, and Zenan Liu.
Qwen-image technical report, 2025.
- Xiang et al. (2025)
Jianfeng Xiang, Zelong Lv, Sicheng Xu, Yu Deng, Ruicheng Wang, Bowen Zhang, Dong Chen, Xin Tong, and Jiaolong Yang.
Structured 3d latents for scalable and versatile 3d generation.
In *Proceedings of the Computer Vision and Pattern Recognition Conference*, pages 21469–21480, 2025.
- Yang et al. (2023)
Kairui Yang, Enhui Ma, Jibin Peng, Qing Guo, Di Lin, and Kaicheng Yu.
Bevcontrol: Accurately controlling street-view elements with multi-perspective consistency via bev sketch layout.
*arXiv preprint arXiv:2308.01661*, 2023.
- Yang et al. (2024)
Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao.
Depth anything: Unleashing the power of large-scale unlabeled data.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 10371–10381, 2024.
- Yu et al. (2025)
Beike Yu, Dafang Wang, Jiang Cao, Pengyu Zhu, and Yifei Zhao.
Vehiclesim: realistic and 3d-aware video editing with one image for autonomous driving.
*Multimedia Systems*, 31(4):316, 2025.
- Yu and Smith (2019)
Ye Yu and William AP Smith.
Inverserendernet: Learning single image inverse rendering.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 3155–3164, 2019.
- Zanjani et al. (2025)
Farhad G Zanjani, Davide Abati, Auke Wiggers, Dimitris Kalatzis, Jens Petersen, Hong Cai, and Amirhossein Habibian.
Gaussian splatting is an effective data generator for 3d object detection.
*arXiv preprint arXiv:2504.16740*, 2025.
- Zhang et al. (2023a)
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala.
Adding conditional control to text-to-image diffusion models.
In *Proceedings of the IEEE/CVF international conference on computer vision*, pages 3836–3847, 2023a.
- Zhang et al. (2023b)
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala.
Adding conditional control to text-to-image diffusion models.
In *Proceedings of the IEEE/CVF international conference on computer vision*, pages 3836–3847, 2023b.
- Zhang et al. (2023c)
Peng Zhang, Xin Li, Liang He, and Xin Lin.
3d multiple object tracking on autonomous driving: A literature review.
*arXiv preprint arXiv:2309.15411*, 2023c.
- Zhang et al. (2018)
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang.
The unreasonable effectiveness of deep features as a perceptual metric.
In *Proceedings of the IEEE conference on computer vision and pattern recognition*, pages 586–595, 2018.
- Zhao et al. (2025a)
Guosheng Zhao, Xiaofeng Wang, Zheng Zhu, Xinze Chen, Guan Huang, Xiaoyi Bao, and Xingang Wang.
Drivedreamer-2: Llm-enhanced world models for diverse driving video generation.
In *Proceedings of the AAAI Conference on Artificial Intelligence*, pages 10412–10420, 2025a.
- Zhao et al. (2025b)
Jingyuan Zhao, Yuyan Wu, Rui Deng, Susu Xu, Jinpeng Gao, and Andrew Burke.
A survey of autonomous driving from a deep learning perspective.
*ACM Computing Surveys*, 57(10):1–60, 2025b.

## Appendix A Implementation Details

### A.1 Model and Training Details

DriveObj3D Generation Pipeline.
We adopt Grounded-SAMRen et al. (2024) for image segmentation, Qwen-Image-EditWu et al. (2025) for image generation, and Hunyuan3D 2.0Hunyuan3D et al. (2025) for 3D asset generation.

Inpainting Diffusion Model.
For video synthesis, Dream4Drive is constructed upon a DiT-based architecture. The backbone is initialized with pretrained weights from MagicDriveDiT (Gao et al., 2024a),
which originally conditioned on BEV maps and 3D bounding boxes.
We extend its ControlNet branch by incorporating novel 3D-aware guidance maps,
and fine-tune the model on the nuScenes dataset.
The generated videos have a resolution of $512\times 768$ with 33 frames.

Training is conducted in PyTorch using 8 NVIDIA H200 GPUs with mixed-precision acceleration for 2000 iterations.
We employ the AdamW optimizer with a weight decay of 0.01 and adopt a cosine annealing learning rate schedule
with linear warm-up over the first 10% of steps.
The learning rate is set to $2\times 10^{-4}$, and the batch size per GPU is 1.
During training, we introduce additional Mask Loss and LPIPS Loss to enhance perceptual consistency and local reconstruction quality.

Mask Loss.
In the VAE-decoded space, we compute the mean squared error (MSE) between prediction and ground truth only within the foreground mask region.
Let $\hat{\mathbf{x}}$ and $\mathbf{x}$ denote the predicted and ground-truth decoded images, and $\mathbf{M}$ be a binary mask (1 for constrained pixels, 0 otherwise). The masked MSE is defined as:

$$ $\mathcal{L}_{\text{mask}}=\frac{\left|\mathbf{M}\odot(\hat{\mathbf{x}}-\mathbf{x})\right|_{2}^{2}}{\sum\mathbf{M}+\epsilon},$ (7) $$

where $\odot$ denotes element-wise multiplication, and the denominator normalizes over valid pixels.

LPIPS Loss.
To further improve perceptual quality, we adopt the LPIPS (Learned Perceptual Image Patch Similarity) loss (Zhang et al., 2018), which measures feature-level perceptual differences between prediction and ground truth:

$$ $\mathcal{L}_{\text{LPIPS}}=\text{LPIPS}(\hat{\mathbf{x}},\mathbf{x}),$ (8) $$

where the LPIPS network is pretrained on large-scale perceptual similarity data.

### A.2 Experiment Details

Scene Insertion. We select video frames where no other vehicles appear along the insertion trajectory. For training downstream perception models, inserted scenes are strictly drawn from the nuScenes training split.

Asset Insertion. In the setting of Sec. [5.2](https://arxiv.org/html/2510.19195v2#S5.SS2), we use 420 samples evenly distributed across object categories, which provide the necessary coverage while keeping the synthetic data below 2% of the training set.

Ablation Studies. Experiments on insertion direction, insertion distance, and asset source in Sec. [5.3](https://arxiv.org/html/2510.19195v2#S5.SS3) are conducted under the 1× training epoch and 512×768 resolution setting.

**Table 6: AP comparison across different categories for Real and Ours (+420) at 1× training epoch.**
| Method | car | truck | bus | trailer | construction_vehicle | pedestrian | motorcycle | bicycle | traffic_cone | barrier |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Real | 0.562 | 0.323 | 0.296 | 0.067 | 0.111 | 0.422 | 0.377 | 0.371 | 0.569 | 0.515 |
| Ours | 0.600 | 0.354 | 0.341 | 0.106 | 0.135 | 0.468 | 0.402 | 0.411 | 0.626 | 0.565 |

**Table 7: AP comparison across different categories for Real and Ours (+420) at 2× training epoch.**
| Method | car | truck | bus | trailer | construction_vehicle | pedestrian | motorcycle | bicycle | traffic_cone | barrier |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Real | 0.622 | 0.371 | 0.375 | 0.154 | 0.132 | 0.493 | 0.430 | 0.400 | 0.651 | 0.589 |  |
| Ours | 0.633 | 0.383 | 0.389 | 0.168 | 0.147 | 0.497 | 0.446 | 0.434 | 0.647 | 0.604 |  |

**Table 8: AP comparison across different categories for Real and Ours (+420) at 3× training epoch.**
| Method | car | truck | bus | trailer | construction_vehicle | pedestrian | motorcycle | bicycle | traffic_cone | barrier |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Real | 0.627 | 0.362 | 0.367 | 0.154 | 0.132 | 0.488 | 0.436 | 0.454 | 0.656 | 0.632 |  |
| Ours | 0.639 | 0.377 | 0.408 | 0.190 | 0.164 | 0.501 | 0.446 | 0.439 | 0.654 | 0.621 |  |

## Appendix B Detailed AP Metrics

Tab. [3](https://arxiv.org/html/2510.19195v2#S5.T3) compares detection metrics across different training epochs. Detailed per-class AP scores are provided in Tab. [6](https://arxiv.org/html/2510.19195v2#A1.T6), [7](https://arxiv.org/html/2510.19195v2#A1.T7), and [8](https://arxiv.org/html/2510.19195v2#A1.T8).

## Appendix C Additional Experiments on Generation Quality

**Table 9: Domain gap. Comparison of detection performance on Real and Gen-nuScenes validation sets at 512×768 resolution. Values are reported in percentage.**
| Method | mAP $\uparrow$ | mATE $\downarrow$ | mAOE $\downarrow$ | mAVE $\downarrow$ | NDS $\uparrow$ |
| --- | --- | --- | --- | --- | --- |
| Real | 36.1 | 69.2 | 56.7 | 28.5 | 47.9 |
| Gen-nuScenes | 24.4 | 78.5 | 66.1 | 34.9 | 39.1 (82%) |

**Table 10: Quantitative comparison on video generation quality with other methods. Our method achieves the best FVD and FID score.**
| Method | FPS | Resolution | FVD$\downarrow$ | FID$\downarrow$ |
| --- | --- | --- | --- | --- |
| MagicDrive Gao et al. (2023) | 12Hz | 224$\times$400 | 218.12 | 16.20 |
| Panacea Wen et al. (2024) | 2Hz | 256$\times$512 | 139.00 | 16.96 |
| SubjectDrive Huang et al. (2024a) | 2Hz | 256$\times$512 | 124.00 | 15.98 |
| DriveWM Wang et al. (2024b) | 2Hz | 192$\times$384 | 122.70 | 15.80 |
| Delphi Ma et al. (2024a) | 2Hz | 512$\times$512 | 113.50 | 15.08 |
| MagicDriveDiT Gao et al. (2024a) | 12Hz | 224$\times$400 | 94.84 | 20.91 |
| DiVE Jiang et al. (2024) | 12Hz | 480$\times$854 | 94.60 | - |
| UniScene Li et al. (2024a) | 12Hz | 256$\times$512 | 70.52 | 6.12 |
| CoGen Ji et al. (2025) | 12Hz | 360$\times$640 | 68.43 | 10.15 |
| DriveDream-2 Zhao et al. (2025a) | 12Hz | 512$\times$512 | 55.70 | 11.20 |
| Ours | 12Hz | 512$\times$768 | 31.84 | 5.80 |

**Table 11: Ablation study on 3D-aware guidance maps.**
| Method | FVD$\downarrow$ | FID$\downarrow$ |
| --- | --- | --- |
| No cutout&mask | 66.36 | 14.38 |
| No depth&normal | 34.64 | 7.33 |
| No edge | 49.45 | 8.44 |

Controllability. The controllability of our approach is quantitatively evaluated using perception metrics from StreamPETR Wang et al. (2023a). We first generate the entire nuScenes validation set with Dream4Drive, and then measure perception performance using a pre-trained StreamPETR model. The relative metrics, compared to those obtained on real data, indicate how well the generated samples align with the original annotations. As shown in Tab. [9](https://arxiv.org/html/2510.19195v2#A3.T9), Dream4Drive achieves a relative performance of 82%, demonstrating precise control over foreground object locations.

Generation Quality. As shown in Tab. [10](https://arxiv.org/html/2510.19195v2#A3.T10), our method achieves FVD 31.84 and FID 5.80, outperforming layout- and 3D semantics-guided methods Gao et al. (2023, 2024a); Li et al. (2024a); Ji et al. (2025). The results indicate improved motion consistency and preserved visual fidelity, confirming that our 3D-aware guidance maps effectively generate both foreground and background content. Notably, no post-processing or selective filtering was applied to the generated videos.

Additional Ablation Studies. We perform ablation experiments on the components of the Multi-condition Fusion Adapter, with results presented in Tab. [11](https://arxiv.org/html/2510.19195v2#A3.T11). The findings indicate that all components contribute significantly to overall performance.

## Appendix D Additional Visualization

We present more comparisons between the naive insert and ours in Fig. [9](https://arxiv.org/html/2510.19195v2#A6.F9), [10](https://arxiv.org/html/2510.19195v2#A6.F10), asset insertion videos across different categories in Fig. [11](https://arxiv.org/html/2510.19195v2#A6.F11), [12](https://arxiv.org/html/2510.19195v2#A6.F12), [13](https://arxiv.org/html/2510.19195v2#A6.F13), [14](https://arxiv.org/html/2510.19195v2#A6.F14), [15](https://arxiv.org/html/2510.19195v2#A6.F15), and [16](https://arxiv.org/html/2510.19195v2#A6.F16), where all inserted assets are highlighted with red bounding boxes. In addition, we showcase several corner-case examples, such as imminent collision and close-following scenarios, also illustrated in Fig. [17](https://arxiv.org/html/2510.19195v2#A6.F17) and Fig. [18](https://arxiv.org/html/2510.19195v2#A6.F18).

## Appendix E Limitation and Future Works

Although Dream4Drive is capable of inserting arbitrary assets into diverse scenes, automatically ensuring that the inserted trajectories remain within drivable areas and avoid collisions with pedestrians or other vehicles remains an open challenge. Addressing this issue would enable more flexible generation of diverse corner cases.

## Appendix F Statement on LLM Usage

During the preparation of this manuscript, large language models (LLMs) were used solely for language refinement. All scientific ideas, experiments, analyses, and conclusions are the authors’ original contributions.

Figure: Figure 9: A case study of contrast between reflection and shadow.
Refer to caption: x10.png

Figure: Figure 10: A case study of contrast between reflection and shadow.
Refer to caption: x11.png

Figure: Figure 11: Insertion of a car in the right-side region.
Refer to caption: x12.png

Figure: Figure 12: Insertion of a truck in the left-side region.
Refer to caption: x13.png

Figure: Figure 13: Insertion of a barrier in the left-side region.
Refer to caption: x14.png

Figure: Figure 14: Insertion of a bus in the back-side region.
Refer to caption: x15.png

Figure: Figure 15: Insertion of a traffic cone in the left-side region.
Refer to caption: x16.png

Figure: Figure 16: Insertion of a construction vehicle in the back-side region.
Refer to caption: x17.png

Figure: Figure 17: Corner case. Insertion of a barrier in the front-side region, where the ego vehicle is about to collide with it.
Refer to caption: x18.png

Figure: Figure 18: Corner case. Insertion of a truck in the close-range front-side region.
Refer to caption: x19.png

## Peer Reviews

confidence: 4
contribution: 4
initial_rating: 8
presentation: 4
questions: This work demonstrates the effectiveness of out-of-distribution (OOD) scene data generation for downstream task training, focusing specifically on *scene-level* OOD—namely, inserting vehicles into originally empty scenes. However, the scope of OOD scenarios could be further expanded. For instance, one could explore modifying existing scenes by altering vehicle trajectories or swapping vehicle categories to investigate how such diverse OOD variations impact downstream task performance.
soundness: 4
strengths: - Identifies and analyzes the limitations of current generative methods in downstream tasks, showing that even a small amount (as little as 2%) of high-quality generated data can lead to significant performance gains.
- Introduces DriveObj3D, a large-scale 3D asset dataset encompassing typical object categories in driving scenarios, which supports diverse and 3D-aware video editing applications.
summary: This paper primarily investigates the limitations of existing generation methods in downstream tasks, demonstrating that significant improvements can be achieved using only 2% of generated data. It also introduces DriveObj3D, a large-scale 3D asset dataset covering typical categories in driving scenarios and enabling diverse 3D-aware video editing.
weaknesses: - The method relies on scene insertion and asset insertion that currently lack sufficient automation and are primarily dependent on manual effort.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 2
questions: Please refer to the weaknesses above
soundness: 2
strengths: 1. Dream4Drive introduces a combination of dense 3D-aware guidance maps with generative video editing, enabling realistic, geometry-consistent, multi-view scene synthesis.

2. Demonstrating that fewer than 2% synthetic samples can enhance real data training is a strong and practically relevant result, suggesting high efficiency in data augmentation.

3. The proposed DriveObj3D fills a gap in open 3D asset resources for driving research, supporting reproducibility, and future extension in video-level generative modeling.
summary: The paper presents Dream4Drive, a 3D-aware synthetic data generation framework designed to improve downstream perception tasks such as 3D object detection and tracking in autonomous driving. Dream4Drive employs dense 3D-aware guidance maps to edit real driving videos by rendering high-quality 3D assets, ensuring multi-view geometric consistency and photorealism. A new dataset, DriveObj3D, is introduced to provide diverse 3D vehicle and traffic assets for large-scale, geometry-consistent scene editing. Dream4Drive demonstrates that injecting fewer than 2% synthetic samples consistently improves detection and tracking metrics, outperforming existing augmentation baselines.
weaknesses: 1. The paper does not clarify how inserted 3D assets are constrained to remain physically valid: ensuring no collisions, adherence to drivable areas, and correct orientations, nor how occlusion layers between objects are resolved across depth, normal, and edge maps.

2. There is no qualitative criterion for evaluating 3D asset realism or consistency, raising uncertainty about the reliability of the generated dataset and edited scenes.

3. As shown in Table 3, the method’s performance fluctuates across epochs; in some settings, direct insertion outperforms Dream4Drive, and differences between methods are within statistical noise, suggesting the need for repeated trials with mean–variance reporting.

4. The introduction is verbose, and the method section lists procedural steps without sufficient motivation or justification. Figures contain minor layout inconsistencies.

5. Missing Related Work: SimGen (NeurIPS 2024).

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: Additional questions:

Q1: Given that naive insertion performs within 0.6-0.7 mAP of your full method (Tables 3-4), can you provide a compute or latency cost-benefit analysis? The qualitative improvement in image quality is clear (Figure 8) but the actual metric benefits less so.

Q2: How much manual effort is required to select 420 valid insertion positions and trajectories? Can you quantify the human time required and discuss path to full automation?

Q3: Building off Q2, do the authors have an evidence or intuition as to whether the approach will scale with more augmented scenes using their library? W2 suggests that the increase in training data doesn't necessarily correspond with a proportional increase in performance.
soundness: 3
strengths: - The paper identifies a key shortcoming in most generative simulators for autonomous driving data: improvements in downstream task from training on both real and synthetic data have in the past been attributed only to the addition of synthetic data while ignoring that increasing training epochs improves performance even just using the base dataset (nuScenes).
- The paper provides a large-scale dataset (DriveObj3D) along with a method to insert assets from the dataset in a natural manner to match scene lighting and shadows.
- The results show some improvement over existing methods in downstream detection tasks while being sample efficient (only adding <2% data over the original dataset)
summary: This paper proposes Dream4Drive, a framework for generating synthetic training data for autonomous driving perception tasks. The key insight is that previous work evaluated synthetic data unfairly by pretraining on synthetic data and then finetuning on real data (2x training epochs), obscuring true benefits. Dream4Drive decomposes videos into 3D-aware guidance maps (depth, normal, edge, object image, mask) and uses a fine-tuned diffusion transformer to render edited multi-view videos after inserting 3D assets. The authors contribute DriveObj3D, a dataset of 3D assets across driving scenarios. With only 420 synthetic samples (<2% of training data), they demonstrate improvements in detection and tracking at 1x, 2x, and 3x training epochs.
weaknesses: W1. The paper lacks novelty beyond the Epoch related finding. The actual data generation framework is a composition of existing models and methods rather than a technical contribution. While adding multiview images as conditions to generate a 3D mesh for 3D assets does improve asset quality over meshes generated from single view, it seems like an unsurprising improvement. Moreover, the actual improvement in scores from Table 5 compared to other 3D asset generation methods is relatively small, despite using multiview images vs Hunyuan3D's single view method.

W2. The detection performance results using the method are marginal improvements over previous works. In Tables 1, 2, and 3, the relative improvement in metrics like mAP in the 2x and 3x Epochs are small, even over just using the original nuScenes dataset. In fact, this result shows that even the DriveObj3D falls prey to the core discovery the authors note about normalizing for # of Epochs trained; the improvement over training with only real data diminishes with increasing Epochs. The increase in performance in mAP vs Real data in Table 1 (2x Epochs) is <1% improvement despite including >1% increase in training data. 

W3. Appendix E acknowledges that "automatically ensuring insertions are in drivable areas and avoid collisions remains an open challenge." This is a significant limitation, with manual effort required preventing this method from generating corner cases on a significant scale.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 2
questions: 1. Regarding the `DriveObj3D` creation pipeline: Could you quantify the robustness of this pipeline? What is the approximate failure rate, and what are the common failure modes? How much manual filtering or intervention was required to curate the final high-quality dataset?
2. The scene selection for insertion is described as choosing frames "where no other vehicles appear along the insertion trajectory." How is this selection performed (manually or automatically)? Could this process introduce a bias, for example, by preferentially selecting less cluttered scenes, thereby limiting the complexity of the generated data?
3. The paper highlights improved realism via shadows and reflections. Does the generative model also learn to synthesize more complex physical interactions? For example, does an inserted vehicle generate splashes when moving through a puddle, or create dust on a dirt road? To what extent can the model handle nuanced lighting effects beyond direct shadows, such as colored light from traffic signals reflecting on the vehicle?
soundness: 3
strengths: 1. **Important and Timely Critique:** The central argument regarding the unfair evaluation of synthetic data is a significant contribution. By demonstrating that simply increasing training epochs on real data can match or exceed the performance of hybrid-data training, the paper forces the community to reconsider how the value of synthetic data is measured. This provides a strong and compelling motivation for the proposed work.
2. **Comprehensive and Convincing Experiments:** The experimental validation is thorough and directly supports the paper's claims.
    - The head-to-head comparisons under 1x, 2x, and 3x epoch settings provide clear evidence for the effectiveness of Dream4Drive over baselines.
    - The ablation studies are insightful, systematically analyzing the impact of insertion position, distance, 3D asset source, and different components of the guidance maps. This provides a deeper understanding of what makes the synthetic data effective.
    - The evaluation on both detection and tracking tasks, at multiple resolutions, demonstrates the general applicability of the generated data.
summary: This paper critically re-evaluates the utility of synthetic data generated by driving world models for downstream perception tasks. The authors argue that previous works often rely on an unfair evaluation protocol where models trained on hybrid (real + synthetic) data use twice the training epochs of real-data-only baselines. They demonstrate that when training epochs are matched, the benefits of existing synthetic data augmentation methods become negligible or even negative.

To address this, the paper introduces **Dream4Drive**, a novel framework for generating high-quality, 3D-aware synthetic data. Instead of relying on sparse conditioning signals like BEV maps, Dream4Drive first decomposes a real video into a set of dense, 3D-aware guidance maps (depth, normal, edge, etc.). It then renders 3D assets into these maps and fine-tunes a Diffusion Transformer-based world model to generate photorealistic, multi-view videos that incorporate these new objects. This approach allows for precise, instance-level control and ensures geometric and visual consistency.

Furthermore, the authors contribute **DriveObj3D**, a large-scale 3D asset dataset tailored for driving scenarios, along with an automated pipeline for its creation. Through extensive experiments on detection and tracking tasks, the paper shows that augmenting the training set with a very small fraction (<2%) of data generated by Dream4Drive consistently and significantly improves perception performance, even under fair, epoch-matched comparisons.
weaknesses: 1. **Limited Scope of Generated Corner Cases:** The paper defines "corner cases" primarily as the insertion of new objects into a scene. While this is an important class of long-tail events, it does not cover other critical scenarios, such as adverse weather conditions (heavy rain, snow, fog), unusual lighting (lens flare, low sun), complex multi-agent interactions, or environmental changes (e.g., road construction). The framework's applicability to these other types of corner cases is not explored.
2. **Potential Bottlenecks in the Asset Generation Pipeline:** The `DriveObj3D` pipeline is a cascade of multiple sophisticated models (segmentation, multi-view generation, mesh generation). The final asset quality is contingent on the successful execution of every step. This pipeline may be brittle; for instance, a failure in multi-view generation could lead to an incomplete or distorted 3D mesh. A discussion on the robustness, failure modes, and potential need for manual curation of this pipeline would strengthen the paper.
3. **Limited Technical Contribution.** While the specific application and the system-level design are effective in research application, the work does not introduce a new core generative modeling technique or a new research framework. For some reviewers, the novelty might be perceived as incremental, lying more in the clever combination of existing parts than in foundational innovation.
4. **Computational Cost Analysis is Missing:** The proposed data generation process is multi-staged and involves fine-tuning a large Diffusion Transformer model. This suggests a significant computational cost. While the paper argues for fairness in terms of training *epochs*, a discussion on fairness in terms of total *compute* (cost of data synthesis + cost of training) would provide a more complete picture. For instance, how does the cost of generating 420 synthetic samples compare to the cost of training the perception model for an additional epoch?


Decision: Accept (Poster)