Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformers
Authors: Gangwei Xu
Haotong Lin
Hongcheng Luo
Xianqi Wang
Jingfeng Yao
Lianghui Zhu
Yuechuan Pu
Cheng Chi_
Haiyang Sun
BING WANG
Guang Chen
Hangjun Ye
Sida Peng
Xin Yang
Venue: NeurIPS 2025 poster ()
Keywords: monocular depth estimation; generative depth estimation models; diffusion transformers; relative depth
Primary area: deep_learning
TL;DR: We present Pixel-Perfect Depth, a monocular depth estimation model based on pixel-space diffusion transformers.

Abstract: This paper presents **Pixel-Perfect Depth**, a monocular depth estimation model based on pixel-space diffusion generation that produces high-quality, flying-pixel-free point clouds from estimated depth maps. Current generative depth estimation models fine-tune Stable Diffusion and achieve impressive performance. However, they require a VAE to compress depth maps into the latent space, which inevitably introduces flying pixels at edges and details. Our model addresses this challenge by directly performing diffusion generation in the pixel space, avoiding VAE-induced artifacts. To overcome the high complexity associated with pixel-space generation, we introduce two novel designs: 1) **Semantics-Prompted Diffusion Transformers** (**SP-DiT**), which incorporate semantic representations from vision foundation models into DiT to prompt the diffusion process, thereby preserving global semantic consistency while enhancing fine-grained visual details; and 2) **Cascade DiT Design** that progressively increases the number of tokens to further enhance efficiency and accuracy. Our model achieves the best performance among all published generative models across five benchmarks, and significantly outperforms all other models in edge-aware point cloud evaluation. Project page: https://pixel-perfect-depth.github.io/.

Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformers
Gangwei Xu
Haotong Lin
Hongcheng Luo²
Xianqi Wang'
Jingfeng Yao Bing Wang"
Lianghui Zhu'
Yuechuan Pu²
Cheng Chi
Haiyang Sun
Guang Chen²
Hangjun Ye²
Sida Peng* Xin Yang! +04
HHazhoog University of Science and Technology "Xiaomi EV 'Zhejiang University
http://pxxxxlppeeecctt -eephh github. io
uell
Image
Ours
Marigold
Depth Anything
Depth
Figure I: We present Pixel-Perfect Depth, a monocular depth estimation model with pixel-space diffusion transformers. Compared existing discriminative [82 4 and generative (4 models, its
estimated depth maps can produce high-quality, flying-pixel-free point clouds.
Abstract
This paper presents Pixel-Perfect Depth, monocular depth estimation model based on pixel-space diffusion generation that produces high-quality, flying-pixel- free point clouds from estimated depth maps. Current generative depth estimation models fine-tune Stable Diffusion and achieve impressive performance. However, they require VAE compress depth maps into the latent space, which inevitably introduces flying pixels at edges and details. Our model addresses this challenge by directly performing diffusion generation in the pixel space, avoiding VAE-induced artifacts. To overcome the high complexity associated with pixel-space generation, we introduce two novel designs: 1) Semantics- Prompied Diffusion Transform- ers (SP-DiT), which incorporate semantic representations from vision foundation models into DiT to prompt the diffusion process, thereby preserving global se- mantic consistency while enhancing fine-grained visual details; and 2) Cascade DIT Design that progressively increases the number of tokens further enhance efficiency and accuracy. Our model achieves the best performance among all published generative models across five benchmarks, and significantly outperforms all other models in edge-awe adge-aware point cloud evaluation.
Equal contribution, Project leader, Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Image
Marigold
GT (VAE)
Ours
GT
Figure 2: Qualitative comparisons. GT(VAE) denotes the ground truth depth map after VAE reconstruction. Existing generative models B4] use VAE to compress inputs into the latent space, inevitably introducing, flying pixels at edges and details. In contrast, our model directly performs diffusion in pixel space, avoiding these issues. Depth maps are visualized on the point clouds.
Introduction
Monocular depth estimation (MDE) is fundamental task with wide range of downstream ap- plications, such as 3D reconstruction, novel view synthesis, and robotic manipulation. Due to its significance, a large number of depth estimation models [44 81
82 [8) have emerged recently. These models achieve high-quality results in most zero-shot scenarios or regions, but suffer from flying pixels around object boundaries and fine details when converted into point clouds [39], as shown in Figure and 5S] which limits their practical applications in tasks such as free-viewpoint broadcast, robotic manipulation, and immersive content creation.
Current models suffer from the flying pixels problem due to different reasons. For discriminative models 82 4 88, 33), flying pixels mainly arise from their tendency to output an intermediate deetth-disoontinuuu edges, in order
(average) depth value between the foreground and background
bypass direct regression by modeling pixel-wise depth distributions, allowing them to preserve sharp edges and recover fine structures more faithfully. However, current generative depth models typically fine-tune Stable Diffusion [51] for depth estimation, which requires a Variational Autoencoder (VAE) to compress depth maps into latent space. This compression inevitably leads to the loss of edge sharpness and structural fidelity, resulting in a significant number of flying pixels, as shown in Figure 2
to minimize regression loss. In contrast, generative models 34
17 23
A trivial solution could be training a diffusion- based monocular depth model in pixel space, bypassing the use of a VAE. However, we find this highly challenging, due to the increased complexity and instability of modeling both global semantic consistency and fine- ane-ggrined visual details, leading to extremely low-quality depth predictions (Table and Figure To further investigate this limitation, we examine prior studies on high-resolution image generation. Several works 29 61 94), through signal-to-noise ratio (SNR) analysis, have pointed out that adding noise with higher intensity is more likely to disrupt the global structures or low-frequees frequency components of high-resolution images, thereby improving generation. This reveals that the primary difficulty in high-resolution pixel-space generation lies in effectively perceiving and modeling global image structures.
In this paper, we present Pixel-Perfect Depth, a framework for monocular depth estimation using pixel-space diffusion transformers. Recognizing that the major difficulty in high-resolution pixel-space generation lies in perceiving and modeling global image structures. To address this challenge, we propose the Semantics- Promiitd Diffusion Transformers (SP-DiT) that incorporate high-level semantic representations into the diffusion process to enhance the model's ability to preserve global structures and semantic coherence. Equipped with SP- our model can more effectively preserve global semantic consistency while generating fine-grained visual details in high-resolution pixel space. However, the semantic representations obtained from vision foundation models [44 DIT, leading to training instability and convergence issues. To address this, we introduce a simple yet effective regularization technique for semantic representations, which ensures stable training and facilitates convergence to desirable solutions. As shown in Table and Figure 6 SP- significantly improves overall performance, with up to a 78% gain on the NYUv2 58 AbsRel metric.
and flying-pinn pixel-free
82
65 24 often do not align well with the internal representations of
Furthermore, we introduce the Cascade DřT Design (Cas-DiT), an efficient architecture for diffusion transformers. We find that in diffusion transformers, the early blocks are primarily responsible for capturing and generating global or low- frequeeyy structures, while the later blocks focus

---

have demonstrated impressive results in generating tigh-frequency details. Based on this insight, Cas-DiT adopts a progressive patch size strategy: larger patch size used in the early DiT blocks reduce the number of tokens and facilitate global image structure modeling; in the later DiT blocks, we increase the number of tokens, which is equivalent using a smaller patch size, allowing the model focus on the generation of fine-grained spatial details. This coarse-to-fine cascaded design not only significantly reduces computational costs and improves efficiency, but also delivers substantial improvements in accuracy.
We highlight the main contributions of this paper below:
We present Pixel-Perfect Depth, monocular depth estimation model with pixel-space diffusion generation, capable of producing syiing-pixel-free point clouds from estimated depth maps.
We introduce Demantics-Promplee DiT, which integrates normalized semantic representations into the DiT to effectively preserve global semantic consistency while enhancing fine- ine-grained visual details. This significantly boosts overall performance. We further propose novel Cascade DiT Design to enhance the efficiency and accuracy of our model.
Our model achieves the best performance across five benchmarks among all published generative depth estimation models.
We introduce an edge-aware point cloud evaluation metric, which effectively assesses flying pixels at edges. Our model significantly outperforms previous models in this evaluation.
2 Related Work
2.1 Monocular Depth Estimation
Depth estimation can be broadly categorized into monocular
82
69 stereo 75 72 77
76
22
7 D. and sparse depth completion [41] methods. Early monocular depth estimation methods relied primarily on manually designed features (52
28 The advent of neural networks revo- struggled with cross-dataset generaliza- tion. To address this limitation, scale-invariant and relative loss 49 are introduced, enabling
lutionized the field, though initial approaches
[15
14
multi-dataset 336
86
10
74
71
68
64
73
training. Recent methods focus on im- 677. depth consistency [80 6
50
proving the gencralization ability 21
82
66
31 33 and metric
scale 37 38
88
89
30
of depth estimation. These methods converge towards using
transformer-bes based architectures 48 Concurrent works [69] [69,70, 79] explore point cloud representa-
have attempted to use diffusion models for metric depth estimation. In contrast, our method focuses on relative depth and demonstrates improved generalization and fine- ane-gnained detail across a wide range of real-world scenes. Furthermore, our model significantly differs from these methods by introducing Domantics-Prompler DiT, which incorporates pretrained high-level semantic representations into the diffusion process, greatly enhancing performance.
tions improve depth estimation performance. Several recent methods [32 12
55
53
54
93
More recently, 34 brought the new insight the field by fine-tuning pretrained Stable Diffusion [5] for depth estimation, which demonstrated impressive zero-shot capabilities for relative depth. The following works 23 20
60 92 2 attempt improve its performance and inference speed. However, they are all based on the latent diffusion model [51], which is trained in the latent space and requires a VAE to compress the depth map into a latent space. We focus on a pixel-space diffusion model that is trained directly in the pixel space without requiring any VAE.
2.2 Diffusion Generative Models
Diffusion generative models 25 59 45
90
84
85
96
26 such as DDPM (25 operate directly the pixel space, enabling high-fidelity generation but incurring significant computational costs, especially at high resolutions. To address this limitation, Latent Diffusion Models perform the diffusion process in diwenn-imeniinnal latent space obtained via VAE, popularized by Stable Diffusion [51]. This design significantly improves training and inference efficiency and has been widely adopted in recent works [16. 85| 90
image and video generation. Early approaches 255
27
95
35
47
83
Diffusion models for monocular depth estimation typically follow similar trend. For instance, Marigold (34 and its follow-ups 23 20 fine-tune pretrained Stable Diffusion (51) models for depth
3

---

Vision Foundation Models (DINOv2, Depth Any., VGGT.
Norm
Image
Concat
Add Noise
DiT Blocks
Depth Pred
Semantics Promiied
DiT Blocks
Cascade DiT Design
Used only
during training
Depth
Overview of Pixel-Perfect Depth. Given an input image, we concatenate it with noise and feed it into the proposed Cascade DiT. Meanwhile, the image is also processed by pretrained encoder from Vision Foundation Models to extract high-level semanties, forming our Comaantics-rroppeen DiT. We perform diffusion generation directly in pixel space without using any VAE.
Figure 3:
estimation, benefiting from fast convergence and strong priors learned from large-scale datasets. However, the VAE's latent compression leads to flying pixels in the resulting point clouds. In contrast, pixel-space diffusion avoids such artifacts but remains computationally intensive and slow to converge DiT and Cascade DiT Design,
high resolutions. To address this, propose which enables efficient high-resolution depth estimation without latent compression.
Method
3.1 Pixel-Perfect Depth
Given an input image, our goal is estimate a pixel-perfect depth map that is free of flying pixels when converted to point clouds. Existing models 34 17 23 82 4 often suffer from flying pixels due to their inherent modeling paradigms. Discriminative models tend to smooth object edges and blur fine details because of their mean- mean-prediction bias, which results in noticeable flying pixels in the reconstructed point clouds. Generative models, in theory, can better capture the multi-modal depth distribution at object edges. However, current generative models typically fine-tune Stable Diffusion (5T) for depth estimation, relying on strong image priors. This requires compressing the depth map into latent space via VAE, inevitably causing flying pixels.
To unleash the potential of generative models for depth estimation, we propose Pixel-Perfect Depth that performs diffusion directly in the pixel space instead of the latent space. It allows us to directly model the pixel-wise distribution of depth, such as the discontinuities object edges. However, training generative diffusion model directly in the high-resolution pixel space (e.g., (024x768) computationally demanding and hard optimize. To overcome these challenges, we introduce Cemantics-Prompter DIT and Cascaded DiT Design, detailed in the following sections.
3.2 Generative Formulation
as the generative core our depth estimation framework. Flow Matching learns a continuous transformation from Gaussian noise to data sample via a first-order Ordinary Differential Equation (ODE). our case, we model the transformation from Gaussian noise to a depth sample. Specifically, given clean depth sample Xo ~ 'D and Gaussian noise X1 ~ (0, 1), we define an interpolated sample at continuous time E [o, 1] as:
adopt Flow Matching 42
X = ===-x (1 X0-
(I)

---

Image
Depth Anything v2
Depth Pro
MoGie 2
Ours
Figure 4: Comparison with existing depth foundation models open-world images. Our model preserves more fine-grained details than Depth Anything v2 [82] and MoGe 2 [70], while
demonstrating significantly higher robustness compared to Depth Pro 4
This defines a velocity field:
dx dt
V
X1
Xo-
which describes the direction from clean data to noise. Our model ve(x,, ł, c) is trained to predict the velocity field, based the current noisy sample Xx, the time step and the input image c. The training objective is the mean squared error (MSE) between the predicted and true velocity:
Lvelocity()) = Exn.xi. [Ivooxe, t, - vell?]
(3)
At inference, we start from noise x; and solve the ODE by discretizing the time interval [o, 1] into steps t, iteratively updating the depth sample as follows:
Xt-
Xt, + vo(x,, tes (4+-1 1z),
where t, decreases from 1 to 0, gradually transforming the initial noise x, into the depth sample xo-

---

Image
Ours
Marigold
Depth Anything v2
Depth Pro
Figure 5: Qualitative point cloud results in complex scenes. Our model produces significantly fewer flying pixels compared to other depth estimation models 34 on the point clouds for visualization.
with depth maps overlaid
3.3 Semantics- Prompted Diffusion Transformers
Our Prommtted DiT builds DiT 45 for its simplicity, scalability, and strong performance in generative modeling. Unlike previous depth estimation models such as Depth Anything v2 [82] and Marigold (34), our architecture is purely transformer-based, without any convolutional layers. By integrating high-level semantic representations, SP-DiT enables model to preserve global semantic consistency while enhancing fine-grained visual details, without sacrificing the simplicity and scalability of DiT.
Specifically, given the interpolated noise sample x; and the corresponding image c, we first concate- nate them into a single input: a = B C, where the image e serves as condition. Then, we directly feed a, into the DIT. The first layer of DiT is patchify operation, which converts the spatial input a, into a 1D sequence of T tokens (patches), each with a dimension of D, by linearly embedding each patch of size p x p from the input a, Subsequently, the input tokens are processed by a sequence of Transformer blocks, called DiT blocks. After the final DiT block, each token is linearly projected into a X p tensor, which is then reshaped back to the original spatial resolution to obtain the predicted velocity V, (i.e.,
xo). with channel dimension of 1.
Unfortunately, performing diffusion directly in the pixel space leads poor convergence and highly inaccurate depth predictions. As shown in Figure 6 the model struggles to model both global image structure and fine-grained details. To address this, we extract high-level semantic representations e as guidance from the input image using vision foundation model f, as follows:
(5) where T' and D' are the number of tokens and the embedding dimension of f(c), respectively. These high-level semantic representations are then incorporated into our DiT model, enabling it to more effectively preserve global semantic consistency while enhancing fine-grained visual details. However, we found that the magnitude of the obtained semantics e differs significantly from the magnitude of the tokens in our DiT model, which affects both the stability of the model's training and its performance. To address this, we normalize the semantic representation e along the feature dimension using L2 norm, follows:
f(c) E R TT xD
|ell2
Subsequently, the normalized semantic representation is integrated into the tokens z of our DiT model via multilayer perceptron (MLP) layer hgg
z' = h B(ê)),

---

NYUv2
KITTI ETHED
ScanNet
DIODE
Image
wio SP-DiT
w SP Di
Figure 6: Qualitative ablations for the proposed SP-DiT. Without SP-DiT, the vanilla DiT model struggles with preserving global semantics and generating fine-grained visual details.
where ((-) denotes the bilinear interpolation operator, which aligns the spatial resolution of the semantic representation é with that of the DiT tokens. The resulting z' denotes the DiT tokens enhanced with semantics. After the fusion, the subsequent DiT blocks are prompted by semantics to effectively preserve global semantic consistency while enhancing fine-grained visual details in the high-resolution pixel space. We refer to these subsequent DiT blocks as Semantics- Srmaattcs-Prompted DiT.
In this work, experiment with various pretrained vision foundation models, including DINOv2 [44), VGGT [65], MAE (24), and Depth Anything v2 [2). All them significantly boost performance and facilitate more stable and efficient training, shown in Table 3 Note that we only utilize the encoder of each vision foundation model, e.g., a 24-layer Vision Transformer encoder (ViT-L/14) for
both DINOv2 44 and Depth Anything v2 82
3.4 Cascade DiT Design
Although the proposed Semantics Prooicted- DiT significantly improves accuracy performance, performing diffusion directly in the pixel space remains computationally expensive. To address this issue, We propose a novel Cascaded DiT Design to reduce the computational burden of the model. We observe that in DIT architectures, the early blocks primarily responsible for capturing global image structures and low-frequency information, while the later blocks focus on modeling fine-grained, high-frequency details.
To optimize the efficiency and effectiveness of this process, we adopt large patch size in the early DiT blocks. This design significantly reduces the number of tokens that need to be processed, leading to lower computational cost, Additionally, it encourages the model to prioritize learning and modeling global image structures and low- now-frequency information, which also better aligns with the high-level semantic representations extracted from the input image. In the later DiT blocks, we increase the number of tokens, which is equivalent to using smaller patch size. This allows the model to better focus on fine-grained spatial details. The resulting coarse-to-fine cascaded design mirrors the hierarchical nature of visual perception and improves both the efficiency and accuracy of depth estimation.
Specifically, for our diffusion model with total of N DiT blocks, the first N/2 blocks constitute the coarse stage with larger patch size, while the remaining N/2 blocks (i.e., SP-DiT) form the fine stage using smaller patch size.
3.5 Implementation Details
In this section, we provide essential information about the model architecture details, depth normal- ization, and training details.

---

Table 1: Zero-shot relative depth estimation. Better: AbsRel 81 Bold numbers are the best.
Our model outperforms other generative models on five benchmarks. Ours (512) represents 512 x 512 model, and Ours (1024) represents 1024 768 model.
Training Data
NYUv2
KITTI
ETH3D
ScanNet
DIODE
Type
Method
AbsRely 611
AbsRel 811
AbsRet d1t
AbsRell 811
AbsRelj 811
DiverseDepth
87
320K
11.7
87.5
19.0
70.4
22.8
69.4
10.9
88.2
Discriminative
MiDaS 49
2M
11.1
88.5
23.6
63.0
18.4
75.2
12.1
84.6
LeReS 89
354K
9.0
91.6
14.9
78.4
17.1
77.7
9.1
91.7
Omnidata 13
12M
94.5
14.9
83.5
16.6
77.8
7.5
HD
300K
6.9
94.8
11.5
86.7
12.1
83.3
8.0
93.9
DPT 48
1,2M
9.8
90.3
10.0
90.1
7.8
94.6
8.2
93.4
DepthAny. v2 82
54K
5.4
97.2
8.6
92.8
12.3
88.4
8.8
93.7
DepthAny. v2
62M
4.5
97.9
7.4
94.6
13.1
86.5
6.5
97.2
6.6
95.2
Marigold
74K
5.5
96.4
9.9
91.6
6.5
96.0
6.4
95.1
10.0
90.7
Generative
GeoWizard
280K
96.6
9.7
92.1
6.4
96.1
6.1
12.0
DepthFM
74K
5.5
96.3
8.9
91.3
5.8
96.2
6.3
95.4
GenPercept 78
90K
5.2
96.6
9.4
92.3
6.6
95.7
5.6
96.5
Lotus 23
54K
5.4
96.8
8.5
92.2
5.9
97.0
5.9
95.7
9.8
92.4
Ours (512)
54K
4.3
97.4
8.0
93.1
4.5
97.7
4.5
97.3
7.0
95.5
Ours (1024)
125K
97.7
7.0
95.5
4.3
98.0
4.6
97.2
6.8
Model architecture details. In our implementation, we use total of N = 24 DIT blocks, each operating at hidden dimension of 1024. The first 12 blocks are standard DiT blocks with patch size of 16, corresponding to (H/16) x (W/16) tokens for input of size H W. After the 12th block, we employ an MLP layer to expand the hidden dimension by a factor 4, followed by reshaping to obtain (H/8) (W/8) tokens. The remaining 12 SP-DiT blocks then further process these (H/8) x (W/8) tokens. Finally, we employ an MLP layer followed by a reshaping operation to transform the processed tokens into H W depth map. In contrast to prior monocular depth models, such as Depth Anything and Depth Pro, our model does not rely on any convolutional layers.
Depth normalization. The ground truth depth values are normalized to match the scale expected by the diffusion model. Before normalization, we convert the depth values into log scale to ensure more balanced capacity allocation across both indoor and outdoor scenes. Specifically, we apply the transformation d log(d + e), where d denotes the transformed depth, dl is the original depth value, and a small positive constant (e.g., 1) to ensure numerical stability. We then normalize the log-scaled depth using:
à dmin 0.5,
à dmax dmin
(8)
where dmin and dmaz are the 2% and 98% depth percentiles of each map, respectively.
Training details. We train two variants of the diffusion model at different resolutions: one at 512 x 512 and the other 1024 768. We train all models 8 NVIDIA GPUs with a per-GPU batch size of 4, using the AdamW optimizer with a constant learning rate x 10-4. The training loss is the MSE loss between the predicted and true velocity, shown in Equation and the gradient matching loss, which adopted from [82].
Experiments
4.1 Experimental Setup
Training datasets. Our objective is to estimate pixel-perfect depth maps, which, when converted to point clouds, are free of flying pixels and geometric artifacts. To achieve this, is essential train on datasets with high-quality ground truth point clouds. We adopt Hypersim [50], a p synthetic dataset with accurate and clean geometry, which contains approximately 54K samples, to train the 512 x 512 model. For the 1024 x 768 model, we additionally leverage four datasets, UrbanSyn [19] (7.5K), UnrealStereo4K 62 (8K), VKITTI (25K), and TirtanAir [7 (30K), to
further enhance the model's generalization and robustness.

---

Table Ablation studies on five zero-shot benchmarks. All metrics are presented in percentage terms, bold numbers are the best. Inference time was tested on an RTX 4090 GPU. All results were obtained using the 512 x 512 model.
NYUv2
KITTI
ETH3D
ScanNet
DIODE
Method
Time(s)
AbsRel4
8.1
AbsRell
611
AbsRell
61
AbsRel
611
AbsRel
611
DiT (vanilla)
22.5
72.8
27.3
63.9
12.1
87.4
25.7
65.1
23.9
76.5
0.19
SP-DiT
4.8
96.7
8.6
92.2
4.6
97.5
6.2
94.8
8.2
94.1
0.20
SP-DiT+Cas-DIT
4.3
97.4
8.0
93.1
4.5
97.7
4.5
97.3
7.0
95.5
0.14
Evaluation setup. Following the majority of previous depth estimation models [34
23 we
evaluate the zero-shot relative depth estimation performance on five real-world datasets: NYUv2 [58].
KITTI 18 ETH3D [56]. ScanNet [TT], and DIODE 633, covering both indoor and outdoor scenes. To assess the quality of depth estimation, we adopt two widely-used evaluation metries: Absolute Relative Error (AbsRel) and 61 accuracy. To demonstrate that our model generates point clouds without flying pixels, we convert the estimated depth maps into 3D point clouds and evaluate them using the proposed edge-aware metric. For simplicity, the majority of quantitative evaluations are conducted using the 512 x 512 model. We employ the 1024 x 768 model for the quantitative evaluations in Table T as well as for qualitative comparisons.
4.2 Ablations and Analysis
Component-wise ablation analysis. We adopt the vanilla DiT [45] model as our baseline and conduct ablations performing diffusion generation in high-resolution pixel space is highly challenging due to substantial computational costs and optimization difficulties, leading significant performance degradation. As illustrated in Figure 6 the baseline model struggles with preserving global semantics and generating fine-grained visual details. In contrast, the proposed Semantics-Prone Prompted DiT (SP-DiT) addresses these challenges, achieving significantly improved accuracy, for example, a 78% gain on the NYUv2 AbsRel metric. We further introduce novel Cascaded DIT Design (Cas-DiT) that progressively increases the number of tokens. This coarse-to-fin design not only significantly improves efficiency, for example, reducing inference time by 30% on an RTX 4090 GPU, but also better models global context, leading to noticeable gains in accuracy.
proposed modules. Quantitative results are shown in Table 2 Directly
Ablations vision foundation models (VFMs). We evaluate the performance of SP-DiT using pretrained vision encoders from different VFMs, including MAE [24], DINOv2 [44], Depth Anything v2 and VGGT [65], as illustrated in Table 3
All of them significantly boost performance.
4.3 Zero-Shot Relative Depth Estimation
To evaluate model's zero-shot generalization, we compare it with recent depth estimation mod- els [82] 4 34 23, (00) on five real-world benchmarks. As shown in Table our model outperforms all other generative depth estimation models for all evaluation metries. Unlike previous generative models, we do not rely on image priors from pretrained Stable Diffusion [51] model. Instead, our diffusion model is trained from scratch and still achieves superior performance. Our model generalizes well to wide range of real-world scenes, even when trained solely on synthetic depth datasets. Visual comparisons are shown in Figure 4 our model (1024) preserves more fine-ge ane-grained details than Depth Anything v2 [82] and MoGe 2 (70). Moreover, it demonstrates significantly higher robustness than Depth Pro 4. especially in challenging regions with complex textures, cluttered backgrounds, or large sky areas.
4.4 Edge-Aware Point Cloud Evaluation
Our objective is to estimate pixel-perfect depth maps that yield clean point clouds without flying pixels, which often occur object edges due to inaccurate depth predictions these regions. However, existing evaluation benchmarks and metrics often struggle to reflect flying pixels at object edges. For

---

Table 3: Ablation studies on Vision Foundation Models (VEMs). Note that we only utilize pretrained encoder from these VFMs, such as 24-layer VIT from DINOv2 or Depth Anything v2.
NYUv2
KITTI
ETH3D
ScanNet
DIODE
VFM Type
AbsRell
511
AbsRell
t.1
AbsRel
d1t
AbsRell
81
AbsRell
d11
w/o SP-DiT
22.5
72.8
27.3
63.9
12.1
87.4
25.7
65.1
23.9
76.5
SP-DiT (MAE 24
6.4
95.0
14.4
84.9
7.3
94.8
7.7
92.5
11.6
91.3
SP-DIT (DINOv2 44
4.8
96.4
9.3
91.2
5.6
96.2
5.1
96.9
9.2
93.5
SP-DIT (VGGT (55
4.7
96.7
7.6
94.1
4.1
97.8
3,8
98.0
7,8
94.9
SP-DiT (DepthAny. v2
82
4.3
97.4
8.0
93.1
4.5
97.7
4.5
97.3
7.0
95.5
Table 4: Edge-aware point cloud evaluation. Our model achieves the best performance on the high-quality Hypersim test set. To further verify that VAE compression leads to flying pixels, we evaluate the ground truth depth maps after VAE reconstruction, denoted as GT(VAE).
Marigold 34)
GeoWizard DepthAny. v2 82
DepthPro 4 GT(VAE) Ours
Chamfer Dist.
0.17
0.16
0.18
0.14
0.12
0.08
example, benchmarks like NYUv2 or KITTI usually lack edge annotations, while metrics such as AbsRel and 6, are dominated by flat regions, making difficult to assess depth accuracy at edges.
To address these limitations, we evaluate on the official test split of the Hypersim [50] dataset, which provides high-quality ground-truth point clouds and is not used during training. We further propose an edge-aware point cloud metric that quantifies depth accuracy at edges. Specifically, we extract edge masks from ground-truth depth maps using the Canny operator and compute the Chamfer Distance between predicted and ground-truth point clouds near these edges.
Quantitative results in Table 4 show that method achieves the best performance. Diseriminative models like Depth Pro [ and Depth Anything v2 [22 tend to smooth edges, causing flying pixels. Generative models such Marigold (34] rely VAE compression, which blurs edges and details, causing artifacts in the reconstructed point clouds. To illustrate this, we encode and decode the ground-truth depth using VAE (GT(VAE)) without any generative process. Table 4 and Figure show that VAE compression introduces flying pixels, leading to a larger Chamfer Distance than ours.
5 Conclusion
We presented Pixel-Perfect Depth, a monocular depth estimation model that leverages pixel-space diffusion transformers to produce high-quality, flying-pixel-free point clouds. Unlike prior generative depth models that rely on latent-space diffusion with VAF, our model performs diffusion directly in the pixel space, avoiding flying pixels caused by VAE compression. To tackle the complexity and optimization challenges of pixel-space diffusion, we introduce Comnantics-Prmppeer DiT and Cascade DiT Design, which greatly boost performance. Our model significantly outperforms prior models in edge-aware point cloud evaluation.
Limitations and future work. This work has two known limitations. First, like most image- based diffusion models, it lacks temporal consistency when applied to video frames, resulting in a little flickering depth across frames. Second, its multi-step diffusion process leads to slower inference compared to discriminative models like Depth Anything v2. Future works can address these limitations by exploring video depth estimation methods [57 31 80 63 to improve temporal consistency and adopting DiT acceleration strategies to speed up inference.
Acknowledgemments. This research is supported by the National Key R&D Program of China (2024YFE0217700), the National Natural Science Foundation of China (623B2036, 62472184), the Fundamental Research Funds for the Central Universities, and the Innovation Project Optics Valley Laboratory (Grant No. 2002005570055.
10

---

References
[1] Michael S Albergo and Eric Vanden- Building normalizing flows with stochastic interpolants. arXiv preprint arXiv:2209.15571, 2022.
[2] Yunpeng Bai and Qixing Huang. Fiffdepth: generators for detailed depth estimation. arXiv preprint arXiv:2412.00671, 2024.
transformation of diffusion-baxye
[3] Shariq Farooq Bhat, Reiner Birkl, Diana Wofk, Peter Wonka, and Matthias Müller. Zoedepth: Zero-shot transfer by combining relative and metric depth. arXiv, 2023.
[4] Aleksei Bochkovskii, AmaÂGI Delaunoy, Hugo Germain, Marcel Santos, Yichao Zhou, Stephan R Richter, and Vladlen Koltun. Depth pro: Sharp monocular metric depth in less than second. arXiv preprint arXiv:2410.02072 2024.
[5] Yohann Cabon, Naila Murray, and Martin Humenberger. Virtual kitti 2. arXiv preprint arXiv:2001.10773, 2020.
[6] Sili Chen, Hengkai Guo, Shengnan Zhu, Feihu Zhang, Zilong Huang, Jiashi Feng, and Bingyi Kang. Video depth anything: Consistent depth estimation for super-long videos. arXiv preprint arXiv:2501. /235, 2025.
[7] Junda Cheng, Wenjing Liao, Zhipeng Cai, Longliang Liu, Gangwei Xu, Xianqi Wang. Yuzhou Wang, Zikang Yuan, Yong Deng. Jinliang Zang, Yangyang Shi, Jinhui Tang, and Xin Yang. Unified stereo matching, multi-view stereo, and real-time stereo with monodepth priors. arXiv preprini arXiv:2501 00640,, 2025.
Monster++:
[8] Junda Cheng, Longliang Liu, Gangwei Xu, Xianqi Wang, Zhaoxing Zhang, Yong Deng, Jinliang Zang, Yurui Chen, Zhipeng Cai, and Xin Yang. Monster: Marry monodepth to stereo unleashes power. CVPR, 2025.
[9] Junda Cheng, Wei Yin, Kaixuan Wang, Xiaozhi Chen, Shijie Wang, and Xin Yang. Adaptive fusion of single-view and multi-view depth for autonomous driving. In CVPR, pages 10138- 10147, 2024.
[10] Jachoon Cho, Dongbo Min, Youngjung Kim, and Kwanghoon Sohn. Diml/cvl rgb-d dataset: 2m rgb-d images of natural indoor and outdoor scenes. arXiv, 2021.
[11] Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-amotated 3d reconstructions of indoor scenes. In CVPR, pages 2017.
[12] Yiquan Duan, Xianda Guo, and Zheng Zhu. Diffusiondepth: Diffusion denoising approach for monocular depth estimation. In ECCV, pages 432 449. Springer, 2024.
[13] Ainaz Eftekhar, Alexander Sax, Jitendra Malik, and Amir Zamir. Omnidata: scalable pipeline for making multi-task mid-level vision datasets from 3d scans. In ICCV, pages 10786-10796, 2021.
[14] David Eigen and Rob Fergus. Predicting depth, surface normals and semantic labels with common multi-scale convolutional architecture. In ICCV, pages 2650-2658, 2015.
[15] David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from single image using a multi-scale deep network. NeurlPS, 27, 2014.
[16] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transform- ers for high-resolution image synthesis. In ICML, 2024.
[17] Xiao Fu, Wei Yin, Mu Hu, Kaixuan Wang, Yuexin Ma, Ping Tan, Shaojie Shen, Dahua Lin, and Xiaoxiao Long. Geowizard: Unleashing the diffusion priors for 3d geometry estimation from single image. In ECCV, pages 241-258. Springer, 2025.
[18] Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In CVPR. pages 3354-3361. IEEE, 2012.
11

---

[19] Jose L Gómez, Manuel Silva, Antonio Scoane, Agnès Borrás, Mario Noriega, Germán Ros, Jose A Iglesias- Iglesias-Guitian, and Antonio M López. All for one, and one for all: Urbansyn dataset, the third musketeer of synthetic driving scenes. Neurocomputingg 637: 137:130038, 2025.
[20] Ming Gui, Johannes Schusterbauer, Ulrich Prestel, Pingchuan Ma, Dmytro Kotovenko, Olga Grebenkova, Stefan Andreas Baumann, Vincent Tao Hu, and Björn Ommer. Depthfm: Fast generative monocular depth estimation with flow matching. In AAAI, volume 39, pages 3203- 3211, 2025.
[21] Vitor Guizilini, Igor Vasiljevic, Dian Chen, Rares Ambruş, and Adrien Gaidon. Towards zero-shot scale-aware monocular depth estimation. In ICCV, pages 9233-9243, 2023.
[22] Xianda Guo, Chenming Zhang, Juntao Lu, Yiqi Wang, Yiqun Duan, Tian Yang, Zheng Zhu, and Long Chen. Openstereo: A comprehensive benchmark for stereo matching and strong baseline. arXiv preprint arXiv:2312.00344. 2023.
[23] Jing He, Haodong Li, Wei Yin, Yixun Liang, Leheng Li, Kaiqiang Zhou, Hongbo Liu, Bingbing Liu, and Bing-Cong Chen. Lotus: Diffusion- based visual foundation model for high-quality dense prediction. arXiv, 2024.
[24] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In CVPR, pages 16000-16009, 2022.
[25] Jonathan Ho. Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. NIPS, 33:6840-6855 2020.
[26] Jonathan Ho, Chitwan Saharia, William Chan, David Fleet, Mohammad Norouzi, and Tim Salimans. Cascaded diffusion models for high fidelity image generation. JMLR, 23(4)):1-33, 2022.
[27] Jonathan Ho and Tim Salimans. arXiv:2207.12598. 2022.
Classifier-free diffusion guidance.
arXiv preprint
[28] Derek Hoiem, Alexei A Efros, and Martial Hebert. Recovering surface layout from an image. IJCV, 75:151-172, 2007.
[29] Emiel Hoogeboom, Jonathan Heek, and Tim Salimans. simple diffusion: End-to-end diffusion for high resolution images. In ICML, pages 13213-13232. PMLR, 2023.
[30] Mu Hu, Wei Yin, Chi Zhang, Zhipeng Cai, Xiaoxiao Long, Hao Chen, Kaixuan Wang, Gang Yu, Chunhua Shen, and Shaojie Shen. Metric3d v2: A versatile monocular geometric foundation model for shro-shot metric depth and surface normal estimation. TPAMI, 2024.
[31] Wenbo Hu, Xiangjun Gao, Xiaoyu Li, Sijie Zhao, Xiaodong Cun, Yong Zhang, Long Quan, and Ying Shan. Depthcrafter: Generating consistent long depth sequences for open-world videos. arXiv preprint 240.002409.00005, 2024.
[32] Yuanfeng Ji, Zhe Chen, Enze Xie, Lanqing Hong, Xihui Liu, Zhaoqiang Liu, Tong Lu, Zhenguo Li, and Ping Luo. Ddp: Diffusion model for dense visual prediction. ICCV, pages 21741- 21752, 2023.
[33] Bingxin Ke, Dominik Narnhofer, Shengyu Huang, Lei Ke, Torben Peters, Katerina Fragkiadaki, Anton Obukhov, and Konrad Schindler. Video depth without video models. arXiv preprint arXiv:2411.19189, 2024.
[34] Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Metzger, Rodrigo Caye Daudt, and Kon- rad Schindler. Repurposing diffusion- image generators for monocular depth estimation. 2024.
In CVPR, pages
[35] Black Forest Labs.
2024.
[36] Zhengqi Li and Noah Snavely. MegaDepth: Learning single-view depth prediction from internet photos. In CVPR, 2018
12

---

[37] Zhenyu Li, Shariq Farooq Bhat, and Peter Wonka. Patchfusion: An end-to-end tile- based framework for high-resolution monocular metric depth estimation. In CVPR, pages 10016- 10025, 2024.
[38] Zhenyu Li, Shariq Farooq Bhat, and Peter Wonka. Patchrefiner: Leveraging synthetic data for real-domain high-resolution monocular metric depth estimation. In ECCV, pages 250-267. Springer, 2024.
[39] Dingkang Liang, Tianrui Feng, Xin Zhou, Yumeng Zhang, Zhikang Zou, and Xiang Bai. Parameter-efficient fine-tuning in spectral domain for point cloud learning. TPAMI, 2025.
[40] Dingkang Liang. Wei Hua, Chunsheng Shi, Zhikang Zou, Xiaoqing Ye, and Xiang Bai. Sood++: Leveraging unlabeled data to boost oriented object detection. TPAMI, 2025.
[41] Haotong Lin, Sida Peng, Jingxiao Chen, Songyou Peng, Jiaming Sun, Minghuan Liu, Hujum Bao, Jiashi Feng, Xiaowei Zhou, and Bingyi Kang. Prompting depth anything for 4k resolution accurate metric depth estimation. In CVPR, pages 17070-17080, 2025.
[42] Yaron Lipman, Ricky TQ Chen, Heli Ben- Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:221000274,, 2022.
[43] Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.0300) 2022.
[44] Maxime Oquab, Timothée Dareet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07//99 2023.
[45] William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4195-4205, 2023.
[46] Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Mattia Segu, Siyuan Li, Lue Van Gool, and Fisher Yu. UniDepth: Universal monocular metric depth estimation. In CVPR, 2024.
[47] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint 2rXivv2307.0//92, 2023.
[48] René Ranfil, Alexey Bochkovskiy, and Vladlen Koltun. Vision transformers for dense prediction. In ICCV, pages 12179-12188 2021.
[49] René Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. TPAMI, 2020.
[50] Mike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar, Miguel Angel Bautista, Nathan Paczan, Russ Webb, and Joshua M. Susskind. Hypersim: A photorealistic synthetic dataset for holistic indoor scene understanding. In ICCV, 2021.
[51] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In CVPR, pages 10684-10695, 2022.
[52] Ashutosh Saxena, Min Sun, and Andrew Y Ng. Make3d: Learning 3d scene structure from single still image. TPAMI, (155)222--400 2008
[53] Saurabh Saxena, Charles Herrmann, Junhwa Hur, Abhishek Kar, Mohammad Norouzi, Deqing Sun, and David J Fleet. The surprising effectiveness of diffusion models for optical flow and monocular depth estimation. NIPS, 36:39443-39469, 2023.
[54] Saurabh Saxena, Junhwa Hur, Charles Herrmann, Deqing Sun, and David Fleet. Zero-shot metric depth with vield-of-view conditioned diffusion model. arXiv preprint arX1v:2312.13252. 2023.
[55] Saurabh Saxena, Abhishek Kar, Mohammad Norouzi, and David J Fleet. Monocular depth estimation using diffusion models. arXiv preprint 2302..2200..481166 2023.
13

---

[56] Thomas Schops, Johannes L Schonberger, Silvano Galliani, Torsten Sattler, Konrad Schindler, Mare Pollefeys, and Andreas Geiger. multi-view stereo benchmark with high-resolution images and multi-camera videos. In CVPR, pages 2260-3269) 2017.
[57] Jiahao Shao, Yuanbo Yang. Hongyu Zhou, Youmin Zhang, Yujun Shen, Vitor Guizilini, Yue Wang, Matteo Poggi, and Yiyi Liao. Learning temporally consistent video depth from video diffusion priors. arXiv preprint arXivv2406.01
2024.
[58] Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. In ECCV, pages 746-760. Springer, 2012.
[59] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020.
[60] Ziyang Song, Zerong Wang, Bo Li, Hao Zhang, Ruijie Zhu, Li Liu, Peng- Teng-Tao Jiang, and Tianzhu Zhang. Depthmaster: Taming diffusion models for monocular depth estimation. arXiv preprint arXiv:2501.02576 2025.
[61] Jiayan Teng, Wendi Zheng, Ming Ding, Wenyi Hong, Jianqiao Wangni, Zhuoyi Yang. and Jie Tang. Relay diffusion: Unifying diffusion process across resolutions for image synthesis. arXiv preprins arXiv: 2023.
[62] Fabio Tosi, Yiyi Liao, Carolin Schmitt, and Andreas Geiger. Smd-nets: Sterco mixture density networks. In CVPR, pages 8942-8952, 2021.
[63] Igor Vasiljevic, Nick Kolkin, Shanyi Zhang, Ruotian Luo, Haochen Wang, Falcon Z Dai, Andrea Daniele, Mohammadreza Mostajabi, Steven Basart, Matthew R Walter, et al. Diode:
A dense indoor and outdoor depth dataset. arXiv preprint arXiv: 1rXiv:108.0044 2019.
[64] Chaoyang Wang, Simon Lucey, Federico Perazzi, and Oliver Wang. Web stereo video supervi- sion for depth prediction from dynamic scenes. In 3DV, 2019.
[65] Jianyuan Wang. Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. arXiv preprint arXiv:2.503. //61,, 2025.
[66] Jiyuan Wang, Chunyu Lin, Cheng Guan, Lang Nie. Jing He. Haodong Li, Kang Liao, and Yao Zhao. Jasmine: Harnessing diffusion prior for self self-supervised depth estimation. arXiv preprint arXiv:2503.15905, 2025.
[67] Yuan Wang, Chunyu Lin, Lei Sun, Rongying Liu, Lang Nie, Mingxing Li, Kang Liao, Xiangxiang Chu, and Yao Zhao. From editor to dense geometry estimator. arXiv preprint arXiv:2509.04338, 2025.
[68] Qiang Wang, Shizhen Zheng, Qingsong Yan, Fei Deng, Kaiyong Zhao, and Xiaowen Chu. Irs: A large naturalistic indoor robotics stereo dataset to train deep models for disparity and surface normal estimation. In ICME, 2021.
[69] Ruicheng Wang. Sicheng Xu, Cassie Dai, Jianfeng Xiang, Yu Deng, Xin Tong, and Jiaolong Yang. Moge: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision. In CVPR, pages 5261-5271, 2025.
[70] Ruicheng Wang, Sicheng Xu, Yue Dong, Yu Deng, Jianfeng Xiang, Zelong Lv, Guangzhong Sun, Xin Tong, and Jiaolong Yang. Moge-2: Accurate monocular geometry with metric scale and sharp details. arXiv preprint Miiv:2507.0254:: 2025.
[71] Wenshan Wang, Delong Zhu, Xiangwei Wang, Yaoyu Hu, Yuheng Qiu, Chen Wang, Yafei Hu, Ashish Kapoor, and Sebastian Scherer. Tartanair: A dataset to push the limits of visual slam. In IROS, 2020.
[72] Xianqi Wang, Gangwei Xu, Hao Jia, and Xin Yang. Selective-stereo: Adaptive frequency information selection for stereo matching. In CVPR, pages 19701-19710, 2024.
[73] Ke Xian, Chunhua Shen, Zhiguo Cao, Hao Lu, Yang Xiao, Ruibo Li, and Zhenbo Luo. Monoc- ular relative depth perception with web stereo data supervision. In CVPR, 2018.
14

---

[74] Ke Xian, Jianming Zhang, Oliver Wang, Long Mai, Zhe Lin, and Zhiguo Cao. ceuuttrre-uiided ranking loss for single image depth prediction. In CVPR, 2020.
[75] Gangwei Xu, Xianqi Wang. Xiaohuan Ding, and Xin Yang. Iterative geometry encoding volume for stereo matching. In CVPR, pages 21919-200 21928, 2023.
[76] Gangwei Xu, Xianqi Wang, Zhaoxing Zhang, Junda Cheng, Chunyuan Liao, and Xin Yang. Igev++: Iterative multi-range geometry encoding volumes for stereo matching. TPAMI, 2025.
[77] Gangwei Xu, Yun Wang. Junda Cheng, Jinhui Tang, and Xin Yang. Accurate and efficient stereo matching via attention concatenation volume. TPAMI, 2023.
[78] Guangkai Xu, Yongtao Ge, Mingyu Liu, Chengxiang Fan, Kangyang Xie, Zhiyue Zhao, Hao Chen, and Chunhua Shen. What matters when repurposing diffusion models for general dense perception tasks? arXiv preprint M.Xi::24433060000 2024.
[79] Tian-Xing Xu, Xiangjun Gao, Wenbo Hu, Xiaoyu Li, Song-Hai Zhang, and Ying Shan. Geome- trycrafter: Consistent geometry estimation for open-world world videos with diffusion priors. arXiv preprint arXiv:2504.01016, 2025.
[80] Honghui Yang, Di Huang, Wei Yin, Chunhua Shen, Haifeng Liu, Xiaofei He, Binbin Lin, Wanli Ouyang, and Tong He. Depth any video with scalable synthetic data, arXiv, 2024.
[81] Lihe Yang, Bingyi Kang, Zilong Huang. Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In CVPR, pages 10371- 1- 10381, 2024.
[82] Lihe Yang. Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. NIPS, 219111
[83] Zhuoyi Yang. Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang. Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint 2xXi:22408.06072, 2024.
[84] Jingfeng Yao, Cheng Wang, Wenyu Liu, and Xinggang Wang. Fasterdit: Towards faster diffusion transformers training without architecture modification. NIPS, 37:56166-56188 2024.
[85] Jingfeng Yao, Bin Yang. and Xinggang Wang. Reconstruction vs. generation: Taming optimiza- tion dilemma in latent diffusion models. arXiv preprini arXiv:
2025.
[86] Yao Yao, Zixin Luo, Shiwei Li, Jingyang Zhang, Yufan Ren, Lei Zhou, Tian Fang, and Long Quan. Blendedmvs: A large-scale dataset for generalized multi-view stereo networks. In CVPR, 2020.
[87] Wei Yin, Xinlong Wang, Chunhua Shen, Yifan Liu, Zhi Tian, Songcen Xu, Changming Sun, and Dou Renyin. Diversedepth: Affine-invarintt depth prediction using diverse data. arXiv preprint arXiv:2002.00566 2020.
[88] Wei Yin, Chi Zhang, Hao Chen, Zhipeng Cai, Gang Yu, Kaixuan Wang. Xiaozhi Chen, and Chunhua Shen. Metric3d: Towards zero-shot metric 3d prediction from a single image. In CVPR, pages 9043-9053, 2023.
[89] Wei Yin, Jianming Zhang, Oliver Wang, Simon Niklaus, Long Mai, Simon Chen, and Chunhua Shen. Learning to recover 3d scene shape from a single image. CVPR, pages 204-213, 2021.
[90] Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is easier than you think. arXiv preprint wxXiv:2410.0694/ 2024.
[91] Chi Zhang, Wei Yin, Billzb Wang, Gang Yu, Bin Fu, and Chunhua Shen. Hierarchical normal- ization for robust monocular depth estimation. NIPS, 35:14128-14139, 2022.
[92] Xiang Zhang, Bingxin Ke, Hayko Riemenschneider, Nando Metzger, Anton Obukhov, Markus Gross, Konrad Schindler, and Christopher Schroers. Betterdepth: Plug-and-play diffusion refiner for shot monocular depth estimation. arXiv preprint arXiv:2407.17952. 2024.
15

---

[93] Wenliang Zhao, Yongming Rao, Zuyan Liu, Benlin Liu, Jie Zhou, and Jiwen Lu. Unleashing text-to-image diffusion models for visual perception. In ICCV, pages 5729-5739, 2023.
[94] Wendi Zheng, Jiayan Teng, Zhuoyi Yang, Weihan Wang, Jidong Chen, Xiaotao Gu, Yuxiao Dong, Ming Ding, and Jie Tang. Cogview3: Finer and faster text-to-image generation via relay diffusion. In ECCV, pages 1-22. Springer, 2024.
[95] Lianghui Zhu, Zilong Huang, Bencheng Liao, Jun Hao Liew, Hanshu Yan, Jiashi Feng, and Xinggang Wang. Dig: Scalable and efficient diffusion models with gated linear attention. arXiv preprint 2rX1v:2405.18428, 2024.
[96] Lianghui Zhu, Zilong Huang, Bencheng Liao, Jun Hao Liew, Hanshu Yan, Jiashi Feng, and Xinggang Wang. Dig: Scalable and efficient diffusion models with gated linear attention. In CVPR, pages 7664-7674, 2025.
16

---

Image
MoGe
Figure 7: Qualitative comparisons with MoGe [69]. Top: input images taken from four test
sets: Hypersim [50), DIODE [63). ScanNet [I]]. and ETH3D [56]. Middle: results of MoGe [69]
Bottom: our results. As a discriminative model, MoGe [69], like other discriminative models [82
also suffers flying pixels at edges and details.
Table 5: Quantitative comparisons with REPA 190.. Our model significantly outperforms
REPA 90]. To ensure fair comparison, the pretrained vision encoder used in both DiT+REPA and DiT+Ours kept the same.
NYUv2
KITTI
ETH3D
ScanNet
DIODE
Method
AbsRel.
61
AbsRel4
611
AbsRel
611
AbsRell
61
AbsRel.
611
DiT (vanilla)
22.5
72.8
27.3
63.9
12.1
87.4
25.7
65.1
23.9
76.5
DIT+REPA 900
17.6
78.0
23.4
70.6
9.1
91.2
20.1
74.3
14.6
86.9
DiT+Ours
4.3
97.4
8.0
93.1
4.5
97.7
4.5
97.3
7.0
95.5
A Qualitative Comparisons with MoGe
We provide qualitative comparisons of reconstructed point clouds, shown in Figure 7 MoGe 669 as a discriminative model, suffers from flying pixels at edges and fine structures, a common issue observed in other discriminative models [82 4 Our model produces significantly fewer flying pixels compared to MoGe 669.
B Additional Discussion with REPA
We provide an additional discussion on the recent image generation method REPA 90). REPA 990
aligns intermediate tokens in diffusion models with pretrained vision encoder, significantly improving training efficiency and generation quality for image generation tasks. We compare our method with REPA 190), and the quantitative evaluation results presented in Table DiT+REPA refers training the DiT model with REPA'S representation alignment, while DiT+Ours denotes training Semantics-Promptee the DiT model using our Semantics Prompted DIT. For fair comparison, the pretrained vision encoder used in both DiT+REPA and DiT+Ours is kept the same. Experimental results show that our Cemantics-Prompted DiT significantly outperforms REPA [90]. We attribute our model's superiority over REPA to two factors. First, during training. REPA's implicit alignment of DIT tokens with the pretrained vision encoder is suboptimal, making difficult for DiT effectively leverage semantic prompts from the pretrained vision encoder. In contrast, our Semantics -rountt DiT directly integrates semantic cues, resulting more effective prompts. Second, at inference, REPA cannot leverage the pretrained vision encoder to provide semantic prompts, whereas our method effectively incorporates high-level semantics into the Semantics Semantics-Prompted DiT during inference to prompt the diffusion process.
17

---

Table Runtime comparison on RTX 4090 GPU. The runtime is measured using the 512 512 model with denoising steps.
Depth Anything v2
82
DepthPro
4
PPD-Large
PPD-Small
Time (ms)
18
170
140
40
Table Quantitative comparisons between PPD-Large and PPD-Small.
NYUv2
KITTI
ETH3D
ScanNet
DIODE
Method
AbsRell.
61
AbsRell
61
AbsRel.
61
AbsRel)
6.t
AbsRel..
ő1
PPD-Small PPD-Large
4.5
97.3
8.3
92.8
4.6
97.4
4.7
97.2
7.3
95.3
4.3
97.4
8.0
93.1
4.5
97.7
4.5
97.3
7.0
95.5
C Analysis of Flying Pixels in Different Types of VAEs
To better understand the emergence of flying pixels in VAE-based reconstructions, we analyze VAEs with different latent dimensions (i.e., channel) by using them to reconstruct ground truth depth maps. Figure shows that both VAE variants exhibit flying pixels at object edges and details, revealing common weakness VAE reconstructions preserving precise geometric structures. VAE-d4 (SD2) denotes the reconstruction of ground truth depth maps using the VAE from Stable Diffusion 2, with latent dimension of 4, which is also used in Marigold [34]. VAE-d16 (SD3.5) uses the VAE from Stable Diffusion 3.5, which has latent dimension of 16.
D Efficiency and Lightweight Variant
Our Pixel-Perfect Depth (PPD) model is slower than Depth Anything v2 182 owing to the multi- step diffusion process, but its inference time remains comparable to Depth Pro 4 as shown in Table To further accelerate inference, we develop a lightweight variant, PPD-Small, which achieves substantially faster runtime with only marginal accuracy loss, shown in Table 7 PPD-Large, PPD-Small is built upon DiT-Smal with a reduced number parameters, making more suitable for efficient inference.
In contrast to
Image
VAE-d4 (SD2)
VAE-d16 (SD3.5)
Ours
Figure 8: Validation of flying pixels in different types of VAEs. We present further qualitative comparisons showing that increasing the latent dimension in VAEs fails to eliminate flying pixels. VAE-d4 (SD2) denotes the reconstruction of ground truth depth maps using the VAE from Stable Diffusion with latent dimension 4, which also used in Marigold. VAE-d16 (SD3.5) uses the VAE from Stable Diffusion 3.5, which has latent dimension of 16.
18

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly present the contributions of the paper, which are subsequently supported by experimental results in the main body.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of model the Conclusion section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sell-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't It acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
19

---

Justification:
The paper does not contain formal theoretical results or proofs. The focus is on the design, implementation, and empirical evaluation the proposed method.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof's provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The paper provides sufficient implementation details to reproduce the main
experimental results, including model architecture, training settings, datasets, evaluation metrics, and ablation studies. We also plan to release the code and models to further facilitate reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
20

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
The datasets used in our experiments are publicly available, ensuring accessi- bility of the data. Although the code is not provided submission time, we plan release the code, models, and detailed instructions to facilitate full reproducibility.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://niis. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed run to reproduce the results. See the NeurIPS code and data submission guidelines (https www.eeeeeeooeemmmeeceeee
/ //wpps.ccppusiie.e wwtpcccppppccccceccoec
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: The paper provides comprehensive details on the experimental setup, including dataset splits, hyperparameter settings and optimizer type.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: We report quantitative metrics for the depth estimation task and provide qualitative visualizations of the depth map and point clouds to demonstrate the effectiveness our method. However, the current experiments do not include error bars or formal statistical significance analysis.
Guidelines:
The answer NA means that the paper does not include experiments.
21

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eπror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We report the number of GPUs used during training. We provide the inference time measured on an RTX 4090 GPU.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental rums well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: We have reviewed the NeurlPS Code of Ethics and confirm our research fully complies with its principles. No ethical issues arise.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [NA]
22

---

This work primarily focuses on fundamental algorithm/moder improvements and technical methods. There are currently no direct or significant societal impacts associated with the research; therefore, no discussion on positive or negative societal impacts is included.
Justification:
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative socictal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymeett of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper does not involve the release of models or datasets that pose high risk of misuse. Therefore, no specific safeguards are necessary.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We use publicly available datasets and pretrained models, all of which are properly eited in the paper.
Guidelines:
23

---

The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperssiipps thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Although the assets (code and model weights) are not released at submission time, we plan to release them upon acceptance. The released assets will properly documented, including usage instructions, license information, and known limitations. No new datasets are introduced this work.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details the dataset/cod//model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification: This paper does not involve crowdsourcing experiments or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects

---

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve research with human subjects crowdsourcing, and therefore no IRB or equivalent ethical approval is required.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method of this research does not involve any important, original, or non-standard usage of large language models (LLMs). Any use of LLMs was limited to writing assistance and does not impact the scientific contributions of the paper.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our LLM /ttp:p////eaarpp.cccc//ooeeeeees for what should or should not be described.
25


Reviews:
[{"review_id": "n2xVNyMPXr", "reviewer": "Reviewer_p5DD", "summary": "The paper presents Pixel-Perfect Depth, a monocular depth estimation model that leverages pixel-space diffusion transformers to generate high-quality, flying-pixel-free depth maps. Unlike prior generative models (e.g., Marigold) that use Variational Autoencoders (VAEs) for latent-space compression, this framework avoids VAE-induced artifacts by performing diffusion directly in the pixel space. Key contributions include:\n\nSemantics-Guided Diffusion Transformers (SG-DiT): Integrates high-level semantic representations from vision foundation models (e.g., DINOv2, Depth Anything v2) to guide the diffusion process, enabling accurate modeling of global structures and fine-grained details. This is achieved via L2-normalized semantic features and a bilinear interpolation mechanism to align with transformer tokens .\n\nCascade DiT Design (Cas-DiT): Adopts a coarse-to-fine strategy where early transformer blocks use large patch sizes (16×16) for global structure modeling, and later blocks use smaller patches (8×8) for fine details. This reduces computational cost by 30% while improving accuracy .\n\nState-of-the-Art Performance: Trained on 54K samples from Hypersim, the model outperforms all published generative models on five benchmarks (NYUv2, KITTI, etc.), achieving an AbsRel of 4.3 on NYUv2. It also introduces an edge-aware point cloud metric, demonstrating 0.08 Chamfer Distance (vs. 0.12 for VAE-based GT) .", "questions": "Question1: How does the model perform with depth data from consumer-grade devices (e.g., iPhone LiDAR) vs. clinical systems? Have you tested for device-specific noise patterns?\n\nQuestion2: The paper notes flickering depth in video applications. What strategies are planned to integrate temporal consistency (e.g., optical flow or video diffusion priors)?\n\nQuestion3: Are there plans to implement techniques like knowledge distillation or token caching to reduce inference time for edge deployment?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\nOriginality:\n\nThe direct pixel-space diffusion approach eliminates flying pixels caused by VAE compression, a critical limitation in prior work .\nSemantic guidance from pretrained models and the cascade design are novel, addressing the challenge of balancing global-local features in high-resolution generation .\n\nMethodological Rigor:\n\nAblation studies validate the necessity of SG-DiT and Cas-DiT, showing up to 78% improvement in AbsRel when both components are used .\nCross-dataset evaluation on diverse scenes (indoor/outdoor) confirms generalization .\n\nPractical Significance:\n\nThe edge-aware metric and flying-pixel-free results are crucial for real-world applications like 3D reconstruction and robotic manipulation .\n\nWeaknesses\n\nComputational Overhead:\n\nTraining requires 8 NVIDIA GPUs for 800K steps, limiting accessibility for small labs .\n\nDataset Limitations:\n\nThe training dataset (Hypersim) is synthetic, and real-world datasets lack diversity in recording devices or demographics .\n\nInference Speed:\n\nMulti-step diffusion leads to slower inference (0.14s on RTX 4090) compared to discriminative models like Depth Anything v2 .", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for recognizing the **originality**, **methodological rigor**, and **practical significance** of our work. We address your questions as follows.\n\n### **Q1: Computational overhead.**\n\nWe will release both the pretrained models and code publicly, allowing users to directly use our model without retraining. In addition, we will release a smaller variant (**PPD-Small**, as shown in the table below) of our model that can be trained on consumer-grade GPUs (e.g., 4×4090), making it accessible to small labs. \n\n### **Q2: Training dataset limitations.**\n\nBenefiting from the semantic priors provided by the vision foundation model, our method achieves strong generalization even when trained only on the synthetic Hypersim dataset. Similarly, prior work such as Marigold has also shown that diffusion priors can enable strong generalization when trained on the synthetic dataset alone. \n\n### **Q3: Multi-step diffusion leads to slower inference.**\n\nTo significantly reduce inference time, we have developed a smaller variant (**PPD-Small**) of our model presented in the paper (**PPD-Large**). As shown in the table below, **PPD-Small** achieves significantly faster inference than Depth Pro and is only slightly slower than Depth Anything v2.\n\n|      Method      |   Time (ms)   |\n|--------|:--------:|\n|    Depth Anything v2   |    18    |\n|    Depth Pro   |    170    |\n|    **PPD-Large (Ours)**   |    140    |\n|    **PPD-Small (Ours)**   |    40    |\n\nAs shown in the table below, **PPD-Small** achieves comparable accuracy to the larger **PPD-Large** model presented in our paper.\n\n|      Model      |      NYUv2      |      KITTI      |      ETH3D      |      ScanNet      |      DIODE      |\n|:----------:|:----------:|:----------:|:----------:|:----------:|:----------:|\n|           |  AbsRel↓    δ₁↑  |  AbsRel↓    δ₁↑  |  AbsRel↓    δ₁↑  |  AbsRel↓     δ₁↑  |  AbsRel↓     δ₁↑  |\n|  **PPD-Large**  | 4.3   /   97.4  | 8.0   /   93.1  | 4.5   /   97.7  | 4.5   /   97.3  | 7.0    /  95.5  |\n|  **PPD-Small**  | 4.5   /   97.3  | 8.3   /   92.8  | 4.6   /   97.4  | 4.7    /  97.2  | 7.3    /  95.3  |\n\n### **Q4: How does the model perform with depth data from consumer-grade devices?**\n\nOur model performs well on images captured by consumer-grade cameras, as demonstrated in Figure 5 of our paper using the NYUv2 dataset, which was collected with a Kinect camera. Since our model only requires a single RGB image as input, it naturally generalizes to a wide variety of consumer-grade RGB cameras. We will include additional results from other consumer devices, such as iPhone cameras, in the final version of the paper.\n\n### **Q5: The paper notes flickering depth in video applications. What strategies are planned to integrate temporal consistency?**\n\nOur method can be extended to produce temporally consistent depth maps by incorporating strategies such as the temporal alignment technique used in Video Depth Anything (Chen et al., CVPR 2025) or the video diffusion priors introduced in DepthCrafter (Hu et al., CVPR 2025). These strategies can help our model mitigate flickering artifacts and produce temporally consistent depth without flying pixels. This could be one of the interesting future works.\n\n### **Q6: Are there plans to implement techniques like knowledge distillation or token caching to reduce inference time for edge deployment?**\n\nYes, we have already distilled a lightweight model, **PPD-Small**, to facilitate faster inference for edge deployment. While our original model, **PPD-Large**, adopts the DiT-Large architecture, PPD-Small is built upon the more efficient DiT-Small architecture. As shown in the table below, **PPD-Small** significantly reduces inference time while maintaining competitive accuracy compared to **PPD-Large**.\n\n|      Model      |      NYUv2      |      KITTI      |      ETH3D      |      ScanNet      |      DIODE      |      Time (ms)      |\n|:----------:|:----------:|:----------:|:----------:|:----------:|:----------:|:----------:|\n|           |  AbsRel↓    δ₁↑  |  AbsRel↓    δ₁↑  |  AbsRel↓    δ₁↑  |  AbsRel↓     δ₁↑  |  AbsRel↓     δ₁↑  |            | \n|  **PPD-Large**  | 4.3   /   97.4  | 8.0   /   93.1  | 4.5   /   97.7  | 4.5   /   97.3  | 7.0    /  95.5  |    140    |\n|  **PPD-Small**  | 4.5   /   97.3  | 8.3   /   92.8  | 4.6   /   97.4  | 4.7    /  97.2  | 7.3    /  95.3  |    40    |"}, {"review_id": "hHmnu7VUJU", "reviewer": "Reviewer_MxaR", "summary": "This paper analyzes the limitations of current generative and discriminative models for monocular depth estimation. Specifically, it identifies two major issues: (1) the training instability and poor performance of pixel-based diffusion models, and (2) the boundary over-smoothing problem in discriminative models. To address these challenges, the authors propose a novel geometry-guided function model, which effectively stabilizes the training of diffusion models and preserves sharp depth boundaries. The experimental results support the authors' analysis and demonstrate that their method achieves state-of-the-art performance with a relatively simple network architecture.", "questions": "1. What is the inference time for a 512x512 images.\n2.The paper mentions that the geometry features are normalized due to their smaller magnitude compared to diffusion features. However, if the diffusion features are not globally normalized either, wouldn't the scale mismatch still affect feature fusion? Would it be more appropriate to normalize both or apply a learnable scaling? Would you mind do an ablation about this?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. To the best of my knowledge, this is the first work that combines a foundation geometry model with a pixel-space diffusion model for monocular depth estimation. Pixel-based diffusion model is particularly meaningful, as latent-space representations inevitably suffer from information loss. Therefore, the proposed approach is both novel and potentially impactful.\n2. The paper provides insightful analysis of the limitations in current depth estimation models, which may inspire future research directions. Moreover, the proposed solution is simple yet effective. By adopting a pure-transformer architecture, the method remains lightweight and easily extensible, which is beneficial for follow-up work.\n3.The experiments are comprehensive and convincingly demonstrate that the proposed method achieves state-of-the-art performance.\n4. The writing is clear, well-organized, and easy to follow. \n\nWeakNesses.\n\n1.Although the simple geometry integration achieves SOTA, it is unclear if more efficient or better alternatives than normalize + concat exist. The improvement over DepthAnything v2 is not very significant, and more ablation studies are needed to justify this design choice.\n\n2. The paper lacks a thorough analysis of efficiency. While it mentions that diffusion models inevitably slow down inference, it does not quantify how much slower they are. A more precise comparison of inference time would be necessary.\n\n3. The paper lacks deeper theoretical insight. While it is somewhat expected that geometry priors can enhance diffusion-based depth estimation, the paper does not clearly analyze the underlying challenges or implications of combining the two. As a result, the novelty seems to reside more in the practical combination of known components than in a fundamentally new conceptual contribution.", "was_revised": true, "final_justification": "The authors have addressed my problem. It is good work.", "rebuttal": "Thank you for your positive feedback and recognition of our work’s **novelty**, **simplicity**, and **effectiveness**. We address your questions as follows.\n\n### **Q1: Simple geometry integration achieves SOTA, it is unclear if more efficient or better alternatives than normalize + concat exist.**\n\nWe observed that geometry features (semantic features) have much larger magnitudes than diffusion features, which caused training instability and frequent NaNs when fused directly. To address this, we normalize the geometry features to match the scale of diffusion features. This simple strategy stabilizes training and achieves strong performance. We also experimented with more complex alternatives, such as learnable scaling or adaptive fusion weights, but they did not bring clear performance gains.\n\n### **Q2: Improvement over Depth Anything v2.**\n\nThe main goal of our work is to address the flying pixels problem commonly observed in discriminative models such as Depth Anything v2. As shown in Figures 1 and 4 and Table 4 of our paper, our model significantly reduces flying pixels. While standard benchmarks and metrics (e.g., Table 1 in our paper) typically fail to assess flying pixels at object edges, we propose an edge-aware point cloud metric to quantify flying pixels specifically at edges. This is explained in Section 4.4 of the paper. As illustrated in Table 4 of our paper, our method achieves a Chamfer distance of 0.08, which is a substantial improvement compared to 0.18 from Depth Anything v2.\n\n### **Q3: Analysis of efficiency.**\n\nWe provide a quantitative comparison of inference speed against other representative models. For a fair comparison, all models are evaluated on an RTX 4090 GPU with an input resolution of 512×512. As shown in the table below, our model is slower than Depth Anything v2, primarily due to the multi-step nature of diffusion-based inference. To offer a clearer view of the computational overhead, we report the runtime of each component: the semantics encoder takes 10 ms, and each diffusion iteration takes approximately 32.5 ms. With 4 iterations, the total inference time sums to 140 ms. \n\n|      Method      |   Time (ms)   |\n|--------|:--------:|\n|    Depth Anything v2   |    18    |\n|    Depth Pro   |    170    |\n|    Marigold   |    210    |\n|    Ours   |    140    |\n\n\n\n|      Module      |   Time (ms)   |\n|--------|:--------:|\n|    Semantics encoder   |    10    |\n|    1 iter (diffusion)   |    32.5    |\n|    4 iter (diffusion)    |    130    |\n|    Ours (total)   |    140    |\n\n### **Q4: Deeper theoretical insight.**\n\nOur work is grounded in a principled analysis of the flying pixels problem commonly found in discriminative models such as Depth Anything v2 and Depth Pro, which tend to predict averaged depth values at depth-discontinuous edges due to the objective of minimizing regression loss. In contrast, our pixel-space diffusion model estimates pixel-wise depth distributions, allowing it to preserve sharper edges and avoid flying pixels. Unlike existing diffusion-based methods that rely on latent-space Stable Diffusion and suffer from VAE-induced flying pixels, our model avoids latent compression entirely.\n\nWhile our model effectively eliminates flying pixels, it introduces new optimization challenges due to the increased difficulty of learning in pixel space. To improve convergence, we integrate geometry priors into the diffusion process. This innovative solution significantly enhances convergence and contributes to the strong performance of our model.\n\nOur design reflects the underlying modeling challenges and contributes new insights into how diffusion can be effectively applied to structured prediction tasks such as depth estimation.\n\n### **Q5: Inference time for 512x512 images.**\n\nThe inference time for a 512×512 image is 140ms, as shown in Table 2 of the paper.\n\n### **Q6: Normalizing both geometry and diffusion features.**\n\nWe conducted an ablation study to evaluate the effect of normalizing both geometry and diffusion features. As shown in the table below, doing so yields no clear performance improvement. In fact, normalizing geometry features alone already provides stable training and effective integration with diffusion features.\n\n|                 |      NYUv2      |      KITTI      |      ETH3D      |      ScanNet      |      DIODE      |\n|:----------:|:----------:|:----------:|:----------:|:----------:|:----------:|\n|           |  AbsRel↓    δ₁↑  |  AbsRel↓    δ₁↑  |  AbsRel↓    δ₁↑  |  AbsRel↓     δ₁↑  |  AbsRel↓     δ₁↑  |\n|  **Norm geometry**  | 4.3   /   97.4  | 8.0   /   93.1  | 4.5   /   97.7  | 4.5   /   97.3  | 7.0    /  95.5  |\n|  **Norm both geometry and diffusion**  | 4.2   /   97.6  | 7.9   /   93.9  | 4.4   /   98.0  | 4.6    /  97.1  | 7.5    /  95.1  |"}, {"review_id": "ukPmpmyJCL", "reviewer": "Reviewer_fjSa", "summary": "This paper introduces Pixel-Perfect Depth, a monocular depth estimation framework that operates in pixel space rather than the more common latent space, thereby avoiding flying-pixel artifacts introduced by VAE compression. To address the complexity of high-resolution generation, the authors propose two novel architectural components: Semantics-Guided Diffusion Transformers (SG-DiT) and a Cascade DiT design (Cas-DiT). This method achieves SOTA results and introduces an edge-aware evaluation metric for assessing depth quality near object boundaries. I find the results promising, and I believe the paper would be further strengthened if the experimental issues noted in the Weaknesses section are addressed.", "questions": "Please refer to the Weaknesses. If the authors can adequately address those concerns in the rebuttal, I would consider increasing my score from 4 to 5.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths:\n1. The motivation is clear and well-justified: the method targets the flying pixel issue in depth estimation models, particularly at object boundaries. By proposing a pixel-space diffusion framework with semantic guidance, the authors offer a novel and effective solution. The qualitative and quantitative results across diverse benchmarks demonstrate strong performance.\n2. The paper introduces a new edge-aware evaluation metric designed to quantify depth estimation accuracy specifically at object boundaries. This addresses a gap in current evaluation protocols and is a meaningful contribution to the field.\n\nWeaknesses:\n1. In the qualitative ablation analysis (Figure 5), only depth map visualizations are shown. Including corresponding point cloud visualizations (as in Figure 1) would help better assess how this component impacts the flying-pixel artifacts.\n2. Since the core motivation of the paper is to reduce flying pixels, and a new edge-aware evaluation metric is proposed for that purpose, it would strengthen the paper to conduct ablation studies using this metric. This would provide more direct evidence of how each module contributes to mitigating boundary errors.", "was_revised": true, "final_justification": "I have read the rebuttal and appreciate the authors’ clarifications. They have addressed my main concerns, and I am increasing my score from 4 to 5.", "rebuttal": "Thank you for your very positive feedback! We truly appreciate you highlighting the **well-motivated** design of our method, its **novel solution** to the flying pixel issue, and the **significant contribution** of our new edge-aware evaluation metric. We've addressed all your comments in detail below.\n\n### **Q1: Point cloud visualizations for ablation analysis.**\n\nThank you for the valuable suggestion. We have visualized the corresponding point clouds for the ablation studies  (with and without SG-DiT) to assess the impact on flying-pixel artifacts. Our SG-DiT not only effectively reduces flying pixels but also better preserves the global structure of the scene. Due to space limitations in the rebuttal, we are unable to include these figures here, but we will ensure that these visualizations are included in the final version of the paper.\n\n### **Q2: Ablation studies on edge-aware evaluation metric.**\n\nThank you for the insightful suggestion. We have already included ablation studies using the proposed edge-aware evaluation metric, as shown in the table below. \n\n|        Method      |   Chamfer Distance ↓   |\n|--------|:--------:|\n|  DiT (baseline)  |  0.13  |\n|  SG-DiT  |  0.09  |\n|  SG-DiT+Cas-DiT (full)  |  0.08  |\n\nExperimental results demonstrate that our proposed components consistently reduce flying pixels and enhance the structural integrity of the reconstructed point clouds. Specifically, SG-DiT improves the Chamfer Distance from 0.13 to 0.09, while the full model further improves it to 0.08, confirming the effectiveness of our design in mitigating boundary errors."}, {"review_id": "j5EDeqREbi", "reviewer": "Reviewer_icJf", "summary": "The authors present an innovative approach to relative depth estimation using Semantics-Guided Diffusion Transformers (SG-DiT). Focused on generating high-quality depth maps, the paper effectively addresses challenges such as flying pixels and edge artifacts. The claims are robustly supported by both theoretical foundations and experimental results throughout the study, showcasing the model's ability to generalize across diverse scenarios. Furthermore, the paper provides insights into the computational efficiency of the proposed methods. Overall, it is well-structured, thoroughly addressing vital issues concerning generalization and reproducibility, thereby marking a significant advancement in the domain of more precise relative depth estimation.", "questions": "See in the weaknesses part.", "limitations": "yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths: The paper presents detailed theoretical and experimental validation, thoroughly discussing the model's architecture while providing quantitative evaluation metrics and visual quality demonstrations. It proposes an innovative solution for monocular relative depth estimation, with considerable potential for practical applications. The paper effectively addresses typical issues related to flying pixels and edge artifacts, offering robust comparisons and solutions.\n\nWeaknesses:\n\n- It seems that post-processing techniques, such as filtering depth at certain positions using image gradient information, could also resolve the flying pixels issue?\n\n- Compared to relative depth, monocular metric depth estimation is more practical. Can the method proposed in this paper be extended to monocular metric depth estimation?\n\n- How does this method perform in extremely detailed regions of an image, such as hair details showcased in Depth Pro?", "was_revised": true, "final_justification": "The author's response has addressed my concerns, so I have decided to accept the manuscript.", "rebuttal": "Thank you for your positive feedback and recognition of our **detailed theoretical and experimental validation**, **innovative approach**, and **practical potential**. We've addressed all your comments in detail below.\n\n### **Q1: Filtering depth.**\n\nThank you for the thoughtful question. While post-processing techniques, such as filtering depth based on image gradients, may offer some mitigation, they are often ineffective in handling severe flying-pixel artifacts. Moreover, such techniques typically smooth or replace flying pixels using neighboring values, which can result in edge blurring or structural distortion.\n\nIn contrast, our model addresses flying pixels directly during the prediction stage, enabling it to preserve sharp object boundaries and produce structurally accurate and reliable depth maps, without relying on heuristic post-hoc fixes.\n\n### **Q2: Monocular metric depth estimation.**\n\nOur method can be extended to metric depth estimation using metric depth normalizing techniques proposed in Metric3D [Yin et al., ICCV 2023] and Depth Pro [Bochkovskii el al., ICLR 2025]. Specifically, our method could serve as backbone network to predict metric depth normalized by focal length, obtaining metric depth without flying pixels. This could be one of the interesting future works.\n\n### **Q3: Performance in extremely detailed regions.**\n\nOur method performs well in fine-detail regions such as hair structures, which is one of the core strengths of diffusion-based generative models, as they are particularly effective at modeling high-frequency details. We have visualized hair details similar to those in Depth Pro. Due to the limitations of the rebuttal format, we are unable to include these qualitative visualizations here, but we will ensure that they are incorporated into the final version of the paper."}]

Author rebuttal:

Meta-review: The final ratings are 4 accepts. The AC have read the reviews and rebuttal, and discussed the submission with the reviewers. The reviewers raised a number of points during the review phase including limitations to dataset, analysis of efficiency, and technical insight. The authors were able to address these points during the rebuttal and discussion phases by demonstrating a smaller variant of the model, extending evaluation, and providing additional analyses. The AC recommends the authors to incorporate the feedback and suggestions provided by the reviewers, and the materials presented in the rebuttal, which would improve the next revision of the manuscript.
Decision: Accept (poster)