PatchRefiner V2: Fast and Lightweight Real-Domain High-Resolution Metric Depth Estimation
Authors: Zhenyu Li
Wenqing Cui
Shariq Farooq Bhat
Peter Wonka
Venue: ICLR 2026 Poster (2026)
Keywords: Depth Estimation
High Resolution
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: While current high-resolution depth estimation methods achieve strong results, they often suffer from computational inefficiencies due to reliance on heavyweight models and multiple inference steps, increasing inference time. To address this, we introduce PatchRefiner V2 (PRV2), which replaces heavy refiner models with lightweight encoders. This reduces model size and inference time but introduces noisy features. To overcome this, we propose a Coarse-to-Fine (C2F) module with a Guided Denoising Unit for refining and denoising the refiner features and a Noisy Pretraining strategy to pretrain the refiner branch to fully exploit the potential of the lightweight refiner branch. Additionally, we propose to adopt the Scale-and-Shift Invariant Gradient Matching (SSIGM) loss within local windows to enhance synthetic-to-real domain transfer. PRV2 outperforms state-of-the-art depth estimation methods on UnrealStereo4K in both accuracy and speed, using fewer parameters and faster inference. It also shows improved depth boundary delineation on real-world datasets like CityScapes, demonstrating its effectiveness.

Published as a conference paper at ICLR 2026
PATCHREFINER V2: FAST AND LIGHTWEIGHT REAL- DOMAIN HIGH-RESOLUTION METRIC DEPTH ESTI- MATION
Zhenyu Li, Wenqing Cui, Shariq Farooq Bhat, Peter Wonka
KAUST
(zhenyu.li, 1i.., wenging. cui, shariq .iaat, peter. wonka (kaust edu. sa
ABSTRACT
While current high-resolution depth estimation methods achieve strong results, they often suffer from computational inefficiencies due to reliance on heavyweight models and multiple inference steps, increasing inference time. To address this, we introduce PatchRefiner V2 (PRV2), which replaces heavy refiner models with lightweight encoders. This reduces model size and inference time but introduces noisy features. To overcome this, we propose a Coarse-to-Fine (C2F) module with Guided Denoising Unit for refining and denoising the refiner features and a Noisy Pretraining strategy to pretrain the refiner branch to fully exploit the po- tential of the lightweight refiner branch. Additionally, we propose adopt the Scale-and-Shin Invariant Gradient Matching (SSIGM) loss within local windows enhance synthetic-to
domain transfer. PRV2 outperforms state-of-the -he-----thhrrr art depth estimation methods on UnrealStereo4K in both accuracy and speed, using fewer parameters and faster inference. It also shows improved depth boundary de- lineation real-world datasets like CityScapes, demonstrating its effectiveness.
INTRODUCTION
Accurate nigh-resolution depth estimation from single image is critical for advancements in fields such as autonomous driving, augmented reality, and 3D reconstruction Eigen et al. (2014); Zhang al. (2023); Bhat et al. (2023):
al. (2024c). Current tht depth estimation models typically operate at relatively low resolutions (e.g.. 0.3 megapixels). High memory requirements, especially 4K resolution, pose a significant challenge for training depth estimation models that can natively support high-resolution inputs. Recent 4K depth estimation approaches like PatchRe- finer Li et al. (204b) (PRV1) use a tile-based strategy where the high-resolution image is divided into patches. The patch-level depth predictions (fine, local outputs) are then fused with the depth prediction of a downsampled version of the input image (coarse, global output) to obtain a single, consistent, high-resolution output.
Despite its success, the PatchRefiner framework faces critical computational efficiency and scalabil- ity challenges for weal-wrldd applications. It employs the same architecture (a pre-trained base depth model) to extract both global and patch-level features. This amounts to at least 16 forward passes of the base model for single high-resolution input. As the base model used [Bat al. (0033); Yang et al. (2 Yang et al. is often large, this results in two major issues: 1) High inference time of more than second per image, and more importantly 2) High memory requirement, making the end-to-end training infeasible. Therefore, the PRV1 framework has to adopt stage-wiss wise train- ing, where global and local branches are trained sequentially, leading to long training time and suboptimal results.
To alleviate these issues, we propose to substitute the large foundational models, such as ZoeDepth Bhat al. (2023) or DepthAnything Yang et al. (2024); Yang al.) used in the refiner branch Li et al. (2024b) with lightweight encoders like MobileNet Howard et al.] (2019); Qin et al.) 2024) and EfficientNet Tan & Le (2019). This change significantly reduces the number of param- eters and memory usage, decreases inference time, and enables end-to-en training without bells and whistles. However, this modification introduces trade-off: the model capacity is reduced, and

---

Published as a conference paper at ICLR 2026
1.05
Patches Poller
@
Base
F2C
Offset
1.00
Whole
Base
Copy
rol
Coarse Pred.
Depth
RMSE (ooer la
Image
(i) PatchRefiner Framework
0.95
PBV
PRV
RMSE
Patches
Lite
C2F
F2C
Offset
PP
1864
452-1M
3,44c
BRV2c
200/2
PRVT
1003
30U05
72,134
47,034
0.374
0.324
1345-
Whole
Image
Base
rol
Coarse Predi
Depth
0.85
iii Our PatchRefiner V2 Framework
0.0
0.5
1,0
1.5
2,0
2.5
3.0
3.5
4.0
Indereace Time [seconds)
(a) Comparison on UnrealStereo4K.
(b) A Comparison of PRVI and PRV2.
Figure I: PatchRefiner V2 (PRV2) significantly outperforms previous high-resolution frameworks. PF and PRV1 are short for PatchFusion Li et al. (2024a) and PatchRefiner Li et al. (2024b), re- spectively. We adopt a lightweight encoder for the refiner branch, which alleviates the inference speed bottleneck, reduces the number of parameters for high-resolution estimation, and facilitates end-to-end training. A novel coarse-to-fiee (C2F) module is proposed to denoise features from the lite model and further boost performance.
the refiner branch now lacks the depth-aligned feature representation otherwise provided by the pre- viously used pre-trained depth estimation base models. While end-to-end training alleviates some of this limitation, the lack of repth-aligned feature representation remains concern. Indeed, we observe that the features generated by these lightweight encoders tend to be 'noisy' (see Fig. 2 even after ImageNet initialization Deng et al.) (2009) and end-to-end training. This causes the original Fine-to-Coars, module (F2C) used in PRV1 to struggle to inject rich, high-resolution information for the final depth prediction.
We propose two components to improve the feature representation in the refiner branch: 1) The Coarse-to-fine module (C2F), which incorporates novel Guided Denoising Units (GDUs) a bottom-to-top manner Lin et al. (2017;; Xian et al. (2018); Ranftl et al. (2021). GDUs utilize coarse depth features as guidance to denoise and enhance the high-resolution refiner features. Together with the original Fine-to-Coarse module (F2C), this establishes bidirectional fusion process: C2F initially denoises and refines high-resolution features using coarse features, followed by F2C's en- hancement of the predicted coarse depth map via residual prediction. 2) Noisy Pre-training Given that the C2F and F2C modules require initialization from scratch, we propose a simple pre-training strategy for the entire refiner branch including the encoder, C2F, and F2C modules to en- hance feature representation and accelerate learning. During Noisy Pre-traine te-rriiniin, we replace the input coarse depth features for GDUs with random noise, essentially forcing the refiner branch to learn to extract depth-relevant features from the high-resolution input.
Given
Finally, the PRV1 framework Li et al. (202bb) employs the Detail and Scale Disentangling (DSD) training strategy to adapt the high-resolution depth estimation framework to real-domain datasets, which enables learning 'detail' from synthetic data and 'scale" from the real domain. To isolate the scale from the synthetic data, the DSD strategy uses ranking loss and Scale-and-Shift Invariant (SSI) loss Ranftl et al. (022). To further improve the transfer of high-frequency knowledge, we introduce gradient-level loss Li & Snavely (2018): Ranftl al. (2022) applied after scale-and- shift alignment within local windows Bhat et al. (2022); Wang et al. (2025), which we term the local Scale-and-Shii Invariant Gradient Matching (local SSTGM) loss. While the formulation of the gradient loss follows & Snavely (0188: Ranftl et al. (2022), our method differs in two key aspects: (1) supervision is derived from pseudo labels generated by a teacher model trained on a synthetic dataset, and (2) the loss is computed within local windows rather than across the entire depth map. These modifications are designed to mitigate potential distortions in accurate scale estimation and to encourage the model to focus on fine-grained local structures.
(2018)
We use the 'pre-training' loosely, as this process accurs prior the final training phase.
2

---

Published as a conference paper at ICLR 2026
(a) Input, Prediction
(b) Coarse Feature
(c) Refiner Feature (wo C2F) (d) Refiner Feature (w/ C2F)
Figure 2: Visualization of F2C Input Feature Maps. We showcase the first 16 channels of the F2C input features. (c) Without the C2F module (setting @ in Tab, 2 the refiner features are 'noisy' and hard to interpret. (d) The C2F module helps denoise the refiner features, leading to clear boundaries and better results.
Experiments demonstrate that our advanced framework, PatchRefiner V2 (PRV2), performs ef- fectively across various lightweight architectures. As summarized in Fig. la PRV2 significantly outperforms other high-resolution metric depth estimation frameworks on the UnrealStereo4K Tosi al. 202T) dataset in terms of both quantitative results and inference speed. Additionally, uate the effectiveness of our local SSIGM loss on the real-world dataset CityScapes Cordts et al. (2016). Our method reveals significant improvements in depth boundary delineation (e.g.. +25.1% boundary F1 w.e/ [i et al.] (2024b)) while maintaining accurate scale estimation, showcasing its adaptability and effectiveness.
eval-
RELATED WORK
2.1
RISH-RESOLUTION MONOCULAR DEPTH ESTIMATION
Monocular depth estimation (MDE) is fundamental computer vision task and has recently seen
impressive progress with advanced network design Eigen et al. (2014); Bhat et al. [0]22202] Li et al.
2023 2024c); Bhat al. (2023); Yang et al. (2021), supervision Lee & Kim (2020); Liu et al.
2023a Xian et al. (2020); Ranftl et al. (2022) Godard et al. (2019), formulation Fu et al. (2018);
Diaz & Marathe (2019) Bhat et al. (2021 Xian al. (2020): Li et al. (2024c)
; Bhat et 2022
training strategy Petroval & Nedevschi (2022); Godard et al. (2019):
| Fan et al. (2023): 3: Ranftl et al.
2022). , public datasets Silberman et 2012); Geiger et (2013)
; Dai et (2017); Cordts et
2016 ; Roberts et al. (2021), etc. Recently, most SOTA frameworks Bhat et al. (2023): Yang
al. (2024); Yang et al. et al. (2024) build on the top of heavy backbones |ao al. (2022);
Dosovitskiy et al. (2021); Oquab et al. (2025): Rombach et al. (2022). leading to the limitation of low-resolution input. For example, Depth Anything V2 Yang et al. uses [isovitDsy et al. 2021); Oquab et al.) (2025) and can only infer 756x (about 0.75 megapixels) images an NVIDIA VTO0 32G GPU. Another recent work, DepthPro Bochkovskii et al. (2025), presents a high-resolution framework whereas the input resolution is fixed 1536. While another line of re- search utilizing the generative model for MDE achieves fine-graing results Ke et al. (2024); Phhm let al. 2025); Xu et al. (2025). , similar dilemma exists. For instance, Marigold Ke al. (2024)
based on Stable Diffusion Rombach et al. (2022) runs with ~-.33 megapixels as default.
This contrasts with the advancements in modern imaging devices, which increasingly capture images at higher resolutions, reflecting the growing demand for high-resolution depth estimation Li et al. (022a)). To relax the constraints, initial efforts utilize Guided Depth Resou--eoo (GDSR) Met- Zger et al. (2023); ; Hui et al. 2016); Zhong et al. (2023) and Implicit Functions Mildenhall et al. (2021); Chen al. (2021). Recent works adopt a tile-based method segment images into patches for estimation and then reassemble all predictions into comprehensive depth map Miangolch et al. (202 Li et al.] (2024a b): Kwon & Kim (2025). Since all these methods adopt the dual-branch architecture and utilize the same SOTA depth model in both branches, the frameworks are heavy and slow at inference time. By contrast, we aim to achieve fast, high-resolution metric depth es- timation using the tile-based method with fewer additional parameters. More specifically, Kwon & Kim (2025) proposes the grouped patch consistency training and bias-free masking to improve patch consistency and mitigate dataset-specific biases. Their approach focuses consistency learn- ing, which orthogonal and complementary to our focus on lightweight architectural design for efficient high-resolution depth estimation.
Chen
3

---

Published as a conference paper at ICLR 2026
Guided
GDU
Course Feature C2F Ougutt Footure
Duniiaai Unss
Roltha Contr Rompune
Sum
27F layer
Corcaterate
Rasioual Comy Lint
C2P Loyer
Cony Block
steed Danonant Lime
Nikyeer
Rostae Pro eotion
Sigmaid
Mw
122 Lagar
Layer (C2F)
fa
Figure 3: Left: Searse-to-Fine (C2F) module overview. It processes refiner features in bottom- to-top manner with N successive C2F layers. Each layer guided by coarse features with corre- sponding resolution and outputs denoised features for the Fine-to-Coarse (F2C) module. Center:
C2F layers combine multi-level features with Residual Convolutional Units Lin et al. (2017): |aantl
ct al. (2022) and denoises the features using Guided Denoising Units (GDU). Right: Guidance
formation from the coarse branch is introduced through a concatenation followed by convolutional
block and then converted weight map ranging from 0 to I through the sigmoid operator. We then adopt an elementwise multiplication to denoise the shortcut feature.
2.2 KENTHETIC-TO-REATT TRANSFER FOR MDE
The challenge of obtaining high-quality, real-domain data for training high-resolution depth mod- els has led recent efforts to utilize synthetic datasets, thereby encountering significant domain gaps during real-world inference RRjpal et al. (2023); Li et al. (2024a). To address this issue, PatchRe- finer combines labeled data from both synthetic and real domains, enhancing depth estimation in real-world, high-resolution settings Li et al. (2024b). Inspired by the successes of semi-supervised learning Van Engelen & Hoos (2020) Yang et al. (2024): Kirillov et al. (2023), it employs pseudo- labeling approach Pseudo-Labed (20ee (2013); Saito et al. (2017) (0117:(C Chhhn et al. (2019) Pastore et al. (202T); Shin al. (2022) along with the Detail and Scale Disentangling (DSD) loss. This strategy facili- tates the transfer of fineggrined knowledge from synthetic to real domains. In our work, we extend this concept by incorporating supervision the gradient space and within local windows, thereby significantly enhancing the effectiveness of knowledge transfer.
METHOD
3.1 REVISITING PATCHREFINER
We first revisit the PatchRefiner framework Li et al. (2024b) (named as PRV1). The PRV1 frame- work adopts tile-based approach to address the high memory and computational demands of high- resolution depth estimation Li et al.) (2024a); MMinggole et al. (2021). It utilizes a two-step process: Depth Refinement, as shown in Fig. Tb
(i) Coarse Depth Estimation and (ii)
This step involves a coarse depth estimation network, Ná. which pro- cesses downsampled inputs generate global depth map, D.. This map captures the overall scene structure and provides a baseline for further refinement. Notably, Nr can be any depth estimation model and is kept fixed after this stage.
(i) Coarse Depth Estimation:
(ii) Fine-Grained Depth Refinement: PRVI introduces a unified refinement network, Ay,, in place of separate fine depth networks and fusion mechanisms Li al. (2024a); Poucin et al. (202T). This network refines the coarse depth map by recovering details and enhancing depth precision at a patch level.
The refinement process begins with the cropped input image I, processed by a base depth model Na, which shares the same architecture as A.. Multi-scale features from both N., and A are collected as Fa = (fāke and F. (f:H Following Li al. 2024a), the roi He et al. 2017) operation extracts features from the cropped area j roi (f:)
4

---

Published as a conference paper at ICLR 2026
These features then aggregated by lightweight decoder through concatenation and convolu- tional blocks, referred to as the Fine-to-Coarse (F2C) module in this paper, which injects fine- grained information into the coarse refinement process. The F2C module constructs the resid- ual depth map D, at the input resolution, and the final patch-wise depth map is computed as D Drroi (Dz) + D,-
As the second contribution, PRV1 introduces a teacher-student framework to transfer the fine- grained knowledge learned from the synthetic data to the real domain. The Detail and Scale Disen- tangling (DSD) loss is designed to help the model balance detail enhancement with scale accuracy by integrating both the ccale-consistent ground truth supervision and the detail-focused pseudo la- bels. Both ranking loss Xian et al. (2020) and the scale-and ande-and--shft invariant loss Ranftl et al. (2022) can be adopted for pseudo-label supervision.
Limitations of PRVI. Similar to other tile-based methods Poucin et al. (2021) Li al. (2024a), the PRV1 framework encounters significant challenges with the computational efficiency and scalability for real-world applications due to the shared usage of the base depth model (c.g., Manthingganggao ZoeDepth[Bhan [Bhat et al. (2023). Depth Anything Yang et al. (2024): Yang et al.) across both the coarse and refiner branches. For given input image, while the coarse branch processes the downsampled image to gather global information, the refiner branch requires multiple inferences (at least 16 in PRVI's default mode) for the patches. Since both branches share the same architecture, the refiner branch becomes the primary efficiency bottleneck. Our goal is alleviate this bottleneck as much as possible.
Moreover, heavy framework makes end-to-end training infeasible due to GPU memory limita- tions. The PRVI framework has to adopt two stages for training the framework, where global and local branches are trained sequentially. This results in long training time and suboptimal perfor- mance. While the authors claim that multiple-stage stage training could potentially lead to stage- stage-wise local optima Li et al. (2024b), our goal is pursue end-to-end training.
3.2 PATCHREFINER V2 FRAMEWORK
3.2. 1
LITE FRAMEWORK FOR FASTER INFERENCE AND END-TO-END TRAINING
We propose simple solution to address PRVI's limitations: lightweight architecture for the refiner branch. Given that the coarse branch already provides a reliable base depth estimation D,, using the same heavy model for the refiner might be unnecessary. This substitution significantly increases inference speed, reduces the model size, and enables end-to-end training. However, it also results in noticeable decline in refinement quality compared to previous methods Li al. (2024a b). We attribute this decline to the lack of depth-aligned feature representation in the refiner branch, as shown in Fig.
To compensate for the loss in model capacity and pepph-pretriinin by the proposed substitution, we introduce better architecture design, a Coarse-to- (C2F) Module, and fast and simple pre-training strategy, Noisy Pretraining (NP).
3.2.2 COARSE-TO-FINE MODULE
Since the refiner branch no longer includes a pretrained depth model, we propose utilizing informa- tion from the global coarse branch to guide the selection of relevant details from the fine, patch-level features.
The proposed Coarse-to-Fine (C2F) module shown in Fig. 3 processes the multi-scale features extracted from the lightweight encoder through N successive C2F layers in a bottom-to-up man- ner Ronneberger et al.] ) Lin et al. (2017). mirroring the design of the module Li et al. (2024b). Each C2F layer is designed to progressively enhance and denoise the refiner features with the help of coarse feature representations.
(F2C)
Each layer in the C2F module consists of two components: our proposed Guided Denoising Unit (GDU) and the Residual Convolutional Unit Lin et al. (2017); Ranftl et al. (2022). The GDU intro- duces coarse feature maps fa at each stage to refine and denoise the refiner features. Specifically, the coarse features as guidance, which are incorporated via the concatenation operation (Cat)
5

---

Published as a conference paper at ICLR 2026
ZoeDepth
PatchRefiner
PRV2c
GT. Input
Figure 4: Qualitative Comparison on UnrealStereo4K. We show the depth prediction and corre- sponding error map, respectively. The qualitative comparisons showcased here indicate our PRV2c outperforms counterparts Bhat et al. (2023); Li et al. (2024b) with sharper edges and lower error around boundaries while achieving faster inference. emphasize details near depth boundaries.
We show individual patches in all images to
followed by the convolutional block (CB). The output of these blocks is passed through a sigmoid activation function (o) to obtain weight map M.., which ranges from 1. This weight map is then applied to the shortcut features fs through elemental multiplication @, effectively denoising the shortcut features. This process can be formulated as
Mw = (CCB(Cat(foo
(1)
fa Mw & fa:
where the fa indicates the denoised feature. Associated with the Residual Convolutional Unit, it al- lows the model to filter out irrelevant noise and enhance the quality of the refined features iteratively across the network layers. After that, we utilize the F2C module to inject the denoised fine-grained information for coarse features, leading more effective and better refinement process.
3.2.3 NOISY PRETRAINING
In PRV1, the framework's efficacy largely depends on the comprehensive pretraining of the base models in both the coarse and refiner branches Li et al. (2024b). During the subsequent high- resolution training stage, only the Fine-to-Coarse (F2C) module is trained from scratch, representing minor portion of the overall refiner branch (24.0M vs. 369.0N parameters). In other words, significant portion (~944% of the refiner branch is pretrained for depth estimation.
By our substitution, this pretraining is also lost. While the lightweight encoder used can be pre- trained on a large-scale dataset with complex strategies, it now constitutes only small part of the refiner branch (1.3M vs. 47.0M parameters for PRV2 and lacks the depth-aligned feature rep- resentation. In other words, even if we pre-train the encoder, significant portion (~98%) of the refiner branch must still be trained from seratch.
To address this issue, we propose a novel approach called Noisy Pretraining (NP). Prior to the high-resolution training, we pretrain the lightweight encoder along with the C2F and F2C modules. However, critical aspect of our framework is that both the C2F and F2C modules rely on features from the base model in the coarse branch. These features, however, are challenging to omit during the pretraining process. We propose straightforward yet effective solution: randomly generate the coarse features using a normal distribution N N(0, 1) as inputs. This forces the refiner branch to learn depth--elevant features without guidance from the coarse branch.

---

Published as a conference paper at ICLR 2026
Table 1: Quantitative Comparison on UnrealStereo4K. Best results are marked bold. PF, PRVI and PRV2 are short for PatchFusion Li et al. (0224)), PatchRefiner Li et al. (2024b) and PatchRe- finer V2, respectively. We report the P 16 mode for these high-resolution depth estimation frameworks Li et al. (2024a). Gray lines present numbers from the original paper with vanilla pretraining settings. : indicates the pretraining aligned version, where we remove the non-public Midas pretraining stage Ranfil et al. (2022) adopted for the fine refiner branch in PRVI and PF to make fair comparisons with our PRV2. The coarse branch is NOT modified. #param, and T denote the number of additional parameters adopted for high resolution estimation and the inference time of the fine or refiner branch for one input image. Best results are in bold. SharpDepth Pham et al. 2025 is not involved in this benchmark different training and evaluation protocols.
Mothod
(x())
RELI
RMSEL
SiLoogi
SEEL
#param]
TL
Refermnce
Piccinelli
SMD-Nol Tost of KPTT
97.774
0.044
1,282
7,389
0.833
CVPR
Graph-(USssin Jilmel
70223
97,932
0044
1.264
7.469
0.872
CVPR 2022
Boosting Minkgglar
98.104
0.044
1,123
6.662
0.939
CVPR 2021
ZoeDepth
97.717
0.046
1,289
2eeDe Lier
20244
98.419
0.040
1.088
6.212
0.838
432.7M
3.44s
CVPR 2024
ZeeDepth+P
PoeDejch+PRV of 20246
ZoeDeph++PRV Li
2024b
98.680
99.821
0.034
0.033
0.892
0.941
5.684
5.417
0.750
0.771
309.0M
1.45%
BCCV 2024
PRV
98.610
0.034
47.0M
ZxDepth+PRY2
98.728
0.034
0.948
0.816
72.1M
0.57
Ours
ZeDDech+PPV2
98.863
0.032
0.884
5.281
0.787
245.8M
0.625
Unlike other strategies Liu al. (2023b) Ozguroglu et al. (2024) Brooks et al. (2023), which often require careful selection and modification of convolutional layers and their corresponding parame- ters, our NP method avoids altering the framework's architecture. As a result, the pretraining and subsequent training stages proceed seamlessly, preserving the integrity of the overall model struc- ture while ensuring that all components of the refiner branch are well-prepared for high-resolution training.
3.3 LOCAL SCALE-AND-SHEE INVARIANT GRADIENT MATCHING
In the synthetic-to-real transfer stage, PRV1 employs the scale-and-shift invariant (SSI) loss Lssi as the pseudo-label supervision within the Detail and Scale Disentangling (DSD) loss. To better transfer frequeryy information, we extend the supervision the gradient domain and compute it locally.
Let d be the predicted depth and the pseudo label generated by teacher trained on synthetic data. J.N
For each training patch, we randomly sample N square windows { of side length L. For each window we estimate local scale-shift pair (sk, fx) by least-squares alignment Ranftl et al. (022
(SL. tx) = arg min 8,
Σ (sd(p) + ?
(3)
PEf
Our local Scale-and-Shift Invariant Gradient Matching (local SSIGM) can be formulated as
Liocal SSIGM
+V
(4)
14
PEft
where V. and V, are finite-difference gradients as in Li & Snavely 2018 Here, rx(p) is the residual of aligned prediction and ground-truth depth, caleulated by T (p) +t e Qg. The loss is combined with a weight À to control the strength pscudo-label supervision, as in PRV1 al. (2024b).
By aligning and matching gradients within windows rather than over the entire map, local SSIGM reduces the influence of global scale biases and forces the model to focus on fine-grained structures (e.g., boundaries and thin objects) during fighffrequency knowledge transfer. Note that setting ==1 and Q to the full image recovers the original (global) SSIGM.

---

Published as a conference paper at ICLR 2026
4 EXPERIMENTS
4.1 DATASETS AND METRICS
We evaluate the effectiveness of our proposed framework on the UnrealStereo4K dataset Tosi et al. (02I) (Synthetic), which offers synthetic stereo images at 4K resolution (2160 3840). each paired with accurate, boundary-complete pixel-wise ground truth. Adhering to the dataset splits in Tosi et al. (202T), we employ a default patch size of 540x 960 for compatibility with Li et al. 2024b). In terms of the symthetic-to-reat transfer part, we use the Cityscapes [Cordts et al. (2016)
dataset following PRVI [i et al.] (2024b). Following Li et al. (2024a), we adopt standard depth evaluation metrics from Eigen et al. (2014) and the Soft Edge Error (SEE) from Tosi et al. (2021)
for scale evaluation. As for the real-domain datasets (Cityscapes), we adopt the standard protocol
introduced in Li et al. (2024b) and utilize the Fl score to evaluate the boundary quality.
4.2 IMPLEMENTATION DETAILS
PRV2 on Synthetic Dataset: For training the synthetic dataset, employ the scale-invariant log loss Lasing. as introduced in Eigen et al. (014). We initialize the coarse network No with pretrained weights from the NYU-v2 dataset SSilberman
al.] (2012), adhering the approach in Li et al. 2024a b) for a fair comparison. As for the refiner branch, we employ the MobileNetV4-Smme al. (2024). EfficientNet-BS Tan & Le (2019). and Convnext-Large for PRV2M, PRV2g, and PRV2c. respectively. We perform the noisy pretraining for the refiner branch for 96 epochs. The No independently trained for 24 epochs and fine-tuned with the refiner branch in a fully end-to-end manner for another 48 epochs on the synthetic dataset. During inference, we use the Consistency- Aware Inference, as described in Li et al. (2024a).
Qin
Learning on Real-Domain Dataset: Following Li et al. 2024b), we first train the full PRV2 framework on the target real-domain dataset with the same setting as the synthetic dataset. During this stage, we perform impact on the final results. After that, we fine-tune the model with the Detail and Scale Disentangling loss for three epochs to refine depth estimations. The weight of the DSD loss is set as 0.8.
ablation the NP strategy by toggling it and off and reporting its
4.3 EXPERIMENTAL RESULTS AND DISCUSSION
Main Results: As shown in Tab. 1 our most lightweight model, PRV2 not only improves RMSE by 22.2% compared to the base depth model but is also 9.2x smaller and 10.7x faster than PatchFu- sion (PF) in terms of parameter count and inference speed, respectively. Our middleweight model, PRV2g, achieves comparable RMSE to the previous SoTA PRVI while being 2.5x faster and 5.1x smaller, offering an excellent balance between performance and efficiency. With ConvNext as the backbone, PRV2c sets new SoTA with an RMSE of 0.884, while being 2.3x faster than PR. Qual- itative results in Fig. 4 demonstrate PRV2g's 's superior boundary delineation.
We ablate and discuss the contributions of individual components proposed for PRV2. We employ the MobileNet in the refiner branch for experiments on the synthetic dataset and EfficientNet on the real-domain dataset. By default, we adopt
16 patches for clarity and ease of comparison.
Framework Design: As shown in Tab. Za we start with a baseline framework (0) in which we only substitute the base depth model in the refiner branch in PR with the lightweight encoder. While the inference time and model size are drastically reduced, quality is also degraded by large margin. Simply adding more parameters to scale up the F2C cannot improve the performance, as shown in @ Adopting an end-to-end training strategy can help improve the model performance (3). With the help of our proposed C2F that denoises the refiner features effectively, as shown in Fig. model's RMSE is reduced by 12.2% while only introducing a satisfactory overhead (3) 4 ). We also adopt different variants for C2F to evaluate the effectiveness. Firstly, we remove the GDU so that the C2F degrades to simple bottom-to-top aggregation module (@). While it can still improve the model performance, there is a large margin compared with the complete C2F module. Then, replace the GDU with the fusion module used in F2C (D). This results in significant drop in performance. We argue this is due to the coarse features dominating the fusion process. The nigh-frequency information cannot be preserved correctly, leading to performance on par with @.
the
8

---

Published as a conference paper at ICLR 2026
Table 2: Ablation Study of Design Choice. F2C and C2F denote the fine-to-coarse and coarse-to- fine module in the bi-directional fusion module, respectively. E2E, NP, GM, and wins. are short for end-to-end training. noisy pretraining, gradient matching, and local windows, respectively. Time: average inference time of the refiner branch for one image. ranking and SSI are the major contribu-
tions from PRV1 Li et al. 2024b).
(a) Ablation Study on UnrealStereo4K.
(b) Ablation Study on Cityscapes.
Method
RMSE
#param.
T(s)
Method
RMSE
F1
cuans baseline
9.097
19.15
Coarse Baseline
1,289
F2C
B26
C2F
F2C
E2E
C2F
NP
8.890
22.27
1,201
8.849
22.87
1.214
70.2M
8,513
27.98
40
12
1,041
1.184
47.00l
27.5M
0.32s
0.08s
NP
rankiag
SSI
DSD Loss
GM
1,003
47.0M
0,32s
5.513
27.98
w/o CDU
8.533
78.27
replace with Patchitefiner fusion
1.202
47.0M
8.533
70.22
NP, only load encoder
1.029
47.0M
0.32s
9.022
33.47
w/o ImageNet pretraining
1.059
47.0M
0.32s
8.527
8.534
36.54
35.32
Table 3: Ablation Study of Local SSIGM on Cityscpaes. When varying window size, the number of windows is fixed to the best setting, and vice versa.
witx, size
191
Variants RMSB/1 /P
8,5287 34,28
8.323/36.00 36.04
/3.5/
/36.06
35.79
8,538/ 35.66
of wöne
20
200
300
Varizats RMSE/PP
8,534/ 35.32
8.325/ 36.43
8.532/36.52
36.52
8.527 /36.54
8,529/ / 36.51
8,529/ 36.49
When equipped with NP (*),, our model achieves the best performance with 22.2% improvement over the coarse baseline in terms of RMSE. To prove our claim in the method section, we conduct the experiment by only loading the encoder part parameters after the NP process (8). The discrepancy in performance indicates that the pretraining of C2F and F2C modules is also crucial for the model performance, which often ignored in the current depth estimation commu- nity. Then, we discard the ImageNet Deng et al. (2009) pretrained parameters for the lightweight encoder and train the entire refiner branch from scratch (9). The result validates our assumption that pretraining is crucial for the refiner. Moreover, shown in Tab. 2b the NP also plays crucial role for training a real-domain high-resolution model (④, 5).
Noisy Pretraining:
Local SSIGM for Real-Domain Dataset: Tab. 2b illustrates the performance gains achieved with our proposed local SSIGM loss. While maintaining a comparable scale RMSE to the ranking and SSI losses used in Li et al. (022b)) local SSIGM significantly improves boundary Fl scores, with gains of 25.1%. 1.. The detailed ablation study demonstrates the effectiveness of both applying the gradient matching after scale-and-shift alignment and the local window strategy (5, *). As shown in Tab. 3 we further vary the window width É and observe an optimum at 2-23: larger windows weaken the locality and boundary precision, whereas smaller windows (2 23) lack sufficient context and also degrade performance. In addition, we ablate the number of local windows. Using too few windows provides insufficient supervision, leading to worse results. Once the number of windows exceeds 100, the performance plateaus and remains comparable across settings, indicating that additional windows bring little benefit.
CONCLUSION
We presented PatchRefiner V2, an enhanced and efficient framework for high-resolution monocu- lar metric depth estimation. Building on the strengths of the original PatchRefiner, PRV2 introduces a lightweight refiner branch, dramatically improving inference speed and reducing model size. With the novel Coarse-to-Fine (C2F) module and Noisy Pretraining strategy, our framework successfully mitigates the challenges posed by noisy features and the lack pre-training of the refiner branch. Furthermore, we introduced the local Scale-and-Shii Invariant Gradient Matching (local SSIGM) loss to enhance boundary accuracy and improve synthetic- -o-reeal transfer. Our framework signif- icantly outperforms previous methods the UnrealStereo4K dataset, achieving up to 9.2x fewer parameters and 10.7x faster inference. PRV2 also demonstrates considerable improvements in depth boundary delineation on real-world datasets.
9

---

Published as a conference paper at ICLR 2026
6 ACKNOWLEDGEMENTS
The research reported in this publication was supported by funding from King Abdullah University of Science and Technology -KAUST)-Cente of Excellence for Generative AI, under award number 5940 and a gift from Google.
REFERENCES
Ibraheem Alhashim and Peter Wonka. High quality monocular depth estimation via transfer learn- ing. arXiv preprint 1111:1112..1941,, 2018.
Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. BEÎT: BERT pre-training of image transform- ers. In ICLR, 2022.
Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka. Adabins: Depth estimation using adap- tive bins. In CVPR, pp. 4009-4018, 2021.
Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka. Localbins: Improving depth estimation by learning local distributions. ECCV, pp. 480-496. Springer, 2022.
Shariq Farooq Bhat, Reiner Birkl, Diana Wofk, Peter Wonka, and Matthias Müller. Zoedepth: Zero- shot transfer by combining relative and metric depth. arXiv preprint arXiv:2302. 2023.
Aleksei Bochkovskii, AmaÂGl Delaunoy, Hugo Germain, Marcel Santos, Yichao Zhou, Stephan Richter, and Vladlen Koltun. Depth pro: Sharp monocular metric depth in less than a second. In ICLR, 2025.
Tim Brooks, Aleksander Holynski, and Alexei Efros. Instructpix2pix: Learning to follow image editing instructions. In CVPR, pp. 18392-18400 18402, 2023.
Chaoqi Chen, Weiping Xie, Wenbing Huang, Yu Rong, Xinghao Ding, Yue Huang, Tingyang Xu, and Junzhou Huang. Progressive feature alignment for unsupervised domain adaptation. In CVPR, pp. 627-636, 2019.
Yinbo Chen, Sifei Liu, and Xiaolong Wang. Learning continuous image representation with local implicit image function. In CVPR, pp. 8628-8638, 2021.
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In CVPR, pp. 3213-3223, 2016.
Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In CVPR, pp. 5828- 5839, 2017.
Riccardo De Lutio, Alexander Becker, Stefano D' Aronco, Stefania Russo, Jan D Wegner, and Kon- rad Schindler. Learning graph regularisation for guided super-resolution. In CVPR, pp. 1979- 1988, 2022.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. CVPR, pp. 248-255. leee, 2009.
Raul Diaz and Amit Marathe. Soft labels for ordinal regression. In CVPR, pp. 4738-4747, 2019.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. An image is worth 16x 16 words: Transformers for image recognition at scale. In ICLR, 2021.
David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from single image using multi-scale deep network. NewrIPS, 27, 2014.
Rizhao Fan, Matteo Poggi, and Stefano Mattoccia. Contrastive learning for depth prediction. In CVPRW, pp. 3225-3236, 2023.
10

---

Published as a conference paper at ICLR 2026
Huan Fu, Mingming Gong, Chaohui Wang, Kayhan Batmanghelich, and Dacheng Tao. Deep ordinal regression network for monocular depth estimation. In CVPR. pp. 2002-2011, 2018.
Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The kitti dataset. IJRR, (2(11):2231-2377 2013.
Clément Godard, Oisin Mac Aodha, Michael Firman, and Gabriel Brostow. Digging into self- supervised monocular depth estimation. ICCV, pp. 3828-3838, 2019.
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In ICCV, pp. 2961- 2969, 2017.
Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang. Yukun Zhu, Ruoming Pang, Vijay Vasudevan, et al. Searching for mobilenetv3. In CVPR, pp. 1:11--1244 2019.
Mu Hu, Wei Yin, Chi Zhang, Zhipeng Cai, Xiaoxiao Long, Hao Chen, Kaixuan Wang, Gang Yu, Chunhua Shen, and Shaojie Shen. Metric3d v2: A versatile monocular geometric foundation model for zero-shot metric depth and surface normal estimation. IEEE TPAMI, 2024.
Tak-Wai Hui, Chen Change Loy, and Xiaoou Tang. Depth map super-resolution by deep multi-scale guidance. In ECCV, pp. 353-369. Springer, 2016.
Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Metzger, Rodrigo Caye Daudt, and Konrad Schindler: Repurposing diffusion-basse image generators for monocular depth estimation. In CVPR. pp. 9492-9502, 2024.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan Yen Lo, et al. Segment anything. In ICCV, pp. 4015-4026, 2023.
Byeongjun Kwon and Munchurl Kim. One look is enough: Seamless patchwise refinement for zero- shot monocular depth estimation on high-resolution images. In ICCV, pp. 8077-8087, October 2025.
Jae-Han Lee and Chang-Su Kim. Multi-loss rebalancing algorithm for monocular depth estimation. In ECCV, pp. 785-801. Springer, 2020.
Zhengqi Li and Noah Snavely. Megadepth: Learning single-view depth prediction from internet photos. CVPR. pp. 2041-2050, 2018.
Zhenyu Li, Zehui Chen, Xianming Liu, and Junjun Jiang. Depthformer: Exploiting long-range correlation and local information for accurate monocular depth estimation. Machine Intelligence Research, pp. 1-18, 2023.
Zhenyu Li, Shariq Farooq Bhat, and Peter Wonka. Patchfusion: An end-to-end tile-based framework for high-resolution monocular metric depth estimation. In CVPR, pp. 10016-10025, 2024a.
Zhenyu Li, Shariq Farooq Bhat, and Peter Wonka. Patchrefiner: Leveraging synthetic data for real-
domain high-resolution monocular metric depth estimation. In ECCV, pp. 250-267, 2024b.
Zhenyu Li, Xuyang Wang. Xianming Liu, and Junjun Jiang. Binsformer: Revisiting adaptive bins for monocular depth estimation. IEEE TIP. 33:3964-3976. 2024c.
Guosheng Lin, Anton Milan, Chunhua Shen, and Ian Reid. Refinenet: Multi-path refinement net- works for resoluttotttion semantic segmentation. In CVPR, pp. 1925-1934, 2017. 2017.
Ce Liu, Suryansh Kumar, Shuhang Gu, Radu Timofte, and Lue Van Gool. Single image depth prediction made better: A multivariate gaussian take. In CVPR, pp. 17346-1779 11346617336, June 2023a.
Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl Vondrick. Zero-1-to-3: Zero-shot one image to 3d object. In ICCV, pp. 9298-9309, 2023b.
Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer. Trevor Darrell, and Saining Xie. A convnet for the 2020s. In CVPR, pp. 11976-1 1986, 2022.

---

Published as a conference paper at ICLR 2026
Nando Metzger, Rodrigo Caye Daudt, and Konrad Schindler. Guided depth super-resolution by deep anisotropic diffusion. In CVPR, pp. 18237-18246, 2023.
S Mahdi H Miangoleh, Sebastian Dille, Long Mai, Sylvain Paris, and Yagiz Aksoy. Boosting monoc- ular depth estimation models to high-resolution via content-adaptive multi culli-reesoluion merging. In CVPR. pp. 9685-9694 2021.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes neural radiance fields for view synthesis. Communications of the ACM, 65(1):99-106, 2021.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. In ICIR,, 2025.
Ege Ozguroglu, Ruoshi Liu, Dídac Surís, Dian Chen, Achal Dave, Pavel Tokmakov, and Carl Von- drick. pix2gestalt: Amodal segmentation by synthesizing wholes. In CVPR, pp. 3931-3940.
IEEE Computer Society, 2024.
Giuseppe Pastore, Fabio Cermelli, Yongqin Xian, Massimiliano Mancini. Zeynep Akata, and Bar- bara Caputo. A closer look at self-training for zero-label semantic segmentation. In CVPR, pp. 2693-2702, 2021.
Andra Petrovai and Sergiu Nedevschi. Exploiting pscudo labels in a self-supervised learning frame- work for improved monocular depth estimation. In CVPR, pp. 1578-1588, 2022.
Duc-Hai Pham, Tung Do, Phong Nguyen, Binh-Son Hua, Khoi Nguyen, and Rang Nguyen. Sharpdepth: Sharpening metric depth predictions using diffusion distillation. In CVPR, pp. 17060-17069, 2025.
Luigi Piccinelli, Christos Sakaridis, and Fisher Yu. idisc: Internal discretization for monocular depth estimation. In CVPR, pp. 21477-21487, 2023.
Florentin Poucin, Andrea Kraus, and Martin Simon. Boosting instance segmentation with synthetic data: A study to overcome the limits of real world data sets. In Int. Conf. Comput. Vis. Worksh. pp. 905-9455555 953, 2021.
Dong-Hyun Lee Pseudo-Label. The simple and efficient semi-supervised learning method for deep neural networks. In ICML 2013 Workshop: Challenges in Representation Learning, pp. 1-6, 2013.
Danfeng Qin, Chas Leichner, Manolis Delakis, Marco Fornoni, Shixin Luo, Fan Yang, Weijun Wang, Colby Banbury, Chengxi Ye, Berkin Akin, et al. Mobilenetv4-uniierrsll models for the mobile ecosystem. In ECCV, 2024.
Aakash Rajpal, Noshaba Cheema, Klaus Illgner-Fehns, Philipp Slusallek, and Sunil Jaiswal. High- resolution synthetic rgb-d datasets for monocular depth estimation. CVPR, pp. 1188-1198, 2023.
René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vision transformers for dense prediction. In ICCV, pp. 12188, 2021.
René Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. IEEE TPAMI, 44(3), 2022.
Mike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar, Miguel Angel Bautista, Nathan Paczan, Russ Webb, and Joshua M Susskind. Hypersim: A photorealistic synthetic dataset for holistic indoor scene understanding. In ICCV, pp. 10912-10922, 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In CVPR, pp.
2022.
12

---

Published as a conference paper at ICLR 2026
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedi- cal image segmentation, 2015.
Kuniaki Saito, Yoshitaka Ushiku, and Tatsuya Harada. Asymmetric tri-training for unsupervised domain adaptation. In International Conference on Machine Learning, pp. 299872997.. PMLR, 2017.
Daniel Scharstein, Heiko Hirschmüller, York Kitajima, Greg Krathwohl, Nera Nešić, Xi Wang, and Porter Westingg High-resolution stereo datasets with subpixel-accurate ground truth. In Pattern Recognition: 36th German Conference, GCPR 2014, Münster, Germany, September 2-5, 2014, Proceedings 36, pp. 31-42. Springer, 2014.
Thomas Schops, Johannes L Schonberger, Silvano Galliani, Torsten Sattler, Konrad Schindler, Marc Pollefeys, and Andreas Geiger. A multi-view stereo benchmark with high-resolution images and multi-camera videos. In CVPR, pp- 3260-3269, 2017.
Inkyu Shin, Yi-Hsuan Tsai, Bingbing Zhuang, Samuel Schulter, Buyu Liu, Sparsh Garg, In So Kweon, and Kuk-Jin Yoon. Mm-tta: multi-modal test-time adaptation for 3d semantic segmenta- tion. In CVPR, pp. 16928-16937, 2022.
Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and sup- port inference from rgbd images. In ECCV, pp. 746-760. Springer, 2012.
Shuran Song, Samuel P Lichtenberg, and Jianxiong Xiao. Sun rgb-d: A rgb-d scene understanding benchmark suite. In CVPR, pp. 567-576, 2015.
Mingxing Tan and Quoc Le. EfficientNet: Rethinking model scaling for convolutional neural net- works. In ICML, volume 97, pp. 6105-6114, 2019.
Fabio Tosi, Yiyi Liao, Carolin Schmitt, and Andreas Geiger. Smd-nets: Stereo mixture density networks. In CVPR. pp. 8942-8952, 2021.
Jesper E Van Engelen and Holger H Hoos. A survey on semi-supervised learning. Machine learning. 109(2)3373-440, 2020.
Ruicheng Wang, Sicheng Xu, Cassie Dai, Jianfeng Xiang, Yu Deng, Xin Tong, and Jiaolong Yang. Moge: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision. In CVPR, pp. 5261-5271, 2025.
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE TIP, 13(4):600-613 2004.
Ke Xian, Chunhua Shen, Zhiguo Cao, Hao Lu, Yang Xiao, Ruibo Li, and Zhenbo Luo. Monocular relative depth perception with web stereo data supervision. In CVPR, pp. 311-320, 2018.
Ke Xian, Jianming Zhang, Oliver Wang, Long Mai, Zhe Lin, and Zhiguo Cao. Structure-guided ranking loss for single image depth prediction. In CVPR, pp. 61 1-620, 2020.
Guangkai Xu, Yongtao Ge, Mingyu Liu, Chengxiang Fan, Kangyang Xie, Zhiyue Zhao, Hao Chen, and Chunhua Shen. What matters when repurposing diffusion models for general dense percep- tion tasks? In ICLR, 2025.
Guanglei Yang, Hao Tang, Mingli Ding, Nicu Sebe, and Elisa Ricci. Transformer-base attention networks for continuous pixel-wise prediction. In ICCV, pp. 16269-16279, 2021.
Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. NeurlPS, 37:21.75-21911.
Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In CVPR, pp. 10371-10381, 2024.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, pp. 3836-3847, 2023.
Zhiwei Zhong, Xianming Liu, Junjun Jiang, Debin Zhao, and Xiangyang Ji. Guided depth map super-resolution: survey, ACM Computing Surveys, 2023.
13

---

Published as a conference paper at ICLR 2026
Input
Zoc+PRV1
Zoc+PRV2Mr
Figure 5: Qualitative Comparison with PRV1. Images are from the internet. Though our PRV2 is 7.6x smaller and 4.5x faster than RRV1, it achieves satisfactory and comparable results. Zoom in to better perceive details near boundaries.
A DATASET
UnrealStereo4K (Synthetic, 4K): The UnrealStereo4K dataset Tosi et al. 2022 consists of syn- thetic stereo images with resolution of 2160x3840 pixels, each paired with precise, boundary- complete pixel-wise ground truth. Images with labeling inaccuracies are excluded based on the Structural Similarity Index (SSIM) Wang et al. (2004). process adapted from Li et (2024a b). Ground truth depth maps are computed from the provided disparity maps using specific camera parameters. Consistent with the splits suggested in Tosi et al. (202T): Li et al. 2004a b), the experi- ments utilize patch size of 540 pixels for fair comparison.
CityScapes (Real): The CityScapes dataset Cordts et al. (2016) provides a diverse collection of urban scene images, segmentation masks, and disparity maps at resolution of 1024x2048 pixels. This dataset surpasses many in its domain in terms of image density, volume, and resolution Sil- berman al. (2012); Song al. (2015); [Schops al. (2017); Scharstein al. (2014). For experiments, we use a standard patch size of 256x pixels, primarily focusing on this dataset for testing our models following Li et al. (0024b).
14

---

Published as a conference paper at ICLR 2026
Table 4: Ablation study of PRV2g on UnrealStereo4k. F2C and C2F denote the fine-to-coarse and coarse-to- to-nee module in the bi-directional fusion module, respectively. E2E and NP are short for end-to-end training and noisy pretraining. Time: average inference time of the refiner branch for one image.
Method
RMSE
#param.
T(s)
Course Baseline
1.289
F2C
F2F
C2F
NP
1.118
51.7M
0.29%
95.6M
1.100
51.7M
0.29
0.985
7M
0.57s
0.947
72.1M
0.57s
Table 5: Ablation study of PRV2c on UnrealStereo4K.
Method
RMSE
#param.
T(s)
Coarse Baseline
1.289
F2C
EZB
C2F
NP
L,095
226.9M
0.38s
2
1.151
270.9M
0.61s
1.089
226.9M
0.38s
0.966
245 8M
0.62s
0.883
245.8M
0.62s
B QUALITATIVE COMPARISON WITH PRVI
We present the qualitative comparison with PRV1 in Fig. Though our PRV2M is 7.6x smaller and 4.5x faster than RRVI, it achieves satisfactory and comparable results.
C ABLATION STUDY
Framework Design: We present more ablation studies about framework design based on PRV2 and PRV2c as shown Tab. 4 and Tab. 5 respectively. we start with a baseline framework (0) in which only substitute the base depth model in the refiner branch in PR with the lightweight encoder. While the inference time and model size are drastically reduced, quality is also degraded. Simply adding
parameters to scale up the F2C cannot improve the performance, shown in @ Adopting an end-to-end training strategy can help improve the model performance (3). With the help of our proposed C2F that denoises the refiner features effectively, the model's RMSE is reduced while only introducing a satisfactory overhead 3 ①), When equipped with the NP (*), ), our model achieves the best performance.
PRV2 with Stronger Base Model: We fine-tuned DepthPro Bochkovskii et al. 2025) separately on
UtrealStereo4KK and CityScapes using the same training protocol as PRV2. Integrating our PRV2 refinement module (e.g., PRV2 on top of DepthPro as the coarse branch yields notable gains on both trained domain and zero-shot context (see Tab. 6b. demonstrating that our approach is comple- mentary to existing SOTA coarse models under a fair resource setting.
We believe our method extensible toward improving its zero-shot capability, Specifically: 1) can be scaled up using stronger coarse backbones, leading to better generalization ability. 2) It can be trained jointly across multiple real domains, benefiting from our lightweight design and the effective local SSIGM loss. We regard this promising path for future work toward high-resolution zero- shot depth estimation. (3) Tab, (bb] also suggests that our method has the potential to improve the zero-shot ability of DepthPro with the same training data.
We adopt DenseDepth Alhashim & Wonka (0018) as the coarse model and train our PRV2M. As shown in Tab. 77, our framework can consistently boost the model performance for riighressoutionn depth estimation, indicating the effectiveness of PRV2 even based on weaker coarse estimator.
Weaker Base Model:
15

---

Published as a conference paper at ICLR 2026
Table 6: PRV2 with DepthPro as Base Model.
(b) UnrealStere4K and ETH3D.
(a) Cityscapes.
Metric RMSEL
Boundary Flf
Method
u4k
ETHED
RMSEL
SEEJ
61
AboRelj.
DepthPro
7.341
29. .666
DepthPro Ours
7.257
37.51
DepthPro
1.285
0.872
93.61
0.086
DepthPro PRV2
0.824
0.692
94.12
0.077
Table Framework Performance with Weaker Base Model on UnrealStereo4K. Our frame- work can consistently boost the model performance for high-resolution depth estimation.
Method
RMSEL
SEEJ
DenseDepth Alhashim & Wonka (0188)
2.552
1.842
DenseDeptt PRV2ar
1.898
1.436
Table 8: NP v.s., Metric3D Pretrained Weights on UnrealStereo4K. It indicates the effectiveness of our NP strategy.
Method
RMSE,
SEE
Metric3D Pretrained Hu et al. 2024 ours (with NP)
0.931
0.798
0.883
0.787
NP v.S., Other Depth Pretrained Weights: In this experiment, adopt the Metric3D Hu et al. 2024) pretrained ConvNext Liu et al. (2022) as the refiner encoder. As shown Tab. g our NP demonstrates its effectiveness with 5.15 5.1% lower RMSE. Note that the Metric3D pretramndd Con- vNext is able to provide satisfactory depth-related features. Hence, such a discrepancy may indicate the importance of including the F2C and C2F modules in the pretraining stage, which is part of our NP strategy.
D LLM USAGE
We use ChatGPT to polish the paper writing.
16


Reviews:
[{"review_id": "IldGx3D97S", "reviewer": "Reviewer_k7TN", "summary": "This paper proposed PatchRefiner V2(PRV2) for real-domain high-resolution metric depth estimation. PRV2 replaces a heavyweight refiner  with lightweight encoder and adds a coarse-to-fine (C2F) block with a guided denoising unit. The paper also present noisy pretraining. Experiments on UnrealStreao4K  (synthetic) and Cityscapes (real) report the efficiency of PRV2.", "questions": "No questions", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Experimental results\nOn UnrealStereo4K, PRV2 delivers state-of-the-art accuracy while using fewer parameters and achieving faster inference than strong baselines.\n\n2. Simple design & recipe\nA modular architecture and a minimal noisy-pretraining scheme make the method easy to reproduce and extend into existing pipelines.", "weaknesses": "1. Benchmark coverage is narrow (mainly UnrealStereo4K)\nThe main SOTA claims are substantiated primarily on one synthetic dataset, while the real-domain evidence is limited to Cityscapes.\n\n2. Incomplete comparators for 2024–2025 SOTA\nUnrealStereo4K comparisons largely focus on ZoeDepth/ZoeDepth+PF/ZoeDepth+PRV1. Please add or discuss comparisons (or a justified protocol mismatch) against strong recent depth estimation methods (e.g., Marigold, SharpDepth, ...) matched resolution/compute, and clarify where a fair comparison is infeasible.\n\n3. Ablations split across datasets create interpretation friction\nCore architectural ablations (C2F/NP) are on UnrealStereo4K, while loss/boundary analyses are on Cityscapes, which makes it hard to see how each module contributes on the same data. Please add a unified ablation table on one dataset (preferably a real set) so readers can read row-wise improvements coherently\n\n4. Contribution novelty leans engineering rather than conceptual\nGDU-style gating and the NP strategy are practical and effective, but feel incremental relative to prior guided fusion paradigms\n\n5. Timing definition under-reports full pipeline cost\nThe paper defines T as the refiner-branch time per image; please also report end-to-end wall-clock (coarse + all patch refinements) and memory for a fair, reproducible comparison for follow-up research", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fn6vjrxTmu", "reviewer": "Reviewer_JEWx", "summary": "This paper presents a refined version of PatchRefiner, aiming to accelerate inference by replacing the original patch refiner with a lightweight network. The proposed approach incorporates a coarse-to-fine module, a guided denoising unit, and a noisy pre-training strategy. With these enhancements, PatchRefiner V2 achieves substantial speed improvements while delivering superior performance compared to its predecessor.", "questions": "1. Regarding the noisy pre-training strategy, it would be helpful to clarify whether the type of noise used during training influences the final performance. For instance, how would the results change if Gaussian noise were replaced with uniform noise?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well written and clearly organized.\n- The proposed improvements to PatchRefiner demonstrate both significant acceleration in inference speed and enhanced performance.", "weaknesses": "1. At Line 377, the authors state that the Cityscapes dataset is used for synthetic-to-real transfer evaluation. However, quantitative comparisons with other methods are missing in both the main text and the supplementary material, which limits the completeness of the evaluation.\n2. Only quantitative results on the in-domain UnrealStereo4K dataset are reported. Considering that the base model, ZoeDepth, is a generalizable depth estimator, it would be valuable to include experiments under cross-dataset settings to provide a more comprehensive assessment of the proposed method. \n3. At Line 365, the authors claim that local SSIGM performs better than matching gradients over the entire map. However, as shown in Table 3, the variant with zero windows does not exhibit significant performance degradation compared to local variants, and the influence of window size and number of windows on performance appears minimal. This observation reduces the perceived effectiveness of the proposed local SSIGM loss.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Iprnd78qLi", "reviewer": "Reviewer_JkXG", "summary": "This paper presents PatchRefiner V2 (PRV2), an enhanced framework for high-resolution monocular depth estimation that aims to address the computational inefficiencies of its predecessor, PatchRefiner (PRV1). The core contributions are a lightweight refiner branch, a novel Coarse-to-Fine (C2F) module with Guided Denoising Units (GDUs), a Noisy Pretraining (NP) strategy, and a local Scale-and-Shift Invariant Gradient Matching (local SSIGM) loss. The paper is well-structured, the problem is clearly motivated, and the experimental evaluation is comprehensive.", "questions": "While the term noisy features is used to describe the output of the lightweight encoder, a more precise characterization would strengthen the argument. Is this noise in the traditional sense (random, high-frequency artifacts), or is it a lack of depth-specific semantic structure? A brief quantitative analysis (e.g., using feature similarity metrics) could complement the visual evidence.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper effectively identifies the critical bottlenecks of PRV1—high inference time, large memory footprint, and the inability for end-to-end training—due to using a heavyweight base model for patch-level refinement. The motivation for replacing it with a lightweight encoder is well-justified and addresses a practical need for real-world applications.\n2.The idea of using coarse features to \"denoise\" the features from a lightweight encoder is intuitive and effective. The GDU mechanism is clearly explained and visualized showing a tangible improvement in feature quality.\n3. Noisy Pretraining is a simple yet clever strategy to force the refiner branch to learn robust, depth-relevant features from the high-resolution input itself. The ablation studies strongly validate its importance.\n4. Extending the SSI loss to the gradient domain and applying it within local windows is a thoughtful approach to improve boundary accuracy without compromising global scale. The significant improvement in the boundary F1 score on CityScapes is a key result.\n5. The paper provides thorough quantitative and qualitative evidence. The results on UnrealStereo4K are impressive, demonstrating that PRV2 can achieve state-of-the-art or comparable accuracy with a massive reduction in parameters (up to 9.2x) and inference time (up to 10.7x faster). The ablation studies are systematic and clearly demonstrate the contribution of each proposed component (C2F, NP, E2E training, local SSIGM).\n6. The inclusion of experiments on a real-world dataset (CityScapes) and the analysis of boundary quality are highly valuable and demonstrate the method's practical utility.\n7. The method is described in sufficient detail, with clear diagrams and mathematical formulations for the GDU and local SSIGM loss. The implementation details provided in Section 4.2 are adequate for reproduction.", "weaknesses": "1The GDU is a central component, but the ablation only compares it to one alternative. A more detailed analysis, for instance, comparing the proposed sigmoid-based gating to an additive fusion or an attention-based mechanism, would provide deeper insights into why the current design is optimal.\n2.  While the overall framework is much faster, the specific computational cost introduced by the C2F module and the local SSIGM loss (during training) is not discussed. A brief note on their relative overhead would be useful for readers considering implementation.\n3. The experiments are focused on UnrealStereo4K and CityScapes. While the results on CityScapes show good synthetic-to-real transfer, a brief zero-shot evaluation on other standard depth benchmarks (e.g., KITTI, NYUv2) would more strongly demonstrate the generalizability and robustness of the learned representations, especially given the use of the local SSIGM loss.\n4. The field of efficient high-resolution vision is rapidly evolving. A discussion of how PRV2 compares to other contemporary lightweight or patch-based refinement approaches (beyond PRV1 and PatchFusion) would better situate its contributions within the current research landscape.\n5. Some arxiv papers are actually published on important conferences, and please cite the published information, not arxiv.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BAVb119Lxl", "reviewer": "Reviewer_pdNs", "summary": "This paper extends PatchRefiner (ECCV 2024) for high-resolution monocular metric depth estimation. PatchRefiner V2 replaces the heavy refinement encoder with a lightweight alternative to significantly reduce runtime and memory usage during both training and inference. To compensate for the reduced capacity, the method introduces a coarse-to-fine module that injects coarse-level features to guide and denoise refinement features, along with a noisy pretraining strategy to improve the refiner's robustness. The paper also adopts an improved local-window SSIGM loss to enhance synthetic-to-real transfer. Experiments on UnrealStereo4K and Cityscapes demonstrate improved accuracy and efficiency compared to the original PatchRefiner.", "questions": "Please refer to the weaknesses.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "- The proposed PatchRefiner V2 achieves impressive improvements in runtime and memory efficiency over the original PatchRefiner, while also achieving equal or better accuracy. This makes high-resolution monocular depth estimation more feasible for practical use.\n\n- The coarse-to-fine module is well-motivated and effectively compensates for the reduced capacity of the lightweight refiner. The guided denoising design is clearly presented, and its contribution is supported by thorough ablation studies.\n\n- The noisy pretraining strategy is simple yet empirically effective. Pretraining the refinement branch with randomized coarse features leads to a more robust model without requiring additional external data or complex setups.\n\n- The local SSIGM loss provides more precise supervision. Enforcing scale-and-shift consistency at a local spatial level results in sharper depth boundaries and avoids introducing scale error.\n\n- The paper is well-written and easy to follow, with clear figures and architectural diagrams that effectively explain the design choices.\n\n- The visual annotations (e.g., the snails and lightning bolts in Fig. 1) clearly and intuitively highlight the performance and efficiency differences. The visualization of features in Fig. 2 clearly presents the motivation and effectiveness of the fusion model. These figures improved readability and made the narrative smoother and more engaging.", "weaknesses": "- Discussion of Related Work Could Be Expanded.\nThe motivation and effectiveness of the proposed coarse-to-fine module are clearly presented, and the design is well-justified. However, there are existing two-branch feature fusion strategies in related areas (e.g., [1] and [2]). While these works focus on different tasks and modalities, a brief discussion comparing the design philosophy or fusion flow direction could further clarify the novelty of the proposed C2F module and situate the contribution more explicitly in the broader literature.\n\n[1] Bi-SSC: Geometric-Semantic Bidirectional Fusion for 3D Scene Completion\n\n[2] FFB6D: Full Flow Bidirectional Fusion for 6D Pose Estimation\n\n- Clarification of PRV2’s Advantage Over High-Resolution Backbone Models.\nThe improvement of PRV2 over DepthPro is quite substantial, which strongly supports the value of the refinement design. Since DepthPro is already a high-resolution metric depth model, it would be helpful for the paper to provide a bit more insight into why PRV2 achieves such notable gains when refining DepthPro outputs. \n\n- While PRV2 is not intended to be a general-purpose “zero-shot” depth refiner, a short discussion of the expected generalization behavior could help guide future follow-up work aiming toward more foundational refinement pipelines.\n\n\n- There is a small typo in Table 2: in the caption, “GM and wins.” can be removed for clarity.\n\n- Since the updated local-window SSIGM loss is one of the key improvements, adding a brief pseudo-code snippet in the supplement would make re-implementation easier. This would improve the usability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)