Hyden: A Hybrid Dual-Path Encoder for Monocular Geometry of High-resolution Images
Authors: Zaiwei Zhang
Marc Mapeke
Wei Ye
Rakesh Ranjan
JQ Huang
Venue: ICLR 2026 Poster (2026)
Keywords: 3D vision
Monocular Depth Estimation
Monocular Surface Normal Estimation
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We propose a hybrid dual-path encoder design for monocular geometry prediction of high-resolution images, which achieves SOTA results with low inference latency.

Abstract: We present a hybrid dual-path vision encoder (Hyden) for high-resolution monocular depth, point map and surface normal estimation, surpassing state-of-the-art accuracy with a fraction of the inference cost. The architecture pairs a low-resolution Vision Transformer branch for global context with a full-resolution CNN branch for fine details, fusing features via a lightweight MLP before decoding. By exploiting the linear scaling of CNNs and constraining transformer computation to a fixed resolution, the model delivers fast inference even on multi-megapixel inputs. To overcome the scarcity of high-quality high-resolution supervision, we introduce a self-distillation framework that generates pseudo-labels from existing models at both lower resolution full images and high-resolution crops—global labels preserve geometric accuracy, while local labels capture sharper details. To demonstrate the flexibility of our approach, we integrate Hyden and our self-distillation method into DepthAnything-v2 for depth estimation and MoGe2 for surface normal and metric point map prediction, achieving state-of-the-art results on high-resolution benchmarks with the lowest inference latency among competing methods.

Published as a conference paper at ICLR 2026
HYDEN: A HYBRID DUAL-PATH ENCODER FOR MONOCULAR GEOMETRY OF HIGH-RESOLUTION IMAGES
Zaiwei Zhang Marc Mapeke & Wei Ye & Rakesh Ranjan JQ Huang Meta Reality Labs
322 Airport Blvd. Burlingame, CA 94010, USA { aza Paawweiie thange mmapeke, wei rakeshr, jqhuang) @meta com
ABSTRACT
We present hybrid dual-path vision encoder (Hyden) for high-resolution monoc- ular depth, point map and surface normal estimation, surpassing state-of-the-art accuracy with fraction of the inference cost. The architecture pairs a low- resolution Vision Transformer branch for global context with full-resolution CNN branch for fine details, fusing features via a lightweight MLP before de- coding. By exploiting the linear scaling of CNNs and constraining transformer computation to a fixed resolution, the model delivers fast inference even on multi- megapixel inputs. To overcome the scarcity of high-quality high-resolution su- pervision, we introduce a self-distillation framework that generates pseudo-labels from existing models at both lower resolution full images and high-rese regoutioolu crops-global labels preserve geometric accuracy, while local labels capture sharper details. To demonstrate the flexibility of our approach, we integrate Hy- den and our self-distillation method into DepthAnything-v2 v2 for depth estimation and MoGe2 for surface normal and metric point map prediction, achieving state- of-the-art art results on high-resolution benchmarks with the lowest inference latency among competing methods.
INTRODUCTION
Monocular depth, pointmap, and surface normal estimation are core 3D perception in driving, robotics, and mixed reality. Models like MiDaS Ranfil et al. (2020) and DepthAnything Yang el al. (2024) show strong results from a single RGB image, but most are trained at low resolution, causing degraded predictions on megapixel inputs Wang et al. (2025).
To close this gap. recent work partitions images into tiles and blends ViT features (Depth- Pro |Bochkovkii et al.) (2024), PatchFusion Li et al.] (2024b)) designs multi-branch ViTs (FlashDepth Chou et al. (2025). Yet, VIT inference scales quadratically with resolution. Super- vision is also problematic: real high-res supervisions are often noisy sparse, while synthetic labels are perfect but introduce domain gaps, making model generalization difficult.
To address these issues we present Hyden-a Hybrid Dual-path Encoder coupling full-resolution CNN with a low-resolution VIT. CNN features preserve local detail, while upscaled VIT tokens provide global context, fused through lightweight layers before task-specific decoding. This design substantially lowers inference latency while maintaining sharp predictions.
For supervision, Hyden uses self-distillation unlabeled high-resolution images are pseudo-labeled by a frozen teacher at 518 x 518 for both full images and high-res crops. The original ViT branch is kept frozen, and only the CNN branch, fusion layer, and decoder are optimized using both global loss (on the downsampled full image) and a local crop loss (on masked crop regions).
Our contributions is summarized as follows:

---

Published as a conference paper at ICLR 2026
Inpat resolution 3188318
Input reselution
Input resolution
20124a122
(a) Example inferences with Hyden-DA2 (row 1), Hyden-MoGee-Normal (row 2), Hyden-MoGe2- Pointmap (row 3) across inputs with different resolutions
Teachopth
Hydee PUci2
Modm?
(b) Geometry Accuracy (1) (c) Geometry Sharpness (L) (d) Latency (log-scale) (L)
Figure I: Performance comparison across inference resolutions for geometric foundation mod- els: (a) Example inferences with relative depth, surface normal and point map prediction models illustrate the tradeoff between latency and sharpness across resolutions.
(b) shows average rank- ing across datasets for relative depth accuracy, (c) reports average ranking for geometry sharpness across depth models, and (d) plots inference latency (log-scale) measured on an NVIDIA A100 GPU with FP16 precision. Lower is better for all plots. Compared to base models, both Hyden-DA2 and Hyden-MoGe2 achieve improved accuracy at high-resolution inference and deliver significant inference speedups. Hyden-MoGe2 achieves the best geometry accuracy and sharpness compared to other state-of-the-art models and consumes significantly lower inference latency. (*DepthPro evaluated fixed resolution due model constraint.)
We introduce Hyden, the first encoder that combines a fixed-resolution ViT for global con- text and full-resolution CNN for fine detail, significantly reducing inference cost while preserving high-resolution accuracy.
We propose self-distillation framework that uses global pseudo-labels to preserve accurate geometry across resolutions and local pseudo-labels to capture sharper fine details in high- resolution predictions.
By integrating the Hyden encoder into two leading models: DepthAnything-v2 Yang al. (2024) and MoGe2 Wang et al. (2025)-om) approach establishes new state-of-the-an performance for high-resolution depth, point map, and surface-normal prediction, while achieving average 3x lower inference latency at 2K and nearly 10x lower at 4K resolution compared the original models (see Figure
RELATED WORK
2.1 ZERO-SHOT MONOCULAR GEOMETRY ESTIMATION
Traditional monocular models Bhat et al. 2021; Eigen et al. 2014): Li et al. 2022 Eigen & Fergus (2015); (axena et al, (2008) were traied on single datasets for specific domains (e.g., indoor street-view) and generalized poorly due to limited diversity and fixed camera setups.
2

---

Published as a conference paper at ICLR 2026
Relative depth To improve generalization, MegaDepth Li & Snavely (2018) and Di- verseDepth Yin et al. (2020) scaled supervision with Internet-scale data. MiDaS Ranftl et al. (2020) introduced scale- and shift-invariant losses, later extended with transformers Ranfil al. (2021);
Birkl et al. (2023). Dnythnng Yang et al. (0224) distilled pseudo labels for 62M images, while
generative priors adapted diffusion models Ke et al. (2024); RRombach al. (2022) joint atten-
tion Fu et al. (022)). These methods generalize broadly but remain limited by scale/shift ambiguity.
Metric depth Scarce metric annotations hinder absolute scale. ZoeDepth Bhat et (2033) ) fine- tunes metric heads on relative models. Metric3D Hu et (2024) and DepthPro Bochkovskii et al. (2024) resolve cross-camera ambiguity via canonical transformations. UniDepth Piccinelli et al. 2025 learns implicit camera models, while MoGe2 Wang et al. (2025) predicts scale-invariant
pointmaps with scale recovery.
Metric point map Another direction is predicting 3D pointmaps. Many works Yin ct al. (2021);
Piccinelli et al. (2025); Hu et al. (2024); BBcckooskii et al. (2024) decouple depth and camera
recovery, e.g., LeRes Yin et al.
2021) regresses depth and intrinsics, UniDepth |Piccinelli et al.)
(2024) uses camera embeddings. DUSI3R Wang et al. (2024) predicts stereo pointmaps end-to-end,
and MoGe2 Wang et al. (2025) combines scale-invariant maps with scale factors.
Despite progress, most models are trained at low resolution (S 518 x 518), losing detail when downsampled and incurring high cost at full scale.
2.2 ZERO-SHOT SURFACE NORMAL ESTIMATION
Normals avoid metric ambiguity and capture local shape for localization |Bhhley & Stachniss
(2018)
mapping Wang et al. (2019), and reconstruction Yu et al. (2022): Wang et al. (2022). p. Early work
derived them from RGB-D scans Silberman et al. (2012); Eigen & Fergus (201 5):55: Qi al. (2020)
and denoised via consistency Qi et al. (2018) 88, adaptive constraints Long et al. (024 2021 I), or un- certainty Bae et al. (202T) OmnilDta Eftekhar et al. (202T) scaled to 1.3B frames, while Normal-in- Hie-Wild|on Chen et al. (2017) expanded to outdoor scenes. DSINE Bac & Davison (2024) introduced a normals-specific architecture, and recent transformer-batt approaches Hu et al. (2024): ) Wang
al. (2025) unify depth, normals, and pointmaps. Yet most remain constrained low-resolution training reducing sharpness.
2.3 HIGH-RESOLUTION DEPTH AND SURFACE NORMAL ESTIMATION
To recover fine details, SMD-Net Tosi et al. 2021) and Toisson-fusiog Dai et al. (2023): ; Li et al. (2024a) sharpen boundaries, while patch pipelines BoostingDepthh Miangolch et (2021). Patch- Fusion Li et al. (2024b), PatchRefiner Li et al (2024c)-boos local detail but introduce artifacts and latency. PRO Kwon Kim (2025) cuts computation but lags end-to-end models Bochkovskil et al. 2024]; Chou et al. (2025). DepthPro Bochkovskil et al. (2024) improves patch efficiency, and
Chou
Dephh Chou et (2025) uses buaa-b ViTs, though both rely on refinements or synthetic pretraining.
In contrast, our Hyden framework integrates a full-resolution CNN with low-resolution VIT and self-distillation, leveraging CNNs' linear scaling to produce sharp, efficient, and generalizable pre- dictions for depth and normals at megapixel scales.
APPROACH
3.1 HYBRID DUAL-PATH VISION ENCODER
Our architecture employs hybrid dual-path encoder that combines low-resolution Vision Trans- former (ViT) with a full-uesolution CNN to balance global context and fine-detail preservation (Fig- The VIT branch processes uniformly downsampled input (up 518 x 518) capture long- range dependencies at constant cost, leveraging any pretrained backbone (c.g., DepthAnything Yang et al. (2024)).
ure2) 2.
3

---

Published as conference paper at ICLR 2026
VIT Encoder
Feature Maps
CLS Feature
DPT Decoder
(DA2)
ConvBlock
(MoGe-v2)
Bilinear Upsampling
Downsample
Multi-stage Fused Features
Aggregated Fused Features
Feature Fusion Network
Decoder
Decoder
T
Original resolution
CNN
Encoder
Feature Pyramid (Coarse to Fine)
Pooled Feature
Relative Depth
Metric Points
Surface
Hybrid Dual-path Vision Encoder
Different Fusion Network Designs
The ViT encoder takes in down-sampled images while the CNN encoder takes in images with original resolution. To recover the high resolution features, the target ViT feature maps are upsampled with bilinear interpolation. CNN and VIT feature maps will be concatenated and fed into a feature fusion network. The fused features are used for down-stream tasks. Depending on the decoder architecture, the fusion logic needs to be slightly modified. For example, DA2 (DepthAnything-v2) uses all intermediate features from ViT and we fuse the cor- responding CNN layers for each ViT features. MoGe2 uses an aggregated VIT feature map and similarly we aggregate multi-scale CNN feature maps with upsampling and concatenation, and only fuse the aggregated CNN and VIT features. For global-level feature, apply average pooling to the CNN maps and concatenate the result with the CLS token for downstream tasks.
Figure 2: Network Architecture:
In parallel, the CNN branch directly processes the full-resolution image, efficiently extracting high- frequency features such edges and textures. We adopt a ResNet-like encoder with hierarchical downsampling stages.
The two streams are fused by upsampling VIT features the CNN resolution, concatenating them, and applying a lightweight two-layer convolution. This enables joint reasoning over global trans- former context and local detail with minimal overhead.
A key advantage is scalability: VIT cost remains fixed while CNN scales linearly with resolution, enabling efficient inference on multi- magtiimeegapixl images-unlics unlike pure-ViT VIT models with quadratic cost. The encoder is modular and task task-agnostic, and we integrate it into DepthAnything! jtt al. (2024) MoGe-Vi V2 Wang al. (2025) with minimal modifications to the fusion logic, preserving resolution robustness and efficiency (see supplemental materials).
Yang
3.2 SELF-DISTILLLTIOON TRAINING
Motivation. High-resolution supervision difficult obtain in practice: real datasets rarely pro- vide dense, clean depth or surface-normal labels at megapixel scales due to hardware and annotation constraints, while synthetic datasets introduce a domain gap relative to real imagery. general training pipeline that upgrades an existing depth, point map surface normal model to our hybrid qual-path encoder and scales gracefully to high-resolution inputs- we introduce self- distillation framework.
build
Overview. From set of unlabeled high-resolution images { we generate pseudo labels with a target model (e.g., a strong zero-shot predictor). We extract (i) global labels from the down- sampled full image (518 x 518), and (ii) local labels from 518 518 crops, which recover sharper details. Local labels may vary in scale and shift, so we align them to the global prediction before training (Figure 3)

---

Published as a conference paper at ICLR 2026
Label Generation
Model Training
Hyden
Model
Downsampled
Rendom Crops
Original Resolution
Target Model
Inference lower resolution
Global label
labels
Global loss: Ensure geometry accuracy with global label
Local crop loss: Improve prediction sharpness with local label
Globally accurate
Globally accurate
Sharpness
Sharpness
Figure 3: Self-distillation: (1) Label generation: Our pipeline samples multiple low-resolution views of the input, using downsampling for global context and random cropping for local details. The target model produces predictions on each view, which are mapped back to the original reso- lution via up-sampling indexing. This yields pseudo labels that preserve geometric accuracy and local sharpness. (2) Model training: Using hyden encoders, models are trained on high-resolution inputs their native resolution. global loss is applied on downsampled predictions with global labels to retain geometry, while local loss on full-resolution predictions enhances sharpness.
We then replace the target model's encoder with our Hyden encoder, adapt the decoder, and freeze the ViT training only the CNN branch, fusion module, and decoder. Since the ViT sees only the downsampled view, it requires no additional fine-tuning.
Notation. Let I E 3xxx33 be a high-resolution image and S denote the fixed low resolution (S=518), Denote by Is (.) uniform downsampling to SxS, and by crop the k-th high- resolution crop operator with spatial support S4 C (1.... H) x [1, ,..... W); its resized version to SxS is r.)op, The teacher T produces global pseudo labels y = T (7)) and local pseudo labels y = (rcrrop (!)). Our student (hybrid) network Fa outputs a dense prediction y Fo()) the native resolution.
3.2. TASK-SPECIFIC LOSSES
We use the original task objectives of the target models for both global and local supervision.
Relative depth sccale/hift-invariinn Given predicted depth d and teacher depth on a pixel set M, align the prediction by scale and shift
(add + (adp+b-d))
a" b arg min a.b [M
PEM
and compute robust alignment loss (e.g., l1):
Eeepth (d, d; M)
a dbdd -d
[M
PEM
Surface normals (angular/cosine), For predicted unit normals ny and teacher normals ngg.
Gnormal (n, M)
Σ (1 (mpp:ny)
(2)
[M
5

---

Published as a conference paper at ICLR 2026
Affine-ivvariant point map loss. Let Poff R be the predicted affine- invariant point map and Paf the teacher/GT affine-invariant point map for pixel p E M. The affine-invariant point map loss is:
ppoint pall (palf ;M,;M)
Σ PEM
paff
paf
|M
Scale prediction loss. Given the predicted global scale 0, we supervise it against the optimal alignment scale s* between paft and the metric GT points Ppi
Cacale = ||og 8 stopgrad(log s >
(4)
Please refer to DA2 Yang et al. (2024) and MoGee Wang et al. (025 for more details in task- specific loss formulation. We write tack to denote either Edpphh or mormal depending on the task.
3.2.2 GLOBAL LOSS
We downsample the student prediction S' S and compare with the global pseudo label:
Lglobal = feastd 4s (y), y Mgg
(5)
where Mg is the valid-pixel mask the global resolution (e.g., invalid depth/normal entries re- moved).
3.2.3 LOCAL CROP LOSS
For each crop k. we compare the high-resolution student prediction with the teacher labels projected back to the crop region. Let ta, (-) denote injecting local labels at Sx S the high-resolution support Q2, and Mg be the binary mask of f2::
K Crast y, ts (yī);; Ms)
Liocal
Σ 1=1
K
where
M = M valid(tae (yT))
(6)
This formulation applies the task loss directly the native image resolution but only within each crop's support. The loss is averaged across all crops.
3.2.4 TOTAL OBJECTIVE
The final objective combines global and local terms:
Liotal Ay. Lglohal Xe Liocal,
with weights Ag, A 1. all experiments we freeze the VIT branch and optimize the CNN encoder, fusion layer, and task decoder end-to-end using Listal-
3.3 IMPLEMENTATION DETAILS
We adapt our Hyden architecture to DA2 and MoGe2, denoted as Hyden-DA2 and Hyden-MoGe2. The additional CNN encoder introduces 10M parameters, incurring only a minor computational overhead. During self distillation, we randomly resize the input images from 518 to 2072 resolution
for better geometry consistency (see Section 4.2).. We use local crops for all our self-distillation experiments. For the unlabeled high-resolution images { sampled 50 million images from a publicly available repository of crawled web data and we resized all the images to 2072x2072 resolution. We train models for 300k iterations with batch size 192 64 NVIDIA H100 GPUs. We use an initial learning rate of le-5 for CNN encoder and le-6 for feature fusion and decoder module. We use adam W Kinga al. (2015) optimizer and use polynomial learning rate scheduler.

---

Published as a conference paper at ICLR 2026
4 RESULTS
4.1 BASELINE AND EVALUATION METRICS
We benchmark Hyden models against DepthAnythingg22 Yang al. (2024) for relative depth and MoGe2 Wang et al. 2025) for metric-scale pointmaps and normals. We also compare with high- resolution depth methods DepthPro Bochkovskii et al. (2024) and Feash Chou et al. (2025). and with normals-focusee DSINE Bae & Davison (2024) and Metric3Dv2 Hu et al. (2024).
Evaluation Metrics For relative depth and metric pointmaps, we evaluate on 9 datasets: NYUv2 Nathan Silberman & Fergus (2012), KITTI Geiger et al. (203), ETH3D [Schoss et al. 2017), iBims-1 Koch et al. (2018) (020), DDAD Guizilini et al. (2020), DIODE Vasiljevic et al.
(2019). HAMMER Jung et al. (2023), Booster Ramirez et al. (2022), and Middlebury [Scharstein
et al. 2014). These span indoor, street-view, and object domains, with ETH3D, Booster, and Mid- dlebury providing 2K+ ground truth for high-resolution evaluation. We report the average relative
error for point maps and depth:
|p 1P-ll |pl2
2 -
Relp
Rela
along with the percentage of inliers
P pllz
d d d
8.
< 0.25,
of max
1.25.
|p/2
For surface normal, we evaluate on NYUv2 Nathan Silberman & Fergus 2012). iBims-l Koch et al. (2018 2020), Scannet[Dai et al. (2017). and vkitti Cabon et al. (2020), reporting mean angular error. For boundary sharpness, we follow MoGe2 Wang et al. (2025) ) and evaluate on iBims-1, Sintel Butler ct al. 2012), HAMMER, and Spring Mehl al. (2023).
For all evaluation benchmarks, we resize the largest side of the input images to target image resolu- tion and the predictions are evaluated the original groundtruth resolution.
Table 1: Zero-shot depth & point map accuracy. We report the average relative error (lower is better) and 82 score per dataset (higher is better) and aggregate performance across datasets via the average rank (lower is better). *DepthPro is evaluated at 1536x1536 and all other models are evaluated with 2K resolution input.
Depth
Inference Lanency
NYUv2
iBims-1 Rell
Middlebury Rell S1
HAMMER &.1
Bell
Rell
Rell
Rell 6
Rel., 65t
Rell 62t
Raakk!
Relative depth man
DA2 Vang et K00
#08. I
5.4
92.3
8.3
92.3
5.5
91.3
4.1
95.5
30
98.8
73
$7.8
15.8
82.4
5.4
98.8
6.3
96.1
4.5
DepdlPen Bocnkorake
2004
$41,3*
4.4
96.5
5.7
953
7.5
93.1
4.
96.7
3.2
98.6
92
84.8
15.1
90.1
49
94.3
5.3
9%.3
4.3
FanDepp
98.
83
5.
11.0
75.5
90.4
87.1
6.7
MaGe2 Warg 2013
90.5
Hydai-DA2 (Dus)
100.7
4.6
06.5
7.6
05.3
5.1
05.
4.1
97.8
3.0
98.7
10.4
83.2
13.4
K3.7
4.7
96.2
5.4
97.3
3.9
Hyden Matic2 (Ours)
171.6
3.7
98.5
4.9
97.8
3.5
98.3
32
98.6
2.1
9900
2.0
95.8
11.1
91.6
3.7
97.1
3.2
99.1
1.3
depth
intrinsiss)
Depthiho Bochkoroka clal. 2024
[31,33
25.8
38.1
38.6
MiGia Wing Wingerar
9.2
19.6
24.9
55.6
00.4
68.3
Hodai-Moii2 (Oass
171.6
7.7
96.1
164
86.2
18.3
85.7
12.4
91.2
20.2 47.3
34.1
99.2
17.7
71.4
36,0
64.6
1.3
point map
DepthP Bedikovaki aral.
$41.3
26.1
189
72.8
34.9
MoGe Waig crale 2043
476.8
9.7
93.8
16.8
85.7
20.3
92.6
16.2
84.6
03.3
38.3
26.5
53.4
18.0
09.6
25.2
69.5
1.8
Hylm-MMail (Ours)
171.6
8.3
95.7
17.9
83.1
19.1
94.7
14.1
90.7
50.8
49.3
25.4
$7.9
17.7
72.9
78.7
66
1.3
4.2 PERFORMANCE COMPARISON ON IMAGES WITH DIFFERENT RESOLUTIONS
We first evaluate robustness of test-time input scaling. As illustrated in Figure I at low reso- lution (518x518), DA2 Yang et al. (2024) and MoGe2 Wang et al. (2025) surpass Hyden, but above 784x784 all baselines, including Metric3Dv2 Hu et al. (202)) degrade sharply while Hy- den stays accurate. Although FeashDepth Chou al. (225)) offers relatively low inference latency,
7

---

Published as a conference paper at ICLR 2026
Hyden-DA2 325ms
DA2 413tms
FlashDepth-HHy Hybidd 79tms
aa) Relative depth maps
Hyden-MoGe2
430ms
4224ms
440mm
(b) Normal maps MoGe2
Hyden-MoGe2
DepthPro
563ms
341ms"
0.3 metera
26,1 meters
Point maps
Figure 4: Qualitative comparison of geometry predictions on in-the-wild images 4004 x 4004. Left: Hyden models; Middle: base models (DepthAnythingg22 V2, MoGe2); Right: other SOTA mod- els. Hyden models produces sharper, more accurate geometry with much lower latency. (*DepthPro is evaluated at 1536x1536 resolution due to model constraints.)
its lightweight decoder and limited supervision result in significant performance drops. Hyden mod- els maintain consistent depth and pointmap accuracy across resolutions, with only marginal latency increases. Trained on mixed resolutions (Section 5.2). it combines stability with efficient high-res inference. In particular, Hyden-MoGe) delivers the best 2K+ accuracy and low latency, and at 4K (Figure D outperforms VIT baselines while running 10x faster.
4.3 PERFORMANCE COMPARISON ON HIGH-RESOLUTION IMAGES
Zero-shot depth & point map As shown in Figure 4 Dyden-DA2 yields sharper geometry and Hyden-MoGe2 predict better metric scale at high resolution. In Table I Hyden-MoGG attains the highest average accuracy, while both Hyden variants improve accuracy and reduce latency their baselines. At 2K. Hyden-MoGel achieves the best geometric accuracy and runs ~3x faster, highlighting the effectiveness of our self-distillation for high-resolution accuracy and efficiency.
8

---

Published as a conference paper at ICLR 2026
Zero-shot surface normal As shown in Table Hyden-MoGe2 improves accuracy most benchmarks while keeping low latency, with qualitative results Figure 4 DSINE Bae & Davison (2024) runs at similar speed but trails ViT-based models. In out-of-domain tests, Hyden-MoGe2 surpasses Metric3Dy2 on three benchmarks and significantly outperforms MoGe2 at 2K resolution.
Table 2: Zero-shot surface normal accuracy. We report the mean angular errors (lower is bet- ter). *ScanNet evaluation is in-domain for Met- ric3Dv2, as it is included in the training
Table Zero-shot boundary sharpness. We report Fl score and recall for all datasets (higher is better).
Speing
Moddd
Inference
NYU1Z
[llin11
Sconnet Meon,
skdid
16.
15.3
Midel
Rank
Noc
11.3
11.2
25.3
38.7
68
59
302
17.6
57
DSINE Darion RIC
149.4
180
4.0
Deputt
Menacibo 2017
12.8
13.4
21:3
24A
49
4.0
31.1
17.5
14.1
8.0
Mesic
606.7
159
154
11.4
29.5
2.3
McGre Watreris
438.2
156
160
13.7
27.3
2.5
Hyden MaGrl (Oa))
127.4
146
148
13.0
27.0
1.2
Hyden MoGiet
289
Zero-shot boundary sharpness As shown in Table Hyden models achieve notably higher F1 and recall than their baselines while running faster at 2K. Hyden-MoGe2 further surpasses other SOTA methods without extra high-resolution supervision.
Additional results and model size comparisons are provided in the supplemental material.
ABLATION STUDY
5.1 IMPORTANCE OF LOCAL CROP LOss
As shown in Table (all models are evaluated with 2K resolution input), removing the local crop loss causes Hyden-DA2 DA2 lose sharpness due to reliance low-resolution global supervision. Increas- ing the number of crops improves sharpness, and we find that using four crops offers best trade-off' between labeling cost and model performance.
Table 4: Ablation on local crop loss. (We re- port F1 score and recall for zero-shot boundary sharpness evaluation.)
Table 5: Ablation mixed-resoluse train- ing & fusion network design.
Depch Model
NYUv2 KITTI
ETHDD HAMMER
Rell
Rell
Rell
Reli
Depth Model
iBims-
Sintel
HAMMER
Spring
Hydan- DM2 smined from 518-1036
5.14
8.83
5.27
7.10
Hyden- DA2 trined from 318-2072
4,60
7.63
5.12
5.44
Hyden-DAB DA2 wis local crop loss
11.8
184
27.9
382
78
13.1
14.7
13.8
Hyden-DAA2 cmps
14.4
20.9
318
40.5
87
168
155
14.7
Hyden-Doe DA2 Fasion
4.72
7.92
5.31
5.93
Hyden DA2
15.8
21.3
460
10.7
19.3
159
16.6
Hydee DA2 w/ L-Jayer CNN Fusion
4,63
7,88
5.22
5.87
Hyden-DAS crops
Hyden-DA2 2-layer CNN Fusion
4,60
7.63
5.12
5.44
5.2 IMPORTANCE OF MIXED-RESOLUTION TRAINING
Table 5 compares Hyden models trained with input resolutions ranging from 518-1036 and 518-2072. Matching the training resolution to the test-time resolution improves depth accuracy, highlighting the value of nexolttesolution training. However, for ViT encoders, training at very high resolutions (c.g., over 20K tokens at 2K resolution) is computationally prohibitive. By con- straining the ViT branch to low-resolution input, Hyden enables practical high-resolution training.
5.3 FEATURE FUSION NETWORK DESIGN
We evaluate several fusion designs for feature projection: a single linear layer, single CNN layer, and two CNN layers with ReLU activation. As shown in Table [S] the two-layer CNN achieves the best performance. We adopt this configuration in our model as it offers superior accuracy without incurring excessive computational overhead.

---

Published as a conference paper at ICLR 2026
6 CONCLUSION
We presented Hyden, hybrid dual-path vision encoder that delivers high-resolution depth, point map and surface-normal estimation with low latency. By combining global and local pseudo-label self-distillation, Hyden preserves geometric accuracy while enhancing fine details, without relying on high-resolution ground truth. Integrated into leading baselines, Hyden achieves state-of-the-arr accuracy across resolutions while maintaining fast inference, offering scalable solution for dense prediction tasks.
REFERENCES
Gwangbin Bae and Andrew J Davison. Rethinking inductive biases for surface normal estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9535-9545, 2024.
Gwangbin Bae, Ignas Budvytís, and Roberto Cipolla. Estimating and exploiting the aleatoric uncer- tainty in surface normal estimation. In Proceedings of the IEEE/CVE International Conference Computer Vision pp. 13137-13146, 2021.
Jens Behley and Cyrill Stachniss. Efficient surfel-based slam using 3d laser range data in urban environments. In Robotics: science and systems, volume 2018. pp. 59, 2018.
Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka. Adabins: Depth estimation using adap- tive bins. In Proceedings the pp. 4009-4018, 2021.
the IEEE/CVF conference on computer vision and pattern recognition,
Shariq Farooq Bhat, Reiner Birkl, Diana Wofk, Peter Wonka, and Matthias Müller. Zoedepth: Zero- shot transfer by combining relative and metric depth. arXiv preprint arXiv:2302. 2023.
Reiner Birkl, Diana Wolk, and Matthias Müller. Midas v3. I-a model zoo for robust monocular relative depth estimation. arXiv preprint arXiv:2307.14400 2023.
Aleksei Bochkovskii, AmaÂGI Delaunoy, Hugo Germain, Marcel Santos, Yichao Zhou, Stephan R Richter, and Vladlen Koltun. Depth pro: Sharp monocular metric depth in less than a second. arXiv preprint 2rX0022410.02073, 2024.
D. J. Butler, J. Wulff, G. B. Stanley, and M. Black. A naturalistic open source movie for optical flow evaluation. In A. Fitzgibbon et al. (Eds.) (ed.), European Conf. on Computer Vision (ECCV), Part IV, LNCS 7577, pp. 611 6-1-52.. Springer- Verlag, October 2012.
Yohann Cabon, Naila Murray, and Martin Humenberger. Virtual kitti 2, 2020.
Weifeng Chen, Donglai Xiang, and Jia Deng. Surface normals in the wild. In Proceedings of the IEEE International Conference Computer Vision, pp. 1557-1566, 2017.
Gene Chou, Wenqi Xian, Guandao Yang, Mohamed Abdelfattah, Bharath Hariharan, Noah Snavely, Ning Yu, and Paul Debevec. Flashdepth: Real-time streaming video depth estimation at 2k reso- lution. arXiv preprint 2rXiv:2504.07099 2025.
Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Rinoty-dnnotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8828-5839,200
Yaqiao Dai, Renjiao Yi, Chenyang Zhu, Hongjun He, and Kai Xu. Multi-resolution monocular depth map fusion by self superviud gradient-based composition. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 488 098.996,11 2023.
Ainaz Eftekhar, Alexander Sax, Jitendra Malik, and Amir Zamir. Omnidata: A scalable pipeline for making multi-tase mid-level vision datasets from 3d scans. In Proceedings of the IEEÉ/CVF International Conference Computer Vision, pp. 10786-10796, 2021.
10

---

Published as a conference paper at ICLR 2026
David Eigen and Rob Fergus. Predicting depth, surface normals and semantic labels with common multi-scale convolutional architecture. In Proceedings of the IEEE international conference on computer vision, pp. 2650-2658 2015.
David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from a single image using multi-scale deep network. Advances in neural information processing systems, 27, 2014.
Xiao Fu, Wei Yin, Mu Hu, Kaixuan Wang, Yuexin Ma, Ping Tan, Shaojie Shen, Dahua Lin, and Xiaoxiao Long. Geowizard: Unleashing the diffusion priors for 3d geometry estimation from single image. In European Conference on Computer Vision, pp. 241-258. Springer, 2024.
Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The kitti dataset. International Journal of Robotics Research (IJRR), 2013.
Vitor Guizilini, Rares Ambrus, Sudeep Pillai, Allan Raventos, and Adrien Gaidon. 3d packing for self--upervised monocular depth estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2485-2494 2020.
Mu Hu, Wei Yin, Chi Zhang, Zhipeng Cai, Xiaoxiao Long, Hao Chen, Kaixuan Wang, Gang Yu, Chunhua Shen, and Shaojie Shen. Metric3d v2: A versatile monocular geometrie foundation model for zero-shot metric depth and surface normal estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
HyunJun Jung. Patrick Ruhkamp, Guangyao Zhai, Nikolas Brasch, Yitong Li, Yannick Verdie, Jifei Song. Yiren Zhou, Anil Armagan, Slobodan Ilie, et al. On the importance of accurate geometry data for dense 3d vision tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 780-791, 2023.
Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Metzger, Rodrigo Caye Daudt, and Konrad Schindler. Repurposing diffusion-based image generators for monocular depth estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9492- 9502, 2024.
Diederik Kinga, Jimmy Ba Adam, et al. A method for stochastic optimization. In International conference on learning representations (ICLR), volume 5. California;, 2015.
Tobias Koch, Lukas Liebel, Friedrich Fraundorfer, and Marco Korner. Evaluation of cnn-based single-image depth estimation methods. In Proceedings of the European Conference Computer Vision (ECCV) Workshops, pp. 0-0, 2018.
Tobias Koch, Lukas Liebel, Mareo Körner, and Friedrich Fraundorfer. Comparison of monocular depth estimation methods using geometrically relevant metrics on the ibims-1 dataset. Computer Vision and Image Understanding, 191:102877, 2020.
Byeongjun Kwon and Munchurl Kim. One look enough: A novel seamless patchwise refine- ment for zero-shot monocular depth estimation models on high-resolution images. arXiv preprint arXiv:2503.22351, 2025.
Jiaqi Li, Yiran Wang, Jinghong Zheng. Zihao Huang, Ke Xian, Zhiguo Cao, and Jianming Zhang. Self-distilled depth refinement with noisy poisson fusion. Advances in Neural Information Pro- cessing Systems, 37:69999-70025. 2024a.
Zhengqi Li and Noah Snavely. Megadepth: Learning single-view depth prediction from internet photos. In Proceedings of the IEEE conference on computer vision and pultern recognition, pp- 2041-20500 2018
Zhenyu Li, Xuyang Wang. Xianming Liu, and Junjun Jiang. Binsformer: Revisiting adaptive bins for monocular depth estimation. arXiv preprint arXiv:2204.00987, 2022.
Zhenyu Li, Shariq Farooq Bhat, and Peter Wonka. Patchfusion: An end-to-end tile-based frame- work for high-resolution monocular metric depth estimation. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 10016-10025. 2024b.

---

Published as a conference paper at ICLR 2026
Zhenyu Li, Shariq Farooq Bhat, and Peter Wonka. Patchrefiner: Leveraging synthetic data for real real-domain high-resolution monocular metric depth estimation. In European Conference on Computer Vision, pp. 250-267. Springer, 2024c.
Xiaoxiao Long, Cheng Lin, Lingjie Liu, Wei Li, Christian Theobalt, Ruigang Yang, and Wenping Wang. Adaptive surface normal constraint for depth estimation. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 12849-12858, 2021.
Xiaoxiao Long, Yuhang Zheng, Yupeng Zheng, Beiwen Tian, Cheng Lin, Lingjie Liu, Hao Zhao, Guyue Zhou, and Wenping Wang. Adaptive surface normal constraint for geometric estimation from monocular images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46 (9):6263 1166663-279,, 2024.
Lukas Mehl, Jenny Schmalfuss, Azin Jahedi, Yaroslava Nalivayko, and Andrés Bruhn. Spring: A high-resolution high-detail dataset and benchmark for scene flow, optical flow and stereo. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
Mahdi H Miangoleh, Sebastian Dille, Long Mai, Sylvain Paris, and Yagiz Aksoy, Boosting monoc- ular depth estimation models to high-resolution via content-adaptive multi-resolution merging. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 9685-9694, 2021.
Pushmeet Kohli Nathan Silberman, Derek Hoiem and Rob Fergus. Indoor segmentation and support inference from rghd images. In ECCV, 2012.
Luigi Piccinelli, Yung- Hsu Yang. Christos Sakaridis, Mattia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. Unidepth: Universal monocular metric depth estimation. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 10106-101 2024.
Luigi Piccinelli, Christos Sakaridis, Yung-Hsu Yang, Mattia Segu, Siyuan Li, Wim Abbeloos, and Luc Van Gool. Unidepthv2: Universal monocular metric depth estimation made simpler. arXiv preprint arXiv22502.201 2025.
Xiaojuan Qi, Renjie Liao, Zhengzhe Liu, Raquel Urtasun, and Jiaya Jia. Geonet: Geometric neural network for joint depth and surface normal estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 283-291, 2018.
Xiaojuan Qi, Zhengzhe Liu, Renjie Liao, Philip HS Torr, Raquel Urtasun, and Jiaya Jia. Geonet++; Iterative geometric neural network with edge-aware refinement for joint depth and surface normal estimation. IEEE Transactions Pattern Analysis and Machine Intelligence, 44(2):969-9) 984, 2020.
Pierluigi Zama Ramirez, Fabio Tosi, Matteo Poggi, Samuele Salti, Stefano Mattoccia, and Luigi Di Stefano. Open challenges in deep stereo: the booster dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21 1168--21,7, 2022.
René Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. IEEE transac- fions pattern analysis and machine intelligence, 44(3):1623-1633 2020.
René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vision transformers for dense prediction. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 12179-12188, 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and puttern recognition, pp. 10684-10695, 2022.
Ashutosh Saxena, Min Sun. and Andrew Y Ng. Make3d: Learning 3d scene structure from single still image. IEEE transactions on pattern analysis and machine intelligence, 31(5):824-88) 2008.
12

---

Published as a conference paper at ICLR 2026
Daniel Scharstein, Heiko Hirschmülleer, York Kitajima, Greg Krathwohl, Nera Nešić, Xi Wang, and Porter Westling. High-resolution stereo datasets with subpixel-accuratee ground truth. In German conference on pattern recognition, pp. 31 31-42. Springer, 2014.
Thomas Schops, Johannes L Schonberger, Silvano Galliani, Torsten Sattler, Konrad Schindler, Marc Pollefeys, and Andreas Geiger. A multi-view stereo benchmark with high-resolution images and multi-camera videos. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3260-3269, 2017.
Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and sup- port inference from rgbd images. European conference on computer vision, pp. 746-760. Springer, 2012.
Fabio Tosi, Yiyi Liao, Carolin Schmitt, and Andreas Geiger. Smd-nets: Stereo mixture density net- works. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8942-8952, 2021.
Igor Vasiljevic, Nick Kolkin, Shanyi Zhang, Ruotian Luo, Haochen Wang, Falcon Z Dai, Andrea Daniele, Mohammadreza Mostajabi, Steven Basart, Matthew R Walter, et al. Diode: A dense indoor and outdoor depth dataset. arXiv preprint arXiv: vrXiivI908..0 00463, 2019.
Jiepeng Wang, Peng Wang. Xiaoxiao Long, Christian Theobalt, Taku Komura, Lingjie Liu, and Wenping Wang. Neuris: Neural reconstruction of indoor scenes using normal priors. In European conference on computer vision, pp. 139-155. Springer, 2022.
Kaixuan Wang, Fei Gao, and Shaojic Shen. Real-time scalable dense surfel mapping. In 2019 International conference on robotics and automation (ICRA), pp. 6919-6925. IEEE, 2019.
Ruicheng Wang, Sicheng Xu, Yue Dong, Yu Deng, Jianfeng Xiang, Zelong Lv, Guangzhong Sun, Xin Tong, and Jiaolong Yang. Moge-2: Accurate monocular geometry with metric scale and sharp details. arXiv preprint
0it.:02547002566 2025.
Shuzhe Wang. Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Ge- ometric 3d vision made easy. In Proceedings of the IEEF/CVF Conference on Computer Vision and Puttern Recognition, pp. 20697-20709, 2024.
Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. Advances Neural Information Processing Systems, 37:21875-21911 2024.
Wei Yin, Xinlong Wang, Chunhua Shen, Yifan Liu, Zhi Tian, Songcen Xu, Changming Sun, and Dou Renyin. Diversedepth: Affine-invarian depth prediction using diverse data. arXiv preprint arXiv:2002.00569, 2020.
Wei Yin, Jianming Zhang, Oliver Wang, Simon Niklaus, Long Mai, Simon Chen, and Chunhua Shen. Learning to recover 3d scene shape from a single image. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 204-214 213, 2021.
Zehao Yu, Songyou Peng, Michael Niemeyer, Torsten Sattler, and Andreas Geiger. Monosdf: Ex- ploring monocular geometric cues for neural implicit surface reconstruction. Advances in neural information processing systems, 15:25018-25035 2022.
13


Reviews:
[{"review_id": "qP1fFzxojR", "reviewer": "Reviewer_nhZ4", "summary": "This paper proposes a hybrid dual-path vision encoder for high-resolution geometry prediction. In addition, the authors employ off-the-shelf models to generate low-resolution depth for the entire image and high-resolution depth for local patches, which serve as the ground truth to supervise the network training. Experiments demonstrate that the proposed method achieves state-of-the-art performance and efficiency on the high-resolution geometry estimation benchmark.", "questions": "Please refer to Weaknesses.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The idea of hybrid dual-path architecture and self-distillation is simple and effective.\n\n2.  The proposed method achieves state-of-the-art performance and efficiency.", "weaknesses": "1.  The novelty of the proposed hybrid dual-path encoder appears limited, as FlashDepth also adopts a similar methodology of fusing features from different resolution branches for high-resolution prediction. What is the ​fundamental difference​ in the encoder design between the proposed method and FlashDepth?\n\n2.  The novelty of the proposed self-distillation also appears limited. Using cropping for self-distillation is also a common practice in self-supervised learning. In the field of depth estimation, Distill Any Depth [1] similarly employs a local-global distillation strategy based on cropping. A clear discussion of the core distinctions between the proposed method and Distill Any Depth is required.\n\n3.  In the ablation study, the local crop loss is key to achieving fine-grained depth estimation. I'm curious to see what would happen if we only used the local crop loss and did away with the global loss altogether.\n\n[1] Distill Any Depth: Distillation Creates a Stronger Monocular Depth Estimator, arXiv preprint arXiv: 2502.19204", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eZWU8vnm3u", "reviewer": "Reviewer_yCEW", "summary": "HYDEN is a high-resolution monocular depth, point map, and surface normal estimation model that achieves state-of-the-art accuracy with substantially reduced inference cost. Its architecture combines a low-resolution Vision Transformer (ViT) branch for global context with a full-resolution CNN branch for fine-grained details, and employs a self-distillation framework to address the scarcity of high-quality ground-truth supervision. Integrated into leading models such as DepthAnything-v2 and MoGe2, HYDEN delivers top performance on high-resolution benchmarks while providing significantly lower inference latency compared to competing methods.", "questions": "- How sensitive is the self-distillation performance to the teacher model’s quality or prediction bias? Have the authors experimented with weaker teachers or ensemble teachers?\n- Have the authors considered applying HYDEN to video sequences or multi-view inputs to assess temporal coherence in predictions?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- First of all, the key idea is interesting. The hybrid dual-path design effectively combines global ViT features and local CNN features without incurring the quadratic cost typical of pure ViT models. The modular encoder design makes HYDEN adaptable across multiple downstream tasks and backbones such as DA2 or MoGe2.\n- Also, HYDEN achieves substantial speed-ups while preserving accuracy, making it practical for deployment in real-time or resource-constrained settings (e.g., robotic perception, self-driving planning etc.). This contrasts with prior methods that trade efficiency for high-resolution detail.\n- Lastly, the model’s performance on zero-shot benchmarks highlights the robustness of the self-distillation framework and the hybrid encoder’s scalability.", "weaknesses": "- While the proposed framework is well designed and practically effective, its technical novelty is relatively limited. The core components such as the ViT–CNN fusion and pseudo-label-based self-distillation largely build upon existing approaches. The contribution primarily lies in the integration and scalability of existing techniques rather than introducing fundamentally new architectural or algorithmic innovations.\n- Although the component-wise ablation studies are well organized, the paper lacks sufficient detail on the design rationale and training loss formulations for each module, making full reproduction of the method challenging.\n- The effectiveness of the self-distillation pipeline relies heavily on the quality of the teacher model. However, the paper does not extensively discuss how weaker or biased teachers affect performance or convergence stability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Dr4KBD4u55", "reviewer": "Reviewer_2xjn", "summary": "This is a method for extending a dense predictor (based on ViT) to high resolution, applied to depth, surface normals and point maps prediction.  Given an already-trained network on a medium resolution (e.g. 518x518), the network is extended with a convnet (that scales linearly with image area) by fusing features from the original global network and the convnet.  This second-stage model is trained using a self-supervised method that predicts pseudolabels on crops using the original model, and aligns the crops to the global image prediction to form the training targets.\n\nNotably, for any of the higher resolutions, the ViT's resolutions is held constant at 518.  This takes advantage of two properties/assumptions of the image captures: (1) the camera captures a scene framed by the image extent, so while increased resolution supplies more details, the world scene in the image doesn't change and can be handled by the ViT at fixed res; and (2) crops of the image are themselves scenes with smaller (or more distant) objects that are represented well enough in the original training set to be accurately predicted.\n\nThe method is incorporated into DepthAnything-v2 and MoGe2, with evaluations showing excellent improvements in latency vs resolution, accuracy measures and visual details at high res up.", "questions": "To what extent do the ViT predictions of the global image vs crop disagree?  Some is of course inherently necessary, details and boundaries not present at lower res.  But are there times when the two scales disagree in larger areas?  How often does that occur and does it need to be handled in the alignment or by filtering?\n\nDo you think this could be used for semantic segmentation and/or panoptic segmentation?  In particular, crops may or may not have enough of the objects present to identify them, depending on their size.  So the alignment between crops and global scale may need to account for that, as well as the possibility that the smaller crops' segmentations by the original model could output finer segmentation levels (e.g. car globally vs car/door/window in crops) depending on the model.\n\n3.2.3:  I think this description is little confusing with the description of injecting into a support set, though I think I understand it.  Equivalently, though, is this the same as cropping the student SxS predictions and comparing against the corresponding teacher labels for the crop, summing over the crops in the task loss?\n\nFig 1 plots:  what are the stars in the plots?  also this plot is a little too small, I needed to zoom in a lot to read it.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "This is an exceptionally simple method with great results, naturally combining the strengths of ViT and convnets to form a performant system.  The self-supervised training technique leverages existing models and data, making this extension mechanism fairly general.  I also found the presentation and writing very clear (except in a couple parts mention below), I much enjoyed reading it.", "weaknesses": "While this is an excellent extension method, it requires an already performant model to distill from.  And in particular, the original model must be performant at its full original (518) res on crops which may have a different scene framing or subjects from the original images.  While these are indeed the case for the applications here, it would be good to see this discussed as a requirement.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SqE7PL1kUv", "reviewer": "Reviewer_R67Q", "summary": "The paper presents a conv-transformer dual-path framework for high resolution depth estimation. The transformer processes lower resolution input whereas the conv processes higher one. After fusing the features, the model provides high resolution depth. Due to the scarcity of HR depth GT in the wild, the paper adopts a self-distillation method for labeling. Experiments show promising results.", "questions": "Please check the weakness.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The dual-branch design is well-motivated and the overall framework is technically sound.\n\n- The framework shows strong performance compared with current sota methods.\n\n- The paper is well-writen and it's easy for me to follow the ideas.", "weaknesses": "- It would be better to add discussions about papers sharing the same ideas. For self-distillation: consider [1]. For dual branch, consider [2]. From my point view, discussing comparisons with [1] is kind of crucial as [1] was released 8 months ago and it was a trending paper. Though Hyden additionally presents a good architecture, the self-distillation is another major contribution claimed by the paper. [2] is an old paper but using the same dual-branch plus feature-level fusion idea. It would be good to add it in the related work.\n\n    [1] Distill Any Depth: Distillation Creates a Stronger Monocular Depth Estimator\n    [2] DepthFormer: Exploiting Long-Range Correlation and Local Information for Accurate Monocular Depth Estimation\n\n- Since the scale and shift alignment is not perfect (is kind of noisy as far as I see), it would be better to present some clues about the error introduced by this alignment process, giving readers more insights about the upper bound of this method. It would benifit future potential follow-up works.\n\n- One follow-up question: what if we use the way generating pseudo labels for direct evaluation? Given a HR image, we use DAV2/Moge for coarse prediction. Then, we crop patches from the HR image, use DAV2/Moge for patch-wise prediction. Finally, we align the patch-level prediction with coarse one and ensemble predictions to get final depth. I know this method leads to a super long inference time, but it would be good to have this one in the paper to indicate the performance and inference time gap.\n\n- It's hard to reproduce the experimental results, majorly because the introduction to the training data is vague. Since the unlabeled data can be from any dataset, why don't use public available ones for the sake of potential follow-up works? Also, it would be good to include the results with different size of training data.\n\n- I was wondering how many parameters are frozen vs, trainable in the architecture. Given a frozen transformer, I assume that only convolutional encoder and the fusion decoder are trainable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)