RPG360: Robust 360 Depth Estimation with Perspective Foundation Models and Graph Optimization
Authors: Dongki Jung
Jaehoon Choi
Yonghan Lee
Dinesh Manocha
Venue: NeurIPS 2025 poster ()
Keywords: monocular depth estimation
omnidirectional
foundation model
scale alignment
graph optimization
Primary area: applications

Abstract: The increasing use of 360$^\circ$ images across various domains has emphasized the need for robust depth estimation techniques tailored for omnidirectional images. However, obtaining large-scale labeled datasets for 360$^\circ$ depth estimation remains a significant challenge. In this paper, we propose RPG360, a training-free robust 360$^\circ$ monocular depth estimation method that leverages perspective foundation models and graph optimization. Our approach converts 360$^\circ$ images into six- face cubemap representations, where a perspective foundation model is employed to estimate depth and surface normals. To address depth scale inconsistencies across different faces of the cubemap, we introduce a novel depth scale alignment technique using graph-based optimization, which parameterizes the predicted depth and normal maps while incorporating an additional per-face scale parameter. This optimization ensures depth scale consistency across the six-face cubemap while preserving 3D structural integrity. Furthermore, as foundation models exhibit inherent robustness in zero-shot settings, our method achieves superior performance across diverse datasets, including Matterport3D, Stanford2D3D, and 360Loc. We also demonstrate the versatility of our depth estimation approach by validating its benefits in downstream tasks such as feature matching 3.2 ∼ 5.4% and Structure from Motion 0.2 ∼ 9.7% in AUC@5$^\circ$.

RPG360: Robust 360 Depth Estimation with Perspective Foundation Models and Graph Optimization
Dongki Jung Jachoon Choi Yonghan Lee Dinesh Manocha University of Maryland, College Park [jdk9405, kevchoi, 1yhan12, dmanocha] @umd edu
360MonoDepth
Depth Anywhere
Ours
Ground Truth
Prior Knowledge
Omnidirectional Camera
Image
Point
Perspective Foundation Model
ERP Image
Figure 1: (Left) 360 images can be transformed into multiple undistorted tangential plane images, and then acale-anbiguous depth maps can be predicted using perspective-foundee foundation models 46
68 25 (Right) 360MonoDepth [7 and Depth Anywhere (00 have addressed the scale ambiguity using optimization learning-bused methods. However, these reconstructed point cloud visualization. proposed method leverages the prior knowledge the perspective foundation model along with graph optimization, therehy enhances 3D
approaches suffer from 3D structural dogradation,
structural awareness and achieves superior performance.
Abstract
The increasing use of 360 images across various domains has emphasized the need for robust depth estimation techniques tailored for omnidirectional images. However, obtaining large-scale labeled datasets for 360 depth estimation remains a significant challenge. In this paper, we propose RPG360, a training free robust 360 monocular depth estimation method that leverages perspective foundation models and graph optimization. Our approach converts 360" images into six- face cubemap representations, where a perspective foundation model is employed to estimate depth and surface normals. To address depth scale inconsistencies across different faces of the cubemap, introduce novel depth scale align- ment technique using graph-based optimization, which parameterizes the pre- dicted depth and normal maps while incorporating an additional per-face scale parameter. This optimization ensures depth scale consistency across the six-face foundation cubemap while preserving 3D structural integrity. Furthermore, models exhibit inherent robustness in zero-shot settings, our method achieves supe- rior performance across diverse datasets, including Matterport3D. Stanford2D3D. and 360Loc. We also demonstrate the versatility of our depth estimation ap- proach by validating its benefits in downstream tasks such as feature matching 3.2 ~ 5.4% and Structure from Motion 0.2 ~ 9.7% in AUC@5" Project Page: https://jaks40s. wtt.hh.........000
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
360 sensors offer significant advantages due to their wide field of view, allowing the capture of rich
contextual information within single image [74 9 22 66). The growing adoption of such panoramic images in various applications, such as robot navigation [8, 65]. virtual reality [51. autonomous vehicles [41, and immersive media [10], necessitates robust depth estimation, generalizing to zero- shot settings, techniques specifically tailored for 360 images. However, applying conventional perspective depth estimation methods [67 68
71
25 62 to 360 images directly is challenging
due to fundamental differences in camera models, which introduce distortions. To address these challenges, previous research 360 depth estimation has broadly followed two approaches: I) learning-based methods and 2) optimization-based methods.
Existing learning-basse -aaedd approaches 77
43
27
59
72 28 have focused on
designing 360 depth estimation networks trained on labeled 360 datasets. These methods attempt to mitigate distortions through adaptive network architectures 778 53 61. or by leveraging cubemap and tangent image [37 5D projections during training. However, a major limitation of these approaches is the scarcity of labeled datasets for 360 depth estimation. few works have proposed self-supervised training methods [79] or generating pseudo ground truth depth maps 60) to
[38 59 27
overcome this limitation.
In contrast, optimization-busseee methods leveraging perspective foundation models 47 42 offer an
alternative approach that does not require labeled datasets. Recently, monocular depth estimation methods based on foundation models in the perspective image domain [45] 67 have demonstrated robust performance in zero-shot scenarios. 360 images can be transformed into multiple tangential plane images to reduce distortions. This transformation enables respective depth and normal estimation on each tangential plane, which can then be recombined into a single equirectangular projection. Leveraging pre-trained robust foundation models helps address the limitations associated with the lack of labeled 360 datasets. However, key challenge remains: Monocular depth estimation encounters scale ambiguity when merging independently estimated depth maps from different viewpoints. While several methods (47 42 for 360 depth estimation propose blending overlapped regions of tangent-plane depth maps from perspective networks, this simple blending overlooks crucial 3D structural information, as shown in Fig. | A Although the depth maps may exhibit high quality in 2D space, their corresponding 3D point clouds often suffer from 3D structural inconsistencies and performance degradation.
668
32
71 25 62
We present a novel structure-aware optimization technique for 360 depth estimation using perspective foundation models [71 25 II. Our approach converts an equirectangular projec- tion image into perspective images via cubemap projections, which represent the 360 scene without overlapping regions. Depth and normal estimation
Main Results:
performed using a perspective foundation model. Since monocular depth estimation provides only relative depth information, the depth scales across different faces of the cubemap may be inconsistent. To address this issue, we introduce a novel graph optimization approach that parameterizes the predicted depth maps and surface normals with additional per-face scale parameters. This parameterization ensures depth scale consistency across different perspective images of the cubemap. Our approach demonstrates robust depth estimation per- formance across diverse datasets, including indoor and outdoor environments such Matterport3) 5 Stanford2D3D 4. and 360Loc 26 The main contributions include:
We propose robust aptimization-basee 360 depth estimation method leveraging perspec- tive foundation models to address the scarcity of labeled 360 dataset.
We introduce a graph optimization formulation that integrates depth and surface normals with additional per-view scale parameters to enforce depth scale consistency. This optimization preserves the 3D structure and demonstrates superior performance terms of 3D evaluation metrics.
Most learning-baid -baeed methods exhibit performance degradation when trained on a dataset that differs from the test scenes. In contrast, our method does not require training dataset and is unaffected by domain gap.
We demonstrate the versatility and benefits of our 360 depth estimation approach by apply- ing it to downstream tasks such feature matching (30] and structure-froms soree-frrommoooioon [31].

---

2 Related Work
2.1 Perspective Foundation Models
Foundation models possess remarkable adaptability, allowing them to generalize effectively across different contexts. CLIP
has exhibited outstanding versatility across wide range of vision- introduced a powerful visual encoder designed for various downstream 2D vision applications, such segmentation, object detection, and depth estimation. DUSt3R [63 further explores the applicability of foundation models in 3D vision tasks, leveraging CroCo 64 pre-
44
language tasks. DINOv2 39
training enhance performance. Furthremore, learning-han haaedd monocular depth estimation methods have evolved from optimizing training techniques for various model architectures [2] 16 19
20 6
enhancing zero-shot capabilities [45] 46
I| 67
68 32 225, enabling strong generalization
across diverse domains, Notably, Metric3D 771 25 extends this direction by specifically exploring
zero-shot performance for metric depth estimation and surface normal prediction.
2.2
360 Depth Estimation
360 depth estimation suffers from the inherent distortions introduced by the equirectangular pro- jection. OmniDepth 78 leverage Spherical Convolutions [33 to mitigate this projection distortion. Other methods improve performance by combining multiple projection techniques, such as fusing cquirectangular and cubemap projections [58 59 27 or adopting icosahedral projections [. Addi- tionally, dilated convolutions have been explored to enhance feature extraction in distorted panoramic images 777.. Gravity-aligned feature learning has also been proposed to improve 360 depth estima- 2 S1 have recently gained traction for capturing global panoramic context more effectively. Despite these advancements, supervised learning methods for 360 depth estimation still heavily rely on labeled datasets. Due to the lack of large-scale wooldw annotations, these methods are often dependent on synthetic datasets []] 75 3]
tion
43 Meanwhile, transformer-basee approaches [28, 73
potentially causing performance degradation in real-world scenarios.
To mitigate this issue, some works explore self-supervised learning via view synthesis 57
Another line of research focuses on leveraging pre-trained perspective monocular depth estimation models for panorama depth estimation. For example, the pre-trained weights of perspective monocular depth estimation model are fine-tuned on 360 images using distortion-aware convolutional filters [56]. More recently, perspective foundation models have been incorporated, alleviating the dependency on large-scale labeled 360 datasets. 360MonoDepth [47] utilizes foundation models trained on perspective images 45 46) to predict depth on tangent images, which are later merged using a blending algorithm. Depth Anywhere [60] adopts a different approach by predicting depth for cubemap images using foundation model
67 and generating pseudo ground truth, which is then refined through affine-invariant loss. However, while these methods incorporate certain distortion- aware techniques, they lack explicit consideration of 3D structural consistency in the estimated depth maps. We propose novel approach that leverages perspective foundation models to estimate depth while explicitly promoting 3D structure awareness for 360 depth estimation.
3
Our Approach: RPG360
The overall process is illustrated in Fig. In Section 3.1 the Equirectangular Projection (ERP) image captured by 360 camera is first converted into distortion-free perspective (PER) images for individual monocular depth estimation. We adopt cubemap projection to represent PER images due to its computational efficiency, as every pixel is treated as parameter and the cubemap representation avoids overlapping regions. The estimated depth maps are then merged back into the ERP domain using the refinement method described in Section 3.2
3.1 Transformation of Coordinate Systems
An ERP image a 2D plane representation that depicts normalized rays on spherical sur- face in flattened form. The coordinate transformation can be formulated as E =S)).
where & = (0, Φ) denotes the spherical coordinates with 0 E [- & E corresponding 3D Cartesian coordinates are given by the unit vector S (S S S.). For
1-
r The
3

---

PER images obtained from six-face cubemap projection, E {O, 5) indicates the cor- responding face of the cubemap. The pixel coordinates of PER images are denoted as ps.
Progacthe
CrvvidinctioniiCoo Canwra
S sin cos
S, = sin(
P. K R S. (1)
S² = cos cos(Ø).
Figure 2: Differences in depth definitions between cam- era models. (Left) In perspective camera, depth is defined the distance along the z-axis from the camera center. (Right) In an omnidirectional camera, depth is measured as the Euclidean distance from the camera cen- ter to the 3D point.
where P. is the homogeneous coordinate of P.- The rotational extrinsic parameter for each face is repre- sented as R., while K E R3x3 denotes the intrinsic camera matrix, which the focal length and the principal point at half of the image size. Using this coordinate transformation, we can convert the depth maps between PER and ERP space. Notably, the definition of depth differs between perspective cameras and omnidirectional cameras. As illustrated in Fig depth PER cameras is measured as the distance along the z-direction, whereas in 360 cameras, it represents the radial distance from the sensor. Thus, a scaling factor p should be applied during the warping process to compensate for the variation in depth definitions.
(E)P(() = DD.
(E)
PER (p.).
p= - Pall-
where D is the predicted depth map from the pre-trained PER depth estimation model 25 and () represents the nearest neighbor interpolation. We can obtain 3D points P from the ERP depth map ERP
P = ERPS S.
3.2 Graph-based Optimization for Scale Alignment
Scale and Shift Alignment Monocular depth estimation inherently suffers from scale ambiguity, leading inconsistent depth scales across different PER images. To address this inconsistency. previous studies have proposed methods such as estimating scale and shift between frames [0 23 This can be formulated as a linear equation.
D +
(4)
However, ensuring consistent depth D" across different frames is challenging, as we rely on single scalar values À IR and J E IR for all pixels in the depth map D.
Graph Optimization for Local Scale Alignment We employ a graph optimization algorithm that parameterizes the predicted depth and normal maps. This optimization refines the depth maps by adjusting the local scale, which is analogous to the shift term, but the adjustment is applied to each pixel rather than relying single scalar value. We define this optimization function based simple equation that represents a plane defined by (Po, no),
Too (P - Po) 0,
(5)
where Po is a 3D point and is its corresponding normal vector of the surface. P indicates any point lying on the same plane. Inspired by (50). we define the 3D points corresponding to each pixel in the predicted depth maps as nodes and assign edge weights based on the likelihood that two adjacent points belong to the same plane,
Lp = ΣΣ way I|: (P, (P)-l + a ΣΣ wig wiyling n, nill2 j.s
(6)
where j i denotes the neighbor pixels of i in the graph, and Wij represents the edge weight between pixel i and its adjacent pixel j in the ERP space. We set a as 0.5. In Eq. the first term ensures two points are positioned in the same plane, while the second term ensures that neighboring points maintain a consistent plane based on the likelihood Wij between adjacent pixels. Based on the assumption that areas with the same texture correspond the same object surface (34 [4 49 50 we define the edge weights using local patches and the spatial distance between pixels,
Q
17
|| -ii-ill
Wi exp
20 2
exp
20 2
where
ja = ja mod W, Jy jy: (7)
apa

---

ERP Image
Perspective Domain
Inconsistent Depth Scale
Perspective
Weightt
Foundation
Model
Sistarity
Cubemaa Projection
Depth Maps
Normal Maps
High Low
Equirectangular Projection Domain
Consistent Depth Scale
ERP Depth
Graph Optimization
Parameter
@ 30 poirts o
Local Surfece e
Parameter
Figure 3: Overview of depth estimation method. The input cquirectangular (ERP) image projected six-face cubemap, where each non-overlapping face is independently processed by a perspective foundation model generate depth and nornal maps. The normal maps are transformed into a commmon voondinate system and merged into a single ERP normal map. However, due to the inherent limitations of monocular depth estimation, the merged depth maps exhibit depth scale inconsistencies
faces. To resolve this issue, the merged depih and normal maps porameterized, incorporating an additional scale parameter Using these parameters, we perform graph optimization under local planar approximation obtain a scale-aligned ERP depth map. For the construction of graph, we use the intensity values and the distances between a pixel i (white node) and its adjacent pixels j (red nodes) to define the edge weights usy. Based on these weights, 3D points are aligned on the surface using both depth and normal information.
where Q; represents a patch centered at pixel i of the image, || || denotes the Frobenius norm, and iin and o spa are set to 0.07 and 3.0, respectively. Since ERP image has its left and right extremities aligned, the r z-coordinate ju of pixel should be updated using the modulo operation with the ERP image width W ensure periodic boundary conditions.
Per-face Parameter for Global Scale Alignment Simply using Eq. 6 may result in over-smoothed depth propagation in neighboring points, as shown in Fig. [ Therefore, we exploit regularization terms for depth and normal maps to preserve the high fidelity of the input geometry. Here, we incorporate the per-face scale parameter Ae E IR6 to achieve global scale alignment for the depth of each cubemap face. This additional parameter Åe functions similarly to the scale term in Eq. The per-face scale parameter X is expanded to match the size each cubemap face, and then transformed and integrated into the ERP space as a scale map À E BHWW
La Σ -D. I, Σ
T2. m;
(8)
where and Ti denote input ERP depth and normal maps. The confidence masks m are determined based the cosine similarity between the input normal and the normal computed from the input depth map using the 8-neighbor convention 69 29]. The total loss function formulated as a weighted sum of the proposed loss terms,
Ltotal 7pp Lp + Td La The Lns
(9)
where Tp. Nd, Tn are weights selected through grid search.
4 Experiments
4.1
Experiments Settings
Datasets and Evaluation Metries We conduct extensive experiments on benchmark datasets Matterport3D [5), Stanford2D3D [4) and 360Loc 26) using images of a resolution 1024 x 512. For Matterport3D and Stanford2D3D, we adopt the official train and test splits. We additionally evaluate on Matterport3D-2K (2048 1024) for high-resolution benchmarks, fol- lowing 47 360Loc consists of four scenes, including both indoor and outdoor environments, each providing panoramic sequences of mapping and query images. We employ all mapping se- quences from Loc, which include ground truth poses and depth maps, to evaluate the zero-shot performance of depth estimation. As monocular depth estimation increasingly emphasizes 3D
5

---

Table 1: Quantitative comparison the Matterport3D (M) and Stanford2D3D (S) test sct. M indicates the combination M and StructuredaD (7). We evaluate 360 depth estimation performance under three different settings: (a) when the training and test scenes are identical, (b) when the training and test scenes differ, and (c) in a training-free sctup. The best methods for each cutegory shown in bold faces. Most learning- tearning-based methods exhibit performance degradation when trained on dataset that differs from the test scenes. In contrast, the optimization-basee bastedd methods using perspective foundation models (1)) do not require a training dataset, making them unaffected by domain gap. Under the training-free setting, RPG360 improves the Chamfer distance by 46.7% on Matterporti (M) and 64.2% on Stanford2DaD (S), compared to 360Mono
Method
Train Test
Dansee
Chamter
3D metrics F Score
lol
Meibod
Train -t Test
Duiaset
Chamfer
3D metrics
F Scoore"
IaU
SliceNet
18
M M
1,197
21.247
13.304
SliceNat
S S
0.291
70.700
57.453
UniFuse
M -> M
0.384
42.721
28,575
UniPuse
2
S S
0.331
43,881
28:555
(4)
BiFuscte
ACDNe
[0
M M
M M
0.334
0.554
39.383
448863
26.066
30.725
ACDNe:
Bloe360D
1
S
S
0.271
0,223
66,031
56,832
51.764
42.0017
ElRe300D
Depth Angwhere [0
M* M
M M
0.291
0.420
67.189
$3.997
54,663
39.482
UniFusc
SliceNat
21
43
M +
M S
0.277
1,205
56.150
21,681
12.763
40.675
SlioeNa
[3]
S
0.873
24.433
14.734
ACDNe
bM
M
0.495
43.497
26 585
(8)
ACDNet [X
UniFrsee 2t
8
0.543
0.580
32,277
$3.830
21,711
19,985
Flac3600
RiFuus+
[5)
U
M 11-55
M 5
0.770
0.184
73,501
46.650
31.731
68.175
0.539
34,499
21.548
Depth Anywbrr
M S
0.498
50,927
35.021
69)
Depth
XH
0.632
28, 28.056
16.998
360MenaDepth
1
0.576
25,230
14:672
RPG360 (Oun)
M
0,337
58.816
44.912
RPG330 (Curs)
0.206
73.917
61,880
structural awareness for practical applications 152 40]. we adopt 3D metrics, such Chamfer Distance, F-Score and IoU, rather than 2D metrics to better assess improvements in 3D structure and geometry. Further details of the 3D metrics are described in the supplementary materials.
Table 2: Quantitative comparison on 360Loc oo (L.. For Depth Anywhere, we train and test on 3601 oc using its pscuuo labeling (Lp) technique. Among optimization- based methods leveraging perspective foundation models ) RPG360 improves the Chamfer distance by 17.2% compared to Depthh.
Comparison Models
As also noted in 60) many
recent methods 1 2
73
36
37
51
have not
fully released their pre-trained models or provided their code and implementation details. Some also use different datasets, making direct comparison difficult. Therefore, we evaluate our method on fully accessible benchmarks, comparing it with both learning-
Method
Dutect
metrica F-Scre
SleehNer
L
2.441
K,164
4,405
443 27 77 59, I 60) and optimization- based [47] approaches using 3D metrics. In addition, to demon- strate the effectiveness of our approach in relation to
UniFaed
M4
2,533
10,373
5.670
ACTON 17
M-
2.435
10:372
5.724
(6)
B.Fuse++ 50
2.579
5,682
4.647
M-
1.790
14:777
recent methods, we also include evaluations on 21) metrics. For each method, We adopt the backbone or pre-trained model that achieved the highest perfor- mance as reported in the original paper (e.g., EfficientNet-B5 [55) for Elite360D [1 and +iFuse++ for Depth Anywhere (59) ). To mitigate scale ambiguity in monocular depth estimation, we apply median alignment (76) in all evaluations. Further details are provided in the supplementary material.
Dopth Auprlerr
11,889
6.5l
Dopuh Anywhune
368MosssDeph
-L
2.274
4.573
RJG336 (Cund)
1.883
19,806
11.302
Implementation Details We use the Adam optimizer 33 to perform gradient descent on a single RTX A5000 GPU. To accelerate convergence, adopt multi-scale approach following (50). The
ERP depth map DERP! E R1//22Ix((/e"" where / E {0, L - 1}. is downsampled by a factor of 2'. In this experiment, we set L = 3 and use learning rates of 5 x 10'-1. Each optimization step is performed for 300, 150, and 30 iterations. The weights of the loss terms Mp. nd. and 7/,, set to 50, 0.5, and 10, respectively.
4.2 Experimental Results
As shown in Table T although supervised methods perform well when the training and testing domains are the same, their performance degrades significantly when applied to different domains. Interestingly, Table [ also reveals the opposite trend: some of the methods, such as EliDe360D [. trained in Matterport3D achieve better performance on the Stanford2D3D test set than those trained on the Stanford2D3D train set. Given that the Matterport3D dataset is much larger than Stanford2D3D, we can infer that the available volume of labeled 360 data in Stanford2D3D might be insufficient for effective learning, depending on network capacity. This demonstrates that the lack of labeled 360 datasets poses a significant challenge for supervised learning approaches.
As potential solution to this problem, the perspective foundation models offer advantages because of their inherent robustness across diverse scenes, and they leverage knowledge from large-scale datasets. 360MonoDepth [47). which employs the perspective foundation model with blending algorithm,

---

ERP Image
360MoneDepth
Depth Anywhere
Elite360D
Ours
Ground Truth
Matterports
Stanford2D3D
360Loc
Figure Qualitative comparison point clouds reconstructed depth maps across different datasets, Depth Anywhere and Elite360D are learning-base based methods, while DephhnoDepth and Ours are optimization- based methods with perspective foundation models. Our RPG360 demonstrates superior structural accuracy, most closely matching the ground truth in 3D space.
However, its performance in 3D metrics is suboptimal, as it ignores 3D struetural information during optimization. Depth Anywhere [60] addresses the lack of labeled datasets by generating pseudo ground truth through perspective foundation model. While it demonstrates competitive performance compared to other supervised learning methods, its structural information appears distorted when visualized in 3D point clouds, as shown in Fig. 4 By incorporating our graph-based optimization with the additional per-face parameter, we demonstrate that our method achieves comparable or superior performance across diverse datasets. Furthermore, Fig. 4 shows our method produces point clouds with significantly fewer artifacts in 3D space, further validating its effectiveness.
achieves high-quality depth estimation in 2D-based metrics 47
Table 2 presents zero-shot quantitative results on 360Loc, which consists both indoor and outdoor scenes. For a fair comparison, trained Depth Anywhere (60) using its pseudo-labeling method on 360Loc and evaluated its performance. Elite360D [ performs well in indoor scenes like its training domain but exhibits performance degradation in zero-shot tests. Our method achieves better performance in terms of F-Score and IoU, demonstrating its robustness in diverse environments. We conduct 2D metric evaluations the Matterport3D dataset as shown in Table 3 Learning-baal -aaed methods generally demonstrate stronger performance on 2D metrics than optimization-basel methods. We also evaluate optimization-oan based methods using high-resolution 2K images, following 47 42].
To ensure fair comparison, we implement 360MonoDepth [47] with Metric3D v2 (25). However, since 360MonoDepth is originally designed output inverse depth (as in MiDaS v3 [46]) using model that predicts standard depth results in degraded blending performance. In contrast, our method requires depth predictions. Therefore, we only evaluate methods that directly produce depth outputs [1). in order to avoid additional normalization issues when converting inverse depth to depth. Since Peng and Zhang [42 leverages both a perspective and a 360 network, it outperforms 3e0MonoDepth, which relies solely perspective network. Although our method relies exclusively on a perspective model, still achieves superior performance, benefiting from both the robustness of foundation models and the structural awareness introduced by graph optimization.

---

Table Quantitative comparison using 2D metries the MatterporttD (M. 1024 512) and (c. d) MaterportS-22K (M-2K, 2048 x 1024) test sets. In the 2K high-resolution setting. (c) learninghasee methods (360 Train.) exhibit performance degradation, whereas (d) optimization-based methods (Opti.), including ours, demonstrate stronger performance.
Method
Promited Medd
Backhwns
360 Train
Oyti.
Theen -
Thaowe
Tatwer hoiee
Hligher Is briter
1,207
41.55
0.178
LinFase 20
BooNe34 20
M4
0.106
0.494
0..90
0962
6963
BCFemeer 13
Inasfoomss
14
0.167
0.003
0881
0939
0.946
ACUNNe
0958
10
RoNNerS
10
01117
11.01
01867
0550
0.855
M
0.105
08682
11800
D371
6,991
Deyth Asye 80
BiFusc 11
Deydi Acythng 3V
1M
0.865
0.917
0.76
0,991
+
XSDr5 <2
86
33
0.264
0.916
0.612
0.554
0.41
RUPG-568 (Oun)
Omridda v2
1
M
0.315
11577
11.96
DS3S
0.015
IUPG360 (Ouo)
Mamn:10 vA 13
0.383
0.667
0.889
0.53
8.977
EaczNIi 0
Mi-K
0.309
0.979
0.525
0.92
0.910
69
Deyth Auyyllere
X0
BiRuet [40] Doyh Anythnn 71
MT - MM3K
0.167
0.790
0.515
D547
0.95
Peng and Zhong
91
UniFare M Lakes
M5-2K
0.115
0.611
01871
B354
6,932
Deydo
31
MiDuS <
M5:KK
0.308
0.701
0.606
0,500
0.911
MamelD v2
33
X5-IK
0.500
1.141
0.455
8766
0.897
IUP,360 (Ouzs)
Orsadaaa v2
1
M-2
0.147
0.545
0.20
0947
0,950
RPG.360 (Our))
Mami:10 v2
M-X
0.107
0.464
0,89
6.969
0.987
Feature Matching
Spherical
Image
Warp before Tuning
Warp after Tuning
YVUCAY
Mesh
Ess.
SnP
Figure 5: Qualitative results of the downstream tasks using our method. For dense feature matching, EDM X0 estimates more confidential correspondences after tuning with pseudo ground truth generated by RPG360, compared to the initial results. For initial estimation in IM360 31]. SnP with RPG360 reconstructs slightly more accurate 3D structure than two-view
geometry- bused estimation (Ess.).
4.3
Experimental Analysis
To highlight the significance and utility of robust depth estimation, we apply our method down- stream 360" vision tasks. By leveraging our 360 depth estimation method, we can estimate the relative poses between multiple view frames. accomplish this, we implement the Spherical-n-PPi (SnP) algorithm [21] in conjunction with RANSAC [K]
Dense Feature Matching Since obtaining ground truth correspondences in 360 images is chal- lenging, most existing learning-bast based feature matching approaches for 360 datasets are primarily restricted to indoor environments [17] [8] 30). Consequently, when evaluating the feature matching method 30 in outdoor scenes, such as 360Loc, we observe performance degradation. To address this issue, utilize our robust depth estimation method, which leverages the perspective foundation model. With the aid of SnP and RANSAC, pseudo ground truth correspondences are generated using the relative pose and depth maps from RPG360 in 360Loc query scenes. We then finetune EDM for several iterations and evaluate it on the 360Loc mapping sequences. Table 4 and Figure showcase the advantages of our proposed depth module.
Structure from Motion IM360 31 demonstrates significant improvements in localization and mapping for large-scale indoor scenes sparsely captured with 360 cameras. During the localization process, relative poses are initialized using two-view geometry estimation based on epipolar geometry and essential matrix decomposition in spherical cameras. Instead of relying on this two-view geometry, we employ SnP (1 with RANSAC [13), utilizing our predicted depth maps before performing spherical incremental triangulation. As shown in Table G our approach achieves improvements in AUC errors across most scenes, demonstrating that robust depth estimation can boost the performance of other vision tasks. The qualitative result is presented in Fig,
8

---

Table 4: Performance of dense feature matching Table Ablation study different foundation models and the impact [0 with and without tuning using RPG360. Our of the per-face parameter A on 3D reconstruction performance. While robust depth estimation improves A by 5,4% graph optimization alone provides limited performance improvement, the hall scene of 360Loc.
incorporating propused per-face parameter Ae leads to superior results, highlighting its effectiveness 3D reconstruction quality.
Methed
Scme
Tuming
185"
A
Foundation
Graph
Por-face
3D metrics F-Score
EDM EDM
atrium
X
37.19
65.01
82.11
Model
Opt.
Chamfer
atrium
38.37
65.99
82.62
EDM EDM
concoume
X
33.13
56.97
75.81
Marigold 12
0.916
21.962
14.262
30
concourse
/
34,53
58.52
77.26
GooWizard [5
0.752
27.923
16.919
RDM EDM
50
ball
X
57.83
63.20
80.92
Omnidata 22
/
/
0.402
50.035
35.968
30
hall
/
39.88
64.67
81.57
Metric3D v2
0.380
51.306
36.971
EDM HDM
platrium
X
44.45
69.91
84.56
Metric3D v2
25
x
0.382
61.605
37,303
30
piatrium
/
46.49
70.34
85.06
Metric3D v2
25
0.337
58.816
44,912
Table 6: Performance of spherical Structure from Motion
Without Scale
With Scale Param.
using Epipolar geometry (Ess.) Spherical-n- Point (SnP) 2 the initial pose. RPG360 enables the SuP. resulting in a 9.7% improvement in AUC @ for the YVUC3YcDteY scene in Matterport3D. demonstrating the effectiveness of our robust depth estimation in enhancing spherical SIM performance.
(SfM) [1
Method
Scene
Init Pose
03"
AUC 1
0'10"
IM360
31
2r7WUuleka?
Ess.
49.16
69.05
84.53
IM
11
217
SoP
51.17
70.48
85.24
IM360
8194nk5LaLH
IM360
11
819
SnP
34:45
46.56
67.25
1M360
31
Ple44aQe7ggG
Ess.
73.95
84.37
92.18
IM360
74.39
92.26
Figure 6: Effect of the per-face scale parameter de. (Left)
IM360
1
RPmr22Hlmar
Ess.
53.44
71.87
85.93
Without Ae Fg Jonly smooths the gap between 3D points from different face views within the cubemap, leading noticeable distortions and misaligned structures. (Right)
IM 1
RPan22HHmyY
SoP
52.92
71.54
85.77
IM360
YVUCHY<DIEY YVUCHYeDIeY
72.40
91.71
JM360
31
SuP
79.44
87.67
93.83
By incorporating Ae in Eq. g depth scale consistency is improved, resulting in enhanced structural integrity.
IM360
31
2
Ess.
52.35
71.14
85.49
2*N04HB99L7
53.50
85.86
Ablation Study We analyze the performance variations of our method when using different perspective foundation models, as shown in Table 5 Marigold 32) and GeoWizard [15] utilize Stable Diffusion [48), whereas Omnidata [1]] and Metric3D (25 are trained on large-scale datasets using multi-task learning. All these models are capable of predicting both depth and surface normal maps. Although diffusion-basce models [32 [1] generate high-fidelity depth maps, their 3D point cloud reconstructions suffer from quality degradation when used for scene reconstruction. Table 5 alaso demonstrates the effectiveness of the per-face parameter Ae in graph optimization. Without Aes performance remains similar regardless of whether graph optimization is applied. However, incorporating our proposed per-face parameter de leads to noticeable improvements, confirming its contribution reconstruction quality. The corresponding qualitative improvements are illustrated in Fig. 6 The per-face scale parameter A provides flexibility for 3D points to move collectively within each face, preventing adjacent points from different faces from being simply normalized. This encourages depth consistency while preserving the original structure.
5 Conclusion, Limitations, and Future Work
In this paper, we propose a novel framework for 360 depth estimation. Leveraging foundation models for perspective images, we convert a single ERP image into six-face cubemap images and predict depth and surface normal maps for each face. To ensure depth scale consistency across different faces of the eubemap, we parameterize the predicted depth and normal maps and employ a graph optimization technique with the proposed per-face scale parameter for depth scale alignment. Our optimization-bant based approach may produce less detailed depth estimations for thin structures compared to training-based methods. However, by leveraging prior knowledge from perspective foundation models, it demonstrates strong robustness in zero-shot settings. Furthermore, as more powerful foundation models emerge, we expect the advantages of our method to become even more pronounced. In the future, we aim to extend the robustness of our approach beyond SfM to applications such as multiview image reconstruction.

---

Acknowledgements This work was supported in part by ARO Grant W911NF2310352 and Army Cooperative Agreement 191
References
[1] Hao Ai and Lin Wang. Elite360d: Towards efficient 360 depth estimation via semantic-and distance- aware bi-projection fusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattemn Recognition, pages
2024.
[2] Hao Ai, Zidong Cao, Yan-Pei Cao, Ying Shan, and Lin Wang. Hrdfuse: Monocular 360deg depth estimation by collaboratively learning molistic-with-rege Conference Computer Vision and Pattern Recognition, pages 13273- 13282, 2023.
depth distributions. In Proceedings of the IECE/CVF
[3] Georgios Albanis, Nikolaos Zioulis, Petros Drakoulis, Vasileios Gkitsas. Vladimiros Sterzentsenko, Federico Alvarez, Dimitrios Zarpalas, and Petros Daras. Pano3d: A holistic benchmark and a solid baseline for 360deg depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3727-3737, 2021.
[4] Tro Armeni, Sasha Sax, Amir R Zamir, and Silvio Savarese. Joint 2d-3d-semantic data for indoor scene 702.01 105, 2017.
understanding. arXiv preprint
[5] Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data indoor environments. arXiv preprint urXiv:1709.066 779.06158, 2017.
[6] Jaehoon Choi, Dongki Jung, Donghwan Lee, and Changick Kim. Safenet: Self-supervised monocular depth estimation with semantic-wware feature extraction. arXiv preprint arXiv:2010.0289) 2020.
[7] Jaehoon Choi, Dongki Jung, Yonghan Lee, Deokhwa Kim, Dinesh Manocha, and Donghwan Lee. Selídeco: Sepe-supe monocular depth completion in challenging indoor environments. In Proc. of the IEEE International Conference on Robotics and Automation, 2021.
[8] Jachoon Choi, Dongki Jung. Yonghan Lee, Deokhwa Kim, Dinesh Manocha, and Donghwan Lec. Sclftune: Metrically scaled monocular depth estimation through self- superviedd learning. In 2022 International Conference on Robotics and Automation (ICRA), pages 651 1-6518, 18. IEEF, 2022.
[9] Thiago LT da Silveira, Paulo GL Pinto, Jeffri Murrugarra- Llerena, and Cláudio R Jung. 3d seene geometry estimation from 360 imagery: A survey. ACM Computing Surveys, 1-(4) 2022.
[10] Marek Domańíki, Olgierd Stankiewicz, Krzysztof Wegner, and Tomasz Grajek. Immersive visual media-mpeg-it 360 video, virtual navigation and beyond. In 2017 International conference systems, signals and image processing (IWSSIP), pages 1-9. IEEE, 2017.
[11] Ainaz Eftekhar, Alexander Sax. Jitendra Malik, and Amir Zamir. Omnidata: A scalable pipeline for making multi-task mid-leve level vision datasets from 3d scans. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10786- 10796, 2021.
[12] David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from single image using multi-scale deep network. Advances neural information processing systems, pages 2366 23744, 2014,
[13] Martin A Fischler and Robert C Bolles. Random sample consensus: paradigm for model fitting with applications image amalysis and automated cartography. Communications of the ACM, 24(6):381-395, 1981.
[14] Alessandro Foi and Giacomo Boracchi. Foveated self-similarrty in nonlocal image filtering. In Human Vision and Electronic Imaging XVII, volume 8291, pages 296-307. SPIE, 2012.
[15] Xiao Fu, Wei Yin, Mu Hu, Kaixuan Wang, Yuexin Ma, Ping Tan, Shaojie Shen, Dahua Lin, and Xiaoxiao Long. Geowizard: Unleashing the diffusion priors for 3d geometry estimation from single image. In European Conference on Computer Vision, pages 241-258. Springer, 2024.
[16] Ravi Garg, Vijay Kumar BG, Gustavo Carneiro, and Ian Reid. Unsupervised enn for single view depth estimation: Geometry to the rescue. In European Conference on Computer Vision, pages 740-756. Springer, 2016.
[17] Christiano Gava, Vishal Mukunda, Tewodros Habtegebrial, Federico Raue, Sebastian Palacio, and Andreas Dengel. Sphereglue: Learning keypoint matching on high resolution spherical images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6134-6144, 2023.
10

---

[18] Christiano Gava, Yummin Cho, Federico Raue, Sebastian Palacio, Alain Pagani, and Andreas Dengel. Spherecraft: A dataset for spherical keypoint detection, matching and camera pose estimation. In Proceedings of IEEE/CVF Winter Conference on Applications of Computer Vision, pages 4408-4417, 2024.
[19] Clément Godard, Oisin Mac Aodha, and Gabriel Brostow. Unsupervised monocular depth estimation with left-right consistency. Proceedings of the IEEE Conference on Computer Vision and Pattem Recognition, pages 270-279, 2017.
[20] Clément Godard, Oisin Mac Aodha, Michael Firman, and Gabriel Brostow. Digging into self-supervised monocular depth estimation. In Proceedings of the IEEE international conference on computer vision, pages 3828 3838, 2019.
[21] Hao Guan and William AP Smith. Structureffrom-mmotion in spherical video using the von mises-fisher distribution. IEEE Transactions on Image Processing. 26(2):111772) 2016.
[22] Julia Guerrero- Viu, Clara Fernandez-Labrade Cédric Demonceaux. and Jose J Guerrero. What's in my room? object recognition on indoor panoramic images. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 567-573. IEEE, 2020.
[23] S Mahdi H. Miangoleh, Mahesh Reddy, and Yaguz Aksoy. Scale-invariant monocular depth estimation via ssi depth. In ACM SIGGRAPH 2024 Conference Papers, pages 2024.
[24] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[25] Mu Hu, Wei Yin, Chi Zhang, Zhipeng Cai, Xiaoxiao Long, Hao Chen, Kaixuan Wang, Gang Yu, Chunhua Shen, and Shaojie Shen. Metric3d v2: A versatile monocular geometric foundation model for zero-shot metrie depth and surface normal estimation. arXiv preprint arXiv:2404. 15506, 2024.
[26] Huajian Huang, Changkun Liu, Yipeng Zhu, Hui Cheng, Tristan Braud, and Sai Kit Yeung. 360loc: A dataset and benchmark for omnidirectional visual localization with cross-device queries. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22314-22324, 2024.
[27] Hualie Jiang. Zhe Sheng, Siyu Zhu, Zilong Dong, and Rui Huang. Unifuse: Unidirectional fusion for 360 panorama depth estimation. IEEE Robotics and Automation Letters, 6(2):1519-1526, 2021.
[28] Masum Shah Junayed, Arezoo Sadeghzadeh, Md Baharul Islam, Lai-Kuan Wong, and Tarkan Aydın. Himode: A hybrid monocular onnidirectional depth estimation model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5212-5221, 2022.
[29] Dongki Jung, Jaehoon Choi. Yonghan Lee, Deokhwa Kim, Changick Kim. Dinesh Manocha, and Dongh- wan Lee. Dnd: Dense depth estimation in crowded dynamic indoor scenes. In Proceedings the JEEE/CVF International Conference on Computer Vision, pages 12797-12807, 2021.
[30] Dongki Jung, Jachoon Choi, Yonghan Lee, Somi Jeong, Taejae Lee, Dinesh Manocha, and Suyong Yeon. Edm: Equirectangular projection-oriented dense kernelized feature matching. arXiv preprint arXiv:2502.20665. 2025.
[31] Dongki Jung, Jaehoon Choi, Yonghan Lee, and Dinesh Manocha. Im360: Textured mesh reconstruction for large-scale indoor mapping with 360 cameras, 2025. URL https///:////r org/ abs/ http:/bbb22522... 2502. 1.22..200 12545
[32] Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Metzger, Rodrigo Caye Daudt, and Konrad Schindler. Repurposing diffusionn bastussioo-baedddddee image generators for monocular depth estimation. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9492-9502, 2024.
[33] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.69MM) 2014.
[34] Philipp Krähenbühl and Vladien Koltun. Efficient inference in fully connected crfs with gaussian edge potentials. Advances in neural information processing systems, 24, 2011.
[35] Laurent Lescop. 360 vision, from panoramas to vr. In Envisioning architecture: space/time/meaning volume 1. pages 226-232. Published by the Mackintosh School of Architecture and the School of .... 2017.
[36] Meng Li. Senbo Wang, Weihao Yuan, Weichao Shen, Zhe Sheng, and Zilong Dong. S2net: Accurate panorama depth estimation on spherical surface. IEEE Robotics and Automation Letters, 8(2): 1060. 2023.

---

[37] Yuyan Li, Yuliang Guo. Zhixin Yan. Xinyu Huang, Ye Duan, and Liu Ren. Omnifusion: 360 monocular depth estimation via geomelry-aware fusion. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 2801-2810, 2022.
[38] Emanuele Menegatti, Takeshi Maeda, and Hiroshi Ishiguro. Image-based memory for robot navigation using properties of omnidirectional images. Robotics and Autonomous Systems, 47(4):251-267 2004.
[39] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafranice, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint xtiv:2304.07195, 2023.
[40] Evin Pinar Örnek, Shristi Mudgal, Johanna Wald, Yida Wang, Nassir Navab, and Federico Tombari. From 2d to 3d: Re-thinking benchmarking of monocular depth prediction. arXiv preprint arXiv:2203.0812 12,, 2022.
[41] Gaurav Pandey, James R McBride, and Ryan M Eustice. Ford campus vision and lidar data set. The International Journal of Robotics Research, 30(13):1543-1552, 2011.
[42] Chi-Han Peng and Jiayao Zhang. High-resolution depth estimation for 360deg panoramas through perspective and panoramic depth images registration. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 31 3116-3125, 2023.
[43] Giovanni Pintore, Marco Agus, Eva Almansa, Jens Schneider, and Enrico Gobbetti. Slicenet: deep dense depth estimation from a single indoor panorama using a slice-based representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11536-11345, 2021.
[44] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. International conference on machine learning, pages 8748-8763, PMI.R, 2021.
[45] René Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. IEEE transactions on pattern analysis and machine intelligence, 44(3):122316677, 2020.
[46] René Ranftl, Alexey Bachkovskiy, and Vladlen Koltun. Vision transformers for dense prediction. In Proceedings of the IEEF/CVF international conference on computer vision, pages 12179-12188, 2021.
[47] Manuel Rey-Arca, Mingze Yuan, and Christian Richardt. 360monodepth: High-resolution 360deg monocular depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3762-3772, 2022.
[48] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björ Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CV conference computer vision and pattern recognition, pages 10684-10695, 2022.
[49] Mattia Rossi, Mireille El Gheche, and Pascal Frossard. A nonsmooth graph-based approach to light field super-resolution. In 2018 25th IEEE International Conference on Image Processing (ICIP), pages 2590-2594. IEEE, 2018
[50] Mattia Rossi, Mireille El Gheche, Andreas Kuhn, and Pascal Frossard. Joint graph-based depth refinement and normal estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12154-12163, 2020.
[51] Zhijic Shen, Chunyu Lin, Kang Liao, Lang Nic, Zishuo Zheng, and Yao Zhao. Panoformer: Panorama transformer for indoor 360 depth estimation. In European Conference on Computer Vision, pages 195-211. Springer, 2022.
[52] Jaime Spencer, Chris Russell, Simon Hadfield, and Richard Bowden. Deconstructing sule-supe monocular reconstruction: The design decisions that matter. arXiv preprint arXiv:2208.0144 2022.
[53] Yu-Chuan Su and Kristen Grauman. Learning spherical convolution for fast features from 360 imagery. Advances in neural information processing systems, 30, 2017.
[54] Cheng Sun, Min Sun, and Ewaan--Tzong Tzong Chen. Hohonet: 360 indoor holistic understanding with la- tent horizontal features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattem Recognition, pages 2573-2500 2582, 2021.
12

---

[55] Mingxing Tan and Quoe Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference machine learning, pages 6105-6114. PMLR, 2019.
[56] Keisuke Tateno, Nassir Navab, and Federico Tombari. Distortion -atortion-aware convolutional filters for dense prediction in panoramic images. In Proceedings of the European Conference on Computer Vision (ECCV), pages 707-722, 2018.
[57] Fu-En Wang, Hou-Ning Hu, Hsien-Tzu Cheng, Juan-Ting Lin, Shang- Ta Yang, Meng-Li Shih, Hung- Kuo Chu, and Min Sun. Self-supervised learning of depth and camera motion from 360 videos. In Asian Conference on Computer Vision, pages 53-68. Springer, 2018.
[58] Fu-En Wang. Yu- Ych, Min Sun, Wei-Chen Chiu, and Yi-Hsuan Tsai. Bifuse: Monocular 360 depth estimation via bi-projection fusion. In Proceodings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 462 471, 2020.
[59] Fu-En Wang, Yu-Hsuan Ych, Yi-Hsuan Tsai, Wei-Chen Chiu, and Min Sun. Bifuse++: Self-supervised and efficient bi-projection fusion for 360 depth estimation. IEEE transactions on pattern analysis and machine intelligence, 45(5):5448-5460, 2022.
[60] Ning Hsu Wang and Yu-Lun Liu. Depth anywhere: Enhancing 360 monocular depth estimation via perspective distillation and unlabeled data augmentation. arXiv preprint (tt.22066112499 2024.
[61] Ning-Hsu Wang. Bolivar Solarte, Yi-Hsuan Tsai, Wei-Chen Chiu, and Min Sun. 360sd-net: 360 stereo depth estimation with learnable cost volume. 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 582-588. IEEE, 2020.
[62] Ruicheng Wang. Sicheng Xu, Cassie Dai, Jianfeng Xiang. Yu Deng, Xin Tong, and Jiaolong Yang. Moge: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision. arXiv preprint MXXv:22400.19115. 2024.
[63] Shuzhe Wang, Vincent Leroy, Yohann Cahon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made casy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20697-20709, 2024.
[64] Philippe Weinzaepfel, Thomas Lucas, Vincent Leroy, Yohann Cabon, Vaibhav Arora, Romain Brégier, Gabriela Csurka, Leonid Antsfeld, Boris Chidlovskii, and Jérôme Revaud. Croco v2: Improved cross- view completion pre-training for stereo matching and optical flow. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 17969-17980, 2023.
[65] Niall Winters, José Gaspar, Gerard Lacey, and José Santos-Victor. Omni-directional vision for robot navigation. In Proceedings IEEE Workshop Omnidirectional Vision (Cat. No. PR00704), pages 21-28. IEEE, 2000.
[66] Mai Xu, Chen Li, Shanyi Zhang, and Patrick Le Callet. State-of-the-att art in 360 video/image processing: Perception, assessment and compression. IEEE Journal of Selected Topies in Signal Processing, 14(1): 5-26, 2020.
[67] Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power large-scale unlabeled data. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10371-10381, 2024.
[68] Lihe Yang. Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. arXiv preprint arXiv:2406.09414 2024.
[69] Zhenheng Yang, Peng Wang, Wei Xu, Liang Zhao, and Ramakant Nevatia. Unsupervised learning of geometry from videos with edge-aware depth-normal consistency. Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
[70] Wei Yin, Jianming Zhang, Oliver Wang, Simon Niklaus, Long Mai, Simon Chen, and Chunhua Shen. Learning to recover 3d seene shape from a single image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 204-213, 2021.
[71] Wei Yin, Chi Zhang, Hao Chen, Zhipeng Cai, Gang Yu, Kaixuan Wang, Xiaozhi Chen, and Chunhua Shen. Metric3d: Towards zero-shot metric 3d prediction from single image. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9043-9053, 2023.
[72] 1w Yun, Jae Lee, and Chae Eun Rhee. Improving 360 monocular depth estimation via non-l dense prediction transformer and joint supervised and self supervieed learning. In Proceedings of the AAAI Conference Artificial Intelligence, volume 36, pages 3224-3233 2022.
13

---

[73] 1wwi Yun. Chanyong Shin, Hyunku Lee, Hyuk-Jae Lee, and Chae Eun Rhee. Egformer: Equirectangular geometry
reometry-biased biased transformer for 360 depth estimation. In Proceedings of the IEED/CVF International Conference on Computer Vision, pages 6101-6112, 2023.
[74] Fanglue Zhang, Junhong Zhao, Yun Zhang, and Stefanie Zollmann. A survey 360 images and videos in mixed reality: algorithms applications. Journal Computer Science and Technology, 38(3):473-491 2023.
[75] Zheng, Junfei Zhang, Jing Rui Tang, Shenghua Gao, and Zihan Zhou. Structured3d: large photo-realistic dataset for structured 3d modeling. Computer Vision ECCV 2020: 16th European Conference, Glasgow, UK. August 23 28, 2020, Proceedings, Part IX 16, pages 519 535. Springer, 2020.
[76] Tinghui Zhou, Matthew Brown, Noah Snavely, and David Lowe. Unsupervised learning of depth and ego- motion from video. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1858, 2017.
[77] Chuanqing Zhuang. Zhengda Lu. Yiqun Wang. Jun Xiao, and Ying Wang. Acdnet: Adaptively combined dilated convolution for monocular panorama depth estimation. In Proceedings of the AAAI Conference Artificial Intelligence, volume 36, pages 3653-3661, 2022.
[78] Nikolaos Zioulis, Antonis Karakottas, Dimitrios Zarpalas, and Petros Daras. Omnidepth: Dense depth estimation for indoors spherical panoramas. In Proceedings the European Conference Computer Vision (ECCV), pages 448-465, 2018.
[79] Nikolaos Zioulis, Antonis Karakottas, Dimitrios Zarpalas, Federico Alvarez, and Petros Daras. Spherical view synthesis for seff-supervised 360 depth estimation. In 2019 International Conference on 3D Vision (3DV), pages 690-699. IEEE, 2019.
14

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providedd proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Sec.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Sec.
15

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
Theory assumptions are not included in this work.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
Our proposed methods are described in Sec. Details are included in Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
16

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. w
For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification:
We use the public datasets and cite them in the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (nttps://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ 7/nips. /ttppp.cc/pblic/guideeee http://oeeuuiiissooooooooooo
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
17

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer:
[Yes]
Justification: Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification:
The experiments are done using the fixed random seed.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Sec.1
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
18

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htttps:/neurrps:: http://wwwggg..coooaaalc/aatttaa
Answer: [Yes]
Justification:
We follow the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Details are included in Appendix.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Answer: [NA]
We do not use pretrained language models, image generators, or scraped datasets.
Justification:
Guidelines:
19

---

The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Details are included in Appendix. Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- ce-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
Our proposed method is included in Sec.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
20

---

Crowdsourcing and research with human subjects are not included in this work.
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based the requirements of your country institution) were obtained?
Answer: [NA]
Justification:
This is not related our work.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
Declaration LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: This research does not involve LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our LLM policy http://eeerrrr..e/oooooeeeeeeees for what should or should be described.
21


Reviews:
[{"review_id": "0VmSlZrwEJ", "reviewer": "Reviewer_s1NL", "summary": "This paper proposes a depth map estimation method from omnidirectional(360) images. The method utilizes the strength of recent foundation models to estimate accurate depth maps. Since those foundation models are trained based on perspective images rather than omnidirectional images, the method starts with partitioning the image into cube maps. After that, the method estimates the ERP space scale map using graph optimization with depth maps and normal maps. By using scale, the final resulting depth map is obtained. The method is evaluated on three datasets, using standard 3D and 2D metrics.", "questions": "There is some missing information and details:\n- What is the running time? How long does it take for inference? How long is the optimization time?\n- Neither the main paper nor the supplementary material includes the estimated ERP space scale map. It may be correct, since the downstream tasks and depth map metrics, but providing this along with the initial/final results (Figure 2 in supp.) may help readers to understand what the updates are.\n- When merging cube maps to the ERP image again, how is the interpolation handled? There may be possibilities for choosing an improper sampling method for such remapping, which can lead to artifacts in edge cases.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The proposed method uses perspective foundation models for the omnidirectional image, which seems to work. The core contribution to this is graph optimization for scale alignment. The proposed graph formulation, weight estimation, and optimization equation seem technically sound. The paper also includes comprehensive comparisons and ablation experiments, including variations on foundation models (Table 5) and other partitioning methods (Table 3). The resulting quantitative metrics and images show the state-of-the-art performance. There are no major technical flaws; only minor considerations are listed in the Questions section below.", "was_revised": true, "final_justification": "The rebuttal properly answered my questions. I'm positive about the paper when focusing on the method that attempts to utilize foundation models for 360 images. Thus, I keep my ratings. However, I also agree with the other reviewer's opinion on the concern for the comparison methods. As a result, I cannot strongly claim the acceptance of the paper, and there is room to lower the score to Borderline Accept.", "rebuttal": "We thank the reviewer for their time and careful review. We appreciate the recognition of the technical soundness of our graph optimization for scale alignment, as well as the thoroughness of our evaluations across models (Table 5), partitioning strategies (Table 3), and datasets.\n\n## **Q1. Running time**\nThe inference time of our method depends on the input image resolution and the number of optimization iterations.\nFor example, with an image size of $1024 \\times 512$ and multi-scale iterations (Line 209)  set to $(300, 150, 30)$, the total inference time is approximately 4.6 seconds.\nWe summarize the relationship between the number of iterations and performance in the Table below.\n\nThe total number of iterations is $300 + 150 + 30 = 480$. To analyze the impact of iteration distribution, we define several configurations with the same total iterations: allocating more iterations to lower resolutions (coarse), allocating more to higher resolutions (fine), and distributing iterations evenly across all scales (equal).\nWe also include configurations with double and half the number of iterations for comparison.\n\n|  Type  |   # Iterations  | Chamfer ↓ | a1 ↑ | time (sec) |\n|:--|:--:|:--:|:--:|:--:|\n| original | (300, 150, 30)   | 0.33827  | 0.85503 | 4.5898 |\n| coarse   | (400, 40, 40)    | 0.33684  | 0.85288 | 4.4416 |\n| fine     | (40, 40, 400)   | 0.37439  | 0.82867 | 8.7129 |\n| equal    | (160, 160, 160)  | 0.34565  | 0.85124 | 6.1151 |\n||||||\n| half     | (150, 75, 15)    | 0.34897  | 0.84841 | 2.5647 |\n| double   | (600, 300, 60)   | 0.33499  | 0.85550 | 8.5421 |\n\nWe tested RPG360 with Metric3D v2 on the Matterport3D dataset.\nDue to rebuttal time constraints, we sampled 100 images from Matterport3D to show performance trends.\nWe will include full evaluation metrics in the supplementary material of the final version.\n  \n## **Q2. Visualization of Scale Map**\nWe originally believed the ablation in Table 5 and Figure 6 sufficiently demonstrated the scale map’s impact since it was simply used as an intermediate representation of scale parameters for optimization.\nWe will add the estimated scale map to help readers better understand the effect of the global scaling.\n\n\n## **Q3. Concerns about Interpolation**\nWhen merging the cubemap predictions back into the ERP space, we use nearest-neighbor interpolation for both depth and normal maps, which is a standard practice. \nWhile this method may not fully address interpolation deviations caused by projection distortions, the subsequent joint graph optimization helps compensate for such issues. \nIn our experiments, this interpolation strategy did not result in noticeable performance degradation."}, {"review_id": "KcIbr1DaKx", "reviewer": "Reviewer_78uu", "summary": "The paper proposes RPG360, a training-free pipeline for monocular 360-degree depth estimation. A panorama is converted to a 6-face cubemap; each face is fed to an off-the-shelf perspective foundation model to predict depth and normals. A subsequent graph optimisation introduces a learnable per-face scale λ_c and planar depth–normal constraints, aligning all faces into a single, scale-consistent ERP depth map.", "questions": "1. Considering that graph optimization may involve complex iterative calculations and may be limited by computational resources in practical applications.  Please report average optimisation time and GPU/CPU memory per 1024×512 panorama.  A plot of objective vs iterations would help.\n2. How does the optimization of the per-face scale parameter λc in Eq 8 prevent depth discontinuity at cubemap face boundaries? Is there a conflict between λc (global scale alignment) and local smoothness constraints (Eq 6)?\n3. Provide a robustness table for (η_p, η_d, η_n, σ_int, σ_spa) across datasets, or propose an automatic heuristic.", "limitations": "yes—Authors honestly mention smoothing of thin structures and slower inference vs feed-forward networks.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "The method is solid and yields large 360-depth gains without retraining, giving high practical value; yet novelty is mostly λ_c + graph optimisation, with no runtime/memory/hyper-parameter study, limited performance on thin structures, no comparison to recent diffusion approaches, and some notational clutter.", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your thoughtful and constructive feedback. \nWe are glad that our method was found to be solid and effective for 360 depth estimation without retraining, and we appreciate the recognition of our graph optimization with per-face scale alignment as the core contribution.\n\n## **Q1. Time and Memory Usage** \nThe **Table** showing inference time with respect to iteration count is included in our response to **Reviewer s1NL**’s **Q1**.\nHowever, allocating more iterations to high-resolution scales does not always yield better performance, as seen in the **fine** configuration.\nOur default setting (**original**), which follows a coarse-to-fine strategy (more iterations at lower scales, then refinement at higher scales), achieves the best balance between performance and efficiency.\nDepending on the application, one may reduce the total number of iterations (**half**) for faster inference with reasonable performance, or slightly improve performance by increasing iterations (**double**), though performance gains begin to saturate.\n\nRegarding memory usage, our method requires approximately 1680 MiB on an RTX A5000 GPU with an 8-core CPU when processing a $1024 \\times 512$ panorama.\n\n## **Q2. The role of $\\lambda_c$**\nThe overall optimization process can be interpreted as performing affine transformation of the form \n$$D^* = \\lambda D + \\tau ,$$\nwhere $D$ is the initial input depth and $D^*$ is the final consistent depth (**Line 141**).\nIn this formulation, the per-face scale parameter $\\lambda_c$ in Eq. 8 serves as the global scaling factor $\\lambda$, which aligns the depth across cubemap faces.\nIn contrast, the local smoothness constraint Eq. 6 effectively corrects the pixel-wise offsets, acting to estimate the bias term $\\tau$.\n\nAs a result, rather than conflicting, these two components work in a complementary fashion.\n$\\lambda_c$ enforces global consistency across faces, while the local constraints refine local geometric coherence.\nFurthermore, Eq. 6 serves as the main loss function, whereas Eq. 8 acts as a regularization term that prevents the optimization from diverging too far from the input or collapsing to trivial solutions.\n\n## **Q3. Hyperparameters**\nFollowing Rossi et al. [50], we set $\\sigma_{int} = 0.07$ and $\\sigma_{spa} = 3.0$. \n\nWe conducted a grid search to empirically determine the hyperparameters $\\eta_p = 50$, $\\eta_d = 0.5$, and $\\eta_n = 10$. \nThe table below shows the performance with different hyperparameter values. While there are some variations, the overall performance distribution remains relatively stable.\n\n| Type     | ($\\eta_p$, $\\eta_d$, $\\eta_n$) | Chamfer ↓ | a1 ↑ |\n|:----------|:------------------:|:----------:|:----------:|\n| original | (50, 0.5, 10)    | 0.33827       | 0.85503      |\n| | | | |\n| $\\eta_p$ ↓       | (10, 0.5, 10)    | 0.35356  | 0.85073  |\n| $\\eta_p$ ↑      | (100, 0.5, 10)   | 0.33572  | 0.85157  |\n| $\\eta_d$ ↓      | (50, 0.05, 10)   | 0.36480  | 0.82762  |\n| $\\eta_d$ ↑      | (50, 5.0, 10)    | 0.35525  | 0.84832  |\n| $\\eta_n$ ↓      | (50, 0.5, 5.0)   | 0.33533  | 0.85569  |\n| $\\eta_n$ ↑      | (50, 0.5, 100)   | 0.34166  | 0.85307  |\n\nHere, $\\eta_p$ controls the weight of the local planar approximation, while $\\eta_d$ and $\\eta_n$ constrain the optimized depth from deviating too far from the original input.\nAmong these, we observed that the optimization process is generally robust to $\\eta_p$ and $\\eta_n$, but slightly more sensitive to $\\eta_d$.\n\nIf $\\eta_d$ is set too high, the optimization resists deviation from the original input, resulting in almost no difference between the input depth and the optimized depth. \nConsequently, the discrepancies between cubemap faces persist.\n\nConversely, if $\\eta_d$ is too small, the influence of the per-face parameters diminishes, weakening the constraint from the original input. This causes over-smoothing across cubemap faces and structural collapse near seams (as illustrated in **Fig. 6**, \"**without $\\lambda$**\").\n\nUltimately, we selected the hyperparameter values reported in the main paper because we believe that qualitative results are as important as numerical performance in 3D vision tasks, and the chosen values produced strong visual results.\nTo ensure fair and consistent evaluation, we used the same set of parameters across all datasets and found that these values led to stable performance.\nWhile we cannot include figures in the rebuttal, we plan to add additional visualizations in the supplementary material, similar to **Figure 6**, to illustrate the effects of different hyperparameter settings."}, {"review_id": "qDLsSb7yce", "reviewer": "Reviewer_vRvv", "summary": "The authors propose a method that offers a training-free 360° depth estimation method. They leverage perspective foundation models that predict depth for cubemap representations of the 360° scene. The predicted depth maps are aligned through graph-based optimization addressing scale inconsistencies. The method shows superior zero-shot performance compared to training-free foundation models on datasets like Materport3D, Stanford2D3D and 360Loc.", "questions": "Could the authors clarify whether the model is intended for real-time deployment or for generating pseudo-ground truth, and provide runtime measurements to support this?\n\nGiven that RPG360 is pre-trained on a large and diverse dataset corpus, whereas competing methods like Elite360D are trained only on Stanford2D3D—which you note may be too small to generalize well—have the authors considered whether training such methods on more data (excluding Matterport, which is used only for zero-shot evaluation) would significantly improve their performance on Matterport? Without this comparison, it remains unclear whether RPG360’s superior zero-shot performance stems from its architecture or simply from access to more training data.\n\nCould the authors evaluate the model’s performance on challenging outdoor 360° datasets such as Dur360BEV or scenes to demonstrate its generalization beyond indoor environments where surface assumptions are valid only to a limited extent?\n\nCould the authors provide ablation studies showing the trade-off between the number of optimization iterations and performance, as well as the specific impact of incorporating surface normals?\n\nWhy did the authors choose to reimplement 360MonoDepth with a different backbone (Metric3D V2) instead of evaluating your method using 360MonoDepth's original foundation model (e.g., MiDaS V3) and standard 2D metrics?\n\nCan the authors demonstrate whether your method improves existing models like Elite360D when used to generate pseudo-ground truth, in line with approaches such as Depth Anything?", "limitations": "No, runtime is potentially a limitation and should be addressed.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\nThe 360-degree foundation model appears useful and relevant.\nFigure 4 presents compelling results.\nThe manuscript is well-written and easy to follow.\nThe authors plan to release the code, which will benefit the community.\n\nWeaknesses:\n\nA 360-degree foundation model with zero-shot performance is promising, but its value depends on how it can be used. It should be made clear whether the goal is real-time application or generating pseudo-ground truth. Since runtime is not reported, this remains uncertain.\n\nWhile the zero-shot performance on Matterport is strong, RPG360 lags behind Elite360D by over 10% on Stanford2D3D and on 360Loc. The authors suggest Elite360Ds limited performance on Matterport may be due to Stanford2D3D’s limited training size, which limits generalization for models like Elite360D. However, this comparison seems unfair, as RPG360's foundation models were pre-trained on a much larger and more diverse dataset corpus. It should be examined whether training competing methods such as [1] on more data (excluding Matterport, as it’s used only for zero-shot evaluation) would significantly improve their performance on Matterport. Not addressing this leaves open whether a properly trained alternative could outperform the proposed method as a foundation model.\n\nUnclear how model performs in outdoor environments where surface assumptions (walls etc) don’t apply as much such as indoors -> Dur360BEV (360 autonomous driving dataset) or similar dataset evaluation would be interesting\n\n* Yuan, Chao, et al. \"Dur360BEV: A Real-world 360-degree Single Camera Dataset and Benchmark for Bird-Eye View Mapping in Autonomous Driving.\" ICRA 2025\n\nAblation studies on both the efficiency trade-off (number of optimization iterations vs. performance) and the impact of incorporating surface normals (and the extent of their improvement) would be helpful.\n\nIn line 244ff, the authors note that using a model predicting standard (non-inverse) depth leads to degraded performance in 360MonoDepth. However, their reimplementation uses Metric3D V2, which they acknowledge performs suboptimally. While I agree that 360MonoDepth’s limited adaptability to standard depth values is a drawback, it would be more appropriate to evaluate the proposed method by adapting it, and the evaluation metrics, to 360MonoDepth as-is, rather than reimplementing 360MonoDepth with a different model. A comparison with the proposed method using the foundation model of 360MonoDepth (e.g., MiDaS V3, with surface normals computed via post-processing if needed) and traditional metrics (e.g., AbsRel, MAE, δ₁,₂,₃) would make the reported performance gains more convincing.\n\nIt would be valuable to demonstrate that this method can generate high-quality pseudo-ground truth and enhance 360° depth estimation approaches like those in Depth Anything, which rely on perspective-based pseudo labels. For example, evaluating whether zero-shot performance of top-performing models like Elite360D can be further improved using this approach would be interesting.\n\n\nMinor:\n\n- “For methods not covered in these papers, we referred to the original 30 publications to obtain their reported values.” \nWhy do the authors mention this? The reader might not have the time to go through all related papers to retrieve reported value. Relevant methods should be included, if not enough space in the main paper they should be included in the supplemental material.", "was_revised": true, "final_justification": "The rebuttal addressed most concerns, including runtime analysis, use case clarity, iteration trade-offs, and design choices. The method is technically sound, well-presented, and useful. Outdoor robustness is only partially evaluated, and pseudo-ground-truth usage is not tested. Therefore, I raise my score to borderline accept.", "rebuttal": "We are sincerely grateful for your thoughtful and detailed feedback.\nWe are glad that the proposed optimization method leveraging a perspective foundation model was found to be useful and relevant for 360 depth estimation.\nWe plan to release the code upon acceptance to further support the community.\n\n## **W1Q1) Main Purpose and Runtime**\nOur primary goal is not real-time deployment.\nInstead, we aim to develop a robust 360 depth estimation that can serve offline mapping itself, as well as downstream applications such as visual localization (e.g., SfM), as demonstrated in the main paper.\nDue to the lack of large-scale training datasets for 360 images, it is particularly challenging to build a generalized model directly for this modality.\nTo address this, we leverage an existing perspective-view foundation model and introduce our proposed optimization technique to enable robust zero-shot 360 depth estimation.\nThe downstream tasks are presented to illustrate the importance and utility of such robustness: even though the model is not designed for real-time use, a robust 360 depth estimation can still benefit many 360 vision tasks, such as feature matching and SfM.\n\n## **W2Q2) Response to Unfair Comparison Concern**\nWhile RPG360 shows slightly lower Chamfer Distance scores than Elite360D on Stanford2D3D and 360Loc, it actually outperforms Elite360D on other metrics such as F-Score and IoU.\nMoreover, although RPG360 leverages a foundation model pretrained on diverse perspective view datasets, it is not trained on any 360 datasets, unlike Elite360D. \nWe acknowledge that training Elite360D on a larger and more diverse corpus (excluding Matterport) could potentially improve its zero-shot performance. \nHowever, such training assumes access to large-scale 360 training datasets, which is currently unavailable due to the scarcity of 360 ground truth depth data.\nOur goal, instead, is to demonstrate a training-free approach that enables robust 360 depth estimation even under such data-scarce conditions.\nWe believe this setup highlights the practical relevance of RPG360 as a viable approach when 360 training data is unavailable or limited.\n\n## **W3Q3) Performance on Outdoor Dataset**\nOur method is based on local plane approximation at the patch level, which allows it to handle various surface geometries, not only flat structures such as walls but also curved objects, by approximating local tangential planes. \nAs a result, it is applicable to both indoor and outdoor scenes, including autonomous driving environments. \nMost prior works on 360 depth estimation adopt indoor datasets such as Matterport3D or Stanford2D3D as standard benchmarks. \nFor a fair comparison with existing methods, we followed this convention.\n\nWe agree with the reviewer that performance in outdoor environments is important.\nTo address this, we additionally evaluated our method on 360Loc [26] (**Line 61**, **Figure 4**), a recently released dataset that includes outdoor scenes. \nRegarding Dur360BEV, we appreciate the reviewer’s suggestion. As this dataset was introduced at ICRA 2025, which took place after our full paper submission, we were not aware of it at the time. \nNonetheless, we recognize its significance as the first large-scale autonomous driving dataset featuring spherical imagery and plan to include relevant experiments using Dur360BEV in the final version of the paper. \n\n## **W4Q4) Ablation Study on Iteration and Surface Normal**\nWe have added a **Table** summarizing the relationship between the number of iterations and performance in our response to **Reviewer s1NL's Q1**.\nOur results show that performing a sufficient number of iterations at lower resolution, followed by a small number of refinement steps at higher resolution, provides the best trade-off between accuracy and runtime.\nWhile increasing the number of iterations can further improve performance (type: **double**), we observe that the gains begin to saturate, making the additional runtime less cost-effective.\n\nIn our method, surface normals are an essential component for enforcing the local plane approximation and are therefore not an optional module that can be ablated.\nMoreover, surface normals can be either estimated by a separate network or directly computed from the predicted depth, meaning that their use does not require additional supervision or compromise the training-free nature of our approach.\nTherefore, we believe that incorporating normals does not pose a limitation in our framework.\n\n## **W5Q5) Reimplementation of 360MonoDepth**\nMiDaS V2 and V3, the original backbones used in 360MonoDepth, predict inverse depth. \nHowever, directly converting inverse depth to standard depth is not straightforward. \nSimply inverting the values does not yield an accurate depth map, as it ignores important factors such as focal length, leading to structural distortions in 3D space.\nInverse depth represents only a relative inverse distance; thus, naively inverting it does not recover true relative depth or distance.\nAs a result, the reconstructed 3D geometry (e.g., point clouds) becomes unreliable, and surface normals computed from such depth maps are also inaccurate. \n\nSince our method relies on the structural integrity of both depth and normals through local plane approximation, using MiDaS V2 or V3 would not result in a meaningful or fair comparison. \nTherefore, we chose to reimplement 360MonoDepth using Metric3D V2, which directly predicts standard depth and preserves structural consistency.\nWhile Metric3D V2 may not be optimal, we believe it provides a more appropriate and fairer basis for comparison with our method.\n\n## **W6Q6) Use of Pseudo Ground Truth from RPG360 for Training Elite360D**\nWe acknowledge that using RPG360 to generate pseudo-ground truth and further train models like Elite360D would be valuable. \nHowever, such an approach is inherently limited by the quality of the generated pseudo-ground truth.\nIn other words, the maximum achievable performance is bounded by the accuracy of the pseudo labels themselves.\n\nIn contrast to Depth Anywhere [60], which rely on pseudo-label training from perspective foundation models, our goal is to demonstrate that a perspective foundation model itself can directly perform robust 360 depth estimation in a zero-shot manner when combined with our proposed optimization technique.\nWe believe this highlights a distinct and meaningful contribution that goes beyond merely serving as a pseudo-label generator.\n\n## **W-minor)**\nTo clarify, the sentence means that we included the performance of all relevant methods in our tables. \nIn 360 depth estimation research, the same methods are often reported with slightly different performance values across different papers, which can lead to confusion. \nTo ensure consistency and fairness, we followed the values reported in the most recent and comprehensive benchmark papers [1] and [18] when available. \nFor methods not covered in [1] or [18], we referred to their original papers to obtain the reported performance. \nOur intention was to avoid inconsistencies in reported metrics, not to imply that any prior works were excluded from our comparison. \nWe will revise the wording in the final version to make this point clearer and will include the full set of reference values in the supplementary material for transparency."}, {"review_id": "tCyu89kPj5", "reviewer": "Reviewer_PqBQ", "summary": "This paper introduces RPG360, a novel training-free method for monocular 360-degree depth estimation. The core idea is to leverage the robust capabilities of pre-trained perspective foundation models, circumventing the need for large, labeled 360-degree datasets. The approach begins by converting an equirectangular (ERP) image into a six-face cubemap representation, where each face is processed independently by a perspective foundation model to predict initial depth and surface normal maps. The central contribution lies in a graph-based optimization framework designed to address the scale inconsistencies that arise when merging these independently estimated depth maps from different cubemap faces. This optimization refines the depth and normal maps by incorporating a per-face scale parameter, ensuring global depth scale consistency and preserving 3D structural integrity. The authors demonstrate the method's effectiveness and robustness in zero-shot settings across diverse datasets like Matterport3D, Stanford2D3D, and 360Loc, and illustrate its utility in downstream tasks such as feature matching and Structure-from-Motion.", "questions": "1. Specificity of Foundation Model Version: In Ln. 276, the paper references \"Metric3D v2 [25]\". Could the authors confirm that all instances where Metric3D is utilized as the foundation model throughout the paper, particularly for the \"RPG360 (Ours)\" results in Tables 1, 2, and 3, consistently refer to and use \"Metric3D v2\" as detailed in reference [25]?\n\n2. Could the authors reformat Table 4. and Table 5. for readability:\nThe presentation of quantitative results in Table 4 and Table 5 would benefit from enhanced readability. We suggest the authors consider reformatting these tables to improve clarity, such as by adjusting column spacing, ensuring consistent value alignment, or utilizing more distinct visual separations between logical sections. This would facilitate easier interpretation of the presented data.\n\n3. Comparison with Planar-Based 360 and 3D Reconstruction Methods: \nGiven the paper's emphasis on 3D metrics and structural correctness in depth estimation, a comparison with established planar-based 3D reconstruction methods would provide valuable context. These methods also prioritize 3D alignment and geometric accuracy. Could the authors include a comparative analysis against representative planar-based approaches to further contextualize RPG360's performance in 3D structural awareness?\n\n4. Inference Time and Computational Efficiency: \nThe paper currently lacks a comprehensive analysis of inference time. To fully assess the practical applicability and computational efficiency of RPG360, please provide the end-to-end processing time for a single 360-degree image. This should include the time taken for depth estimation by the foundation model and the subsequent graph optimization step.\n\n5. Robustness to Image Degradation: \nThe paper highlights the robustness of RPG360. While zero-shot generalization to diverse scenes is demonstrated, it is crucial to understand the method's performance under various image degradations (e.g., low-light conditions, noise, motion blur). Given that the initial depth and normal maps from perspective foundation models may degrade in such scenarios, how does the proposed graph optimization specifically perform and affect the final depth quality under these challenging input conditions? Does it exhibit graceful degradation, amplify errors, or provide resilience?\n\n6. Limitations in Thin Structures and Fine Details: In Section 5, the authors mention, 'Our optimization-based approach may produce less detailed depth estimations for thin structures compared to training-based methods.' Could the authors expand on the specific reasons for this limitation? Is it inherent to the graph optimization's smoothing properties, limitations of the cubemap projection near edges, or a characteristic inherited from the foundation models? Discussing potential strategies or future work to mitigate this limitation would enhance the paper's completeness.\n\n7. Visualizations of Failure Modes/Limitations: Beyond quantitative metrics, qualitative visualizations of the method's limitations or failure modes would be highly beneficial. Specifically, could the authors provide examples (e.g., in supplementary material) illustrating scenarios where 'less detailed depth estimations for thin structures' are evident, or where distortions or inconsistencies persist despite the graph optimization? This would offer crucial insights into the method's current boundaries.", "limitations": "yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "### Strengths of RPG360\n\n* **Effective Adaptation of Perspective Foundation Models**: The paper presents an innovative strategy for leveraging powerful, pre-trained perspective depth models for 360-degree imagery. This \"training-free\" approach is a significant advantage, as it inherently bypasses the challenges of limited large-scale, labeled 360-degree datasets and mitigates domain generalization issues commonly faced by supervised learning methods in this domain.\n* **Novel Graph Optimization for Consistent Scale Alignment**: A key technical strength is the introduction of a novel graph-based optimization framework, notably incorporating a per-face scale parameter ($\\lambda_c$). This method offers an effective solution to the critical problem of depth scale inconsistencies that arise when fusing independently estimated depth maps from multiple cubemap faces. The ablation study (Table 5) clearly demonstrates the substantial positive impact of this per-face parameter on the quality of 3D reconstruction.\n* **Robustness and Strong Zero-Shot Generalization**: RPG360 consistently exhibits robust performance in zero-shot settings across a variety of datasets, including Matterport3D, Stanford2D3D, and 360Loc. This is a notable benefit, as the method effectively operates without requiring 360-degree training datasets and is consequently less susceptible to domain gaps often encountered by other approaches.\n* **Demonstrated Utility in Downstream Applications**: The paper effectively showcases the practical applicability of its high-quality depth maps by validating their benefits in common 360-degree vision tasks such as dense feature matching and Structure-from-Motion (SfM). This provides compelling evidence of the method's real-world impact beyond raw metric scores.\n\n### Weaknesses of RPG360\n\n* **Implicit Foundation Model Specification for Main Results**: While the ablation study (Table 5) lists and evaluates several foundation models, the specific model (including its version and backbone configuration) used to generate the primary \"RPG360 (Ours)\" results in Tables 1, 2, and 3 is not explicitly stated. For complete clarity and reproducibility, this crucial detail should be directly declared in the respective table captions or accompanying text.\n* **Limited Comparative Analysis of Optimization Strategies**: The paper thoroughly describes its proposed graph optimization approach and its internal components. However, it lacks a broader discussion or comparative analysis against other established or alternative global scale alignment or fusion optimization techniques present in the existing literature. This omission makes it challenging to fully assess the distinct advantages and necessity of their particular graph formulation compared to other potential solutions in the domain.\n* **Partial Reproducibility Details for Certain Baselines**: Although some baseline models are identified (e.g., \"BiFuse++ [59]\" for \"Depth Anywhere [60]\"), comprehensive retraining specifics, such as the exact number of epochs or precise learning rate schedules, are not provided for all models, particularly those that undergo training on pseudo-labeled data (e.g., \"Depth Anywhere [60]\" on 360Loc). This lack of detail could hinder the exact reproduction of certain baseline results.\n* **Lack of Explicit Surface Normal Ablation**: While surface normals are integrated into the graph optimization's formulation (Eq. 6), the paper does not present a dedicated ablation study that directly isolates and quantifies the performance impact of using normals versus entirely omitting them from the optimization framework. The current ablation (Table 5) primarily highlights the substantial impact of the per-face scale parameter, leaving the individual contribution of surface normals less clear.\n* **Imbalanced Space Allocation**: While the demonstrations of the method's utility in downstream tasks are valuable and well-executed, the considerable space dedicated to these aspects could potentially be optimized for conciseness. This might allow for more in-depth discussions, additional ablation studies focused directly on the core depth estimation methodology, its theoretical underpinnings, or deeper analyses of failure modes within the main paper.\n* **Limited Comparative Scope against 3D-Focused Methods**: While the paper effectively compares against learning-based 360-degree depth estimation methods, the absence of a direct comparison with established planar-based 3D reconstruction techniques, which also prioritize geometric alignment and structural correctness, represents a missed opportunity to fully contextualize RPG360's claimed superior 3D structural awareness.\n* **Unanalyzed Robustness to Image Degradations**: Although the paper claims \"robustness\" and demonstrates strong zero-shot generalization to diverse scenes, it does not provide an analysis of the method's performance when presented with common image degradations such as low-light conditions, noise, or blur. It remains unclear how the proposed graph optimization, which relies on initial depth and normal maps, performs or is affected under these challenging input scenarios.\n* **Suboptimal Table Formatting**: The visual presentation and separation of information within Tables 4 and 5 are suboptimal. The uneven or off-center alignment of elements within these tables can be distracting and hinder quick interpretation of the presented quantitative results.", "was_revised": true, "final_justification": "### Justification for Final Decision\n\nAfter reviewing the authors' rebuttal, my initial concerns regarding the paper's scientific rigor remain, as the explanations were not persuasive on key issues of originality, experimental fairness, and robustness.\n\n---\n### Addressed Points\nI would like to acknowledge that the authors have committed to updating the model specifications and adding visualizations of limitations, which will improve the paper's clarity. The new robustness validation against image degradation is a welcome addition, as is the detailed explanation for the \"thin structures\" limitation (though this would be more convincing with direct experimental support).\n\n---\n### Originality Claim and Literature Gap\n\nThe authors' originality claim is overstated. While their specific combination of a training-free approach with graph optimization is novel, the fundamental concept of incorporating structural correctness (e.g., via planar priors or structural losses) is not, as shown by prior work [1, 2]. The paper mischaracterizes the existing literature by not acknowledging these precedents.\n\n---\n### Fairness of Comparison\n\nThe experimental design has significant issues that undermine the paper's conclusions.\n\n* **Unbalanced Baselines:** A key premise of this work is that leveraging a massive perspective foundation model should lead to superior generalization in zero-shot scenarios where supervised methods struggle. However, the evaluation shows the proposed method achieves only similar results to the state-of-the-art Elite360D[4] on the Stanford2D3D dataset (when the baseline is trained on Matterport3D). This lack of a significant performance gap, despite the massive data advantage of the foundation model, makes it difficult to ascertain the true effectiveness of the proposed graph optimization.\n\n* **Invalid Justification:** The rebuttal on this point was particularly concerning. While the authors state in their response that they \"included both the original performance of Depth Anywhere and additional results using Metric3D V2 to allow for better contextualization,\" these specific results are not present in the submitted paper nor rebuttal. For a meaningful comparison between methods that both leverage perspective foundation models, the underlying models must be aligned. Justifying an unfair comparison by referencing non-existent data does not resolve the core methodological issue.\n\n---\n### Conclusion\n\nIn conclusion, while I acknowledge the effort in the rebuttal and the new experiments provided, the core concerns regarding the paper's positioning on novelty and the fairness of key experimental comparisons remain. Because these issues significantly impact the confidence in the paper's conclusions, I have adjusted my score from Borderline Accept to Borderline Reject.\n\n***\n[1] Indoor Panorama Planar 3D Reconstruction via Divide and Conquer\\\n[2] Pano3D: A Holistic Benchmark and a Solid Baseline for 360° Depth Estimation\\\n[3] Enhancing 360 Monocular Depth Estimation via Perspective Distillation and Unlabeled Data Augmentation\\\n[4] Hao Ai and Lin Wang. Elite360d: Towards efficient 360 depth estimation via semantic-and distance301 aware bi-projection fusion.", "rebuttal": "We appreciate your time and careful evaluation of our work.\nWe are glad that the effectiveness of our training-free approach and the core contribution of our graph-based optimization framework with per-face scale alignment were well recognized.\nWe also appreciate the recognition of our method’s zero-shot generalization and its utility in downstream applications.\n\n## **W1Q1) Model Specification**\nIn **Table 3** in the main paper, we explicitly indicate the backbone or pretrained model used for each method, including Metric3D V2 and Omnidata V2 for RPG360.\nFor the main results in Tables 1 and 2, RPG360 (Ours) consistently uses Metric3D V2 as the foundation model.\nWe will revise the manuscript to explicitly state the foundation model used in Tables 1 and 2 to ensure clarity and reproducibility.\n\n## **W2Q3) Analysis of Optimization Strategies**\nTo the best of our knowledge, there are no existing methods that apply graph-based optimization specifically for 360 depth estimation. \nAmong prior works, 360MonoDepth [47] and Peng and Zhang [42] incorporate optimization techniques, but their approaches are limited to blending-based depth alignment without explicit consideration of structural consistency or global 3D geometry (**Lines 47–51**). \nIn contrast, our method introduces a graph optimization framework that leverages both depth and surface normals to enforce local planarity and global structural coherence, resulting in more accurate 3D reconstruction. \nMoreover, we analyze the effectiveness of our proposed optimization strategy through an ablation study presented in Table 5.\n\n## **W3) Reproducibility Details for Depth Anywhere [60]**\nFor models that we reimplemented, we followed the original codebase and training configurations as closely as possible to ensure fair comparison.\nSpecifically, for Depth Anywhere, we used its best-performing backbone BiFuse++ (**Line 205**), and followed the original settings: training for 200 epochs with a batch size of 4, a learning rate of 3e-4, and the Adam optimizer.\nWe selected the best-performing checkpoint based on validation performance for testing.\nWe will include these training details in the supplementary material to enhance clarity and reproducibility.\n\n## **W4) Lack of Surface Normal Ablation**\nIn our proposed optimization, surface normals are an essential component and not an optional module.\nRemoving normals would effectively mean disabling the core of our method. \nThe performance in this setting, without applying our optimization framework, is reported in Table 5, fourth row (Chamfer = 0.380).\nFurthermore, while surface normals can be predicted using a separate network, they can also be computed directly from the predicted depth. \nThus, incorporating normals does not introduce additional inference overhead, and we do not consider their use a burden to the overall method.\n\n## **W5) Space Allocation**\nOur primary contribution lies in proposing a robust 360 depth estimation method using a perspective foundation model. \nGiven this, we believe it is important to demonstrate not only why robustness matters, but also how the proposed method can be meaningfully applied to various 360 vision tasks. \nThese downstream tasks help validate the generalization capability of our method and provide empirical insight into its practical effectiveness. \nTherefore, we allocated a portion of the main paper to downstream applications, which we believe contribute to a more comprehensive understanding of the methodology.\n\n## **W6Q3) Limited Comparison**\nAs noted in our response to W2, our work is the first to explicitly incorporate structural correctness and local planar approximation into a 360 depth estimation framework. \nInstead, we compared our approach with prior optimization-based blending algorithms such as [42, 47]. \nWe believe this comparison supports the validity and advantages of our graph-based optimization approach.\n\nTo the best of our knowledge, most planar-based 3D reconstruction techniques fall under the category of multi-view stereo (MVS), which operates under the assumption of overlapping views. \nIn contrast, our graph optimization method is designed to operate on adjacent but non-overlapping cubemap faces in a single 360 image. \nTherefore, these MVS-based approaches are not directly applicable to our problem setting.\n\nIf the reviewer was referring to other specific planar-based techniques beyond MVS, we would be happy to consider them and include a discussion in the supplementary material.\n\n## **W7Q5) Image Degradation**\nOur primary focus in this work is to explore how robust 360 depth estimation can be achieved in the absence of large-scale 360 training datasets, across both indoor and outdoor scenes (**Line 60**). \nWhile we agree that evaluating robustness under image degradation such as low-light, noise or motion blur would provide additional insights, acquiring 360 datasets with controlled degradations is challenging. \n\nAlso, to ensure fair comparison, we conducted experiments using standard benchmark datasets in 360 depth estimation: Matterport3D [5], Stanford2D3D [4], and recently released 360Loc [26]. \nAlthough they may not frequently contain the specific types of degradations suggested, their diversity in scene types allows us to test generalization across varied conditions. \nFurthermore, because our method builds upon a foundation model trained on large-scale perspective data with strong generalization capabilities, we expect it to remain relatively robust to common image degradations, especially compared to models trained solely on limited 360 data.\n\n## **W8Q2) Table Formatting**\nThank you for the helpful suggestion. \nDue to differences in caption length and table layout, aligning and visually separating Tables 4 and 5 precisely at the center was challenging in the current submission.\nWe agree that improving visual clarity is important, and we will reformat the tables to enhance readability.\n\n## **Q4) Inference Time**\nThe time required for depth estimation by the foundation model depends entirely on the specific model used. \nThe subsequent graph optimization stage introduces additional computation, and its runtime varies depending on the number of optimization iterations. \nWe have included a runtime versus performance trade-off **Table** in our response to **Reviewer s1NL’s Q1**. \nWith the parameters used in our main experiments, the proposed optimization takes approximately 4.6 seconds per image based on our Python implementation. \nIf real-time performance is desired, further speed-ups can be achieved by reducing the number of iterations.\n\n## **Q6) Thin Structures and Fine Details**\nLearning-based methods often preserve edge-level details better because they directly regress depth from image features, although the resulting 3D points may lack geometric accuracy.\nIn contrast, our optimization-based approach focuses on enforcing geometric consistency, but may underperform in preserving very thin structures due to the smoothing nature of graph optimization.\nThis limitation could potentially be mitigated by incorporating additional priors, such as semantic segmentation, to selectively break graph connections near object boundaries.\nHowever, this would increase computational complexity and was considered beyond the scope of this work.\n\n## **Q7) Visualization of Limitation**\nIn some cases, such as thin structures like chair legs that occupy only a small number of pixels, our optimization may smooth out the depth and reduce detail.\nTo help clarify this limitation, we will add qualitative examples of such failure cases in the supplementary material."}]

Author rebuttal:

Meta-review: The paper presents a training-free method for 360-degree depth estimation by combining perspective foundation models with graph-based scale alignment. Strengths include technical soundness, robust zero-shot generalization across datasets, and practical benefits in downstream tasks. Concerns were raised about overstated originality, the fairness of comparisons with baselines trained on smaller datasets, and limited evaluation on outdoor or degraded imagery. The rebuttal addressed runtime and robustness questions convincingly, and the authors committed to clarifying model details and expanding evaluations. Overall, the method is well-motivated, impactful, and reproducible, leading to the recommendation of acceptance.
Decision: Accept (poster)