Title: ST$^2$360D: Spatial-to-Temporal Consistency for Training-free 360 Monocular Depth Estimation
Venue: NeurIPS 2025
Accepted: yes
Keywords: 360 camera

monocular depth estimation
Abstract: 360-degree monocular depth estimation plays a crucial role in scene understanding owing to its 180-degree by 360-degree field-of-view (FoV). To mitigate the distortions brought by equirectangular projection, existing methods typically divide 360-degree images into distortion-less perspective patches. However, since these patches are processed independently, depth inconsistencies are often introduced due to scale drift among patches. Recently, video depth estimation (VDE) models have leveraged temporal consistency for stable depth predictions across frames. Inspired by this, we propose to represent a 360-degree image as a sequence of perspective frames, mimicking the viewpoint adjustments users make when exploring a 360-degree scenario in virtual reality. Thus, the spatial consistency among perspective depth patches can be enhanced by exploiting the temporal consistency inherent in VDE models. To this end, we introduce a training-free pipeline for 360-degree monocular depth estimation, called ST²360D. Specifically, ST²360D transforms a 360-degree image into perspective video frames, predicts video depth maps using VDE models, and seamlessly merges these predictions into a complete 360-degree depth map. To generate sequenced perspective frames that align with VDE models, we propose two tailored strategies. First, a spherical-uniform sampling (SUS) strategy is proposed to facilitate uniform sampling of perspective views across the sphere, avoiding oversampling in polar regions typically with limited structural details. Second, a latitude-guided scanning (LGS) strategy is introduced to organize the frames into a coherent sequence, starting from the equator, prioritizing low-latitude slices, and progressively moving toward higher latitudes. Extensive experiments demonstrate that ST²360D achieves strong zero-shot capability on several datasets, supporting resolutions up to 4K.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nST²360D aims for monocular depth estimation from 360° images by incorporating the video depth estimation model (VDE). The authors first reproject an equirectangular image into perspective patches at sampled regions. Given a sequence of sampled perspective patches, VDE estimates depth with temporal consistency across patches. The estimated patch depths are reprojected to the equirectangular image and mean-/Poisson-blended to get a final 360° depth map. Authors proposed spherical uniform sampling and latitude-guided scanning strategy to reduce redundant sampling at polar region and feed richful information in the initial frame into VDE. These strategies enhance the depth estimation performances of the suggested approach. As a result, ST²360D achieves high zero-shot performance on high resolution images without additional training or fine-tuning.\n\nSTRENGTHS AND WEAKNESSES:\nStrength\n+ ST²360D investigated the first-time approach of incorporating a video depth estimation model to 360° monocular depth estimation. The authors have suggested simple but effective strategies to enhance 360° depth estimation with existing foundation models. The proposed contributions also improved the depth estimation accuracies.\n+ Authors show that temporal consistency by VDE across patches on 360° images is as effective as additional fine-tuning with 360° synthetic datasets (Table 1). Moreover, this method is generalizable since it does not need training or fine-tuning.\n+Authors have conducted thorough evaluations and ablation on the method. Experiments show the strength of ST²360D on the zero-shot performance and high-quality results in high-resolution images. Ablation shows the importance of temporal consistency and suggested strategies in the final performance while searching for the best variable sets for achieving the best performance.\n\nWeakness\n\n+ Evaluation is focused on quantitative evaluation, while qualitative comparison is insufficient. Figures mostly show the depth map result of ST²360D variations without visualizing comparisons methods’ depth maps.\n\nQUESTIONS:\n1. I suggest including figures that compare depth maps for each comparison method with an error map. This will better depict the geometrical accuracy of ST²360D.\n2. I am curious about the depth quality of ST²360D at edge regions. Does temporal consistency and Poisson blending guarantee the sharpness of depth maps then fine-tuning the network with synthetic 360° datasets?\n3. In Figure 6, is there an expected reason for degrading accuracy even with SUS and LGS as the number of samples increases? In my sense, temporal consistency across the 360° images should be strengthened thereby reducing the RMSE.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel 360 depth estimation pipeline called ST^2360D. To cover the nature of 360 images that usually contain high-resolution spatial information, the proposed method treats a single 360 image as a sequence of perspective images and successfully utilizes existing video depth estimation methods. The authors suggest two strategies (SUS and LGS) for the seamless adaptation of the video depth estimation model into the 360 image domain. The comprehensive experimental results demonstrate that the proposed pipeline outperforms the previous monocular 360 depth estimation methods and shows the effectiveness of its design choices with two proposed modules.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n- Problem formulation: The authors newly reformulate the 360 depth estimation task as a video depth estimation task and successfully improve the depth quality, especially in terms of depth consistency, where the previous methods suffer from seamless aggregation.\n\n- Training-free pipeline: The proposed pipeline is training-free, and it shows fast inference speed compared to the previous method. Moreover, thanks to its training-free pipeline, the proposed method can easily be expanded to other tasks by utilizing various 4D perception models.\n\n- Well-constructed writing: The paper is well-written and easy to follow. Experimental results are helpful to understand the design choices and effectiveness of each proposed module.\n\n\n### Weaknesses\n- Lack of its technical novelty: Although the proposed pipeline is novel and practical, the technical contribution of the paper is insufficient. Specifically, spherical-uniform sampling (SUS) is already commonly used in 360 3D perception fields (SpherePHD [CVPR’19], PanoFormer [ECCV’22], Elite360D [CVPR’24]).\n\n- Comparison methods: In Table 2, data-specific methods (HoHoNet, SliceNet, UniFuse) are pretty outdated. To validate the effectiveness of the proposed method, it is recommended to compare it with recent 360 depth estimation methods (e.g., HRDFuse [CVPR’23], Elite360D [CVPR’24], PanDA [CVPR’25], etc.).\n\nQUESTIONS:\n- More details about depth-controlled generation experiments are not described in the supplementary material (refer to L310 in the main paper). I wonder how the authors obtained the results in Fig. 7.\n- Could you describe the dead-end cases in the viewpoint searching procedure?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a training-free method for 360 monocular depth estimation. Existing training-based methods rely on expensive 360 depth estimation and have limited zero-shot capability. Existing training-free methods are time-consuming or rely on 360 depth models. The proposed method addresses these limitations through sequentializing the input equirectangular projection (ERP) image into a sequence of patches and processing the sequence via a pre-trained large zero-shot video depth estimation model. A Spherical-Uniform Sampling (SUS) Strategy and a Latitude-Guided Scanning (LGS) Strategy are proposed to sequentialize the ERP image. Experiment results show that the proposed method possesses comparable zero-shot capability to SOTA training-based methods, and outperforms existing training-free methods by a large margin.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength**\n+ The idea of using video depth estimation models to solve the 360 monocular depth estimation task is interesting and novel.\n+ The experiment results are impressive, compared to both training-based and training-free methods.\n+ Substantial designs of sequentializing strategies are tested.\n\n**Weakness**\n+ The illustration of latitude-aware traversing is not clear enough. It is better presented using an algorithm block or in equations. \n+ The inference efficiency of the proposed method is only compared with one baseline method. More methods should be included. Furthermore, GPU memory consumption, which is also important, is not given.\n\nQUESTIONS:\n1. Table 10 shows that 360monodepth takes 20 views as input, which is far less than the proposed method. How will it perform if more views are used?\n2. Can you compare the inference time efficiency of your method with more baselines and include GPU memory consumption?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a training-free method for depth estimation from 360-degree images by leveraging video depth estimation models on undistorted image patches. The approach employs spherical uniform sampling and latitude-guided scanning to achieve efficient sampling, avoiding oversampling at polar regions and prioritizing the equator to enhance temporal consistency in depth predictions. Resulting in state-of-the-art performance on Stanford2D3D and Replica360 datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n* The method is innovative in its ability to directly utilize established foundational models without requiring additional training, enhancing its practical applicability.\n* The use of spherical uniform sampling and latitude-guided scanning effectively ensures temporal consistency, with experimental results demonstrating superior performance compared to existing methods.\n* The paper provides thorough experimental evidence to support the novelty and significance of the proposed method, strengthening its contributions.\n* The paper is well-structured and accessible, making the methodology and results easy to follow for readers.\n\nWeaknesses\n* The latitude-guided scanning method assumes that information near the equator (zero latitude) is dominant and serves as an optimal starting point for achieving temporal consistency. This assumption may not hold for outdoor 360-degree datasets, where regions near 90-degree latitudes could contain critical information, potentially limiting the method’s generalizability.\n* The latitude-guided scanning approach introduces randomness (noted in line 209, particularly that the algorithm could encounter a \"dead end\" during scanning), which may lead to instability or inconsistent results.\n\nQUESTIONS:\n* How robust is the LGS method when applied to outdoor 360-degree datasets, where high-latitude regions may contain significant information?\n\n* Why does the LGS method incorporate randomness rather than a deterministic scanning strategy? What specific measures could be implemented to ensure more consistent results?\n\n* Table 10 indicates that the proposed method has a significantly shorter inference time compared to 360MonoDepth, despite requiring sequential inference. What factors contribute to this efficiency, given that 360MonoDepth does not rely on sequential processing?\n\nLIMITATIONS:\nyes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}