IEBins: Iterative Elastic Bins for Monocular Depth Estimation
Authors: Shuwei Shao
Zhongcai Pei
Xingming Wu
Zhong Liu
Weihai Chen
Zhengguo Li
Venue: NeurIPS 2023 poster (2023)
Keywords: Monocular depth estimation
Iterative refinement
Deep learning
TL;DR: A novel iterative elastic bins strategy for the classification-regression-based monocular depth estimation.

Abstract: Monocular depth estimation (MDE) is a fundamental topic of geometric computer vision and a core technique for many downstream applications. Recently, several methods reframe the MDE as a classification-regression problem where a linear combination of probabilistic distribution and bin centers is used to predict depth. In this paper, we propose a novel concept of iterative elastic bins (IEBins) for the classification-regression-based MDE. The proposed IEBins aims to search for high-quality depth by progressively optimizing the search range, which involves multiple stages and each stage performs a finer-grained depth search in the target bin on top of its previous stage. To alleviate the possible error accumulation during the iterative process, we utilize a novel elastic target bin to replace the original target bin, the width of which is adjusted elastically based on the depth uncertainty. Furthermore, we develop a dedicated framework composed of a feature extractor and an iterative optimizer that has powerful temporal context modeling capabilities benefiting from the GRU-based architecture. Extensive experiments on the KITTI, NYU-Depth-v2 and SUN RGB-D datasets demonstrate that the proposed method surpasses prior state-of-the-art competitors. The source code is publicly available at https://github.com/ShuweiShao/IEBins.

IEBins: Iterative Elastic Bins for Monocular Depth Estimation
Shuwei Shao', Zhongeai Pei Xingming Wu', Zhong Liu', Weihai Chen and Zhengguo Li
School Automation Science and Electrical Engincering, Beihang University, China School of Electrical Engineering and Automation, Anhui University, China SRO department, Institute for Infocomm Research, A A*STAR, Singapore * *Corresponding author, email:whchen @ buaa.edu.cn
Abstract
Monocular depth estimation (MDE) is fundamental topic of geometric computer vision and technique for many downstream applications. Recently, several methods reframe the MDE as a cnaifficaatoo-rereesssiii problem where a linear combination of probabilistic distribution and bin centers is used to predict depth. this paper, we propose a novel concept of iterative elastic bins (IEBins) for the rerffiiattion-reessiiooo----oo high-quality depth by progressively optimizing the search range, which involves multiple stages and each stage performs finer-grained depth search in the target bin on top of its previous stage. To alleviate the possible error accumulation during the iterative process, target bin, the width of which is adjusted elastically based on the depth uncertainty. Furthermore, we develop a dedicated framework composed of a feature extractor and an iterative optimizer that has powerful temporal context modeling capabilities benefiting from the GRU-based architecture. Extensive experiments on the KITTI, NYU-Depth-v2 and SUN RGB-D datasets demonstrate that the proposed method surpasses prior state-of-the competitors. The source code is publicly available at /https:://gtthii com/
MDE. The proposed IEBins aims to search for
utilize a novel elastic target bin to replace the original
IEBiNS
Introduction
Monocular depth estimation (MDE) is a long-standing and fundamental topic in geometric computer vision, with many applications in robotics [I]. 3D reconstruction (2), scene understanding [3, etc. It consists in inferring the depth map from a single RGB image, which is ill ill-posed and has the challenge of scale ambiguity, because the same 2D image can be projected from infinitely many 3D scenes. Recently, more and more learning-
approaches have been proposed to promote the development of MDE 4 5 6 g 9 TO 11). Without loss of integrity, these methods can be grouped into three categories: regression, classification and classification-reeresssi
12
Regression is the most primitive and straightforward formulation 4 directly generates continuous pixel-wise depth under the supervision regression loss. Despite model suffers from unsatisfactory
16 which
its great success as universal paradigm, the regression- results [5). Classification is proposed in (3) and [17] to formulate the MDE as per-pixel classification and predict the optimal depth interval. More specifically, it discretizes the full depth range into multiple intervals (bins) in uniform/Log-unitnn
space (Fig (a) and (b)) and takes the bin center (depth candidate) of classified target bin the final depth prediction. While the classification makes the MDE easier and significantly improves the model performance, the poor visual quality with discontinuity artifacts tends to appear [6].
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Uniform Bins
Space Increasing Bins
(c) Adaptive Bins
Iterative Elastic Bins
Depth Candidate
Target
Bin Edge
Elastic Target Din
Elaskk Turget Bin
Target
Figure 1: Illustaaioon of different bin types comprising uniform bins [ space increasing bins [5]. adaptive bins [6] and the proposed iterative elastic bins.
In order to overcome the discontinuity artifacts in classification, some methods 6 18 12
19
20] reframe the MDE as a per-pixel Daasification-regresste problem, learning the probabilistic distribution each pixel and using the linear combination with depth candidates the final depth prediction. Theoretically, can achieve the sub-pixel depth estimation. On top that, Bhat et
al. [6] noticed the extreme fluctuations in depth distribution across different scenes and proposed to
derive adaptive bins (Fig. (c)) from the image content. Li et al. [12] and Bhat et al. [19] further improved [6] by disentangling bins generation and probabilistic distribution learning performing local predictions of depth distributions a gradual step-wise manner. Moreover, Agarwal et al. [20]
developed bin center predictor that uses pixel queries at the coarsest level to predict bins.
In this paper, we introduce novel concept termed iterative elastic bins (IEBins, Fig. (d)), tailored -bsed MDE. The IEBins leverages multiple small number of bins, instead of one standard number bins, to search for high-quality depth by progressively reducing the search range. To specify, the proposed IEBins involves multiple stages, where each stage predicts depth different granularities and performs finer-grained depth search the target bin (the bin in which the predicted depth is located in case) of its previous stage. Unfortunately. the depth ground- truth may fall outside the target bin due to wrong depth predictions, resulting in unstable optimization and degraded accuracy. To mitigate the eπror accumulation during iterations, clastically adjust the width of the target bin according to the depth uncertainty that indicates the potential depth errors. Inspired by (21), utilize the variance of the probabilistic distribution quantify the uncertainty. Last but not least, we develop a dedicated framework (Fig. 2 consisting of two main components: a feature extractor that generates strong feature representations and gated recurrent unit (GRU)-bassee iterative optimizer with powerful temporal context modeling capabilities that predicts the per-pixel probabilistic distribution from its hidden state for the depth dlasiffiationnreeressio.
for the
To summarize, our main contributions are three-fold:
We introduce a novel iterative elastic bins strategy for the classification ceatffccatioo-reresssion based MDE. The IEBins performs an iterative elastic search using multiple small number of bins in light of the depth uncertainty.
We develop a framework to instantiate the proposed IEBins, where a feature extractor attains strong feature representations and GRU-based iterative optimizer predicts the probabilistic distribution.
Extensive experiments are conducted on the KITTI 22), RGB-D [4]] datasets, and the experimental results show that the proposed method exceeds previous state-of-the-art competitors.
NYU-Depth-v2 [33 and SUN
2 Related work
Monocular depth estimation. Learning-basee MDE has witnessed tremendous progress in recent years. Saxena et al. (27) proposed pioneering work that uses Markov Random Field capture critical local- and global-image features for depth estimation. Later, Eigen et al. [4] introduced convolutional neural network (CNN)-based architecture to attain multi-scale depth predictions. Since then, CNNs have been extensively studied in MDE. For instance, Laina et al. (28) utilized residual CNN [29 for better optimization. Recently, Transformer (30) has attracted widespread attention
in the computer vision community 3] 32 33 Following the success of visual Transformer in

---

Feature Extractor
Iterative Optimizer
Input. Image
Probabilistic Distribution
Depth Map
Canditatts
Inear
Depth
Combination
GRU
lEBins
Hidden Stale
Probabilistic Distribution
Depth Map
Depths Candidatess
Linear
C Combination
GRU
Stageo
Hidden Probabilistic Distribution
Depth Map
Cax
Imear C Combination
Deyth
Projection
GRU
psample
An overview of the whole framework. The upsample stands for the pixel shuffle [25]. The projection is achieved using four 3 x convolutional layers followed by the ReLU activation (26
Figure 2:
other tasks, Yang er al. [4], Ranftl et al. [34 and Yuan er al. [8] replaced CNN with Transformer, further improving the performance. However, the above methods suffer from sub-optimal solutions induced by the inherent drawback of regression [5). Fu el al. and Cao et al. [17] proposed
to formulate MDE classification problem and discretized the full depth range into multiple
bins to predict the optimal depth interval. Diaz al.
(355 softened the classification label in
during the training phase. In addition, Bhat al. [6] Johnston al. [18]. Li et al. [12). Bhat er
al. [19] and Agarwal et al. (20) reframed the MDE as per-pixel classification regressioonnrreesooonnnoooo to mitigate the discontinuity artifacts caused by depth discretization. Among them, [6] introduced an adaptive bins strategy to boost the performance. In contrast, we propose an iterative elastic bins paradigm for the classification-regnneee regreesionn based MDE, which uses multiple small number of bins rather than one standard number of bins like 6 [9] also refines the binning structure in an iterative manner. However, [19 divides all bins from the previous stage at each stage, and when the stage increases, the number of bins increases, while the proposed IEBins locates and divides the target bin only, and the number of bins is not changed at different stages.
Iterative refinement. Recently, Teed et al. (36) proposed to iteratively refine a displacement vector field through the GRU for optical flow estimation, emulating the first-order optimization. This idea gradually receives attention in other tasks, such as stereo [37 38], structure from motion [39] and scene flow [40]. In this paper, deploy GRU-based iterative optimizer to predict the per-pixel probabilistic distribution, where the hidden state is updated at each stage for more accurate estimation.
Methodology
In this section, first introduce the iterative elastic bins tailored for the classification- -eegressioo--- based MDE. Then, we demonstrate a detailed description regarding the feature extractor and iterative optimizer in our framework. Finally, we present the training loss function.
3.1 Iterative Elastic Bins
The proposed IEBins embodies the idea iterative division of bins, and composed of two parts, initialization and update. In the initialization stage, we perform coarse and uniform discretization of the full depth range. During each subsequent stage, we follow an iterative process to locate and uniformly discretize the target bin by using the target bin as the depth range. The details presented below.
Initialization. To initialize the IEBins, we discretize the full depth range [diiin, dmax] into N bins in the uniform space,
en = duin + nB, n 00 1,.,, N,
with
B dmax dmin
(2)
3

---

Stage 1
Stage2
3tagp3
Stagp4
Stage
Stagg
Illustration of depth and uncertainty maps for each stage. The upper row is depth maps, which we display using the bin center of every target bin to better visualize the refinement process. The lower row is uncertainty maps (yellow/red: high/highest uncertainty; blue: low uncertainty).
Figure 3:
where en denotes the n-th bin edge, B denotes the bin width, dmaa and daai are set to 80 and 0.1, and 10 and 0.1 for KITTI [22] and NYU-Depth-v2 [23] datasets, respectively, and N is set to 16 when not otherwise specified, significantly fewer number of bins than the 256 bins used in AdaBins [6]. As we cannot directly use discrete bins to predict depth, we take the bin centers to represent the depth candidates of bins,
D, = Cn + Cn-1 n = 1.....,,N 1
2
where D,, denotes the n-th depth candidate. Once the per per-pixel probabilistic distribution associated with depth candidates is predicted, we can acquire depth prediction via their linear combination
N-1
D
Σ D. Pm (p),
(4)
where D (p) denotes the predicted depth, denotes the pixel coordinate and P., denotes the n-th depth probability.
Update. Each subsequent stage searches at finer-grained granularity in the target bin of its previous stage. The target bin represents the corresponding bin in which the predicted depth is located, which obtain by comparing the predicted depth with bin edges
Gn S (p) Cn+1
(S)
and denote as [en, 1n. However, discussed in the introduction section, the depth ground-truth may fall outside the target bin due to depth prediction errors. In this case, the errors will gradually accumulate the stage increases, resulting in unstable optimization and decreased accuracy. To
account for such a failure case and make the paradigm more robust, we propose to leverage the elastic target bin update the depth candidates. The width of the elastic target bin is adjusted flexibly based on the depth uncertainty reflecting the likelihood of depth inaccuracies. Inspired by [21], we capture the uncertainty via the variance of the probabilistic distribution. In particular, the variance V (p) is calculated as
N-1
V (p)
Σ Dr - -D(p)) 2 P.) (p) n=
(6)
The corresponding standard deviation is Ô (p) = V (p). Given the target bin [en, [en,en+] en+il and standard deviation ā (p), we can acquire the elastic target bin by
[en - kō (p), ent1 + Kō (p)]
where x is coefficient that determines the error tolerance and set to 0.5. For a pixel with severe depth errors, the standard deviation will become larger such that the elastic target bin has higher immunity against errors. On top of that, we renew dini and d as e,, -0.55 (p) and 6n+1+ +0.56 (p). respectively. Through Eqs [ 3 and _ we can achieve the updated depth candidates and prediction. The update step will repeated until reaching to the final stage. should be noted that although we do not show p behind D in above formulas, the depth candidates D can actually be spatially-varying due to different elastic target bins subsequent update steps. In Fig. we display examples of depth maps and associated uncertainty maps each stage.

---

3.2 Feature Extractor
The feature extractor adopts an mecodr-ddcoodr structure with sxip-connections. The encoder uses the recently proposed Swin-Trame Transforoerr family backbones [33, which take an RGB image of size x W as input and generate a four-level feature pyramid. Then, the skip-connections propagate the pyramid features into the decoding phase. The decoder uses three neural conditional random field (CRF) modules [ to capture the vital long-range correlation. We alternate between the neural CRF module and pixel shuffle [25] up the !! x W resolution. Next, the ! W resolution feature maps from the encoder and the decoder are respectively sent to the iterative optimizer as the context feature and the initialization of GRU hidden state.
3.3 Iterative Optimizer
To efficiently predict the probabilistic distribution each stage, we deploy a GRU-based iterative optimizer taking inspiration from (36), since the GRU capable retaining the information from history stages and can fully exploit the temporal context during iterations. The optimizer operates H. W resolution.
T T
More specifically, we first project the depth candidates D into the feature space using four 3 x 3 convolutional layers and each convolutional layer is followed by ReLU activation [26]. We then concatenate the projected feature and the context feature to constitute tensor Ist as the input. The structure inside GRU is
k sigmoid (Conusx5 (n+ [+] W.))
x+1 sigmoid (Coneixs [ht, 1 ..,))
kk+1 tanh (Conusxs (1Rtt ② h², 1h, Wn)): h
(10)
k+1 = (1- (1-z h +
(11)
where k is the stage index, z is the update gate, r is the reset gate, Conusxs is the separable 5 x convolution, is the element-wise multiplication, W., W, and W, stand for learnable parameters. The hidden state h initialized by the H 4 x 4 resolution output in the decoder. The probabilistic 3 convolutional layers.
W
distribution is finally predicted from the updated hidden state with Meanwhile, the feature maps are regularized by ReLU and Softmax activations, respectively. A linear combination of probabilistic distribution and depth candidates is applied obtain the depth prediction, which is further upsampled to the original resolution by the bilinear interpolation.
Equipped with this optimizer, initiating coarse prediction, the predicted depth is iteratively refined and eventually converges to the final result.
3.4 Training Loss Function
Pixel-wise depth loss. Following 8 6. we leverage a scaled Scale- Invariant loss for depth supervi-
sion 7.
K Σ
2
Cpinel =
a 1 Eg(P))
Eg(p) p
(12)
[T|
where g (p) log D (p) log ad (p), K is the maximum number of stages and is set to 6. T stands for the set pixels having valid boound-truth values, a and B are set to 10 and 0.85 based on 77.
Experiment
We evaluate the proposed method on both outdoor and indoor datasets, which include KITTI [22). NYU-Deptth-2 [23] and SUN RGB-D 24). the following, we start by introducing the relevant datasets, evaluation metrics and implementation details. Then, we present quantitative and quali- tative comparisons to prior sfate-of-the competitors, generalization and ablation studies, model parameters and inference time comparison.
5

---

Method
Backbone
Abs Rel
Rel.
RMSE
RMSE
F L.25
d< 1,257
6xc 1.25
ResNer-10l 181
0.072
0.084
0.994
ResNeXI-101
0.908
DenseNer-61
0.060
0.249
2.758
0.995
0.988
PWA 5
ResNeXt-101
0.060
0.221
2,604
0.093
0,958
0.994
11.999
TransDophh
14
R-50-VIT-R/161
0.064
0.252
2,755
0.008
0.956
0.094
0.999
AdaBins
E-B5+mini-VV VIT
0.058
0.190
2,360
0.964
0.995
0.999
P3Depth
NeWCRFs
Swin Swin-Large"
0.052
0.155
2.129
0,974
0.997
0,999
BinsFormer
P
Swin-Tiny
0.058
0,183
2.286
0.088
0,968
0.995
0,999
BinsFormer
P2
Lange
0.052
0.151
2,098
0.079
0.974
0.997
0,999
PixelFormer
20
Swin Swin-Large"
0.051
0.149
2,081
0.077
0.976
0.997
0.999
Ours
Boii--Tin
0.056
2.205
0.970
0.996
0.999
Swin-Lange
Table Quantitative depth comparison on the Eigen split of KITTI dataset. We provide results of the proposed method based on Swin-Large and Swin-Tiny backbones [33. The maximum depth is capped at 80m. R-50 and E-B5 are the abbreviations of ResNet-50 [29] and EfficientNct-BB [45), respectively. f indicates that the models pre-trained by ImageNet-22K. s-' means not applicable. The best results are marked in bold.
Method
datnset
SIl
Abs Rel
Sq Rel
IRMSF
RMSE
1.25
4 1.25"
& 1.25%
BTS
validation
10.67
7.51
8.10
3.37
0:987
0.996
BA-Pull Md
validation
10.64
8.25
1.81
8.47
3.30
0.938
0.988
0.997
NeWCRFs X
validition
8.31
5.54
0.89
6.34
2.55
0.968
0.905
0.998
Ours
validation
7.58
5.10
0.75
5.90
2.37
0.974
0.996
0.999
DORN [5]
online test
11,77
8.78
2.23
12.98
BTS M
unline test
11.67
9.04
2,21
12:23
BA Full 66
unline test
11.61
9.38
2.29
12.23
12.32
Vip-Deeplub
+
online
NeWCRF:
3
online test
10.39
8.37
1,83
11.03
PiselFormer
200
online test
10.28
8.16
1.82
10.84
BinfFormer
unline test
10.14
8.23
1.69
10.50
online test
Table 2: Quantitative depth comparison on the official split of KITTI dataset. The SILog is the main ranking metric.
4.1 Datasets and Evaluation Metrics
KITTI an outdoor dataset captured by equipment mounted moving vehicle, providing stereo images and corresponding 3D laser scans. The images are around 376 x 1241 resolution. Here we adopt two data splits, Eigen training/testing split [4] and official benchmark split [48]. The former uses 23488 left view images for training and 697 images for testing. The latter consists of 85898 training images, 1000 validation images and 500 test images without the depth ground-truth, The evaluation results on the official benchmark split are generated by the online server.
NYU-Depth-y2 indoor dataset that has RGB images and ground-truth depth maps 480 x 640 resolution. We evaluate the proposed method on the official data split, which involves 36253 images for training and 654 images for testing.
SUN RGB-D is collected from indoor scenes with high diversity using four sensors, containing roughly 10K images. The dataset is only used for zero-shot generalization study and the official 5050 test images are adopted.
Evaluation metrics. Similar to previous work g. we leverage the standard evaluation protocol to validate efficacy of the proposed method experiments, i.e., relative absolute error (Abs Rel), relative squared error (Sq Rel), root mean squared error (RMSE), root mean squared logarithmic error (RMSE log), inverse root mean squared error (iRMSE), logio error (log1o). threshold accuracy (8 < 1.25, 8 < 1.25 and 1.253) and square root of the scale invariant logarithmic error (SILog).

---

Method
Rackhome
Abs Rel
Rel
RMSEL
10g
4 1:25
1,25"
DORN
ResNes-101 101
0.115
0.509
0.081
0.828
0,968
0.992
VNL
22
ResNeXt-100
0.108
0.416
0.048
0.875
0,976
0.994
BTS
DemseNes-161
0.110
B.066
0.392
0.047
0.885
0.788
0.994
DenseNes-161
Long
HRNaa-18
Depoh
R-50+ ViT-R/66
0.365
AdaBins
E.B5-mini-V VIT
0.103
0.364
0.044
0,903
0,984
0.997
P3Dephh
ResNe-101 101
0,104
0.356
0,043
0
0
0.996
L LocalBins
1v
E-B5
0.099
0.357
0.042
0.907
0.987
0.998
NeWCRFss
Swin Larget
0.095
0.045
0.334
0.041
0.922
0,992
0.998
BinsFormeri
Swin-Tiny
0.113
0.379
0.047
0.890
0.983
0.996
BinsFormert
Swin-Largei
0.094
0.330
0.040
0.925
0,989
0.997
PixelFomer
XI
Swin-1 larget
Swin-Thy
0.108
0.375
0.046
0.803
0.996
Swin-Largef
Table 3: Quantitative depth comparison on the NYU-Depth-v2 dataset. The maximum depth is capped 10m. $ stands for that the model is trained using auxiliary scene class information.
Method
Backbone
Abs
RMSE
10g
5 1.25
8< 1.2551
1,257
SENet
0.494
0.071
0.757
4M
ResNeXt-101
BTS
DemseNet116 161
0.172
0,515
0.075
0,740
0,933
0,980
AdaRins
E-B
0.159
0.476
0088
0.771
0.944
0,983
LocalBins
P
E-B5
0.156
0.470
0.077
0.777
0.949
0.985
PixelFormer
20
Swin-Large
0.141
0.411
0.022
0.802
0.962
0,990
BinsFormer:
Swin: Tiny
P2
Swin-Lange
Ours
Swin: Tiny
0.157
0.476
0.069
0.768
0,950
0.987
Oues
Swin-1 ange"
0.135
0.485
0.059
0.822
0,971
0.993
Table 4: Generalization to the SUN RGB-D dataset in zero-shot setring with models trained on the NYU-Depth-v2 dataset.
4.2 Implementation Details
Our framework is implemented in the PyTorch library (50) and trained on 4 NVIDIA A5000 24GB GPUs. The training process runs a total number of 20 epochs and takes around 24 hours. We utilize the Adam optimizer [5T] and a batch size of 8. The learning rate is gradually reduced from 2e-5 to 6 via the polynomial decay strategy.
4.3 Comparison to previous state-of-the-are competitors
KITTL. We report results the Eigen split and official benchmark split, as summarized in Tables | [ respectively. For the Eigen split, the proposed method exceeds the leading approaches by a large margin, e.g., compared to PixelFormer, which is also based on the seesificaatin-regressioi In terms of the official benchmark split, the results are generated by the online server and the proposed method outperforms previous approaches again. It is worth noting that the main ranking metric SILog is reduced considerably.
In Fig. 4 we present qualitative depth comparison on the KITTI dataset. As we can see, the proposed method is capable of correctly predicting the depth of foreground objects, such as poles, even when the background is very complex, e.g., lush foliage and bushes. In such complex scenes, PixelFormer and NeWCRFs are prone to foreground and background depth aliasing. Morcover, we notice that the compared methods, especially PixelFormer, tend to have mosaic-like artifacts the top of the depth maps, but the proposed method does not.
NYU-Depth-y2. To further present the superiority of the proposed method in the indoor scenario, we evaluate it on the NYU-Depth-v2 dataset. The results are reported in Table indicating that the proposed method surpasses prior competing approaches and achieves consistent improvements on most metrics. particular, improves BinsFormer by 7.4% and 4.8% on the Abs Rel and RMSE, respectively, which emphasizes the efficacy of our IEBins. In Fig. 5 demonstrate qualitative depth
7

---

Input
Pixelf ormer
NeW CRFs
Ours
Figure 4: Qualitative depth comparison on the KITTI dataset. The red boxes show the regions to focus on.
Input
GT
BTS
AdaBins
PixelFormer
NeWCRFs
Ours
Figure 5: Qualitative depth comparison on the NYU-Depth-v2 dataset. The white boxes indicate the regions focus
comparison. As can be seen, the proposed method preserves fine- aeee-graied details, such as boundaries and generates more continuous depth values in planar regions.
4.4 Zero-shot Generalization
Similar to previous approaches 6 12]. conduct cross coss-ettaset evaluation in zero-shot setting where the models are trained on the YYU-Depth-v2 dataset but evaluated on the SUN RGB-D dataset. As shown in Table the proposed method achieves superior results than the compared approaches. Besides, we notice that the proposed method with Swin-Tiny backhone performs slightly worse than AdaBins the NYU-Depth-v2 dataset, but the SUN RGB-D even surpasses AdaBins some metrics like Abs Rel, which is an indicator its excellent generalization ability.
4.5 Ablation Study
To better indicate the influence of each individual component, we conduct several ablation studies, divided into IEBins, bin types and effect of bin numbers.
IEBins. The importance of IEBins is first evaluated by comparing it with standard regression and IBins. We construct baseline by removing the iterative optimizer from the whole framework. The standard regression is achieved by using the baseline to directly predict the depth map. The IBins represents that replace the elastic target bin with the original target bin at each stage. Table shows that the standard regression performs worse than IBins and IEBins. Benefiting from the elastic target bin, the more robust IEBins takes the IBins step further.
Bin types. We then compare the IEBins against other choices including uniform bins [5) space increasing bins [5) adaptive bins 6 and local bins [19 To make fair comparison, we remain all other configurations the same except for bin types. As listed Table 55 the baseline equipped with IEBins improves the performance markedly and surpasses other variants.

---

Method
Abs Rel
RMSE L
login
8 < 1:25
8 < 1.25
8 1.25
Baseline Regression
0.095
0.337
0.041
0,921
0.991
0.998
Baseline- + UBins [S]
0.091
0.328
0.040
0.925
0.991
0.998
Baseline SIBins 5
0.090
0.326
0.039
0,928
0.992
0.998
Baseline + AdaBins 6
0.089
0.320
0.038
0,931
0.991
0.998
Baseline LocalBins
0.319
0.992
Bascline IBins
0,090
0.317
0.038
0.932
0.991
0.998
Baseline IRBins
0.087
0.314
0.038
0.936
0.992
0.998
Table 5: Comparison of different bin types on the NYU-Depth-r22 dataset. The baseline is built by removing the iterative optimizer from our whole framework. UBins: uniform bins; SIBins: space increasing bins; AdaBins: adaptive bins; LocalBins: local bins; IBins: iterative bins, which stands for replacing the elastic target bin with the original target bin at each stage. The number of bins for UBins, SIBins, AdaBins and LocalBins is set to 256 following 6 [9].
Number of Bins
Abs Rel 4
RMSE І
1og10
< 1.25
1.253
<< 1.25*
4
0.105
0.335
0.043
0,900
0.982
0,999
8
0.089
0.317
0.038
0.934
0.992
0.998
16
0.087
0.314
0.038
0.936
0.992
0.998
32
0,088
0.317
0.038
0,932
0.992
0,998
Table 6: Effect of bin numbers on the NYU-Depth-v2 dataset.
Method
Backbone
Abs Rel |
Parameters (M)
Inference Time (s)
NeWCRFss
8
Swin-Largef
0.095
270
0.052
BinsFormer;
12
Swin-Largef
0.094
0.216
Ours
Swin-Largef
0.087
273
0.085
Table 7: Comparison model parameters and inference time on the NYU-Depth-v2 dataset.
Effect of bin numbers. To examine how the number of bins affects the performance, we train our model using different values of the bin number. The results reported in Table The error drops sharply as the number of bins increases, and then this drop disappears when is greater than 16 bins. Therefore,
16 bins in our final model.
4.6 Model Parameters and Inference Time
We conduct a comparison between the proposed method, NCWCsFs and BinsFormer based on their inference time and the number of model parameters in Table 7 with the Swin-Large backbone. We measure the inference time on the NYU-Depth-Ves v2 test set with a batch size of 1. can be seen that the number of parameters of the proposed method is almost equal to that of NeWCRFs and slightly more than that of BinsFormer. Nevertheless, the proposed method is nearly 60% faster than BinsFormer in inference time, also the ressfffaationrreresssion based method. Meanwhile, the proposed method achieves much better performance than these two counterparts. Hence, the proposed method provides better balance between performance, number of parameters and inference time.
5 Limitations
We acknowledge the following limitations: First, we use the classification- regrressoonnrr to predict depth. Compared with the classification, the depth acquired by classification-regress regressson smoother and more continuous, but at the same time it may blur the boundaries due to the weighted average of all depth candidates. Second, we use the pixel-wise loss supervise depth without imposing direct supervision signal on the probabilistic distribution. This does not guarantee that the depth candidate corresponding to the peak point the probability distribution is the real optimal depth candidate, thereby affecting the depth estimate.

---

6 Conclusion
In this paper. we introduce a novel concept of iterative elastic bins for the classification-regeessii based monocular depth estimation. The proposed iterative elastic bins uses multiple small number of bins to progressively search for high-quality depth and can be plugged into other frameworks as a strong baseline. In addition, present dedicated framework composed of feature extractor and an iterative optimizer. The performance is evaluated on two popular datasets from both outdoor and indoor scenarios and the proposed method exceeds previous state-of-the-art competitors. Furthermore, its generalization ability is verified in a zero-shot setting on the SUN RGB-D dataset.
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China under grant U1909215 and 51975029, in part by the S*SRAR Singapore through Robotics Horizontal Technology Coordinating Office (HTCO) under Project C2215 18005, in part by the Key Research and Devel- opment Program of Zhejiang Province under Grant 2021 0021C03550 in part by the Scientific Research Project of Agriculture and Social Development of Hangzhou under Grant No. 20212013BI 1,11.00 and in part by the National Natural Science Foundation of China under grant 61620106012 and 61573048.
References
[1] Weibin Jia, Wenjie Zhao, Zhihuan Song, and Zhengguo Li. Object servoing of differential- service robots using switched control. Journal of Control and Decision, 10(3):114-325, 2023.
[2] Shahram Izadi, David Kim, Otmar Hilliges, David Molyneaux, Richard Newcombe, Pushmeet Kohli, Jamie Shotton, Steve Hodges, Dustin Freeman, Andrew Davison, al. Kinectfusion: real-time 3d reconstruction and interaction using moving depth camera. In Proceedings of the 24th annual ACM symposium on User interface software and technology, pages 559 0698566,55 2011.
[3] Po-Yi Chen, Alexander H Liu, Yen-Cheng Liu, and Yu-Chiang Frank Wang. Towards scene understanding: Unsupervised monocular depth estimation with semantic-aware representation. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 6624-2633 2019.
[4] David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from single image using a multi-scale deep network. In Advances in Neural Information Processing Systems, pages 2014.
[5] Huan Fu, Mingming Gong, Chaohui Wang, Kayhan Batmanghelich, and Dacheng Tao. Deep ordinal regression network for monocular depth estimation. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 2002-2011, 2018.
[6] Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka. Adabins: Depth estimation using adaptive bins. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4009-4018, 2021.
[7] Jin Han Lee, Myung-Ky Kyu Han, Dong Wook Ko, and II Hong Suh. From big to small: Multi-scale local planar guidance for monocular depth estimation. arXiv preprint arXiv: 2019.
[8] Weihao Yuan, Xiaodong Gu, Zuozhuo Dai, Siyu Zhu, and Ping Tan. Neural window fully- connected crfs for monocular depth estimation. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 9916-3925, June 2022.
[9] Zhenda Xie, Zigang Geng, Jingcheng Hu, Zheng Zhang, Han Hu, and Yue Cao. Revealing the dark secrets of masked image modeling. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 14475-1442 14485, 2023.
[10] Shuwei Shao, Zhongeai Pei, Weihai Chen, Ran Li, Zhong Liu, and Zhengguo Li. Urede-depth: Uncertainty rectified cross-distillation with cutflip for monocular depth estimation. IEEE Transations on Multimedia, 2023.
10

---

[11] Shuwei Shao, Zhongcai Pei, Weihai Chen, Xingming Wu, and Zhengguo Li. Nddepth: Normal- distance assisted monocular depth estimation. Proceedings of the IEEE International Conference on Computer Vision, 2023.
[12] Zhenyu Li, Xuyang Wang, Xianming Liu, and Junjun Jiang. Binsformer: Revisiting adaptive bins for monocular depth estimation. arXiv preprint arXiv:2204.0000 00987, 2022.
[13] Lam Huynh, Phong Nguyen- Ha, Jiri Matas, Esa Rahtu, and Janne Heikkilä. Guiding monocular depth estimation using depth- depth-attention volume. In European Conference on Computer Vision, pages 581-597, Springer, 2020.
[14] Guanglei Yang, Hao Tang, Mingli Ding, Nicu Sebe, and Elisa Ricci. Transformer-basel attention networks for continuous pixel-wise prediction. In Proceedings of the IEEE International Conference Computer Vision, pages 16269-16279, October 2021.
[15] Xiaoxiao Long, Cheng Lin, Lingjie Liu, Wei Li, Christian Theobalt, Ruigang Yang, and Wenping Wang. Adaptive surface normal constraint for depth estimation. In Proceedings of the IEEE International Conference on Computer Vision, pages 12849-12858, October 2021.
[16] Shuwei Shao, Ran Li, Zhongeai Pei, Zhong Liu, Weihai Chen, Wentao Zhu, Xingming Wu, and Baochang Zhang. Towards comprehensive monocular depth estimation: Multiple heads are better than one. IEEE Transactions on Multimedia, 2022.
[17] Yuanzhouhan Cao, Zifeng Wu, and Chunhua Shen. Estimating depth from monocular images as classification using deep fully convolutional residual networks. IEEE Transactions on Circuits and Systems for Video Technology, 28(1 (8311433774-1188 2017.
[18] Adrian Johnston and Gustavo Carneiro. Self-supervised monocular trained depth estimation using self-attention and discrete disparity volume. In Proceedings of the IEEE International Conference on Computer Vision, pages 4756-4765, 2020.
[19] Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka. Localbins: Improving depth estimation by learning local distributions. In Proceedings of the European Conference Computer Vision, pages 480-496. Springer, 2022.
[20] Ashutosh Agarwal and Chetan Arora. Attention attention everywhere: Monocular depth prediction with skip attention. Proceedings of the IEEE Winter Conference on Applications of Computer Vision, pages 5861-5870, 2023.
[21] Dongting Hu, Liuhua Peng, Tingjin Chu, Xiaoxing Zhang, Yinian Mao. Howard Bondell, and Mingming Gong. Uncertainty quantification depth estimation via constrained ordinal regression. In Proceedings of the European Conference Computer Vision, pages 237-256. Springer, 2022.
[22] Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The kitti dataset. The International Journal of Robotics Research, 32(11):1231-1237 2013.
[23] Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. In European Conference Computer Vision, pages 746-760. Springer, 2012.
[24] Shuran Song, Samuel P Lichtenberg, and Jianxiong Xiao. Sun rgb-d: A Agb-d scene understand- ing benchmark suite. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 567-576, 2015.
[25] Wenzhe Shi, Jose Caballero, Ferenc Huszár, Johannes Totz, Andrew P Aitken, Rob Bishop. Daniel Rueckert, and Zehan Wang. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1874-1883, 2016
[26] Xavier Glorot, Antoine Bordes, and Yoshua Bengio. Deep sparse rectifier neural networks. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pages 315-323, JMLR Workshop and Conference Proceedings, 2011.

---

[27] Ashutosh Saxena, Sung H Chung, Andrew Y Ng, et al. Learning depth from single monocular images. In Advances in Neural Information Processing Systems, volume 18, pages 1-8, 2005.
[28] Iro Laina, Christian Rupprecht, Vasileios Belagiannis, Federico Tombari, and Nassir Navab. Deeper depth prediction with fully convolutional residual networks. In 2016 Fourth International Conference on 3D Vision, pages 239-248. IEEE, 2016.
[29] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for im- age recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June 2016.
[30] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, tukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Informa- fion Processing Systems, pages 5998-6008, 2017.
[31] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 16 words: Transformers for image recognition at scale. International Conference on Leurning Representations, 2021.
[32] Zhiliang Peng, Wei Huang, Shanzhi Gu, Lingxi Xie. Yaowei Wang, Jianbin Jiao, and Qixiang Ye. Conformer: local features coupling global representations for visual recognition. In Proceedings of the IEEE International Conference on Computer Vision, pages 367-376, October 2021.
[33] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang. Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE International Conference on Computer Vision, pages 10012-10022, October 2021.
[34] René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vision transformers for dense prediction. In Proceedings of the IEEE International Conference on Computer Vision, pages 12179-12188, 2021.
[35] Raul Diaz and Amit Marathe. Soft labels for ordinal regression. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4738-4747, 2019.
[36] Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Proceedings of the European Conference on Computer Vision, pages 402-419. Springer, 2020.
[37] Lahav Lipson, Zachary Teed, and Jia Deng. Raft-stereo: Multilevel recurrent field transforms for stereo matching. In 2021 International Conference on 3D Vision (3DV), pages 218-227. IEEE, 2021.
[38] Fangjinhua Wang, Silvano Galliani, Christoph Vogel, and Marc Pollefeys. Itermvs: iterative probability estimation for efficient multi-view stereo. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8606-8615, 2022.
[39] Xiaodong Gu, Weihao Yuan, Zuozhuo Dai, Siyu Zhu, Chengzhou Tang, Zilong Dong, and Ping Tan. Dro: Deep recurrent optimizer for video to depth. IEEE Robotics and Automation Letters, 8(5):2844-2888 28511, 2023.
[40] Zachary Teed and Jia Deng. Raft-3d: Scene flow using rigid-moting embeddings. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8384, 2021.
[41] Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, Furu Wei, and Baining Guo. Swin transformer v2: Scaling up capacity and resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 12009-12019 June 2022.
[42] Wei Yin, Yifan Liu, Chunhua Shen, and Youliang Yan. Enforcing geometric constraints of virtual normal for depth prediction. In Proceedings of the IEEE International Conference on Computer Vision, pages 5684 5693, 2019.
[43] Sihaeng Lee, Janghyeon Lee, Byungju Kim, Fojindl Yi, and Junmo Kim. Patch-wise attention network for monocular depth estimation. Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 1873-1881, 2021.
12

---

[44] Vaishakh Patil, Christos Sakaridis, Alexander Liniger, and Luc Van Gool. P3depth: Monocular depth estimation with a piecewise planarity prior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1610-1621, 2022.
[45] Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural networks. International Conference on Machine Learning, pages 6105-66 6114. PMLR, 2019.
[46] Shubhra Aich, Jean Marie Uwabeza Vianney, Md Amirul Islam, and Mannat Kaur Bingbing Liu. Bidirectional attention network for monocular depth estimation. In 202/ IEEE International Conference on Robotics and Automation, pages 11746-1179 1752. IEEE, 2021.
[47] Siyuan Qiao, Yukun Zhu, Hartwig Adam, Alan Yuille, and Liang-Chich Chen. Vip-deeplab: Learning visual perception with depth-aware video panoptic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3997-4008, 2021.
[48] Jonas Uhrig, Nick Schneider, Lukas Schneider, Uwe Franke, Thomas Brox, and Andreas Geiger. Sparsity invariant enns. 20/7 International Conference on 3D Vision, pages 11-20. IEEE, 2017.
[49] Xiaotian Chen, Xuejin Chen, and Zheng-Ju Jun Zha. Structure- residual pyramid network for monocular depth estimation. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, pages 694-700, 2019.
[50] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary De Vetioo, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. Advances in Neural Information Processing Systems Workshop Autodiff. 2017.
[51] Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization. In Interna- lional Conference on Learning Representations, 2015.
13


Reviews:
[{"review_id": "O4JIbiPuMi", "reviewer": "Reviewer_s85F", "summary": "The paper proposes an iterative elastic bins (IEBins) strategy for monocular depth estimation. The IEBins use a small number of bins adaptively at each iteration. It use a GRU to predict the depth distribution at each stage. The authors conduct experiments on 3 commonly used datasets and the proposed method shows better results than previous methods. ", "questions": "L44. \"This learning ambiguity grows rapidly as the number of bins increases, exacerbating the difficulty of model convergence and prone to overfitting.\" Why the ambiguity grows in this case? And why the proposed iterative method doesn't lead to increased ambiguity?\n\n", "limitations": "1. Currently only the depth maps are shown. It would be great to compare the point cloud with other methods since it's much easier to see the 3D structure in point cloud.\n2. How is the improved depth estimation benefits in downstream task (compared to other MDE methods)? For example, could it be used for SLAM or 3D reconstruction from multiview images. ", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well written and easy to follow. \nThe idea of using iterative update is interesting.\nThe experiments and ablation study are thorough and validate the proposed method's effectiveness.\n\n", "weaknesses": "1. The motivation of iterative updates is not very clear. RAFT used iterative updates because at each iteration, a new cost volume can be constructed based on current prediction. But I don't see where the new information comes from in MDE. \n2. Based on 1. Maybe the improved performance come from the depth candidates projects since the model knows what the depth is. Would be interesting to see how the model performance by using a positional encoding of depth candidates to other non-iterative methods, such as adabins. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### __We thank our reviewer for the constructive feedback and comments.__\n\n### _W1: The motivation of iterative updates is not very clear. I don't see where the new information comes from in MDE._\nA1: The IEBins embodies the idea of iterative division of bins. Each stage first divides the elastic target bin (the bin in which the current depth prediction is located) and then feeds the updated depth candidates into the iterative optimizer as new information for finer-grained depth search.\n\n### _W2: Would be interesting to see how the model performance by using a positional encoding of depth candidates to other non-iterative methods, such as adabins._\nA2: Nice comment. We have verified the model performance by using a positional encoding of depth candidates to AdaBins while there is little change in performance.\n\n### _Q1: Why the ambiguity grows in this case? And why the proposed method doesn't lead to increased ambiguity?_\nA3: As presented in [1], the depth candidate corresponding to the peak point of the probabilistic distribution (or complementary cost distribution) may not be the true optimal depth candidate. However, the desired depth prediction can still be obtained after a linear combination of the probabilistic distribution and the depth candidates. In other words, there are many linear combinations for a set of depth candidates that can yield the desired depth prediction. When the number of bins increases, the combination between the probabilistic distribution and the depth candidates becomes more and more complex. Hence, we point out that ``this learning ambiguity grows rapidly as the number of bins increases”. The proposed IEBins uses multiple small number of bins, instead of one large standard number of bins. To avoid potentially misleading, we will rephrase or remove this claim in the revision. \n\n[1]. Adaptive Unimodal Cost Volume Filtering for Deep Stereo Matching, AAAI 2020.\n\n### _L1: It would be great to compare the point cloud with other methods._\n\nA4: We have shown point cloud comparison in Figs. 3 and 4 of supplementary material.\n\n### _L2：How is the improved depth estimation benefits in downstream task (compared to other MDE methods)?_ \n\nA5: We will add necessary discussion on our future works according to this nice advice in the revised version.\n\n\n\n"}, {"review_id": "tDhJ9NLIC6", "reviewer": "Reviewer_4WPj", "summary": "This paper proposes iterative elastic bins (IEBins), a multi-stage coarse-to-fine method for monocular depth estimation (MDE). It progressively searches the target depth bin on top of the previous step. To reduce the error accumulation in the iterations, an elastic bin is proposed whose width is adjusted based on the depth prediction uncertainty. Experiments on the KITTI, NYU-v2, and SUN RGB-D datasets show improved performances on MDE.", "questions": "- Have other depth uncertainty presentations been investigated or compared with the variance of the probabilistic distribution used in Eq 6?", "limitations": "- N/A for the limitations.\n- I suggest the authors provide the failure cases (and the corresponding explanations) of the proposed IEBins, w.r.t. for example, the number of bins and the elastic width of bins.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The depth bin is adjusted elastically based on the depth prediction uncertainty.", "weaknesses": "- The contributions are not enough to my knowledge. I think only the elastic bins are new. The iterative manner for depth estimation by GRUs has been proposed in RAFT[28] and adopted in many follow-up works, e.g., RAFT-Stereo[29] and Itermvs[30]. The depth bins as shown in Eq 1,2,3&4 have been investigated in Adabins [3]. As for the framework of the feature extractor and GRU-based layers, standard implementations are involved. The multi-stage depth estimation pattern was also seen in previous works, e.g., CasMVS [Gu et al, cvpr2020] and IterMVS[30].\n- Some minor ones:\n  - line 141: a RGB --> an RGB\n  - line 151: during iteration --> during iterations", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### __We thank our reviewer for the constructive feedback and comments.__\n\n### _W1: The contributions are not enough to my knowledge._\n\nA1: Apologies for not stating the contributions clearly. In this work, we introduce a novel iterative elastic bins (IEBins) strategy for monocular depth estimation.  Instead of using only one set of bins like AdaBins [3], the IEBins refines binning structure in an iterative manner. The initialization stage makes a coarse uniform discretization of the full depth range and each subsequent stage iteratively locates and uniformly discretizes the target bin by using the target bin as the new depth range. While previous works CasMVS [Gu et al, cvpr2020] and IterMVS [30] also adopt a multi-stage fashion to estimate depth, they acquire a new depth range by centering on the current depth prediction and empirically setting the range size. In addition, the depth interval size of different stages at the same level in IterMVS [30] remains the same, while the bin width of our target bin is gradually reduced for a finer-grained depth search. To cope with the possible error accumulation during iterations, we further make the target bin elastic based on the depth uncertainty. To instantiate the IEBins, we design the network architecture by borrowing ideas from other fields such as optical flow estimation (RAFT [28]). Last but not least, the IEBins ranks 2nd and outperforms all previously published methods on the KITTI benchmark leaderboard at the submission time. \n\n### _W2: Some minor ones: a RGB --> an RGB, during iteration --> during iterations._\n\nA2: We will fix these grammatical errors in the revision.\n\n### _Q1: Have other depth uncertainty presentations been investigated or compared with the variance of the probabilistic distribution used in Eq 6?_\n\nA3: We have experimented with the predictive uncertainty [1] as well as using variance directly in our framework and find that using standard deviation of the probabilistic distribution works better.\n\n[1] . Depth Completion from Sparse LiDAR Data with Depth-Normal Constraints, CVPR 2019.\n\n### _L1: I suggest the authors provide failure cases (and the corresponding explanations) of IEBins._\n\nA4: We will revise our paper according to this nice advice.\n\n\n\n"}, {"review_id": "Fi3BiVUuUe", "reviewer": "Reviewer_iovA", "summary": "The paper tackles the task of monocular depth estimation which is of fundamental importance in computer vision and has many downstream applications. Several recent works use bin-based approaches and follow the adaptive binning framework where the distribution of bin centers on the depth interval (that are treated as depth candidates) can vary per image or per-pixel. \n\nThis paper proposes a novel approach to the adaptive binning framework called IEBins. Instead of working with only one (potentially large) set of bins (per image or per pixel), IEBins refines the binning structure in an iterative manner. Starting with a coarse uniform division of the original depth interval (dmin, dmax), the idea is to recursively find and uniformly divide the ‘target’ bin by using the ’target’ bin as the new target depth interval for the next step. In addition, the target bin is made elastic (the new target depth interval’s ends are allowed to change) based on the uncertainty estimate. \n\nThe work achieves state-of-the-art results on popular benchmarks including KITTI and NYU-Depth-v2 and the authors promise to release the code and models publicly. ", "questions": "* How do the bin-widths actually evolve during the iterative refinement? What is the mean of elasticity factor (new adjusted width divided by the bin-width at that stage with no elasticity)?\n* What is the maximum bin-width in meters reached at the final stage e.g. for the NYU test set? This can also answer the elasticity to some extent.\n* Why is a large number of bins undesirable? What is the evidence?\n* How does IEBins compare with LocalBins?", "limitations": "The authors do not explicitly reflect upon the limitations or scope of the work. Refer to \"weaknesses\" for major concerns.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 4, "strengths": "* State-of-the-art results. The work achieves SOTA in a highly competitive space of monocular depth estimation with over 7% improvement in RMSE over the prior SOTA. IEBins idea is also validated and shows about 2% improvement to adaptive bins baseline in fair settings.\n* Interesting idea. The idea of recursive division of bins is obvious in retrospect yet creative. Elastic nature of bins based on uncertainty also makes sense and subtly introduces the ‘adaptive’ nature of bins. \n* Well exploited ideas and good architecture design. Authors have well exploited the ideas from other works to obtain their goal. e.g. Iterative refinement using GRU, developed in separate field of optical flow estimation, has been introduced here in a well designed architecture. \n* Potential to be foundational. Recently, depth estimation has seen foundational ideas like adaptive binning that serve as a starting point for a series of several follow up works. I believe the iterative refinement of bins can prove to be as influential. \n* Good to see ablations on all bin types in fair settings.\n", "weaknesses": "* W1 **Incomplete literature review**. Authors have missed some important published works that are highly relevant to this work:  \n    * [a] - LocalBins (ECCV’22) also introduces the idea of “splitting” the bins and step-wise refinement of bins in a coarse-to-fine manner. IEBins and LocalBins are highly related and pursue the same goal but follow different approaches. Quantitative comparison with [a] and providing the insights to differences and similarities is highly suggested. \n    * [b] - PixelFormer (WACV’23) is also based on the adaptive binning framework and introduces layer-wise refinement of “pixel queries”. \n* W2 **Absence of quantitative evidence towards the working of the fundamental idea**. According to the proposed idea of iteratively making the ‘target’ depth interval smaller, the bin-width (or its median across an image) should exponentially decrease stage-by-stage, with some flexibility introduced by the elastic nature. At the same time, if the bins are too elastic i.e. final bin-widths are comparable to initial bin-widths (e.g. only 50% smaller than previous or so), then the iterative restrictive nature of depth search is invalidated. Although authors visualise the refinement process in Fig. 3 but these results can still be explained by ‘too elastic’ bins as the elastic-bin-adjustment varies spatially. Also, are the uncertainty 'heat maps' scaled individually or globally? If individually (which seems to be the case), then the uncertainty visualizations only show that uncertainty tends to take much higher values near edges, rather than an overall absolute decrease and therefore tell nothing about the bin widths. The authors are suggested to provide evidence to show that method actually works as described to avoid misleading conclusions. For example, via a simple line plot of evolution of target bin-width (multiple lines for pixels across a row or single line plot of median elastic bin width across image etc) through the course of refinement, or any other form that authors deem suitable that delivers the evidence clearly. \n* W3 **Unsubstantiated claim at L44**. Authors claim that large number of bins are undesirable and use this as the main motivation for the work. Authors state at L43-44 that “Learning ambiguity grows rapidly as the number of bins increases” but fail to provide any explanation, reference or evidence. As this information can be potentially misleading, authors should either remove the claims or provide an explanation, reference or evidence towards the claim.\n\n[a] Bhat, Shariq Farooq, Ibraheem Alhashim, and Peter Wonka. \"Localbins: Improving depth estimation by learning local distributions.\" European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022.\n\n[b] Agarwal, Ashutosh, and Chetan Arora. \"Attention attention everywhere: Monocular depth prediction with skip attention.\" Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### __We thank our reviewer for the constructive feedback and comments.__\n\n### _W1&Q4: Incomplete literature review. Comparison with LocalBins._\n\nA1: We will add these two interesting and relevant works to our revised version. \n\nSimilarities with LocalBins: Both IEBins and LocalBins use a multi-stage fashion to refine the binning structure.\n\nDifferences between them: At each stage, LocalBins divides all bins from the previous stage, and when the stage increases, the number of bins increases, while IEBins locates and divides the target bin only, and the number of bins is not changed at different stages. LocalBins refines the binning structure on multiple resolutions, while IEBins on the same resolution.\n\nQuantitative comparison with LocalBins: We follow the experimental settings in Table 5 and the results of Baseline + LocalBins on Abs Rel, RMSE, log10, $\\delta  < 1.25$,  $\\delta  < {1.25^2}$, $\\delta  < {1.25^3}$ are 0.090, 0.319, 0.038, 0.932, 0.992, 0.998 on the NYU-Depth-v2 dataset, which are worse than those of Baseline + IEBins (0.087, 0.314, 0.038, 0.936, 0.992, and 0.998).\n\n### _W2&Q1&Q2: Quantitative evidence towards the working of the fundamental idea, elasticity factor and maximum bin-width in meters reached at the final stage._\n\nA2: Nice comments. We randomly choose a sample from the NYU-Depth-v2 test set, and show the median elastic target bin width across the image for each stage, and corresponding uncertainty values and elasticity factors. We note that 9.9 is the original range size ($d_{max}$ 10 - $d_{min}$ 0.1) .\n\n|&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp; Stage1    &nbsp;  Stage2  &nbsp;   Stage3   &nbsp;  Stage4  &nbsp;   Stage5   &nbsp;   Stage6||\n|:---|:---|\n|Width (median)   &ensp;&ensp;&ensp; 9.9 &ensp;&ensp;&ensp; 2.561 &ensp;&ensp;&ensp; 0.756 &ensp;&ensp; 0.228 &ensp;&ensp; 0.073 &ensp;&ensp; 0.024| \n|Uncertainty (std)  &ensp;&ensp;&ensp; - &ensp;&ensp;&ensp;&ensp;&nbsp;  0.971 &ensp;&ensp;&nbsp; 0.281 &ensp;&ensp; 0.087 &ensp;&ensp; 0.029 &ensp;&ensp; 0.010| \n|Elasticity factor &ensp;&ensp;&ensp;&nbsp; &nbsp; - &ensp;&ensp;&ensp;&ensp; 4.139 &ensp;&ensp;&nbsp;&nbsp; 3.917 &ensp;&ensp; 4.222 &ensp;&ensp; 4.562 &ensp;&ensp;  4.800|\n\nIt can be seen that as the stage increases, the elastic target bin widths and uncertainty values continue to decrease. The elasticity factors are between 3.9 and 4.8.\n\nThe maximum bin widths of elastic target bin and newly divided bins for final stage are 0.046m and 0.0029m (0.046/16).\n\n### _W3&Q3: Unsubstantiated claim at L44. Why is a large number of bins undesirable? What is the evidence?_ \n\nA3: As presented in [1], the depth candidate corresponding to the peak point of the probabilistic distribution (or complementary cost distribution) may not be the true optimal depth candidate. However, the desired depth prediction can still be obtained after a linear combination of the probabilistic distribution and the depth candidates. In other words, there are many linear combinations for a set of depth candidates that can yield the desired depth prediction. When the number of bins increases, the combination between the probabilistic distribution and the depth candidates becomes more and more complex. Hence, we point out that ``this learning ambiguity grows rapidly as the number of bins increases''. Intuitively, it is much easier to classify the optimal candidate from a small set of depth candidates than from a large set of depth candidates. Hence, we choose small number of bins. To avoid potentially misleading, we will rephrase or remove this claim in the revision. \n\n[1]. Adaptive Unimodal Cost Volume Filtering for Deep Stereo Matching, AAAI2020.\n"}, {"review_id": "3IlPJRoxBi", "reviewer": "Reviewer_LZ7t", "summary": "This paper introduces a classification-regression-based monocular depth estimation pipeline.\nPrevious classification or classification-regression-based MDE approaches often suffer from huge complexity and loss of generalizability issue, due to the nature of requiring more depth hypothesis for better performance.\nIn this paper, it propose iterative elastic bins, ie, IEBins technique, which iteratively adjusts bin ranges as the prediction converges, so that with less number of bins and fast inference time, the method can still perform on the state-of-the-art level.\nIt additionally introduces dedicated transformer-based feature extractor and GRU-based iterative optimizer.", "questions": "1. I have some questions regarding updating rules of the proposed IEBins.\n\n1-1. In Eq. 7, it denotes that elastic target bin edges are modified using uncertainty from previous stage's probability distribution. Does it mean that for each pixel, bin edges are set differently? If this is the case, the network should remember separate bin ranges for every pixel, which seems like to require huge memory. Correct me if I'm wrong.\n\n1-2. In L133, the paper says that d_min and d_max values are updated with new bin edges. In the next iteration, are 16 bins additionally set within this new min-max range? Then, does it mean that in 2nd iteration, for example, new bin range are approximately 1/256 from original min-max range?\n\n2. In Fig 5, the range of the proposed method seems significantly off compared to GT. How is the visualization done?\n\n3. In Tab 6, why does the performance drop when using 32 bins instead of 16? Is this aligned with overfitting issue stated in the introduction section? If it's true, than doesn't it mean that the proposed method already suffers from 32 bins while AdaBins, for example, can operate until 256 bins? How can this phenomenon be analyzed?", "limitations": "The authors did not address any limitations. Yet, other than weakness and questions I commented above, I don't see any severe limitation to this paper.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "This paper is well-written and easy to follow.\nThe proposed IEBins is an intuitive and interesting idea, and the performance also backs up its effectiveness.", "weaknesses": "More in-depth analysis on 'iterative' perspective of IEBins will be appreciated.\n\n1) For example, what happens when iteration is less or more than 6? How much does the number of iterations affect the final performance?\n\n2) The authors claim that the proposed GRU-based iterative optimizer is also one of their contributions, saying that it is helpful to capture temporal information during IEBins-based depth estimation. Yet, there is no ablation study regarding this. Comparison between adding/excluding GRU unit, alongside its effectiveness regarding the number of iteration will help strengthening the claimed contribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### __We thank our reviewer for the constructive feedback and comments.__\n\n### _W1: What happens when iteration is less or more than 6? How much does the number of iterations affect the final performance?_\n\nA1: The results of IEBins at different stages are shown below:\n|Stage  &nbsp;  Abs Rel  &nbsp;   RMSE  &nbsp;  log10  &nbsp;   $\\delta  < 1.25$  &nbsp;   $\\delta  < {1.25^2}$ &nbsp; $\\delta <1.25^3$||\n|:---|:---|\n|Stage1 &nbsp; 0.093 &emsp; 0.333 &ensp; 0.041 &emsp; 0.921 &emsp;&ensp;&ensp; 0.991 &emsp;&ensp;&ensp; 0.998|\n|Stage2 &nbsp; 0.090 &emsp; 0.325 &ensp; 0.040 &emsp; 0.927 &emsp;&ensp;&ensp; 0.991 &emsp;&ensp;&ensp; 0.998|\n|Stage3 &nbsp; 0.089 &emsp; 0.320 &ensp; 0.039 &emsp; 0.931 &emsp;&ensp;&ensp; 0.991 &emsp;&ensp;&ensp; 0.998|\n|Stage4 &nbsp; 0.088 &emsp; 0.317 &ensp; 0.038 &emsp; 0.933 &emsp;&ensp;&ensp; 0.992 &emsp;&ensp;&ensp; 0.998|\n|Stage5 &nbsp; 0.087 &emsp; 0.315 &ensp; 0.038 &emsp; 0.935 &emsp;&ensp;&ensp; 0.992 &emsp;&ensp;&ensp; 0.998|\n|Stage6 &nbsp; 0.087 &emsp; 0.314 &ensp; 0.038 &emsp; 0.936 &emsp;&ensp;&ensp; 0.992 &emsp;&ensp;&ensp; 0.998|\n|Stage7 &nbsp; 0.087 &emsp; 0.313 &ensp; 0.038 &emsp; 0.935 &emsp;&ensp;&ensp; 0.992 &emsp;&ensp;&ensp; 0.998|\n\nAs the stage increases, the performance gradually improves until saturated, and when the number of iterations exceeds 6, the performance changes very little.\n\n### _W2: Ablation study on the GRU-based iterative optimizer._\n\nA2:  Nice comment. We have presented the results at each stage after excluding GRU unit from the iterative optimizer below. For results of the whole framework, please refer to A1 to W1.\n\n|Stage  &nbsp;  Abs Rel  &nbsp;   RMSE  &nbsp;  log10  &nbsp;   $\\delta  < 1.25$  &nbsp;   $\\delta  < {1.25^2}$ &nbsp; $\\delta <1.25^3$||\n|:---|:---|\n|Stage1 &nbsp; 0.093 &emsp; 0.334 &ensp; 0.041 &emsp; 0.920 &emsp;&ensp;&ensp; 0.991 &emsp;&ensp;&ensp; 0.998|\n|Stage2 &nbsp; 0.091 &emsp; 0.327 &ensp; 0.040 &emsp; 0.925 &emsp;&ensp;&ensp; 0.991 &emsp;&ensp;&ensp; 0.998|\n|Stage3 &nbsp; 0.090 &emsp; 0.323 &ensp; 0.039 &emsp; 0.928 &emsp;&ensp;&ensp; 0.991 &emsp;&ensp;&ensp; 0.998|\n|Stage4 &nbsp; 0.089 &emsp; 0.320 &ensp; 0.039 &emsp; 0.930 &emsp;&ensp;&ensp; 0.991 &emsp;&ensp;&ensp; 0.998|\n|Stage5 &nbsp; 0.088 &emsp; 0.318 &ensp; 0.039 &emsp; 0.931 &emsp;&ensp;&ensp; 0.992 &emsp;&ensp;&ensp; 0.998|\n|Stage6 &nbsp; 0.088 &emsp; 0.317 &ensp; 0.039 &emsp; 0.932 &emsp;&ensp;&ensp; 0.992 &emsp;&ensp;&ensp; 0.998|\n\nThe results verify the efficacy of the GRU-based iterative optimizer.\n\n### _Q1.1: Confusion over bin edges._\n\nA3: The bin edges vary between pixels besides the initialization stage. The iterative optimizer operates at 1/4 resolution in our design, making the memory consumption affordable.\n\n### _Q1.2: Are 16 bins additionally set within this new min-max range? And in 2nd iteration, new bin range are approximately 1/256 from original min-max range?_\n\nA4: Yes, 16 bins are set within the target bin by using the target bin as a new min-max range at each stage. For the case of target bin, the new bin range are 1/256 from original min-max range in 2nd stage. While for the case of elastic target bin, this ratio varies from pixel to pixel and image to image due to the adaptive nature introduced by elastic target bin.\n\n### _Q2: How is the visualization done?_\n\nA5: We scale each depth map individually here to acquire the colormap. The GT depth maps are not completely dense and have missing values in border regions, which may affect the final colormap.\n\n### _Q3: In Tab 6, why does the performance drop when using 32 bins instead of 16? Is this aligned with overfitting issue stated in the introduction section?_\n\nA6: This phenomenon may not be induced by the overfitting issue. As the number of bins increases, it becomes more and more difficult to classify the true optimal candidate for the next stage from a larger number of depth candidates. When the target bin classified deviates so far from the true/GT depth that the true depth also cannot be included in the elastic target bin, large depth errors will be generated in subsequent stages, thereby affecting the final performance.\n"}, {"review_id": "4kHlP2kpnh", "reviewer": "Reviewer_jxnL", "summary": "The paper introduces a method for monocular depth. The method uses a recurrent network based on RAFT to predict a probability distribution over a set of bins, which enables the depth at each iteration to be computed as the expectation over all bins and the margins of each bin to be adjusted using the computed variance. In the subsequent iterations, a finer-grained search is performed in the adjusted bins. This new strategy is introduced as \"IEBins\". The authors motivate this strategy as being robust to uncertain initial predictions. ", "questions": "None", "limitations": "I did not see discussion of limitations", "rating": 8, "confidence": 2, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The method outperforms prior work across multiple standard evaluation settings.\n\nThe method is interpretable, producing confidence for different regions and steadily improving depth predictions.\n\nTo the best of my knowledge, the iterative design is substantially different than the usual approach for monocular depth.\n\nThe introduced method is very fast.\n\nThe IEBins are compared against other binning strategies in ablation experiments.", "weaknesses": "No obvious weaknesses.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### __We thank our reviewer very much for the highly positive feedback.__"}, {"review_id": "KXm8hNTHUY", "reviewer": "Reviewer_TtXH", "summary": "The paper introduces an Iterative Elastic Bins (IEBins) approach for monocular depth estimation. Many conventional monocular depth estimation approach uses a soft-argmax representation (Eq. (4)) that sums the product between depth probability and pre-defined depth bins. However, the large number of pre-defined bins hinders model convergence (@L.43). To alleviate the problem, the paper proposes to use adaptive ranges of depth bins for each pixel based on previous intermediate depth estimates. Along with the number of iteration steps, the depth ranges get smaller for fine-detail estimation (yet, taking uncertainty into account). The paper shows empirically good accuracy over previous methods.", "questions": "- In Fig. 2 the three depth maps on the right look the same. Probably it may need to be changed to the actual experiment results. \n- In the last example (chair) in Fig. 5, the object boundary near the chair is still blurry. (Just out of curiosity) I wonder if the proposed representation can resolve this blurry object boundary problem because it limits the depth range, so I would expect it outputs much clearer depth around object boundaries. \n- In Table 7, what is the source of the faster inference time than BinsFormer?", "limitations": "The paper didn't include its limitations or societal impact.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "+ Good results\n\n  Table 1, 2, 3, and 4 show that the method achieves better accuracy than published methods in the public benchmark datasets (KITTI Eigen, KITTI, NYU-Depth-v2, and SUN RGB-D). \n\n+ Good ablation study\n\n   Table 5 and 6 demonstrates how each design choice affects/improves the accuracy. Table 5 shows the strength of the proposed IEBin idea over previous approaches (UBins, SIBins, AdaBins, and IBins). Table 6 also justifies the design choice of the number of bins.", "weaknesses": "- Question on the number of stages (Fig. 3)\n\n  The method uses 6 stages (iteration steps) in total. I wonder what happens if it uses more iteration steps during the inference (i.e., 6 iterations during training, but 8 or 10 iterations during test time). Will the method keep improving the accuracy with more iteration steps? Or, even during training, what if it uses more iteration steps other than 6? How the number 6 is set?\n\n- Possibly an unfair comparison? Table 5, Comparison with different bin types.\n\n  I wonder if other methods (UBins, SIBins, AdaBins, and IBins) also use the iterative optimizer. For a fair comparison, I think it's also good to prepare a baseline with the iterative optimizer and only change the depth bin types. Then it can differentiate the performance gain of bin types from the iterative optimizer. \n\n\nDespite the good results, I would like the paper Borderline Reject for now, mainly due to the possible unfair comparison in Table 5. The source of gain is not so clear if it's from the iterative optimizer or the depth bin types. \n\n----\n\nI share the same concern with Reviewer 4WPj that the main contributions don't seem to be so strong. The iterative refinement ideas have been proposed in other literature (e.g., optical flow, SfM), and the paper demonstrates empirical accuracy gain. That said, I am raising the score to Borderline Accept as the authors' response resolves my main concerns. I hope all the comments and concerns during the discussion phase are reflected in the updated version.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### __We thank our reviewer for the constructive feedback and comments.__\n### _W1: Question on the number of stages (Fig. 3)_\n\nA1: During both training and inference phases, we find that when the number of stages exceeds 6, the performance changes very little. As we know, more stages require longer training and inference time, greater memory consumption. Hence, we set the number of stages to 6.\n\n### _W2: Possibly an unfair comparison? Table 5, Comparison with different bin types._\n\nA2: We apologize for not stating the settings clearly. Because UBins, SIBins and AdaBins are non-iterative methods, we do not add iterative optimizer for them. On the other hand, Baseline + AdaBins (276M) has more parameters than Baseline + IEBins (273M) because AdaBins requires an additional Transformer architecture to generate adaptive bins. IBins is a variant of IEBins and is acquired by replacing the elastic target bin with original target bin at each stage. We have used iterative optimizer for IBins when reporting its results in Table 5. To further elaborate results, we take well-based AdaBins as an example and feed its adaptive depth candidates into iterative optimizer. The corresponding results of Abs Rel, RMSE, log10, $\\delta <1.25$, $\\delta <1.25^2$, $\\delta <1.25^3$ are 0.089, 0.321, 0.038, 0.932, 0.991, and 0.998 on the NYU-Depth-v2 dataset, which are worse than those of Baseline + IEBins (0.087, 0.314, 0.038, 0.936, 0.992, and 0.998).\n\n### _Q1: Probably it may need to be changed to the actual experiment results for three depth maps in Fig.2._\n\nA3: We will modify our paper according to this nice advice.\n\n### _Q2: In the last example (chair) in Fig. 5, the object boundary near the chair is still blurry. (Just out of curiosity) I wonder if the proposed representation can resolve this blurry object boundary problem because it limits the depth range, so I would expect it outputs much clearer depth around._\n\nA4: Yes, this is possible because the proposed representation of iterative division of bins is sensitive to the object boundaries due to large depth variations in these regions. \n\n### _Q3: In Table 7, what is the source of the faster inference time than BinsFormer?_\n\nA5: BinsFormer contains two decoders, a pixel decoder and a transformer decoder, and frequent interactions occur between these two decoders, which may increase the inference time significantly. Although our method is iterative, the developed iterative optimizer is lightweight and operates at 1/4 resolution.\n\n\n\n"}]

Author rebuttal:

Meta-review: This paper received 4x positive ratings (2x borderline accepts, 1x accept and 1x strong accept) and 2x negative ratings (2x borderline rejects). The reviewers who gave the positive comments mention that: The idea of using iterative update is interesting, the idea of recursive division of bins is obvious in retrospect yet creative. The method outperforms prior work across multiple standard evaluation settings; The method is interpretable, producing confidence for different regions and steadily improving depth predictions; The introduced method is very fast; Good results and good ablation study on all bin types in fair settings; Well exploited ideas and good architecture design. The negative comments are: Elastic bins are new. The iterative manner for depth estimation by GRUs has been proposed in RAFT[28] and adopted in many follow-up works. Not totally convinced by additional contributions (such as GRU-based optimizer) and analysis. One of the positive reviewer also mentioned that the idea is not totally new, but it's from other areas, e.g. optical flow, sfm, etc. Considering that the positive comments outweigh the negative comments in the reviews, the AC decide to follow the majority vote to accept the paper.
Decision: Accept (poster)