Hierarchical  Normalization for Robust Monocular Depth Estimation
Authors: Chi Zhang
Wei Yin
Billzb Wang
Gang YU
BIN FU
Chunhua Shen
Venue: NeurIPS 2022 Accept (2022)
Keywords: depth prediction
depth estimation

Abstract: In this paper, we address monocular depth estimation with deep neural networks. To enable training of deep monocular estimation models with various sources of datasets, state-of-the-art methods adopt image-level normalization strategies to generate affine-invariant depth representations. However, learning with the image-level normalization mainly emphasizes the relations of pixel representations with the global statistic in the images, such as the structure of the scene, while the fine-grained depth difference may be overlooked. In this paper, we propose a novel multi-scale depth normalization method that hierarchically normalizes the depth representations based on 
spatial information and depth distributions. Compared with previous normalization strategies applied only at the holistic image level, the proposed hierarchical normalization can effectively preserve the fine-grained details and improve accuracy. We present two strategies that define the hierarchical normalization contexts in the depth domain and the spatial domain, respectively. Our extensive experiments show that the proposed normalization strategy remarkably outperforms previous normalization methods, and we set new state-of-the-art on five zero-shot transfer benchmark datasets. 

Hierarchical Normalization for Robust Monocular Depth Estimation
Chi Zhang',
Wei Yin', Tecent PCG, China
Zhibin Wang',
Gang Yu Bin Fu',
Chunhua Shen
DJI Technology, China
*Zhejang University, China
1 (Johnczhang, brianfu, skicyyu)@tencent.com com; 2 yvanwy@outlook.com com; Chunhua@iclod..comm com
Abstract
In this paper, we address monocular depth estimation with deep neural networks. To enable training of deep monocular estimation models with various sources of datasets, state-of-the-arr methods adopt image-level normalization strategies generate affine-invariant depth representations. However, learning with the image-level normalization mainly emphasizes the relations of pixel representations with the global statistic in images, such as the structure of scene, while the fine-grained depth difference may be overlooked. In this paper, propose a novel multi-scale depth normalization method that hierarchically normalizes the depth representations based spatial information and depth distributions. Compared with previous normalization strategies applied only at the holistic image level, the proposed hierarchical normalization grained details and improve accuracy. We present two strategies that define the hierarchical normalization contexts in the depth domain and the spatial domain, respectively. Our extensive experiments show that the proposed normalization strategy remarkably outperforms previous normalization methods, and set new state-of-the-art on five zero-shot transfer benchmark datasets.
effectively preserve the fine-
551
SSI
551
Ours
Ours
Qur
Figure propose a hierarchical depth normalization stralegy to improve the training monocular depth estimation models. Compared with the previous normalization strategy SSI 24], our design effectively improves the details and smoothness of predictions. We visualize the predictions of close regions with heat maps to observe details.
Introduction
Data-driven deep learning based monocular depth estimation has gained wide interest in recent years, due to its low requirements of sensing devices and impressive progress, Among various learning
"Correpooding author
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

objectives of deep monocular estimation, zero-shot transfer carries the promise of learning generic depth predictor that can generalize well across a variety of scenes. Rather than training and evaluation on the subsets of individual benchmarks that usually share similar characteristics and biases, zero-shot transfer expects the models to be deployed for predictions of any in-the- wiil images.
To achieve this goal, large-scale datasets with equally high diversity for training are necessary to enable good generalization. However, collecting data with high-quality depth annotations is expensive, and existing benchmark datasets often show limitations in scales or diversities. Many recent works 24 37 seek mataset training, where datasets captured by various sensing modalities and in diverse environments can be jointly utilized for model training, which largely alleviates the difficulty of obtaining diverse annotated depth data at scale. Nevertheless, the mix-data training also comes with its challenges, as different datasets may demonstrate inconsistency in depth representations, which causes incompatibility between datasets. For example, the disparity map generated from web stereo images [2) or 3D movies [24] can only provide depth annotations up to scale and shift, due to varied and unknown camera models.
To solve this problem, state-of-the-art methods 24 37 seek training objectives invariant to the scale- and-shift changes in the depth representations by normalizing the predictions or depth annotations based on statistics of the image instance, which largely facilitates the mix-data learning of depth predictors. However, as the depth represented by the magnitude of values, normalization based on the instance inevitably squeezes the fine- eeneggrinedd depth difference, particularly in close regions. Suppose that an object-centric dataset with depth annotations is available for training, where the images sometimes include backgrounds with high depth values. Normalizing the depth representations with global statistics can distinctly separate the foreground and background, but it may meanwhile lead to an overlook of the depth difference in objects, which may be our real interest. As the result, the learned depth predictor often excels at predicting the relative depth representations of each pixel location with respect to the entire scene in the image, such as the overall scene structure, but struggles to capture the fine-grained details.
Motivated by the bias issue in existing depth normalization approaches, we aim to design a training objective that should have the flexibility to optimize both the overall scene structure and fine-grained depth difference. Since depth estimation is a dense prediction task, we take inspiration from classic local normalization approaches, such as local response normalization in AlexNet [16), local histogram normalization in SIFT 20 and HOG features [6. and local deep features in DeepEMD 40 which rely on normalized local statistics to enhance local contrast or generate discriminative local descriptors.
By varying the size of the local window, we can control how much context is involved to generate a normalized representation. With such insights, we present a hierarchical depth normalization (HDN) strategy that normalizes depth representations with different scopes of contexts for learning. Intuitively, a large context for normalization emphasizes the depth difference globally, while a small context focuses the subtle difference locally. We present two implementations that define the multi-scale contexts in the spatial domain and the depth domain, respectively. For the strategy in the spatial domain, we divide the image into several sub-regions and the context is defined as the pixels in individual cells. In this way, the fine-grained difference between spatially close locations is emphasized. By varying the grid size, we obtain multiple depth representations of each pixel that rely on different contexts. For the strategy in the depth domain, we group pixels based on the ground truth depth values to construct contexts, such that pixels with similar depth values can be differentiated. Similarly, we can change the number of groups to control the context size. By combining the normalized depth representations under various contexts for optimization, the learner emphasizes both the fine-grained details and the global scene structure, as shown in Fig. |
To validate the effectiivnesss of our design, conduct extensive experiments various benchmark datasets. Our empirical results show that the proposed hicrarchical depth normalization remark- ably outperforms the existing instance-level normalization qualitatively and quantitatively. Our contributions are summarized as follows:
We propose hierarchical depth normalization strategy to improve the learning of deep monocular estimation models.
We present two implementations to define the multi-scale normalization contexts of HDN based on spatial information and depth distributions, respectively.
Experiments on five popular benchmark datasets show that our method significantly outper- forms the basclines and sets new state-of-the-at results.

---

Next we review some works that are closest to ours and then present our method in Section 3. In Section we empirically validate the effectiveness of our method on several public benchmark datasets.
2 Related Work
Deep monocular depth estimation. As opposed early works monocular depth estimation based on hand-crafted features, recent studies advocate end-to-end learning based on deep neural networks. Since the pioneer work [7] first adopts deep neural networks to undertake monocular depth estimation, significant progress has been made from many aspects, such as network architectures [17] 22
[8]
large-scale and diverse training datasets [4 38 loss functions [24 37), multi-task learning 39]
synthesized dataset geometry constraint 37..
35 36
and various sources of supervisions 33
For supervised training, collecting high-diversin data with ground truth depth annotations at scale is expensive. Recent works based ranking loss 2 33 and scale-and-shift invariant losses 24
37
enable network training with other forms of annotations, such as ordinal depth annotations 2 32
generated by uncalibrated stereo images using optical flow algorithms (28). In particular, scale-and-shine invariant (SSI) loss (24) and image-level normalization loss (37 allow data from multiple sources be learned in fully supervised depth regression manner, which largely facilitates large-scale training and improves the generation ability of learning based depth estimators. The SSI loss removes the major incompatibility between various datasets, i.e., the scale and shift changes, by transforming the depth representation into canonical space through normalization. With such advances, zero-shot transfer is made possible, where the network learned on a large-scale database with high diversity can be directly evaluated on various benchmarks without seeing their training samples, which is the focus of this paper.
relative inverse depth map B3 34 24
Furthermore, some literature without sensor-captured ground truth but leverages the training signal from consecutive temporal frames or stereo videos. However, most of these methods need the camera intrinsic parameters for supervision.
26 proposes to solve the monocular depth estimation problem
Normalization in CNNs. Normalization is widely adopted deep neural networks, while different normalization strategies employed for different purposes. For instance, batch normalization (BN) [13] normalizes the feature representations along the batch dimension to stabilize training and accelerate convergence. BN usually prefers large normalization contexts to obtain robust feature representation. On the other hand, another line of normalization methods relies on local statistics. For example, instance normalization 29) and its variants [12 21] based on instance-level statistics dominate the style transfer task, as they emphasize the unique styles of individual images. A collection of literature seeks normalization in local regions. For example. the well-known SIFT feature [20] and HOG features (6) are based on the normalized local statistics to generate diseriminative local features. DeepEMD 40 41 computes the optimal transport between local normalized deep features as a distance metric between images. Since the local details and the overall scene structure are both important for depth estimator, we incorporate the ideas of both global normalization and local normalization in the monocular depth estimation models.
3 Method
In this section, we first briefly summarize the preliminaries of the task. Then we define unified form of depth normalization and show that the normalization strategy scale-and-shift invariant loss 24 is special case. Finally, present two implementations of our proposed hierarchical depth normalization approaches based on the spatial domain and the depth domain, respectively.
3.1 Preliminaries
We aim to boost the performance of zero-shot monocular depth estimation with diverse training data. In our pipeline, we input single RGB image / E H x3 to the depth prediction network F(.) to generate a depth map D E R xxxx33 Instead of directly regressing the output map with the ground- truth depth supervision, state-of-the-art methods [34 24 37 normalize the depth representations

---

HON Sportall
HDN Death
representzaiion
Globally nor representations
S-1
Sm3
Set
(c) Herrachhccay ized representations
Figure 2: Left: An illustration of different normalization strategies. Given the raw representations (a), previous methods (b) normalize the representations globally. Our proposed hierarchical depth normalization (c) strategy divides representations into S groups, denoted by different colors, and apply normalization individual groups generate representations. We select different obtain multiple representations for learning. By doing so, we can emphasize both the global data distribution and the fine- fine-grained difference between representations during training. Right: Our two normalization strategies for the depth estimation task that group pixels in the spatial domain (HDN-Spatial) and the depth domain (HDN-Depth), respectively. The HDN-Spatin groups pixels in sub regions, while the HDN-Depth groups pixels based depth distributions. The red cells denote the selected pixels for normalization.
before computing the regression loss. In our work, we focus on the depth normalization part, which means our methods can be easily combined with state-of taae-offthhe-rt network structures or loss functions.
3.2 Unified Form of Depth Normalization
Let d E RM and d' E R denote the vectorized predicted depth map and the ground truth depth map, respectively, where M the number pixels with valid annotations. Our goal is to generate the normalized representations, Nu. (d;) and N., (d;) for computing the regression loss of each location i, where u, the set of location indexes that constitute the context of location i, and Nu, is the normalization function based the context u, For example, the normalization employed in the scale-and-shift invariant loss (SSI) 24 is to explicitly remove the estimated scale and shift from the raw depth representations:
d; median,, (d)
dj median (d")
N... (d+)
Nu, (d))
(1)
[
median,, (d)
M4 d
median., (d+))
where the median,, operator computes the median depth of locations that belong to u4- The instance- level normalization in SSI assigns global context Mglo for each pixel that involves all pixel locations with valid depth annotations, i.e., Uj Mglla Vj Vyllo- Finally, the SSI loss L computes the mean absolute error between the normalized prediction and the ground truth, and losses over all locations averaged to obtain the final loss SSI,
M Σ
SSI
css where C. = NN., (d.) - N.., (dt)
M
Intuitively, large normalization context that covers large scope of locations emphasizes the relative depth representations with the global statistics, while small context emphasizes the depth difference more locally. Based on this observation, we propose assign multiple contexts at different scales for each location.
3.3 Hierarchical Depth Normalization
We next present two strategies that define the multi-scale contexts the spatial domain and the depth domain, respectively, which are illustrated in Fig.

---

In the spatial domain (HDN-S). Since images have 2D grid-like representations, we can define the context based on spatial positions. In this way, the context corresponds to a group of pixels in the sub-region. Defining context in spatial domain based on grids was also seen in segmentation models, where poolings are used to explicitly enlarge the effective receptive fields of features 43 Here we adopt a similar strategy that evenly divides the image plane into several sub-regions based on a S x S grid. Then the pixels belonging to the same region share the same context for normalization. By varying the grid size, we can obtain multiple contexts for each pixel location. In our design, we select the grid size Sppatial from {2", 21, 2 ...} and apply normalization at each level. For example, at the highest level, the normalization is essentially the instance-level normalization that covers all pixels as the context, while at the lowest level, the normalization is limited to an H W image patch.
5 8
X
In the depth domain (HDN-D). The other way to group close pixels is based on their ground-trutt depth distributions, such that pixels with similar depth values share the same context. Here we present two strategies. The first one, denoted by HDN-DP, is to sort all the pixels hased on their ground-truth depth values and evenly divide them into S bins, where the numbers of pixels in each bin are M The g. second strategy, denoted by HDN-DR, is to evenly divide the depth range presented in image into bins, and classify pixels into different bins. Pixels belonging to the same bin share the same context. Since pixels with similar depth values may not be spatially close, long-range spatial dependency can be established by the shared contexts to promote global coherency. The bin number Stepth is also selected from (2"; 22, 2², .2....
Suppose that the multi-scale contexts of location constitutes set Us. we compute the SSI losses generated by each context from Ug and average them obtain the final loss LID.N
M
C HDN
CDDN where L
HDN
Σ DEtt
-
(3)
M
IU:1
As we can see, by combing the contexts at different scales, the proposed learning objective emphasizes the depth difference at different levels, which can preserve the local fine-grained details while maintaining structural consistency. In contrast, the SSI loss is special case that only focuses on the global structure.
We can also define the hierarchical normalization contexts based on irregular patterns, e.g., segmenta- tion masks. In fact, our design provides useful way explicitly utilize semantic knowledge for improving predictions. For example, if we define normalization context that covers stuff and objects based on panoptic segmentation, emphasizes the spatial relations between different instances and stuff. On the other hand, if the normalization context is defined based on the instance masks, we can emphasize the fine-gras -raiiid depth difference in object appearance, such as the structures of vehicles. As we aim to design generic depth normalization algorithm, we choose to define the hierarchical normalization based on regular patterns without seeking extra knowledge in this paper.
Implementation. Our proposed hierarchical normalization strategies are lightweight and easy to implement. Since the SSI loss functions are usually implemented function of ground truth depth maps, predicted depth maps, and valid- -ixel maps, our approaches can be casily implemented by only modifying the valid-pixe pixel maps to obtain batched computations, where the pixels out of the context are masked out.
4 Experiments
Datasets We follow LeReS 37 construct mix-data training set, which includes 114K images from Taskonomy dataset, 121K images from DIML dataset, 48K images from Holopix50K, and 20K images from HRWSI [33). We withhold 1K images from all datasets for validation during training. The details of each dataset are as follows.
Taskonomy. Taskonomy 38 is high-quality and large-scale dataset, which contains over 4 million images of indoor scenes from about 600 buildings. It includes annotations for over 20 tasks. In our experiments, we sampled around 114k RGB-D pairs for training.
DIML. DIML. [14 contains synchronized RGB-D frames from Kinect v2 or Zed stereo camera. For the outdoor split, they are mainly captured by the calibrated stereo cameras. It contains various outdoor places, e.g., offices, rooms, dormitory, exhibition center, street, road and so on. We used GANet 42 to recompute the disparity and depth maps for training.
5

---

DIODE
ETH3D
KITTI
NYU
ScanNel
Mean
Norm. Method
AbsRell
Improv.
Instance
47.1
19.4
13.2
9.6
10.22
Batch
45.0 (-4%)
25.2 (+30%)
14.9 (+12%)
11.5 (+19%)
12.2 (+19%)
(+15%)
Local-S
33.1 (-30%)
14.5 (-25%)
19.3 (+45%)
11.8 (+22%)
11.5 (+12%)
(+5%)
Local-DR
25.7 (-45%)
12.7 (-35%)
14.7 (+11%)
9.4 (-2%)
9.9 (-3%)
(15%)
Local-DP
28.8 (-39%)
(-
23.3 (+76%)
10.4 (+8%)
10.95 (+7%)
(+5%)
HDN-S
36.4 (222%)
14.5 (-25%)
12.5 6%)
8.8 (-8%)
9.5 (-6%)
14%)
HDN-DR
24.9 (47%)
12.7 35%%
11.9 10%)
8.7 (00%
9.4 (-8%)
22%)
HDN-DP
25.1 (-47%)
12.8 (34%)
13.2 (0%)
8.7 (-9%)
9.3 (8)
19%)
Table I: Comparison of different depth normalization strategies. Our proposed HDN outperforms the instance- level normalization baseline remarkably. Relying only on local normalization produces unstable results across benchmarks.
HRWSI and Holopix50k. HRWSI [3 and Holopix50k || are both diverse relative depth datasets. Although they contain diverse scenes and various camera settings, their provided stereo images are uncalibrated. Therefore, the stereo matching methods are inapplicable to recover the metric depth
information. We use RAFT 28
to recover their relative depth representations for training.
Implementation details. We use the state-of-the-art monocular depth estimation network DPT- Hybrid (3 in our experiments. The input image size is 480 x 480 both training and testing time. For evaluation on benchmarks, we re-size the outputs to the raw image resolutions with the bilinear interpolation. Random horizontal flip and random crop are employed for data augmentation. The default random crop size in all experiments is sampled in [0.5, 1] of the raw image size, with the aspect ratio restricted in [3/4, 4/3]. and the random cropped patches are re-sized the input resolution for training. We use the Adam optimizer with learning rate of 10-5, The model is trained on 8 V100 GPU with the batch size of 32. each mini-batch, we sample the equal number of images from different training data sources. For HDN in the spatial domain, we select the grid size Sppnial from f 22, 2 to construct the hierarchical contexts. For HDN in depth domain, the group number Sdepth is chosen from {
2 .2²).
Evaluation Metrics. We include 5 popular benchmarks that are unseen during training, including.
DIODE (30, ETH3D (25) KITTI [9, NYU [77.. and ScanNet [5]. We follow the previous works to evaluate our model on zero-shot cross-dataset transfer. We use the mean absolute value of the relative M
4 error (AbsRel): (1/M) > Cim1 |d; dīi /dj and the percentage of pixels with 6 max क _ <
1.25. Following Midas [24] and LeReS [37], we align the predictions and ground truth in scale and shift before evaluation. Please refer to our supplementary material for more experiment results and analysis.
4.1 Analysis
For ablation study and analysis, we sample a subset of 16K images evenly from our four training sets, and the input image size is 384 x 384.
Comparison of normalization strategies. At the beginning, we compare different normalization strategies. The first baseline is the instance-level normalization, i.e., scale-and-shift invariant loss. We also include the batch-level depth normalization for a comprehensive study. For our proposed HDN, we report the results of models based on the spatial domain (HDN-S) and the depth domain (HDN-DP and HDN-DR). We also take the finest normalization level of the proposed HDN to validate whether local contexts alone help the training, which is denoted by Local. A detailed record of the experiment results is presented in Table
As we can see, normalization along the batch dimension yields the worst results, which means that the sale--addshiff changes between data can not be addressed by utilizing batch-level statistics. Our proposed HDNs in both the depth domain and the spatial domain remarkably outperform the instance-level normalization baseline, which validates the effectiveness of our design. The HDNs in the depth domain are more effective. In particular, HDN-DR reduces the error by an average of 22% on the five benchmarks, so we use HDN-DR as the default model in rest experiments. We find that the instance normalization bascline performs poorly on DIODE and ETH3D datasets, particularly in

---

LOss
AbsRelf
61 t
LI
14.7
80.1
LI+ + SSI 24
14.6
0.6%%
79.1
L1 HDN
13.3 9.5%)
82.8
Table 2: Experiments results under the fully supervised setting. The experiment is conducted on NYU V2 dataset [27]. Our design also improve the training of monocular estimation models under the fully supervised setting.
their outdoor splits. This may be caused by dataset bias or the shortage of diversity in the training datasets. In contrast, the proposed HDNs perform well on all benchmark datasets and even reach state-of-the--rt with only 16K training data, which are dozens of times fewer than the training seis used in recent methods. Relying solely on local normalization, i.e., the finest normalization level, can improve the baseline on some benchmarks, e.g., DIODE and ETH3D, but it may meanwhile cause significant performance drops on other benchmarks, e.g.. KITTI. This suggests that the combination of local normalization and global normalization is the optimal strategy for robust training and good generalization.
Delving into HDN. We next study the characteristies of the proposed HDN, As the key difference between HDNs and instance-level normalization is the extra local normalization contexts, first validate whether our methods still work well when different strength of random cropping is applied for data augmentation. We select the lower bound of cropping size from {1/8, 1/4, 1/2] of the raw size to observe the improvements. We then gradually add more fine-grained normalization context levels and observe the performance changes. For example, if three levels are adopted for HDN-Depth, the group number Sdepth is chosen from {20, 2 2²). When only the top level is adopted, the HDN degrades to the instance-level normalization baseline. We also report performance changes of our method relying on the finest levels alone, which is also the Local model variants in Table [] For all experiments, we report the mean relative error drop rate (AbsRel) over the instance-level normalization baseline in T whose lower bound of random crop is 0.5.
The results presented in Fig. Removing the random crop (No Crop) harms the overall performance. which can be seen from the starting point of different charts that denote the instance- level normalization baselines. Although instance-level normalization with aggressive random crop (Crop 1/8 and Crop 1/4) can also enforce local spatial normalization contexts, it may meanwhile lose useful contexts for feature encoding and causes performance drops. Our proposed HDNs can consistently and significantly outperform the baselines under various augmentation strengths, which indicates that data augmentation with random cropping and our HDNs are complementary.
For our HDNs, adding more fine-grained normalization context levels can not continually improve the performance. By observing the results of normalization relying on the local contexts alone, find that they can still outperform the instance-leve, normalization baseline when the appropriate level is assigned. However, the performance is likely to get worse significantly when the contexts become too local, as can be seen from the result of Local-DP.
The effectiveness of HDN under the fully supervised setting. We next validate whether our proposed HDN can help the model training under the fully supervised setting, where the metric depth annotations are provided in the training set. We validate our design the popular NYU V2 dataset, which contains 795 training images in Eigen split (7). by adding our proposed HDN loss as an auxiliary loss to a standard L.I regression loss. The result is shown in Table Our proposed HDN can still effectively improve the performance, while the SSI bascline can hardly boost the performance, which shows the generalization capability of our methods in monocular depth estimation tasks.
Qualitative results. We present some qualitative comparisons in Figure 4 with the instance-level normalization baseline, i.e., the scale-shift-invarian (SSI) loss 24 All comparison cases are sampled from the zero-shot testing datasets. We can observe that our hier- archical normalization methods make better predictions in the smoothness of object surface and the sharpness of edges. Therefore, compared to the global instance-level normalization, our extra local normalization in the depth domain or the depth domain can force the network to master better knowledge of local geometries. Furthermore, we randomly sample 2K pixels from each image and plot the predicted depth value and ground truth depth value in the last two columns. Idcally, the
We mainly compare
7

---

Norm. Method
auce DBB
SDBE
AbsRell
Instance-level
2.55
5.84
9.81
HDN-S
2.35
5.15
8.58
HDN-DR
2.41
5.57
8.76
HDN-DP
2.47
5.67
8.99
Table 3: Results on iBims-1 dataset 15.. metries for evaluating the depth boundaries.
The proposed normalization methods outperform the baseline in
10
10
5
5
0
-5
-10
-10
-10
-15
-15
-20
-25
-25
-25
Nam. of Levels
Num.
Num
(a) No Crop
(b) Crop 1/4
(c) Crop 1/8
10
30
20
-5
10
HDN-S ..... Local-S
-DN-DDDR
Local-DR
-20
-20
-25
-30
Local-DP
Num. of Levels
Num. of Levels (e) Crop 1/2
(d) Crop 1/2
Figure 3: Different numbers context levels for the proposed Hierarchical Depth Normalization and variants based solely on the lowest level (Local). Lower means better. The proposed normalization approaches consistently outperform the baselines (number of level 0) under various data augmentation strengths. Relying only local normalization with very small contexts may harm the performance.
predicted affine-invarian depth should be strictly linear to the ground truth, i.e., the red diagonal line. We can observe that the linearity of our method much better than the baseline. Note that even the baseline could achieve similar prediction accuracy sometimes (such as the second example), our methods generate more find-grained details.
Evaluation of depth boundaries. We have demonstrated the advantages of our proposed methods in generating sharper edges and better details through visualization. We next quantitatively evaluate the quality of edges based on experiments on iBims-1 dataset [15]. In particular, we choose the metrics specifically used for evaluating depth boundaries, i.e., the edges. Please refer to [[5] for detailed definitions of the metrics. The result is presented in Table 3 As we can see, our proposed normalization strategies outperform the baselines.
4.2 Comparison with State-of-the-Art
Finally, we compare our depth estimator (HDN-DR) with the state-of-the-ar methods on five benchmark datasets, including, DIODE (30), ETH3D (55) KITTI NYU [27,, and ScanNet (5), which are unseen during training. As the result shows in Table our method outperforms previous methods by large margin on multiple benchmarks with fewer training data than recent state-of-the-art methods, such as LeReS (37) and Midas (24].

---

Inputs
SSI
Ours
SSI
Ours
Prodictdd Docth
Cheelctee Deeth
Ground Dopth
Ground Depth
Weddeted Depth
Predd Redtleed Depnis
Ground Depth
Ground Depth
Predieted Denth
Pred Doech
Ground
Ground Depth
Predte red Depitt
Redilted Derith
Depth
Ground Depth
Reed Deeph
Prodeted Seythh
Ground Depth
Ground
Predictd Depih
Pedbered Bhetth
Ground Depth
Ground Depth
Predeted Depis
sned Deepth
Ground Depth
Ground Depth
Figure 4: Qualitative comparison of method and the baseline replying instance-level normalization (SSI (4)). The red diagonal lines denote the ground-truth, and closer it means better. Our method generates smoother surfaces, sharper edges and more details.
Limitations
We find that for datasets with very high image resolutions, such as ETH3D with 4032 6048 raw resolutions, relatively small inference resolution, e.g., 384 x 384, is worse than relatively large resolution, e.g., 800 x 800, but further increasing the resolution can not consistently improve the performance. Therefore, there is lack of principle for setting the optimal input resolutions at test time. We believe that research on adaptive input resolutions and learnable test-time augmentations would be promising solutions in the future.
Our future work also includes better strategies to define the hierarchical normalization contexts, such utilizing cross-domain knowledge and extra knowledge, e.g., segmentation maps.

---

NYU
KITTI
DIODE
ScanNet
ETH3D
Method
Training Data
AbsRel
61
AbsRell d
AbsRel 811
AbsRell d1
AbsRel. 61t
OASIS
140K
21.9
66.8
31.7
43.7
48.4
53.4
19.8
69.7
29.2
59.5
MegaDepth
19
150K
19.4
71.4
20.1
66.3
39.1
71.2
26.0
64.3
39.8
52.7
Xian et al. [33
30K
16.6
77.2
27.0
52.9
42.5
61.8
17.4
75.9
27.3
63.0
WSVD 31
1.5M
22.6
65.0
24.4
60.2
35.8
63.8
18.9
71.4 26.1
61.9
Chen
795K
16.6
77.3
32.7
51.2
37.9
66.0
16.5
76.7
23.7
67.2
DevtrseDepth
34
300K
87.5
19.0
70.4
37.6
63.1
22.8
69.4
MiDaS
24
2M
11.1
88.5
23.6
63.0
33.2
71.5
11.1
88.6
18.4
75.2
Leres
37
360K
9.0
91.6
14.9
78.4
27.1
76.6
9.5
91.2
17.1
77.7
Ours
300K
6.9
94.8
11.5
86.7
24.6
78.0
8.0
93.9
12.1
83.3
Table 4: Comparison with state-of-the-aa methods on six zero-shot transfer benchmark datasets, Our model significantly outperforms previous methods and sets new state-of of satte-of-hhh-rte the-hhttrttt art in many benchmarks. We use the MiDaS model reported in their original paper for comparison.
Conclusion
In this paper, we have presented a novel hierarchical depth normalization strategy for monocular depth estimation tasks. Compared with the existing instance-level normalization strategy that mainly focuses on the global structure of the scene in the image, our normalization and loss preserve both the fine-grained details and overall structure. Extensive experiments validate the effectiveness of two implementations the spatial domain and the depth domain, and new state-of-the-arr performance is set on multiple benchmarks.
Acknowledgments C. Shen's participation was in part supported by a major grant from Zhejiang Provincial Government.
References
[1] Jia- Ba-Wang Bian, Zhichao Li, Naiyan Wang, Huangying Zhan, Chunhua Shen, Ming-Ming Cheng, and Tan Reid. Unsupervised scale-consistent depth and ego-motion learning from monocular video. In Proc. Advances in Neural Inf. Process. Syst. 2019.
[2] Weifeng Chea, Zhao Fu, Dawei Yang, and Jia Deng. Single-image depth perception in the wild, In Proc. Advances in Neural Inf. Process, Syst. pages 730-738, 2016.
[3] Weifeng Chen. Shengyi Qian, and Deng, Learning single-image depth from videos using quality assessment networks. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 5604-5613, 2019.
[4] Weifeng Chen, Shengyi Qian, David Fan, Noriyuki Kojima, Max Hamilton, and Jia Deng. Oasis: A large-scale dataset for single image the wild. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 679 079,688,000 2020.
[5] Angela Dai, Angel Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießnnr. Scannet: Richly-annotated 3d reconstructions of indoor scenes. Proc. IEFE Conf. Comp. Vis. Patt. Recogn., pages 5828-5839, 2017.
[6] Navneet Dalal and Bill Triggs. Histograms of oriented gradients for human detection. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn. Ieee, 2005.
[7] David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from a single image using a multi-scale deep network. In Proc. Advances in Neural Inf. Process. Syst., pages 2366-2374, 2014.
[8] Adrien Gaidon, Qiao Wang. Yohann Cabon, and Eleonora Vig. Virtual worlds as proxy multi-object tracking analysis. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 4340-4349, 2016.
[9] Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are ready for autonomous driving? the kitti vision benchmark suite. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn. pages 3354 3361. JEEE, 2012.
[10] Clément Godard, Oisin Mac Aodha, Michael Firman, and Gabriel Brostow. Digging into self- monocular depth estimation. In Proc. IEEE Int. Conf. Comp. Vis., pages 3828 3838,, 2019.
10

---

[11] Yiwen Hua, Puneet Kohli, Pritish Uplavikar, Anand Ravi, Saravana Gunaseelan, Jason Orozco, and Edward Li. H0lopixSOk: A large-scale tollh stereo image dataset. In IEEE Conf. Comput. Vis. Puttern Recog. Worksh., June 2020.
[12] Xun Huang and Serge Belongie, Arbitrary style transfer in real time with adaptive instance normalization. In Proc. IEFF Int. Conf. Comp. Vis., pages 1501-1510, 2017.
[13] Sergey loffe and Christian Szegedy. Batch normalization: Accelcrating deep network training by reducing internal covariate shift. In Proc. Int. Conf. Mach. Learn., 2015.
[14] Youngjung Kim, Hyungjoo Jung, Dongho Min, and Kwanghoon Sohn. Deep monocular depth estimation via integration of global and local predictions. IEEE Trans. Image Process. 27(8):1131-44 14444, 2018
[15] Tobias Koch, Lukas Liebel, Friedrich Fraundorfer, and Marco Körner. Evaluation of CNN-based single- image depth estimation methods. In Eur. Conf. Comput. Vis. Worksh. pages 331-348, 2018.
[16] Alex Krizhevsky, Ilya Sutskever, and Geoffre E Hinton. Imagenet classification with deep convolutional neural networks. In Proc. Advances in Neural Inf. Process. Syst., 2012.
[17] Jin Han Lee, Myung-Ky Kyu Han, Dong Wook Ko, and Il Hong Suh. From big small: Multi-scale local planar guidance for monocular depth estimation, arXiv: Comp. Res. Repository, page 10326, 2019.
[18] Ruibo Li, Ke Xian, Chunhua Shen, Zhiguo Cao, Hao Lu, and Lingxiao Hang. Deep attention- classification network for robust depth prediction. In Proc. Asian Conf. Comp. Vis., pages 663-678. Springer, 2018
[19] Zhengqi Li and Noah Snavely. Megadepth: Learning single-view depth prediction from internet photos. In Proc. IEEE Conf. Comp. Vis. Patr. Recogn., pages 2041-200102 2050, 2018
[20] David G Lowe. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision, 60(2):91- 110, 2004.
[21] Taesung Park, Ming- Yu Liu, Ting-Chun Wang, and Yan Zhu. Semantic image synthesis with spatially- adaptive normalization. Proc. IEEE Conf. Comp. Vis. Patt. Recogn. pages 2337-2346, 2019.
[22] Xiaojuan Qi, Renjie Liao, Zhengzhe Liu, Raquel Urtasun, and Jiyy Jia. Geonet: Geometric neural network for joint depth and surface normal estimation. Proc. IEFE Conf. Comp. Vis. Part. Recogn., pages 283-291, 2018.
[23] René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vision transformers for dense prediction. In Proc. IEEE Int. Conf. Comp. Vis., pages 12179-12188, 2021.
[24] René Ranftl, Katrin Lasinger. David Hafner, Konrad Schindler. and Vladlen Koltun. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. IEEE Trans. Pattern Anal. Mach. Intell., 2020.
[25] Thomas Schops, Johannes Schonberger, Silvano Galliani, Torsten Sattler, Konrad Schindler, Marc Pollefeys, and Andreas Geiger. A multi-view stereo benchmark with resolttin images and multi- camera videos. In Proc. JEEE Conf. Comp. Vis. Patt. Recogn., pages 3260-3269, 2017.
[26] Chang Shu, Kun Yu, Zhixiang Duan, and Kuiyuan Yang. meature-metric loss for self-supervised learning of depth and egomotion. In Proc. Eur. Conf. Comp. Vis., pages 572-588, 2020.
[27] Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. In Proc. Eur. Conf. Comp. Vis., pages 146-760. Springer, 2012.
[28] Zachary Teed and Jia Deng. Raft: Recurrent pal-s field transforms for optical flow. In Proc. Eur. Conf. Comp. Vis., pages 402-419. Springer, 2020.
[29] Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Instance normalization: The missing ingredient for fast stylization. arXiv preprint Xiv: 660.00002, 2016.
[30] Igor Vasiljevic, Nick Kolkin, Shanyi Zhang, Ruotian Luo, Haochen Wang. Falcon Z Dai, Andrea Daniele, Mohammadreza Mostajabi, Steven Basart, Matthew R Walter, et al. Diode: A dense indoor and outdoor depth dataset. arXiv: Comp. Res. Repository, page 1908.00463, 2019.
[31] Chaoyang Wang, Simon Lucey, Federico Perazzi, and Oliver Wang. Web stereo video supervision for depth prediction from dynamic scenes. In Inf. Conf. 3D. Vis., pages 157- IEEE, 2019.
11

---

[32] Ke Xian, Chunhua Shen, Zhiguo Cao, Hao Lu, Yang Xiao, Ruibo Li, and Zhenbo Luo. Monocular relative depth perception with web stereo data supervision. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 311-320, 2018.
[33] Ke Xian, Jianming Zhang, Oliver Wang, Long Mai, Zhe Lin, and Zhiguo Cao. Structure- guided ranking loss for single image depth prediction. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 611-620, 2020.
[34] Wei Yin. Yifan Liu, Chunhua Shen. Virtual normal: Enforcing geometric constraintsfor accurate and robust depth prediction. IEEE Trans. Pattern Anal. Mach. Intell., 2021.
[35] Wei Yin. Yifan Liu, Chunhua Shen, and Youliang Yan. Enforcing geometric constraints of virtual normal for depth prediction. In Proc. IEEE Int. Conf. Comp. Vis., 2019.
[36] Wei Yin, Jianming Zhang, Oliver Wang. Simon Niklaus. Simon Chen, Yifan Liu, and Chunhua Shen. Towards accurate reconstruction of 3d scene shape from a single monocular image. IEEE Trans. Pattern Anal. Mach. Intell., pages 1 1-2 2022.
[37] Wei Yin, Jianming Zhang, Oliver Wang, Simon Niklaus, Long Mai, Simon Chen, and Chunhua Shen. Learning to recover 3d scene shape from single image. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., 2021.
[38] Amir Zamir, Alexander Sax, William Shen, Leonidas Guibas, Jitendra Malik. and Silvio Savarese. Taskonomy: Disentangling task transfer learning. In Proc. IEEE Conf. Comp. Vis. Patr. Recogn. IEEE, 2018.
[39] Amir R. Zamir, Alexander Sax. Nikhil Cheerla, Rohan Suri, Zhangjie Cao, Jitendra Malik, and Leonidas J. Guibas. Robust learning through cross-task consistency. Proc. IEEE Conf. Comp. Vis. Patt. Recogn.. June 2020.
[40] Chi Zhang, Yujun Cai, Guosheng Lin, and Chunhua Shen, DeepEMD: Few-shot image classification with differentiable earth mover's distance and structured classifiers. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., 2019.
[41] Chi Zhang, Yujun Cai, Guosheng Lin, and Chunhua Shen. Decpemd: Differentiable earth mover's distance for few-shot lcarning, 2020.
[42] Feihu Zhang, Victor Prisacariu, Ruigang Yang. and Philip Torr. Ga-net: Guided aggregation net for end-to-end stereo matching. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 185-194, 2019.
[43] Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang. and Jiaya Jia. Pyramid scene parsing network. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 2881-2881 1-990, 2017.
[44] Tinghui Zhou, Matthew Brown, Noah Snavely, and David Lowe. Unsupervised learning of depth and ego-motion from vidco. In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 1851-1858, 2017.
12


Reviews:
[{"review_id": "xhjGETmP95y", "reviewer": "Reviewer_v9VC", "summary": "This paper proposes a normalization method for monocular-based depth estimation before computing the regression loss. It is based on scale-and-shift invariant loss. Instead of computing the median and doing the normalization globally, it split the depth map into multiple size patches and compute scale-and-shift invariant loss for each patch. They propose two strategies to generate the patches consisting of spatial domain splitting and depth domain splitting. In the zero-shot setting, the results show the model with the proposed normalization can outperform all baselines. ", "questions": "1. Because it will compute multiscale scale-and-shift invariant loss, will the proposed loss make the training significantly slower?", "limitations": "None", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "1. This paper is well written. It is easy to read.\n2. The proposed idea is clean and simple. We can easily apply it to other depth estimation models.\n3. The improvement on the zero-shot monocular depth estimation task is significant. \n\nWeakness:\n1. Table 1 does not show the ablation when combining the different domains. It is unclear if combing all of them can get the best performance.\n2. The proposed method can be applied to all monocular-based models. Have the authors tested the loss on fully supervised monocular/stereo depth models? This paper only presents the results in a zero-shot setting.\n3. The proposed method is very simple, so the contribution might be limited. To improve the contribution, it is important to show the generalization of different models and datasets.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "s9GblvLaZCj", "reviewer": "Reviewer_rDoq", "summary": "The paper studies zero-shot monocular depth estimation. One way to improve zero-shot performance is to train on a combination of datasets. The problem with this strategy is that different datasets has different scales and shifts. One way to address this issue is to train with a scale and translation invariant loss (SSI [22]). This paper argues that the SSI loss is normalized over an entire image. A more effective way is to normalize over a small local window or a certain depth range. This paper thus propose a hierarchical depth normalization training loss, which helps their model to achieve state-of-the-art performance in zero-shot depth estimation on five major benchmarks.", "questions": "1. The formulation in equation (1) seems to be different from the original one in the SSI [22] paper. SSI solves for an optimal scale and shift between pred and gt depth, while equation (1) is normalizing using simple global stats such as median and mean. Why?", "limitations": "The authors have adequately addressed the limitations and potential negative societal impact of their work.", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n+ This paper proposes a simple yet very effective version of the scale-shift-invariant loss for training depth estimation network. This loss proves to help in improving zero-shot depth estimation (Table 1). I like this idea.\n\nWeakness:\n- It is not very clear if the improvement really comes from the proposed hierarchical depth loss (Table 2). The \"Ours\" model differs from SOTAs in the training data, the number of training samples, and the network architecture. The conclusion would be more solid if only one thing is changed at a time.  For example it is unclear if it is fair to claim that \"our method outperforms previous methods by a large margin on multiple benchmarks with fewer training data than recent state-of-the-art methods, such as Leres [33] and Midas [22].\", because it could be a result of using higher quality training data.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "4DOeKoHwQOf", "reviewer": "Reviewer_uzns", "summary": "This paper presents a novel normalization technique for learning monocular depth estimation networks with various sources of datasets that may have different depth scales. Unlike the previous approach that learns with the image-level normalization, which often disregards the fine-grained depth difference, this paper presents a multi-scale depth normalization that hierarchically normalizes the depth representations based on both spatial information and depth distributions. Experimental results have shown the superiority of this method even though the technique is relatively simple.", "questions": "- Using a hierarchical normalization has been already used in many other fields, as the authors also described, and applying this for monocular depth estimation is the first attempt in our knowledge, so it would be fine. However, due to its simplicity, I think more through experiments should be conducted. For instance, why not dividing the depth representation irregularly? or is there any way to consider additional semantic information, e.g., object boundaries, and divide the depth representation with respect to this semantic information? Without this kinds of through experiments, it would be hard to say this regular hierarchical representation is optimal. \n-  Fig. 2 is hard to follow. In the left, what the authors tries to say? Maybe different colors are coming from the different spatial location? It would be better if it is clarified.\n- It would be interesting if we only used a single dataset for learning monocular depth estimation networks which does not suffer from depth scale or shift problem, e.g., KITTI itself. Maybe performance is degraded with this additional local normalization layers? \n- To overcome depth scale and shift problem, other loss functions may be used, for instance, AbsRel. It would be better if the other loss functions are compared. \n- Computation complex may be evaluated as well.\n\nMinor comments:\n- It would be great if the authors clarify which datasets are used for each evaluation of Table 2. \n- Other backbone models such as Monodepth2 can also be used? In many applications, the CNN-based models are still popularly used, so this experiment would be very interesting.\n\nOverall I like such a simple idea, but more through evaluations would be required to argue this is optimal. I really want to see the rebuttal.", "limitations": "The authors well mentioned the limitations of this paper.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "+ The idea of using multi-scale normalization, inspired by conventional normalization methods such as AlexNet, SIFT, and HOG, is very simple, but effective.\n+ Dividing the normalization candidates as spatial and depth axis is interesting and makes sense.\n+ The state-of-the-art performance is attained.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "3yBC9hcgzJ2", "reviewer": "Reviewer_ftki", "summary": "The authors introduce a novel strategy for monocular depth estimation that focuses on both global structure and fine-grained details. Their implementation is effective at preserving fine-grained details via the spatial and depth domains using Hierarchical Normalization. Ultimately, performing well in terms of accuracy, even against state-of Art benchmarks from before our work was published.", "questions": "Why the median in (1)? More elaboration on the insights could be worthwhile.", "limitations": "A lack of limitations in the ability to be extended to MVS; lack of demonstrating the final reconstruction results (just depth images depicted).", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "An interesting solution to a renowned problem in monocular depth estimation.\n\nClearly motivated by the shortcomings of existing SOTA.\n\nThe presentation is nice, for the most part; the language is okay but should be further improved if accepted for publication.\n\nWeaknesses:\nSeems the authors ran out of space, with many means to improve figures and tables to increase the data-to-ink ratio.\n\nFigure 3-4 : why not share x and y axes labels and make graphs easier to see in less space.\n\nAdd future work to the Limitations section.\n\nBe consistent with style: different tables use different schemes (some use arrows, others use colors).\n\nAvoid orphans (single words taking up entire lines, like line 205).\n\nLack of depth in analysis for the many ablations conducted.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept