Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposed predicting the Long-Range Distance Estimation of objects by exploiting the relative distance between objects. It demonstrated results on two novel datasets to point out the advantage of the algorithm.


main_review: Advantages:

The paper is well written and the problem of long distance object prediction is well motivated.

The evaluation proves that using R4D shows improvement over the previous dept prediction methods.

Results on out-of-distribution data shows that the method generalizes well with different settings shows the robustness of the method.

Weaknesses:

I think there is a fundamental issue with the data augmentation. with the addition of distance to objects the relative distance between the objects remains the same but the image should also scale with the addition of such distances. how is this issue handled during the training.

How are the small objects detected?, current vision algorithms cannot detect such small objects. since the network is very dependent on such detections it would be interesting to know this problem is tackled.

what happens when there are multiple objects occluding the other object how are such scenarios tackled and how does the accuracy improve?
summary_of_the_review: The paper tackles an interesting problem of long distance estimation by introducing a novel dataset and an attention based learning framework. The authors need to clarify some my concerns before acceptance.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes to utilize camera images to further increase the perception capabilities for self-driving vehicles, especially in long-range settings. The paper builds on top of the assumption that there exist objects with accurate distance information. Then, pairwise relationships can be established between such objects and the anchoring objects. By formulating such relationships as a graph, distances can be better estimated for long-range objects.
main_review: 1. The formulation of pairwise relationships between objects is nice. It can be really hard to use a global feature representation to model the interaction between different objects/agents within the 3D scene around the self-driving vehicles. Using such a consensus-based formulation, it is easier for the framework to figure out what can be better estimated.
2. One of the weaknesses is the dependencies on accurate active sensors such as lidars and radars. This to some extent limits the application scenario for the proposed method. It might be hard to deploy the proposed method to many vehicles with limited low-level ADAS capabilities.
3. The statement of lidar sensor limitations might be inaccurate with respect to current industrial applications. Although open-source academic datasets can only provide lidar measurements for up to 80 meters, in practical industrial applications much better lidar sensors are available. Accurate measurements can be provided for at least 200 meters, even 300 meters for certain advanced models. This actually raises an interesting question as well. If lidar can generate point clouds covering a 200-meter radius, with sparser coverage at far away distances, how does this camera-based approach help in these middle-range scenarios (like 150-200 m)? For instance, we get few points covering the objects, which can lead to inaccurate depth estimation. Can the proposed method increase the estimation accuracy and robustness?
summary_of_the_review: Overall, this paper proposed an interesting solution to a very hard visual perception problem. By modeling the pairwise relationship between targets and reference objects, the proposed framework can achieve good performance for the problem of interest. The paper is well written, experimental results are solid.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a method for the task of long-range object distance estimation. It proposes a framework to predict long-range distance by modeling pairwise relationship between short-range objects and long-range objects. To evaluate the method, two datasets are also introduced. Experimental results are mainly compared to the baseline Zhu & Fang (2019) and several distance metrics are evaluated.
main_review: Pros:
1. The long-range distance estimation task is practical for industry applications. It will benefit the community definitely if the proposed datasets will be released as claimed by the authors.

2. The proposed idea of extrapolating long-range distance from cues of short-range objects is great. The motivation behind the model design is also explained.

3. Experiments are sufficient. The ablation studies on data augmentation, relative distance supervision, number of reference objects are very helpful for better understanding of the contribution of each component.

4. The experiments on OOD data is great as the proposed method is shown to have better generalization ability, which is critical for autonomous driving applications.

Cons:
1. The paper uses a LiDAR-based detector to generate 3D distance of reference objects. Although the baseline method Zhu & Fang (2019) is modified to add LiDAR depth as input for fair comparison. However, I think it's equally important to experiment with pure image-based setting, i.e., generating distance of reference objects with monocular image based method. It's unclear how the quality of reference object distance affects the performance of long-range distance estimation.

2. The metrics may need further clarification. If I understood correctly, the distance metrics are computed on the set of "correctly matched" predicted objects with ground truth objects. However, the "correct match" criteria between predicted and ground truth objects is not defined, which may affect the results of distance metrics.

3. It's unclear how each component (data augmentation, contextual modeling, etc.) affects the generalization ability of the method. It would be better if more detailed analysis are added to the OOD experiments.
summary_of_the_review: The work addresses a practical problem in autonomous driving. The motivation and the proposed method are clearly desribed. Experimental results are pretty good on two datasets. Of course, there are also room for improvement in both the experimental setting and evaluations as described in the above reviews. Nevertheless, I lean to acceptance to the paper.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper addresses the task of Long-Range Distance Estimation. In particular, it proposes R4D, a framework to estimate the distance of long-range objects by utilizing the pair-wise relations between the reference objects (objects with known distance) and the target objects (objects of which the distance is to be estimated). In addition, the authors also present two new datasets, pseudo long-range KITTI dataset and anonymous internal long-range dataset, for this task. The evaluation results show that R4D achieves better results than the previous methods.
main_review: ## Strengths
1. The paper is well written and one can follow smoothly.
2. The idea of utilizing the known distance of the near objects to estimate the unknown distance of the distant objects is very intuitive and reasonable.
3. The network design considers various types of embeddings to explore the features from the target object, reference object, appearance relations, and geometric relations. 

## Weaknesses
1. The proposed method utilizes more inputs than the other compared methods, e.g, Zhu & Fang (2019), so the better performance of R4D is expected. Specifically, R4D needs the 2D bounding boxes as well as the distance of the reference object provided by an additional 3D object detector. Also, it is not mentioned how the 2D bounding boxes are obtained: are they from another 2D detector or just projections of the 3D bounding boxes?
2. It is unclear how robust R4D is with respect to the noise of the input 2D bounding box information and the object distance provided by the 3D detector. It would be great if the authors can evaluate this aspect.
3. It is unclear whether the training/test splits are the same for all the evaluated methods.
4. The authors did not describe very clearly how the ground-truth labels for the distant objects are created. In the paper, it reads "labelers use smooth object trajectory constraints to make sure the boxes are consistent over time and space" and it would be better if the authors can elaborate more on it.
5. The distance augmentation is quite interesting. However, it can also confuse the network since the appearances of the objects should look different when the distances of the objects change. For instance, the same car should look smaller when its distance is changed from 200m to 220m. Also, when the reference object and the target object are moved together by, e.g., 20m, they will look closer on the image, which will be reflected by the union embedding.
6. In Table 4, no evaluations are shown for the experiment "We study the effect of removing the relative distance supervision" as stated in Section 5.3.
7. Since the architecture design of Attention & MLP module expects a fixed number of target-reference pairs, how it is handled in practice when there are fewer or more reference objects?
summary_of_the_review: In general, I lean towards accepting the paper since it tackles an important but underexplored task with a reasonable design of the network. Also, the proposed method achieves better results than the previous works (not a surprise though) and the authors verify the design choices with the ablation studies.

However, since I do have some concerns about the paper, my final recommendation will be heavily dependent on the responses of the authors.
review_title: null

Meta-review: This paper focuses on using reference objects for long distance estimation by introducing a novel dataset and an attention based learning framework. While the presentation flows well and the methodology is practically useful, it is only marginally significant and novel. Some of the practical data augmentation aspect raise some question on whether the process wouldn't confuse the network -- the authors provide empirical evidence of the contrary in their response, although I find the principled argument to be somewhat lacking.
Decision: Accept (Poster)