Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: The paper introduces foveabox, a method that performs "keypoint" like object detection -- instead of "anchor" based detection (to be discussed later). The idea is simple: predict class labels for pixels that fall within (a reduced version) the GT boxes of the instance; and predict bounding box offsets for those positive pixels. The idea is built on top of the FPN backbone, where a set of feature maps (each representing a specific scale) are used to detect object boxes in multiple scales.  The method is mainly compared against RetinaNet (which is "anchor" driven), and also compared against other more recent methods (ExtremeNet, CenterNet, FCOS) etc. 

+ The paper is quite well written and structured, the illustrations are also clear;
+ I have also read its previous version, and the new version has added a significant amount of work improving it -- e.g. added feature alignment and group norm;
+ I haven't fully checked the results section of other concurrent papers for full comparison, but the current results are among the state-of-the-art for one-stage detectors.

- I don't think the paper breaks away from the notion of "anchors". The current approach is in fact implicitly defining a *single* anchor for each feature map, and changing the assignment rule from IoU based to distance/scale based. I firmly believe that such modification can lead to concrete improvements -- for example recent work has shown that changing assignment rule can improve AP even with fewer anchors (TensorMask); but to say that the paper breaks away from the usage of anchors, it is a bit far for me. I think I will be way more convinced if the experiments are done on a single-scale feature map (anchor also works with single-scale but I am not sure without defining different anchors it can work just on C4 for example).
- Also because such assignment rule has been there before (e.g. DeepMask has used x, y, scale for assignment), and with other recent works (CenterNet, Objects as Keypoints), the contribution of this work is fairly limited

Therefore, I vote for reject.
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: This paper introduces an anchor-free object detection framework that aims at simultaneously predicting the object position and the corresponding boundary. To achieve this, the proposed FoveaBox detector predicts category-sensitive semantic maps for the object existing possibility, and  produces category-agnostic bounding box for each position that is likely to contain an object. The scales of target boxes are associated with feature pyramid representations. Experiments are performed on MS COCO detection benchmark.

Pros:
The proposed approach is simple and is shown to avoid most computation and hyper-parameters related to anchor boxes. The paper is well written and easy to follow. 
Cons:
The main issue with the paper is the main idea is similar to [1,2, 3, 4, 5]. For instance, CenterNet also represents each object instance by its features at the center point and achieves similar detection performance compared to the proposed detector. Further, no speed comparison with these approaches is provided in the paper. Without a fair speed comparison and with similar detection performance, it is difficult to fully assess the merits of the proposed approach. Though inference speed comparison is reported with RetineNet. However, a proper and detailed comparison with [1, 2, 3, 4, 5] is missing. 

1: Xingyi Zhou, Dequan Wang, Philipp Krähenbühl: Objects as Points. CoRR abs/1904.07850 (2019).
2: Xingyi Zhou, Jiacheng Zhuo, Philipp Krähenbühl: Bottom-Up Object Detection by Grouping Extreme and Center Points. CVPR 2019.
3: Zhi Tian, Chunhua Shen, Hao Chen, Tong He: FCOS: Fully Convolutional One-Stage Object Detection. CoRR abs/1904.01355 (2019).
4: Hei Law and Jia Deng: Cornernet: Detecting objects as paired keypoints. ECCV 2018.
5: Kaiwen Duan, Song Bai, Lingxi Xie, Honggang Qi, Qingming Huang, Qi Tian: CenterNet: Keypoint Triplets for Object Detection. CoRR abs/1904.08189 (2019).


review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: ## Overview
The paper tackles object detection without predefined anchors (sliding windows). 
The paper is well motivated and existing detection methods often rely on anchors, which may limits their potential. So this paper is solving an interesting problem and seems novel.
The paper provides experiments to support the proposed architecture empirically.

## Summary of the contribution:
1. The paper proposed an object detection approach called FoveaBox that does not rely on anchors (sliding widows). 
2. The paper shows that FoveaBox outperforms some existing object detection methods.
3. The paper shows that FoveaBox can also be used for object proposals by changing the classification target to class-agnostic head.

## My feedback:
I think the paper is well motivated and has value to the object detection community. So I am leaning positive to accept this paper. The major reasons are:
1. The paper studies an interesting architecture which does not rely on sliding windows and shows its effectiveness. The problem seems not very well studied in the existing object detection literature.
2. The paper provides sufficient ablation study to analyze and understand the proposed methods.
However, I believe the paper could be significantly improved especially in the writing. So my position is not strong.

## Improvements
1. The writing can be significantly improved. The paper reads a bit confusing and unclear especially at the technical part. The paper could benefit from a clear overview figure about the proposed approach. Figure 4 seems to be doing that illustration but giving only the tensor shape seems quite confusing. I also listed some questions below about the actual algorithm.
2. The paper could be improved with experiments and ablation on another dataset. Current ablation shows that \eta should be 2.0 for best performance but without a second dataset it is hard to say this value is general. So one may have to tune this parameters in different datasets.
3. The scale is still discretized. So that is essentially anchors in the scale space. I wonder how could the approach applies to scale to?

## Questions
1. How would the proposed approach address the issue when there are multiple bounding boxes around the same pixels? It seems the current approach is predicting a box per pixel?
2. What is the inference computation cost? And do you have a comparison with existing methods on the compute cost?
review_title: Official Blind Review #2

Meta-review: The paper proposes a method for object detection by predicting category-specific object probability and category-agnostic bounding box coordinates for each position that's likely to contain an object. The proposed idea is interesting and the experimental results show improvement over RetinaNet and other baselines. However, in terms of weakness, (1) conceptually speaking it's unclear whether the proposed method is a big departure from the existing frameworks; and (2) although the authors are claiming SOTA performance, the proposed method seems to be worse than other existing/recent work. Some example references are listed below (more available here: https://paperswithcode.com/paper/foveabox-beyond-anchor-based-object-detector). 

[1] Scale-Aware Trident Networks for Object Detection
https://arxiv.org/abs/1901.01892

[2] GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond
https://arxiv.org/abs/1904.11492

[3] CBNet: A Novel Composite Backbone Network Architecture for Object Detection
https://arxiv.org/abs/1909.03625

[4] EfficientDet: Scalable and Efficient Object Detection
https://arxiv.org/abs/1911.09070

References [3] and [4] are concurrent works so shouldn't be a ground of rejection per se, but the performance gap is quite large. Compared to [1] and [2] which have been on arxiv for a while (+5 months) the performance of the proposed method is still inferior. Despite considering that object detection is a very competitive field, the conceptual/technical novelty and overall practical significance seem limited for ICLR. For a future submission, I would suggest that a revision of this paper being reviewed in a computer vision conference, rather than ML conference.

Decision: Reject