FoveaBox: Beyound Anchor-based Object Detection
Authors: Tao Kong
Fuchun Sun
Huaping Liu
Yuning Jiang
Lei Li
Jianbo Shi
Venue:  (2020)

Abstract: We present FoveaBox, an accurate, flexible, and completely anchor-free framework for object detection. While almost all state-of-the-art object detectors utilize predefined anchors to enumerate possible locations, scales and aspect ratios for the search of the objects, their performance and generalization ability are also limited to the design of anchors. Instead, FoveaBox directly learns the object existing possibility and the bounding box coordinates without anchor reference. This is achieved by: (a) predicting category-sensitive semantic maps for the object existing possibility, and (b) producing category-agnostic bounding box for each position that potentially contains an object. The scales of target boxes are naturally associated with feature pyramid representations. We demonstrate its effectiveness on standard benchmarks and report extensive experimental analysis.  Without bells and whistles, FoveaBox achieves state-of-the-art single model performance on the standard COCO detection benchmark. More importantly, FoveaBox avoids all computation and hyper-parameters related to anchor boxes, which are often sensitive to the final detection performance.  We believe the simple and effective approach will serve as a solid baseline and help ease future research for object detection. 

Under review as a conference paper at ICLR 2020
FOVEABOX: BEYOUND ANCHOR-BASED OBJECT DE- TECTION
Anonymous authors Paper under double-blind review
ABSTRACT
We present FoveaBox, an accurate, flcxible, and completely anchor-free frame- work for object detection. While almost all statc-of-the-ar object detectors utilize predefined anchors to enumerate possible locations, scales and aspect ratios for the search of the objects, their performance and generalization ability are also limited to the design of anchors. Instead, FoveaBox directly learns the object existing possibility and the bounding box coordinates without anchor reference. This is achieved by: (a) predicting category-sensitive semantic maps for the object exist- ing possibility, and (b) producing category-agnostic bounding box for each posi- tion that potentially contains an object. The scales of target boxes are naturally associated with feature pyramid representations. We demonstrate its effective- ness on standard benchmarks and report extensive experimental analysis. Without bells and whistles, FoveaBox achieves state-of-the-arr single model performance on the standard COCO detection benchmark. More importantly, FoveaBox avoids computation and hyper-parametert related to anchor boxes, which are often sensitive to the final detection performance. We believe the simple and effective approach will serve as solid baseline and help ease future research for object detection.
INTRODUCTION
Object detection requires the solution of two main tasks: recognition and localization. Given an arbitrary image, an object detection system needs to determine whether there are any instances of semantic objects from predefined categories and, if present, to return the spatial location and extent. To add the localization functionality generic object detection systems, sliding window approaches have been the method choice for many years (Lampert et al., 2008 Felzenszwalb et al., 2010 Liu
ct al. 2018).
Recently, deep learning techniques have emerged as powerful methods for learning feature repre- sentations automatically from data (Simonyan & Zisserman 2014 He al. 2016: Huang et al.) 2017ap. For object detection, the anchor-based Region Proposal Networks (Ren al. (015) are widely used to serve as a common component for searching possible regions of interest for modern object detection frameworks (Liu et al. 2016 He et al.) 2017] Lin et al., 2018). In short, anchor method suggests dividing the box space into discrete bins and refining the object box in the corre- sponding bin. Most state-of-the-arr detectors rely on anchors to enumerate the possible locations, scales, and aspect ratios for target objects (Liu al.) 2018). Anchors are regression references and classification candidates to predict proposals for two-stage detectors or final bounding boxes for single-stage detectors. Nevertheless, anchors can be regarded as a feature- sharing sliding window scheme to cover the possible locations of objects.
loU-based matching with ground-truth
Anchor design
Training target
Figure The anchor-basee object detection frameworks need (a) design anchors according to the ground-truth box distributions; (b) match anchors with ground-truth boxes to generate training target (anchor classification and refinement); and (c) utilize the target generated by (b) for training.

---

Under review as a conference paper at ICLR 2020
However, anchors must be carefully designed and used in object detection frameworks. (a) One of the most important factors in designing anchors is how densely it covers the instance location space. To achieve good recall rate, anchors carefully designed based on the statistics com- puted from the training/validation set (Lin et al. 2018). (b) One design choice based on a particular dataset is not always applicable other applications, which harms the generality (Yang al. 2018). (c) At training phase, anchor-methods rely on the intersection-over-miioo (IoU) to define the posi- tive/negative samples, which introduces additional computation and hyper--arameters for an object detection system (Wang et al. 2019).
In contrast, our human vision system can recognize the instance in space and predict the boundary given the visual cortex map, without any pre-defined shape template (Bear et al. (007). In other words, we human naturally recognize the object the visual scene without enumerating the candi- date boxes. Inspired by this, an intuitive question to ask is, is the anchor scheme the optimal way to guide the search of objects? And further, could we design an accurate object detection frame- work without anchors candidate boxes? Without anchors, one may expect complex method is required to achieve comparable performance. However, we show that a surprisingly simple and flexible system match, even surpass the prior state-of- oottt---fthhe---e the--hhrrrrr art object detection results without any requirement of candidate boxes.
faven area
dog
fovea in eye
classificution
output
(Fw21)
(tautt)
box prediction
Figure 2: FoveaBox object detector. For each output spacial position that potentially presents an object, FoveaBox directly predicts the confidences for all target categories and the bounding box.
To this end, we present FoveaBox, a completely anchor-free framework for object detection. Fove- aBox is motivated from the fovea of human eyes: the center of the vision field is with the highest visual acuity (Fig 2 left), which is necessary for activities where visual detail is of primary impor- tance (Iwasaki & Inomata 1986). FoveaBox jointly predicts the locations where the object's center area is likely to exist as well as the bounding box at each valid location.
In FoveaBox, each target object is predicted by category scores center area, associated with 4-d bounding box, as shown in Fig 2
right. At training phase, we do not need to utilize anchors, or loU matching to generate training target. Instead, the training target is directly generated by ground-truth boxes.
To demonstrate the effectiveness of the proposed detection scheme, we combine the recent progress of feature pyramid networks and our detection head to form the framework of FoveaBox. Without bells and whistles, FoveaBox gets sfate-of-the single-model results on the COCO object detec- tion task. Compared with the inchor-based RetinaNet, FoveaBox gets 2.2 AP gains, which also surpasses most of previously published anchor based single-model results. We believe the simple training/inference manner of FoveaBox, together with the flexibility and accuracy, will benefit future research on object detection and relevant topics.
2 FOVEABOX
FoveaBox is conceptually simple: It contains a backbone network and fovea head network. The backbone is responsible for computing a convolutional feature map over an entire input image and is an off-the-sheel convolutional network. The fovea head is composed of two nub-branches, the first branch performs per pixel classification on the backbone's 's output; the second branch performs box prediction for each position that potentially covered by object.
2

---

Under review as a conference paper at ICLR 2020
2.1 REVIEW OF FPN AND ANCHORS:
We begin by briefly reviewing the Feature Pyramid Network (FPN) used for object detection (Lin ctt al., 2017). In general, FPN uses a top-down architecture with lateral connections to build an in-network feature pyramid from single-scale input. FPN is independent of specific task. For object detection, each level of the pyramid in FPN is used for detecting objects at a specific scale. On each feature pyramid, anchor-based methods uniformly place A anchors each of the I W spacial position. After computing the IoU overlap between all anchors and the ground-truth boxes, the anchor-based methods define training targets. Finally, the pyramid features utilized optimize the targets.
Positive Anchor
Negative Anchor
GT
Positive area (ga4))
Start point
Figure 3: Anchor-based object detection v.s. FoveaBox object detection. left: The anchor-based method uniformly places A (A = 3 in this example) anchors on each output spacial position, and uti- lizes IoU define the positive/negative anchors; right: FoveaBox directly define positive/negative samples for each output spacial position by ground-truth boxes, and predicts the box boundaries from the corresponding position.
2.2 FOVEABOX
FoveaBox directly predicts the object existing possibility and the corresponding boundary for each position potential contained by an instance. In this section, we introduce the key components step- by-step.
2.2.1 OBJECT OCCURRENCE POSSIBILITY
Given a valid ground-truth box denoted as (I1, 02, U2). We first map the box into the target feature pyramid Pt
F1 2
91
122
1/2
81
x2
1/2
81
91
(1)
c = 0.5(g + 1) cy = 0.5(y2 + v1),
w = I2
h
V1
where s; is the down-sample factor. The positive area Rpos on the score map is designed to be roughly a shrunk version of the original one (Fig right):
2100 c. 0.50w v
0.5ah',
(2)
x2 C. + 0.5mw 20 Cy 0.5ah'
pos
where a is the shrunk factor. At training phase, each cell inside the positive area is annotated with the corresponding target class label. The negative area is the whole feature map excluding area in Rpos For predicting, each output set of pyramidal heat-map has C' channels, where C' is the number of categories, and is of size H x W. Each channel a binary mask indicating the possibility for class, like FCNs in semantic segmentation (Long et al. 2015). The positive area usually accounts for a small portion of the whole feature map, so we adopt Focal Loss (Lin et al. 2018) to train this branch.
3

---

Under review as a conference paper at ICLR 2020
2.2.2 SCALE ASSIGNMENT
While our goal is to predict the boundary of the target objects, directly predicting these numbers not stable, due to the large scale variations of the objects. Instead, we divide the scales of objects into several bins, according to the number of feature pyramidal levels. Each pyramid has a basic scale 7 ranging from 32 512 on pyramid levels P3 to Pr, respectively. The valid scale range of the target boxes for pyramid level I computed as
[rt/n,ri ri m/)
where 7) is empirically control the scale range for each pyramid. Target objects not in the corresponding scale range are ignored during training. Note that an object may be detected by multiple pyramids of the networks, which is different from previous practice that maps objects to only one feature pyramid (He et al. 2017).
2.2.3 BOX PREDICTION
Each grothd-truth bounding box is specified the way G (21, V1. 22, V2). Starting from a positive point (z, y) in Roos, FoveaBox directly computes the normalized offset between (z, y) and four boundaries:
st(r + 0.55)
21 byn = llog
sc 0.5)) 9/
tz, log
11.
r1
(4)
T2
s1(x + 0.5) lya log - s: + 0.5)
lmg
log
71
r
This function first maps the coordinate (2r, 3) to the input image, then computes the normalized offset between the projected coordinate and G. Finally the targets are regularized with the log-space function. rı is the basic scale defined in section 2.2.2
For simplicity, we adopt the widely used Smooth L1 loss (Ren et al. 2015) to train the box prediction Lboz. After targets being optimized, we can generate the box boundary for each cell (zz y) on the output feature maps"] [1] In box branch, each output set of pyramidal heatmap has 4 channels, for jointly prediction of (ta) by tays (-2).
2.2.4 NETWORK ARCHITECTURE
To demonstrate the generality of our approach, we instantiate FoveaBox with multiple architectures. For clarity, we differentiate between: (i) the convolutional backbone architecture used for feature extraction over an entire image, and (ii) the network head for computing the final results.
cls branch
HXW x256
-
HeW x256
FPN is feature
HoW x256
x
HzWi
X4
box branch
Figure 4: On each FPN feature level, FoveaBox attaches two subnetworks, one for classifying the corresponding cells and one for predict the (I1, V1, F2, 32) of ground-truth object box. Right is the score output map with their corresponding predicted boxes before feeding into non-maximum suppression (NMS). The score probability in each position denoted by the color density. More examples are shown in Fig 5
'Eq. and its inverse transformation can be easily implemented by element-wise layer in modern deep learning frameworks (Paszke al. 2017 Chen et al. 2016).
4

---

Under review as a conference paper at ICLR 2020
Most of the experiments based on the head architecture shown in Fig We also utilize different head variants to further study the generality. More complex designs have the potential to improve performance but are not the focus of this work.
2.2.5 IMPLEMENTATIONS
We adopt the widely used FPN networks for fair comparison. Concretely, we construct a pyramid with levels {Pi}, 3, 3,4,....7 where indicates pyramid level. Pt has 1/2' resolution of the input. All pyramid levels have C 256 channels. Fovea head is attached on each pyramid level. Parameters are shared across all pyramid levels.
FoveaBox is trained with stochastic gradient descent (SGD). We use synchronized SGD over GPUs with total of 16 images per minibatch (4 images per GPU). Unless otherwise specified, all models are trained for 12 epochs with an initial learning rate of 0.01, which is then divided by 10 at 8th and again at 1 11th epochs. Weight decay of 0.0001 and momentum of 0.9 are used. Only standard horizontal image flipping is used for data augmentation. During inference, we first use confidence threshold of 0.05 to filter out predictions with low confidence. Then, we select the top 1000 scoring boxes from each prediction layer. Next, NMS with threshold 0.5 is applied for each class separately. Finally, the top- 100 scoring predictions selected for each image. Although there are more intelligent ways to perform post- post-processing, such as bbox voting (Gidaris & Komodakis/ 2015), Soft-NMS (Bodla al. 2017) or test-time image augmentations, in order keep simplicity and to fairly compare against the baseline models, we do not use those tricks here.
3 EXPERIMENTS
We present experimental results on the bounding box detection track of the MS COCO benchmark. All models are trained on MS COCO trainval35k. If not specified, ResNet-50- backbone and 600 pixel train and test image scale are used to do the ablation study. We report lesion and sensitivity studies by evaluating on the minival split. For our main results, we report COCO AP on the test-dev split, which has no public labels and requires use of the evaluation server.
3.1 ABLATION STUDY
Various anchor densities and FoveaBox: One of the most important design factors in an anchor- based detection system is how densely it covers the space of possible objects. As anchor-based detectors use fixed sampling grid, popular approach for achieving high coverage of boxes is to use multiple anchors at each spatial position. One may expect that we can always get better performance sweep
when attaching denser anchors on each position. To verify this assumption, number of scale and aspect ratio anchors used at each spatial position and each pyramid level in RetinaNet, including single square anchor each location 12 anchors per location (Table [a)). Increasing beyond 6-9 anchors does not show further gains. The saturation of performance w.r.f density implies the handcrafted, over-density anchors do not offer an advantage.
the
Over-density anchors not only increase the foreground-backgroundd optimization difficulty, but also likely to cause the ambiguous position definition problem. For each output spatial location, there are A anchors whose labels are defined by the IoU with the ground-truth. Among them, some of the anchors are defined as positive samples, while others are negatives. However they are sharing the same input features. The classifier needs to not only distinguish the samples from different positions, but also different anchors at the same position.
In contrast, FoveaBox explicitly predicts one target at each position and gets no worse performance than the best anchor-based model. Compare with the anchor based scheme, FoveaBox enjoys several advantages. (a) Since only predict one target at each position, the output space has been reduced to 1/A of the machor-based method. (b) There is no ambiguous problem and the optimization target is more straightforward. (c) FoveaBox has fewer paper-parameter,, and is more flexible, since do not need to extensively design anchors to see a relatively better choice.
One of the major benefits of FoveaBox is the robust prediction of bounding boxes. To verify this, we divide the boxes in the validation set into three groups according to the ground-truth aspect ratios u max( h w ). We compare FoveaBox and
FoveaBox is more robust to box distribution:
5

---

Under review as a conference paper at ICLR 2020
Table 1: Ablation experiments for FoveaBox. All models trained on trainva 135k, test minival. If not specified. default values are 2.0 and ø = 0.4. (a) Our anchor-free FoveaBox get 0.9 AP gains compared with the best model of anchor-based RetinaNet; (b) FoveaBox is more robust to bounding box distributions; (c) FoveaBox could also generate high-quality region propos- als; (d) Accuracy of FoveaBox for various network depths and image scales; (e) and (f): FoveaBox
gets best performance with / 2.0 and 0.4. See Section 3.1 for details.
(a) Varying anchor density and FoveaBox.
(h) Detection with different aspect ratios
method
AP
APuc3
AP3suc5
APo25
method
#sc
#ar
AP
AP5o
AP7s
RetinaNet
34.2
36.5
10.2
RetinaNet
30.2
49.0
31.7
RetinaNet
31.9
50.0
34,1
FoveaBox
35.1
36.8
26.8
16.4
RetinaNet
31.9
49.4
33,8
(c) Region proposal performance.
RetinaNet
3
34.2
53.1
36,5
RetinaNet
3
34.2
53.2
36,9
method
backbone
AR
AR300
AR3000
RetinaNet
4
3
33,9
52.1
36,2
RPN
ResNet-50
44.5
51.1
56.6
FoveaBox
35.1
54.3
37.1
FoveaBox
ResNet-50
52.9
57.3
61.5
(d) Different input resolutions and models.
(e) Varying (a = 0.4).
(f)
Varying a (1) 2.0).
net-depth-scale
AP
AP50
AP7s
AP
AP50
AP7s
AP
APso
AP7s
FoveaBox -5-4400
31.9.14
49.6
33.8
1.0
32.0
50.4
31.8
0.2
34.1
53.2
36.0
FoveaBox 5006000
35.1.09
54.3
37.1
1.5
34.1
53.3
36.0
0.3
34.8
54.0
36.7
FoveaBox 50-800
ToveaBox-SO-880
36.4.09
56.2
38.7
2.0
35.1
54.4
37.0
0,4
35.1
54.4
37.0
FoveaBox- 101-400
33.3-14
51.0
35.0
2.5
35.0
54.2
36.8
0.5
34.8
53,9
36,6
FoveaBox- 101-600
37.0-1o
56.4
39.3
3.0
34.3
53.3
36.5
0.6
34.1
53.1
36,0
FoveaBox-104 100-800
38.6-a9
58.0
41.2
4.0
32.8
51.0
34.5
0.7
33.3
52.5
34,9
RetinaNet different aspect ratio thresholds, shown in Table [b). We that both methods get best performance when u is low. Although FoveaBox also suffers performance decrease when increases, much better than the baseline model.
Generating high-quality region proposals: Changing the classification target to class-agnostic head is straightforward and could generate region proposals. We compare the proposal performance against FPN-based RPN (Lin et al. (2117) and evaluate average recalls (AR) with different numbers of proposals on mi n val set, as shown in Table [c). Surprisingly, our method outperforms the RPN baseline by large margin, among all criteria. Specifically, with top 100 region proposals, FoveaBox gets 52.9 AR, outperforming RPN by 8.4 points. This validates that our model's capacity in generating high quality region proposals.
Across model depth and scale: TableTil shows FoveaBox utilizing different backbone networks and input resolutions. The train/inference settings exactly the same as the baseline method (Lin et al. 2018). Under the same settings, FoveaBox consistently gets 0.9~1.4 higher AP. When comparing the inference speed, find that FoveaBox models about 1.1~1.3 times faster than the RetinaNet counterparts.
Analysis of 1) and In (3. 17 controls the scale assignment extent for each pyramid. As increases, each pyramid will response to more scales of objects. Table (T)) shows the impact of on the final detection performance. Another important hyper-parameter the shrunk factor which controls the positive/negative samples. Table ft) shows the model performance with respect to changes. In this paper, o 0.4 and 1) are used in other experiments.
loU-based assignment v.s. fovea area: Another choice of defining the positive/negative samples is firstly gets the predicted box from the box branch, and then assign the target labels based on the IoU between the predicted boxes and ground-truth boxes. As shown in Table 2 the shrunk version gets better performance (+0.4 AP) than the loU-based assignment process.
Better head and feature alignment: The most recent works Chen et al. 2019 Yang et al. 2019) suggest to align the features in one-stage object detection frameworks with anchors. In Fovea we adopt deformable convolution (Dai et al., (017)) based on the box offset learned by Eq. (4. to refine the classification branch FoveaBox works well when adding such techniques. Specifically,
"ee supplementary for network details.

---

Under review as a conference paper at ICLR 2020
Table 2: Label assignment strategy (ResNet-50, 800 scale).
assign method
AP
A0
AP75
IoU (0.5/0.4)
35.6
54.7
37.7
IoU (0.6/0.5)
35.9
54.7
38.4
IoU (0.5/0.5)
36.0
54.9
38.4
Fovea (ar 0.4)
36.4
56.2
38.7
when change the classification branch to heavier head, together with feature alignment and GN, FoveaBox gets 40.1 AP using ResNet-50 as backbone! This experiment demonstrates the generality of our approach to the network design (Table 3).
Table Feature alignment and group normalization (ResNet-50, 800 scale).
cls branch
alignment
GN
AP
AP50
AP75
x5 +256(3x3) +256(xx) -256gxx))+226xxx (2566x)))-26(xxx) +256(333)
36.4
56.2
38.7
23)
+25666x3)
36.8
56.5
38.9
x36(xx3) -25663x3) -2566xxx)))-566x33 -256) +256(333)
37.1
56.7
39.2
36.7
57.0
39.1
-024axx))+0244xxx -1024(xx)>
37.2
57.4
39.4
1024(3x3) (100xxxx)>112220 +0024(xx)
37.5
58.2
39.5
-1244xx))-=1024xxxx 2x epochs
37.9
58.4
40.4
1024(x33 -1124)xx)00014 epochs, mstrain
40.1
60.8
42.5
3.2 MAIN RESULTS
We compare FoveaBox to the state-of-the-ar methods in Table 4 All instantiations of our model outperform baseline variants of previous tate-of-the-an models. The first group of detectors on Ta- aare two-stage detectors, the second group one-stage detectors, and the last group the FoveaBox detector. FoveaBox outperforms all single-stage detectors under ResNet-101 backbone, under all evaluation metrics. This includes the recent one-stage CornerNet and ExtremeNet (Law & Deng. 2018 Zhou et al. 2019b). FoveaBox also outperforms most two-stage detectors, including FPN (Lin et al. 2017). Mask R-CNN (He et al. 2017) and IoU-Net (Jiang et al.) 2018).
Two-stage detectors rely on region-wise sub-networks to further classify the sparse region proposals. Since FoveaBox could also generate region proposals by changing the model head class agnostic scheme (Table [[c),, we believe could further improve the performance of two-stage detectors, which beyond the focus of this paper.
IN CONTEXT OF RELATED WORK
Our work is related previous works in different aspects. Before closing, we discuss the relations and differences in details.
The anchor-based object detection frameworks be gener- ally grouped into two factions: two-stage, proposal driven detectors and one-stage, proposal free methods. Anchors are regression references and classification candidates to predict proposals for
Anchor-based Object Detection:
two-stage detectors(Ren et al. 2015 Lin et al., 2017 He et al. 2017 Cai & Vasconcelos 2018) or final bounding boxes for
detectors (Liu et al. 2016)] Lin al. 2018 (0018)Redmmo Redmon & Farhadi 2017). Most top one-stage detectors rely on the anchor boxes to enumerate the possible locations of target objects.
Anchor-Free Explorations: There are also some prior works trying to remove the dependence of anchors. Due to the absence anchors or region proposals, usually they lack the ability to deal with complex scenes and cases (Huang al. 2015 Redmon et al., 2016). In text detection, the score mask technique has been used due to the arbitrary shape of target text (Zhang et al., 2016; Hu et al.] 2017 Zhou al. 2017). Such works usually utilize the fully convolutional networks predict the
7

---

Under review as a conference paper at ICLR 2020
Table 4: Object detection single-model results v.s. state-of-the-arts on COCO test-dev. We show results for our FoveaBox models with 800 input scale. FoveaBox-align indicates utilizing feature alignment discussed in Section 3.1
backbone
AP
AP50
AP
APs
APM
AP
two-stage methods
Faster R-CNN w FPN (Lin al., 2017
ResNet-101
36.2
59.1
39.0
18.2
39.0
48.2
Mask R-CNN He al. 2017
ResNet-101
38.2
41.7
20.1
41.1
50.2
Faster R-CNN by G-RMI (Hungg al. 2017b
Preppion-RReNNeteee
34.7
55.5
36.7
13.5
38.1
52.0
Faster R-CNN w TDM (Shrivastava et 2016
ResNet-v2
36.8
57.7
39.2
16.2
39.8
52.1
Relation Network (Ilu el 2018
DCN-101
39.0
58.6
42.9
loU-Net (iang et ai. 2018
ResNet-101
40.6
59.0
Cascade R-CNN(CCS (Cal & Vasconcelos 2018
ResNet-101
42.8
62.1
46.3
23.7
45.5
55.2
one-stage methods
YOLOv2 (Redmon al. 2016
Darknet-IB
21.6
44.0
19.2
5.0
35.5
YOLOv (Reumon Farhadt 2018
Darknet-53
33.0
57.9
34.4
18.3
35.4
41.9
SSD513 [uu of al. 2017
ResNet-101
31.2
50.4
33.3
10.2
34.5
49.8
DSSDS Fü cta 2017
ResNet-101
33.2
53.3
35.2
13.0
35.4
51.1
RetinaNet Lan
2018
ResNet-100 101
39.1
59.1
42.3
21.8
42.7
50.2
RetinaNet Lin al. 2018
ResNeXt-101
40.8
61.1
44.1
24.1
44.2
51.2
RPDet Yang a..
RcsNcXt-101
62.9
44.3
23.6
44.1
51.7
FCOS Iaa 2019
ResNeXt-101
42.1
62.1
45.2
25.6
44.9
52.0
CornerNet (Law Deng 2018
Hourglass- 104
40.5
56.5
43.1
19.4
42.7
53.9
ExtremeNet houu ct al. 20196
Hourglass- 104
40.1
55.3
43.2
20.3
43.2
53.1
CenterNet (Duan et al. 2019
Hourglass-100 104
42.1
61.1
45.9
24.1
45.5
52.8
0073
FoveaBox
ResNet-101
40.8
44.0
24.1
53.2
FoveaBox
ResNeXt-101
42.3
62.9
45.4
25.3
46.8
55.0
FoveaBox-align
ResNet-101
42.1
62.7
45.5
25.2
46.6
54.5
FuveaBox-align
ResNeXt-101
43.9
63.5
47.7
26.8
46.9
55.6
existence of target scene text the quadrilateral shapes. Guided- Ancloriinn (Wang et al., 2019) jointly predicts the locations where the center of objects likely to exist well as the scales and aspect ratios centered at the corresponding locations. Mhided-Anchoring still relies on predefined anchors optimize the object shape, and utilizes the center points give the best predictions. In contrast, FoveaBox predicts the (left, top, right, bottom) boundaries of the object for each foreground position.
Also there are contemporary works (Tian et al. 2019 Zhou et al. 2019a) similar the idea of FoveaBox. FCOS relies on the proposed centerness map for better learning of the instance. Instead, FoveaBox directly predict the final class probability without centerness voting, which more simple. The CenterNet (Zhou al.) (2019)) represent each instance by its features at the center point, which could also get comparable performance when adopting heavier networks (Law & Deng 2018). Compare with single point, our fovea-based positive sample definition process is more reasonable (The performance drops dramatically when we decrease σ). Again, we note that FoveaBox, CenterNet and FCOS are concurrent works.
Contemporary Works:
Bottom-up Methods: In CornerNet (Law & Deng 2018), the authors propose to detect an object bounding box as a pair of key-points, the top-left corner and the bottom-right corner. CornerNet adopts the Associative Embedding (Newell et al. 2017) technique to separate different instances. Also there are some following works bottom-up grouping manner (Zhou et al. [019b] Duan al. 2019). It should be noted that the bottom-up methods also do not need anchors during training and inference.
5 CONCLUSION
We have presented FoveaBox, a simple, effective, and completely anchor-free framework for generic object detection. By simultaneously predict the object position and the corresponding boundary, FoveaBox gives a clean solution for detecting objects without prior candidate boxes. We demonstrate its effectiveness standard benchmarks and report extensive experimental analysis. We believe the simple and effective approach will serve as a solid baseline and help ease future research for object detection.
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Mark F Bear, Barry W Connors, and Michael A Paradiso. Neuroscience, volume 2. Lippincott
Williams & Wilkins, 2007.
Navaneeth Bodla, Bharat Singh, Rama Chellappa, and Larry Davis. Sof--mms-improving ob- ject detection with line of code. In Proceedings of the IEEE International Conference Computer Vision, pp. 5561-5569, 2017.
Zhaowei Cai and Nuno Vasconcelos. Cascade r-cnn: Delving into high quality object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6154-6162, 2018.
Hong- Yunn Chen et al. Tensorflow- system for large-scale machine learning. In OSDI, volume 16, pp. 265-283,20 2016.
Hong-Yunn
Yuntao Chen, Chenxia Han, Naiyan Wang, and Zhaoxiang Zhang. Revisiting feature alignment for one-stage object detection. arXiv preprint
2019.
Jifeng Dai, Haozhi Qi, Yuwen Xiong, Li, Guodong Zhang, Han Hu, and Yichen Wei. Deformable convolutional networks. In Proceedings of the IEEE International Conference on Computer Vi- sion, pp. 764-773, 2017.
Kaiwen Duan, Song Bai, Lingxi Xie, Honggang Qi, Qingming Huang, and Qi Tian. Centernet: Keypoint triplets for object detection. arXiv preprint arXiv:1904.08/ 2019.
Pedro F Felzenszwalb, Ross B Girshick, and David McAllester. Cascade object detection with de- formable part models. In Computer vision and pattern recognition (CVPR), 2010 IEEE conference on, pp. 2241-22122222 -2248. IEEE, 2010.
Cheng- Yang Fu, Wei Liu, Ananth Ranga, Ambrish Tyagi, and Alexander C Berg. Dssd: Deconvo- lutional single shot detector. arXiv preprint (X0106600006655 2017.
Spyros Gidaris and Nikos Komodakis. Object detection via a multi-region and semantic segmentation- -wwree enn model. In Proceedings of the IEEE International Conference on Com- puter Vision, pp. 1134-1145 1142, 2015.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp- 770-778, 2016.
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Computer Vision (ICCV), 2017 IEEE International Conference on, pp. 2980-2988. IEEE, 2017.
Han Hu, Chengquan Zhang, Yuxuan Luo, Yuzhuo Wang. Junyu Han, and Errui Ding. Wordsup: Exploiting word annotations for character based text detection. In Proceedings of the IEEE Inter- national Conference on Computer Vision, pp. 4940-4949, 2017.
Han Hu, Jiayuan Gu, Zheng Zhang, Jifeng Dai, and Yichen Wei. Relation networks for object detection. In Computer Vision and Pattern Recognition (CVPR), volume 2, 2018.
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 4700-4708, 2017a.
Jonathan Huang, Vivek Rathod, Chen Sun, Menglong Zhu, Anoop Korattikara, Alireza Fathi, Ian Fischer, Zbigniew Wojna, Yang Song, Sergio Guadarrama, et Speed/accuracy trade-off's for modern convolutional object defectors. In IEEE CVPR, volume 4, 2017b.
Lichao Huang, Yi Yang, Yafeng Deng, and Yinan Yu. Densebox: Unifying landmark localization with end to end object detection. arXiv preprint arXiv:/509.0489 2015.
Masayuki Iwasaki and H Inomata. Relation between superficial capillaries and foveal structures in the human retina. Investigative ophthalmology & visual science, 17982):198811708 1986.

---

Under review as a conference paper at ICLR 2020
Borui Jiang, Ruixuan Luo, Jiayuan Mao, Tete Xiao, and Yuning Jiang. Acquisition of localization confidence for accurate object detection. In Proceedings of the European Conference on Com- puter Vision, Munich, Germany, pp. 8-14, 2018.
Christoph H Lampert. Matthew B Blaschko, and Thomas Hofmann. Beyond sliding windows: Object localization by efficient subwindow search. In 2008 IEEE conference on computer vision and pattern recognition, pp. 1-8. IEEE, 2008.
Hei Law and Jia Deng. Cornernet: Detecting objects paired keypoints. In Proceedings of the European Conference on Computer Vision (ECCV), volume 6, 2018.
Tsung-Yi Yi Lin, Piotr Dollár, Ross B Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In CVPR, volume 1, pp. 4, 2017.
Tsung- Yi Lin, Priyal Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. IEEE transactions on pattern analysis and machine intelligence, 2018.
Li Liu, Wanli Ouyang, Xiaogang Wang, Paul Fieguth, Jie Chen, Xinwang Liu, and Matti Pietikäinen. Deep learning for generic object detection: A survey. arXiv preprint arXiv: 1809.02 065000666666666 2018.
Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng- Yang Fu, and Alexander C Berg. Ssd: Single shot multibox detector. In European conference computer vision, pp. 21-37. Springer, 2016.
Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In Proceedings the IEEE conference on computer vision and pattern recognition, pp. 3431-3440, 2015.
Alejandro Newell, Zhiao Huang, and Jia Deng. Associative embedding: End-to-end learning for joint detection and grouping. In Advances in Neural Information Processing Systems, pp. 2277- 2287, 2017.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary Detioo, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
Joseph Redmon and Ali Farhadi. Yolo9000: better, faster, stronger. In Proceedings of the IEEE
conference on computer vision and pattern recognition, pp. 7263-7271, 2017.
Joseph Redmon and Ali Farhadi.
Yolov3: An incremental improvement. arXiv preprint
02X70 2018
Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 779-788, 2016.
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances in neural information processing systems, pp. 91-99, 2015.
Abhinav Shrivastava, Rahul Sukthankar, Jitendra Malik, and Abhinav Gupta. Beyond skip connec- tions: Top-down modulation for object detection. arXiv preprint 161iv:1612.06851, 2016.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recogmition. arXiv preprint arXiv:1409.1556 2014.
Zhi Tian, Chunhua Shen, Hao Chen, and Tong He. Feos: Fully convolutional one-stage object detection. arXiv preprint arXiv:1944.0133 2019.
Jiaqi Wang, Kai Chen, Shuo Yang, Chen Change Loy, and Dahua Lin. Region proposal by guided anchoring. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2965-2974, 2019.
10

---

Under review as a conference paper at ICLR 2020
Tong Yang, Xiangyu Zhang, Zeming Li, Wenqiang Zhang, and Jian Sun. Metaanchor: Learning to detect objects with customized anchors. In Advances in Neural Information Processing Systems, pp. 318-3288 2018.
Ze Yang. Shaohui Liu, Han Hu, Liwei Wang, and Stephen Lin. Reppoints: Point representation for object detection. arXiv preprint arXiv: 1490, 2019.
Zheng Zhang, Chengquan Zhang, Wei Shen, Cong Yao, Wenyu Liu, and Xiang Bai. Multi-oriented text detection with fully convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4159-4167, 2016.
Xingyi Zhou, Dequan Wang, and Philipp Krähenbühh. Objects as points. arXiv preprint arXiv: 1904v:9904.078500 2019a.
Xingyi Zhou, Jiacheng Zhuo, and Philipp Krahenbuhl. Bottom-up object detection by grouping extreme and center points. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pp. 850-859, 2019b.
Xinyu Zhou, Cong Yao, He Wen, Yuzhi Wang, Shuchang Zhou, Weiran He, and Jiajun Liang. East: an efficient and accurate scene text detector. In Proceedings the IEEE conference Computer Vision and Pattern Recognition, pp. 5551-5560, 2017.
A
APPENDIX
A.1 QUALITATIVE RESULTS
Fig shows the detection outputs of FoveaBox. Points and boxes with class probability larger than 0.5 are shown (before feeding into NMS). For each object, though there are several active points, the predicted boxes are very close to the ground-truth. These figures demonstrate that FoveaBox could directly generate accurate, robust box predictions, without the requirement of candidate anchors.
Figure 5: FoveaBox results on the COCO minival set. These results are based on ResNet- 101, achieving single model box AP of 38.6. For each pair, left is the detection results with bounding box, category, and confidence. Right is the score output map with their corresponding bounding boxes before feeding into non-maximum suppression (NMS). The score probability in each position is denoted by the color density.

---

Under review as a conference paper at ICLR 2020
A.2 PER-CLASS DIFFERENCE:
Fig. shows per-class AP difference of FoveaBox and RetinaNet, Both of them are with ResNet-50- FPN backbone and 800 input scale. The vertical axis shows AP Favea Box AP RetinaNet- FoveaBox shows improvement in most of the classes
Figure 6: AP difference of FoveaNet and RetinaNet on COCO dataset. Both models use ResNet- FPN-50 as backbone and 800 input scales.
A.3 MORE TRAINING TIME:
Table shows the detection performance under 2x setting (24 epochs training). FoveaBox also outperforms the anchor-based baseline method.
Table 5: 2x epochs training (800 scale).
method
AP
AP50
AP75
RetinaNet-50
36.4
56.3
39.4
RetinaNet- 101
38.1
57.8
41.1
FoveaBox-50
37.1
56.7
39.3
FoveaBox-
38.7
58.4
41.1
A.4 FEATURE ALIGNMENT
For feature alignment, we adopt a 3 deformable convolutional layer to implement the transforma- tion, shown in Fig 7 The offset input the deformable convolutional layer is (fr, fy, faz &yz) of the bbox output.
FPN e*
deformable convolution
Aligned feature
cls branch
HdW xC
cls autput
feature
HkW x256
HXW
box output
x296
box branch
Figure Feature alignment process.
12


Reviews:
[{"review_id": "rJlOCkwbqH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "## Overview\nThe paper tackles object detection without predefined anchors (sliding windows). \nThe paper is well motivated and existing detection methods often rely on anchors, which may limits their potential. So this paper is solving an interesting problem and seems novel.\nThe paper provides experiments to support the proposed architecture empirically.\n\n## Summary of the contribution:\n1. The paper proposed an object detection approach called FoveaBox that does not rely on anchors (sliding widows). \n2. The paper shows that FoveaBox outperforms some existing object detection methods.\n3. The paper shows that FoveaBox can also be used for object proposals by changing the classification target to class-agnostic head.\n\n## My feedback:\nI think the paper is well motivated and has value to the object detection community. So I am leaning positive to accept this paper. The major reasons are:\n1. The paper studies an interesting architecture which does not rely on sliding windows and shows its effectiveness. The problem seems not very well studied in the existing object detection literature.\n2. The paper provides sufficient ablation study to analyze and understand the proposed methods.\nHowever, I believe the paper could be significantly improved especially in the writing. So my position is not strong.\n\n## Improvements\n1. The writing can be significantly improved. The paper reads a bit confusing and unclear especially at the technical part. The paper could benefit from a clear overview figure about the proposed approach. Figure 4 seems to be doing that illustration but giving only the tensor shape seems quite confusing. I also listed some questions below about the actual algorithm.\n2. The paper could be improved with experiments and ablation on another dataset. Current ablation shows that \\eta should be 2.0 for best performance but without a second dataset it is hard to say this value is general. So one may have to tune this parameters in different datasets.\n3. The scale is still discretized. So that is essentially anchors in the scale space. I wonder how could the approach applies to scale to?\n\n## Questions\n1. How would the proposed approach address the issue when there are multiple bounding boxes around the same pixels? It seems the current approach is predicting a box per pixel?\n2. What is the inference computation cost? And do you have a comparison with existing methods on the compute cost?", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "rJlAJhYL9r", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper introduces foveabox, a method that performs \"keypoint\" like object detection -- instead of \"anchor\" based detection (to be discussed later). The idea is simple: predict class labels for pixels that fall within (a reduced version) the GT boxes of the instance; and predict bounding box offsets for those positive pixels. The idea is built on top of the FPN backbone, where a set of feature maps (each representing a specific scale) are used to detect object boxes in multiple scales.  The method is mainly compared against RetinaNet (which is \"anchor\" driven), and also compared against other more recent methods (ExtremeNet, CenterNet, FCOS) etc. \n\n+ The paper is quite well written and structured, the illustrations are also clear;\n+ I have also read its previous version, and the new version has added a significant amount of work improving it -- e.g. added feature alignment and group norm;\n+ I haven't fully checked the results section of other concurrent papers for full comparison, but the current results are among the state-of-the-art for one-stage detectors.\n\n- I don't think the paper breaks away from the notion of \"anchors\". The current approach is in fact implicitly defining a *single* anchor for each feature map, and changing the assignment rule from IoU based to distance/scale based. I firmly believe that such modification can lead to concrete improvements -- for example recent work has shown that changing assignment rule can improve AP even with fewer anchors (TensorMask); but to say that the paper breaks away from the usage of anchors, it is a bit far for me. I think I will be way more convinced if the experiments are done on a single-scale feature map (anchor also works with single-scale but I am not sure without defining different anchors it can work just on C4 for example).\n- Also because such assignment rule has been there before (e.g. DeepMask has used x, y, scale for assignment), and with other recent works (CenterNet, Objects as Keypoints), the contribution of this work is fairly limited\n\nTherefore, I vote for reject.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "N/A", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "H1e8TZVM9S", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper introduces an anchor-free object detection framework that aims at simultaneously predicting the object position and the corresponding boundary. To achieve this, the proposed FoveaBox detector predicts category-sensitive semantic maps for the object existing possibility, and  produces category-agnostic bounding box for each position that is likely to contain an object. The scales of target boxes are associated with feature pyramid representations. Experiments are performed on MS COCO detection benchmark.\n\nPros:\nThe proposed approach is simple and is shown to avoid most computation and hyper-parameters related to anchor boxes. The paper is well written and easy to follow. \nCons:\nThe main issue with the paper is the main idea is similar to [1,2, 3, 4, 5]. For instance, CenterNet also represents each object instance by its features at the center point and achieves similar detection performance compared to the proposed detector. Further, no speed comparison with these approaches is provided in the paper. Without a fair speed comparison and with similar detection performance, it is difficult to fully assess the merits of the proposed approach. Though inference speed comparison is reported with RetineNet. However, a proper and detailed comparison with [1, 2, 3, 4, 5] is missing. \n\n1: Xingyi Zhou, Dequan Wang, Philipp Krähenbühl: Objects as Points. CoRR abs/1904.07850 (2019).\n2: Xingyi Zhou, Jiacheng Zhuo, Philipp Krähenbühl: Bottom-Up Object Detection by Grouping Extreme and Center Points. CVPR 2019.\n3: Zhi Tian, Chunhua Shen, Hao Chen, Tong He: FCOS: Fully Convolutional One-Stage Object Detection. CoRR abs/1904.01355 (2019).\n4: Hei Law and Jia Deng: Cornernet: Detecting objects as paired keypoints. ECCV 2018.\n5: Kaiwen Duan, Song Bai, Lingxi Xie, Honggang Qi, Qingming Huang, Qi Tian: CenterNet: Keypoint Triplets for Object Detection. CoRR abs/1904.08189 (2019).\n\n", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a method for object detection by predicting category-specific object probability and category-agnostic bounding box coordinates for each position that's likely to contain an object. The proposed idea is interesting and the experimental results show improvement over RetinaNet and other baselines. However, in terms of weakness, (1) conceptually speaking it's unclear whether the proposed method is a big departure from the existing frameworks; and (2) although the authors are claiming SOTA performance, the proposed method seems to be worse than other existing/recent work. Some example references are listed below (more available here: https://paperswithcode.com/paper/foveabox-beyond-anchor-based-object-detector). 

[1] Scale-Aware Trident Networks for Object Detection
https://arxiv.org/abs/1901.01892

[2] GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond
https://arxiv.org/abs/1904.11492

[3] CBNet: A Novel Composite Backbone Network Architecture for Object Detection
https://arxiv.org/abs/1909.03625

[4] EfficientDet: Scalable and Efficient Object Detection
https://arxiv.org/abs/1911.09070

References [3] and [4] are concurrent works so shouldn't be a ground of rejection per se, but the performance gap is quite large. Compared to [1] and [2] which have been on arxiv for a while (+5 months) the performance of the proposed method is still inferior. Despite considering that object detection is a very competitive field, the conceptual/technical novelty and overall practical significance seem limited for ICLR. For a future submission, I would suggest that a revision of this paper being reviewed in a computer vision conference, rather than ML conference.

Decision: Reject