FoveaBox: Beyound Anchor-based Object Detection

We present FoveaBox , an accurate , flexible , and completely anchor-free framework for object detection . While almost all state-of-the-art object detectors utilize predefined anchors to enumerate possible locations , scales and aspect ratios for the search of the objects , their performance and generalization ability are also limited to the design of anchors . Instead , FoveaBox directly learns the object existing possibility and the bounding box coordinates without anchor reference . This is achieved by : ( a ) predicting category-sensitive semantic maps for the object existing possibility , and ( b ) producing category-agnostic bounding box for each position that potentially contains an object . The scales of target boxes are naturally associated with feature pyramid representations . We demonstrate its effectiveness on standard benchmarks and report extensive experimental analysis . Without bells and whistles , FoveaBox achieves state-of-the-art single model performance on the standard COCO detection benchmark . More importantly , FoveaBox avoids all computation and hyper-parameters related to anchor boxes , which are often sensitive to the final detection performance . We believe the simple and effective approach will serve as a solid baseline and help ease future research for object detection . 1 INTRODUCTION . Object detection requires the solution of two main tasks : recognition and localization . Given an arbitrary image , an object detection system needs to determine whether there are any instances of semantic objects from predefined categories and , if present , to return the spatial location and extent . To add the localization functionality to generic object detection systems , sliding window approaches have been the method of choice for many years ( Lampert et al. , 2008 ; Felzenszwalb et al. , 2010 ; Liu et al. , 2018 ) . Recently , deep learning techniques have emerged as powerful methods for learning feature representations automatically from data ( Simonyan & Zisserman , 2014 ; He et al. , 2016 ; Huang et al. , 2017a ) . For object detection , the anchor-based Region Proposal Networks ( Ren et al. , 2015 ) are widely used to serve as a common component for searching possible regions of interest for modern object detection frameworks ( Liu et al. , 2016 ; He et al. , 2017 ; Lin et al. , 2018 ) . In short , anchor method suggests dividing the box space into discrete bins and refining the object box in the corresponding bin . Most state-of-the-art detectors rely on anchors to enumerate the possible locations , scales , and aspect ratios for target objects ( Liu et al. , 2018 ) . Anchors are regression references and classification candidates to predict proposals for two-stage detectors or final bounding boxes for single-stage detectors . Nevertheless , anchors can be regarded as a feature-sharing sliding window scheme to cover the possible locations of objects . However , anchors must be carefully designed and used in object detection frameworks . ( a ) One of the most important factors in designing anchors is how densely it covers the instance location space . To achieve a good recall rate , anchors are carefully designed based on the statistics computed from the training/validation set ( Lin et al. , 2018 ) . ( b ) One design choice based on a particular dataset is not always applicable to other applications , which harms the generality ( Yang et al. , 2018 ) . ( c ) At training phase , anchor-methods rely on the intersection-over-union ( IoU ) to define the positive/negative samples , which introduces additional computation and hyper-parameters for an object detection system ( Wang et al. , 2019 ) . In contrast , our human vision system can recognize the instance in space and predict the boundary given the visual cortex map , without any pre-defined shape template ( Bear et al. , 2007 ) . In other words , we human naturally recognize the object in the visual scene without enumerating the candidate boxes . Inspired by this , an intuitive question to ask is , is the anchor scheme the optimal way to guide the search of objects ? And further , could we design an accurate object detection framework without anchors or candidate boxes ? Without anchors , one may expect a complex method is required to achieve comparable performance . However , we show that a surprisingly simple and flexible system can match , even surpass the prior state-of-the-art object detection results without any requirement of candidate boxes . To this end , we present FoveaBox , a completely anchor-free framework for object detection . FoveaBox is motivated from the fovea of human eyes : the center of the vision field is with the highest visual acuity ( Fig.2 left ) , which is necessary for activities where visual detail is of primary importance ( Iwasaki & Inomata , 1986 ) . FoveaBox jointly predicts the locations where the object ’ s center area is likely to exist as well as the bounding box at each valid location . In FoveaBox , each target object is predicted by category scores at center area , associated with 4-d bounding box , as shown in Fig.2 right . At training phase , we do not need to utilize anchors , or IoU matching to generate training target . Instead , the training target is directly generated by ground-truth boxes . To demonstrate the effectiveness of the proposed detection scheme , we combine the recent progress of feature pyramid networks and our detection head to form the framework of FoveaBox . Without bells and whistles , FoveaBox gets state-of-the-art single-model results on the COCO object detection task . Compared with the anchor-based RetinaNet , FoveaBox gets 2.2 AP gains , which also surpasses most of previously published anchor based single-model results . We believe the simple training/inference manner of FoveaBox , together with the flexibility and accuracy , will benefit future research on object detection and relevant topics . 2 FOVEABOX . FoveaBox is conceptually simple : It contains a backbone network and a fovea head network . The backbone is responsible for computing a convolutional feature map over an entire input image and is an off-the-shelf convolutional network . The fovea head is composed of two sub-branches , the first branch performs per pixel classification on the backbone ’ s output ; the second branch performs box prediction for each position that potentially covered by an object . 2.1 REVIEW OF FPN AND ANCHORS : . We begin by briefly reviewing the Feature Pyramid Network ( FPN ) used for object detection ( Lin et al. , 2017 ) . In general , FPN uses a top-down architecture with lateral connections to build an in-network feature pyramid from a single-scale input . FPN is independent of a specific task . For object detection , each level of the pyramid in FPN is used for detecting objects at a specific scale . On each feature pyramid , anchor-based methods uniformly place A anchors on each of the H ×W spacial position . After computing the IoU overlap between all anchors and the ground-truth boxes , the anchor-based methods can define training targets . Finally , the pyramid features are utilized to optimize the targets . 2.2 FOVEABOX . FoveaBox directly predicts the object existing possibility and the corresponding boundary for each position potential contained by an instance . In this section , we introduce the key components stepby-step . 2.2.1 OBJECT OCCURRENCE POSSIBILITY . Given a valid ground-truth box denoted as ( x1 , y1 , x2 , y2 ) . We first map the box into the target feature pyramid Pl x ′ 1 = x1 sl , y ′ 1 = y1 sl , x ′ 2 = x2 sl , y ′ 2 = y2 sl , c ′ x = 0.5 ( x ′ 2 + x ′ 1 ) , c ′ y = 0.5 ( y ′ 2 + y ′ 1 ) , w ′ = x ′ 2 − x ′ 1 , h ′ = y ′ 2 − y ′ 1 , ( 1 ) where sl is the down-sample factor . The positive area Rpos on the score map is designed to be roughly a shrunk version of the original one ( Fig.3 right ) : xpos1 = c ′ x − 0.5σw ′ , ypos1 = c ′ y − 0.5σh ′ , xpos2 = c ′ x + 0.5σw ′ , ypos2 = c ′ y + 0.5σh ′ , ( 2 ) where σ is the shrunk factor . At training phase , each cell inside the positive area is annotated with the corresponding target class label . The negative area is the whole feature map excluding area in Rpos . For predicting , each output set of pyramidal heat-map has C channels , where C is the number of categories , and is of size H ×W . Each channel is a binary mask indicating the possibility for a class , like FCNs in semantic segmentation ( Long et al. , 2015 ) . The positive area usually accounts for a small portion of the whole feature map , so we adopt Focal Loss ( Lin et al. , 2018 ) to train this branch . 2.2.2 SCALE ASSIGNMENT . While our goal is to predict the boundary of the target objects , directly predicting these numbers is not stable , due to the large scale variations of the objects . Instead , we divide the scales of objects into several bins , according to the number of feature pyramidal levels . Each pyramid has a basic scale rl ranging from 32 to 512 on pyramid levels P3 to P7 , respectively . The valid scale range of the target boxes for pyramid level l is computed as [ rl/η , rl · η ] , ( 3 ) where η is set empirically to control the scale range for each pyramid . Target objects not in the corresponding scale range are ignored during training . Note that an object may be detected by multiple pyramids of the networks , which is different from previous practice that maps objects to only one feature pyramid ( He et al. , 2017 ) . 2.2.3 BOX PREDICTION . Each ground-truth bounding box is specified in the way G = ( x1 , y1 , x2 , y2 ) . Starting from a positive point ( x , y ) in Rpos , FoveaBox directly computes the normalized offset between ( x , y ) and four boundaries : tx1 = log sl ( x+ 0.5 ) − x1 rl , ty1 = log sl ( y + 0.5 ) − y1 rl , tx2 = log x2 − sl ( x+ 0.5 ) rl , ty2 = log y2 − sl ( y + 0.5 ) rl . ( 4 ) This function first maps the coordinate ( x , y ) to the input image , then computes the normalized offset between the projected coordinate and G. Finally the targets are regularized with the log-space function . rl is the basic scale defined in section 2.2.2 . For simplicity , we adopt the widely used SmoothL1 loss ( Ren et al. , 2015 ) to train the box prediction Lbox . After targets being optimized , we can generate the box boundary for each cell ( x , y ) on the output feature maps1 . In box branch , each output set of pyramidal heatmap has 4 channels , for jointly prediction of ( tx1 , ty1 , tx2 , ty2 ) . 2.2.4 NETWORK ARCHITECTURE . To demonstrate the generality of our approach , we instantiate FoveaBox with multiple architectures . For clarity , we differentiate between : ( i ) the convolutional backbone architecture used for feature extraction over an entire image , and ( ii ) the network head for computing the final results . 1Eq . ( 4 ) and its inverse transformation can be easily implemented by an element-wise layer in modern deep learning frameworks ( Paszke et al. , 2017 ; Chen et al. , 2016 ) . Most of the experiments are based on the head architecture as shown in Fig.4 . We also utilize different head variants to further study the generality . More complex designs have the potential to improve performance but are not the focus of this work .

Summary: The paper introduces foveabox, a method that performs "keypoint" like object detection -- instead of "anchor" based detection (to be discussed later). The idea is simple: predict class labels for pixels that fall within (a reduced version) the GT boxes of the instance; and predict bounding box offsets for those positive pixels. The idea is built on top of the FPN backbone, where a set of feature maps (each representing a specific scale) are used to detect object boxes in multiple scales.  The method is mainly compared against RetinaNet (which is "anchor" driven), and also compared against other more recent methods (ExtremeNet, CenterNet, FCOS) etc.