Paper:
R-FCN: Object Detection via Region-based Fully Convolutional Networks
We present region-based, fully convolutional networks for accurate and efficient object detection. In contrast to previous region-based detectors such as Fast/Faster R-CNN [7,19] that apply a costly per-region subnetwork hundreds of times, our region-based detector is fully convolutional with almost all computation shared on the entire image. To achieve this goal, we propose position-sensitive score maps to address a dilemma between translation-invariance in image classification and translation-variance in object detection. Our method can thus naturally adopt fully convolutional image classifier backbones, such as the latest Residual Networks (ResNets) [10], for object detection. We show competitive results on the PASCAL VOC datasets (e.g., 83.6% mAP on the 2007 set) with the 101-layer ResNet. Meanwhile, our result is achieved at a test-time speed of 170ms per image, 2.5-20× faster than the Faster R-CNN counterpart. Code is made publicly available at: https://github.com/daijifeng001/r-fcn.

Review:
This paper is an important extension of recent seminal object detection work Faster R-CNN proposed by Ren et al [18]. The main contribution of the paper is to redesign Faster R-CNN into a fully convolutional variant R-FCN through introducing so called position-sensitive score maps. Benefiting from the property of fully convolutional architecture, the proposed method can not only bring about 2.7 & 2.5X speeding up in training & testing, but also can obtain on par performance with standard Faster R-CNN method. Extensive experimental results are also provided to show its effectiveness and to analyze various choices regarding the network design, training and testing.
Faster R-CNN [18] is one of the latest top-performing object detection methods on several popular benchmark datasets such as PASCAL VOC, Microsoft COCO and ImageNet. The main novelty of this paper is to redesign Faster R-CNN method into a fully convolutional variant R-FCN through introducing so called position-sensitive score maps. Note that the basic idea of position-sensitive score maps is extended from instance-sensitive score maps proposed by Dai et al. [3] recently. However by properly combining it with Faster R-CNN [18] and powerful ResNet [9], the proposed R-FCN method can not only bring about 2.7X & 2.5X speeding up in training & testing, but also can obtain on par performance with standard Faster R-CNN method. Extensive experiments are conducted for evaluation and analysis.  Overall, this paper is a good submission. But it can be further improved if the following concerns can be addressed.  - Obviously, the performance improvement of R-FCN is not just from position-sensitive score maps. The modification of ResNet with “hole algorithm” [2, 15] and the improved training with OHEM [22] also play important role. In Table 4, 5, 6 and two additional Tables (without captions) on page 8, it is important to clearly clarify that OHEM is used or not used in R-FCN. According to the results shown in Table 3, I think OHEM should be used for all results of R-FCN in above mentioned Tables. This is also important for fair comparisons with standard Faster R-CNN and Faster R-CNN +++ since they do not use OHEM. - As the authors describe that Faster R-CNN +++ further incorporates iterative BBR, global context and multi-scale testing, yet R-FCN incorporates “hole algorithm” and OHEM (and also multi-scale testing on Microsoft COCO dataset), how about object detection accuracy of R-FCN when using the completely same settings regarding its best results or all tricks, e.g., adding iterative BBR and global context. This is important to show the best potential of R-FCN. Will this provide best results on PASCAL VOC and Microsoft COCO datasets? - In line 250 and 251, the authors claim that R-FCN performs better on the objects of small sizes (in Microsoft COCO dataset) than standard Faster R-CNN. But according to Table 6, two methods show very similar performance, i.e., ~0.5% accuracy difference with respect to mAPs, does this mean that R-FCN performs less better/similar on the objects of middle sizes or large sizes? If yes, this is somewhat confusing because larger object instances usually activate more in feature maps compared with smaller object instances.  - Note that R-FCN still adopts 4-step (even 5-steps alternating training strategy on Microsoft COCO dataset) proposed in standard Faster R-CNN, it would be interesting if the authors can provide a single-run training scheme and make it more straightforward to train the detector, as like YOLO [17] and SSD [14]. - A minor typo should be corrected. According to Table 6, in line 252, “51.5%/29.2%”->”51.9%/29.9%”. - One related work (Gidaris and Komodakis, Object detection via a multi-region & semantic segmentation-aware cnn model, in ICCV 2015), in which the use of multi-region contextual information was first explored, should be cited. - The authors provide detailed info on training and testing, it will be better if the entire training time can be given for each benchmark dataset.