Revisiting a single-stage method for face detection

arXiv:1902.01559v1 [cs.CV] 5 Feb 20191

Nguyen Van Quang1,2 , Hiromasa Fujihara2

Faculty of Information Sciences, Tohoku University, Sendai, Japan
2
Laboro.AI Inc., Tokyo, Japan

Abstract— Although accurate, two-stage face detectors usually require more inference time than single-stage detectors do.
This paper proposes a simple yet effective single-stage model
for real-time face detection with a prominently high accuracy.
We build our single-stage model on the top of the ResNet101 backbone and analyze some problems with the baseline
single-stage detector in order to design several strategies for
reducing the false positive rate. The design leverages the context
information from the deeper layers in order to increase recall
rate while maintaining a low false positive rate. In addition, we
reduce the detection time by an improved inference procedurefor decoding outputs faster. The inference time of a VGA(640×480) image was only approximately 26 ms with a Titan X
GPU. The effectiveness of our proposed method was evaluatedon several face detection benchmarks (Wider Face, AFW, Pascal
Face, and FDDB). The experiments show that our method
achieved competitive results on these popular datasets with afaster runtime than the current best two-stage practices.

I. INTRODUCTION
Face detection has attracted much attention as it constitutes
the fundamental step of many common face-related tasks.
Since the pioneering work of Viola-Jones [9], face detection
has progressively improved [15], [16], [19], but still relying
on laborious feature engineering to train the face detectors.
However, in challenging datasets such as the recently introduced Wider Face [6], these approaches show the nonrobustness to a wide range of facial variations. Recently,
the CNN-based methods have been increasingly adopted todeal with the facial variations and have surpassed the former
methods. Nevertheless, there is a trade-off between computational cost and accuracy in face detection. For instance, theHR detector [5] needs to forward an image pyramid into ashared CNN model which consumes more than 1 second for
accurate results.
Inherited from object detection, two main approaches
have been applied successfully to face detection, namely,
one-stage (single-stage) and two-stage methods. Two-stage
methods follow a common pipeline: (1) produce a set of
region proposals with their local features (or pixels), (2) passthem to the second network for classifying and regressing thebounding boxes of the detected faces. Though very accurate,
these systems require numerous intensive computations. For
example, Chenchen Zhu et al. [1] recently proposed a twostage detector which obtains the state-of-the-art performanceon face benchmarks but requires about 150ms to proceed animage of size 600×1000. On the other hand, single-stage
methods extract the feature maps of convolutional layers atseveral depths from a base network, in which each layer isassociated with a set of predefined anchors. Convolutions
are then performed on these feature maps for classification
and regression tasks directly. Although these methods significantly improve the runtime of object detection over two-stage

methods while delivering comparative performance, applying
single-shot detectors (SSD) [11] directly to face detection
does not yield the acceptable performance [1]. However, SFD[4] and SSH [3], the one-stage models proposed for facedetection using VGG-16 [13] as a backbone framework withseveral improvements, could obtain very competitive results,
outperforming even the top ResNet-based models.
Inspired by SSD, SFD and SSH, we build a simple singlestage model with the ResNet-101 backbone for face detection
as a baseline. After investigating the baseline model, we
attribute the high rate of false positives to the following
causes.
Lack of context information. First, small faces appear
to lack deterministic facial parts due to their low resolution. Moreover, the corresponding feature maps of small
faces have less context information in the shallower anchorassociated layers than in the deeper layers. Consequently,
small-scale faces are difficult to detect. According to [5],large receptive fields can increase the context information
for detecting small faces.
Very large receptive field. In contrast to the above,
excessively large receptive fields might provide redundant
information for detection which increases the false positive
rate. For a given stride, the the anchor-associated layers in
ResNet-101 based networks have larger receptive fields than
VGG-16 based networks do. In SFD, the anchor sizes andlayouts are designed reasonably such that the anchor area
matches the receptive field for each stride.
Shared feature maps for the classification and regression tasks in detection. In SSD, two conv layers are
performed directly on the extracted feature maps to do bothclassification and regression tasks. Therefore, the network
might not easily and immediately learn the mappings andbalance the losses in both tasks.
Dense anchors for each ground truth box, and foreach cell in the feature maps. The number of anchors
assigned to small faces was increased by the matchingstrategy proposed elsewhere [1], [4]. Moreover, there areseveral anchors of different aspect ratios per cell in SSD.
Although these strategies help increase the recall rate, it
probably also contributes the high false positive rate.
All of the above factors lessen the capacity of the ResNet
backbone in single-stage models. Herein, we propose several
simple but efficient strategies that reduce the false positive
rate of our single-stage model with a ResNet-101 base forface detection. Overall, reducing the false positives, especially those with high confidence scores, boosts the average
precision (AP) score of the model. After implementing thesestrategies, our method achieved the state-of-the-art resultson AFW, PASCAL face, FDDB, and also competitive results
on WIDER FACE at faster runtime speed than the current
best two-stage methods. We also improve the decoding

II. RELATED WORK

Extra layers

procedure to make the inference faster up to about 10% withthe speed of 38 fps for VGA images of size 640×480.

In the past decades, Viola and Jones [9] pioneered a
real-time face detector, using Haar-like features to train
a cascade of Adaboost models. After that, face detection
has continuously improved in various works from [15],[16], to deformable part models (DPM) [17]–[19]. Most ofthese methods train traditional classifiers using a selective
set of hand-crafted features. Recently, CNN-based methods
have significantly boosted the performance of face detection
over traditional methods which struggle to deal with facial
variations. These recent state-of-the-art face detectors can bedivided into two main approaches: one-stage detectors andtwo-stage detectors. We follow the first approach to design
our face detector.
B. Single-stage networks for face detection
Several works follow the design of one-stage detection
architecture, having achieved very remarkable results oncommon benchmarks for face detection. SFD [4] adopts the
VGG-16 but tiling and assigning the anchors more tightly
for small faces to increase the recall score. Mahyar Najibi
et al. [3] proposed a network called SSH which removes
the fully connected layers from the VGG-16 base network,and provides the context information for detection module byfusing several convolutional layers. To improve the detectionof occluded faces, Wang et al. [14] proposed FAN model
which employs the RetinaNet [10] using feature pyramid
integrated with the attention mechanism.
III. OUR PROPOSED NETWORK
A. The network architecture
Following the SFD design, we also build a single-stage
framework which is robust to face scales. We extract the feature maps of different convolutional layers from the network.
Each cell in the feature map from each layer is associatedwith one square anchor (aspect ratio of 1:1). The anchor
size from each extracted conv layer increases exponentiallywith the power of 2, ranging from 16 to 512 pixels. As
implemented in SFD, the strides of anchors are also kept
unchanged to guarantee that faces of different scales are
assigned to the equally adequate number of matched anchorsfrom the feature maps. The model consists of the feature
extractor, fusion modules and detection modules. The feature
extractor is constructed by several of following layers:
Base convolutional layers. We design a very deep CNN
network using ResNet-101 [12] which is well known for
producing highly representative features for extraction. All
the layers of ResNet-101 before the first fully connected
layers are retained as our feature extractor.
Extra convolutional layers. For detecting large faces, we
add the extra convolutional layers to the ResNet-101 basein order. The additional Conv 6 and Conv 7 blocks aresequential blocks of two 3×3 conv layers of N = 256 filters,
in which the latter layer with a stride of 2 is followed by
ReLU activation.

ResNet layers

A. Face detection

Conv7 block

Detection module

Conv6 block

Detection module

Conv5 block

Detection module

Conv4 block
Conv3 block
Conv2 block

Detection module
Detection module
Detection module

Conv layers

Fig. 1: The proposed model architecture.
Detection convolutional layers. For further detection,
we extract feature maps of several layers from base convolutional layers and extra convolutional layers as anchorassociated layers for further detection. These extracted layers(Conv2 3, Conv3 3, Conv4 3, Conv5 3, Conv6 2, andConv7 2) are the last conv layers in their corresponding
blocks. We convolve the selected layers with 1×1 conv layerof 256 filters to ensure that all the extracted layers have 256
channels if their number of channels differs from 256.
B. Strategies for reducing the false positives
1) Fusing features from the higher layer: Although the
ResNet-101 backbone provides the larger receptive field
for each detection convolutional layer than VGG-16 base,the shallower layers still lack context information for thedetection. Fusing the features from higher layers is thecommon practice to enlarge the receptive fields and increase
the context information for shallower layers. For instance,
RetinaNet [10] designed the feature pyramid architecture
which allows a layer to enjoy the flow of context informationfrom all higher layers in the hierarchy. For face detection,
the feature pyramid architecture might provide redundant
context features for small face detection, increasing the false
positive rate. In order to increase the context information of
the feature maps in the lower layers, we fuse the current
detection layer with only one consecutive higher detection
layer in the hierarchical order. Therefore, we upsample thehigher detection layer by factor of 2 and fuse it with thecurrent detection layer by element-wise addition as depicted
Figure 2(a). This strategy approximately doubles the receptive field of the corresponding layer approximately which
provides sufficient context information for detecting small
faces while reducing the risk of false positive detection.
2) Shared features for detection tasks: In SFD, the detection layers are directly convolved with classification andregression filters. This makes the network harder to learn
and optimize the loss of two tasks. To avoid this problem,
we separate the detection tasks into two branches. Prior tothe classification and regression layer, we add a sequential
block including several 3×3 conv layers of 256 filters foreach layer on each branch as implemented in RetinaNet.
For each anchor, we need to regress the 4 offsets related
to its coordinates and N confidence scores for classification

)!!!

Upsample x2
The fused layer(a) Fusion module
conv block

cls layerclassiﬁcation
output*+,-+.-./012-3+1454621(!!!

<=;->+920 ?@2-A/120482

'!!!
&!!!
%!!!
$!!!
#!!!
"!!!
!

!"# $ !"%&!"% $ !"'&!"' $ !"(&!"(&$)"!&

7+8.4928:2- 1:+;2conv block

reg layer

regression
output(b) Detection module

Fig. 2: Fusion and Detection modules.

(as we have only the background label and the face label,
we set N = 2). Therefore, we apply 3×3 convolutional
layers of 4 and 2 filters on the regression branch andclassification branch, respectively. Figure 2(b) describes thedetection module.
3) Anchor assigning strategy: In the training phase, weneed to assign a ground truth box of faces to an anchor
or a set of anchors. In [1], [4], they adopt a strategy formatching which anchors to a ground truth box by decreasing
the threshold of Jaccard overlap between a face boundingbox and anchors. This strategy significantly increases thenumber of anchors per a ground truth faces, especially thesmall faces. Although this strategy increases the recall rate
for small faces, it produces more false positives. Following
the same matching strategy as SFD, we increase the numberof anchors per small faces but maintaining the threshold ofJaccard overlap in the first step at 0.5. In the second step, at
most 4 anchors that most overlap the hard faces (i.e., have thehighest Jaccard overlap) are selected as the matched anchors.
The remaining anchors are labeled as negative.
C. Improved decoding strategy
We adopt an efficient decoding strategy during the inference although single-stage models like SSD enjoy very fast
detection speed. During the inference, transfering raw outputs of detection modules from the GPU to CPU also incurs
a time burden. For an image with the size of 640×640, we
need to transfer the output tensor with size of 37, 500 whichcorresponds to the number of generated anchors, in which
only few of the anchors are associated with the detected
faces. Hence, we compute the classification scores for all
the anchors, and keep only the feature maps corresponding toanchors with confidence scores greater than a threshold λ1 =0.1. Then, we regress the offsets for coordinates of boundingboxes corresponding to these anchors and transfer them to
the CPU for further steps. The overlapped bounding boxes
are filtered by non-maximum suppression with a thresholdof λ2 = 0.3. The improved strategy decreases the inferencetime by approximately 10%, improving the runtime to 36 fps
on VGA images.

Fig. 3: No. of false positives at different confidence scoresof our model and the baseline on the Wider Face val set.
IV. EXPERIMENTS AND RESULTS
A. Experimental setup
Training dataset. The Wider Face dataset [6] comprises
32,203 images of 393,703 annotated faces with widely
varying scales, light conditions, expressions, poses and occlusions. The dataset is divided into train set, validation set,
and test set (ratio 40:10:50). The validation and test sets
are split into three subsets with varying detection-difficultylevels: easy, medium, and hard. We trained our model on
the Wider Face train set, and evaluated on the Wider Facevalidation and test sets. Following [4], we performed color
jittering and selected a random square crop with a size ratio
from 0.3 to 1.0 (relative to the original image’s shortest side).
The cropped patch was resized into to 640×640, along withrandom horizontal flip.
Loss function. The multi-task loss was formulated asfollows:
 1
 1
Lcls (pi , p∗i ) + λLreg (ti , t∗i ) (1)
L=
N
N
cls
reg
i
i

where i is the index of an anchor and pi denotes the predictedprobability that anchor i is a face, ti denotes the vector of
predicted offsets for anchor i while p∗i , t∗i are the groundtruth label and offsets, respectively.
We computed the cross entropy loss for the classification
task Lcls (pi , p∗i ) over two classes (background and face),and divided it by the number of anchors Ncls taken accountinto the loss. During the training process, the number ratio ofnegative examples (background) to positive examples (faces)
was maintained at 3 : 1 via online negative hard mining
procedure. The regression loss Lreg is a smooth L1 loss,
and Nreg is the number of anchors assigned to the groundtruth (p∗i = 1). We set λ = 4 as to optimally balance the twolosses.
Training settings. We train our model with the ResNet
module pre-trained on Imagenet dataset [12] with the batch
size of 6 on 1 GPU. SGD with momentum of 0.9 and weightdecay of 1e−4 is used as our optimizer. The model wastrained in 300 full epochs with the initial learning rate at
1e−3 which was divided by 10 when the multi-task loss
L plateaued. Meanwhile, the ResNet-101 based single-stage
baseline was constructed with the same feature extractor as
our model and the same training settings but not applied
some proposed strategies. The details of the baseline will bepresented in the supplementary part.

(a) Precision-recall curves on AFW(b) Precision-recall curves on Pascal Face(c) Discontinuous ROC curves on FDDB

Fig. 4: Results on common face detection benchmarks
TABLE I: The AP scores of different methods on the Wider
Face Test and Validation set
MethodFaceness [32]
LDCF+ [30]MT-CNN [28]CMS-RCNN [26]
ScaleFace [29]HR [5]SFD [4]
SSH [3]Detector in [1]
Ours

Easy
71.1 (71.3)
79.7 (79.0)
85.1 (84.8)
90.2 (89.9)
86.7 (86.8)
92.3 (92.5)
93.5 (93.7)
92.7 (93.1)
94.9 (94.9)
94.0 (94.2)

AP Test (Val)%
MediumHard
60.4 (63.4)
31.5 (34.5)
77.2 (76.9)
56.4 (52.2)
82.0 (82.5)
60.7 (59.8)
87.4 (87.4)
64.3 (62.4)
86.6 (86.7)
76.2 (77.2)
91.0 (91.0)
81.9 (80.6)
92.1 (92.5)
85.8 (85.9)
91.5 (92.1)
84.4 (84.5)
93.5 (93.3)
86.5 (86.1)
92.8 (92.8)
85.9 (86.1)

B. Results on Common benchmarks
Wider Face To demonstrate the effectiveness of our proposal, Figure 3 compares the false positves of our proposedmodel and the ResNet baseline. The detection result was divided into three subsets with different image size thresholds.
Our model is evaluated on the Wider Face validation andtest sets following the standard evaluation protocol. Table
I compares the average precision scores (AP) obtained byour model and several face detectors in [1], SFD [4], SSH[3], ScaleFace [29], HR [5], CMS-RCNN [26], Multitask
Cascade CNN [28], LDCF+ [30], and Faceness [32] on
Wider Face test and validation sets. Among the comparedmethods, our detector achieved the second best AP scoreson all face cases.
To show the consistency of our model’s performance, we
evaluated the model on other common dataset benchmarks
FDDB [7], AFW [8], and Pascal Faces [2]. Following [3], we
resize one side of images to 640 and keep the image aspectratio unchanged. Interestingly, our model achieved state-ofthe-art results on small resized images of these datasets
without constructing an image pyramid.
AFW The dataset comprises 205 images in which 473
faces are annotated. We compared our model with several
popular methods [17], [19], [24], [25] and commercial
face detectors (Face.com, Picasa and Face++). Our model
achieved a state-of-the-art result with an AP score of 99.92%.
Figure 4(a) depicts the comparisons.
Pascal Face This dataset comprises 851 images in which
1,335 faces are annotated. We compared our model withsome popular methods [17], [19], [24] and commercial facedetectors (Picasa, Face++). Again, our model outperformed
the other detectors by a large margin with an AP score of

Fig. 5: Inference times versus AP scores of our method andvarious methods on Wider Face Validation Hard set.
99.37% (Figure 4(b)).
FDDB This dataset comprises of 2,845 images in which
5,171 faces are annotated in elliptic bounding boxes. Rather
than learning an elliptic regressor, we directly regressed therectangular boxes for faces, and compared our method withother methods [5], [17], [27]–[44] which use no additional
self-annotations. Our detector also achieves the top AP scoreof 97.9% (depicted in Figure 4(c)).
C. Runtime analysis
Finally, we evaluated the inference time of our model. Following [1], we recorded the AP scores and the corresponding
inference times of variously sized images with the same
hardware configuration (a single NVIDIA Titan X GPU witha batch size of 1). Figure 5 shows that our model achievedvery fast inference time with competitive accuracy comparedwith the detector in [1]. In fact, the AP-versus-runtime curveof our model enveloped those of the others except that of[1]. On the Wider Face Validation hard set, our detector
forwarded one image within 36 ms and achieved 75.0% APscore. Meanwhile, the detector in [1] needs more than 150ms
to process an image to obtain 75.7% AP score. Our speed
advantage is mainly gained from the single-stage design with
one predefined anchor per cell whereas the detector in [1]adopts a two-stage design with several predefined anchors
per cell.
V. CONCLUSIONSThis work introduced a simple single-stage model forface detection which reasonably handles the speed-accuracy
trade-off. Understanding the causes of the high false positive
rate in single-stage methods was important for designingan effective, accuracy face detection framework. With SSD
fashion, our model uses ResNet-101 as the feature extractor
and incorporates several effective strategies that lower therate of false positive together with fast inference time. Ourmodel consistently achieved competitive results on common
benchmarks for face detection with superiorly real-timeinference.
R EFERENCES[1] Chenchen Zhu, Ran Tao, Khoa Luu, Marios Savvides. Seeing Small
Faces from Robust Anchor’s Perspective. In Conference on Computer
Vision and Pattern Recognition (CVPR), 2018.
[2] B. Yang, J. Yan, Z. Lei, and S. Z. Li. Fine-grained evaluation on facedetection in the wild. In Automatic Face and Gesture Recognition(FG), 11th IEEE International Conference on. IEEE, 2015.
[3] M. Najibi, P. Samangouei, R. Chellappa, and L. S. Davis. Ssh: Single
stage headless face detector. In The IEEE International Conference on
Computer Vision (ICCV), Oct 2017.
[4] S. Zhang, X. Zhu, Z. Lei, H. Shi, X. Wang, and S. Z. Li. S3fd:
Single shot scale-invariant face detector. In The IEEE International
Conference on Computer Vision (ICCV), Oct 2017.
[5] P.Hu and D.Ramanan. Finding tiny faces. In Proceedings of the IEEE
Conference Computer Vision and Pattern Recognition (CVPR), 2017.
[6] S. Yang, P. Luo, C.-C. Loy, and X. Tang. Wider face: A face detection
benchmark. In Proceedings of the IEEE Conference on Computer
Vision and Pattern Recognition, pages 5525-5533, 2016.
[7] V. Jain and E. Learned-Miller. Fddb: A benchmark for face detection in unconstrained settings. Technical Report UM- CS-2010-009,
University of Massachusetts, Amherst, 2010.
[8] P. M. R. M. Koestinger, P. Wohlhart and H. Bischof. Annotated facial
landmarks in the wild: A large-scale, real-world database for facial
landmark localization. 2011.
[9] P. Viola and M. J. Jones. Robust real-time face detection. International
Journal of Computer Vision (IJCV), 57(2):137-154, 2004.
[10] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollar. Focal loss for
dense object detection. In Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition (CVPR), pages 2980-2988,
2017.
[11] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.Y. Fu, and A.
C. Berg. Ssd: Single shot multibox detector. In European conferenceon computer vision, pages 21-37. Springer, 2016.
[12] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning forimage recognition. In Proceedings of the IEEE conference on computer
vision and pattern recognition (CVPR), pages 770-778, 2016.
[13] K. Simonyan and A. Zisserman. Very deep convolutional networks forlarge-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
[14] Jianfeng Wang, Ye Yuan, Gang Yu. Face Attention Network: An
Effective Face Detector for the Occluded Faces. arXiv preprint
arXiv:1711.07246, 2017.
[15] P. Felzenszwalb, R. Girshick, and D. McAllester. Cascade object
detection with deformable part models. In Proceedings of the IEEE
Conference on Computer Vision and Pattern Recognition (CVPR),
2010.
[16] X. Yu, J. Huang, S. Zhang, W. Yan, and D. Metaxas. Pose-free facial
landmark fitting via optimized part mixtures and cascaded deformableshape model. In Proceedings of the IEEE International Conference on
Computer Vision, pages 1944-1951, 2013.
[17] M. Mathias, R. Benenson, M. Pedersoli, and L. Van Gool. Face
detection without bells and whistles. In Computer Vision-ECCV, pages
720-735, 2014.
[18] J. Yan, Z. Lei, L. Wen, and S. Z. Li. The fastest deformable part modelfor object detection. In Computer Vision and Pattern Recognition(CVPR), pages 2497-2504, 2014.
[19] X. Zhu and D. Ramanan. Face detection, pose estimation, andlandmark localization in the wild. In Computer Vision and Pattern
Recognition (CVPR), 2012 IEEE Conference on pages 28792886,
2012.
[20] M. Pham, Y. Gao, V. Hoang, and T. Cham. Fast polygonal integrationand its application in extending haarlike features to improve object
detection. In Conference on Computer Vision and Pattern Recognition(CVPR), 2010.
[21] B. Yang, J. Yan, Z. Lei, and S. Z. Li. Aggregate channel features
for multi-view face detection. In International Joint Conference on
Biometrics (IJCB), 2014.
[22] Q. Zhu, M.-C. Yeh, K.-T. Cheng, and S. Avidan. Fast human detection
using a cascade of histograms of oriented gradients. In Conference on
Computer Vision and Pattern Recognition (CVPR), 2006.

[23] R. Ranjan, V. M. Patel, and R. Chellappa. A deep pyramid deformablepart model for face detection. In International Conference on Biometrics Theory, Applications and Systems (BTAS), 2015.
[24] J. Yan, X. Zhang, Z. Lei, and S. Z. Li. Face detection by structuralmodels. Image and Vision Computing, 32(10):790-799, 2014.
[25] X. Shen, Z. Lin, J. Brandt, and Y. Wu. Detecting and aligning faces byimage retrieval. In Proceedings of the IEEE Conference on Computer
Vision and Pattern Recognition, pages 3460-3467, 2013.
[26] C. Zhu, Y. Zheng, K. Luu, and M. Savvides. Cms-rcnn: contextual
multi-scale region-based cnn for unconstrained face detection. In Deep
Learning for Biometrics, pages 57-79. Springer, 2017.
[27] X. Sun, P. Wu, and S. C. Hoi. Face detection using deep learning:
An improved faster rcnn approach. arXiv preprint arXiv:1701.08289,
2017.
[28] K. Zhang, Z. Zhang, Z. Li, and Y. Qiao. Joint face detection andalignment using multitask cascaded convolutional networks. IEEE
Signal Processing Letters, 23(10):1499-1503, 2016.
[29] S. Yang, Y. Xiong, C. C. Loy, and X. Tang. Face detection
through scale-friendly deep convolutional networks. arXiv preprint
arXiv:1706.02863, 2017.
[30] E. Ohn-Bar and M. M. Trivedi. To boost or not to boost? on the limitsof boosted trees for object detection. In Pattern Recognition (ICPR),
2016 23rd International Conference on, pages 3350-3355. IEEE, 2016.
[31] R. Ranjan, V. M. Patel, and R. Chellappa. A deep pyramid deformablepart model for face detection. In Biometrics Theory, Applications andSystems (BTAS), 2015 IEEE 7th International Conference on, pages
1-8. IEEE, 2015.
[32] S. Yang, P. Luo, C.-C. Loy, and X. Tang. From facial parts responses toface detection: A deep learning approach. In Proceedings of the IEEE
International Conference on Computer Vision, pages 3676-3684, 2015.
[33] Y. Li, B. Sun, T. Wu, and Y. Wang. face detection with endto-end
integration of a convnet and a 3d model. In European Conference on
Computer Vision, pages 420-436. Springer, 2016.
[34] R. Ranjan, V. M. Patel, and R. Chellappa. Hyperface: A deep multitask learning framework for face detection, landmark localization, pose
estimation, and gender recognition. arXiv preprint arXiv:1603.01249,
2016.
[35] D. Triantafyllidou and A. Tefas. A fast deep convolutional neuralnetwork for face detection in big visual data. In INNS Conference
on Big Data, pages 61-70. Springer, 2016.
[36] D. Chen, S. Ren, Y. Wei, X. Cao, and J. Sun. Joint cascade facedetection and alignment. In Computer Vision-ECCV 2014, pages 109122. Springer, 2014.
[37] B. Yang, J. Yan, Z. Lei, and S. Z. Li. Convolutional channel features.
In Proceedings of the IEEE International Conference on Computer
Vision, pages 82-90, 2015.
[38] H. Li, Z. Lin, X. Shen, J. Brandt, and G. Hua. A convolutional
neural network cascade for face detection. In Proceedings of the IEEE
Conference on Computer Vision and Pattern Recognition (CVPR),
pages 5325-5334, 2015.
[39] B. Yang, J. Yan, Z. Lei, and S. Z. Li. Aggregate channel features
for multi-view face detection. In Biometrics (IJCB), 2014 IEEE
International Joint Conference on, pages 1-8. IEEE, 2014.
[40] G. Ghiasi and C. C. Fowlkes. Occlusion coherence: Detecting andlocalizing occluded faces. arXiv preprint arXiv:1506.08347, 2015.
[41] H. Li, Z. Lin, J. Brandt, X. Shen, and G. Hua. Efficient boosted
exemplar-based face detection. In Proceedings of the IEEE Conference
on Computer Vision and Pattern Recognition (CVPR), pages 18431850, 2014.
[42] S. S. Farfade, M. J. Saberian, and L.-J. Li. Multi-view face detection
using deep convolutional neural networks. In Proceedings of the 5th
ACM on International Conference on Multimedia Retrieval, pages
643-650. ACM, 2015.
[43] E. Zhou, H. Fan, Z. Cao, Y. Jiang, and Q. Yin. Extensive facial landmark localization with coarse-to-fine convolutional network cascade.
In Proceedings of the IEEE International Conference on Computer
Vision Workshops, pages 386-391, 2013.
[44] H. Li, G. Hua, Z. Lin, J. Brandt, and J. Yang. Probabilistic elastic partmodel for unsupervised face detector adaptation. In Proceedings of theIEEE International Conference on Computer Vision, pages 793-800,
2013.
[45] N. Markus, M. Frljak, I. S. Pandzic, J. Ahlberg, and R. Forchheimer.
A method for object detection based on pixel intensity comparisons.
In 2nd Croatian Computer Vision Workshop, 2013.

FG 2019
804
805
806
807
808
809
810
811
812
813
814
815
816

FG 2019

FG 2019 Submission. CONFIDENTIAL REVIEW COPY. DO NOT DISTRIBUTE.

#****

#****

VI. S UPPLEMENTARY M ATERIALS
To avoid cluttering, we describe some details of ourbaseline here. The feature extractor of our baseline has thesame architecture like that of our proposed model with theconvolutional layers (ResNet-101), and extra convolutional
layers. The detection layers extracted from the feature extractor are convolved directly with classification and regression
conv layers. The anchor assigning and training on the model
1
1
are0.9 followed that of the single shot detector 0.9(SSD).

1
0.9

0.8

B. 0.8Precision-recall curves on Wider Face dataset
0.7
0.7

0.8
0.7

0.8

Precision

Precision

FG 2019
885

#****
886

1

(c) Hard

0.8

0.8

0.7

0.7

0.6

0.6

0.5

Zhu et al.-0.949
Ours-0.942
SFD-0.937
SSH-0.931
CMS-RCNN-0.899
ScaleFace-0.868
Multitask Cascade CNN-0.848
LDCF+-0.790
Faceness-WIDER-0.713
Multiscale Cascade CNN-0.691

0.4
0.3
0.2
0.1
0

0

0.2

0.4

0.6

Recall

0.8

1
0.9
0.8
0.7

0.5

Zhu et al.-0.933
Ours-0.928
SFD-0.925
SSH-0.921
CMS-RCNN-0.874
ScaleFace-0.867
Multitask Cascade CNN-0.825
LDCF+-0.769
Multiscale Cascade CNN-0.664
Faceness-WIDER-0.634

0.4
0.3
0.2
0.1
0

1

Precision1
0.9

Precision

Precision

1
0.9

0

0.2

(a) Easy

0.4

Recall

0.6

0.8

0.6
0.5

Ours-0.861
Zhu et al.-0.861
SFD-0.859
SSH-0.845
ScaleFace-0.772
CMS-RCNN-0.624
Multitask Cascade CNN-0.598
LDCF+-0.522
Multiscale Cascade CNN-0.424
Faceness-WIDER-0.345

0.4
0.3
0.2
0.1
0

1

0

0.2

0.4

(b) Medium

Recall

0.6

0.8

1

(c) Hard

Fig. 6: Precision and recall curves on Wider Face Validation set with Easy, Medium, and Hard levels
0.9

0.8

0.8

0.8

0.7

0.7

0.7

0.6

0.6

Zhu et al.-0.949
Ours-0.940
SFD-0.935
SSH-0.927
HR-0.923
CMS-RCNN-0.902
ScaleFace-0.867
Multitask Cascade CNN-0.851
LDCF+-0.797
Faceness-WIDER-0.716
Two-stage CNN-0.657

0.5
0.4
0.3
0.2
0.1
0

0

0.2

0.4

Recall0.6

0.8

Zhu et al.-0.935
Ours-0.928
SFD-0.921
SSH-0.915
HR-0.910
CMS-RCNN-0.874
ScaleFace-0.866
Multitask Cascade CNN-0.820
LDCF+-0.772
Faceness-WIDER-0.604
Two-stage CNN-0.589

0.5
0.4
0.3
0.2
0.1
0

1

Precision1
0.9

Precision

1
0.9

0

0.2

(a) Easy

0.4

0.6

Recall0.8

0.6
Zhu et al.-0.865
Ours-0.859
SFD-0.858
SSH-0.844
HR-0.819
ScaleFace-0.764
CMS-RCNN-0.643
Multitask Cascade CNN-0.607
LDCF+-0.564
Faceness-WIDER-0.315
Two-stage CNN-0.304

0.5
0.4
0.3
0.2
0.1
0

1

0

0.2

(b) Medium

0.4

Recall

0.6

0.8

1

(c) Hard

Fig. 7: Precision and recall curves on Wider Face Test set with Easy, Medium, and Hard levels

1

1

0.9

0.9

0.9

0.8

0.8

0.8

0.7

0.7

0.7

0.6

0.6

Zhu et al.-0.949
Ours-0.940
SFD-0.935
SSH-0.927
HR-0.923
CMS-RCNN-0.902
ScaleFace-0.867
Multitask Cascade CNN-0.851
LDCF+-0.797
Faceness-WIDER-0.716
Two-stage CNN-0.657

0.5
0.4
0.3
0.2
0.1
0

0

0.2

0.4

Recall0.6(a) Easy

0.8

Zhu et al.-0.935
Ours-0.928
SFD-0.921
SSH-0.915
HR-0.910
CMS-RCNN-0.874
ScaleFace-0.866
Multitask Cascade CNN-0.820
LDCF+-0.772
Faceness-WIDER-0.604
Two-stage CNN-0.589

0.5
0.4
0.3

0.1

1

0

0.2

0.4

Recall0.6(b) Medium

0.8

0.6
Zhu et al.-0.865
Ours-0.859
SFD-0.858
SSH-0.844
HR-0.819
ScaleFace-0.764
CMS-RCNN-0.643
Multitask Cascade CNN-0.607
LDCF+-0.564
Faceness-WIDER-0.315
Two-stage CNN-0.304

0.5
0.4
0.3

7

0.2

0

Precision

852
836
853
837
854
838
855
839
856
840
857
841
858
842
859
843
860
844
861
845
862
846
863
847
864
848
865
849
866
850
867
851
868
852

884

Ours-0.861
Zhu et al.-0.861
SFD-0.859
SSH-0.845
ScaleFace-0.772
CMS-RCNN-0.624
Multitask Cascade CNN-0.598
LDCF+-0.522
Multiscale Cascade CNN-0.424
Faceness-WIDER-0.345(b) Medium

Precision

845
829
846
830
847
831
848
832
849
833
850
834
851
835

Zhu et al.-0.933
Ours-0.928
SFD-0.925
SSH-0.921
CMS-RCNN-0.874
ScaleFace-0.867
Multitask Cascade CNN-0.825
LDCF+-0.769
Multiscale Cascade CNN-0.664
Faceness-WIDER-0.634(a) Easy

Precision

828
812
829
813
830
814
831
815
832
816
833
817
834
818
835
819
836
820
837
821
838
822
839
823
840
824
841
825
842
826
843
827
844
828

Zhu et al.-0.949
Ours-0.942
SFD-0.937
SSH-0.931
CMS-RCNN-0.899
ScaleFace-0.868
Multitask Cascade CNN-0.848
LDCF+-0.790
Faceness-WIDER-0.713
Multiscale Cascade CNN-0.691

C. Some qualitative results
Fig. 6: Precision and recall curves on Wider Face Validation set with Easy, Medium, and Hard levels
In this section, we show some qualitative results on several
benchmark datasets (see in the next pages) in Figure VI.6
and Figure VI.7. We denote the red bounding boxes as thedetected faces by our models. Best view the results in color.

Precision

825
809
826
810
827
811

Precision

0.6
0.6
0.6
In
this section, we present the precision-recall
curves on
0.5
0.5
817
Wider Face validation and test set which are0.5omitted in our
0.4
0.4
0.4
FG
2019
818
paper.
The results were obtained
using
the standard
evalua- REVIEW COPY. DO NOT DISTRIBUTE.
0.3
FG 2019
Submission.
CONFIDENTIAL
0.3
0.3
#****
819
tion
server (test
0.2
0.2
0.2 protocol (validation set) and from the test
0.1
0.1
820
0.1 The precision-recall curves on Wider Face
804
set).
validation
0
0
821
805
0
0.2
0.4
0.6
0.8
1
set 0isin 0.4Figure0.6 6 whilethe1 precision-recall
curves
on
0
0.2
0.4
0.6
0 shown
0.2
0.8
RecallRecallRecall822
806
Wider Face test set is shown in Figure 7.

823
807
824
808

871
872
873
874
875
876
877
878
879
880
881
882
883

A. The details of our baseline

0.2
0.1
1

0

0

0.2

0.4

Recall0.6(c) Hard

0.8

1

887
871
888
872
889
873
890
874
891
875
892
876
893
877
894
878
895
879
896
880
897
881
898
882
899
883
900
884
901
885
902
886
903
887
904
888
905
889
906
890
907
891
908
892
909
893
910
894
911
895
912
896
913
897
914
898
915
899
916
900
917
901
918
902
919
903
920
904
921
905
922
906
923
907
924
908
925
909
926
910
927
911
928
912
929
913
930
914
931
915
932
916
933
917
934
918
935
919

Fig. VI.6: Several qualitative results on several datasets. Best view in color.

Fig. VI.7: Several qualitative results on several datasets. Best view in color.