# End-to-End Object Detection with Enhanced Positive Sample Filter
## 3 Method
### Overall Pipeline  
The overall detection pipeline is presented in Figure 1. Following FCOS [3], we first constructed a Feature Pyramid Network (FPN) [36] on a backbone network (ResNet50 [37]) to generate multi-scale feature maps. We denote feature maps from five pyramid levels as \(P_{3}\), \(P_{4}\), \(P_{5}\), \(P_{6}\), \(P_{7}\), which have strides 8, 16, 32, 64, 128, respectively, and will be passed to a shared head for final detection. All feature maps from different pyramid levels have 256 channels. The shared head has three branches: a regression branch, an objectiveness branch and a classification branch. The regression and classification branches remain the same as in FCOS [3]. To provide credible and robust representations of objects, we constructed another binary classification branch called the objectiveness branch, which aimed to identify whether the prediction at each location belonged to foreground or background objects. All three branches were constructed with four convolution layers as well as another convolution layer for final outputs. The regression branch produces a 4-\(d\) vector \(t_{x,y}=(l,t,r,b)\) for each location (\(x\),\(y\)) involved with a foreground sample, which encodes distances of the current location to four sides of corresponding GT box. With the predictions, we can obtain the coordinates of the predicted bounding boxes. The objectiveness branch generates a 1-\(d\) vector for each location, which represents the probability that the involved prediction is a foreground object. It should be noted that the output of the objectiveness branch does not contribute to inference directly, it is only used for providing rich information. The classification branch generates \(C\)-\(d\) (C = 80 for COCO, C = 20 for PASCAL VOC, C = 1 for CrowdHuman and Caltech) probability logits for each location to denote the probabilities that the prediction belongs to each labeled category. The proposed Enhanced Positive Sample Filter generates a 1-\(d\) vector \(m_{x,y}\) for each location. Its output serves as a selection mask to filter out the positive sample. The final detection results 'Dets' can obtained as follows:  
\[\text{Dets}=\mathcal{H}(\Phi(I))=\{\text{Reg, Obj, Cls, EPSF}\}(\Phi(I))=\{\mathcal{B},\mathcal{P}\},\]  
where \(I\) represents the input image and \(\Phi(\cdot)\) denotes the backbone network with FPN. The generated \(\{\mathcal{B},\mathcal{S}\}\) represents the set of predicted bounding boxes \(\mathcal{B}\) and corresponding classification probabilities \(\mathcal{P}\). \(\mathcal{H}(\cdot)\) represents the shared detection head that consists of four components, i.e., \(\text{Reg}(\cdot)\), \(\text{Obj}(\cdot)\), \(\text{Cls}(\cdot)\) and \(\text{EPSF}(\cdot)\), which denotes the regression branch, objectiveness branch, classification branch and the proposed Enhanced Positive Sample Filter, respectively. They can be formulated as follows:  
\[\text{Reg}(\Phi(I))=\{\mathcal{B},f_{\text{reg}}\},\] \[\text{Obj}(\Phi(I))=\{\mathcal{P}_{obj},f_{obj}\},\] \[\text{Cls}(\Phi(I))=\{\mathcal{P}_{cls}\},\] \[\text{EPSF}(f_{\text{reg}},f_{obj})=\mathcal{M}_{\text{epsf}},\] \[\mathcal{P}=\mathcal{P}_{cls}\cdot\mathcal{M}_{\text{epsf}},\]  
where \(f_{\text{reg}}\) and \(f_{obj}\) denote feature maps from the fourth convolutional layers in regression and objectiveness branches, respectively. \(\mathcal{M}_{\text{epsf}}\) represents the selection mask generated by EPSF.  
The overall training objectives can be formulated as follows:  
\[L=L_{cls}(\mathcal{P}_{cls})+L_{obj}(\mathcal{P}_{obj})+L_{\text{reg}}( \mathcal{B})+L_{\text{o}to}(\mathcal{P}),\]  
where \(L_{cls}\), \(L_{obj}\) and \(L_{\text{reg}}\) are applied to classification, objectiveness and regression branches, respectively. Specifically, \(L_{\text{o}to}\) is applied to the products of outputs from classification and the proposed Enhanced Positive Sample Filter. \(L_{cls}\), \(L_{obj}\) and \(L_{\text{o}to}\) are a focal loss in [2] while \(L_{\text{reg}}\) is the IoU loss in [38].