# Efficient Shot Detector: Lightweight Network Based on Deep Learning Using Feature Pyramid
## 3 Proposed Method
### Network Architecture  
The overhead should be low in the process after the backbone network for fast and efficient object detection. The proposed network leads to a lightweight pyramid structure with features extracted from EfficientNet. The scaling value of EfficientNet mentioned in Equation (1) was set as \(\alpha=1.2\), \(\beta=1.1\), \(\gamma=1.15\). The max-pooling process at the top scale S3 of the pretrained backbone network was applied twice to generate S4 and S5 with \(8\times 8\) and \(4\times 4\) scales, respectively. In general, CNN-based object detection is performed using a prior box based on the center point of the feature map grid. Therefore, the highly compressed high-level feature map compared to the input image scale has information about large objects. Conversely, a low-level feature map has information about small objects because it is not compressed with the context information about the image. The proposed lightweight pyramid structure performs detection with a total of five heads from S1 to S5 scales. First, the fusion process was performed with features with adjacent scales through the upsampling path. This serves to supplement the semantic information of the object by utilizing the context information of the feature. Second, the location information of the object was supplemented from each feature channel through the downsampling path. Between the two paths, efficient training is possible by designing a shortcut structure that reuses input features. Finally, the detection was performed through classification and localization of each head. Table 1 shows the scale of the five heads. Figure 2: Efficient Shot Detector (ESDet) architecture. We used depthwise and pointwise instead of standard convolution in the lightweight pyramid structure to minimize the network computation cost. A convolution operation is performed on one channel of a feature as a unit instead of performing a filter operation on multiple channels of a feature map, reducing the computation cost compared to the standard convolution. Equations (2) and (3) show the comparison of standard convolution and depthwise and pointwise convolution operations. \[conv_{std}=F_{w}\times F_{h}\times C_{input}\times C_{output} \tag{2}\]  
\[Conv_{dw+pw}=DW+PW\]  
\[=(F_{w}\times F_{h}\times C_{input})+(C_{input}\times C_{output}) \tag{3}\]  
The standard convolution requires \(3\times 3\times 112\times 40\) = 40,320 parameters when S3 is calculated as an example, as shown in Table 1. On the other hand, \(112\times(3\times 3+40)=5488\) parameters are required when using the depthwise and pointwise convolution. Therefore, the parameter cost can be reduced by about eight times. Based on this, all convolutions of networks are replaced with the depthwise and pointwise convolution. The features of adjacent scales are fused based on the feature refining process in the feature pyramid structure shown in Figure 2. The feature fusion adds features with different information to the same input image, as shown in Equation (4). This method classifies object sizes in various ways compared to using a single feature. \[F_{i}^{up}=Conv*(\sum_{x=1}^{W}\sum_{y=1}^{h}(F_{i}\oplus\text{bilinear}(F_{i- 1})) \tag{4}\]  
This process shows an example of an upsampling path. The small scale \(F_{i-1}\) is adjusted to the same size as \(F_{i}\) using the bilinear interpolation method. Then, the element summing is performed for each pixel. A low-cost convolution for the fused features was used as shown in Equation (3). However, all input features are treated equally when a refinement process is added between layers by simply convolution. Therefore, the proposed feature refinement process is performed as in Equation (5). \[F_{i}^{DW}=\text{DW}(\text{Conv}_{1\times 1}(F_{i}^{up})) \tag{5}\]  
The feature \(F_{i}^{up}\) fused with two scale features is extended to \(1\times 1\) convolution \(Conv_{1\times 1}\) to preserve the channel information.