# Deep-Learning Based Algorithm for Detecting Targets in Infrared Images
## 3 CMF Net
### Feature Fusion Strategy  
Two multiscale feature extraction methods use different strategies to extract multiscale feature maps. The final output of the feature map has its unique advantages. The first multiscale feature extraction method ultimately retains the mixed features of target location and target recognition extracted from the middle convolutional layer. Still, it loses some high-level semantic features and affects target recognition. The second multiscale feature extraction method preserves the semantic features of a high-level convolutional layer but loses some mixed features, which affects the target location. Therefore, it is necessary to fuse the two kinds of feature maps so that the final output feature map simultaneously contains rich mixed features and semantic features.  
Figure 3: Multiscale feature extraction II.  
Figure 2: Multiscale feature extraction I.  
The final feature maps obtained by the multiscale feature extraction mechanism suffer from inconsistent resolution and inconsistent amplitude of feature values. Therefore, these feature maps cannot be directly fused with features. The feature fusion method proposed in this paper makes the feature maps output by the first multiscale feature extraction mechanism consistent with the resolution of the feature maps output by the second multiscale feature extraction mechanism through a pooling process. Secondly, L2 normalization is performed in the feature maps outputted by the two feature extraction methods, so that the amplitudes of feature values in the two feature maps are consistent. Then, we connect two feature maps to get a feature map containing rich visual features, mixed features, and semantic features. Finally, we input it to RPN to extract ROI information.  
We assign a binary class label to each box (including the target or excluding the target). We set a binary class label (include target or not) to each box. We assign a positive title to a box with an IoU threshold higher than 0.7 with any ground truth box and then assign a negative label to a box with an IoU threshold lower than 0.3 with all ground truth boxes. Our goal is to minimize a multitask loss function.  
\[L(k,k^{*},t,t^{*})=L_{cls}(k,k^{*})+\lambda L_{reg}(t,t^{*}) \tag{1}\]  
\(L_{cls}\) is the classification loss, \(L_{reg}\) is the coordinate regression loss of the box with a positive label assigned. \(k^{*}\) and \(k\) are true to label and predicted labels separately, respectively. \(L_{reg}(t,t^{*})=R(t-t^{*})\) where R is the smoothed loss function defined in Faster R-CNN. We express the coordinates of the positive box as \(t=\left(t_{x},t_{y},t_{w},t_{h}\right)\) and the coordinates of the predicted box as \(t^{*}=\left(t_{x^{*}}^{*},t_{y^{*}}^{*},t_{h}^{*}\right)\).  
\[\begin{split} t_{x}=\left(G_{x}-P_{x}\right)/P_{w}\cdot t_{y}& =\left(G_{y}-P_{y}\right)/P_{h}\\ t_{w}=\log(G_{w}-P_{w})\cdot t_{h}&=\log(G_{h}-P_{ h})\end{split} \tag{2}\]  
where \(P^{i}=\left(P_{x},P_{y},P_{w},P_{h}\right)\) specifies the coordinates of the center point of the predicted box. \(G^{i}\) specifies the coordinates of the center point of the positive box.  
The RPN module first resamples the unbalanced sample set of positive and negative samples, using the oversampling method in random sampling to obtain more sample first data balance by randomly repeating examples from a small number of class sample sets. Then, the gradient descent method is used for training, and the classification loss error \(L_{cls}\) and regression loss error \(L_{reg}\) are back-propagated to update the model parameters until the RPN module converges. The parameters for the training of the RPN module are set as shown in Table 1.