## 3 Approach
### Salient Region Localization Module  
Our model uses bilinear CNN to extract features. Then, a weighted gradient-based algorithm for class activation mapping is applied on the resulting features. This process is inspired by the process used in the Grad-CAM model, which was proposed by [PERSON] et al. [26]. This process enables our model to eliminate the influence brought by varying structures of convolutional neural networks. Figure 2: Overview of our neural network’s structure. The whole neural network is a two-level attention model. The saliency module is used for locating salient regions of the target images, then the bilinear attention module is used as our feature extractor. Finally, the derived features are used to calculate the outer product to achieve the final fused feature. Additionally, it grants our model the ability to generate visually interpretable feature maps. It also makes our model capable of giving a score to a specific label when only one image and the target labels are fed into our bilinear CNN model without training from the ground up or changing the original CNN model's structure. The score of the labels is obtained through the calculation of specific tasks. For all labels, except the required, the labels' gradients are set to 1; the rest of the gradients are set to 0, and then the gradient is propagated back to the entire convolutional feature map. All feature maps are combined by a precise method for obtaining heat maps of the given image. The resulting heat map reveals the part that needs to pay more attention. Finally, we apply element-wise multiplication to the heatmap and the directed backpropagation, using bilinear interpolation to up-sample the input image's resolution. Then, we merge the backpropagation results and visualization results to obtain saliency maps, which are shown in Figure 3.  
After obtaining the generated saliency map, an adaptive maximum inter-class variance algorithm is used to obtain the threshold according to the calculation [31], and the threshold is used for converting the saliency feature map into a binary mask. Thereby, we can distinguish the background from the foreground and highlight the differences between these two parts of the image. We use \"1\" for meaning the specific position of the provided image is foreground, and \"0\" for meaning that the position is background. Then, we apply the eight-connected region labelling algorithm to the foreground to locate the target and label the target coordinates. The mentioned processes of the saliency model are shown in Figure 4.  
Figure 4: Saliency module based on weakly supervised learning. Figure 3: Saliency maps of different images. We locate and obtain the most distinct regional area from the input image to generate the heat map. We choose to generate heat maps as they can be visualized directly by adding to the original image. We use the bilinear interpolation method to generate a heat map for the original image. The heat map and the input image have the same size. The heat map will be combined with the original image. However, different feature maps have a different region of response on original images, and the key regional features are found to be not salient enough after visual analysis, so we cannot localize salient targets with them. To solve this problem, we decide to sum over the \(d\) dimension of three-dimension tensor \(D\), which has the size \(h\times w\times d\), turning it into a two-dimension tensor \(B\), which has the size \(h\times w\), to better localize salient targets. The addition equation is as follows:  
\[B=\sum_{i=1}^{d}A_{i}. \tag{1}\]  
In the equation above, \(A_{i}\) is a feature map of the \(i\)-th channel. The fusion of multiple feature maps through Equation (2) helps our model to enhance the feature information of salient areas, which in turn makes it easier to locate regional salient areas more accurately. Each saliency map, having the size of \(h\times w\), corresponds to all pixels in the \(h\times w\) area.