# An End-to-End Deep Learning Image Compression Framework Based on Semantic Analysis
## 3 Network Structure  
### Semantic Analysis Network  
To enable the differentiated compression of the image, it is necessary to perform semantic analysis on the input image to identify an area of interest to the human eye and then allocate it more compressed bits. Although the existing techniques are able to accurately outline the boundary of the objects in the picture, this segmentation of the hard boundary of the object is not important for semantic compression, and it is critical to locate the approximate range of the object. In this paper, we use deep learning methods for image classification to achieve the annotation of semantic areas. The class activation mapping (CAM) method is proposed by [PERSON] in [20] to describe the possibility that each pixel belongs to a specific category. Based on the CAM method, the probability that each pixel belongs to a specific class can be converted into the semantic importance of the pixel. The higher the possibility, the higher the importance of the pixel for the picture content, and the more compression bits need to be allocated.  
The semantic analysis network structure used in this paper is shown in Figure 2, using a classification-based training network architecture similar to Visual Geometry Group 16 (VGG16). The first five convolutional layers are used to extract the characteristics of the input image, and a global average pooling (GAP) layer is placed after the last convolution layer instead of the fully connected layer (FC) to convert the feature map into a feature vector. Compared to FC, GAP reduces a large number of network parameters, preventing over-fitting, and most importantly, GAP can save spatial information of images, which is significant to generate the semantic map. Finally, the weighted linear sum of the feature vector and its corresponding weight are input to the softmax layer to obtain the class activation map.  
Given an input image \(I\) and a set \(C\) of categories to be identified, we can extract feature maps \(f_{u}\) by utilizing the CNN network, which obtains the same number as categories in set \(C\). \(f_{u}\) is the feature map of \(I\) for the convolutional unit \(u\) after the last convolutional layer _Conv5_, and each \(f_{u}\) has the ability to extract features of a certain category in \(I\). Each \(f_{u}\) will be averaged in GAP, and multiplied by the weight \(\alpha_{u}^{c}\)--the weight of convolution unit \(u\) corresponding to the class \(c\)--to obtain the possibility \(P_{c}\) of \(f_{u}\) belonging to \(c\).  
\[P_{c}=\sum_{u}\alpha_{u}^{c}G(f_{u}) \tag{1}\]  
where \(G(f_{u})\) is the GAP operation of \(f_{u}\). We use \(P_{c}\) as the input to softmax layer; then, the final classification result \(R_{c}\) can be calculated by Formula (2).  
\[R_{c}=\frac{\exp(P_{c})}{\sum\limits_{c}\exp(P_{c})}=\frac{\exp(\sum\limits_{u }\alpha_{u}^{c}G(f_{u}))}{\sum\limits_{c}\exp(\sum\limits_{u}\alpha_{u}^{c}G(f _{u}))} \tag{2}\]  
Figure 2: Semantic analysis network structure.  
The class \(c\) with the highest probability is chosen as the classification result of the image \(I\), and the cross-entropy of the result and the label corresponding to \(I\) are used as the loss function to train the network. Finally, we can obtain the trained weight \(a_{u}^{c}\).  
To visualize the effect of the network on image \(I\), for the specific class \(c\), we use the linear weighted sum \(H_{c}\) of \(a_{u}^{c}\) and \(f_{u}\) to represent the probability that each pixel in the image belongs to class \(c\).  
\[H_{c}=\sum_{u}a_{u}^{c}f_{u} \tag{3}\]  
By up-sampling \(H_{c}\) to the size of picture \(I\) and superimposing it with \(I\), we can get the thermal maps shown in Figure 3 as the result of semantic analysis. The degree of highlighting indicates the level of semantic importance.