# SSNet: Learning Mid-Level Image Representation Using Salient Superpixel Network
## 3 The Proposed Method for Image Representation
### The Mid-Level Image Representation Based on SSNet
Thus, we propose a novel representation method that roughly divides an image into object and background region first by saliency detection technology and then only choses the saliency regions to represent an image. By this way, the side effect of burstiness and noise information in the background can be alleviated. To this end, a fast saliency detection algorithm based on the Gestalt grouping principle is proposed. Figure 6 presents some saliency maps produced by the proposed saliency detection algorithm on the sample images taken from the Pascal VOC 2007 and 15-Scene datasets. As mentioned in Section 3.3, in the proposed method, only the patches with a top-\(S\) large salient value are used to describe an image. The value of the \(S\) is a key parameter, which affects the performance of the representation. In Equation (33), we assume that the \(NumSuppixel_{img}\) is a constant, and the value

Figure 6: Saliency maps for the images from VOC 2007 and 15-Scene. (**a**) Saliency maps for images taken from Pascal VOC 2007; (**b**) Saliency maps for images taken from 15-Scene. of \(S\) is in proportion to \(Ratio_{sal}\). So, the performance of our method can be considered as a function of \(Ratio_{sal}\). Figure 7 shows the curve of the mAP of our approach on the benchmark datasets with the different \(Ratio_{sal}\). Here, we take the LLC encoding method [4] as an example and integrate it with the saliency detection scheme to present the images. In the experiments, all the settings for LLC are in accordance with the work [4] except that the SPM technology can only be adopted under the condition of \(Ratio_{sal}=100\%\) (note that \(Ratio_{sal}=100\%\) denotes the original LLC method, i.e., without introducing the saliency detection scheme). As shown in Figure 7, the LLC encoding method achieves the best result on the VOC2007, Caltech 101, and Caltech 256 datasets when \(Ratio_{sal}=80\%\), and on the 15-Scene dataset when the \(Ratio_{sal}=85\%\). So, it is clear that the saliency strategy is capable of enhancing the discriminative power of image representation. For the other encoding methods, the results follow similar trends. Thus, in the following experiments, we choose the same settings for the saliency detection method. ### Effectiveness Evaluation of Weighted Second-Order Pooling

In order to evaluate the performance of the weighted second-order pooling method, we integrate it with different BoVW baseline methods such as ScSPM [3], LLC [4], FV [22], and VLAD [5], and analyze the classification results. In the experiments, for the ScSPM [3] and LLC [4], the size of each codebook is set to 1000, and the other settings are accordance with the works [3] and [4], respectively. Here, when ScSPM [3] is aggregated by the weighted second-order pooling method, only Sparse Coding (SC) vector is used and the spatial pyramid matching (SPM) scheme is ignored. For the FV [22] and VLAD [5], the Dense SIFT features are reduced to 64 dimensions first, and the size of codebook is set to 128. The obtained FV and VLAD encoding vectors are reduced to 1000 dimensions. Finally, the two encoding vectors mentioned above are aggregated by the weighted second-order pooling method respectively to build a 100,000 dimension (reducing the dimension from 500,500 to 100,000) representation vectors. The regularization parameters in Equation (25) are set to \(\lambda_{1}=10^{2}\) and \(\lambda_{2}=10^{1}\) by the cross-validation method. The classification accuracy is summarized in Table 1. In the following subsection, if not especially specified, the accuracy is measured in terms of Mean Average Precison (MAP in %), and the values highlighted in bold signify the best results between the approaches in the table. Figure 7: Performance of the improved LLC representation with different saliency region ratios. This figure illustrates the classification accuracy of LLC representation, formed by the LLC encoding method conducting on different numbers of saliency regions, on the benchmark datasets. Table 1 summarizes the results of the four benchmark datasets.