# SSNet: Learning Mid-Level Image Representation Using Salient Superpixel Network
## 3 The Proposed Method for Image Representation
### The Mid-Level Image Representation Based on SSNet
In the meanwhile, we find that even if the saliency detection intermediate step is not adopted, weighted second-order pooling can also improve the performance (FV+WSOMP, 55.8% on PFID61, and 51.6% on Food-101). In the second set of experiments, we evaluate the performance of the SSNet framework using the off-the-shelf CNN local features. All the methods adopt the same settings as in the previous experiments. The proposed FV+SSNet yields an accuracy of 65.3% on PFID61 and 60.6% on Food-101, outperforming other two methods, MOP [8] and FC-CNN [36]. These results validate the effectiveness of our proposed method. According to the classification results, Figure 8 lists some sample images from the Food-101 dataset. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Features** & **Methods** & **PFID61** & **Food-101** \\ \hline \multirow{3}{*}{Dense SIFT} & FV [22] & 40.2 & 38.9 \\  & FV (using MASK) & 42.9 & NA \\  & FV + WSOMP & 55.8 & 51.6 \\  & FV + SSNet & 57.1 & 52.4 \\ \hline \multirow{3}{*}{Off-the-shelf CNN features} & FC-CNN [36] & 48.2 & 54.1 \\  & MOP [8] & 61.7 & 59.7 \\ \cline{1-1}  & FV + SSNet & 65.3 & 60.6 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison of different image encoding methods using Dense SIFT, off-the-shelf CNN features on PFID61 and Food-101 datasets (MAP in percentage). Figure 8: Some sample images from the Food-101 dataset. (**a**) These four images are selected to build the SSNet representation using the Dense SIFT features. (**b**) These four images are selected to build the SSNet representation using the CNN local features. The check mark indicates that this image can be recognized correctly by our proposed method, and the cross indicates that our method cannot recognize this image correctly. ### Limitations

In the previous sections, the performance of the SSNet has been discussed. Next, we will analyze its limitations. Compared with the traditional BoVW representation model, the introduction of the saliency detection and the weighted second-order pooling operation enhance the discriminative ability of the SSNet image representation. However, both of the introduced modules have their own limitations. In practical application, the parameter \(Ratio_{sal}\) in Equation (33) is important, which determines the size of the selected regions of an image for building the image representation. An inappropriate parameter setting may have a negative effect on the classification results. So, if a target in the recognition task exists in the non-saliency regions, we should assign a large value to \(Ratio_{sal}\) so that the target can be included in the selected region. At present, the setting for the parameter \(Ratio_{sal}\) only depends on people's prior experience. In the future, we hope to find a solution that can adjust this parameter adaptively. In addition, the weighted second-order pooling operation increases the dimension of the final representation, which will consume more memory resources. To solve this problem, people can try the following three approaches: (1) adopting low-dimension local features; (2) adopting small size codebooks; and (3) reducing the dimension of the final representation vector by dimensionality reduction methods. ## 5 Conclusions

In order to deal with the burstiness problem of features and introduce high-order information for image representation, we have proposed a novel framework, named Salient Superpixel Network (SSNet), to learn the mid-level image representation. It can be regarded as an extension of the BoVW representation model to exploit both the saliency detection and the weighted second-order pooling technologies. Specifically, we first propose a fast saliency detection method to choose the regions with high saliency values for representing the image. As a result, the interference information in the background is alleviated effectively. Secondly, the weighted second-order pooling (WSOP) method is applied to perform the feature aggregation.