# SSNet: Learning Mid-Level Image Representation Using Salient Superpixel Network
## 3 The Proposed Method for Image Representation
### The Mid-Level Image Representation Based on SSNet
```

**Algorithm 1**Generating the SSNet Mid-Level Image Representation

## 4 Experiments and Results

In this section, we carry out a series of experiments to investigate the effectiveness of the SSNet framework. First, the experiment setting and the datasets used are introduced. Then, integrating with four BoVW baseline methods, including ScSPM [3], LLC [4], FV [22], and VLAD [5], we assess the benefit of the saliency detection scheme and the weighted second-order pooling method. Finally, we evaluate the overall performance of the SSNet mid-level representation with the handcrafted and off-the-shelf CNN local features, respectively. ### Experimental Setting

In the experiment, we use the Dense SIFT local features and the off-the-shelf CNN local features respectively to evaluate the proposed methods. All the algorithms are implemented by the VLFeat [57] and Matconvnet toolbox [58]. The setting for the CNN local features follows the description in Section 2.2. For Dense SIFT features, they are extracted densely from local patches with a stride of four pixels on each image, and PCA is applied to reduce the dimensionality to 64. In the BoVW representation, a codebook containing \(K\) visual words is built via \(K\)-means clustering, and the value \(K\) varies across different datasets. In addition, the \(\sigma\) in Equation (24) is set to 1. With the same settings as [16], the \(\lambda_{1},\lambda_{2}\) in Equation (25) is chosen by cross-validation from the set \(\left\{10^{1},10^{2},10^{3},10^{4},10^{5}\right\}\), and the \(\alpha\) in Equation (30) is set to 0.5. The setting for other parameters will be discussed in the following subsection. For each dataset, the experiments are repeated five times, and the average accuracy is reported. ### Benchmark Datasets

**15-Scene** This dataset consists of 4485 scene images that belong to 15 different categories. Each category contains about 200-400 images, with an average size of \(300\times 250\). In the experiments, we test our method by randomly selecting 100 training images per class and using all the rest for testing. **Pascal VOC 2007** This dataset is a challenging benchmark, which consists of 9963 images from 20 classes. There are large in-class divergence, including variation on scale, illumination, and deformation, as well as severe object occlusions. Here, we choose this dataset as a sample to demonstrate the parameter setting for the SSNet. **Caltech 101** This dataset is one of the most commonly used datasets for image classification. It contains 9146 images from 101 object categories and one background category. The number of images per category varies from 40 to 800. In our experiments, we randomly select 30 training images and at most 50 testing image and ignore the background class. **Caltech 256** This dataset consists of 30,607 images divided in 256 categories, containing at least 80 images each. Compared to Caltech 101, it presents a much higher variability in object size, location, and pose. In the experiments, we randomly select 45 training images per class and all the rest for testing without evaluating the background class. **PFID61** The dataset contains 1098 fast-food images belonging to 61 different food categories, with masked foreground [59]. Each food category includes three different instances. For each instance, six images are collected from six viewpoints. In the experiments, 12 images from two instances are used for training and six images are used from the third instance for testing. **Food101** The dataset contains 101,000 images divided in 101 food categories [60]. Each category contains 1000 images, and all the images are rescaled to have a maximum side length of 512 pixels. In the experiments, 750 images are used for training and 250 images are used for testing. ### Effectiveness Evaluation of the Proposed Saliency Detection Scheme for Image Classification

Generally speaking, for an image, the objects often exist in the saliency region and the background corresponds to the non-saliency region.