We propose a locality-constrained contextual sparse coding (LCCSC) for top-down saliency estimation where higher saliency scores are assigned to the image locations corresponding to the target object. Three locality constraints are integrated in to this novel sparse coding. First is the spatial or contextual locality constraint in which features from adjacent regions have similar code, second is the feature-domain locality constraint in which similar features have similar code, and third is the category-domain locality constraint in which features are coded using similar atoms from each partition of the dictionary, where each partition corresponds to an object category. Person Bicycle Sheep Horse Sofa Cat Train TV monitor Figure 1 : Our Saliency maps on Graz-02 and PASCAL VOC-07 datasets. The pipeline of the proposed method is shown in fig. 2 , which is similar to the widely used sparse coded spatial pyramid matching (ScSPM) image classifier, i.e., feature extraction, feature coding, pooling and feature classifier. Spatial neighborhood of a feature is divided into a reg- ular grid and the codes in each cell of the grid are max-pooled individually. These max-pooled vectors are vertically concatenated to form a context max-pooled vector representing the patch. Logistic regressionbased feature classifier is learnt using these context max-pooled vectors. For saliency inference on a test image, the class-conditional probability of context max-pooled vectors are estimated from the learnt logistic regression model. This probability is the saliency value of a patch in the image. The pixel-level saliency map is obtained using a novel Gaussian-weighted interpolation of the patch-level saliency map. Locality-constrained contextual sparse coding (LCCSC) Various coding schemes aim for specific objectives like sparsity, featuredomain locality and spatial-domain locality. Here, all these desired properties are integrated into a single objective function. LCCSC ensures that features representative of salient regions are not ignored even if they are not discriminative. Given a feature vector f and dictionary D, LCCSC coding searches for the codeword z that satisfies the following criteria: The first two terms are the conventional sparse coding of feature f with l 1 constraint. The third term imposes locality constraint in the feature domain as well as in the spatial domain. The fourth term imposes category-domain locality, i.e the number of atoms that contribute to the non-zero values of z are distributed among all object categories. Detailed discussion about eq. (1) and an approximate solution is available in the paper. We test our method on two challenging datasets -Graz-02 and PAS-CAL VOC-07. Table 1 compares the pixel-level results of our method with other top-down saliency models. The proposed method is called as LCCSC-pooled, where pooled indicates contextual max pooling of the code. It is to be noted that we achieve state-of-the-art performance by using simpler feature coding and contextual max-pooling in comparison with computationally complex dictionary learning and graph-based approaches of [1, 2] . As in [2] , all models are evaluated on the entire 210 segmentation test images of PASCAL VOC-07 dataset, irrespective of the presence or absence of target. We outperform [2] in 15 out of 20 classes. On averaging across all classes, in patch-level, we achieve a mean precision rate at EER of 23.4% which is better than 16.15% of [2] .
Cholakkal et al. REF designed a top-down saliency detection approach via locality-constrained contextual sparse coding.
Top-down Saliency with Locality-constrained Contextual Sparse Coding
BMVC
Computer Science