Contexts play an important role in the saliency detection task. However, given a context region, not all contextual information is helpful for the final task. In this paper, we propose a novel pixel-wise contextual attention network, i.e., the PiCANet, to learn to selectively attend to informative context locations for each pixel. Specifically, for each pixel, it can generate an attention map in which each attention weight corresponds to the contextual relevance at each context location. An attended contextual feature can then be constructed by selectively aggregating the contextual information. We formulate the proposed PiCANet in both global and local forms to attend to global and local contexts, respectively. Both models are fully differentiable and can be embedded into CNNs for joint training. We also incorporate the proposed models with the U-Net architecture to detect salient objects. Extensive experiments show that the proposed PiCANets can consistently improve saliency detection performance. The global and local PiCANets facilitate learning global contrast and homogeneousness, respectively. As a result, our saliency model can detect salient objects more accurately and uniformly, thus performing favorably against the state-of-the-art methods. * Corresponding author Recently, convolutional neural networks (CNNs) have been introduced into saliency detection to learn effective contextual representation. Specifically, several methods [18, 24, 47] first directly use CNNs to extract features from multiple image regions with varying contexts and subsequently combine these contextual features to infer saliency. Some other models [17, 19, 23, 22, 37, 9, 26, 45, 46, 38] adopt fully convolutional networks (FCNs) [25] for feature representation at each image location and generate saliency map in a convolutional way. In these models, the first school extracts contextual features from each input image region, while the second one extracts features at each image location from its corresponding receptive field. However, all the existing models utilize context regions holistically to construct contextual features, in which the information at every contextual location is integrated. Intuitively, for a specific image pixel, not all of its contextual information contribute to its final decision. Some related regions are usually more useful, while other noisy responses should be discarded. For example, for the red dot pixel in the first row in Figure 1 , we need to compare it with the background to infer its global contrast. If we want to
Further, a pixel-wise contextual attention model was proposed to learn global and local contextual maps in order to only include helpful context areas for the final saliency prediction REF .
PiCANet: Learning Pixel-Wise Contextual Attention for Saliency Detection
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
Computer Science