  Recently, massive saliency detection methods have achieved promising results
by relying on CNN-based architectures. Alternatively, we rethink this task from
a convolution-free sequence-to-sequence perspective and predict saliency by
modeling long-range dependencies, which can not be achieved by convolution.
Specifically, we develop a novel unified model based on a pure transformer,
namely, Visual Saliency Transformer (VST), for both RGB and RGB-D salient
object detection (SOD). It takes image patches as inputs and leverages the
transformer to propagate global contexts among image patches. Apart from the
traditional transformer architecture used in Vision Transformer (ViT), we
leverage multi-level token fusion and propose a new token upsampling method
under the transformer framework to get high-resolution detection results. We
also develop a token-based multi-task decoder to simultaneously perform
saliency and boundary detection by introducing task-related tokens and a novel
patch-task-attention mechanism. Experimental results show that our model
outperforms existing state-of-the-art results on both RGB and RGB-D SOD
benchmark datasets. Most importantly, our whole framework not only provides a
new perspective for the SOD field but also shows a new paradigm for
transformer-based dense prediction models.
