DeepCT: A novel deep complexvalued network with learnable transform for video saliency prediction Abstract   The past decade has witnessed the success of transformed domain methods for image saliency prediction. However, it is intractable to develop a transformed domain method for video saliency prediction, due to the limited choices on spatiotemporal transforms. In this paper, we propose learning the transform from training data, rather than the predefined transform in the existing methods. Specifically, we develop a novel deep Complexvalued network with learnable Transform (DeepCT) for video saliency prediction. The architecture of DeepCT includes the Complexvalued Transform Module (CTM), inverse CTM (iCTM) and Complexvalued Stacked Convolutional Long ShortTerm Memory network (CSConvLSTM). In the CTM and iCTM, multiscale pyramid structures are introduced, as we find that transforms at multiple receptive scales can improve the accuracy of saliency prediction. To make the CTM and iCTM xe2x80x9cinvertiblexe2x80x9d, we further propose the cycle consistency loss in training DeepCT, which is composed of frame reconstruction loss and complex feature reconstruction loss. Additionally, the CSConvLSTM is developed to learn the temporal saliency transition across video frames. Finally, the experimental results show that our DeepCT method outperforms other 13 stateoftheart methods for video saliency prediction.

EMLNET:An Expandable MultiLayer NETwork for Saliency Prediction Abstract   Existing advanced saliency systems have been proposed using Convolutional Neural Networks (CNNs), the prior knowledge of objectness is crucial for saliency detection. However, we show that the use of objectness may also limit the power of CNNs on the images which do not contain a salient object. Besides, one previous study has shown applying a deeper CNN model may not improve the performance, the reason behind could be due to the limited training data. In this work, we aim at investigating the effect of prior knowledge from other domains and proposing a multimodality system. Our work shows the depth of the model still plays an important role in saliency detection when the training data is large enough. To do this in a sophisticated manner can be complex, and also result in unwieldy networks or produce competing objectives that are hard to balance. For the scalability, our multimodality system is trained in an almost endtoend piecewise fashion. The encoder and decoder components are separately trained to deal with complexity tied to the computational paradigm and required space. Therefore, our system can be easily extended to include a variety of prior knowledge for saliency detection. Besides, we also propose a combined saliency loss based on modifications of Pearson correlation and normalized scanpath saliency. Our experiment shows the combined loss can train a CNN model more comprehensively for saliency detection. We denote our expandable multilayer network as EMLNET and our method achieves the stateoftheart results on the public saliency benchmarks, SALICON, MIT300 and CAT2000.