Recent results suggest that state-of-the-art saliency models perform far from optimal in predicting fixations. This lack in performance has been attributed to an inability to model the influence of high-level image features such as objects. Recent seminal advances in applying deep neural networks to tasks like object recognition suggests that they are able to capture this kind of structure. However, the enormous amount of training data necessary to train these networks makes them difficult to apply directly to saliency prediction. We present a novel way of reusing existing neural networks that have been pretrained on the task of object recognition in models of fixation prediction. Using the well-known network of Krizhevsky et al., 2012, we come up with a new saliency model that significantly outperforms all state-of-the-art models on the MIT Saliency Benchmark. We show that the structure of this network allows new insights in the psychophysics of fixation selection and potentially their neural implementation. To train our network, we build on recent work on the modeling of saliency as point processes. By understanding how humans choose eye fixations, we can hope to understand and explain human behaviour in a number of vision-related tasks. For this reason human eye movements have been studied for more than 80 years (e.g., Buswell, 1935) . During the last 20 years, many models have been developed trying to explain fixations in terms of so called "saliency maps". Recently, it has been suggested to model saliency maps probabilistically using point processes (Barthelmé et al., 2013) and to evaluate them using loglikelihood (Kümmerer et al., 2014) . This evaluation revealed that state-of-theart models of saliency explain only one third of the explainable information in the spatial fixation structure (Kümmerer et al., 2014) . Most of the existing models use low-level cues like edge-detectors and color filters (Itti et al., 1998)
Deep Gaze I REF utilizes CNNs for the fixation prediction task by treating saliency prediction as point processing.
Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet
ICLR 2015
arXiv: Computer Vision and Pattern Recognition
Computer Science Biology Mathematics