What Do Deep Saliency Models Learn about Visual Attention?
Authors: Shi Chen
Ming Jiang
Qi Zhao
Venue: NeurIPS 2023 poster (2023)
Keywords: Saliency prediction
human attention
low-level vision
TL;DR: An analytic study on how deep saliency models prioritize visual cues to predict saliency.

Abstract: In recent years, deep saliency models have made significant progress in predicting human visual attention. However, the mechanisms behind their success remain largely unexplained due to the opaque nature of deep neural networks. In this paper, we present a novel analytic framework that sheds light on the implicit features learned by saliency models and provides principled interpretation and quantification of their contributions to saliency prediction. Our approach decomposes these implicit features into interpretable bases that are explicitly aligned with semantic attributes and reformulates saliency prediction as a weighted combination of probability maps connecting the bases and saliency. By applying our framework, we conduct extensive analyses from various perspectives, including the positive and negative weights of semantics, the impact of training data and architectural designs, the progressive influences of fine-tuning, and common error patterns of state-of-the-art deep saliency models. Additionally, we demonstrate the effectiveness of our framework by exploring visual attention characteristics in various application scenarios, such as the atypical attention of people with autism spectrum disorder, attention to emotion-eliciting stimuli, and attention evolution over time. Our code is publicly available at \url{https://github.com/szzexpoi/saliency_analysis}.

What Do Deep Saliency Models Learn about Visual Attention?
Shi Chen Ming Jiang Qi Zhao
Department of Computer Science and Engineering, University of Minnesota {chen4595, mjiang, qzhao) edu
Abstract
In recent years, deep saliency models have made significant progress in predicting human visual attention. However, the mechanisms behind their success remain largely unexplained due to the opaque nature of deep neural networks. In this paper, we present a novel analytic framework that sheds light on the implicit features learned by saliency models and provides principled interpretation and quantifica- tion of their contributions to saliency prediction. Our approach decomposes these implicit features into interpretable bases that are explicitly aligned with semantic attributes and reformulates saliency prediction as a weighted combination of prob- ability maps connecting the bases and saliency. By applying our framework, we conduct extensive analyses from various perspectives, including the positive and negative weights of semantics, the impact of training data and architectural designs, the progressive influences fine-tuning, and common failure patterns of state-of- demonstrate the effectiveness of
the-art deep saliency models. Additionally, our framework by exploring visual attention characteristics in various application scenarios, such as the atypical attention of people with autism spectrum disorder, attention to emotion-eliciting stimuli, and attention evolution over time, Our code is publicly available at btttps://githut wwt.huuu.oooeeeeeeeeeeeeeeeeeeee wtt.tt/...oooooeeeoooooa//aaaaaa
Introduction
Attention deployment is a complex and fundamental process that enables humans to selectively attend important sensory data in the visual environment. Scientists have been fascinated by the question of what drives human visual attention for decades. Understanding the mechanisms of visual attention not only sheds light on the human visual system but also helps computational methods to localize critical sensory inputs more efficiently.
One approach predicting human attention is through saliency models, which have received consid- erable research attention. saliency model predicts the most visually important regions in an image that are likely capture attention. Earlier models follow feature integration approach 2 3 4 extract low-level features (e.g., colors, intensity, orientations) or higher-level features (e.g., objects, semantics) from the input image to infer saliency 6 While these models show initial success, their performance is limited by the difficulty of engineering relevant visual features. In contrast, follow data-driven approach, leveraging large datasets [122 and to learn discriminative features. These models achieve human-
and
recent saliency models K
10
deep neural networks 13
14 15
performance on several saliency benchmarks [2
17.. thanks their accurate detection of 19 However, due to the lack of transparency. it is still unclear what semantic features these models have captured to predict visual saliency.
16
important objects and high-level semantics [8
To understand how deep neural networks predict visual saliency, in this paper, we develop a principled analytic framework and address several key questions through comprehensive analyses:
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

How do deep saliency models differentiate salient and non non-salient semantics?
How do data and model designs affect semantic weights in saliency prediction?
How does fine-tuning saliency models affect semantic weights?
Can deep saliency models capture characteristics of human attention?
What is missing to close the gap between saliency models and human attention?
Our method connects implicit features learned by deep saliency models to interpretable semantic attributes and quantifies their impact with a probabilistic method. It factorizes the features into trainable bases, and reformulates the saliency inference as a weighted combination of probability maps, with each map indicating the presence of basis. By measuring the alignment between the bases and fine- tme-grai semantic attributes (e.g,, concepts in Visual Genome dataset (0) it quantifies the relationships between diverse semantics and saliency. This unique capability enables us to identify the impact of various key factors on saliency prediction, including training datasets, model designs, and fine-tuning. It can also identify common failure patterns with state-of-the-art saliency models, such as SALICON [9] DINet [1I and TranSalNet [1O]. Beyond general saliency prediction, the framework also shows promise in analyzing fine-grained attention preferences in specific application contexts, such as attention in people with autism spectrum disorder, attention to amotional-elicitten stimuli, and attention evolution over time. In sum, our method offers an interpretable interface that enables researchers to better understand the relationships between visual semantics and saliency prediction, as well as a tool for analyzing the performance of deep saliency models in various applications.
Related Works
Our work is most related to previous studies on visual saliency prediction, which make progress in both data collection and computational modeling.
Saliency datasets. With the overarching goal of understanding human visual perception, considerable efforts have been placed on constructing saliency datasets with diverse stimuli. The pioncering study [16] proposes eye-tracking dataset with naturalistic images, which later becomes popular online benchmark [21] Several subsequent datasets characterize visual saliency into finer categories based on visual scenes [22] visual semanties [17], sentiments (23 24 temporal dynamics [25]. J. to study the impact of different experimental factors on attention. To overcome the difficulties of tracking eye movements, Jiang et al. [12] leverage crowd-sourcing techniques and use mouse-tracking an approximation for eye gaze, which results in currently the largest saliency dataset. Recent works also consider broader ranges of visual stimuli, including videos [26] 27), graphical designs (28 29],
web pages 330 31. crowds (32), driving scenes (33 and immersive environments 34 35 In this
study, we focus on visual saliency for naturalistic images, which serves as the foundation of saliency
prediction studies.
Saliency models. Prior works have developed saliency prediction models quantitatively study human attention. Inspired by seminal works on saliency modeling (5 6 36 early saliency models [ΣΣ 4 (37 typically adopt bottom-up approach, integrating handcrafted features (e.g., colors, intensity, and orientations). On the other hand, recent approaches take a different route and leverage deep neural networks [13
38) to automatically learn features and predict saliency. Vig 39 is one of the first attempts to utilize convolutional neural networks (CNNs) for saliency al. prediction. Huang al. [9] consider features learned from multi-scale inputs model the coarse- to-fine semantics. Kruthiventi et al. [40] leverage convolutional layers with diverse kernel sizes to convolution. Kümmerer et al. [41] demonstrate the usefulness of features of visual objects in saliency prediction. Coria et al. [ develop recurrent neural network iteratively refine features for saliency prediction. Yang et al. 111 improve saliency prediction with dilated convolution to capture information from
capture multi-scale features and incorporate positional biases with location-dependee
broader regions. Lou et al. [10] study the usefulness of self attention [[5] for saliency prediction.
Instead of building new models, several works 42 43 44 45 46) study the behaviors of models. By analyzing predictions on different categories of stimuli, they identify key factors behind the successes and failures of existing saliency models (e.g., accurate detection of semantic objects, incorporation of low- and high-level features, contrast between diverse visual cues, detection of Add-One-Out target, and etc.), and propose directions for improvements. A recent work [19] also

---

W
Probabilistic Distribution
W
Input Image
Deep Saliency Model
Traimable Buses
Saliency Map
Basis face 10% female huppy
Basis 30% sky 5% cloudy
2% blue
Basis 70% jersey + 10% black 5% white
Measurement
sky
hair black
jersey black
happy
bluue
femalo
cloudy
white
Basis N: 60% hair 25% black 10% brown
Semantic Anmorations R2
Semantic Alignment
Figure I: Illustration of our method. factorizes implicit features into trainable bases, and interprets the meanings of bases by aligning them with diverse semantics. Each basis can be interpreted as weighted combination of semanties (e.g,, face, female, and happy). By reformulating the saliency inference with probabilistic method, the relationships between semantics and saliency can be quantified by integrating the model weights Weal (i.e, the weight of each basis) and the semantic alignment O (i.e., the composition of semantics for each basis).
analyzes the features learned by saliency models by aligning activation maps with segmentation for selection of objects (e.g., body parts, food, and vehicles in (7)..
Our research contributes to the field of attention research by introducing a rigorous methodology
42 43], our study has three key distinctions: First, while previous analyses of deep saliency models were restricted to predefined set of salient objects (e.g., object segmentation used in [9)), our method automatically identifies both salient and non-salient semantics from vocabulary of objects, parts, actions, and attributes. Second, different from previous qualitative analyses, our method quantifies the weights of these semantics in saliency prediction. It allows us to investigate the impacts of various factors (e.g., the contributions of positive/negative semantics, the characteristics of datasets and model designs, and the process of fine-tuning) on saliency prediction, offering insights into the development of future deep saliency models. Third, our approach goes beyond analyzing the general visual saliency and demonstrates its strength in characterizing human visual attention under specific conditions such as the attention of people with autism spectrum disorder, the saliency of emotion eeliciiingi images, and time-course attention evolution.
for analyzing deep saliency models. In contrast to previous studies [8 19
Methodology
Human visual attention is influenced by spectrum of visual cues, from low-level contrasts to high- level semantic attributes [5]. However, current deep learning-based saliency models [8] 9 [T] remain opaque in terms of the semantic attributes they have learned and how these attributes contribute to saliency prediction. To address this gap, we propose a method that decomposes neural network features into discriminative bases aligned with wide range of salient non-salient semantic attributes, and quantifies their weights saliency prediction.
As illustrated in Figure our method decomposes visual features by projecting them onto collection of trainable bases, and uses the probabilistic distribution of bases to infer visual saliency. The overall idea is to identify both salient and non-salient semantics and quantify their impact on saliency prediction. To achieve this, we start with a deep saliency model and compare the features learned at the penultimate layer with different bases. This is done using a dot product between features V E xC (M and C are the spatial resolution and dimension of features) and bases E NCC (N 1000 is the number of bases defined based the number of units in the final layers of deep saliency models 9 10), which corresponds to their cosine similarity:
a & BT))
(I)
3

---

where @ denotes the dot product. is the Sigmoid activation for normalization, ar E [0, 1]
represents the probability of jMh basis by detected at the 1t region P(b, = 1.)) Inspired by 47 for decomposing model weights, we factorize the features as a weighted combination of matched bases:
N On Bj
V
(2)
where V E M are factorized features used to predict the saliency map S = W/Vs (S ERM W are weights of the last layer).
Upon building the connections between visual features and discriminative bases, we then re-route the final saliency prediction by (1) freezing all model weights including the bases, and (2) adjusting the last layer for saliency inference based on the probabilistic distribution a. We train a new layer (with weights W* for predicting the saliency map:
S Σ
W 0.
1=1
Intuitively, the method formulates the problem of saliency prediction as learning the linear correlation between the detected bases and visual saliency, which can be denoted as learning P(S by, by With the intrinsic interpretability of the design, i.e., a as the probabilistic distribution of bases and Wssa encoding the positive/negative importance bases, we are able to investigate the weights of different bases to visual saliency.
The final step is understand the semantic meanings of each basis. Unlike previous studies [8] 19 42, 43 that focus on predefined salient objects, we take into account a comprehensive range of semantics without assumptions on their saliency. Specifically, our method leverages the factorization paradigm to measure the alignment between each basis and the semantics. Given an image and the regions interest for different semantics (e.g.. bounding box annotations in Visual Genome (20)) we (1) compute the probabilistic map for each jth basis a:.j E IR M and (2) measure its alignment Oj, with the regions of interest Rp for each semantic p. Following [48), we binarize the probabilistic map with threshold tj and measure the alignment with Intersection over Union (IoU):
O₂, = Ma. > 4g) O Rp [a. [g] U Pep
(4)
We use an adaptive threshold t, for each individual basis, which defined to cover the top 20% of regions of probabilistic maps. Through iterating the measuring process for all images within the dataset, we are able to link bases learned from saliency prediction to a variety of visual semantics. We consider the top-5 semanties matched with each basis, and incorporate their average alignment scores Öj for determining the weight / to saliency:
Weal O
4
(5)
Z
where Z is the normalization factor that normalizes the contribution of the semantics to the range of [-1, 1] (for semantics with positive/negative contributions, Z denotes the maximal/minimum contribution among all semantics). This approach enables us to capture a broader range of contributing semantics, while avoiding an overemphasis on dominant salient/non- malient/non-salientt salient semantics (e.g., face and cloudness).
Overall, our method establishes the foundation for bridging implicit features learned by deep saliency models with interpretable semantics. goes beyond existing studies that analyze models' predictive behaviors a selection of object categories, as it considers comprehensive range of semantics without assuming their relevance to saliency. provides insights into how well the models capture both salient and non-salient semantics, and how they quantitatively contribute to saliency prediction.

---

1.00
action
vehicle
sky
other color
0.75
social
clothing
ground
0.50
animal
text
scene
0.25
0.00
0.25
0.50
0.75
-1.00
surfiing
airpline
hourd
zehrr
soo weowwbudddh douarriiin
Harebbourdi
decker er
train
writeen
blond
firsee
male
mooth
eye
heliii
tactl
barefpal
glowing
strcer sign
Tester
staring
asiaat
hardwwod
whkty
tun
hulding
chogrs
kntcheen
posing
cutly
road
folld
cat pet
oc geilinn
happyy myy
smiling-
room
par en
paved
dloudy
dv
ooyd pal deeee moo
cloudlee almilics
diuble
Figure 2: Important semantics learned by the deep saliency model and their weights. We visualize
the top-60 semantics with significantly positive or negative weights.
Experiment
4.1 Implementation
Semantic annotations. To correlate implicit features with interpretable visual semantics, leverage the Visual Genome (20) dataset. has (1) multiple objects in the same scene to derive relative importance; and (2) broad coverage of semantics in the naturalistic context, including objects, parts, attributes, and actions. We use the bounding box annotations of semantics (i.e., R in Equation to measure the alignment between bases and semantics.
titthh-ar art saliency prediction models, including SALICON (9] DINet [II] and TranSalNet [1O]. All models are optimized with a combination of saliency evaluation metrics (i.e., Normalized Scanpath Saliency (NSS) 49], Correlation Coefficient
Model configuration. We experiment with three state-of- siite-offthh-ait
(CC) [50]. and KL--Divergence (KLD) (51) as proposed in [8) and use ResNet-50 [3] as the backbone. Model training follows two-step paradigm: (1) The model is optimized to factorize features with trainable bases, where the weighted combination of bases (i.e., V in Equation 2 is used for predicting the saliency map. Note that we do not use pretrained and fixed deep saliency models, but optimize the corresponding model architecture with the proposed factorization modules. (2) We freeze the model weights learned the previous step and reroute the saliency inference to derive the saliency map from the probabilistic distribution (see Equation so that the interpretation on features learned by the same saliency model. Only the last layer Wam fine-tuned to learn the correlation between the distribution of bases and visual saliency.
4.2 How Do Deep Saliency Models Differentiate Salient and Non-Salient Semantics?
Deep saliency models are powerful tools for predicting visual saliency, and their ability to incorporate semantic information is crucial for closing the gap between computational modeling and human behaviors [17] To gain insight into the semantics that deep saliency models learn and their contribu- tions to saliency prediction, we apply our framework to the state-of-the-art DINet [11] trained on the SALICON [12] dataset (see Section 4.3 for results on other models and datasets), and explicitly measure the weights of diverse semantics during the inference process (i.e., Lp in Equation 55)
Figure 2 shows that the DINet model effectively captures variety of semantics that are closely related to visual saliency. These include social cues such as faces, noses, and beards, actions like having meeting, snowboarding, and jumping, clothing such as goggles, and salient object categories like animals, vehicles, and text. These findings resonate with previous research [18
19
43 that
saliency models learn to recognize salient cues. More importantly, they showcase the versatility of our approach in automatically identifying key contributing factors of saliency models without any preconceived assumptions 4 19), enabling attention analyses across diverse array of scenarios (see Section 4.5).
5

---

SALICON
OSIE
MIT
1.00
0.75
0.50
0.25
0.00
0.25
00.00
0.75
-1.00
action color
social
clothing
vehicle
animal
text
other
ground
sky
Figure 3: Semantic weights for DINet trained on different datasets.
Another unique advantage of our approach is to simultaneously derive semantics that both positively and negatively contribute to saliency, while previous studies commonly focus on the positive side. Our results reveal clear separation between the semantics that contribute positively and negatively to saliency. Specifically, the model considers social and action cues to have positive contribution to saliency, while semantics related to backgrounds such as sky (e.g.. hazy and overcast), ground (e.g.. pavement and carpet), and scene (e.g., bathroom and park) have negative weights. The observation demonstrates that deep saliency models' success is not only due the accurate detection of salient objects [8, 19 but also strongly related to the ability to distinguish salient and non-salient semantics.
4.3 How Do Data and Model Designs Affect Saliency Prediction?
Training data and model designs play crucial roles in determining how well deep saliency models perform [18]. To understand these roles better, we conduct a comparative analysis of the effeets of different training datasets and model architectures saliency prediction.
Figure compares the results of three DINet models trained on different datasets: SALICON [2], OSIE [17]. and MIT [16]. It reveals that the shifts in semantics weights are tightly coupled with the characteristics of the training data. For instance, the model trained on OSIE pays more attention to social cues and non-salient semantics, because the OSIE dataset collects attention semantic- rich stimuli with diverse social cues. Differently, the model trained on MIT assigns a less positive weight to social cues and a more negative weight to the vehicle category, which is likely due to the dataset's less emphasis on social semantics and the co-occurrence of vehicles and more salient objects. Therefore, differences in the semantic weights across models trained different datasets reflect the variations in the semantics presented the respective datasets.
Figure 4 compares the semantic weights of three different saliency models trained on the same SALICON [2] dataset: i.e., DINet [ SALICON D. and TranSalNet [1O]. While models place the highest weights on actions and the lowest weights ground, scene, and sky, the differences in model designs are reflected in their semantic weights. For instance, compared to the other models, TranSalNet considers clothing and text to be strongly salient but sky to be less non-salient. The results
DINet
SALICON
TranSalNet
1.00
0.75
0.50
0,25
0.50
-0.75
-1.00
action
social
clothing
vehiele
animal
text
color
other
socne
ground
sky
Figure 4: Semantic weights for different saliency models.

---

Wio fine-tuning
Single-epoch fine-tuning
Complete fine- fine-tuning
1.00
0.75
0.50
0.25
0.00
0.25
-0.50
0.75
-1.00
action
social
clothing
vehicle
animal
text
color
other
ground
sky
Figure 5: Evolution of semantic weights throughout the fine-tuning process.
shed light the relationships between the designs and behaviors of saliency models, and show the usefulness of our framework in helping researchers tailor their models for specific applications.
Despite the differences in model behaviors across datasets and models, all models excel at automati- cally identifying salient semantics, e.g., action and social cues, and differentiating foreground from background semantics. The observation is consistent with our findings in Section (42 and indicates that the ability to correlate semanties with saliency and quantify their positive/negativ contributions is a systematic advantage of deep saliency models.
4.4 How Does Fine-tuning Saliency Models Affect Semantic Weights?
The process of fine-tuning is crucial for training deep saliency models, highlighted in previous studies [9]. To better understand the evolution of feature weights during fine-tuning, we conduct experiments on models trained in three scenarios of fine-tuning. These scenarios include models with fixed ImageNet [52) features (W/o fine-tuning). models that are fine-tuned for single epoch (Single-epoch fine-tuning), and fine-tuned models with the best validation performance (Complete fine-tuning). We experiment with three different models, namely SALICON [9]. DINet [[I], and TranSalNet [T0]. and report the average results obtained from our experiments.
As shown on the left side of Figure models with fixed ImageNet features already have the ability to identify salient semanties, such action and social cues, highlighted with strong positive weights. This is consistent with previous findings 19] which validates the effectiveness of our approach.
When comparing the results of models with single-epoch fine-tuning (shown in the middle of Figure 5) to those without fine-tuning (shown on the left), we notice a significant shift in the negative weights. Specifically, while models with fixed ImageNet features identify scene-related semantics to contribute most negatively to saliency, after being fine-tuned for single epoch, the models now focus sky-related semantics and have weaker emphasis on scene-related semantics. This can be attributed to the fact that ImageNet features are learned from iconic images, which are inherently insensitive to the sky background. However, since the sky is strong indicator of low saliency values, is necessary to learn this semantic to achieve accurate saliency prediction (note that the largest performance gain also occurs during the first epoch).
Finally, when examining the fully-tuned models (shown on the right side of Figure 5). we find that semantic weights continue to evolve during the later epochs of fine-tuning. Unlike the first epoch which imposes larger changes on a few negative semantics, the subsequent fine-tuning mostly plays a role in refining the weights of a broader range of semantics, enabling the models to become more sensitive to salient cues (e.g. action and text) and continuously adjusts the weights of negative semantics (e.g. ground and other).
These observations offer comprehensive view of how deep saliency models progressively adapt ImageNet features through fine-tuning. They show that fine-tuning first concentrates on semantics with negative weights to saliency, which are not well captured in the pretrained features, and then gradually adjusts the relative weights of diverse semantics. Understanding the evolution of model behaviors during training can provide insights into optimizing the learning recipes for saliency models and enabling them to progressively encode the knowledge of diverse semantics.
7

---

autism
w/o autism
Positive
Negative
Neutral
1.0
social
action
0.6
vehicle
0.4
animal
0.2
color
0.0
social
action
clothing (b)
animal
0.5s
(a)
(c)
Figure 6: Semantic weights for characterizing the attention on three different settings. From left to right results for the attention of people with and without autism, attention stimuli eliciting different emotions, and the attention of different time periods. Figures share the same y-axis. Note that the results in the three figures are derived from different datasets selected based on the context of analyses.
4.5
Can Deep Saliency Models Capture Characteristics of Human A Attention?
Human attention influenced by several factors, such as the visual preferences of viewers, character- istics of visual stimuli, and temporal dynamics. We investigate the ability of DINet to capture the impact these factors by training on different attention data for each factor. We visualize results for the most discriminative semantics, and include the complete analysis in supplementary materials.
Firstly, we study the impact of visual preferences on atlention with two subject groups, i.e., people with autism and those without, using a dataset with attention data of subjects from the two groups [53]. We train a DINet model on each set compare their corresponding semantic weights. As shown in Figure o4, both models assign significant weights to social cues, but the model for the autism group has a considerably weaker emphasis on action cues. This can be linked to the deficits Additionally, the model for the autism group also assigns a strong positive weight to the "other" category. The specific objects highlighted in the category are related to gadgets, e.g., digital, illuminated, and metal (see supplementary materials for details), which is consistent with the findings about the special interests people with autism (53).
in joint attention for people with autism [54 55
Next, we explore the impact of stimuli with diverse characteristics with images exhibiting positive, negative, and neutral sentiments, using the EMOd dataset [23]. Figure shows that models trained on stimuli eliciting strong emotions (positive and negative) exhibit a higher emphasis on social and action cues (e.g.. face and nose, see our supplementary materials for details) than the one for neutral sentiment. Their behaviors align with previous findings that human attention generally prioritizes emotion-elicne elicitingg content over -mottiiioo--liiiiinnng content (e.g. clothing) [66 57.. We also identify that models for positive/negative sentiments place more focus on human-related objects (e.g., social cues and clothing) than objects less related humans (e.g., animals), which is driving factor for characterizing the emotion prioritization effect attention deployment.
Finally, we investigate the effects of temporal dynamics by conducting experiments on the CodeCharts K (25) dataset that provides attention annotations collected at 0.5, 3, and seconds. As depicted in Figure o shift of focus from dominantly salient cues (e.g., action and social cues) to more diverse semantics (e.g., vehicle, animal, and color) is depicted by the gradually increasing weights for the latter group. cues at the beginning of visual exposure (i.e., within 3 seconds) before broadening their focus.
because viewers usually engage their attention to the most salient
Overall, findings suggest that deep saliency models encode the fine-grained characteristics of diverse attention. They also validate the usefulness of our approach in revealing the discriminative patterns of attention and shed light how visual attention is influenced by various factors.
4.6
What Is Missing to Close the Gap Between Saliency Models and Human Attention?
have identified a collection of common mistakes saliency models by probing the predicted saliency maps. In this paper, we aim to complement these studies by analyz- ing the failure patterns within the intermediate inference process using the proposed factorization framework.
Previous studies 18 43

---

Distribution
Distribution Bases
Stimuli
Ground Truth
Prediction
Stimuli
Ground Truth
Prediction
of Bases
Successful Examples
Failure Examples
Figure 7: Successful (left) and failure (right) examples of deep saliency models. We visualize the predictions from DINet, as those for the other two models are similar. For the distribution of bases, from red blue, the probabilities of positive bases decrease while those for negative bases increase.
To conduct our analysis, we first select the common success and failure examples where three tested models (i.e., SALICON [9) DINet [1]], and TranSalNet (T0) consistently have high/low NSS scores [49] Then we perform qualitative analysis by visualizing the spatial probabilistic distribution (a
in Equation 3) of the bases for semantics with positive (red) and negative (blue) weights (see our supplementary materials for details), which is used to derive the final saliency maps.
In the successful examples (see the left panel of Figure 7. we find that accurate saliency prediction correlates with the differentiation of diverse semantics. Specifically, stimuli for these examples typically have salient and non-salient regions belonging to different semantics. Therefore, with the ability to distinguish positive and negative semantics (i.e with discriminative distributions of the corresponding bases), models can readily determine their saliency distribution.
However, in the failure examples (see the right panel of Figure 7). models commonly struggle to determine the saliency within objects (e.g., bicycle in the 1 failure example) among objects with similar semantics (e.g., elephants in the 2nd failure example). Investigation of the probabilistic distribution of bases shows that models typically have uniform-like distribution of bases on object parts or among objects of the same category, thus inherently incapable of determining the importance of the bases construct an accurate saliency map. We also note that existing models have difficulty with scenes without salient objects, which is illustrated in the 3rd failure examples with a relatively empty scene.
As shown in Figure 7 (second column of the right panel), the ground truth human attention of the failure patterns is scattered. We further look into the inter-subject variability of these cases and found that their inter-subject variability is high, suggesting that human viewers may not agree on where look, and therefore the ground truth maps are less valid. In this case, one assumption about saliency modeling (i.e., certain commonality about human attention patterns) may not be true, and the validity of using the ground truth human map for training and evaluation (i.e., the standard leave-one-sune -o--ssbjeec--out approach), and the expected behavior of targeted models are interesting and open questions.
Overall, while high-level semantics learned in existing deep saliency models are powerful. we hypothesize that leveraging more structured representations encode the contextual relationships between semantics, and integrating mid- and low-level cues will be helpful for accommodating challenging scenarios in saliency prediction.
4.7 Quantitative Evaluation of the Interpretable Model
The fundamental objective of our study is to develop principled framework for understanding the underlying mechanism behind deep saliency models without altering their inherent behaviors. For this, we only introduce minimal architectural modifications, limited to the last two layers of the saliency models, thereby ensuring that their performance aligns seamlessly with the original models across all datasets. To complement our aforementioned analyses and further substantiate the efficacy

---

OSIE 17
MIT [66
SALICON
12
NSS
CC
NSS
CC
NSS
CC
SALICON 9
2.75
0.63
2.56
0.70
1.89
0.86
SAM
2.70
0.65
2.47
0.69
1.84
0.86
DINet [1I
2.88
0.63
2.54
0.70
1.92
0,87
Ours
2.91
0.64
2.53
0.70
1.89
0.86
Table I: Comparative results of saliency prediction three popular datasets. Models evaluated with two common metrics, including Normalized Scanpath Saliency (NSS) Coefficient (CC) (50)
49 and Correlation
of our methodology, we quantitatively evaluate the saliency prediction performance of our method (using DINet backbone trained the SALICON training split example) three commonly used saliency datasets, including OSTE [17], MIT [16), and SALICON [12. Comparative results reported in Table [ demonstrate the competitive nature of our approach with respect ofate-of-the-art methods, and validate its effectiveness in achieving the balance between interpretability and model performance.
5 Conclusion, Limitations, and Broader Impacts
As deep saliency models excel in performance, is important understand the factors contributing to their successes. Our study introduces a novel analytic framework to explore how deep saliency models prioritize visual cues to predict saliency, which is crucial for interpreting model behavior and gaining insights into visual elements most influential in performance. We discover that the models' success can attributed to their accurate feature detection and their ability differentiate semanties with positive and negative weights. These semantic weights are influenced by various factors, such as the training data and model designs. Furthermore, fine-tuning the model advantageous, particularly in allocating suitable weights to non-salient semantics for optimal performance. Our framework also serves as a valuable tool for characterizing human visual attention diverse scenarios. Additionally, our study identifies common failure patterns in saliency models by examining inference processes, discusses challenges from both human and model attention, and suggests modeling with holistic incorporation of structures and lower-level information.
Despite advancing attention research from different perspectives, our work still has room for im- provement. Specifically, the current study focuses on visual attention deployment in real-world scenarios and employs the commonly used natural scenes as visual stimuli. We are aware that certain applications (e.g., graphical design and software development) may also involve artificial stimuli, such as advertisements, diagrams, and webpages, and extension of the proposed framework to broader domains can be straightforward and interesting.
We anticipate several positive impacts stemming from this research. Firstly, by advancing the understanding of deep saliency models, valuable insights can be applied to optimize interfaces for human-come computer interactions. This, in turm, will enhance the efficiency and reliability of the next generation of computer-aided systems, leading to improved user experiences and increased productivity. Secondly, the accurate capture of visual importance can have significant implications for individuals with visual impairments. Saliency models can effectively assist visually impaired individuals in navigating and interacting with both people and environments. This could empower them to engage more fully in various aspects of daily life, fostering independence and inclusivity. In sum, this comprehensive exploration of computational saliency modeling holds great potential for broader societal benefits.
Acknowledgements
This work is supported by NSF Grants 2143197 and 2227450.
10

---

References
[1] Neil Bruce and John Tsotsos. Saliency based information maximization. In NewrIPS, volume 18, 2005.
[2] Jonathan Harel, Christof Koch, and Pietro Perona. Graph-based visual saliency. In Advances in Neural Information Processing Systems, volume 19, MIT Press, 2006.
[3] Laurent and Christof Koch, A saliency-based search mechanism for overt and covert shifts visual attention, Vision Research, 40( 10): 108:1499-, 2000.
[4] Eleonora Vig, Michael Dorr, and David Cox. Large-scale optimization of hierarchical features for saliency prediction in natural images. In CVPR, pages 2798-28808 2014.
[5] Christof Koch and Shimon Ullman. Shifts in selective visual attention: Towards the underlying neural circuitry. Matters of Intelligence, 15-:11,,141, 1987.
[6] Anne M. Treisman and Garry Gelade. A feature-integration theory of attention. Cognitive Psychology, 12(1):97-136, 1980.
[7] Jeremy M Wolfe. Guided search 2.0: The upgrade. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 37(19):295--1999 1993.
[8] Marcella Cornia, Lorenzo Baraldi, Giuseppe Serra, and Rita Cucchiara. Predicting Human Eye Fixations via LSTM-based Saliency Attentive Model. TIP, 27(10))5142 5154, 2018.
[9] Xun Huang. Chengyao Shen, Xavier Boix, and Qi Zhao. Salicon: Reducing the semantic gap in saliency prediction by adapting deep neural networks. In ICCV, pages 262-270, 2015.
[10] Jianxun Lou, Hanhe Lin, David Marshall, Dietmar Saupe, and Hantao Liu. Transalnet: Towards perceptu- ally relevant visual saliency prediction. Neurocompucting 494:455-467, 2022.
[11] Sheng Yang. Guosheng Lin, Qiuping Jiang, and Weisi Lin. A dilated inception network for visual saliency prediction. TMM, 22(8) 2020.
[12] Ming Jiang, Shengsheng Huang, Juanyong Duan, and Qi Zhao. Salicon: Saliency in context. In CVPR, pages 1072-1080, 2015.
[13] Kaiming He, Xiangyu Zhang. Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 770- 778. 2016.
[14] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. NeurlPS, volume 25, 2012.
[15] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NewrIPS, volume 30, 2017.
[16] Tilke Judd, Krista Ehinger, Frédo Durand, and Antonio Torralba. Learning to predict where humans look. In ICCV, pages 2106-21 211,, 2009.
[17] Juan Xu, Ming Jiang, Shuo Wang. Mohan S. Kankanhalli, and Qi Zhao. Predicting human gaze beyond pixels. Journal of Vision, 14(1):1-20, 2014.
[18] Ali Borji. Saliency prediction in deep learning era: Successes and limitations. TPAMI, 43(2):679-700, 2021.
[19] Sen He. Hamed R. Tavakoli, Ali Borji, Yang Mi, and Nicolas Pugeault. Understanding and visualizing deep visual saliency models. In CVPR, pages 10198-1000 10207, 2019.
[20] Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalanditis, Li-Jia Li, David A Shamma, Michael Bernstein, and Li Fei-Fei. Visual genome: Connecting language and vision using crowdsourced dense image annotations. In IJCV, volume 123, pages 32-73, 2017
[21] Tilke Judd, Frédo Durand, and Antonio Torralba. A benchmark of computational models of saliency to predict human fixations. In MIT Technical Report, 2012.
[22] Ali Borji and Laurent Itti. Cat2000: A large scale fixation dataset for boosting saliency research. CVPR workshop, 2015.
11

---

[23] Shaojing Fan, Zhiqi Shen, Ming Jiang, Bryan L Koenig, Juan Xu, Mohan S Kankanhalli, and Qi Zhao. Emotional attention: study of image sentiment and visual attention. In CVPR, pages 7521-7531, 2018.
[24] Subramanian Ramanathan, Harish Katti, Nicu Sebe, Mohan Kankanhalli, and Tat-Seng Chua. An eye fixation datahase for saliency detection in images. In ECCV, pages 30-43. Springer, 2010.
[25] Camilo Fosco, Anclise Newman, Pat Sukhum, Yun Bin Zhang, Nanxuan Zhao, Aude Oliva, and Zoya Bylinskii. How much time do you have? Modeling multi-duration saliency. In CVPR, pages 4472-4481, 2020.
[26] Lai Jiang, Mai Xu, Tie Liu, Minglang Qiao, and Zulin Wang. Deepvs: A deep learning based video saliency prediction approach. In ECCV, pages 602-617, 2018.
[27] Wenguan Wang, Jianbing Shen, Fang Guo, Ming-Ming Cheng, and Ali Borji. Revisiting video saliency: large-scale benchmark and a new model. In CVPR, pages 4894 4903, 2018
[28] Zoya Bylinskii, Nam Wook Kim, Peter O' D'onovan, Sami Alsheikh. Spandan Madan, Hanspeter Pfister, Frédo Durand, Bryan C. Russell, and Aaron Hertzmann. Learning visual importance for graphic designs and data visualizations. Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology, 2017.
[29] Camilo Fosco, Vincent Casser, Amish Kumar Bedi, Peter O'Donovan, Aaron Hertzmann, and Zoya Bylinskii. Predicting visual importance across graphic design types. In Proceedings the 33rd Annual ACM Symposium on User Interface Software and Technology, page 249-260, 2020.
[30] Souradeep Chakraborty, Zijun Wei, Conor Kelton, Seoyoung Ahn, Aruna Balasubramanian, Gregory J. Zelinsky, and Dimitris Samaras. Predicting visual attention in graphic design documents. TMM, pages
1-1, 2022.
[3]] Chengyao Shen and Qi Zhao. Webpage saliency. In ECCV, pages 33-46, 2014.
[32] Ming Jiang, Juan Xu, and Qi Zhao. Saliency in crowd. ECCV, pages 17-32. Springer, 2014.
[33] Andrea Palazzi, Davide Abati, Francesco Solera, Rita Cucchiara, et al. Prodicting the driver's focus of attention: the dr (cye) ve project. TPAMI, 41(7):1720-177 2018.
[34] Vincent Sitzmann, Ana Serrano, Amy Pavel, Maneesh Agrawala, Diego Gutierrez, Belen Masia, and Gordon Wetzstein. Saliency in vr: How people explore virtual environments? IEEE Transactions on Visualization and Computer Graphics, 24(4):1633-1642, 2018.
[35] Ziheng Zhang, Yanyu Xu, Jingyi Yu, and Shenghua Gao. Saliency detection in 360 videos. ECCV, pages 504 520, 2018.
[36] L. Itti. C. Koch, 20(1 20(11):2244-1299 1998.
Niebur. model saliency- based visual attention for rapid scene analysis. TPAMI,
[37] Jaanming Zhang and Stan Sclaroff. Saliency detection: A boolean map approach. In ICCV, pages 153-160, 2013.
[38] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recogni- tion. ICLR, 2015.
[39] Eleonora Vig. Michael Dorr, and David Cox. Large-scale optimization of hierarchical features for saliency prediction in natural images. In CVPR, pages 2798-2805, 2014.
[40] Srinivas S. S. Kruthiventi, Kumar Ayush, and R. Venkatesh Babu, Deepfix: A fully convolutional neural network for predicting human eye fixations. /7IP, 2609):444644455 2017.
[41] Matthias Kümmerer, Thomas S. A. Wallis, and Matthias Bethge. Deepgaze II: reading fixations from deep features trained on object recognition. Arxiv, 2016.
[42] Ali Borji, Dicky N. Sihite, and Laurent Itti. Quantitative analysis of human-model agreement in visual saliency modeling: A comparative study. TIP, 22(1 22(1):55-66, 2013.
[43] Zoya Bylinskii, Adrià Recasens, Ali Borji, Aude Oliva, Antonio Torralba, and Frédo Durand, Where should saliency models look next? In ECCV, pages 809 824, 2016.
[44] Neil D. B. Bruce, Christopher Catton, and Sasa Janjic. A deeper look at saliency: Feature contrast, semantics, and beyond. In CVPR, pages 316-524, 2016.
12

---

[45] Iuliia Kotseruba, Calden Wloka, Amir Rasouli, and John K. Tsotsos. Do saliency models detect odd-one-out targets? new datasets and evaluations. In BMVC, 2019.
[46] Matthias Küimmerer, Thomas S.A. Wallis, Leon A. Gatys, and Matthias Bethge. Understanding low- and high-level contributions fixation prediction. In 2017 IEEE International Conference on Computer Vision (ICCV), pages 4799-4808, 2017.
[47] Bolei Zhou, Yiyou Sun, David Bau, and Antonio Torralba. Interpretable basis decomposition for visual explanation. In ECCV, pages 119 134, 2018.
[48] David Bau, Bolei Zhou, Aditya Khosla. Aude Oliva, and Antonio Torralba. Network dissection: Quantify- ing interpretability deep visual representations. CVPR, pages 3319-3327, 2017.
[49] Robert Peters, Asha Iyer, Laurent Itti, and Christof Koch. Components of bottom-up gaze allocation in natural images. Vision Research, 45(18):2397 2416, 2005.
[50] Olivier Le Meur, Patrick Le Callet, and Dominique Barba. Predicting visual fixations video based on low-level visual features. Vision Research, 47(11):2483 - 2498, 2007.
[51] Solomon Kullback. Information Theory and Statistics. Wiley, 1959.
[52] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248-255, 2009.
[53] Shuo Wang, Ming Jiang, Xavier Morin Duchesne, Elizabeth A. Laugeson, Daniel P. Kennedy, Ralph Adolphs, and Qi Zhao. Atypical visual saliency in autism spectrum disorder quantificd through model-based eye tracking. Neuron, 88(3):044 616, 2015.
[54] Julie Osterling and Geraldine Dawson. Early recognition of children with autism: A study of first birthday home videotapes. Journal of Autism and Developmental Disorders, 24(3):247-257, 1994.
[55] Morton Ann Gernsbacher, Jennifer L Stevenson, Suraiya Khandakar, and H Hill Goldsmith. Why does joint attention look atypical autism? Child Development Perspectives, 2(1):38-45 2008.
[56] Rehecca J. Compton. The interface between emotion and attention: review of evidence from psychology and neuroscience. Behavioral and Cognitive Neuroscience Reviews, 122:115--29, 2003.
[57] Patrik Vuilleumier. How brains beware: neural mechanisms of emotional attention. Trends in Cognitive Sciences, 9:585-594, 2005.
13


Reviews:
[{"review_id": "fhY3ARyR8u", "reviewer": "Reviewer_u4m5", "summary": "The paper presents a novel analytic framework that provides a principled interpretation and quantification of the implicit features learned by deep saliency models, which are used for predicting human visual attention. The framework decomposes these features into interpretable bases aligned with semantic attributes, and reformulates saliency prediction as a weighted combination of probability maps. The authors conducted extensive analyses to understand the factors contributing to the success of saliency models, including the positive and negative weights of semantics, the impact of training data and architectural designs, and the effects of fine-tuning. They also explored visual attention in various application scenarios, such as autism spectrum disorder, emotion-eliciting stimuli, and attention evolution over time. The study identifies the accurate feature detection and differentiation of semantics as key factors in the models' success, influenced by training data and design choices. The framework is also useful for characterizing human visual attention and understanding common failure patterns in saliency models. The authors suggest incorporating structures and lower-level information for improved modeling. The research has potential impacts on optimizing human-computer interfaces, assisting visually impaired individuals, and enhancing societal benefits.", "questions": "I am curious to know if the authors could perform and share the results of comparison with existing methods using the standard metrics for measuring saliency.", "limitations": "The authors analyzed the failure patterns of deep saliency models within the intermediate inference process using their proposed factorization framework. They select common success and failure examples where three tested models consistently have high/low NSS scores. They then perform a qualitative analysis by visualizing the spatial probabilistic distribution of the bases for semantics with positive and negative weights.\n\nIn the successful examples, they find that accurate saliency prediction is correlated with the differentiation of diverse semantics. The stimuli in these examples typically have salient and non-salient regions belonging to different semantics. Therefore, the models, with the ability to distinguish positive and negative semantics, can readily determine the saliency distribution. On the other hand, in the failure examples, the models struggle to determine saliency within objects or among objects with similar semantics. Investigation of the probabilistic distribution of bases reveals that models often have a uniform-like distribution of bases on object parts or among objects of the same category, making it difficult to construct accurate saliency maps.\n\nThe analysis also highlights that existing models have difficulty with scenes without salient objects. These scenes are challenging for the models as they lack clear focal points. The ground truth human attention in these failure patterns exhibits high inter-subject variability, suggesting that human viewers may not agree on where to look, making the ground truth maps less reliable. The authors raise the question of whether models may perform as well as or even better than humans in these challenging situations.\n\nBased on their observations, the authors hypothesize that leveraging more structured representations to encode contextual relationships between semantics and integrating mid- and low-level cues may be beneficial in addressing these failure patterns in certain scenarios. This suggests that incorporating additional information and incorporating contextual understanding may help improve the performance of deep saliency models.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper presents a novel analytic framework that provides interpretation and quantification of the implicit features learned by deep saliency models for predicting human visual attention. The framework decomposes implicit features into interpretable bases aligned with semantic attributes, allowing for a weighted combination of probability maps connecting the bases and saliency in saliency prediction. The framework effectively identifies a variety of semantics learned by deep saliency models, including social cues, actions, clothing, and salient object categories, showcasing its versatility in analyzing attention across diverse scenarios.\n2. The framework reveals the positive and negative contributions of semantics to saliency, highlighting the ability of deep saliency models to distinguish between salient and non-salient semantics.\n3. The analysis demonstrates how training data and model designs impact saliency prediction, with shifts in semantic weights reflecting the characteristics of the datasets and models.\n4. The study also investigates the effects of fine-tuning on semantic weights, showing how deep saliency models progressively adapt features during training to better capture salient cues and refine the weights of negative semantics.\n5. The framework is also applied to explore the capture of human attention characteristics, such as the impact of visual preferences, characteristics of visual stimuli (e.g., emotions), and temporal dynamics, providing insights into the factors influencing attention deployment.\n6. The findings validate the effectiveness of deep saliency models in automatically identifying salient semantics, differentiating foreground from background, and encoding fine-grained characteristics of attention.", "weaknesses": "The work has a lot of novelty with good empirical analyses. The paper is also nicely written. However, I find that the authors have missed comparing the performance of their model with other saliency models using standard saliency metrics as can be found in this leaderboard: http://saliency.mit.edu/results_mit300.html.\n\nThe authors have also failed to cite some important works from like DeepGaze, DeepFix, BMS, etc.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**1. Q**: The work has a lot of novelty with good empirical analyses, but has missed comparing the performance of their model. \n\n**R**:  We acknowledge the importance of performance comparison with existing methods using standard metrics for measuring saliency, and have added comparisons in the global response. Results show that our approach is able to achieve competitive performance among three popular datasets. We are committed to including these comparisons and results in the revised version of the paper.\n\n**2. Q**: Cite some important works from like DeepGaze, DeepFix, BMS, etc.\n\n**R**: We appreciate the suggestion to highlight relevant works. These references will certainly enrich our discussion on diverse saliency techniques and we will add them in the revision."}, {"review_id": "CtfWuv6K6Z", "reviewer": "Reviewer_ZMfD", "summary": "This paper attempts to decompose the learned representation of a data-driven saliency model into a constituent set of bases that are mapped onto semantic concepts, thereby providing insight into what is driving the model's representation of saliency. This method is applied to three different saliency models of varying formulation over several datasets. Some discussion is then provided of the results, as well as some qualitative discussion of examples of model failures.", "questions": "- Although drawing on the architectures of SALICON, DINet, and TranSalNet, the model configurations in this paper are distinct from the instantiations of the original publications in order to accommodate the need for trainable bases. How does that change the ultimate behaviour of the models with respect to standard measures of saliency performance? Alternatively, even a quantified value for the change in saliency maps when compared within model (e.g. the paper's version of SALICON correlated against the standard instantiation of SALICON) would help put the paper's results in context with the existing literature.\n\n- What is the value of N (the number of bases)? How stable are the results with respect to N (i.e. does the mapped semantic content change substantially with even small changes in N)?\n\n- Each basis is mapped onto a top-5 semantic mix. Why not onto a single concept? How was 5 selected? Similar to the previous question, how does this choice affect the subsequent analysis?\n\n- How is \"action\" defined in a static image? The primary example given, \"having meeting\", seems like a social activity, which was a separate category. I get that it is challenging to relate the messy details of semantic categorization in a short paper, but given that this is central to the topic of the paper, I think it needs a clearer explanation.\n\n- How do the insights provided in this paper relate to model performance? When the models show a markedly different breakdown of salient factors (e.g. Figure 4, which shows SALICON emphasizing vehicles much more strongly than TranSalNet, while TranSalNet emphasizes clothing more than any other model), does this correlate with predictive accuracy?\n\n- Related to the previous question, could you use the IoU Measurement process used to assign labels to the bases to provide an approximate breakdown of the factors leading to human fixations directly? This might provide another point of comparison to better put the results of this paper in context.\n\n- Given the range of behaviours across the models shown in Figure 4, why does Figure 5 average the semantic weights across models (also, this should be noted in the caption; when I first read the paper I was quite confused which model was being shown in Figure 5)? What is the justification for this? Do the models tend to converge after fine-tuning?\n\n- Figure 6 (b.) and (c.): are these results for subjects without autism? If so, this should be more clearly noted.\n\n- What was meant by line 304: \"Whether models... open question.\"? What would lead to a model behaving \"even better than humans\", given that humans are the system trying to be modelled?", "limitations": "The limitations discussed seem clear.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The paper takes on a challenging and open-ended problem in saliency modelling, namely the difficulty of teasing apart the different contributions to attentional capture. Similarly, the work provides an example of continued work in general explainability within deep learning methods, which is an important issue within the field.\n\nThe paper is clearly written and relatively easy to follow, although there are a few details that are missing (such as the choice of N).", "weaknesses": "There are some references that seem pertinent that were not discussed. In particular, significant work on failure modes of modern saliency models was poorly represented.\n- Bruce et al., \"A Deeper Look at Saliency: Feature Contrast, Semantics, and Beyond\", CVPR 2016\n  -- This paper digs into some of the failure modes common to deep learning models, including object vs. background and semantic vs. feature contrast elements (e.g. see Figure 7), providing pertinent insights to the discussion in the submission.\n- Kümmerer et al., \"Understanding Low- and High-Level Contributions to Fixation Prediction\", ICCV 2017\n  -- This paper explicitly explores constituent elements of saliency representation in deep networks from the perspective of high-level vs. low-level features. Given the way the paper attempts to tease apart the representation of saliency between different feature classes, it is conceptually highly relevant background for the current submission.\n- Kotseruba et al., \"Do Saliency Models Detect Odd-One-Out Targets? New Datasets and Evaluations\", BMVC 2019\n  -- This paper provides a psychophysical (P^3) and natural image (O^3) dataset with targets explicitly defined by low-level salient features (e.g. colour, orientation, shape, or size singletons), and finds that saliency models (including deep learning-based models) largely perform quite poorly. For exploring failure rates the O^3 dataset would be a potentially useful (albeit ground truth was defined by semantic object annotation and not fixation data), but even if the dataset is not used the examination of model failures in the submission should include the context of this prior exploration.\n- Tatler et al., \"Visual correlates of fixation selection: effects of scale and time\", Vision Research, 2005\n  -- This paper explores the evolution of fixations through time, including aspects such as central vs. peripheral distribution and inter-subject consistency of fixation location. Given that this is one area that the submission claims novelty, it would be good to put it in context with prior explorations in this area of the temporal evolution of low-level human attention.\n\nOverall, while the paper is interesting and tackles an exceedingly challenging problem, I think there are a number of conceptual issues that it needs to overcome. While some specific issues are given in the questions below, the primary issue is that while the submission encodes the positive/negative importance of the various bases extracted, it is well established within psychophysics that the relative importance of elements to saliency is contextual (e.g. see Nothdurft (1993) \"Saliency effects across dimensions in visual search\"; Nothdurft (2000) \"Salience from feature contrast: additivity across dimensions\" for some low level examples), and so these relative attributes are likely to change from image to image. Within the current submission, these attributes change from model to model and dataset to dataset; what conclusions are to be drawn from this? Is the technique shedding light on dataset composition, model bias, or some deeper aspect of relative aspects of saliency? Much of the analysis is presented without clear connection to either human behaviour (with the exception of Section 4.5, which cleverly makes use of the technique to explore the representations learned from data from different human subject populations or conditions) or model performance in a traditional sense, making it difficult to put into context or derive deeper insight.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**1. Q**: The relative importance of elements to saliency is contextual. These attributes change from model to model and dataset to dataset; what conclusions are to be drawn from this? Is the technique shedding light on dataset composition, model bias, or some deeper aspect of relative aspects of saliency?\n\n**R**: We agree that the relative importance of elements to saliency is indeed contextual, as demonstrated in the referred psychophysics research on low-level features (Nothdurft, 1993 and 2000) as well as recent ones including higher-level semantics [17, 19, 41]. Instead of investigating individual images, this study aims to reveal general conclusions from learning-based methods based on a large-scale dataset featuring diverse objects in natural contexts. For example, a small far-away pedestrian face in a street image can be less salient than a nearby car, but with many images with faces of different sizes and locations co-occurring with various objects in natural contexts, faces show overall high saliency values. This allows us to make statistical inferences regarding the relative significance of semantics, quantifying and visualizing their contributions to saliency prediction. It further enables a collection of interesting analyses and important conclusions including (1) differentiating the positive and negative contributions of semantics is critical to saliency prediction; (2) deep saliency models learn key properties of attention in different settings (different participant groups, stimuli, and time durations), mostly aligned with findings in human vision studies; and (3) common failures of deep saliency models can be attributed to the inability to differentiate semantics.\n\nThe reviewer is right that  “the technique is shedding light on dataset composition, model bias…”. While we highlight the general conclusions, the proposed framework can also be applied to different data or models to reveal certain data- or model-specific conclusions or biases, so readers may be aware of the differences and select them accordingly. For example, Section 4.3 analyzes the same model trained on different datasets and reveals the data characteristics based on semantic weights.\n\n**2. Q**: Some references were not discussed.\n\n**R**: The references are indeed relevant. Our work complements them by differentiating high-level semantics based on their contributions and understanding the general mechanisms of deep saliency models. We will include these papers and add a dedicated section to discuss more comprehensively about failure modes, saliency representation, and temporal evolution in saliency models. \n\n**3. Q**: How does the method change the saliency performance?\n\n**R**: We include the results in the global comment, showing that our method does not compromise saliency performance.\n\n**4. Q**: What is the value of N?\n\n**R**: We set N based on the number of units in the final layers of deep saliency models [9-11]. We explored two settings of N (512 and 1000) and found that their differences in contributions were not substantial. Therefore, we proceed with N=1000, which strikes a balance between granularity and computational efficiency. \n\n**5. Q**: Why not project bases onto a single concept? \n\n**R**: This approach resonates with [42] that uses the top k concepts for interpreting image classification. The top-1 semantic alone may only explain ~30% of the alignment score (sum of normalized IoU). Alternatively, the top-5 semantics account for 90-100% of the score, capturing a broader range of contributing semantics while avoiding an overemphasis on dominant salient/non-salient semantics (e.g., face and cloudness). \n\n**6. Q**: How are semantic categories like action defined?\n\n**R**: Our categorization of semantics considers both their association with objects and meanings. For categories related to actions, we first identify human-centric social semantics (i.e., objects and attributes related to humans), and then distinguish actions (e.g. doing something) from non-actions (e.g., body parts). We also offer detailed results for each semantic in the supplementary materials.\n\n**7. Q**: How do the insights relate to model performance?\n\n**R**:  The paper sheds light on how deep saliency models prioritize visual cues to predict saliency, which is crucial for interpreting model behavior and gaining insights into visual elements most influential in performance: For example, Figure 5 analyzes the evolution of semantic weights through fine-tuning where performance increases with epochs. The fine-tuning improves the weight difference between salient and non-salient semantics, leading to enhanced model performance. \n\n**8. Q**: Could you use the IoU Measurement directly for factorization?\n\n**R**: Directly using IoU for factorization and employing the cosine similarity $\\alpha$ as a measure of contribution (which is a form of machine attention) may not quantify the actual contributions of bases (different attention weights lead to the same prediction, see “Attention is not Explanation, 2019”). We address the issue by reformulating saliency prediction with a probabilistic framework, to explicitly measure the contributions and reveal deeper insights into saliency models. \n\n**10. Q**: Are the results in Fig. 6bc for subjects without autism?\n\n**R**: The two subfigures analyze the effects of stimuli and time duration on the observers’ attention, which are unrelated to autism. We will provide clearer explanations to avoid misunderstanding.\n\n**11. Q**: What was meant by line 304?\n\n**R**:  When inter-subject variability is high, human observers do not have a consensus on where to look. In this case, one assumption about saliency modeling (i.e., certain commonality about human attention patterns) may not be true, and the validity of using the ground truth human map for training and evaluation (i.e., the standard leave-one-subject-out approach), and the expected behavior of targeted models are interesting and open questions. "}, {"review_id": "GOXW9Ljp27", "reviewer": "Reviewer_srbe", "summary": "This paper examines the problem of predicting visual saliency in images. Unlike many other works, it focuses on determining what leads to the predictions made, including underlying features that are learned, and formulating the prediction as a combination of bases. Through this, one is able to garner an understanding of how the models make their decisions and attach this to semantic information or other concepts.", "questions": "1. If possible, can you comment on how this formalism for saliency prediction extends to other domains?\n\n2. In figure 5, it seems that a single epoch of fine tuning is closer to careful fine tuning than the original weights. Is this from using a relatively large learning rate?\n\n3. Related to the above, how does this vary with changes to the learning rate?\n", "limitations": "The authors have addressed limitations of their approach in Section 5 of the paper.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Strengths of this paper are as follows:\n1. It approaches saliency from a different point of view; rather than competing to marginally beat an ROC score, it takes a step towards truly understanding the success of deep learning models in this domain\n2. By explicitly modelling features correlated with attention, and those inversely correlated with attention, one can model both what is salient and what is not in a way that allows these concepts to be quantified\n3. Because the model constructs it's prediction from a set of bases, this effectively allows human attention to be modelled subject to a change to these bases, as demonstrated in an example involving autism.", "weaknesses": "Ultimately, it would be nice to see some quantitative results on how well the model performs compared to some of the state-of-the-art models, although I understand this is not the point of the paper. Nevertheless, a detailed set of results along this dimension could add value but I wouldn't insist on it.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**1. Q**: It would be nice to see some quantitative results on how well the model performs.\n\n**R**: We thank the reviewer for the suggestion and include the results in the global comment, which shows that our model achieves state-of-the-art performance.\n\n**2. Q**: Can you comment on how this formalism for saliency prediction extends to other domains?\n\n**R**: The key components of our framework, such as feature factorization and probabilistic inference, are general and adaptable, which can be readily extended to other domains and applications. For instance, our method can be directly applied to similar regression tasks in other domains, e.g., visual aesthetic estimation, to gain insights into the contributions of different semantics to the predicted scores. It can also be extended to classification tasks such as image classification, and help identify the common/distinct semantics for different classes. For this, we can adjust our paradigm based on the general methodology discussed in [42], and interpret the relationship between object classes and fine-grained concepts by iteratively analyzing the contributions of semantics for predicting each class.\n\n**3. Q**: It seems that a single epoch of fine-tuning is closer to careful fine-tuning than the original weights. Is this from using a relatively large learning rate? How do the results vary with the learning rate?\n\n**R**: In general, the shift of semantic contributions in saliency prediction models during fine-tuning is larger in the first epoch compared to the subsequent epochs. This is because the pre-trained ImageNet features provide a good initialization of model weights for saliency models to converge quickly. The learning rate used in our experiments is 1e-4, following the learning recipe of the original paper [11]. We also experimented with a learning rate of 4e-4 and observed similar convergence behaviors and results."}, {"review_id": "s95lzd0VmU", "reviewer": "Reviewer_UKcT", "summary": "This paper proposes a new framework, which can decompose the learned features of a saliency model into trainable bases (using [42]), those bases are combined to formulate the final saliency map, and the weight of the combination indicates the contribution of each basis. The semantic meaning of each basis can be explored by matching the bases to a probe dataset (Visual Genome is used in this draft), Thus, saliency can be studied based on visual concepts, .e.g, different types of visual relationship, or objects, exploring what the key factors are behind the saliency region. ", "questions": "See above", "limitations": "Exploring unknown bases to a dataset is questionable, which requires further study. The finding of those bases may not explain saliency, but the biases of the training and probing datasets.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "This draft is well written, the organization is clear, and the method is explained in details. The proposed method attempts to solve a fundamental and important question, what features are important to saliency.\n\n", "weaknesses": "I like this study attempts to solve a very interesting question, however, some of the designs may need more discussion. This study is based on the existing basis de-composition[42] technique. The main contribution is to link those bases to other visual concepts by computing thresholded IOU. This design heavily relies on the probing dataset, which makes the hyper-parameter threshold a little tricky. What if other datasets are used, will the bases also match other concepts, how one can validate the matched concept (objects or events) is valid. Some studies attempt to explore the learned feature based on known probing concepts, e.g., designing a dataset specifically for texture and shape concepts(Geirhos, Robert, et al. 2018). Exploring visual concepts (bases) by matching to other datasets is not very convincing. Should one exhaustively match to all concepts to find the best match (I highly doubt if this is feasible or how to define the universe), or setting a threshold and matching to a “local”’ solution(a specific dataset)? Thus, matching bases to a specific dataset is not very solid to show what the bases are. \n\nMore generally, assuming we can find the “ground-truth” (the true meaning of those bases), I believe saliency is way more complicated than simply based on visual concepts. From Figure 2, we can see that street sign is less important than faces. We can imagine the street sign could be the most salient region if the picture only contains a street sign in the middle. Given two images, one has a face in the middle and one has a street sign in the middle, they could be equally salient. Thus, one explanation for figure 2 could also be it shows the bias of the dataset which used to train the model, more faces appeared in that dataset. There also could be biases in the probe dataset, Visual Genome in this study. One concept, say “having meeting”, is more salient could be the visual features happen to match one of the bases. The real most salient basis does not exist in that dataset. \n\nAnother problem in this study is that saliency may not depend on the semantic. Imagine we have ten faces in a picture, the middle could be the most salient, while the others are not. Thus, study which visual basis is the most salient is not convincing. This may also happen to the Visual Genome dataset. The region for the visual relation is the union of the head and the tail, which is larger than the two objects. The bigger union region may better correlate with the well-known centre bias regardless of what the actual content. This is why “action” is the most salient in all of the datasets in Figure 3.\n\nOverall, applying basis decomposition [42] is not new, the finding of the semantic importance given saliency may simply showing dataset biases.\nGeirhos, Robert, et al. \"ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness.\" arXiv preprint arXiv:1811.12231 (2018).\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**1. Q**: What if other datasets are used, will the bases also match other concepts? Should one exhaustively match all concepts to find the best match? \n\n**R**: We appreciate the thoughtful questions raised about the potential limitations of matching bases to a single dataset. We agree that the matching of bases and concepts may depend on specific datasets or thresholds used, and it is challenging to define or exhaust all concepts in the universe. Our study follows the common practice of leveraging big datasets with key concepts at scale [42, 43]. In particular, we use Visual Genome, the biggest and most diverse naturalistic dataset with a sufficiently large number of images (100,000 images) and fine-grained annotations, providing a rich and diverse set of objects, scenes, attributes, relationships, and contexts. Our approach automatically captures salient and non-salient semantics (Figure 2) that agree with previous studies [19, 40, 41], as well as discriminative attention patterns [49-52] in various contexts (e.g., for different participant groups, stimuli, and time durations in Figure 6). These findings validate the effectiveness of our method in drawing key insights into saliency.\n\n**2. Q**: I believe saliency is way more complicated than simply based on visual concepts. Given two images, one has a face in the middle and one has a street sign in the middle, they could be equally salient. \n\n**R**: Saliency suggests relative importance and thus the respective capability in attracting attention when multiple objects or semantics co-occur in a scene.  We agree with the reviewer that it is dependent on other factors such as the context of an image and the locations of the objects. We also agree that with a face in the middle and a street sign in the middle, both are salient in their individual contexts. It is in fact difficult to understand the relative importance of objects or semantics with iconic images with one or few dominant objects in the center, and this is indeed the reason that we leverage datasets with images that (1) have multiple objects co-occurring in images, and (2) include diverse objects in naturalistic context; so the confounder effects such as center bias are neutralized to some degree, and statistical conclusions of their importance can be derived. Therefore, instead of looking at saliency in an individual image, our framework derives conclusions from a more global perspective from a large set of naturalistic images with various objects and contexts.\n\n**3. Q**: One explanation for Figure 2 could also be it shows the bias of the dataset which used to train the model, more faces appeared in that dataset.\n\n**R**: Our method uses the average IoU score across diverse images annotated with a specific concept to measure its importance in Equation 5, which accounts for the different frequencies of concepts. Therefore, higher frequencies of a concept do not necessarily result in higher importance. For instance, the \"mouth\" has an equivalent frequency to \"face\" but is assigned much lower importance, the “floor” has fifteen times more frequency than “cloudless” but their importance shows the opposite.\n\n\n**4. Q**: The finding of those bases may not explain saliency, but the biases of the training and probing datasets.\n\n**R**: We acknowledge that both the training dataset and the probing dataset may introduce biases in the saliency analysis. However, the large-scale Visual Genome dataset mitigates the risk of biases from limited data samples present in smaller-scale studies. This approach helps in understanding the general strategies employed by deep saliency models for capturing attention across diverse visual contexts. As highlighted in Section 4.5, our method results in meaningful results that are consistent with human vision studies [49-52], validating its effectiveness in explaining saliency instead of biases.\n\n**5. Q**: The region for the visual relation is the union of the head and the tail, which is larger than the two objects. The bigger union region may better correlate with the well-known center bias regardless of the actual content.\n\n**R**: Our approach takes advantage of two key strategies to counter center biases and the size of semantics, and thus ensure a faithful measurement of the alignment between the probabilistic distribution of a semantic and the segmentation of different semantics.\n\nFirst, size-insensitive measurement. We follow the general methodology of [43] and measure the alignment with the IoU score and adaptive thresholding (i.e., thresholds that capture the top 20% quantile level of distributions for different bases). This method ensures that the alignment score will be high only if the probabilistic distribution of the basis aligns with the majority of the segmentation mask, regardless of the average size of the semantics. For instance, on average, a \"train\" is over seven times larger than a \"face\", but our approach correctly recognizes the significantly higher importance of \"face\" in Figure 2.\n\nSecond, the choice of dataset.  Unlike many image datasets (e.g., ImageNet and MIT) focusing on iconic images where few objects are centered, Visual Genome used in our study is designed to encompass a rich set of semantic concepts in the same scene. Due to a more balanced distribution of visual features, the most salient object is not always at the center of the scene. Therefore, this design alleviates center biases.\n"}]

Author rebuttal:
We thank the reviewers for their insightful feedback. We are encouraged that they recognize our work as solving a fundamental, important, challenging, and open-ended problem, which approaches explainability and takes a step towards truly understanding the success of deep learning models (ZMfD, srbe, UKcT). We appreciate that they find our work with a lot of novelty and good empirical analysis, on multiple aspects including both salient and non-salient semantics, data and model impact, and the exploration of human attention characteristics (u4m5, ZMfD, srbe, UKcT). All reviewers identify the importance of feature decomposition and explicit modeling of the connections between bases and semantics, indeed two key components of our framework for bridging the gap between visual semantics and saliency. We are also glad that they consider our paper nicely written (u4m5, ZMfD, UKcT).

The key objective of our study is to develop a principled framework for interpreting and understanding the underlying mechanism behind deep saliency models, rather than designing saliency models that strive for enhanced performance. As pointed out by Reviewer srbe, instead of competing to marginally beat an ROC score, it approaches saliency from a different point of view and works toward a true understanding of deep saliency models. With this objective in mind, by explicitly quantifying the contributions of diverse semantics, our approach provides key insights into the impacts of different factors (e.g., datasets, model architecture, learning process in Section 4.2-4.4) on saliency prediction and attention deployments in various settings (e.g., different participant groups, stimuli with diverse sentiment, and varying temporal dynamics in Section 4.5) and also complements studies on model design [9, 10, 11] with an interpretable tool for investigating the behaviors of models (e.g., Section 4.6).

Although not the focused point of the paper as acknowledged by the reviewers, our approach also shows promise in achieving competitive performance in saliency prediction. To complement our analyses in the main paper and further substantiate the efficacy of our methodology in accurately interpreting saliency models without altering their inherent behaviors, we follow reviewers' suggestions and include an additional performance comparison. The table below shows the comparative performance across three commonly used datasets: OSIE, MIT, and SALICON (our model is trained using the SALICON training split with a DINet backbone). These results demonstrate the competitive nature of our approach compared with state-of-the-art techniques. It is noteworthy that our approach introduces minimal architectural modifications, limited to the last two layers of the saliency models (see Section 4.1 for details), thereby ensuring that its performance aligns seamlessly with the original DINet model across all datasets.


| | | CC | NSS |
|----------|----------|----------|----------|
| OSIE  | SALICON| 0.63 | 2.75 |
| | SAM | 0.65 | 2.70 |
| | DINet| 0.63 | 2.88 |
| |  Ours (DINet) | 0.64 | 2.91 |
| MIT | DVA| 0.64 | 2.38 |
| | SALICON | 0.70 | 2.56 |
| | SAM| 0.69 | 2.47  |
| | DINet | 0.70 | 2.54 |
| |  Ours (DINet)  | 0.70 | 2.53 |
| SALICON | DeepNet| 0.86 | 1.61 |
| | SAM | 0.86 | 1.84 |
| | SALICON | 0.86 | 1.89 |
| | UNISAL | 0.88 | 1.95 |
|  | EML-Net | 0.87 | 1.95 |
| | DINet  | 0.87 | 1.92 |
|    | Ours (DINet) | 0.86 | 1.89 |

We address the questions (**Q**) raised by each reviewer in the individual responses (**R**) below, and will incorporate the comments in the revision.


Meta-review: Paper received 1 x Accept, 2 x Weak Accept and 1 x Borderline Reject ratings. A number of concerns were raised in the initial reviews. However, provided rebuttal largely addressed concerns of [ZMfD] and [u4m5] as noted in their post-rebuttal discussion; [srbe] did not participate in discussion but was already positive about the paper pre-rebuttal. Reviewer [UKcT] engaged in lengthy discussion with the authors but found the responses only partially convincing at the end. Remaining concerns of [UKcT] were carefully reviewed and discussed by AC and SAC. While valid, they were deemed not significant enough to prevent acceptance of the paper. Ultimately, AC agrees with the consensus that paper presents interesting and valuable findings and would be a valuable addition to the conference. Authors are encouraged to revise the manuscript to include discussion and additional results provided in the rebuttal.
Decision: Accept (poster)