Auditory Saliency Using Natural Statistics

Auditory Saliency Using Natural Statistics Tomoki Tsuchida (ttsuchida@ucsd.edu) Garrison W. Cottrell (gary@ucsd.edu) Department of Computer Science and Engineering 9500 Gilman Drive, Mail Code 0404 La Jolla CA 92093-0404 USA Abstract In contrast to the wealth of saliency models in the vision lit- erature, there is a relative paucity of models exploring audi- tory saliency. In this work, we integrate the approaches of (Kayser, Petkov, Lippert, & Logothetis, 2005) and (Zhang, Tong, Marks, Shan, & Cottrell, 2008) and propose a model of auditory saliency. The model combines the statistics of natural soundscapes and the recent past of the input signal to predict the saliency of an auditory stimulus in the frequency domain. To evaluate the model output, a simple behavioral experiment was performed. Results show the auditory saliency maps cal- culated by the model to be in excellent accord with human judgments of saliency. Keywords: attention; saliency map; audition; auditory percep- tion; environmental statistics Introduction In general, attention plays a very important role in the survival of an organism, by separating behaviorally relevant signals from irrelevant ones. One approach to understanding how attention functions in the brain is to consider the “saliency map” over the sensory input space, which may determine sub- sequent motor control targets or selectively modulate percep- tual contrast thresholds. The brain’s putative saliency maps can be thought of as interest operators that organisms use to enhance or filter sensory signals. Many visual saliency models have been investigated, but relatively little attention has been paid to modeling auditory saliency. However, since the fundamental necessity for per- ceptual modulation remains the same regardless of modal- ity, the principles of visual saliency models should apply equally well to auditory saliency with appropriate sensory input features. Two representative visual saliency models are the center-surround contrast model (Itti, Koch, & Niebur, 2002) and the SUN (Salience Using Natural Statistics) model (Zhang et al., 2008). Itti et al.’s model is neurally-inspired, with the response of many feature maps (e.g., orientation, motion, color) combined to create a salience map. The SUN model uses a single feature map learned using Indepen- dent Components Analysis (ICA) of natural images, and the salience at any point is based on the rarity of the feature re- sponses at that point - novelty attracts attention. Here, rarity is based on statistics taken from natural images, so the model assumes experience is necessary to represent novelty. Previous works that apply the visual saliency paradigm to the auditory domain include the models of (Kayser et al., 2005) and (Kalinli & Narayanan, 2007). Both adapt the visual saliency model of (Itti et al., 2002) to the auditory domain by using spectrographic images as inputs to the model. Al- though this is a reasonable approach, these models fail to cap- ture several important aspects of the auditory modality. First, this approach treats time as simply another dimension within the spectrographic representation of the sound. Even though these models utilize asymmetric temporal filters, the resulting saliency map at each time point is contaminated by informa- tion from the future. Second, spectrographic features are not the most realistic representations of human auditory sensa- tions, since the cochlea exhibits complex nonlinear responses to sound signals (Lyon, Katsiamis, & Drakakis, 2010). Fi- nally, Itti et al.’s model determines the saliency values from the current input signal, with no contribution from the life- time experience of the organism. This makes it impossible for the model to account for potential perceptual differences induced by differences in individual experience. The Auditory Saliency Model In this work, we propose the Auditory Salience Using Natural statistics model (ASUN) as an extension of the SUN model. The extension involves (1) using realistic auditory features instead of visual ones, and (2) combining long-term statis- tics (as in SUN) with short-term, temporally local statistics. Although the SUN model has both a top-down, task-based component and a bottom-up, environmentally driven compo- nent, here we restrict ourselves to just the bottom-up portion of SUN. SUN defines the bottom-up saliency of point x in the image at time t as: s x (t) ∝ − log P(F x = f x ) Here, f is a vector of feature values, whose probability is computed based on prior experience. This is also known as the “self-information” of the features, and conveys that rare feature values will attract attention. In the SUN model, this probability is based on the lifetime experience of the organ- ism, meaning that the organism already knows when feature values are common and when they are rare. Assuming the primary purpose of attention is to separate remarkable events from the humdrum, it is logical to equate the rarity of the event with the saliency of it. For example, a loud bang may be salient not only because of its physical energy content, but also because of its relative rarity in the soundscape. An or- ganism living under constant noise may not find an explosion to be as salient as another organism acclimated to a quieter environment.