Early Visual Processing of Feature Saliency Tasks: A Review of Psychophysical Experiments

The visual system is constantly bombarded with information originating from the outside world, but it is unable to process all the received information at any given time. In fact, the most salient parts of the visual scene are chosen to be processed involuntarily and immediately after the first glance along with endogenous signals in the brain. Vision scientists have shown that the early visual system, from retina to lateral geniculate nucleus (LGN) and then primary visual cortex, selectively processes the low-level features of the visual scene. Everything we perceive from the visual scene is based on these feature properties and their subsequent combination in higher visual areas. Different experiments have been designed to investigate the impact of these features on saliency and understand the relative visual mechanisms. In this paper, we review the psychophysical experiments which have been published in the last decades to indicate how the low-level salient features are processed in the early visual cortex and extract the most important and basic information of the visual scene. Important and open questions are discussed in this review as well and one might pursue these questions to investigate the impact of higher level features on saliency in complex scenes or natural images.

Related papers:

title: Mechanisms of visual attention in the human cortex.
abstract: A typical scene contains many different objects that, because of the limited processing capacity of the visual system, compete for neural representation. The competition among multiple objects in visual cortex can be biased by both bottom-up sensory-driven mechanisms and top-down influences, such as selective attention. Functional brain imaging studies reveal that, both in the absence and in the presence of visual stimulation, biasing signals due to selective attention can modulate neural activity in visual cortex in several ways. Although the competition among stimuli for representation is ultimately resolved within visual cortex, the source of top-down biasing signals derives from a network of areas in frontal and parietal cortex.

title: Focal attention produces spatially selective processing in visual cortical areas V1, V2, and V4 in the presence of competing stimuli.
abstract: 1. The activity of single neurons was recorded in Macaca mulatta monkeys while they performed tasks requiring them to select a cued stimulus from an array of three to eight stimuli and report the orientation of that stimulus. Stimuli were presented in a circular array centered on the fixation target and scaled to place a single stimulus element within the receptive field of the neuron under study. The timing of the cuing event permitted the directing of visual attention to the spatial location of the correct stimulus before its presentation. 2. The effects of focal attention were examined in cortical visual areas V1, V2, and V4, where a total of 672 neurons were isolated with complete studies obtained for 94 V1, 74 V2, and 74 V4 neurons with receptive-field center eccentricities in the range 1.8-8 degrees. Under certain conditions, directed focal attention results in changes in the response of V1, V2, and V4 neurons to otherwise identical stimuli at spatially specific locations. 3. More than one-third of the neurons in each area displayed differential sensitivity when attention was directed toward versus away from the spatial location of the receptive field just before and during stimulus presentation. Both relative increases and decreases in neural activity were observed in association with attention directed at receptive-field stimuli. 4. The presence of multiple competing stimuli in the visual field was a major factor determining the presence or absence of differential sensitivity. About two-thirds of the neurons that were differentially sensitive to the attending condition in the presence of competing stimuli were not differentially sensitive when single stimuli were presented in control studies. For V1 and V2 neurons the presence of only a few (3-4) competing stimuli was sufficient for a majority of the neurons studied; a majority of the V4 neurons required six to eight stimuli in the array before significant differences between attending conditions occurred. 5. For V1 and V2 neurons the neuronal sensitivity differences between attending conditions were observed primarily at or near the peak of the orientation tuning sensitivity for each neuron; the differences were evident over a broader range of orientations in V4 neurons. 6. In conclusion, neural correlates of focal attentive processes can be observed in visual cortical processing in areas V1 and V2 as well as area V4 under conditions that require stimulus feature analysis and selective spatial processing within a field of competing stimuli.(ABSTRACT TRUNCATED AT 400

title: The Brain's Visual World: Representation of Visual Targets in Cerebral Cortex
abstract: Microelectrode recordings from behaving monkeys have shown that neuronal responses in the visual cerebral cortex can depend greatly on which aspect of the scene is the target of the animal's attention. Accumulating evidence suggests that while the early stages of the visual pathway provide a faithful representation of the retinal image, later stages of processing in the visual cortex hold representations that emphasize the viewer's current interest. By filtering out irrelevant signals and adding information about objects whose presence is remembered or inferred, the cortex creates an edited representation of the visual world that is dynamically modified to suit the immediate goals of the viewer.

title: The Psychology of Attention
abstract: Part 1 Attention and perception: selective attention divided attention attentional set capacity and selection - theorizing about attention. Part 2 Attention, memory and action: central processing limitations in sensorimotor tasks attention and memory automaticity, effort and control.

title: What attributes guide the deployment of visual attention and how do they do it?
abstract: As you drive into the centre of town, cars and trucks approach from several directions, and pedestrians swarm into the intersection. The wind blows a newspaper into the gutter and a pigeon does something unexpected on your windshield. This would be a demanding and stressful situation, but you would probably make it to the other side of town without mishap. Why is this situation taxing, and how do you cope?

title: Visual attention: The past 25 years
abstract: This review focuses on covert attention and how it alters early vision. I explain why attention is considered a selective process, the constructs of covert attention, spatial endogenous and exogenous attention, and feature-based attention. I explain how in the last 25 years research on attention has characterized the effects of covert attention on spatial filters and how attention influences the selection of stimuli of interest. This review includes the effects of spatial attention on discriminability and appearance in tasks mediated by contrast sensitivity and spatial resolution; the effects of feature-based attention on basic visual processes, and a comparison of the effects of spatial and feature-based attention. The emphasis of this review is on psychophysical studies, but relevant electrophysiological and neuroimaging studies and models regarding how and where neuronal responses are modulated are also discussed.

title: Guided search for triple conjunctions
abstract: A key tenet of feature integration theory and of related theories such as guided search (GS) is that the binding of basic features requires attention. This would seem to predict that conjunctions of features of objects that have not been attended should not influence search. However, Found (1998) reported that an irrelevant feature (size) improved the efficiency of search for a Color × Orientation conjunction if it was correlated with the other two features across the display, as compared to the case in which size was not correlated with color and orientation features. We examined this issue with somewhat different stimuli. We used triple conjunctions of color, orientation, and shape (e.g., search for a red, vertical, oval-shaped item). This allowed us to manipulate the number of features that each distractor shared with the target (sharing) and it allowed us to vary the total number of distractor types (and, thus, the number of groups of identical items: grouping). We found that these triple conjunction searches were generally very efficient—producing very shallow Reaction Time × Set Size slopes, consistent with strong guidance by basic features. Nevertheless, both of the variables, sharing and grouping, modulated performance. These influences were not predicted by previous accounts of GS; however, both can be accommodated in a GS framework. Alternatively, it is possible, though not necessary, to see these effects as evidence for “preattentive binding” of conjunctions.

title: Visual attention: the where, what, how and why of saliency
abstract: Attention influences the processing of visual information even in the earliest areas of primate visual cortex. There is converging evidence that the interaction of bottom-up sensory information and top-down attentional influences creates an integrated saliency map, that is, a topographic representation of relative stimulus strength and behavioral relevance across visual space. This map appears to be distributed across areas of the visual cortex, and is closely linked to the oculomotor system that controls eye movements and orients the gaze to locations in the visual scene characterized by a high salience.

title: Stimulus Saliency Modulates Pre-Attentive Processing Speed in Human Visual Cortex
abstract: The notion of a saliency-based processing architecture  underlying human vision is central to a number of current theories of visual selective attention [e.g., 2]. On this view, focal-attention is guided by an overall-saliency map of the scene, which integrates (sums) signals from pre-attentive sensory feature-contrast computations (e.g., for color, motion, etc.). By linking the Posterior Contralateral Negativity (PCN) component to reaction time (RT) performance, we tested one specific prediction of such salience summation models: expedited shifts of focal-attention to targets with low, as compared to high, target-distracter similarity. For two feature-dimensions (color and orientation), we observed decreasing RTs with increasing target saliency. Importantly, this pattern was systematically mirrored by the timing, as well as amplitude, of the PCN. This pattern demonstrates that visual saliency is a key determinant of the time it takes for focal-attention to be engaged onto the target item, even when it is just a feature singleton.

title: Fast and Conspicuous? Quantifying Salience With the Theory of Visual Attention
abstract: Particular differences between an object and its surrounding cause salience, guide attention, and improve performance in various tasks. While much research has been dedicated to identifying which feature dimensions contribute to salience, much less regard has been paid to the quantitative strength of the salience caused by feature differences. Only a few studies systematically related salience effects to a common salience measure, and they are partly outdated in the light of new findings on the time course of salience effects. We propose Bundesen’s Theory of Visual Attention (TVA) as a theoretical basis for measuring salience and introduce an empirical and modeling approach to link this theory to data retrieved from temporal-order judgments. With this procedure, TVA becomes applicable to a broad range of salience-related stimulus material. Three experiments with orientation pop-out displays demonstrate the feasibility of the method. A 4th experiment substantiates its applicability to the luminance dimension.

title: Complementary effects of gaze direction and early saliency in guiding fixations during free viewing.
abstract: Gaze direction provides an important and ubiquitous communication channel in daily behavior and social interaction of humans and some animals. While several studies have addressed gaze direction in synthesized simple scenes, few have examined how it can bias observer attention and how it might interact with early saliency during free viewing of natural and realistic scenes. Experiment 1 used a controlled, staged setting in which an actor was asked to look at two different objects in turn, yielding two images that differed only by the actor's gaze direction, to causally assess the effects of actor gaze direction. Over all scenes, the median probability of following an actor's gaze direction was higher than the median probability of looking toward the single most salient location, and higher than chance. Experiment 2 confirmed these findings over a larger set of unconstrained scenes collected from the Web and containing people looking at objects and/or other people. To further compare the strength of saliency versus gaze direction cues, we computed gaze maps by drawing a cone in the direction of gaze of the actors present in the images. Gaze maps predicted observers' fixation locations significantly above chance, although below saliency. Finally, to gauge the relative importance of actor face and eye directions in guiding observer's fixations, in Experiment 3, observers were asked to guess the gaze direction from only an actor's face region (with the rest of the scene masked), in two conditions: actor eyes visible or masked. Median probability of guessing the true gaze direction within ±9° was significantly higher when eyes were visible, suggesting that the eyes contribute significantly to gaze estimation, in addition to face region. Our results highlight that gaze direction is a strong attentional cue in guiding eye movements, complementing low-level saliency cues, and derived from both face and eyes of actors in the scene. Thus gaze direction should be considered in constructing more predictive visual attention models in the future.

title: The role of temporal structure in human vision.
abstract: Gestalt psychologists identified several stimulus properties thought to underlie visual grouping and figure/ground segmentation, and among those properties was common fate: the tendency to group together individual objects that move together in the same direction at the same speed. Recent years have witnessed an upsurge of interest in visual grouping based on other time-dependent sources of visual information, including synchronized changes in luminance, in motion direction, and in figure/ ground relations. These various sources of temporal grouping information can be subsumed under the rubric temporal structure. In this article, the authors review evidence bearing on the effectiveness of temporal structure in visual grouping. They start with an overview of evidence bearing on temporal acuity of human vision, covering studies dealing with temporal integration and temporal differentiation. They then summarize psychophysical studies dealing with figure/ground segregation based on temporal phase differences in deterministic and stochastic events. The authors conclude with a brief discussion of neurophysiological implications of these results.

title: Modeling perceptual grouping and figure-ground segregation by means of active reentrant connections.
abstract: The segmentation of visual scenes is a fundamental process of early vision, but the underlying neural mechanisms are still largely unknown. Theoretical considerations as well as neurophysiological findings point to the importance in such processes of temporal correlations in neuronal activity. In a previous model, we showed that reentrant signaling among rhythmically active neuronal groups can correlate responses along spatially extended contours. We now have modified and extended this model to address the problems of perceptual grouping and figure-ground segregation in vision. A novel feature is that the efficacy of the connections is allowed to change on a fast time scale. This results in active reentrant connections that amplify the correlations among neuronal groups. The responses of the model are able to link the elements corresponding to a coherent figure and to segregate them from the background or from another figure in a way that is consistent with the so-called Gestalt laws.

title: Visual features that vary together over time group together over space
abstract: The visual system perceives objects as coherent even when some parts are hidden or discontinuous. How this representation is constructed from local features of many nearby objects is termed the 'binding problem.' Here we manipulate contrast in several drifting gratings that can be perceived as either independent objects or parts of a single object. Contrast modulations that are correlated in time enhance perceptual coherence, whereas uncorrelated modulations impair coherence. Presumably, correlated contrast modulations produce correlated responses in cortical neurons. Therefore, our results are consistent with the hypothesis that temporal correlation of neural activity is important for feature binding.

title: Age, target-distractor similarity, and visual search.
abstract: Younger and older adults were asked to find a single target in both feature- and conjunction- search conditions. Display size varied between 2 and 8 items, and target-distractor similarity ranged from relatively low to high levels. The accuracy data indicated that older adults had particular difficulty finding targets in high-similarity conjunction-search displays containing a large number of distractors. The reaction time (RT) analyses found larger age deficits in many of these same conditions. For both groups, predictions of conjunction search based on Treisman and Sato's additive model (Journal of Experimental Psychology: Human Perception and Performance, 1990: 16, 459-478) departed significantly from actual performance. The RT data of older observers were, in large part, predicted as a simple linear function of the young adults' data. These results are discussed with respect to age differences in selective attention, generalized slowing, and an age-related loss in search efficiency.

title: An inter-item similarity model unifying feature and conjunction search
abstract: We propose a model of visual search to address the hitherto unresolved issue of reconciling serial deployment of attention accounts with inter-item similarity effects. Target-distractor and distractor-distractor similarity were systematically varied in 85 (17x5) set type-size conditions over seven experiments, including univariate feature and bivariate conjunction search. The model, a power (square root) function of dimension-specific target-distractor and distractor-distractor similarity in linear combination with set size, accounted for 98% of the variance on type-size means. It suggests that much of efficient and inefficient search can be unified under a single theory involving item similarity.

title: Effects of part-based similarity on visual search: The Frankenbear experiment
abstract: Do the target-distractor and distractor-distractor similarity relationships known to exist for simple stimuli extend to real-world objects, and are these effects expressed in search guidance or target verification? Parts of photorealistic distractors were replaced with target parts to create four levels of target-distractor similarity under heterogenous and homogenous conditions. We found that increasing target-distractor similarity and decreasing distractor-distractor similarity impaired search guidance and target verification, but that target-distractor similarity and heterogeneity/homogeneity interacted only in measures of guidance; distractor homogeneity lessens effects of target-distractor similarity by causing gaze to fixate the target sooner, not by speeding target detection following its fixation.

title: Salience-Based Selection: Attentional Capture by Distractors Less Salient Than the Target
abstract: Current accounts of attentional capture predict the most salient stimulus to be invariably selected first. However, existing salience and visual search models assume noise in the map computation or selection process. Consequently, they predict the first selection to be stochastically dependent on salience, implying that attention could even be captured first by the second most salient (instead of the most salient) stimulus in the field. Yet, capture by less salient distractors has not been reported and salience-based selection accounts claim that the distractor has to be more salient in order to capture attention. We tested this prediction using an empirical and modeling approach of the visual search distractor paradigm. For the empirical part, we manipulated salience of target and distractor parametrically and measured reaction time interference when a distractor was present compared to absent. Reaction time interference was strongly correlated with distractor salience relative to the target. Moreover, even distractors less salient than the target captured attention, as measured by reaction time interference and oculomotor capture. In the modeling part, we simulated first selection in the distractor paradigm using behavioral measures of salience and considering the time course of selection including noise. We were able to replicate the result pattern we obtained in the empirical part. We conclude that each salience value follows a specific selection time distribution and attentional capture occurs when the selection time distributions of target and distractor overlap. Hence, selection is stochastic in nature and attentional capture occurs with a certain probability depending on relative salience.

title: Modeling the Effects of Perceptual Load: Saliency, Competitive Interactions, and Top-Down Biases
abstract: A computational model of visual selective attention has been implemented to account for experimental findings on the Perceptual Load Theory (PLT) of attention. The model was designed based on existing neurophysiological findings on attentional processes with the objective to offer an explicit and biologically plausible formulation of PLT. Simulation results verified that the proposed model is capable of capturing the basic pattern of results that support the PLT as well as findings that are considered contradictory to the theory. Importantly, the model is able to reproduce the behavioral results from a dilution experiment, providing thus a way to reconcile PLT with the competing Dilution account. Overall, the model presents a novel account for explaining PLT effects on the basis of the low-level competitive interactions among neurons that represent visual input and the top-down signals that modulate neural activity. The implications of the model concerning the debate on the locus of selective attention as well as the origins of distractor interference in visual displays of varying load are discussed.

title: On the Temporal Relation of Top–Down and Bottom–Up Mechanisms during Guidance of Attention
abstract: Two mechanisms are said to be responsible for guiding focal attention in visual selection: bottom–up, saliency-driven capture and top–down control. These mechanisms were examined with a paradigm that combined a visual search task with postdisplay probe detection. Two SOAs between the search display and probe onsets were introduced to investigate how attention was allocated to particular items at different points in time. The dynamic interplay between bottom–up and top–down mechanisms was investigated with ERP methodology. ERPs locked to the search displays showed that top–down control needed time to develop. N2pc indicated allocation of attention to the target item and not to the irrelevant singleton. ERPs locked to probes revealed modulations in the P1 component reflecting top–down control of focal attention at the long SOA. Early bottom–up effects were observed in the error rates at the short SOA. Taken together, the present results show that the top–down mechanism takes time to guide focal attention to the relevant target item and that it is potent enough to limit bottom–up attentional capture.

title: Forced to remember: When memory is biased by salient information
abstract: The last decades have seen a rapid growing in the attempt to understand the key factors involved in the internal memory representation of the external world. Visual salience have been found to provide a major contribution in predicting the probability for an item/object embedded in a complex setting (i.e., a natural scene) to be encoded and then remembered later on. Here I review the existing literature highlighting the impact of perceptual- (based on low-level sensory features) and semantics-related salience (based on high-level knowledge) on short-term memory representation, along with the neural mechanisms underpinning the interplay between these factors. The available evidence reveal that both perceptual- and semantics-related factors affect attention selection mechanisms during the encoding of natural scenes. Biasing internal memory representation, both perceptual and semantics factors increase the probability to remember high- to the detriment of low-saliency items. The available evidence also highlight an interplay between these factors, with a reduced impact of perceptual-related salience in biasing memory representation as a function of the increasing availability of semantics-related salient information. The neural mechanisms underpinning this interplay involve the activation of different portions of the frontoparietal attention control network. Ventral regions support the assignment of selection/encoding priorities based on high-level semantics, while the involvement of dorsal regions reflects priorities assignment based on low-level sensory features.

title: Parietal cortex integrates contextual and saliency signals during the encoding of natural scenes in working memory
abstract: The Brief presentation of a complex scene entails that only a few objects can be selected, processed indepth, and stored in memory. Both low‐level sensory salience and high‐level context‐related factors (e.g., the conceptual match/mismatch between objects and scene context) contribute to this selection process, but how the interplay between these factors affects memory encoding is largely unexplored. Here, during fMRI we presented participants with pictures of everyday scenes. After a short retention interval, participants judged the position of a target object extracted from the initial scene. The target object could be either congruent or incongruent with the context of the scene, and could be located in a region of the image with maximal or minimal salience. Behaviourally, we found a reduced impact of saliency on visuospatial working memory performance when the target was out‐of‐context. Encoding‐related fMRI results showed that context–congruent targets activated dorsoparietal regions, while context–incongruent targets de‐activated the ventroparietal cortex. Saliency modulated activity both in dorsal and ventral regions, with larger context‐related effects for salient targets. These findings demonstrate the joint contribution of knowledge‐based and saliency‐driven attention for memory encoding, highlighting a dissociation between dorsal and ventral parietal regions.

title: Bayesian surprise attracts human attention
abstract: We propose a formal Bayesian definition of surprise to capture subjective aspects of sensory information. Surprise measures how data affects an observer, in terms of differences between posterior and prior beliefs about the world. Only data observations which substantially affect the observer's beliefs yield surprise, irrespectively of how rare or informative in Shannon's sense these observations are. We test the framework by quantifying the extent to which humans may orient attention and gaze towards surprising events or items while watching television. To this end, we implement a simple computational model where a low-level, sensory form of surprise is computed by simple simulated early visual neurons. Bayesian surprise is a strong attractor of human attention, with 72% of all gaze shifts directed towards locations more surprising than the average, a figure rising to 84% when focusing the analysis onto regions simultaneously selected by all observers. The proposed theory of surprise is applicable across different spatio-temporal scales, modalities, and levels of abstraction.

title: Bottom-up and top-down attention are independent.
abstract: What is the relationship between top-down and bottom-up attention? Are both types of attention tightly interconnected, or are they independent? We investigated this by testing a large representative sample of the Dutch population on two attentional tasks: a visual search task gauging the efficiency of top-down attention and a singleton capture task gauging bottom-up attention. On both tasks we found typical performance--i.e., participants displayed a significant search slope on the search task and significant slowing caused by the unique, but irrelevant, object on the capture task. Moreover, the high levels of significance we observed indicate that the current set-up provided very high signal to noise ratios, and thus enough power to accurately unveil existing effects. Importantly, in this robust investigation we did not observe any correlation in performance between tasks. The use of Bayesian statistics strongly confirmed that performance on both tasks was uncorrelated. We argue that the current results suggest that there are two attentional systems that operate independently. We hypothesize that this may have implications beyond our understanding of attention. For instance, it may be that attention and consciousness are intertwined differently for top-down attention than for bottom-up attention.

title: Dissociating the effects of similarity, salience, and top-down processes in search for linearly separable size targets
abstract: In two experiments, we explored the role of foreknowledge on visual search for targets defined along the size continuum. Targets were of large, medium, or small size and of high or low similarity relative to the distractors. In Experiment 1, we compared search for known and unknown singleton feature targets as a function of their size and similarity to the distractors. When distractor similarity was high, target foreknowledge benefited targets at the end of the size continuum (i.e., large and small) relatively more than targets of medium size. In Experiment 2, participants were given foreknowledge of what the target was not. The beneficial effect of foreknowledge for endpoint targets was reduced. The data indicate the role of top-down templates in search, which can be “tuned” more effectively for targets at the ends of feature dimensions.

title: Saliency and context play a role in infants’ texture segmentation
abstract: We investigated whether young infants orient reliably towards more salient vs. less salient objects in a visual scene. Subjects were tested with stimuli presented on textured fields, one side showing a target stimulus (a 'more salient' or 'less salient' texture patch) and the other a background stimulus. Infants typically preferred the more salient, but not the less salient target. Their behaviour depended on the configuration of the background stimulus. In contrast, 3-4 year-old children always showed a preference for the target stimulus, regardless of the configuration of the background. We conclude that both saliency of a target stimulus and its context play a role in early texture segmentation.

title: Neural Activities in V1 Create a Bottom-Up Saliency Map
abstract: The bottom-up contribution to the allocation of exogenous attention is a saliency map, whose neural substrate is hard to identify because of possible contamination by top-down signals. We obviated this possibility using stimuli that observers could not perceive, but that nevertheless, through orientation contrast between foreground and background regions, attracted attention to improve a localized visual discrimination. When orientation contrast increased, so did the degree of attraction, and two physiological measures: the amplitude of the earliest (C1) component of the ERP, which is associated with primary visual cortex, and fMRI BOLD signals in areas V1-V4 (but not the intraparietal sulcus). Significantly, across observers, the degree of attraction correlated with the C1 amplitude and just the V1 BOLD signal. These findings strongly support the proposal that a bottom-up saliency map is created in V1, challenging the dominant view that the saliency map is generated in the parietal cortex.

title: Superior colliculus encodes visual saliency before the primary visual cortex
abstract: Theories of visual attention postulate the existence of a saliency map that guides attention/gaze toward the most visually conspicuous stimuli in complex scenes. This study compared saliency coding in the two dominant visual gateways: the primary visual cortex (V1) and the evolutionarily older visual system that exists in the midbrain superior colliculus. Our results show that neurons in the superficial visual layers of the superior colliculus (SCs) encoded saliency earlier and more robustly than V1 neurons. This was surprising, because the dominant input to the SCs arises from V1. This result is in line with models that place a feature processing stage (V1) before the feature-agnostic saliency map in SCs. Models of visual attention postulate the existence of a bottom-up saliency map that is formed early in the visual processing stream. Although studies have reported evidence of a saliency map in various cortical brain areas, determining the contribution of phylogenetically older pathways is crucial to understanding its origin. Here, we compared saliency coding from neurons in two early gateways into the visual system: the primary visual cortex (V1) and the evolutionarily older superior colliculus (SC). We found that, while the response latency to visual stimulus onset was earlier for V1 neurons than superior colliculus superficial visual-layer neurons (SCs), the saliency representation emerged earlier in SCs than in V1. Because the dominant input to the SCs arises from V1, these relative timings are consistent with the hypothesis that SCs neurons pool the inputs from multiple V1 neurons to form a feature-agnostic saliency map, which may then be relayed to other brain areas.

title: A feature-integration theory of attention
abstract: A new hypothesis about the role of focused attention is proposed. The feature-integration theory of attention suggests that attention must be directed serially to each stimulus in a display whenever conjunctions of more than one separable feature are needed to characterize or distinguish the possible objects presented. A number of predictions were tested in a variety of paradigms including visual search, texture segregation, identification and localization, and using both separable dimensions (shape and color) and local elements or parts of figures (lines, curves, etc. in letters) as the features to be integrated into complex wholes. The results were in general consistent with the hypothesis. They offer a new set of criteria for distinguishing separable from integral features and a new rationale for predicting which tasks will show attention limits and which will not.

title: Computational modelling of visual attention
abstract: Five important trends have emerged from recent work on computational models of focal visual attention that emphasize the bottom-up, image-based control of attentional deployment. First, the perceptual saliency of stimuli critically depends on the surrounding context. Second, a unique 'saliency map' that topographically encodes for stimulus conspicuity over the visual scene has proved to be an efficient and plausible bottom-up control strategy. Third, inhibition of return, the process by which the currently attended location is prevented from being attended again, is a crucial element of attentional deployment. Fourth, attention and eye movements tightly interplay, posing computational challenges with respect to the coordinate system used to control attention. And last, scene understanding and object recognition strongly constrain the selection of attended locations. Insights from these five key areas provide a framework for a computational and neurobiological understanding of visual attention.

title: Mechanisms of top-down attention
abstract: Attention exhibits characteristic neural signatures in brain regions that process sensory signals. An important area of future research is to understand the nature of top-down signals that facilitate attentional guidance towards behaviorally relevant locations and features. In this review, we discuss recent studies that have made progress towards understanding: (i) the brain structures and circuits involved in attentional allocation; (ii) top-down attention pathways, particularly as elucidated by microstimulation and lesion studies; (iii) top-down modulatory influences involving subcortical structures and reward systems; (iv) plausible substrates and embodiments of top-down signals; and (v) information processing and theoretical constraints that might be helpful in guiding future experiments. Understanding top-down attention is crucial for elucidating the mechanisms by which we can filter sensory information to pay attention to the most behaviorally relevant events.

title: What stands out in a scene? A study of human explicit saliency judgment
abstract: Eye tracking has become the de facto standard measure of visual attention in tasks that range from free viewing to complex daily activities. In particular, saliency models are often evaluated by their ability to predict human gaze patterns. However, fixations are not only influenced by bottom-up saliency (computed by the models), but also by many top-down factors. Thus, comparing bottom-up saliency maps to eye fixations is challenging and has required that one tries to minimize top-down influences, for example by focusing on early fixations on a stimulus. Here we propose two complementary procedures to evaluate visual saliency. We seek whether humans have explicit and conscious access to the saliency computations believed to contribute to guiding attention and eye movements. In the first experiment, 70 observers were asked to choose which object stands out the most based on its low-level features in 100 images each containing only two objects. Using several state-of-the-art bottom-up visual saliency models that measure local and global spatial image outliers, we show that maximum saliency inside the selected object is significantly higher than inside the non-selected object and the background. Thus spatial outliers are a predictor of human judgments. Performance of this predictor is boosted by including object size as an additional feature. In the second experiment, observers were asked to draw a polygon circumscribing the most salient object in cluttered scenes. For each of 120 images, we show that a map built from annotations of 70 observers explains eye fixations of another 20 observers freely viewing the images, significantly above chance (dataset by Bruce and Tsotsos (2009); shuffled AUC score 0.62±0.07, chance 0.50, t-test p<0.05). We conclude that fixations agree with saliency judgments, and classic bottom-up saliency models explain both. We further find that computational models specifically designed for fixation prediction slightly outperform models designed for salient object detection over both types of data (i.e., fixations and objects).

title: Saliency-Based Color Accessibility
abstract: Perception of color varies markedly between individuals because of differential expression of photopigments in retinal cones. However, it has been difficult to quantify the individual cognitive variation in colored scene and to predict its complex impacts on the behaviors. We developed a method for quantifying and visualizing information loss and gain resulting from individual differences in spectral sensitivity based on visual salience. We first modeled the visual salience for color-deficient observers, and found that the predicted losses and gains in local image salience derived from normal and color-blind models were correlated with the subjective judgment of image saliency in psychophysical experiments, i.e., saliency loss predicted reduced image preference in color-deficient observers. Moreover, saliency-guided image manipulations sufficiently compensated for individual differences in saliency. This visual saliency approach allows for quantification of information extracted from complex visual scenes and can be used as an image compensation to enhance visual accessibility by color-deficient individuals.

title: Salience from feature contrast: temporal properties of saliency mechanisms
abstract: Single cell recordings in area V1 of the macaque monkey had suggested that saliency effects from orientation contrast might be delayed compared to the representation of other stimulus properties. This conjecture was tested in three series of experiments on regular line patterns. Experiment 1 investigated the time courses of saliency effects evoked either by the onset of a single line or by a target that popped out from orientation contrast. Saliency effects from orientation contrast developed later than saliency effects related to stimulus onset. Experiment 2 measured the detectability of such targets in brief presentations. As expected, single line targets were detected at shorter presentation times than popout targets with orientation contrast. Experiment 3 finally investigated the temporal resolution of saliency effects from feature contrast in different dimensions. Line arrays with a popout target (e.g. an orthogonal line) were alternated with complementary line arrays in which the target and the non-target features were exchanged (e.g. all lines were orthogonal to those in the previous pattern). Thus, although feature contrast was present in every single stimulus display, saliency effects could only develop when alternation rates were slow enough to be resolved by the underlying saliency mechanisms. Feature flicker of this sort was tested in orientation, motion (direction), color and luminance. Saliency mechanisms encoding orientation contrast were slower than those encoding differences in luminance or color; motion contrast produced intermediate results that also differed between subjects.

title: Quantifying object salience by equating distractor effects
abstract: It is commonly believed that objects viewed in certain contexts may be more or less salient. Measurements of salience have usually relied on asking observers "How much does this object stand out against the background?". In this study, we measured the salience of objects by assessing the distraction they produce for subjects searching for a different, pre-specified target. Distraction was measured through response times, but changes in response times were not assumed to be a linear measure of distracting potency. The analysis rested on measuring the effects of varying disparities--in size, luminance, and both-between a target object, a key distractor, and other background items. Our results indicate: (1) object salience defined by luminance or size disparity is determined by the ratio between its defining feature value and the corresponding feature value of background items; this finding is congenial to Weber's law for discrimination thresholds. (2) If we define salience as the logarithm of a feature value ratio, then salience increases approximately as fast due to increase in area as due to increase in luminance. (3) The sum of salience arising from object-background disparity in both size and luminance is larger than their vector sum (orthogonal vectors), but smaller than their scalar sum.

title: Global Precedence, Spatial Frequency Channels, and the Statistics of Natural Images
abstract: A great deal of evidence suggests that early in processing, retinal images are filtered by parallel, spatial frequency selective channels. We attempt to incorporate this view of early vision with the principle of global precedence, which holds that Gestalt-like processes sensitive to global image configurations tend to dominate local feature processing in human pattern perception. Global precedence is inferred from the pattern of reaction times observed when visual patterns contain multiple cues at different levels of spatial scale. Specifically, it is frequently observed that global processing times are largely unaffected by conflicting local cues, but local processing times are substantially lengthened by conflicting global cues. The asymmetry of these effects suggests the dominant role of global configurations. Since global spatial information is effectively represented by low spatial frequencies, global precedence potentially implies a low frequency dominance. The thesis is that low spatial frequencies tend to be available before information carried by higher frequency bands, producing a coarse-to-fine temporal order in visual spatial perception. It is suggested that a variety of factors contribute to the prior entry of low frequency information, including the high contrast gain of the magnocellular pathway, the amplitude spectra typical of natural images, and inhibitory interactions between the parallel frequency-tuned channels. Evidence suggesting a close relationship between global precedence and spatial frequency channels is provided by observations that the essential features of the global precedence effect are obtained using patterns consisting of low and high frequency sinusoids. The hypothesis that these asymmetric interference effects are due to interactions between parallel spatial channels is supported by an analysis of reaction times (RTs), which shows that RTs to redundant low and high frequency cues produce less facilitation than predictions that assume the channels are independent. In view of previous work showing that global precedence depends upon the low frequency content of the stimuli, we suggest that low spatial frequencies represent the sine qua non for the dominance of configurational cues in human pattern perception, and that this configurational dominance reflects the microgenesis of visual pattern perception. This general view of the temporal dynamics of visual pattern recognition is discussed, is considered from an evolutionary perspective, and is related to certain statistical regularities in natural scenes. Potential adaptive advantages of an interactive parallel architecture that confers an initial processing advantage to low resolution information are explored.

title: Size tuning in the absence of spatial frequency tuning in object recognition
abstract: How do we attend to objects at a variety of sizes as we view our visual world? Because of an advantage in identification of lowpass over highpass filtered patterns, as well as large over small images, a number of theorists have assumed that size-independent recognition is achieved by spatial frequency (SF) based coarse-to-fine tuning. We found that the advantage of large sizes or low SFs was lost when participants attempted to identify a target object (specified verbally) somewhere in the middle of a sequence of 40 images of objects, each shown for only 72 ms, as long as the target and distractors were the same size or spatial frequency (unfiltered or low or high bandpassed). When targets were of a different size or scale than the distractors, a marked advantage (pop out) was observed for large (unfiltered) and low SF targets against small (unfiltered) and high SF distractors, respectively, and a marked decrement for the complementary conditions. Importantly, this pattern of results for large and small images was unaffected by holding absolute or relative SF content constant over the different sizes and it could not be explained by simple luminance- or contrast-based pattern masking. These results suggest that size/scale tuning in object recognition was accomplished over the first several images (<576 ms) in the sequence and that the size tuning was implemented by a mechanism sensitive to spatial extent rather than to variations in spatial frequency.

title: Salience from feature contrast: additivity across dimensions
abstract: Test targets ('singletons') that displayed orientation, motion, luminance, or color contrast, or pairwise combinations of these, were presented in line texture arrays, and their saliences were quantified in comparison to reference targets at defined luminance levels. In general, saliency effects in different stimulus dimensions did add, but did not add linearly. That is, targets with feature contrast in two dimensions were generally more salient than targets with only one of these properties, but often less salient than predicted from the sum of the individual saliency components. Salience variations within a dimension were compared with and without a second saliency effect added. The resulting gain reduction in the combined stimulus conditions was interpreted to reflect the amount of overlap between the respective saliency mechanisms. Combinations of orientation and color contrast produced the strongest gain reduction (about 90% for color in orientation) thus indicating the strongest overlap of underlying saliency mechanisms. Combinations of orientation and motion contrast revealed about 50% overlap, slightly smaller rates were found for combinations of color and motion. All combinations with luminance contrast (orientation and luminance, motion and luminance) produced only little gain reduction (<30%) thus indicating a higher degree of independence between the underlying saliency mechanisms than for other stimulus dimensions.

title: Attention shifts to salient targets
abstract: To investigate the role of salience in fast visual search, time courses for the detection and identification of salient targets were measured in six subjects, using texture-like line arrays. Single lines were made salient from luminance contrast, motion contrast, or by an added circular cue that is known to attract focal attention. Three major findings are reported: (1) Identification of target orientation required longer presentations than detection of the saliency effect itself, consistent with the model that target salience attracts focal attention for target analysis. (2) Different saliency mechanisms produced similar effects, suggesting that salience from feature contrast is functionally equivalent to salience evoked from a visual cue. (3) Circular cues were most effective when presented close to the target; performance in target identification decreased when the diameter was enlarged so that the cue was presented farther away and on a different spatial scale. All together, these findings suggest that popout targets in visual search may be detected fast and independent of set size because (a) they are salient and attract focal attention, and (b) their salience is produced on the same spatial scale and at the same location in the visual field where target properties are encoded.

title: Feature-specific interactions in salience from combined feature contrasts: evidence for a bottom-up saliency map in V1.
abstract: Items that stand out from their surroundings, that is, those that attract attention, are considered to be salient. Salience is generated by input features in many stimulus dimensions, like motion (M), color (C), orientation (O), and others. We focus on bottom-up salience generated by contrast between the feature properties of an item and its surroundings. We compare the singleton search reaction times (RTs) of items that differ from their surroundings in more than one feature (e.g., C + O, denoted as CO) against the RTs of items that differ from their surroundings in only a single feature (e.g., O or C). The measured RTs for the double-feature singletons are compared against "race model" predictions to evaluate whether salience in the double-feature conditions is greater than the salience of either of its feature components. Affirmative answers were found in MO and CO but not in CM. These results are consistent with some V1 neurons being conjunctively selective to MO, others to CO, but almost none to CM. They provide support for the V1 hypothesis of bottom-up salience (Z. Li, 2002) but are contrary to expectation from the "feature summation" hypothesis, in which different stimulus features are initially analyzed independently and subsequently summed to form a single salience map (L. Itti & C. Koch, 2001; C. Koch & S. Ullman, 1985; J. M. Wolfe, K. R. Cave, & S. L. Franzel, 1989).

title: Attention capture by eye of origin singletons even without awareness--a hallmark of a bottom-up saliency map in the primary visual cortex.
abstract: Human observers are typically unaware of the eye of origin of visual inputs. This study shows that an eye of origin or ocular singleton, e.g., an item in the left eye among background items in the right eye, can nevertheless attract attention automatically. Observers searched for a uniquely oriented bar, i.e., an orientation singleton, in a background of horizontal bars. Their reports of the tilt direction of the search target in a brief (200 ms) display were more accurate in a dichoptic congruent (DC) condition, when the target was also an ocular singleton, than in a monocular (M) condition, when all bars were presented to the same single eye, or a dichoptic incongruent (DI) condition, when an ocular singleton was a background bar. The better performance in DC did not depend on the ability of the observers to report the presence of an ocular singleton by making forced choices in the same stimuli (though without the orientation singleton). This suggests that the ocular singleton exogenously cued attention to its location, facilitating the identification of the tilt singleton in the DC condition. When the search display persisted without being masked, observers' reaction times (RTs) for reporting the location of the search target were shorter in the DC, and longer in the DI, than the M condition, regardless of whether the observers were aware that different conditions existed. In an analogous design, similar RT patterns were observed for the task of finding an orientation contrast texture border. These results suggest that in typical trials, attention was more quickly attracted to or initially distracted from the target in the DC or DI condition, respectively. Hence, an ocular singleton, though elusive to awareness, can effectively compete for attention with an orientation singleton (tilted 20 or 50 degrees from background bars in the current study). Similarly, it can also make a difficult visual search easier by diminishing the set size effect. Since monocular neurons with the eye of origin information are abundant in the primary visual cortex (V1) and scarce in other cortical areas, and since visual awareness is believed to be absent or weaker in V1 than in other cortical areas, our results provide a hallmark of the role of V1 in creating a bottom-up saliency map to guide attentional selection.

title: Salience is only briefly represented: evidence from probe-detection performance.
abstract: Salient objects in the visual field tend to capture attention. The present study aimed to examine the time-course of salience effects using a probe-detection task. Eight experiments investigated how the salience of different orientation singletons affected probe reaction time as a function of stimulus onset asynchrony (SOA) between the presentation of a singleton display and a probe display. The results demonstrate that salience consistently affected probe reaction time at the shortest SOA. The effect of salience disappeared as SOA increased. These results suggest that contrary to the assumption of major theories on visual selection, salience is transiently represented in our visual system allowing the effects of salience on attentional selection to be only short-lived.

title: Visually guided pointing movements are driven by the salience map.
abstract: Visual salience maps are assumed to mediate target selection decisions in a motor-unspecific manner; accordingly, modulations of salience influence yes/no target detection or left/right localization responses in manual key-press search tasks, as well as ocular or skeletal movements to the target. Although widely accepted, this core assumption is based on little psychophysical evidence. At least four modulations of salience are known to influence the speed of visual search for feature singletons: (i) feature contrast, (ii) cross-trial dimension sequence and (iii) semantic pre-cueing of the target dimension, and (iv) dimensional target redundancy. If salience guides also manual pointing movements, their initiation latencies (and durations) should be affected by the same four manipulations of salience. Four experiments, each examining one of these manipulations, revealed this to be the case. Thus, these effects are seen independently of the motor response required to signal the perceptual decision (e.g., directed manual pointing as well as simple yes/no detection responses). This supports the notion of a motor-unspecific salience map, which guides covert attention as well as overt eye and hand movements.

title: Salience-based integration of redundant signals in visual pop-out search: evidence from behavioral and electrophysiological measures.
abstract: Behavioral and electrophysiological evidence is presented suggesting that, in visual search for feature singleton targets, multidimensional signals are integrated at a preselective stage of processing. Observers searched for a target that was consistently defined by the same features, but differed from the variable context either nonredundantly by one or redundantly by two dimensionally different features. The behavioral results showed reaction time redundancy gains and evidence of coactive processing, and the electrophysiological analyses revealed the latency of the N2pc component of the event-related potential (ERP) to be expedited by redundant relative to nonredundant displays, while the response-related lateralized readiness potential (LRP) remained unaffected. These findings suggest that target signal integration in singleton search paradigms occurs pre-attentively, that is, prior to focal-attentional target selection, with observers basing their responses on the detection of featureless saliency signals, even under conditions in which the target features remain constant and are known in advance. These results have implications for theories assuming top-down influences in feature detection.

title: Summation of texture segregation across orientation and spatial frequency: electrophysiological and psychophysical findings
abstract: Objects are usually segregated from ground by several visual dimensions. We studied texture segregation in checkerboards defined by gradients in spatial frequency, orientation or both frequency and orientation, using Gabor-filtered noise patterns. Saliency was measured electrophysiologically using the visual evoked potential (VEP) associated with texture segregation ('tsVEP') (an associated component in the visual evoked potential), and psychophysically by a 2AFC task. Spatial frequency and orientation stimuli evoked percepts of texture segregation and tsVEPs in all 11 subjects. The tsVEPs to combined stimuli were larger than those to each dimension alone, but smaller (74%) than the algebraic sum of tsVEPs to both individual dimensions. Psychophysical detection rates differed significantly between all conditions (P < 0.001), with highest rates for the combined stimuli. The findings suggest that segregation based on a combination of 'orientation' and 'spatial frequency' is more salient than that based on either of these alone. The significant deviation from full additivity in the tsVEPs suggests that simultaneous contrasts in spatial frequency and orientation have a common processing stage.

title: Faster Thalamocortical Processing for Dark than Light Visual Targets
abstract: ON and OFF visual pathways originate in the retina at the synapse between photoreceptor and bipolar cells. OFF bipolar cells are shorter in length and use receptors with faster kinetics than ON bipolar cells and, therefore, process information faster. Here, we demonstrate that this temporal advantage is maintained through thalamocortical processing, with OFF visual responses reaching cortex ∼3–6 ms before ON visual responses. Faster OFF visual responses could be demonstrated in recordings from large populations of cat thalamic neurons representing the center of vision (both X and Y) and from subpopulations making connection with the same cortical orientation column. While the OFF temporal advantage diminished as visual responses reached their peak, the integral of the impulse response was greater in OFF than ON neurons. Given the stimulus preferences from OFF and ON channels, our results indicate that darks are processed faster than lights in the thalamocortical pathway.

title: Response Properties of Local Field Potentials and Neighboring Single Neurons in Awake Primary Visual Cortex
abstract: Recordings from local field potentials (LFPs) are becoming increasingly common in research and clinical applications, but we still have a poor understanding of how LFP stimulus selectivity originates from the combined activity of single neurons. Here, we systematically compared the stimulus selectivity of LFP and neighboring single-unit activity (SUA) recorded in area primary visual cortex (V1) of awake primates. We demonstrate that LFP and SUA have similar stimulus preferences for orientation, direction of motion, contrast, size, temporal frequency, and even spatial phase. However, the average SUA had 50 times better signal-to-noise, 20% higher contrast sensitivity, 45% higher direction selectivity, and 15% more tuning depth than the average LFP. Low LFP frequencies (<30 Hz) were most strongly correlated with the spiking frequencies of neurons with nonlinear spatial summation and poor orientation/direction selectivity that were located near cortical current sinks (negative LFPs). In contrast, LFP gamma frequencies (>30 Hz) were correlated with a more diverse group of neurons located near cortical sources (positive LFPs). In summary, our results indicate that low- and high-frequency LFP pool signals from V1 neurons with similar stimulus preferences but different response properties and cortical depths.

title: Mixing of Chromatic and Luminance Retinal Signals in Primate Area V1.
abstract: Vision emerges from activation of chromatic and achromatic retinal channels whose interaction in visual cortex is still poorly understood. To investigate this interaction, we recorded neuronal activity from retinal ganglion cells and V1 cortical cells in macaques and measured their visual responses to grating stimuli that had either luminance contrast (luminance grating), chromatic contrast (chromatic grating), or a combination of the two (compound grating). As with parvocellular or koniocellular retinal ganglion cells, some V1 cells responded mostly to the chromatic contrast of the compound grating. As with magnocellular retinal ganglion cells, other V1 cells responded mostly to the luminance contrast and generated a frequency-doubled response to equiluminant chromatic gratings. Unlike magnocellular and parvocellular retinal ganglion cells, V1 cells formed a unimodal distribution for luminance/color preference with a 2- to 4-fold bias toward luminance. V1 cells associated with positive local field potentials in deep layers showed the strongest combined responses to color and luminance and, as a population, V1 cells encoded a diverse combination of luminance/color edges that matched edge distributions of natural scenes. Taken together, these results suggest that the primary visual cortex combines magnocellular and parvocellular retinal inputs to increase cortical receptive field diversity and to optimize visual processing of our natural environment.

title: Consciousness is not necessary for visual feature binding
abstract: When visual information enters the brain, it is relayed to different specialized regions, processing features such as shape, color, or motion. And yet, in our conscious experience of a colored, moving shape, all the different features seem to be integrated into one unified percept. Therefore, it has been hypothesized that consciousness and feature binding share an intimate relationship. To study this relationship, we used a paradigm in which the behavioral effects of feature binding can be measured. Using masks, we investigated whether spontaneous binding between the orientation and location of a Gabor patch takes place when the Gabor patch is processed consciously or unconsciously. The results of our study suggest that orientation and location of a visually presented object are automatically integrated, even when subjects are unaware of that object. We conclude that binding and consciousness share a less intimate relationship than previously hypothesized, since consciousness is not a necessary condition for binding to occur.

title: Feature binding across different visual dimensions
abstract: The human brain represents different kinds of visual feature dimensions in different ways. For example, surface features exhibit some properties that are very different from contour features, and some feature dimensions may be represented more extensively in either the dorsal or the ventral visual stream. Given such differences, we investigated feature binding across different feature dimensions and whether some feature dimensions might be more easily bound together than others. In Experiment 1, we looked at cross-dimension bindings for all combinations of color, orientation, and shape dimensions, while at the same time controlling for feature discriminability. Rates of correct binding, illusory conjunctions, and feature errors were equivalent in all cases. There was no bias so that some feature dimensions were easier to combine than others. In Experiment 2, we manipulated the difficulty of feature discrimination for the key, target-defining feature and the report feature. Rates of binding errors increased with difficulty of the key feature, but not with that of the report feature. The accuracy of feature discrimination could be dissociated from the accuracy of binding the feature to an object. Across both experiments, the accuracy of feature binding was independent of specific feature dimensions or perceptibility. These findings are discussed in relation to both feature integration and multiple-stage accounts of visual feature integration.

title: The Psychophysical Evidence for a Binding Problem in Human Vision
abstract: Boston, Massachusetts 02115 †Department of Psychology tions. Illusory conjunctions occurred with both letters University of Southampton and abstract shapes and included all the features tested Highfield (color, shape, size, and solidity). Treisman and Schmidt Southampton SO17 1BJ concluded that when attention is not available to comUnited Kingdom bine features correctly, they can be put together to form

title: Features in visual search combine linearly.
abstract: Single features such as line orientation and length are known to guide visual search, but relatively little is known about how multiple features combine in search. To address this question, we investigated how search for targets differing in multiple features (intensity, length, orientation) from the distracters is related to searches for targets differing in each of the individual features. We tested race models (based on reaction times) and co-activation models (based on reciprocal of reaction times) for their ability to predict multiple feature searches. Multiple feature searches were best accounted for by a co-activation model in which feature information combined linearly (r = 0.95). This result agrees with the classic finding that these features are separable i.e., subjective dissimilarity ratings sum linearly. We then replicated the classical finding that the length and width of a rectangle are integral features-in other words, they combine nonlinearly in visual search. However, to our surprise, upon including aspect ratio as an additional feature, length and width combined linearly and this model outperformed all other models. Thus, length and width of a rectangle became separable when considered together with aspect ratio. This finding predicts that searches involving shapes with identical aspect ratio should be more difficult than searches where shapes differ in aspect ratio. We confirmed this prediction on a variety of shapes. We conclude that features in visual search co-activate linearly and demonstrate for the first time that aspect ratio is a novel feature that guides visual search.

title: Spatial frequency selectivity of cells in macaque visual cortex
abstract: We measured the spatial frequency contrast sensitivity of cells in the primate striate cortex at two different eccentricities to provide quantitative statistics from a large population of cells. Distributions of the peak frequencies and bandwidths are presented and examined in relationship to (a) each other, (b) absolute contrast sensitivity, (c) orientation tuning, (d)retinal eccentricity, and (e) cell type. Simple and complex cells are examined in relationship to linear/nonlinear (that is, X/Y) properties; a procedure is described which provides a simple, reliable and quantitative method for classifying and describing striate cells. Among other things, it is shown that (a) many stirate cells have quite narrow spatial bandwidths and (b) at a given retinal eccentricity, the distribution of peak frequency covers a wide range of frequencies; these findings support the basic multiple channel notion. The orientation tuning and spatial frequency tuning which occurs at the level of striate cortex (in a positively correlated fashion) suggests that the cells might best be considered as two-dimensional spatial filters.

title: Spatial interactions in apparent contrast: Inhibitory effects among grating patterns of different spatial frequencies, spatial positions and orientations
abstract: Suppression of the apparent contrast of a small 4 cycle wide suprathreshold sine wave grating patch by a high contrast sine wave grating surround pattern was studied as a function of the spatial frequency, orientation and spatial extent of the surround. The data are consistent with the existence of a complex network of inhibitory interconnections among mechanisms that mediate contrast perception. These connections must extend over spatial distances equivalent to more than 12 cycles of the central grating patch.

title: Misbinding of Color and Motion in Human Visual Cortex
abstract: A fundamental challenge for the visual system is to integrate visual features into a coherent scene, known as the binding problem. The neural mechanisms of feature binding are hard to identify because of difficulties in separating active feature binding from feature co-occurrence. In previous studies on feature binding, visual features were superimposed and presented simultaneously. Neurons throughout the visual cortex are known to code multiple features. Therefore, the observed binding effects could be due to the physical co-occurrence of features and the sensory representation of feature pairings. It is uncertain whether the mechanisms responsible for perceptual binding were actually recruited. To address this issue, we performed psychophysical and fMRI experiments to investigate the neural mechanisms of a steady-state misbinding of color and motion, because feature misbinding is probably the most striking evidence for the active existence of the binding mechanisms. We found that adapting to the color-motion misbinding generated the color-contingent motion aftereffect, as well as the color-contingent motion adaptation effect in visual cortex. Notably, V2 exhibited the strongest adaptation effect, which significantly correlated with the aftereffect across subjects. Furthermore, effective connectivity analysis using dynamic causal modeling showed that the misbinding was closely associated with enhanced feedback from V4 and V5 to V2. These findings provide strong evidence for active feature binding in early visual cortex and suggest a critical role of reentrant connections from specialized intermediate areas to early visual cortex in this process.

title: Misbinding of color and motion in human early visual cortex: Evidence from event-related potentials
abstract: One of the central tasks for the visual system is to integrate visual features into objects, which is referred to as the binding problem. To study the binding mechanisms, it has been suggested to use phenomena of feature misbinding to separate active feature binding from feature co-occurence. Taking advantage of a steady-state misbinding of color and motion, we performed psychophysical and event-related potential (ERP) adaptation experiments to investigate the neural mechanisms of the misbinding (i.e., the active color-motion binding). Human subjects adapted to the misbinding of color and motion, as well as their correct binding that was used for identifying neural processes associated with the co-occurrence of color and motion. We found that adaptation to the misbinding and the correct binding could generate color-contingent motion aftereffects (CCMAEs), but in opposite directions. ERP adaptation effects manifested in the earliest ERP component C1. The C1 latency in the misbinding condition was 11ms longer than that in the correct binding condition. In the correct binding condition, the C1 adaptation effect (i.e., the C1 amplitude reduction after adaptation) took place in the peak phase of the C1. The dipole source of the adaptation effect was located in V1. In the misbinding condition, the C1 adaptation effect occurred in the descending phase of the C1 and its dipole source was in V2. In both conditions, the C1 adaptation effects correlated with the CCMAEs across individual subjects. These findings provide human electrophysiological evidence that active feature binding takes place in early visual cortex, but at later processing stages than feature co-occurrence.

title: The right temporo-parietal junction contributes to visual feature binding
abstract: We investigated the neural basis of conjoined processing of color and spatial frequency with functional magnetic resonance imaging (fMRI). A multivariate classification algorithm was trained to differentiate between either isolated color or spatial frequency differences, or between conjoint differences in both feature dimensions. All displays were presented in a singleton search task, avoiding confounds between conjunctive feature processing and search difficulty that arose in previous studies contrasting single feature and conjunction search tasks. Based on patient studies, we expected the right temporo-parietal junction (TPJ) to be involved in conjunctive feature processing. This hypothesis was confirmed in that only conjoined color and spatial frequency differences, but not isolated feature differences could be classified above chance level in this area. Furthermore, we could show that the accuracy of a classification of differences in both feature dimensions was superadditive compared to the classification accuracies of isolated color or spatial frequency differences within the right TPJ. These data provide evidence for the processing of feature conjunctions, here color and spatial frequency, in the right TPJ.

title: From Thought to Action: The Parietal Cortex as a Bridge between Perception, Action, and Cognition
abstract: The lateral intraparietal area (LIP) is a subdivision of the inferior parietal lobe that has been implicated in the guidance of spatial attention. In a variety of tasks, LIP provides a "salience representation" of the external world-a topographic visual representation that encodes the locations of salient or behaviorally relevant objects. Recent neurophysiological experiments show that this salience representation incorporates information about multiple behavioral variables-such as a specific motor response, reward, or category membership-associated with the task-relevant object. This integration occurs in a wide variety of tasks, including those requiring eye or limb movements or goal-directed or nontargeting operant responses. Thus, LIP acts as a multifaceted behavioral integrator that binds visuospatial, motor, and cognitive information into a topographically organized signal of behavioral salience. By specifying attentional priority as a synthesis of multiple task demands, LIP operates at the interface of perception, action, and cognition.

title: Masking within and across visual dimensions: Psychophysical evidence for perceptual segregation of color and motion
abstract: Visual masking can result from the interference of perceptual signals. According to the principle of functional specialization, interference should be greatest when signal and mask belong to the same visual attribute (e.g., color or motion) and least when they belong to different ones. We provide evidence to support this view and show that the time course of masking is visual attribute specific. First, we show that a color target is masked most effectively by color (homogeneous target-mask pair) and least effectively by motion (heterogeneous pair) and vice versa for a motion target. Second, we show that the time at which the mask is most effective depends strongly on the target-mask pairing. Heterogeneous masking is strongest when the mask is presented before the target (forward masking) but this is not true of homogeneous masking. This finding supports a delayed cross-feature interaction due to segregated processing sites. Third, lengthening the stimulus onset asynchrony between target and mask leads to a faster improvement in color than in motion detectability, lending support for a faster color processing system and consistent with reports of perceptual asynchrony in vision. In summary, we present three lines of psychophysical evidence, all of which support a segregated neural coding scheme for color and motion in the human brain.

title: Timing, Timing, Timing: Fast Decoding of Object Information from Intracranial Field Potentials in Human Visual Cortex
abstract: The difficulty of visual recognition stems from the need to achieve high selectivity while maintaining robustness to object transformations within hundreds of milliseconds. Theories of visual recognition differ in whether the neuronal circuits invoke recurrent feedback connections or not. The timing of neurophysiological responses in visual cortex plays a key role in distinguishing between bottom-up and top-down theories. Here, we quantified at millisecond resolution the amount of visual information conveyed by intracranial field potentials from 912 electrodes in 11 human subjects. We could decode object category information from human visual cortex in single trials as early as 100 ms poststimulus. Decoding performance was robust to depth rotation and scale changes. The results suggest that physiological activity in the temporal lobe can account for key properties of visual recognition. The fast decoding in single trials is compatible with feedforward theories and provides strong constraints for computational models of human vision.

title: Evoked brain responses are generated by feedback loops
abstract: Neuronal responses to stimuli, measured electrophysiologically, unfold over several hundred milliseconds. Typically, they show characteristic waveforms with early and late components. It is thought that early or exogenous components reflect a perturbation of neuronal dynamics by sensory input bottom-up processing. Conversely, later, endogenous components have been ascribed to recurrent dynamics among hierarchically disposed cortical processing levels, top-down effects. Here, we show that evoked brain responses are generated by recurrent dynamics in cortical networks, and late components of event-related responses are mediated by backward connections. This evidence is furnished by dynamic causal modeling of mismatch responses, elicited in an oddball paradigm. We used the evidence for models with and without backward connections to assess their likelihood as a function of peristimulus time and show that backward connections are necessary to explain late components. Furthermore, we were able to quantify the contribution of backward connections to evoked responses and to source activity, again as a function of peristimulus time. These results link a generic feature of brain responses to changes in the sensorium and a key architectural component of functional anatomy; namely, backward connections are necessary for recurrent interactions among levels of cortical hierarchies. This is the theoretical cornerstone of most modern theories of perceptual inference and learning.

title: Salience from feature contrast: variations with texture density
abstract: The salience of popout targets was measured in regular line arrays as a function of texture density. Test targets (singletons with orientation, motion, or luminance contrast) presented at different raster widths were compared with reference lines (lines brighter than surrounding lines) presented at fixed raster width. The luminance at which the reference target appeared as salient as the particular test target was taken as a measure of the relative salience of the test target. For orientation or motion contrast, targets at medium to small raster widths were far more salient than targets in sparse or very dense line arrangements. For targets defined by luminance contrast, salience variations with texture density were less pronounced. Some subjects also reported salience for lines in sparse arrangements even when these did not display feature contrast. When such non-specific saliency effects were subtracted from the actual measurements, salience curves for orientation or motion contrast revealed peaks of increased sensitivity at line spacings below 2-3 deg and flat curves at larger grid sizes. In an additional experiment, saliency effects from orientation contrast were measured using texture lines of different size. Salience variations were commonly observed. However, the curves were not found to scale with the different sizes of texture elements but were constantly related to the free space between neighbouring lines. This suggests that peaks in the salience profiles reflect the limited spatial extent of the underlying neural mechanisms.

title: Neuronal and Perceptual Differences in the Temporal Processing of Darks and Lights
abstract: Visual information is mediated by two major thalamic pathways that signal light decrements (OFF) and increments (ON) in visual scenes, the OFF pathway being faster than the ON. Here, we demonstrate that this OFF temporal advantage is transferred to visual cortex and has a correlate in human perception. OFF-dominated cortical neurons in cats responded ∼3 ms faster to visual stimuli than ON-dominated cortical neurons, and dark-mediated suppression in ON-dominated neurons peaked ∼14 ms faster than light-mediated suppression in OFF-dominated neurons. Consistent with the neuronal differences, human observers were 6-14 ms faster at detecting darks than lights and better at discriminating dark than light flickers. Neuronal and perceptual differences both vanished if backgrounds were biased toward darks. Our results suggest that the cortical OFF pathway is faster than the ON pathway at increasing and suppressing visual responses, and these differences have parallels in the human visual perception of lights and darks.

title: Attention to Banner Ads and Their Effectiveness: An Eye-Tracking Approach
abstract: As with all forms of advertising, exposure is a necessary prerequisite for Internet banner ad effectiveness. However, exposure does not guarantee a user's attention, an issue especially relevant to the Internet, where ad avoidance occurs most frequently. And if an ad is noticed, the message may or may not remain in the consumer's memory after cognitive processing. However, even if the advertising message is not consciously remembered, the exposure can be unconsciously processed and subsequently change the user's affective state. To investigate how attention levels influence users, this study uses eye tracking to measure the level of attention that results from an advertisement exposure and explores how different levels of attention influence users in conscious and unconscious ways. Also, we examine the effect of animation—one of the most popular attention-grabbing tools—on attention and how it moderates cognitive processing. By measuring and analyzing users' actual eye-movement data, we found that animation in banner ads not only attracts less attention than static ads but also reduces the positive effect of attention on memory. In addition, although more than half of the participants could not recognize the advertised brand, the animated banner ad was unconsciously processed and did influence attitudes toward the brand. The results suggest that animation in banner ads does not necessarily increase user attention, but that even if a user does not consciously notice a banner ad, the user's attitude toward the brand is influenced.

title: Salience of unique hues and implications for color theory.
abstract: The unique hues--blue, green, yellow, red--form the fundamental dimensions of opponent-color theories, are considered universal across languages, and provide useful mental representations for structuring color percepts. However, there is no neural evidence for them from neurophysiology or low-level psychophysics. Tapping a higher prelinguistic perceptual level, we tested whether unique hues are particularly salient in search tasks. We found no advantage for unique hues over their nonunique complementary colors. However, yellowish targets were detected faster, more accurately, and with fewer saccades than their complementary bluish targets (including unique blue), while reddish-greenish pairs were not significantly different in salience. Similarly, local field potentials in primate V1 exhibited larger amplitudes and shorter latencies for yellowish versus bluish stimuli, whereas this effect was weaker for reddish versus greenish stimuli. Consequently, color salience is affected more by early neural response asymmetries than by any possible mental or neural representation of unique hues.

title: Perceptive fields of saliency.
abstract: Visual saliency plays an important role in early vision. Humans automatically orient to salient information, via covert attentional shifts and overt eye movements. Here, we measured saliency using a novel psychophysical method. The stimulus was a grid of colored oriented lines. Line luminance varied continuously over the image. Using a mouse, participants adjusted line luminance at locations in the image, until all lines appeared homogeneously luminant. Participants tended to increase (or decrease) luminance of lines where perceptually salient information was absent (or present), thus line luminance setting correlates with perceived saliency. Perceptually less salient image regions were more homogenous in color and orientation, consistent with iso-feature suppression. Perceptual fields of contextual modulation are derived, showing increased saliency surrounding color and/or orientation changes, increased saliency for collinear and end-stopping lines, and a nonlinear integration of saliencies across dimensions. It took 3 or more surround items identical to a target to generate a measurable inhibitory effect, beyond which every additional identical item had a monotonic effect. These novel findings allow a revision of current models of visual saliency. In particular, we found evidence of sustained saliency. Moreover, this new method is sensitive within the normal functioning range, unlike most current research methods.

title: Is contrast just another feature for visual selective attention?
abstract: The biased-competition theory of attention [Annual Review of Neuroscience 18 (1995) 193] suggests that attention and stimulus contrast trade off, and implies that high-contrast stimuli should be easy to attend to and hard to ignore. To test this, observers searched displays for a target digit. Observers were well able to exclude high-contrast distractors when attempting to search only among low-contrast stimuli (Experiment 1). In Experiments 2 and 3, location determined which stimuli were relevant. When contrast of relevant and irrelevant stimuli was uncertain (due to contrast varying between trials, Experiment 2), increasing the contrast of distractors impaired performance. However, when contrast was certain (due to blocking of trials, Experiment 3) and targets were of low contrast, high contrast distractors produced less interference than low contrast distractors. The ability of subjects to attend selectively to low vs. high contrast items in Experiments 1 and 3 suggests that selectivity for stimulus contrast might be similar to other types of feature selectivity (e.g., color and location). Such findings are inconsistent with the biased competition theory regarding the interplay of contrast and attention. However, results from Experiment 2 suggest that, when target contrast varies, the default tendency is to attend to high-contrast items.

title: Saliency effects across dimensions in visual search
abstract: In previous work it has been shown that search for an orientation target can only be done fast, and in parallel, when the target's orientation differs sufficiently from that of neighbouring lines. Targets with an increased local orientation contrast appear as salient and are immediately detected. In the present study, the effect of saliency from other stimulus dimensions was investigated. Five subjects were asked to look for vertical lines in texture-like displays, which were presented either embedded in the local orientation flow ("non-salient" target presentation) or at an increased feature contrast ("salient" target presentation). Saliency was obtained from local differences in orientation ("within dimension" saliency) or other visual cues ("cross dimensions" saliency), such as colour, luminance, motion, and disparity. Targets displaying cross-dimensional saliency were always non-salient in the orientation domain. The results show that subjects were slower in detecting targets from non-salient than from salient presentations. The effect of saliency on visual search is not feature-specific and, in particular, is not restricted to the stimulus dimension in which target features are searched for.

title: Feature analysis and the role of similarity in preattentive vision
abstract: Texture arrays of line elements at various orientations were used to study three phenomena of preattentive vision. Subjects were asked (1) to discriminate texture areas and to distinguish their form (experiments on texture segmentation); (2) to detect salient or vertical line elements (experiments on pop-out); and (3) to identify configurations of similar or dissimilar targets (experiments on grouping). Within the patterns, line orientation was systematically varied to distinguish the effect of differences between areas from the effect of similarity within areas. In all of the experiments, performance was found to depend on local orientation contrast at texture borders rather than on the analysis of line orientation itself. Texture areas were correctly identified only when the orientation contrast at the border well exceeded the overall variation of line orientation in the pattern. Similarly, only target elements with high local orientation contrast were detected fast and “in parallel.” Targets with an orientation contrast lower than background variation required serial search. Preattentive grouping was found to depend on saliency, as defined by local orientation contrast, but not on the similarity of line elements. In addition to local orientation contrast, which played an important role in all of the visual phenomena studied, influences from the alignment of line elements with the outline of a figure were also seen.

title: A saliency map in primary visual cortex
abstract: I propose that pre-attentive computational mechanisms in primary visual cortex create a saliency map. This map awards higher responses to more salient image locations; these responses are those of conventional V1 cells tuned to input features, such as orientation and color. Hence no separate feature maps, or any subsequent combination of them, is needed to create a saliency map. I use a model to show that this saliency map accounts for the way that the relative difficulties of visual search tasks depend on the features and spatial configurations of targets and distractors. This proposal links psychophysical behavior to V1 physiology and anatomy, and thereby makes testable predictions.

title: Psychophysical Tests of the Hypothesis of a Bottom-Up Saliency Map in Primary Visual Cortex
abstract: A unique vertical bar among horizontal bars is salient and pops out perceptually. Physiological data have suggested that mechanisms in the primary visual cortex (V1) contribute to the high saliency of such a unique basic feature, but indicated little regarding whether V1 plays an essential or peripheral role in input-driven or bottom-up saliency. Meanwhile, a biologically based V1 model has suggested that V1 mechanisms can also explain bottom-up saliencies beyond the pop-out of basic features, such as the low saliency of a unique conjunction feature such as a red vertical bar among red horizontal and green vertical bars, under the hypothesis that the bottom-up saliency at any location is signaled by the activity of the most active cell responding to it regardless of the cell's preferred features such as color and orientation. The model can account for phenomena such as the difficulties in conjunction feature search, asymmetries in visual search, and how background irregularities affect ease of search. In this paper, we report nontrivial predictions from the V1 saliency hypothesis, and their psychophysical tests and confirmations. The prediction that most clearly distinguishes the V1 saliency hypothesis from other models is that task-irrelevant features could interfere in visual search or segmentation tasks which rely significantly on bottom-up saliency. For instance, irrelevant colors can interfere in an orientation-based task, and the presence of horizontal and vertical bars can impair performance in a task based on oblique bars. Furthermore, properties of the intracortical interactions and neural selectivities in V1 predict specific emergent phenomena associated with visual grouping. Our findings support the idea that a bottom-up saliency map can be at a lower visual area than traditionally expected, with implications for top-down selection mechanisms.

title: Effects of surrounding frame on visual search for vertical or tilted bars.
abstract: It is easier to find a tilted bar among vertical bars than vice-versa, but this asymmetry can be abolished or reversed by surrounding the bars with a tilted frame. The frame effect is important because it challenges bottom-up models of saliency. We conducted two experiments to investigate the causes of this effect. In Experiment 1, we removed different components of a square frame, and concluded that the frame effect was caused by a combination of (1) high-level configural cues that provided a frame of reference, and (2) bottom-up iso-orientation competition from the sides of the frame parallel to the bars. The iso-orientation competition could have arisen from (1) diversion of attention to the parts of the frame parallel to the target, or (2) iso-orientation suppression between nearby units selective for the same orientation. Experiment 2 investigated the nature of the iso-orientation competition process. In this experiment, we used a single line (the "axis") embedded in a circular field of bar elements, rather than a square frame surrounding them. The effect of the axis declined rapidly to zero with increasing target-axis distance, suggesting that the iso-orientation competition was caused entirely by iso-orientation suppression between nearby units tuned to the same orientation.

title: Visual search for a target against a 1/fβ continuous textured background
abstract: We present synthetic surface textures as a novel class of stimuli for use in visual search experiments. Surface textures have certain advantages over both the arrays of abstract discrete items commonly used in search studies and photographs of natural scenes. In this study we investigate how changing the properties of the surface and target influence the difficulty of a search task. We present a comparison with Itti and Koch's saliency model and find that it fails to model human behaviour on these surfaces. In particular it does not respond to changes in orientation in the same manner as human observers.

title: Search asymmetries? What search asymmetries?
abstract: In order to establish a search asymmetry, one must run an experiment with a symmetric design and get asymmetric results. Given an asymmetric design, one expects asymmetric results, and such results do not imply an asymmetry in the search mechanisms. In this paper, I argue that a number of experiments purporting to show search asymmetries contain built-in design asymmetries. A saliency model of visual search predicts the results of these experiments, using only a simple measure of target-distractor similarity, without reliance on asymmetric search mechanisms. These results have implications for search mechanisms and for other experiments purporting to show search asymmetries.

title: Saliency in a suppressed image affects the spatial origin of perceptual alternations during binocular rivalry
abstract: During binocular rivalry, perception alternates between dichoptically presented incompatible images. With larger images, such perceptual alternations will typically start locally and then gradually spread across the image, known as traveling waves of perceptual dominance. Several image-features (such as local contrast) are known to determine where in the image a traveling wave originates. Here we investigate whether orientation contrast in the suppressed image affects these spatial origin(s) of perceptual alternations. The results show that the origins are increasingly biased towards locations of increasing orientation contrast in the suppressed image. This increase in bias is related to the efficiency of visual search for the orientation contrast, tested offline: we find large biases towards orientation contrast when visual search for it is efficient, and small biases when search for it is inefficient. Our results imply that rivalry suppression is not homogenous across the suppressed image, but is dependent on local image-features in the suppressed image. The relation between spatial bias and visual search performance suggests that spatial origins of perceptual alternations are biased to salient locations in the suppressed image. Moreover, the finding that saliency affects the spatial origin of a perceptual alternation is in agreement with the idea that saliency is represented at a monocular, unconscious level of visual processing.

title: Objects and attributes in divided attention: Surface and boundary systems
abstract: A series of experiments investigated concurrent discriminations of surface and nonsurface attributes, including color, brightness, texture, length, location, and motion. In all cases but one, results matched those previously reported: Interference occurred when two discriminations concerned different objects, but not when they concerned the same one. In the two-object case, interference was the same whether discriminations were similar (e.g., two surface discriminations) or different (e.g., one surface, one boundary). Such results support a model of visual attention in which separate visual subsystems are coordinated, converging to work on surface and boundary properties of the same selected object. A partial exception is color: For reasons that are unclear, color escapes twoobject interference except from other, concurrent surface discriminations

title: Top-down and bottom-up mechanisms in biasing competition in the human brain
abstract: The biased competition theory of selective attention has been an influential neural theory of attention, motivating numerous animal and human studies of visual attention and visual representation. There is now neural evidence in favor of all three of its most basic principles: that representation in the visual system is competitive; that both top-down and bottom-up biasing mechanisms influence the ongoing competition; and that competition is integrated across brain systems. We review the evidence in favor of these three principles, and in particular, findings related to six more specific neural predictions derived from these original principles.

title: Visual attention as a multilevel selection process
abstract: Natural visual scenes are cluttered and contain many different objects that cannot all be processed simultaneously. Therefore, attentional mechanisms are needed to select relevant and to filter out irrelevant information. Evidence from functional brain imaging reveals that attention operates at various processing levels within the visual system and beyond. First, the lateral geniculate nucleus appears to be the first stage in the processing of visual information that is modulated by attention, consistent with the idea that it may play an important role as an early gatekeeper in controlling neural gain. Second, areas at intermediate cortical-processing levels, such as V4 and TEO, appear to be important sites at which attention filters out unwanted information by means of receptive field mechanisms. Third, the attention mechanisms that operate in the visual system appear to be controlled by a distributed network of higher order areas in the frontal and parietal cortex, which generate top-down signals that are transmitted via feedback connections to the visual system. And fourth, the pulvinar of the thalamus may operate by integrating and coordinating attentional functions in concert with the fronto-parietal network, although much needs to be learned about its functional properties. The overall view that emerges from the studies reviewed in this article is that neural mechanisms of selective attention operate at multiple stages in the visual system and beyond and are determined by the visual processing capabilities of each stage. In this respect, attention can be considered in terms of a multilevel selection process.

title: No apparent influence of psychometrically-defined schizotypy on orientation-dependent contextual modulation of visual contrast detection
abstract: We investigated the relationship between psychometrically-defined schizotypy and the ability to detect a visual target pattern. Target detection is typically impaired by a surrounding pattern (context) with an orientation that is parallel to the target, relative to a surrounding pattern with an orientation that is orthogonal to the target (orientation-dependent contextual modulation). Based on reports that this effect is reduced in those with schizophrenia, we hypothesised that there would be a negative relationship between the relative score on psychometrically-defined schizotypy and the relative effect of orientation-dependent contextual modulation. We measured visual contrast detection thresholds and scores on the Oxford-Liverpool Inventory of Feelings and Experiences (O-LIFE) from a non-clinical sample (N = 100). Contrary to our hypothesis, we find an absence of a monotonic relationship between the relative magnitude of orientation-dependent contextual modulation of visual contrast detection and the relative score on any of the subscales of the O-LIFE. The apparent difference of this result with previous reports on those with schizophrenia suggests that orientation-dependent contextual modulation may be an informative condition in which schizophrenia and psychometrically-defined schizotypy are dissociated. However, further research is also required to clarify the strength of orientation-dependent contextual modulation in those with schizophrenia.

title: Neural activities in V1 create the bottom-up saliency map of natural scenes
abstract: A saliency map is the bottom-up contribution to the deployment of exogenous attention. It, as well as its underlying neural mechanism, is hard to identify because of the influence of top-down signals. A recent study showed that neural activities in V1 could create a bottom-up saliency map (Zhang et al.  2012). In this paper, we tested whether their conclusion can generalize to complex natural scenes. In order to avoid top-down influences, each image was presented with a low contrast for only 50 ms and was followed by a high contrast mask, which rendered the whole image invisible to participants (confirmed by a forced-choice test). The Posner cueing paradigm was adopted to measure the spatial cueing effect (i.e., saliency) by an orientation discrimination task. A positive cueing effect was found, and the magnitude of the cueing effect was consistent with the saliency prediction of a computational saliency model. In a following fMRI experiment, we used the same masked natural scenes as stimuli and measured BOLD signals responding to the predicted salient region (relative to the background). We found that the BOLD signal in V1, but not in other cortical areas, could well predict the cueing effect. These results suggest that the bottom-up saliency map of natural scenes could be created in V1, providing further evidence for the V1 saliency theory (Li in  2002).

title: Visual search in spatial neglect studied with a preview paradigm
abstract: Impaired visual search is a hallmark of spatial neglect. When searching for an unique feature (e.g., color) neglect patients often show only slight visual field asymmetries. In contrast, when the target is defined by a combination of features (e.g., color and form) they exhibit a severe deficit of contralesional search. This finding suggests a selective impairment of the serial deployment of spatial attention. Here, we examined this deficit with a preview paradigm. Neglect patients searched for a target defined by the conjunction of shape and color, presented together with varying numbers of distracters. The presentation time was varied such that on some trials participants previewed the target together with same-shape/different-color distracters, for 300 or 600 ms prior to the appearance of additional different-shape/same-color distracters. On the remaining trials the target and all distracters were shown simultaneously. Healthy participants exhibited a serial search strategy only when all items were presented simultaneously, whereas in both preview conditions a pop-out effect was observed. Neglect patients showed a similar pattern when the target was presented in the right hemifield. In contrast, when searching for a target in the left hemifield they showed serial search in the no-preview condition, as well as with a preview of 300 ms, and partly even at 600 ms. A control experiment suggested that the failure to fully benefit from item preview was probably independent of accurate perception of time. Our results, when viewed in the context of existing literature, lead us to conclude that the visual search deficit in neglect reflects two additive factors: a biased representation of attentional priority in favor of ipsilesional information and exaggerated capture of attention by ipsilesional abrupt onsets.

title: Salient-but-irrelevant stimuli cause attentional capture in difficult, but attentional suppression in easy visual search.
abstract: Search for a shape target is difficult when its shape is similar to the shape of the surrounding nontargets and easy when it is dissimilar. We asked whether interference from a salient but irrelevant color singleton depended on search difficulty as manipulated by target-nontarget similarity. We found that interference was strong in difficult searches, and the occurrence of an electrophysiological index of attentional selectivity (the N2pc component) confirmed that attention was captured by the distractor. In contrast, interference from the distractor was weak with easy searches, and the occurrence of the PD component confirmed that saliency signals from the distractor were suppressed. The results suggest that attentional suppression of salient but irrelevant distractors is only possible when search is efficient (i.e., with low target-nontarget similarity), otherwise, attentional capture occurs. Further, we analyzed the Ppc component, a positivity occurring between 100 and 200 ms after stimulus onset contralateral to a salient feature discontinuity. It has been suggested that the Ppc reflects bottom-up saliency signals, but we found the Ppc to occur only when the salient stimulus was unlikely to be selected, as in efficient searches. Thus, attentional requirements modulate the Ppc component.

title: Attentional modulation of visual processing.
abstract: Single-unit recording studies in the macaque have carefully documented the modulatory effects of attention on the response properties of visual cortical neurons. Attention produces qualitatively different effects on firing rate, depending on whether a stimulus appears alone or accompanied by distracters. Studies of contrast gain control in anesthetized mammals have found parallel patterns of results when the luminance contrast of a stimulus increases. This finding suggests that attention has co-opted the circuits that mediate contrast gain control and that it operates by increasing the effective contrast of the attended stimulus. Consistent with this idea, microstimulation of the frontal eye fields, one of several areas that control the allocation of spatial attention, induces spatially local increases in sensitivity both at the behavioral level and among neurons in area V4, where endogenously generated attention increases contrast sensitivity. Studies in the slice have begun to explain how modulatory signals might cause such increases in sensitivity.

title: Selective visual attention and perceptual coherence
abstract: Conscious perception of the visual world depends on neural activity at all levels of the visual system from the retina to regions of parietal and frontal cortex. Neurons in early visual areas have small spatial receptive fields (RFs) and code basic image features; neurons in later areas have large RFs and code abstract features such as behavioral relevance. This hierarchical organization presents challenges to perception: objects compete when they are presented in a single RF, and component object features are coded by anatomically distributed neuronal activity. Recent research has shown that selective attention coordinates the activity of neurons to resolve competition and link distributed object representations. We refer to this ensemble activity as a "coherence field", and propose that voluntary shifts of attention are initiated by a transient control signal that "nudges" the visual system from one coherent state to another.

title: Mechanisms of Spatial Attention Control in Frontal and Parietal Cortex
abstract: Theories of spatial attentional control have been largely based upon studies of patients suffering from visuospatial neglect, resulting from circumscribed lesions of frontal and posterior parietal cortex. In the intact brain, the control of spatial attention has been related to a distributed frontoparietal attention network. Little is known about the nature of the control mechanisms exerted by this network. Here, we used a novel region-of-interest approach to relate activations of the attention network to recently described topographic areas in frontal cortex [frontal eye field (FEF), PreCC/IFS (precentral cortex/inferior frontal sulcus)] and parietal cortex [intraparietal sulcus areas (IPS1–IPS5) and an area in the superior parietal lobule (SPL1)] to examine their spatial attention signals. We found that attention signals in most topographic areas were spatially specific, with stronger responses when attention was directed to the contralateral than to the ipsilateral visual field. Importantly, two hemispheric asymmetries were found. First, a region in only right, but not left SPL1 carried spatial attention signals. Second, left FEF and left posterior parietal cortex (IPS1/2) generated stronger contralateral biasing signals than their counterparts in the right hemisphere. These findings are the first to characterize spatial attention signals in topographic frontal and parietal cortex and provide a neural basis in support of an interhemispheric competition account of spatial attentional control.

title: Visual attention to advertising: A segment-level analysis
abstract: We propose a methodology to study the effects of physical ad properties on consumers' visual attention to advertising that accounts for heterogeneity in these effects across consumers. In an illustrative experiment, we monitor consumers' eye movements during naturalistic exposure to a consumer magazine, in which experimentally designed ads are inserted. A latent class regression model accounting for heterogeneity across consumers through unobserved segments is used to analyze the eye movement data in detail. Three consumer segments are identified that exhibit distinct patterns of visual attention as well as different profiles of product involvement, brand attitude, and advertising recall. Implications for visual attention theory and for advertising research are discussed.

title: Attention Capture and Transfer in Advertising: Brand, Pictorial, and Text-Size Effects
abstract: The three key ad elements (brand, pictorial, and text) each have unique superiority effects on attention to advertisements, which are on par with many commonly held ideas in marketing practice. This is the main conclusion of an analysis of 1363 print advertisements tested with infrared eye-tracking methodology on more than 3600 consumers. The pictorial is superior in capturing attention, independent of its size. The text element best captures attention in direct proportion to its surface size. The brand element most effectively transfers attention to the other elements. Only increments in the text element's surface size produce a net gain in attention to the advertisement as a whole. The authors discuss how their findings can be used to render more effective decisions in advertising.

title: Visual Salience Improves Spatial Working Memory via Enhanced Parieto-Temporal Functional Connectivity
abstract: In everyday life, the brain is bombarded with a multitude of concurrent and competing stimuli. Only some of these enter consciousness and memory. Attention selects relevant signals for in-depth processing depending on current goals, but also on the intrinsic properties of stimuli. We combined behavior, computational modeling, and functional imaging to investigate mechanisms supporting access to memory based on intrinsic sensory properties. During fMRI scanning, human subjects were presented with pictures of naturalistic scenes that entailed high levels of competition between possible target objects. Following a retention interval of 8 s, participants judged the location (same/different) of a target object extracted from the initial scene. We found that memory performance at retrieval increased with increasing object salience at encoding, indicating a “prior entry” for salient information. fMRI analyses revealed encoding-related activation in the posterior parietal cortex, selectively for salient objects that were later remembered. Moreover, parietal cortex showed increased functional coupling with the medial-temporal lobe, for remembered objects only. These findings reveal a parietotemporal circuit that integrates available sensory cues (based on attention-grabbing saliency signals) and current memory requirements (storing objects' locations) to encode object-related spatial information in working memory.

title: Perceptual salience affects the contents of working memory during free-recollection of objects from natural scenes
abstract: One of the most important issues in the study of cognition is to understand which are the factors determining internal representation of the external world. Previous literature has started to highlight the impact of low-level sensory features (indexed by saliency-maps) in driving attention selection, hence increasing the probability for objects presented in complex and natural scenes to be successfully encoded into working memory (WM) and then correctly remembered. Here we asked whether the probability of retrieving high-saliency objects modulates the overall contents of WM, by decreasing the probability of retrieving other, lower-saliency objects. We presented pictures of natural scenes for 4 s. After a retention period of 8 s, we asked participants to verbally report as many objects/details as possible of the previous scenes. We then computed how many times the objects located at either the peak of maximal or minimal saliency in the scene (as indexed by a saliency-map; Itti et al., 1998) were recollected by participants. Results showed that maximal-saliency objects were recollected more often and earlier in the stream of successfully reported items than minimal-saliency objects. This indicates that bottom-up sensory salience increases the recollection probability and facilitates the access to memory representation at retrieval, respectively. Moreover, recollection of the maximal- (but not the minimal-) saliency objects predicted the overall amount of successfully recollected objects: The higher the probability of having successfully reported the most-salient object in the scene, the lower the amount of recollected objects. These findings highlight that bottom-up sensory saliency modulates the current contents of WM during recollection of objects from natural scenes, most likely by reducing available resources to encode and then retrieve other (lower saliency) objects.