Bounded Optimal State Estimation and Control in Visual Search: Explaining Distractor Ratio Effects

Bounded Optimal State Estimation and Control in Visual Search: Explaining Distractor Ratio Effects Christopher W. Myers (christopher.myers.29@us.af.mil) Air Force Research Laboratory Wright-Patterson AFB, OH USA Richard L. Lewis (rickl@umich.edu) University of Michigan Ann Arbor, Michigan USA Andrew Howes (howesa@bham.ac.uk) University of Birmingham Birmingham, UK Abstract We demonstrate that an ideal observer model bounded by known limitations of the human visual system can explain empirical evidence concerning two effects of distractor ratios on visual search—effects that have previously been explained with salience-based models. The model makes optimal state estimations based on Bayesian estimates of stimuli localiza- tion and optimal control decisions of where to fixate in order to maximize task performance. Analysis of the model’s be- havior under different task strategies and different constraints on the visual system reveal which aspects of the model are re- sponsible for the effects: the distractor-ratio effects on number of fixations is a signature of optimal state estimation in the face of noisy spatial information, and the saccadic-bias effect is a signature of both optimal control and estimation under these same bounds. Keywords: optimal state estimation, optimal control, ideal ob- server models, visual salience, distractor ratios Introduction Visual search is so ubiquitous that we probably hardly notice ourselves doing it. We search for our car keys on a cluttered desk, for our family at the market, or for a reference in text. The current paper addresses how one adapts during visual search by determining what information to visually inspect. We address each of these questions through the development of a model based on the optimal integration of perceived in- formation given a set of known constraints on the human vi- sual system. In a review of the literature on eye movements, Kowler (2011) describes two general approaches to modeling visual search processes. First are map-based approaches, such as salience maps (Itti, 2006; Itti & Koch, 2000) and activation maps (Pomplun, 2003; Wolfe, 2007), where information is accumulated and processed to produce a topogrpahical map. Peaks in the map represent areas/items that differ from their surround, that contain attributes of the target, or both. Map peaks are used to guide search through the display using some peak selection routine, such as a greedy heuristic (Pomplun, 2003) or winner-take-all algorithm (Itti & Koch, 2000). In general, the map-based approach assumes that saccades are programmed to move the fovea to an area of a stimulus that stand out from its surroundings or that is similar in some way to a search goal. Alternatively, visibility models (Kowler, 2011) such as ideal observer/searcher models (Geisler, 2011; Myers, Gray, & Sims, 2011; Najemnik & Geisler, 2008; Baron & Klein- man, 1969), assume that saccades are programmed to direct foveated vision to areas of impoverished acuity in order to maximize information gained in service of task performance. Najemnik and Geisler (2005) found that the number, and spa- tial distribution, of saccades to find a target could be predicted by a model in which each saccade was directed to the ideal location (i.e., the highest probability of finding the target). Their model was sensitive to known human constraints on vi- sion (e.g., decreasing contrast sensitivity with increasing ec- centricity). Hence, saccadic selectivity could be considered a process that maximizes search performance by considering the effect of the eyes’ subsequent fixation location. In the current paper we build on the ideal observer ap- proach by deriving a boundedly optimal adaptive visibil- ity model capable of capturing empirical phenomena that demonstrate adaptation to changes in the proportion of avail- able environmental features. More specifically, we use the model to explain phenomena associated with the distractor ratio paradigm (Bacon & Egeth, 1997; Shen, Reingold, & Pomplun, 2000; Zohary & Hochstein, 1989)—phenomena that have previously been given interpretations in terms of salience maps. Key to this explanation is the incorporation of constraints on the representation of spatial information in the periphery into an ideal observer analysis . The adaptive visibility model has a simple structure that decomposes visual search into optimal state estimation (the integration of perceptual evidence into a task-relevant repre- sentation of the external stimulus) and optimal control (the choice of overt task responses and information gathering ac- tions; Stengel, 1994). The model incorporates a small number of constraints intended to abstractly characterize important properties of a noisy, foveated vision system (Tanner, 1961). Bayesian state estimation is used to optimally integrate the noisy percepts across fixations in service of two control deci-