Towards a Flexible, Reusable Model for Predicting Eye Movements During Visual Search of Text

Towards a Flexible, Reusable Model for Predicting Eye Movements During Visual Search of Text Tim Halverson (thalvers@cs.uoregon.edu) Anthony J. Hornof (hornof@cs.uoregon.edu) Department of Computer and Information Science, 1202 University of Oregon Eugene, OR 97403-1202 USA directly on a number of previous studies of structured, menu- like visual layouts. The purpose of this modeling effort is to further clarify and build a framework for understanding (scientifically) and predicting (scientifically and for design purposes) how people integrate perceptual, strategic and motor processes in visual search. This paper describes the evolution of a visual search model from a constrained, random search strategy into a robust and flexible model of menu search that accounts for a wide variety of eye movement data. We believe the resulting model, while developed using data from one task, has been evolved by the analysts with sufficiently few task-specific requisites. That is, the model is flexible and reusable. Abstract Visual search is an integral component in many human activities. The eye movements produced during such activities can provide valuable information about people’s cognitive processes. This research investigates, with detailed eye movement data analysis and computational cognitive modeling, the perceptual, strategic, and oculomotor processes people use to visually search. A cognitive model is evolved in a principled manner based on eye movement data, past modeling efforts, and recent psychological literature. In the model, re-usable, parsimonious, local strategies interact with perceptual-motor constraints to predict the bulk of the eye movement data, including aspects of the data that appear to require task-specific global strategies in addition to fixation-to-fixation local strategies. The analysts evolve a base level model with a random strategy into a robust and reusable model with a flexible strategy that could work with a wide range of visual stimuli. Building on Previous Visual Search Models This research builds directly on previous research of menu search. Hornof (2004) studied the visual search of layouts with and without a useful visual hierarchy. The task relevant to the current research is the visual search of layouts without a visual hierarchy. Figure 1 shows a sample layout from the experiment. Sixteen participants searched four different screen layouts for a precued target object. Each layout contained one, two, four, or six groups. Each group contained five objects. The groups always appeared at the same physical locations on the screen. One-group layouts used group A. Two-group layouts used groups A and B. Four-group layouts used groups A through D. Each trial proceeded as follows: The participant studied and clicked on the precue; the precue disappeared and the layout appeared; the participant found the target, moved the mouse to the target, and clicked on the target; the layout disappeared and the next precue appeared. Hornof (2004) presented models that predicted and Keywords: cognitive modeling; visual search; EPIC; eye movements Introduction The visual search strategies people employ have a substantial effect on the time it takes people to find a target in a visual layout. A fair amount of research has been done on visual search strategies people use. For example, Shen, Reingold, and Pomplun (2003) found that people tend to shift their visual search strategy very quickly based on which visual feature is most informative for a given layout. Burke, et al. (2005) found that people ignore the most salient objects that do not relate to the task, flashing banner advertisements, in simulated web pages. One way to better understand what visual search strategies people use, and why they use them, is through computational cognitive modeling. The models instantiate the theory, make testable numeric predictions, and facilitate identification of unanswered questions. Several computational models of visual search have been proposed (e.g. Pomplun, Reingold, & Shen, 2003; Wolfe, 1994). For the most part, these computational models of visual search account for one or two of the perceptual, strategic, or oculomotor processes involved in visual search, but not all three. Ideally, a model of visual search would explain some aspect of each process involved in visual search. This research proposes a flexible and reusable computational cognitive model of text search that builds Figure 1. A 6-group layout. The precue, in the top left, would disappear when the layout appeared. The gray text did not appear during the experiment.