Computer Science at Kent Attentional capture in stimulus rich computer interfaces

To explore the degree which attentional set in one task interferes with performance in a secondary task, we have developed a prototype SRRI test system. This comprises a central task involving driving through a virtual maze and the presentation of an intermittent stream of competing stimuli of varying levels of salience. Centrally presented arrows are followed in the driving task and the stream of competing stimuli is presented via a head mounted display. The colour relationship between the central arrows and stimuli in the competing stream is varied. How this "task prescribed" colour relationship impinges upon attentional capture by stimuli in the competing stream is investigated. This experiment has revealed no discernible carryover effect of attentional set from one task to the other in either accuracy or reaction time for detecting the targets in the stream. Thus, the attention system seems to be adept at allocating feature based attention in a way that does not spontaneously migrate from one task to another. Introduction The visual world contains more information than can be processed by any real-time cognitive system. The human brain uses a multi-stage attention mechanism to deal with this problem. At the input level, the eyes can be moved around, selecting parts of the visual world around us with saccades. This visual input from the retina is then subject to a spatial form of attention, which selectively amplifies some regions of the visual field (Posner 1980). Beyond the spatial filter, the attention system can select items for processing based on certain features, such as a particular shape or color. All of these mechanisms are under the control of top-down volitional control, allowing a person to tune their attentional filters as necessary to detect targets that are relevant to their current goals. These mechanisms are clearly critical in allowing us to function in situations of information overload. If a user is aware of the salience of a red light in a cockpit display, attention can be configured to enhance the probability that red stimuli will be selected for further processing by the attentional system. 1 The full reference for this article is, B. Wyble, H. Bowman, and P. Craston. "Attentional capture in stimulus rich computer interfaces. " Technical Report 7-06, Computing Lab, University of Kent at Canterbury, September 2006. Transient Attention and Contingent Capture In addition to relatively static spatial and feature based filters, the visual system also has a reactive method of deploying attention. Within just 50 ms of the onset of a salient visual cue, target detection is enhanced at the cued location, indicating a rapid and reactive deployment of attention (Nakayama & Mackeben 1989). This may be related to the phenomenon of attentional capture (Theeuwes 1994), the theory that a window of attention can be deployed in response to salient visual objects, which makes concurrent target detection more difficult at other spatial locations because this reactive mechanism is already occupied. A debate has centered around the degree to which this reactive attention mechanism is under top-down control. Recent work has demonstrated that the attention settings of the subject, for a particular color, makes it more likely that distractors of that color will be able to capture attention, a phenomenon known as contingent capture (Folk, Leber & Egeth 2002, Serences et al 2005). Recent computational work has demonstrated that this form of attention plays a critical role in our ability to perceive rapidly presented stimuli, especially in the context of Rapid Serial Visual Presentation (Bowman & Wyble 2007; Wyble, Bowman & Potter, submitted). What this work suggests is that the attentional settings of the subject (e.g. in search of a particular colour or shape) affect the ability of stimuli with those features to capture attention. This idea is critical in the present work, in which we test the ability of the attentional set specified in one task, to affect the ability to detect rapidly presented targets in another task. Dual-Task Research There is a large literature on the ability of subjects to switch from one task to another and the delays imposed by the switch (e.g. Arrington & Logan, 2004). The ability of the attentional system to mediate two different tasks at the same time is less well studied, as these paradigms make it difficult to isolate a single cognitive function. However, simultaneous performance is of critical importance for complex interfaces, such as in cockpits. One critical question that arises in such research is how flexible the attentional system truly is. For example, if a subject is looking for a certain color in one task, does that affect their ability to perceive targets in a second task for which that color distinction is irrelevant or reversed? In other words, is there a single attentional filter mechanism that needs to be shared across multiple tasks? Or can each task utilize its own dedicated filter? It is well known that there are fundamental limitations in our ability to do multiple things concurrently. For example, ideas such as central capacity theory (Kahneman 1973, Tombu & Jolicoeur 2003) are firmly established theoretical frameworks. Many of the experimental paradigms that study such central limitations report differences in response times that exist between performing two unrelated tasks. It is consistently found that doing two things at the same time results in slower performance, decreased accuracy, or both. For example, the psychological refractory period (PRP) refers to the increased time to respond to a second stimulus after having responded to a first one (Ruthruff, Pashler, & Hazeltine 2003), a cost which can be observed even for two tasks in different modalities. What is less well studied is the specific pattern of interference between tasks that use the same perceptual system. The question we ask here is to what degree do filters in one perceptual system (e.g. vision) cross tasks that rely on the same system; see Figure 1. This question is of vital importance for the design of any complex interface. If one display requires users to attend to red and ignore green, while another requires the opposite, what costs do these conflicting instructions incur if the user is to process both of these displays simultaneously? Some work has been done that is relevant to this question. Olivers & Watson (In Press) had subjects monitor an RSVP stream of characters for letters of a particular colour. Immediately following this task, subjects had to rapidly count the number of dots of a particular colour in a display. It was found that if the colour of the dots to be counted matched the colour of the distractors in the RSVP task, performance was impaired, indicating that the colour filter established for one task had carry over effects onto a secondary task, even though that distinction was entirely irrelevant. However these two tasks were performed sequentially, not simultaneously. Most & Astur (Under Revision) addressed this question directly, by having subjects maneuver a virtual car through a city, attending to signposted arrows of one colour, while ignoring arrows of another colour. While driving, subjects would occasionally have to break to avoid hitting a motorcyclist, appearing suddenly in front of the car. It was found that subjects were faster at braking in response to cyclists of the same color as the arrows to which they were attending. These results are suggestive of cross-task interference although it could be argued that this is not a true case of dual-task performance, as steering and braking could be considered parts of the same task. What we address here is the degree to which the visual system exhibits interference between concurrent, and unrelated tasks fed from the same modality. The question proposed is whether the same kind of interference demonstrated by Most & Astur will be found for independent tasks. Combining Rapid Presentation and Driving Tasks To test the interference between independent tasks that use a common sensory modality, we combine a colored-arrow guided navigation task, as in Most & Astur, with a task requiring rapid identification of digits presented concurrently in a heads-up display. The colour of each digit is randomly chosen, but subjects are to ignore the color. If the interaction between the colors of the arrows and the motorcyles in the Most & Astur task was due to the fact that subjects were sharing the same visual input between the tasks, we should expect to observe the same effect here. On the other hand, it is possible that if the visual system is being used for two distinct tasks, that a visual filter can be applied to the visual input to one of the tasks without strongly affecting the other. In our experiment, one group of subjects navigate the maze guided by black arrows and the other guided by white. Meanwhile, in a secondary display, they attempt to detect digits that are white, black or coloured. A critical aspect of the current task is that, on average, subjects in both the black and the white arrow groups are seeing the same visual stimulus. The only difference between them is the instructions that they are given at the start of the experiment (e.g. to attend to white or to black arrows). Thus, any differences observed would necessarily be the result of top-down task set.