The Effect of Immediate Accuracy Feedback in a Multiple-Target Visual Search Task

The Effect of Immediate Accuracy Feedback in a Multiple-Target Visual Search Task Nada Attar (nattar@cs.umb.edu) Chia-Chien Wu (chiachie@cs.umb.edu) Marc Pomplun (marc@cs.umb.edu) Department of Computer Science, University of Massachusetts at Boston 100 Morrissey Boulevard, Boston, MA 02125-3393, USA Abstract Visual search is one of the most common behaviors in daily life. Studies of visual search, however, have mainly focused on how visual properties of stimuli affect search efficiency. The current study examined the effects of immediate auditory feedback for each selection during a multiple-target visual search task. In a feedback condition, one of two different sounds was played, indicating whether the subject had reported a correct detection, i.e., was fixating a target. In the neutral sound condition, subjects always received the same sound, regardless of whether they had visually selected a target or a distractor. We analyzed overall performance measures such as trial duration and the proportion of correct target detections and correctly completed trials. Furthermore, we analyzed pupil size as a measure of cognitive effort. The results show that pupil dilation was greater and search accuracy was better when subjects were given feedback than when they only received a neutral sound. In summary, the present study demonstrates that immediate feedback may increase cognitive effort, leading to more accurate task performance, with enhancement of specific components of search behavior. Keywords: Attention; visual search; eye movements; pupil dilation; auditory feedback. Introduction Visual search is arguably the most common task in our daily life. It has therefore received substantial research interest, with a focus on the factors that may affect the search performance. In a typical visual search task, participants are asked to report if a visually distinctive target is present among a set of distractors in a given scene. The gauge for the task difficulty is the accuracy and the “search slope”, i.e., the slope of the response time (RT) that typically increases linearly with greater search set size. Wolfe (1998) used the term “search efficiency” as a term describing how search for a target can be influenced by distractors being added to the visual search display. Most studies of visual search have varied either the visual features of the stimuli or the prior experience participants bring to the task. For example, Duncan and Humphreys (1989) manipulated the similarity between targets and distractors in their study, and Cohen and Ivry (1991) used the density of objects to vary the search difficulty instead. Variation in previous knowledge included experiments in which the features (i.e., color or shape) of the target and distractors were maintained from trial to trial and then were reversed unpredictably (Bravo & Nakayama, 1992). Another way of influencing visual search performance is the presentation of feedback to the subject. Feedback was used in many previous studies after each trial only to indicate if the trial has been passed or failed (see Bravo & Nakayama, 1992; Donnelly et al., 2007; Giesbrecht, Sy, & Guerin, 2013). Such “between trials reinforcement” rarely occurs in the real world. That is, during natural viewing, observers typically bring their line of sight to the location of an expected search target. If the fixated object is consistent with their initial expectation, observers will keep the same viewing strategy. Otherwise, the search strategy has to be modified. These confirming and modifying processes based on the instant feedback seem to be commonly used in our search performance. The role of feedback on this sequential search behavior, however, is not well understood. Some prior studies have found that receiving feedback can facilitate search performance. Chun and Wolfe (1996) suggested that viewers use feedback to set their decision criterion in visual search tasks. This feedback can either be implicit (the viewers already know they were correct because they were able to identify the target) or explicit (the viewers are told whether they were correct or incorrect). Wolfe et al. (2007) showed that explicit feedback can have an impact on tasks with low target prevalence, i.e., tasks in which targets appear infrequently and typically yield high proportions of missed targets. Schwark, Sandry, MacDonald, and Dolgov (2012) examined the effect of false feedback on finding targets in a visual search task. They controlled participants' perceived number of misses through explicitly misleading feedback. In the false-feedback condition, participants were falsely informed that they had missed a target even though they had indicated “target absent” correctly. Their result indicated that participants committed a higher number of false alarms due to a shifted criterion but they were able to find low prevalence targets more successfully. Though these studies have shown that feedback can change searchers’ decision criterion and thereby affect search performance, they only used feedback after each trial. As noted above, however, in natural viewing we rarely receive feedback after search performance is terminated. In fact, we normally receive visual feedback on each fixation. This within-trial based feedback has been neglected in most studies. Finally, it is also important to note that sound can affect search performance. Only very few studies investigated the effect of the sound presented during visual search.