Preventing Postcompletion Errors: How Much Cue Is Enough?

Preventing Postcompletion Errors: How Much Cue Is Enough? Michael D. Byrne (byrne@acm.org) Departments of Psychology and Computer Science Rice University, 6100 Main St., MS-25 Houston, TX 77005 USA Abstract as well as significant in the real world. For example, one form of postcompletion error is reportedly common in electronic voting machines: voters forget to press the final “cast vote” button after making all their selections, thus failing to actually vote (see Everett, Greene, Byrne, Wallach, Derr, Sandler, & Torous, 2008). Legitimate practical consequences naturally lead to questions about what can be done to mitigate such errors. In many cases, the solution is obvious: re-design the task so that there are no postcompletion steps. This is the strategy now used in many ATMs: the cash is not dispensed until the user has retrieved his or her card. While this is almost certainly the optimal solution in many cases, there are many other cases where this approach is not possible, such as already-deployed hardware-based systems. The next most obvious mitigation strategy is to provide some kind of cue to let the user know that such an error has occurred (or is possible). Again, this is the solution adopted by many ATMs; they emit a loud beeping sound and/or flash a light near the card slot after dispensing cash. But how effective are such cues? This is exactly the question asked by Chung and Byrne (2004; 2008). In one of their experimental conditions, they introduced a cue which completely eliminated postcompletion errors by their experimental subjects. The cue in this case was a pair of colored, blinking arrows which pointed to the button which needed to be pressed at the time it needed to be pressed. This cue has three properties that Chung and Byrne considered critical: • Salience. The display used was a grayscale display, so the blinking colored arrows had a high level of perceptual salience. • Specificity. Unlike the beeping of the ATM, which is non-specific, the cue used made it clear to the user exactly what to do: the arrows pointed directly to the relevant button on the display. • Just-in-time. The cue appeared right at the moment that action on the critical button needed to be taken. Thus, it did not rely on the user to have to remember to take action at a specific time. Chung and Byrne (2008) also describe an ACT-R model of the postcompletion error which demonstrates the effectiveness of the cue. As the model was constructed, it depended critically on all three of these properties to prevent the error. The model had to notice the cue (salience), had to have knowledge about what the cue meant (specificity), and had to be able to act on that knowledge immediately (just- Postcompletion error (Byrne & Bovair, 1997) is a robust form of routine procedural error that can be difficult to prevent. However, Chung and Byrne (2004, 2008) reported the complete elimination of this error with a highly aggressive cue, one which was just-in-time, highly specific, and highly visually salient. This paper reports three experiments designed to investigate the role of each one of these factors by using cues which have only two of the three key properties. Surprisingly, salience appears to be the least critical property, and being just-in-time seems to be the most vital. This has important implications for the mitigation of real-world postcompletion errors. Keywords: human error; postcompletion error; routine procedures; visual cues; visual salience Introduction Even in the execution of known routine procedures, people make non-random errors. Everyone has had this experience, whether it is leaving one’s bank card in an automated teller machine (ATM) or failing to attach a promised file to an email message. While many such errors have little or no real cost, many such errors have dire consequences, including loss of human life (Casey, 1998). A particularly robust type of routine procedural error is the postcompletion error (or PCE; Byrne & Bovair, 1997). Postcompletion errors occur when the primary goal of a task is fulfilled and terminal step (or steps) of the procedure, which occur after this, are omitted. Standard examples of this error include the aforementioned bank card left in an ATM, leaving the original document on a photocopier or flatbed scanner, and forgetting to replace the gas cap when filling up the car; most people are personally familiar with at least one postcompletion error. While postcompletion errors have been discussed in the human-computer interaction literature for some time (e.g., Young, Barnard, Simon, & Whittington, 1989; Polson, Lewis, Reiman, & Wharton, 1992), it is only in the last few years that they have begun to receive more substantial attention, both theoretical (e.g., Altmann & Trafton, 2002) and empirical (Byrne & Davis, 2006; Li, Blandford, Cairns, & Young, 2005; Li, Cox, Blandford, Cairns, & Abeles, 2006; Ratwani, McCurry, & Trafton, 2008). This makes postcompletion errors unique among routine procedural errors, as errors in such tasks are generally not robust enough to lend themselves to controlled experimentation. Postcompletion errors, however, are both pervasive and robust, making them both tractable to study in the laboratory