Human Sensitivity to Variability Information in Detection Decisions

INTRODUCTION Technological developments often result in the automatization of many operations, particularly those that involve complex processing, handling of hazardous materials, and remote operations. Many of the automated systems include data acquisition subsystems (DAS), which comprise a variety of sensors that monitor important system parameters. The sensor readings are generally sent to a global control system that implements decision rules for control of the system and fine-tunes information to be displayed to a human operator (Yamasaki, 1996). The role of the human operator is usually that of monitoring events and diagnosing and rectifying system failures that the system was not designed to handle. Although automated systems tend to be highly reliable, the increase in complexity associated with automation often generates an increased likelihood for component failures and changes in system reliability. One of the most common bottlenecks found in complex system operation is associated with proper sensor function. As a result, reinforcements such as arrays of sensors to measure a particular variable and microprocessors at the local level of measurement (Ohba, 1992) are included in systems, and access to the DAS is made available to human operators who have to diagnose system problems. In diagnosing these problems, operators may have to evaluate visual displays of sensor data about multiple system variables (e.g., energy meter readings and transformer ratios in power system operation and control). When operators have to use displayed information to form a decision, they will make good decisions if they are sensitive to differences in source reliability. The accuracy with which the operator can perform these tasks may be reduced if any reliability estimates depend on statistical estimates made by the human decisionmaker. One statistical property that determines the reliability of information is the underlying variability in the source. An informational source, or sensor, that yields a consistent measure for a given quantity (x) is more reliable than an informational source that yields an inconsistent measure (x [+ or -] n) of the same quantity. An optimal decision maker is sensitive to this fact and places greater weight on sources that show less variability. However, in both auditory (Berg, 1990) and visual (Montgomery & Sorkin, 1996) signal detection tasks for which reliability was defined in terms of the underlying source variability, observers were relatively ineffective at weighting sources in terms of reliability. In these two studies, observers had to be sensitive to the underlying source variability and also had to use this information to make a decision between two options (a two-alternative decision). The observed loss in weighting efficiency may have been linked to an inability of observers to identify differences in source variability or an inability to use this information to weight sources accordingly. Knowing where the limitations lie can provide insights into how to display information for more efficient data utilization. The current set of studies investigated the effects of these tasks (i.e., extracting and using reliability information) on observer efficiency in combining information from graphical elements presented on a visual display to make a two-alternative decision. The first study addressed observers' accuracy in identifying relative differences in source reliability based on the trial-to-trial variability of graphical elements that were similar to those used in the Montgomery and Sorkin (1996) study. The second study examined observers' efficiency in extracting and using this information with a multi-element two-alternative decision task. The third study investigated observers' ability to adjust weights to match the predicted ideal weights. EXPERIMENT 1 Early evidence suggests that humans are less than ideal intuitive estimators of variability, because factors other than the underlying variability in the data influence perceived variation. …