Detection of operator performance breakdown in a multitask environment

Yoo, Hyo-Sang. Ph.D., Purdue University, December 2015. Detection of Operator Performance Breakdown in a Multitask Environment. Major Professor: Steven Landry. The purpose of this dissertation work is: 1) to empirically demonstrate an extreme human operator’s state, performance breakdown (PB), and 2) to develop an objective method for detecting such a state. PB has been anecdotally described as a state where the human operator “loses control of the context” and “cannot maintain the required task performance. ” Preventing such a decline in performance could be important to assure the safety and reliability of human-integrated systems, and therefore PB could be useful as a point at which automation can be applied to support human performance. However, PB has never been scientifically defined or empirically demonstrated. Moreover, there exists no method for detecting such a state or the transition to that state. Therefore, after symbolically defining PB, an objective method of potentially identifying PB is proposed. Next, three human-in-the-loop studies were conducted to empirically demonstrate PB and to evaluate the proposed PB detection method. xvi Study 1 was conducted: 1) to demonstrate PB by increasing workload until the subject reports being in a state of PB, and 2) to identify possible parameters of the PB detection method for objectively identifying the subjectively-reported PB point, and determine if they are idiosyncratic. In the experiment, fifteen participants were asked to manage three concurrent tasks (one primary and two secondary tasks) for 18 minutes. The primary task’s difficulty was manipulated over time to induce PB while the secondary tasks’ difficulty remained static. Data on participants’ task performance was collected. Three hypotheses were constructed: 1) increasing workload will induce subjectivelyidentified PB, 2) there exists criteria that identify the threshold parameters that best detect the performance characteristics that maps to the subjectively-identified PB point, and 3) the criteria for choosing the threshold parameters are consistent across individuals. The results show that increasing workload can induce subjectively-identified PB, although it might not be generalizable — 12 out of 15 participants declared PB. The PB detection method was applied on the performance data and the results showed PB can be identified using the method, particularly when the values of the parameters for the detection method were calibrated individually. Next, study 2 was conducted: 1) to repeat the demonstration of inducing PB, 2) to evaluate whether the threshold parameters established in study 1 for the PB detection method can be used in a subsequent study, or whether they have to be re-calibrated for each study, and 3) to examine whether a specific physiological measure (pulse rate) can be used to identify the subjectively-reported PB point. Study 2 was conducted in the same task environment (three concurrent tasks) as study 1. Three hypotheses were constructed: 1) xvii increasing workload will induce subjectively-identified performance breakdown, 2) the threshold parameters established from study 2 will be the same as those from study 1 for all participants and will perform approximately as well or better, and 3) there exists criteria for choosing the threshold parameters that captures the characteristics at the subjectively-reported PB point using the PB detection method on pulse rate data. The results show that increasing workload induces the same participants (12 out of 15) from study 1 to declare PB. Also, it was found that the threshold parameters established in study 1 for the PB detection method cannot be reliably used in a subsequent study, and suggest that it may require re-calibration for each study. The results provided no evidence that pulse rate data can be used to detect PB. Study 3 was conducted: 1) to determine if PB is induced by the primary task workload or is affected by the presence of the secondary tasks, and 2) to re-test whether threshold parameters from study 1 can be used in a subsequent study. In study 3, the same participants from study 1 and 2 were only asked to perform the primary task while its difficulty increased in a similar manner to the first two studies. Two hypotheses were established: 1) PB will occur without the secondary tasks being present, and 2) the threshold parameters established from study 3 will be the same as those from study 1 and/or study 2 for all participants and will perform approximately as well or better. No participants declared PB without the secondary tasks present, even though the primary task workload was the same. Again, it was verified that the threshold parameters established in study 1 and 2 for the PB detection method cannot be used in a subsequent study, and suggest that it may require re-calibration for each study.