# Presentation: 714142

## Fortuitous Limitations to the Utility of Training Systems 

- Robert S. Kennedy
- RSK Assessments
- Kay M. Stanney
- University of Central Florida

**Cognitive Systems:  Human Cognitive **
**Models in System Design**
- Hosted by Sandia National Laboratories, University of New Mexico,
- and United States Naval Research Laboratory
**Santa Fe, New Mexico / July 6th - July 8th, 2005**

## Who will truly be able to benefit from advanced training technologies? 

- Tools for training and evaluation
- Tools for augmenting performance of individuals and teams
- Tools for automating human performance
- All of these technologies are intended to enhance training and operational performance
- Yet, there may be unforeseen limitations to their ubiquitous use

## Tools for Training and Evaluation

## Tools for training and evaluation

- Advanced training systems, such as virtual environments, allow for unprecedented realism and interactivity
- Yet, some individuals may not be able to fully benefit from such technology due to:
- Emetic response
- Dropouts
- Individual factors (gender vs. susceptibility)
- Sopite syndrome
- Flashbacks
- Long-term aftereffects
- Perceptual illusions
- At-risk individuals
- Negative training

## Emetic response

- Simulator Exposure		0.1%
- Riding in automobiles		1.0%
- Virtual Environments		1.5%
- Of  1028 participants, 15 (1.46%) vomited during or after exposure
- 7 M (1.2% of M); 8 F (1.9% F)

- Kingdon, K., Stanney, K.M., & Kennedy, R.S. (2001).  Extreme responses to virtual environment exposure.
*The 45th Annual Human Factors and Ergonomics Society Meeting* (pp. 1906-1910).  Minneapolis/St. Paul MN, October 8-12, 2001

## Time to emetic response 

- 73% of those who had an emetic response did so within 30 min of exposure
- Average exposure:
-   26.9 min (+/- 15.7 min)
- Range: 3 – 60 min

- Kingdon, K., Stanney, K.M., & Kennedy, R.S. (2001).  Extreme responses to virtual environment exposure.
*The 45th Annual Human Factors and Ergonomics Society Meeting* (pp. 1906-1910).  Minneapolis/St. Paul MN, October 8-12, 2001

## Dropouts

- Of 1097 participants, 133 participants (~12%) requested early termination of session (64 M[12.1%] / 69 F[14.9%])
- Dropout distribution increased as exposure duration increased
- 15min, 9 dropouts[16.7%]
- 60 min, 57 dropouts[42.8%]

- Stanney, K.M., Kingdon, K., & Kennedy, R.S. (2002).  Dropouts and aftereffects:  examining general accessibility to VE technology.
*The 46th Annual Human Factors and Ergonomics Society Meeting* (pp.2114-2118).   Baltimore, MD, September 29-October 4, 2002.

## 50.7% of dropouts did so                                                 within first 20 min of exposure
Average dropout time
   was 24 min (SD 13.6)
Range was 
   3 – 59 minutes

- Average dropout time
-    was 24 min (SD 13.6)
- Range was
-    3 – 59 minutes

- Time to dropout

- Stanney, K.M., Kingdon, K., & Kennedy, R.S. (2002).  Dropouts and aftereffects:  examining general accessibility to VE technology.
*The 46th Annual Human Factors and Ergonomics Society Meeting* (pp.2114-2118).   Baltimore, MD, September 29-October 4, 2002.

## Individual factors

- Individual factors make a difference
- Susceptibility may be more telling than gender

- D. Graeber, Dissertation Data

## Sopite Syndrome

- Of 960 participants, 43.8% experienced drowsiness after VE exposure
- Drowsiness positively correlated with VE duration
- 60-min group experienced 54% more severe drowsiness as compared to 15-min group
- May be an indication of sopite syndrome, characterized by lowered arousal or mood during or after VE use
- If sopite syndrome occurs among VE users, likely to affect performance without being fully detected by afflicted person

- Stanney, K.M., & Kennedy, R.S. (1998). Aftereffects from virtual environment exposure:  How long do they last? *Proceedings *
*of the 42nd Annual Human Factors and Ergonomics Society Meeting *(pp.  1476-1480).  Chicago, IL, October 5-9.

## Flashbacks

- Flashbacks (i.e., visual illusion of movement or false sensations of movement, when _**not**_ in VE)  experienced immediately after VE exposure by 144 of 960 participants (15.0%)
- High incidence level for what is thought to be a rare outcome from VE exposure
- Could affect post-exposure performance

- Stanney, K.M., Kingdon, K., Nahmens, I., & Kennedy, R.S. (2003).  What to expect from immersive virtual environment exposure:
-  Influences of gender, body mass index, and past experience.  *Human Factors*, 45(3), 504-522

## Long-term aftereffects

- Adverse affects post exposure could compromise human performance
- At 2-4 hr post-exposure, 73% of participants still had symptoms substantially higher than pre-VE exposure
- More than 4 hr after VE exposure, 35% of participants still reported SSQ symptoms higher than pre-VE exposure levels

## Long-term aftereffects

| SSQ | N | O | D | TS |
| --- | --- | --- | --- | --- |
| Pre-exposure* (n=960) | 2.11 | 3.26 | 0.12 | 2.47 |
| 0 min (n=960) | 21.42 | 20.93 | 29.75 | 26.72 |
| 15 min (n=960) | 15.02 | 16.66 | 21.84 | 19.97 |
| 30 min (n=960) | 14.20 | 16.94 | 19.42 | 19.14 |
| 45 min (n=960) | 13.13 | 16.81 | 17.59 | 18.17 |
| 60 min (n=960) | 12.56 | 16.49 | 16.79 | 17.57 |
| 2-4 hr (n=366) | 16.81 | 14.50 | 12.89 | 17.21 |
| > 4 hr (n=366) | 5.43 | 9.81 | 6.88 | 8.82 |
| ~ 24 hr (n=366) | 2.23 | 4.08 | 3.59 | 3.85 |

- n=366, number that returned take-home SSQ.

- Stanney, K.M., Kingdon, K., Nahmens, I., & Kennedy, R.S. (2003).  What to expect from immersive virtual environment exposure:
-  Influences of gender, body mass index, and past experience.  *Human Factors*, 45(3), 504-522

## Total SSQ Severity

- 18.75

- 32.76

- 1.94

- Time Post-Exposure
- (min)

- SSQ Mean Score

- Stanney, K.M., & Kennedy, R.S. (1998). Aftereffects from virtual environment exposure:  How long do they last? *Proceedings *
*of the 42nd Annual Human Factors and Ergonomics Society Meeting *(pp.  1476-1480).  Chicago, IL, October 5-9.

## It’s Not Over When It’s Over

**Self-report **
**of symptoms**

**Repeated Exposures**

**Severity of Symptoms**

**Objectively**
**measured**
**postural**
**instability**

## Perceptual illusions

- When sensorial transpositions are used in simulators of VEs (e.g., replace one sense with another), there is an opportunity for perceptual illusions to occur
- With perceptual illusions, certain perceptual qualities perceived by one sensory system are influenced by another sensory system
- This can lead to misperceptions upon post-exposure that could affect operational performance

## Negative training

- Adopt behaviors in training system that could negatively impact real world performance
- For example, minimize head movements to minimize pseudo-Coriolis (i.e., experienced when head is tilted during illusory self-rotation induced by moving visual stimuli) while doing flight maneuvers in simulator
- Normally pilots do a lot of head and eye movements, thus may adopt adverse habits – negative habit acquisition to get around simulator problems

## Tools for Augmenting Performance of Individuals and Teams 

## Tools for augmenting performance of individuals and teams

- Augmented cognition seeks to substantially extend human abilities / performance via computational technologies explicitly designed to address human information processing (HIP) limitations
- Leverage diagnostic psychophysiological (e.g., EEG, fNRI) sensors to gauge and detect HIP bottlenecks and then employ augmentation strategies to overcome limitations

## Realizing augmented cognition

- To realize augmented cognition, first must characterize “cognitive state” to monitor and appropriately regulate HIP bottlenecks
- Use neural signatures as diagnostic tool of “cognitive load,” which can be measured in real-time while an individual interacts with training / operational system
- Yet, some individuals may not be able to fully benefit from such technology due to:
- Neurological, psychiatric, sleep disorders, drug use
- Fatigue, environmental stressors, sleep loss
- Effects of practice

## Neurological, psychiatric, sleep disorders, drug use

- May not be able to obtain diagnostic neural signatures for those with neurological, psychiatric and sleep disorders, and those using illegal or prescription drugs that may affect the brain (e.g. narcotics, barbiturates or anti-psychotics)

## Subject state

- Subject states can change how brain carries out its jobs, thereby altering neural signatures
- Fatigue
- Environmental stressors
- Sleep loss

## Effects of practice

- Practice can change what neural signature looks like
- Parts of brain that are activated may change based on how practiced an individual is
- Part of brain used in acquisition of skill may change to another part of the brain after skill has been acquired
- Neural signatures will have to make allowance for human practice issue (i.e., habituation)

## Tools for Automating Human Performance

## Tools for automating human performance

- Well-designed automation can enhance human- system performance
- Yet, adaptive automation changes nature of work
- Sometimes given function executed by human, at other times by automation, and at still others by both human and computer
- Vigilance-arousal continuum
- If offload tasks from executive function, then ability to monitor those tasks reduced - complacency
- Thus, automation indices should be sensitive to changes in operator arousal

## Automation surprises

- “...28 of the 58 controllers responding to the survey indicated instances in which they had been ''surprised" by a reconfiguration of the system that had been carried out by a remote operator; at the time they were not aware of the reconfiguration, but only discovered it later, when they tried to perform operations that failed in the new reconfigured mode. The potential for such mode errors... is perhaps an inevitable downside of the flexible aspects of some automation functions.”

- Wickens, C.D., Mavor, A.S., Parasuraman, R., & McGee, J.P. (Eds.). (1998). The Future of Air Traffic Control: Human Operators and Automation.
- Commission on Behavioral and Social Sciences and Education.  Washington, DC:  National Academy Press.

## Automation and trust

- “...excessive trust of or excessive mistrust of automation on the part of controllers can lead to problems. The former can lead to complacency and reduced situation awareness, the latter to disuse or under-utilization.” (Wickens et al., 1998
- Yet, there are individual differences in inclination to trust automation – Complacency-Potential Rating (Singh, 1993) measures propensity to display complacent behavior and is related to boredom potential and cognitive failure (Prinzel et al., 2001)

- Prinzel, L.J., DeVries, H., Freeman, F.G& ., Mikulka, P. (2001).  Examination of automation induced complacency and individual differences.  Technical
- Report  NASA/TM-2001-211413. Langley Research Center,, Hampton, VA.
- Singh, I.L., Molloy, R., & Parasuraman, R. (1993).  Automation-induced “complacency”:  Development of a complacency-potential scale.  International
- Journal of Aviation Psychology, 3, 111-122.
- Wickens, C.D., Mavor, A.S., Parasuraman, R., & McGee, J.P. (Eds.). (1998). The Future of Air Traffic Control: Human Operators and Automation.
- Commission on Behavioral and Social Sciences and Education.  Washington, DC:  National Academy Press.

## Automation and culture

- Automation is viewed very differently by  different cultures (McClumpha & James, 1994) and thus may be differentially affective based on a user’s cultural background

- McClumpha, A.J., & James, M. (1994). Understanding automated aircraft. In M. Mouloua & R. Parasuraman (Eds.),
*Human Performance in Automated Systems. * Hillsdale, NJ: Erlbaum.

## Individual factors

- Thus, automation strategy may have to vary by:
- Operator arousal
- Complacency potential
- Culture
- and other individual factors

## Conclusions

- While advanced training technologies are intended to enhance training and operational performance
- Unforeseen limitations to their ubiquitous use include:
- Adverse effects during and after exposure that may preclude some individuals from being able to use the technology
- Inability to obtain diagnostic neural signatures to drive augmentation strategies from some individuals
- Inability to effectively modulate automation due to individual differences such as operator arousal, complacency, and culture