ARMY RESEARCH LA

Aviator Behavior and Performance as

Affected by Aircrew Life Support and

Protective Equipment

John D. Waugh

Linda T. Fatkin

Debra J. Patton

Linda L. Mullins

Pamela A. Burton

Daniel J. Barker

David A. Mitchell

I :

I .

Approved for public release; distribution is unlimited.
Jack@ is a registered trademark of the University of Pennsylvania.

The findings in this report are not to be construed as an offkial Department of the Army position

unless so designated by other authorized documents.

Citation of manufacturer’s or trade names does not constitute an official endorsement or approval of

the use thereof.

Destroy this report when it is no longer needed. Do not return it to the originator.
t

REPRODUCTION QUALITY NOTICE &

This document is the best quality available. The copy furnished

to DTIC contained pages that may have the following quality

problems:

l Pages smaller or larger than normal.

l Pages with background color or light colored printing.

l Pages with small type or poor printing; and or

l Pages with continuous tone material or color

photographs.

Due to various output media available these conditions may or

may not cause poor legibility in the microfiche or hardcopy output

you receive.

w If this block is checked, the copy furnished to DTIC

contained pages with color printing, that when reproduced in

Black and White, may change detail of the original copy.
Abstract

A methodology for quantifying Army rotary wing aviator performance as

influenced by aircrew life support, survival, and nuclear-biologicalchemical clothing and equipment ensembles was examined in a set of

experimental trials conducted in an AH-64 (Apache) combat mission

simulator. The methodology was based on an aircrew evaluation

procedure originally developed for use in the crew coordination training of

all Army aviators. It uses a set of 13 basic qualities, each with behavioral

anchors and a 7-point rating scale, and it is administered by specifically

trained senior aviator evaluators. Ten crews, two aviators in each, while

fully encumbered, performed three combat missions for record,

representative of typical operational tasks, with one “variation” trial

conducted without the over-water components of the ensemble. Measures

of effectiveness and flight data, as well as stress assessment and equipment

“complaints” citations, were recorded. The results indicated that the

behavior-anchored scores were not sensitive enough to statistically

-discriminate among the independent variables of repeated measures and the

variation trials even though graphically, differences were readily apparent.

Attempts to apply transformations to the data, based on the aviator

subjects’ relative flying experience and their apparent accommodation to

the trials were also statistically unsuccessful. The additional measures

collected did not yield statistically significant discriminations nor did they

correlate well with the evaluation scores. A number of options for

improving the technique are offered.

ii
1

ACKNOWLEDGMENTS

The authors offer thanks to all the people who helped in this project. From the Human

Research and Engineering Directorate (HRED) of the U.S. Army Research Laboratory,

Mr. Gordon Herald, for his expertise in computers, modern and not so modern, and to Mr.

Thomas Davis and Mr. Judah Katznelson for assistance in data reduction. From the U.S. Army

Aviation Warfighting Center, Ft. Rucker, Alabama, Ms. Laura Thompson and CW4 Larry

Thompson, for helping us understand the AH-64 combat mission simulator (CMS). From the

HRED Field Element, Ft. Rucker, Mr. Richard Armstrong and Mr. David Durbin, for their

technical reviews and all-round assistance in all phases of this project. From the Army Research

Institute (ART), also in Ft. Rucker, Dr. Robert Wright, for sharing insights and data from ART’s

allied investigations. To CW5 Kenneth Donahue, Training and Doctrine Command Systems

Manager Longbow, Ft. Rucker, for his technical review, and to MAJ Steve Walters and CW4

Steve Paris, U.S. Army Aviation Technical Test Center, Ft. Rucker, for working closely with us

in the initial survey of aircrew training manual (ATM) tasks. To CW5 Thomas Connell and

CW5 Howard Swaim, Flight Standardization, Ft. Hood, Texas, for their advice, counsel, and

assistance in our survey of ATM tasks.

We also offer an appreciative and hearty “well done” to that highly motivated group of

associate investigators who were major contributors to this investigation by working long, long

hours during the trials while maintaining their sense of humor throughout: Ms. Jean Breitenbach

and Mr. Ronald Whittaker as well as co-authors Mr. Daniel Barker, Ms. Debra Patton, and Ms.

Linda Mullins, HRED; and to those highly skilled instructor-operators and evaluators manning

the simulator, CW3 C. David Pope, Jr., CW4 Michael Hillwig, CW5 Gary Kilker, Mr. Charles

(Chip) Decker, W02 Joseph O’Neill, W02 Michael Wanamaker. Of course, we are extremely

grateful to the aviators who participated as subjects, whom we cannot thank by name in this

report. Nevertheless, they are highly commended for their dedication to the investigation as

well as their willingness to tolerate strange hours and large doses of hard work and discomfort.

Special gratitude is given to the people at the Western Area Aviation Training Site

(WAATS), Marana, Arizona, who most graciously offered their simulation facilities for the

experiment and who assisted, along with the Arizona National Guard, in the logistics of its

execution: COL George Gluski, CDR, WAATS, LTC James Braman, Chief of the AH64 CMS

facility; his second in command and co-author MAJ David Mitchell; Platoon Leader and master of

the schedule CW4 Don Curry; W02 William Mercer who made the right things happen, day after

day after day, and the whole WAATS staff who cheerfully saw to our many needs and logistics for

the entire 6 weeks worth of trials. Special thanks to MAJ Kenneth Nettles, CDR Arizona Army

National Guard (ARNG) Army Aviation Support Facility, Silver Bell Army Heliport (next door to

WAATS) and his staff for supplying participants, equipment and all manner of support, often on a

short fuze basis.

Also, most worthy of mention are the people we only knew via the telephone but without

whom we would not have been able to conduct the project: From Headquarters, National Guard

Bureau, MAJ Brian West who secured the National Guard’s approval to employ the WAATS

facilities and broadcast our need for participants, and Mr. Howard Manwieler who obtained the

. . . 111
pay and allowance dollarsto support those participants. To Ms. Colleen Heck, ARNG Budget

Officer (Phoenix) who performed all the financial wizardry between the National Guard Bureau,

HRED, and the participants. To the various state National Guard points of contact who did

recruiting as well as arranging, scheduling, and the paper work for the participants: CW3 Russell

Thacker, Utah ARNG; CPT Dale Hall, South Carolina ARNG; W02 David Kelcevick, North

Carolina ARNG.

John D. Waugh

Principal Investigator

iv
CONTENTS

.

.

EXECUTIVE SUMMARY ..........................................

INTRODUCTION ................................................

OBJECTIVE ....................................................

METHOD ......................................................

Subjects ..................................................

APPARATUS.. .................................................

Simulation Facilities ..........................................

Instrumentation .............................................

ALSE ....................................................

Procedures .................................................

RESULTS.. ....................................................

Behavior-Anchored Ratings .....................................

Crew Position ..............................................

Measures of Effectiveness and Flight Data ...........................

Stress Assessment of Aviator Subjects. .............................

Post-Mission Interviews .......................................

DISCUSSION ...................................................

Behavior-Anchored Ratings .....................................

Aviator Experience ...........................................

Post-Mission Interviews .......................................

Stress Assessment of Aviator Subjects. .............................

Measures of Effectiveness and Flight Data ...........................

Discussion Summary. .........................................

CONCLUSIONS .................................................

RECOMMENDATIONS ...........................................

REFERENCES ...................................................

APPENDICES

A. Description of ALSE Components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

B. Hasty Attack Into Horse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

1

9

10

11

11

12

12

12

13

13

18

18

20

22

37

56

60

60

68

69

69

73

78

79

80

83
C. Surveyed ATM Tasks ...................................... 95

D. Sampling of Behavioral Anchors ............................... 10 1

E. Aircrew Coordination Evaluation Grade Slip ....................... 111

DISTRIBUTION LIST ............................................. 115

REPORT DOCUMENTATION PAGE ................................. 117

FIGURES

1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

11.

12.

13.

14.

15.

16.

17.

18.

19.

20.

21.

22.

23.

24.

25.

26.

27.

28.

29.

30.

31.

32.

33.

34.

35.

36.

AH-64 Combat Mission Simulator .............................. 13

SME Survey Results. ....................................... 14

Flying Experience of Participating Subjects ......................... 2 1

Mean Times and Standard Deviations of CPG Tasks .................. 23

Kills, Hits, and Misses ...................................... 25

Percentage Kills, Hits, and Misses .............................. 26

Ownship Exposure Zones .................................... 29

Events: Maximum Exposure Zones .............................. 30

Total Exposure Times and Times Shot At ......................... 30

Acquisition and Hit Probabilities ............................... 3 1

Histogram, MAACL-R Today Combined Data Means ................ 38

Mean Scores for MAACL-R Subscales Interaction ................... 39

Histogram, Mean Anxiety Scores by Time ......................... 40

Histogram, Mean Depression Scores by Time ...................... 40

Mean Depression Scores by Time and Trial ........................ 41

Histogram, Mean Hostility Scores by Time ........................ 42

Histogram, Mean Dysphoria Scores by Time ....................... 42

Histogram, Mean Positive Affect Scores by Time .................... 43

Mean Positive Affect Scores by Time and Trial ..................... 44

Histogram, Subjective Stress Scores by Time ....................... 45

Mean Positive Affect Scores by Trial and Seat ...................... 45

Histogram, SRE Scores by Time ................................ 46

Histogram, Amylase Data Scores by Time ......................... 47

Comparison of Mean Pre-Stress Anxiety Scores ..................... 48

Comparison of Mean Post-Stress Anxiety Scores .................... 49

Comparison of Mean Pre-Stress Depression Scores. .................. 49

Comparison of Mean Post-Stress Depression Scores .................. 50

Comparison of Mean Pre-Stress Hostility Scores .................... 50

Comparison of Mean Post-Stress Hostility Scores ................... 5 1

Comparison of Mean Pre-Stress Negative Affect Scores. ............... 5 1

Comparison of Mean Post-Stress Negative Affect Scores ............... 52

Comparison of Mean Pre-Stress Positive Affect Scores ................ 52

Comparison of Mean Post-Stress Positive Affect Scores ............... 53

Comparison of Mean Pre-Stress Subjective Stress Scores ............... 53

Comparison of Mean Post-Stress Subjective Stress Scores .............. 54

Comparison of Mean Post-Stress Specific Rating of Events Scores ........ 54

2

.
37

38

39

40

41

.

.

.

.

.

I

1

,

I

42.

43.

44.

45.

46.

47.

48.

49.

50.

Comparison of Mean Pre-Stress Amylase Scores .................... 55

Comparison of Mean Post-Stress Amylase Scores ................... 56

Post-Mission Citations-Equipment and Non-comfort ................ 57

Post-Mission Citations by ARI Questionnaire Section. ................ 59

Post-Mission Citations from the ARI Questionnaire Mission

Performance Section ........................................ 60

Standard Deviations of Behavior-Anchored Rating Scores ............... 6 1

Standard Deviations of Rating Scores for the Third Sequential Trial ........ 62

Scores by Trial Sequence ..................................... 63

Scores by Trial Type ....................................... 63

Mean Adjusted Scores by Trial Type ............................ 65

Standard Deviations of Adjusted Rating Scores, Overall and by Trial Type ... 67

Histogram and Normal Curve for Unadjusted and Adjusted Scores-Basic

Quality 12 .............................................. 67

Histogram and Normal Curve for Adjusted Scores With SecondR and VR

Trial Standard Deviations-Basic Quality 12 ....................... 68

Behavior-Anchored Scores by AH-64 Flying Experience-Pilot and CPG .... 70

TABLES

1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

11.

12.

13.

14.

15.

16.

Trials Order of Presentation by Crew and Subject No. ................. 19

Trials Scored by Participating Evaluators .......................... 19

Flying Experience Categories .................................. 22

Results of Both Tukey and Duncan Post Hoc Tests for Platform StabilityCrews .................................................. 27

Mean Platform Stability Range of Velocities-Trial Sequence ............ 28

Mean Platform Stability Range of Velocities-Trial Type .............. 28

Results of Both Tukey and Duncan Post Hoc Tests for Summed Exposure

Zones-Trial Sequence ...................................... 3 1

Mean Sums of Maximum Exposure Zones-Trial Type ................ 32

Results of Both Tukey and Duncan Post Hoc Tests for Probability of

Acquisition-Trial Sequence .................................. 32

Mean Maximum Probabilities of Acquisition-Trial Type .............. 33

Results of Both Tukey and Duncan Post Hoc Tests for Maximum

Probability of Hit-Trial Sequence .............................. 33

Results of Both Tukey and Duncan Post Hoc Tests for Maximum

Probability of Hit-Trial Type ................................ 34

Means for Number of Events, Ownship Hits, Ownship Misses, No Shot at

Ovynship, Percent Shot at, Total Exposure Time, Average Exposure Time

per Event, Sum of Exposure Zones for Trial Sequence ................. 35

Means for Number of Events, Ownship Hits, Ownship Misses, No Shot

at Ownship, Percent Shot At, Total Exposure Time, Average Exposure

Time per Event, Sum of Exposure Zones, by Trial Type ............... 36

Threat Measures Intercorrelations .............................. 37

MAACL-R Today Combined Data Means ........................ 38

3
17. Mean Scores for MAACL-R Subscales: Anxiety, Depression, Hostility,

Positive Affect and Dysphoria . . . . . . . . . . . . . . . . . . . . . . . . . ‘. . . . . . . . 38

18. Mean Anxiety Scores by Time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

19. Mean Depression Scores by Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

20. Mean Depression Scores by Time and Trial . . . . . . . . . . . . . . . . . . . . . . . . 41

2 1. Mean Hostility Scores by Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

22. Mean Dysphoria Scores by Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

23. Mean Positive Affect Scores by Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

24. Mean Positive Affect Scores by Time and Trial . . . . . . . . . . . . . . . . . . . . . . 43

25. Subjective Stress Scores by Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

26. Mean Subjective Stress Scores by Trial and Seat . . . . . . . . . . . . . . . . . . . . . 45

27. SRE Scores by Time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . L . . . . . 46

28. Amylase Data Scores by Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

29. Post-Mission Citation of ALSE Items by Trial Type . . . . . . . . . . . . . . . . . 58

30. Results of Tukey Post Hoc Test, Basic Quality 12, for Trial Type . . . . . . . . 66

3 1. Spearman’s p Correlations: Basic Quality Scores by Flying Experience . . . . . 69

32. Spearman’s p Correlations: Adjusted Basic Quality Scores by Equipment

Citations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

33. Spearman’s p Correlations: Basic Quality Scores by Stress Assessment

Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

34. Spearman’s p Correlations: Adjusted Basic Quality Scores by Weapons

Engagement Measures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

35. Spearman’s p Correlations: Adjusted Basic Quality Scores by Threat

Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
EXECUTIVE SUMMARY

.

This report describes an experiment conducted by the Human Research and Engineering

Directorate of the U.S. Army Research Laboratory (ARL) in support of the Project Manager, Air

Crew Integrated Systems. The purpose of the experiment was to conduct research into the

development of a technique for use in the assessment of future rotary wing aviation aircrew life

support (ALSE) ensembles and components, including survival and nuclear-biological-chemical

(NBC) clothing and equipment.

.

The experiment consisted of a set of trials with ten male AH-64 Apache crews flying

tactical, combat missions in the AH-64 combat mission simulator located at the Western Area

Aviation Training Site, Marana, Arizona. The crews were outfitted in the Army’s current

inventory of NBC, and ALSE, in missions described as hostile threat, over-water survival, and

NBC environments.

Each crew flew four trials or missions. The first trial was a familiarization trial done

primarily to bring the subjects into the condition of having to don and perform with the entire

collection of equipment never before experienced. Three trials for record were then conducted,

two for a repeated measures application and the third trial being a “variation” trial where the

crews did not don the over-water components of the ensemble. The three trials for record were

counterbalanced in their order of presentation to the crews. The mission scenario for the three

trials for record was the same for each crew and each trial. The familiarization trials were all done

with a separate scenario and terrain.

The primary measure of individual crew member’s performance was an adaptation of the

successful crew coordination evaluation procedure developed for Army Crew Coordination

Training in 1995- 1996 by the Army Research Institute for the Behavioral Sciences. The technique

is based on 13 sets of “behavioral anchors,” for each of 13 basic qualities against which crews are

evaluated at the end of their crew coordination training. Aviators are rated along a 7-point scale

from “very poor” through “acceptable” to “superior”. In this experiment, individual performance

was scored by certified evaluators, not against the individual himself, but against the effects of the

encumbering ensemble on the individual, which could possibly degrade his performance. To that

end, the behavioral anchors were edited to make the ensemble the object of the evaluator’s

judgment. A number of other measures were recorded concurrently, including measures of

effectiveness and flight data from the simulator, pre- and post-trial questionnaires and interviews,

and an assessment of the stress levels present in the subjects by a battery of questionnaires and

the assay of salivary amylase, all intended to corroborate the mainline evaluation findings.

5
The evaluation scores that were gathered did not have sufficient sensitivity to the

experimental independent variables for two fundamental reasons: first, too large a variation was

present from crew to crew in the scores, well exceeding the span of a single point of the 7-point

rating scale. Second, the range of awarded scores was too narrow in terms of its overall distribution.

The end points of the scale never appeared in the scores posted in the trials; they all clustered about

the mid-point, between 4 (acceptable) and 5 (good) on the scale. Some of the other measures

collected showed statistical significance with respect to the experiment’s conditions, usually the

second of the two fully encumbered trials and the variation trial. Some systemic problems were

encountered in gathering those data, but then, the data were not intended to be subjected to the

degree of statistical scrutiny reserved for the primary measure. The aviators made a great number of

citations where the ensemble components caused problems or subjectively caused their performance

to be degraded or contributed significantly to their discomfort. Principally, the protective mask,

followed by the body armor and protective overgloves were cited, as well as combinations of

component such as helmet and mask or mask and armor. In the copilot-gunner’s position, the body

armor in front plus the life raft on the back physically prevented full aft cyclic displacement from

either seat. The collection of stress assessment measures showed that the aviators experienced

.significantly more stress in performance of their missions versus pre-trial and recovery after-trials

measures, but the level of stress was not as great as that encountered by participants in other

investigated situations. A moderate amount of psychological stress was concluded to be present.

When the subjects were encumbered in ALSE in a low physical work environment, physiological

stress created by the ensembles was minimal with respect to that in other investigations.

While not statistically viable in this experiment, graphically, it is very apparent that the

subjects improved their performance from the first through the third trials for record in sequence.

Rather than calling this improvement “learning” in the classical sense, “accommodation” is the term

of choice because the subjects already have learned to fly, fight, and work in ALSE. A most

puzzling observation of the evaluation scores revealed that the subjects performed more poorly

during their variation trials, when they wore less equipment and no over-water gear, than during the

second trial for record (recall that these trials were counterbalanced for order of presentation). It

was fully expected that they would perform best in this trial, but it was not the case. It was also

thought that the more experienced pilots (two had at least 5000 total hours logged) would perform

at a higher level than their less experienced brethren. This was not the case either, and the notion of

perhaps weighting the scores in this fashion to potentially increase their sensitivity had to be

abandoned. Adjusting the scores to remove the influence of the suggested accommodation effect

appeared to increase their sensitivity but not quite enough to bring them into statistical significance.
.

.

This is not to say that the concept of employing behaviorally anchored performance

evaluations as a tool will not work for assessing the merit of prototype or proposed future ALSE

ensembles or components. Rather, that the technique should be modified in order to eliminate

those properties that caused its insensitivity. Recommended solutions include reduction of the 7-

point scale to a 5- or even a 3-point scale, leaving less room for variation and encouraging the

certified evaluators to be rather ruthless in their evaluations and to be willing to use the end points

of the scale more so than was observed in this set of trials. The adjusting of scores to cancel the

conjectured performance increases from trial to trial should be continued at least until we have more

knowledge of the aviator’s ability to accommodate to the fully encumbered environment. Finally,

future trials should be conducted, probably with proposed ensembles or components on a

comparative basis, evaluating missions against the current ALSE that would be replaced. That

‘way, a current equipment baseline can be further defined, and meaningful assessments of advanced

technology equipment can be conducted. Include the technique(s) that may be available to assess

the relative levels of the aviator’s situation awareness as another potentially useful factor.

7
AVIATOR BEHAVIOR AND PERFORMANCE AS AFFECTED BY AIRCREW

LIFE SUPPORT AND PROTECTIVE EQUIPMENT

INTRODUCTION

.

Military rotary wing aviators are required, depending on their mission, to don a large

variety of life support and protective equipment and components that are intended to increase

their probability of survival in tactical situations. Among this equipment are items for survival in

the event of over-water “ditching,” protection against nuclear-biological-chemical (NBC) agents,

enemy fire, crashing, fire, eye-damaging lasers, and extreme temperatures, hot and cold. Add to

this equipment an emergency locator transmitter, a side arm, survival knife, flashlight, and so

forth. The average weight of the standard flight uniform, clothing, helmet, and boots is 17.7 lb,

and the average weight of a mission-oriented protective posture (MOPP) IV fully encumbered

ensemble is 57.1 lb (Reardon et al., 1996).

The investigation cited above concluded that discomfort experienced by aviator

participants, in terms of increased core temperatures and dehydration because of increased sweat

rates and equipment pressure points, affected mood and mission endurance. An investigation in a

simulator emulating the RAH-66 Comanche (Sharkey & Schwirzke, 1995) made similar

conclusions and highlighted a significant decrement in the time it took to perform data entry tasks

in MOPP IV. A report of the man-machine integration design and analysis system (MIDAS)

modeling investigation of the AH-64D Longbow Apache cockpit (Shively et al., 1995), which

presented a scenario of flying and fighting tasks for the Longbow Apache mission in concert with

employment of the Jack@ human figure model (no human subjects), stated that the results .“...

clearly show some of the performance decrements associated with today’s protective ensemble.”

Finally, a U.S. Army Research Institute (ART) investigation (Wright, Hartson, & Couch, 1996) in

an AH-64 configured simulator cited the same discomfort, heat stress, “hot spot” pressure points

and endurance difficulties, performance decrements in tasks as well as equipment-cockpit

interferences.

Universal acknowledgment of the problems associated with aviators and ALSE has led to

the establishment of the Project Manager for Air Crew Integrated Systems (PM ACIS) and the

Air Warrior Program described as “... the rotary wing aviation focus for providing a missiontailorable system that standardizes and integrates Aviation Life Support Equipment for Army,

Navy, and Marine Corps aircrew personnel during flight and ground operations” (Metzler, 1995).

The PM ACIS has retained the Human Research and Engineering Directorate (HRED) of the U.S.

Army Research Laboratory (ARL) to develop the capability to conduct early engineering analyses

9
of emerging technologies and designs aircrew life support equipment (ALSE) and protective

components1 and to estimate any potential payoff in mission performance improvement. This

capability is expected to allow both the user proponency and materiel developers to better

standardize and integrate all this equipment as a soldier-based system for the rotary wing aviation

mission. The ARL focus was on the AH-64 Apache as the basis of investigations because it

represents the platform with the most confined cockpit space and highest workload; the

knowledge gained could potentially be extrapolated to the AH-64D Apache Longbow, successor

to the Apache.

The investigation described herein is a fundamental research effort for the Air Warrior

Metrics program performed on behalf of the PM ACIS. In addition to becoming a behavioral and

performance baseline for ALSE affects, the results of the Air Warrior Metrics investigations are

also expected to contribute to improved performance research integration tool (IMPRINT)

analyses of current and future helicopter systems.

OBJECTIVE

The object of this investigation will be to develop performance-based measurement scales

which may then be used to estimate the merits (or lack thereof) of future ALSE components and

their use on rotary wing aircrews. The assessment of aircrew ensemble technologies should be

provided in a standardized framework that will allow the accumulation of a database across

airframe systems and ALSE systems and components, which is keyed to soldier (human)

performance. To be useful, these measurement scales should be (a) developed around the

intended mission(s) for each current or proposed helicopter type, as defined in its aircrew

training manual (required aviator and aircrew tasks), (b) sensitive to ALSE technologies and

designs, (c) easily observable and measurable, and (d) have demonstrated validity and reliability.

The helicopter agreed upon by PM ACIS and ARL to be the initial platform to be studied

is the attack helicopter AH-64 Apache. The required aviator and aircrew tasks in the AH-64

aircrew training manual (ATM) assigned to this aircraft (Headquarters, Department of the Army,

1992) represent both flying and fighting tasks to be performed by a two-person crew, both rated

aviators, positioned in tandem with the copilot-gunner (CPG) in the forward seat.

The total number of tasks listed in the ATM is 88, but not all these tasks were

necessarily exercised in the investigation. Preflight, postflight, and certain redundant tasks have

*Use of the acronym ALSE will imply all life support, survival, and MOPP equipment and components for the

remainder of this report.

10
been eliminated. Further culling of tasks has been accomplished by the surveying of subject

matter experts (SMEs) before experimentation (see Method and Procedure). It was also agreed

that the investigation and subsequent measurement scales consider as the definitive baseline, the

condition of aviators fully encumbered in current ALSE. This is somewhat counterintuitive in

that investigations of this kind usually seek comparisons between “unloaded” and “loaded” levels

of an independent variable. That human performance is degraded via the employment of MOPP

ensembles and ALSE has been amply demonstrated and documented. It is, by far, more

appropriate to estimate changes in performance in reference to current equipment and doctrine.

The approach to developing performance-based measurement scales is grounded in a

framework of behavior-anchored ratings (Cain Smith & Kendall, 1963) of mission flights

conducted in a simulator. This technique has been successfully and recently employed as a

means of evaluating aircrews upon completion of aircrew coordination training, required of all

Army aviators (U.S. Army Aviation Center, 1992). This investigation employs behavioranchored rating scales that are already in use. The narrative text comprising the behavioral

anchors has been modified or edited to reflect the fact that the ALSE, worn by the aviators while

performing their mission flight, is the object of the evaluation. The behavior-anchored rating

techniques and procedures of evaluation were adapted to the Air Warrior metrics purpose. It is

not known, however, if this kind of behavioral metrics estimation technique has been employed

to evaluate the influence, merits, or effect of a physical system such as ALSE.

The parent crew coordination evaluation methodology was validated in test bed

development in 1992 (Simon, Grubb, & Leedom, 1993). Reliabilities of the original behavioral

items were reported as “exceptionally high” (Simon, Grubb, & Leedom, 1992).

For obvious reasons, this investigation was a laboratory type effort obliged to be done in

a flight simulation facility. The trials, however, were conducted in the AH-64 combat mission

simulator (CMS), which is a fielded system intended for training of tactical units flying AH-64

Apache helicopters. The AH-64CMS is located at the Western Area Aviation Training Site

(WAATS), Marana, Arizona.

METHOD

Subjects

Subjects consisted of ten crews of pilot and copilot-gunner pairs, involving a total of 20

aviators. They were all male Army National Guard pilots holding current medical certification, both

rated and current in the AH-64 Apache and having mission experience in flying or in simulation

11 .
while wearing full ALSE ensembles. Participating aviators were from units in Arizona, Idaho, South

Carolina, and Utah. Apache aircrews are normally considered partnerships and do much of their

training together. Except for one or two crews, paired crews were unavailable for this experiment;

however, each aviator is trained to fly and fight from either seat, and they all had taken required

crew coordination training within the preceding year. The rear seat pilot is usually the crew member

on the flight controls and is the pilot in command. The front seat CPG aids in clearing the aircraft of

obstacles, performs navigation, and normally acts as the sensors and weapons operator because of

the intended panoramic outside view from this position. With very few exceptions, all operations

may be conducted from either seat. Individual aviators begin their Apache flying in the front seat as

CPGs and as they gain in training and experience, matriculate to the rear seat as those positions

become available. This is true for both officers and warrant offricers.

While not defined as subjects, the other participants in the investigation were the

evaluators or scorers of the trials flown using the behavior-anchored rating scales. These were

certified (U.S. Army Aviation Center, 1992, p. 3-1) crew coordination training evaluators.

Evaluators may be flight examiners, instructor pilots, and unit trainers. There was one of each kind

participating, all senior Warrant Officers, W-5, W-4, and W-3, respectively. All the evaluators

were qualified AH-64CMS operators. Unfortunately, only one of these was able to participate in

the entire number of trials, as shall be seen later in this report.

-APPARATUS

Simulation Facilities

The flight simulator used in this investigation was the AH-64CMS, the combat mission

simulator for the AH-64 Apache. It is normally employed as a tactical training, recurrency, and

check ride platform for unit aircrews in lieu of actual aircraft. A single simulator combines two

six-degree-of-freedom motion platforms (see Figure I). The AH-64 rear seat cockpit is installed

on one motion platform and the front seat on the other. This allows independent training in each

cockpit or integrated training for crews and combat missions. The latter configuration was

applied to this investigation.

Instrumentation

The AH-64CMS is capable of recording both event data (weapons selection, firing, hits

or misses, ownship hits, ground contact, etc.) and flight data (heading, ground track, altitude,

airspeed, etc.) to be used as measures of effectiveness and quality of each flight trial.

12
Figure 1. AH-64 combat mission simulator.

ALSE

Appendix A contains a listing of the equipment components to be worn by the subjects.

Procedures

SME Survey of Aviator and Aircrew Tasks

Given that here is a “strong correlation” between aircrew task performance and

crew coordination behaviors (Simon, 1991), the presence of ALSE might be expected to have an

impact on the aircrew’s performance of these tasks. Because 88 aviator and aircrew tasks are

listed and described in the ATM for the AH-64, it made sense to attempt to discover if some

tasks might be influenced to a greater degree than others via a survey taken among regular Army

senior Apache aviators. If this were true, a mission scenario could embody those tasks most

likely to be affected. Also, a scenario requiring that all 88 tasks be performed would impractical

to the point of impossibility. The number of tasks selected for the survey was culled to 35 by

several test pilot aviators at the U.S. Army Aviation Technical Test Center, Ft. Rucker,

Alabama, eliminating inappropriate tasks, such as flight planning (done outside the simulator),

and redundant tasks, such as a variety of instrument approaches, as well as tasks that cannot be

performed in the simulator. A sampling of some 12 other regular Army senior AH-64 aviators

13
was surveyed to determine the relative influence that ALSE has on selected tasks by scoring both

the task and its parts or subtasks and rank ordering the set of tasks placed in each of the scoring

categories (very much affected, moderately affected, or slightly affected). Figure 2 illustrates the

fact that there was no evidence of agreement among the SMEs as to which tasks might be affected

by ALSE. Appendix B is a listing of the 35 surveyed ATM tasks in median ranked order.

I- 35

30

25

Range and Median ranking

of Surveyed ATM Tasks

I I

I

12 Senior Army aviators

I ”

I ”

II

Figure 2. SME survey results.

The extremely wide range of SME rankings for the listed ATM tasks made the

data unusable for building mission scenarios. However, scenario guidance and samples are

provided in the Crew Coordination Evaluation Exportable Package (U.S. Army Aviation Center,

1992, Sect. 5).

Scenario

The Army training community, headed by the Directorate of Training and

Simulation, Ft. Rucker, Alabama, has developed a standardized set of mission scenarios for the

AH-64CMS, perhaps better described as tactical situations, Enemy and friendly forces are

arrayed on modeled and mapped terrain, and battle missions are devised for one or more Apaches

in a flight, ownship and simulated other ships. Standardized published operational plans,

operational orders, and pre-mission briefs are employed; however, the mission flight within this

framework is completely open ended. This allowed the simulator operator-instructor, for

1 4
example, to apply an engine failure or other system failure after the crew disengaged from the

battle area in order to bring emergency procedure tasks into play. The standardized scenario

employed for each crew’s familiarization was named “Raid on Regimental Artillery Group” while

the trials for record were conducted for “Hasty Attack Into Horse”.

Methodology

The employment of behavior-anchored rating scales in the evaluation of aircrew

performance as affected by ALSE, was accomplished by adapting the behavioral anchors (i.e., the

text employed in the crew coordination evaluation methodology) to place the burden on

behaviors as impacted by the equipment as opposed to pure coordination behaviors. All active

duty Army and National Guard aviators are required to complete crew coordination training.

Behavioral anchors were written for each of 13 basic qualities against which the evaluations are

scored. Each basic quality contains three anchors with attendant text: Score value 1 (lowest),

“very poor rating”; Score value 4 (midrange), “acceptable rating”; and Score value 7 (highest),

“superior rating”. Each of the original crew coordination anchors (U.S. Army Aviation Center,

1992, Sect. 6) has been edited or modified to reflect the effect of ALSE, as behavior overcame or

succumbed to its intluence. Appendix B contains an overview statement, a sampling of the

modified behavioral anchors, plus a copy of the rating form (grade slip) for aircrew coordination

evaluations, which was used without modification. Rating scores collected on grade slips, one for

each crew member, for each trial, were to be the primary data for the investigation. Each trial also

yielded measures of effectiveness data and flight data to be used in the analysis.

Grading and Scoring

The Army’s standard simulator check ride procedure calls for evaluation

and grading of each individual crew member against ATM tasks d crew coordination evaluation

grading, that is, a single grade slip for the aircrew as an entity (not as individual aviators), and

three grade slips in all. Because the wearing of ALSE may have different impacts on the

individual aviators occupying the front and rear seats, the evaluators were required to execute an

aircrew coordination evaluation grade slip for each individual but not to score and till in the two

grade slips for ATM tasks. The resulting workload for evaluators was thought to be

approximately the same as with the standard check ride.

Trials Procedure

Subjects were first be briefed about the purpose of the investigation and procedures

they were to follow and were allowed to ask questions of the investigators. It was made clear that

15
the subjects were not on trial so much as was the current ensemble of ALSE components. The

subjects were asked to do the best job that they could, however. They then read and signed the

volunteer agreement. The standard procedure for launching a tactical mission was followed,

including the dissemination of operations orders (OPORD) (see Appendix C), presentation of the

air mission brief, scripted for the instructor and keyed to the scenarios and terrain residing in the

simulator and pre-mission, flight and post-mission phases. Typically, each session lasted

approximately 3-l/4 hours as follows: pre-mission planning and donning of ALSE, 1 hour; flight,

l-1/4 hours; post-mission, 1 hour.

Stress Assessment of Aviator Subjects2

The psychological and physiological state of soldiers while they performed

combat-relevant tasks during stressful conditions may be critical to the outcome of a successful

mission. The data presented here were designed to assess the psychological and physiological

stress levels of aviators as affected by ALSE. A standardized battery of psychological state

questionnaires was administered to the subjects in conjunction with a noninvasive physiological

stress measure, salivary amylase. Both types of measures have been used in previous ARL

research and have proved to be sensitive to the degree of stress experienced in a variety of

situations.

Psychological State Measures

A 5-minute battery of stress perception measures was administered at

strategic time points before, immediately following, and 1 hour after the each trial session. The

following state measures were included:

Multiple Affect Adjective Check List

Revised Today Form (MAACL-R; Zuckerman & Lubin, 1985). The

MAACL-R Today form consists of the following subscales: Anxiety, Depression, Hostility,

Positive Affect. The scores are derived from a one-page list of 132 adjectives in which soldiers

check all words that describe how they feel during a specified time period. This is used to

examine changes in specific affects in response to stressful situations. In addition, an overall

distress score Dysphoria or Negative Affect was calculated by combining the Anxiety,

Depression, and Hostility scores.

*The treatment of stress assessment information in the following sections was prepared by the following members of

the HRED Soldier Stress and Cognitive Performance Team: Fatkin, Patton, Mullins, and Burton.

16
1

.

Specific Bating of Events scale (SRE)

The SIXE requires the subjects to rate (on a scale of 0 to 100) how stressed

they feel at the present moment or how they felt during a specified time period.

Subjective Stress Scale

This scale is designed to detect significant affective changes in stressful

conditions (Kerle & Bialek, 1958). Soldiers are instructed to select one word from a list of 15

adjectives that best describes how they feel at the present moment or how they felt during a

specified time period.

Physiological Assessment

Salivary amylase data were collected before, immediately following, and

1 hour after each trial session, including the familiarization procedure. These times coincide with

the completion of the psychological state measures.

Amylase is an enzyme that hydrolyzes starch to oligosaccharides and then

slowly to maltose and glucose. Measurement of amylase in saliva involves chemical color

changes according to standard photometric procedures developed by Northwestern University

(Chatter-ton, Vogelsong, Lu, Elhnan, & Hudgens, 1996). This method combines time lapse and

temperature data to derive a quantifiable level of stress.

Saliva samples for amylase assay were obtained from the subjects by using

small, clean rectangular sponges (1 in. by 0.5 in. by 0.5 in.) in small plastic cups. The subjects

were instructed to roll the sponges in their mouths for 1 minute while they completed the state

questionnaires; then, upon instruction, they were to place the sponges back in the cups, snap the

cover on them, and hand them to the investigator. The pre-labeled cups containing the sponges

were then put in an insulated bag until the assays could be conducted.

Subject Training

The subjects were qualified and current in the AH-64 and were familiar with

performing training exercises, practice missions, tactics, and taking check rides in an AH-64CMS

(as well as in the AH-64 Apache). Annual recurrent training in MOPP IV is normally required of

all Army aviators, but the requirement is often waived. Additional ALSE components such as

ballistic protection plates, life rafts, helicopter emergency egress device (HEED) (under water) air

bottles, sidearms, and so forth are not normally worn during NBC training; therefore, few of the

subjects had experienced flying or simulation missions dressed in the complete collection of

17
encumbering components. A familiarization trial with the “Raid on Regimental Artillery Group”

scenario was conducted before all trials for record.

Evaluator Training

Instructor pilots (IPs) and unit trainers (UTs) certified and serving as evaluators

were trained in the specific procedures and scenarios associated with this investigation. Review

and understanding of the modified basic qualities behavioral anchor narratives was emphasized.

The evaluators practiced scoring trials during the first familiarization trials.

Experimental Design

The design for the conduct of trials was a Treatments x Subjects with repeated

measures layout (Lindquist, 1953), where two repeated measures trials with the subjects fully

encumbered were interleaved with one trial, per aircrew, with the over-water mission aspect

eliminated, the variation trial. Ten AH-64 aircrews, that is, 20 aviators, participated. The

individual members of an aircrew occupied the seat, front or rear, that they ordinarily had for the

majority of their AH-64 experience. Rear seat subjects were designated Si through Sio, and the

front seat subjects designated Sii though S20. The series of three trials took place after the first

familiarization trial had been completed. Table 1 illustrates the counterbalancing schemes .

RESULTS”

Behavior-Anchored Ratings

Evaluators

Evaluators were a very rare commodity when it came to recruiting them for these

trials. Evaluator No. 1, the CW5 Flight Examiner, was required to travel to Germany after only

the first week of trials (Crews 1 through 3). He missed two individual trials during the time he

was present. Evaluator No. 2 , the CW3 Unit Trainer, was present for the entire time trials were

conducted. Both No. 1 and 2, however, missed the very first trial involving Crew No. 1.

Evaluator No. 3, the CW4 Instructor Pilot, was recruited midway through the trials (Crew 7

through lo), except he missed one of the trials for Crew No. 10. Table 2 summarizes the trials

scored by each of the evaluators.

3The counterbalancing scheme is not complete. Twelve crews would have provided a complete and balanced

design, but only ten crews were available.

4An alpha level of .05 was used for all statistical tests.

18
Table 1

Trials Order of Presentation by Crew and Subject No.

Crew No. Variation trial

1 @I, Sll)

2 02, S12)

3 @3, S13)

4 (S4, Sl4)

5 @5, w

6 06, s16)

7 @7, S17)

8 @s, s18)

9 @9, S19)

10 @lo, S20)

1

3

2

1

3

2

1

3

2

1

3

2

1

3

2

1

3

2

1

3

Table 2

Trials Scored by Participating Evaluators

Trial 1 Trial 2 Variation trial

Evaluator No. 1 2 3 1 2 3 1 2 3

Crew No.

1 (Sl, Sll)

2 (S2, S12)

3 @3> S13)

4 @4, S14)

5 es, w

6 (s6, s16)

7 (S7, S17)

8 @ST s18)

9 (S9, S19)

10 @lo, s20>

x x

x x

X

X

X

x x

x x

x x

x x

x x

x x x

x x

X

X

X

x x

x x

x x

X

x x

X

x x

X

X

X

x x

x x

x x

x x

The evaluation scores on a Likert 7-point scale for the 13 bashqualities, common

to Evaluator No. 1 and 2, were subjected to a hierarchical iog linear (frequency) analysis. No

statistically significant one-way effects for evaluators nor two-way associations (E < .05) were

found. Scoring by Evaluator 1 and Evaluator 2 was not different.

19
The evaluation scores on a Likert 7-point scale for the 13 basic qualities, common

to Evaluator No. 2 and 3, were subjected to a hierarchical log linear (frequency) analysis. No

statistically significant one-way effects for evaluators nor two-way associations (p < .05) were

found. Scoring by Evaluator 2 and Evaluator 3 was not different.

Crews

The evaluation scores on a Likert 7-point scale for the 13 basic qualities, collapsed

or aggregated5 for all evaluators, were subjected to a hierarchical log linear (frequency) analysis.

Crew was the factor analyzed. No statistically significant one-way effects for individual crews

nor two-way associations (r! < .05) were found. Individual crew scores were not different.

r

Crew Position (pilot, rear seat versus copilot-gunner, front seat)

The evaluation scores on a Likert 7-point scale for the 13 basic qualities, aggregated for all

evaluators, were subjected to a hierarchical log linear (frequency) analysis. Crew position was

the factor analyzed. No statistically significant one-way effects for crew position nor two-way

associations (p < .05) were found. Pilot and copilot-gunner scores were not different.

Trial Sequence

Given the counterbalanced design to accommodate the variation trial (no overwater equipment), the sequence of the three trials for record for each crew was examined for

learning or adjustment effects. The evaluation scores on a Likert 7-point scale for the 13 basic

qualities, aggregated for all evaluators, were subjected to a hierarchical log linear (frequency)

analysis. Trial sequence was the factor analyzed. No statistically significant one-way effects for

trial sequence nor two-way associations @ < .05) were found. Trial sequence scores were not

different.

Trial Type

c

The evaluation scores on a Likert 7-point scale for the 13 basic qualities,

aggregated for all evaluators, were subjected to a hierarchical log linear (frequency) analysis. Trial

type was the factor analyzed. No statistically significant one-way effects for trial sequence were

found; however, two-way associations were noted for Basic Qualities 5,7, 11 b < .05). A

Wherever it was considered appropriate in the analysis of data, the evaluation scores rendered by all the evaluators

were employed. The first three crews and the last four crews had two sets of evaluation scores, recorded by two

evaluators, for most of their trials.

20
l

.

further two-way association was noted for Basic Quality 12 when the first and second fully

encumbered trial types only were analyzed. Overall, scores for trial type were not different.

Flying Experience

It was thought that the flying experience, in terms of both total flight hours or

flight hours in the Apache, logged by the subjects was a potential factor for influencing the

evaluation scores. The participating subjects represented a rather wide range of flying experience

as seen in Figure 3.

The subjects were placed into the following experience categories (see Table 3) in

order to facilitate a log linear analysis of the experience factor. There are 4 f 1 subjects in each

category.

The evaluation scores on a Likert 7-point scale for the 13 basic qualities, aggregated

for all evaluators, were subjected to a hierarchical log linear (frequency) analysis. Trial sequence

and trial type by total flight experience categories were the factors analyzed. No statistically

significant one-way effects for trial sequence and trial type were found; however, some two-way

and three-way associations were noted (p < .05). Total flight experience is not a factor with

respect to trial sequence or trial type.

5000

4500

4000

3500

v) 3000 L

0’ 2500

= 2000

1500

1000

500

0

I

Total and AH-64 Hours Logged by Subject No.

13 15 14 12 19 11 2 17 2 0 6 18 3 16 5 1 7 8 9 4 10

Subject No.

II TOTAL Hrs

n AH-64 Hrs

I

Figure 3. Flying experience of participating subjects.

21
Table 3

Flying Experience Categories

Total flight categories Total hours AH-64 flight category AH-64 hours

1 1 to 500 1 1 to 250

2 501 to 1000 2 251 to 500

3 1001 to 1500 3 501 to 750

4 1501 to 2000 4 751 to 1000

5 2001 to 5000 5 1001 to 2500

The evaluation scores on a Likert 7-point scale for the 13 basic qualities, aggregated

for all evaluators, were subjected to a hierarchical log linear (frequency) analysis. Trial sequence

and trial type by AH-64 flight experience categories were the factors analyzed. No statistically

significant one-way effects for trial sequence and trial type were found; however, some two-way

and three-way associations were noted (p < .05). AH-64 flight experience is not a factor with

respect to trial sequence or trial type.

Measures of Effectiveness and Flight Data

Data in a number of different formats were collected during the conduct of trials in the

AH-64CMS, reflecting the performance of the subjects in their tasks. Data such as the task

times performed by the CPG in bringing the various Apache systems on line before takeoff were

recorded by observation. Other measures were extracted from the simulator mainframe during the

mission via hard copy requested via computer print command by the simulator operatorsinstructors-evaluators. In the press of conducting the trials, not all these data sets were recorded

or recovered for each and every trial. At times, the simulator mainframe (1970’s architecture)

was rather temperamental, and data were lost from working memory before they could be

recovered. Main power fluctuations and maintenance problems in the computer room

environmental control units exacerbated the situation. The following analyses were performed

with knowledge that the number of cases examined does not represent a complete data set for

each and every trial.

CPG Task Times

The AH-64 crew’s check list called for the CPG to initiate and check the

operation of the navigation, sensor suite, fire control and weapons systems before takeoff and

22
h

E 5.00

i

I- 4.00

z

B0 3.00

z

2.00

I A

1.00 -

l I

0.00 J 1

PGM IHADSS PGM TADS TADS TADS FLIR WNS FUEL T/oDEK DOPP OP INT cur OP INIT cx MIN

Systems

i

immediately after the pilot has started the ship’s auxiliary power unit and brought it on line. The

major tasks are

l Program the digital entry keyboard (DEK) weapons and ammunition data

l Helmet: integrated helmet and display sight system (IHADSS) operational check

l Program Doppler navigation set with navigation way points

l Target acquisition designation sight (TADS) operational check

l TADS initialization

l Forward looking infrared (FLIR) initialization

l Weapons initialization and check

l Fuel check

Also, the time from when the subjects were strapped in the cockpit to time of takeoff was

recorded. Figure 4 illustrates the mean times and their standard deviations for the recorded data.

Pre-takeoff Systems Initialization and ChecksAverage- Time & Standatd Deviation

Figure 4. Mean times and standard deviations of CPG tasks.

23
Times for the data were subjected to one-way analyses of variance (ANOVAs)

with CPG subjects as the between-groups effect. No statistically significant effect was found for

any of these times with respect to trial sequence & < .05). CPG subjects are considered a part of

the same population.

Times for the data were subjected to one-way ANOVAs with trial sequence as the

between-trials effect. No statistically significant effect was found for any of these times with

respect to trial sequence (Iz < .05). The distributions of data points across trial sequence for

these tasks were not necessarily even, possibly contributing to the lack of significance.

Times for the data were subjected to one-way ANOVAs with trial type as the

between-trials effect. No statistically significant effect was found for any of these times with

respect to trial type (Q < *OS). The distributions of data points across trial sequence for these

tasks were not necessarily even, possibly contributing to the lack of significance.

Weapons Engagements

The Apache crews were ordered to fly from their starting point and conduct a hasty

attack and destroy enemy forces in engagement area “HORSE,” consisting of a tank battalion and a

motorized rifle battalion. The enemy force provided opportunity for crews to use all their

weaponry, that is, HELLFIRE guided missiles, 2.75~inch rockets, and the 30-mm gun. Mission

data were gathered for the total number of engagements, missile engagements, rocket engagements,

and gun engagements; number of kills, hits, and misses, and the percentages of each; and the number

of kills, hits, and misses with each type of weapon. Figure 56 summarizes the number of kills, hits,

and misses for each type of weapon. Figure 6 summarizes the percentage of kills, hits, and misses

and the total number of engagements for each weapon.

All these measures were subjected to one-way ANOVAs with crews as the

between-groups effect. No statistically significant effect was found for any of these times with

respect to trial sequence (p < .05). Crews are considered a part of the same population.

All these measures were subjected to one-way ANOVAs with trial sequence as

the between-trials effect. No statistically significant effect was found for any of these times with

respect to trial sequence @ < .05). There were no differences in any of these measures with

respect to trial sequence.

6FirstR denotes the first trial for record fully encumbered, 2ndR is the second trial for record fully encumbered, VR

is the variation (no over-water components) trial for record. These are the three trial types referenced.

24
.

Kills, Hits and Misses by Crew and Trial Type

1 2 3 3 3 5 5 5 6 6 7 7 7 6 6 9 9 9 10 10

VR PndR VR 1slR PndR 1stR VR PndR RV 2ndR 1stR 2ndR VR 1stR VR VA 1stR 2nrR 2ndR VR

Crew and Trial Type

30

25

20

15

10

5

0

25

1

VR

0

Kills, Hits and Misses by Crew and Trial Type

2 3 3 3 5 5 5 6 6 7 7 7 6 6 9 9 9 10 10

ZndR VR 1stR PndR 1stR VR 2ndR RV PndR 1stR PndR VR 1stR VR VR 1stR PnrR 2ndR VR

Crew and Trial Type

Kills, Hits and Misses by Crew and Trial Type

I ROCKETS-M I

n ROCKETS-H 1

. ROCKETS-K j

1 2 3 3 3 5 5 5 6 6 7 7 7 6 6 9 9 9 10 10

VR PndR VR 1stR PndR 1stR VR PndR RV PndR 1stR PndR VR 1stR VR VR 1stR 2nrR PndR VR

Crew and Trial Type

Figure 5. Kills, hits, and misses.

25
Percent Kills, Hits, Misses by Crew and Trial Type

All Weapons, All Engagements

100%

90%

80%

70%

60%

50%

40%

30%

20%

10%

0 %

6Pb

1 2 3 3

VR 2ndR VR lstR2ndR 1stR VR

5 6

2ndR

6 77 7 BQ

I RV 2ndR lstR2ndR VR 1stR VR

Crew No. & Trial Tyoe

9

V R

i

l&E?

I

9

1stF

94%

1CGX

9 10 10

I 2nrR 2ndR VR

6 0

5 5

5 0

4 5

UI

4 0 E Q) E

3 5 g

z

3 0 5

z

2 5

8

2 0 f

:

1 5

10

5

0

PERCENT MISSE!

PERCENT HITS

I PERCENT KILLS

*ENGAGEMENTS

;:I/

Figure 6. Percentage kills, hits, and misses.

All these measures were subjected to one-way ANOVAs with trial type as the

between-trials effect. No statistically significant effect was found for any of these times with

respect to trial type (p. < .05). There were no differences in any of these measures with respect

to trial type.

Weapons Platform Stability

The AH-64CMS momentarily records the Apache’s motion during engagements

in terms of pitch, yaw, and roll rates and vertical and lateral translation rates. All are in units of

feet per second, plus (right and upward velocities) and minus (left and downward velocities)

movement of the launcher rail or gun muzzle. These data are not available for missile rounds.

This is done from 5 seconds before trigger pull until the last rocket or gun round has left the ship

and constitutes an “engagement”. For both rockets and gun, anengagement may be a burst of

rounds with 5 seconds to the next trigger pull. Data were collected for the maximum velocities in

the plus direction and the maximum velocities in the minus direction and algebraically summed,

regardless of whether these maximums occurred in the pitch, yaw, roll, vertical or lateral axes.

The resulting sums then represent a range of rates from plus to minus for further analysis. It

26
would have been more rigorous to perform a vector summation of the rate data; however, the

results would have been more conservative and compacted.

.

The platform stability range of velocities for AH-64CMS recorded trials was

subjected to one-way ANOVAs with crews as the between-trials effect. A statistically

significant effect was found with respect to trial sequence (Q = .OOl). Both Tukey and Duncan

post hoc tests were applied to these data. They yielded slightly different results (see Table 4).

Note, however, that the number of trials representing the simulator recorded data is 2 to 3,

extremely small.

Table 4

Results of Both Tukey and Duncan Post Hoc Tests for Platform Stability-Crews

Crew No. 5 3 9 10 6 7

N 3 2 3 2 2 3

Means 1.030 1.040 1.063 1.095 1.135 1.153 .

Tukey

Duncan

p < .05

Means underscored by the same line are not significantly different.

Means not underscored by the same line are significantly different

The Tukey test says Crew No. 5,3,9, and 10 were not significantly different

from each other, but grouped, they were different from Crew No. 6 and 7; Crew No. 9, 10, and 6

were not significantly different from each other, but grouped, they were different from Crew No.

3,2, and 7 and Crew No. 6 and 7; Crew No. 10,6, and 7 were not significantly different from

each other, but grouped, they were different from Crew No. 5,3, and 9. The Duncan test says

Crew No. 5,3, and 9 were not significantly different from each other, but grouped, they were

different from Crew No. 10,6, and 7; Crew No. 9 and 10 were not significantly different from

27
each other, but grouped, they were different from Crew No. 5.,3,6, and 7; Crew No. 10 and 6

were not significantly different from each other, but grouped, they were different from Crew No.

5,3,9, and 7; and Crew No. 6 and 7 were not significantly different from each other, but

grouped, they were different from Crew No. 5,3,9, and 10.

The platform stability range of velocities for AH-64CMS recorded trials was

subjected to one-way ANOVAs with trial sequence as the between-trials effect (see Table 5).

No statistically significant effect was found for any of these times with respect to trial sequence

(E < .05). There were no differences in any of these measures with respect to trial sequence.

The sample sizes for the trial means are small.

Table 5

Mean Platform Stability Range of Velocities-Trial Sequence

Trial sequence 1 2 3

N 5 6 6

Means 1.072 1.070 1.117 +

The platform stability range of velocities for AH-64CMS recorded trials was

subjected to one-way ANOVAs with trial type as the between-trials effect (see Table 6). No

statistically significant effect was found for any of these times with respect to trial type (Q <

.05). There were no differences in any of these measures with respect to trial type. Again, the

sample sizes for the trial means are small.

Table 6

Mean Platform Stability Range of Velocities-Trial Type

Trial type First R Second R VR

N 4 6 7

Means 1.058 1.100 1.093

28
Enemy Forces Threat to Ownship

The enemy forces deployed in EA HORSE could shoot back at the Apache crews

during their mission and did so if and when the crew’s ownship was searched, found, and acquired

for a long enough period. While the enemy’s tanks and troop carriers and command vehicles could

engage with guns, the ownship was out of their range more often than not. The bulk of the threat

was from a variety of air defense weapons accompanying the forces, which employed guns and

rockets and missiles, optics and radar. The probabilities of acquisition and hit, given ownship shot

at is a function of how much of the ownship is unmasked (by exposure zone), the time of

exposure, time of flight of the enemy projectiles, and whether the ownship can remask. Figure 7

describes the ownship exposure zones portion of the probability functions.

PROBABILITY of ACQUISITION (PA)

0 0.000

1 0.500

2 0.626

3 0.750

4 0.875

5 1 .ooo

100% EXPOSED

ZONE 5

ZONE 4

ZONE 3

ZONE 2

ZONE 1

ZONE 0

100% MASKED

MEAN ACQUISITION RANGE

.

Figure 7. Ownship exposure zones.

The AH-64CMS computes a threat “event” with its attendant acquisition and hit

probabilities whenever the ownship is exposed, with line of sight in exposure zone one or higher

for 10 seconds or more, with 5 seconds or more continuous (U.S. Army Materiel Command,

Simulation, Training, and Instrumentation Command, 1996). The number of events occurring on

a given mission is uncontrolled and solely dependent on the pilot’s flying maneuvers. Measures

29
of summed maximum exposure zones (over all the events), of maximum probabilities of enemy

acquisition, maximum probabilities of enemy hits, total exposure time, average exposure time per

event, percent of events where shot at, hits, misses, and no-shots were examined. Figures 8, 9,

and 10 show the threat data gathered.

AH64 CMS Recorded Exposure to Threat systems by Crew and Trial Type

Number of Events

1 :oo:oo

0:50:00

:

ln. .

.; 0:40:00

k

F 0:30:00

!!

B:, 0:20:00

1

0: 1o:oo

o:oo:oo

AH64 CMS Recorded Total Exposure Times

& Number of times shot at, by Crew and Trial Type

#B 56:07 8

S6:43

28:16

t-t 2522 25:Ol

1 2 3 3-3

VR 2nd VR 1stR 2nd

R R

Figure 8. Events: Maximum exposure zones.

5556677788

1stR VR 2nd VR 2nd 1stR 2nd VR 1stR VR

R R R

7

44:56 2

37:02

l-l

9 9 9 10

VR 1stR 2nd 2nd

R R

Crew and Trial Type

Figure 9. Total exposure times and times shot at.

30
Maximum Probability of Acquisition (PA) & Hit (PH) for AH64 CMS

Recorded Threat Events by Crew and Trial Type

1 .oo

0.90

0.80

0.70

0.60

0.50

0.40

0.30

0.20

0.10

0.00

4

1 I 8.1 I 1 I 1 / 1 1 I I I I / I 1 10

1 2 3 3 3 5 5 5 6 6 7 7 7 8 6 9 9 9 IO

VR 2ndR VR IstR 2ndR IstR VR 2ndR VR 2ndR 1 stR 2ndR VR IstR VR VR IstR 2ndR 2ndR

Crew and Trial Type

Figure 10. Acquisition and hit probabilities.

The sum of maximum exposure zones for AH-64CMS recorded trials was

subjected to one-way ANOVAs with trial sequence as the between-trials effect. A statistically

significant effect was found with respect to trial sequence (p. = .048). Both Tukey and Duncan

post hoc tests were applied to the data. Both yielded the same results (see Table 7).

Table 7

Results of Both Tukey and Duncan Post Hoc Tests for

Summed Exposure Zones-Trial Sequence

Trial sequence 1 2 3

N 6 6 7

Means 20.00 31.00 47.14

Tukey and Duncan

Means underscored by the same line are not significantly different.

Means not underscored by the same line are significantly different

31
The first trial for record was not significantly different from the second but was different from

the third. The second trial for record was not significantly different from either of the other two.

The sum of maximum exposure zones for AH-64CMS recorded trials was

subjected to one-way ANOVAs with trial type as the between-trials effect (see Table 8). No

statistically significant effect was found for any of these times with respect to trial type (g <

.05). There were no differences in any of these measures with respect to trial type.

Table 8

Mean Sums of Maximum Exposure Zones-Trial Type

Trial type First R Second R VR

N 5 7 7

Means 27.20 46.71 24.71

The maximum probability of acquisition for AH-64CMS recorded trials was

subjected to one-way ANOVAs with trial sequence as the between-trials effect. A statistically

significant effect was found for with respect to trial sequence (g = ,017). Both Tukey and Duncan

post hoc tests were applied to the data. They yielded slightly different results (see Table 9).

Table 9

Results of Both Tukey and Duncan Post Hoc Tests for

Probability of Acquisition-Trial Sequence

Trial sequence 1 2 3

N 6 6 6

Means .3517 .4317 .6233

Tukey

Duncan

Means underscored by the same line are not significantly different.

Means not underscored by the same line are significantly different

32
The Tukey test says that the first trial for record was not significantly different from

the second but was different from the third, and the second trial for record was not significantly

different from either of the other two. The Duncan test says the first trial for record was not

significantly different from the second, but the third trial for record was different from both the first

and second.

The maximum probability of acquisition for AH-64CMS recorded trials was subjected

to one-way ANOVAs with trial type as the between-trials effect (see Table 10). No statistically

significant effect was found for any of these times with respect to trial type (g < .05). There were

no differences in maximum probability of acquisition measures with respect to trial type.

Table 10

Mean Maximum Probabilities of Acquisition-Trial Type

Trial type First R Second R VR

N 5 6 7

Means .3700 .5300 .4871

The maximum probability of hit for AH-64CMS recorded trials was subjected to oneway ANOVAs with trial sequence as the between-trials effect. A statistically significant effect

was found for with respect to trial sequence (g = .002). Both Tukey and Duncan post hoc tests

were applied to the data. They yielded the same results (see Table 11).

Table 11

Results of Both Tukey and Duncan Post Hoc Tests for

Maximum Probability of Hit-Trial Sequence

Trial sequence 1 2 3

N 6 6 6

Means 20.00 31.00 47.14

Tukey and Duncan

g < .05

Means underscored by the same line are not significantly different.

Means not underscored by the same line are significantly different

33
Both tests note that the first trial for record was not significantly different from the second, but

the third trial for record was different from both the first and second.

The maximum probability of hit for AH-64CMS recorded trials was subjected to oneway ANOVAs with trial type as the between-trials effect. A statistically significant effect was

found for with respect to trial type (g = .031). Both Tukey and Duncan post hoc tests were

applied to the data. They yielded the slightly different results (see Table 12).

Table 12

Results of Both Tukey and Duncan Post Hoc Tests for

Maximum Probability of Hit-Trial Type

Trial type First R

N 5

Means .oooo

Second R VR

6 7

.2464 .3257

Tukey

Duncan

p < .05

Means underscored by the same line are not significantly different.

Means not underscored by the same line are significantly different

The Tukey test says the first trial for record was not significantly different from

the second but was different from the VR trials, and the second trial for record was not

significantly different from either of the other two. The Duncan test says the first trial for record

was significantly different from the second and the VR trials. Note that for the FirstR trials, the

hit probabilities were all zeros, but that is for only five crews.

The data (number of events, ownship hits, ownship misses, no shot at ownship,

percent shot at, total exposure time, average exposure time per event, sum of exposure zones for

AH-64CMS recorded trials) were subjected to one-way ANOVAs with both trial sequence and

trial type as the between-trials effect. Means are shown in Tables 13 and 14. No statistically

significant effect was found for any of these measures with respect to trial sequence or trial type

(Q < .05). There were no differences in any of these measures with respect to trial sequence or

trial type.

34
. 1)

.

Table 13

Means for Number of Events, Ownship Hits, Ownship Misses, No Shot at Ownship, Percent Shot At, Total

Exposure Time, Average Exposure Time per Event, Sum of Exposure Zones for Trial Sequence

Ntlm- Own- Ownber of ship ship

events hits misses

No shot

at ownshipPercent

shot at

Average

Total exposure sum of

exposure time exposure

time per event zones

Trial 1 Mean 6.50 .17 1.00 5.33

sequence N 6 6 6 6

SD 3.45 .41 1.10 2.66

17.35 13:58

6 6

15.45 OS:46

02:lO 20.00

6 6

01:os 11.73

2 Mean 10.00 .17 .50 9.33 6.25 25:23 03:07 31.00

N 6 6 6 6 6 6 6 6

SD 4.20 .41 .84 3.93 10.46 14:55 02:37 15.86

3 Mean 12.14

N 7

SD ’ 5.18

.oo 3.00 9.14 23.50 31:20 02:28 47.14

7 7 7 7 7 7 7

.oo 2.83 4.14 18.55 18:17 00:54 23.52

Total Mean 9.68 .ll 1.58 8.00 16.11 23:58 02:35 33.47

N 19 19 19 19 19 19 19 19

SD 4.78 .32 2.12 3.93 16.29 15:48 01:38 20.68

SD = standard deviation
Table 14

Means for Number of Events, Ownship Hits, Ownship Misses, No Shot at Ownship, Percent Shot At, Total

Exposure Time, Average, Exposure Time per Event, Sum of Exposure Zones, by Trial Type

Number of

events

Ownship

hits

Ownship

misses

No shot

at ownshipPercent

shot at

Total

exposure

time

Average

exposure sum of

time exposure

per event zones

Trial

type

firstR Mean 8.60 .20 1.40 7.00

N 5 5 5 5

SD 5.46 -45 1.14 4.58

18.32 23:06 03:Ol 27.20

5 5 5 5

14.60 12:55 02:25 17.51

secondR Mean 11.86 .14 2.29 9.43 18.74 30:13 02:33 46.71

N 7 7 7 7 7 7 7 7

SD 5.37 .38 2.87 4.08 15.95 20:41 01:37 23.80

.oo 1.00 7.29 11.90 18:20 02:17 24.7 1

7 7 7 7 7 7 7

.oo 1.83 3.45 19.16 11:16 01:06 13.61

VR Mean 8.29

N 7

SD 3.35

Total Mean 9.68 .ll 1.58 8.00 16.11 23:58 02:35 33.47

N 19 19 19 19 19 19 19 19

SD 4.78 .32 2.12 3.93 16.29 15:48 01:38 20.68

.
An intercorrelation matrix of the threat measures is shown in Table 15. O f

interest is that Maximum Probability of Acquisition by Exposure Zone Maximum (r = .729) and

Maximum Probability of Acquisition by Maximum Probability of Hit (r =.771) exceed the r * 2

.50 commonality rule of thumb.

Table 15

Threat Measures Intercorrelations

Pearson’s p

correlation

Average

Maximum Maximum Total exposure

Ownship probability probability exposure time per Percent

hits of acquisition of hit time event shot at

Ownship hits

Maximum proba- -.146

bility of acquisition

Maximum proba- .057 .771**

of hit

Total exposure

time

Average exposure

time per event

Percent shot at

Exposure zone

maximum

.225 .159 .372

.256 .081 .085 .564*

.282 .213 .266 .424 .128

-.1.51 .729** .610** .338 -.lOl .449

* Correlation is significant at pc.05 (two-tailed).

** Correlation is significant at pc.01 (two-tailed).

Stress Assessment of Aviator Subjects

A repeated measnres ANOVA was conducted on the state data with two factors: time

(pre, post, recovery) and trial type (familiarization, first fully encumbered trial, second fully

encumbered trial, and variation trial) with seat (rear, pilot versus front, CPG) included as a

between-group variable. The combined MAACL-R data revealed a main effect of time, E(2,lS) =

11.25, E < .OOl, with post measures being significantly higher than pre or recovery measures. In

addition, there was a Subscore x Time interaction, E(8,72) = 9.06, B < .OOl . Table 16 and Figure 11

present the overall means, pre, post and recovery for the MAACL-R combined data. Table 17 and

31
Figure 12 show the mean scores for the MAACL-R subscales and the Subscore x Time interaction.

There was no effect of seat for the combined data.

Table 16

MAACL-R Today Combined Data Means

Means MAACL-R Combined data

Pre 50.12

Post 61.67

Recovery 49.40

MAACL Scores

Pre Post Recovery

Figure 11. Histogram, MAACL-R today combined data means.

Table 17

Mean Scores for MAACL-R Subscales: Anxiety, Depression,

Hostility, Positive Affect, and Dysphoria

Means MAACL data Pre

Anxiety 48.29 53.05 46.58

Depression 47.04 69.72 47.44

Hostility 49.15 67.68 48.70

Positive affect 58.44 51.06 57.78

Dysphoria 47.68 66.81 46.51

38
70 -

65 -

60 -

MAACL Scores

Time x Subscore Interaction

i+Anx

1+Dep

-i- Hos

--e-PA

i-X- DYS

55 -

50 -

Pre

I , /

Post Recovery

Figure 12. Mean scores for MAACL-R subscales interaction.

A separate repeated measures ANOVA was conducted for each of the MAACL-R subscores as

well as the subjective stress data and the SRE data with two factors: time (pre, post, recovery)

and trial type (familiarization, first fully encumbered trial, second fully encumbered trial, and

variation trial) and with seat (rear, pilot versus front, CPG) included as a between-group variable.

There was no main effect of trial or seat for any of the subscores.

MAACL-R Anxiety

There was a significant main effect of time on the MAACL-R Anxiety subscale

&2,18) = 7.26, p = .005. The level of anxiety was higher immediately following a trial than

before or 1 hour after a trial (see Table 18 and Figure 13).

Table 18

Mean Anxiety Scores by Time

Means Anxiety scores

Pre 48.10

Post 53.21

Recovery 46.47

39
Anxiety Scores

Main Effect of Time

Pre Post Recovery

Histogram, mean anxiety scores by time. Firmre 13.

MAACL-R Depression

There was a significant main effect of time on the M!kL-R Depression

subscale E(2,18) = 11.94, E = .OOl . The level of depression was higher immediately following a

trial than before or 1 hour after a trial (see Table 19 and Figure 14). Furthermore, there was a

significant Trial x Time interaction E(6,54) = 2.71, p = .023 (see Table 20 and Figure 15).

Table 19

Mean Depression Scores by Time

Means Depression scores

Pre 47.04

Post 69.72

Recovery 47.44

Depression Scores

Main Effect of Time

Pre Post

Tlme

Recovery

Fkure 14. Histogram, mean depression scores by time.

40
Table 20

Mean Depression Scores by Time and Trial

Means

Pre

Depression scores

Post Recovery

.

Trial familiarization 45.92 82.70 46.75

Trial firstR 48.75 65.00 47.00

Trial secondR 46.50 69.00 49.00

Trial VR 47.00 62.20 47.00

Depression Scores / +Pre I

Trial by Time Interaction / +-Post I

/ +-Recovervi

40 ’ I I I I

Trial Fam Trial 1stR Trial PndR Trial VR

Mean depression scores by time and trial. Figure 15.

MAACL-R Hostility

There was a significant main effect of time on the Hostility subscale l(2,18) = 7.53,

p = .004. The level of hostility was higher immediately following a trial than before or 1 hour after

a trial (see Table 21 and Figure 16).

Table 21

Mean Hostility Scores by Time

Hostility scores

Pre 49.37

Post 67.46

Recovery 48.65

41
70

60

50

f 40

s 30

20

10

0

IL

Hostility Scores

Pre Post Recovery

L

Figure 16. Histogram, mean hostility scores by time.

MAACL-R Dysphoria (Negative Affect)

There was a significant main effect of time on the Dysphoria or Negative AfYect

score &2, 18) = 12.04, g = .OO 1. The level of negative affect was higher immediately following a

trial than before or 1 hour after a trial (see Table 22 and Figure 17).

Table 22

Mean Dysphoria Scores by Time

Dysphoria scores

Pre 47.68

Post 66.81

Recovery 46.51

DYSPHORIA

Pre Post Recovery

Finure 17. Histogram, mean dysphoria scores by time.

42
.

MAACL-R Positive Affect

There was a significant main effect of time on the Positive Affect subscale &2,18)

= 9.03, p = .002. The level of Positive Affect was lower immediately following a trial than before

or 1 hour after a trial (see Table 23 and Figure 18). There was also a significant Trial x Time

interaction F(6,54) = 2.59, p = .028 (see Table 24 and Figure 19).

Table 23

Mean Positive Affect Scores by Time

Positive affect scores

Pre 58.6

Post 51.4

Recovery 57.9

Positive Affect Scores

Main Effect of Time

Pre Post Recovery

Histogram, mean positive affect scores by time. Figure 18.

Table 24

Mean Positive Affect Scores by Time and Trial

Means

Pre

Positive affect scores

Post Recovery

Trial familiarization 60.63 50.22 60.83

Trial firstR 54.90 54.33 57.48

Trial secondR 61.53 49.30 53.93

Trial VR 56.70 50.38 58.88

43
-

65

63

Positive Affect

Trial x Time Interaction

6 1

5 9

257 -

g55

=53

5 1

4 9

fl:

Pre Post Recovery

-Q-Trial Fam ~

+Trial 1 stR ~

--t-Trial 2ndR ~

-Trial VR I

;

Fipure 19. Mean positive affect scores by time and trial.

Subjective Stress Scale

There was a significant main effect of time on the Subjective Stress scale E(2,36)

= 22.83, p = .OOO. Subjects reported higher levels of stress immediately following a trial than

before or 1 hour after a trial (see Table 25 and Figure 20). No significant main effects were found

for either trial type or seat; however, a Trial x Seat interaction E(3,54) = 2.93, p = .042 revealed

that pilots in the rear seat reported higher stress for the first trial for record than did the CPGs in

the front seat. Conversely, during the second trial, the rear seat pilots claimed less stress whereas

the front seat CPGs reported an increase in subjective stress (see Table 26 and Figure 21). A

three-way Seat x Trial x Time interaction was also noted E(6, 108) = 3.41, Q = .004.

Table 25

Subjective Stress Scores by Time

Subjective stress scores

Pre 22.24

Post 46.53

Recovery 18.76

44
.

50

45

40

35

E

30

8 25

r

20

15

10

5

0

Subjective Stress Scores

Pre Post Recovery

Figure 20. Histogram, subjective stress scores by time.

Table 26

Mean Subjective Stress Scores by Trial and Seat

Means Subjective stress scores

Front Rear

Trial x Seat Front Rear

Trial familiarization 31.40 28.40

Trial firstR 23.93 36.83

Trial secondR 31.67 33.20

Trial VR 27.03 20.93

4 0

35

2 5

20

Subjective Stress

Trial x Seat Interaction

i-O_ Trial Fam

--C Trial PndR

Front Rear

Fiwre 21. Mean positive affect scores by trial and seat.

4 5
Specific Rating of Events Scale (SRE)

There was a significant main effect of time on the overall stress levels as measured

by the SRE scale &2,36) = 53.34, Q = .OOO. Subjects reported higher overall stress immediately

after a trial than they did before or 1 hour after a trial (see Table 27 and Figure 22).

There were no significant findings of main effects for seat or trail, nor two-way

interactions for seat, trial or time. There was a three-way Seat x Trial x Time interaction E(6,

108) = 2.36, Q = ,035.

Table 27

SRE Scores by Time

SRE scores

Pre 17.26

Post 55.66

Recovery 12.00

L

60

50

40

2

$ 30

5

20

SRE Scores

Pre Post Recovery

Fipure 22. Histogram, SRE scores by time.

Salivary Amylase

A repeated measures ANOVA was conducted on the salivary amylase data with

two factors: time (pre, post, recovery) and trial type (familiarization, first fully encumbered

46
trial, second fully encumbered trial, and variation trial) with seat (front versus back) included as a

between-group variable (see Table 28 and Figure 23).

The effect of trial type did not reach significance nor were there any significant

differences between seats for the amylase data. Only a main effect of time was noted 52,20) =

3.60, p = .046. Amylase measures were significantly higher immediately following a trial than

they were before or 1 hour after testing.

Table 28

Amylase Data Scores by Time

Means Amylase data scores

Pre 230.77

Post 238.41

Recovery 226.75

240 r

Amylase Data

Main Effect of Time

235

225

Pre Post Recovery

J

Histogram, amylase data scores by time. Figure 23.

Stress Assessment Batteries

The psychological state data were compared with data from five referent

protocols (Torre et al., 1991; Fatkin & Hudgens, 1994; Tauson, Doss, Rice, Tyrol, & Davidson,

1995). Each of these referent protocols included a pre-stress measurement and a post-stress

measurement. The five referent protocols for the present evaluation are

47
1. ONCOL SURG-men visiting a hospital on a day when their wives were

facing cancer surgery under general anesthesia.

2. ABDOM SURG-men visiting a hospital on a day when their wives were

facing abdominal surgery under general anesthesia,

3. WR EXAM-third year male medical students taking a written examination

required for completion of the clerkship portion of their medical training.

4. SS COMPET-male soldiers representing elite units in marksmanship (sharp

shooter) competition.

5. INDEP CONTROL-men investigated during normal work days when they

were experiencing no unusual stress.

The ONCOL SURG and ABDOM SURG protocols represent a relatively high

stress level when compared with the WR EXAM and SS COMP protocols, which represent a

relatively moderate level of stress. The INDEP CONTROL protocol represents a relatively low

stress level to a condition of no stress. The INDEP CONTROL pre-stress responses are an

independent, unstressed measure, which was used as a basis for comparison against the current

results (see Figures 24 through 36). Error bars in these figures are standard error of the means.

I

I 90

I

85

80

p

1 g70

; v)85

I 380

: 55

50

45

40

MAACL-R ANXIETY PRE

ABDOM

SURG SURG

WI ss INDEP ALSE PRE ALSE

EXAM COMPET CONlROL FiEcovEFlY

Figure 24. Comparison of mean pre-stress anxiety scores.

48
MAACL-R ANXIETY POST

SuFiG SUFG EXAM COMPET CONTROL

,Figure Comparison of mean post-stress anxiety scores. 25.

MACCL-R DEPRESSION

PRE

EXAM COMP CNTFiL

Figure 26. Comparison of mean pre-stress depression scores.

49
90 -

80 -

MAACL-R DEPRESSION

POST

z

7 0 - .>>J;.;.:.?

....~ ...

.... ....

............

o 6 0 - ~~~:~:~~~;;~;

;;;;g&;;

.................. .........

......... ......... .........

.................. ..................

g50 -

.................. .................. .......... .......

\ II: ...... ......

......... .........

..................

..................

..................

..................

.................. ...........................

...........................

......... ..........

.........

..................

.................. .........

...........................

........................... .................

..................

2 4 0 - *:*:*:.:.:*:*:.:*:

...........................

.................

...................

:

........ .........

..................

......... .........

.................. .........

.................. .................. ..................

.:*:.:.:*:*:*:_:.:* :_&i.;.;.;.;.i.

.........

.................................... i

......... ..........

.........

...................

........................... ......... .........

..................

......... ...................

.........

...........................

..................

.......... ..................

..................

......... ...................

...................

..................

.........

.........

..................

.........

..........

............................

...........................

.................. ...........................

..................

..................

.............................

..................

..................

.........

..................

...................

..................

......... ..........

.........

.................. ................... .........

.........

1 0 - *:*:.:*:*:*:*:*:*:

..................

..................

:

........

..............................

.........

.........

................

.........

.................. .................

*:*:*:.:.:*:*:*:*: ..................

.........

0

........................... .................

:_~~;~~*~*~*~,~~~~ ~_:~:*:*~~:_:_~_:*~

.............................. .........

ABDOM WR SS INDEP

Suffi suffi EXAM COMPET CONTFiOL

ALSE

Figure 27. Comparison of mean post-stress depression scores.

MAACL-R HOSTILITY

90

80

g 70

8

* 60

2

w 50

a

40

PRE

!xRG SUFG EXAM COMP CMRL

FiPure 28. Comparison of mean pre-stress hostility scores.

50
.

I -

MAACL-R HOSTILITY

POST

ABDOM v&i

SURG EXAM

ss

COMP

INDEP

CNmL

ALSE

Comparison of mean post-stress hostility scores. Fieure 29.

MAACL-R NEGATIVE AFFECT

ABDOM

!3JFK3 SWIG

vm ss INDEP ALSE PRE ALSE

EXAM COMP CNmL RECOVERY

Fbure 30. Comparison of mean pre-stress negative affect scores.

51
MAACL-R NEGATIVE AFFECT

POST

Figure 3 1. Comparison of mean post-stress negative affect scores.

MAACL-R POSITIVE AFFECT

PRE

60

r

ABDOM vm ss

SUFG EXAM COMP

INDEP ALSE PRE ALSE

CNTRL l7EcmERY

52

Firrure 32. Comparison of mean pre-stress positive affect scores.
.

I - Figure 34. Comparison of mean pre-stress subjective stress scores.

MAACL-R POSITIVE AFFECT

. . . ....... ..........

.......... ..........

.......... ..........

..........

..........

.......... ..........

.......... ..........

.......... ..........

.......... ..........

.......... ..........

.......... ..........

.......... ..........

.......... ..........

.......... ..... . .... .......... . ......... .......... ..........

.......... ..... . . ...

. . ... . . . . . . . . . . . . . . . . . . . . . . . . . .......... .... . . . . . . .... ..... . . . . . ...... .......... ....... ... ... ..... . . ...... ... . .......... ...... .... .......... .......... ... . .... . . ........ . . ..........

ABDOM WI ss INDEP

SWIG EXAM COMP CNTFtL

ALSE

Comparison of mean post-stress positive affect scores. Figure 33.

60

SUBJECTIVE STRESS SCALE PRE

ABDOM vm ss INDEP ALSE PRE ALSE

SURG EXAM CoMPEr CoNmoL REaxEFw

53
SUBJECTIVE STRESS SCALE POST

70 r

I T

ABDOM VW ss INDEP

SURG EXAM COMPET CONTROL

ALSE

I

Firrure 35. Comparison of mean post-stress subjective stress scores.

SPECIFIC RATING OF EVENTS

POST

. . . . . . . . . . e......... . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

SuFiG suffi EXAM COMP CNTRL

Fiwre 36. Comparison of mean post-stress specific rating of events scores.

54
.

A mean pre-stress specific rating of events scores histogram for the five referent protocols is not

available.

The amylase assay data were compared with data from two referent protocols

(Blewett, Redmond, Fatkin, Popp, & Rice, 1995; Hudgens, Chatterton, Terre, Fatkin, & King,

1990). Each of these referent protocols included a pre-stress measurement and a post-stress

measurement. The two referent protocols for the present evaluation are

1. L-P DECON-Litter Patient Decontamination Group-soldiers in MOPP4

processed 40 litter-borne, fully clothed mannequins (1 S-pound average weight) from a triage

area to a hotline.

2. PARA JUMP-Parachute Jump Group-a group of inexperienced males

performed their first parachute jump. Amylase levels were measured immediately upon

touchdown.

The L-P DECON and PARA JUMP protocols represent low to moderate prestress responses and moderate to high stress levels, post trial (see Figures 37 and 38).

Amylase Pre

900

2 800

1

y 700 1

-ti

800

0 500

1

f 400 1

a

j

300 j

L-P DECN PARA

JUMP

ALSE PRE

I

ALSE

FIECCW

Comparison of mean pre-stress amylase scores. Fipure 37.

55
800

T

1 700

k 600

Amylase Post I !

:

:

3 500 -

”

E" 400 -

a

300 -

5

al 200 - H

100

0L

L-P DECN PARA JUMP ALSE POST

FiPure 38. Comparison of mean post-stress amylase scores.

Post-Mission Interviews

Each crew was interviewed separately within 15 minutes of leaving the AH-64CMS

cockpits. The format of the interviews was developed by and was shared courtesy of the Army

Research Institute for the Behavioral Sciences (ARI) Unit at Ft. Rucker, Alabama, and Dr. Robert

Wright of that unit (Wright, Hartson, & Couch, 1996). The investigators conducting the interviews

focused on eliciting problems that each individual aviator encountered and what, if any,

components or combinations of equipment worn were associated with or caused those problems.

This was a somewhat formal opportunity for the subjects to “blow off a little steam” regarding

their discomforts during the trials, interferences the ensemble may have caused between themselves

and the cockpit, and what improvements might make the job a little easier for them. The subjects’

responses and commentary were transcribed in spreadsheet form. Equipment citations as well as

discomfort citations were counted. Figure 39 summarizes those citations. Not shown in the figure

is the fact that the subjects used the general term “bulk” or words to that effect 137 times, when

citing individual components, combinations of components, or the entire ensemble.

56
POST MISSION EQUIPMENT CITATIONS

Based on 80 Individual Interviews

‘DISCOMFORT’ j30

‘UNCOMFORTABLE i54

TOTAL NON. t COMFORT ,B”’

0 100 200 300 ‘loo 500 60[

Number 0, c,tattom

Figure 39. Post-mission citations-equipment and non-comfort.

The ensemble items and combinations of items citations were subjected to a hierarchical

log linear (frequency) analysis. Trial type including familiarization trials and the items were the

factors analyzed. Statistically significant one-way effects for trial type and items were found,

but no two-way association was noted for trial type by ALSE item (p < .OS). Table 29 shows

the number of citations by trial type.

The distribution of items appears to be approximately even across the trials except, of

course, the VR trial in which no over-water equipment was worn. The VR trial may be

responsible for a significant effect of trial type. The mask was overwhelmingly cited in all trials

and is probably responsible for the finding of significance; however, the overgloves and body

armor also have a high number of citations.

The AR1 interview questionnaire format consists of the following sections:

l Physical l Visual l Tactile

l Comfort l Speech l Mission

57
Table 29

Post-Mission Citation of ALSE Items by Trial Type

Item Familiarization FirstR SecondR VR

Overgarment 14

Overboots 5

Overgloves 76

Mask 167

Gloves & mask 19

Helmet 24

Mask & helmet 18

Survival vest 2

Life preserver* 4

HEEDS bottle* 2

Life raft* 26

Body armor 68

Mask & armor 17

Raft & armor* 10

* Over-water items not worn during VR trials

15 17 16

3 1 5

54 46 35

155 134 124

9 7 5

29 22 27

24 16 22

1 1 1

3 4 0

2 2 0

16 22 0

63 47 54

23 17 19

5 10 0

Figure 40 summarizes the number of citations made by the subjects in the various sections.

Note that an overwhelming number of citations were lodged against the mask, with the

helmet-mask combination being cited heavily in the section concerning comfort. Note also that

the section on speech, primarily intelligibility, hearing and being heard, was apparently not a

problem. Further, the armor plate was cited heavily in the physical section.

The mission section of the interview questionnaire inquires about judgments of the subject’s

own performance and is subdivided into

l Situational awareness l Target acquisition l Optical relay tube, heads out

l Radio procedures l Target identification l Optical relay tube, heads down

l Navigation l Target Engagement l Additional time for tasks

l Reporting l Weapons operation l Alternate heads-up display

l Crew coordination l Flight tasks techniques

The items that are underlined are deemed the more critical items in terms of overall mission

performance. For the underlined items, Figure 41 shows the separate citations from the pilots

and CPGs.

58
.

POST MISSION EQUIPMENT CITATIONS

by Questionnaire Section

Based on 80 Individual Interviews

NBC

It

OVERGARMENT ’

NBC OVERBOOTS

NBCOVERGLOVES

MASK (INCL HOSE)

HOSE (ALONE)

GLOVES&MASK 1

HELMET

MASK 8 HELMET1

SURVIVAL VEST

1

LIFE PRESERVER

AIR BOlTLE

ems Cited LIFE RAFT

SECT. 1 VISUAL

PHYSICAL

200

NumbNer of Citations

The counts of the major components and combinations of components were subjected to a

hierarchical log linear (frequency) analysis. Trial type was the factor analyzed. Statistically

significant one-way effects for trial type but no two-way associations (p < .05) were found. A

trial type effect is to be expected for the VR trial and the citations for the over-water components,

as they were not worn for the VR trials. The rest of the components and combinations were rather

evenly distributed across trials.

Figure 40. Post-mission citations by ART questionnaire section.

Again, the mask, the mask and helmet in combination, and the armor plate as well as

discomfort appear to be foremost in the opinions of the subjects. The CPGs had more citations

than did pilots.

59
POST MISSION EQUIPMENT CITATIONS

for Questionnaire, Section 6:

Situational Awareness, Crew Coordination, Flight Task Difficulties l? Additional

Time Required to dompiete Tasks

Based on 80 Individual interviews

,

of Citations

Figure 4 1. Post-mission citations from the ARI questionnaire mission performance section.

DISCUSSION

Behavior-Anchored Ratings

Evaluators

It would have been ideal if three or four qualified evaluators had been present for

the entire experiment. Analysis of the overlapping trials between Evaluator No. 1 and 2 and

between Evaluator No. 2 and 3 demonstrated that the scoring that they performed was

essentially the same. As we shall see, however, the variations in the scores from crew to crew

and from trial to trial were too large.

l i

60
.

As a hedge against the collapsing or aggregating of all the evaluators’ scores for the

rest of the log linear analyses performed, a sampling of the 13 basic quality scores was subjected

to hierarchical log linear (frequency) analyses for the scores of Evaluator No. 2 alone. Subjects,

crews, trial sequence, and trial type were the factors analyzed. No statistically significant main

effects for any of these factors (p < .OS) were found.

Log Linear Frequency Analyses

None of the log linear analyses employing the behavior-anchored rating scores

yielded a statistically significant main effect finding for any of the factors examined. It is believed

that the distribution of these scores, regardless of how they were cross tabulated for the various

factors was too large to be a sensitive measure. Figure 42 is a computation of the standard

deviations of the scores, of the combined evaluators, for the 13 basic qualities. It shows standard

deviations for all the subjects as well as for the pilots and CPGs alone.

Standard Deviations of Evaluation Scores,

All Basic Qualities

0.95

0.90 4

0.85

0.80’

0.75

0.70

0.65

0.60 I I I

BQ1 862 BQ3 B&l Bd5 BQ6 Bd7 Bd8 B;19 BQlO Bdl 1 BQ12 Bd13

Basic Quality

Figure 42. Standard deviations of behavior-anchored rating scores.

-4-Clverall

I -

Figure 43 shows the standard deviations of the aggregated scores, of the combined

evaluators, for the 13 basic qualities, for the third sequential trial alone. This third trial has what

appears to be the lowest set of standard deviations among trial sequence or type. The situation

is the same as discussed before.

61
Standard Deviations of Evaluation Scores, All

Basic Qualities - Third Sequential Trial

BQl BQ2 BQ3 BQ4 BQ5 BQ6 BQ7 BQ8 BQ9 BQIO BQll BQ12 BQ13

Basic Quality

Figure 43. Standard deviations of rating scores for the third sequential trial.

Assuming any kind of normality for the scores distributions, one standard deviation,

or 67% of the scores, even for the lowest value, Basic Quality 13, exceeds the bounds of one point

on a Likert scale of seven points. If three standard deviations account for 98% of the distribution,

then ideally, the computed standard deviation of any of the basic qualities, in order to be sensitive,

ought to be around one-sixth or 0.167.

When viewed graphically, the rating scores arrayed in terms of trial sequence and

trial type appear to exhibit a rather interesting set of trends, even if the findings were not

statistically significant (see Figures 44 and 45).

With the aggregate of scores arranged by trial sequence, without regard for where

the variation trial appeared in the order of presentation, it would appear that there was a general

increase in the values, signifying better performance according to the general hypothesis behind

the Likert scales. This increase is reasonable in the classical sense of more trials, more learning, or

perhaps more to the point in this case, more accommodation to the ALSE ensemble. When the

aggregate of scores is arranged by trial type, the apparent trend is not intuitive. Recall that in the

variation trial, the subjects did not don the over-water components of the ensemble (i.e., the life

raft, flotation vest, and HEEDS bottle). The life raft on the back, when not worn, left the

aviators with approximately an inch more room between the seat back and the instrument panel

and controls, which was greatly appreciated in the aviator’s post-trial interviews. The reduced

62

- I
5.50

5.00

2

8

tn

4.50

4.00

Mean Evaluation Scores, All Basic Qualities by

Trial Sequence

1 2

Trial Sequence

Figure 44. Scores by trial sequence.

5.50

5.00

Q)

8

z

4.50

4.00

Mean Evaluation Scores, All Basic Qualities by

Trial Type

1st R 2nd R

Trial Type

Figure 45. Scores by trial type.

63
ensemble should presumably have made the job easier and performance better. Yet the scores

reflect that, in general, the subjects faired better in the second trial for record, and their variation

trial scored somewhere in between the first and second fully encumbered trials. Surely they did

not have to accommodate to the lesser encumbered, that is, easier ensemble, all over again. Had

the subjects performed considerably better in the variation trial, one might consider the renowned

Hawthorne Effect (Urwick & Brech, 1965) to be a factor because the investigators were doing

something for the aviators-encumbering them to a lesser degree. It may be that for the trial

where the subjects were dressed with fewer encumbering components, they decided that they

could “take it easy this time” and thus performed more poorly than they could have. It is also

felt that the evaluators, who regularly evaluate and rate other aviators, were somewhat reluctant

to go the extremes of the rating scale. Of all the scores for all 13 battle qualities, only one 7 was

posted, no 1s and only three 2s were posted, The subjects knew intellectually that they were

scoring the ALSE and not the aviators, but it is difficult to keep that subjective element separated

when observing the aviators at work in the mission. Dermis K. Leedom7 has suggested that the

use of “Critical Incident” language as related to check-ride evaluation procedures, placed in the

behavioral anchors text, may serve to remove a degree of ambiguity from the scoring as well

(personal communication with the first author, March 1998).

Because of the trend noted previously, an exercise removing the variation in the

evaluation scores attributed to trial sequence was performed. That is, the scores were adjusted by

subtracting the average difference between the first and second trials in the sequence from the

second trial, and subtracting the average difference between the first and third trials in the sequence

from the third trial, for the 13 basic qualities. This “flat-lined” the scores when arrayed by trial

sequence. When arrayed by trial type, the scores exhibited the same characteristics as seen before

but with the effects of sequence or accommodation removed (see Figure 46).

For the purposes of this exercise, a rather risky assumption is that the adjusted

scores now lie along an equal interval scale (they certainly no longer lie along a Likert scale),

allowing ANOVAs to be performed.

Each of the adjusted basic quality scores was subjected to a series of one-way

ANOVAs with subjects, crew, seat (pilot or CPG) and evaluators each as the between-groups

effects.

‘Lead researcher for ARI in development of the Crew Coordination Evaluation Methods and Materials.

64
Mean Adjusted Evaluation Scores, All Basic Qualities

by Trial Type

--BQ 12

5.20

5.00

2 4.80

b

&j 4.60

:

g

4.40

4.20

4.00

3.80 1

1 stR 2ndR

Trial Type

Figure 46. Mean adjusted scores by trial type.

A statistically significant effect was found for each of the basic qualities (BQ)

scores except BQ 8 with respect to subjects (E < .OS). Tukey post hoc tests were applied to the

data. The results were typically 12 to 16 subjects lumped (not significantly different from one

another) together, with an overlap of 6 to 12 subjects in the next non-significance group. This is

to be expected when attempting to separate a large group of means.

A statistically significant effect was found for each of the basic qualities scores

with respect to crews (p < .05). Tukey post hoc tests were applied to the data. The results

were typically five to seven crews lumped (not significantly different from one another) together,

with an overlap of four to six crews in the next non-significance group. This is to be expected

when attempting to separate a large group of means.

No statistically significant effect was found for each of the basic qualities scores

with respect to seat (p < .OS). There were no differences in any of these adjusted scores with

respect to seat.

A statistically significant effect was found for 8 of the 13 basic qualities scores

with respect to evaluator (g < .05). Tukey post hoc tests were applied to the data. In all but

one (BQ 2) of the eight significance instances, Evaluators 2 and 3 were lumped together.

Evaluators 1 and 3 were never grouped together.

6.5
Each of the adjusted basic quality scores was subjected to one-way ANOVAs

with trial type as the between-groups effects. A statistically significant effect was found for

only one of the 13 basic qualities scores, BQ 12 (emphasized in Figure 46), with respect trial

type (B < .05). Tukey post hoc tests were applied to the BQ 12 data (see Table 30), which

point to the difference in the VR and SecondR trials as yielding a significant statistic. In other

words, this is the one instance when the difference in the means was large enough and the amount

of variation around these means was small enough to make a difference. Discounting this one

exception, there were no differences in any of these adjusted scores with respect to trial type.

Table 30

Results of Tukey Post Hoc Test, Basic Quality 12, for Trial Type

Trial type VR

N 30

Means 4.36

First R

30

4.61

Second R

26

4.86

Tukey

p < .05

Means underscored by the same line are not significantly different.

Means not underscored by the same line are significantly different

Figure 47 depicts the standard deviations of the adjusted scores for the 13 basic

qualities, overall and by trial type. They, like the unadjusted scores, range from slightly over a

half point to close to one full point (or just over in one case) on the 7-point scale. Three standard

deviations would then range from one and a half to three points, too wide a range to discriminate.

Figure 48 depicts this for both the unadjusted and the adjusted scores for BQ 12, and Figure 49

has the adjusted scores’ standard deviations for the secondR and VR trials superimposed.

BQ 12 concerns advocacy and assertion where “... crew members advocate a course

of action they consider best even when it may differ with the one being followed or proposed” (U.S.

Army Aviation Center, 1992, Sect. 6). There is not very much rationale for why this particular BQ

should exhibit the largest mean difference, 0.50 or half a point on the 7-point scale, for the VR and

secondR trials, of all the BQs arranged by trial type, unless advocacy happened to be a particularly

important point in the minds of the evaluators during their scoring.

66
l

1.10

0.80

0.70

0.60

0.50

Standard Deviations of Adjusted Evaluation Scores

bY Basic Quality - Overall and Trial Type

BQ2 BQ3 BQ4 BQ5 BQ6 BQ7 BQ8 BQ9 BQIO BQl 1 BQ12 BQ13

Basic Quality

+OveralI

+1sm

0 2ndR

--b-W

Figure 47. Standard deviations of adjusted rating scores, overall and by trial type.

Sld De”= .76

Mean = 4.7

0 N=.9000 0

3.0 4.0 50 0.0 2.Y) 3co 3.50 4cc 4.50 ml 5.50 6.03

8012 m12

Figure 48. Histogram and normal curve for unadjusted and adjusted scores-Basic Quality 12.

67
BQ12

Ntd. Dev = .72

lean=4.60

I= 66.00

Figure 49. Histogram and normal curve for adjusted scores with secondR and VR trial standard

deviations-Basic Quality 12.

The trend of the having the SecondR trial, even with the adjusted scores as the

criteria, be the best of the three trials for record instead of the VR trial is readily apparent. An

explanation for this is not apparent. In the post-trial interviews, all the subjects, to a man,

expressed that the VR trial was a relief from being fully encumbered in the preceding trial(s).

Aviator Experience

The group of subjects participating had a rather complete range of experience in both total

flying hours logged and in Apache flying hours logged. It might be expected that in general, the

subjects would perform in proportion to their level of experience, that is, they would have higher

scores if they had more experience and lower scores if they had a lesser number of hours in their

logbooks. Perhaps a regression could be applied. The log linear analyses examining experience

categories, both total and for the AH-64, weighted by each of the 13 basic quality scores, failed

to conclude that a relationship exists. A sampling of basic quality scores against raw flying

experience, not categorized, also failed to find a relationship. The aggregated scores were

subjected to a Spearman p correlation computation against both total and AH-64 experience,

categorized and raw hours (see Table 3 1).

68
Table 3 1

Spearman’s p Correlations: Basic Quality Scores by Flying Experience

BQl BQ2 BQ3 BQ4 BQS BQ6 BQ7 BQ8 BQ9 BQlO BQll BQ12 BQ13

.

Total .093 .I39 .018 .075 ,126 .059 .083 .007 .106 .121 .095 .056 -.144

hrs.cat

Total

hours

.074 .106 .OOl .038 ,106 .061 .067 .006 .099 .124 .098 .027 -.I80

AH-64 ,083 .154 .093 .065 .161 .117 .143 .037 .155 .171 .197 .094 -.131

hrs.cat

AH-64 .088 .195 .102 .067 .178 .138 ,170 .059 .I55 .199 .222* .126 -.134

hours

* Correlation is significant at the .05 level (two-tailed).

Flying experience does not at all correlate with the behavior-anchored scores. While not

significant, it is curious that BQ 13, “Crew-level after-action reviews accomplished,” had all

negative coefficients. Figure 50 graphically illustrates why no correlation is present.

Post-Mission Interviews

While there was an abundance of equipment citations and complaints, there did not

appear to be a connection between the equipment components cited and any particular trials

encountered. Pearson’s p correlation coefficients were computed for the number of times each

component or combination of components was cited and each of the 13 basic quality adjusted

scores (aggregated) was rendered for each subject and trial for record (see Table 32). No

correlation coefficients approach the customary .70 level that signifies a potentially definitive

relationship among the variables.

Stress Assessment of Aviator Subjects

An evaluation of the stress experience revealed that pilots wearing the ALSE were only

moderately stressed following the test sessions. The Time x Subscore interaction found in the

MAACL-R combined data is readily explained by the fact that as one’s anxiety, hostility,

depression, and negative affect levels increase, one’s positive affect level will naturally decrease.

This relationship can easily be seen in Figure 12.

69
Mean Evaluation Scores for Pilots,

All Basic Qualities by AH-64 Flying Hours

5.50

5.00

: 4.50

s

4.00

3.50

3.00

AHCat 1 AHCat 2 AHCat 3 AHCat 4 AHCat 5

AH-64 Flying Hours Categc

Mean Evaluation Scores for Copilot-Gunners,

All Basic Qualities by AH-64 Flying Hours

6.00

5.50

5.00

g 4.50

s

4.00

3.50

3.00 

AHCat

-PI_

 1 AHCat 2 AHCat 3 AHCat 4

Y

AHCat !

AH-64 Flying Hours Categc

Figure 50. Behavior-anchored scores by AH-64 flying experience-pilot and CPG.

The depression subscore showed a Trial x Time interaction. This is consistent with

previous research (Blewett, Redmond, Fatkin, Popp, 22 Rice, 1995; Mullins, Fatkin, & Patton,

1998) in that higher levels of depression on the first day of testing typically result when soldiers’

expectations exceed their actual performance.

70
.

.

Table 32

Spearman’s p Correlations: Adjusted Basic Quality Scores by Equipment Citations

BQ-I BQ-2 BQ-3 BQ-4 BQ -5 BQ-6 BQ-7 BQ-8 BQ-9 BQ-10 BQ-I 1 BQ-12 BQ-13

Overgarment

Overboots

Overgloves

Mask

Gloves & mask

Helmet

Mask 8z helmet 4

* Survival vest

Life preserver

Air bottle

Life raft

Body armor

Mask & armor

Raft& armor

0.013 -0.070 0.021

-0.22 1* -0.246* -0.189

0.161 0.104 0.167

-0.032 -0.110 -0.029

-0.037 0.057 -0.002

0.006 -0.213* 0.022

-0.096 -0.275* -0.069

-0.094 0.001 -0.121

-0.216* -0.112 -0.207

-0.050 -0.066 -0.073

-0.037 -0.115 0.042

-0.075 0.072 -0.071

-0.206 -0.072 -0.164

-0.135 -0.097 -0.049

-0.042 -0.071 -0.023 -0.077 -0.136

-0.221* -0.330* -0.168 -0.198 -0.191

0.089 0.070 0.072 0.038 0.019

-0.150 -0.181 -0.059 -0.180 -0.109

-0.104 -0.073 -0.089 -0.001 -0.126

-0.030 -0.020 0.001 -0.147 -0.069

-0.080 -0.135 -0.151 -0.251* -0.177

0.014 -0.015 -0.130 0.038 -0.118

-0.133 -0.196 -0.260* -0.151 -0.129

-0.064 -0.081 -0.012 -0.040 -0.001

0.013 -0.042 0.012 -0.003 0.054

-0.056 -0.147 -0.187 -0.100 -0.140

-0.178 -0.310** -0.227* -0.139 -0.224*

-0.038 -0.176 -0.089 -0.046 -0.024

-0.149 -0.233* -0.094

-0.066 -0.188 -0.138

-0.117 -0.136 0.021

-0.160 -0.179 -0.137

-0.154 -0.074 -0.071

-0.166 0.018 -0.148

-0.349** -0.160 -0.295**

-0.119 0.015 -0.114

-O-285** -0.086 -0.206

-0.108 0.084 -0.067

0.037 0.010 -0.022

-0.129 -0.316** -0.178

-0.157 -0.323** -0.242*

0.083 -0.130 -0.063

0.011 0.109

-0.114 -0.194

0.040 0.100

-0.039 -0.071

-0.026 -0.105

-0.007 0.099

-0.096 0.031

-0.281** -0.158

-0.358** -0.139

-0.251* 0.029

0.105 0.006

0.019 -0.162

-0.086 -0.297**

0.136 -0.215*

* Correlation is significant at pc.05 (two-tailed).

** Correlation is significant at ~c.01 (two-tailed).
Psychological and Physiological Stress Assessment

Both the stress perception and salivary amylase measures were used to quantify

individual reactions to potential physical and mental stressors using noninvasive methods with

no interference with the performance of the aviators. When administered concurrently, as in the

present study, these measures provide diagnostic information that helps to identify specific

components of stress. The amylase measure has provided data that indicate “how much” or

“how high” the individual’s physiological stress reactions are, yet do not give information about

which stress component is at work. The stress perception measures such as the MAACL-R

subscales and the Subjective Stress Scale provide information about the probable source of stress.

An evaluation of the stress experience revealed that pilots wearing the ALSE were

only moderately stressed following the test sessions. The Time x Subscore interaction found in

the MAACL-R combined data is readily explained by the fact that as one’s anxiety, hostility,

depression, and negative affect levels increase, one’s positive affect level will naturally decrease.

This relationship can easily be seen in Figure 12.

The depression subscore showed a Trial x Time interaction. This is consistent

with previous research (Blewett, Redmond, Fatkin, Popp, & Rice, 1995; Mullins, Fatkin, &

Patton, 1998) in that higher levels of depression on the first day of testing typically result when

soldiers’ expectations exceed their actual performance.

Comparison With Other Protocols

The data obtained in the current study were compared with data obtained in

previous studies conducted by ARL. These comparisons provide a method for estimating the

relative stress experienced in a given situation. The two surgical protocols represent a relatively

high stress level; the written exam and the soldier competition protocols represent a relatively

moderate level of stress, and the independent control protocol represents a condition ranging

from no stress to low stress.

The stress perception measures completed before testing (pre) and those obtained

1 hour after testing were equivalent to the INDEP CNTRL group, indicating that the pilots were

experiencing little or no stress at those time points. However, post measures more closely

resemble the levels of stress experienced by the WR EXAM and SS COMP groups. Both of

these groups indicate low to moderate levels of stress. Moreover, the stress induced by the WR

EXAM and the SS COMP groups is similar in nature to that of the pilots in the current study in

that it creates a competitive situation in which the individuals are in control of the outcome.

72
Although the aviators’ salivary amylase concentrations were significantly higher

during the trials than during their pre-trial or recovery time points, this increase does not indicate

that the aviators were physically stressed during their trials. When the data are compared with

other studies identified as moderate to high stress scenarios, L-P DECON and PAIU JUMP

protocols, the actual values of amylase concentration for the aviators are acknowledged as

relatively low. A recent study was conducted using both experienced flight instructors and

relatively inexperienced cadets performing real and simulated British Aerospace Hawk MK 5 1

flights (Ylonen, Lyytinen, Leino, Leppaluoto, & Kuronen, 1997). The effects of psychological

workload on physiological symptoms, such as heart rate, were investigated. No statistical

differences were found between the groups, and the only differences in heart rate occurred during

precision instrument final approaches to landing, in minimum weather, which was considered a

cognitively loaded portion of the flight. As in the current study, the changes in physiological

responses were not attributed to a high level of physical workload.

Further examination of the stress levels obtained from the psychological measures

reveals that the moderate stress experienced by the aviators was primarily attributable to a sense

of frustration or discouragement regarding their performance during adverse conditions. The

aviators were required to perform while wearing an uncomfortable ensemble in a confined crew

station, within a battlefield mission and related threat environment. Hence, the stress experienced

was attributable to the psychological or emotional component. Although a moderate level of

psychological stress is typical during a state of arousal or vigilance, it was not dramatic enough to

create an influence on aviator performance. The low levels of anxiety reported by the aviators

indicate that they were certain about the tasks they were asked to perform but may have been

somewhat frustrated or critical of their own performance.

Stress Assessment and Evaluation Scores

Spearman’s p correlations were computed for the post-psychological and post-

.

. amylase measures and the 13 basic qualities scores. The computed coefficients did not have any

patterns of association with any particular measure, and no coefficients reached the .70 level,

signifying a potentially definitive relationship (see Table 33).

Measures of Effectiveness and Flight Data

This set of measures, when analyzed, failed to discriminate among either trial sequence or

trial type. It does represent a baseline of sorts that may be useful in the assessment of prototype

or future ensembles and components. One set of measures stands out in an anecdotal fashion:

73
Table 33

Spearman’s p Correlations: Basic Quality Scores by Stress Assessment Measures

BQ-I BQ-2 BQ-3 BQ-4 BQ -5 BQ-6 BQ-7 BQ-8 BQ-9 BQ-10 BQ-11 BQ-12 BQ-13

Subjective stress

SRE

Anxiety

Hostility

Depression

Positive affect

Negative affect

2 Amylase

-0.090 -0.184 -0.05 1 -0.034 -0.080 -0.275* -0.128 -0.197 -0.356**

0.172 -0.194 0.166 0.131 0.067 0.038 -0.099 -0.026 -0.140

0.038 -0.046 0.124 0.043 0.141 -0.044 -0.115 0.054 -0.049

-0.154 -0.197 -0.233 -0.284* 0.005 -0.114 -0.104 -0.065 -0.150

-0.022 -0.112 -0.020 -0.039 0.104 -0.136 -0.033 -0.056 -0.300*

0.117 0.335** 0.068 -0.028 -0.047 0.072 0.061 0.030 0.072

-0.050 -0.166 -0.085 -0.092 0.123 -0.079 -0.085 -0.040 -0.169

-0.052 -0.118 -0.03 1 0.055 -0.106 -0.084 -0.198 -0.079 -0.185

-0.352** -0.326** 0.023 0.218

-0.022 -0.042 0.197 0.345**

-0.018 -0.015 0.177 0.152

-0.101 -0.277* -0.079 0.160

-0.289* -0.293* 0.022 0.284*

0.106 0.189 -0.056 -0.029

-0.100 -0.228 0.009 0.187

-0.315* -0.192 -0.124 0.208

* Correlation is significant at pc.05 (two-tailed).

** Correlation is significant at ec.01 (two-tailed).
.

during the initial survey of ATM tasks, before the conducted trials at WAATS, several of the

surveyed Apache aviators at Ft. Hood, Texas, as well as at Ft. Rucker, Alabama, volunteered that

the usual time taken to “boot up” the AH-64 was on the order of 20 minutes even with raised

cockpit temperatures when the helicopters were standing on a ramp in bright sunlight. The time

taken in the AH-64CMS with temperatures around 68” F, was in the neighborhood of 30 minutes.

In general, the AH-64CMS computed measures, when analyzed, were found to have

significant test statistics for trial sequence, namely, the first and third trials were different, except

for the events, hits, misses, and no-shots data which showed no significant differences in trial

sequence nor in trial type. The only significant test statistic for trial type occurred when the

means for Maximum Probability of Hit were examined; then the firstR trial differed from the VR

trial. Counter to expectation, the crews (only five) whose missions were captured by the CMS

all had zero hit probabilities assigned to their fast fully encumbered trials for record.

Spearman’s p correlation coefficients were computed for the weapons engagement data and

for each of the 13 basic quality adjusted scores (aggregated) rendered for subjects and their trials for

record (see Table 34). Among the highest computed were the negative coefficients associated with

missile shots that missed the intended targets; however, no correlation coefficients reached the .70

level, signifying a potentially definitive relationship among the variables.

Spear-man’s p correlation coefficients were computed for the threat measures data and

each of the 13 basic quality adjusted scores (aggregated) rendered for subjects and their trials for

record (see Table 35). Among the highest computed were the negative coefficients associated

with total and average exposure times along with the mean exposure zone per event; however, no

correlation coefficients reached the .70 level, signifying a potentially definitive relationship among

the variables.

In general, the data extracted from the CMS mainframe contained gaps and were incomplete

across trials, often beyond the control of the instructor-operators. The job of collating these data

by hand was also long and tedious. Ideally, investigators should have had the ability to extract and

collate these data electronically via software programming. If further work is anticipated in this

venue, it would be worthwhile pursuing this course of action.

Both the engagement measures and threat measures able to be gathered in this investigation

are indicators of the crew’s performance during their missions. Additionally, it became quite

apparent to the investigators, the evaluators, and even to the simulator instructor-operators that

the ALSE ensembles and their attendant distractions were probably responsible for a reduction in

75
Table 34

Spearman’s p Correlations: Adjusted Basic Quality Scores by Weapons Engagement Measures

BQ-I BQ-2 BQ-3 BQ-4 BQ -5 BQ-6 BQ-7 BQ-8 BQ-9 BQ-IO BQ-11 BQ-12 BQ-13

Engagements

Engagementsmissile

Engagementsrocket

Engagements-gun

Kills

Hits

2 Misses

Percent kills

Percent hits

Percent misses

Missile kills

Missile hits

Missile misses

Rocket kills

Rocket hits

Rocket misses

Gun kills

Gun hits

Gun misses

-0.091 -0.043 -0.268* -0.180 -0.23 1 -0.159 -0.079 -0.156 0.072 -0.011 0.066 -0.047 -0.108

-0.157 -0.220 -0.223 -0.140 -0.429** -0.338** -0.188 -0.266* -0.191 -0.142 -0.150 -0.232 -0.161

-0.242 -0.010 -0.172 -0.103 0.036 -0.119 -0.016

-0.054 -0.097

-0.106 0.058

-0.071 0.116

-0.141 -0.198

0.025 0.144

-0.067 0.133

-0.065 -0.288*

-0.017 0.180

-0.086 -0.153

-O-548** -0.525**

-0.193 -0.165

-0.038 0.162

-0.093 -0.103

-0.037 0.023

-0.124 -0.041

0.038 -0.002

-0.079 -0.069 -0.202 -0.295* -0.090 -0.226

-0.024 -0.161 -0.193 -0.119 -0.192

0.111 -0.07 1 -0.07 1 0.093 -0.180

0.035 0.168 -0.123 0.003 0.019

-0.228 -0.145 -0.253 -0.290* -0.199

0.239 -0.037 0.155 0.286* -0.044

0.062 0.201 -0.086 0.045 0.043

-0.237 -0.145 -0.124 -0.272* -0.133

0.151 0.096 0.069 0.118 -0.100

-0.052 -0.040 0.008 -0.032 -0.056

-o-404** -0.404** -0.426** -0.364** -0.639**

-0.200 -0.222 -0.25 1 -0.134 -0.087

0.044 0.182 -0.102 -0.025 0.040

-0.26 I -0.09 1 -0.113 -0.102 -0.029

0.118 -0.050 -0.073 0.056 -0.140

0.059 0.009 -0.159 0.126 -0.159

0.008 -0.114 -0.090 -0.105 -0.094

-0.078 0.082 0.072 0.156 0.005 -0.093

-0.112 0.014 0.040 0.125 0.035 -0.050

-0.048 -0.101 -0.086 -0.069 -0.063 -0.047

-0.179 0.040 -0.019 -0.028 -0.124 -0.162

0.047 0.054 0.100 0.195 0.171 0.081

-0.011 -0.054 -0.058 -0.038 -0.023 0.070

-0.094 -0.044 -0.052 -0.130 -0.166 -0.089

0.015 0.090 0.069 0.166 0.067 -0.009

0.000 -0.234 0.114 -0.132 -0.214 0.040

-0.485** -0.372** -0.362** -0.376** -0.410** -0.307*

-0.236 -0.285* -0.182 -0.33 1* -O-274* -0.166

-0.052 -0.076 -0.123 -0.040 -0.033 -0.047

-0.035 -0.029 -0.111 -0.277 -0.085 -0.189

-0.054 0.119 0.118 0.242 0.129 -0.003

-0.016 0.055 -0.07 1 0.006 0.029 -0.053

0.007 0.107 0.136 0.195 0.030 -0.039

* Correlation is significant at pc.05 (two-tailed).

** Correlation is significant at p<.OI (two-tailed).
Table 35

Spearman’s p Correlations: Adjusted Basic Quality Scores by Threat Measures

BQ-1 BQ-2 BQ-3 BQ-4 BQ -5 BQ-6 BQ-7 BQ-8 BQ-9 BQ-10 BQ-11 BQ-12 BQ-13

Number of events -.233

Number of hits -.102

Number of misses -. 136

Number of no shots -.094

Percent shot at -.044

Total exposure -.374**

time

Average exposure -.428**

2 time

Mean exposure .243

zone

Sum of exposure -.182

zones

Max probability -.009

of acquisition

Max probability -.194

of hit

-.300 -.209 -.068 -.lOO -.135 -.05 1 -.173 .282 .139 .019 .196

-.116 .Oll -.022 -.165 -.057 -.277* .080 -.05 1 .026 -.102 -.186

-.234 -.077 -.091 -.296* -.143 -.341** -.141 .108 .127 .024 -.108

-.136 -.127 .057 .087 -.006 .123 -.061 .280* .164 .066 .238

-.070 -.022 -.069 -.280* -.122 -.339** -.082 .Oll .074 .034 -.180

-.567** -.342** -.261* -.309* -.276* -.226 -.323* -.036 -.091 -.241 -.043

.002

-.l 11

-.071

.031

-.087

-080

-.587** -.402** -.374** -.385** -.347** -.274* -.390** -.235 -.309* -.385** -.157 .037

.195 .282* .187 .189 .220 .024 .091 .195 .343** .289* .033 .095

-.218 -.035 -.070 .Oll -.135 .328* .214 .094 .237 .019

.003

-.404**

-.131

.033

-.107

-.049

-.127

-.007

.150

-.095

.048 .167 -.093 .033

-.136 -.146 ~252 -.026

-004

.040

.052 .075 -.023

-.073 -.016 .021

* Correlation is significant at pc.05 (two-tailed).

** Correlation is significant at ec.01 (two-tailed).
the collective as well as the individual aviator’s situation awareness. Evidence ranged from

instances of too great an exposure and time of exposure to a threat, to munitions fired into the

intervening terrain, to having switches and selectors in the wrong position, to the inability to see

and react to an illuminated master caution light. The National Research Council (1997) states that

in aircraft simulation there is evidence of correlation between mission performance and situation

awareness, but because of the many factors influencing this link, performance measures can only

provide an indirect indication situation awareness. It is universally acknowledged that there is a

definite relationship between performance, workload, and situation awareness albeit a complex one,

and that techniques are advocated to address the quantification of situation awareness (Endsley,

1990). Any further research along these lines ought to include the influence that ALSE technology,

or for that matter, the influence of any proposed new aircraft systems would have on situation

awareness and especially systems whose stated purpose is to enhance aviator’s situation

awareness. The raising of an aviator’s situation awareness in one particular information set could

precipitate a reduction in situation awareness along one or more other, perhaps critical, channels.

Discussion Summary

The behavior-anchored rating scores were not found to have main effect significances.

They did not discriminate among individual aviators, crews, or seat position occupied, which

may be the fault of the wider than required variations in their distributions or may be that there

are no real differences among individual aviators, crews, or seat position occupied. The behavioranchored rating scores were not found to be significant in terms of trial sequence or trial type. In

general, aviator perforrnance.could be judged between “adequate” (scored as 4) and “good”

(scored as 5). Graphically, however, it is quite apparent that an upward trend existed in terms of

the sequence of trials. This could be attributable to the subject’s continued accommodation (not

necessarily learning) over the three trials for record. Also very apparent was the fact that the

variation trials (no over-water equipment) did not produce the highest of the scores by type of

trial, when intuitively, they should have. The subjects may have been exhibiting a degree of

complaisance during that trial or may have had to re-accommodate to the changed ensemble.

Intuition also tells us that the more experienced the aviator, the higher that aviator as a

subject would be likely to score in the trials, and if true, the evaluation scores could be weighted

and thence be more apt to discriminate. Such was not the case. There were no correlations of

experience by basic quality scores higher than .222, and the distribution appears to be almost

random. This is not an avenue to be pursued.

78
.

The adjustment of the evaluation scores to level the assumed effect of accommodation

appeared to improve the analysis but did not quite surpass the probability boundary into

significance except for BQ 12. This adjustment probably should be part of the analysis routine

in any further research along this line.

The three components of the ALSE ensemble giving the subjects the most problems were

the protective mask, body armor, and protective overgloves, followed by the helmet, life raft, and

the combinations of helmet-mask and mask-armor. The combination of the armor in front and the

raft in back caused interference between the CPG and the front seat cyclic stick, preventing full

aft displacement for both crew members. “Pitch-up” maneuvers such as hovering quick stops

and landing flares had to be well anticipated because of this dangerous limitation. This has been

previously observed in the Apache cockpit of the ARI simulator at Ft. Rucker (Wright, Hartson,

& Couch, 1996).

While the stress perception measures, salivary amylase concentrations, and psychological

measures indicated a higher level of stress in the aviators immediately following trials, than did

either pre-trial or recovery levels, these levels were universally lower than those recorded in other

studies characterized as moderate to high stress scenarios.

The capturing of data arising from the AH-64CMS mainframe is full of gaps and not user

friendly in the gathering and reduction of data. If further research or comparative assessments are

to be conducted on this type of facility, automated capturing of electronic data streams of

performance-related parameters should be pursued and accomplished. The data gathered tend to

form a baseline to accompany the other findings of the experiment.

While not formally addressed in the investigation, evidence of situation awareness

degradation, as an effect of the ensembles worn, was observed. Inclusion of this factor in

subsequent research would be well advised.

CONCLUSIONS

The main-line concept of using the framework of crew coordination evaluation, behavioranchored rating system yielded a current equipment baseline set of scores in the neighborhood of 4

to 5 (adequate to good) on a 7-point rating scale. As employed in this initial research into examining

their use as an assessment tool, they did not quite do the intended job. It is concluded that

79
1. The variation in the scoring was much wider than that required to have the system of

scoring be sensitive to the independent variables.

2. At the same time, the distribution of the scoring across the 7-point Likert scale was

much narrower than might have been expected, probably because of an inherent tendency of the

evaluators to avoid the extreme ends of the rating scale.

3. The purpose of rating with behavioral anchors is to enhance the objectivity of the

rating system. Subjectivity remains a significant part of the scoring done in this experiment,

the authors’ opinion.

in

4. The other measures gathered, while forming a somewhat hazy baseline for future

assessments, in the case of the CM&generated data, did not by themselves discriminate among

the various independent variables (nor were they necessarily intended to).

5. It is not concluded that the behavior-anchored rating concept for the assessment of

ALSE has no merit. It can, in our opinion, be useful if the framework is modified to remove or

minimize the conditions that were responsible for generating the imprecision seen in this first

research attempt. The following recommendations are made.

RECOMMENDATIONS

The behavioral anchors generated for this experiment should be retained, but they should

be reviewed toward editing, to perhaps make them even stronger at each end of their range. The

addition of “critical incident” descriptors to their text has been suggested.

The 7-point rating scale should be reduced to either 5 or 3 points. A 3-point scale would

simply be l-poor, unacceptable, a definite decrement in performance relative to current

equipment; 2-acceptable, no significant improvement nor decrement in the case of comparisons

to current ALSE; 3-good, better than may have been expected, a significant improvement over

current equipment. It is believed that this will reduce the standard deviation or variation

observed with the 7-point scale.

The evaluators need to learn to be rather ruthless in their scoring of individual

performance as affected by the ensemble-no compensating for a subject for any reason. The

ALSE is the object of the trials.

.

8 0
.

It would be desirable to conduct enough future trials to carry encumbered subject aviators

to an accommodation asymptote, if there is one. It may be that performance as measured by the

behavior-anchored scales, or by any system, would peak and decrease rapidly just because

mission flying while encumbered is a very tough job, even in a simulator. Barring such an

exercise, continue to adjust evaluation scores for accommodation as was done here.

If a future set of trials is to be conducted to assess prototype or proposed future

ensembles or components, the initial series of trials should contain current and proposed ALSE

as independent variables in order to perform direct comparative analyses.

Include established techniques for the assessment of situation awareness as it may be

influenced by the independent variables.

81
REFERENCES

Blewett, W.K., Redmond, D.P., Fatkin, L.T., Popp, K., & Rice, D.J. (1995). A P2NBC2 report:

Patient decontamination at mobile medical facilities (ERDEC-TR-255). Aberdeen Proving

Ground, MD: U.S. Army Chemical and Biological Defense Agency.

. Cain Smith, P., & Kendall, L. (1963). Performance anwaisal: Assessing human behavior at work

In Bernardin & Beatty (1984).

Chatterton, R.T., Vogelsong, K.M., Lu, Y., Ellman, A.B., & Hudgens, G.A. (1996). Salivary

amylase as a measure of endogeneous adrenergic activity. Clinical Phvsiolom. 16.433448.

Endsley, M.R. (1990). Predictive utility of an objective measure of situation awareness.

Proceedines of the Human Factors Societv 34th Annual Meeting, p 41-45. Santa Monica,

CA: Human Factors Society.

Fatkin, L.T., & Hudgens, G.A. (1994). Stress nercentions of soldiers narticipating in training at

the Chemical Defense Training Facilitv: The mediating: effects of motivation. experience. and

confidence level (Technical Report ARL-TR-365). Aberdeen Proving Ground, MD: U.S.

Army Research Laboratory.

Headquarters, Department of the Army (1992). Aircrew Training Manual Attack Helicopter,

AH-64, (Training Circular TC 1-214). Washington, DC: Author.

Hudgens, G.A., Chatterton, R.T. Jr., Torre, J.P. Jr., Fatkin, L.T., & King, J.M. (1990). Situation

characteristics determine stress response profiles. In Proceedings of the Annual Meeting of

the American Association for the Advancement of Science, New Orleans :

Kerle, R.H., & Bialek, H. M. (1958). The construction. validation. and annlication of a

Subiective Stress Scale (Staff Memorandum Fighter IV, Study 23). Presidio of Monterey,

CA: US Army Leadership Human Research Unit.

Lindquist, E.F. (1953). Design and analvsis of exneriments in nsvcholoav and education.

Boston, MA: Houghton Mifflin.

Metzler, T.P. (1995). Presentation to air warrior management working: proun. St. Louis, MO:

Project Manager, Army Command Information System, U.S. Army Aviation and Troop

Command, Research, Development, & Engineering Command.

Mullins, L.L., Fatkin, L.T., Patton, D.J. (1998). Soldier nerceptions and corrnitive performance

during: nhvsical exertion and heat stress. Unpublished manuscript.

National Research Council (1997). Tactical disnlav for soldiers human factors considerations.

Panel on Human Factors in the Design of Tactical Display Systems for the Individual Soldier.

Washington DC: National Academy Press.

83
Reardon, M., Smythe III, N., Hager, J.D., Helms, B., Omer, J., Freeze, M., & Buccannan, D.

(1996). Evaluation of the effects of thermally stressful UH-60 simulator cockpit conditions

on aviators wearing current MOPPO and encumbered MOPP IV flight uniforms (ARL TN-96-

38). Ft. Rucker, AL: U.S. Army Aeromedical Research Laboratory.

Reardon, M., Smythe III, N., Hager, J.D., Helms, B., Omer, J., Freeze, M., & Buccannan, D.

(1996). Phvsiolonical and nsvcholoaical effects of thermallv stressful UH-60 simulator

cocknit conditions on aviators wearing; standard and encumbered flight uniforms (Report

No. 97-06). Ft. Rucker, AL: U.S. Army Aeromedical Research Laboratory.

Sharkey, T.J., & Schwirzke, M.F.J. (1995). Investigation of the effects of MOPP IV gear on

flight nerformance in the crew station research and develonment facilitv (CSRDF) flight

simulator (Report Number MTI-TR-95940506(1 l)-01). Moffet Field, CA: Monterey

Technologies, Inc., for USAATCOM Aeroflight Dynamics Directorate.

Shively, J., Atencio, A., Shankar, R., Bunzo, M., Bushnell, H., Neukom, C., Wong, C., & Banda,

C. (1995). MIDAS evaluation of AH-64D Longbow Apache crew nrocedures in an airground flight segment: MOPP versus unencumbered Moffet Field, CA: USAATCOM

RDEC Aeroflight Dynamics Directorate.

Simon, R. (199 1). Results of the data analvsis armv aircrew coordination measures testbed

conducted in snring 1990. Wilmington, MA: Dynamics Research Corporation.

Simon, R., Grubb, G., & Leedom, D. (1992). Develonment of crew coordination evaluation

methods and materials (DRC Report E-2 1455U). Wilmington, MA: Dynamics Research

Corporation.

Simon, R., Grubb, G., & Leedom, D. (1993). Develonment of candidate crew coordination

evaluation methods and materials (DRC Report E-21 867U). Wilmington, MA: Dynamics

Research Corporation.

Tauson, R.A., Doss, N.W., Rice, D.J., Tyrol, D.E., & Davidson, D. (1995). The effect of vehicle

noise and vibration (caused bv movinp onerations) on cognitive nerformance in the command

and control vehicle (Technical Report ARL-MR-279). Aberdeen Proving Ground, MD: U.S.

Army Research Laboratory.

Torre, J.P., Wansack, S., Hudgens, G.A., King, J.M., Fatkin, L.T., Mazurczak, J., & Breitenbach,

J.S. (1991). Effects of comnetition and mode of fire on nhvsiological resnonses,

psvcholof$cal stress reactions. and shooting nerformance (Technical Memorandum 1 l-9 1).

Aberdeen Proving Ground, MD: U.S. Army Research Laboratory.

U.S. Army Aviation Center (1992). Crew coordination exnortable evaluation package for Army

aviation. Ft. Rucker AL: Prepared by Army Research Institute for the Behavioral Sciences

and Dynamics Research Corporation.

84
U.S. Army Materiel Command (1996). Instructor utilization handbook AH-64 combat mission

simulator (Revision Al. Orlando FL: Simulation, Training, and Instrumentation Command.

Urwick, L., & Brech, E.F.L. (1965). The making of scientific management: Vo13. The

Pawthorne investigations. London: Pitmans.

Wright, R.H., Hartson, R.R., & Couch, M.E. (1996). Air warrior baseline evaluation. Vol. I,

Surnmarv. Ft. Rucker, AL: U.S.Army Research Institute for the Behavioral Sciences.

. Ylonen, H., Lyytinen, H., Leino, T., Leppaluoto, J., & Kuronen, P. (1997). Heart rate responses

to real and simulated BA Hawk MK 5 1 flight. Aviation Snace and Environmental Medicine,

68,601-605.

Zuckerman, M., & Lubin, B. (1985). Manual for the multinle affect adiective checklist-revised.

San Diego, CA: Educational and Industrial Testing Service. ,

85
APPENDIX A

DESCRIPTION OF ALSE COMPONENTS

87
DESCRIPTION OF ALSE COMPONENTS

Aircraft: AH-64

Crew Position: Pilot-Front seat, Pilot-Gunner Rear seat

Mission motile: Temperate environment, Combat, Over water, NBC Threat

1.

.

2.

3.

4.

5.

6.

7.

8.

9.

10.

11.

12.

13.

14. .

Coverall: Battle Dress Overgarment, NBC or Coverall CWU-27P or CWU-21A/p

Boots: Leather jump boot; Danner; ML-B-44152

Overshoes: NBC Rain boot

Gloves: Std. nomex aircrew MIL-G-8 1188

Overgloves: Butyl Rubber NBC

Mask: M43 Type I

Helmet: Integrated Helmet And Display Sight System (IHADSS)

Side arm: M-9 Beret@ holster

Survival Vest

Floatation vest: Preserver, life, underarm, LPU-10/P

Underwater breathing: HEED SRU3 6/P

Raft: SRU-37P (worn anterior)

Body Armor: Front (only) chicken plate

Radio: Survival AN/PRC- 112A

BOrder Coverall, Survival vest, Floatation vest, Raft, Body armor

89
APPENDIX B

SURVEYED ATM TASKS

91
SURVEYEDATMTASKS

TASK No. Category Task Name

1106 Front Seat

2044 Emerg Pro

1101 Front Seat

1051 Emerg Pro

1145 Both/either

2049 Front Seat

1037 Terrain Flt

1100 Front Seat

1035 Terrain Flt

1054 Emerg Pro

1017 ,Hover

1103 Both/either

1140 Both/either

1141 Both/either

2052 Front Seat

1026 Terrain Flt

1062 Emerg Pro

1142 Both/either

1081 TO/Ldg

1032 Hover

1052 Emerg Pro

1055 Emerg Pro

1079 Both/either

1090 Hover

1098 Back Seat.

1031 TO/Ldg

1078 Emerg Pro

1007 Both/either

1019 TO/Ldg

1020 TO/Ldg

1015 TO/Ldg

1039 Cruise Flt

1028 TO/Ldg

1018 TO/Ldg

1021 Cruise Flt

Perform data entry procedures

Perform actions on contact

Perform TADS boresighting

Perform standard autorotation

Perform IHADSS target tracking

Search for and identifjr targets with the TADS

Perform NOE deceleration

Perform TADS operational checks

Perform terrain flight

Perform simulated single-engine failure, OGE hover

Perform hovering flight

Perform IHADSS boresighting

Engage target with Hellfire

Engage target with the ARCS

Perform target tracking with the TADS

Perform doppler navigation

Perform ECU lockout operations

Engage target with the AWS

Perform nonprecision approach

Perform slope operations

Perform simulated engine failure, IGE hover

Perform single-engine landing

Perform radio communication procedures

Perform masking and unmasking

Perform after-landing tasks

Perform confined area operations

Perform unusual attitude recovery

Perform engine-start, run -up, hover, and before-takeoff checks

Perform a rolling takeoff (minimum power takeoff)

Perform simulated maximum

Perform ground taxi

Perform high-speed flight

Perform VMC approach

Perform a normal takeoff

Perform deceleration/acceleration

93
APPENDIX C

HASTY ATTACK INTO HORSE

95
. I

AIRCREW MISSION BRIEFING

For use of this form. See AR 95-l and Unit SOP.

UNIT: A Co. DATE: PRESENT MISSION NO.: HAHOR-IC-128/TEE-516

BRIEFING: See FRAGO See FRX+Q

1. ENEMY: HAHOROl FRIENDLY: HAHOROl TASK ORG: X CORPS

*2. MISSION: Conduct a hastv attack to destrov enemv tank and

motorized rifle units in the vie of EA HORSE.

.

3. EXECUTION:

*a. Mission Type: A

*b. Authorized Conditions

*c.

*d.

Authorized Fligzodr!e *w$'w

Aircraft/crews/duty status:

A/C TYPE/TAIL # PC/SEAT PI/SEAT

(1) AH-64 /G16 POTTS/ HODGKINS/

(2) AH-64 /Gl2 CMS CREW

(3)

(4)

(5)

*e. Special mission equipment: Qperational AT,O-136.144 and APR 39

f. Authorized Loads: PAX m Cargo/Sling KLB AMMO SEE REMKS

*g. Flight Route: D?O. Deoart HA Astros alona ti Axis O~I to * . OCCUT~V BPS 001 and O-2. Maintain 50 ktas and 100 ft AGL

mtil 2 kms from BP. Formation - co-se. Return same

route 120 ktas and 100 ft AGL.

*h. Mission Restrictions: Do not oo east of N/S Grid 78

*i . Safety Considerations: ID all taraets. 8/17 CAV in VIC of BPS

*j. Risk assessment for this mission is: MEDIUM LOW

4. SERVICE SUPPORT:

,":

Refuel/Rearm location: PRI-MURPH VK 7052 6940/SEC-JUMP VK 7460 6645

Assembly Area/Bivouac/RON Locations: FAA BRAVES VK 8785 8091

C . Maintenance Support: T,imited-HA JUDY and ASTROS - Full Cao.-HA BETH

5. COMMAND AND SIGNAL:

*a. Command:,

(1) Air Mission Commander: CPT POTTS

(2) Command or Support Relationship: 8/17 Attached X CORPS OPCON

b. Signal (except published Freq.) Pee Commo Card. Reoort BP arrival

and de-oarture. Reoort oassina all Phase Lines. and reoort FARP

arrival and denarture to 8/17th Souadron S-3.

6. ADDITIONAL REMARKS: LASER CODES: G-12: A (1112). G16: b (1116)

8/17 CAV GLLD: c28: C (1128)

* MATORY FOR JiJ& FLIGHTS

/s/ Homer S. Reno

BRIEFER'S SIGNATURE

MAJ, AVN

s-3

97
UNCLASSIFIED

COPY_ of -Copies

TF DIGGER,Western Coalition

oyal Moldovian Kingdom

051400sJUN9_

MAJ Payne

FRAGO HAHOROl

Reference: Simulator Map 2317 series M741S ‘TROUT LAKE" 1:50,000

Time Zone used throughout Sierra (Local)

1. SITUATION:

a. Enemy: Early this morning Motorized Rifle and Tank Forces of the

Peoples Krasnovian Republic launched a major attack along the southern border

of the Moldovian Kingdom. The main effort of the attack is at Avenue of

Approach B and C. Currently, lead elements of the 9th GMRR and 21st MRR are

being heavily engaged by units of the 38th and 80th ID in vie of EA BIRD and

DOG. A faint attack by the 3 MRR into the 2nd BDE of the 36th ID operational

area has caused a shift of forces to the west creating a large gap between the

1st and 2nd BDE of the 80th ID. A Tank BN and Motorized Rifle BN Task force

has broken away from the 21st MRR and attempting to split the 1st and 2nd BDE

and gain access to the Division Rear.

b. Friendly: 1st BDE 80th ID has shifted west, closer to the western

boundary of the Division and the 2nd BDE has shifted to protect the Division

Center and Eastern boundary. This repositioning has caused a major gap

between the 1st and 2nd BDE and has thin the 2nd BDE lines. The 38th ID has

counter attacked and pushed the 21st MRR just south of EA BIRD and now are in

defensive positions along the East/West grid 59. Units of the 80th ID are

still in a pitched battle along the East/West grid 60 with heavy casualties to

both sides. The 3rd BDE 80th ID is being held in reserve by Corps and

mechanized infantry units of the 8/17 Cav are being rushed to the vie EA

HORSE.

2. MISSION: OPCON to the 8/17th Cav will conduct a hasty attack and destroy

enemy forces in EA HORSE.

3. EXECUTION: A co l-633 ATKHB moves to contact along Air Axis OWL as soon

as possible and attack motorized rifle and tank units in EA HORSE from BPS O-l

and O-2.

4. Service Support: No change

5. Command and Signal: No change

UNCLASSIFIED

98

..a-- _ -yI..
UNCLASSIFIED

.

6. WEATHER: Clear and 15 NM' Baro Press: 30:oo

TEMP: 15 c PA: 438

BENTEEN

COL

8/17th CAV

Official:

RENO

s3

Annex: A X Corps and 8/17th Cav OPNS Graphics

UNCLASSIFIED

99
APPENDIX D

SAMPLE OF BEHAVIORAL ANCHORS AND GRADE SLIP

101
.

SAMPLE OF BEHAVIORAL ANCHORS AND GRADE SLIP

Basic Qualities and Behavioral Anchors

for the Assessment of Aircrew Life Support Equipment (ALSE)

as it Affects Aircrew Performance

Overview

This method of assessment of ALSE is based on the highly successful Crew Coordination

Training program instituted by the U.S. Army Aviation Center (USAAVNC) and the Crew

Coordination Exportable Evaluation Package for Army Aviation developed by the Army

Research Institute for the Behavioral Sciences. The latter will serve as a framework for the

evaluation of aircrew behaviors as influenced by, and aircrew performance as may be degraded by

the wearing of ALSE during operations. The purpose of conducting such assessments and

evaluations is to: (1) establish baseline bodies of data reflecting the expected performance,

measures of effectiveness and qualities of mission operations employing the current collection of

ALSE components and (2) conduct comparative assessments of developing ALSE which may

take advantage of emerging technologies and the initiative to more thoroughly integrate ALSE

components to each other, the aircraft and most importantly, the Air Warrior.

The Basic Oualitv items below are the behavioral anchors to be employed in the assessment.

They have been adapted for the evaluation of behaviors affect by ALSE by modification and

editing of the Crew Coordination Evaluation items as they appear in the Exportable Package. It

was the intent of the research investigators to retain the crew coordination qualities of the items

as the underlying basis of the assessment methodology, but to focus on the affect of ALSE as

reflected in aircrew behaviors. In-other-words, the aviators both individually and collectively as

the aircrew, will have become the indicators, the yardstick, employed to measure the influence

and ultimate acceptability of ALSE in the cockpit.

Personnel trained and experienced in conducting Crew Coordination Evaluations should, with a

minimum of additional training, be able to adapt and serve as evaluators for this Air Warrior

project. In performing mission flight, the primary considerations that the evaluators must keep

in mind are summarized in the following questions:

Is the aircrew able to overcome the influence or affect of the ALSE in terms of crew

coordination and performance of tasks, or is their behavior, in fact, degraded because of the

ALSE?

Is the aircrew mentally preoccupied or distracted by the ALSE during the mission, which,

in turn, is hurting their crew coordination and performance?

Are members of the aircrew physically hampered in crew coordination efforts and the

performance of tasks because of interferences between the ALSE, the aircraft and their own

bodies?

103
=============================================================

BASIC QUALITY 1. Establish and maintain flight team leadership and crew climate (Crew

Climate)

____--_______--__---~---------_----__--___--____----_------~~ --__---___-________-~~----~~--------~---__----__-----------~~

Explanation:

This rating assesses the quality of relationships among the crew and the overall climate of the

flight deck.

Aircrews are teams with a designated leader and clear lines of authority and responsibility. The

pilot-in-command sets the tone of the crew and maintains the working environment. Effective

leaders use their authority but do not operate without the participation of other crewmembers.

When crewmembers disagree on a course of action, rate the crew’s effectiveness in resolving the

disagreement. Note: Traditional leadership centralizes leadership in the leader with followers

fully dependent on the leader. Functional leaderships assigns leadership and followership roles

as the situation evolves. Flight team leadership recognizes the impact of leadership style on the

working environment. Regardless of leadership style, the pilot-in-command retains final decision

and direction authority.

Superior Rating (7)

The crewmembers have very good interpersonal relationships, with no detriment from ALSE

caused stressors. They respect each others’ skills and appear to enjoy being with each other.

The climate is very open; crewmembers freely talk and ask questions. Crewmembers encourage

the individual with the most information about the situation-at-hand to participate. There is a

genuine concern for good working relationships. No degrading comments or negative voice tones

are used in interactions. Disagreements are perceived as a normal part of crew interactions, and

the crew directly confronts the issues over which the disagreement began. Arguments or

disagreements focus on behaviors or solutions, rather than on personalities. Disagreements do

not arise due to irritation and frustration created by ALSE caused stressors. Each crewmember

carefully listens to others’ comments. Senior crewmembers accept challenges from junior

crewmembers. Alternative solutions are explored. The solution produced is a “win-win’

situation in which all crewmembers’ opinions are considered. The crewmembers have no hard

feelings at the conclusion of the incident.

Acceptable Rating (4)

The crewmembers have sound interpersonal relationships and seem to respect each others’ skills.

The climate is an open one, in which ALSE stressors play little or no role, and crewmembers are

free to talk and ask mission questions. Regardless of rank or duty position, the individual with

the most information about the situation-at-hand is allowed to participate. When disagreements

arise, it may be because of the ALSE. The crew directly confronts the issues over which the

disagreements began. The primary focus is on behaviors or solutions not on ALSE, and no

personal attacks are made in the heat of discussion. The solution is generally seen as reasonable.

Problem resolution ends on a positive note with very little hostility or grumbling among

crewmembers. Mutual respect is clearly intact.

.

104
Very Poor Rating (1)

.

.

Crew interactions are often awkward and uncomfortable due to ALSE caused stressors. The

crewmembers do not appear to like or respect each other. Crewmembers may be curt and

impolite to each other as a result of ALSE stressors. Requirements for assistance are made as

commands rather than as requests for support. When problems arise, the crew fails to directly

confront the issues due to distraction or preoccupation with the ALSE and fail to recognize that

the ALSE may be the underlying cause. Personal attacks may arise. Senior crewmembers are

resistant to recommendations from junior crewmembers. Crewmembers do not explore the range

of possible solutions. They may shout and argue without finding a solution. One or more

crewmembers may retreat and say nothing at all. A “win-lose” situation develops in which one

crewmember is shown to be right and the other to be wrong. The crewmembers show little

respect to one another except for deferring to formal rank. All such behaviors are attributed to

the ALSE.

105
=============================================================

BASIC QUALITY 8. Decisions and actions communicated and acknowledged (Corn&Act)

Explanation:

Rate the extent to which decisions and actions are actually made and announced to the

crewmembers after input is solicited from them. Crewmembers should respond verbally or with

the appropriate adjustment to their behaviors, actions, or control inputs to clearly indicate that

they understand when a decision has been made and what it is. Failure to do so may confuse

crews and lead to uncoordinated operation. Note: Due to time constraints in certain situations,

there is often little or no more time for crews to make inputs to a decision. In such cases, raters

should focus on the extent to which decisions are acknowledged verbally or through coordinated,

pre-planned action.

Examples:

l UH-60 Task 2086, Perform masking and unmasking: P* will announce his intent to

unmask. The P and CE will acknowledge that they are prepared to execute the

maneuver.

l AH-64 Task 1038, Perform terrain flight approach: P* will announce intention of a

go-around...whether approach will terminate to a hover or to the ground. P will

acknowledge

use of manual stabilator or any intent to deviate from the approach.

Superior Rating (7)

The pilot-in-command states, without interference of or preoccupation with ALSE, decisions and

actions and, time permitting,~explains the reasons and intent. Crewmembers acknowledge the

decisions with a clear verbal response, without ALSE interference and ask questions to clarify

any confusion. The pilot-in-command answers all questions in a positive, straight-forward, easy

to hear manner. Crewmembers keep the pilot-in-command informed of the results of their

activities and changing responsibilities - especially visual area of responsibility or task focus, as

may be influenced by ALSE. The crew clearly acknowledges, without ALSE influence, results of

actions, or changes, and then states its intended adjustments based on the information provided.

If crewmembers do not acknowledge or adjust, the pilot-in-command requests acknowledgment.

Crewmembers are particularly attentive to the communication of workload, stress, heat stress

etc. buildups. When assuming control of the aircraft or making control inputs, notification is

always given and acknowledgment received regardless of ALSE interference.

Acceptable Rating (4)

.

The pilot-in-command states, with a minimum of interference of or preoccupation with ALSE,

decisions and actions along with, time permitting, a brief explanation of the reasons and informs

the crew of the adjustments they are expected to make. The crew acknowledges, without ALSE

influence, its awareness of the decisions and directions. Crewmembers may ask questions to

106
.

.

clarify confusion. The pilot answers questions clearly, with little ALSE or speech and hearing

interference, and quickly and the crew adjusts to the new situation. When assuming control of

the aircraft or making control inputs, notification is given and acknowledged with a minimum of

ALSE interference.

Very Poor Rating (1)

Decisions and actions of a crewmember are often not passed on to the crew, due to interference

of or preoccupation with ALSE. The pilot-in-command takes unilateral action and can not

clearly explain or inform the crew of his intended purpose due to ALSE influences, like poor

speech intelligibility. The crew is often not aware that a decision has been made. The crew

infrequently asks questions for clarification. The pilot-in-command may not acknowledge or

respond to questions. The crew may not know how to react to changed circumstances.

Crewmembers may take uncoordinated actions without being able to clearly state intentions or

results, due to ALSE induced confusion. Two pilots may attempt to simultaneously take control

of the aircraft, because of degraded speech intelligibility and poorer hearing ability, when flight

control authority is unclear. Neither crewmember may be clearing the aircraft.

107
_________-______-_--~~~~------_~~~~------~~~~~~~---~--------- _____________-__----~~~-----_-_~~~~------~~~~~~~---~---------

BASIC QUALITY 12. Advocacy and assertion practiced (Advoc/Assert)

==============================------=========================

Explanation:

This rating evaluates the extent to which crewmembers advocate a course of action they consider

best, when it may differ with the one being followed or proposed. Note: Except under extreme

emergency conditions where time is absolutely critical, it is usually in the crew’s best interest to

hear the full range of viewpoints available.

Examples:

l UH-60 and AH-64 Task 2083, Negotiate wire obstacles: Crew will discuss the

characteristics

of the wires...to determine the method of crossing.

l AH-64 Task 2044, Perform actions on contact: Crew will discuss options for

developing

the situation.

Superior Rating (7)

Crewmembers clearly state to the rest of the crew a course of ALSE influenced action that they

consider best. They clearly explain their reasons for believing this to be the best course. Other

crewmembers listen to the argument before presenting any criticism or proposing alternate ALSE

based actions. Discussions focus on the strengths and weaknesses of the proposed course of

ALSE based action, without lengthy discourse on ALSE caused stressors. Crewmembers call the

crew’s attention to changes in the situation, as may be influenced by ALSE, and provide

information that is essential to the proper execution of another crewmember’s task.

Crewmembers pursue feedback to ensure that their views are heard and understood. Other

crewmembers expect such open comments and view them as positive contributions to mission

performance in spite of AWE.

.

.

108
Acceptable Rating (4)

.

Crewmembers state their support for a course of action or suggest improvements to other

proposed ALSE influenced actions. Each crewmember makes an effort to explain his position

and convince others to concur with him on the course of the ALSE based action to be taken.

Other crewmembers may interrupt with their views and alternatives. Crewmembers usually

speak out when they recognize a departure from the best plan, as influenced by ALSE, or

standard procedures or when they have a piece of information that is important to another’s task

execution, e.g., using the HDV and not the ORT. Crewmembers seek assurances that presented

information has been received. Other crewmembers view such comments as constructive and not

a challenge to authority.

Very Poor Rating (1)

The crew almost never suggest course of ALSE influenced action. Crewmembers attempting to

propose a course of action may be cut-off before they can propose the ALSE based action or

explain the rationale for that action. Crewmembers proposing courses of action, as influenced by

ALSE, may receive personal attacks due to frustration caused by ALSE stressors like heat buildup or confinement. The crew raises few, if any ALSE concerns. Crewmembers may even fail to

intervene when ALSE caused risks arise.

109
1

APPENDIX E

AIRCREW COORDINATION EVALUATION GRADE SLIP

111
AIRCREW COORDINATION EVALUATION GRADE SLIP

.

.

AIRCREW COOROINATION EVALUATION (ACE] CHECKLIST

For use of this form, see Aircrew Coordination Exportable Evaluation

Package for Army Aviation.

PC Date

PI

NCM

NO CREW COOROINATlON BASIC QUAUTIES

1 Establish and maintain flight team leadership and crew climate

(Crew Ctimatei

2 Premission planning and rehearsal accomplished (Plan It Rehearse)

/ RATING

5 (Managem ent of ~~unexpected events (Unexp Events)

and directives ctear, timely, relevant, complete, and verified

7 1 Maintenance of mission situational awareness (Sit Aware} I

8 Decisions and actions communicated and acknowledged (Comm/Ack)

9 Supporting information and actions sought ftom crew (Info Sought)

10 Crewmember actions mutually cross-monltored (Cross-Monitor) I

I1 Supporting informanon and actions offered by crew (Info Offered)

I2 1 Advocacy and assertion practiced (AdvocjAssert)

I3 1 Crew-level afteractIon revtews accomplished (AAft) I

valuator’s Signature.

otes:

onsult the behavioral anchored rating guidance. Enter a summary rating (1. 2 ?) in the

sting black for each Basic Quality. Refer to the rating scale below.

RATING SCALE

1

Jery Poor Poor Marginal Acceptable Good Very Good Superior

1 2 3 4 5 6 7

I 1

AIRCREW COORDlNATlOEI EVALtJATiON (ACE) CHECKLIST

113
l

NO. OF

COPIES

2

ORGANIZATION

ADMINISTRATOR

DEFENSE TECHNICAL INFO CENTER

ATTN DTIC OCP

8725 JOHN J KINGMAN RD STE 0944

FT BELVOIR VA 22060-62 18

DIRECTOR

US ARMY RESEARCH LABORATORY

ATTN AMSRL CS AL TA REC MGMT

2800 POWDER MILL RD

ADELPHI MD 20783-l 197

DIRECTOR

US ARMY RESEARCH LABORATORY

ATTN AMSRL CI LL TECH LIB

2800 POWDER MILL RD

ADELPHI MD 207830- 1197

DIRECTOR

US ARMY RESEARCH LABORATORY

ATTN AMSRL D DR R WHALIN

2800 POWDER MILL RD

ADELPHI MD 20783- 1197

DIRECTOR

US ARMY RESEARCH LABORATORY

ATTN AMSRL DD J J ROCCHIO

2800 POWDER MILL RD

ADELPHI MD 20783-l 197

WALTER REED ARMY INST OF RSCH

ATTN SGRD UWI C (COL REDMOND)

WASHINGTON DC 20307-5 100

COMMANDER

USA AEROMEDICAL RESEARCH LAB

ATTN LIBRARY

FORT RUCKER AL 36362-5292

CHIEF

ARMY RESEARCH INSTITUTE

AVIATION R&D ACTIVITY

ATTN PER1 IR

FORT RUCKER AL 36362-5354

MR R BEGGS

BOEING-HELICOPTER CO

P30-18

PO BOX 16858

PHILADELPHIA PA 19 142

NO. OF

COPIES

1

ORGANIZATION

DIRECTOR

US ARMY AEROFLIGHT DYNAMICS DIR

ATTN SAVRT AF D (A W KERR)

AMES RESEARCH CENTER (MS 2 15- 1)

MOFFETT FIELD CA 94035-1099

PROGRAM MANAGER RAH-66

ATTN SFAE AV

BLDG 5300 SPARKMAN CENTER

REDSTONE ARSENAL AL 35898

DENNIS L SCHMICKLY

CREW SYSTEMS ENGINEERING

MCDONNELL DOUGLAS HELICOPTER

5000 EAST MCDOWELL ROAD

MESA AZ 85205-9797

JON TATRO

HUMAN FACTORS SYSTEM DESIGN

BELL HELICOPTER TEXTRON INC

PO BOX 482 MAIL STOP 6

FT WORTH TX 76101

CHIEF CREW SYSTEMS INTEGRATION

SIKORSKY AIRCRAFT M/S S3258

NORTH MAIN STREET

STRATFORD CT 06602

COMMANDER

US ARMY AVIATION CENTER

ATTN ATZQ CDM S (MR MCCRACKEN)

FT RUCKER AL 36362-5 163

DIRECTOR

US ARMY AEROFLIGHT DYNAMICS DIR

MAIL STOP 239-9

NASA AMES RESEARCH CENTER

MOFFETT FIELD CA 94035-1000

DIRECTOR

ARMY NATIONAL GUARD

ATTN NGB AVN MAJ WEST

LTC BACON

111 SOUTH GEORGE MASON DRIVE

ARLINGTON VA 22204

CDR U S ARMY AVIATION

TECHNICAL TEST CENTER

ATTN STEAT CO

CAIRNS ARMY AIRFIELD

BLDG 30601

FT RUCKER AL 36362

115
NO. OF

COPIES

1

ORGANIZATION

DIRECTOR

DIRECTORATE OF COMBAT

DEVELOPMENTS

US ARMY AVIATION CENTER

ATTN ATZQ CD (MR G HARRISON)

FORT RUCKER AL 36362

DIRECTOR

DIRECTORATE OF EVALUATION

AND STANDARDIZATION

US ARMY AVIATION CENTER

ATTN ATZQ DPT ES

FORT RUCKER AL 36362

US ARMY RESEARCH INSTITUTE

ROTARY WING AVIATION RSCH UNIT

ATTN TAPC AR1 IR (DR R WRIGHT)

FORT RUCKER AL 36362-5000

COMMANDING GENERAL

US ARMY AVIATION CENTER

ATTN ATZQ TSM LB

FORT RUCKER AL 36362-5000

DIRECTOR

DIRECTORATE OF TRAINING

DOCTRINE & SIMULATION

ATTN ATZQ TDS GT (MS L THOMPSON)

FORT RUCKER AL 36362

HHC (USAG)

ATTN AFZF AV ST (CW5 T CONNELL)

FT HOOD TX 76544

PROGRAM MANAGER

AIR CREW INTEGRATED SYSTEMS

ATTN SFAE AV LSE (MR T METZLER)

BLDG 5681

REDSTONE ARSENAL AL 35898

CDR

ARMY AVIATION SUPT FACILITY #2

ATTN MAJ K NETTLES

SILVER BELL ARMY HELIPORT AZ 85653

CDR

WESTERN ARNG AVTN TRAINING SITE

SILVER BELL ARMY HELIPORT AZ

85653-9598

NO. OF

COPIES ORGANIZATION

CDR

WESTERN ARNG AVTN TRAINING SITE

AVTN SUPPORT & TRAINING TASK FORCE

ATTN LTC J BRAMAN

SILVER BELL ARMY HELIPORT AZ

85653-9598

CDR

WESTERN ARNG AVTN TRAINING SITE

AVTN SUPPORT & TRAINING TASK FORCE

ATTN MAJ D MITCHELL

SILVER BELL ARMY HELIPORT AZ

85653-9598

ABERDEEN PROVING GROUND

DIRECTOR

US ARMY RESEARCH LABORATORY

ATTN AMSRL CI LP (TECH LIB)

BLDG 305 APG AA

LIBRARY

ARL BLDG 459

APG-AA

USATECOM

RYAN BUILDING

APG-AA

CHIEF ARL HRED ERDEC FIELD

ELEMENT

ATTN AMSRL HR MM (R MCMAHON)

BLDG 459 APG-AA

US ARMY EDGEWOOD RDEC

ATTN SCBRD TD J VERVIER

APG MD 21010-5423

ABSTRACT ONLY

DIRECTOR

US ARMY RESEARCH LABORATORY

ATTN AMSRL CS AL TP TECH PUB BR

2800 POWDER MILL RD

ADELPHI MD 20783-l 197

116
.

REPORT DOCUMENTATION Form Approved

OMB No. 07044188

1. AGENCY USE ONLY (Learn blank) 2. REPORTDATE 3. REPORT TYPE AND DATES COVERED

March 1999 Final

4. TllLE AND SUBl’llLE 5. FUNDINQ NUMBERS

Aviator Behavior and Performance as Affected by Aircrew Life Support and Protective AMS Code 6 11102.74AOOO 11

Esuipmat PR: lL1611102.74A

6. AUTHOR(S) . PE: 6.11.10

Waugh, J.D.; Fatkin, L.T.; Patton, DJ.; Mullins, L.L.; Burton, PA.; Barker, DJ. (all ofARL);

Mitchell, DA. (Arizona ANG)

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS 8. PERFORMING ORQANKATION

REPORT NUMBER

U.S. Army Research Laboratory

Human Research & Engineering Directorate

Aberdeen Proving Ground, MD 21005-5425

3. SPONSORINWMONITORING AQENCY NAME(S) AND ADDRESS

U.S. Army Research Labomtory

Human Research & Engineering Directorate

Aberdeen Provinpr Ground MD 2 1005-5425

I 1. SWPLEMENTARY NOTES

10. SPONSORlNG/MONlTORlNG

AGENCY REPORT NUMBER

ARL-MR-440

2a. DlSlRlBUTlONIAVAlLABlLrrY STATEMENT

Approved for public release; distribution is unlimited.

12b. DlBlRlBUllON CODE

3. ABSTRACT (Maxlmum2tKJwnds)

A methodology for quantifying Army rotaty wing aviator performance as influenced by aircmw life support, survival, and nuclearbiological-chemical clothing and equipment ensembles was examined ht a set of experimental trials conducted ia an AH-64

(Apache) combat mission simulator. The methodology was based on an aircrew evaluation procedure originally developed for use

in the crew qordimmon training of all Atmy aviators. It uses a set of 13 basic qualities, each with behavioral anchors and a 7-point

rating scale, and it is administered by specifically trained senior aviator evaluators. Ten mews, two aviators in each, while fully

encumbered, performed three combat missions for mcord, mpresentative of typical operational tasks, with one “variation” trial

conducted without the over-water components of the ensemble. Measures of effectiveness and flight data, as well as stress

assessment and quipment “complaints” citations, were mcorded. The results indicated that the behavior-anchored scores were not

sensitive enough to statistically discriminate among the independent variables of repeated measures and the variation trials even

though graphically, differences were readily apparent Attempts to apply transformations to the data, based on the aviator subjects’

nlative flying experience and their apparent accommodation to the trials were aiso statistically unsuccessful. The additional

measures collected did not yield statistically significant discrimiions nor did they correlate well with the evaluation scores. A

number of options for improving the technique are offered.

1. SUBJECT TERMS 15. NUMBER OF PAQES

viator performance Apache attack helicopter life support quipment survival equipment 120

ir crew integrated systems behavioral anchors -NBC ensemble 16. PRICE CODE

ir watrior crew coordination evaluation rotary wing aviation

‘. SECURlTY CLASSlFlCATlON 18. SECURllY CLASSlFlCAl-lON 19. SECURITY CLASSlFlCAflON 20. LlMllAllON OF ABSlRACl

OF REPORT OF MIS PAGE OF ABSTRACT

Unclassified Unclassified Unclassified

‘?I____., r__ -,..___ ,. __I