AH-64D Apache Longbow Aircrew Workload Assessment 

for Unmanned Aerial System (UAS) Employment 

by David B. Durbin and Jamison S. Hicks 

ARL-TR-4707 January 2009 

Approved for public release; distribution unlimited. 
NOTICES 

Disclaimers 

The findings in this report are not to be construed as an official Department of the Army position 

unless so designated by other authorized documents. 

Citation of manufacturer’s or trade names does not constitute an official endorsement or 

approval of the use thereof. 

Destroy this report when it is no longer needed. Do not return it to the originator. 
Army Research Laboratory 

Aberdeen Proving Ground, MD 21005-5425 

ARL-TR-4707 January 2009 

AH-64D Apache Longbow Aircrew Workload Assessment 

for Unmanned Aerial System (UAS) Employment 

David B. Durbin and Jamison S. Hicks 

Human Research and Engineering Directorate, ARL 

Approved for public release; distribution unlimited. 
ii 

REPORT DOCUMENTATION PAGE Form Approved 

OMB No. 0704-0188

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing 

data sources, gathering and maintaining the data needed, and completing and reviewing the collection information. Send comments regarding this burden estimate 

or any other aspect of this collection of information, including suggestions for reducing the burden, to Department of Defense, Washington Headquarters Services, 

Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be 

aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not 

display a currently valid OMB control number. 

1. REPORT DATE (DD-MM-YYYY)

January 2009 

2. REPORT TYPE 

Final 

3. DATES COVERED (From - To) 

July - August 2007 

5a. CONTRACT NUMBER 

5b. GRANT NUMBER 

4. TITLE AND SUBTITLE 

AH-64D Apache Longbow Aircrew Workload Assessment For Unmanned 

Aerial System (UAS) Employment

5c. PROGRAM ELEMENT NUMBER 

5d. PROJECT NUMBER 

5e. TASK NUMBER 

6. AUTHOR(S) 

David B. Durbin and Jamison S. Hicks 

5f. WORK UNIT NUMBER 

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 

U.S. Army Research Laboratory 

ATTN: AMSRD-ARL-HR-MJ 

Aberdeen Proving Ground 21005-5425

8. PERFORMING ORGANIZATION 

 REPORT NUMBER 

ARL-TR-4707

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR'S ACRONYM(S) 

11. SPONSOR/MONITOR'S REPORT 

 NUMBER(S) 

12. DISTRIBUTION/AVAILABILITY STATEMENT 

Approved for public release; distribution unlimited. 

13. SUPPLEMENTARY NOTES 

14. ABSTRACT 

This study assessed whether workload was tolerable for AH-64D pilots when they employed an unmanned aerial system 

(UAS) during simulated missions. Ten AH-64D pilots participated in the study. Pilot workload, situation awareness, crew 

coordination, crewstation interface, switch actuations, simulator sickness, visual gaze and dwell times (using a head-eye 

tracker), audio-video, and tactics, techniques and procedures data were collected and analyzed. Pilot workload was found to 

be tolerable for the tasks they performed during the simulated missions. The workload ratings provided by the pilots were 

lower than the Objective and Threshold workload ratings requirements listed in the AH-64D Apache Longbow Block III 

Capability Development Document. Pilots reported that employing the UAS increased their overall task workload, but the 

situation awareness provided by the UAS sensor decreased the workload required to detect and engage targets and decreased 

overall target engagement timelines. 

15. SUBJECT TERMS

AH-64D, unmanned aerial system, crewstation, workload, situation awareness, eye tracker 

16. SECURITY CLASSIFICATION OF:

19a. NAME OF RESPONSIBLE PERSON

David B. Durbin 

a. REPORT 

U 

b. ABSTRACT 

U 

c. THIS PAGE 

U 

17. LIMITATION 

OF 

ABSTRACT 

UU 

18. NUMBER

OF 

PAGES 

76 19b. TELEPHONE NUMBER (Include area code) 

334-255-2069 

Standard Form 298 (Rev. 8/98) 

Prescribed by ANSI Std. Z39.18
iii

Contents 

List of Figures vi

List of Tables vii

1. Introduction 1

1.1 Background .....................................................................................................................1 

1.2 System Description..........................................................................................................1 

1.3 Assessment Overview .....................................................................................................2 

1.4 RACRS Cockpits.............................................................................................................3 

1.5 One Semi-Automated Forces (OneSAF).........................................................................4 

1.6 UAS Control Station .......................................................................................................4

1.7 AH-64 #2 Control Station ...............................................................................................4 

1.8 Terrain Location..............................................................................................................5 

2. Method 6

2.1 Data Collection................................................................................................................6 

2.2 Demographics..................................................................................................................6 

2.3 Assessment of Crew Workload .......................................................................................6 

2.3.1 Bedford Workload Rating Scale..........................................................................7 

2.3.2 Visual Workload..................................................................................................7 

2.4 Assessment of Crew Situation Awareness ......................................................................7 

2.5 Situation Awareness Rating Technique ..........................................................................8 

2.6 UAS Crewstation Interface .............................................................................................8 

2.7 Assessment of Simulator Sickness..................................................................................8 

2.8 Simulator Sickness Questionnaire...................................................................................9 

2.9 Subject Matter Experts ....................................................................................................9 

2.10 Evaluation Design ...........................................................................................................9 

2.11 Overhead Cockpit Cameras...........................................................................................10 

2.12 Switch Actuations..........................................................................................................10 

2.13 Head and Eye Tracker System ......................................................................................10 
iv

2.14 Data Analysis ................................................................................................................11 

2.15 Evaluation Limitations ..................................................................................................11

2.16 Participants ....................................................................................................................11 

3. Results 12

3.1 Crew Workload—Average Bedford Workload Ratings for Flight and Mission Tasks 12 

3.1.1 CPG Task Shedding ..........................................................................................13 

3.1.2 Impact of Workload on Aircrew Coordination .................................................13 

3.1.3 SME Ratings of Aircrew Coordination .............................................................13 

3.1.4 SME Workload Ratings.....................................................................................14 

3.1.5 Visual Workload................................................................................................15 

3.1.6 Comparison of Eye Tracker Data......................................................................18 

3.2 Crew Situation Awareness—Situation Awareness Ratings ..........................................19 

3.3 SME SA Ratings ...........................................................................................................20 

3.4 SME Ratings of Mission Success and Mission Objectives...........................................20 

3.5 UAS Crewstation Interface ...........................................................................................21 

3.6 Switch Actuations..........................................................................................................21 

3.7 Top Improvements Recommended by Pilots ................................................................22 

3.8 Simulator Sickness ........................................................................................................22 

3.9 Comparison of RACRS Simulator SSQ Scores to Other Helicopter Simulators..........23 

4. Conclusion 24

4.1 Crew Workload .............................................................................................................24 

4.2 Crew Situation Awareness ............................................................................................25 

4.3 Crew Coordination ........................................................................................................25

4.4 UAS-Crewstation Interface ...........................................................................................25 

4.5 Simulator Sickness ........................................................................................................26 

5. Recommendations 26

6. References 27

Appendix A. Bedford Workload Rating Scale Scores and Pilot Comments 29

Appendix B. Subject Matter Expert Workload Comments 39
v

Appendix C. Situation Awareness Ratings and Comments 43

Appendix D. UAS-Crewstation Interface Ratings and Comments 49

Appendix E. Top Improvements for UAS Integration 53

Appendix F. Switch Actuations 57

Acronyms 61

Distribution 64 
vi

List of Figures 

Figure 1. Apache RACRS simulator (Camber Corp). ....................................................................3 

Figure 2. Apache RACRS CPG cockpit (Camber Corp)................................................................3 

Figure 3. UAS control station (Camber Corp)................................................................................4 

Figure 4. AH-64 #2 control station (Camber Corp)........................................................................5 

Figure 5. Apache RACRS Afghanistan database screenshot (Camber Corp). ...............................5 

Figure 6. Eye tracker, pupil/camera monitors, and control panel interface..................................10 

Figure 7. PI and CPG workload ratings........................................................................................12 

Figure 8. SME ratings for PI and CPG aircrew coordination.......................................................14 

Figure 9. Comparison of crew and SME Bedford workload ratings. ...........................................14 

Figure 10. CPG visual gaze and dwell times during UAS missions.............................................16 

Figure 11. CPG visual gaze and dwell times during baseline mission (non-UAS mission).........17 

Figure 12. PI visual dwell times estimate. ....................................................................................18 

Figure 13. Overall SART scores for PI and CPG. ........................................................................19 

Figure 14. SME ratings of mission success and mission objectives.............................................21 

Figure A-1. The BWRS. ...............................................................................................................29 

Figure A-2. Visually inside the cockpit vs. non-UAS missions. ..................................................36 
vii

List of Tables 

Table 1. Test variables. ...................................................................................................................9 

Table 2. Pilot demographics (N = 10)...........................................................................................11 

Table 3. SME Bedford workload ratings. .....................................................................................15 

Table 4. Comparison of eye tracker results for AH-64D, ARH, and UH-60M simulations. .......19 

Table 5. SME SA rating................................................................................................................20 

Table 6. MPD page changes during missions...............................................................................22 

Table 7. SSQ ratings. ....................................................................................................................23 

Table 8. Categorization of simulator sickness symptoms.............................................................23 

Table 9. Comparison of RACRS simulator SSQ ratings with other helicopter simulators. .........24 

Table 10. Pilot workload requirements and ratings. .....................................................................24 

Table A-1. Mean Workload Ratings for UAS missions and comparable non-UAS missions. ....30 

Table C-1. SA of battlefield elements during missions. ...............................................................45 

Table F-1. Switch actuations per mission.....................................................................................57 

Table F-2. Time and percentage of time that menu pages were displayed on MPDs...................58 

Table F-3. Time of movement of switches per mission................................................................59 
viii

INTENTIONALLY LEFT BLANK. 
1 

1. Introduction 

1.1 Background 

The AH-64D Apache Longbow Block III upgrade program will provide aircrews with the 

capability to conduct level II (receive unmanned aerial system (UAS) sensor video), level III 

(control UAS sensor), and level IV (control of UAS sensor and air vehicle) interoperability with 

an UAS during missions. Aircrews will employ the UAS as a remote sensor, adding an 

additional sensor capability to the AH-64D reconnaissance and targeting systems. Testing has 

established that AH-64D aircrews can perform basic UAS operations from the crewstation 

during flight; however, there has been no comprehensive examination of the impact of UAS 

operations on aircrew workload. 

The U.S. Army Research Laboratory (ARL), Human Research and Engineering Directorate 

(HRED) conducted an assessment of mental workload that AH-64D aircrews experienced when 

controlling an UAS during simulated missions. The assessment was conducted to determine if 

the pilot (PI) and copilot-gunner (CPG) experienced tolerable workload while they controlled an 

UAS and performed standard flight and mission tasks (e.g., navigation, communication) during 

missions. The assessment also addressed the workload requirement listed in the AH-64D Block 

III (AB3) Capability Development Document (CDD). The CDD requirement is that “Longbow 

Block III crewstation management and cognitive decision aiding systems must provide a 

workload environment for the crew with a Bedford workload rating not to exceed 6.0 

(Threshold) and 5.0 (Objective).” 

1.2 System Description 

The AH-64D Apache is a twin-engine, tandem-seat, aerial weapons platform built by Boeing 

Integrated Defense Systems. Aircraft armament includes a belly-mounted slewable 30-mm chain 

gun, Hellfire missiles, and 2.75 in. aerial rockets. The aircraft integrated sensor suite includes a 

mast-mounted Longbow fire control radar (FCR) and a nose-mounted modernized target 

acquisition designation sight/pilot night vision sensor (MTADS/PNVS). There are two multipurpose displays (MPDs) in each cockpit; the MTADS electronic display and control in the CPG 

crewstation; and the integrated helmet and display sight system. The aircraft has a flight control 

system with a fully articulated, four-bladed main rotor system. The flight control system consists 

of conventional cockpit controls: cyclic, collective, and pedals connected mechanically to 

hydromechanical actuators for the main and tail rotors; a limited authority automatic stabilization 

system; and an electrically actuated stabilator. 
2 

The Block III Apache is a modernized version of the AH-64D Block II Apache aircraft. The 

design incorporates technology advancements from other development programs and advances in 

processing technology since the fielding of the Block I and II AH-64D Apache. The planned 

upgrades should result in improved aircraft performance, reduced operating costs, and improved 

mission performance. One of the major upgrades is the provision for control of an UAS. 

Integration of the Tactical Common Data Link (TCDL) in the AH-64D will provide the aircrew 

with the capability of level II, level III, and level IV control of an UAS. 

1.3 Assessment Overview 

The assessment consisted of operational missions conducted by Apache aircrews in the 

AH-64D Risk and Cost Reduction Simulator (RACRS). The simulator was modified to 

represent the UAS Level 2-4 functionality to the maximum extent practicable based on the 

maturity of the Apache Block III design at the time of the assessment. 

Pilots received two days of training prior to the beginning of the assessment. The training 

consisted of classroom instruction and hands-on flight training in the RACRS. The pilots flew 

the same types of missions during training that they later flew during the record trials. The 

mission scenario was based on a battlefield environment simulating southwest Asia. Each 

successive mission increased in difficulty in order to impose progressively greater workload on 

the pilots. The aircrews performed specific Aircrew Training Manual (ATM) tasks during each 

mission (appendix A). Each ATM task had prescribed conditions and standards to which both 

crewmembers had to perform to help ensure mission accomplishment. 

During the formal evaluation, the aircrews performed Air Escort missions. The mission 

scenarios were developed by the Training and Doctrine Command (TRADOC) Capability 

Manager, Reconnaissance Attack (TCM RA) office in Fort Rucker, AL. The scenarios were 

developed in accordance with established aircraft tactics, techniques, and procedures (TTP). 

The pilots completed the Simulator Sickness Questionnaire (SSQ) before and after each flight. 

They also completed the Bedford Workload Rating Scale (BWRS), the Situation Awareness 

Rating Technique (SART), and the UAS-Crewstation Interface (UCI) questionnaire after each 

mission. During each mission, the CPG wore an eye tracker, which was used to assess pilot 

visual workload. In addition to the pilot data, subject matter experts (SMEs) provided an 

independent assessment of aircrew workload, situation awareness (SA), and mission success. 

The SMEs completed an aircrew workload, SA, and mission success survey after each mission. 

A mission debriefing and after-action review (AAR) were completed after each mission. 

During the simulation, aircrew actions within the cockpit were recorded for post-test analysis. 

Video recordings of each crewstation and all displays were kept as a permanent record. All 

button presses, switch activations, and MPD page displays were recorded during the simulation. 
3 

1.4 RACRS Cockpits 

The RACRS cockpits consisted of high fidelity aircraft flight controls and displays (figures 1 and 

2). The CPG used Target Acquisition and Designation System (TADS) Electronic Display and 

Control (TEDAC) grips to select and control the sensor’s field of view, azimuth, elevation, gain, 

and level. These controls were also selectable for adjustment of the UAS sensor. The TEDAC 

and MPD displays were used to monitor the sensor view from the Apache and/or the UAS. 

Figure 1. Apache RACRS simulator (Camber Corp). 

Figure 2. Apache RACRS CPG cockpit (Camber Corp). 
4 

1.5 One Semi-Automated Forces (OneSAF) 

The OneSAF simulation provided the ability to generate threats and targets on the battlefield. 

Scenarios were developed that incorporated the detection, identification, and acquisition of 

simulated threats and targets. OneSAF produced outputs on the simulation network that enabled 

all connected simulations to receive and display the threats and targets. 

1.6 UAS Control Station 

A stand-alone workstation (figure 3) was developed to allow the UAS operator to independently 

fly an UAS during the scenarios. This was a desktop computer with visual system representing 

the same terrain location as the RACRS. A commercial joystick and keyboard provided user 

input for UAS control. The operator was linked with the Apache crew via audio 

communications using the existing lab intercom. The UAS positional information was output 

onto the simulation network allowing the other components (e.g., wingman) to be aware of its 

location. 

Figure 3. UAS control station (Camber Corp). 

1.7 AH-64 #2 Control Station 

A stand-alone workstation (figure 4), similar to the UAS control station, was developed to allow 

an operator to independently fly a second AH-64 (wingman) during the scenarios. This was a 

desktop computer with visual system representing the same terrain location as the Apache 
5 

simulation. A commercial joystick and keyboard provided user input for the second AH-64 

control. The operator was linked with the Apache crew via audio communications using the 

existing lab intercom. The AH-64 positional information was output onto the simulation 

network allowing the other components to be aware of its location. 

Figure 4. AH-64 #2 control station (Camber Corp). 

1.8 Terrain Location 

The simulator visual system was configured to fly the existing Bagram, Afghanistan, visual 

database (figure 5). This is a geo-specific large gaming area built from satellite acquired highresolution imagery and detailed terrain relief. It also contained several terrain and cultural 

features to increase realism for the Apache pilots. 

Figure 5. Apache RACRS Afghanistan database screenshot (Camber Corp). 
6 

2. Method

2.1 Data Collection 

Pilot workload, SA, crew coordination, UAS crewstation interface, switch actuations, simulator 

sickness, visual gaze and dwell times (head-eye tracker), audio-video, and TTP data were 

collected and analyzed. These areas were assessed to determine if (1) pilot workload was 

tolerable when interacting with an UAS, (2) pilot workload in the AH-64D was higher, lower, or 

comparable to pilot workload when not interacting with an UAS, (3) pilots have adequate SA 

when interacting with the UAS, (4) the UAS control interface was easy to understand and 

navigate, and (5) pilots experienced simulator sickness symptoms. The data were used to 

recommend design improvements to the UAS menu system, training, and associated switchology 

and to refine TTP. 

The BWRS, SART, UCI, SSQ, and SME questionnaires were developed in accordance with 

published guidelines for proper format and content (O’Brien and Charlton, 1996). A pre-test was 

conducted to refine the questionnaires and ensure that they could be easily understood and 

completed by the pilots and SMEs. 

The pilots completed the workload, UCI, and SA questionnaires after each mission. The pilots 

completed the SSQ before and after each mission. The SMEs completed workload, SA, crew 

coordination, and mission success questionnaires after each mission. Additional data were 

obtained from the pilots and the SMEs during post-mission discussions and AARs. 

Questionnaire results were clarified with information obtained during post-mission discussions 

and the AARs. 

2.2 Demographics 

A demographics questionnaire was used to collect basic information on each pilot’s experience 

and flight qualifications. The demographic data documented the range of pilot experience levels 

and qualifications. 

2.3 Assessment of Crew Workload 

A common definition of pilot workload is “the integrated mental and physical effort required to 

satisfy the perceived demands of a specified flight task” (Roscoe, 1985). It is important to assess 

pilot workload because mission accomplishment is related to the mental and physical ability of 

the crew to effectively perform their flight and mission tasks. If one or both pilots experience 
7 

excessively high workload while performing flight and mission tasks, the tasks may be 

performed ineffectively or abandoned. In order to assess whether the pilots are task-overloaded 

during the missions, the level of workload for each pilot must be evaluated. 

2.3.1 Bedford Workload Rating Scale

The pilots completed the BWRS (appendix A) immediately after each mission to rate the level of 

workload that they experienced when performing flight and mission tasks. The tasks were 

selected because they were estimated to have the most impact on aircrew workload during the 

missions. 

The BWRS has been used extensively by the military, civil, and commercial aviation 

communities for pilot workload estimation (Roscoe and Ellis, 1990). It requires pilots to rate the 

level of workload associated with a task based on the amount of spare capacity they feel they 

have to perform additional tasks. Spare workload capacity is an important commodity for pilots 

because they are often required to perform several tasks concurrently. For example, pilots often 

perform navigation tasks, communicate via multiple radios, monitor aircraft systems, and assist 

the pilot on the controls with flight tasks (e.g., maintain airspace surveillance) within the same 

time interval. Mission performance is reduced if pilots are task saturated and have little or no 

spare capacity to perform other tasks. Integration of the UAS control with the AH-64D 

crewstation should help ensure that pilots can maintain adequate spare workload capacity while 

performing flight and mission tasks. 

2.3.2 Visual Workload 

An eye tracker was used during the evaluation to assess visual gaze and dwell times for the 

pilots. The data were collected to help determine how often the CPG was able to maintain visual 

focus outside the aircraft to assist with navigation (e.g., identification of terrain features), local 

security, terrain flight, etc., when having the added task of controlling an UAS. Visual gaze and 

dwell time data help identify whether pilots experience excessive visual workload or cognitive 

capture because they had problems interpreting information presented to them on the crewstation 

displays (for example). 

2.4 Assessment of Crew Situation Awareness 

SA can be defined as the pilot’s mental model of the current state of the flight and mission 

environment. A more formal definition is “the perception of the elements in the environment 

within a volume of time and space, the comprehension of their meaning, and the projection of 

their status in the near future” (Endsley, 1988). It is important to assess SA because of its 

potential to directly impact pilot and system performance. Good SA should increase the 

probability of good decision making and performance by aircrews when conducting flight and 

mission tasks in the AH-64D. 
8 

2.5 Situation Awareness Rating Technique 

The SART (appendix C) is a multi-dimensional rating scale for operators to report their 

perceived SA. The SART was developed as an evaluation tool for the design of aircrew systems 

(Taylor, 1989) and examines three components of SA: understanding, supply, and demand. 

Taylor proposed that SA is dependent on the pilot’s Understanding (U) (e.g., quality of 

information they receive), and the difference between the Demand (D) on the pilot’s resources 

(e.g., complexity of mission) and the pilot’s Supply (S) (e.g., ability to concentrate). When D 

exceeds S, there is a negative effect on U and an overall reduction of SA. The formula 

SA = U - (D - S) is used to derive the overall SART score. The SART is one of the most 

thoroughly tested rating scales for estimating SA (Endsley, 2000). 

2.6 UAS Crewstation Interface 

The UCI impacts crew workload and SA during a mission. A UCI that is designed to augment 

the cognitive and physical abilities of crews will minimize workload, enhance SA, and contribute 

to successful mission performance. To assess the UCI, the pilots reported any problems that 

contributed to high workload and low SA at the end of each mission. They also completed a 

lengthy questionnaire at the end of their final mission. The questionnaire addressed usability 

characteristics of the UCI. 

2.7 Assessment of Simulator Sickness 

Simulator sickness has been defined as a condition where pilots suffer physiological discomfort 

in the simulator, but not while flying the actual aircraft (Kennedy, Lilienthal, Berbaum, Balzley, 

and McCauley, 1989). It is generally believed that simulator sickness is caused by a mismatch 

either between the visual and vestibular sources of information about self-motion, or between the 

sensory information (e.g., acceleration cues) presented by the simulator and the sensory 

information presented by the primary aircraft that the pilot operates. When the sensory 

information presented by the simulator does not match the aircraft, the pilot’s nervous system 

reacts adversely to the sensory mismatch and the pilot begins to experience discomfort. 

Characteristics of simulator sickness include nausea, dizziness, drowsiness, and several other 

symptoms (Kennedy et al., 1989). It is important to assess simulator sickness because the 

discomfort felt by pilots can be distracting. Pilot distraction is one of the operational 

consequences of simulator sickness listed by Crowley (1987). If pilots are distracted by the 

discomfort they feel during missions, their performance is likely to suffer. Additionally, the 

discomfort could influence the perceived levels of workload and SA that the pilots experienced 

during a mission.
9 

2.8 Simulator Sickness Questionnaire

The SSQ was administered to the pilots to estimate the severity of physiological discomfort that 

they experienced during missions and help assess whether they were being distracted by the 

discomfort. The SSQ (Kennedy, Lane, Berbaum, and Lilienthal, 1993) is a checklist of 16 

symptoms. The 16 symptoms are categorized into three subscales. The subscales are 

Oculomotor (e.g., eyestrain, difficulty focusing, blurred vision), Disorientation (e.g., dizziness, 

vertigo), and Nausea (e.g., nausea, increased salivation, burping). The three subscales are 

combined to produce a Total Severity score. The Total Severity score is an indicator of the 

overall discomfort that the pilots experienced during the mission. 

2.9 Subject Matter Experts

Two SMEs observed each mission and rated crew workload, crew SA, and mission success. The 

SMEs provided an independent assessment of the workload and SA levels experienced by the 

crews. They also helped identify whether problems with crew workload or crew SA contributed 

to lack of mission success. 

The SMEs were pilots assigned to the TCM RA and the AH-64D program management office. 

They held the rank of CW4 and each had over 3000 h of flight experience. The pilots had 

substantial experience conducting attack missions and were familiar with the UAS mission. 

They observed each mission using a suite of monitors that showed all crewstation displays and 

the location of the aircraft, friendly forces, and enemy forces. They also listened to all audio 

communications between crewmembers and outside sources (e.g., wingman, ground 

commander) during the missions. A SME conducted an AAR with the pilots at the end of each 

mission. During the AAR, the SME reviewed the positive and negative aspects of the mission to 

provide instruction to the pilots and develop and refine TTP. 

2.10 Evaluation Design 

While the evaluation was operational in nature rather than experimental, multiple variables were 

controlled in order to maximize the validity of the conclusions regarding the areas of evaluation. 

Table 1 summarizes the variables that were controlled during the simulation. 

Table 1. Test variables. 

Factor Control Conditions 

Mission Constant Air assault 

Flight profile Tactically varied Nap-of-Earth (NOE), contour 

Light conditions Constant Day 

Scenario Constant Southwest Asia 

Crew Constant Maximize crew familiarity 

Seat position Varied Front, back 

Flight uniform Constant Air Warrior Gen 3 Combat-Basic 
10 

2.11 Overhead Cockpit Cameras 

An overhead camera was mounted in the front seat and a small camera was mounted on the 

glareshield in the back seat to record pilot actions. This aided in determining what the pilots 

were doing during different phases of the mission. The cameras had a time stamp added so that 

the elapsed time could be compared with other data that was collected. 

2.12 Switch Actuations 

When initiated, the data collection software collected switch actuations at a 10 Hz rate. The 

collected data were time stamped and written to a comma delimited log file. The log file was 

then post-processed to an Excel spreadsheet format. The data collection software provided an 

interface to allow the operator to initiate and cease collection during an executed run. 

2.13 Head and Eye Tracker System 

CPG visual gaze and dwell times were collected through a head and eye tracking system from 

Applied Science Laboratories (ASL). The ASL system was used because it was capable of 

integrating a laser head tracker to allow unrestricted head movement during data collection and it 

was compatible with the integrated helmet and display sight system (IHADSS) flight helmet. 

The ASL Eye-Head Package included a Model 501 eye tracker and an Ascension Laserbird head 

tracker. This technology allowed us to collect data that specified point of gaze with respect to 

stationary objects (e.g., MPDs) within the CPG crewstation. The ASL software allowed data 

collectors to continuously monitor the eye position of the pilots by crosshairs superimposed over 

live imagery (figure 6). The software also included a built-in analysis tool that allowed data to 

be viewed in tabular or graphical format.

Figure 6. Eye tracker, pupil/camera monitors, and control panel interface. 
11 

2.14 Data Analysis 

Pilot responses to the BWRS, SART, SSQ, and UCI questionnaires were analyzed with means 

and percentages. Their responses to the BWRS, SART, and SSQ were further analyzed with the 

Wilcoxon Signed Rank Test (WSRT) to compare the ratings between the CPG and pilot and 

between ratings for the UAS missions vs. comparable non-UAS missions to determine if the 

differences were statistically significant (≤.05). 

The eye tracker data were summarized by calculating the total percentage of fixations that 

occurred for the different areas of interest (AOIs). Six AOIs were created for the CPG: right 

MPD, left MPD, TEDAC, keyboard, and kneeboard. Visual gaze and dwell times were also 

recorded for out-the-window. A final category, called “Other,” captured eye fixations not 

focused on a specific AOI. 

2.15 Evaluation Limitations 

The primary limitations included the small sample size of pilots (N=10) who participated in the 

assessment, the limited training they received (2 days), limited number of missions they 

conducted, and mix of AH-64D software versions in the RACRS. 

These limitations are not uncommon when replicating a complex aviation system in a simulator. 

However, the information and data listed in the Results and Summary sections of this report 

should be interpreted based on these limitations. Additional data should be collected during 

future simulations and tests to augment and expand the findings contained in this report. 

2.16 Participants 

Ten AH-64D pilots participated in the assessment. Six pilots were assigned to the 3-101 

Aviation Regiment, Fort Campbell, KY; two pilots were assigned to the 21st Cavalry Brigade, 

Fort Hood, TX; and two pilots were assigned to the Forces Command (FORSCOM) HQ G-3 

Office, Fort McPherson, GA. Nine pilots held the rank of warrant officer (CW2 = 2 pilots, 

CW3 = 3 pilots, CW4 = 3 pilots, CW5 = 1 pilot) and one pilot held the rank of Captain. Four 

pilots were rated Flight Activity Category (FAC) 1 and six pilots were FAC 2, and eight pilots 

were Readiness Level (RL) 1 and two pilots were RL 3. They represented a broad range of 

experience with total flight hours from 650 to 3875 h. The relevant demographic characteristics 

of the pilots are shown in table 2. 

Table 2. Pilot demographics (N = 10). 

Summary of 

Demographic 

Characteristics 

Age 

(yrs) 

Flight Hours 

in AH-64D 

Total Flight Hours 

in Army Aircraft 

Mean 

Median 

Range 

36 

37 

25–43

860 

750 

550–1620

1897 

1400 

650–3875
12 

3. Results

3.1 Crew Workload—Average Bedford Workload Ratings for Flight and Mission Tasks

The average Bedford workload rating (for all tasks) was 2.6 for the CPG and 2.9 for the PI 

(figure 7). These ratings indicate that the CPGs and PIs typically felt that workload was 

tolerable for the tasks and they had enough spare mental capacity for all desirable additional 

tasks. The difference in workload ratings between the CPG and PI was not statistically 

significant (WSRT, Z = –1.856, p = .093). This suggests that pilots perceived that workload 

levels were comparable for the CPG and PI during most missions. 

2.6 2.9 3 2.8

0

2

4

6

8

10

UAS Mission Non-UAS Mission

Pilot Bedford Workload Ratings

CPG

PI

Figure 7. PI and CPG workload ratings. 

The estimated average workload ratings for comparable non-UAS missions (using MTADS 

and/or FCR only) was 3.0 for the CPG and 2.8 for the PI. These ratings indicate that the pilots 

typically believed that workload was tolerable for the tasks and they had enough spare mental 

capacity for all desirable additional tasks when performing comparable missions without 

employing a UAS. The differences in workload ratings between UAS missions and comparable 

non-UAS missions was statistically significant for the CPG (WSRT, Z = –2.840, p = .006), but 

not statistically significant for the PI (WSRT, Z = –.577, p = .774). The majority of pilots 

commented that the workload levels they experienced during UAS missions were comparable to 

workload levels they experienced during missions using only the MTADS and/or FCR. They 

reported that having to manage an additional sensor (UAS sensor) increased their overall task 

workload, but the SA provided by the UAS sensor typically decreased the workload required to 

detect and engage targets. They stated that (1) it was easier for them to detect and engage targets 

using the UAS sensor because of the steep visual aspect angle (“God’s Eye” view) that the UAS 
13 

sensor provided compared to the shallower visual aspect angle that the MTADS and/or FCR 

typically provides during missions; (2) the SA provided by the UAS sensor reduced the time 

required to detect and engage targets; and (3) while they had more tasks to complete because of 

the additional sensor, they had more time to complete target detection and engagement tasks 

because of the greater stand-off range that the UAS sensor provided vs. the stand-off range that 

the MTADS/FCR provides during missions. 

3.1.1 CPG Task Shedding 

The CPGs reported that there were very few tasks that they had to ask the PIs to perform (due to 

high CPG workload). ARL personnel observed only a few instances when the CPG asked the PI 

to perform a task because he was experiencing high workload. However, the CPGs occasionally 

shed tasks such as (responding to) radio calls, did not maintain positive control of the MTADS 

and let it drift into its limit stops, and asked for Level 4 control (but did not take Level 4 control) 

of the UAS because they were busy with other tasks. 

3.1.2 Impact of Workload on Aircrew Coordination 

The majority of PIs (80%) reported that the required level of crew coordination was higher 

during UAS missions than comparable non-UAS missions. They commented that having to 

manage an extra sensor increased their workload and required them to interact more with the 

CPG. The CPGs were split on whether having to manage an extra sensor increased the required 

level of crew coordination: 50% of CPGs reported that the required level of crew coordination 

was higher, 30% reported it was lower, and 20% reported it was comparable to a non-UAS 

mission. 

3.1.3 SME Ratings of Aircrew Coordination 

The SMEs provided ratings (figure 8) of how well the crews performed aircrew coordination 

tasks (e.g., positive communication) per Training Circular 1-210. The SMEs rated aircrew 

coordination during most missions as “Good” (50%) or “Average” (35%). 
14 

15%

50% 35%

0%

20%

40%

60%

80%

100%

Excellent Good Average

SME Ratings of Aircrew Coordination

Figure 8. SME ratings for PI and CPG aircrew coordination. 

3.1.4 SME Workload Ratings 

SMEs provided an overall Bedford workload rating for each mission that they observed (figure 

9). The average SME Bedford workload rating (table 3) was 4.9 for the CPG and 4.3 for the PI 

for all missions. These ratings indicate that the SMEs believed that (1) workload was tolerable 

for the CPG and PI but that they had reduced spare workload capacity, (2) the CPG could not 

give the desired amount of attention to additional tasks, and (3) the PI had insufficient spare 

capacity for easy attention to additional tasks. The difference in SME workload ratings between 

the CPG and PI was not statistically significant (WSRT, Z = –1.806, p = .088). 

2.6 2.9

4.9 4.3

0

2

4

6

8

10

Pilot Ratings SME Ratings

Pilot and SME Bedford Workload Ratings

CPG

PI

Figure 9. Comparison of crew and SME Bedford workload ratings. 
15 

Table 3. SME Bedford workload ratings. 

Workload Ratings PI CPG Combined 

Average 

Median 

4.30 

4.00 

4.90 

5.00 

4.60 

4.50 

Standard Deviation (S.D.) 1.49 1.37 1.43 

In previous simulations and operational tests that ARL has helped conduct, SMEs have typically 

rated pilot workload higher than the ratings provided by the pilots. SMEs have more information 

(e.g., location of all threat and friendly vehicles) available to them to assess pilot performance 

and workload than the information that the pilots have available to them. SMEs are often more 

aware of pilot mistakes (that may be attributable to workload) than the pilots. The additional 

information that the SMEs have likely results in a more critical assessment of pilot performance 

and workload. This area requires further study to identify the reasons why SMEs typically rate 

pilot workload higher than the ratings provided by the pilots. 

3.1.5 Visual Workload 

Figure 10 shows the average percentage of time that the CPGs were visually focused on each 

AOI during the missions. The CPGs were visually focused on the right MPD (UAS sensor 

video) for 49% of the time during missions. They were visually focused on the left MPD 

(Tactical Situation Display (TSD)) for 19% of the time and the TEDAC (MTADS sensor) for 

14%. It is interesting to note that the CPGs typically spent only 6% of the time visually focused 

out-the-window during missions. The majority of CPGs (80%) reported on post-mission surveys 

that they were “inside” the cockpit more than during a non-UAS mission; however, they 

commented that the amount of time they were visually inside the cockpit was not much more 

than a comparable non-UAS mission. They stated that they did not believe that the amount of 

time they were visually inside the aircraft increased the probability of accidents. Two CPGs 

reported that not having the visual scene represented on the Helmet Display Unit (HDU) kept 

them visually inside the aircraft more than if they were able to use the HDU for tasks such as 

local security and navigation. 
16 

Figure 10. CPG visual gaze and dwell times during UAS missions. 

One baseline mission (MTADS/FCR only, no UAS) was conducted to compare how much time 

the CPG was visually focused outside the aircraft during the Air Assault mission (figure 11). 

The CPG spent just 2% of the time visually focused out-the-window during the baseline mission. 

The baseline mission suggests that the amount of time that CPGs are visually focused outside the 

aircraft during comparable missions (e.g., day, visual flight rules) when employing the UAS and 

MTADS/FCR is similar to missions when employing only the MTADS/FCR. 

MPD 

49% 

MPD 

19% TEDAC 

14% 

Keyboard 

2% Kneeboard/ 

Printed Map 

6%

Out-The-Window 

6%

Other 4% 
17 

Figure 11. CPG visual gaze and dwell times during baseline mission (non-UAS mission). 

The PIs were split when asked if they were “inside” the cockpit more than during a non-UAS 

mission: 50% of the PIs reported they were inside more than during a comparable non-UAS 

mission, 40% reported they were inside “about the same” as a comparable non-UAS mission, 

and 10% said they were inside less than a comparable non-UAS mission. A small camera was 

mounted on the glareshield in the rear cockpit so that ARL personnel could observe how much 

time the PIs spent visually focused inside vs. outside the aircraft. ARL personnel observed that 

the PIs were visually focused outside the aircraft approximately 75% of the time and inside 25% 

of the time (figure 12). The PIs confirmed the observations made by ARL personnel during 

post-mission discussions. 

MPD 

4% 

MPD 

16% TEDAC 

78% 

Keyboard 

<1% Kneeboard/ 

Printed Map 

0%

Out-The-Window 

2%

Other 

<1%
18 

Figure 12. PI visual dwell times estimate. 

During the missions, the CPGs were not able to maintain visual focus outside the aircraft to 

assist with navigation (e.g., identification of terrain features), local security, terrain flight, etc. 

For 90%+ of the time, the CPGs were visually inside the aircraft performing (mostly) target 

detection and engagement tasks. It should be cautioned that the simulation was conducted with a 

small sample size of pilots, pilots did not get all of the peripheral visual cues that they would in 

an aircraft, pilots know that they cannot die if they crash the simulator (vs. aircraft), and only one 

baseline mission (MTADS only) was conducted. 

3.1.6 Comparison of Eye Tracker Data 

Table 4 shows a comparison of AH-64D, Armed Reconnaissance Helicopter (ARH), and UH60M eye tracker data for Visual Flight Rules (VFR) flight during simulations. The ARH and 

UH-60M simulations were conducted to assess the human factors characteristics of the aircraft as 

part of system development and/or operational testing. While the aircraft, missions, training, and 

personnel experience levels were different for each simulation evaluation, it is interesting to note 

the differences in visual gaze and dwell times for each evaluation. 

Out-The-Window 

75% 

In The Cockpit 

25% 

Camera Used For 

Observing Pilot Visual 

Gaze and Dwell Times 
19 

Table 4. Comparison of eye tracker results for AH-64D, ARH, and UH-60M simulations. 

AH-64D/UAS 

Workload 

Assessment 

UH-60M Early 

User 

Demonstration 2 

UH-60M Limited 

User Test 

UH-60M Limited 

Early User 

Evaluation 

ARH HFE-CAAS 

Evaluation 

Flying 

Pilota CPG 

Flying 

Pilot 

Non 

Flying 

Pilot 

Flying 

Pilot 

Non 

Flying 

Pilot 

Flying 

Pilot 

Non 

Flying 

Pilot 

Flying 

Pilot 

Non 

Flying 

Pilot 

Outside 75% 6% 69% N/A 86% 28% 61% 26% 61% 7% 

Inside 25% 94% 31% N/A 14% 72% 39% 74% 39% 93% a

Estimate from watching PI gaze and dwell times with video camera during missions (non-eye tracker). 

3.2 Crew Situation Awareness—Situation Awareness Ratings 

The overall SART scores provided by the pilots were 24.0 for the CPG and 25.7 for the PI 

(figure 13). These scores indicate that the CPG and PI felt they had moderate levels of overall 

SA during the missions. The difference between SART scores for the CPG and PI was not 

statistically significant (WSRT, Z = –.612, p= 0.572). The estimated SART scores for 

comparable non-UAS missions (MTADS/FCR only) were 18.4 for the CPG and 23.2 for the PI. 

The difference between SART scores for the UAS missions and comparable non-UAS missions 

was not statistically significant for the CPG (WSRT, Z = –1.620, p= 0.133) or for the PI (WSRT, 

Z = –1.127, p= 0.289). The pilots stated that they had higher SA during UAS missions (vs. nonUAS missions) mostly because of the “God’s Eye” view that the UAS sensor video provided 

during missions. The UAS sensor video also gave the pilots good SA earlier in the mission (vs. 

non-UAS missions) because they often received the video prior (or just after) take-off of their 

aircraft. 

24

25.7

18.4

23.2

0

5

10

15

20

25

30

35

40

45

UAS Mission Non-UAS Mission

SART Scores For UAS Mission vs. Non-UAS 

Mission

CPG

PI

Figure 13. Overall SART scores for PI and CPG.
20 

The pilots reported that they had high levels of SA of most of the battlefield elements (appendix 

C) during the missions. The battlefield elements included location of enemy and friendly units, 

location of own ship, location of cultural features (e.g., bridges), and route information (e.g., 

waypoints). However, there were several instances when the aircraft flew near (or over) the 

target(s) and onboard missiles were fired outside of the aircraft/UAS constraints. This was likely 

caused by the lack of extensive training and experience with the UAS and the need for improved 

cueing symbology to help pilots understand where their aircraft and the UAS are located in 

reference to the targets. 

3.3 SME SA Ratings 

The SMEs provided an independent assessment of aircrew SA based on the scale shown in table 

5. The mean SMEs SA rating for aircrews was 2.40, indicating that the SMEs perceived that the 

aircrews typically had adequate levels of SA with minor or insignificant variation between 

perception and reality. The SMEs noted that there were several instances when the aircraft flew 

near (or over) the target(s) and onboard missiles were fired outside of the aircraft/UAS 

constraints. 

Table 5. SME SA rating. 

SME SA Ratings 

1 Crew was consistently aware of all entities on the battlefield. 

2 Crew was aware of the battlefield with minor or insignificant 

variation between perception and reality. 

3 Crew was aware of the battlefield. Variation between reality 

and perception did not significantly impact mission success. 

4 SA needs improvement. Lack of SA had some negative effect 

on the success of the mission. 

5 Lack of SA caused mission failure. 

3.4 SME Ratings of Mission Success and Mission Objectives 

At the end of each mission, SMEs rated whether the mission was a success or failure. The 

criteria that the SMEs used to rate mission success or failure was whether the aircrew completed 

most or all of their mission objectives and did not get shot down or crash. The SMEs rated all 10 

missions as “successful” (figure 14). They also rated whether the aircrew completed their 

mission objectives. The mission objectives were given to the pilots during the pre-mission brief. 

The SMEs believed that the aircrews completed their objectives during 90% of the missions. 

One SME rated two aircrews as not meeting all of their objectives during two missions. 

Mean Rating

2.40 

(SD = 0.99)
21 

100%

0%

90%

10%

0%

20%

40%

60%

80%

100%

Successful

Missions

Failed

Missions

Objectives

Completed

Objectives

Not

Completed

SME Ratings of Mission Success & Mission Objectives

Figure 14. SME ratings of mission success and mission objectives. 

3.5 UAS Crewstation Interface 

The CPGs were generally favorable in their ratings of the UCI. They reported that they were 

able to quickly navigate through the UAS menu pages on the right MPD, quickly use the 

switches on the TEDAC grips and buttons on the right MPD to control the UAS, felt that the 

overall design of the UCI did not significantly hinder them from controlling the UAS, and 

reported that the UCI did not contribute to high workload. The CPGs also reported that they 

“Never” (69%) or “Occasionally” (29%) forgot the steps required for navigating through the 

UAS display pages on the right MPD and experienced some problems switching between the 

operation of the MTADS/PNVS and UAS sensor due (mostly) to lack of experience. However, 

there were several improvements that the pilots recommended be made to the UCI to increase 

usability and decrease workload and the time required to complete tasks. These 

recommendations are listed in appendix D. 

3.6 Switch Actuations 

Switch actuations were recorded to help determine whether there were CPG tasks (e.g., UAS 

menu navigation) that need to be streamlined because of excessive switch actuations. The CPGs 

made an average of 609 switch actuations per mission (appendix F). This equates to 

approximately 7–8 switch actuations per minute or one switch actuation per 8 s for each mission. 

The switch actuations were often clumped together within specific time intervals. The switches 

that were actuated most often by the CPG were the MTADS Field of View (FOV) select switch, 

right hand grip slave select switch, left hand grip tracker switch, left hand grip cursor up/down 

switch, right MPD bezel buttons, and right hand grip laser trigger. The pilots reported that they 

did not think that the number of switch actuations per mission was excessive. Many of the 

switch actuations were momentary actuations of the MTADS FOV select switch and right hand 

grip slave select switch. 
22 

Table 6 summarizes the number of MPD page changes per mission. The average number of 

right and left MPD page changes was 73 for the CPG and 69 for the PI per mission. The TSD 

and TCDL pages were the pages most often displayed by the CPG and PI during missions. 

Table 6. MPD page changes during missions. 

 Number of Page Changes Per Mission 

Mission CPG 

(Right MPD) 

CPG 

(Left MPD) 

PI 

(Right MPD) 

PI 

(Left MPD) 

1 24 55 28 69 

2 16 36 11 34 

3 50 42 8 26 

4 21 54 75 35 

5 52 54 13 29 

6 12 44 10 39 

7 12 52 41 58 

8 17 45 17 18 

9 11 22 25 71 

10 36 76 47 32 

 

Avg 25.1 48 27.5 41.1 

Std. Dev 15.53 14.15 21.36 18.38 

Min 11 22 8 18 

Max 52 76 75 71 

3.7 Top Improvements Recommended by Pilots 

The pilots recommended that several improvements be made to the UCI and UAS to increase 

usability and decrease workload and the time required to complete tasks. Following are the most 

significant improvements that the pilots recommended be made to improve UAS employment 

(examples for each improvement are listed in appendix E): 

• Improvements to UAS Cueing Functions/Symbology 

• Improvements to Display Menu Pages 

• Need Linear Motion Control (LMC) for UAS 

• Incorporate UAS Operator into Pre-Mission Planning 

• Aviation Mission Planning System (AMPS) Integration of UAS Functionality 

3.8 Simulator Sickness 

Pilots reported that they typically experienced very mild to mild simulator sickness symptoms 

during the evaluation. The overall mean Total Severity (TS) score (post mission) for the pilots 

was 8.51 (table 7). The mean TS score for the CPGs was 9.72 and the mean TS score for the PIs 
23 

was 7.01. The difference between the TS scores for the CPG vs. the PIs was not statistically 

significant (WSRT, Z = –.210, p = .875). Based on the categorization of simulator sickness 

symptoms proposed by Kennedy et al (2002) (table 8), the pilots experienced “minimal” 

simulator sickness symptoms during the missions. 

Table 7. SSQ ratings. 

Condition Nausea 

Subscale 

Oculomotor 

Subscale 

Disorientation 

Subscale 

Total Severity 

Score (Mean) 

Standard 

Deviation 

Pre-Mission (CPG) .095 4.54 2.78 3.36 6.22 

Post-Mission (CPG) 8.58 9.09 6.96 9.72 15.88 

 

Pre-Mission (PI) 1.19 0.94 3.48 1.87 2.82 

Post-Mission (PI) 9.54 5.68 1.74 7.01 11.28 

 

Pre-Mission CPG and PI 1.06 2.94 3.09 2.70 4.93 

Post-Mission CPG and PI 9.01 7.58 4.64 8.51 13.98 

Table 8. Categorization of simulator sickness symptoms. 

SSQ Total Score Categorization 

0 No symptoms 

< 5 Negligible symptoms 

5–10 Minimal symptoms

10–15 Significant symptoms 

15–20 Symptoms are a concern 

> 20 A problem simulator 

Note: Categorization of symptoms based on central tendency (mean or median) 

using military aviation personnel in each simulator (Kennedy, 2002). 

3.9 Comparison of RACRS Simulator SSQ Scores to Other Helicopter Simulators 

To assess whether the SSQ ratings provided by the pilots during the assessment were similar or 

different to ratings obtained in other helicopter simulators, the mean SSQ scores for the RACRS 

simulator were compared to the mean SSQ scores for several other helicopter simulators (table 

9). The other helicopter simulators were the AH-64A (Army Research Institute, non-motion 

simulator), ARH-70, S-3H, CH-46E, CH-56D, CH-56F, Sikorsky RAH-66 Engineering 

Development Simulator (EDS), RAH-66 Comanche Portable Cockpit (CPC), and the simulator 

used during the UH-60M for the Early User Demo (EUD) and Limited Early User Evaluation 

(LEUE). In comparison, the RACRS induced fewer simulator sickness symptoms than most of 

the helicopter simulators listed in table 9. 
24 

Table 9. Comparison of RACRS simulator SSQ ratings with other helicopter simulators. 

Simulator Nausea 

Subscale 

Oculomotor 

Subscale 

Disorientation 

Subscale 

Total Severity Score 

(Mean) 

AH-64Aa – – – 25.81 

ARH-70 18.02 21.48 9.28 20.15 

SH-3H 14.70 20.00 12.40 18.80 

RAH-66 EDS 11.84 14.98 4.54 13.25 

CH-53F 7.50 10.50 7.40 10.00 

RAH-66 CPC 3.29 12.94 7.89 9.80 

UH-60M (LEUE) 6.36 11.81 3.09 9.15 

RACRS 9.01 7.58 4.64 8.51 

UH-60M (EUD) 13.88 6.89 0 8.50 

CH-53D 7.20 7.20 4.00 7.50 

CH-46E 5.40 7.80 4.50 7.00 a

SSQ subscale data not available. 

Based on pilot SSQ ratings, observation by ARL HRED personnel during missions, feedback 

during post mission interviews, and comparison of SSQ ratings with ratings from other 

helicopter simulators, it is reasonable to assume that the simulator sickness symptoms the pilots 

experienced did not (1) cause them significant discomfort, (2) distract them during missions, or 

(3) contribute to an increase in perceived workload. 

4. Conclusion

4.1 Crew Workload 

Pilots reported that they typically experienced tolerable workload when performing missions 

while controlling the UAS. They reported that the workload they experienced was comparable 

to workload they experience during “non-UAS” missions (MTADS and/or FCR only). They 

commented that having to manage an additional sensor (UAS sensor) increased their overall task 

workload, but the SA provided by the UAS sensor decreased the workload required to detect and 

engage targets and decreased overall target engagement timelines. The SMEs reported that the 

pilots typically experienced tolerable workload when controlling the UAS during missions, but 

had reduced spare workload capacity. The workload ratings provided by the pilots and SMEs 

were lower than the Objective and Threshold workload ratings requirements listed in the AB3 

CDD (table 10). 

Table 10. Pilot workload requirements and ratings. 

CDD Bedford Workload 

Rating Requirements 

Pilot Bedford Workload 

Ratings 

SME Bedford Workload 

Ratings 

 Objective Req. – 5.0 

Threshold Req. – 6.0 

PI – 2.9 

CPG – 2.6 

PI – 4.3 

CPG – 4.9 
25 

4.2 Crew Situation Awareness 

Pilots typically experienced moderate levels of SA during missions. They reported that they had 

high levels of SA of most of the battlefield elements (e.g., threat location) during the missions. 

However, there were several instances when they flew near (or over) the targets and fired 

missiles outside of the aircraft/UAS constraints. This was likely caused by the lack of extensive 

training and experience with the UAS and the need for improved cueing symbology to help 

pilots understand where their aircraft and the UAS are located in reference to the targets. The 

pilots stated that they had higher SA during UAS missions (vs. non-UAS missions) mostly 

because of the “God’s Eye” view that the UAS sensor video provided during missions. The 

UAS sensor video also gave the pilots good SA earlier in the mission (vs. non-UAS missions) 

because they often received the video prior (or just after) take-off of their aircraft. The SMEs 

reported that the aircrews typically had adequate levels of SA.

4.3 Crew Coordination 

The majority of PIs reported that the required level of crew coordination was higher during UAS 

missions than comparable non-UAS missions. They commented that having to manage an extra 

sensor increased their workload and required them to interact more with the CPG. The CPGs 

were split on whether having to manage an extra sensor increased the required level of crew 

coordination. The SMEs rated aircrew coordination during most missions as “Good” or 

“Average.” 

4.4 UAS-Crewstation Interface 

The CPGs were generally favorable in their ratings of the UCI. They reported that they were 

able to quickly navigate through the UAS menu pages on the right MPD, quickly use the 

switches on the TEDAC grips and buttons on the right MPD to control the UAS, commented that 

the number of switch actuations per mission was not excessive, felt that the overall design of the 

UCI did not significantly hinder them from controlling the UAS, and reported that the UCI did 

not contribute to high workload. 

The pilots recommended that the following improvements should be made to enhance UAS 

employment: 

• Improvements to UAS Cueing Functions and Symbology 

• Improvements to Display Menu Pages 

• Need LMC for UAS 

• Incorporate UAS Operator into Pre-Mission Planning 

• AMPS Integration of UAS Functionality 
26 

4.5 Simulator Sickness 

Pilots reported that they typically experienced very mild to mild simulator sickness symptoms 

during the evaluation. The simulator sickness symptoms the pilots experienced did not (1) cause 

them significant discomfort, (2) distract them during missions, or (3) contribute to an increase in 

perceived workload.

5. Recommendations

The following recommendations are made to enhance the overall effectiveness and suitability of 

the UAS integration into the AH-64D: 

• Address and incorporate the recommended improvements (e.g., enhanced cueing) provided 

by the pilots. 

• Use the Crew Station Working Group to address and incorporate the recommended 

improvements. 

• Upgrade the RACRS simulator to make it representative of the AH-64D Block 3 design to 

enhance future simulations. 

• Maximize the amount of pilot training for future UAS evaluations. 

• Use the same data collection methodology (e.g., Bedford, SART) during future simulations 

and tests for Apache Longbow Block 3. Standardizing the data collection methodology 

will help identify changes that work (e.g., changes that reduce workload), identify areas 

that still need more work, amd help drive continuous incremental improvements. 
27 

6. References 

Crowley, J. S. Simulator sickness: A problem for army aviation. Aviation Space and 

Environmental Medicine 1987, 58, 355–357. 

Endsley, M. R. Situation awareness analysis and measurement; Mahwah, NJ: Lawrence 

Erlbaum Associates, 2000. 

Endsley, M. R. Design and evaluation for situation awareness enhancement. Proceedings of the 

Human Factors Society 32nd Annual Meeting, Vol. 1, 92–101, 1988.

Kennedy, R. S.; Drexler, J. M.; Compton, D. E.; Stanney, K. M.; Lanham, D. S.; Harm, D. L. 

Configural scoring of simulator sickness, cybersickness, and space adaptation syndrome: 

Similarities and differences. In Virtual and adaptive environments: Applications, 

Implications, and Human Performance, L. J. Hettinger, & M. W. Haas, eds.; Erlbaum, 

Mahwah, NJ, 2002, pp 247-278. 

Kennedy, R. S.; Lane, N. E.; Berbaum, K. S.; Lilienthal, M. G. Simulator sickness 

questionnaire: An enhanced method for quantifying simulator sickness. International 

Journal of Aviation Psychology 1993, 3, 203–220. 

Kennedy, R. S.; Lilienthal, M.;G.; Berbaum, B. A.; Balzley, B. A.; McCauley, M. E. Simulator 

sickness in U.S. navy flight simulators. Aviation Space and Environmental Medicine 1989, 

60, 10–16. 

O’Brien, T. G.; Charlton, S. G. Handbook of Human Factors Testing and Evaluation; Lawrence 

Erlbaum Associates, Mahweh, NJ, 1996. 

Roscoe, A. H. The airline pilots view of flight deck workload: A preliminary study using a 

questionnaire; Technical Memorandum No. FS (B) 465, ADA116314; Royal Aircraft 

Establishment: Bedford, UK, 1985. 

Roscoe, A. H.; Ellis, G. A. A Subjective Rating Scale For Assessing Pilot Workload In Flight: A 

Decade Of Practical Use, Royal Aerospace Establishment, Bedford, UK, 1990. 

Taylor, R. M. Situational awareness rating technique (SART): The development of a tool for 

aircrew systems design. Proceedings of the NATO Advisory Group on Aerospace Research 

and Development (AGARD) Symposium on Situational Awareness in Aerospace Operations, 

AGARD-CP-478, 3/1 - 3/17, NATO – AGARD: Neuilly Sur Seine, France:, 1989.
28 

INTENTIONALLY LEFT BLANK. 
29 

Appendix A. Bedford Workload Rating Scale Scores and Pilot Comments 

 Workload Description “Rating”

 

Pilot Decisions 

 

 Was it possible to 

 complete the task? 

Was workload tolerable 

for the task?

Was workload 

satisfactory 

without reduction in 

spare (workload) capacity?

NO 

NO

NO 

Workload insignificant 1 

Workload low 

Enough spare capacity for all 

desirable additional tasks 

 2 

 3

Insufficient spare capacity for easy 

attention to additional tasks 

Reduced spare capacity. Additional 

tasks cannot be given the desired 

amount of attention 

Little spare capacity: level of effort 

allows little attention to additional 

tasks 

YES

YES

YES

 4 

 

 5 

 6 

Very little spare capacity, but 

maintenance of effort in the primary 

tasks not in question 

Extremely high workload. No spare 

capacity. Serious doubts as to ability 

to maintain level of effort 

Very high workload with almost no 

spare capacity. Difficulty in 

maintaining level of effort 

 

 7 

 

 8 

 9 

Task abandoned. Pilot unable to 

apply sufficient effort 

 

10 

Figure A-1. The BWRS. 
30 

Table A-1. Mean Workload Ratings for UAS missions and comparable non-UAS missions. 

Flight and Mission Tasks Mean Workload Ratings for 

UAS Missions 

Mean Workload Ratings for 

Comparable Non-UAS 

Missions 

 CPG Pilot CPG Pilot 

Observing Named Areas of Interest 2 2 3 3 

Target Detection 2 3 4 3 

Target Acquisition 3 3 4 3 

Target Engagement 3 3 4 3 

Movement To Contact 3 3 3 3 

Actions On Contact 4 4 4 3 

Battle Damage Assessment 

And Reporting 2 2 3 3 

Mission Change 3 2 3 3 

Battle Handover 2 3 2 3 

Tactical Navigation (Contour/NOE) 2 3 2 2 

Communications (Tactical Ops Center, Wingman) 2 3 2 2 

NOE/Contour/Low Level Flight 2 3 2 2 

Maintain Airspace Surveillance 3 3 3 3 

VMC Flight Maneuvers 2 2 2 2 

Electronically Aided Navigation 2 2 2 2 

Terrain Flight Navigation 2 3 2 2 

Evasive Maneuvers 3 5 4 4 

MTADS/PNVS Operations 3 2 3 2 

Route Recon 2 3 3 3 

Area Recon 2 3 3 3 

Level 2 UAS Control 2 3 – – 

Level 3 UAS Control 3 3 – – 

Level 4 UAS Control 3 2 – – 

Data Entry Procedures 2 3 2 3 

Engage with Hellfire 3 4 3 4 

Engage with 30mm AWS 4 2 4 2 

Multi-ship Operations 3 3 3 3 

Transmit Tactical Reports 3 3 2 3 

Identify Major U.S./Allied and Threat Equipment 3 3 4 3 

Information Management In The Front Seat (CPG) 3 – 3 – 

Information Management In The Back Seat (PI) – 4 – 3 

Air Escort Mission (overall) 3 3 3 3 

Security Mission (overall) 3 3 4 3 

Overall Workload Ratings 

Avg: 2.6 

Median: 3 

Std. Dev.: 0.6 

Avg: 2.9 

Median: 3 

Std. Dev.: 0.6

Avg: 3 

Median: 3 

Std. Dev.: 0.7 

Avg: 2.8 

Median: 3 

Std. Dev.: 0.5
31 

Pilot Workload Comments 

If you gave a workload rating of ‘5’ or higher for any task in the UAS mission column, explain 

why the workload was high for the task. 

Explanations as to why the pilots assigned a rating of ‘5’ or higher for any task in the UAS 

column. 

Evasive Maneuvers: 

• Task intensive, there is no way to add additional tasks and still be capable of performing or 

reacting to evasive maneuvers. 

• EM too task intensive to also be using UAS. Once completed or out of danger area and 

continuing to engage it becomes much easier. 

• EM caused CPG to totally concentrate on UAV while physiological functions and visual 

cues in the cockpit caused visual rivalries. 

Actions on Contact: 

• Task intensive, there is no way to add additional tasks and still be capable of performing or 

reacting to actions on contact. 

• All concentration goes to maintaining LOS centered on target and communicating. 

• AoC too task intensive to also be using UAS. Once completed or out of danger area and 

continuing to engage it becomes much easier. 

• Weapon engagements are always intensive. 

• Difficulty maintaining SA with wingman/UAV in relation to self when engaging TGTs. 

Target Detection/Acquisition/Engagement: 

• All concentration goes to maintaining LOS centered on target and communicating. 

• Concentrated effort of controlling the TFC (Thumb Force Control) on OBJ/TGT to 

maintain LOS during Hellfire engagement. 

• Workload is lower for detection/identification/acquisition and engagement using the UAS. 

Crew can create a 3-D picture of the battle space vs. the 2-D (TADS) picture. 

• CPG remains focused on TGT with UAV unable to perform other tasks until engagement is 

complete. 

• TGT detection and Acquisition are easier with UAS; TGT engagement has a higher 

workload due to hellfire designation basket. 
32 

Hellfire/Rockets/30mm: 

• All concentration goes to maintaining LOS centered on target and communicating. 

• Increased workload due to remote hellfire basket. 

Movement to Contact: 

• Weapon engagements are always intensive. 

MTADS/PNVS Operations: 

• Note: Full MTADS (Multi-target tracking) not available. MTADS layout when controlling 

UAS. TFC directly located under the UAS FOV selector. Not ergonomically designed 

causing higher workload during TGT engagement (Target Tracking) using the TFC. 

Multiship Operations: 

• Unable to concentrate on wingman’s location. Relied on PI for SA. 

Information Management: 

• Still feels like the CPG has a large workload that can be managed with an opposite crew 

member. 

Mission Change: 

• MSN changes are easier with UAS than with TADS. 

• Increased workload due to decision-making process on how to maximize employment of 

UAS. 

Other Comments: 

• Processing the additional information the UAS provided, there was a heightened sense of 

urgency because this mission involved friendly forces on the objective. 

• Difficult processing information from extra sensor. 

• With legacy target ID it’s (mission)?? darn near impossible. 

List any flight and/or mission tasks that you had to ask your crewmember to accomplish 

because your workload was too high.

PI Comments: 

Other Tasks: 

• Workload shared by crew based on task(s). 
33 

CPG Comments: 

Flight Maneuvers/Airspace Surveillance: 

• There were none he “had” to do, but the PC aided in many tasks. 

• All flt maneuvers and 90% airspace surveillance accomplished by PC.

Communication Tasks: 

• Some communication (radio calls). 

• I could have done it, but knew PC had less to do and could offer assistance. 

• Change freq, change direct, de-was missile. 

• Division of communication duties. 

Tactics: 

• Tactics (TMPR): we as a crew discussed this plan prior to initiating the maneuvers. I relied 

on the PI to keep my ACFT safe from wingman in relation to UAV. 

Target handover/transfer: 

• Briefed with PI, that he would transfer/handover TGTs I stored to wingman or ARTY to 

relieve the workload on CPG. This did assist, but I felt I had enough iterations in this 

profile/mission that I could have assisted with this task. 

Hellfire/Rockets/30mm: 

• 2-Hellfire missile shots. Busy assessing BDA. 

Were you ‘inside’ the cockpit more, less, or about the same compared to a non-UAS mission? 

PI Comments: 

• Video from UAV and ensuring LOS of laser is in constraints to fire a hellfire requires a 

little bit more inside time, but not enough to adversely affect the ability of the pilot to 

remain in control of the aircraft. 

• I disciplined myself outside the acft due to the CPG increased time inside. 

• Helping front seat line up targets, acquire targets draws you inside similar to TADS shots 

when back seat is watching tads video on MPD. 

• Only inside more when CPG was loosing SA. After SA was reestablished I was back 

outside. 

• Processing UAS information. 
34 

• Heads out more today based on MSN. Some head-in time was trying to figure out what was 

wrong with HMD symbology-RACRS (sim) issues. 

• More inside to correlate UAS data with other sensors. 

CPG Comments: 

• No HDU. If I had UAV video in HDU would split more attention outside. 

• A little more along the route recon with the mountains. I would have slaved TADS to my 

helmet and reconned by eye. The UAS allowed me to recon using it instead. Better recon, 

but inside more. 

• Obviously inside more because of the third sight. However, after my 4th flt. Using the 

system, this is becoming much easier and more familiar. With a little time, this system will 

be integrated into an Apache mission without any ill effects. 

• No HDU feed, so all CPG msn tasks involved sights viewed on MPD. 

• Taking advantage of greater stand-off distance to accomplish task of DIL (Detecting, 

Identifying, and Locating)/ Acquiring targets using sights and sensors. 

• Info management: CPG task saturated. 

• Level 3 control coupled with the security mission required CPG to have eyes on for the 

duration of the mission. 

• Tracking Targets. 

Was the required level of crew coordination higher, lower or about the same as a comparable 

non-UAS mission? 

PI Comments: 

• Inside cockpit a little more and coordinating our runs on target require more talking 

between crew members in order to ensure first time engagement success.

• Increased workload for CPG drives pilot to ensure CPG is not task overloaded. This is done 

by offering assistance. 

• Drawn inside, so more cross talk between crews, but SA was highly increased on objective 

area so it seemed well worth it. 

• Back up CPG when his workload increases. 

• Extra sight, more information, weapons engagements are more complex with UAS as the 

sight. 
35 

• Additional sight adds to workload and crew coordination – comparable to FCR. However, 

SA is much greater with UAS than without and somewhat better than FCR. 

• As with previous missions, weapons engagements were more difficult due to FS controlling 

other sensor.

CPG Comments: 

• Higher because there were more tasks to complete. Yet had more time to complete tasks 

due to increased ranges and SA. 

• Better SA in both seats equals less required communication, other than simple 

acknowledgement of same SA. 

• More steps and tasks to complete, yet more time and safer distance to do them. 

• Better SA for both seats = less workload. 

• Lower because of a good crew briefing to include UAS ground operator. 

• Keeping SA. 

• Keeping weapons MGMT/SA between wingman and UAV. 

• We are still having to put the AH64 wpns in constraints with UAS. Coordination is a must. 

• Change of plan/mission – Aircraft positioning for proper target engagements.

In the mission you just flew, list any flight and/or mission tasks that you had to ask your 

crewmember to accomplish because your workload was too high: 

• There were none he “had” to do, but the PI aided in many tasks. 

• All flt maneuvers and 90% airspace surveillance accomplished by PI. 

• I could have done it, but knew PI had less to do and could offer assistance. 

• Change freq, change direct, de-was missile. 

• Some communication (Radio Calls) 

• Tactics (TMPR): we as a crew discussed this plan prior to initiating the maneuvers. I relied 

on the PI to keep my ACFT safe from wingman in relation to UAV. 

• None 

• Workload shared by crew based on task(s). 

• Briefed with PI, that he would transfer/handover TGTs I stored to wingman or ARTY to 

relieve the workload on CPG. This did assist, but I felt I had enough iterations in this 

profile/mission that I could have assisted with this task. 
36 

• Division of commo duties. 

• 2-Hellfire missile shots. Busy assessing BDA. 

In the mission you just completed, were you visually ‘inside’ the cockpit more, less, or about 

the same compared to a comparable non-UAS mission (figure A-2))? 

80%

50%

0% 10%

20%

40%

0%

20%

40%

60%

80%

100%

Inside More Inside Less About The

Same

Visually Inside The Cockpit vs. Non-UAS Missions

CPG

PI

Figure A-2. Visually inside the cockpit vs. non-UAS missions. 

• Video from UAV and ensuring LOS of laser is in constraints to fire a hellfire requires a 

little bit more inside time, but not enough to adversely affect the ability of the pilot to 

remain in control of the aircraft. 

• As the backseater, I disciplined myself outside the acft due to the CPG increased time 

inside. 

• A little more along the route recon with the mountains. I would have slaved TADS to my 

helmet and reconned by eye. The UAS allowed me to recon using it instead. Better recon, 

but inside more. 

• Obviously inside more because of the third sight. However, after my 4th flt. Using the 

system, this is becoming much easier and more familiar. With a little time, this system will 

be integrated into an Apache mission without any ill effects. 

• Helping front seat line up targets, acquire targets draws you inside similar to TADS shots 

when back seat is watching tads video on MPD. 

• No HDU feed, so all CPG msn tasks involved sights viewed on MPD. 

• Only inside more when CPG was loosing SA. After SA was reestablished I was back 

outside. 
37 

• Taking advantage of greater stand-off distance to accomplish task of DIL (Detecting, 

Identifying, and Locating)/ Acquiring targets using sights and sensors. 

• Processing UAS information. 

• Info management: CPG task saturated. 

• Heads out more today based on MSN. Some head-in time was trying to figure out what was 

wrong with HMD symbology-RACRS (sim) issues. 

• Level 3 control coupled with the security mission required CPG to have eyes on for the 

duration of the mission. 

• No HDU. If I had UAV video in HDU would split more attention outside. 

• Tracking Targets. 

• More inside to correlate UAS data with other sensors. 

Was the required level of crew coordination higher, lower, or about the same as a comparable 

non-UAS mission? 

Much Higher level of crew coordination required – Front seat – 10% Back seat – 20%

Somewhat Higher level of crew coordination required – Front seat – 40% Back seat – 60%

About the Same level of crew coordination required – Front seat – 20% Back seat – 20%

Somewhat Lower level of crew coordination required – Front seat – 30% Back seat – 0%

Much Lower level of crew coordination required – Front seat – 0% Back seat – 0%

Pilot comments: 

• Inside cockpit a little more and coordinating our runs on target require more talking 

between crew members in order to ensure first time engagement success. 

• Higher because there were more tasks to complete. Yet had more time to complete tasks 

due to increased ranges and SA. 

• Increased workload for CPG drives pilot to ensure CPG is not task overloaded. This is done 

by offering assistance. 

• Better SA in both seats equals less required communication, other than simple 

acknowledgement of same SA. 

• Almost same reason as 8. Drawn inside, so more cross talk between crews, but SA was 

highly increased on objective area so it seemed well worth it. 
38 

• More steps and tasks to complete, yet more time and safer distance to do them. 

• Back up CPG when his workload increases. 

• Better SA for both seats = less workload. 

• Lower because of a good crew briefing to include UAS ground operator. 

• Extra sight, more information, weapons engagements are more complex with UAS as the 

sight. 

• Keeping SA. 

• Keeping weapons MGMT/SA between wingman and UAV. 

• Additional sight adds to workload and crew coordination – comparable to FCR. However, 

SA is much greater with UAS than without and somewhat better than FCR. 

• We are still having to put the AH-64 wpns in constraints with UAS. Coordination is a 

must. 

• Change of plan/mission – Aircraft positioning for proper target engagements. 

• As with previous missions, weapons engagements were more difficult due to FS controlling 

other sensor. 
39 

Appendix B. Subject Matter Expert Workload Comments 

If you assigned a workload rating of ‘5’ or higher for either crewmember, explain why: 

• Front seater shed tasks to operate UAS. 

• Backseater – had good situation awareness. However, backseat was looking inside at the 

UAS video display during level 3&4 control to help the front seater find the targets. 

• The front seat is 90% or better inside during the mission. 

• Crew continuously flies within 1KM of the target area. 

• Requested level 4, never took control. 

• During pick up system initiation. 

• CPG – did not call BN Ops back during run-up for operation update. 

• CPG – launched two missiles at tgt3  Planned rapid but sim poor video feedback. 

• CPG – Asked for LVL 4, which was approved, but CPG did not take LVL 4 – remained 

LVL 3. 

• Did crew decide to remain LVL 3 – No – CPG perceived he was in LVL 4 guide mode. 

• Understanding what the UAV was doing under Level 4 control. Guide made us PROG 

mode – caught it after first missile. Reprogrammed a plan and then CPG selected PROG. 

• Crew did not know Level 4 was in guide mode, not plan mode. 

• WL-low along route to and from objective (good). Changed LVL 3 to LVL 2 to reduce 

workload  shifted route recon to UAS operator during MSN change – allowed CPG to 

setup next MSN. Remained LVL 3 control for most of the mission. CPG initially checked 

WPN page prior to engagement – did not check WPN page as mission progressed  CPG 

decided this check was not required prior to shot  Omitted task during last half of MSN. 

• Front seat shed engagement tasks of storing TGTS and had difficulty re-acquiring targets. 

• WAS/De-WAS weapons 

• LOAL LO-DIR confusion (little) 

• FXD ACQ – and driving the missile constraints bey IAT FOV TGT drives the sight when 

beyond the FOV 
40 

• Storing TGT not practiced – if done, this would have reduced workload of CPG having to 

re-acquire. 

• One sim- problem increased WL when CPG tried to drop track the UAS FOV went to 

zoom causing the CPG to lose the TGT. 

• Hellfire remote excusum and designator zones exceeded 

• Crew tactic (launch UAV early) gave more time to work the targets and contributed to less 

stress. 

• Crew members had difficulty determining their position with the position of the UAS with 

the targets. 

• PLT loitered over the targets. Workload seemed low, but crew had an SA breakdown. 

• Aircrew lost SA and flew over targets and got shot at. Crew did not maintain standoff. 

• When aircrew had level 3 and 4 crew let sensor drift with no control. Crew did not use 

UAS ground controller to search for and designate targets. Crew shed the task of 

controlling UAS when they had level 3 or 4. 

• PLT performed HF MSL engagements and flying ownship entered MBT WEZ several 

times while maneuvering. 

SME Aircrew Coordination Comments: 

• Crew coordination continues to improve from previous missions. 

• The pilot (back seat) in this crew worked mission priorities and kept the CPG directed. 

• Crew briefed engagement plan – first TGT LVL2 control – TGT destroyed. 

• What were you using to line up MSL constraints? TGT as an ACQ source. 

• Real good job working the cockpits. 

• Very good division of Com duties. 

• Excellent division of duties between MUM team. 

• Very good SA in both crewstations. 

• Excellent crew coordination and division of duties. Assigned tasks to UAS operator when 

appropriate. 

• Aircrews fired multiple missiles outside the missile constraints – payload tracking – 

aircrew had difficulty tracking targets with UAS payload. 

• PL enabled CPG to have a lower workload by performing tasks. 
41 

Describe any problems that aircrews had with situation awareness: 

• TADS Operations were shed to operate UAS. 

• Did not know if they had level 4 control or not. 

• CPG fired second MSL with TADS FXD FWD – did you use UAS distance and HOG for 

MSL constraints used TGT as an ACQ source. 

• Crew closed to under 1KM during gun engagement. Crew discussion to pursue target. 

• Circled closer to EA-1-3KM from UAV LOS. 

• Crew awareness of UAV location 64 – TGT – UAS location. 

• Crew lost SA on UAV location during mission during missile engagements. Missiles 6, 7, 

and 8 were all outside the laser offset performance. 

• Crew fired 3 missiles outside the 60deg gun target line. 

• Front seat had problems knowing whether or not he had lased a weapon during missile 

engagements. 

• Using the UAS acq should have been used to maintain TGT position – lase and store. 

• Maintained excellent SA on the battle and other elements. 

• Crew was prompted to hurry in when AA force holding. 

• Crew prompted during AWS running foe by wingman to hurry up. 

• Could not properly determine there position and the position of the UAS prior to missile 

engagements. 

• HF missile engagement with AATF further downrange along GTL. 

• Crew members had difficulty tracking with UAS payload and the crew had difficulty 

determining the location in regards to the UAS. 

• Loitered over the targets. 

• Crew over flew enemy targets. Did not maintain standoff. 

• Aircrew flew within 1km of targets. 

Did the aircrew complete their mission objectives? 

If no, why weren’t the mission objectives completed? 

• Enemy forces still alive after AA inserted delayed AAGF movement to objective. 

• Crew was engaged by enemy at known location. 
42 

INTENTIONALLY LEFT BLANK. 
43 

Appendix C. Situation Awareness Ratings and Comments 

C-1. CPG 

‘Demand’ During Missions 

Low 1-------3-------5-------7-------9-------11-------13-------15-------17-------19-------21 High 

‘Supply’ During Missions 

Low 1-------4--------7--------10--------13--------16--------19--------21--------24-------28 High 

‘Understanding’ During Missions 

Low 1-------3-------5-------7-------9-------11-------13-------15-------17-------19-------21 High 

SART SCORE: 24.0 

13.40

20.50

16.90
44 

C-2. PI 

‘Demand’ During Missions 

Low 1-------3-------5-------7-------9-------11-------13-------15-------17-------19-------21 High 

‘Supply’ During Missions 

Low 1-------4--------7--------10--------13--------16--------19--------21--------24-------28 High 

‘Understanding’ During Missions 

Low 1-------3-------5-------7-------9-------11-------13-------15-------17-------19-------21 High 

SART SCORE: 25.7 

13.70

21.80

17.60
45 

Table C-1. SA of battlefield elements during missions. 

Battlefield 

Elements 

Very High 

Level of 

Situation 

Awareness 

Fairly High 

Level of 

Situation 

Awareness 

Intermediate 

Level of 

Situation 

Awareness 

Fairly Low 

Level of 

Situation 

Awareness 

Very Low 

Level of 

Situation 

Awareness 

PI CPG PI CPG PI CPG PI CPG PI CPG 

Location of 

Enemy Units 70% 80 % 30% 10% 0% 10% 0% 0% 0% 0% 

Location of 

Friendly Units 30% 70 % 60% 10% 10% 20% 0% 0% 0% 0% 

Location of My 

Aircraft During 

Missions 

90% 80% 0% 20% 10% 0% 0% 0% 0% 0% 

Location of 

Other Aircraft 

In My Flight 

30% 10% 40% 40% 30% 30% 0% 20% 0% 0% 

Location of 

Cultural 

Features (e.g., 

bridges) 

50% 30% 40% 40% 10% 10% 0% 20% 0% 0% 

Route 

Information 

(ACPs, BPs, 

EAs, RPs, etc.) 

90% 80% 10% 20% 0% 0% 0% 0% 0% 0% 

Status of My 

Aircraft Systems 

(e.g., fuel 

consumption) 

70% 0 % 20% 20% 10% 60% 0% 10% 0% 10% 

Note: BP = Battle position, EA = Engagement area, ACP = Air control point, and RP = Release point. 
46 

C-3 Aircrew Situational Awareness Comments 

Describe any instances when you felt you had low situational awareness during the mission: 

• Little airspace surveillance, busy mission not a lot of attention to acft systems status other 

than sights weapons and radios. 

• Only because of probable computer sim limitations. Non-combatants and friendly forces 

not in a vehicle or building made it hard to determine where exactly they were at. 

• Not sure where our flight escorts were and I had very little aircraft info because I did not 

have my HDU. 

• None other than possible convoy freq and location at end. 

• Wasn’t monitoring acft sys or wingman much during intensive engagements. 

• Difficulty finding last vehicle due to simulator issue. *** COMMENT*** Not familiar 

with Lot 4 load. 

• Searching for last target vehicle. 

• During relief-on-station due to lack of exercising the task. 

• No view on non-combatants. 

• CPG was too involved with mission, Required PI to update me on wingman location. 

• Cultural features: I did not bother to make note of them. I was too focused on mission to 

care! 

• *No digital capability in BFT in simulator. 

• I did not hear the UH A/C depart the holding area for the LZ. Not untypical for the Lift 

folks MGT to make that call regardless of a UAS. They will come in based on their fuel or 

the ground CMDR in the A/C ordering them to do so. 

• *Simulation limitation, no present pos report and BFT. 

• Simulation failure of high action display caused lower SA until pilot was able to ID 

problem and find correct info in other sources (TSD). 

• Overall: SA was much higher with UAS than without in this msn. Typically msn changes 

are the hardest task that a crew/team/company must perform. With the UAS, team was able 

to use standoff and develop the situation quicker than without the UAS. 

• Determining the BDA of enemy vehicles destroyed and its position/location. 
47 

C-4 SME Situation Awareness Comments 

• TADS Operations were shed to operate UAS. 

• More SA was maintained. The crew used UAS efficiently to engage targets while 

maintaining SA. 

• Did not know if they had level 4 control or not. 

• CPG fired second MSL with TADS FXD FWD – did you use UAS distance and HOG for 

MSL constraints used TGT as an ACQ source. 

• Crew closed to under 1KM during gun engagement. Crew discuss to pursue target. 

• Circled closer to -1-3KM from UAV LOS. 

• Crew awareness of UAV location 64 – TGT – UAS location. 

• Crew lost SA on UAV location during mission during missile engagements. Missiles 6, 7, 

and 8 were all outside the laser offset performance. 

• Crew fired 3 missiles outside the 60deg gun target line. 

• Good overall SA. 

• Good. 

• Front seat had problems knowing whether or not he had lased a weapon during missile 

engagements. 

• Using the UAS acq should have been used to maintain TGT position – lase and store. 

• Maintained excellent SA on the battle and other elements. 

• Crew was prompted to hurry in when AA force holding. 

• Crew prompted during AWS running foe by wingman to hurry up. 

• Could not properly determine there position and the position of the UAS prior to missile 

engagements. 

• HF missile engagement with AATF further downrange along GTL. 

• Crew members had difficulty tracking with UAS payload and the crew had difficulty 

determining the location in regards to the UAS. 

• Loitered over the targets. 

• Crew over flew enemy targets. Did not maintain standoff. 

• Aircrew flew within 1km of targets. 
48 

INTENTIONALLY LEFT BLANK. 
49 

Appendix D. UAS-Crewstation Interface Ratings and Comments 

P1. How quickly were you able to navigate through the UAS PVI menu pages (below) to 

accomplish a task? 

Very 

Quickly 

Somewhat 

Quickly Borderline Somewhat Slowly 

Very 

Slowly 

UAS Flight Mode 

Options (HOLD, DEST, 

GUIDE, PROG) on 

TCDL Page 

100% 

UTIL 80% 20% 

PLAN 60% 20% 20% 

CODE 80% 20% 

SET 60% 40% 

TSD 60% 40% 

VIDEO 80% 20% 

P2. How often did you forget the steps required for navigating through the UAS PVI menu 

pages to accomplish a task? 

 Frequently Often Occasionally Never 

UAS Flight Mode 

Options (HOLD, DEST, 

GUIDE, PROG) on 

TCDL Page 

60% 40% 

UTIL 40% 60% 

PLAN 20% 40% 40% 

CODE 100% 

SET 40% 60% 

TSD 20% 80% 

VIDEO 100% 
50 

P3. How quickly were you able to use the switches on the grips and displays to control the 

UAS? 

Very 

Quickly 

Somewhat 

Quickly Borderline Somewhat Slowly 

Very 

Slowly 

UAS FOV 40% 60% 

Map Symbols 75% 25% 

FLIR Polarity 100% 

Sensor Slave 60% 40% 

Manual Tracker 40% 40% 20% 

Laser Trigger 80% 20% 

Sight Select 80% 20% 

If you answered “Somewhat Slowly”, “Very Slowly”, “Frequently”, or “Often” to any of the 

questions, explain why you had problems (e.g., “navigating through the UAS menu pages was a 

slow process due to having to page through too many display screens”). 

• Plan- was similar to RTE menu page and thought selecting “plan 2” was enough. Kept 

forgetting to press R4 on top page to select plan. 

• Manual tracker on system; rate should be similar to TADS. 

• Lack of exercise and training. 

P4. Would you be able to more quickly and easily control the UAS if the sensor image was 

displayed on the TEDAC vs. the right MPD? 

Yes - 20% No- 80% 

If yes, explain why it would be quicker/easier to control the UAS on the TEDAC: 

• Physiologically it would be easier. Head is not turned towards right MPD constantly. It 

would also be natural for the CPG to use the TEDAC.

P5. Did you experience any problems switching between the operation of the MTADS/PNVS 

and the UAS sensor? 

 Yes – 60% No – 40% 

If yes, explain the problems: 

• Just had to develop muscle memory. 

• Just a momentary freeze and lock up. CPG needs to double check (usually) to ensure the 

correct sensor needed was displayed. 

• Lack of exercise (muscle memory). 
51 

P6. Was there any UAS symbology or wording depicted on the displays that was difficult to 

quickly and easily understand? 

 Yes -40% No – 60% 

If yes, explain which symbology or wording was difficult to understand and why: 

• UAV field of regard is tough and unnecessary with TSD depiction. 

• The North (N) symbology should be a little brighter for SA. Occasionally, it was hard to 

break out from the video context. 

P7. On average, how often did the design of the UAS PCI significantly hinder you from quickly 

and easily controlling the UASs? (Circle one) 

 

 1 2 3 4 

 __________________________________________________________________ 

 Never – 20% Seldom – 80% Often Frequently 

If ‘Often or Frequently’, describe how the design of the UAS PCI significantly hindered you 

from quickly and easily controlling the UASs: 

• Position of thumb force controller and UAS (FCR) FOV Switch. Again, with exercise 

(TFC Practice) this did not pose a significant problem. 

P8. On average, how often did the design of the UAS PVI contribute to high workload when 

controlling the UASs? (Circle one) 

 

 1 2 3 4 

 __________________________________________________________________ 

 Never – 20% Seldom – 60% Often – 20% Frequently 

If ‘Often or Frequently’, describe how the design of the UAS PVI contributed to high workload 

during missions: 

• Almost too many holding options. 

• TFC sensitivity seemed too high in the RACR. 

P9. Did you feel that lack of linear motion compensation (LMC) for the UAS sensor increased 

the time required to detect, acquire and engage targets vs. having LMC for the UAS sensor? 

Significantly Increased Time – 60%

Somewhat Increased Time – 40% 

No Difference – 0%
52 

If you checked ‘Significantly or Somewhat Increased Time’, explain: 

• On long range 7000+ moving tgt was hard to track. Had to IAT terrain to have any fire 

control, which didn’t leave IAT for TGT. 

• It took longer to acquire multiple targets because it forced you to use IAT which limited 

your ability to pan to the next target. 

• LMC works great all the time for both TGT detection and engagement. I think it should 

have it. 

• Tracking and engaging the TGTS would be much easier with this capability. 

• With exercise/practice without LMC TFC is manageable. LMC will somewhat increase 

time due to stabilization/compensation of TFC. 

P10. List any other UAS PVI usability features that hindered your performance during the 

missions: 

• Can’t fly UAV to TGT. 

• Can’t slave to WP/CM 

• Lack of TGT data number when utilizing TGT store procedures. Required CPG to 

navigate there other MPD pages to verify if TGT was stored and which are. 

◦ CPG should be able to add points directly into the plan page of the UAS. 

◦ FLT profiles of the UAS (Racetrack, Figure 8 and Orbit) should be changed to lower 

values or configurable in the aircraft UAS UTIL page. The size of these profiles can 

limit the use of the UAS laser when engaging TGTs with missiles. Aircraft (AH64) ma 

need to match the UAV’s position in order to meet missile constraints. (Too much 

moving around to meet this requirement). 

• Lack of visual cues on TSD which indicate constraints for Hellfire missile shots. (Ownship 

 UAS tgt visual display). 

• Position of TFC and UAS FOV switch. 
53 

Appendix E. Top Improvements for UAS Integration 

Top Improvements Recommended By Pilots 

Improvements to UAS Cueing Functions/Symbology 

• When TADS LOS is slaved to UAV LOS and 30/60 safety/laser fans are within limits 

horizontal and vertical, LOS symbol should change to indicate within limits. SP or WP 

should know both alt, pos, range, and bearing and compute for crew. 

• Need symbology that allows 60deg “60-no” FQU for Hellfire 

• Slave UAV LOS to waypoint/control point 

• Allow UAV LOS to be slaved to any CM/WPT/HZ/TGT 

• More selectable UAS symbology. To aide with SA of UAS to our aircraft. 

• Allow all NAV points (WPT, HZ, CM) to be selectable for targeting the UAV sensor line 

of sight. AKA so I can have them for an acquisition source. 

• Have range info and heading of UAS on a TSD page, added to show page in attack. 

• Have range info and heading of UAS in HDU symbology. 

• UAS Laser line on TSD should be - - - - (dashed) when not in range ____ (solid line) when 

in range and white when designating. 

• When you delete an IAT gate, the LOS should stay in the field of view and at the same spot 

it is when the gates are deselected. 

• UAS sensor indications on the TSD page (ex |-o<) like the FCR footprint. 

• A dashed or solid route while in a plan. 

• Add a show function on the TSD attack page to indicate heading and laser range from UAS 

to LOS. 

• Laser range to change color when active lasing, while leaving auto green. Possibly 

matching Arm color. 

• TGT data # symbology displayed on the UAS. 

• Laser designation window (basket) on TSD. MSL constraints box should be broken until 

UAS and aircraft are in the basket. Likewise the laser basket should be white until all 

constraints are met and then go green. 
54 

• When slaving UAS to a TGT, WP, HZ, or CM that info should appear next to the point on 

the UAS video. Likewise, shot at (X) should appear. 

• WPNS symbology (WAS’d) on display. 

• When range goes from auto to laser, range should change color. 

• Inverse symbology for UAV. 

• Shot file should say UAS if LOS is UAS, like it does for the FLIR. 

• UAS interface should work just like TADS and FCR. CAQing a TGT or icon on the TSD 

page should automatically make that TGT/icon the UAS AQC source. 

Improvements to Display Menu Pages 

• Ability to “Freeze” the UAS page (to include video) during Level 2 

• Divide symbology on UAS page to have all information that pertains to UAS up top of 

display, information that pertains to own ship on bottom of display. 

• Allow declutter of UAS symbology. 

• When adding a waypoint, TGT, CM, be able to select GRID/Location by using the CAQ 

function. 

• Have UAV video have only LOS, Laser, Field of Regard, and HAD symbology. Just like 

the TADS for back seat. 

• When you store a target it should be hot key selectable as an acquisition source. 

• Add a fully functional COORD page to the TCDL page. 

• Be able to add WP/CM directly to plan page of the UAS using coordinate file or directly to 

plan page. 

• Options for missile codes. At least 2 selectable. 

• Add CORD access to UAS page at T-5 

• Allow for Show options on the UAS page to allow crew member to toggle on and off parts 

of the display. 

• LVL 3 / 4  Freeze/Replay function 

• Plan RTE building function enabling the cursor select option to input (WP, CM, HZ, TGT). 

Need Linear Motion Control (LMC) for UAS 

• LMC for UAV 
55 

• LMC for UAS 

• Needs LMC for UAV. 

• LMC for UAS Sensor 

• Need LMC 

• UAS Sensor package needs more stabilization and green/solid box for in-constraints. 

Move UAS FOV switch to the Left Hand Grip 

• UAS FOV switch should be moved to the TADS FOV (Left Hand Grip) from the FCR 

FOV switch. This allows CPG to maintain manual track with right hand while changed 

FOV’s with left. 

• Re-positioning of UAS FOV switch to the LHG. 

Incorporate UAS Operator into Pre-Mission Planning 

• Incorporate UAS crewmember in mission planning and briefs. 

• Incorporate UAS ground operator with all inbriefs, FLT. Planning, etc.. to ensure all 

available TM players are situationally aware as possible. That will lighten the load for the 

aircraft crewmembers. 

AMPS Integration 

• Ability to pre-plan multiple UAS systems in AMPS before launching our mission. 

• DTC/AMPS input for pre-mission planning. Routes, targets, etc... 

Other Improvements 

• Need the ability to send a UAS hellfire handover (UAS HO) to other aircraft. This UAS 

HO should constrain TGT, laser code UAS data. Once the A/C accepts the UAS HO, then 

the aircraft should automatically put the hellfire MSL in a remote mode. A display the laser 

basket (white/broken box for out of constraints and green/solid box for in-constraints.) 

• With sight select UAS, the TADS should go to fixed forward just like FCR and the link 

function should work just like FCR. 

• UAS hold pattern settings should allow for the lowest settings that each different UAS can 

perform. 

• Multiple IAT for UAS like MTADS. 

• Incorporate RFI with a UAS equipped aircraft. 

• Slew rates for UAS should match more close to TADS 

• allow UAV to fly to TGT. 
56 

INTENTIONALLY LEFT BLANK. 
57 

Appendix F. Switch Actuations 

Table F-1. Switch actuations per mission. 

Switch Description 1 2 3 4 5 6 7 8 9 10 Avg Std Dev Max

Left Hand Grip TAD FOV Select 56 24 58 60 84 119 278 24 268 58 102.9 93.72 278

Right Hand Grip Slave Select 51 13 75 76 100 82 274 34 232 66 100.3 84.79 274

Left Hand Grip Tracker Switch 60 12 20 138 74 127 86 160 106 158 94.1 53.17 160

Left Hand Grip Cursor Up/Down 0 0 39 2 74 2 1 107 252 201 67.8 92.13 252

CPG Left Bezel Buttons 57 25 46 58 62 66 61 38 39 72 52.4 14.78 72

Left Hand Grip Cursor Right/Left 0 0 19 2 42 1 1 116 181 138 50 68.63 181

CPG Right Bezel Buttons 46 31 77 35 78 40 27 24 29 53 44 19.75 78

Right Hand Grip Laser Trigger 34 34 42 31 36 43 48 66 47 48 42.9 10.28 66

Left Hand Grip Target Store 12 46 23 18 0 22 37 13 10 17 19.8 13.35 46

Left Hand Grip Weapon Select 15 12 24 14 20 3 28 8 18 4 14.6 8.21 28

UAV Field of View 8 10 0 10 24 25 6 2 6 40 13.1 12.58 40

TEDAC Gain Rocker Switch 1 0 0 0 2 11 2 0 0 1 1.7 3.37 11

TEDAC TADS Video Select 0 0 0 0 0 0 14 0 2 0 1.6 4.40 14

TEDAC Level Rocker Switch 1 0 0 0 2 6 0 0 0 2 1.1 1.91 6

TEDAC Display Brightness Rocker 0 0 2 1 1 1 1 0 0 1 0.7 0.67 2

Left Hand Grip LMC Select 3 0 4 0 0 0 0 0 0 0 0.7 1.49 4

TEDAC PNVS Video Select 0 0 0 0 0 0 6 0 0 0 0.6 1.90 6

TEDAC Display Contrast Rocker Switch 1 1 0 0 0 0 0 3 0 0 0.5 0.97 3

TEDAC Symbology Brightness Rocker 0 0 0 0 0 0 0 2 0 0 0.2 0.63 2

Left Hand Grip Scan Switch 0 0 0 0 0 0 2 0 0 0 0.2 0.63 2

TEDAC UAV Video Select 00000000 0 0 0 0 0

Left Hand Grip Sensor Select 00000000 0 0 0 0 0

Right Hand Grip Map Symbols Select 00000000 0 0 0 0 0

Right Hand Grip Sight Select 00000000 0 0 0 0 0

Right Hand Grip Enter for Cursor Control 00000000 0 0 0 0 0

Right Hand Grip Polarity Select 00000000 0 0 0 0 0

Sum 345 208 429 445 599 548 872 597 1190 859 609.2 291.16 1190

Number of Switch Actuations Per Mission
58 

Table F-2. Time and percentage of time that menu pages were displayed on MPDs. 

CPG 

(Left MPD) 

CPG 

(Right 

MPD) 

 PI 

(Left MPD) PI (Right MPD) 

Time (s) Title % Time (s) Title % Time (s) Title % Time (s) Title % 

28135.4 TSD 79% 29678.1 TCDL 83% 13306.6 TSD 37% 18158.1 TSD 51% 

2173.8 Wpn 6% 931.1 Wpn 3% 8780.3 TCDL 25% 12033.1 TCDL 34% 

1480.1 Coord 4% 840.8 TCDL Util 2% 3930.4 Video 11% 1673.4 Video 5% 

1479.8 Tre 4% 738.6 Com 2% 3878.8 Flt 11% 920.7 Point 3% 

848.9 Point 2% 715.7 TCDL Plan 2% 1967.7 Eng 6% 839.9 Eng 2% 

467.1 Eng 1% 709.1 Flt 2% 956.1 Wpn 3% 644.4 Tre 2% 

201.6 Rpt 1% 598.3 Menu 2% 584.1 TCDL Plan 2% 497.8 DTU 1% 

156.2 Map <1% 509.7 DTU 1% 520.7 Com 1% 235 Com 1% 

143 AC Util <1% 493.2 TSD 1% 467 Tre 1% 202.2 Coord 1% 

132.6 Shot <1% 114.3 Video <1% 366.9 Sight Boresight 1% 118.1 TCDL Util <1% 

113.4 Menu <1% 93.8 Tre <1% 322.1 Ctrlm 1% 82.7 TCDL Plan <1% 

108.9 Flt <1% 63.3 Freq <1% 126.4 WCA <1% 82.5 TSD Util <1% 

67.9 Ctrlm <1% 58.6 Xpndr <1% 109 Coord <1% 51.7 Sight Boresight <1% 

49.4 Show <1% 42.5 ASE <1% 64.2 Show <1% 43.5 Rpt <1% 

32.6 Code <1% 30.6 Map <1% 54.6 Wpn Util <1% 38.7 Show <1% 

25.9 TCDL Util <1% 22.8 Show <1% 46.3 AC Util <1% 22.6 Route Menu <1% 

23.5 TCDL <1% 21.5 Route Menu <1% 45.6 Perf <1% 19.9 Wpn <1% 

19.3 Route Menu <1% 16.9 Wpn Util <1% 36.5 TCDL Util <1% 16.3 Menu <1% 

17.2 Wpthz <1% 12.9 Code <1% 31.6 Menu <1% 15 Man <1% 

13.2 ADF <1% 11.8 Man <1% 27.7 Code <1% 14 ASE Util <1% 

12 TSD Util <1% 7.2 AC Util <1% 24 ASE <1% 13.1 Wpthz <1% 

10.5 ASE <1% 7 Coord <1% 22.6 TSD Util <1% 9.1 Net <1% 

9.6 Video <1% 6.9 WCA <1% 20.8 Route Menu <1% 6.4 Edit <1% 

5.4 Zn <1% 5.2 Fuel <1% 18.4 Fuel <1% 6.1 ASE <1% 

4.2 Fault <1% 5.2 Perf <1% 14 Sys <1% 2.8 Map <1% 

4 Wpn Util <1% 4.1 Sinc <1% 9.1 Map <1% 2.7 Zn <1% 

3.6 Chan <1% 4 Eng <1% 8.2 Edit <1% 2 DMS <1% 

3.5 WCA <1% 3.8 TSD Util <1% 7.7 Point <1% 1.1 Flt <1% 

3.5 Fuel <1% 3.2 HQ2 <1% 5.4 Net <1% 

2.9 DMS Util <1% 1.2 TCDL Code <1% 

2.3 DMS <1% 0.9 DMS <1% 
59 

Table F-3. Time of movement of switches per mission. 

 Time of Movement

Column Detailed Description 1 2 3 4 5 6 7 8 9 10 Avg Std Dev Max 

WpnTrig Weapon Trigger 9.3 14.1 23.4 8.8 10.2 3.4 9.9 2.4 2.1 7 9.06 6.3 23.4 

TFCAz Right Hand Grip Thumb 

Force Controller Azimuth 1387.7 545 901.7 1232.1 1255.3 813.2 966.5 732.9 959.7 982.6 977.67 256.1 1387.7

TFCEI Right Hand Grip Thumb 

Force Controller Elevation 1490.7 531.5 957.4 1344.7 1394 832.9 1022.3 747.7 1068.7 990.8 1038.07 301.5 1490.7
60 

INTENTIONALLY LEFT BLANK. 
61 

Acronyms 

AAR After Action Review 

AB3 AH-64D Block III 

ACP Air Control Point 

AMPS Aviation Mission Planning System 

AOI area of interest 

ARH Armed Reconnaissance Helicopter 

ARL U.S. Army Research Laboratory 

ASL Applied Science Laboratories 

ATM Aircrew Training Manual 

BP Battle Position 

BWRS Bedford Workload Rating Scale 

CDD Capability Development Document 

CPC Comanche Portable Cockpit 

CPG Copilot-Gunner 

D Demand 

EA Engagement Area 

EDS Engineering Development Simulator 

EUD Early User Demo 

FAC Flight Activity Category 

FCR fire control radar 

FORSCOM Forces Command 

FOV Field of View 

HDU Helmet Display Unit 

HRED Human Research & Engineering Directorate 
62 

LEUE Limited Early User Evaluation 

LMC Linear Motion Control 

LOS line of sight 

MANPRINT Manpower and Personnel Integration 

MPD Multi-Purpose Display 

MTADS Modernized Target Acquisition Detection System 

NAI Named Area of Interest 

NOE Nap-of-Earth 

OneSAF One Semi-Automated Forces 

OTW Out-the-Window 

PI Pilot (pilot in back seat who flew the aircraft) 

PMO Product Manager’s Office 

PNVS pilot night vision sensor 

RACRS Risk and Cost Reduction Simulator 

RL Readiness Level 

RP Release Point 

S Supply 

SASA Situation Awareness 

SART Situational Awareness Rating Technique 

SME Subject Matter Expert 

SSQ Simulator Sickness Questionnaire 

TADS Target Acquisition and Designation System 

TCDL Tactical Common Data Link 

TCM RA TRADOC Capability Manager, Reconnaissance Attack 

TCM TRADOC Capability Manager 

TEDAC TADS Electronic Display and Control 

TFC Thumb Force Control 
63 

TRADOC Training and Doctrine Command 

TS Total Severity 

TTP tactics, techniques, and procedures 

U Understanding 

UAS Unmanned Aerial System 

UCI UAS-Crewstation Interface 

VFR Visual Flight Rules 

VMC Visual Meteorological Conditions 

WSRT Wilcoxon Signed Ranks Test 
NO. OF NO. OF 

COPIES ORGANIZATION COPIES ORGANIZATION 

64 

 1 DEFENSE TECHNICAL 

 (PDF INFORMATION CTR 

 only) DTIC OCA 

 8725 JOHN J KINGMAN RD 

 STE 0944 

 FORT BELVOIR VA 22060-6218 

 1 DIRECTOR 

 US ARMY RESEARCH LAB 

 IMNE ALC IMS 

 2800 POWDER MILL RD 

 ADELPHI MD 20783-1197 

 1 DIRECTOR 

 US ARMY RESEARCH LAB 

 AMSRD ARL CI OK TL 

 2800 POWDER MILL RD 

 ADELPHI MD 20783-1197 

 1 DIRECTOR 

 US ARMY RESEARCH LAB 

 AMSRD ARL CI OK PE 

 2800 POWDER MILL RD 

 ADELPHI MD 20783-1197 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR ML J MARTIN 

 MYER CENTER RM 2D311 

 FT MONMOUTH NJ 07703-5601 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MZ A DAVISON 

 320 MANSCEN LOOP STE 115 

 FT LEONARD WOOD MO 65473 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MD T DAVIS 

 BLDG 5400 RM C242 

 REDSTONE ARSENAL AL 35898-7290 

 1 COMMANDANT USAADASCH 

 ATSA CD 

 AMSRD ARL HR ME DR HAWLEY 

 5800 CARTER RD 

 FT BLISS TX 79916-3802 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MM DR V J RICE 

 BLDG 4011 RM 217 

 1750 GREELEY RD 

 FT SAM HOUSTON TX 78234-5002 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MG R SPINE 

 BLDG 333 

 PICATINNY ARSENAL NJ 07806-5000 

 1 ARMY RSCH LABORATORY - HRED 

 ARMC FIELD ELEMENTT 

 AMSRD ARL HR MH C BURNS 

 THIRD AVE BLDG 1467B RM 336 

 FT KNOX KY 40121 

 10 HRED 

 USAACE FIELD ELEMENT 

 AMSRD ARL HR MJ D DURBIN 

 BLDG 4506 (DCD) RM 107 

 FT RUCKER AL 36362-5000 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MK MR J REINHART 

 10125 KINGMAN RD 

 FT BELVOIR VA 22060-5828 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MY M BARNES 

 2520 HEALY AVE 

 STE 1172 BLDG 51005 

 FT HUACHUCA AZ 85613-7069 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MP D UNGVARSKY 

 POPE HALL BLDG 4709 

 BCBL 806 HARRISON DR 

 FT LEAVENWORTH KS 66027-2302 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MJF J HANSBERGER 

 JFCOM JOINT EXPERIMENTATION J9 

 JOINT FUTURES LAB 

 115 LAKEVIEW PKWY STE B 

 SUFFOLK VA 23435 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MQ M R FLETCHER 

 NATICK SOLDIER CTR 

 AMSRD NSC WS E BLDG 3 RM 343 

 NATICK MA 01760-5020 

 1 ARMY RSCH LABORATORY-HRED 

 AMSRD ARL HR MT J CHEN 

 12423 RESEARCH PKWY 

 ORLANDO FL 32826 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MT C KORTENHAUS 

 12350 RESEARCH PKWY 

 ORLANDO FL 32826 
NO. OF NO. OF 

COPIES ORGANIZATION COPIES ORGANIZATION 

65 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MS C MANASCO 

 SIGNAL TOWERS 

 BLDG 29808A RM 303 

 FT GORDON GA 30905-5233 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MU 

 6501 E 11 MILE RD MS 284 

 BLDG 200A 2ND FL RM 2104 

 WARREN MI 48397-5000 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MF C HERNANDEZ 

 2421 NW AUSTIN RD STE 220 

 FT SILL OK 73503-9042 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MV 

 S MIDDLEBROOK 

 91012 STATION AVE RM 348 

 FT HOOD TX 76544-5073 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MN R SPENCER 

 DCSFDI HF 

 HQ USASOC BLDG E2929 

 FT BRAGG NC 28310-5000 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MW E REDDEN 

 BLDG 4 ROOM 332 

 FT BENNING GA 31905-5400 

 1 ARMY G1 

 (CD DAPE MR B KNAPP 

 only) 300 ARMY PENTAGON RM 2C489 

 WASHINGTON DC 20310-0300 

 1 US ARMY RSRCH DEV & ENGR 

 CMND 

 SYS OF SYS INTEGRATION 

 AMSRD SS T 

 6000 6TH ST STE 100 

 FT BELVOIR VA 22060-5608 

ABERDEEN PROVING GROUND

 1 DIR USARL 

 AMSRD ARL CI OK TP (BLDG 4600) 

 5 DIR USARL 

 AMSRD ARL CI OK TP 

 S FOPPIANO 

 AMSRD ARL HR 

 T LETOWSKI 

 AMSRD ARL HR MD 

 D HARRAH 

 AMSRD ARL HR MR 

 F PARAGALLO 

 AMSRD ARL HR SD 

 B AMREIN 

TOTAL: 42 (1 PDF, 1 CD, 40 HCs) 
 

66 

INTENTIONALLY LEFT BLANK.