H uman Factors of UAVs 1

H uman Factors Implications of UAVs in the National Airspace

J ason S. McCarley C hristopher D. Wickens

I nstitute of Aviation A viation Human Factors Division U niversity of Illinois at Urbana
- C hampaignH uman Factors of UAVs 2 I NTRODUCTION U nmanned aerial vehicles (UAVs) ar e quickly becoming a part of the national a irspace system (NAS) as they transition from primarily military and hobbyist a

pplications to mainstream flight applications such as security monitoring, satellite transport, and cargo hauling. Before the full pote ntial of UAV flight in the NAS can be r ealized, however, FAA standards and regulations for UAV operations must be e stablished. Given the experience of the U.S. military that mishap rates for UAVs are s everal times higher than for manned aircraft (Williams, 2004)

— ove r thirty times higher, i n some cases (Department of Defense, 2001)

—the importance of carefully designed s tandards and regulations is clear. I ssues related to human factors are likely to be of particular concern in e stablishing guidelines for UA V flight. As noted by Gawron (1998), UAV flight presents hum an factors challenges different from and in some ways greater than those of manned f light. These arise primarily from the fact that operator and aircraft are not co
- l ocated. As di scussed in more d e tail below, the separation of operator and vehicle imposes a number of barriers to optimum human performance, including loss of sensory cues valuable for f light control, delays in control and communications loops, and difficulty in scanning the vi sual env i ronment surrounding the vehicle. Unmanned flight also allows the possibility that a single operator might control multiple vehicles simultaneously, a task likely to i mpose unique and heavy workload demands. T he goal of the current work was to examine the e xisting research literature on the hum an factors of unmanned flight, and to delineate issues for future research to address. T he topics discussed below are divided into the categories A utomation ; P erceptual and C

ognitive Aspects of Pilot Interface ; A ir Tr af

fic Management Procedures ; and C rew Q ualifications
. As will be clear, however, the issues covered within the various c ategories are highly interrelated. Answers to questions about crew complement, for e xample, will be contingent on the nature and reliabi l ity of automation provided to s upport UAV operators. Likewise, decisions about interface design will depend on the e xtent to which flight control is automated, with manual flight mode demanding traditional stick
- and
- r udder controls and automated flight mo de allowing for point
- and
c lick menu
- ba sed control or other forms of non
- traditional interface. I t is also important to note that unmanned aircraft will likely serve a range of pur poses in civilian airspace, and that the demands placed on human operators w ill vary w ith characteristics of the flight mission. Proposed uses for UAVs include agricultural, ge ological, and meteorological data collection; border surveillance; long distance transport; search and rescue; disaster monitoring; traffic monitoring; and t elecommunications relay. Furthermore, military UAVs will increasingly be required to transition through civilian airspace en route to their missions. In some of these cases, the ve hicle is likely to operate solely within line
- o f
- s ight communications rang e and only ove r relatively short periods of time (i.e., on the time scale of several hours or less). In other cases, the vehicle will operate at distances demanding over
- the
- ho rizon c ommunications, and will potentially remain airborne for many days on end. These m ission characteristics will modulate concerns about communications delays between gr ound control station and vehicle, and about the need for transfer of vehicle control be tween crews. For some applications, additionally, operators will likely be re qui red toH uman Factors of UAVs 3

m ake frequent control inputs, adjusting flight parameters or selecting new waypoints “ online” in response to changing task demands or conditions. For other applications, f light path will be predetermined and less susceptible to modification, redu c ing the i mmediacy and frequency with which operators are required to intervene in flight control and allowing for a heavier reliance on automated vehicle guidance. T ECHNICAL APPROACH Our technical approach involved three parallel efforts. (1) We ac qui red a large body of l iterature, both in published sources and in technical reports, that addressed any aspects of human factors in UAVs. This literature is documented in an annotated bibliography in Appendix A. (2) We identified laboratories where UAV hum an factors work is in pr ogress. These laboratories, and points of contacts, are listed in Appendix C. (3) We be came acquainted with UAV human factors issues in civilian airspace by familiarizing our selves with Access 5 documents. (4) We applied our own s ubj ect - m atter expertise of both aviation human factors in general, and UAV operations in particular, to identify 18 ke y human factors research topics, that we believed were r elatively unique to UAV ope rations. This uniqueness constraint is critical. There are for example numerous human f

actors issues that should be applied equally to manned as well as to unmanned aircraft, r elating to topics such as display legibility, CRM and communications, checklist design, e tc. We did not include these in our effort, but note their enduring importance for UAV c ertification. Research topics are described in the text below, and in Appendix B are c ross - i ndexed with relevant sources from the research literature described in Appendix A. Having identified issues, and exam ined written documents that described human f

actors research, our final product was to map research needs against existing research doc uments, where such documents contained empirically valid findings. This material, c ontained in table 1 of the report bel ow , provides an identification of the key research areas that we believe should be funded, in order to proceed on the path toward safe c ertification of UAVs in civilian airspace. We have not explicitly prioritized these areas i n terms of their importance. A UTOMATION ISSUES 1. To what extent should en route flight control be automated? C urrent UAV systems vary in the degree to which en route flight control is a

utomated. In some cases the aircraft is guided manually using stick and rudder controls, w ith the ope rator receiving visual imagery from a forward looking camera mounted on the vehicle. In other cases control is partially automated, such that the operator selects the de sired parameters or behaviors through a computer menu or rotary dial interface in th e gr ound control station. In other cases still control is fully automated, such that an a

utopilot maintains flight control using preprogrammed fly
- t o coordinates. At least one s ystem (Pioneer), finally, allows the operator to switch between full manual, hyb rid, and f ull automation control modes. T hese various modes of flight control each present benefits and drawbacks ( Mouloua, et al, 2003). Full manual control would seem to impose the highest and most c ontinuous level of cognitive workload on UAV operators
. Moreover, manual control w ill be degraded by communication delays between UAV and GCS (see #8, #13). C onversely, fully automated control can prevent an operator from rapidly interveningH uman Factors of UAVs 4

when necessary, (e.g., upon loss of communications) and by leaving the operator largely “ out of the loop” (Wickens & Holland, 2000), can produce degraded situation awareness ( e.g., noticing a change of handling qualities due to icing). Flight planning can also be e xcessively time consuming for fully automated systems, some t imes requiring many w eeks (Williams, 2004). F or reasons like those described above, Mouloua, et al (2003) recommended hybr id manual/automated control systems for military UAVs. A blanket r ecommendation, however, may not be appropriate for UAV flight in c i vilian airspace. R ather, the optimal flight control system seems likely to vary with the characteristics of the flight operation, either within or across flights. UAV operations that entail primarily l ong - e ndurance station
- ke eping (ACCESS 5, 2003), for ex ample, are not likely to impose e specially high demands on operator situation awareness. Fully automated control might therefore be more appropriate for such operations that either hybrid or manual a

utomation. The optimal level of automation may also depen d on the number of UAVs that a single operator is required to control, the communication delays between operator and UAV, and the quality of visual imagery and other sensory information provided to the operator from the UAV. A number of questions related t o the method of UAV flight control thus remain to be addressed. Research is recommended to: • D etermine the circumstances under which various modes of UAV flight control— f ully automated, partially automated, manual

— are appropriate. • D etermine whether or not the level of automated flight control should be r econfigurable, such that the operator can alternate between levels of control when he/she deems appropriate. • D etermine whether the reconfiguration of flight control should itself be adaptively a

utomated, such that the UAV system adjusts the level of automated flight control t o match the current circumstances (e.g., the current communications delay be tween UAV and GCS). • D etermine how and when the UAV operator will be allowed to override the a

utomated flight cont rol system. T he output of this work would be a set of rules advising what level of automation should be available/required, during what phases of flight and types of operations. 2. What are the consequences of degraded reliability of automated UAV functi on s f

or performance of the automated task and of concurrent tasks? A s the discussion in #1 above makes clear, UAV operations are likely to be hi ghly automated. It is widely acknowledged, however, that often the effect of a

utomation is not to reduce the hum an operator’s task demands but rather to change them, imposing new forms of cognitive workload and modifying the operator’s pe rformance strategies (Parasuraman, 2000). Such changes, and occasional increases in c ognitive workload, often result in circumstan c es when automation is imperfect. This i mperfection does not refer to issues such as software reliability (e.g., “10 – 5 r equirements”), but rather, to circumstances where correctly functioning automation is i ncapable of perfectly carrying out the functions asked of it. Examples include on
- boa rd c onditions (e.g., icing) for which stability control cannot fully compensate, diagnostic s ystems based on imperfect cues, or conflict detection/avoidance algorithms based uponH uman Factors of UAVs 5 f uture trajectory estimates in a probabi l istic environment (Xu, et al, 2002; Kuchar, 2001). P ast work has indicated that imperfect automation at a reliability level greater than around 0.80 can continue to support performance on the automated task as well as c oncurrent tasks (Dixon and Wickens, 2004; Wickens & Dixon, 2005), although this “ threshold” estimate remains far from an absolute value, and other factors, such as the nature and priority of the automated task, appear to modulate pilot tolerance for i mperfection. To allow the optimal design of automated support systems for UAV ope rators, research is thus recommended to: • D etermine the minimum acceptable reliability levels for automated functions that r elatively unique to UAV operations. • A nticipate potential forms of system failure, and delineat e their likely c onsequences. • E stimate the means and standard deviations of operators’ response times to various failures. T echniques such as Failure Modes Effects Criticality Analysis can be used in these e ndeavors.
3. How will see and avoid requiremen t s be addressed in UAV flight? Can automated d etect, see, and avoid ( D

SA ) technology allow a UAV operator to maintain ac ceptable levels of separation? What are the consequences of imperfectly reliable D SA automation on conflict detection and on performance of concurrent tasks? T he ability to maintain adequate separation between aircraft is a prerequisite for the safe integration of unmanned vehicles into the NAS. While safe separation from other a ircraft can generally be assured through standard ATC operati ons in operations under I FR and IMC (but see issue #13 below), there will be times in which UAVs may be f lying under VFR (or a corresponding designation) in which detect, see and avoid (DSA) c apabilities are essential. In such circumstances, separation may often be maintained through emerging CNS (communications, navigation, surveillance) technology supported by GPS navigation and ADS
- B communications. However, these conditions do not a ccommodate unequipped (non
- c ooperating) air vehicles that are unable to a ccurately transmit (or transmit at all) their position and trajectory through the 3D airspace, and which may be uncooperative or non
- r esponsive in negotiating conflict avoidance m aneuvers. It is for this reason that automated DSA functions are required. T he need for s uch functions raise two critical human factors concerns. F irst, operators will be asked to interact with error prone systems. It is likely that a

utomatic target recognition capabilities will be fallible, particularly if they are asked to ge ner ate early alerts (i.e., at sufficient distance that avoidance maneuvers are possible). A s a consequence, this form of automation will be imperfect (see # 2 above; Thomas, W ickens, & Rantanen, 2003), leading to either misses (late alerts) or false alerts. G i ven the high costs of misses, and low base
- r ate of events (Parasuraman, Hancock, and O bofinbaba, 1996), the false alarm rate will be potentially quite high (Krois, 1999). The e ffects of such automation errors will have to be considered in designing DSA sy s tems. S econd, operators will be required to interact with the imperfectly reliable DSA system while also maintaining responsibility for airframe and payload control. These concurrent r esponsibilities will determine the degree to which the operator can be e xpected toH uman Factors of UAVs 6

ove rsee the DSA, monitoring the raw data of the UAV sensor images of the 3D airspace upon which the DSA algorithms are based. In light of these concerns, research is r ecommended to: • D etermine how operators will respond to alert imperfections in DSA. • D elineate the conflict geometries and visibility conditions that are likely to de grade the reliability of DSA automation. • E stablish procedures by which the output of human perception and automated t arget analysis can be combined to maximize the sensi t ivity of the two c omponent (pilot and algorithm) system given the pilot’s concurrent r esponsibility for flight control.
4. To what extent should takeoff and landing be automated? C urrent UAV systems differ in their manner of takeoff and landing. Some (e
. g., the Hunter and Pioneer) are controlled by an on
- s ite external pilot. Others (e.g., the P redator) are controlled by an air vehicle operator within the GCS. For others still (e.g., the Global Hawk) takeoff and landing are fully automated. These differen c es appear to be c onsequential; takeoff and landing errors constitute a higher proportion of human factors
r elated accidents for the Hunter (67%) and Pioneer (78%) systems, both of which are c ontrolled by an external pilot, than for other systems (Williams , 2004). Research is therefore recommended to: • D etermine what method of UAV control during takeoff and landing is appropriate f

or aircraft in civilian airspace. • D elineate the responsibilities that the human operator can and will be expected to a ssume in th e case of automation failure. • E stablish guidelines to for how and how will the human operator will be allowed t o override automated control systems. P ILOT INTERFACE: PERCEPTUAL AND COGNITIVE ISSUES 5. Through what form of control interface should internal and external pilots m anipulate a UAV? A s noted above, UAV systems will vary in the degree to which airframe control is a

utomated either en route or during takeoff and landing. For any system that is not fully a

utomated

— i ncluding systems that allow for a hu man operator to intervene in vehicle c ontrol by overriding automation

— i t will be necessary to provide operators with a c ontrol interface through which to manipulate the vehicle. The form of this interface will di ffer for internal pilots, those who interac t with the vehicle through an interface of s ensor displays and controls inside a ground control station, and external pilots, those who interact with the vehicle while in visual contact with it at the site of takeoff or l anding. In the case of full manual c ontrol by an internal pilot, the seemingly obvious c hoice of control design is a stick and rudder interface like that used for control of m anned aircraft. In cases of partially automated flight control, or of fully automated flight c ontrol where the pilot is provided authority to override the automation when deemed ne cessary, the optimal design of control interface is less clear. Current UAV systems vary i n control design, with some systems allowing interaction through knobs or position s witches and others through mouse
- dr iven point
- a nd
- c lick computer menus (Williams,H uman Factors of UAVs 7 2004) . Alternative designs may be possible, however, and should be explored. A dditionally, it is important to ensure that any interface be tailored according to e stablished human factors guide l ines; data suggest that some existing UAV system i nterfaces are poorly designed for human interaction. S imilarly, research is necessary to assess and improve the design of controls for e xternal pilots. Currently, an external pilot manipulates the UAV usin g joystick controls s imilar to those used by radio
- c ontrolled aircraft hobbyists (Williams, 2004). These de signs are problematic, however, in that the mapping of vehicle movement to control i nput varies depending on the heading of the vehicle relative to that of the EP. When the he ading of the vehicle and pilot are the same, a rightward input to the joystick control pr oduces rightward motion from the aircraft relative to the pilot. When the heading of the a ircraft and pilot deviate, however, this is no long er true. In the most extreme case, where the heading of the UAV and pilot differ by 180°, a rightward input on the joystick pr oduces leftward motion of the vehicle relative to the pilot. Joystick controls for external pi lots are thus not designed to conform consistently to the well
- e stablished human factors pr inciple of m otion compatibility ( Wickens & Holland, 2000; Wickens, Vincow, & Yeh, 2005) . Not surprisingly, this violation appears to be contributing factor in a high number of UAV mishaps (Williams, 20 04) . Quigley and colleagues (Quigley, Goodrich, & Beard, 2004) have designed and tested a variety of control interfaces to address this problem. F urther is now necessary to: • E xplore and optimize the design of control interfaces for internal and external pi l ots’ control of UAVs. • D elineate the performance benefits and drawbacks of various forms of UAV c ontrol interface so as to determine which design should be adopted. 6. What compromises should be adopted between spatial resolution, temporal r esolution, tim e delay, and field - o f - vi ew (FOV) in the display of visual imagery for f light control and/or conflict detection? A UAV operator generally relies on imagery from onboard sensors for manual c ontrol of vehicle and payload and for visual target detection. The qua lity of this visual i nformation, however, may be degraded due to datalink bandwidth limits and transmission delays. Specific degradations include poor spatial resolution, limited FOV, l ow update rates, and delayed image updating (Van Erp, 2000). These co ndi tions will i mpair both vehicle control and the visual detection of air traffic. For example, low image update rates will degrade perception of motion information that is useful for drawing attention to air traffic in the visual field. Low update rates and long communication de lays, likewise, will produce discontinuous and slow visual feedback in response to ope rator control inputs, leading to instability of manual UAV control or camera image c ontrol and encouraging operators to adopt a “go
- and
- w ait strat e gy” in manual control ( Van Erp & Van Breda, 1999). Poor spatial resolution, obviously, will impair detection of obj ects that occupy only a small visual angle within an image, reducing stand
- of f distance i n detection of potential traffic conflicts (see 4 a bove ). A small field of view (FOV) will not only eliminate ambient visual information useful for assessing ego
- m otion necessary at low level flight (Gibson, 1979; Wickens & Holland, 2000), but will also impose a de mand for greater amounts of camera scannin g for successful traffic detection. A well
de signed system for display of sensory imagery will be required to balance the benefitsH uman Factors of UAVs 8

and costs of temporal resolution, spatial resolution, and FOV. To guide the design of vi sual information displays, research i s recommended to: • D etermine what information is “task
- c ritical” in manual airframe control, payload s ensor control, and visual traffic detection (Van Breda, 2000). • E stablish the optimal compromise between spatial resolution, temporal resolution, and FOV i n the display of visual imagery. A s part of this work, it will be important to establish a catalogue of mission payload r equirements that may compromise the quality of visual information for flight, and e stablish the minimum necessary information (time del ay and image quality) for manual c ontrol. For different functions, sensitivity curves should be established to show pe rformance quality or function degradation as a function of spatial and temporal r esolution. 7. Can augmented reality displays or synthet i c vision systems successfully c ompensate for the degraded visual imagery provided by onboard sensors? A s noted above (#6), low temporal resolution and delayed updating of visual i magery received from onboard sensors will degrade manual control of airfram e and payload tasks. The judicious choice of spatial and temporal image parameters may attenuate these effects, but is unlikely to mitigate them in full. An alternative approach to i mproving visual information display may be through the use of “augmented r e ality” ( Milgram & Colquhoun, 1999) or “synthetic vision” (Draper, et al, 2004), in which the r eal
- w orld imagery provided by a sensor is embedded within a display of computer
ge nerated landmarks or objects representing the same scene. The virtue of augment e d r eality in the context of UAV flight is that the computer
- ge nerated component of a di splay can be updated immediately in response to control inputs from a UAV operator, pr oviding rapid feedback to improve manual tracking. For example, Van Erp & Van B red a (1999) provided subjects in a simulated payload sensor control task camera i magery overlaid by a computer
- ge nerated grid of perpendicular lines, oriented so as to c onform to the imaged scene. The synthetic grid shifted in real time following input from the operator, giving visual feedback as to the direction and magnitude of camera m ovement. As compared to a control condition with no virtual grid, augmented displays s ignificantly improved target tracking at low camera update rates (i.e, long sensory de lay s ). A study by Veltman and Oving (2002) produced similar benefits by embedding c urrent and predicted camera footprints within a larger map (either 2D or 3D) of the t errain to be scanned. A still more sophisticated form of display is a fully virtual synthet i c vi sion system, in which terrain information is stored in databases and rendered based on G PS position. An important issue here concerns the degree of realism with which s ynthetic imagery should be presented, whether minimalist (e.g., the grid used by Van E rp and Van Breda), or highly realistic, such as employed in current SVS systems ( Prinzel et al, 2004). The danger of the latter is that pilots may place undue trust in the i magery, leading to cognitive tunneling and neglect of information not available w ithin s uch a high imagery display (e.g., a “transponder off” aircraft; Thomas and Wickens, 2004) . Augmented displays thus present a promising method of enabling good UAV ope rator performance, but are not without potential costs. Research is thus recommend e d t o:H uman Factors of UAVs 9

• F urther develop and test predictive augmented displays to improve airframe and payload sensor control. • D etermine the effects of display format/fidelity on the UAV operator’s level of trust in the automated system.
8. Can multimodal display techno l ogy be used to compensate for the dearth of s ensory information available to a UAV operator? O ne of the primary consequences of the separation between aircraft and operator i s that the operator is deprived of a range of sensory cues available to the pilot of a m anned vehicle. Rather than receiving direct sensory input from the environment in which his vehicle is embedded, the UAV pilot receives only that information provided by onboard sensors via datalink. As noted above, this consists primarily of potent i ally de graded visual imagery covering a relatively small FOV. Sensory cues that are lost thus i nclude ambient visual input, kinesthetic/vestibular information, and sound, all of which are valuable in maintaining operator awareness of the environmental and system c onditions (e.g., turbulence, icing). As compared to the operator of a manned aircraft, therefore, a UAV operator can be said to function in relative “sensory isolation” from the a ircraft under his control. It is critical in light of this for UAV s ys tem developers to de sign displays and alarms to keep operators well
- informed of system status and aware of pot ential system failures. V isual displays provide one method of presenting a UAV operator with sensor i nformation beyond that conveyed by imagery from a vehicle
- m ounted camera. Data s uggest, however, that UAV operators may not optimally modify their visual scanning s trategies to compensate for the absence of multisensory cues (Tvaryanas, 2004). M oreover, the task of creating an “ecological”, intui t ively
- i nterpreted visual r epresentation for such information is often difficult. An alternative way to compensate U AV operators for the lack of direct sensory input from the vehicle’s environment could be through the use of multimodal (e.g., tactile or au di tory) information displays. For e xample, fly by wire controls have long been equipped with augmented force feedback m imicking the forces experienced on the air surfaces of manned aircraft, and roughly c apturing the changes in handling quality. Ruff, et a l (2000) examined the value of haptic di splays for alerting UAV operators to the onset of turbulence. Their data revealed that ha ptic alerts, conveyed via the UAV operator’s joystick, could indeed improve self
- r ated s ituation awareness during turbulent con di tions in a simulated UAV approach and l anding task. Interestingly, this was true despite the fact that the haptic signals were not de signed to closely simulate or mimic the veridical haptic information experienced by the pi lot of a manned vehicle. The be ne fits of haptic displays, however, were obtained only unde r limited circumstances (specifically, only when turbulence occurred far from the runway), and were offset by an increase in the subjective difficulty of landing. These r esults suggest some value o f multi
- m odal displays as a method of compensating for s ensory cues typically denied to a UAV operator, but also indicate that such displays may not be universally valuable and may carry costs as well as benefits. A related point is that multimodal displays may be useful not simply as a means t o compensate for the UAV operator’s impoverished sensory environment, but more ge nerally to reduce cognitive
- pe rceptual workload levels. Studies by Calhoun, et alH uman Factors of UAVs 10 ( 2002), Sklaar and Sarter (1999), and Wickens and Dix on (2002; Dixon & Wickens, 2003; Wickens, Dixon, & Chang, 2003), for example, have found that auditory and tactile di splays can improve aspects of flight control and system monitoring. I n sum, research is necessary to: • F urther develop techniques for multi m odal information display. • A ssess the value of multimodal displays in countering UAV operators’ sensory i solation. • A ssess the more general value of multimodal displays in distributing workload opt imally across cognitive
- pe rceptual channels. 9. To what ext e nt can displays and controls be standardized across UAV systems? Wh at level of standardization should be mandated? (Basic T instrument panel? H UD overlay?) W e anticipate a tendency for vendors to produce novel designs for the interface, particularly, give n the diversity of specialized payload missions for which UAVs may be de signed. It is essential to establish certain commonalities across all interfaces. Exactly what these should be remains a question for research. Questions to be considered include, but are not limited to: • S hould the “basic T” always be maintained? • I s an inside
- out attitude display necessary, given that the pilot is no longer inside the vehicle? • S hould certain information always be visible (never hidden to be retrieved by m enu navigatio n) ? • S hould all aspects of the payload display be kept spatially separated from the pr imary flight display, or are HUD overlays advisable? • S hould certain controls (e.g., a joystick) be mandatory for certain functions, and s hould others be prohibited (e.g. , mouse for flight control)? Identification of these issues recognizes that no single display layout or control a ssignment is optimal for all tasks, but also recognizes that certain cases of inconsistency c an lead to negative transfer and pilot erro r , as pilots transfer from one interface to another. Similar issues have been addressed in assigning common type ratings and di fferences training in commercial manned aircraft. 10. Wh at are the consequences for system safety of pilot judgment when the pi l ot no l onger has a “shared fate” with the vehicle? Will there be subtle shifts in risk taking t hat might affect overall airspace safety? U AV pilots will not be at risk for injury or death if their aircraft crashes, in c ontrast to the circumstances of mann e d aircraft pilots, who “share fate” with their a ircraft. This difference could, in theory, impose a substantial difference in risk taking t endencies, in such areas as the decision to carry out a flight into bad weather (Goh & W iegmann, 2001; Weigmann, Goh , & O’Hare, 2002). Such differences may be further amplified by the sensory isolation described previously. Research is thus recommended t o: • D etermine how the UAV operator’s risk perception and risk taking behavior are a ffected by absence of shared fate wi th his/her vehicle.H uman Factors of UAVs 11

• D etermine how the UAV operator’s risk perception and risk taking behavior are a ffected by the absence of sensory/perceptual cues.
A IR TRAFFIC MANAGEMENT PROCEDURAL ISSUES 11. How will hand - o ffs between crews be accomplished during lo n g - e ndurance f lights? L ong - di stance and/or long
- e ndurance UAV flight will require the frequent transfer of control between operators, generally taking one of three forms (Kevin W illiams, personal communication). First, control may be passed from one ground c ontrol station to another. Second, control may be passed from one crew of operators to another within the same ground control station. Finally, control may be passed from one ope rator to another within a crew. The transfer of control will likely constitut e a critical and high
- w orkload phase of UAV operations. Indeed, a number of military UAV a ccidents have occurred during transfer of control from one team of operators to another, ge nerally because the station receiving control was not properly configured ( Williams, 2004) . Research is necessary to establish procedures for the safe handoff of control be tween UAV crews. More specifically, this work should: • D evelop and test formal procedures for handoff of UAV control between teams of ope rators. • D evelop and te s t displays, automation, and procedures to ensure that the operators r eceiving UAV control are adequately informed of system status and are alerted to di screpancies in system configuration between control stations relinquishing and a ssuming vehicle control
. 12. What are the effects of variable total loop time delays on response to ATC i nstructions? D atalink delays may be expected to add as many as several seconds to the c ommunications loop between UAV operators and ATC. The magnitude of these delays, how e ver, will be variable, and may not always be predictable to human operators. Thus, c ontrollers may have greater difficulty in compensating for these delays than they do in c ompensating for the fixed response characteristics of a given class of aircraft. P ot ential c ompensatory responses to communication delays are changes in the timing with which A TC commands are issued and acted upon, and changes in the communications flow be tween ATC and UAV operators (Kiekel, Gorman, & Cooke, 2004; Rantanen McCarley & Xu , 2004). To anticipate and accommodate the effects of communications delays, it w ill be necessary to understand and take account of these compensatory behaviors and their consequences for system performance. Research should thus be conducted to: • D etermine what compensatory behaviors, if any, air traffic controllers’ and UAV ope rators’ adopt in response to communication delays. • D etermine the effects of communications delays on the flow of air traffic. C omputer simulation models of communications may be parti c ularly effective tools here, s o long as such models are based on empirically validated estimates of human response t ime, variability, and reliability (probability of communications error).H uman Factors of UAVs 12 13. What form of predictable autonomous behavior should a UAV ad op t following a l oss of ground - t o - ai r communications? How should the UAV operator be alerted to a loss of ground - t o - ai r communications? O ne particularly disruptive scenario of UAV automation failure is a total severing of the GC
- U AV control loop. It is imp ortant that the vehicle behave predictably under such c ircumstances. This is a human factors issue because such default rules are of critical i mportance to the ATC/ATM who must manage traffic based on the knowledge of these rules (Shively, 2004). It is als o important, clearly, that the UAV operator become aware of the communications loss as rapidly as possible. Research is therefore necessary to: • D etermine what behavior UAV be programmed to adopt by default in case of a t otal communications loss with ground control station (Continue to fly on a s traight path? Descend? Fly toward the nearest equipped airfield?). • D evelop displays, automation, and/or procedures by which the UAV operator can be made aware of a communications loss, and be provided estimates of it s pot ential causes and consequences.
C REW QUALIFICATIONS 14. How many members will each crew comprise, and what will be each c rewmember’s responsibilities? Can an operator supervise multiple UAVs s imultaneously while maintaining an acceptable level of p e rformance? M ilitary UAV crews for reconnaissance missions typically include both an air ve hicle operator and a mission payload operator (Draper, et al, 2000; Mouloua et al, 2003) . Such crew structure is reasonable in light of findings that the assignment of a irframe and payload control to the same operator can substantially degrade performance ( Van Breda, 1995, cited in Van Erp & Van Breda, 1999). For UAV flight in civilian a irspace, however, the size of the crew complement necessary for each vehicle is li ke ly to be contingent on the nature and goals of the flight task (e.g., surveillance vs. long
di stance transport vs. station keeping for telecommunications). Although some research ha s demonstrated possibility of single pilot
- multiple UAV (1
- t o - m any) contr ol ( Cummings and Guerlain, 2004; Galster, et al, 2001; Wickens, et al, 2003), these s uccesses pre - s uppose three circumstances: (1) closely coordinated and correlated a ctivities among the multiple UAVs (Cummings & Guerlain, 2004), (2) operation in a di sturb ance free (closed) environment, such as very high altitudes, (3) high levels of r eliable automation (Dixon and Wickens, 2004). When any of these three characteristics are not in force, and, in particular, when one UAV enters a failure mode, the ability of the pi lot to monitor others in a 1
- t o - m any configuration is severely compromised. F urthermore, even in a 1
- t o - 1 configuration, performance of concurrent tasks is dr amatically degraded when heavy demands are imposed on the single operator by c omplex payload operations (e.g., manipulating camera imagery) (Dixon and Wickens, 2004) . In light of this, r esearch is necessary to: • D elineate circumstances under which multiple responsibilities (e.g., flight control, c onflict detection, payload control) be safely assig ne d to a single UAV operator, and circumstances under which such responsibilities should be distributed across two or more crew members. A nd, by extension:H uman Factors of UAVs 13

• D elineate circumstances under which a single operator can safe hold r esponsibility for multiple UAVs simultaneously. I t is crucial that such research consider circumstances under which automation is i mperfect (#2), and that it address the potential cost of communications and teamwork be tween multiple operators (Kiekel, et al, 2004). 15. What are the co r e knowledge, skills, and abilities (KSAs) that should be r equired for UAV pilot certification? What KSAs should be required for c ertification to fly particular UAV systems or classes of systems? R esearch is necessary to: • D etermine the general KSAs that w ill be required of all UAV operators. • D etermine KSAs required for certification to operate specified classes or systems of UAV.
16. How should UAV operators be trained? What constitutes an appropriate r egimen of ground school, simulator, and flight ex p erience for UAV flight c ertification? S afe flight of unmanned vehicles in the national airspace will demand effective pr ocedures for UAV pilot training. Ryder, et al (2001) note that because the task demands of operating a UAV from a ground control conso l e are similar during simulated and real f light, simulator experience is likely to constitute a greater portion of training for pilots of unm anned vehicles than for pilots of manned aircraft. As noted below (#17), furthermore, e xperience piloting manned air craft appears to produce positive but imperfect positive transfer to UAV flight (Schreiber, et al, 2002). Research is needed to: • O ptimize simulation systems for UAV pilot training and test their adequacy • E stablish requirements for flight training outside t he simulator. • D etermine to what extent manned pilot experience should offset training r equirements for UAV certification.
17. Should experience piloting a manned aircraft be prerequisite for UAV pilot c ertification? P ast research has come to conflicting conclusions as to whether UAV operators w ill benefit from experience piloting a manned aircraft. Schreiber, et al (2002) examined the effects of prior flight experience on novice operators’ skill acquisition and transfer in a Predator UAV simulation. In g e neral, flight experience reduced the number of training trials required for operators to reach a criterion level of performance on a set of basic m aneuvering and landing tasks, and improved operator performance on a subsequent r econnaissance task. Other f i ndings, however, have suggested that UAV operators need not be rated aviators. Using the Army’s Job Assessment Software System (JASS), B arnes, et al (2000) elicited Hunter UAV operators ratings of the relative importance of various cognitive skills in UAV air vehicle operators. Ratings indicated that outside of c ommunication skills, raters did not consider flight
- r elated skills of great importance to U AV operations, leading the authors to conclude that selection of rated aviators as air ve hicle operators w oul d be of little value.H uman Factors of UAVs 14 T he apparent discrepancy in the conclusions reached by Schreiber, et al (2002) and Barnes, et al (2000) may be due, at least in part, to differences in the operation of the U AV systems under consideration; while the Predator is pi l oted manually via a stick and rudder interface similar to that of a manned aircraft, the Hunter is guided by automation that allows the operator to select flight parameters using knobs on the GCS console. The va lue of prior flight experience to a UAV oper ator, that is, may depend in part on s imilarity between the manned and unmanned systems. Research is necessary to: • D etermine whether and how much experience piloting a manned system should be required for UAV pilot certification. • D etermine whether prerequi s ite levels of flight experience, if any, should vary a cross UAV platforms. 18. What medical qualifications should a UAV operator be required to meet? A lthough issues of high altitude physiology and medication induced vestibular di sruption are not relev ant to UAV pilots, some forms medical qualifications are likely to r emain necessary. Research is necessary to: • D etermine whether medical standards for UAV operators should be in any ways l ess or more strict than for pilots of manned aircraft. • E stablish sp e cial duty limits for long duration missions.H uman Factors of UAVs 15 R eferences A ccess 5 (2003). H ALE ROA Concept of Operations
. B arnes, M. J., Knapp, B. G., Tillman, B. W., Walters, B. A., Velicki, D. (2000). C rew s ystems analysis of unmanned aerial vehicle (UAV) future jo b and tasking e nvironments ( Technical Report ARL
- T R
- 2081 ). Aberdeen Proving Ground, MD: A rmy Research Laboratory. C alhoun, G.L., Draper, M.H., Ruff, H.A., & Fontejon, J.V. (2002). Utility of a tactile di splay for cueing faults. P roceedings of the Human Fa c tors and Ergonomics Soc iety 46th Annual Meeting, 2144
- 2148
. C ummings, M. L., & Guerlain, S. (2004). Developing operator capacity estimates for s upervisory control of autonomous vehicles
. Manuscript under review at Human F actors. D epartment of Defense (2 001 ). U nmanned aerial vehicles roadmap, 2002

- 2025
. Office of the Secretary of Defense, Department of Defense, Washington, DC, April 2001. D ixon, S., & Wickens, C.D. (2003). I mperfect automation in unmanned aerial vehicle f

light control (Technical report A HFD
- 03
- 17/ MAD
- 03
- 2) . Savoy, IL: University of Illinois, Institute of Aviation, Aviation Human Factors Division. D ixon, S. R., & Wickens, C. D. (2004). R eliability in automated aids for unmanned ae rial vehicle flight control: Evaluating a model of automat i on dependence in hi

gh workload ( Technical report AHFD
- 04
- 05/ MAAD
- 04
- 1) . Savoy, IL: U niversity of Illinois, Institute of Aviation, Aviation Human Factors Division. D raper, M. H., Geiselman, E. E., Lu, L.G., Roe, M. M., & Haas, M. W. (2000). Display c oncep t s supporting crew communications of target location in unmanned air ve hicles. Proceedings of the IEA 2000/ HFES 2000 Congress , 3.85
- 3.88. D raper, M.H., Nelson, W.T., Abernathy, M.F., Calhoun, G.L. (2004). Sy nthetic vision ov erlay for improving UAV oper at ions
. G alster, S. M., Duley, J. A., Masalonis, A. J., & Parasuraman, R. (2001). Air traffic c ontroller performance and workload under mature free flight: Conflict detection and resolution of aircraft self
- s eparation. I nternational Journal of Aviation P

s y chology, 11 , 71
- 93. G awron, V.J. (1998). Human factors issues in the development, evaluation, and operation of uninhabited aerial vehicles. A UVSI '98: Proceedings of the Association for U

nmanned Vehicle Systems International , 431
- 438
.H uman Factors of UAVs 16 G ibson, J. J. (197 9)
. T he ecological approach to visual perception
. Hillsdale, NJ: E rlbaum. G oh, J., and Wiegmann, D (2001). Visual flight rules flight into instrument m eteorological conditions. I nternational Journal of Aviation Psychology , 11 , 359
- 379. K iekel, P. A., Gorm an, J. C., & Cooke, N.J. (2004). Measuring speech flow of co
- l ocated and distributed command and control teams during a communication channel gl itch. P roceedings of the Human Factors and Ergonomics Society 48th Annual M eeting , 683
- 687. M ilgram, P., & Col quhoun , H. (1999). A taxonomy of real and virtual world display i ntegration. In Y. Ohta & H. Tamura (Eds.), M ixed Reality

— M erging real and v irtual worlds
. Tokyo, Ohmsha and Berlin, Germany: Springer
- V erlag. M ouloua, M., Gilson, R., & Hancock, P. (2003). H um an - c entered design of unmanned a erial vehicles. Ergonomics in Design, 11 , 6
- 11. P arasuraman, R. (2000). Designing automation for human use: empirical studies and quantitative models. Ergonomics, 43 , 931
- 951. P arasuraman, R., Hancock, P.A., & Obofinbaba , O. (1997). Alarm effectiveness in driver c entered collision warning systems. Ergonomics , 4 0, 390
- 399
. P rinzel, L. J., K ramer, L. J., B ailey, R. E.,. Parrish, R.V., & Arthur, J. J., (2003) C FIT P revention using Synthetic Vision. P roceedings of SPIE

- The International Soc iety for Optical Engineering , 146
- 157
. Q uigley, M., Goodrich, M.A., & Beard, R.W. (2004). Semi
- a utonomous human
- U AV i nterfaces for fixed
- w ing mini
- U AVs. P roceedings of IROS 2004 , Sep 28 – Oct 2, S endai, Japan. R antanen, E. M., McCarley, J. S., & Xu, X. (2004 ) . Time delays in air traffic control c ommunication loop: Effect on controller performance and workload. I

nternational Journal of Aviation Psychology, 14 , 369
- 394. R

uff, H.A., Draper, M.H., Lu, L.G., Poole, M.R., & Repperger, D.W. (2000). Haptic f

eedback a s a supplemental method of alerting UAV operators to the onset of turbulence. P roceedings of the IEA 2000/ HFES 2000 Congress , 3.41
- 3.44. S chreiber, B. T., Lyon, D. R., Martin, E. L., & Confer, H. A. (2002). Impact of prior f

light experience on learnin g Predator UAV operator skills ( AFRL
- H E
- A Z
- T R
2002
- 0026) . Mesa, AZ: Air Force Research Laboratory, Warfighter Training R esearch Division.H uman Factors of UAVs 17 S hively, R (2004) Panel chair: Human Factors Issues in UAV Systems: Proceedings 48 t h A nnual Meeting of the Human Fac t ors and Ergonomics Society. S klar A.E., & Sarter, N.B. (1999). Good Vibrations: Tactile feedback in support of attention allocation and human
- a utomation coordination in event
- d riven domains. H

uman Factors, 41( 4), 543
- 552. T homas, L.C., & Wickens, C.D. (2004). Eye
- t racking and individual differences in off
normal event detection when flying with a Synthetic Vision System Display. P roceedings of the Human Factors and Ergonomics Society 47 th Annual Meeting, 223
- 227
. T homas, L. C., Wickens, C. D., Rantane n, E. M (2003). Imperfect Automation in A viation Traffic Alerts: A Review of Conflict Detection Algorithms and Their I mplications for Human Factors Research. Proceedings of the Human Factors and Ergonomics Society 47 th Annual Meeting, 344
- 348. T varyanas, A .P. (2004). Visual scan patterns during simulated control of an uninhabited a erial vehicle (UAV). A viation, Space, and Environmental Medicine, 75 , 531
- 538. V an Erp, J.B.F. (2000). Controlling unmanned vehicles: The human factors solution. R TO Meeting Pro c eedings 44 (RTO

- M P

- 44 ) , B8.1
- B 8.12. V an Erp, J. B. F., & Van Breda, L. (1999). H uman factors issues and advanced interface design in maritime unmanned aerial vehicles: A project overview
. TNO
- r eport T M
- 99
- A 004. V eltman, J.A., & Oving, A.B. (2002). Augme nt ing camera images for operators of unm anned aerial vehicles. R TO Meeting Proceedings (RTO

- M O

- 088)
. W iegmann, D.A., Goh, J., and O’Hare, D (2002). The role of situation assessment and f light experience in pilot’s decisions to continue visual flight rules in to adverse w eather. H uman Factors 44, 189
- 197
. W ickens, C. D. (2004). Aviation displays. In P. S. Tsang & M. A. Vidulich (Eds.), P rinciples and Practice of Aviation Psychology (pp. 147
- 200 ). Mahwah, NJ: E rlbaum. W ickens, C.D., & Dixon, S. (September, 2002). W orkload demands of remotely piloted v ehicle supervision and control: (1) Single vehicle performance (Technical report A HFD
- 02
- 10/ MAD
- 02
- 1) . Savoy, IL: University of Illinois, Institute of Aviation, A viation Human Factors Division. W ickens, C.D., & Dixon, S. (2005). I s there a magic number 7 (to the minus 1)?: The be nefits of Imperfect Diagnostic Automation: A synthesis of the Literature
. ( Technical Report. MAAD 05
- 1) Savoy, IL: University of Illinois, Institute ofH uman Factors of UAVs 18 A viation, Aviation Human Factors D ivision. W ickens, C. D., Dixon, S., & Chang, D. (2003). U sing interference models to predict pe rformance in a multiple

- task UAV environment

- 2 UAVs (Technical report A HFD
- 03
- 09/ MAAD
- 03
- 1) . Savoy, IL: University of Illinois, Institute of A viation, Aviatio n Human Factors Division. W ickens, C. D., & Hollands, J. G. (2000). Engineering psychology and human pe rformance (3rd ed.). W ickens, C. D, Vincow, M. and Yeh, M (2005). Design Implications of visual
- s patial thinking: the importance of frames of reference
. In A. Miyake and P. Shah (Eds) H

andbook of Visual spatial thinking
. Oxford U Press. W illiams, K. W. (2004). A summary of unmanned aircraft accident/incident data: Human f

actors implications
. (Technical report DOT/FAA/AM
- 04/ 24). Washington, DC: O ffice of Aerospace Medicine, FAA. Xu, X., Wickens, C.D., & Rantanen, E. (2004). Imperfect conflict alerting systems for the c ockpit display of traffic information (Technical report AHFD
- 04
- 08/ NASA
- 04
- 2) . Savoy, IL: University of Illinois, Institute of Aviation, A vi ation Human F actors Division.H uman Factors of UAVs 19 A ppendix A B ibliography 1) B all, J.T., Gluck, K.A., Krusmark, M.A., & Rodgers, S.M. (2001). C omparing three variants of a computational process model of basic aircraft m aneuvering. P roceedings of the 12th Conference on B ehavior Representation in M odeling and Simulation . T he paper uses an ACT
- R model to examine expert/novice differences and effects of c ontrol strategy on Predator UAV flight. Three models were developed. Model P ( performance only) lacked the knowledge of c ontrol instrument settings that is c haracteristic of expert pilots, and therefore could only rely on performance indicators in m aneuvering the aircraft. Model CP (Control + Performance) had knowledge of control and performance settings needed to achieve ai r craft behavior, and therefore could rely on a control and performance strategy. However, the model did not remain focused on c ontrol indicator after making adjustment to it, but continued with normal crosscheck and c hecked to see if manipulation had its i nt ended when attention eventually returned to the i ndicator. Model CFP (Control Focus & Performance) was similar to model CP, except that it remained focus on control instrument until it was properly set. This was in addition t o normal crosscheck. T o exam i ne expert/novice differences, the authors compared Model P to Model CP. To e xamine strategic effects, they compared Model CP to Model CFP. R esults P erformance was better for Model P than for Model CP on 6 of 7 maneuvers. Model P w as better on the most c omplex (three
- axis) maneuver, though its not clear why. P erformance was better for Model CFP than for Model CP on 5 of 7 maneuvers. P erformance on other two maneuvers was similar. Overall, Model CFP performed the m ost like human subject matter experts.

2) B arnes, M.J., Knapp, B.G., Tillman, B.W., Walters, B.A., Velicki, D. (2000). C

rew systems analysis of unmanned aerial vehicle (UAV) future job and tasking environments (Technical Report ARL - T R - 2081) . Aberdeen Proving Ground, MD: A rmy Research Laborator y . E xperiment 1 A ssessed the importance of using rated aviators for air vehicle operator (AVO) and e xternal pilot (EP) positions in the Hunter UAV. The AVO tasks are to design mission pl ans in collaboration with commander, fly the UAV after take
- o ff, and set course to w aypoints. The AVO must be able to read the instruments, understand flight status, c oordinate with the mission payload operator (MPO) when reaching target area, andH uman Factors of UAVs 20

r espond to emergencies and make course changes when necessary. However, the A VO doe s not fly the vehicle using stick
- and
- rudder controls, in the manner of a typical pilot. T he EP is responsible for take
- o ff and landing, using a controller similar to that for model a irplanes. Most flight safety problems occur during the times that the EP is in control, pr imarily because take
- of f and landing are inherently difficult. T he study used the Army's Job Assessment Software System (JASS) to determine what c ognitive skills are important for the AVO and EP positions. JASS collects ratings abo ut the degree to which various skills and abilities are necessary to perform a given task. The s kills/abilities rated by JASS fall into six categories: communication, speed
- l oaded, r easoning, visual, auditory, and psychomotor. JASS data were supplemented w i th e nhanced computer
- a ided testing (ECAT) data from an earlier study. The ECAT data w ere one - and two
- handed tracking scores, which were correlated with failure rates for E P training. 21 subjects were AVO & MPO designated; 11 gave ratings of AVO task structure, 10 gave ratings of MPO task structure. 9 subjects were certified EPs, and gave ratings of EP t ask structure. 16 fixed
- or rotary - w ing Army aviators also rated EP skills. ECAT data c ame from a sample of 28 students in Pioneer and Hunter UAV EP trai ning courses. Six of these failed the course. R esults A VO raters did not rate flight
- r elated tasks as overly demanding on any of the six skill s ets except communication. The EP task was rated as more demanding than AVO task on a ll skill sets. EP subjects w ere broken into 4 groups: EP low experience, EP high e xperience, fixed
- w ing aviators, and rotary
- w ing aviators. Aviators gave slightly higher r atings to reasoning skills than did EPs. The EP low experience group gave especially hi gh ratings to vision, aud i tion, and psychomotor skills. Experienced EPs reported using m ostly conceptual skills during emergency situations. Inexperienced EPs reported relying on visual & psychomotor skills. E CAT tracking data were correlated with EP course success rates; 5 of the 6 students who failed had tracking data near bottom of sample distribution. These findings are c onsistent with the finding (noted above) that inexperienced EPs find visual and ps ychomotor skills to be particularly important. E xperiment 2 E xamined the p ot ential value of imagery and intelligence analysts as components of the U AV. The method used was to measure overlap between JASS ratings for imagery analysts, intelligence analysts, and UAV crew task duties. Imagery analyst skill rankings w ere significant l y correlated with those for 2 out of 16 UAV crew duties, intelligence analyst skill rankings were correlated with those for 14 out 16 UAV crew duties. Results s uggest that imagery analysts would complement UAV crew skill sets. E xperiment 3H uman Factors of UAVs 21 U sed a computat ional model of human cognition (Micro Saint) to investigate workload throughout the course of a simulated Outrider UAV flight mission. Also considered r emarks from subject matter experts (SMEs). Results suggest that candidate tasks for a

utomation included pre
- and post
- f light procedures & checks, verification of system s ettings, and computer checks of mission plans. SMEs reported that they did not want full a

utomation, but preferred instead to retain decision making authority themselves. To r educe workload , they suggested making the computer interface faster and letting the a

utomation provide backup check for safety problems.

3) B ell, B., & Clark, J.G. (2002). Bringing command and control of unmanned ai r vehicles down to earth. P roceedings of the 21st D i gital Avionics Systems C

onference (DASC) , Irvine, CA. D escribes an automated system to assist in UAV search area planning. System is called the Automated Search Area Planning System (ASAPS), and is meant to reduce search area by modeling terrain and targe t mobility then helping operator to plan a search route f

ocusing on areas where target is most likely to be found. 4) C alhoun, G.L., Draper, M.H., Ruff, H.A., & Fontejon, J.V. (2002). Utility of a t actile display for cueing faults. P roceedings of the Hum an Factors and Ergonomics Society 46th Annual Meeting, 2144 - 2148 . S ubjects performed a compensatory tracking task in conjunction with a monitoring task. S tudy compared the value of visual, tactile, and combined visual/tactile alerts for i dentifying which o f four scales exceeded normal range in the monitoring task. In the vi sual condition, subject was required to monitor the scales. In the tactile condition, s ubject received pulse train alert of fault, with location and frequency of train indicating which sc a le was beyond normal range of values. R esults R esponse time to faults was faster and RMS tracking error was reduced with tactile cues c ompared to visual cues. Subjective ratings also strongly preferred the tactile cues.

5) Calhoun, G.L., Fontejon, J . V., Draper, M.H., Ruff, H.A., & Guilfoos, B.J. ( 2004). Tactile versus aural redundant alert cues for UAV control applications. P

roceedings of the Human Factors and Ergonomics Society 48th Annual Meeting, 137 - 141. E xperiment 1 E xamined the value of aural and tactile alerts, presented redundantly with visual cues, to s ignal warnings during a simulated UAV control task. Subjects performed a continuousH uman Factors of UAVs 22 U AV control task. While doing this, they were required to respond to occasional (3
- 4 per m inute) data entry t asks. Two kinds of data entry task were used: 1) Warning response task Subject determined whether warning level was caution (20
- 24 pe r trial) or critical (3 per trial), then responded by entering an appropriate sequence of ke ys. 2) Radio frequency task S ubj ects heard call signs, followed by a combination of color & number. On events with call sign Eagle, subject was required to select the appropriate c olor/number coordinate on the HDD using a mouse. On low auditory load trials, only the c all sign Eagle ev e nts were used. On high auditory load trials, distractors events with di fferent call signs were interposed. 3) Data query task Simultaneous visual/voice commands specified data for subject to r etrieve from HDD and enter via keypad. P rimary manipulation o f interest was in the warning response task. In baseline condition, c aution signals were specified by yellow visual cue and redundant Type I aural cue, c ritical signals were specified by red visual cue and the Type I aural cue. In +2nd aural c ondition, caut ion signals were specified by yellow visual cue and redundant Type I aural c ue, critical signals were specified by red visual cue and the Type 2 aural cue. In +Tactile c ondition, caution signals were specified by yellow visual cue and redundant Type I aur a l c ue, critical signals were specified by red visual cue and tactile cue. R esults + 2nd aural and +Tactile conditions produced shorter RTs than baseline condition (p < .05 and p < .10). Baseline was also subjectively rated as less salient than the other tw o c onditions. +2nd aural and +Tactile were not significantly different. +2nd aural and + Tactile improved performance in the radio frequency task under high auditory load c onditions. Flight performance was not affected by alert condition. E xperiment 2 T he second experiment was conducted to examine the interaction of cue format and a

uditory load more closely. Method was similar to that the Experiment 1, except that 1) no auditory cue was used in the baseline condition, 2) only critical cues (no caution cu e s) w ere used, 3) high auditory load was more difficulty, 4) aural and tactile cues were m atched for salience by a pilot study, 5) a visual IFF status probe task was added. R esults + Aural and +Tactile conditions produced shorter RTs than baseline conditio n, and were r ated as more salient. No differences obtained between +Aural and +Baseline conditions. B aseline condition also produced higher subjective workload. Low auditory load in ge neral produced better self
- r ated SA and task performance and lower workl oa d, but did not interact with cue format, contrary to findings of Experiment 1.H uman Factors of UAVs 23 6) C ooke, N.J., & Shope, S.M. (2004). Synthetic task environments for teams: C ERTT's UAV - S TE. H andbook on human factors and ergonomics methods . Taylor F rancis. D etails ste ps involved in creating a synthetic task environment, and illustrates the process by describing the development of CERTT's Predator UAV STE.

7) C ummings, M.L. (2004). Human supervisory control of swarming networks. P

aper presented at the Second Annual A utonomous Intelligent Networked Systems C

onference . D iscusses issues related to supervisory control of swarming UAVs, i.e., groups of UAVs w ith some level of autonomous inter
- ve hicle collaboration. Collaboration between UAVs i ntroduces another layer of aut omation into UAV control task. At the minimum level of of network autonomy, there is no collaboration between UAVs. At the maximum level, ve hicles are in full collaboration and there is no need for human intervention in emergent s ituations. I ncreasing int er - ve hicle collaboration does not necessarily increase automation level for the system as a whole. At the lowest level of inter
- ve hicle collaboration, automation can r ange from SV levels 1
- 10. At highest levels, it can range from levels 7
- 10. The effects of a

utomation of full system and of inter
- ve hicle communication must be considered in s ystem design. D OD recognizes the need for a standard UAV interface that provides critical SA and l ocation data to support airspace integration. Swarming UAVs will be ta s ked to optimize multi
- obj ective cost functions, and central issue in maintaining SA will be to provide vi sualization tools that communicate cost function info to UAV operator. It will also be ne cessary to provide interactive sensitivity analysis tools to de termine how human a djustments of variables could change overall cost function.

8) Cummings, M.L., & Guerlain, S. (2004). D eveloping operator capacity estimates f

or supervisory control of autonomous vehicles . Manuscript under review. A n experiment ass e ssed operators' ability to control multiple autonomous aircraft. S ubjects performed a supervisory task that required them to control and occasionally re
t arget multiple Tomahawk missiles. Commands and occasional queries were presented in an onscreen chat box. Chat box responses served as a secondary task measure of w orkload. Retargeting was done with a decision matrix (looks like a spreadsheet) that a llowed subjects to view information on all retargetable missiles, including how long it w ould take missiles to get to target and the time remaining for the operator to retarget the m issile. Available missiles were listed in rows, potential targets were listed in columns. C ell at the intersection of a given row and column gave info about that missile/targetH uman Factors of UAVs 24 pa ir i ng. Retargeting commands arrived at two tempos, low (one event every 4 minutes) and high (one event every 2 minutes). Difficulty of task scenarios was easy, medium, or hard. D ependent variables were decision time for retargeting; Figure of Merit (FOM), a w eighted measure of overall performance; utilization, an objective workload measure gi ven by % busy time; and NASA
- T LX ratings. Participants were 42 active and retired duty Navy personnel. R esults D ecision time, FOM, and utilization scores were similar w i th 8 and 12 missiles, but were de graded with 16 missiles. The effect of number of missiles on decision time was larger a s the scenario became more difficult. Subjective workload scores were not affected by number of missiles. Results suggest that operator s can manage up to 12 missiles with no de gradation. See papers by Galster, et al (2001) and Hilburn, et al (1997) for similar c onclusions from ATC domain.

9) d e Vries, S. C. (2001). H ead - s laved control versus joystick control of a remote c amera (TNO- re por t TM - 01 - B 008) . Soesterberg, The Netherlands: TNO Human F actors Research Institute. E xperiment compared benefits of head
- s laved HMD control of UAV camera vs. joystick c ontrol. Camera joystick was either passive, active (force feedback), or combined with U AV control joystick. Dynamics of the joystick were either position or velocity control. I n some conditions, reference marks were included to aid perception of camera or ientation. R esults A nalysis of joystick manipulations indicated that best performance c ame from a passive j oystick providing position control without vehicle references. Performance on almost all m easures was superior with joystick control relative to head
- s laved control.

10) d e Vries, S.C., & Jansen, C. (2002). Situational awareness of UAV operators on board moving platforms. Proceedings HCI - A ero 2002 . A n experiment examined spatial awareness of operators controlling a UAV from onboard a moving helicopter. Displays presented a 2
- D electronic map of terrain including the U AV, helicopter, f ootball stadium, and a column of tanks. In some conditions, a 3
- D map w as presented to provide self
- m otion info from perspective of operator inside the he licopter. 2
- D maps varied in their center (heli
- c entered vs. UAV centered) and or ientation (north up vs . helicopter heading up vs. UAV heading up). The subject's task w as to monitor displays through a 40
- 60 s automated flight period then answer questionsH uman Factors of UAVs 25

a

bout locations of various items. Questions could ask about either absolute (world
c entered) orientati on or relative positions of the four items onscreen. R esults N orth
- up displays were better for absolute orientation questions, as assessed by angular j udgment errors and by RTs. In general, absolute judgments were slower than relative j udgments, except in case where map is north
- up and there was no 3
- D self
- m otion. 3
- D s elf motion increased errors in most conditions (perhaps producing an SAT in some c ases) but improved judgments of relative direction from helicopter, and had no effect on j udgments relative to the UAV.
11) Wickens, C.D., & Dixon, S. (2003). I mperfect automation in unmanned aerial v ehicle flight control (Technical report AHFD - 03 - 17/ MAD - 03 - 2) . Savoy, IL: U niversity of Illinois, Institute of Aviation, Aviation Human Factors Division. E mplo ye d the single
- U AV task of Wickens & Dixon (2002) to examine the effects of i mperfect automated aids for detecting system failures and controlling UAV flight path. S ubjects flew simulated UAV missions to a command target (CT) locations while c oncurrently s e arching for targets of opportunity (TOOs) and monitoring a set of gauges f

or system failures (SFs). In a baseline manual condition, subjects flew without a

utomated aids. Three groups of subjects were provided an aid to signal system failures. F or one grou p, aid was perfectly reliable. For another group, aid was 67% reliable and pr one to committing false alarms. For the third group, aid was 67% reliable and prone to c ommitting misses. Two additional groups were provided an autopilot to control UAV f light path. For one group, autopilot was perfectly reliable. For the other group, autopilot w as 67% reliable (i.e., prone to going off
- c ourse). A final group was provided both forms of automation, with both being perfectly reliable. R esults D ata indicated that pe r fectly reliable aids improved performance relative to baseline, and that even the imperfect autopilot was beneficial. Furthermore, automated flight control i mproved performance on the concurrent TOO search task. In contrast, imperfectly r eliable aids for S F detection produced no gains relative to baseline, and even perfectly r eliable SF detection failed to improve TOO detection. Results suggest that the benefits of later stage automation (i.e., automation of task execution) may be greater and more r obust than the benefits of early stage automation.

12) D ixon, S.R., & Wickens, C.D. (2004). R eliability in automated aids for u nmanned aerial vehicle flight control: Evaluating a model of automation dependence i n high workload (Technical report AHFD - 04- 05 /MAA D - 04- 1) . Savoy, IL: University of Illinois, Institute of Aviation, Aviation Human Factors Division.H uman Factors of UAVs 26 E mployed the single
- U AV task of Wickens & Dixon (2002) to examine the effects of an i mperfect automated aid for detection of system failures. Subjects flew s imulated UAV m issions to a command target (CT) locations while concurrently searching for targets of opportunity (TOOs) and monitoring a set of gauges for system failures (SFs). In A80 c ondition, automated aid was 80% reliable and was equally likely to co m mit a miss or a f

alse alarm. In A60f condition, aid was 60% reliable and was 3x more likely to commit a f

alse alarm than a miss. This should have encouraged high reliance/low compliance. In A 60m condition, aid was 60% reliable and was 3x more likely to co m mit a miss than a f

alse alarm. This should have encouraged low reliance/high compliance. In a baseline c ondition, subjects performed with no automated aid. R esults T racking error was unaffected by automation condition. T he number of instruction refreshe s (presented visually) was higher in the A60m (M = 8. 5) condition than in the baseline (M = 3). The number of refreshes for A80 (M = 5.57) and A60f (M = 5.25) conditions were marginally lower than in A60m condition, and w ere non - s ignificantly higher than i n baseline condition. T OO detection rate was higher in the A80 condition than in baseline. No other di fferences in detection rate between groups were significant. TOO detection times were hi gher in the A60f and A60m conditions than in baseline. Data showe d a non - s ignificant trend toward larger decrement in A60f condition, suggesting that a high false alarm rate i nduced subjects to invest more visual resources in inspecting gauges in response to an a larm than a high miss rate did. C T detection times were s i gnificantly and substantially (2 seconds) longer in A60f and A 60m conditions than in the baseline and A80 conditions. S F detection rates were higher when workload was high (i.e., during loitering/inspection), but this did not interact with automation co ndi tion. SF detection times were also higher when workload was high, and showed an interaction with condition, reflecting the fact that load increased detection times in A60f condition more than in any other condition. E ffects of load were similar on all o ther conditions. Comparison of A60f and A60m c onditions showed that in both cases, detection times were increased when automation m issed the SF. Detection times when the automation detected the failure were longer in the A60f condition than in the A60m, re f lecting greater compliance with alarms in the l ater condition. A computational model accounted for the data well. Results of the modeling suggest that c ompliance and reliance are linearly related to the automation's FAR and HR, r espectively, and are lar ge ly independent of one another.H uman Factors of UAVs 27 13) D owell, S.R., Shively, R.J., & Casey, D.P. (2003). UAV ground control station p ayload symbology evaluation. P aper presented at the Annual AUVSI Conference , Ju ly 15 - 17 , Baltimore, MD. C ompared the effects of floating c ompass rose and heading tape symbology on mission payload operators' ability to respond to change commands and SA queries. Symbology f

ormats also differed in their representation of sensor pitch: compass rose displays gave pi tch as a digital readout, head i ng tape displays depicted it with a wedge representation i ndicating the angle of declination. Commanded changes could be to sensor heading, s ensor pitch, sensor heading relative to air vehicle (sensor bearing angle), or to AV he ading. MPO did not perform A V heading changes, but called them out to confederate pi lot. Subjective measures of workload (NASA
- T LX) and SA (SART) were also c ollected. R esults C hanges to sensor heading and sensor bearing angle were more accurate with heading t ape than with compass r ose symbology, with no SAT. Unexpectedly, changes to sensor pi tch were more accurate with compass rose symbology. Post
- e xperiment interviews with s ubjects suggest this might be due to size and gradient of marked increments on heading t ape symbology. Contr ol reversals were more frequent with compass rose than with he ading tape. SA probes didn't show much, and no differences were found in subjective r atings. 14) D raper, M., Calhoun, G., Ruff, H., Williamson, D., & Barry, T. (2003). M anual versus speech inp u t for unmanned aerial vehicle control station operations. P

roceedings of the Human Factors and Ergonomics Society 47th Annual Meeting, 109 - 113. E mployed a UAV simulation to examine benefits of manual (keyboard) and speech input m odalities for intermitted data entry tasks during a continuous flight/navigation control t ask. A manual command comprised a series of button presses. A voice command c omprised a single word or short phrase. Subjects received intermittent signals to perform data entry tasks during f l ight task. A response to each alert was required within 10 s econds, and the task was required to be completed within an allotted amount of time thereafter. A failure to acknowledge alert was considered a miss, and a failure to c omplete the task was consid e red a time
- out . R esults T ask completion times were on average 40% shorter with voice commands. Benefits r anged in magnitude from 3.14 to 21.43 seconds. Number of time
- out s was almost 10 t imes higher with manual entry (M = .95 vs. M = .1), and the number of tasks completed i ncorrectly was approximately 23 times greater. Response time to alerts was faster for m anual entry mode, but difference was less than 1 second. RMS airspeed error, path e rror, and altitude error were all smaller in speech entry conditio ns . Subjective data also f

avored speech entry.H uman Factors of UAVs 28 A s study was devised, speech entry mode generally required fewer commands than m anual entry. Data do not indicate if speech entry would still be superior if modalities w ere equated for number of steps require d to execute commands.
15) D raper, M.H., Geiselman, E.E., Lu, L.G., Roe, M.M., & Haas, M.W. (2000). D isplay concepts supporting crew communications of target location in unmanned ai r vehicles. P roceedings of the IEA 2000/ HFES 2000 Congress , 3.85 - 3.88 . U AVs for intelligence, reconnaissance, and surveillance (ISR) usually have two ope rators, a sensor operator (SO) and an air vehicle operator (AVO). The AVO controls the airframe, monitors subsystems, and communicates with the ground control station ( GCS)
. The SO searches for targets using a UAV
- m ounted camera. T he AVO generally views scene with a larger FOV than the SO, and can therefore assist i n target detection by directing the SO's attention to targets outside the SO's current FOV. U sually, the AVO attempts to communicate the target location verbally. The goal of pa per was to assess a pair of display concepts meant to facilitate AVO/SO c ommunication about target location. Two kinds of advanced displays were tested: • C ompass rose overlay on the SO's cam e ra display Allows AVO to give direction i n world
- c entered references (N, S, E, W), and should make translation to screen
c entered references easier for the SO • T elestrator Allows AVO to designate target location on his display using a mouse, then presents a locator line on the SO's display indicating the direction and di stance in which SO should shift camera to find target F our conditions were tested: baseline (control), compass rose, telestrator, compass rose + t elestrator R esults T elestrator reduced the time necessary to designate the target, improved camera path e fficiency improved, and reduced workload. Compass rose was of little benefit. 16) D raper, M.H., Nelson, W.T., Abernathy, M.F., Calhoun, G.L. (2004). S ynthetic vision overlay for improving UA V operations. T he authors discuss potential benefits of synthetic vision systems (SVSs) for UAVs. T hese include: • S VS could improve SA by highlighting items of interest in camera image. • S VS could allow operator to maintain SA if visual datalink is lost. • S V S could facilitate communications between users who are not co
- l ocated.H uman Factors of UAVs 29 17) D raper, M.H., Ruff, H.A., Fontejon, J.V., & Napier, S. (2002). The effects of h ead - c oupled control and a head - m ounted display (HMD) on large - ar ea search t asks. P roceedings of t h e Human Factors and Ergonomics Society 46th Annual M eeting , 2139 - 2143. C ompared effects of various head
- c oupled and manually
- c ontrolled camera/display c onfigurations on ability to locate targets in a 360
- de gree search task in a simulated U AV. Target acqui s ition was better with manual joystick/stationary CRT combination than with head
- c oupled HMD configurations. Workload ratings, SA ratings, and s imulator sickness data also generally favored the non
- H MD configurations.

18) D raper, M.H., Ruff, H.A., & La F leur, T. (2001). The effects of camera c ontrol and display configuration on teleoperated target search tasks. P roceedings of t he Human Factors and Ergonomics Society 45th Annual Meeting, 1872 - 1876 . S ubjects performed forward
- f ield and rear
- f ield search ta s ks in a UAV simulation using e ither A) joystick controlled camera with stationary CRT display, B) 1.0x gain head c oupled camera with HMD, C) 1.5x gain head coupled camera with HMD, or D) 1.0x ga in head coupled camera with HMD in conjunction with combined w ith manual joystick c ontrol. R esults C onfiguration A produced best performance for forward
- f ield search. Data showed no s ignificant differences between configurations for rear
- f ield search.

19) F ong, T., & Thorpe, C. (2001). Vehicle teleoperation i n terfaces. A utonomous R obots, 11 , 9 - 18. P aper discusses various forms of interfaces for vehicle teleoperation. These include: • D irect Operator manually controls aircraft, typically using controls that are s imilar to those of a manned vehicle. This form of c ontrol/interface is appropriate when 1) real
- t ime human control or intervention is required, and 2) the control s tation and vehicle are connected by a high
- bandwidth, low
- de lay c ommunications link. • Multimodal/multisensor Multimodal interfaces "provide the ope rator with a variety of control modes (individual actuator, coordinated motion, etc.) and di splays (text, visual, etc.)...Multisensor displays combine information from s everal sensors or data sources to present a single integrated view." • S upervisory control Operator specifies subtasks which the vehicle then performs on its own. This is appropriate when datalink bandwidth is low or c ommunications are delayed. • N ovel These include psychophysiologically
- dr iven control, gesture
- ba sed control, w eb
- ba sed inte r faces, PDA
- ba sed interfaces.H uman Factors of UAVs 30

20) F ontejon, J., Calhoun, G., Ruff, H., Draper, M. & Murphy, K. (2004). Tactile al erts for monitoring tasks in complex systems. A n earlier study (Calhoun, et al, 2002) found that tactile alerts could speed detection of s y s tem faults in a multi
- t ask environment. In that experiment, subjects were required to de tect & identify system faults while also performing a manual tracking task. Two tactors w ere used to signal four possible system faults: combination of tactor location and vi bration frequency signaled which of four system parameters was in fault. Performance w as best (RT shortest) when one tactor was located on each arm. When both were on a s ingle arm, performance was better with the right than the left arm. I n the stu dy described above, all participants were right
- handed. Additionally, manual tracking was performed with the right hand. The current study was conducted to de termine if similar results would obtain for A) left
- handed subjects, and B) when subject pe rformed the tracking task using the left hand. R esults RTs were shortest when tactors were located on different arms. When they were on the s ame arm, there was no significant difference in RTs for left & right arms. Hand used for tracking did not have any affect on RT to faults. 21) G awron, V.J. (1998). Human factors issues in the development, evaluation, an d operation of uninhabited aerial vehicles. A UVSI '98: Proceedings of the A ssociation for Unmanned Vehicle Systems International , 431 - 438 . T he author disc us ses a number of unique human factors concerns unique to UAV flight. T hese include: • D ata link drop outs may be difficult for operator to notice. • U AV mission times may exceed human vigilance capability. • H umans can attend to/inspect only one stream of image s at a time, while some U AVs may provide multiple image streams. • O perators are sometimes given with unprioritized lists of multiple of targets to s earch for. This may be especially problematic when the operator is asked to c ontrol multiple UAVs simultaneou s ly. • C rew coordination depends on appropriate communications flow between crew m embers, which can be difficult when crew is large or when crew members are not co - l ocated. • V isual imagery is difficult to obtain during rocket launching or UAV, and during ne t or cable arrest. Workload is also high during launch and recovery. Finally, s mall sensor FOV can reduce SA and make navigation, target acquisition, and traffic detection difficult. • M anual control of vehicles with time delays is difficult. • C ontrol interface on some systems is poorly designed. • S oftware is not standardized, even between instances of the same UAV system.H uman Factors of UAVs 31 P roposed military uses for UAVs include special operations; point reconnaissance, cued s urveillance, and target acquisition. Non
- m ilitary use s are possible in the fields such as l aw enforcement, fire fighting, agriculture, construction, archaeology, geology, and postal de livery.
22) G luck, K.A., Ball, J.T., Krusmark, M.A., Rodgers, S.M., & Purtee, M.D. ( 2003). A computational process model o f basic aircraft maneuvering. P roceedings of the Fifth International Conference on Cognitive Modeling, 117 - 122 . P aper presents an ACT
- R model of Predator UAV flight. The model is based on s imulation used to train Air Force Predator operators. The simulatio n involves three t asks: basic maneuvering, landing, and reconnaissance. The modeling effort presented in this paper focuses on basic maneuvering. Pilot is required to make constant
- r ate changes i n airspeed, altitude, and heading. A total of seven maneuvers are involved. The first three require pilot to change one flight parameter and hold the other two constant. The s econd three maneuvers require pilot to change two flight parameters and hold third c onstant. The seventh maneuver requires the pilot to change all three flight parameters s imultaneously. T he model uses a flight strategy called "Control and Performance Concept". First, the ope rator establishes appropriate control settings for desired performance. Next, the ope rator cross checks instruments to de termine if the desired performance is being a chieved. The rationale is that control instruments have a first
- o rder effect on aircraft be havior, which shows up only as a second
- or der effect in performance instruments. R esults RMSD for airspeed, altitude , and heading were normalized and summed to provide an ove rall measure of performance. Grand mean performance on this measure over 20 runs of the model was almost identical to grand mean performance of 7 subjects. Across m aneuvers, r - s quared for predicting human performance from model was .64. The model w as also sensitive to maneuver complexity in the same way that human subjects were, s howing better performance for one
- axis maneuvers than for two axis
- m aneuvers and be tter performance for two
- axis maneuvers than for three
- axis maneuvers. 23) G orman, J.C., Foltz, P.W., Kiekel, P.A., Martin, M. J., & Cooke, N. J. (2003). E valuation of latent - s emantic analysis - b ased measures of team communications. P

roceedings of the Human Factors and Ergonomics Society 47th A nnual Meeting , 424 - 428. T he authors used Latent Semantic Analysis to develop methods of assessing c ommunications content between team members in a Predator UAV simulation. M easures used were communications density (CD), the average task relevance of the t e am's communications, and lag coherence (LC), a measure of task
- r elevant topic
s hifting. Data came from two experiments in which teams of three
- ope rators (air vehicle ope rator, payload operator, and navigator) flew simulated Predator UAV reconnaissanceH uman Factors of UAVs 32

m is s ions. In the second experiment, workload levels were manipulated (low vs. high) and s ome teams of operators were distributed rather than colocated. R esults C ommunications density T eam performance in Experiment 1 was related to CD by a quadratic function, indicating that beyond some point performance declined with additional communication. Similar r esults were found for co
- l ocated and low
- w orkload teams in Experiment 2. Under high
w orkload conditions, performance continued to increase as CD increased, show i ng no e vidence of a quadratic trend or an optimal CD level. Data from distributed team c onditions was too noisy to interpret clearly. L ag coherence C oherence was positively correlated with team performance, indicating that low
pe rforming teams tend to sh i ft topics more within a short window than high
- pe rforming t eams.

24) Gugerty, L. & Brooks, J. (2004). Seeing where you are heading: Integrating e nvironmental and egocentric references frames in cardinal direction judgments. Jou rnal of Experimental Ps y chology: Applied, 7 , 251 - 266. N avigational tasks often require operators to make cardinal direction judgments, which data suggest are difficult. The goal of the experiments reported here was to examine the s trategies by which people make cardinal directio n judgments. E xperiment 1 S ubjects performed a static judgment task. Stimuli each trial were A) a north
- up map s howing their aircraft and a footprint of a forward
- f acing vehicle
- m ounted camera, and B ) a 3
- D view of the terrain as seen through the vehicle
- m ounted camera. The view pr esented each trial contained a building with a parking lot on each side (N, S, E, W). O ne of the parking lots contained vehicles while the others were empty. The subjects' job w as to indicate the cardinal direction of the occupie d parking lot, relative to the building. R esults T hree noteworthy effects were evident in both the error rate and the RT data. The first w as a misalignment effect, whereby performance declined as camera heading deviated f rom north
- up . The second was a south
- a dvantage effect, whereby performance was s ubstantially better when camera was oriented south than when it was at nearby or ientations. The third was cardinal
- di rection advantage effect, whereby judgments were m ore slightly accurate when the camera was or i ented east or west than when it was at ne arby orientations. E xperiment 2 T he goal of experiment 2 was to determine if dynamic spatial information, such as that pr ovided by controlling a vehicle, improves cardinal direction judgments. The dynamicH uman Factors of UAVs 33
t ask use d was a simulated UAV mission. Subjects were provided three visual channels, A ) a north
- up map similar to that used in Experiment 1, B) a 3
- D view of terrain from a r otatable vehicle
- m ounted camera, similar to that used in Experiment 1, and C) a standard f l ight display. The subjects' task was to pilot the UAV to a 10 target objects and answer que stions about each one. Three of the 10 questions required cardinal direction j udgments. S ubjects also performed a static judgment task identical to that of Experim e nt 1. R esults S tatic judgment task replicated the results of the first experiment. Cardinal
- di rection j udgments in the dynamic task showed effects similar to those of the static task, though the cardinal direction advantage was weaker. E xperiment 3 S ubj e cts performed the cardinal direction task of Experiment 1 and 2 while providing ve rbal protocols. R esults P rotocols gave evidence for final strategies. The first was a mental rotation strategy, whereby subjects mentally transformed images to be in alignm ent with one another and north
- up. The second was a heading referencing strategy, whereby subjects assigned the c urrent heading to the forward view in the camera, then making judgments relative to that he ading ("If forward is northeast, then this is north [pointing to the upper left lot], and this is east [pointing to the upper right lot]." The third was a south
- r eversal strategy, whereby subjects noted that camera heading was south and then reversed the answers they would have given for a northward headin g (this was possible only when camera was or iented toward the south, obviously). The final strategy was a north
- he ading strategy, whereby subjects noted that camera was oriented to the north and then simply made j udgments within a canonical north
- up frame.
25) G unn, D.V., Nelson, W.T., Bolia, R.S., Warm, J.S., Schumsky, D.A., & C orcoran, K.J. (2002). Target acquisition with UAVs: Vigilance displays and ad vanced cueing interfaces. P roceedings of the Human Factors and Ergonomics Society 46th Annual Meeting, 1541 - 1545 . T he authors note that UAV operators will probably spend much of their time in s upervisory control mode, but will be required to switch to manual control suddenly in r esponse to system malfunctions, target acquisition, enemy actions, and other i ntermittent e vents. As such, UAV operation will be a form of vigilance task. The goals of study were A ) to examine value of display type (cognitive or sensory) on performance in a vigilance t ask and in subsequent manual control mode, B) to compare the eff e cts of visual, a

uditory, and haptic cueing of target location in a 360 degree target acquisition task.H uman Factors of UAVs 34 S ubjects flew simulated UAV missions. In supervisory control mode, they were required t o monitor a stream of digit pairs for a threat warning indicatin g the presence of an enemy a ircraft. In the sensory task, the threat warning was signaled by a size difference between the two digits. In the cognitive task, the threat warning was signaled by an even
- odd digit pa iring. After detecting a threat warning, th e subject was required to target the hostile a ircraft with a joystick
- c ontrolled crosshair. In the visual cueing condition, a locator line on the bottom right of screen indicated the direction of the target. In the auditory cueing c ondition, broadband nois e pulses were presented from the target location. In the haptic c ueing condition, force feedback on control stick guided the subject toward the target. In the control condition, no cueing was provided. R esults H it rates for warnings showed no effect of si gna l rate, but a significant benefit of sensory di splay format relative to the cognitive format. False alarms were lower for cognitive than for sensory displays. Target acquisition times were shorter for sensory displays than f

or cognitive. Visual, auditory , & haptic cue conditions produced similar benefits in t arget acquisition times, all of which were shorter than in control condition. Subjective w orkload was higher with cognitive than with sensory displays. 26) H ansman, R.J., & Weibel, R.E. (2004). Pan e l 1: UAV classification thoughts. P

aper presented at NRC Workshop on UAVs . P roposes a classification scheme for UAVs in NAS. H igh altitude, long endurance • Above FL 500, above majority of commercial air traffic • P otential applications include long
- dw ell mi s sions such as communications relay, pr ecision mapping/imaging, and atmospheric research M edium altitude endurance • F L 180
- FL 500, Class A airspace • P otential applications include meteorology, disaster monitoring, border patrol, and regional mapping T act i cal • 1000 to FL 180/10,000 ft., mixed airspace • P otential applications include law enforcement surveillance, pipeline/rail m onitoring, search and rescue, agriculture M ini • B elow 1200/700 ft. AGL • - - P otential applications include law enforcement, local imagery, and c inematography M icro • B elow 1200/700 ft. AGLH uman Factors of UAVs 35

• P otential applications include recreation, and local imagery R otorcraft • Up to 2000 ft. AGL • P otential applications include search & rescue, law enforcement, traffic m onitoring, cinematography, and agricul ture 27) H ansman, R.J., & Weibel, R.E. (2004). Panel 2: Operating and flight rules. P

aper presented at NRC Workshop on UAVs . P resents safety analyses of UAV, deriving acceptable failure rates (mean numbers of hour s until failures) for varying classes of UAVs. Note a number of safety issues for U AVs operating under instrument and visual flight rules. I FR• C ontrol latency • C ommunication paths • C ontroller workload and representation • S eparation standards • T raffic load • F light plan filing • C ost recovery V FR • S ee and be seen equivalence • R ules of the road • O peration at controlled and uncontrolled airfields 28) Kiekel, P.A., Gorman, J.C., & Cooke, N.J. (2004). Measuring speech flow of c o - l ocated and distributed command and control teams during a communication c hannel glitch. P roceedings of the Human Factors and Ergonomics Society 48th A nnual Meeting , 683 - 687. A n experiment used communication flow measures developed by the authors in earlier pa pers to examine effects of co
- l ocation and communication channel disruptions on team c ommunications in a simulated UAV recon task. Teams of three members flew a s imulated UAV, taking pictures of target items. Each team comprised three members: the data exploitation, mission planning and control (DEMPC) member planned the route, air ve hicle operator (AVO) flew the aircraft, and the payload operator (PLO) controlled the c amera and took pictures. Manipulations were A) teams colocated vs. teams distributed, B ) workload high vs. low (workload effect not further discussed in this paper), and C) c ommunications normal or disrupted by glitch in channel from DEMPC to AVO. T hree types of analysis were conducted. The first used Pathfinder algorithm to identify c ommon sequences of communications events: XLoop (person X begins and ends a c ommun i cation, then begins another), XYcycle (person X produces a completeH uman Factors of UAVs 36

c ommunication, then person Y does), and XiY (person X interrupts person Y). CHUMS analysis measured the stability of communications, as reflected in the relative proportion of speech prod uc ed by each member in a one
- m inute window. Analysis of dominance m easured the influence that each team member's communications had over other m ember's. E xpectation was that occurrence of glitch would modify communication pattern, that D EMPC should have h i gh dominance score, and that dominance of DEMPC should drop i n distributed teams and when glitch occurs. R esults P athfind analysis found that colocated teams produced more utterances in general than di stributed teams. Glitch causes decrease in DAcycles ( c ommunications between D EMPC and AVO), increase in DPcycles, increase in PAcycles, and decrease in P Dcycles. This is generally what would be expected if communications that normally w ould have gone from DEMPC to AVO were re
- r outed through the PLO following the gl itch. C HUMS analysis produced more models for distributed teams, suggesting less stable c ommunications patterns. The communications glitch also reduced stability. A nalysis of dominance found that in co
- l ocated teams under normal conditions, the D E M PC is moderately dominant and the AVO is reactive. In distributed teams under normal conditions, AVO is distributed and DEMPC is reactive. During the c ommunications glitch, co
- l ocated teams become AVO dominant and PLO reactive. 29) L aFleur, T., Draper, M.H., & Ruff, H.A. (2001). Evaluation of eye d ominance effects on target - ac quisition tasks using a head - c oupled monocular H MD. P roceedings of the Human Factors and Ergonomics Society 45th Annual M eeting , 1433 - 1437. S ubjects performed a target acquisition t ask in a UAV simulation. A large FOV display w as presented with a monocular HMD. Image was provided by the UAV's gimbaled c amera, with camera control by operator's head movements. After spotting a potential t arget in the HMD, the operator was required to I D and designate it on a high
- r esolution, s mall FOV CRT display with camera view controlled by joysticks. Several dependent variables recorded. Of primary interest were effects of dominance of HMD viewing eye on performance. R esults E ye - dom inance had no e f fect. 30) M iller, C.A., Funk, H.B., Goldman, R.P., & Wu, P. (2004). A "playbook" for var iable autonomy control of multiple heterogeneous unmanned air vehicles.H uman Factors of UAVs 37 P

roceedings of the Second Human Performance, Situation Awareness, and A utomation Conference ( H

PSAA II) , Daytona Beach, FL, March 22 - 25 . D iscusses "delegation" as a technique for control of multiple UAVs. Characteristics of de legation are that supervisor sets agenda for subordinates, but subordinates are given a

uthority to decide exactly how to carry out commands. The authors note five m anners/components of delegation that can be employed in varying combinations: 1. Stipulation of goal 2. Stipulation of a plan to perform 3. Stipulation of constraints (via specification of actions or states that shou l d be avoided) 4. Stipulation of actions or states to be achieved (i.e., subgoals) 5. Stipulation of an objective function that will allow the subordinate to assess the de sirability of various states and actions T he authors describe their work on developi ng a "playbook" architecture for delegating to U AVs. Playbook would involve assigning a name to complex behavior patterns, then a llowing UAVs to autonomously implement a play when it is called. The Playbook s ystem would assess the feasibility of a commande d behavior before attempting to pe rform it. When given a high
- l evel command, Playbook would assess various methods of achieving goal, then would issues specific commands to vehicles under its control. When given more highly
- s pecified lower
- l evel commands, P laybook would report to the hum an operator if the commands were infeasible, or would issues the commands to the ve hicles if they were feasible, filling in additional details as necessary. 31) M orphew, M.E., Shively, J.R., & Casey, D. (2004). Helmet moun t ed displays f

or unmanned aerial vehicle control. P aper presented at the International Society for O

ptical Engineering (SPIE) conference , April 12 - 16, Orlando, FL. C ompared performance on a target search & ID task when subjects used a conventional C RT disp l ay & joystick control versus when they used a head
- s laved HMD. UAV flight w as automated. Subjects' task was to search for items in the virtual world display and ID them as target, non
- t arget, or distractor. After spotting a target or non
- t arget, subject w a s t o center crosshairs on the item and press a button on control box to classify it. I ndependent variables were method of display/control (CRT/joystick vs. HMD/head
s laved), virtual world search width (2500 vs. 5000 ft.), and mission duration (3 vs. 9 m inut es). Dependent variables were target detection accuracy (HR, CRR), cursor distance ( distance of crosshairs from center of object when object was classified), slant range ( distance from aircraft at which subjects were able to classify target), R esults A c curacy was high (>98%) for both forms of display/control. However, cursor distance w as smaller and slant range was larger (i.e., performance was better in both cases) for the C RT/joystick configuration. HMD configuration also produced higher levels of nau s ea, e yestrain, disorientation, and over simulator sickness rating than the CRT configuration. W ide search width produced smaller cursor distance than did narrow width, but effect did not interact with any others.H uman Factors of UAVs 38

32) M ouloua, M., Gilson, R., Daskaroli s - K ring, E., Kring, J., & Hancock, P. ( 2001). Ergonomics of UAV/UCAV mission success: Considerations for data link, c ontrol, and display issues. P roceedings of the Human Factors and Ergonomics Society 45th Annual Meeting, 144 - 148 . L ists a number of consid e rations and recommendations for optimal design of U AV/UCAV system interface and data transfer. D ata - l ink timing I f satellite
- U AV or UAV
- U AV relays are used, variable time delays of 1 second or l onger are possible. This eliminates real
- t ime feedback to co ntrols inputs. One way of c ircumventing this problem is to task operator with supervisory monitoring of on
- board a

utomation using pre
- p rogramming flight parameters such as speed, altitude, direction. P redictive graphics displays may also be useful. C ontr ol s N either full automated control nor full manual control is practical. Full automation pr events the operator from intervening in flight control when necessary, while full m anual control can produce excessive workload and make control susceptible to c ommuni cations delays. The authors recommend hybrid control in which human operator s upervises automation by calling subroutines of pre
- pr ogrammed software. D isplay/control interfaces should be based on a standardized group of core functions de scribed with co m mon terminology. Keyboards, touchscreens, pointing devices, and j oysticks/pedals are appropriate controls, but must be designed to resist dirt, damage, etc., e specially for field operations. Keyboard inputs should be replaced with menu
- or s peech
- inputs for on
- l ine vehicle control. A ssuming semi
- a utomatic flight, flight
- m anagement systems and terminology should e mulate that of ARINC and DOD. Since commands are anticipatory, this approach allows f

or preview and escape actions. If manual flight control is used, a GCS with full
- t ime j oystick/pedal/power controls will be necessary, and real
- t ime communication with U AV/UCAV will be required. I f menus are used for in
- f light supervisory control, it will be necessary to determine opt imal number of menus and me nu items. Five seems like reasonable starting point, ba sed on Miller's magic number. D isplays D isplays need to reduce and format data for easy interpretation. Other principles to f

ollow include minimizing scene movement & unnecessary changes in viewpoint ; using hi gh
- qua lity displays to help ID areas of interest; employing alerts/alarms for system f

aults; providing mechanisms of selection, comparisons, parsing, scaling of displayed i nfo.H uman Factors of UAVs 39

I n addition to high
- qua lity sensor
- i mage displays, content should in c lude: • s ystem conditions and communications status • f light data • threat advisories • w eapons status 33) M ouloua, M., Gilson, R., Kring, J., & Hancock, P. (2001). Workload, s ituation awareness, and teaming issues for UAV/UCAV operations. P roceedings of t he Hu m an Factors and Ergonomics Society 45th Annual Meeting, 162 - 165 . T he authors discuss UAV design considerations relevant to workload, SA, and teaming. S ome of these issues pertain specifically to combat UAVs. P hysical & cognitive workload A ssuming that the UAV control will be highly automated, then the operator's task will c onsist primarily of supervisory monitoring and making small course adjustments. This is l ikely to be tedious, producing vigilance failures. UAV systems must therefore be de signed not jus t to avoid overload, but underload as well. This might be done by c ombining manual and automated control. Operator would be responsible for higher
or der tasks (target recognition, munitions deployment) and automation would be r esponsible for lower
- or der ta s ks (flight control, obstacle avoidance). On
- board a

utomation should require operator action only as needed. S ituation awareness P oor SA is likely to contribute to UAV mishaps. One way to help maintain SA is to pr ovide displays that keep the operator awar e of the processes being controlled by the a

utomation, with the goal "to make the deep relational structure of the system e nvironment visible to operators and help to identify options for action and indicate the boundaries for successful performance." UAVs may also be able have and advantage r elative to manned systems in providing good SA since large numbers of on
- boa rd and of f

- board sensor data streams are available. T eaming R esearch is needed to determine the appropriate crew size and structure for UAV c ont rol, and to ensure effective communications. One particular source of miscommunication is the large amount data provided to the various UAV operators. This demands that i mportant information be shared appropriately among operators. Ways to do this inclu de " creating a mechanism for communicating understanding the real
- time situation at a hi gher level across several connected teams or individuals", creating teams of specialists f

or target detection/authentication and for emergency operations. 34) N elson, W. T., Anderson, T.R., McMillan, G.R. (2003). A lternative control t echnology for uninhabited aerial vehicles: Human factors considerations . Book c hapter.H uman Factors of UAVs 40 D iscusses potential alternative control technologies for UAVs. These include position and orientation tracking, eye
- pos ition tracking, speech recognition, gesture recognition, and electrophysiological measures. The authors advocate increasingly immersive e nvironments for UAV pilots, with eventual possibility that alternative control t echnologies will repl a ce traditional controls. Possible impediments to these goals include t ime delays in display updating, simulator sickness. 35) Nelson, J.T., Lefebvre, A.T., & Andre, T.S. (2004). Managing multiple u ninhabited aerial vehicles: Changes in number of vehicl e s and type of target s ymbology. I nterservice/Industry Training, Simulation, and Education Conference ( I/ITSEC) . T he authors describe an experiment conducted with two goals. The first was to examine c hanges in performance that result from increasing the number of UAVs under an ope rator's supervision. Contrary to expectations, past research (Draper, Calhoun, & Ruff, 2003) found limited performance consequences when the number of UAVs under a single ope rator's supervision increased from 2 to 4. The present e xpe riment compared pe rformance using 3 and 5 UAVs, in an effort to increase the workload demands of the hi gher
- l oad condition. The second was to compare performance with a novel set of s tylized icons to performance with a set of standardized icons (MIL
- S TD _2525B ). The a

uthors hypothesized that the stylized icons, designed to have a physical resemblance to the objects they represented, would produce better performance. A fter training, subjects flew two missions in the Multi
- m odal Immersive Intelligent I nter f ace for Remote Operation (MIIIRO). One mission involved control of 3 UAVs, the other involved control of 5 UAVs (order of missions counterbalanced). Flight control was a

utomated. Subjects were responsible for several additional tasks: identifying unknown a ircraft, approving replans of UAV routes, identifying and selecting targets in the i magery from UAV sensors, completing tasks on a mission mode indicator, and counting s ymbols on the Tactical Situation Display (top
- do wn map of terrain with UAVs and r oute s depicted) for later recall. Subjective workload measures were also collected. R esults D ata showed no difference between the 3 UAV and 5 UAV conditions in the number of e nemy targets identified and selected. Other dependent variables showed significant e f

fects favoring the 3 UAV condition. Specifically, the time to respond to unidentified a ircraft was shorter, the number of Mission Mode Indicator tasks completed was higher, the time to check and approve UAV replan routes was shorter, and all subjective pe r formance measures (ratings of situation awareness, perceived task difficulty, pe rceived performance, and perceived workload level) were better. R ecall of symbols on the TSD was better with the standardized icons than with the s tylized set. The authors sp e culate that this might have happened because the s tandardized icons were easier to perceptually segregate from the background.H uman Factors of UAVs 41 36) P urtee, M.D., Gluck, K.A., Krusmark, M.A., Koffe, S.A., & Lefebvre, A.T. ( 2003). Verbal protocol analysis for validation o f UAV operator model. P roceedings of the 25th Interservice/Industry Training, Simulation, and Education Conference , 1741 - 1750. U sed concurrent and retrospective verbal reports from subject matter experts piloting a P redator UAV simulation to determine how a ccurately the lab's ACT
- R model of P redator pilot performance represents the cognition/information processing of actual pi lots. R esults O verall, attention to performance instruments was verbalized more often than attention to c ontrol instruments during c oncurrent reports. However, retrospective reports suggested that SMEs were using the Control and Performance concept implemented by the model. R esults also demonstrate that distribution of operator attention, as reflected in concurrent ve rbal reports, is i nfluenced by goals/demands of maneuver being implemented. Ideas for i mproving the cognitive model include incorporating use of trim and a metacognitive awareness of passage
- of
- t ime to improve use of timing checkpoints for monitoring pr ogress toward goal.
37) Quigley, M., Goodrich, M.A., & Beard, R.W. (2004). Semi - au tonomous human U AV interfaces for fixed - w ing mini - U AVs. Proceedings of IROS 2004 , Sep 28 – Oct 2, Sendai, Japan . T he paper describes work prototyping and testing several forms of interface f or control of s mall (32" wingspan, in this case) semi
- a utonomous UAVs. C ontrol techniques Numeric parameter
- ba sed interfaces provide text boxes in which operator types desired f light parameters. P DA direct manipulation interface presents fixed
- h orizon w i ng - vi ew representation from the viewpoint of an observer behind the UAV. Display also includes a compass and s peedometer alongside the wing
- vi ew display. The user controls the UAV through drag
and
- dr op manipulation of the UAV icon or the compass/speedom e ter. Color differences ( blue vs. red) are used to distinguish current state of UAV from desired state specified by us er manipulation. This makes the future state of the UAV easy to predict. This interface w as also tested with a laptop using trackpad and m ous e. V oice controller is allows UAV control using a grammar of twenty words (e.g., "climb", " go north"). Voice synthesizer provides immediate feedback in present progressive tense ( e.g., "climbing", "going north").H uman Factors of UAVs 42 A ttitude joystick controller mimics a f ly
- by
- w ire controller by mapping deflection in j oystick to deflection in aircraft attitude. This form of interaction is especially good for novi ces, non - e xperience pilots. T rackpoint controller uses a trackpoint pointing device from ThinkPad laptop, with hor izontal inputs mapped to UAV roll and vertical inputs mapped to UAV pitch. T he physical icon interface is a hand
- he ld model of UAV. Orientation of model in 3
- D s pace is tracked, and converted into pitch / roll commands for UAV. Should provide good S A since the operator is holding a representation of UAV in its actual orientation. An a ccompanying (optional) display helps the user distinguish actual and desired UAV states by presenting the current state in one color and the desired state (i.e., state of physical i con control) in another. A ssessments T he attitude joystick, physical icon, and trackpoint controller produce the fastest operator r esponse times for UAV manipulations. However, these re problematic when UAV is traveling toward the operator, si nc e they require reversal of control inputs. D irect manipulation interfaces are useful because they don't required sustained attention. A fter the user has specified the desired UAV state, no further interaction is necessary unt il another change of state i s desired. This can make these interfaces especially a

ppropriate for control of multiple UAVs. The only difficulty is that subject must shift ga ze between interface and UAV. This creates demand for frequent accommodation c hanges. V oice control allows oper ator to maintain gaze on UAV, and, assuming commands can be issued in world
- c entered coordinates (e.g., "go north"), does not require mental transformation of commands depending on heading of UAV. P arameter based interfaces are difficult because they requ i re operator to type, and also de mand mental math.

38) R uff, H.A., Calhoun, C.L., Draper, M.H., Fontejon, J.V., Guilfoos, B.J. ( 2004). Exploring automation issues in supervisory control of multiple UAVs. P

roceedings of the Second Human Performance, Situ ation Awareness, and A utomation Conference (HPSAA II) , Daytona Beach, FL, March 22 - 25 . E xperiment 1 E xamined the effects of automation on task completion time and subjective workload l evels in control of 2 or 4 UAVs. Two levels of automation (LOA) were c ompared, m anagement by consent (MBC) and management by exception (MBE). Automation was pr ovided for two tasks, image prosecution and route re
- pl ans. Time limits for overridingH uman Factors of UAVs 43

a

utomation in the MBE condition were 40 s for image prosecution and 15 s for rout e re pl an. Automation reliability was either 75% or 98% for image prosecution, 75% or 100% f

or route re - p lan. O verall task involved four sub
- t asks 1. Responding to unidentified aircraft 2. Route re
- pl anning 3. Image prosecution 4. Monitoring Mission Mode Indicator (MMI) R esults A n increase in number of UAVs increased completion times for image prosecution, route r e - pl anning, and MMI monitoring, and also increased the amount of time was spent in threat zones. Subjects were more likely to respond before MB E automation acted (i.e., s ubjects were more likely to carry out the automation's recommendation than to simply l et the automation carry out the action itself) when there were 2 UAVs than when there w ere 4. Subjective workload ratings were higher for 4 UAV s than for 2. F ewer images were prosecuted and more errors committed when automation was less r eliable. Subjective trust ratings were also lower. N o consistent differences obtained between MBC and MBE . This might be because the t ime limit for overriding MBE automation was too long. As noted above, subjects often pr e - e mpted the automation under MBE by carrying out the recommended action themselves. Expt 2 therefore reduced time limits for MBE. E xperiment 2 Automation reliability and LOAs were same as in E xpt 1, but the time limit for ove rriding MBE was now included added as a dependent variable. Time limits for image pr osecution were 15 and 40 seconds. Time limits for route re
- pl an were 10 and 15 s econds. Execution of route re
- pl an task was also made more difficult. Only the 4 UAV c ondition was used. R esults O nly one measure, the percentage of images correctly prosecuted, showed an effect of a

utomation reliability. W ith MBC, subjective ratings were similar across time limits. With MBE, shorter time l imi t s produced higher workload ratings and poorer performance on both tasks. Subjects w ere faster to complete tasks in MBE/short time limit condition than in any other c ombination of automation/time limit, suggesting that they felt time pressure. This may hav e contributed to higher subjective workload. MBE automation was more likely to e xecute action in short time limit condition.H uman Factors of UAVs 44 39) R uff, H.A., Draper, M.H., Lu, L.G., Poole, M.R., & Repperger, D.W. (2000). H aptic feedback as a supplemental method of alert i ng UAV operators to the onset of t urbulence. P roceedings of the IEA 2000/ HFES 2000 Congress , 3.41 - 3.44. U AV operators are denied many of the sensory cues available to the pilot of a manned a ircraft. One instance in which this might be consequential is in detecting turbulence. D uring flight of a manned aircraft, the onset of turbulence typically produces ki nesthetic/haptic feedback. During UAV flight, turbulence is signaled to the operator onl y by perturbations of camera image. C urrent experiment measur ed value of haptic alert (via control stick) for detection of turbulence onset. Participants flew simulated UAV landings. When turbulence occurred, s ubjects rated their level of SA. After each trial, participants rated the difficult of di fficulty, assesse d their performance, and judged the strength (mild or severe) and axis of perturbation (horizontal or vertical) of the turbulence. Note that the multimodal display di d not mimic the haptic signals experienced in real flight, but was simply meant as an a lert ing signal. R esults H aptic feedback improved SA ratings, but if when turbulence occurred when UAV was f

ar from the runway. When UAV was near runway, no benefit of feedback. Authors s uggest that heightened alertness near runway might facilitate turbulence detection, m itigating the effects of haptic feedback. RTs for turbulence detection would have pr ovided useful data to test this hypothesis. S ubjective ratings of landing performance were unaffected by haptic alert, but ratings of l anding difficulty incre a sed when haptic alert was provided. Perceived turbulence s trength and judgments of turbulence direction were unaffected by haptic feedback.

40) Ruff, H.A., Narayanan, S., & Draper, M.H. (2002). Human interaction with l evels of automation and decision - ai d fidelity in the supervisory control of multiple s imulated unmanned air vehicles. P resence, 11 , 335 - 351 S ubjects flew simulated UAV missions, with task of acquire four targets at unknown l ocations (3 enemy and 1 friendly) while avoiding enemy fire. Numbe r of UAVs was 1, 2, or 4. Flight path was preprogrammed. Subjects were required to respond to/manage e vents as they occurred through course of scenario. Automation was provided to some s ubjects to help manage events. Two forms of automation were used, in addition to the no - a utomation baseline: management by consent, and management by exception. Two l evels of automation reliability were used: 100% and 95%. R esults M anagement by consent produced the highest level of mission efficiency (number of e nemy targ e ts destroyed divided by number of missiles fired). Management
- by
- e xceptionH uman Factors of UAVs 45

and manual monitoring produced similar efficiency scores. Decision aid false alarms in the 95% reliability automation condition were more likely to be detected under m anagement - by
- c onsent than management
- by
- e xception. I n manual condition, event management became poorer as the number of UAVs i ncreased. Subjective workload estimated by NASA_TLX also increased with number of U AVs in the manual and management by consent conditions. SWO RD ratings of w orkload were higher for manual control than for either form of automation, and were hi gher for management
- by
- c onsent than for management
- by
- e xception when reliability w as less than perfect. Sword ratings also increased as the number of UAVs i ncreased. M anagement - by
- c onsent produced higher levels of self
- r ated situation awareness than did m anual control or management
- by
- e xception. Management
- by
- e xception produced e specially low ratings of SA when automation reliability was only 95%. SA ratin gs also de creased as the number of UAVs increased. T rust in automation decreased as the number of UAVs increased.

41) R yder, J.M, Scolaro, J.A., Stokes, J.M. (2001). An instructional agent for U AV controller training. U AVs- S

ixteenth International Con f

erence , 3.1 - 3 .11. Bristol, U K: University of Bristol. D escribes development of an automated agent, EAGLE, to train pilots on a simulated P redator UAV landing task. The authors note that because there are minimal differences be tween operating a console dur ing real missions and simulations, simulations may be i deal for UAV operator training. The current instructional agent was developed using C HI Systems' COGNET framework.

42) Schreiber, B.T., Lyon, D.R., Martin, E. L., & Confer, H.A. (2002). I mpact of pr ior flight experience on learning Predator UAV operator skills (AFRL - H

E

- A Z - T R 2002 - 0026) . Mesa, AZ: Air Force Research Laboratory, Warfighter Training R esearch Division. E xamined subjects ability to learn & perform maneuvers on a Predator UAV. Compared s everal groups of subjects including experienced Predator pilots; experienced USAF pi lots selected to fly the Predator; students who had recently completed USAF T
- 38 training; students who had recently completed USAF T
- 1 training; students who recently c ompl eted single
- e ngine instrument training at Embry
- Riddle; students who recently c ompleted requirements for private pilot's license; Embry
- Riddle ROTC students who pl anned to join USAF but had no flight training or experience.H uman Factors of UAVs 46
S ubjects flew basic maneuver s and landings until reaching a criterion level of pe rformance, then flew 30 reconnaissance missions. Of interest was the number of trials ne cessary to reach criterion performance on the training task, and the transfer of training pe rformance to the reconn a issance task. R esults T raining A s expected, predator pilots required the fewest trials to reach criterion performance, and nonpi lot ROTC students required the most. This comparison demonstrates the validity of the simulation and task. Predator selectees and civilian instrument pilots required fewer trials than T
- 1 grads, required fewer trials. T
- 38 grads and private pilots were not s ignificantly worse than Predator selectees/civilian instrument pilots, nor were they s ignificantly better than T
- 1 grads. R e sults demonstrate that prior flight experience can r educe the number of trials to become proficient at maneuvering & landing the Predator s imulation.
T ransfer D ependent variable was mean amount of time that sensor was focused on target during e ach trial
. Predator pilots, Predator selectees, and T
- 38 grads had more time on target than other groups. ROTC nonpilots had less time on target than Predator pilots, selectees, T
- 38 grads, and T
- 1 grads. Results show that even after subjects achieved matched level s of performance on the training task, prior flight experience improved performance in the r econ task. Authors suggest that good performance of T
- 38 grads as compared to T
- 1 gr ads may reflect the degree to which performance characteristics of the T
- 38 and T
- 1 are s imilar to those of the Predator.

43) T varyanas, A.P. (2004). Visual scan patterns during simulated control of an u ninhabited aerial vehicle (UAV). A viation, Space, and Environmental Medicine, 75, 531 - 538. A n experiment examined pilots' eye mo ve ments during a simulated Predator UAV flight t ask. Goals were to determine A) how efficiently operators process the moving textbox i nstrument displays used in the Predator HUD, B) whether workload (as determined by w indiness during flight and by the diff i culty of flight maneuvers) affected scan patterns, and C) whether the absence of auditory and haptic cues caused UAV pilots to increase their dwell frequency on the engine instrument (RPM) relative to pilots in a manned a ircraft. Of particular interest wa s whether or not the moving textbox instruments would be processed as digital/quantitative displays or as analog/qualitative displays. Past data ha s suggested that quantitative displays are processed less efficiently (i.e., require longer dw ell times) than qualitative displays.H uman Factors of UAVs 47 S ubjects were 5 instrument rated pilots. Subjects flew an eight
- l eg flight plan twice, once i n no - w ind conditions and once in windy conditions (order randomized). Different s egments of the flight profile included changes in heading , altitude, and airspeed, s ometimes singly and sometimes in combination. Changes involving multiple parameters w ere presumed to impose higher workload, as were windy flight conditions. R esults D ependent variables were dwell times and dwell frequencies. In de pendent variables were f light conditions (no wind vs. windy), flight segment (one, two, or three parameters c hanged), and instrument. Both dwell times and dwell frequencies showed a significant m ain effect of instrument, and neither showed any other main effects or interactions. D well frequency was highest for the ADI, followed by the VSI, AS, HI, ALT, RPM, and A OA. T o determine whether moving textboxes were processed as qualitative or quantitative di splays, dwell times for these instruments were comp ared to dwell time for the heading i ndicator, an instrument that is clearly quantitative. If moving textboxes are processed as qua litative displays, they should have dwell times shorter than those for the HI. Only the A OA and RPM had significantly shorter dw ell times, suggesting that other moving t extboxes were processed as quantitative displays. T he author also compares the current data to the results of earlier studies of instrument s canning in manned aircraft. In the present study, ADI was the most fre que ntly fixated i nstrument, but still accounted for only 30% of all dwells. In contrast, previous research ha s found that the ADI can account for over 50% of all dwells in manned flight. C onversely, VSI was fixated more often in the present study (16% of a l l dwells) than in s tudies of manned flight (22%). The author suggests two possible reasons for these di fferences. First, ADI in the Predator HUD was very simple, showing only a horizontal l ine without any pitch or bank scale. VSI scanning might therefore be necessary to a cquire or verify climb and descent rates. Second, pilots might rely on the VSI more he avily in UAV given the inherent system delays. In other words, delays in the system r esponses to control inputs might require operators to rely more heav i ly on the predictive V SI instrument than in manned flight. D ata also suggest that the engine instrument was not scanned more heavily in the current t ask than in previous studies of scanning in manned flight. This suggests that operators di d not use the RP M to compensate for the absence of auditory and haptic information, and may indicate a sub
- opt imal performance strategy. T he author speculates that the high skill level of the participants in the current experiment m ight explain the null effect of worklo a d on scanning behavior.H uman Factors of UAVs 48 44) V an Erp, J.B.F. (2000). Controlling unmanned vehicles: The human factors s olution. R TO Meeting Proceedings 44 (RTO- M P

- 44) , B8.1 - B 8.12. T he author notes that bandwidth constraints on the datalink between a ground control s tat ion and UAV will limit the quality of sensor information displayed to UAV operators. T wo remedies to this problem are possible. The first is to reduce bandwidth needs by i dentifying task
- c ritical sensor information and ensuring that only this is transmitt e d. The s econd is to design advanced interfaces that assist the operator in compensating de gradations or limits in sensor imagery. S everal specific ways in which the information provided to a UAV operator is degraded are described. First, this information typically includes only imagery from an on
- board c amera. Input from other sensory modalities (audition, kinesthesia) is lost. Second, the s ensor imagery provided to the operator is often of low resolution, achromatic, and l imited to a small FOV. Third, se ns or imagery is often of low temporal resolution. F ourth, the control devices used to manipulate sensor cameras do not provide pr oprioceptive/kinesthetic feedback similar to that obtained in using the scanning through he ad and eye movements. T he author ne xt delineates a variety of sensor image characteristics that contribute to ve hicle control: field size, magnification factor, chromaticity, temporal resolution, spatial r esolution, monoscopic vs. stereoscopic presentation, fixed vs. variable viewing direct i on, and placement/aiming. To optimize performance, operator can be given the capability to m anually adjust the temporal and spatial display resolution, reduce the image field size, and toggle between color/grayscale and between stereoscopic/monoscopic dis pl ay modes. T he UAV operator is also confronted with difficulties in payload sensor control. First, c ontrols do not provide feedback on camera responses to user inputs. Second, the ope rator does not receive vestibular feedback to specify vehicle attitude
. Third, the ope rator has no proprioceptive feedback to indicate viewing direction. Fourth, control i nputs do not produce immediate changes in sensor imagery. Fifth, spatial information w ithin the sensor imagery is low in resolution. Sixth, the sensor FOV i s often small, i mposing the need for additional control inputs to scan a scene and degrading the ope rator's ability to integrate sensor images into a coherent and veridical mental r epresentation. Seventh, camera imagery may be zoomed
- i n, disturbing the no rmal r elationship between camera translation and image motion. Finally, image update rates m ay be low, degrading the temporal resolution necessary for dynamic tracking tasks. The a

uthor discusses a number of advanced display designs to address the difficul t ies in c amera control produced by these degradations. Two of these involve computer
s ynthesized imagery superimposed upon or embedding the camera imagery. The value of s uch "augmented reality" displays is that they computer
- ge nerated components can be upd ated immediately in response to user inputs, even if the sensor imagery itself is not updated until after a delay. The computer
- ge nerated components thus can provide real
t ime feedback to assist in guiding the sensor footprint. The third novel display disc us sed i s a radar image that includes actual and predictive sensory footprints. Thus, motion of the computer
- ge nerated predictive sensor footprint again provides operator with i mmediate feedback to aid camera targeting, despite delays in camera update rate. Head
-H uman Factors of UAVs 49

c oupled camera control, the author notes, does not improve camera control in a search t ask (effect of the head
- c oupled control is a speed
- a ccuracy tradeoff), and may degrade pe rformance because of mismatches in proprioceptive and visual information p r oduced by sensor delays.

45) V an Erp, J.B.F., & Van Breda, L. (1999). H uman factors issues and advanced i nterface design in maritime unmanned aerial vehicles: A project overview. TNOr

eport TM - 99- A 004 . S oesterberg, The Netherlands: TNO Human Factors R e search I nstitute. R eport presents a summary of human factors issues in UAV control, and an overview of r elevant research conducted at TNO. H uman factors concerns T he authors assume that vehicle control will generally be highly automated, and so focus the ir discussion of on manual control of payload camera. The studies reported assume that the most important source of information for camera control will be the imagery f rom the on
- board camera. T he authors note that the perceptual information the operat or receives from the remote e nvironment is likely to be degraded in several ways:

- - no proprioceptive feedback from controls

- - no vestibular input based on attitude

- - no proprioceptive feedback based on viewing direction

- - l imited spatial orientation

- - no direct feedback (i.e., feedback delayed) in response to control inputs

- - no auditory input

- - l imited resolution of camera images

- - l imited geometrical field of view

- - z oomed
- i n camera image

- - f ew points of reference at sea

- - l imited image update rate P oss i ble consequences for human performance include poor tracking; difficulty in j udging camera, platform, and target motion; confusion about direction of platform flight; c onfusion about viewing direction of camera; disorientation; degraded situation awarenes s . E xperiments E xperiment 1 E xamined the benefits of synthetic visual motion in guiding payload camera. A c omputer - ge nerated grid of perpendicular lines was overlaid on camera image, andH uman Factors of UAVs 50

m oved in response to camera inputs. In the first experiment, subjec t s had to track a m oving ship with a simulated UAV sensor camera. Performance was improved by s ynthetic image augmentation, and benefits were largest when the update frequency of the camera was low. In a second experiment, subjects saw a target ship, then ha d to point c amera at after a 15 s delay that included several translations and rotations of the MUAV. A gain, performance was improved by the synthetic image overlay. E xperiment 2 A sked whether a computer
- s ynthesized world embedding the camera image (c a lled an e cological display, based on the notion that visual cues provided by embedding world are di rectly perceived) can aid in guiding camera. Subjects had to search for target ships with c amera. Performance with ecological display was compared to perform ance with he ading/pitch indicators adjacent to camera image. Such indicators require cognitive i nference, in contrast to ecological display. Ecological display reduced search time and t otal number of camera motions. Indicators did not significantly improv e performance r elative to baseline. E xperiment 3 A sked whether an ecological display can allow an operator to control UAV airframe and c amera simultaneously. The task required the subject to track a target ship while circling i t. Four display types were u s ed: two without augmentation (north up & track up), and two with augmentation (2D synthetic world and a 3D synthetic world). Data showed that a

ugmented displays aided airframe control without degrading tracking. Augmented di splays also allowed effective m anual control with high airframe speeds. E xperiment 4 E xperiment examined manual control of sensor under conditions of low update rates and de layed visual feedback, and measured the benefits of a predictive camera footprint. Data s howed that update rates below 2 Hz and delays longer than 2 seconds degraded tracking pe rformance. Predictor display eliminated costs of slow update rate and time delay except at the most extreme values. E xperiment 5 E xamined the effects of head
- s laved camera control, time delays , and advanced interface de sign on situational awareness. The authors speculate that proprioceptive feedback from he ad
- s laved control may aid SA. However, helmet
- m ounted displays might be unc omfortable, and transmission delays could make the perception o f spatial information di fficult. In the experiment, HMD was compared to head
- s laved dome projection. To ove rcome the problems of delayed image transmission, a method of compensation called de lay handling was introduced. Delay handling preserves spatial rel ationships between i nput images by presenting them in the viewing direction of the camera at the moment i mage was recorded, rather than the moment at which image transmission is received. D ata indicate that delay handling improves SA. No benefit was found f or dome pr ojection relative to helmet
- m ounted display. E xperiment 6H uman Factors of UAVs 51 C ompared head
- c oupled control/helmet
- m ounted displays to manual control of camera. S ubjects had to locate five target ships as quickly as possible. In manual control c ondition, imagery w as projected on a dome, so that proprioceptive info was available in both conditions. Head
- s laved camera control increased search speed but enlarged the s earch path as compared to manual control.

46) V eltman, J.A., & Oving, A.B. (2002). Augmenting ca m era images for op erators of unmanned aerial vehicles. R TO Meeting Proceedings (RTO- M O- 088) . U AVs flight path is often pre
- pr ogrammed, but camera must still be steered manually. T his can be difficult because of low camera update rates and communication t i me delays be tween GCS and vehicle. One method of addressing these difficulties is to provide c urrent and predictive camera view footprints on a 2D map. This provides motion f

eedback when camera moves, along with information preview of where the camera wil l be shifting. The authors note that a 2D map provides primarily exocentric (authors use the term "local") spatial information, while a 3D map provides egocentric ("global") info. Authors speculate that providing a predictive camera footprint within a 3D m a p might therefore improve camera steering performance beyond that observed with a 2D map. T he goal of the experiment was to test this speculation. S ubjects flew a simulated UAV recon mission which required them to search for military ve hicles along roads and edges of woods. Two side
- by
- s ide displays were used. On the l eft was a 2D map which presented waypoints and route plan; flight direction; and actual and predicted camera footprints. On the right (in some conditions) was a 3D map which pr esented an imm e rsed view from vantage point of UAV camera, along with actual and pr edicted footprints. In experimental conditions, subjects were provided the 3D display in a ddition to the 2D map. In control conditions, only the 2D map was provided. The c amera image was presented in lower right display. Camera image quality had three l evels: normal, 3 Hz update rate, 1 second delay. In some conditions, subjects also pe rformed a secondary monitoring/memory task. R esults When camera quality was normal, 3D camera produced a small increase in the percentage of roads and wood edges that were inspected (~35% vs. ~40%). When camera image qua lity was degraded, the benefits of the 3D map were larger (~20% vs. ~30%). S econdary task performance was better with 3D map, suggesting th at map produced s pare mental capacity, and subjective workload ratings were lower with 3D camera than w ithout. EOGs indicated that subjects inspected 3D map frequently, especially when c amera quality was low.H uman Factors of UAVs 52 47) Wal ters, B.A., Huber, S., French, J., & Barnes, M.J. (2002). U sing simulation m odels to analyze the effects of crew size and crew fatigue on the control of tactical u nmanned aerial vehicles (TUAVs) (ARL - C R - 04 83) . Aberdeen Proving Ground, MD: A rmy Research Laboratory. A study used simulation mo de ling to determine how fatigue, crew size, and rotation s chedule affect operator workload and performance on a TUAV control task. Simulations w ere conducted using MicroSaint modeling architecture, from Micro Analysis and D esign. 18 subject matter experts ( SMEs) provided A) a list of tasks that are involved in c ontrolling a TUAV during normal operations and during emergencies, B) the order in which the tasks are performed, C) the visual, auditory, cognitive, and psychomotor w orkload imposed by tasks, D) the types of emergencies that can occur during missions, and E) the probabilities of mishaps occurring during emergencies when soldiers are f

atigued. T he fatigue algorithm used by simulation predicts human response capability for tasks ove r an extended perio d of sleep deprivation. The focus of algorithm is the interaction of s leep deprivation with circadian rhythms. T he model used simulates the tactical operations center (TOC) and launch/recover station ( LRS) (including mission commander [MC], aerial vehicl e operator [AVO], and mission payload operator [MPO] duties), and several functions: launch, transfer, recovery, m ission support, emplacement, displacement, emergencies, mishaps, and maintenance dur ing emplacement. The model was used to simulate 12
- and 18
- hr missions over a 24
- hr period under 15 different conditions for three consecutive days. During the missions, there were times with 2 UAVs in
- f light: one observing the targets, and one flying to a ssume control of search. Soldiers were modeled to work 2
- , 3
- , 4
- , or 6
- hr rotation s chedules. The model does not simulate soldier activity in between shifts. M odels simulate one move (jump) per day for the TOC and one move every other day for the LRS. Each move comprises a 1/2 break
- down, 1/2
- hr move, and a 1
- h r setup. The T UAV spends 5 hrs of simulation time in the air: 4 hrs of surveillance and 1 hr in transit t o/from destination. The output of model includes performance times, target detection r ates, and AV mishaps under each simulated condition. Several cre w configurations ( different numbers of MCs and AVOs/MPOs) were tested. S ome conditions that can affect a TUAV mission include

- - type of search: area search, person search, airfield, tanks, building, road search, bridge, m issile site, command post, air def e nse artillery, check points, battle damage assessment on SAM, artillery search

- - e mergencies: icing, generator failure, signal degradation or intermittent link loss, payload failure, AVO or MPO console failure, GPS failure

- - w eather: humidity, sun, gusti ng winds, crosswinds, flat clouds, ragged clouds

- - t errain: high vegetation, desert (sand), high desert, city, town, villageH uman Factors of UAVs 53 W orkload estimates were obtained from SMEs using a scale developed to be compatible w ith Wickens' (1984) multiple resources theory of attention. Four resource pools were a ssumed: visual, auditory, cognitive, and psychomotor. T he model simulates 5 TUAV launches per day for an 18
- hr mission. Each launch, three types of target search were performed. Missions were repeated every day for 3 days for e ach crew rotation schedule. R esults D ecreasing crew size decreased target hit rates and increased target detection times. W orkload estimates suggested that when there was no MC in the LRS, the TOC MC was i nterrupted ~50% of the time to per f orm tasks that the LRS MC would otherwise have pe rformed. When there was 1 MC in the LRS, the TOC MC was interrupted ~20% of the t ime with LRS MC tasks. Adding a third AVO to the LRS (compared to baseline c ondition of 2 AVOs) did not improve performance. T he model was adjusted to simulate 12
- hr mission profiles with and without 1
- hr gaps be tween flights. Three launches were simulated per day instead of 5. Results were similar t o those from 18
- h r mission conditions. No performance differences were produce d by 1
- hr gaps between missions.

48) We eks, J.L. (2000). U nmanned aerial vehicle operator qualifications (AFRL H

E

- A Z - T R - 2000- 0002 ) . Mesa, AZ: Air Force Research Laboratory, Warfighter T raining Research Division. R eport compares selection criteria for U AV operators across branches of the U.S. m ilitary and British army. P ioneer, USNL EP candidates go through a 24
- w eek training course. Payload operator and AVO compete different 8
- w eek courses. Mission commander has to be a flight of ficer. Health conditi ons related to hypoxia or pressure changes are not disqualifying. H ealth standards include corrected visual acuity of 20/20 in each eye, normal color vi sion, normal hearing, clear & distinct speech, and voice well modulated. EP requires normal depth percep t ion. P ioneer, USMC AVO and PO complete the same 8
- w eek training course. Candidates for E P have to demonstrate satisfactory as AVO or PO, demonstrating good 3
- D c ognition/perception, then complete a 19
- w eek training course. MC has to be an aviation of fic e r. Physical standards are the same as for UAN UAV operators. H unter, USA AVO and PO have to complete a 23
- w eek training course. Candidates for E P have to demonstrates satisfactory performance as AVO or PO, and are screened byH uman Factors of UAVs 54 i nterview and by performance using a radio
- c ontrolled model airplane. If selected, they must complete a 16
- w eek training course. The AVO and PO are required to pass a class I V flight physical, which includes requirements for medium physical demands, a normal phys ical profile, and norm al color vision. The EP is required to pass a class III physical, s imilar to that required for air traffic controllers. P hoenix, British Army AVO is required to take a 3
- w eek course. Flight crews are not r equired to take physicals. P redator, USAF AVO candi date has to be a pilot of a fixed
- w ing aircraft or a navigator w ith FAA instrument rated commercial license. Beyond undergraduate flight training, f

ollow
- on training, then 9 weeks of Predator basic training. DEMPC and SO complete 24 w eeks of initial
- s ki l ls training as an Imagery Interpretation Apprentice, then 9 weeks of P redator basic training. AVO has to pass a Class I physical. DEMPC and SO have to pass a Class III flight physical, but with visual acuity and depth perception standards e quivalent to Cl a ss I standards.

49) Wi ckens, C.D., & Dixon, S. (2002). W orkload demands of remotely piloted v ehicle supervision and control: (1) Single vehicle performance (Technical report A HFD - 02 - 10/ MAD - 02 - 1) . Savoy, IL: University of Illinois, Institute of Aviatio n , A viation Human Factors Division. E xamined the benefits of offloading tasks from visual channel in a single
- U AV control t ask, and compared the results to the predictions of single
- c hannel theory (SCT), single
r esource theory (SRT), and multiple
- r esource theory (MRT) of attention. Subjects each f lew a series of missions involving three tasks: mission completion (flight path tracking), i nspecting command targets (CTs) / searching for targets of opportunity (TOOs), and m onitoring system gauges for system fa i lures (SFs) (i.e., out of bounds values). Flight i nstructions (fly
- t o coordinates of next target and a question about the target to be answered) were provided during the task. Instructions could be refreshed with a button pr ess. In baseline condition, pi l ots flew with all manual flight controls and all visual di splays. In auditory condition, SF alerts and flight instructions were provided aurally. In an automation condition, flight control was offloaded to an automated system. All a lerts/instructions/auto m ation were perfectly reliable. R esults F light path tracking was unaffected by auditory offloading. B utton - pr ess refreshes of instructions were reduced in both the auditory offloading and the automation conditions. In auditory condition, this might have been because aural pr esentation reduced visual conflict during time that instructions were first presented. In a

utomation condition, it may have been because the subject was not required to r emember target coordinates.H uman Factors of UAVs 55

T OO detection rates improved under a utopilot flight control both for single and dual
U AV tasks. This was in part due to the fact that the autopilot flew directly over each t arget, while the human operator sometimes did not. However, an autopilot benefit was e vident even restricting analysis to those trials on which the TOO appeared in the 3D di splay. This suggests a role for the autopilot in improving cognitive/attentional pe rformance in TOO detection. Auditory alerts significantly improved SF detection rates and SF response times c ompared t o baseline, except in cases where the subject was loitering/inspecting a target at time of SF occurrence. This suggests that difficult image interpretation produced c ognitive tunneling. Automation condition also improved SF detection rates (though not und e r all circumstances), but did not affect SF response times.

50) Wi ckens, C.D., Dixon, S., & Chang, D. (2003). U sing interference models to predict performance in a multiple- t ask UAV environment - 2 UAVs (Technical report A HFD - 03 - 09/ MAAD - 03 - 1) . Savoy, IL : University of Illinois, Institute of Aviation, A viation Human Factors Division. E xamined the benefits of task offloading to pilots performing single
- and multiple
- U AV t asks and compared the results to predictions of single
- c hannel attention theory (SCT ) , s ingle
- r esource theory (SRT), and multiple
- r esource theory (MRT) of attention. Subjects e ach flew a series of missions involving three tasks: mission completion (flight path tracking), inspecting command targets (CTs) / searching for targets of opportun i ty ( TOOs), and monitoring system gauges for system failures (SFs). Flight instructions (fly
t o coordinates of next target & a question about the target to be answered) were provided dur ing the task. Instructions could be refreshed with a button press. In the baseline c ondition, pilots flew with all manual flight controls and all visual displays. In the a

uditory condition, SF alerts and flight instructions were provided aurally. In the a

utomation condition, flight control was offloaded to an automated syst e m. All a lerts/instructions/automation were perfectly reliable. R esults F light path tracking was unaffected by auditory offloading. Button
- p ress refreshes of f light instructions were reduced in both the auditory offloading and the automation c onditions, s uggesting that these conditions freed up processing resources. The number of r efreshes was higher in dual
- U AV conditions, but the effect was primarily in the baseline & auditory offloading conditions, not in the automation condition. TOO detection rates i mproved under autopilot flight control both for single and dual
- U AV tasks. This was in part due to the fact that the autopilot flew directly over each target, while the human ope rator sometimes did not. However, an autopilot benefit was evident even restri c ting analysis to those trials on which the TOO appeared in the 3D display. This suggests a role f

or autopilot in improving cognitive/attentional performance in TOO detection.H uman Factors of UAVs 56

Auditory alerts improved SF detection rates and reduced detection times. Autopi l ot had no effect. Dual
- U AV costs to SF detection time obtained in the baseline and autopilot c onditions, but not in the auditory alert condition. SF detection times were longer for f

aults that occurred during target inspection than for faults that occurre d under normal f light, lending some support to CST and SRT. However, this effect was mitigated s omewhat by auditory alerts, consistent with MRT.

51) Wi lliams, K.W. (2004). A summary of unmanned aircraft accident/incident d ata: Human factors implication s . E xamines military UAV accident/incident data for various UAV systems. A rmy H unter H unter takes off & lands using an external pilot (EP) in visual contact using controller s imilar to that used for remote controlled hobby planes. After takeoff and climb , internal pi lot (IP) assumes controls from GCS. The IP controls the aircraft using knobs to select a ltitude, heading, & airspeed. 47% of accidents were HF related. The largest percentage ( 47%) of HF issues arose during landing. An additional 20% arise dur i ng takeoff. Control di fficulties are caused in part by the need for the operator to reverse control inputs when a ircraft is headed toward him/her. Other problems include:

- - pi lot
- i n - c ommand issues

- - f ailure of alerts/alarms to inform operator of non
- norma l conditions

- - m ode display errors

- - c rew failure to follow proper procedure S hadow T he Shadow uses a launcher for takeoff and an automated system, the tactical automated l anding system (TALS), for recovery. Landing generally does not require intervention f rom the operator in the GCS. In flight, the aircraft is controlled through a menu
- ba sed i nterface that allows operator to select altitude, heading, & airspeed. During landing, ope rator in GCS has no visual contact with aircraft, and receives no data from onboard s ensors. An external observer is required to communicate to the operator when the craft ha s touched down, at which time the operator gives a command to stop the engine. HF e rrors were less frequent with Shadow than with Hunter. N avy P ioneer T he P ioneer requires an EP for takeoff & landing. After takeoff, IP controls the vehicle f rom the GCS in one of three modes: autonomously using preprogrammed waypoint c oordinates; semi
- a utonomously using airspeed, altitude and heading values specified w ith rot ary knobs; manually, with a joystick. There are plans to implement an automatedH uman Factors of UAVs 57

s ystem for ship
- ba sed landings. 28% of accidents were HF related. Of these, 68% oc curred during landing and 10% during takeoff. An additional 13% were due to aircrew c oordinat i on lapses (procedural & communication errors) and 10% were weather related e rrors resulting from errors in pilot decision making. F ire Scout A vertical takeoff & landing vehicle, the Fire Scout was involved in one accident. A ntenna was damaged during gro und handling (human error), causing incorrect altimeter r eading when vehicle was airborne. A ir Force P redator T he Predator is flown from a GCS using a joystick and rudder pedals and a forward l ooking camera with a 30
- de g FOV. The camera is also used for t akeoffs and landings. H uman factors lapses contributed to 67% of accidents. A majority of these (75%) were pr ocedural errors, including a failure to follow checklist steps during handoff between c rews and an accidental activation of a program that erased the aircraft computer's i nternal RAM. I nterface issues are discussed in 89% of Predator accidents, and are cited as a c ontributing factor in 44%. Four categories of interface issues: design of HUD; design of H DD; alerts and alarms; functioning of the autop i lot.

- - H UD problems: FOV (30 degs) is too narrow; attitude indicator is inadequate; RPM i ndicator needs improvement; symbology obscured during low
- l ink conditions; s ymbology contrast too low; symbology inadequate.

- - H DD problems: too many levels to maneuve r through to reach needed info; info display uni ntuitive; critical commands unprotected or unemphasized; operational value ranges i nconsistent within display.

- - A lerts/alarms problems: do not capture attention; audio warnings insufficient or absent; i nfo provided inadequate or poorly prioritized; info provided invalid; data that need to be c ompared not always collocated on same display page.

- - Autopilot problems: no indication of autopilot status on HUD; flight controls are di sabled while autopilot is eng a ged (i.e., no override capability) and four separate menus have to be traversed in order to deactivate autopilot (requires about M = 7 seconds); a

utopilot tends to command extreme measures and overstress aircraft; autopilot f unctionality does not conform t o AF standards. G lobal Hawk G lobal Hawk is the most fully automated of UAV systems. All phases of flight are a

utomated, including takeoff & landing. The crew's task is to monitor the aircraft and c ontrol payload. This makes flying the relatively easy, b ut makes mission planning e xceedingly difficult. Mission planning process begins up to 270 days prior to mission. O nce the target set is finalized, 3
- 5 weeks are required to write and validate mission plan. O f three accident reports available for Global Haw k, only one involved HF issues. A ircraft was forced to perform an emergency landing at a preprogrammed alternate a irport. At point of airport, a taxi speed of 155 knots had been set due to software bugH uman Factors of UAVs 58 dur ing preprogramming. When aircraft was commanded to begin taxiing for takeoff, it r eached a speed where it was unable to turn at appropriate point and ran off the runway. F undamental HF problem with the Global Hawk is that the system does not encourage c lose monitoring by operators, resulting in reduced SA
. An additional problem is that s tatus reports are provided in hexadecimal and do not include trend data.

52) Wi lson, G.F., & Russell, C.A. (2004). Psychophysiologically determined ad aptive aiding in a simulated UCAV task. P roceedings of the Second Hum an P

erformance, Situation Awareness, and Automation Conference (HPSAA II) , Daytona B each, FL, March 22 - 25 . A n experiment tested the benefits of adaptive aiding based on psychophysiological a ssessment of operator workload. The task required subjects to mon i tor 4 vehicles flying pr eplanned routes. When vehicles reached designated points, radar images of target area w ere presented to subject. Subject searched target area then selected targets for bombing. T he search/target designation task was conducted under time stress. The subjects chose the order in which images from the vehicles were presented. Images were presented at two levels of difficulty. The more difficult level included more distractors and required m ore difficult decisions regarding target priori ty. S ubjects were also required to monitor vehicles for potential emergencies (e.g., loss of c ommunication). Memory load was manipulated by having subject hold up to 4 a ircraft/problem combinations simultaneously until a command was given specifying whic h problem to address. E EG, ECG, and EOG data were recorded. An artificial neural network was trained to r ecognize periods of low and high task difficulty using these data. During criterion task pe rformance, three levels of adaptive aiding were used: 1) n o aiding, 2) aiding during t imes of high workload, and 3) random aiding. Aiding involved decreasing velocity of ve hicle so that time stress was reduced. Subjective workload was measured with NASA
T LX. R esults T he neural net was 70% accurate at classifying high/low task difficulty levels during task pe rformance. For all conditions, the number of correctly selected targets was lower when the task was difficult. The number of designated points of impact was lower for the di fficult task level in the unaided an d the randomly
- a ided conditions, but was unaffected by task difficulty in the adaptively
- a ided condition. Similarly, the number of missed w eapons releases was higher in the difficult level for the unaided and randomly
- a ided c onditions but was unaffected by difficulty in the adaptively aided condition. Differences i n subjective workload were marginal and inconsistent.H uman Factors of UAVs 59 A ppendix B R esearch Matrix T his appendix provides a cross
- i ndex of the research issues discussed in the main body of the text with the re s earch literature described in Appendix A. Only those articles deemed di rectly relevant to each question are included. Bold
- f aced italics indicate research articles that present empirical data. 1. To what extent should en route flight control be automated ? R elevant articles: 2 , 7, 19, 25 , 32, 33, 38 , 40, 49 , 50, 51, 52 2. What are the consequences of degraded reliability of automated UAV functions f

or performance of the automated task and of concurrent tasks? R elevant articles: 11 , 12 , 40 3. How will see and avoid requirements be addressed in UAV flight? Can automated d etect, see, and avoid ( D

SA ) technology allow a UAV operator to maintain ac ceptable levels of separation? What are the consequences of imperfectly reliable D SA automation on conflict detecti on and on performance of concurrent tasks? R elevant articles: 27 4. To what extent should takeoff and landing be automated? R elevant articles: 2 , 51 5. Through what form of control interface should internal and external pilots m anipulate a UAV? R elevant articles: 37 , 51 6. What compromises should be adopted between spatial resolution, temporal r esolution, time delay, and field - o f - vi ew (FOV) in the display of visual imagery for f light control and/or conflict detection? R elevant articles: 10 , 32, 44, 45 , 4 6 , 51 7. Can augmented reality displays or synthetic vision systems successfully c ompensate for the degrade visual imagery provided by onboard sensors? R elevant articles: 3, 13 , 15 , 16, 17 , 18 , 29 , 44, 45 , 46 8. Can multimodal display technology be used t o compensate for the dearth of s ensory information available to a UAV operator? R elevant articles: 4 , 5 , 11 , 12 , 14 , 19, 20 , 25 , 32 , 34, 39 , 43 , 44, 49 , 50 , 51 9. To what extent can displays and controls be standardized across UAV systems? Wh at level of s tandardization should be mandated? (Basic T instrument panel? H UD overlay?) R elevant articles: 21H uman Factors of UAVs 60
10. Wh at are the consequences for system safety of pilot judgment when the pilot no l onger has a “shared fate” with the vehicle? Will there be subtle shifts in risk taking t hat might affect overall airspace safety? R elevant articles: none 11. How will hand - o ffs between crews be accomplished during long - e ndurance f lights? R elevant articles: none 12. What are the effects of variable total loop time delays on r esponse to ATC i nstructions? R elevant articles: 19, 27, 32, 43 , 44, 45 13. What form of predictable autonomous behavior should a UAV adopt following a l oss of ground - t o - ai r communications? R elevant articles: none 14. How many members will each crew comp r ise, and what will be each c rewmember’s responsibilities? Can an operator supervise multiple UAVs s imultaneously while maintaining an acceptable level of performance? R elevant articles: 7, 8 , 15 , 21, 23 , 28 , 30, 33, 38 , 4 7 , 51, 52 15. What are the core k n owledge, skills, and abilities (KSAs) that should be r equired for UAV pilot certification? What KSAs should be required for c ertification to fly particular UAV systems or classes of systems? R elevant articles: 2 , 26, 48 16. Should experience piloting a m anned aircraft be prerequisite for UAV pilot c ertification? R elevant articles: 2 , 42 , 48 17. What medical qualifications should a UAV operator be required to meet? R elevant articles: 25 , 47 , 48H uman Factors of UAVs 61 A ppendix C C ontact Information T his appendix provides av a ilable contact information for first and/or senior authors on the research articles summarized in Appendix A. B all, Jerry T. A ir Force Research Laboratory 6030 S. Kent St. M esa, AZ 85212 ( 480) 988
- 6561 B arnes, Michael J. A rmy Research Laboratory 4656 S. Cherokee S ierra Vista, AZ 85650 ( 520) 538
- 4702 C ooke, Nancy J. C ognitive Engineering Research Institute 5865 S. Sossaman Rd. M esa, AZ 85212 ( 480) 727
- 1331 C ummings, Mary J. M assachusetts Institute of Technology 77 Massachusetts Ave 33
- 305 C ambridge, MA 02139 ( 617) 253
- 4196 de Vries, Sjoerd T NO Human Factors P .O.Box 23 3769 ZG Soesterberg, The Netherlands + 31 346 356 300 de vries@tm.tno.nl D raper, Mark H. A ir Force Research Laboratory/HEC 2255 H St. Wright Patters on AFB, OH 45433 ( 937) 255
- 5779 G luck, Kevin A ir Force Research LaboratoryH uman Factors of UAVs 62 6030 South Kent St. M esa, AZ 85212 ( 480) 988
- 6561 ke vin.gluck@mesa.afmc.af.mil G oodrich, Michael A. B righam Young University 3361 TMCB, BYU P rovo, UT 84602 ( 801) 422
- 6468 m ike@cs. byu. edu Gugerty, Leo C lemson University P sychology Dept. 418 Brackett Hall C lemson, SC 29634 guge rty@clemson.edu H ancock, Peter A. U niversity of Central Florida P artnership II 3100 Technology Pkwy, Suite 337 O rlando, FL 32826
- 0544 ( 407) 823
- 2310 phancock@pegasus.cc.ucf.edu H ansman, R. John M assachusetts Institute of Technology 33
- 303 MIT C ambridge, MA 02139 ( 617) 253
- 2271 r jhans@mit.edu M artin, Elizabeth A ir Force R e search Laboratory 6030 South Kent St. M esa, AZ 85212 ( 480) 988
- 6561 e lizabeth.martin@mesa.afmc.af.mil M ouloua, Mustapha U niversity of Central Florida P hillips Hall 302M O rlando, FL ( 407) 823
- 2910H uman Factors of UAVs 63

m ouloua@ pe gasus.cc.ucf.edu R

yder, Joan C HI Systems 1035 Virgina Dr. F ort Washington, PA 19002 ( 215) 542
- 1400 j rider@chisystems.com S hively, Jay U .S. Army/NASA Ames M S
- 243
- 11 M offett Field, CA 94035 ( 650) 604
- 6249 j shively@mail.arc.nasa.gov T varyanas, Anthony P
. U SAF 2602 Louis Bauer Dr. B rooks
- C ity
- B ase, TX 78235
- 5251 anthony.tvaryanas@brooks.af.mil W eeks, Joseph L. A ir Force Research Laboratory 6030 S. Kent St M esa, AZ 85212
- 6061 W ickens, C.D. U niversity of Illinois I nstitute of Aviation, Aviation Human Fac t ors Division #1 Airport Rd S avoy, IL 61874 ( 217) 244
- 8617 c wickens@uiuc.edu W illiams, Kevin W. F AA CAMI 5801 NW 31 s t Terrace O klahoma City, OK 73122 ( 409) 954
- 6843 ke vin.williams@faa.gov