# Developing a Design Space for Critiquing Autopilot Interfaces

Romane Dubus, Anke Brock, Wendy E. Mackay

► To cite this version:

Romane Dubus, Anke Brock, Wendy E. Mackay. Developing a Design Space for Critiquing Autopilot Interfaces. IHM'24 - 35e Conférence Internationale Francophone sur l'Interaction Humain-Machine, Mar 2024, Paris, France. pp.1-12, 10.1145/3649792.3649805. hal-04449912

HAL Id: hal-04449912

https://hal.science/hal-04449912v1

Submitted on 9 Feb 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Developing a Design Space for Critiquing Autopilot Interfaces
## Espace de Conception et Analyse Critique des Interfaces de Pilote Automatique

ROMANE DUBUS, Université Paris-Saclay, CNRS, Inria, France and Fédération ENAC ISAE-SUPAERO ONERA,
Université de Toulouse, France

ANKE M. BROCK, Fédération ENAC ISAE-SUPAERO ONERA, Université de Toulouse, France

WENDY E. MACKAY, Université Paris-Saclay, CNRS, Inria, France

Autopilot interfaces are designed to help aircrews maintain awareness about the aircraft's current automation mode, the lack of which can lead to fatal accidents. However, even though researchers have identified multiple design flaws, cockpit designs remain largely unchanged. We first review the research literature on autopilot design, with a special emphasis on addressing mode awareness in order to create a design space that characterizes autopilot interfaces. We next follow a generative design approach that analyzes and critiques three existing commercial autopilot interfaces with respect to this design space so as to identify their gaps and limitations. We then extend the design space to include additional design dimensions and conclude by suggesting directions for improving future autopilots.

Les interfaces de pilotage automatique sont conçues pour aider l'équipage à rester conscients du comportement de l'avion, car un manque de cette conscience peut entraîner des accidents graves. Malgré l'identification de défauts de conception, leurs interfaces restent largement inchangées. Nous passons d'abord en revue les travaux de la littérature sur les conceptions de pilotes automatiques afin de créer un espace de conception qui les caractérise. Notre approche de conception générative a guidé l'analyse et la critique de trois cockpits commerciaux existants, conduisant à l'identification de leurs limites. Enfin, nous avons étendu le design space avec de nouvelles dimensions, suggérant ainsi des directions futures pour la conception des interfaces d'autopilotage.

CCS Concepts: • **Human-centered computing** → **User interface design**.

Additional Key Words and Phrases: Design Space, Autopilot, Situation Awareness, Cockpit, Aviation

Mots Clés et Phrases Supplémentaires: Espace de conception, Pilote Automatique, Conscience de la Situation, Cockpit, Aviation

## Reference:

Romane Dubus, Anke M. Brock, and Wendy E. Mackay. 2024. Developing a Design Space for Critiquing Autopilot Interfaces. This is the author's draft version of the work. It is posted here for your personal use. Not for distribution. The definitive version of the record is published in IHM '24: Proceedings of the 35th Conference on l'Interaction Humain-Machine.

# 1 INTRODUCTION

Increasing cockpit automation has brought about a notable shift in pilots' roles, transforming them into system supervisors who monitor automated flight modes rather than operators who perform tasks manually [79]. This shift in responsibilities has taken the aircrew "out of the loop" [29], prompting concerns about pilots' *situation awareness* in general and *mode awareness* in particular. Endsley [28, p.97] defines "Situation awareness" as "the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future". Mumaw [51, p.1] defines "mode awareness" as "the knowledge and understanding of the current and future state and behavior of the automation."

<footnote n="2024">2024. Manuscript submitted to ACM</footnote>
Norman [58] explains that automated systems often lack feedback and argues that inappropriate design increases the risk of human error. Other researchers have identified multiple autopilot interface design flaws, such as ambiguity as to which knob to use [21]; lack of feedback about indirect mode changes; and flight deck errors such as simultaneous use of control input or selecting the wrong mode [59, 101]. Misinterpretation of autopilot behavior — *mode confusion* — has contributed to multiple fatal accidents [25, 29, 37, 50, 61, 85] and raises concerns about aviation safety. Over the past several decades, aircraft manufacturers have implemented minor revisions to address some of these issues, such as the redesign of the vertical speed target after the Air Inter Flight 148 crash [25, 37].

Despite demonstrated problems with autopilot interfaces, the aviation industry usually treats human error as a training issue rather than as a reflection of underlying design flaws [43]. This is not altogether surprising given the high cost of overhauling an established system which would not only be expensive financially, but would also require a complex certification process and extensive additional pilot training. As consequence, airlines prioritize adjustments to training and procedures [43, 92, 104] instead of explicitly redesigning the interface to reduce pilot error, and few believe that a major cockpit redesign is likely in the near future [8, 50, 51]. Instead, new technologies have been introduced gradually, where tinkering with the existing system [81, 83] has turned the flight deck into “a tower of babel” [35, p.11]. Unfortunately, hiding the introduction of a new technology in order to adhere to an older design can lead to disaster, as evidenced by the two Boeing 737 MAX crashes where pilots were unable to respond to a faulty stall protection system [16, 36, 91].

This paper argues that we need to explore new ways of interacting with autopilot systems to improve the safety of commercial aircraft. We first provide background about autopilot interfaces in current flight decks and describe three strategies for increasing situation awareness. We next summarize the literature on autopilot interfaces into a design space with proposed solutions for increasing mode awareness, with three key dimensions: *Touch, Vision and Space*. We use a generative approach to analyze and critique three existing commercial autopilot interfaces and identify five gaps where technology suggested by the literature is not available in current commercial flight decks. We also identify two additional elements — pictorial information and co-location of control and feedback — that would benefit these interfaces. We extend the design space so that it includes *Audition* and *Time* dimensions and identify directions for future flight decks.

# 2 BACKGROUND AND RELATED WORK

## 2.1 Existing autopilot interfaces

An autoflight system consists of multiple subsystems that help to automatically manage the flight path. This paper focuses on the *autopilot*, a short-term mode management system composed of a mode controller and a mode annunciator.

The FCU (Flight Control Unit)<fnref n="1" /> allows pilots to control or delegate flight path management. The pilot uses FCU buttons and knobs to choose the desired flight control mode. Basic modes simply follow a target, such as speed, heading, altitude and vertical speed. Advanced modes follow lateral and vertical flight path constraints, thus providing the highest level of automatic control. A separate system, the FMA (Flight Mode Annunciator), displays the autopilot’s current state, e.g., the authority, as well as altitude, heading, and airspeed targets. The FCU and FMA are located in separate areas in current cockpits (Fig. 1).

<footnote n="1">¹Airbus uses the term FCU (Flight Control Unit) whereas Boeing uses MCP (Mode Control Panel). To ensure clarity and consistency, we use the term FCU to refer to both systems.</footnote>
<figure><img src="image_2.png" /><figcaption>Fig. 1. Autopilot interfaces on commercial aircraft physically separate control (FCU) and feedback (FMA).</figcaption></figure>

## 2.2 Three major solutions for improving mode awareness

The research literature proposes three major solutions for improving awareness and reducing pilots' misunderstandings of aircraft behavior: improved procedures [92, 93], enhanced training [37, 52, 102] and better interface design [58].

2.2.1 *Improved procedures.* Pilots learn a set of procedures that encourage their active engagement during flight, not necessarily by manually manipulating the controls, but rather by “mentally flying” the aircraft whenever an automated system or co-pilot is in control. This lets the pilot anticipate FMA changes before they occur, rather than simply reacting to changes as they happen [90]. These clear, consistent procedures reduce the likelihood of confusion and offer pilots standardized sets of steps that help them avoid dangerous action sequences [58, 85, 92].

Unfortunately, reducing pilots' active involvement in flying the aircraft sometimes reduces their ability to identify and react in abnormal situations. Studies have shown that pilots sometimes deviate from established procedures and develop diverse personal strategies for understanding what the automation is doing [13], especially when procedures are illogical or inconsistent [22]. Roessingh et al. [69] found that airlines use aircraft differently from how they were originally designed, both when they prescribe using automation and when they leave these choices to the discretion of the crew. A survey by Mitchell et al. [46] reported that pilots basically favor using Standard Operating Procedures (SOPs) but do not agree with some aspects of the airlines' and the aircraft manufacturers' strategies. This suggests that establishing new mode control procedures may not fully solve the lack of mode awareness in the FMA and FCU. Even if the procedures are very well designed, pilots may still omit, misinterpret or forget steps, especially in high-stress situations. Procedures are only effective when pilots follow them, which requires extensive training and regular practice.

2.2.2 *Enhanced training.* Pilots receive extensive training to develop an accurate mental model of the autopilot's goals and behavior [12, 29, 51, 53, 75, 82, 85]. Soo et al [87]'s study of pilot learning revealed gaps in their knowledge of automation functionality due to lack of practice and inadequate understanding of how the automated systems work, made worse with long breaks between learning and practice. Pilots need better representations of the underlying
functionality of automated systems to allow them to cope with non-standard situations [72]. Several studies suggest that letting pilots practice nonstandard tasks and explore more diverse scenarios will increase their ability to handle unexpected situations [64, 75, 77, 100]. For example, Sarter and Woods [75] encourage pilots to explicitly learn from surprises experienced by themselves or other pilots.

However, while high-quality training is essential, it cannot address all problems. First, pilot training is already extremely time-consuming and expensive and small airlines that lack the resources and finances of major airlines cannot easily extend their training programs. This poses a challenge for implementing solutions that do not penalize airlines with more limited training resources.

### 2.2.3 Better interface design.
Pilots sometimes lack sufficient information to accurately track the state of the automa-
tion [58] and do not always have the resources to resolve technological failures. Wickens argues that poor situation
awareness may “result from deficient display design that fails to provide effective feedback regarding how automation
is carrying out its tasks” [100, p.368]”.

Some researchers suggest that co-locating information and feedback will increase the pilots’ situation and mode
awareness [33, 35, 40, 70, 100]. Boorman et al. [14] suggest that presenting automation more simply and clearly will not
only improve understanding but also make it easier for pilots to select the optimal automation options [100]. Interfaces
should also present more useful feedback about the state of the aircraft [94], including better interpretation of the
modes [30, 35, 45, 50, 60, 68, 82, 83, 88, 95] and more salient indicators of the aircraft behavior [56, 61].

Other researchers recommend distributing information across multiple modalities. Both auditory and tactile stimuli
can capture the pilot’s attention without necessarily interfering with their visual focus on primary tasks [74, 85]. For
example, Sarter et al. [73] suggest using vibrotactile feedback [98] to convey additional information.

However, incorporating additional feedback risks increasing the complexity of the cockpit display, transforming it into yet another element to monitor. The problems increase when the pilot must interpret multiple, poorly integrated automated systems that address different subtasks. Although adding feedback can be beneficial, implementation details matter. The challenge is to avoid introducing new sources of confusion or higher cognitive workload [42].

Although the goal of adding automation is to reduce the pilots’ workload, cognitive load actually increases when the
pilot’s mental model does not match the aircraft’s current behavior, or when information does not appear in an obvious
and timely manner [9]. Suggestions for improving cockpit designs include reducing automation complexity [66], limiting
the number of options, and reducing interaction to a small set of behaviors [92]. Lintern et al. [41] argue that human
performance problems created by Glass Cockpits — modern cockpits in which analog mechanical instruments are
replaced by electronic displays — are not an inevitable consequence of increased hardware complexity or of automation
but, rather, are the result of nonfunctional design that increases complexity in the cockpit interface. This research
suggests that reassessing the design of cockpit instruments has the potential to significantly improve aviation safety.

In summary, we argue that the time has come to re-evaluate the entire flight ecosystem. Most cockpit interfaces were designed before the introduction of automation and, despite major changes in functionality, few have been upgraded to support the newly created monitoring tasks [83]. Aircraft manufacturers have even explicitly hidden new automation functionality from pilots because they did not want to incur additional training costs associated with learning a new interface [36]. In the long term, aircraft are likely to undergo further waves of new automation, with corresponding opportunities for redesigning cockpit interfaces to match. Although a radical overhaul of the cockpit is unlikely in the
short term [8, 50, 51], the transition from analog cockpits to *Glass Cockpits* demonstrates the feasibility of modifying cockpit designs, provided that the changes offer measurable improvements in safety.

## 3 A DESIGN SPACE FOR AUTOPILOT INTERFACES

The main goal of the autopilot interfaces is to control autopilot modes (input) and provide information about the aircraft's behavior (output). We thus introduce a design space that encompasses both input and output to serve as a framework for exploring new cockpit designs (see Fig. 2). We began by selecting research papers about cockpit mode awareness, especially those that address concepts and approaches related to autopilot interfaces. We used an inductive approach [15] to identify three key design dimensions: *Touch*, *Vision* and *Space*.

<figure><img src="image_3.png" /><figcaption>Fig. 2. The Autopilot Interface Design Space is grounded on the literature and existing cockpits along three principal dimensions, **To Touch**, **V Vision**, **S Space**, that are divided into categories. Some categories may contain multiple elements, such as a tangible interface that uses a touchscreen.</figcaption></figure>

**Touch.** The first dimension relates to **To1 Tactile Input** with **To2 Mode Control Philosophy** and **To3 Haptic Feedback**.

**To1 Tactile Input** approaches include tangible, touchscreen, or hybrid combination of both.

Modern “Glass Cockpits” provide physical tangible controllers — knobs, switches and buttons [97] — for interacting with flight data. For example, Boorman et al. [14] developed a novel autopilot design that lets pilots select targets by turning and pushing knobs. Schutte et al.’s [80] HAPTIC-MULTIMODAL FLIGHT CONTROL uses sticks and throttles as physical controllers to point and “shoot” at various targets.

Emerging avionics concepts are replacing traditional cockpit devices with touchscreens. This transition offers several advantages such as direct manipulation, control performance that could be superior to physical instruments [2, 5], and
the flexibility in the positioning of instruments and controls [3, 39]. While touchscreen technology has potential, some studies have shown mixed results regarding its impact. For example, a study by Rouwhorst et al. [70] introduced a touchscreen control panel for mode selection, but the system did not reduce workload or enhance situation awareness in a statistically significant manner and this worsening effect was aggravated under severe turbulence. In fact, touchscreens are being introduced relatively slowly into the cockpits of commercial aircraft, mainly because of the safety issues associated with use in adverse turbulent conditions [17] that impact the accurate interaction with touch-based interfaces, potentially leading to unintended inputs or errors [17, 19, 27].

By combining tangible and touch features, interfaces can counteract the negative effects of turbulence by offering appropriate solutions [35, 62, 63]. Hutchins [35] presented a graphical interface for autopilot mode management systems. Targets values are set by manipulating physical knobs and modes are selected by touching icons that represent the desired mode. Pauchet et al. [62] introduced GAZEFORM, a system that acts as a touchscreen when users' eyes are focused on interaction, but which dynamically transforms into a physical control that emerges when the user's gaze shifts away from the touch interaction area. The results of their user study demonstrated that GAZEFORM generated a lower subjective mental workload and led to a higher efficiency of execution. In another study, Pauchet et al. [63] explored a foldable interactive surface with the goal to make touch interaction in the cockpit more efficient and reliable. Results showed that the folds helped to reduce physical effort by stabilizing the arm and hand. This physicality was also associated with better performance in the piloting task in a turbulent environment [17].

**TO2 Mode Control Philosophy** highlights two different approaches to mode control, where each button can have a *Single mode* or *Multiple mode* attached. Those approaches are different in terms of their operational philosophy and the level of complexity they introduce to the interface.

In the Single-mode-per-button approach, each button on the interface corresponds to a specific mode, and pressing a button directly activates the associated mode. This approach is straightforward and easy to understand, providing a clear one-to-one mapping between buttons and modes. However, it may require a larger number of buttons when numerous modes are used. Boeing's mode controller uses push buttons—or "switches"—for specific mode engagement. A green light within the switch indicates that the buttons are activated. Likewise, IMMI's modes [35] are engaged by pressing icons, and each icon represents a mode.

On the other hand, the multiple-mode-per-button approach corresponds to one input button with several possible outcomes. A first case corresponds to moded input buttons where a single action leads to different results, as the output depends on the initial state of the system [84]. This design has a high potential for causing confusion of pilots [49]. The second case corresponds to a multifunction button that has several available interactions. For example, Airbus uses buttons that can be pushed or pulled [1]. This approach conserves space and reduces clutter, but it introduces an additional layer of complexity in mode management. Since there are different ways of interacting with the button, pilots have to remember what to do to activate the desired modes, which can lead to inadvertent entries or errors [1, 31, 103].

**TO3 Haptic Feedback** presents the significant advantage of providing eye-free information and catching pilot's attention without impacting the current visual focus which could be on the instruments or exterior of the plane. Haptic Feedback can be *grounded* or *wearable*.

Grounded haptic feedback involves system that is integrated into the aircraft's structure or controls. Some aircraft, such as the ATR 72-500, have their yoke moving when the autopilot is activated. Although this feedback can also be characterized as aural or visual, the pilot can feel the aircraft's steering actions by grabbing the yoke. The same applies
to Boeing's moving auto throttles, which provide important information on thrust management. Even if the information is minimal, at least it lets the pilot know the aircraft's current behavior. Similarly, Schutte et al. proposed a flight control system that provide haptic feedback through a back-driven stick and throttle. These moving controllers allow the pilot to physically feel the aircraft's dynamic actions and understand what the automation is doing [80].

Haptic feedback can also serve as a wearable cockpit instrument that provide information on aircraft behavior [96]. Vibrotactile devices convey information such as aircraft attitude, by generating vibrating feedback directly onto the pilot's skin [18]. Sklar and Sarter [86] conducted a simulator study to determine the effectiveness of visual, tactile and combined tactile and visual feedback in detecting and identifying uncommanded mode changes. The study compared the visual feedback of the FMA, a wrist-based vibration and a combination of both visual and tactile cues. Results showed that both the tactile-only and tactile-visual conditions led to higher detection rates and faster reaction times than visual feedback alone. While tactile-only feedback resulted in a few misidentifications, the visual-only and tactile-visual conditions consistently ensured proper identification when transitions were detected. These results are in line with a previous study [57] that compared visual feedback with tactile feedback. A series of studies have delved into the application of tactile stimulation in the torso area. Van Erp et al. [96] introduced a tactile vest made of vibrating tactors that provide information about yaw rotation and demonstrated its potential in mitigating spatial disorientation. Rupert [71] developed the TACTICAL SITUATION AWARENESS SYSTEM (TSAS), a vest that provides orientation information to the pilot through torso tactors, and investigated the impact on pilot's situation awareness of position and orientation. In an evaluation in simulated shipboard departure, pilots reported a better situation awareness with TSAS compared to departure without it.

**Vision.** The visual dimension has two elements that based on the literature are relevant to describe: the **V1 Mode Representation** and the **V2 Field of view for Feedback**.

**V1 Mode Representation** highlights two different approaches that can be used to display mode information: *labels* (written text) or *graphic symbols*.

Autopilots use labels and textual displays to indicate targets, abstract modes used by the system [94] and describe the aircraft's behavior [30]. Unambiguous labels that clearly distinguish between different types of signals help the pilots quickly identify the meaning and involvement of the system [14, 48, 65, 82]. This reduces the risk of feedback ambiguity, which can otherwise lead to a lack of understanding [85].

An alternative approach is the use of icons or graphical representations [35, 50, 56, 83]. Oliver [60] claimed that the most reliable, effective and acceptable means of increasing situational awareness is the use of intuitive pictorial displays "in which the subject matter is shown in graph or picture form, in a manner which does not require a thought process for understanding [60, p.5]". Earlier, Stokes and Wickens [88] provided an extensive discussion of pictorial display formats and discussed about the use of monochrome or color for pictures and schematics. Both the opinion and performance data indicate that the concept of color pictorial formats is better than monochrome pictorial formats in terms of performance [99] and especially in high workload contexts and when the pilot had to recall information from a previously viewed format display [89]. Studies also compared traditional alphanumeric formats with pictorial formats and concluded that when a higher number of modes were presented, the icon-based FMA format improved mode recall over the baseline [95]. Moreover, pictorial display formats outperformed alphanumeric formats at short exposure durations [89]. This advantage tended to diminish with longer exposure durations. Funabiki [32] proposed
a perspective display format that help the pilot to understand the autopilot behavior thank to icons and graphical representations of the current and future trajectory and autopilot modes.

**V2 Field of view for Feedback** refers to the position within the cockpit of the output that can be in the *peripheral vision* or in the *foveal vision*. We define *output* as any component that provides the pilot with information about the currently engaged modes. In modern cockpit configurations, this role is typically fulfilled by the FMA that plays a central role in providing this output.

The location of the FMA on top of the Primary Flight Display (PFD) (Fig. 1) is in peripheral vision most of the time due to the fact that pilots look more often at the PFD [34, 51]. However, the FMA is designed for foveal vision and so it is not salient enough to compete with other visual elements in a dynamic and data-rich context such as PFD [54–56]. The further away pilots’ attentional focus is from the FMA, the less likely they are to notice mode changes. However, Alaman et al. [67] claimed that it is incorrect to state that pilots do not see peripheral changes while gazing on the PFD. The authors concluded that the perceptual span of pilots expands to peripheral vision. Nikolic and Sarter [56] compared the effectiveness of current foveal feedback of the FMA and two implementations of peripheral visual feedback for keeping pilots informed about uncommanded changes in the status of an automated cockpit system. The two peripheral visual feedback types signaled a transition in a more luminous way than the current FMA feedback to capture attention more effectively. Both peripheral visual displays resulted in higher detection rates of unexpected events and faster response times, without interfering with the performance of concurrent visual tasks any more than does the current FMA.

An alternative approach is to place the feedback within the pilots’ foveal vision, i.e. within the PFD. Horn et al. [33] proposed to merge mode information with flight data on the PFD by adding a green box next to the different flight data if the system controls it. This approach embeds the FMA within the natural scanning pattern of pilots, ensuring that the relevant information is presented in the foveal vision, as the stimulus is applied directly to the raw data field frequently viewed by the pilots. Similarly, Sherry and Mauro [83] presented a new design to show airspeed modes on the speed tape of the PFD. Funabiki [32] integrated a set of shapes into the PFD, describing the ideal flight path, flight information and autopilot modes.

**Space.** The spatial dimension has three elements that are considered relevant based on the literature: the **S1 Control distribution**, the **S2 Relative position** between the control and feedback, and the **S3 Feedback distribution**.

**S1 Control distribution** refers to the position of the input within the cockpit that controls modes and targets. The control position could be either *shared* or *independent*.

In the current flight deck, both pilots share the mode control interfaces with the autopilot, located in the middle between the two pilots, in the center of the cockpit (Fig. 1). Placing the mode control panel in a shared and centralized position allows both pilots to interact on the same system, and to observe the actions of the other pilot when using the control interface [7, 11].

Independent location refers to having the control interface in at least two different positions where each pilot has its own control interface. Interfaces can be identical or different. Barbé et al. [6] studied physical ergonomic needs to optimize the integration of touch interaction in the future cockpit concept and found that some display locations, such as in front of the pilots, should be recommended as they require no physical effort to access them. However, this arrangement can result in errors when pilots use the interface simultaneously without communicating with each other, such as the sidesick to manually control aircraft pitch and roll, that have been used simultaneously by both pilots in
the AF447 crash [20, 26]. Rouwhorst et al. [70] presented a touchscreen control panel and initially proposed placing it in front of each pilot. However, prototype design sessions revealed the need to move it into a crew-centered location, since the use of the system in real operational situations required rapid information sharing among crew members.

**S2 Relative position** refers to the position of the output relative to the input, either *delocated* or *colocated*.

In the current flight deck, feedback is physically separated from the control interface. The FMA, located at the top of the PFD provides information about the modes used by the system (Fig. 1). The physical separation of the FMA from the FCU requires pilots to perform actions in one location and to verify the result in another. It may make it difficult to use the FMA information to verify the correct autopilot mode, potentially affecting pilots’ situational awareness because the verification can be forgotten or omitted [61]. A survey investigated problems and operational experiences during flight operations [76]. Some pilots mentioned related problems with verifying the status and behavior of the automation, attributable to the fact that information on the status and behavior of different system components is presented across the flight deck rather than in an integrated manner in a single location.

Previous work suggests that co-locating control and feedback might improve pilot’s situation awareness without adversely affecting performance [40]. Hutchins [35] introduced the Integrated Mode Management Interface (IMMI), a novel mode controller panel designed to facilitate the composition, execution, and evaluation of mode management actions in one centralized location. Hutchins argues that colocating the depiction of mode states and the mode controls, should reduce the frequency of mode selection errors resulting from the crew’s failure to notice changes indicated by the flight mode annunciator. Albinsson and Zhai emphasized that the co-location of input and output, control and feedback, hand action and eye gaze, makes touchscreens highly intuitive to use [3].

**S3 Feedback distribution** refers to the position of the output that provides information about the system's behavior. The feedback position could be either *shared* or *independent*.

In a shared feedback configuration, the output is set so that both pilots can access it simultaneously, enabling them to share the same interface and consequently hopefully have identical mode awareness. Hutchins [35] installed his autopilot mode management interface in a central position which allows pilots to share the same feedback. Mumaw [50] proposed a feedback-oriented display that provides information about the state of the autopilot system. The display was shared and positioned centrally for better visibility. However, both works raise questions about the ideal placement of a new design in a cockpit already overloaded with interfaces.

In an independent feedback configuration, the output is presented in at least two different positions. Feedback may be identical or different depending on position. Pilots do not use the same interface for collecting information, and hence mode awareness might differ. This is the case on commercial aircraft, where the FMA is used for system status awareness. The same information is displayed, but each pilot has an FMA in front of him. In the literature, several studies have used PFD as a support for integrating independent and duplicated feedback designs for mode awareness [32, 33, 83].

# 4 GENERATIVE APPROACH TO USING THE DESIGN SPACE

We follow the generative design approach suggested by Beaudouin-Lafon et al. [10] to first analyze three existing autopilot interfaces with respect to the design space and then critique them to identify current limitations. We then generate new design dimensions that extend the design space and identify directions for future research.
## 4.1 Design Space Analysis of three existing commercial cockpits

We illustrate how to apply the design space by using it to analyze the autopilot systems of the three most well-known commercial cockpits: Airbus A320, Boeing B777 and ATR 72-500. Table 1 illustrates the design configuration of each of the three systems and compares them. It outlines the major and minor design choices of each dimension, details of which are provided below. Major choices are those that define the fundamental characteristics of the system, while minor choices represent aspects that have a more limited impact on the overall use of the system.

<figure><img src="image_4.png" /><figcaption>Table 1. ATR and Boeing share a common design configurations, and differ from Airbus in the **To2 Mode Control Philosophy** (Airbus = Multiple, ATR / Boeing = Single). ATR and Boeing differ slightly in the **S2 Relative Position** where ATR is both **Colocated** (minor) and **Delocated** (major).</figcaption></figure>

**Touch.** All three cockpits use tangible buttons and knobs on the FCU as **To1 Tactile Input**. Pilots interact with the physical interface to select targets and engage autoflight modes. However, there are notable differences in the design of the FCU (Fig. 3).

First, the organization of the buttons is different. Airbus and Boeing use a structured layout, grouping buttons and knobs in clusters for each of the three axes—longitudinal, lateral and vertical. Each knob is located below a target display window that indicates the target only when actively in use.<fnref n="2" />. ATR has a less structured arrangement, with knobs on either side of the push buttons panel. Knobs are not in an individual display window, but instead on an upper display located just above the buttons panel. The altitude is always displayed, the airspeed target is contingent on mode engagement, and the heading target is absent from this display, requiring the pilot to check somewhere else, on another screen.

Second, the design of each knob differs from one manufacturer to another, which is marked by variations in knob shape or engraving. Moreover, Boeing and ATR combine a thumb wheel and conventional knobs (potentiometers) while Airbus uses only conventional knobs.

Third, knobs on Airbus and Boeing are detented, whereas the knobs on ATR are smooth.

The three manufacturers use different **To2 Mode Control Philosophies** to engage modes. Airbus uses the *Multiple-mode-per-button* approach, while ATR and Boeing use the *Single-mode-per-button* approach. Airbus employs a push/pull philosophy where each knob is associated with multiple modes. In broad terms, the four knobs on the FCU (Fig. 3a) are

<footnote n="2">²The altitude target is always displayed.</footnote>
associated with speed, heading, altitude and vertical speed (left to right). The functionality of the first three buttons consists of pushing to give control to the system (managed mode) and pulling to take control and select the target (selected mode). The vertical speed knob operates differently: pushing it levels off the aircraft, and pulling it activates the vertical speed mode. By contrast, ATR and Boeing use push-buttons with each button associated with a *single* mode, providing a distinct mode engagement mechanism.

None of the three cockpits proposes **To3 Haptic Feedback**.

<figure><img src="image_5.png" /><figcaption>Fig. 3. The three FCU use *tangible* knobs. Airbus associates *multiple* modes with one knob, whereas ATR and Boeing opt for the approach of dedicating one button to a *single* mode.</figcaption></figure>

**Vision.** All three cockpits use an FMA for conveying the state of the system, with *textual* **V1 Mode Representation** and *peripheral* **V2 Field of view for Feedback**.

However, there are slight variations in design across manufacturers. Airbus FMA consists of a five-column layout, with the first three columns dedicated to mode awareness. Each column represents *textual* information indicating speed/thrust mode, lateral mode, and vertical mode. Boeing FMA comprises three columns, while ATR adopts a two-column format<fnref n="3" />. Airbus incorporates a few *pictorial* elements for the **V1 Mode Representation** on the target display windows located just above the four knobs (Fig. 3a). The display shows a dot and a dotted line if the mode is managed (target chosen by the system) or displays the target without dot if the mode is selected (target chosen by the pilot). However, this is not optimal to maintain mode awareness.

**Space.** All three cockpits present a *shared* **S1 Control distribution** with the FCU centrally positioned in the cockpit coupled with *independent* **S3 Feedback distribution** with a FMA in front of each pilot (Fig. 1).

The three cockpits depict a *delocated* **S2 Relative position** of the control and feedback elements where the FMA and FCU are separated (Fig. 1). However, the upper central display on the ATR's FCU (Fig. 3b) also displays feedback

<footnote n="3">³ATR 72-500 has no autothrust. Hence, the speed axis is not shown on the FMA. However, it is possible to select a speed target, which the aircraft will maintain not by modifying thrust, but by varying pitch. Selecting a target is therefore an option for climbing or descending.</footnote>
of modes, even if this is not the recommended method for maintaining mode awareness. Consequently, ATR can be characterized as having a *colocated* **S2 Relative position**.

## 4.2 Critical Analysis of the Design Space

Although the three commercial cockpit designs share a common overall design configuration, they differ significantly in details such as shape and button layout. More importantly, none of these cockpits include the following design space elements: *Touchscreen* **To1**, *Wearable* **To3**, and *Grounded* **To3**, *Foveal* **V2**, *Independent* **S1**, and *Shared* **S3**. Two elements, *Pictorial* **V1** and *Colocated* **S2** are only partially used (Table 2). The literature suggests that these eight elements should be integrated or further developed in future autopilot interfaces because they offer considerable advantages for mode awareness.

<figure><img src="image_6.png" /><figcaption>Table 2. Yellow stripes identify elements partially used or omitted in each of three commercial cockpits.</figcaption></figure>

## 4.3 Extending the Design Space

The proposed design space is based on an analysis of prior research with existing cockpits and is not exhaustive. We generate new design dimensions that extend the design space and identify directions for future research. We present an extended version of the design space with additional dimensions that suggest new cockpit design possibilities (Fig. 4).

Specifically, we have added **V3 Gaze-Based Input** to the visual dimension and suggest two new dimensions: *Audition* with **A1 Audio-based Input** and **A2 Audio-based Output**, *Time* with **T1 Feedback delay**. The revised dimensions include:

**Touch** refers to physical touch interaction with the autopilot.

**To1 Input Interaction** uses touch as input.

**To2 Mode Control Philosophy** refers to the number of modes associated to input elements.

**To3 Haptic Feedback** uses touch as output.

**Vision** refers the visual elements of the autopilot's input controls and output displays.

**V1 Mode Representation** describes how modes and autopilot information are presented visually to the pilot.

**V2 Field of view for Feedback** specifies location of visual feedback within the pilot's field of vision.
<figure><img src="image_7.png" /><figcaption>Fig. 4. The Generative Design Space for Autopilot Interfaces adds **V3 Gaze-Based Input** to the visual dimension and introduces two new dimensions: **A** **Audition** and **T1** **Time**. Some categories may contain multiple elements, such as a tangible interface that uses a touchscreen.</figcaption></figure>

**V3 Gaze-Based Input** uses the pilot's gaze as input, including *Grounded* systems [4] and *Wearable* devices, e.g. eye-trackers and head-mounted displays [44].

**Space** refers to the spatial layout of the autopilot input controls and output displays.

**S1 Control distribution** refers to the spatial arrangement of mode control interfaces between the two pilots.

**S2 Relative position** refers to the spatial relationship between input controls and output displays.

**S3 Feedback distribution** refers to the spatial arrangement of displays with respect to the two pilots.

**Time** refers to the temporal aspects of input controls and output displays.

**T11 Feedback delay** refers to the relationship between input location and where feedback appears. The system may present feedback immediately *after the pilot completes an action* to indicate that it has received the input but not necessarily that the input has been fully processed [78]. Alternatively, the system may present feedback *after the system has processed the input* to indicate that the requested action is complete, and unambiguously indicates the current state of the aircraft.
**Audition** provides auditory information without interfering with the pilot's primary visual focus [74, 85].

**A1** *Audio-based Input* uses *verbal* and *non-verbal* cues as an input to control the autopilot.

Although we know of no existing aircraft that lets pilots command the autopilot verbally, Jones [38] compared a prototype voice-based interface with a trackball and touchscreen to manage a multi-window aircraft management system, and concluded that neither is optimal. However, one could imagine using *Verbal* voice-based commands to control modes or convey mode input, beyond the standard inter-pilot information exchange or communication with air traffic controllers.

**A2** *Audio-based Output* refers to *verbal* and *non-verbal* cues as output that describes autopilot behavior. Autoflight systems use alarms and alerts to notify pilots of unexpected or irregular behavior. Although the aviation literature does not address auditory announcements, Monsainger et al. [47] did explore *earcons* to indicate modes within partially automated vehicles. Aircraft cockpits with excessive auditory alerts and warnings can interfere with the pilot's ability to hear critical messages and cause "inattentional deafness" [23, 24].

## 4.4 Limitations and Future work

Although we sought completeness, we clearly may have missed several relevant studies that address additional design space categories or dimensions. We thus propose this design space as a starting point for reflection which should be updated as the technology evolves.

We applied the design space to the three most prevalent commercial cockpits. However, these do not include all existing cockpit instruments, even among these three manufacturers, much less other aircraft. Future work would benefit from a more comprehensive review of cockpits from other manufacturers. This paper is also restricted to commercial autoflight systems. Future exploration of other aircraft types, such as light aviation or helicopters and paradigms such as *Single Pilot Operation* would expand the design space.

Since the key goal of the design space is to inspire new ideas, we do not consider implementation, technical aspects and certification. The cockpit is a complex and dynamic environment and many external conditions can complicate or hinder the integration of new technologies. For example, turbulence, smoke and brightness might affect performance of eye-tracking or touchscreen use. The noisy cockpit setting may influence the effectiveness of auditory input and output. Future work should assess the feasibility of ideas from this design space in real-world aviation and assess their potential for certification. Finally, future studies with pilots' reactions to different combinations of designs based on the design space would provide valuable insights as to the acceptability, appropriability and importance of each dimension.

## 5 CONCLUSION

This paper presents a design space for autopilot interfaces drawn from the existing research literature on autopilot. We use a generative theory approach to systematically analyze and critique three existing commercial flight decks which resulted in the identification of eight novel or rarely used interface designs. We extend the design space with new categories and dimensions to include a total of five dimensions: *Touch, Vision, Audition, Space and Time*. Our goal is to provide a wider range of options for cockpit designers and researchers to create safer, more effective flight decks in the future.

## REFERENCES

[1] Kathy Abbott, Stephen M Slotte, Donald K Stimson, et al. 1996. The interfaces between flightcrews and modern flight deck systems. (1996).
[2] Alexandre Alapetite, Rune Fogh, David Zammit-Mangion, Christian Zammit, Ian Agius, Marco Fabbri, Marco Pregnolato, and Loic Becouran. 2012. Direct tactile manipulation of the flight plan in a modern aircraft cockpit. In *Proceedings of International Conference on Human-Computer Interaction in Aerospace (HCI Aero 2012)*. Association for Computing Machinery, New York, NY, USA.

[3] Pär-Anders Albinsson and Shumin Zhai. 2003. High Precision Touch Screen Interaction. In *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems* (Ft. Lauderdale, Florida, USA) (CHI '03). Association for Computing Machinery, New York, NY, USA, 105–112. https://doi.org/10.1145/642611.642631

[4] Roland Alonso, Mickaël Causse, François Vachon, Robert Parise, Frédéric Dehais, and Patrice Terrier. 2013. Evaluation of head-free eye tracking as an input device for air traffic control. *Ergonomics* 56, 2 (2013), 246–255.

[5] Huseyin Avsar, Joel E. Fischer, and Tom Rodden. 2016. Designing touch screen user interfaces for future flight deck operations. In *2016 IEEE/AIAA 35th Digital Avionics Systems Conference (DASC) (Sacramento, CA, USA)*. IEEE, 1–9. https://doi.org/10.1109/DASC.2016.7779796

[6] Jérôme Barbé, Nicolas Chatrenet, Régis Mollard, Marion Wolff, and Patxi Bérard. 2012. Physical Ergonomics Approach for Touch Screen Interaction in an Aircraft Cockpit. In *Proceedings of the 2012 Conference on Ergonomie et Interaction Homme-Machine* (Biarritz, France) (ErgoIHM '12). Association for Computing Machinery, New York, NY, USA, 9–16. https://doi.org/10.1145/2652574.2653402

[7] Maxime Bardou, Catherine Letondal, Mickaël Causse, Jean-Paul Imbert, and Mathieu Poirier. 2023. Collaborative Design to Support Mutual Awareness in Single-Pilot Operation and Remote Collaboration. In *Proceedings of the 34th Conference on l'Interaction Humain-Machine* (TROYES, France) (IHM '23). Association for Computing Machinery, New York, NY, USA, Article 22, 12 pages. https://doi.org/10.1145/3583961.3583983

[8] Gordon Baxter, Denis Besnard, and Dominic Riley. 2007. Cognitive mismatches in the cockpit: Will they ever be a thing of the past? *Applied ergonomics* 38, 4 (2007), 417–423.

[9] Gordon Baxter, John Rooksby, Yuanzhi Wang, and Ali Khajeh-Hosseini. 2012. The Ironies of Automation: Still Going Strong at 30?. In *Proceedings of the 30th European Conference on Cognitive Ergonomics* (Edinburgh, United Kingdom) (ECCE '12). Association for Computing Machinery, New York, NY, USA, 65–71. https://doi.org/10.1145/2448136.2448149

[10] Michel Beaudouin-Lafon, Susanne Bødker, and Wendy E Mackay. 2021. Generative theories of interaction. *ACM Transactions on Computer-Human Interaction (TOCHI)* 28, 6 (2021), 1–54.

[11] Valentin Becquet, Catherine Letondal, Jean-Luc Vinot, and Sylvain Pauchet. 2019. How Do Gestures Matter for Mutual Awareness in Cockpits? Disclosing Interactions through Graphical Representations. In *Proceedings of the 2019 on Designing Interactive Systems Conference* (San Diego, CA, USA) (DIS '19). Association for Computing Machinery, New York, NY, USA, 593–605. https://doi.org/10.1145/3322276.3322306

[12] Charles E. Billings. 1997. *Aviation automation : the search for a human-centered approach*. Lawrence Erlbaum Associates Publishers, Mahwah, N.J.

[13] Caroline M Björklund, Jens Alfredson, and Sidney WA Dekker. 2006. Mode monitoring and call-outs: An eye-tracking study of two-crew automated flight deck operations. *The International journal of aviation psychology* 16, 3 (2006), 263–275.

[14] Daniel J Boorman, Randall J Mumaw, A Pritchett, and A Jackson. 2004. A new autoflight/FMS interface: Guiding design principles. In *Proceedings of the international conference on human-computer interaction in aeronautics* (Toulouse, France). 303–321.

[15] Virginia Braun and Victoria Clarke. 2012. *Thematic analysis*. American Psychological Association.

[16] Aircraft Accident Investigation Bureau. 2020. Investigation Report On Accident to the B737-MAX8 Reg. Et-Avj Operated By Ethiopian Airlines. 10 March, 2019.

[17] Alma Cantu, Jean-Luc Vinot, Catherine Letondal, Sylvain Pauchet, and Mickaël Causse. 2022. Does Folding Improve the Usability of Interactive Surfaces in Future Airliner Cockpits An Evaluation under Turbulent Conditions and Varying Cognitive Load. In *Proceedings of the 32nd Conference on l'Interaction Homme-Machine* (Virtual Event, France) (IHM '21). Association for Computing Machinery, New York, NY, USA, Article 4, 10 pages. https://doi.org/10.1145/3450522.3451246

[18] Sylvain Cardin, Frédéric Vexo, and Daniel Thalmann. 2006. Vibro-tactile interface for enhancing piloting abilities during long term flight. *Journal of Robotics and Mechatronics* 18, ARTICLE (2006), 381–391.

[19] Andy Cockburn, Carl Gutwin, Philippe Palanque, Yannick Deleris, Catherine Trask, Ashley Coveney, Marcus Yung, and Karon MacLean. 2017. Turbulent Touch: Touchscreen Input for Cockpit Flight Displays. In *Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems* (Denver, Colorado, USA) (CHI '17). Association for Computing Machinery, New York, NY, USA, 6742–6753. https://doi.org/10.1145/3025453.3025584

[20] Stéphane Conversy, Stéphane Chatty, Hélène Gaspard-Boulinc, and Jean-Luc Vinot. 2014. The Accident of Flight AF447 Rio-Paris: A Case Study for HCI Research. In *Proceedings of the 26th Conference on l'Interaction Homme-Machine* (Villeneuve d'Ascq, France) (IHM '14). Association for Computing Machinery, New York, NY, USA, 60–69. https://doi.org/10.1145/2670444.2670459

[21] Renwick E Curry. 1985. *The introduction of new cockpit technology: A human factors study*. Technical Report. NASA.

[22] Asaf Degani and Earl L Wiener. 1994. *On the design of flight-deck procedures*. Technical Report. NASA.

[23] Frédéric Dehais, Mickaël Causse, François Vachon, Nicolas Régis, Eric Menant, and Sébastien Tremblay. 2014. Failure to Detect Critical Auditory Alerts in the Cockpit: Evidence for Inattentional Deafness. *Human Factors* 56, 4 (2014), 631–644. https://doi.org/10.1177/0018720813510735 arXiv:https://doi.org/10.1177/0018720813510735 PMID: 25029890.

[24] F. Dehais, R.N. Roy, and S. Scannella. 2019. Inattentional deafness to auditory alarms: Inter-individual differences, electrophysiological signature and single trial classification. *Behavioural Brain Research* 360 (2019), 51–59. https://doi.org/10.1016/j.bbr.2018.11.045

[25] Bureau d'Enquêtes et d'Analyse. 1992. Accident survenu le 20 janvier 1992 près du Mont Sainte-Odile (Bas Rhin) à l'Airbus A 320 immatriculé F-GGED exploité par la compagnie Air Inter.
[26] Bureau d'Enquêtes et d'Analyse. 2012. Rapport final-Accident survenu le 1er juin 2009 à l'Airbus A330-203 immatriculé F-GZCP exploité par Air France vol AF 447 Rio de Janeiro-Paris.

[27] Sonia Dodd, Jeff Lancaster, Andrew Miranda, Steve Grothe, Bob DeMers, and Bill Rogers. 2014. Touch Screens on the Flight Deck: The Impact of Touch Target Size, Spacing, Touch Technology and Turbulence on Pilot Performance. *Proceedings of the Human Factors and Ergonomics Society Annual Meeting* 58, 1 (2014), 6–10. https://doi.org/10.1177/1541931214581002

[28] Mica R Endsley. 1988. Design and Evaluation for Situation Awareness Enhancement. *Proceedings of the Human Factors Society Annual Meeting* 32, 2 (1988), 97–101. https://doi.org/10.1177/154193128803200221

[29] Mica R. Endsley. 1999. Situation Awareness in Aviation Systems. *Handbook of Aviation Human Factors* 11 (1999), 257–276.

[30] Michael Feary, Martin Alkin, Peter Polson, Daniel McCrobie, Lance Sherry, and Everett Palmer. 1999. Aiding vertical guidance understanding. *Air & Space Europe* 1, 1 (1999), 38–41.

[31] Thomas Ferris, Nadine Sarter, and Christopher D Wickens. 2010. Cockpit automation: Still struggling to catch up.... In *Human factors in aviation*. Elsevier, 479–503.

[32] Kohei Funabiki. 2004. Design of tunnel-in-the-sky display and curved trajectory. In *24th International Congress of the Aeronautical Science, Yokohama, Japan*.

[33] Andreas Horn, Wen-Chin Li, and Graham Braithwaite. 2018. Human-Centered Design of Flight Mode Annunciation for Instantaneous Mode Awareness. In *Engineering Psychology and Cognitive Ergonomics*, Don Harris (Ed.). Springer International Publishing, Cham, 137–146.

[34] G Huettig, G Anders, and A Tautz. 1999. Mode awareness in a modern glass cockpit. In *Proceedings of the 10th International Symposium on Aviation Psychology*, Vol. 1. Ohio State University, Columbus, OH, USA, 130–135.

[35] Edwin L. Hutchins. 1996. *The Integrated Mode Management Interface*. Technical Report. Ames Research Center of the National Aeronautics and Space Administration in the Aviation Safety/ Automation Program.

[36] Greg A Jamieson, Gyrd Skraaning, and Jeffrey Joe. 2022. The B737 MAX 8 Accidents as Operational Experiences With Automation Transparency. *IEEE Transactions on Human-Machine Systems* 52, 4 (2022), 794–797.

[37] Eric N Johnson and Amy R Pritchett. 1995. Experimental study of vertical flight path mode awareness. *IFAC Proceedings Volumes* 28, 15 (1995), 153–158.

[38] Denise R Jones. 1990. *Three input concepts for flight crew interaction with information presented on a large-screen electronic cockpit display*. Technical Report. National Aeronautics and Space Administration.

[39] Sridher Kaminani. 2011. Human computer interaction issues with touch screen interfaces in the flight deck. In *2011 IEEE/AIAA 30th Digital Avionics Systems Conference*. IEEE, 6B4–1–6B4–7. https://doi.org/10.1109/DASC.2011.6096098

[40] Wen-Chin Li, James White, Graham Braithwaite, Matt Greaves, and Jr-Hung Lin. 2016. The Evaluation of Pilot's Situational Awareness During Mode Changes on Flight Mode Annunciators. In *Engineering Psychology and Cognitive Ergonomics*, Don Harris (Ed.). Springer International Publishing, Cham, 409–418.

[41] Gavan Lintern, Tyler Waite, and Donald A Talleur. 1999. Functional interface design for the modern aircraft cockpit. *The International Journal of Aviation Psychology* 9, 3 (1999), 225–240.

[42] Sara E McBride, Wendy A Rogers, and Arthur D Fisk. 2014. Understanding human management of automation errors. *Theoretical Issues in Ergonomics Science* 15, 6 (2014), 545–577.

[43] Andrew J McClumpha and Marianne Rudisill. 2000. Certification for civil flight decks and the human-computer interface. *Human Factors in Certification* (2000), 289–299.

[44] S Merchant and T Schnell. 2000. Applying eye tracking as an alternative approach for activation of controls and functions in aircraft. In *19th DASC. 19th Digital Avionics Systems Conference. Proceedings (Cat. No. 00CH37126)*, Vol. 2. IEEE, 5A5–1.

[45] Steven P Miller, Sarah Barber, Timothy M Carlson, David L Lempia, and Alan C Tribble. 2002. A methodology for improving mode awareness in flight guidance design. In *Proceedings. The 21st Digital Avionics Systems Conference*, Vol. 2. IEEE, 7D1–7D1.

[46] Jim Mitchell, Leopold Petrus Vermeulen, and Prevendren Naidoo. 2009. Flying glass: A qualitative analysis of pilot perceptions of automated flight-decks after 20 years. (2009).

[47] Noé Monsaingeon, Loïc Caroux, Sabine Langlois, and Céline Lemercier. 2023. Earcons to reduce mode confusions in partially automated vehicles: Development and application of an evaluation method. *International Journal of Human-Computer Studies* 176 (2023), 103044.

[48] R Mumaw, Daniel J Boorman, and RL Prada. 2006. Experimental evaluation of a new autoflight interface. In *Proceedings HCI-Aero 2006, International Conference on Human Computer Interaction, Seattle, WA*.

[49] Randall J Mumaw. 2020. Addressing mode confusion using an interpreter display. *NASA Contractor Report* (2020).

[50] Randall J Mumaw. 2021. Plan B for Eliminating Mode Confusion: An Interpreter Display. *International Journal of Human-Computer Interaction* 37, 7 (2021), 693–702.

[51] Randall J Mumaw, N Sarter, and Christopher D Wickens. 2001. Analysis of pilots' monitoring and performance on an automated flight deck. In *Proceedings of the 11th International Symposium on Aviation Psychology*, Vol. 8. Ohio State University, Columbus, OH, USA.

[52] David C Nagel. 1988. Human Error in Aviation Operations. In *Human Factors in Aviation*, Earl L. Wiener and David C. Nagel (Eds.). Academic Press, San Diego, 263–303. https://doi.org/10.1016/B978-0-08-057090-7.50015-1

[53] Prevendren Naidoo et al. 2009. *Airline pilots' perceptions of advanced flight deck automation*. Ph.D. Dissertation. University of Pretoria.
[54] M Nikolic, James Orr, and Nadine B Sarter. 2001. Why onsets don't always capture attention: The importance of context in display design. In *Proceedings of the 11th International Symposium of Aviation Psychology*. Columbus, OH, USA.

[55] Mark I Nikolic, James M Orr, and Nadine B Sarter. 2004. Why pilots miss the green box: How display context undermines attention capture. *The International Journal of Aviation Psychology* 14, 1 (2004), 39–52.

[56] Mark I Nikolic and Nadine B Sarter. 2001. Peripheral visual feedback: A powerful means of supporting effective attention allocation in event-driven, data-rich environments. *Human factors* 43, 1 (2001), 30–38.

[57] Mark I Nikolic, Aaron E Sklar, and Nadine B Sarter. 1998. Multisensory feedback in support of pilot-automation coordination: the case of uncommanded mode transitions. In *Proceedings of the human factors and ergonomics society annual meeting*, Vol. 42. SAGE Publications Sage CA: Los Angeles, CA, 239–243.

[58] Donald A Norman. 1990. The 'problem' with automation: inappropriate feedback and interaction, not 'over-automation'. *Philosophical Transactions of the Royal Society of London. B, Biological Sciences* 327, 1241 (1990), 585–593.

[59] Bureau of Air Safety Investigation. 1998. *Advanced technology aircraft safety survey report*. Technical Report.

[60] Joseph G Oliver. 1990. *Improving situational awareness through the use of intuitive pictorial displays*. Technical Report. SAE Technical Paper.

[61] Everett Palmer. 1995. Oops, it didn't arm-a case study of two automation surprises. In *Proceedings of the Eighth International Symposium on Aviation Psychology*. Ohio State University, Columbus, OH, USA, 227–232.

[62] Sylvain Pauchet, Catherine Letondal, Jean-Luc Vinot, Mickaël Causse, Mathieu Cousy, Valentin Becquet, and Guillaume Crouzet. 2018. GazeForm: Dynamic Gaze-Adaptive Touch Surface for Eyes-Free Interaction in Airliner Cockpits. In *Proceedings of the 2018 Designing Interactive Systems Conference* (Hong Kong, China) (DIS '18). Association for Computing Machinery, New York, NY, USA, 1193–1205. https://doi.org/10.1145/3196709.3196712

[63] Sylvain Pauchet, Jean-Luc Vinot, Catherine Letondal, Alexandre Lemort, Claire Lavenir, Timothée Lecomte, Stéphanie Rey, Valentin Becquet, and Guillaume Crouzet. 2019. Multi-Plié: A Linear Foldable and Flattenable Interactive Display to Support Efficiency, Safety and Collaboration. In *Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems* (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3290605.3300384

[64] Katherine L Plant and Neville A Stanton. 2012. Why did the pilots shut down the wrong engine? Explaining errors in context using Schema Theory and the Perceptual Cycle Model. *Safety science* 50, 2 (2012), 300–315.

[65] L Ricardo Prada, Randall J Mumaw, Deborah A Boehm-Davis, and Daniel J Boorman. 2006. Testing Boeing's Flight Deck of the Future: A Comparison Between Current and Prototype Autoflight Panels. *Proceedings of the Human Factors and Ergonomics Society Annual Meeting* 50, 1 (2006), 55–58. https://doi.org/10.1177/154193120605000112

[66] Amy R Pritchett. 2009. Aviation automation: General perspectives and specific guidance for the design of modes and alerts. *Reviews of human factors and ergonomics* 5, 1 (2009), 82–113.

[67] Jonay Ramón Alamán, Mickaël Causse, and Vsevolod Peysakhovich. 2020. Attentional span of aircraft pilots: did you look at the speed? In *Eye-Tracking in Aviation. Proceedings of the 1st International Workshop (ETAVI 2020)*. ISAE-SUPAERO, Université de Toulouse; Institute of Cartography and Geoinformation (IKG), ETH Zurich, 108–113.

[68] Victor Riley, Bob DeMers, Chris Misiak, and Brian Schmalz. 1999. The cockpit control language: A pilot-centered avionics interface. *Air & Space Europe* 1, 1 (1999), 48–50.

[69] JJM Roessingh, RNHW Van Gent, G Fletcher, H Dudfield, M Lodge, F Koehl, and B Linsenmaier. 1999. Training for the 'glass cockpit' - Trends, problems and solutions. In *International Symposium on Aviation Psychology, 10 th, Columbus, OH*. 985–991.

[70] Wilfred Rouhorst, Ronald Verhoeven, Marieke Suijkerbuijk, Tanja Bos, Anneloes Maij, Mick Vermaat, and Roy Arents. 2017. Use of touch screen display applications for aircraft flight control. In *2017 IEEE/AIAA 36th Digital Avionics Systems Conference (DASC)* (St. Petersburg, FL, USA). IEEE, 1–10.

[71] Angus H Rupert. 2000. An instrumentation solution for reducing spatial disorientation mishaps. *IEEE Engineering in Medicine and Biology Magazine* 19, 2 (2000), 71–80.

[72] NB Sarter. 1991. The flight management system-pilots' interaction with cockpit automation. In *Conference Proceedings 1991 IEEE International Conference on Systems, Man, and Cybernetics* (Charlottesville, VA, USA), Vol. 2. IEEE, 1307–1310.

[73] N Sarter, Christopher Wickens, R Mumaw, S Kimball, R Marsh, M Nikolic, and W Xu. 2003. Modern flight deck automation: Pilots' mental model and monitoring patterns and performance. In *Proceedings of the International Symposium on Aviation Psychology* (Dayton, OH, USA).

[74] Nadine B Sarter. 2000. The need for multisensory interfaces in support of effective attention allocation in highly dynamic event-driven domains: the case of cockpit automation. *The International Journal of Aviation Psychology* 10, 3 (2000), 231–245.

[75] Nadine B Sarter and David D Woods. 1992. Pilot interaction with cockpit automation: Operational experiences with the flight management system. *The International Journal of Aviation Psychology* 2, 4 (1992), 303–321.

[76] Nadine B Sarter and David D Woods. 1997. Team play with a powerful and independent agent: Operational experiences and automation surprises on the Airbus A-320. *Human factors* 39, 4 (1997), 553–569.

[77] Nadine B Sarter and David D Woods. 2000. Team play with a powerful and independent agent: A full-mission simulation study. *Human Factors* 42, 3 (2000), 390–402.

[78] Dominique L. Scapin and J. M. Christian Bastien. 1997. Ergonomic criteria for evaluating the ergonomic quality of interactive systems. *Behaviour & Information Technology* 16, 4-5 (nov 1997), 220–231. http://www.tandfonline.com/doi/abs/10.1080/014492997119806
[79] Albrecht Schmidt and Thomas Herrmann. 2017. Intervention user interfaces: a new interaction paradigm for automated systems. *Interactions* 24, 5 (2017), 40–45.

[80] Paul Schutte, Kenneth Goodrich, and Ralph Williams. 2012. Towards an improved pilot-vehicle interface for highly automated aircraft: evaluation of the haptic flight control system. In *4th AHFE International Conference on Applied Human Factors and Ergonomics 2012*.

[81] Lance Sherry and Oleksandra Sasha Donnelly. 2019. Design of a Display for Alerting Cockpit Automation Function Configuration Changes. In *2019 Integrated Communications, Navigation and Surveillance Conference (ICNS). IEEE*, 1–15.

[82] Lance Sherry, Michael Feary, Peter Polson, and Everett Palmer. 2001. What's it doing now? Taking the covers off autopilot behavior. In *Proceedings of the 11th International Symposium on Aviation Psychology*. Ohio State University, Columbus, OH, USA, 1–6.

[83] Lance Sherry and Robert Mauro. 2014. Design of Cockpit displays to explicitly support flight crew intervention tasks. In *2014 IEEE/AIAA 33rd Digital Avionics Systems Conference (DASC). IEEE*, 2B5–1.

[84] Lance Sherry, Robert Mauro, and Oleksandra Donnelly. 2022. Alert! The Appearance of Moded Input Devices in the Modern Airliner Cockpit. *IEEE Transactions on Human-Machine Systems* 52, 6 (2022), 1186–1195.

[85] Sathya S. Silva and R. John Hansman. 2015. Divergence Between Flight Crew Mental Model and Aircraft System State in Auto-Throttle Mode Confusion Accident and Incident Cases. *Journal of Cognitive Engineering and Decision Making* 9, 4 (Dec. 2015), 312–328. https://doi.org/10.1177/1555343415597344 Publisher: SAGE Publications.

[86] Aaron E Sklar and Nadine B Sarter. 1999. Good vibrations: Tactile feedback in support of attention allocation and human-automation coordination in event-driven domains. *Human factors* 41, 4 (1999), 543–552.

[87] Kassandra Kim Yoke Soo, Timothy J Mavin, and Yoriko Kikkawa. 2021. Mastering Automation: New Airline Pilots' Perspective. *International Journal of Human-Computer Interaction* 37, 7 (2021), 717–727.

[88] Alan Stokes and Christopher Wickens. 1988. Chapter 12 - Aviation Displays. In *Human factors in aviation*, Earl L Wiener and David C Nagel (Eds.). Gulf Professional Publishing, 387–431.

[89] Michael Newton Stollings. 1982. Information processing load of graphic versus alphanumeric weapon format displays for advanced fighter cockpits. (1982).

[90] Robert Sumwalt, David Cross, and Dennis Lessard. 2015. Examining how breakdowns in pilot monitoring of the aircraft flight path. *International Journal of Aviation, Aeronautics, and Aerospace* 2, 3 (2015), 8.

[91] Komite Nasional Keselamatan Transportas. 2019. Aircraft Accident Investigation Report, PT. Lion Mentari Airlines Boeing 737-8 (MAX); PK-LQP Tanjung Karawang, West Java, 29 October 2018.

[92] Sanjay S Vakil and R John Hansman. 2000. *Analysis of complexity evolution management and human performance issues in commercial aircraft automation systems*. Technical Report. MIT International Center for Air Transportation.

[93] Sanjay S Vakil and R John Hansman Jr. 2002. Approaches to mitigating complexity-driven issues in commercial autoflight systems. *Reliability Engineering & System Safety* 75, 2 (2002), 133–145.

[94] Sanjay S Vakil, Alan H Midkiff, and R John Hansman. 1996. *Development and evaluation of an electronic vertical situation display*. Technical Report. International Center for Air Transportation.

[95] Jaap Jan van den Bosch and Harrie GM Bohnen. 1998. An experimental comparison of alphanumeric and iconic formats of the Flight Mode Annunciator. *The International Journal of Aviation Psychology* 8, 4 (1998), 335–349.

[96] Jan BF Van Erp, Eric L Groen, Jelte E Bos, and Hendrik AHC Van Veen. 2006. A tactile cockpit instrument supports the control of self-motion during spatial disorientation. *Human factors* 48, 2 (2006), 219–228.

[97] Jean-Luc Vinot, Catherine Letondal, Sylvain Pauchet, and Stéphane Chatty. 2016. Could Tangibility Improve the Safety of Touch-Based Interaction? Exploring a New Physical Design Space for Pilot-System Interfaces. In *Proceedings of the International Conference on Human-Computer Interaction in Aerospace (HCI-Aero '16)*. Association for Computing Machinery, New York, NY, USA, Article 8, 8 pages. https://doi.org/10.1145/2950112.2964581

[98] Dong-Bach Vo, Sylvain Pauchet, Isabelle Jolly, Florine Simon, Anke M. Brock, and Jérémie Garcia. 2023. Tactilient: Turbulence Resilient Tactile Icons for Pilot Feedback. In *Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems* (Hamburg, Germany) (CHI '23). Association for Computing Machinery, New York, NY, USA, Article 210, 13 pages. https://doi.org/10.1145/3544548.3580951

[99] TC Way, ME Hornsby, JD Gilmour, RE Edwards, and RE Hobbs. 1984. *Pictorial format display evaluation*. Technical Report. Air Force Flight Dynamics Laboratory.

[100] Christopher D Wickens. 1995. Designing for situation awareness and trust in automation. *IFAC Proceedings Volumes* 28, 23 (1995), 365–370.

[101] Earl L Wiener. 1985. *Human factors in cockpit automation: A field study of flight crew transition*. Technical Report. NASA.

[102] Earl L Wiener. 1989. *Human factors of advanced technology (glass cockpit) transport aircraft*. Technical Report. National Aeronautics and Space Administration.

[103] Earl L Wiener. 1993. *Intervention strategies for the management of human error*. Technical Report.

[104] John A Wise. 2002. Human Factors Certification in Civil Aircraft. In *Proceedings of the Human Factors and Ergonomics Society Annual Meeting*, Vol. 46. SAGE Publications, Los Angeles, CA, USA, 1883–1885.