# A human factors' approach for multimodal collaboration with Cognitive Computing to create a Human Intelligent Machine Team: a Review

Charles-Alban Dormoy, Jean-Marc André, Alain Pagani

► To cite this version:

Charles-Alban Dormoy, Jean-Marc André, Alain Pagani. A human factors' approach for multimodal collaboration with Cognitive Computing to create a Human Intelligent Machine Team: a Review. 10th EASN International Conference on Innovation in Aviation & Space to the Satisfaction of the European Citizens (10th EASN 2020), Sep 2020, Online, United States. pp.012105, 10.1088/1757-899x/1024/1/012105. [https://doi.org/10.5285/epub.12899](https://doi.org/10.5285/epub.12899)

HAL Id: hal-04612548

https://hal.science/hal-04612548

Submitted on 14 Jun 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
PAPER • OPEN ACCESS

# A human factors' approach for multimodal collaboration with Cognitive Computing to create a Human Intelligent Machine Team: a Review

To cite this article: Charles Dormoy et al 2021 IOP Conf. Ser.: Mater. Sci. Eng. 1024 012105

View the [article online](#) for updates and enhancements.

You may also like

- A comparison of methods for fully automatic segmentation of tumors and involved nodes in PET/CT of head and neck cancers
Aurora Rosvoll Groendahl, Ingerid Skjei Knudtsen, Bao Ngoc Huynh et al.

- Factors associated with emerging multimodal transportation behavior in the San Francisco Bay Area
Emily McAuliffe Wells, Mitchell Small, C Anna Spurlock et al.

- Some mathematical aspects of Anderson localization: boundary effect, multimodality, and bifurcation
Chen Jia, Ziqi Liu and Zhimin Zhang

<figure><img src="image_1.png" /></figure>

The Electrochemical Society
Advancing solid state & electrochemical science & technology

DISCOVER
how sustainability
intersects with
electrochemistry & solid
state science research

<figure><img src="image_2.png" /></figure>

<figure><img src="image_3.png" /></figure>
PAPER • OPEN ACCESS

# A human factors' approach for multimodal collaboration with Cognitive Computing to create a Human Intelligent Machine Team: a Review

To cite this article: Charles Dormoy et al 2021 IOP Conf. Ser.: Mater. Sci. Eng. 1024 012105

View the [article online](#) for updates and enhancements.

You may also like

- A comparison of methods for fully automatic segmentation of tumors and involved nodes in PET/CT of head and neck cancers
Aurora Rosvoll Groendahl, Ingerid Skjei Knudtsen, Bao Ngoc Huynh et al.

- Factors associated with emerging multimodal transportation behavior in the San Francisco Bay Area
Emily McAuliffe Wells, Mitchell Small, C Anna Spurlock et al.

- Some mathematical aspects of Anderson localization: boundary effect, multimodality, and bifurcation
Chen Jia, Ziqi Liu and Zhimin Zhang

<figure><img src="image_2.png" /></figure>

The Electrochemical Society
Advancing solid state & electrochemical science & technology

DISCOVER
how sustainability
intersects with
electrochemistry & solid
state science research

<figure><img src="image_3.png" /></figure>
# A human factors' approach for multimodal collaboration with Cognitive Computing to create a Human Intelligent Machine Team: a Review

Charles Dormoy<fnref n="1" />, Jean-Marc André<fnref n="2" />, Alain Pagani<fnref n="3" />

**Abstract.** In the Horizon 2020 funded Clean Sky 2 programme, the Cognitive Collaboration for Teaming project aims at positioning a roadmap towards Single Pilot Operations (SPO) and human intelligent machine teaming. Built on top of the states-of-the art of human factors, a careful examination of the factors and parameters to be taken into account to form a Human intelligent Machine Team (HiMT) was carried out. Each parameter can influence positively or negatively teaming. A good HiMT is a HiMT that communicates, shares knowledge, information, collaborates and trusts each other to ensure flight safety at the highest level. This review shows the added value of multimodality for bidirectional communication in the HiMT. Multimodality will allow both verbal and non-verbal communication to be maintained in both directions. A review was conducted on the benefits of each of the modalities and means of interaction under different conditions and how each modality can complement each other for better bidirectional communication that is natural, efficient, and reliable. The objective is to transmit information in a clear, accurate and concise manner, but also to ensure that it is well received by the receiver (i.e. the CCTeammate and the pilot) and well understood. Each modality will allow the CCTeammate to present and/or represent the information in the best possible way to the pilot depending on the context and the task at hand.

## 1. Introduction

During a flight, pilots must master complex situations, while facing increasing system complexity due to the amount and type of information available. As part of the Horizon 2020-funded programme Clean Sky 2, Co2Team (Cognitive Collaboration for Teaming) pursues the idea that a system based on artificial intelligence can effectively support the pilot using cognitive computing towards single pilot operations (SPO). To create collaboration between the pilot and the intelligent agent, an innovative bi-directional communication paradigm and an intelligent allocation of roles and tasks are needed, based on the concept of keeping "pilot in the loop". In this way, it is possible to benefit from the most effective pilot skills, such as decision making, to achieve flight objectives and guarantee a maximum level of safety and better acceptability.

### 1.1 Towards Single Pilot Operations (SPO)

One of the objectives for airlines and aircraft manufacturers in the years to come is to move towards SPO and maintain the level of flight safety at its highest level in all circumstances. But SPO without extra help for the pilot or without a total redesign of the cockpit adapted to SPO is hardly conceivable. A study of single-pilot operations in a legacy cockpit [1] showed that flight performance decreased, and that safety margins and workload were considered unacceptable by pilots, especially in an emergency situation. So, to help the pilot, different strategies are envisaged such as assistance provided by automated systems and replace the co-pilot and his/her role as Pilot Monitoring by automation. Increasing automation in a cockpit would reinforce the automation paradox and increase the burden of

<figure><img src="image_4.png" /></figure>

Content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.

<footnote n="1">¹CATIE 1 Avenue du Dr Albert Schweitzer, 33400 Talence, France</footnote>

<footnote n="2">²Bordeaux INP-ENSC, IMS UMR CNRS 5218, 109 Av. Roul, 33400 Talence, France</footnote>

<footnote n="3">³DFKI Trippstadter Str. 122, 67663 Kaiserslautern, Germany</footnote>
# A human factors' approach for multimodal collaboration with Cognitive Computing to create a Human Intelligent Machine Team: a Review

Charles Dormoy¹, Jean-Marc André², Alain Pagani³

¹CATIE 1 Avenue du Dr Albert Schweitzer, 33400 Talence, France

²Bordeaux INP-ENSC, IMS UMR CNRS 5218, 109 Av. Roul, 33400 Talence, France

³DFKI Trippstadter Str. 122, 67663 Kaiserslautern, Germany

**Abstract.** In the Horizon 2020 funded Clean Sky 2 programme, the Cognitive Collaboration for Teaming project aims at positioning a roadmap towards Single Pilot Operations (SPO) and human intelligent machine teaming. Built on top of the states-of-the art of human factors, a careful examination of the factors and parameters to be taken into account to form a Human intelligent Machine Team (HiMT) was carried out. Each parameter can influence positively or negatively teaming. A good HiMT is a HiMT that communicates, shares knowledge, information, collaborates and trusts each other to ensure flight safety at the highest level. This review shows the added value of multimodality for bidirectional communication in the HiMT. Multimodality will allow both verbal and non-verbal communication to be maintained in both directions. A review was conducted on the benefits of each of the modalities and means of interaction under different conditions and how each modality can complement each other for better bidirectional communication that is natural, efficient, and reliable. The objective is to transmit information in a clear, accurate and concise manner, but also to ensure that it is well received by the receiver (i.e. the CCTeammate and the pilot) and well understood. Each modality will allow the CCTeammate to present and/or represent the information in the best possible way to the pilot depending on the context and the task at hand.

## 1. Introduction

During a flight, pilots must master complex situations, while facing increasing system complexity due to the amount and type of information available. As part of the Horizon 2020-funded programme Clean Sky 2, Co2Team (Cognitive Collaboration for Teaming) pursues the idea that a system based on artificial intelligence can effectively support the pilot using cognitive computing towards single pilot operations (SPO). To create collaboration between the pilot and the intelligent agent, an innovative bi-directional communication paradigm and an intelligent allocation of roles and tasks are needed, based on the concept of keeping "pilot in the loop". In this way, it is possible to benefit from the most effective pilot skills, such as decision making, to achieve flight objectives and guarantee a maximum level of safety and better acceptability.

### 1.1 Towards Single Pilot Operations (SPO)

One of the objectives for airlines and aircraft manufacturers in the years to come is to move towards SPO and maintain the level of flight safety at its highest level in all circumstances. But SPO without extra help for the pilot or without a total redesign of the cockpit adapted to SPO is hardly conceivable. A study of single-pilot operations in a legacy cockpit [1] showed that flight performance decreased, and that safety margins and workload were considered unacceptable by pilots, especially in an emergency situation. So, to help the pilot, different strategies are envisaged such as assistance provided by automated systems and replace the co-pilot and his/her role as Pilot Monitoring by automation. Increasing automation in a cockpit would reinforce the automation paradox and increase the burden of

<figure><img src="image_4.png" /></figure>

Content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.
system monitoring [2, 3] by the pilot alone in the cockpit. To minimize this effect, several research
works [4, 5] are studying the establishment of a relationship of trust by creating a Human Automation
Team (HAT). Co2Team's project goes further by proposing a Human Intelligent Machine Team (HiMT)
with cognitive computing as a teammate for the pilot instead of more automation. The recommendations
proposed by Cummings et al. and Shively et al. [5, 6] for HAT comply with Co2Team Cognitive
Computing Teammate (CCTeammate). It keeps verbal and non-verbal communication, with a good
HMI², and create a HiMT with the human that understands the machine and the machine that understands
the human and establishes mutual trust. The CCTeammate of the Co2Team project aims to help the
transition to SPO but will not replace the pilot in the role of decision maker (captain) and Pilot Flying.

## 2. What is Cognitive Computing (CC)

CC and Artificial Intelligence (AI) use the same technologies i.e. machine learning, deep learning, Natural Language Processing (NLP), neural networks and so on. Both can solve complex problems, but they have different purposes. CC aims at improving human decision-making and focuses on natural interaction with humans. Whereas AI aims at automating processes and solve problems without human intervention. CC attempts to build computer systems modelled after the human brain (using deep learning approach), which possess NLP, learning and decision-making capability. CC systems have six major characteristics:

1) Information adaptability to integrate big data from heterogeneous sources and then creating ideas from them (e.g. multi-sensory integration)

2) Dynamic Training and adaptation - learning and chaining as they receive new information, new analyses, new users, new intersections, or new activity

3) Probabilistic - discovering relevant patterns based on context, statistically generating and evaluating series of evidence-based hypotheses, predicting the probability of valuable connections, and returning answer based on learning inferencing. This also includes detection of unexpected patterns.

4) Highly integrated - automated systems workload management through which all modules contribute to a central learning system and are affected by new data, interactions and historical data.

5) Meaning-based - performing natural language processing and using embedded analytics to leverage language structures, semantics and relationships.

6) Highly interactive - providing tools and interactions design to facilitate advanced communications within the integrated system and incorporating human-computer interactions (HCI), data analysis and visualizations [7].

CC is a barrier-free communication between human and computer in a natural language (in verbal and nonverbal forms) is one of the primary goals of CC systems. The CC systems should augment human abilities assisting their decision-making processes in the real world [8].

The functionality of CC however, strongly depends on the methods and techniques of AI. In such a way, cognitive computing is an AI based system with a strong focus on helping humans to make decisions by communicating with them in a natural manner. That is why a CC system would have more chance to pass through the Turing Test than an AI system.

## 3. Human factors as the main and central approach

The state of the art carried out, examines the different human factors (HF) identified in the literature that are involved in the integration of an intelligent agent for cockpit operations with all that this implies in terms of complexity.

The Human intelligent Machine Team (HiMT) will operate in a legacy cockpit and will therefore deal with the automated systems of the aircraft. The review highlights the benefits and weaknesses of avionics automation [9–12] and how cognitive computing could address or help manage them. The human tendency towards automation bias is of particular concern, as it occurs when a human decision-maker ignores or does not seek conflicting information and favours the computer-generated solution that is accepted as correct [13–15]. This bias affects teams of people as much as individuals [16]. Co2Team wants to take a different approach and see how cognitive computing can help the pilot in a different way than automatization and avoid automation bias. On the contrary of avionics systems that are not designed and does not behave like a team member, the focus here is on how to create a HiMT.

Previous attempts in the field of aviation have been studied, such as the French "Electronic Copilot (FR)" for Rafale [17–20], the US "Pilot Associate" by DARPA (Defense Advanced Research Projects
system monitoring [2, 3] by the pilot alone in the cockpit. To minimize this effect, several research
works [4, 5] are studying the establishment of a relationship of trust by creating a Human Automation
Team (HAT). Co2Team's project goes further by proposing a Human Intelligent Machine Team (HiMT)
with cognitive computing as a teammate for the pilot instead of more automation. The recommendations
proposed by Cummings et al. and Shively et al. [5, 6] for HAT comply with Co2Team Cognitive
Computing Teammate (CCTeammate). It keeps verbal and non-verbal communication, with a good
HMI², and create a HiMT with the human that understands the machine and the machine that understands
the human and establishes mutual trust. The CCTeammate of the Co2Team project aims to help the
transition to SPO but will not replace the pilot in the role of decision maker (captain) and Pilot Flying.

## 2. What is Cognitive Computing (CC)

CC and Artificial Intelligence (AI) use the same technologies i.e. machine learning, deep learning, Natural Language Processing (NLP), neural networks and so on. Both can solve complex problems, but they have different purposes. CC aims at improving human decision-making and focuses on natural interaction with humans. Whereas AI aims at automating processes and solve problems without human intervention. CC attempts to build computer systems modelled after the human brain (using deep learning approach), which possess NLP, learning and decision-making capability. CC systems have six major characteristics:

1) Information adaptability to integrate big data from heterogeneous sources and then creating ideas from them (e.g. multi-sensory integration)

2) Dynamic Training and adaptation - learning and chaining as they receive new information, new analyses, new users, new intersections, or new activity

3) Probabilistic - discovering relevant patterns based on context, statistically generating and evaluating series of evidence-based hypotheses, predicting the probability of valuable connections, and returning answer based on learning inferencing. This also includes detection of unexpected patterns.

4) Highly integrated - automated systems workload management through which all modules contribute to a central learning system and are affected by new data, interactions and historical data.

5) Meaning-based - performing natural language processing and using embedded analytics to leverage language structures, semantics and relationships.

6) Highly interactive - providing tools and interactions design to facilitate advanced communications within the integrated system and incorporating human-computer interactions (HCI), data analysis and visualizations [7].

CC is a barrier-free communication between human and computer in a natural language (in verbal and nonverbal forms) is one of the primary goals of CC systems. The CC systems should augment human abilities assisting their decision-making processes in the real world [8].

The functionality of CC however, strongly depends on the methods and techniques of AI. In such a way, cognitive computing is an AI based system with a strong focus on helping humans to make decisions by communicating with them in a natural manner. That is why a CC system would have more chance to pass through the Turing Test than an AI system.

## 3. Human factors as the main and central approach

The state of the art carried out, examines the different human factors (HF) identified in the literature that are involved in the integration of an intelligent agent for cockpit operations with all that this implies in terms of complexity.

The Human intelligent Machine Team (HiMT) will operate in a legacy cockpit and will therefore deal with the automated systems of the aircraft. The review highlights the benefits and weaknesses of avionics automation [9–12] and how cognitive computing could address or help manage them. The human tendency towards automation bias is of particular concern, as it occurs when a human decision-maker ignores or does not seek conflicting information and favours the computer-generated solution that is accepted as correct [13–15]. This bias affects teams of people as much as individuals [16]. Co2Team wants to take a different approach and see how cognitive computing can help the pilot in a different way than automatization and avoid automation bias. On the contrary of avionics systems that are not designed and does not behave like a team member, the focus here is on how to create a HiMT.

Previous attempts in the field of aviation have been studied, such as the French "Electronic Copilot (FR)" for Rafale [17–20], the US "Pilot Associate" by DARPA (Defense Advanced Research Projects
Agency) and US Air Force's Wright Laboratory [21], Rotorcraft Pilot's Associate (RPA) [22–24]. And also more recent attempts such as IBM and Airbus CIMON [25–27].

## 3.1 Teaming

Co2Team aims to build a HiMT, so a review has been conducted to look at what cooperation is, how it works for human-human cooperation, and for human-machine cooperation [28–32]. A recent experiment in 2019 [33, 34] showed that human-AI cooperation is possible and that the team's performance is improved as a result. But Co2Team is about human- intelligent agent collaboration, which is why a precise study has been carried out on what collaboration is and the differences with cooperation. In both there is an allocation of tasks and roles between the team members, but collaboration is spontaneous and based on commitment. It is more an active and voluntary act where each member of the team is responsible for the whole [35–38]. The cultural effect and context on communication and collaboration in a small-group was considered, based on what is already applied, such as Crew Resource Management. Since CRM trainings have given pilots the tools to move from good cooperation to better collaboration, it may be important to maintain this even in the SPO and to adapt it to the HiMT [39]. More specifically Co2Team means Cognitive collaboration for Teaming, collaboration was presented, but what is teaming from a cognitive point of view for each individual and as a team? Team cognition [40] is a dynamic team activity inextricably linked to context that applies to the team (i.e. shared mental model [5, 37, 41]). This implies that the CCTeammate must also be equipped with the ability to deduce certain elements from a real situation (i.e. theory of mind ToM [42–44]). In other words, it means knowing the capacities and limits of the other teammate and being able for each member of the team to deduce, foresee and adapt his/her behavior, actions, needs.

The use of AI does not escape the need for mutual trust between the pilot and the CCT, nor from the trust the team will establish in the avionics. Therefore, an in-depth review has been carried out on how trust is built [45, 46] and maintained at the right level through positive and negative feedback, to avoid under and over trust [13, 14, 47–56]. Human decision making (DM) process [57–61] has also been considered as well as risk management to understand how the CCT may assist the pilot [62, 63], using the same DM model as the pilots (e.g. FORDEC). CCT will serve all the necessary insight to provide decision makers, the pilot with the information needed to make better and data-driven decisions and minimize the risks of DM bias and avoid some cognitive bias.

## 4. Multimodal Human Machine Interface and Interactions (HMI²)

Always with a human factors (HF) centered approach a review was conducted about the interactions between man and machine and how these interactions would be with a Cognitive Computing Teammate (CCT). The aim being to form a hybrid HiMT, the review focuses on the different aspects of multimodal communication (MMC) to keep the most familiar way of communication and collaboration for the pilot (i.e. Human-Human). It will be important to keep the verbal and non-verbal parts of communication. This last point is all the more important as the absence of non-verbal communication leads to more confusion (especially when discussing charts or briefings), and more difficulties in interpersonal tasks that require knowing the state of the teammate [64]. Bidirectional MMC is therefore promoted, aiming not only at keeping the non-verbal bi-directional communication, but also at task efficiency, fluidity and understanding. And it's through this bi-directional MMC that the HiMT will be possible. The use of bi-directional MMC should also ensure good usability and provide a form of redundancy and diversity (e.g. verbal + visual) which is one of the most useful tools to reduce the probability of a major fault and thus ensure good reliability of the interaction.

Most importantly the CCT will have to adapt the MMC to the context (e.g. ongoing task, gaze) but also adapt the content and representation to the context and modalities chosen. Jones (1990) [65] suggested, that if each modality performs better in relation to a specific task in a specific context, and if they complement each other and address each other's weaknesses, then combining them may be the most natural and effective method of interaction. But the use of MMC can overload the visual and audio space if it is poorly designed. A good MMC with adapted HMIs must not create a cognitive, visual, or auditory overload for the pilot. The means used for MMC must complement each other but not add up on the same channels.

The HF and user-centred approach led us to conduct semi structured interviews and cognitive walkthroughs with pilots to find out their needs and desires concerning the CCTeammate and this MMC.
Agency) and US Air Force's Wright Laboratory [21], Rotorcraft Pilot's Associate (RPA) [22–24]. And also more recent attempts such as IBM and Airbus CIMON [25–27].

## 3.1 Teaming

Co2Team aims to build a HiMT, so a review has been conducted to look at what cooperation is, how it works for human-human cooperation, and for human-machine cooperation [28–32]. A recent experiment in 2019 [33, 34] showed that human-AI cooperation is possible and that the team's performance is improved as a result. But Co2Team is about human- intelligent agent collaboration, which is why a precise study has been carried out on what collaboration is and the differences with cooperation. In both there is an allocation of tasks and roles between the team members, but collaboration is spontaneous and based on commitment. It is more an active and voluntary act where each member of the team is responsible for the whole [35–38]. The cultural effect and context on communication and collaboration in a small-group was considered, based on what is already applied, such as Crew Resource Management. Since CRM trainings have given pilots the tools to move from good cooperation to better collaboration, it may be important to maintain this even in the SPO and to adapt it to the HiMT [39]. More specifically Co2Team means Cognitive collaboration for Teaming, collaboration was presented, but what is teaming from a cognitive point of view for each individual and as a team? Team cognition [40] is a dynamic team activity inextricably linked to context that applies to the team (i.e. shared mental model [5, 37, 41]). This implies that the CCTeammate must also be equipped with the ability to deduce certain elements from a real situation (i.e. theory of mind ToM [42–44]). In other words, it means knowing the capacities and limits of the other teammate and being able for each member of the team to deduce, foresee and adapt his/her behavior, actions, needs.

The use of AI does not escape the need for mutual trust between the pilot and the CCT, nor from the trust the team will establish in the avionics. Therefore, an in-depth review has been carried out on how trust is built [45, 46] and maintained at the right level through positive and negative feedback, to avoid under and over trust [13, 14, 47–56]. Human decision making (DM) process [57–61] has also been considered as well as risk management to understand how the CCT may assist the pilot [62, 63], using the same DM model as the pilots (e.g. FORDEC). CCT will serve all the necessary insight to provide decision makers, the pilot with the information needed to make better and data-driven decisions and minimize the risks of DM bias and avoid some cognitive bias.

## 4. Multimodal Human Machine Interface and Interactions (HMI²)

Always with a human factors (HF) centered approach a review was conducted about the interactions between man and machine and how these interactions would be with a Cognitive Computing Teammate (CCT). The aim being to form a hybrid HiMT, the review focuses on the different aspects of multimodal communication (MMC) to keep the most familiar way of communication and collaboration for the pilot (i.e. Human-Human). It will be important to keep the verbal and non-verbal parts of communication. This last point is all the more important as the absence of non-verbal communication leads to more confusion (especially when discussing charts or briefings), and more difficulties in interpersonal tasks that require knowing the state of the teammate [64]. Bidirectional MMC is therefore promoted, aiming not only at keeping the non-verbal bi-directional communication, but also at task efficiency, fluidity and understanding. And it's through this bi-directional MMC that the HiMT will be possible. The use of bi-directional MMC should also ensure good usability and provide a form of redundancy and diversity (e.g. verbal + visual) which is one of the most useful tools to reduce the probability of a major fault and thus ensure good reliability of the interaction.

Most importantly the CCT will have to adapt the MMC to the context (e.g. ongoing task, gaze) but also adapt the content and representation to the context and modalities chosen. Jones (1990) [65] suggested, that if each modality performs better in relation to a specific task in a specific context, and if they complement each other and address each other's weaknesses, then combining them may be the most natural and effective method of interaction. But the use of MMC can overload the visual and audio space if it is poorly designed. A good MMC with adapted HMIs must not create a cognitive, visual, or auditory overload for the pilot. The means used for MMC must complement each other but not add up on the same channels.

The HF and user-centred approach led us to conduct semi structured interviews and cognitive walkthroughs with pilots to find out their needs and desires concerning the CCTeammate and this MMC.
Based on these interviews and the literature review, the CCTeammate will use as its primary modality of interaction the one favoured by the pilots i.e. oral. Oral communication using NLP as well as the other modalities and HMIs will have to be adapted to the culture of the pilots, the airlines and the manufacturer using the same phraseology, the right units, and the right terms, etc.

The different possibilities of interaction were investigated as well as the interests of each of them under different conditions and for both way of communication. The survey starts with the different inputs the pilot can provide, like the one already implemented like the voice via the microphone of the headset, the KCCU and the touch screen (TS). And those that are interesting to implement such as eye and gestures tracking. In addition speech to text (STT) and convolutional neural networks (CNN) technologies can be used for speech emotion recognition (SER) [66, 67]. Mainly because it is through speech that people communicate emotions, cognitive states and intentions [68]. So oral will be used as main communication modality, it is the one promoted by the pilots, it is the most natural and needs no additional equipment. TS can also be used, allowing a quick direct interaction [69–71]. But it cannot be used as main interaction modality because it requires a free hand and gaze [72, 73]. Eye and gesture tracking [74–82] can be used in a complementary way to enable non-verbal communication like pointing at something. The infrared cameras used for gesture tracking can also be used for facial emotion recognition [83, 84] to complement the SER as the emotions of the pilots cannot be neglected in the HiMT. Emotions affect, among other things, attention, startle effect, motivation, and ToM. Talking to an intelligent machine incapable of recognising and reacting to emotions and without empathy would be like talking to a sociopath you do not want to team up with.

But how the CCT can communicate in a multimodal way with the pilot using the different senses of the pilot? Oral communication will also be chosen as the main modality of communication e.g. NLP Text to Speech and also to convey emotions such as smooth talking to calm the pilot in a stressful situation. It will be possible to use a visual support e.g. if there is too much information at the same time. But it is not interesting or even useless to display information Head-down if the pilot is looking Head-up. For this it will be interesting to use the Augmented Reality to display information wherever the pilot is looking and to draw pilot's attention.

The Figure 1 summarizes the different modalities according to each direction of communication. Human factors and certification recommendations have been considered in this review to select an HMI² with good usability. The goal being for the pilot to have an intelligent teammate who is pleasant to use, reliable, and resilient.

<figure><img src="image_5.png" /><figcaption>Figure 1: Bi-directional MMC</figcaption></figure>

**5. Conclusion**

To move towards SPO, additional assistance is needed to ensure acceptable working conditions for pilots and maintain a high level of safety. This work examines the different human factors identified and involved in the integration of an intelligent agent for cockpit operations with all that this implies in terms of complexity (work, tasks and environment). And the factors involved in forming a hybrid team with a pilot, the HiMT. In the course of reading and research, a selection has been made of the factors that seem to be most relevant in the literature and most adapted to the intelligent agent. The goal being to
Based on these interviews and the literature review, the CCTeammate will use as its primary modality of interaction the one favoured by the pilots i.e. oral. Oral communication using NLP as well as the other modalities and HMIs will have to be adapted to the culture of the pilots, the airlines and the manufacturer using the same phraseology, the right units, and the right terms, etc.

The different possibilities of interaction were investigated as well as the interests of each of them under different conditions and for both way of communication. The survey starts with the different inputs the pilot can provide, like the one already implemented like the voice via the microphone of the headset, the KCCU and the touch screen (TS). And those that are interesting to implement such as eye and gestures tracking. In addition speech to text (STT) and convolutional neural networks (CNN) technologies can be used for speech emotion recognition (SER) [66, 67]. Mainly because it is through speech that people communicate emotions, cognitive states and intentions [68]. So oral will be used as main communication modality, it is the one promoted by the pilots, it is the most natural and needs no additional equipment. TS can also be used, allowing a quick direct interaction [69–71]. But it cannot be used as main interaction modality because it requires a free hand and gaze [72, 73]. Eye and gesture tracking [74–82] can be used in a complementary way to enable non-verbal communication like pointing at something. The infrared cameras used for gesture tracking can also be used for facial emotion recognition [83, 84] to complement the SER as the emotions of the pilots cannot be neglected in the HiMT. Emotions affect, among other things, attention, startle effect, motivation, and ToM. Talking to an intelligent machine incapable of recognising and reacting to emotions and without empathy would be like talking to a sociopath you do not want to team up with.

But how the CCT can communicate in a multimodal way with the pilot using the different senses of the pilot? Oral communication will also be chosen as the main modality of communication e.g. NLP Text to Speech and also to convey emotions such as smooth talking to calm the pilot in a stressful situation. It will be possible to use a visual support e.g. if there is too much information at the same time. But it is not interesting or even useless to display information Head-down if the pilot is looking Head-up. For this it will be interesting to use the Augmented Reality to display information wherever the pilot is looking and to draw pilot's attention.

The Figure 1 summarizes the different modalities according to each direction of communication. Human factors and certification recommendations have been considered in this review to select an HMI² with good usability. The goal being for the pilot to have an intelligent teammate who is pleasant to use, reliable, and resilient.

<figure><img src="image_5.png" /><figcaption>Figure 1: Bi-directional MMC</figcaption></figure>

**5. Conclusion**

To move towards SPO, additional assistance is needed to ensure acceptable working conditions for pilots and maintain a high level of safety. This work examines the different human factors identified and involved in the integration of an intelligent agent for cockpit operations with all that this implies in terms of complexity (work, tasks and environment). And the factors involved in forming a hybrid team with a pilot, the HiMT. In the course of reading and research, a selection has been made of the factors that seem to be most relevant in the literature and most adapted to the intelligent agent. The goal being to
design a HiMT with the human that understands the machine and the machine that understands the human and to maintain mutual trust. Based on a thorough study of the literature, a review and selection of each means and modality of communication was carried out for each of the two directions of communication. The survey studied the advantages and disadvantages of each of them and how they complement each other in their use, adding up their strengths and complementing their weaknesses. The use of multimodality will ensure operational resilience. Human factors and certification recommendations were considered in this review to select an HMI with good usability. The objective being for the pilot to have an intelligent teammate who is pleasant to use, reliable, resilient, understandable with few constraints and with whom s/he will be able to communicate and collaborate effectively. By taking human factors into account, the power of human-machine symbiosis can be enhanced. Pilot and CCT collaborate by doing what they do best. Co2Team is in line with the vision of Horizon Europe's CleanSky2 programme with the use of the CCT, which could serve as an intermediate step towards opening avionics to the open world and intelligent unmanned aircraft.

## 6. Acknowledgement

This work presented is part of the project Co2Team that received funding from the Clean Sky 2 Joint Undertaking under the European Union's Horizon 2020 research and innovation programme under grant agreement No 831891.

## References

[1] Bailey RE, Kramer LJ, Kennedy KD, et al. An assessment of reduced crew and single pilot operations in commercial transport aircraft operations. IEEE, pp. 1–15.

[2] Bainbridge L. Ironies of automation. Automatica 1983; 19: 775–779.

[3] Parasuraman R, Molloy R, Singh I. Performance Consequences of Automation Induced Complacency. International Journal of Aviation Psychology; 3. Epub ahead of print 1 February 1993. DOI: 10.1207/s15327108ijap0301_1.

[4] Matessa M, Vu K-PL, Strybel TZ, et al. Using distributed simulation to investigate human-autonomy teaming. Springer, 2018, pp. 541–550.

[5] Shively RJ, Lachter J, Brandt SL, et al. Why Human-Autonomy Teaming? In: Baldwin C (ed) Advances in Neuroergonomics and Cognitive Engineering. Cham: Springer International Publishing, pp. 3–11.

[6] Cummings ML, Stimpson A, Clamann M. Functional requirements for onboard intelligent automation in single pilot operations. In: AIAA Infotech@ Aerospace. 2016, p. 1652.

[7] Noor AK. Potential of Cognitive Computing and Cognitive Systems. Open Engineering; 5. Epub ahead of print 27 January 2014. DOI: 10/gf4stp.

[8] Rouse M. What is Cognitive Computing? Special Report: Artificial intelligence apps come of age, 2016, https://searchenterprisei.techtarget.com/definition/cognitive-computing (2016).

[9] Chidester TR. Introducing FMS aircraft into airline operations. In: Coping with Computers in the Cockpit. 1999.

[10] Billings CE. AVIATION AUTOMATION: THE SEARCH FOR A HUMAN-CENTERED APPROACH, https://trid.trb.org/view/572335 (1997, accessed 15 October 2018).

[11] Woods D, Sarter N. Learning from Automation Surprises and ‘Going Sour’ Accidents: Progress on Human-Centered Automation.

[12] Woods DD, Johannesen LJ, Cook RI, et al. Behind human error: Cognitive systems, computers and hindsight. Dayton Univ Research Inst (Urdi) OH, 1994.

[13] Parasuraman R, Manzey DH. Complacency and Bias in Human Use of Automation: An Attentional Integration. Human Factors: The Journal of the Human Factors and Ergonomics Society 2010; 52: 381–410.

[14] Parasuraman R, Riley V. Humans and automation: Use, misuse, disuse, abuse. Human factors 1997; 39: 230–253.

[15] Mosier KL, Skitka LJ. 10 Human Decision Makers and Automated Decision Aids: Made for Each Other? Automation and human performance: Theory and applications 2018; 120.

[16] Skitka LJ, Mosier KL, Burdick M, et al. Automation bias and errors: are crews better than individuals? The International journal of aviation psychology 2000; 10: 85–97.

[17] Hourlier S, Grau J-Y, Valot C. A human factors approach to adaptive aids. Virtual and adaptive
design a HiMT with the human that understands the machine and the machine that understands the human and to maintain mutual trust. Based on a thorough study of the literature, a review and selection of each means and modality of communication was carried out for each of the two directions of communication. The survey studied the advantages and disadvantages of each of them and how they complement each other in their use, adding up their strengths and complementing their weaknesses. The use of multimodality will ensure operational resilience. Human factors and certification recommendations were considered in this review to select an HMI with good usability. The objective being for the pilot to have an intelligent teammate who is pleasant to use, reliable, resilient, understandable with few constraints and with whom s/he will be able to communicate and collaborate effectively. By taking human factors into account, the power of human-machine symbiosis can be enhanced. Pilot and CCT collaborate by doing what they do best. Co2Team is in line with the vision of Horizon Europe's CleanSky2 programme with the use of the CCT, which could serve as an intermediate step towards opening avionics to the open world and intelligent unmanned aircraft.

## 6. Acknowledgement

This work presented is part of the project Co2Team that received funding from the Clean Sky 2 Joint Undertaking under the European Union's Horizon 2020 research and innovation programme under grant agreement No 831891.

## References

[1] Bailey RE, Kramer LJ, Kennedy KD, et al. An assessment of reduced crew and single pilot operations in commercial transport aircraft operations. IEEE, pp. 1-15.

[2] Bainbridge L. Ironies of automation. Automatica 1983; 19: 775-779.

[3] Parasuraman R, Molloy R, Singh I. Performance Consequences of Automation Induced Complacency. International Journal of Aviation Psychology; 3. Epub ahead of print 1 February 1993. DOI: 10.1207/s15327108ijap0301_1.

[4] Matessa M, Vu K-PL, Strybel TZ, et al. Using distributed simulation to investigate human-autonomy teaming. Springer, 2018, pp. 541-550.

[5] Shively RJ, Lachter J, Brandt SL, et al. Why Human-Autonomy Teaming? In: Baldwin C (ed) Advances in Neuroergonomics and Cognitive Engineering. Cham: Springer International Publishing, pp. 3-11.

[6] Cummings ML, Stimpson A, Clamann M. Functional requirements for onboard intelligent automation in single pilot operations. In: AIAA Infotech@ Aerospace. 2016, p. 1652.

[7] Noor AK. Potential of Cognitive Computing and Cognitive Systems. Open Engineering; 5. Epub ahead of print 27 January 2014. DOI: 10/gf4stp.

[8] Rouse M. What is Cognitive Computing? Special Report: Artificial intelligence apps come of age, 2016, https://searchenterprisei.techtarget.com/definition/cognitive-computing (2016).

[9] Chidester TR. Introducing FMS aircraft into airline operations. In: Coping with Computers in the Cockpit. 1999.

[10] Billings CE. AVIATION AUTOMATION: THE SEARCH FOR A HUMAN-CENTERED APPROACH, https://trid.trb.org/view/572335 (1997, accessed 15 October 2018).

[11] Woods D, Sarter N. Learning from Automation Surprises and 'Going Sour' Accidents: Progress on Human-Centered Automation.

[12] Woods DD, Johannesen LJ, Cook RI, et al. Behind human error: Cognitive systems, computers and hindsight. Dayton Univ Research Inst (Urdi) OH, 1994.

[13] Parasuraman R, Manzey DH. Complacency and Bias in Human Use of Automation: An Attentional Integration. Human Factors: The Journal of the Human Factors and Ergonomics Society 2010; 52: 381-410.

[14] Parasuraman R, Riley V. Humans and automation: Use, misuse, disuse, abuse. Human factors 1997; 39: 230-253.

[15] Mosier KL, Skitka LJ. 10 Human Decision Makers and Automated Decision Aids: Made for Each Other? Automation and human performance: Theory and applications 2018; 120.

[16] Skitka LJ, Mosier KL, Burdick M, et al. Automation bias and errors: are crews better than individuals? The International journal of aviation psychology 2000; 10: 85-97.

[17] Hourlier S, Grau J-Y, Valot C. A human factors approach to adaptive aids. Virtual and adaptive
environments, LJ Hettinger & MW Haas (eds) Lawrence Erlbaum Associates: Mahwah, New Jersey 2003; 461–482.

[18] Hourlier S, Grau J-Y, Amalberti R. The “Electronic Copilot,” a Human Factors Approach to Pilot Assistance. 0148–7191, SAE Technical Paper, 1999.

[19] Amalberti R, Valot C. From field work analysis to a cognitive model and the design of support systems: assistance to fighter pilots. In: Proceedings of IEEE Systems Man and Cybernetics Conference - SMC. Le Touquet, France: IEEE, pp. 682–688.

[20] Hourlier S. New «Intelligence» Coming to the Cockpit... Again? Springer, 2018, pp. 275–282.

[21] Banks SB, Lizza CS. Pilot's Associate: a cooperative, knowledge-based system application. IEEE Expert 1991; 6: 18–29.

[22] Colucci F. Rotorcraft Pilot's Associate update- The Army's largest science and technology program. Vertiflite 1995; 41: 16–20.

[23] Miller CA, Hannen MD. The Rotorcraft Pilot's Associate: design and evaluation of an intelligent user interface for cockpit information management. Knowledge-Based Systems 1999; 12: 443–456.

[24] Befort K, Baker B. Rotorcraft Pilot's Associate. In: Loper ML (ed) Modeling and Simulation in the Systems Engineering Life Cycle. London: Springer London, pp. 323–331.

[25] CIMON brings AI to the International Space Station. IBM Innovation Explanations, https://www.ibm.com/thought-leadership/innovation_explanations/article/cimon-ai-in-space.html (2019).

[26] CIMON-2 Masters Its Debut on the International Space Station. IBM News Room, https://newsroom.ibm.com/2020-04-15-CIMON-2-Masters-Its-Debut-on-the-International-Space-Station (2020).

[27] CIMON-2 makes its successful debut on the ISS. Airbus, https://www.airbus.com/newsroom/press-releases/en/2020/04/cimon2-makes-its-successful-debut-on-the-iss.html (2020).

[28] Hoc J-M. Towards a cognitive approach to human-machine cooperation in dynamic situations. International Journal of Human-Computer Studies 2001; 54: 509–540.

[29] Pacaux-Lemoine MP, Debernard S. Common work space for human-machine cooperation in air traffic control. Control Engineering Practice 2002; 10: 571–576.

[30] Millot P, Pacaux-Lemoine M-P. A Common Work Space for a mutual enrichment of Human-machine Cooperation and Team-Situation Awareness. IFAC Proceedings Volumes 2013; 46: 387–394.

[31] Pacaux M-P, Godin SAD, Rajaonah B, et al. Levels of automation and human-machine cooperation: Application to human-robot interaction. IFAC Proceedings Volumes 2011; 44: 6484–6492.

[32] Debernard S, Guiost B, Poulain T, et al. Integrating human factors in the design of intelligent systems: an example in air traffic control. IJISTA 2009; 7: 205.

[33] Jaderberg M, Czarnecki WM, Dunning I, et al. Human-level performance in 3D multiplayer games with population-based reinforcement learning. Science 2019; 364: 859–865.

[34] Gent E. Artificial intelligence learns teamwork in a deadly game of capture the flag. Science. Epub ahead of print 30 May 2019. DOI: 10/gf4sh2.

[35] Gangloff-Ziegler C. Les freins au travail collaboratif. Marché et organisations 2009; 10: 95.

[36] Orasanu J, Fischer U, Tada Y, et al. Team Stress and Performance: Implications for Long-Duration Space Missions. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2004; 48: 552–556.

[37] Alberts DS. Understanding information age warfare. Washington, DC: CCRP Publication Series, 2001.

[38] Brangier É, Hammes-Adelé S, Bastien J-MC. Analyse critique des approches de l'acceptation des technologies : de l'utilisabilité à la symbiose humain-technologie-organisation. Revue Européenne de Psychologie Appliquée/European Review of Applied Psychology 2010; 60: 129–146.

[39] Shively RJ, Lachter J, Koteskey R, et al. Crew Resource Management for Automated Teammates (CRM-A). In: Harris D (ed) Engineering Psychology and Cognitive Ergonomics. Cham: Springer International Publishing, pp. 215–229.
environments, LJ Hettinger & MW Haas (eds) Lawrence Erlbaum Associates: Mahwah, New Jersey 2003; 461–482.

[18] Hourlier S, Grau J-Y, Amalberti R. The “Electronic Copilot,” a Human Factors Approach to Pilot Assistance. 0148–7191, SAE Technical Paper, 1999.

[19] Amalberti R, Valot C. From field work analysis to a cognitive model and the design of support systems: assistance to fighter pilots. In: Proceedings of IEEE Systems Man and Cybernetics Conference - SMC. Le Touquet, France: IEEE, pp. 682–688.

[20] Hourlier S. New «Intelligence» Coming to the Cockpit... Again? Springer, 2018, pp. 275–282.

[21] Banks SB, Lizza CS. Pilot's Associate: a cooperative, knowledge-based system application. IEEE Expert 1991; 6: 18–29.

[22] Colucci F. Rotorcraft Pilot's Associate update- The Army's largest science and technology program. Vertiflite 1995; 41: 16–20.

[23] Miller CA, Hannen MD. The Rotorcraft Pilot's Associate: design and evaluation of an intelligent user interface for cockpit information management. Knowledge-Based Systems 1999; 12: 443–456.

[24] Befort K, Baker B. Rotorcraft Pilot's Associate. In: Loper ML (ed) Modeling and Simulation in the Systems Engineering Life Cycle. London: Springer London, pp. 323–331.

[25] CIMON brings AI to the International Space Station. IBM Innovation Explanations, https://www.ibm.com/thought-leadership/innovation_explanations/article/cimon-ai-in-space.html (2019).

[26] CIMON-2 Masters Its Debut on the International Space Station. IBM News Room, https://newsroom.ibm.com/2020-04-15-CIMON-2-Masters-Its-Debut-on-the-International-Space-Station (2020).

[27] CIMON-2 makes its successful debut on the ISS. Airbus, https://www.airbus.com/newsroom/press-releases/en/2020/04/cimon2-makes-its-successful-debut-on-the-iss.html (2020).

[28] Hoc J-M. Towards a cognitive approach to human-machine cooperation in dynamic situations. International Journal of Human-Computer Studies 2001; 54: 509–540.

[29] Pacaux-Lemoine MP, Debernard S. Common work space for human-machine cooperation in air traffic control. Control Engineering Practice 2002; 10: 571–576.

[30] Millot P, Pacaux-Lemoine M-P. A Common Work Space for a mutual enrichment of Human-machine Cooperation and Team-Situation Awareness. IFAC Proceedings Volumes 2013; 46: 387–394.

[31] Pacaux M-P, Godin SAD, Rajaonah B, et al. Levels of automation and human-machine cooperation: Application to human-robot interaction. IFAC Proceedings Volumes 2011; 44: 6484–6492.

[32] Debernard S, Guiost B, Poulain T, et al. Integrating human factors in the design of intelligent systems: an example in air traffic control. IJISTA 2009; 7: 205.

[33] Jaderberg M, Czarnecki WM, Dunning I, et al. Human-level performance in 3D multiplayer games with population-based reinforcement learning. Science 2019; 364: 859–865.

[34] Gent E. Artificial intelligence learns teamwork in a deadly game of capture the flag. Science. Epub ahead of print 30 May 2019. DOI: 10/gf4sh2.

[35] Gangloff-Ziegler C. Les freins au travail collaboratif. Marché et organisations 2009; 10: 95.

[36] Orasanu J, Fischer U, Tada Y, et al. Team Stress and Performance: Implications for Long-Duration Space Missions. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2004; 48: 552–556.

[37] Alberts DS. Understanding information age warfare. Washington, DC: CCRP Publication Series, 2001.

[38] Brangier É, Hammes-Adelé S, Bastien J-MC. Analyse critique des approches de l'acceptation des technologies : de l'utilisabilité à la symbiose humain-technologie-organisation. Revue Européenne de Psychologie Appliquée/European Review of Applied Psychology 2010; 60: 129–146.

[39] Shively RJ, Lachter J, Koteskey R, et al. Crew Resource Management for Automated Teammates (CRM-A). In: Harris D (ed) Engineering Psychology and Cognitive Ergonomics. Cham: Springer International Publishing, pp. 215–229.
[40] Cooke NJ, Gorman JC, Myers CW, et al. Interactive Team Cognition. Cogn Sci 2013; 37: 255–285.

[41] Scali G, Macredie R. Shared Mental Models as a Way of Managing Transparency in Complex Human-Autonomy Teaming: In: Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications. Prague, Czech Republic: SCITEPRESS - Science and Technology Publications, pp. 153–159.

[42] Premack D, Woodruff G. Does the chimpanzee have a theory of mind? Behav Brain Sci 1978; 1: 515–526.

[43] Hall L, Woods S, Aylett R, et al. USING THEORY OF MIND METHODS TO INVESTIGATE EMPATHIC ENGAGEMENT WITH SYNTHETIC CHARACTERS. Int J Human Robot 2006; 03: 351–370.

[44] Lemaignan S, Warnier M, Sisbot EA, et al. Artificial cognition for social human-robot interaction: An implementation. Artificial Intelligence 2016; 247: 45–69.

[45] Karsenty L. Comment maintenir des relations de confiance et construire du sens face à une crise ? Le travail humain 2015; 78: 141.

[46] Körber M, Baseler E, Bengler K. Introduction matters: Manipulating trust in automation and reliance in automated driving. Applied Ergonomics 2018; 66: 18–31.

[47] Kyriakidis M, de Winter JCF, Stanton N, et al. A human factors perspective on automated driving. Theoretical Issues in Ergonomics Science 2019; 20: 223–249.

[48] Liu P, Yang R, Xu Z. Public Acceptance of Fully Automated Driving: Effects of Social Trust and Risk/Benefit Perceptions: Public Acceptance of Fully Automated Driving. Risk Analysis 2019; 39: 326–341.

[49] Hoff KA, Bashir M. Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust. Hum Factors 2015; 57: 407–434.

[50] Koltai K, Ho N, Masequesmay G, et al. Influence of Cultural, Organizational, and Automation Capability on Human Automation Trust: A Case Study of Auto-GCAS Experimental Test Pilots. 2014.

[51] Lee JD, Moray N. Trust, self-confidence, and operators’ adaptation to automation. International Journal of Human-Computer Studies 1994; 40: 153–184.

[52] Fleury S. Le rôle de l'utilisateur dans les systèmes de traitements automatiques. 2014; 204.

[53] Lee J, Moray N. Trust, control strategies and allocation of function in human-machine systems. Ergonomics 1992; 35: 1243–1270.

[54] Madhavan P, Wiegmann DA, Lacson FC. Automation Failures on Tasks Easily Performed by Operators Undermine Trust in Automated Aids. Hum Factors 2006; 48: 241–256.

[55] Bahner JE, Hüper A-D, Manzey D. Misuse of automated decision aids: Complacency, automation bias and the impact of training experience. International Journal of Human-Computer Studies 2008; 66: 688–699.

[56] Lee JD, See KA. Trust in Automation: Designing for Appropriate Reliance. Human Factors: The Journal of the Human Factors and Ergonomics Society 2004; 46: 50–80.

[57] Misulis KE, Frisse ME. Decision-Making. In: Frisse ME, Misulis KE (eds) Essentials of Clinical Informatics. Oxford University Press, pp. 104–125.

[58] Schacter DL, Gilbert DT, Wegner DM. Psychology. 2nd ed. New York, NY: Worth Publishers, 2011.

[59] Inman JJ, Dyer JS, Jia J. A Generalized Utility Model of Disappointment and Regret Effects on Post-Choice Valuation. Marketing Science 1997; 16: 97–111.

[60] Hiel A van, Mervielde I. The Need for Closure and the Spontaneous Use of Complex and Simple Cognitive Structures. The Journal of Social Psychology 2003; 143: 559–568.

[61] Schaefer KE, Perelman BS, Brewer RW, et al. Quantifying Human Decision-Making: Implications for Bidirectional Communication in Human-Robot Teams. In: Chen JYC, Fragomeni G (eds) Virtual, Augmented and Mixed Reality: Interaction, Navigation, Visualization, Embodiment, and Simulation. Cham: Springer International Publishing, pp. 361–379.

[62] Barnes MJ, Lakhmani S, Holder E, et al. Issues in Human-Agent Communication. 2019; 29.

[63] Metge A, Maille N. IA et assistance à la décision humaine : influence des informations transmises sur l'évaluation d'un nouveau plan de vol. 2020.
[40] Cooke NJ, Gorman JC, Myers CW, et al. Interactive Team Cognition. Cogn Sci 2013; 37: 255–285.

[41] Scali G, Macredie R. Shared Mental Models as a Way of Managing Transparency in Complex Human-Autonomy Teaming: In: Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications. Prague, Czech Republic: SCITEPRESS - Science and Technology Publications, pp. 153–159.

[42] Premack D, Woodruff G. Does the chimpanzee have a theory of mind? Behav Brain Sci 1978; 1: 515–526.

[43] Hall L, Woods S, Aylett R, et al. USING THEORY OF MIND METHODS TO INVESTIGATE EMPATHIC ENGAGEMENT WITH SYNTHETIC CHARACTERS. Int J Human Robot 2006; 03: 351–370.

[44] Lemaignan S, Warnier M, Sisbot EA, et al. Artificial cognition for social human-robot interaction: An implementation. Artificial Intelligence 2016; 247: 45–69.

[45] Karsenty L. Comment maintenir des relations de confiance et construire du sens face à une crise ? Le travail humain 2015; 78: 141.

[46] Körber M, Baseler E, Bengler K. Introduction matters: Manipulating trust in automation and reliance in automated driving. Applied Ergonomics 2018; 66: 18–31.

[47] Kyriakidis M, de Winter JCF, Stanton N, et al. A human factors perspective on automated driving. Theoretical Issues in Ergonomics Science 2019; 20: 223–249.

[48] Liu P, Yang R, Xu Z. Public Acceptance of Fully Automated Driving: Effects of Social Trust and Risk/Benefit Perceptions: Public Acceptance of Fully Automated Driving. Risk Analysis 2019; 39: 326–341.

[49] Hoff KA, Bashir M. Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust. Hum Factors 2015; 57: 407–434.

[50] Koltai K, Ho N, Masequesmay G, et al. Influence of Cultural, Organizational, and Automation Capability on Human Automation Trust: A Case Study of Auto-GCAS Experimental Test Pilots. 2014.

[51] Lee JD, Moray N. Trust, self-confidence, and operators’ adaptation to automation. International Journal of Human-Computer Studies 1994; 40: 153–184.

[52] Fleury S. Le rôle de l'utilisateur dans les systèmes de traitements automatiques. 2014; 204.

[53] Lee J, Moray N. Trust, control strategies and allocation of function in human-machine systems. Ergonomics 1992; 35: 1243–1270.

[54] Madhavan P, Wiegmann DA, Lacson FC. Automation Failures on Tasks Easily Performed by Operators Undermine Trust in Automated Aids. Hum Factors 2006; 48: 241–256.

[55] Bahner JE, Hüper A-D, Manzey D. Misuse of automated decision aids: Complacency, automation bias and the impact of training experience. International Journal of Human-Computer Studies 2008; 66: 688–699.

[56] Lee JD, See KA. Trust in Automation: Designing for Appropriate Reliance. Human Factors: The Journal of the Human Factors and Ergonomics Society 2004; 46: 50–80.

[57] Misulis KE, Frisse ME. Decision-Making. In: Frisse ME, Misulis KE (eds) Essentials of Clinical Informatics. Oxford University Press, pp. 104–125.

[58] Schacter DL, Gilbert DT, Wegner DM. Psychology. 2nd ed. New York, NY: Worth Publishers, 2011.

[59] Inman JJ, Dyer JS, Jia J. A Generalized Utility Model of Disappointment and Regret Effects on Post-Choice Valuation. Marketing Science 1997; 16: 97–111.

[60] Hiel A van, Mervielde I. The Need for Closure and the Spontaneous Use of Complex and Simple Cognitive Structures. The Journal of Social Psychology 2003; 143: 559–568.

[61] Schaefer KE, Perelman BS, Brewer RW, et al. Quantifying Human Decision-Making: Implications for Bidirectional Communication in Human-Robot Teams. In: Chen JYC, Fragomeni G (eds) Virtual, Augmented and Mixed Reality: Interaction, Navigation, Visualization, Embodiment, and Simulation. Cham: Springer International Publishing, pp. 361–379.

[62] Barnes MJ, Lakhmani S, Holder E, et al. Issues in Human-Agent Communication. 2019; 29.

[63] Metge A, Maille N. IA et assistance à la décision humaine : influence des informations transmises sur l'évaluation d'un nouveau plan de vol. 2020.
[64] Lachter J, Battiste V, Matessa M, et al. Toward single pilot operations: the impact of the loss of non-verbal communication on the flight deck. In: Proceedings of the International Conference on Human-Computer Interaction in Aerospace - HCI-Aero '14. Santa Clara, California: ACM Press, pp. 1-8.

[65] Jones DR. Three input concepts for flight crew interaction with information presented on a large-screen electronic cockpit display, https://ntrs.nasa.gov/search.jsp?R=19900009078 (1 April 1990, accessed 9 April 2020).

[66] Mao Q, Dong M, Huang Z, et al. Learning Salient Features for Speech Emotion Recognition Using Convolutional Neural Networks. IEEE Trans Multimedia 2014; 16: 2203-2213.

[67] Rouast PV, Adam M, Chiong R. Deep Learning for Human Affect Recognition: Insights and New Developments. IEEE Trans Affective Comput 2019; 1-1.

[68] Zhihong Zeng, Pantic M, Roisman GI, et al. A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions. IEEE Trans Pattern Anal Mach Intell 2009; 31: 39-58.

[69] Stanton NA, Harvey C, Plant KL, et al. To twist, roll, stroke or poke? A study of input devices for menu navigation in the cockpit. Ergonomics 2013; 56: 590-611.

[70] Cockburn A, Masson D, Gutwin C, et al. Design and evaluation of braced touch for touchscreen input stabilisation. International Journal of Human-Computer Studies 2019; 122: 21-37.

[71] Hourlier S, Guérard S, Servantie X. Avionics Touch Screen in Turbulence: Simulator Design and Selected Human-Machine Interface Metrics. Improving Aviation Performance through Applying Engineering Psychology: Advances in Aviation Psychology 2019; 3: 53.

[72] Large DR, Crundall E, Burnett G, et al. Predicting the visual demand of finger-touch pointing tasks in a driving context. In: Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications - AutomotiveUI '15. Nottingham, United Kingdom: ACM Press, pp. 221-224.

[73] Pauchet S, Letondal C, Vinot J-L, et al. GazeForm: Dynamic Gaze-adaptive Touch Surface for Eyes-free Interaction in Airliner Cockpits. In: Proceedings of the 2018 on Designing Interactive Systems Conference 2018 - DIS '18. Hong Kong, China: ACM Press, pp. 1193-1205.

[74] Pongsakornsathien N, Lim Y, Gardi A, et al. Sensor Networks for Aerospace Human-Machine Systems. Sensors 2019; 19: 3465.

[75] Causse M, Lancelot F, Maillant J, et al. Encoding decisions and expertise in the operator's eyes: Using eye-tracking as input for system adaptation. International Journal of Human-Computer Studies 2019; 125: 55-65.

[76] Landy MS, Maloney LT, Pavel M. Exploratory Vision: The Active Eye. Springer Science & Business Media, 2012.

[77] Rajesh J, Biswas P. Eye-gaze tracker as input modality for military aviation environment. In: 2017 International Conference on Intelligent Computing, Instrumentation and Control Technologies (ICICICT). 2017, pp. 558-564.

[78] Palinko O, Rea F, Sandini G, et al. Robot reading human gaze: Why eye tracking is better than head tracking for human-robot collaboration. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Daejeon, South Korea: IEEE, pp. 5048-5054.

[79] Murthy P, Kovalenko O, Elhayek A, et al. 3D Human Pose Tracking inside Car using Single RGB Spherical Camera. 2017; 4.

[80] Liu H, Wang L. Gesture recognition for human-robot collaboration: A review. International Journal of Industrial Ergonomics 2018; 68: 355-367.

[81] Becquet V. Conception d'une représentation graphique des gestes numériques pour le cockpit tactile fondée sur les dimensions participant à la conscience mutuelle entre les pilotes. 2019; 240.

[82] Roider F, Gross T. I See Your Point: Integrating Gaze to Enhance Pointing Gesture Accuracy While Driving. In: Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. Toronto ON Canada: ACM, pp. 351-358.

[83] Du S, Tao Y, Martinez AM. Compound facial expressions of emotion. PNAS 2014; 111: E1454-E1462.

[84] Ko BC. A Brief Review of Facial Emotion Recognition Based on Visual Information. Sensors 2018; 18: 401.
[64] Lachter J, Battiste V, Matessa M, et al. Toward single pilot operations: the impact of the loss of non-verbal communication on the flight deck. In: Proceedings of the International Conference on Human-Computer Interaction in Aerospace - HCI-Aero '14. Santa Clara, California: ACM Press, pp. 1-8.

[65] Jones DR. Three input concepts for flight crew interaction with information presented on a large-screen electronic cockpit display, https://ntrs.nasa.gov/search.jsp?R=19900009078 (1 April 1990, accessed 9 April 2020).

[66] Mao Q, Dong M, Huang Z, et al. Learning Salient Features for Speech Emotion Recognition Using Convolutional Neural Networks. IEEE Trans Multimedia 2014; 16: 2203-2213.

[67] Rouast PV, Adam M, Chiong R. Deep Learning for Human Affect Recognition: Insights and New Developments. IEEE Trans Affective Comput 2019; 1-1.

[68] Zhihong Zeng, Pantic M, Roisman GI, et al. A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions. IEEE Trans Pattern Anal Mach Intell 2009; 31: 39-58.

[69] Stanton NA, Harvey C, Plant KL, et al. To twist, roll, stroke or poke? A study of input devices for menu navigation in the cockpit. Ergonomics 2013; 56: 590-611.

[70] Cockburn A, Masson D, Gutwin C, et al. Design and evaluation of braced touch for touchscreen input stabilisation. International Journal of Human-Computer Studies 2019; 122: 21-37.

[71] Hourlier S, Guérard S, Servantie X. Avionics Touch Screen in Turbulence: Simulator Design and Selected Human-Machine Interface Metrics. Improving Aviation Performance through Applying Engineering Psychology: Advances in Aviation Psychology 2019; 3: 53.

[72] Large DR, Crundall E, Burnett G, et al. Predicting the visual demand of finger-touch pointing tasks in a driving context. In: Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications - AutomotiveUI '15. Nottingham, United Kingdom: ACM Press, pp. 221-224.

[73] Pauchet S, Letondal C, Vinot J-L, et al. GazeForm: Dynamic Gaze-adaptive Touch Surface for Eyes-free Interaction in Airliner Cockpits. In: Proceedings of the 2018 on Designing Interactive Systems Conference 2018 - DIS '18. Hong Kong, China: ACM Press, pp. 1193-1205.

[74] Pongsakornsathien N, Lim Y, Gardi A, et al. Sensor Networks for Aerospace Human-Machine Systems. Sensors 2019; 19: 3465.

[75] Causse M, Lancelot F, Maillant J, et al. Encoding decisions and expertise in the operator's eyes: Using eye-tracking as input for system adaptation. International Journal of Human-Computer Studies 2019; 125: 55-65.

[76] Landy MS, Maloney LT, Pavel M. Exploratory Vision: The Active Eye. Springer Science & Business Media, 2012.

[77] Rajesh J, Biswas P. Eye-gaze tracker as input modality for military aviation environment. In: 2017 International Conference on Intelligent Computing, Instrumentation and Control Technologies (ICICICT). 2017, pp. 558-564.

[78] Palinko O, Rea F, Sandini G, et al. Robot reading human gaze: Why eye tracking is better than head tracking for human-robot collaboration. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Daejeon, South Korea: IEEE, pp. 5048-5054.

[79] Murthy P, Kovalenko O, Elhayek A, et al. 3D Human Pose Tracking inside Car using Single RGB Spherical Camera. 2017; 4.

[80] Liu H, Wang L. Gesture recognition for human-robot collaboration: A review. International Journal of Industrial Ergonomics 2018; 68: 355-367.

[81] Becquet V. Conception d'une représentation graphique des gestes numériques pour le cockpit tactile fondée sur les dimensions participant à la conscience mutuelle entre les pilotes. 2019; 240.

[82] Roider F, Gross T. I See Your Point: Integrating Gaze to Enhance Pointing Gesture Accuracy While Driving. In: Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. Toronto ON Canada: ACM, pp. 351-358.

[83] Du S, Tao Y, Martinez AM. Compound facial expressions of emotion. PNAS 2014; 111: E1454-E1462.

[84] Ko BC. A Brief Review of Facial Emotion Recognition Based on Visual Information. Sensors 2018; 18: 401.