# Human-machine cooperation

Marie-Pierre Pacaux

► To cite this version:

Marie-Pierre Pacaux. Human-machine cooperation: Adaptability of shared functions between Humans and Machines, Design and evaluation aspects. Automatic. Université Polytechnique Hauts-de-France, 2020. tel-03975267

HAL Id: tel-03975267

https://uphf.hal.science/tel-03975267

Submitted on 6 Feb 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

Université
Polytechnique
HAUTS-DE-FRANCE

HABILITATION A DIRIGER DES RECHERCHES

Spécialité : Génie informatique, automatique et traitement du signal

Section CNU : 61ème Section

Présentée par :

Marie-Pierre PACAUX-LEMOINE

Docteur en Automatique des Systèmes Industriels et Humains

HUMAN-MACHINE COOPERATION

Adaptability of shared functions between Humans and Machines

Design and evaluation aspects

Soutenue le 25 Août 2020 devant la commission d'examen constituée de :

<table><tr><td>Max MULDER</td><td>Pr., TU Delft, Netherlands</td><td>Rapporteur</td></tr><tr><td>Gilles COPPIN</td><td>Pr., IMT Atlantique, Brest, France</td><td>Rapporteur</td></tr><tr><td>Bernard GRABOT</td><td>Pr., ENIT, Tarbes, France</td><td>Rapporteur</td></tr><tr><td>Tetsuo SAWARAGI</td><td>Pr., University of Kyoto, Japan</td><td>Examinateur</td></tr><tr><td>Marie BABEL</td><td>MCF HDR, INSA, Rennes, France</td><td>Examinateur</td></tr><tr><td>Jordan NAVARRO</td><td>MCF HDR, Université de Lyon, France</td><td>Examinateur</td></tr><tr><td>Frank FLEMISCH</td><td>Pr., RWTH, Aachen, Germany</td><td>Invité</td></tr><tr><td>Patrick MILLOT</td><td>Pr., UPHF, Valenciennes, France</td><td>Promoteur</td></tr></table>
# Table of contents

GENERAL INTRODUCTION.................................................................................................................. 11

CHAPTER 1. OVERVIEW OF RESEARCH AND TEACHING ACTIVITIES ........................................ 12

1.1 Research activity ................................................................................................................................. 12

1.2 Teaching activity ................................................................................................................................. 14

INTRODUCTION TO RESEARCH ........................................................................................................ 15

CHAPTER 2. HUMAN(S)-MACHINE(S) SYSTEM: STATE OF THE ART........................................ 18

2.1 Human.................................................................................................................................................. 19

2.1.1 Human information processing.................................................................................................. 19

2.1.2 Cognitive dimensions to control dynamic situations ................................................................ 22

2.2 Human and machine organization ................................................................................................. 23

2.2.1 Cooperative structures ............................................................................................................. 23

2.2.2 Cooperative modes.................................................................................................................... 24

2.2.3 Cooperative forms.................................................................................................................... 26

2.2.4 Levels of automation.................................................................................................................. 26

2.2.5 Levels of autonomy.................................................................................................................. 31

2.3 Awareness .................................................................................................................................................. 32

2.3.1 Situation awareness.................................................................................................................. 33

2.3.2 Team situation awareness ........................................................................................................ 34

SYNTHESIS ON THE STATE OF THE ART ................................................................................................. 36

CHAPTER 3. TOWARDS HUMAN-MACHINE COOPERATION ................................................................. 38

3.1 Definition of cooperation .................................................................................................................. 39

3.2 Towards a meta-model of cooperation................................................................................................. 40

3.2.1 The “Know-How” or the abilities to control the process (KH) ................................................ 40

3.2.2 The “Know-How-to-Cooperate” or the agent’s ability to cooperate (KHC) ............................ 42

3.2.3 The Common Work Space............................................................................................................ 44

3.2.4 Towards a generic model of cooperative agents’ interaction .................................................... 47

3.3 Adaptability of levels of automation........................................................................................................ 51

3.4 Multi-level cooperation................................................................................................................................. 56
<table>
  <tbody>
    <tr>
      <td>CHAPTER 4. DESIGN AND EVALUATION METHODS</td>
      <td>62</td>
    </tr>
    <tr>
      <td>4.1 Some methods from the literature</td>
      <td>63</td>
    </tr>
    <tr>
      <td>4.2 Descending phase for design</td>
      <td>64</td>
    </tr>
    <tr>
      <td>4.3 Ascending phase for evaluation</td>
      <td>67</td>
    </tr>
    <tr>
      <td>4.3.1 Experimental protocol</td>
      <td>67</td>
    </tr>
    <tr>
      <td>4.3.1.1 Scenario and experimental design</td>
      <td>67</td>
    </tr>
    <tr>
      <td>4.3.1.2 Simulation of environment</td>
      <td>68</td>
    </tr>
    <tr>
      <td>4.3.2 Data records</td>
      <td>71</td>
    </tr>
    <tr>
      <td>4.3.2.1 Online data records in a simulated environment</td>
      <td>71</td>
    </tr>
    <tr>
      <td>4.3.2.2 Online data records in a real world</td>
      <td>73</td>
    </tr>
    <tr>
      <td>4.3.2.3 Offline data records</td>
      <td>75</td>
    </tr>
    <tr>
      <td>4.3.3 Assistance systems implementation</td>
      <td>78</td>
    </tr>
    <tr>
      <td>4.3.3.1 Online control of scenario</td>
      <td>78</td>
    </tr>
    <tr>
      <td>4.3.3.2 Online control of assistance systems</td>
      <td>79</td>
    </tr>
    <tr>
      <td>4.3.4 Data processing</td>
      <td>81</td>
    </tr>
    <tr>
      <td>4.3.5 Experiments results and their impacts on the Human-Machine Systems</td>
      <td>84</td>
    </tr>
    <tr>
      <td>4.4 Experimental dimensions</td>
      <td>85</td>
    </tr>
    <tr>
      <td>SYNTHESIS ON RESEARCH CONTRIBUTION</td>
      <td>87</td>
    </tr>
    <tr>
      <td>CHAPTER 5. OBJECTIVES FOR THE FUTURE</td>
      <td>89</td>
    </tr>
    <tr>
      <td>5.1 The generic models</td>
      <td>90</td>
    </tr>
    <tr>
      <td>5.1.1 Improvement of the models</td>
      <td>90</td>
    </tr>
    <tr>
      <td>5.1.1.1 More symmetry between Human and machine?</td>
      <td>90</td>
    </tr>
    <tr>
      <td>5.1.1.2 More criteria for function allocation?</td>
      <td>92</td>
    </tr>
    <tr>
      <td>5.1.2 Improvement in the use of the models</td>
      <td>93</td>
    </tr>
    <tr>
      <td>5.2 From generic models to implementation</td>
      <td>95</td>
    </tr>
    <tr>
      <td>5.2.1 Choice of control laws</td>
      <td>95</td>
    </tr>
    <tr>
      <td>5.2.2 Computer sciences approaches</td>
      <td>96</td>
    </tr>
    <tr>
      <td>5.3 From Common Work Space to Human-machine interface</td>
      <td>98</td>
    </tr>
    <tr>
      <td>5.3.1 Human-machine interface</td>
      <td>98</td>
    </tr>
    <tr>
      <td>5.3.2 Human(s)-machine(s) interface(s)</td>
      <td>99</td>
    </tr>
  </tbody>
</table>
<table>
  <tbody>
    <tr>
      <td>CHAPTER 6. GENERAL CONCLUSION</td>
      <td>102</td>
    </tr>
    <tr>
      <td>6.1 Synthesis on the research perspectives</td>
      <td>102</td>
    </tr>
    <tr>
      <td>6.2 Perspectives on teaching aspects</td>
      <td>103</td>
    </tr>
    <tr>
      <td>6.3 Final conclusion</td>
      <td>104</td>
    </tr>
    <tr>
      <td>REFERENCES</td>
      <td>106</td>
    </tr>
    <tr>
      <td>CHAPTER 7. APPENDIX 1: PROJECTS, IMPLEMENTATION TO VARIOUS APPLICATION AREAS</td>
      <td>122</td>
    </tr>
    <tr>
      <td>7.1 Air Traffic Control</td>
      <td>122</td>
    </tr>
    <tr>
      <td>7.2 Car driving</td>
      <td>124</td>
    </tr>
    <tr>
      <td>7.2.1 Car driving - Embedded Driving Assistance Systems</td>
      <td>125</td>
    </tr>
    <tr>
      <td>7.2.2 Car driving - On-road driving assistance system</td>
      <td>127</td>
    </tr>
    <tr>
      <td>7.2.3 Car driving - Driver behavior during pre-crash situation</td>
      <td>129</td>
    </tr>
    <tr>
      <td>7.2.4 Car driving - Infrastructure impact</td>
      <td>131</td>
    </tr>
    <tr>
      <td>7.3 Fighter aircraft</td>
      <td>133</td>
    </tr>
    <tr>
      <td>7.4 Robotics</td>
      <td>135</td>
    </tr>
    <tr>
      <td>7.4.1 Ground robotics in a simulated military context - MIX</td>
      <td>135</td>
    </tr>
    <tr>
      <td>7.4.2 Ground robotics in a military context - TAROT</td>
      <td>137</td>
    </tr>
    <tr>
      <td>7.4.3 Ground robotics in a crisis management context - SUCRé</td>
      <td>140</td>
    </tr>
    <tr>
      <td>7.4.4 Ground Robotics - Handicap</td>
      <td>143</td>
    </tr>
    <tr>
      <td>7.5 Manufacturing systems</td>
      <td>145</td>
    </tr>
    <tr>
      <td>7.6 Manufacturing Systems - Diagnosis Assistance System</td>
      <td>146</td>
    </tr>
    <tr>
      <td>7.6.1 Manufacturing Systems - Supervision and control of Self-organized system</td>
      <td>148</td>
    </tr>
    <tr>
      <td>7.6.2 Manufacturing Systems - Humans-Intelligent systems Cooperation</td>
      <td>150</td>
    </tr>
    <tr>
      <td>7.6.3 Manufacturing systems - Industry 4.0 in Garment factory</td>
      <td>152</td>
    </tr>
    <tr>
      <td>7.7 Railway</td>
      <td>154</td>
    </tr>
    <tr>
      <td>CHAPTER 8. APPENDIX 2</td>
      <td>156</td>
    </tr>
    <tr>
      <td>8.1 International journal</td>
      <td>156</td>
    </tr>
    <tr>
      <td>8.2 National journal</td>
      <td>157</td>
    </tr>
    <tr>
      <td>8.3 Book direction</td>
      <td>157</td>
    </tr>
  </tbody>
</table>
<table><tr><td>8.4</td><td>Proceedings of Conference organized by LAMIH</td><td>157</td></tr><tr><td>8.5</td><td>Book chapter</td><td>158</td></tr><tr><td>8.6</td><td>Proceedings of International Conference with selection committee</td><td>158</td></tr><tr><td>8.7</td><td>Proceedings of National Conference with selection committee</td><td>165</td></tr><tr><td>8.8</td><td>Seminar</td><td>165</td></tr><tr><td>8.9</td><td>Research report</td><td>167</td></tr></table>
List of figures

<table><tr><td>Fig. 1</td><td>Projects presented according to their objective in terms of decisional levels to be assisted and the level of automation implemented defined according to Humans and Machines respective KH and KHC</td><td>13</td></tr><tr><td>Fig. 2</td><td>Teaching activity</td><td>14</td></tr><tr><td>Fig. 3</td><td>AUTOS pyramid (Artifact, User, Task, Organization and Situation) (Guy A Boy, 2006)</td><td>19</td></tr><tr><td>Fig. 4</td><td>Schematic map of the sequences of information processes (Rasmussen & Goodstein, 1987)</td><td>20</td></tr><tr><td>Fig. 5</td><td>KRS-based model (Rasmussen, 1983)</td><td>21</td></tr><tr><td>Fig. 6</td><td>Cooperative structures adapted from (Patrick Millot & Mandiau, 1995)</td><td>24</td></tr><tr><td>Fig. 7</td><td>Cooperative modes (solid lines: commands, dotted lines: information) adapted from (Rieger & Greenstein, 1982) and enriched by (Patrick Millot, 1988)</td><td>25</td></tr><tr><td>Fig. 8</td><td>Assisted explicit mode (solid lines: commands, dotted lines: information) adapted from (M.-P. Lemoine et al., 1996)</td><td>25</td></tr><tr><td>Fig. 9</td><td>Cooperative forms adapted from (Schmidt, 1991)</td><td>26</td></tr><tr><td>Fig. 10</td><td>Supervisory control (T. B. Sheridan, 1984)</td><td>27</td></tr><tr><td>Fig. 11</td><td>Degrees of automation for decision and control (Thomas B. Sheridan, 1992)</td><td>27</td></tr><tr><td>Fig. 12</td><td>Levels of automation for data acquisition and integration (Toshiyuki Inagaki & Sheridan, 2008)</td><td>27</td></tr><tr><td>Fig. 13</td><td>Levels of automation (Endsley & Kaber, 1999)</td><td>28</td></tr><tr><td>Fig. 14</td><td>Types and levels of human interaction with automation (Parasuraman et al., 2000)</td><td>29</td></tr><tr><td>Fig. 15</td><td>Cooperative Guidance & Control with H-Mode (Altendorf et al., 2016)</td><td>29</td></tr><tr><td>Fig. 16</td><td>Autonomous Control Levels (Adapted from the Roadmap proposed by the Department of Defense of the United States to design Unmanned Aerial vehicles (Galster et al., 2007))</td><td>32</td></tr><tr><td>Fig. 17</td><td>SA three-level model adapted from (Endsley, 1995b)</td><td>33</td></tr><tr><td>Fig. 18</td><td>Team Situation Awareness (Salmon et al., 2007)</td><td>35</td></tr><tr><td>Fig. 19</td><td>Synthesis on the state of the art</td><td>36</td></tr><tr><td>Fig. 20</td><td>Model of cooperative agent adapted from (Patrick Millot & Pacaux-Lemoine, 2013)</td><td>40</td></tr><tr><td>Fig. 21</td><td>Updating the Common Frame of Reference, adapted from (M P Pacaux-Lemoine & Loiselet, 2002)</td><td>45</td></tr><tr><td>Fig. 22</td><td>Cooperative activities through agents Know-how (Ag<sub>i</sub> KH), agents Know-how-to-cooperate (Ag<sub>i</sub> KHC), agents situation awareness (Ag<sub>i</sub> SA), Common Frame of Reference (COFOR), team Situation Awareness (Team SA) and Common Work Space</td><td>46</td></tr><tr><td>Fig. 23</td><td>Model of cooperative agents' interaction, adapted from (Marie-Pierre Pacaux-Lemoine & Itoh, 2015) (Marie Pierre Pacaux-Lemoine & Flemisch, 2018)</td><td>48</td></tr><tr><td>Fig. 24</td><td>Human-Machine function allocation regarding KH, adapted from (L. Habib et al., 2017b)</td><td>53</td></tr><tr><td>Fig. 25</td><td>Human-Machine function allocation regarding KHC, adapted from (L. Habib et al., 2017b)</td><td>53</td></tr><tr><td>Fig. 26</td><td>Emulated-haptic feedback experiments: KH of Human and robot (respectively violet and green rectangles) and their KHC (light blue and green rectangles). If the agent has the ability to perform a sub-function, "1" is written in the rectangle</td><td>54</td></tr><tr><td>Fig. 27</td><td>Extension of Rasmussen model to cooperative behavior (L. Habib et al., 2017b)</td><td>55</td></tr></table>
<table>
  <tbody>
    <tr>
      <td>Fig. 28</td>
      <td>Multi-level cooperation, adapted from (Marie-Pierre Pacaux-Lemoine & Itoh, 2015)</td>
      <td>56</td>
    </tr>
    <tr>
      <td>Fig. 29</td>
      <td>Multi-level cooperation in crisis management</td>
      <td>57</td>
    </tr>
    <tr>
      <td>Fig. 30</td>
      <td>Relationship between shared control, shared and cooperative guidance and control and human-machine cooperation (adapted from (F. Flemisch, Abbink, Itoh, Pacaux-Lemoine, & Weßel, 2019))</td>
      <td>59</td>
    </tr>
    <tr>
      <td>Fig. 31</td>
      <td>Layers of cooperation</td>
      <td>59</td>
    </tr>
    <tr>
      <td>Fig. 32</td>
      <td>Method in "U", adapted from (P. Millot, 1995)</td>
      <td>64</td>
    </tr>
    <tr>
      <td>Fig. 33</td>
      <td>Grid to identify current and future human (D: driver) and machine (S: ADAS) tasks according to their KH and KHC: example in automotive domain (KH: IG: Information Gathering, IA: Information Analysis, DS: Decision Making, AI: Action Implementation; KHC: IG: Information Gathering on the other, ID: Interference Detection, IM: Interference management, FA: Function Allocation) (Marie-Pierre Pacaux-Lemoine et al., 2015)</td>
      <td>66</td>
    </tr>
    <tr>
      <td>Fig. 34</td>
      <td>Preparation of mission in the project with DGA and Dassault Aviation (cf. details in §7.3)</td>
      <td>68</td>
    </tr>
    <tr>
      <td>Fig. 35</td>
      <td>MIX (The Mixed Initiative eXperimental testbed)</td>
      <td>69</td>
    </tr>
    <tr>
      <td>Fig. 36</td>
      <td>Workstations of pilot and weapon system officer</td>
      <td>69</td>
    </tr>
    <tr>
      <td>Fig. 37</td>
      <td>Driving simulators of IFSTTAR (a static one on left), UPHF-LAMIH (a mobile one) and Renault (mobile one on the right)</td>
      <td>70</td>
    </tr>
    <tr>
      <td>Fig. 38</td>
      <td>Car driving simulator of the university of Valenciennes (static version) and digital anthropometric model</td>
      <td>70</td>
    </tr>
    <tr>
      <td>Fig. 39</td>
      <td>Confrontation of pilots and weapon system officers to the video recording of their mission</td>
      <td>71</td>
    </tr>
    <tr>
      <td>Fig. 40</td>
      <td>Workload assessment on the radar display of air traffic controllers work position (red scale on the left lower part of the interface)</td>
      <td>72</td>
    </tr>
    <tr>
      <td>Fig. 41</td>
      <td>Participants' situation awareness during experiments conducted about human-robots cooperation (L. Habib, Pacaux-Lemoine, & Millot, 2017a)</td>
      <td>73</td>
    </tr>
    <tr>
      <td>Fig. 42</td>
      <td>Data records during experiments conducted on real roads</td>
      <td>74</td>
    </tr>
    <tr>
      <td>Fig. 43</td>
      <td>Experimental environment to study human-robot cooperation in reconnaissance mission</td>
      <td>75</td>
    </tr>
    <tr>
      <td>Fig. 44</td>
      <td>Descriptors of NASA TLX</td>
      <td>76</td>
    </tr>
    <tr>
      <td>Fig. 45</td>
      <td>Selection of descriptors according to importance perceived by participants</td>
      <td>76</td>
    </tr>
    <tr>
      <td>Fig. 46</td>
      <td>Grid based on the cooperative agent model, support for design and evaluation of human-machine cooperation (IG: Information Gathering, IA: Information Analysis, DS: Decision Selection, AI: Action Implementation)</td>
      <td>77</td>
    </tr>
    <tr>
      <td>Fig. 47</td>
      <td>Control room of the car driving simulator of UPHF</td>
      <td>79</td>
    </tr>
    <tr>
      <td>Fig. 48</td>
      <td>Wizard of Oz platform to conduct experiments with fighter aircraft crews</td>
      <td>79</td>
    </tr>
    <tr>
      <td>Fig. 49</td>
      <td>Workstations to control experiments and play Wizard of Oz roles</td>
      <td>80</td>
    </tr>
    <tr>
      <td>Fig. 50</td>
      <td>Wizard of Oz platform for experiments in robotics</td>
      <td>80</td>
    </tr>
    <tr>
      <td>Fig. 51</td>
      <td>Example of script for experimenters who play Wizard of Oz role for the tactical assistance system (experimenter 2 on Fig. 50)</td>
      <td>81</td>
    </tr>
    <tr>
      <td>Fig. 52</td>
      <td>Example of use of MAC SHAPA software</td>
      <td>82</td>
    </tr>
    <tr>
      <td>Fig. 53</td>
      <td>Software designed to code drivers and assistance systems activities</td>
      <td>83</td>
    </tr>
  </tbody>
</table>
<table>
  <tbody>
    <tr>
      <td>Fig. 54</td>
      <td>RTMaps for SARI: synchronization of video records with vehicles data along three geographical points (position in the lane, speed, distance from other vehicles, size of vehicle...).</td>
      <td>83</td>
    </tr>
    <tr>
      <td>Fig. 55</td>
      <td>Iterative procedure with spiral refinement (Marie-Pierre Pacaux-Lemoine &amp; Flemisch, 2015).</td>
      <td>85</td>
    </tr>
    <tr>
      <td>Fig. 56</td>
      <td>Three dimensions for experimental design and evaluation in the automotive domain (adapted from (Marie-Pierre Pacaux-Lemoine &amp; Flemisch, 2015))</td>
      <td>85</td>
    </tr>
    <tr>
      <td>Fig. 57</td>
      <td>Research contributions.</td>
      <td>87</td>
    </tr>
    <tr>
      <td>Fig. 58</td>
      <td>Where is the equilibrium point for efficient cooperative activity?</td>
      <td>91</td>
    </tr>
    <tr>
      <td>Fig. 59</td>
      <td>Use the complementarity of CWA and CMH to propose a stronger method to design HMS, adapted from (Marie-Pierre Pacaux-Lemoine &amp; Trentesaux, 2019).</td>
      <td>94</td>
    </tr>
    <tr>
      <td>Fig. 60</td>
      <td>Know-How (deep blue) and Know-How-to-Cooperate (light bleu) of robot translate to control laws: example with Model Predictive Control.</td>
      <td>96</td>
    </tr>
    <tr>
      <td>Fig. 61</td>
      <td>Example of the use of a different approach, e.g. AiKiDo approach, to analyze and manage Human-AI interferences</td>
      <td>97</td>
    </tr>
    <tr>
      <td>Fig. 62</td>
      <td>Emulated Haptic Feedback interface.</td>
      <td>98</td>
    </tr>
    <tr>
      <td>Fig. 63</td>
      <td>Tangible tabletop – collective decision making (Merrad, Habib, Héloir, Kolski, &amp; Krüger, 2019; Merrad, Héloir, &amp; Kolski, 2017)</td>
      <td>100</td>
    </tr>
    <tr>
      <td>Fig. 64</td>
      <td>Synthesis on research perspectives</td>
      <td>102</td>
    </tr>
    <tr>
      <td>Fig. 65</td>
      <td>Perspectives on teaching aspects</td>
      <td>103</td>
    </tr>
    <tr>
      <td>Fig. 66</td>
      <td>Air traffic control platform</td>
      <td>123</td>
    </tr>
    <tr>
      <td>Fig. 67</td>
      <td>Driving environment and driving assistance systems.</td>
      <td>124</td>
    </tr>
    <tr>
      <td>Fig. 68</td>
      <td>Control room – Risky driving situations – Driving simulator of Université Hauts-de-France</td>
      <td>126</td>
    </tr>
    <tr>
      <td>Fig. 69</td>
      <td>Technical solutions implemented and evaluated in PREDIT-SARI project</td>
      <td>128</td>
    </tr>
    <tr>
      <td>Fig. 70</td>
      <td>Evaluation of the impact of perceptual treatment in the field and on two driving simulators: fixed-based one on the left and motion-based one on the right</td>
      <td>128</td>
    </tr>
    <tr>
      <td>Fig. 71</td>
      <td>Control of experiment and data recording; Scenario; digital car and OOP.</td>
      <td>130</td>
    </tr>
    <tr>
      <td>Fig. 72</td>
      <td>Scenario of the unavoidable crash situation; Sequence of the falling block on the windscreen</td>
      <td>130</td>
    </tr>
    <tr>
      <td>Fig. 73</td>
      <td>Tunnel virtual mock-up; Tunnel characteristics</td>
      <td>131</td>
    </tr>
    <tr>
      <td>Fig. 74</td>
      <td>Driving simulator in the Technocentre of Renault</td>
      <td>132</td>
    </tr>
    <tr>
      <td>Fig. 75</td>
      <td>Fighter aircraft experimental platform and Common Work Space (more details on Fig. 48).</td>
      <td>133</td>
    </tr>
    <tr>
      <td>Fig. 76</td>
      <td>MIX (The Mixed Initiative eXperimental testbed).</td>
      <td>136</td>
    </tr>
    <tr>
      <td>Fig. 77</td>
      <td>Experimental platform and R-Trooper robot.</td>
      <td>137</td>
    </tr>
    <tr>
      <td>Fig. 78</td>
      <td>Experiments to evaluate Humans-Robots cooperation in crisis management.</td>
      <td>141</td>
    </tr>
    <tr>
      <td>Fig. 79</td>
      <td>Human-BCI-robot cooperation - Shared control with “emulated haptic feedback”.</td>
      <td>144</td>
    </tr>
    <tr>
      <td>Fig. 80</td>
      <td>Experimental platform: ITS-PLC to simulate the operative part of the process (on the left) and the assistance system (on the right) providing explanations about process stop and command errors.</td>
      <td>146</td>
    </tr>
    <tr>
      <td>Fig. 81</td>
      <td>Flexible manufacturing system at Université Polytechnique Hauts-de-France (AIP-PRIMECA FMS) – Emulated system – CWS with the tactical assistance.</td>
      <td>148</td>
    </tr>
    <tr>
      <td>Fig. 82</td>
      <td>Human-Robots-ASO cooperation, Université Polytechnique Hauts-de-France – Humans-Cobot cooperation, Université Bretagne-Sud, Lorient – Human-Diagnosis support system cooperation, Université de Reims.</td>
      <td>150</td>
    </tr>
  </tbody>
</table>
<table><tr><td>Fig. 83</td><td>Industry 4.0 in textile-clothing chain</td><td>152</td></tr></table>

<table><tr><td>Fig. 84</td><td>Remote control of train and human-ADASs cooperation</td><td>154</td></tr></table>

List of tables

<table><tr><td>Table 1</td><td>Comparisons of automation scales (Pacaux et al., 2011)</td><td>30</td></tr></table>

<table><tr><td>Table 2</td><td>Levels of Cooperation according to KH</td><td>50</td></tr></table>

<table><tr><td>Table 3</td><td>Levels of Cooperation according to KHC</td><td>50</td></tr></table>

<table><tr><td>Table 4</td><td>Matrix of possible levels of Cooperation</td><td>51</td></tr></table>

<table><tr><td>Table 5</td><td>Human and robot KH and KHC for LoA identification</td><td>138</td></tr></table>
# General introduction

The objective to obtain the "Habilitation à diriger des recherches" is not a usual way for a French research engineer to pursue a carrier. Nevertheless, since my PhD in 1998, the LAMIH gave me the opportunity to conduct research activities in parallel to the engineering activity. Such an organization allowed being close to system design, experimental environment and system users, as well as the research environment and the ability to debate with researchers in conferences and research groups. Therefore, the present report deals with my Curriculum Vitae, the state of the art in Human-Machine System, two main scientific contributions and perspectives. The first scientific contribution tackles conceptual aspects with the proposal of Human-Machine Cooperation models. The second one details methodological aspects with the design and evaluation of Human-Machine Systems. My activities were mainly focusing on research; however, I had also the opportunity to have a teaching activity. I was temporary assistant professor and then I have been involved in training institutes to oversee students during internships and projects, as well as by teaching courses in particular about research projects.

Therefore, thanks to many discussions with colleagues from foreign and French labs, LAMIH and students, I have always been wanted to go deeper in the scientific approach to design and evaluate Human-Machine System, but at the same time, to be more generic, in the models of Human-Machine Cooperation we proposed. The present report is also the opportunity to make a synthesis of my work and to take advantage of the presentation of the wide evolution in the Human-Machine System design to propose several tracks to continue to improve this research direction, but new complementary directions too.

The document is organized around six chapters:

* Chapter 1 provides a quick introduction of my research and teaching activities.

* Chapter 2 summarizes the state of the art in Human-Machine System and gives the key definitions and some lacks identified in the past and current research studies.

* Chapter 3 provides my contributions regarding improvements in terms of models to define cooperation between human and automation, adaptability of cooperation according to several criteria and cooperation between several humans and several automated systems.

* Chapter 4 proposes to use cooperation models proposed in the Chapter 3 to develop a method for the design of Human-Machine System, and several tools to conduct evaluations of the designed system according to experimental context.

* Chapter 5 deals with the next steps of the research and teaching activities. Several ways to improve and evaluate the models of cooperation have already been identified, especially within the framework of cooperation with several other labs, as well as an interesting connection with teaching activities and the involvement of students in our research work.
# Chapter 1. Overview of research and teaching activities

## 1.1 Research activity

My research activity consists in participating and supervising research projects, supervising PhD students and engineers involved in these projects. In parallel to projects, I contribute to research groups and lead parts of these groups. Those activities have been mainly based on the framework of national and industrial projects conducted since my PhD. With the benefit of hindsight, research projects can be presented according to two dimensions based on the concepts extracted from these activities. The first dimension is the decision level which deals with the type of activity implied in the Human-Machine System, depending on the level of abstraction and the temporal constraints of the tasks, but also on the hierarchical organisation (in particular the authority management and sharing) of the Human(s)-Machine(s) Systems. The second dimension relates the levels of automation of the systems that have been designed during these projects. This dimension is split into two parts, dealing with the Know-How (KH) and the Know-How-to-Cooperate (KHC) implemented in the Human-Machine System.

Therefore, Fig. 1 presents all projects I took part or I am still taking part according to these dimensions. The figure highlights the focus of the projects and the kind of concept the project studied, dealing more with the cooperation between levels of activity, or focusing on communication tools with the Know-How-to-Cooperate (KHC), or defining automated functions and their allocation with the Know-How (KH). While the older projects only responded to few parts on these dimensions, current projects involve more and more aspects from a low control level close to the operative part of the Human-Machine System, to higher decisional levels discussing Human(s)-Machine(s) organisations for a longer-term vision of the activity.
<figure><img src="image_3.png" /><figcaption>Fig. 1 Projects presented according to their objective in terms of decisional levels to be assisted and the level of automation implemented defined according to Humans and Machines respective KH and KHC</figcaption></figure>

Several projects were or are conducted in several domains of application. Such an ability allows the evaluation of the models and the systems we designed with different contexts, different types of human operator and different constraints. The project names are listed below, followed by the particular topic I studied. Further details are provided all along the presentation of the research activity, as well as at the end of the manuscript in the section Chapter 7, in the appendix.

* 2019-2022: FEDER project, Smart Fashion 4.0: Design of smart garment factory in a 4.0 context: from fashion trend to design and manufacturing (cf. §7.6.3).

* 2017-2021: TC-RAIL project: Train remote control (Railenium, SNCF, Thales C&S, Actia Telecom, CNES) (cf. §7.6.3).

* 2017-2021: HUMANISM project: Human-Machine Cooperation for Intelligent Manufacturing System, (cf. §7.6.2).

* 2014-2017: SUCRé project: Human-Human and Human-Robots Cooperation in hostile environment (head of project, cf. §7.4.3).

* 2013-2016: COCOVEA project: Driver - automated vehicle cooperation (PSA, Valeo, Continental), (cf. §7.2.1).

* 2007-2012: ADEXEC project: Approche de Détection et d'Explication d'Erreur de Commande par filtrage robuste, GIS 3SGS, (cf. §7.6).

* 2005-2010: SARI Project: Automated Supervision of the Road to Inform drivers and managers (IFSTTAR), (cf. §7.2.2).

* 2006-2009: TAROT Project: Definition of Autonomy modes for human-robot cooperation, DGA (French Defence), Thalès, (cf. §7.4.2).

* 2004-2008: ST2 Project: Design and evaluation of safety systems during risky and pre-crash situation, (cf. §7.2.3).
* 2002-2005: ARCOS Project: Research Action for Safe Driving Contribution to adaptive cruise control design and evaluation, (cf. §7.2.1).

* 2002-2003: Project with Cofiroute and Renault: Studies dealing with the design of an unusual tunnel for automobile, (cf. §7.2.4).

* 1998-2001: Project with Dassault Aviation: Studies dealing with Human-Human cooperation in the cockpit of two-seater fighter aircraft, (cf. §7.3).

* 1992-1998: SPECTRA Project, dealing with the design and evaluation of assistance system to detect and manage aircraft conflicts, CENA (Centre d'Etudes de la Navigation Aérienne), DGAC (French Civil Aviation), (cf. §7.1).

* 1992-1994: SWIFT Project: Specification for working positions in future air traffic control, Eurocontrol, (cf. §7.1).

## 1.2 Teaching activity

As teaching is not the main activity of Research Engineer, in the French University organization, I am allowed to give courses and lectures, as a second job.

<figure><img src="image_4.png" /><figcaption>Fig. 2 Teaching activity</figcaption></figure>

However, I was pleased to teach our way to design and evaluate Human-Machine Systems. The two axes, design and evaluation were addressed separately or together, regarding the type of course (lecture, exercise, practical work and project) and the number of hours allocated to courses. Therefore, students are trained to be designer (purple part of Fig. 2), they learn several types of model proposed by industry and literature, how designing machines by programing software and hardware systems, as well as Human-Machine Interface.

In parallel, they are trained as experimenter to be able to evaluate the HMS they designed, not only the technical validity, but also the usefulness of their systems through data analysis and scenario (green part of Fig. 2). They learn to design a scenario, to identify and implement recordings, to analyze data and present results in order to evaluate the system they designed. However, they are also participants to experiments in order to evaluate systems designed by other students.
# Introduction to research

Over the last centuries, we have experienced scientific, technological and societal progress that has enabled the design of machines with increasing abilities and autonomy. Such an evolution must be attached to real advances for humans as well as for mankind. This requires a conscious distribution of roles and control between humans and machines. Machines can be more than just either fully automated or manually controlled; they can work together with humans at different levels of assistance and automation in a hopefully beneficial cooperation. One way for studying and reaching such a beneficial cooperation has been explored within the framework of Human-Machine Cooperation. It is one of the main topics studied by the Automation department of LAMIH for over thirty years (P. Millot & Willaeys, 1985; Patrick Millot, 1988). Many works managed with various investigations and exchanges with research partners from different disciplines, including automation, computer sciences, cognitive and social psychology, have allowed exploring several ideas to build efficient and desired cooperation between human and machine.

The main global objective of the research is to answer the question: how might it be possible to provide humans with machine in a way that a kind of “complicity” could emerge from their interaction? Human-machine cooperation principles have been the main means used to study and find the best combination of Human and machine capabilities to meet human needs. My background in automation initially led to take into consideration only human and machine competences to complete the mandated tasks. Nevertheless, the time spent with professional human operators demonstrates to what extent other types of competence and skills have to be involved in the design of efficient and well-accepted cooperation. Therefore, we identified and modeled other aspects of competence. These aspects focus on cooperation and we defined the *Know-How-to-Cooperate*. It is the ability of human and machine to work with each other in order to reach a common goal and to facilitate the task of the other. In the present report, we use the definition of the *Know-How-to-Cooperate* to study human-machine cooperation, an ability usually applied to human-human cooperation, when people are working more or less efficiently and amicably with each other. Thinking of these aspects brings many examples to mind.

Literature usually highlights asymmetry between the human and the machine. Nevertheless, with the increase of machine ability, more and more examples show that when machine suits human needs well, symmetry may increase. This is the reason of our willingness to build up similar models for the human and the machine. We defined the model of a cooperative agent, an agent being a human or a machine. We define an agent according to two abilities. The first ability is the *Know-How*, which deals with the ability to control a part of the process without any help from other agents. It is the capability and the capacity of the agent to gather and analyze information in order to make a decision and to act on the process. The second ability is the *Know-How-to-Cooperate*, which is the ability to cooperate with another agent to reach a common goal. The Know-How-to-Cooperate allows agents to build a representation of other agents, in order to infer intention, to detect and manage interferences that could appear in their mutual understanding of process. The Know-How-to-
Cooperate also leads to identify criteria for a dynamic task allocation between agents. Know-How and Know-How-to-Cooperate are very useful functions, well-assessed by automation and computer science community as well as by human science community. The discussions with human science researchers may sometimes underline that such a model still appears to be too restrictive to represent all cognitive capacities of human. Nevertheless, they also underline that this model often constitutes a cooperative support between Human Science community and engineers, model then applied as a "common language" between human and machine. This report aims to highlight the usefulness of the model to design new systems, as well as to evaluate new or existing systems thanks to a detailed coding of individual and cooperative activities. Several works are currently in progress in LAMIH to improve Know-How and Know-How-to-Cooperate functions. So far, the model has been mainly tested and adapted in dynamic environments like driving tasks (automobile, aeronautic, railway). The model has been improved in robotics (crisis management, manufacturing systems) with a focus on the identification of levels of automation (i.e. role and authority of humans and machines), as well as on the levels of activity linked to organization (low to high decisional levels), and levels of cooperation dealing with task allocation criteria and tools to support cooperation.

Indeed, human and technical agents must be aware of process state, but they also need to be aware of other agent's involvement in the process control. The notion of *Common Work Space* has been defined to reach such needs by providing support for agents' communication, situation awareness and team situation awareness. It was built from workspaces shared between several agents and it gathers different kinds of information from process. However, more than simple shared workspace, we enriched the Common Work Space with information about other agents in order to make explicit their involvement in the process control, and to prevent agents from making too many wrong inferences. The Know-How-to-Cooperate of each agent helps gather agents' information and provide them to the Common Work Space. The enrichment firstly depends on the time agents have to provide information. Secondly, it depends on the ability to extract information from agents in an automatic way. Information about machines' state can be directly extracted, but it is far more difficult to identify human "state". Human monitoring is another topic that must be studied in accordance with human needs if we want to improve human-machine interaction, cooperation and perhaps "complicity".

Therefore, the first chapter having dealt with the Curriculum Vitae, the second chapter is dedicated first to the state of the art in Human-Machine Systems (HMS), the identification of the missing points between our objectives concerning Human-Machine cooperation and the current results in research in different disciplines and domains of application. However, such a topic is very wide and we will focus on the main and generic models used for dynamic situations, considering numerous and different types of shared function between human and machine, i.e. decisional tasks as well as action ones. The state of the art also provides the analysis of existing models from Automation, as well as Human Science. This analysis leads to the need to define cooperation between a human and a machine; definitions are presented in the chapter 3.

*The third chapter* details the definition of cooperation on which we based our research, and compares cooperation to other kinds of interactions, such as the levels of automation approach. The study of this definition from an engineering point of view leads to identify the three main key elements to combine in order to propose different levels of cooperation, i.e. the *Know-How*, the *Know-How-to-Cooperate* and the *Common Work Space*. The off-line and on-line update of levels of cooperation are also discussed. Nevertheless, the
definition of cooperation between two agents raises the question of cooperation between several humans and
several machines, a situation often experienced in the everyday life in both the public and private sphere. The
*multi-level aspect of the cooperation* came from this statement. Indeed, humans are, most of the time,
organized into several hierarchical and activity levels. The end of the third chapter aims to study and support
interactions and cooperation inside and between levels of activity.

*The fourth chapter* sums up main aspects relevant to cooperation previously presented by the means of a
method guiding the design and the evaluation of HMS based on the model of cooperation. The models of
cooperation previously presented are implemented in order to propose a formal and generic method to identify
suitable levels of cooperation, as well as several kinds of experimental tools to evaluate designed HMS based
on these levels.
**Chapter 2. Human(s)-Machine(s) System: State of the art**

Researchers in System engineering and automation have worked for a long time on the design of systems able to replace or compensate human reliability problems. Researchers and engineers are used to adopting techno-centered design approaches favoring the definition and allocation of tasks to automated systems, only considering human operators at the end of the design process once the control system defined (Trentesaux & Millot, 2016). Such designers assumed that this design approach would identify an *a priori* task allocation, each task being allocated to the best agent (Fitts, 1951) and the HMS's performance was therefore presumed to be optimal. This conception has been questioned in a convincing way by several authors (Bainbridge, 1983; Reason, 1988; Thomas B. Sheridan, 1992; D. D. Woods, 1986). Such a Machine-centered strategy, also called residual strategy due to engineers' behavior to automate whatever they can, is still used by researchers and companies, only allocating task that cannot be automated, i.e. the residues, to human operators (Chauvin & Hoc, 2014). New conceptual approaches took shape considering more and more cooperative aspects of human-machine interaction. The Human-Machine system, Human and machine both involved, was the focus of system design (Erik Hollnagel & Woods, 1983), especially with the proposition of "joint cognitive systems" (E. Hollnagel & Woods, 2005). Therefore, several authors proposed other approaches dealing with Human-centered design approaches, such as Human-centered automation (Billings, 1996).

Nevertheless, despite the integration of an increasing number of human and machine characteristics in the design of HMS, an important aspect still lacked, the relevance and the definition of cooperation, until the notion of Human-Machine Cooperation was conceived (Jean-Michel Hoc, 1996; Jean-Michel Hoc, Cacciabue, & Hollnagel, 1995). Indeed, the machine was designed with very limited cooperative features, and the human operator remained responsible for the overall task allocated to the complete HMS. The objective of the Human-Machine Cooperation (HMC) approach is to take into account not only tasks but also functions that can be allocated to human and/or machine according to their individual needs to respond to their common objectives. Coordination tasks needed to be added, to human and machine, in a way that human overload and misunderstanding were avoided, and so in order for human to avoid interacting with "clumsy" automation (D. Woods, 1997).

Therefore, the HMC approach implies the consideration of new operations. In addition to those strictly necessary for task execution by a single agent to control a system, new tasks have to manage role allocation, conflict resolution between goals, and so forth. Organization becomes an important aspect to involve in the design approach. Boy proposed the AUTOS pyramid (Artifact, User, Task, Organization and Situation) to support the life cycle of an HMS from design to decommissioning (Guy A Boy, 2006). The pyramid he designed (cf. Fig. 3) highlights the main points to be taken into consideration when organizing interactions between users and artifacts according to the task to perform for a specific situation. On top of identifying the main actors, the pyramid also underlines the necessary adaptation of the HMS, especially according to the type of situation and the type of task.
<figure><img src="image_5.png" /><figcaption>Fig. 3 AUTOS pyramid (Artifact, User, Task, Organization and Situation) (Guy A Boy, 2006)</figcaption></figure>

Parts of the interactions presented on the AUTOS pyramid are described through the user point of view in the next section, while the different types of organization that can be proposed between several humans and artifacts, called assistance systems are described in the section that follows. The last section pays particular attention to the notion of awareness which is crucial not only for the situation but for all the summits of the pyramid.

## 2.1 Human

Human-Machine System designers are, most of the time, automation or computer science engineers, but since human is part of the system, designers have to be aware of human factors, and even more so, be able to understand and use models provided by cognitive psychology studies. While the activities of a system are usually directly accessible by the examination of the explicit instructions in the program code (even if this access will be less easy in the future with machine learning), such an examination is difficult to conduct with humans. Human cognitive activity has two parts: an explicit part, which is the directly observable overt behavior, and an implicit part, only attainable by interpretation and by determining the behavior structure (Jean Piaget, 1947). Therefore, models may help identify main cognitive functions involved during human activities, but many are hidden and can only be assumed by the designer. Two main aspects of human cognitive activity will now be presented, they tackle the models and characteristics which would be used by designers.

### 2.1.1 Human information processing

Among the tools and approaches proposed by human-centered design and HMS approaches, the model of human information processing put forwards by Rasmussen and Goodstein in 1987 (Rasmussen & Goodstein, 1987) was used as a basis to identify the human ability to perceive, diagnose and control a process (cf. Fig. 4). In the decisional ladder presented by the authors, all the steps followed by a human are detailed, from the detection of a need of action to implement on the process, to the execution of a formulated procedure. In 1995, Hoc and Amalberti proposed a revised version of the decisional ladder, coming from research on diagnosis, both in experimental psychology and in cognitive engineering, with nine main steps.
<figure><img src="image_6.png" /><figcaption>Fig. 4 Schematic map of the sequences of information processes (Rasmussen & Goodstein, 1987)</figcaption></figure>

Intermediate steps might be avoided with shortcut arrows between going up phases and going down phases. The possibility of shortcuts depends on the human's experience and knowledge regarding the situation of the process that they have to manage and the target to be reached. Rasmussen proposed a synthesis of such shortcuts through the KRS-based model (Rasmussen, 1983). The model, presented in Fig. 5, suggests extracting three types of behavior. The more complicated and longer-term loop behavior, called the Knowledge-based behavior (KBB), is used when the process state is unfamiliar to the human and when this human needs to utilize their knowledge to find out a solution. The intermediate behavior, called the Rule-based behavior (RBB), is used when the human has already met similar process problems in the past and had found up global solutions. Rules had been defined that can be applied again on similar situations. The shorter-term loop behavior, called Skill-based behavior (SBB), is no longer a cognitive behavior but a sensorimotor one, close to the automated loops well known by the automation engineers. While KBB and RBB are conscious processes, SBB is a subconscious one.
<figure><img src="image_7.png" /><figcaption>Fig. 5 KRS-based model (Rasmussen, 1983)</figcaption></figure>

Such models (Fig. 4 and Fig. 5) are very useful for automation designers to provide human with assistance systems adapted to each step. The next list presents the nine steps and the several assistance systems that were proposed to support human cognitive activity (Taborin & Millot, 1989). The global steps are:

* *Detection*: assistance systems to keep human process representation and to support process monitoring

* *Explicit search*: assistance system to filter information among and to provide relevant ones regarding the alert

* *Identification*: assistance system to check the certainty of a hypothesis to predict process state

* *Interpretation*: assistance system to infer a hypothesis to anticipate process state

* *Evaluation*: assistance system to test a hypothesis to diagnose process state

* *Task definition*: assistance system to define task to solve process problem

* *Goal definition*: assistance system to define goal to solve process problem

* *Planning*: assistance system to plan goals and tasks identified in the previous steps

* *Execution*: assistance system to implement actions and check that the implementation was successful and had the desired effect to solve process problem

Rasmussen's models support our research in identifying human capabilities according to the type of decision they have to make, and according to their expertise and experience in the tasks to perform. Chapter 3 proposes an extension of these models, for now focusing on the interaction between human and process, to the interaction between human and assistance system.

Humans adapt their behavior according to process state and evolution. Whatever the process, most of the time, it can be considered as a dynamic situation that a human has to supervise and control. The characterization of the type of dynamic situation may help automation designers in the choice of assistance system it would be interesting to provide to the human. These characterizations are now presented.
2.1.2 Cognitive dimensions to control dynamic situations

Hoc proposed five cognitive dimensions influencing humans in the supervision and control of dynamic situations (Jean-Michel Hoc, 2000). These dimensions are the supervisory span, the control directness, the process information accessibility, process speed and are continuity. For each dimension, examples of assistance systems extracted from HMS studies are given in order to mitigate or compensate the negative effect of an excessive value in the dimension (Marie-Pierre Pacaux-Lemoine, Berdal, Enjalbert, & Trentesaux, 2018).

The first dimension relates to the *supervisory span*. It concerns the ability for human to access to process variables due to temporal, causal or spatial restriction or excess and their combination. A restricted supervisory span may result in the difficulty to anticipate process states regarding the speed of action feedbacks. An example of a dynamic situation presenting this characteristic is the control of heavy and long ships. A ship needs time to reach the direction or the speed requested by the pilot, and external unpredictable events like waves, winds and sea currents disturb its trajectory. Assistance systems provide useful support for anticipating disturbances using simulation tools. In contrast, a large supervisory span may lead to difficulty in identifying process state, to build a model and to make decision at the right time before decision begins to be obsolete. A typical example is fighter aircraft piloting. During certain phases of the flight, a pilot is unable to make a decision due to the high speed of the aircraft and the rapid sequences of actions. All actions must be planned before the mission and updated during the flight. Recent fighter aircrafts are now equipped with assistance systems aiming at supporting task plan update on the tactical situation (SITAC display). This dimension deals with system complexity regarding the difficulty to identify causal relationship between variables.

The second dimension relates to the *control directness*. It deals with the length of causal chains and the impact of intermediate variables, not fully controlled or predicted. Control directness affects system controllability, mainly due to a large number of variables. A large control directness, with overly long causal chains, may prevent humans from anticipating what the impact of their actions could be. A process with restricted control is more robust because complications are less likely to arise. Control directness relates then to complicated and dynamic systems. Planes, trains, plants present large control directness, and more and more assistance systems have to be designed in order to restrain this dimension to allow human to grasp global process through its sub-parts. The Human-Machine System approach provides tools to make shorter causal chains with assistance systems like information screening, prediction and simulation supports.

The third dimension concerns the *process information accessibility*. This dimension also concerns accessibility of process variables, but it deals with the absence of the variable that must be calculated or assessed in the base of "surface" variables. System observability is relevant to this dimension. A variable, which is not accessible, implies that the human has to make inferences and hypotheses. It could also be a problem of timing. Indeed, a variable can be accessible, but too late, to diagnose process and make decision. In this case, model-based predictions must be made.

The fourth dimension relates to *process speed*. It concerns typically sampling frequency and the time the human has to monitor processes to be sure not to miss an important piece of information. This dimension also deals with the human's ability to plan during control task if the process is slow, e.g. blast furnace, or fast, such
as in highly automated systems. In the first case, simulators provide useful tools to allow a human to build a representation of the process according to a specific time. In the second case, assistance systems like autopilot replace humans' skills and reduce their workload due to time pressure induced by process speed.

The last dimension concerns *process continuity*, which is the evolution of the process and the ability for human to determine significant variations. Assistance systems aim to highlight important information stemming from data analysis, especially regarding sampling.

A human feels these five dimensions differently according to their own expertise, experiences and capacity, and this last ability can evolve very quickly according to the difficulty of the tasks. Human-Machine Systems approach uses these five dimensions in order to design assistance systems adapted to process characteristics and human needs.

> Cognitive dimensions help guide our work regarding the identification of the complexity of processes from the human capabilities point of view, and how assistance systems may support human to mitigate such complexities. Chapter 3 will present the cooperative agent model that supports the design of adaptable assistance systems. Such types of system take into account the dynamic of these dimensions to suit the current and future needs of human in the supervision and the control of the process.

## 2.2 Human and machine organization

In accordance with the previous section, as soon as automation provides results to be integrated in human decisional processing, automation designers may have to cooperate with Human sciences. Automation and Human sciences have to identify common objectives and share common languages, as well as models. The path to cooperation between researches from different fields is a long one, which the HMS domain has already started. This domain aims to consider Human operator and assistance system in the same way, each one having their own competences and capacities which must be updated if necessary to enable fruitful cooperation. Thomas Sheridan put forwards the first principles of HMS in 1974 (T. Sheridan & Ferrell, 1974). The authors proposed models of human performance related to information, control and decision. Such principles aim to adopt a Human-centered design approach, considering the current and future activity of Humans at the first steps of the HMS design. Human-Machine Cooperation focuses on cooperative goals and aims to take into account objectives, competences and characteristics of each agent, both human and technical, to find the best organization regarding the situation to control. Organization can be defined according to different criteria or objectives. Several have already been suggested and evaluated, and are now presented in the following sections.

### 2.2.1 Cooperative structures

The first works in cooperation focused on task allocation and authority management. Millot and Mandiau defined the “vertical” and “horizontal” structures of cooperation (cf. Fig. 6)(Patrick Millot & Mandiau, 1995). In the vertical structure, the machine ability is limited and is only able to provide an analysis of the situation and a set of solutions to a human. The human can select one solution and implement it. The machine has no access to the control of the process but cooperates with the human to find the best solution. An example is the
GPS in a car, which provides advice to the driver who, however, is not obliged to follow this advice: the driver alone has control.

<figure><img src="image_8.png" /><figcaption>Fig. 6 Cooperative structures adapted from (Patrick Millot & Mandiau, 1995)</figcaption></figure>

In the horizontal structure, both human and machine can fulfill all the tasks of the decisional processing from the analysis of a situation to the implementation of actions. Therefore, structures of cooperation depend on the abilities of each agent to select the best action to solve the problem, but the horizontal structure allows implementation by the automation. Expert systems are typical cases of vertical cooperation. Subsequent sections of this chapter consider both structures as a potential ability to cooperate, but more details about the horizontal one will now be given with a description of the cooperative modes.

### 2.2.2 Cooperative modes

Rieger and Greenstein (1982) proposed “explicit” and “implicit” modes of cooperation (cf. Fig. 7)(Rieger & Greenstein, 1982). In the explicit mode, automation offers its solution and the human has to choose between their solution and the solution proposed by the automation. In the implicit mode, automation implements its solution directly. The principles of explicit task allocation tested by these authors were very specific to discrete tasks, based on queuing models on one hand or on the information theory on the other hand, and very difficult to implement in concrete worlds. Five years later, Millot showed their feasibility in a theoretical micro-world involving tasks of supervision of independent variables (Patrick Millot, 1988). In 1992, Millot started supervising several PhD theses which experimented both modes of cooperation in the Air Traffic Control domain (F. Vanderhaegen, Crevits, Debernard, & Millot, 1994). The objectives were to manage Air Traffic Controller workload by way of an assistance system able to manage conflicts between two planes. The human controller decides the task allocation in the explicit mode, and automation decides the task allocation in the implicit mode. One of the findings was that the human controller was not confident enough in the automation’s decision to abandon the control of task allocation, even if they could be overloaded by task allocation, which is an extra task to fulfill. Moreover, automation solutions were not always the best ones or did not suit human solutions regarding overall traffic (§7.1 provides more details about these experiments).
<figure><img src="image_9.png" /><figcaption>Fig. 7 Cooperative modes (solid lines: commands, dotted lines: information) adapted from (Rieger & Greenstein, 1982) and enriched by (Patrick Millot, 1988)</figcaption></figure>

In order to compensate such difficulties, i.e. managing human workload as well as checking usefulness of automation solutions, Lemoine et al. (M.-P. Lemoine, Debernard, Crevits, & Millot, 1996) proposed to experiment a new mode, called the “assisted explicit” mode (cf. Fig. 8). This third mode would allow two types of assistance system to be distinguished. They focus on different goals; one tackles strategic goals when the other tackles tactical goals. The Fig. 8 provides the schema of such an organization. Automation B suggests implementing the solution given by Automation A to Human B who is at an upper decisional level than the one concerned by the control task, i.e. Human A. Therefore, Human B decides the allocation. The assisted explicit mode involves more acceptability and less workload induced by cooperative tasks. Indeed, Air Traffic controllers work in pairs, one controller focuses on aircrafts inside a sector (Human A) when a second controller focuses on aircrafts entering and exiting a sector (Human B). The time span to make a decision as well as type of information managed by the second controller allow having a good anticipation of the first controller’s workload, solutions and agreement with assistance system’s solutions in order to decide task allocation. Moreover, the first controller trusts the second controller because they choose to work together and they have a good model of each other.

<figure><img src="image_10.png" /><figcaption>Fig. 8 Assisted explicit mode (solid lines: commands, dotted lines: information) adapted from (M.-P. Lemoine et al., 1996)</figcaption></figure>

A dedicated assistance system supported the second controller to present future traffic, shareable tasks (i.e. conflicts that the assistance system can manage) as well as the detection of future controller overload. This assistance system makes the decision to allocate a conflict to the first controller or the automation, but the second controller can modify the decision. Results stemming from experiments conducted on this new mode of cooperation demonstrated the interest, especially regarding acceptability and efficiency. More details about this experiment can be found in §7.1.
Previous sections about structures and modes of cooperation mainly dealt with authority management for task allocation, usually using criteria such as human trust and workload, but that is with a general point of view, without providing possibilities to define shareable tasks in a dynamic way according to situation and the evolution of competences and capacities of human and assistance system. The next section presents forms of cooperation, which are more focusing on complementarity between human and assistance system according to the evolution of such abilities.

### 2.2.3 Cooperative forms

Schmidt (1991) proposed three forms of cooperation, the “integrative”, “augmentative” and “deductive” forms of cooperation (cf. Fig. 9)(Schmidt, 1991). The “augmentative” form is the selected form when both the human and the assistance system have similar competences to perform the task. They can share the task in order to adapt the capacity of the overall human-machine system. The “integrative” form is the selected form when human and assistance system have complementary competences. In that case, they can share the task in order to improve the adaptability of the overall human-machine system. The “deductive” form is the selected form when human and assistance system have similar competences, but in this form, they can debate the best solution in order to improve the reliability of the overall human-machine system (Thomas B. Sheridan, 1992). Therefore, forms of cooperation deal with the identification of criteria to organize task sharing.

<figure><img src="image_11.png" /><figcaption>Fig. 9 Cooperative forms adapted from (Schmidt, 1991)</figcaption></figure>

Structures, modes and forms of cooperation provided a basis to analyze and define task allocation, authority management and organization between the human and the machine. Nevertheless, they do not tackle all possible criteria that can influence human-machine system efficiency and acceptability. Other works and approaches proposed new criteria and organizations defining levels of automation and they are presented in the next section.

### 2.2.4 Levels of automation

Levels of automation or automation modes have been discussed for a long time (Rasmussen, 1981; T. B. Sheridan & Verplank, 1978). Several studies applied them, firstly in aviation (Billings, 1996), and more recently in manufacturing (Lindström & Winroth, 2010), in unmanned aerial vehicles (M L Cummings & Mitchell, 2007) and in nuclear power plants (Jou, Yenn, Lin, Yang, & Chiang, 2009). Indeed, one way to study how human and assistance system can work together consists in knowing detailed competences of each one, before deciding task allocation. Sheridan proposed a machine centered approach by defining firstly five human roles for supervisory control (cf. Fig. 10)(T. B. Sheridan, 1984).
<figure><img src="image_12.png" /><figcaption>Fig. 10 Supervisory control (T. B. Sheridan, 1984)</figcaption></figure>

He defined ten levels of automation for decision and control (Thomas B. Sheridan, 1992)(cf. Fig. 11), and ten levels of automation for information acquisition and integration (Toshiyuki Inagaki & Sheridan, 2008).

The computer:

1. Offers no assistance

2. Offers a complete set of decision/action alternatives

3. Narrows the selection down to a few

4. Suggests one alternative

5. Executes that suggestion if the human approves

6. Allows the human a restricted time to veto before automatic execution

7. Executes automatically, then necessarily informs the human

8. Informs the human only if asked

9. Informs the human only if it, the computer, decides to

10. Decides everything, acts autonomously, ignoring the human

<caption>Fig. 11 Degrees of automation for decision and control (Thomas B. Sheridan, 1992)</caption>

The computer:

1. Offers no assistance

2. Suggests many sources of information, or

3. Narrows the sources to a few, or

4. Guides the human to particular information, and

5. Responds to questions posed in restricted syntax, or

6. Responds to questions posed without restricted syntax, and

7. Integrates the information into a coherent presentation, or

8. Integrates the information into a coherent presentation, with an indication of confidence about each aspects, and

9. Passes it to the human or automation for actions, but allows for other considerations

10. The computer collects information as it sees fit, packages it, and presents it to human or automation for action with no opportunity to consider alternatives

<caption>Fig. 12 Levels of automation for data acquisition and integration (Toshiyuki Inagaki & Sheridan, 2008)</caption>
Inagaki proposed adding a level 6.5 for decision and control when the assistance system executes automatically after telling the human what it is going to do (T. Inagaki, Moray, & Itoh, 1998). The capability to make a decision would be split into three levels (from 2 to 4): a proposition of a set of decisions/alternatives, a selection of decisions or one selected decision. Regarding the capability to execute a decision, two levels of decision-making would be made (from 5 to 10). The first level is the capability for the assistance system to perform the execution. The second level is the capability for the assistance tool to decide the task allocation. The fourth and last levels differ in terms of communication aspects. To sum up, Sheridan organized the ten levels according to the precision of the decision-making, the capability to decide allocation and the capability to communicate to human.

Kaber and Endsley offered another taxonomy of levels of automation, implementing a more human-centered approach. Ten levels were also proposed but according to four types of activity: Monitoring, Generating, Selecting and Implementing (Endsley & Kaber, 1999; Kaber & Endsley, 2004)(cf. Fig. 13). The authors described activities as roles allocated to human or to assistance system, or both. Monitoring includes information gathering to perceive process status. Generating allows for the formulating of task strategies to achieve goals. Selecting leads to choosing a particular strategy. Implementing carries out the chosen strategy.

<figure><img src="image_13.png" /><figcaption>Fig. 13 Levels of automation (Endsley & Kaber, 1999)</figcaption></figure>

Parasuraman et al. proposed similar approach by defining automation at four stages (Parasuraman, Sheridan, & Wickens, 2000). Assistance systems can support a human at four main stages (cf. Fig. 14):

* Acquisition of multiple sources of information, including sensory processing, preprocessing of data, and selective attention.

* Manipulation of information in working memory and cognitive operations such as integration, diagnosis, and inference, occurring prior to the point of decision.

* Decisions based on such cognitive processing.

* Entailment of an action consistent with the decision choice.

Percentage of allocation to automation between low and high involvement defines task sharing between human and machine. In Fig. 14, system A is highly automated and system C designed with little automation.
<figure><img src="image_14.png" /><figcaption>Fig. 14 Types and levels of human interaction with automation (Parasuraman et al., 2000).</figcaption></figure>

With a similar objective of finding the best task sharing between human and automation, Flemisch et al. proposed the H-mode, based on the H(orse)-Metaphor (F. O. Flemisch et al., 2003). The metaphor aims to characterize the roles of human and automation as similar to the roles of a rider and a horse respectively. Mainly applied in the automotive domain, the H-mode describes the haptic-multimodal interaction and execution of the driving task by the human and the automation.

<figure><img src="image_15.png" /><figcaption>Fig. 15 Cooperative Guidance & Control with H-Mode (Altendorf et al., 2016)</figcaption></figure>

Table 1 gives a synthesis of the above definitions, mixing the approaches proposed by Sheridan & Inagaki and by Kaber & Endlsey, by integrating progressively more capability of the assistance system and by taking into account types of activity. Criteria were also formalized to distinguish levels for which an assistance system is able to carry out a solution and/or to decide an allocation. Explicit allocation differs from implicit allocation by whether the human confirms or not the allocation respectively. Moreover, there are three kinds of time constraints to decide allocation. The first one used by the level 5 is “t<sub>c</sub>”. It refers to the assistance system’s capability to control allocation (“c” for control). If the human consents to allocate the implementation of action to the assistance system responding to its question, t<sub>c</sub> is different from zero and implementation is automated. This value is also used by the level 6 but with the goal of comparing it to a time limit. If the human does not veto the allocation of decision and/or action, the assistance system completes the task. The other defined times are “t<sub>i</sub>” and “t<sub>a</sub>”. They are respectively the time to provide or to require information (“i” for information) to the other, and the time to perform the action (“a” for action). Levels from 6.5 to 9 are different according to these times. The assistance system performs the task before or after giving feedback.
Communication between the human and the assistance system only deals with feedback about automated implementation. The feedback is implicit when the assistance system always provides feedback about its decision and action. It is explicit when it is on human demand. It is an assessed feedback when the assistance system assesses the necessity to provide the feedback to the human. Contrary to the degrees defined by Sheridan, Kaber and Endsley do not discuss in terms of elapsed time to decide allocation or to give feedback. They focus on the need for the human to validate the allocation. Moreover, the allocation does not relate to action but also to diagnosis and decision making.

The representation of levels of automation given in Table 1 underlines what would be a sharing of the role according to the type of capability human and assistance tool have. A look at this table (from level 2 until 9) shows that both agents need to gather information. However, information acquisition can be different between the two agents, especially if the human and the assistance tool have not the same access to process information. To sum up, Table 1 underlines the task to allocate to the human, to the assistance tool or both. When only one agent can perform a task, it is a predefined task allocation. When two agents can perform a task, criteria must support the selection of one or the split of the task into subtasks in order to share the load.

<table><caption>Table 1 Comparisons of automation scales (Pacaux et al., 2011)</caption><thead><tr><th></th><th>Levels of automation</th><th>Criteria of task allocation</th><th>Information acquisition</th><th>Information analysis</th><th>Decision and action selection</th><th>Action implementation</th></tr></thead><tbody><tr><td>1-(1)</td><td>No assistance</td><td></td><td>H</td><td>H</td><td>H</td><td>H</td></tr><tr><td>2</td><td>Set of decisions</td><td></td><td>H/M</td><td>M</td><td>H</td><td>H</td></tr><tr><td>3</td><td>Selection of a few decisions</td><td></td><td>H/M</td><td>M</td><td>H</td><td>H</td></tr><tr><td>4</td><td>Selection of one decision</td><td></td><td>H/M</td><td>M</td><td>H</td><td>H</td></tr><tr><td>(2)</td><td>Action support</td><td></td><td>H/M</td><td>H</td><td>H</td><td>H & M</td></tr><tr><td>(3)</td><td>Batch Processing</td><td>Explicit allocation of action</td><td>H/M</td><td>H</td><td>H</td><td>M</td></tr><tr><td>(4)</td><td>Sharing control</td><td>Explicit allocation of action</td><td>H/M</td><td>H & M</td><td>H</td><td>H & M</td></tr><tr><td>(5)</td><td>Decision support</td><td>Explicit allocation of action</td><td>H/M</td><td>H & M</td><td>H</td><td>M</td></tr><tr><td>5</td><td>Carrying out of one solution</td><td>if t<sub>c</sub>(H) ≠ 0</td><td>H/M</td><td>M</td><td>H</td><td>H/M</td></tr><tr><td>(6)</td><td>Blended decision making</td><td>Explicit allocation for each stage</td><td>H/M</td><td>H & M</td><td>H or M</td><td>M</td></tr><tr><td>6</td><td>Carrying out of one solution</td><td>if t<sub>c</sub>(H) &lt; t<sub>limit</sub></td><td>H/M</td><td>M</td><td>H/M</td><td>H/M</td></tr><tr><td>(7)</td><td>Rigid system</td><td>Explicit allocation of action</td><td>H/M</td><td>M</td><td>H</td><td>M</td></tr><tr><td>(8)</td><td>Automated decision making</td><td>Implicit allocation for each stage</td><td>H/M</td><td>M & H</td><td>M</td><td>M</td></tr><tr><td>(9)</td><td>Supervisory control</td><td>Implicit allocation for each stage</td><td>H/M</td><td>M or H</td><td>M</td><td>M</td></tr><tr><td>6.5</td><td>Carrying out of one solution</td><td>t<sub>i</sub>(M) &lt; t<sub>a</sub>(M)</td><td>H/M</td><td>M</td><td>M</td><td>M</td></tr><tr><td>7</td><td>Carrying out of one solution</td><td>t<sub>i</sub>(M) &gt; t<sub>a</sub>(M), implicit feedback</td><td>H/M</td><td>M</td><td>M</td><td>M</td></tr><tr><td>8</td><td>Carrying out of one solution</td><td>t<sub>i</sub>(M) &gt; t<sub>a</sub>(M), explicit feedback</td><td>H/M</td><td>M</td><td>M</td><td>M</td></tr><tr><td>9</td><td>Carrying out of one solution</td><td>t<sub>i</sub>(M) &gt; t<sub>a</sub>(M), assessed feedback</td><td>H/M</td><td>M</td><td>M</td><td>M</td></tr><tr><td>10-(10)</td><td>Full automation</td><td></td><td>M</td><td>M</td><td>M</td><td>M</td></tr></tbody></table>

Table 1 also underlines some shortcomings. For example, the four stages proposed by Parasuraman may help design the functions according to a different combination. Indeed, the combination may be constructed with a sharing or a trading of control (T. B. Sheridan, 2002), and when functions are shared, the combination may be constructed according to the 3 forms of cooperation which define a basis (Grislin-Le Strugeon & Millot, 1999).
Application domains like the automotive and railway industries have already defined their own levels according to their own needs. The Society of Automotive Engineers has 5 levels of automation, while the NHTSA (National Highway Traffic Safety Administration) has 4 levels (SAE, 2013), and the railway industry has 4 Grades of Automation (GoA) (Cappaert-Blondelle, 2012). Studies dealing with such definitions are detailed in §7.2 and §7.7 respectively, but they also highlighted the need to improve how levels of automation are characterized. Human-Machine Cooperation framework can tackle such issues, with even more information, especially regarding the ability to integrate more information about other agents involved in the same process to control. The next section presents parallel approaches, usually conducted in different fields, such as Computer Science, robotics and aviation, and dealing with levels of autonomy.

> This study leads us to question the interest in defining precise levels of automation when most of the time designers have to define levels to suit to their problematics. Chapter 0 proposes following another path to find the best adaptation between human and assistance system, according to more criteria than just time, feedback or decision making of function allocation. Criteria might also be extracted from human, assistance system and process states. Such a reconsideration of levels of automation has also recently been highlighted by Thomas Sheridan and Toshiyuki Inagaki, two of the main authors dealing with these topics (Toshiyuki Inagaki & Sheridan, 2018).

### 2.2.5 Levels of autonomy

With the increase of machine abilities, especially regarding the function of decision making and authority management, new scales have been proposed and these concern levels of autonomy. Authors defined these levels according to specific functions and objectives to reach. Examples can be found in robotics (Cosenzo et al., 2010a; Prewett, Johnson, Saboe, Elliott, & Coover, 2010; D. D. Woods, Tittle, Feil, & Roesler, 2004), in industry (Skjerve & Skraaning, 2004) and in aeronautics (Krobusek, Boys, & Palko, 1988). In robotics, for example, Goodrich et al. proposed 5 levels (fully autonomous, autonomous with goal biases, waypoint methods, intelligent teleoperation, and dormant) (Goodrich, Olsen, Crandall, & Palmer, 2001).

Regarding difference between automation and autonomy, some clarifications have been recently provided by the Joint Air Power Competence Centre. This is a team of multinational experts who provide key decision-makers effective solutions on Air and Space Power challenges, in order to safeguard NATO and the Nations' interests. They worked on the definitions and distinctions of automation, autonomy and autonomicity (Haider & Catarrasi, 2016). They argued that: — "an automated system follows a pre-defined and finite thus predictable sequence of actions according to initial or continued human authorization"; — "an autonomous system independently decides its own courses of action to achieve its given objective without human intervention"; — "an autonomic system selects from a pre-defined and finite set of actions to achieve its given objective without supervision unless a human intervenes". Therefore, autonomous systems are less predictable than automated systems, as well as being more independent from humans, and so might be more competent since they able to complete several combined functions, though they are less adapted to human activities. Therefore, it would be interesting to find an intermediate design with the advantages and without the disadvantages of both. Chapter 4 partly uses studies conducted on levels of autonomy to propose a method to
identify appropriate levels of automation for a specific application, especially regarding their adaptability
according to several criteria.

However, studies about levels of autonomy are also interesting as they deal with higher decisional levels and complex organization with several humans and machines than the ones usually considered by automation. Fig. 16 provides an example presenting the roadmap developped by the Department of Defense of the United States to design Unmanned Aerial vehicles.

<figure><img src="image_16.png" /><figcaption>Fig. 16 Autonomous Control Levels (Adapted from the Roadmap proposed by the Department of Defense of the United States to design Unmanned Aerial vehicles (Galster et al., 2007))</figcaption></figure>

The proposed levels took into account three levels of activity, the operational level for individuals, the tactical level for groups and the strategic level for teams. Such an extension of the levels of automation/autonomy will also be discussed in chapter 4 with the definition of the multilevel aspect of cooperation, as well as with the layers of cooperation.

Levels of autonomy and autonomous systems are being cited more and more frequently in current projects, in particular with the expansion of works involving “artificial intelligence”. Autonomous systems are able to make decisions and have the authority to implement their solutions. In the future, they might also be able to learn and modify their algorithms. The unpredictability and the difficulty for humans to understand autonomous systems has already been underlined more than 20 years ago with an article about “Automation surprises” (Sarter, Woods, & Billings, 1997). As such problems have not been resolved yet, new ability like learning, will again increase the number of human surprises. Nevertheless, some tools already supported system designers to prevent such surprises, among them the notion of awareness, which is presented in the next section.

## 2.3 Awareness

The more the number of agents involved in the control of a process or an environment increases and the more agents delegate part of their functions or tasks to other agents, the less they are aware of the process states and other agents' state and involvement. Awareness quickly appears as a central objective in the design of human-machine systems. Literature underlines different aspects of awareness. It may deal with the situation of the process/environment, but also with the situation of the team involved in the control and especially the management of roles and authority within this team. These aspects are described in the following sections.
### 2.3.1 Situation awareness

Situation awareness (SA) "is the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future" (Endsley, 1995b). For the author, SA is the product of mental mechanisms of perception and information processing based on a three-level model:

* SA level 1 is about the perception of the status, attributes, and dynamics of relevant elements in the environment,

* SA level 2 is about the comprehension of a situation, being aware of such elements to include an understanding of their significance in light of pertinent operator goals, and

* SA level 3 is about the projection of the future status of such elements in the environment.

This model remains the most popular as it is the most understandable and close to Rasmussen's problem solving functional model presented in a previous section (Patrick Millot, 2015). The model depicts SA as a component of information processing that follows perception and leads to decision making and action execution (cf. Fig. 17). Mental models (formed by training and experience) are used to facilitate the achievement of SA levels and the development of their maintenance. The very complete paper by Salmon et al. (2007) describes the numerous attempts for defining SA, the authors state that Endsley's framework remains the most commonly used.

<figure><img src="image_17.png" /><figcaption>Fig. 17 SA three-level model adapted from (Endsley, 1995b)</figcaption></figure>

In complement to description of individual SA, Salmon et al. (2007) also emphasize the interest of Team SA.
## 2.3.2 Team situation awareness

They found extensions of the SA studies to groups of decision makers and to collective works. Several researchers have developed frameworks to understand and describe how Situation Awareness can be managed collectively in a group of humans and artificial agents (Salas, Prince, Baker, & Shrestha, 1995; Salmon et al., 2007; Shu & Furuta, 2005). This research topic is still in progress; several studies have been carried out in several application domains like car driving and very distributed situations involving several agents such as road traffic, troop deployment or air traffic. New theories are proposed and tested in order to fix this complicated problem. That results in a great interest for SA and especially the collective SA, in order to evaluate and maintain the human SA level through an adequate adaptive and cooperative automation. The term collective SA described the different meanings proposed in the literature and related to the SA acquired by several people and/or artificial cognitive systems:

*   **Team SA** : Salas et al. suggested “a shared understanding of a situation among team members at one point in time” (Salas et al., 1995); Endsley (1995a) conceives team SA “whereby each team member has a specific set of SA elements about which he or she is concerned, as determined by each member's responsibilities within the team”. Shu and Furuta (2005), on the base of Endsley’s model, proposed a combination of both individual SA and mutual beliefs, as well as a Team SA inference method to evaluate dynamically the impact of new types of interfaces or support systems.

*   **Shared SA** : according to Endsley and Robertson (2000) it refers to “the same SA across shared requirements”. Bolstad and Endsley (2000) propose a model of Team SA based on four components:

*   Shared SA Requirements as “the degree to which the team members know which information needs to be shared”,

*   Shared SA Devices as “the devices available for sharing this information”,

*   Shared SA Mechanisms as “the degree to which team members possess mechanisms” to be able to interpret information,

*   Shared SA Processes as “the degree to which team members engage in effective processes for sharing SA information ... checking each other for conflicting information or perceptions, setting up coordination and prioritization of tasks, and establishing contingency planning among others”. Salmon et al. (2007) defines Team SA as a combination of individual team member SA, shared SA and the combined SA of the whole team (cf. Fig. 18).

*   **Distributed SA** : Stanton et al. (2006) suggested following set of tenets:

*   SA is held by human and non-human agents,

*   Different agents have different views on the same scene,

*   Whether or not one agent’s SA overlaps with that of another depends on their respective goals,

*   Communication between agents may be non-verbal behavior, customs, and practice,

*   SA holds loosely coupled systems together,

*   One agent may compensate for degradation in SA in another agent. Therefore, each agent has an important role in the development and maintenance of other agents’ SA.
<figure><img src="image_18.png" /><figcaption>Fig. 18 Team Situation Awareness (Salmon et al., 2007)</figcaption></figure>

Salmon et al. (2007) deduce that teams need to share pertinent data at the higher levels of SA, such as the significance of SA, elements to the team goals and projected states. The team performance will thus depend on the shared goals, the interdependence of team member actions and on the division of labor between team members. Fig. 18 details such interdependence between individual and team SA, based on taskwork and teamwork (Morgan, Glickman, Woodard, Blaiwes, & Salas, 1986). The lack of convergence on team-SA definition seems to reflect a lack of agreement on the foundations to model the phenomenon. Nevertheless the construct of team process seems very close to the framework developed on human-machine cooperation (P. Millot & Pacaux-Lemoine, 2014).

Mode is another word sometimes used to deal with level (Baker & Yanco, 2004a). As explained in the sections about level of automation/autonomy, the selection of one level or mode predefines the roles allocated to human and machine. The fact that human is aware of the role of each one, and in particular what the machine is intended to do, was defined as the mode awareness (Sarter & Woods, 1995; Sarter et al., 1997; Vakil, Hansman, Midkiff, & Vaneck, 1995). These authors mainly focused on the mode awareness problems since, in the example of aviation, pilots may not be aware of the engaged mode.

Situation Awareness studies support our research regarding the use of other criteria to define levels of automation. As an extension of the studies dealing with mode awareness, it seems obvious that all agents involved in cooperative activities to control a process or an environment must be aware of the role of each one. Chapter 4 will describe how we can support awareness, whether to be informed and to understand the process situation, or the agents’ state and objectives, or the cooperation between these agents. We called such a support a Common Work Space.
## SYNTHESIS ON THE STATE OF THE ART

The present second chapter presented the key elements that must be part of the design of good cooperation between human and machine (cf. Fig. 19). Models representing humans may provide information about humans depending on the complexity of situation and task to complete. Machine can then integrate such information to adapt itself to human needs in an explicit or an implicit way. Models of organization underline how rich the interaction can be, but at the same time, how complex this interaction could also be if designers do not take precautions. One of precautions is to support general awareness. Therefore, competences and capacities of agents to control process and to be aware of other agents involved in this control must be the bases of the design of human-machine systems. The combination of these bases may organize the human-machine system in a way that it is more reliable and adaptable to the dynamic of process due to new events and changes in agents’ state. The Human-Machine Cooperation framework aims to reach this goal by analyzing and integrating parts or all elements of models previously presented, and by proposing a generic model of agents. The chapter 3 presents this model and Human-Machine Cooperation principles.

<figure><img src="image_19.png" /><figcaption>Fig. 19 Synthesis on the state of the art</figcaption></figure>
**Chapter 3. Towards Human-Machine Cooperation**

Every word or concept, clear as it may seem to be, has only a
limited range of applicability, W. Heisenberg, *Physics and
Philosophy*, 1958

As far as the laws of mathematics refer to reality, they are not
certain, and as far as they are certain, they do not refer to reality.
‘Geometry and Experience’, an expanded form of an address by
Albert Einstein to the Prussian Academy of Sciences in Berlin, 27
January 1921

The further we penetrate into the submicroscopic world, the more
we shall realize how the modern physicist, like the Eastern mystic,
has come to see the world as a system of inseparable and ever-moving
components. *Fritjof Capra, The Tao of Physics, 1975*

“A map is not the territory” (1933). *Alfred Korzybski, Science and
Sanity: An Introduction to Non-Aristotelian Systems and General
Semantics*

1 + 1 = 3 (at least, I hope with all my heart), The Encyclopedia of
Absolute and Relative Knowledge, Edmond Wells, *The Revolution of
the Ants, Bernard Werber, 1996*
The principles of Human-Machine Cooperation (HMC) have been laid down with a focus on Dynamic Task Allocation (Patrick Millot, 1988; Patrick Millot & Kamoun, 1988). Several domains were particularly adapted to apply these principles, with the objective of sharing or trading functions between human and machine, including air traffic control, fighting aircraft, car driving, robotics, manufacturing as well as the railway industry. Studies conducted through the HMC approach aim to evaluate the risk for Human-machine system to reach an unstable and unrecoverable state. Such a goal can be achieved by proposing a model of cooperation and a method of application in order to assess current or future Human-machine interactions. Indeed, risk could arise from the environment or from an inappropriate behavior of humans or machines regarding a situation. The next section details the model a cooperative agent. The model has evolved, since the late 1990s, when we used it for the first time during the study in air traffic control, until today in manufacturing and the railway industry. The fact is that machines offer more and more abilities to control process and situations, but interaction with human and combination of these new abilities with humans' abilities, are most of the time forgotten. The model of cooperative interaction then presents how to identify and test potential combinations of function. The following sections detail the types of ability to combine, i.e. sub-functions of "Know-How" and "Know-How-to-Cooperate" which will be defined, and the definition of a space able to support cooperation through efficient communication tools, called the "Common Work Space". The last section deals with the extension of the principles of HMC to more complex socio-technical systems with the definition of the layers of cooperation.

## 3.1 Definition of cooperation

All the work I did on cooperation began with many discussions, system designs, experiments and analysis conducted with Jean-Michel Hoc, researcher in psychology in LAMIH, Patrick Millot and Serge Debernard, professors in LAMIH. Hoc proposed the next definition inspired by (J. Piaget, 1977): "Human and machine are in a situation of cooperation if: (a) each of them strives towards goals although interfering with the goals of the other (at least based on resources or procedures), and (b) they try to manage such interference to make easier the activities of the other" (Jean-Michel Hoc, 1996; Patrick Millot & Hoc, 1997). In the definition the term "interference" is borrowed from physics in a metaphoric way and its connotation is not purely negative (signals in phase opposition), but also includes a positive aspect (signals in phase). The definition of cooperation echoes the definition of team proposed by Salas et al. as: "distinguishable set of two or more people who interact dynamically, interdependently, and adaptively toward a common and valued goal/object/mission, who have each been assigned specific roles or functions to perform, and who have a limited life span of membership" (Salas et al., 1995). Based on the definition of cooperation, we proposed a model of cooperative agents, humans and/or machines, agents that share a common goal and have to manage interferences to adjust their decisions and actions.
* **Know-How (KH)**

* Internal ability to solve problems (regarding the process)

* capabilities: knowledge, rules, skills / experience, expertise

* processing abilities: inferences, workload, fatigue...

* External ability to:

* get information (from the process and the environment)

* act (on the process)

* **Know-How-to-Cooperate (KHC)**

* Internal ability to:

* build up a model of other agents (KH and KHC)

* deduce the other agents' intentions

* analyze the task and identify the cooperative organization

* produce a common plan regarding tasks and coordination

* External ability to communicate:

* understanding other agents

* providing information to other agents

<caption>Fig. 20 Model of cooperative agent adapted from (Patrick Millot & Pacaux-Lemoine, 2013)</caption>

Therefore, Human-Machine Cooperation studies lead to a definition of the model of an agent based on two dimensions: — the agent's ability to control a process or an environment, also called the “Know-How”, and — the agent's ability to cooperate with other agents concerned by the same process or environment, also called the “Know-How-to-Cooperate” (Patrick Millot & Lemoine, 1998). Fig. 20 proposes a synthesis of cooperation attributes, as presented in its last version.

Human cognitive process merges both types of abilities, and it is usually the case during the design of assistance system. Nevertheless, it is interesting to clarify the distinction by highlighting all the types of interactions between agents in order to avoid the recurrent problem of cooperation.

## 3.2 Towards a meta-model of cooperation

A meta-model of cooperation may support the design and the evaluation of Human-Machine System, especially regarding cooperative aspects. The cooperative agent model is the basis of the meta-model. It specifies and justifies the definition of the three main entities, the “Know-How”, the “Know-How-to-Cooperate” and the “Common Work Space” now presented. A generic model of cooperative agents’ interactions then offers a schematic representation of cooperation as the result of the complementarity of these three entities. They were defined during my PhD when we had to design cooperative systems to support air traffic controllers’ activities when they had to tackle an increase in traffic (Jean-Michel Hoc & Lemoine, 1998a; M. P. Lemoine, 1998) (cf. Appendix 1 - 7.1).

### 3.2.1 The “Know-How” or the abilities to control the process (KH)

The *Know-How* (KH) of an agent focuses the control of a process or an environment, and not the interaction with other agents. A process is composed of a set of passive and active elements. Passive elements refer to infrastructure, simple mechanisms, and basic objects, i.e. elements with no ability to decide and act. Active elements are more or less complex systems, like complicated mechanisms, but there are systems with which an agent considers that is not possible to cooperate; the agent must take it as it is, like the traffic in the case of
transport systems, hostile environment in case of robotics application. In the case of air traffic control, air traffic is the process since we did not consider the cooperation between air traffic controllers and aircraft pilots in our study. Therefore, we consider that the process is not seen as “intelligent” (no decision or action made by the process), but human and technical agents may control it in a cooperative and intelligent way. It is important to define a process in order to be able then to identify who or what is a cooperative agent. In the next parts of the document, we will use the term process, if it is an operative part like in manufacturing or an environment in which vehicle and robots act.

The KH is split up into two parts, one is called the internal KH and the other the external KH (Patrick Millot & Pacaux-Lemoine, 2013). *The internal KH* relates agents’ problem-solving processing, *i.e.* their competences and capacity to control a process. Section §2.1 provided a description of competence and behavior of humans, based on knowledge, rules and skills they gained to control the process, according to their expertise and experience (*i.e.* practices) (Rasmussen, 1983). Expertise relates knowledge and rules, when experience is more skill based and improved by training. The more complex a process is, the more an agent needs competence (or procedure) to control it. The more complicated a process is, the more an agent needs training and practice to control it. Complication relates more to the capacity of the agent. The capacity of an agent deals with the number of tasks the agent can perform in a period. It also addresses the endurance of the agent over a long period of time. Competence and capacity affect each other, such as in the definition of acceptable performance, which could be a function of efficiency and cost, with efficiency focusing on competence and cost on capacity (Patrick Millot & Pacaux-Lemoine, 2013). In our air traffic control study, the internal KH of the assistance systems we designed consisted in detecting and solving the simpler conflicts between aircrafts.

The *external KH* is the ability to access the process information and the ability to act on this process. The external KH especially concerns communication aspects. In our air traffic control study, the external KH of the assistance systems we proposed relates their ability to gather information about traffic and to send command to pilots through air-ground data-link communication. Ergonomics rules provide advice to design interface with visible, readable and comprehensible data. The amount of data must be limited and updated in a controlled way, especially when the dynamic of a process is high (cf. 2.1.2). Nevertheless, we will not focus on such ergonomics aspects even if it is very important in the design of tools to support cooperation. The present work is more focusing on the identification of type of data, functions and sub-functions that can be shared or traded between agents.

The KH of technical agents is “normally” more obvious to extract, especially by having access to its code with the help of their designers. However, the fact is that technical agents can learn and define their own rules and knowledge more and more. The perspectives section will discuss this risky machine evolution from the cooperation point of view. That is why we would now like to specify the distinction between the three terms “automatic”, “automated” and “autonomous”. The term “automatic” usually refers to system with very simple, mechanical responses to environmental input. The term “automated” usually refers to more complex, rule-based systems; while the term “autonomous” refers self-direction and self-learning systems that can have emergent behavior not predictable from its code. The current research deals with systems, which can be self-directed, but not self-learning.
Unlike technical agents, human agents' KH is usually estimated from assessments done before and during work by peers or hierarchy (based on prescribed tasks), but also by the agents themselves. Indeed, human agents build models of themselves thanks to their education, practices and experiences. Self-confidence and trust may depend on their own model. Human agents have to be aware of their competences and capacities regarding the task they have to complete. If they have no confidence in themselves to perform a particular task, they might prefer to cooperate with others and they will ask for help (Goodrich et al., 2001).

Therefore, the internal KH allows agents to build up a representation of the current situation of the process, using their competences and capacities, in order to gather information, to analyze the situation, to make a decision and to implement an action as proposed by (Parasuraman et al., 2000) and presented in §2.2.4. Agents are able to complete these cognitive activities because they interact with process through their external KH. Internal and external KH allow agents to be aware of process state or situation in order to know the current state of the process, to diagnose such a state and to project future states, i.e. the situation awareness presented in §2.3.1. However, because of a lack of competence or a lack of capacity, agents may require the assistance of another agent. In this case, they need another ability, namely the ability to take into account the abilities of other agents and to communicate with them. This specific ability is the Know-How-to-Cooperate. In the air traffic control study, the competence of air traffic controllers was fortunately not in question, but the objective was to share air traffic control with assistance systems in order to manage their workload in case of high number of aircrafts. This was an augmented form of cooperation (cf. §2.2.3).

### 3.2.2 The "Know-How-to-Cooperate" or the agent's ability to cooperate (KHC)

The *Know-How-to-Cooperate* (KHC) allows one agent to take advantage of the complementary KH of other agents. The KHC also has two parts, an external and an internal part. The *internal KHC* allows an agent to build up a model, or at least a representation, of other agents in order to make easier the cooperation with them. This model allows them to be aware of the other actors' concerns, expectations and intentions (Schmidt, 2002). Working, training, interact with others enables the development of such a model. Agents progressively identified, memorized and used KH and KHC of other agents in order to achieve cooperative behavior. In the air traffic study, air traffic controllers already had a KHC regarding their mutual cooperation because they were used to working together and they had procedures they had learned during their training; but they had to build a KHC regarding their cooperation with the assistance systems, in order to know, for example, how they could use them, how they could communicate. Similarly, assistance systems had a KHC by the way of a simple model of air traffic controllers' tasks, and in particular one assistance system was able to give an estimation of the workload of the radar controller (who manages conflicts inside a sector) (M.-P. Pacaux-Lemoine & Debernard, 2002). Nevertheless, even if reliability is high, other characteristics may put trust into question (Lyons & Guznov, 2018). That is the reason why we are analyzing agents' interaction in more depth through the definition of KHC.

With *the internal part of KHC*, agents try to build up shared mental models (Rouse, Cannon-Bowers, & Salas, 1992) or compatible mental models (M P Pacaux-Lemoine & Loiselet, 2002; Flemisch, Kelsch, Löper, Schieben, & Schindler, 2008). In order to build up such a Common Frame of Reference (COFOR) (Jean Michel Hoc, 2001), cooperative agents need to observe and to communicate, so that agents will build up a
model of each other in parallel. For example, a human agent needs to understand the automation reasoning
processes and activities in order to be able to control the overall system (D. D. Woods et al., 2004). With a
model of the others, agents are more confident and trust each other (J. Lee & Moray, 1992). Lee and See's
definition of trust is one of the most widely cited definitions: "the attitude that an agent will help achieve an
individual's goals in a situation characterized by uncertainty and vulnerability" (J. D. Lee & See, 2004). An
agent may trust the abilities of others, but also the quality of their interaction (Rajaonah, Tricot, Anceaux, &
Millot, 2008). Some connections exist between relative trust, self-confidence and confidence in the other (Van
Dongen & Van Maanen, 2013). Having a model of others enables the estimation of their intentions, especially
if they are accountable (Mary L Cummings, 2003).

Regarding technical agents, Balfe et al. identified several characteristics that can contribute to trust (Balfe,
Sharples, & Wilson, 2015). The automation must be reliable, competent, visible, observable, understandable,
directable, robust, and accountable; it must provide proactive control and prevent skill degradation. These
characteristics are at the same time relevant for KH and for KHC, and must be part of the model that a human
may have of a technical agent, and that is what I proposed alongside Makoto Itoh in a Japanese project on car
driving (Makoto Itoh & Pacaux-Lemoine, 2018; Muslim, Itoh, & Pacaux-Lemoine, 2017)(cf. Projects on car
driving §7.2.1). Indeed, with models, other agents' behaviors are more predictable and interaction can be more
constructive. However, too much trust may lead to unacceptable effects, such as "overtrust" and
"overreliance" when an agent overestimates other agents' abilities (Toshiyuki Inagaki & Itoh, 2010, 2013),
"complacency" when an agent underestimates process state evolution (Billings, 1996), and "neglect" effects
when there is not enough mutual control between agents (Goodrich et al., 2001). Such extreme effects may
lead to a total dependency of one agent regarding the high implication of others, and lead to the risk for the
agent to lose the feeling of being in control of the situation (Biester & Bosch, 2005). Nevertheless, involvement
of agent is linked to the willingness to cooperate (Rajaonah et al., 2008)(Nevo, Benbasat, & Wand, 2012), but
also to the responsibility that can be considered such as a motivational factor (Van Dongen & Van Maanen,
2013). That is the affective part of the cooperation (Skjerve & Skraaning, 2004). In the air traffic control study,
participants trusted the assistance systems because they learned their KH and KHC which were clear, easily
usable and responding to some of their needs, especially regarding the organization of their functions, however
responsibility was still an outstanding issue (Frank Flemisch et al., 2012).

One way to build up such a model of others is to communicate with each other or at least to observe the other.
The *external part of KHC* has to support communications or observations. Three main ways allow reaching
these goals: — agents carry out direct observations of others (movements, mimics, emotions...), — agents
have verbal exchanges or they communicate through mediation tools, — agents analyze the activity of others
through their effect of the others' actions on the process.

More and more diagnosis systems to analyze human agent's behavior and state do exist, like (motion capture, analysis of physiological data such as heart rate, electro dermal activity, and eye tracking...). With these types of systems, technical agents receive some information to build up and update their human model. When two humans cooperate, they usually analyze the behavior of the other human's state according to voice intonation and mimics in order to identify the current state. Several examples were found in the project with Dassault aviation and DGA, where we analyzed cooperation in a two-seated fighter aircraft, between a pilot and a
weapon system officer (M P Pacaux-Lemoine & Loiselet, 2002)(cf. Project about fighter aircraft crew §7.3). It was especially useful for the weapon system officer to know when he was allowed to interrupt the pilot's individual activity in order to give him a cooperative activity to take part in. When possible, they could communicate and so make explicit the supposed activity or state of the other that was otherwise only based on visual or audio perceptions. In this case, agents needed to have a common code of communication in order to be able to receive and to understand information from others and to transmit comprehensible information to others. With a model of others, agents can adapt information they want to transmit to the potential receiver. With mediated communication, visual, audio, tactical or haptic aids transmit information. Such aids could be useful for agents to share their understanding of the situation and to negotiate decision making if necessary, mainly when cooperation is asynchronous. This kind of cooperation tool, we called *Common Work Space* (M.-P. Pacaux-Lemoine & Debernard, 2002), is detailed in the next section.

### 3.2.3 The Common Work Space

Several authors in various work domains have already put forwards the notion of work space. In the Air Traffic Control domain, Bentley *et al*. proposed a shared work space which provides an adapted presentation of the air traffic to different users, on different machines, to make the use of shared entities easier (Bentley, Rodden, Sawyer, & Sommerville, 1992). In their work, the shared work space is limited to the external environment and to the representation of the process. Other authors enrich the work space by several elements related to the activity of each agent, and not by raw information but also elaborated ones. Decortis and Pavard define the shared cognitive environment as a set of facts and hypothesis which are a subset of the cognitive universe of each agent (Decortis & Pavard, 1994). In the specific field of Knowledge Based Systems, Brézillon and Pomerol pointed out the necessity for human and assistance system to share a contextual knowledge (Brézillon & Pomerol, 1999). For these authors, it is necessary to share the cognitive representations in order to agree the contextual knowledge of problem solving, to enable the following of each other's reasoning, the exchange of explanation and to control the interaction. Moreover, Royer specifies that a system is all the more cooperative since the cooperation implies more levels, such as perception, analysis, decision and action (Royer, 1994). Indeed, Karsenty underlines the relevance of a shared problem representation to support the process of explanation during which co-workers articulate their individual problem representations and thus, build a richer shared problem representation, rend a deeper analysis of the decision space, and thus reach a better decision (Karsenty, 2000). In the same year, Sonnenwald and Pierce dealt with "Interwoven Situational Awareness", which is "the continuous extraction of environmental information, integration of this information with previous knowledge to form a coherent mental picture in directing further perception and anticipating future events" (Sonnenwald & Pierce, 2000). They propose an interwoven situational awareness that includes the individual, intra group and inter group situational awareness for the Control and Command team members. Indeed, team members need to collect, synthesize and disseminate information to create an understanding of the current battlefield situation and to anticipate future battlefield events. The situational awareness is based on information about the dynamic work goal and situation (the environmental information: mission, enemy, terrain/weather, troops and time available), the work process (tasks, formal and informal task procedures) and specialized field knowledge (information about battle tactics, decision analysis algorithms or methods, biological chemicals, enemy profiles/characteristics, telecommunications networks, civilian government...). Jones and Jasek integrate the same idea into the building of an Intelligent Support for Activity Management
(ISAM) which allows the sharing of goals, contextual information and allocation of functions between the agents (P. Jones & Jasek, 1997). Gutwin and Greenberg highlight the importance for integrating the activity of the other agents and deal with a work space awareness which is “the up-to-the-moment understanding of another person’s interaction with the shared workspace” (Gutwin & Greenberg, 1999). The shared work space allows agents to know who is working, what they are doing, where they are working, when various events happen and how those events occur. The environment provides such knowledge that enables actions to be performed or potential actions to be explored.

Cooperative agents develop and maintain a Common Frame Of Reference (COFOR), which is the common part of their respective current representation of the situation. Common Work Space (CWS) is the external support of the COFOR (M P Pacaux-Lemoine & Loiselet, 2002)(cf. Fig. 21). CWS is the external support of a combination of the representation of process states and agents’ “mental states”, called internal current representation on the figure. CWS is a tool for shared or mutual cognitive environment (Rognin, Salembier, & Zouinar, 2000; Sperber & Wilson, 1987). Sperber and Wilson assert that “A mutual cognitive environment gives each of those who share it evidence of the other’s beliefs, including, to some extent, beliefs about the other’s beliefs, provided they also have evidence of where the other’s attention has been and is going”.

<figure><img src="image_20.png" /><figcaption>Fig. 21 Updating the Common Frame of Reference, adapted from (M P Pacaux-Lemoine & Loiselet, 2002)</figcaption></figure>

Fig. 22 demonstrates a schema detailing the interest of CWS during cooperative activities. A representation of process or situation states provide input to agent 1 and agent 2 by direct observation or through interfaces by gathering information from process. These interfaces also allow the agent to act on the process (bidirectional arrows between Ag₁ internal KH and Ag₁'s interfaces). Therefore, each agent develops and maintains one-agent situation awareness regarding the process state (SA) (unidirectional dotted arrows between Ag₁ SA and Ag₁'s interfaces), but also team situation awareness (team-SA) (Patrick Millot & Pacaux-Lemoine, 2013). CWS can in fact also support cooperative activities. By means of CWS, agents can provide information concerning they own current and future individual activity, but also diagnosis, advices or orders concerning cooperative activity. Supporting such cooperative activities can help humans not to be detracted from their own task (Baker & Yanco, 2004b). Team-SA and COFOR are built in parallel. They result from interference management, i.e. differences which might appear between one-agent situation awareness and the inference of the other-agent situation awareness (red zone between Ag₁ SA and inference of Ag₂ SA). In the
project about fighter aircraft crew and cooperative activity, pilot and weapon systems officers may have a different situation awareness because they have different activities and KH, which lead both to focus their attention on different aspects of the situation (cf. §7.3). Nevertheless, when they have to cooperate on a common task, e.g. pilot controls aircraft trajectory and weapon systems officer illuminates a target, they have to adjust their SA in order to reach a common frame of reference.

<figure><img src="image_21.png" /><figcaption>Fig. 22 Cooperative activities through agents Know-How (Ag<sub>i</sub> KH), agents Know-How-To-Cooperate (Ag<sub>i</sub> KHC), agents Situation Awareness (Ag<sub>i</sub> SA), Common Frame of Reference (COFOR), team Situation Awareness (Team SA) and Common Work Space</figcaption></figure>

The use of the other’s model, so part of the KHC, makes the management of interferences easier. Fig. 22 highlights three ways for an agent to get information about the other and to build such a model. The first one is direct through *communications and observations*. However, this method is time consuming and may disturb the other. The second method is less direct and involves *analyzing Ag<sub>1</sub>'s actions effects* through the process evolution. This method involve too many inferences. The last method is the use of the CWS on which each agent activity is more obvious as well as *other agent's intentions*. Bratman (1992) in (Shu & Furuta, 2005) underlined the benefit to have such information and made the hypothesis that agents attempt to be responsive to both the intention and action of others, and know that others are attempting to be similarly responsive. He called this behavior “mutual responsiveness”.

This section presented the way agents may build and enrich individual and team situation awareness, especially with the use of the CWS. Sasangohar et al. also emphasize benefits to have and maintain SA, especially when events interrupt tasks in progress. They proposed a decision aid similar to a CWS and they demonstrate this assistance that highlights the main points of the situation, and they plot out the evolution of the SA from a chronological account of events (Sasangohar, Scott, & Cummings, 2014). The next section gathers the notions presented in the previous parts, “Know-How”, “Know-How-to-Cooperate” and “Common Work Space”, and combines them in order to propose a generic model of cooperative agents’ interaction, model more suitable for an engineering approach and for automation designers.
### 3.2.4 Towards a generic model of cooperative agents' interaction

Literature on human-machine systems, human decisional processing, human-machine organizations, situation awareness leads us to design tools, KH, KHC and CWS, that we are now able to adjust together as a mechanism that we wish to design and control. This mechanism is an aid for the designer to identify all the necessary functions an assistance system must have, but also that a human may have, training to build models, and how to support interaction between human and assistance system. Moreover, designers can implement this mechanism to have dynamic control over human-machine organization according to function allocation, rules and other types of key elements we will present in the following sections.

Paragraphs 3.2.1 and 3.2.2 gave a detailed presentation of agents' KH and KHC, but as we would like to provide methods to the designer and to be able to implement these entities, we need to bring together the main objectives of the cooperation and propose a few functions as a synthesis of these objectives. From a designer's point of view, KH and KHC can consist of two parallel functions (or task in object-oriented and multi-tasking programming), and each one integrates four sub-functions.

The functions proposed by Parasuraman et al. to specify levels of automation (presented in §2.2.4) define the four sub-functions of the KH. Agent may gather information about the process, analyze gathered information to diagnose/forecast process state, find solutions and select one to make a decision, and implement the decision by sending a command to the process. As explained in §2.1.1 and shown in Fig. 4 (Rasmussen model), shortcuts may prevent the agent from completing the four sub-functions if the agent is skilled with the process or an expert (cf. Fig. 5).

To define the sub-functions of the KHC, we must put together the goals of cooperation given in the definition of cooperation proposed by Hoc (presented in §3.1), and the goal of being aware of others (team situation awareness concept, cf. §2.3.1), and finally, the goal to decide task allocation (concepts about human-machine system organization and authority management, cf. §2.2). Therefore, a cooperative agent has to:

* gather information about teammate agent, in order to have a representation of the current KH and KHC of the other agent (or their usual KH and KHC when models exits),

* detect potential interference between their understanding of the situation (the result stemming from their KH function),

* manage interferences by negotiation, accepting the solution of the other, or imposition of solution to the other,

* decide who manages decisions and actions by task allocation.

Fig. 23 shows a representation of KH and KHC functions in a human-machine system with the automation domain point of view. This model highlights several possible interactions between human and automation (Marie-Pierre Pacaux-Lemoine & Itoh, 2015). KH functions of agents interact by the means of the Common Work Space represented by the blue rectangle with four scales between human and automation parts (scales are based on Parasuraman et al. and Flemisch et al. representations, cf. Fig. 14 and Fig. 15). KHC functions of agents use information provided by the Common Work Space in order to build a model of the other agent (rectangles on the left part of the schema) and to evaluate this agent's involvement in the process control. The
results of this evaluation, compared to the own involvement and model of oneself (interference detection and management), have to lead to the adjustment of the position of sliders that describe the functions allocation (scales represented in the Common Work Space). The scales associated to each KH functions have the same objectives as the scale proposed by Flemisch et al. (2003) (cf. §2.2.4).

The method of representation demonstrated by Fig. 23 makes it possible to gather all the types of organization proposed with the structures, forms and modes of cooperation (§2.2).

<figure><img src="image_22.png" /><figcaption>Fig. 23 Model of cooperative agents’ interaction, adapted from (Marie-Pierre Pacaux-Lemoine & Itoh, 2015) (Marie Pierre Pacaux-Lemoine & Flemisch, 2018)</figcaption></figure>

The position of the slider on the scale defines the degree or the percentage of sharing between human and automation functions. Either automation or human can manage some functions alone according to their competency and capacity (slider completely on one side), but also according to how they can take into account the activity of the other. Therefore, agents have to identify and implement adequate coordination of the four KH functions. The blue rectangles with scale and thin blue straight arrows between each KH function of Fig. 23 represent such a coordination. It must define the best balance between each agent’s involvement, and so find the right place for the slider on the scale for a specific situation. Such a representation may suggest a smooth shifting control authority (Abbink, Mulder, & Boer, 2012). The control of the slider identifies who has the authority. Agents may have the *authority to initiate* and the *authority to terminate* a function (Goodrich et al., 2001). However, this definition focuses on KH functions. Regarding KHC functions, Inagaki defined the *human-initiated automation invocation* and the *system-initiated automation invocation* (T. Inagaki, 2003; Scerbo, 1996). Another way to proceed is to offer the human the possibility of an allocation decision made by the machine, called *autonomy mode suggestion* (Baker & Yanco, 2004b). In the automation invocation strategies, Inagaki proposed initiating cooperation in the case of critical events, the so-called critical-event strategies (T. Inagaki, 2003). Machine prepares an action and waits for human agreement. Information about the human is the basis of the two other strategies proposed by Inagaki. In this case, agents need to have model
of the other agents (rectangles on the left side of Fig. 23). Agents provide information about themselves in order to permit others to update their KHC. The *measurement-based driven strategy* deals with gathering current information about human agent state (workload, attention...) and *model-based driven strategy*, deals with the use of models to predict the performance of human agents (intents, instantaneous capabilities, resources).

This model was proposed, and was very useful, during the project with Thalès and DGA (French Defense) about Human-Robot Cooperation (cf. §7.4.2). We studied the cooperation between a remote human operator who had to do reconnaissance in a risky environment and a robot that was able to control its trajectory during specific kinds of situation. We identified the KH and KHC functions that both human and robot needed to reach efficient cooperation, but also cooperation that is well accepted by the human who was the DGA teleoperators (cf. Table 5 in §7.4.2). Another example of these functions is provided in the chapter about the design method in the case of cooperation study between car driver and driving assistance system (cf. §Fig. 33).

According to the type of situation, events and the state of agents, human and machine have, most of the time, different KH and KHC. Differences can be ranked, which is what we did with the definition of Levels of Cooperation (LoC). LoC use a simple ranking of KH and KHC in order to provide a method to identify possible types of cooperation (Marie-Pierre Pacaux-Lemoine & Vanderhaegen, 2013b). We imagine that an agent builds up a model of the other agent, i.e. their KH and KHC, in terms of Benefit, Cost and Deficit.

The so-called BCD (Benefit/Cost/Deficit) model was initially developed to assess the consequences of human errors on several criteria related to technical or human performance or state (Frédéric Vanderhaegen, 2004). These consequences can be positive or negative. Regarding a reference situation, benefits, costs and possible deficits can assess a given human behavior:

* The benefits due to the success of the control of a given situation.

* The acceptable costs due to the cognitive or physical effort to control a given situation in order to make it successful.

* The unacceptable possible deficit related to the potential occurrence of a hazardous or unprofitable situation, in case of unsuccessful control of a given situation.

As agents compare their own KH and KHC to those of the other, similarities and differences between such abilities emerge. We use the word similarity, instead of equality, because KH or KHC can never be exactly equal, except between two identical machines. KH and KHC define two dimensions that describe Levels of Cooperation, $LoC_{x,y}$. The Table 2 presents the possible LoC according to KH. Cooperation can start if both agents have at least one KH. At $LoC_{1,y}$ one agent has a KHI (internal KH) and the other a KHE (external KH). The first agent analyzes situation and makes decision, and the other gathers information (input) and implements decision (output). A typical example is the remote control of robots. At $LoC_{2,y}$ each agent has a KHI and a KHE, but they are different. Therefore, for the two first levels agents are complementary. At $LoC_{1,y}$ complementarity stands for knowledge and at $LoC_{2,y}$ complementarity stands for process sharing. Both levels can refer respectively to the integrative form and augmentative form of cooperation proposed by Schmidt and presented in §2.2.3. At $LoC_{3,y}$ agents' KH are similar. At this level, agents can debate the results of their
activities. This level can be compared to the debative form of cooperation proposed by Schmidt. The last level, $LoC_{4,y}$, represents a combination of levels 1 to 3.

<table><caption>Table 2 Levels of Cooperation according to KH</caption><thead><tr><th>LoC/KH</th><th>KH<sub>Ag1</sub> and KH<sub>Ag2</sub></th></tr></thead><tbody><tr><td>0</td><td>KH<sub>Ag1</sub> = {Ø} or KH<sub>Ag2</sub> = {Ø}</td></tr><tr><td>1</td><td>KH<sub>Ag1</sub> = KHE<sub>Ag2</sub> = {Ø} and KHE<sub>Ag1</sub> ≠ {Ø} and KH<sub>Ag2</sub> ≠ {Ø} and KH<sub>Ag1</sub> ∩ KH<sub>Ag2</sub> = {Ø}</td></tr><tr><td>2</td><td>KH<sub>Ag1</sub> ≠ {Ø} and KH<sub>Ag2</sub> ≠ {Ø} and KH<sub>Ag1</sub> ∩ KH<sub>Ag2</sub> = {Ø}</td></tr><tr><td>3</td><td>KH<sub>Ag1</sub> ≠ {Ø} and KH<sub>Ag2</sub> ≠ {Ø} and KH<sub>Ag1</sub> ∩ KH<sub>Ag2</sub> = KH<sub>Ag1</sub> = KH<sub>Ag2</sub></td></tr><tr><td>4</td><td>KH<sub>Ag1</sub> ∩ KH<sub>Ag2</sub> ≠ KH<sub>Ag1</sub> ≠ KH<sub>Ag2</sub> ≠ {Ø}</td></tr></tbody></table>

The Table 3 presents LoC regarding KHC. Cooperation can start at the $LoC_{x.1}$ when both agents can communicate but are not able to build up a model of the other. It is usually the case when a human uses a system only occasionally. They can exchange information about the results of their respective KH. At $LoC_{x.2}$ only one agent can build up a model of the other. At $LoC_{x.3}$ both agents can build up a model of the other.

<table><caption>Table 3 Levels of Cooperation according to KHC</caption><thead><tr><th>LoC/KHC</th><th>KHC<sub>Ag1</sub> and KHC<sub>Ag2</sub></th></tr></thead><tbody><tr><td>0</td><td>KHC<sub>Ag1</sub> = {Ø} and KHC<sub>Ag2</sub> = {Ø}</td></tr><tr><td>1</td><td>KHCE<sub>Ag1</sub> = KHCE<sub>Ag2</sub> ≠ {Ø} and KHCI<sub>Ag1</sub> = KHCI<sub>Ag2</sub> = {Ø} and KHC<sub>Ag1</sub> ∩ KHC<sub>Ag2</sub> = KHCE<sub>Ag1</sub></td></tr><tr><td>2</td><td>KHCE<sub>Ag1</sub> = KHCE<sub>Ag2</sub> ≠ {Ø} and KHCI<sub>Ag1</sub> = {Ø} and KHCI<sub>Ag2</sub> ≠ {Ø} and KHC<sub>Ag1</sub> ∩ KHC<sub>Ag2</sub> = KHCE<sub>Ag1</sub></td></tr><tr><td>3</td><td>KHCE<sub>Ag1</sub> = KHCE<sub>Ag2</sub> ≠ {Ø} and KHCI<sub>Ag1</sub> = KHCI<sub>Ag2</sub> ≠ {Ø} and KHC<sub>Ag1</sub> ∩ KHC<sub>Ag2</sub> = KHC<sub>Ag1</sub> = KHC<sub>Ag2</sub></td></tr></tbody></table>

The objective is now to combine levels of each KH and KHC to offer different types of cooperation, so lack of KH can be compensated by better KHC, and vice versa. Table 4 presents such a combination. For example, at $LoC_{1.1}$, agents' individual activities are complementary but dependent, and agents can communicate. Therefore, they can provide or ask each other the results of their respective activities. At $LoC_{1.2}$, there is no change about their KH, they can communicate, but moreover one agent can build up a model of the other. This agent can infer the result that the other would provide. At $LoC_{1.2}$ both agents can make inferences regarding the activity of the other. The cost of $LoC_{1,y}$ is that one agent cannot assess the KH of the other because they do not have the same KH. However, if they have a $LoC_{1.3}$ the benefit is that they have a good model of each other regarding results that they can reach together. In fact, there is a kind of compensation between cost and benefit of each dimension.
<table><caption>Table 4 Matrix of possible levels of Cooperation</caption><thead><tr><th>LoC/KHC</th><th>1</th><th>2</th><th>3</th></tr></thead><tbody><tr><th>LoC/KH</th><td>LoC<sub>1.1</sub></td><td>LoC<sub>1.2</sub></td><td>LoC<sub>1.3</sub></td></tr><tr><th>1</th><td>LoC<sub>2.1</sub></td><td>LoC<sub>2.2</sub></td><td>LoC<sub>2.3</sub></td></tr><tr><th>2</th><td>LoC<sub>3.1</sub></td><td>LoC<sub>3.2</sub></td><td>LoC<sub>3.3</sub></td></tr><tr><th>3</th><td>LoC<sub>4.1</sub></td><td>LoC<sub>4.2</sub></td><td>LoC<sub>4.3</sub></td></tr><tr><th>4</th><td>LoC<sub>4.1</sub></td><td>LoC<sub>4.2</sub></td><td>LoC<sub>4.3</sub></td></tr></tbody></table>

Table 4 is a grid to study possible organizations between human and machine. If that is during the design of a new human-machine system with a selected LoC, designers can identify the best KH and KHC for human and for machine. They also identify criteria (defined by BCD (Marie-Pierre Pacaux-Lemoine & Vanderhaegen, 2013a)) in order to have an efficient human-machine system. Moreover, the grid can also assess LoC during human-machine system activity, when KH and/or KHC of agent or environment change.

This approach has been used to identify criteria that could be implemented to decide task allocation between driver and driving assistance systems in case of risky and pre-crash situations (M. Itoh, Pacaux-Lemoine, Robache, & Morvan, 2013; M.-P. Pacaux-Lemoine, Itoh, Morvan, & Vanderhaegen, 2012).

The next section tackles such a dynamic control of task allocation between human and machine by defining the adaptability of levels of automation.

## 3.3 Adaptability of levels of automation

All the previous sections have asserted the necessity to provide the human-machine system with more flexibility to adapt itself to new events, new environments, changes in the competences and capacities of agents involved in the process control. Adaptability of levels of automation can bring key elements to reach this objective. The previous section dealt with the implementation of a kind of mechanism that may support human-machine system design and dynamic adaptation during its use. This section presents decisional trees able to manage this approach.

Before describing the numerous steps that can lead to adaptability, we would like to present previous studies conducted in the past with the same goal. As written in section §2.2 (page 23), three main performance criteria predefine human-machine system organization: — the reliability resulting after debates, — the adaptability according to competence resulting from adjustment of different *capabilities*, and — the adaptability according to *capacity* resulting from adjustment of workload and energy consumption. Sheridan proposes improving reliability by task sharing or task trading (Thomas B. Sheridan, 1992). With task trading, human agent or artificial agent controls a task (not both), and either of them can transfer the control to the other. With task sharing, human agent and artificial agent can control different aspects of the process in parallel at the same time. Agents can debate the solution because, in this case, they have the same KH.
KH must also be the same when the adaptability is according to capacity. This case appears when the capacity of an agent is not sufficient to perform a task. The relief type of task sharing proposed by Sheridan has such an objective by avoiding human operator overload (Thomas B. Sheridan, 1992). This approach has been used in robotics (Kiener & Stryk, 2010). They distinguish four robot's abilities similar to the ones proposed by Parasuraman, and they define a utility value for each task according to required abilities and cost for completing the task, the preceding and/or the following task and current state of execution (assigned/not assigned or executable/being executed or executed/solved). A robot manages a task if it has the most important utility value for this task. The adaptability is important when the KH of one agent is not sufficient to perform a task. The "extension type of task sharing" proposes such an extension of the capabilities of the human-machine system (Thomas B. Sheridan, 1992).

It is now time to specify the difference between *adaptive* and *adaptable* system. An adaptive system determines and executes the necessary adaptation. For an adaptable system, the human is in charge (Miller & Parasuraman, 2007). Studies usually deal with adaptable systems, even if they attempt to justify, in some specific situations, the interest of providing artificial agent with more authority. Well-known works about adaptability used different notions, such as "sliding autonomy" (Simmons et al., 2007), "adaptive automation" (Cosenzo et al., 2010b; Kaber, Perry, Segall, McClernon, & Prinzel, 2006; D. D. Woods et al., 2004), "adjustable autonomy" (Baker & Yanco, 2004b; Goodrich et al., 2001; Zieba, Polet, & Vanderhaegen, 2011), "situation adaptive autonomy" (T. Inagaki, 2006) or "mixed-autonomy" (Barber, Davis, Nicholson, Finkelstein, & Chen, 2008; Finzi & Orlandini, 2005; Goodrich et al., 2001).

These studies usually tried to respond to a specific objective, in most cases from a specific domain. We would like to propose a generic procedure to examine and implement adaptive and adaptable human-machine system (L. Habib, Pacaux-Lemoine, & Millot, 2017b). This study has been conducted as part of the **PhD of Lydia Habib** (Lydia Habib, 2019). The two next graphs illustrate such a generic procedure based only on the ability for an agent to complete or not (1 or 0) the four functions of the KH and the four functions of the KHC, presented in Fig. 24 and Fig. 25 respectively. It is of course very restrictive to decide if an agent can or cannot complete a function, especially to describe human activity, but it is the first step for applying this method. We will extend the number of possibilities to share functions in order to achieve more adaptability. A fuzzy method, for example, could suit this need. Therefore, using this simple binary approach, three cases are possible, [1 0] only human has the KH or the KHC, [0 1] only machine has the KH or the KHC, and [1 1] both human and machine have the KH and the KHC.

At the first function of the KH, information gathering, the agent who has the KH gets information from the process (cf. Fig. 24). Then if the agent has the ability to cooperate, the agent may interact with the other agent. Cooperation is represented by the blue diamond on both trees (Fig. 24 and Fig. 25). However, the other agent can also trigger the cooperation because, for example, this agent has to make a decision without being able to gather information. The Common Work Space helps manage such cooperation and shares information by providing common language and tools to organize exchanges between agents.
<figure><img src="image_23.png" /><figcaption>Fig. 24 Human-Machine function allocation regarding KH, adapted from (L. Habib et al., 2017b)</figcaption></figure>

<figure><img src="image_24.png" /><figcaption>Fig. 25 Human-Machine function allocation regarding KHC, adapted from (L. Habib et al., 2017b)</figcaption></figure>
The agent who initiates cooperation gathers information about the other agent, or can ask for information if the other agent can answer (cf. Fig. 25). The result of this function is the input of the next KHC function, the interference detection. In this function, the agent who has the interference detection ability detects potential interferences between the results of the KH function of each one. The result of this function supplies the interference management function performed by one of the agents or the both. When both agents manage the interferences, they impose, accept or negotiate their result in the next step dealing with the function allocation. This KHC function decides the allocation of the KH function according to the previous result. After this step, the next KH functions follow the same procedure, until the last KH function.

We had the opportunity to work with the University College of London and Tom Carlson, and we aimed to study how Human-Machine Cooperation principles may help design a cooperative Brain-Computer Interface to control robot (cf. §7.4.4; p.143). Using the approach presented below, we extracted four levels of cooperation between a robot and a human who can only use their “thoughts” to send commands.

<figure><img src="image_25.png" /><figcaption>Fig. 26 Emulated-haptic feedback experiments: KH of Human and robot (violet and green rectangles respectively) and their KHC (light blue and green rectangles). If the agent has the ability to perform a sub-function, “1” is written in the rectangle.</figcaption></figure>

In the above explanation, one piece of information or one event triggers the procedure with the sequence of KH and KHC functions. When agents have no expertise or experience to control a process, or to cooperate with the other agent, usually they strictly observe such a procedure. Nevertheless, most of time some shortcuts can accelerate decisions. It is true for the KH, as described in Fig. 5 (p. 21), but it is also true for the KHC. Therefore, we proposed an extension of Rasmussen model in order to identify the experience and the expertise of an agent to cooperate with other agents (L. Habib et al., 2017b). Three types of cooperative behavior respond to this objective. In the *KHC skill-based behavior*, human and machine are accustomed to working together. They have an adequate model of each other and they can anticipate what the intentions and actions of the other could be, cooperation is implicit (no need to ask, to justify, to negotiate), integrative and/or augmentative (no need to debate). In the *KHC rule-based behavior*, human and machine need to have a little bit more information about the other. They have a model of each other but they need information regarding
how the model of each other could be instantiated in a situation they have not yet come across. Cooperation is more explicit and needs some exchanges to debate the best instantiation. In the *KCH knowledge-based*, human and machine have to build or to improve the model they have of each other. Cooperation is again more explicit and needs negotiations to plan task sharing, task trading and criteria for such decisions.

<figure><img src="image_26.png" /><figcaption>Fig. 27 Extension of Rasmussen model to cooperative behavior (L. Habib et al., 2017b)</figcaption></figure>

Training usually concerns KH functions, *but the training to work with colleagues or with new assistance systems is also a key point of good cooperation as we saw in the projects in Air Traffic Control, with Fighter aircraft crew and with Thalès/DGA staff* (cf. §7.1, §7.3 and §7.4.2). Indeed, as we saw in these applications, humans usually work with same teammates. They know each other very well because they spend time together, they solve problems together, and they know strengths, weaknesses, habits, and many other things about the others. They demonstrate skill-based cooperative behavior in most of the task. When several cooperative tasks are skill-based, the cooperative agents may manage additional complex and difficult cooperative tasks without affecting their performance. Team training issues have not been studied enough. Salas et al. made some proposals about what, when and how to train (Salas et al., 1995), and we will discuss this aspect in the next chapter about methodology.

All presentations and explanations until now, stemming from the literature and models we proposed, have focused on interaction and cooperation between two agents, except when dealing with the assisted explicit mode of cooperation. Indeed, in this mode, we integrate agents from an upper decisional level. In fact, agents are rarely the only two to control process. They control a part of an overall process in which several other agents are involved with similar or complementary objectives. With the increase of process complexity, but at the same time the increase of machine abilities, the gathering of all human-machine systems extends towards socio-cyber-physical-systems. Therefore, we need to consider most humans and machines involved at different decisional levels of the overall process. We believe that the model of cooperative agent and the model of cooperative agents' interaction may help identify individual and cooperative activities of humans and machines in order to reach cooperative socio-cyber-physical-systems. This is the goal of the next section. It deals with cooperation between several levels of activity, or several levels of abstraction, sometimes defined according to hierarchical levels.
## 3.4 Multi-level cooperation

Process control and supervision represent different types of activity, from the more reactive and short-term part close to the commands of the process, to the more planned and long-term part close to the planning level. In the case of automation and especially in the case of the human-machine system, three levels are mainly used (M.-P. Lemoine et al., 1996; Michon, 1985). The planning level, also called the strategic level, has the goal of defining the plan of the activity. The tactical level's goal is to apply the plan by triggering the defined tasks or to update the plan according to new events, changes in the resources or new objectives. The operational level has the objective of controlling the process. One or several agents may manage each level, and an agent may take part of one or several levels. Therefore, it appears important to define such a cooperation that we called multi-level cooperation (M.-P. Lemoine et al., 1996). Indeed, cooperation between levels concerns interactions about KH or KHC of human or machine. Objectives, decisions and orders, managed by the KH of agents, evolve from higher decisional levels (planning or tactical) to lower levels close to command/control of process. In reverse, events and information update stem from lower levels and are sent to upper levels. Regarding the KHC of agents, one main difference between levels is the time the human or the machine can allocate to cooperative task. At the operational level, time pressure to perform individual task is sometimes too high to allow spending time to cooperate. At the operational level, agents usually cooperate for action implementation with traded or shared control. At this level, cooperation is generally prepared because there is no time to debate the best solution. In that case, the human or the machine has the authority to decide action according to the complexity or the emergency of the situation (Marie Pierre Pacaux-Lemoine, 2014). We had paid particular attention to the shared control concept, and we decided to extend the model of cooperation used for two agents to other agents involved in other levels (Marie-Pierre Pacaux-Lemoine & Itoh, 2015). Therefore, cooperation between levels can have several forms, and the CWS used in each level can take into account the CWS of other levels (cf. Fig. 28).

<figure><img src="image_27.png" /><figcaption>Fig. 28 Multi-level cooperation, adapted from (Marie-Pierre Pacaux-Lemoine & Itoh, 2015)</figcaption></figure>
A useful project to exemplify multi-level cooperation with several humans and technical agents is the project conducted in crisis management (cf. §7.4.3, p.140). The study of firefighters' activity and crisis unit management highlighted three main levels of activity (cf. Fig. 29). The crisis unit of a firefighter department or the crisis unit of a city (with a sub-prefect as decision-maker) is at the strategic level and decides missions and allocates resources. The command post vehicle with several officers, usually close to the hostile environment, is at the tactical level. They received commands from the strategical level and they provide contextual and adapted information to the operational level where several other firefighters are involved. Each level is supported by technical agents, i.e. decision support systems at all the levels according to the needs (raw to precise information, time span...), and robots at the operation level. Therefore, cooperation is supported at each level, not only by its own CWS, but also between levels with an adaptation of CWS, like a kind of zoom in to focus on the operational level or zoom out for a broader view at the strategic level.

<figure><img src="image_28.png" /><figcaption>Fig. 29 Multi-level cooperation in crisis management</figcaption></figure>

Several authors have already studied cooperative activities implemented at each level and the model of cooperation we propose may implement these multi-level cooperative activities.

*Cooperation at strategic level* concerns “cooperation in plan” (Loiselet & Hoc, 2001), “Plan based direction” (D. D. Woods et al., 2004), “planning control” (Simmons et al., 2007), “mission management control loop” (M L Cummings & Mitchell, 2007). Cooperative activities may concern the planning of individual tasks as well as the planning of cooperative tasks such like meeting points. In the “planning-based interaction of the mixed-initiative approach” proposed by Finzi and Orlandini, cooperative activities are planned in order to limit interaction between levels (Finzi & Orlandini, 2005).

*Cooperation at tactical level* concerns the cooperation that consists in updating and executing the plan. In the “cooperation-based interaction of the mixed-initiative approach” proposed by Finzi and Orlandini, humans can add new tasks to the plan and the machine validates. Goodrich et al. proposed a similar approach. It concerned the mode of autonomy with “goal-biased autonomy”, for which the geographic zone and a goal are defined by human (Goodrich et al., 2001). In the “Direction through commanders’ intent”, proposed by Woods et al., the cooperative objective is to adapt a plan for disruptions and opportunities (D. D. Woods et al., 2004). “Executive control” (Simmons et al., 2007), “navigation control loop” (M L Cummings & Mitchell, 2007) also refer to tactical level. Allen and Ferguson propose an “automated plan reasoning system” to support collaborative planning in the case of evacuation-planning domain (Allen & Ferguson, 2002). They address tactical levels with three possible problem-solving operations: suggesting a new goal, extending an existing solution and modifying a parameter of a solution.
*Cooperation at operational level* concerns “cooperation in action” (Loiselet & Hoc, 2001). At this level, agents are mainly reactive and they have no time to negotiate, but they must adjust their respective actions. In the “operator-based interaction of the mixed-initiative approach” proposed by Finzi and Orlandini, humans have to trigger interaction frequently (Finzi & Orlandini, 2005). A similar approach, proposed by Goodrich et al. with “the intelligent teleoperation” (safe mode, shared control/allocation of control), promotes continuing interaction between human and machine (Goodrich et al., 2001). Another way to control operational level tasks is to use constraints. With “Constraint based direction” proposed by Woods et al. (D. D. Woods et al., 2004), or waypoints and heuristics proposed by Goodrich et al., attractive or repulsive potential fields and constraints are used to control individual tasks by an indirect cooperation. Therefore, this level relates more behavioral control (Simmons et al., 2007) and motion control loop (M L Cummings & Mitchell, 2007).

These studies highlight several possibilities of cooperation at each level but there is little information dealing with what is cooperation between these levels. In other words, we could write how a human-in-the-loop can cooperate with a human-on-the-loop (Schirner, Erdogmus, Chowdhury, & Padir, 2013), but what about cooperation between agents-in-the-loop and agents-on-the-loop? Tactical and strategic levels implement more and more assistance systems, but problems start to appear, like the loss of flexibility at operational level when the assistance system at tactical level imposes rigid organization. Examples of this problem already come from different domains like in the railway industry (Balfe et al., 2015), and problems of interaction between levels are also already underlined by the military domain in which implementation of new technology is rapid but risky. The UK Ministry of Defense Crown published a report about human-machine teaming that warns about such risks. They wrote, “The interdependencies between technologies, tactics and strategy are likely to be complex. Our understanding must be improved through research, experimentation and training” (UK Ministry of Defence Crown, 2018). That was the objective when we studied links between levels gathering the concepts of shared control, shared and cooperative guidance and control, and human-machine cooperation (F. Flemisch, Abbink, Itoh, Pacaux-Lemoine, & Weßel, 2016) (cf. Fig. 30).

Shared control seems to focus on the common task or function on operational level, while cooperation includes a way of taking increasingly into account the other agent and the other levels. In line with the above, as well as knowing what the other is doing, cooperation creates the possibility of having a model of the partner in order to know how they can cooperate. A cooperative agent (that has KHC) can gather information about the other, analyze this information in order to make a decision about their cooperation. Such activity can go directly back again to the operational level. Nevertheless, tactical and strategic levels should prepare such cooperative activity that may even be prepared and maintained with communication that is not concerned about operational, tactical or strategic levels at all. An additional level should supervise and manage cooperative activity. We called this the cooperational level and it is transversal to the three other cooperation levels.
<figure><img src="image_29.png" /><figcaption>Fig. 30 The relationship between shared control, shared and cooperative guidance and control and human-machine cooperation (adapted from (F. Flemisch, Abbink, Itoh, Pacaux-Lemoine, & Weßel, 2019))</figcaption></figure>

Nevertheless, as levels of cooperation were already defined as a combination of KH and KHC, the cooperation between levels presented in this part is called the layers of cooperation (Marie Pierre Pacaux-Lemoine & Flemisch, 2018). Using the model of cooperative agents’ interaction, Fig. 31 shows one main link between layers, the Common Work Space.

<figure><img src="image_30.png" /><figcaption>Fig. 31 Layers of cooperation</figcaption></figure>
This model does not detail interaction between layers, but underlines all the possible interactions. The three CWSs, one at each level, represent in fact the same environment but with different points of view. The more activity deals with high decisional level (strategic level), the more general the information is, the wider the geographical view is. The more activity deals with short-term control level, the more information focus on specific detailed view and fast feedback.

This chapter presents the state of the art in human-machine system augmented with our contributions on Human-machine Cooperation, with a focus on how to address human factors, how to combine such human characteristics with machines that have more and more capabilities. We try to highlight benefits and risks to put together smart technologies and humans in the same environment with common goals. Several criteria, general rules and examples provides clues for designers to create cooperative assistance systems, and for managers to be careful to organize human-machine interaction in a safe, acceptable and efficient way. The next chapter aims to provide designers with tools that can help reach these objectives.
**Chapter 4. Design and evaluation methods**

"What we observe is not nature in itself but nature exposed to our method of questioning," *Heisenberg* (1958)

Indeed, in the description of atomic phenomena, the quantum postulate presents us with the task of developing a 'complementarity' theory the consistency of which can be judged only by weighing the possibilities of definition and observation. *Bohr N* (1928). '*The Quantum Postulate and the Recent Development of Atomic Theory*'. *Nature*. 121 (3050): 580-590.
Researchers, engineers must evaluate every new machine they design and every model that supports the design, not only the technical requirements, but also their usefulness regarding their integration in human life. That is particularly true when designers tackle human-machine systems, for all the reasons mentioned in the previous chapter. Moreover, sometimes companies and industries re-use existing machines with different objectives, or different humans, or different environments, but even if the difference is very small, it can disturb the overall human-machine system and may lead to catastrophes. Therefore, we need to use and improve tools that can support design and evaluation. These tools mainly concern experimental conditions, measurements and analysis that the designer is able to use to conduct progressive and ecological experiments. Experiment can begin from the very early stages with brainstorming and systems prototyping, to the final stages with integration in real human activities.

Methodology is not the focus of my research, but a tool I need every time I have to design and evaluate human-machine systems as a research engineer. Therefore, the chapter starts with a short state of the art about human-machine system methods, then it presents a method defined and evaluated in LAMIH. Its use during a large number of projects allowed for improving each step and tool, which are described in the following sections. Though it is not at the center of my research, the need to improve the methodology has nevertheless been made clear, thanks to lectures I have done and discussions with other researchers in human-machine system domain, and so the final section deals with such objectives.

## 4.1 Some methods from the literature

Methods provide useful guidelines to manage a project, from the definition of a hypothesis based on the study of objectives, to the analysis of data and results about hypothesis, and more. Deep data analysis may also lead designers to identify the justifications of the results. Several user-centered design methods aim to evaluate the ergonomics of systems. They provide knowledge about the design and evaluation of interfaces, e.g. usability and models of human-machine interaction based on cognitive behavior. Nielsen defines the usability as a sub-part of acceptability, which is the extent to which a product matches customers' expectations (Nielsen, 1993). Gould and Lewis describe usability engineering as a general approach with three strategies: taking into account of users and their tasks early on, empirical measures and iterated design (Gould & Lewis, 1985). This approach includes user aspects in most of steps of project management and defines several norms. Nevertheless, other methods exist, originating from different domains, such as the method presented in social acceptability. Brangier et al. propose a comparison between social and ergonomic approaches (Brangier, Hammes-Adelé, & Bastien, 2010). The authors underline the interest in exploiting their complementarities. This is also the method proposed by Millot, the so-called method in "U" (P. Millot, 1995). This method aims to propose an easier way to design and evaluate system than the V-cycle proposed by Royce (Royce, 1970). V-cycle, stemming from systems engineering, bases the method on the different stages of the cascade model. Nevertheless, it is very difficult to use to model complex environments, such as dynamic environments, the
main application of our research. Moreover, it does not take into account humans and their interactions with
assistance systems. The method in “U” has this objective, as well as balancing human involvement with
machine involvement in the control of the process in a cooperative manner.

Firstly used in telerobotic domain (Patrick Millot & Roussillon, 1991) and in Air Traffic Control (Patrick Millot & Debernard, 1993), other domains applied this method in automotive domain to design and evaluate driving assistance systems (Marie-Pierre Pacaux-Lemoine & Crévits, 2010; Tricot, Rajaonah, Popieul, & Millot, 2006). This method was also the design guide for studies in fighter aircraft domain, robotics as presented in the appendix dealing with projects done for more than 20 years, and the several domains of application.

Fig. 32 presents a new version of the method. Indeed, details about human and machine representations, their interactions through the Common Work Space, and the multi-level aspects of cooperation enrich the previous version.

<figure><img src="image_31.png" /><figcaption>Fig. 32 Method in “U”, adapted from (P. Millot, 1995)</figcaption></figure>

In this method, a descending phase designs Human(s)-machine(s) system and an ascending phase evaluates
it.

**4.2 Descending phase for design**

All the steps of the descending phase use the models of cooperative agents, organizations and cooperation presented in the previous chapter. First, designers need to know the objectives of the human-machine system they have to design (cf. Fig. 32-1). Objectives usually describe the increase of performance, but performance
is a general word that hides many things. It could be the increase of the number of planes air traffic controllers can managed, the decrease of death on the road with driving assistance systems, the increase of the number of products generated by manufacturing systems. However, performance can also mean better comfort for humans in their activities, such as for disable people. Therefore, designers must correctly identify and understand objectives with discussions with managers and experts, but it is sometimes useful to have also discussions with the future end users, especially for the next step of the method regarding the analysis of the process.

Most of the time, process and human activity dealing with the control of this process already exist. Therefore, models of current organization, models of prescribed tasks and functions, and detailed description of current process might inform designers (cf. Fig. 32-0). Less often, only human activity exists and designers have to imagine and create systems to automate parts of human tasks. Therefore, designers must identify the limits of their intervention, defined by a set of human tasks and sub-tasks they have the agreement for modification. Safety and security criteria usually define such limits. Designers study human and technical resources (cf. Fig. 32-3), and may conduct an analysis of the current process with points of view that are sometimes different from the point of view of managers (cf. Fig. 32-2). The objective here is not to avoid the influence of managers, but to have neutral approach to offer novel and original perspectives. Ergonomic and social fields help analyze human resources, extracting cognitive and physical constraints linked to activities, as well as constraints from the social organization. The method highlights the interest in extracting the levels of activity and layers of cooperation from the organization (cf. Fig. 32-4) and the models described in the section about multi-level cooperation (§3.4). The designer has to then identify tasks and sub-tasks of humans and machines at each level (operational, tactical, strategic), as well as tasks or information transferred from one level to another (cf. Fig. 32-5). At each level, the designer suggests modifications of current tasks or creations of new tasks for humans and for machines (cf. Fig. 32-6). It is an iterative procedure analyzing activity level by level, and complementarity between levels (cf. Fig. 32-7). Tasks or functions deal with individual and cooperative activities. Based on the models presented in the previous chapter (§3.2), KH and KHC functions may be analyzed through a grid to identify and define tasks and sub-tasks (cf. Fig. 33).

This grid has been used within the framework of the COCOVEA project (cf. §7.2.1, p.125). The objective was to assist car drivers with different types of Driving Assistance Systems (DAS) on the three levels of activity (operational, tactical and strategic levels) (Marie-Pierre Pacaux-Lemoine, Simon, & Popieul, 2015). The grid points out the shared and/or traded functions between human and machine (yellow part on the grid), the way human and machine can evaluate the other (respectively blue and green part on the grid) and information that should be present in the Common Work Space (pink part on the grid). In this example dealing with the automotive domain, where the human is the same for the three levels of activity (the driver), we propose to merge functions of the levels to simplify the presentation (strategic level in bold, tactical level in italics and operational level in regular font). The completion of such a grid is a difficult exercise for the designer, because each square of the grid requires an answer. For example, the question for the blue cell in the top-left of the DAS's KHC is: How does DAS able to gather information on the driver's ability to gather information for the driving task? Is it possible to use a model of this driver (their habits for example) and their current activity?
<table><caption>Fig. 33 Grid to identify current and future human (D: driver) and machine (S: ADAS) tasks according to their KH and KHC: example in automotive domain (KH: IG: Information Gathering, IA: Information Analysis, DS: Decision Making, AI: Action Implementation; KHC: IG: Information Gathering on the other, ID: Interference Detection, IM: Interference management, FA: Function Allocation) (Marie-Pierre Pacaux-Lemoine et al., 2015)</caption><thead><tr><th colspan="2"></th><th colspan="4">Driver</th><th colspan="4">Know-How</th><th colspan="4">Know-How-to-Cooperate</th></tr><tr><th colspan="2"></th><th>Information Gathering</th><th>Information Analysis</th><th>Decision Selection</th><th>Action Implementation</th><th>Information Gathering on the other</th><th>Interference Detection</th><th>Interference Management</th><th>Function Allocation</th><th>Information Gathering</th><th>Interference Detection</th><th>Interference Management</th><th>Function Allocation</th></tr></thead><tbody><tr><td rowspan="4">ADAS</td><td rowspan="2">Know-How</td><td>IG</td><td>D and/or S: Signs, position, road and traffic, lane, following vehicle</td><td>D: Analysis of situation according to their perception</td><td>D: Trajectory selection according to their analysis</td><td>D: Lateral and longitudinal control</td><td>D: Sensors states</td><td>D: Sensors are broken, unsuitable or improperly adjusted</td><td>D: Modification of sensors</td><td>D: Manual control</td><td>D: Sensors states</td><td>D: Sensors are broken, unsuitable or improperly adjusted</td><td>D: Modification of sensors</td><td>D: Manual control</td></tr><tr><td>IA</td><td>S: Analysis of situation according to available sensors</td><td>D and/or S: Analysis of the driving situation regarding itinerary, lane change, trajectory</td><td>D: Trajectory selection according to ADAS analysis</td><td>D: Lateral and longitudinal control</td><td>D: Perception of ADAS analysis</td><td>D: ADAS analysis procedure is wrong or not the best one</td><td>D: Modification of drivers' preferences</td><td>D: Accept or refuse</td><td>D: Perception of ADAS analysis</td><td>D: ADAS analysis procedure is wrong or not the best one</td><td>D: Modification of drivers' preferences</td><td>D: Accept or refuse</td></tr><tr><td>DS</td><td>S: Trajectory selection according to previous analysis</td><td>S: Trajectory selection according to driver analysis</td><td>D and/or S: Choice of speed, road, lane, lateral position in the lane</td><td>D: Lateral and longitudinal control according to ADAS decision making</td><td>D: Perception of ADAS decision making</td><td>D: ADAS decision making is wrong or not the best one</td><td>D: Modification of drivers' preferences, correction of decision</td><td>D: Accept, refuse or modify</td><td>D: Perception of ADAS decision making</td><td>D: ADAS decision making is wrong or not the best one</td><td>D: Modification of drivers' preferences, correction of decision</td><td>D: Accept, refuse or modify</td></tr><tr><td>AI</td><td>S: Lateral and/or longitudinal control</td><td>S: Lateral and/or longitudinal control</td><td>S: Lateral and/or longitudinal control according to driver choices</td><td>D and/or S: Lateral and longitudinal control</td><td>D: Perception of ADAS actions</td><td>D: ADAS actions are wrong or not the best one</td><td>D: Modification of drivers' preferences, correction of action</td><td>D: Accept, refuse or modify</td><td>D: Perception of ADAS actions</td><td>D: ADAS actions are wrong or not the best one</td><td>D: Modification of drivers' preferences, correction of action</td><td>D: Accept, refuse or modify</td></tr><tr><td rowspan="4">Know-How-to-Cooperate</td><td rowspan="2">IG</td><td>S: Eyes, head, face, hands perception, steering wheel, pedals state</td><td>S: Eyes, head, hands, steering wheel, pedals movement</td><td>S: Eyes, head, hands, steering wheel, pedals movement</td><td>S: Steering wheel, pedals, HMI use, vehicule behavior</td><td>D & S perceptions are different</td><td colspan="4">ADAS justification of sensors functioning...</td></tr><tr><td>ID</td><td>S: Eyes closed, hands off the steering wheel</td><td>S: Driver is distracted, drowsy</td><td>S: Driver intention is wrong</td><td>S: Driver action is wrong, current driving style</td><td>D & S analysis are different</td><td colspan="4">ADAS justification of its analysis...</td></tr><tr><td>IM</td><td>S: Alert message, Wakeup procedure</td><td>S: Alert message, Wakeup procedure</td><td>S: Visual, audio alert message</td><td>S: Haptic alert</td><td>Drivers justification of their presence</td><td>Driver justification of his/her analysis</td><td>D & S decision making is different</td><td>ADAS justification of its decision making...</td><td>Drivers justification of their decision making</td><td>D & S action implementations are different</td><td>D & S action implementations are different</td></tr><tr><td>FA</td><td>S: Vehicle stopped in the lane or on the hard shoulder</td><td>S: Return to the manual control or lower LOA</td><td>S: Haptic alert</td><td>S: Collision avoidance ?</td><td>Drivers justification of their presence</td><td>Driver justification of his/her analysis</td><td>D & S decision making is different</td><td>ADAS justification of its decision making...</td><td>Drivers justification of their decision making</td><td>D & S action implementations are different</td><td>D & S action implementations are different</td></tr></tbody><tfoot><tr><td colspan="12" style="text-align: left; font-weight: bold;">Fig. 33 Grid to identify current and future human (D: driver) and machine (S: ADAS) tasks according to their KH and KHC: example in automotive domain (KH: IG: Information Gathering, IA: Information Analysis, DS: Decision Making, AI: Action Implementation; KHC: IG: Information Gathering on the other, ID: Interference Detection, IM: Interference management, FA: Function Allocation) (Marie-Pierre Pacaux-Lemoine et al., 2015)</td></tr></tfoot></table>

After checking complementarity between levels, the designer proceeds to the programming of machine functions, the programming of the interface (visual, sound, haptic...) and the training of future users (cf. Fig. 32-8). Definition and organization of human and machine functions are important in the design of a human-machine system, but the interface is another crucial part in the success of such a system. Nevertheless, we will not focus of this aspect, but use dedicated literature and especially that which aims to choose or mix different sensory perceptions (e.g. Baltzer & Flemisch, 2019)).

At the end of the descending phase, the ascending phase takes over the work to evaluate the new human(s)-machine(s) system.
## 4.3 Ascending phase for evaluation

The ascending phase starts when the designer has identified a minimum of functions. In sociological approaches, experimenters may ask to people (in the street or on the internet) to imagine a situation and to answer to questions about the interest of a machine briefly or in detail. The advantage of this approach is that experimenters can contact many people. Nevertheless, the disadvantage is the heterogeneity of population, the strong hypothesis that people correctly imagine the situation, and the lack of details regarding the functions (Bordel et al., 2014). Another difficulty is the variety of possible answers. The synthesis work after such a survey is enormous, even if sometimes it could be fruitful as we were able to observe in the SARI project about the design of on-road driving assistance systems (cf. §7.2.2, p.127). In the engineering approach, even if we can take advantage of results from sociological approaches, we usually start the evaluation phase when the human-machine system has a minimum number of implemented functions.

The objectives of the evaluation must answer the questions raised by the stated hypothesis. What do we want to highlight with the evaluations? How can the proposed new human-machine system be assessed and prove its usefulness? Experimental procedures and protocols answer such questions (cf. Fig. 32-9).

### 4.3.1 Experimental protocol

Experimenters define contexts and scenarios, step 9 of the design and evaluation method (cf. Fig. 32, p.64). Context deals with whole or part of the process in which the human-machine system takes part. A starting state of the process and objectives defines a global shape of user activities. Scenarios aim to encourage users to complete some tasks by asking them (and the assistance systems) to reach goals despite triggering unexpected events. The experimental protocol may involve building several scenarios to allow experimenters to compare different experimental conditions, i.e. different configurations or propositions of human-machine systems, like the comparison of several levels of automation.

#### 4.3.1.1 Scenario and experimental design

Scenarios must be different to prevent users from recognizing a situation, from making the same decisions and from interacting with the assistance system in the same way; however, scenarios require similar tasks to make the data comparable. The duration of scenario depends on the tasks and can last a few minutes for a reconnaissance mission in a fighter aircraft or for robotics activities, to several hours when the objective is to evaluate driving vigilance in automotive or industrial process supervision. Scenarios and experimental conditions are, most of the time, counterbalanced using a Graeco-Latin square experimental design to reduce order and learning effects. Such an organization of experiments defines the experimental design, which aims to reduce the impact of evaluating one type of assistance system before another one, and the specificity of a scenario according to the type of assistance system. A condition without new assistance systems defines the control condition. It makes it possible to evaluate potential benefits of the new human-machine system.

Experimental design involves defining experimental procedures, i.e. the various steps experimenters interact with participants taking part in experiments. Firstly, experimenters present scientific objectives and then the experimental platform. Secondly, participants may have to learn process if they are novices in the domain.
They may have to identify context and types of scenario (requested tasks), usually without using new
assistance systems. Therefore, they may have to learn how the assistance system functions with a formal
presentation, and then use it in a dedicated scenario to train themselves. Training scenarios are different from
experimental scenarios, but with similar difficulty.

During the project with Dassault Aviation and DGA, we had the opportunity to conduct very sophisticated experiments (cf. §7.3, p.133). Fig. 34 presents an example of a task that participants had to complete for the experiments. This scenario was one of the four they had to experiment. Each experimental session was composed of five stages: mission-planning, preparation of the assistance systems (each crew member adapted the KH and KHC of the assistance systems according to their individual and cooperative needs), briefing of mission, mission completion, and debriefing (relative to the cooperative activity and to the assessment of the assistance systems). One member of the crew managed the briefing, explaining all information of the mission file following a precise order. The partner listened and spoke up if he had a different interpretation of information or requested tasks. The briefing involved the allocation of tasks between both members. After the briefing, the crew fulfilled the mission. They had two objectives of destroying, or one objective of destroying and another one recognizing, depending on the scenario.

<figure><img src="image_32.png" /><figcaption>Fig. 34 Preparation of mission in the project with DGA and Dassault Aviation (cf. details in §7.3)</figcaption></figure>

4.3.1.2 Simulation of environment

Several kinds of simulators support human-machine systems studies. The level of realism can range from very low level of realism, where participants have only to perform very simple tasks but usually with many repetitions, to high level of realism where simulation only concerns environment but command and control systems are real ones. We work on these different kinds of simulators and study the impact they have on participants, and so on results.

Micro-worlds are ones of the simple simulators but they facilitate the control of many parameters of the experimental protocol (e.g. (Sauer, Wastell, Hockey, Crawshaw, & Downing, 2003)). They usually consist of one piece of software and experiments conducted on one computer with one screen. The simulation of a pasteurization plant (J. Lee & Moray, 1992), the simulation of forest firefighters (J. M. Hoc & Amalberti, 2007), the simulation of a war environment (Barber et al., 2008) are some of the well-known simulations. These kinds of simulators focus on cognitive activities and most of the time on tactical or planning level of activity. At the start of our studies on robotics, we used the Mixed Initiative eXperimental testbed (cf. Fig. 35). This environment allows the impact of levels of automation on participants’ behavior to be studied. Experimenters define the number of vehicles that the human has to control when they are moving in a risky
environment, and their level of autonomy. Participants can plan vehicles trajectory and then supervise the right completion of such trajectories despite unexpected events, while they do reconnaissance (cf. §7.4.1, p.135).

This experimental environment helps to have a good level of control in the experiments but participants, who are usually novices like students (unless experiments take place in an army context), need to learn the activity as well as how to use the simulator.

<figure><img src="image_33.png" /><figcaption>Fig. 35 MIX (The Mixed Initiative eXperimental testbed)</figcaption></figure>

In a similar context, we were able to work on a more realistic micro-world and with professional participants (cf. §7.3, p.133). With the support of Dassault Aviation, we designed a flight simulator to study cooperation between the pilot and the weapon system officer, as well as cooperation between the pilot and another officer who would be in another plane and would have more planning functions. Several types of systems to support cooperation were studied. In this experiment, tasks were the usual ones that officers learned from training and experience, but we simulated the environment. Even if such officers were used to training themselves with high fidelity fighter aircraft simulator, they were fully involved in our experiments. We were working on future displays of Rafale and they were pleased to participate to such experiments (M P Pacaux-Lemoine & Loiselet, 2002). The place where we conducted experiment is another important point that made these experiments successful. Indeed, we stayed for several weeks in air bases close to officers’ operating and break rooms. We were at their disposal to explain the experiments and to find dates to conduct them.

<figure><img src="image_34.png" /><figcaption>Fig. 36 Workstations of pilot and weapon system officer</figcaption></figure>
The more advanced simulators we used were the car driving simulators of University Polytechnique Hauts-de-France (UPHF) (cf. §7.2.1, p.125), IFSTTAR (7.2.2, p.127) and Renault (cf. §7.2.4, p.131). Participants, whether they are beginners or more experienced drivers, understand the driving tasks and can easily complete them, especially in a reference situation without new devices or assistance systems. With such simulators, participants drive the car as if it is a rented car (unless they have the same car). Therefore, the main difference is the driving environment. Nevertheless, it is now possible to design very realistic environments based on real topographic layout, lane widths, road markings, sight distances, and other geometrical characteristics such as trees, houses of the field.

<figure><img src="image_35.png" /><figcaption>Fig. 37 Driving simulators of IFSTTAR (a static one on the left), UPHF-LAMIH (a mobile one) and Renault (mobile one on the right)</figcaption></figure>

In the SARI project (§7.2.2, p.127), we were able to find similar driving behaviors (risky position in a lane on a vertical curve) between participants who drove with one of the two kinds of driving simulator (UPHF and IFSTTAR), and drivers who used the real road but were not aware that their driving data was being recorded (Auberlet, Pacaux, Anceaux, Plainchault, & Rosey, 2010; Auberlet et al., 2012).

A simulated environment proves to be a very useful tool for experimenters to control experiments. Nevertheless, participants may sometimes display unnatural behavior, since they are aware their activities are being recorded and do their best to meet experimenters' expectations. When we conducted experiments during the ST2 project (cf. §7.2.3, p.129), with the objective of studying drivers' behavior during risky situations and their ability to anticipate such situations, we observed that participants needed about 20 minutes to find their normal, comfortable driving position (Hault-Dubrulle, Robache, Drazétic, & Morvan, 2009; Hault-Dubrulle, Robache, Pacaux, & Morvan, 2011), even if the car was a real one (cf. Fig. 38). In this experiment, several types of data such as biomechanical, physiological, psychological data, drivers' actions, all synchronized, support deep analysis (Robache, 2017).

<figure><img src="image_36.png" /><figcaption>Fig. 38 Car driving simulator of the University of Valenciennes (static version) and digital anthropometric model</figcaption></figure>
For each stage of the experiments, designers and experimenters prepare data records. Data can be objective, subjective, qualitative, quantitative, raw, interpreted, and coded, depending on experimental design and experimental platform. Data record definition and control are an important step for experimenters since it is a necessary condition for obtaining useable data to generate significant results. The next section deals with this step.

## 4.3.2 Data records

Data is a more frequently cited word nowadays, as in, for example, big data, database, data mining, data cloud, but experimenters must find the right compromise between recording many data and spending a lot of time cleaning, selecting and analyzing data, or selecting the most appropriate data and the way to analyze them. The following sections present some recommendations to reach such good compromise.

### 4.3.2.1 Online data records in a simulated environment

Online data records and their analysis are at the core of the understanding of human-machine system activities. Recorded data must highlight the states of the human, the assistance system and the process, as well as transitions between such states. Designers can program data records directly on the experimental platform.

All commands sent through human-machine interfaces and information updates in the Common Work Space constitute precious information for experimenters to understand process situations and agents' decision making regarding such situations. Precoding data during execution of simulation and experiments may support analysis. During the experiments we conducted in most projects, we used the model of the cooperative agent to code agents' individual and cooperative activities. An example of coding and use of the coding is presented in section §4.3.4. Even if it is difficult to anticipate all human reactions, precoding provides a strong basis which is then improved by offline data. Of course, coding assistance system activities is easier than human activity since it is the output of a technical agent program. Programed information from scenarios, and especially events, can also provide interesting data to identify process situations.

<figure><img src="image_37.png" /><figcaption>Fig. 39 Confrontation of pilots and weapon system officers to the video recording of their mission</figcaption></figure>

Spontaneous verbal reports and video outputs of user workstations provide other types of data. Spontaneous verbal reports are especially interesting when two humans work together during experiments. That was the case in the experiments conducted in air traffic control (cf. §7.1, p.122) and in fighting aircraft control (cf. §7.3, p.133). Professional human operators, like radar and planning controllers, or pilots and weapon systems
officers, train themselves for their individual tasks, but also for cooperation. They are usually used to working
with the same partners. Therefore, verbal reports are full of information about individual and mutual
information processing. That was especially the case when humans had to perceive or transmit all information
through electronic devices, because they have to be silent to avoid detection by the enemy for example. Online
verbal reports are sometimes not easy to understand for experimenters. Therefore, in addition to these reports,
experimenters use self-observation reports. After task execution, participants watch video records of their
activities and provide comments to explain (but not justify) their decision making.

Like in spontaneous verbal reports, simulated environments give experimenters the opportunity to record other kinds of subjective viewpoints, e.g. online participants' workload and situation awareness. Experimenters evaluate online workload assessment by asking to participants to regularly answer questions requesting information about their workload. Questions are verbal or posed by the way of a scale drawn on a display (cf. Fig. 40). Even though it is an additional task for participants, this question can be part of a cooperative task, especially if participants have to evaluate their own workload to decide to allocate tasks to a partner. We used this method during experiments conducted with air traffic controllers. The radar controller evaluated their own workload, while the planning controller evaluated the workload of the radar controller. The objective was to know if the planning controller was able to evaluate the workload of their partner in order to allocate task to the assistance system if they were overloaded (cf. §7.1, p.122). Results confirmed this hypothesis and advocated studying multi-level cooperation (cf. §3.4).

<figure><img src="image_38.png" /><figcaption>Fig. 40 Workload assessment on the radar display of air traffic controllers work position (red scale on the left lower part of the interface)</figcaption></figure>

Designers and experimenters may compare the evaluation of workload done by participants to calculated
assessment performed by the system using different technics. In the experiment conducted with air traffic
controllers, a scheduling module based on three sub-modules (task set prediction, characterization of predicted
set of tasks, partitioning) focusing on the radar controller's KH, decided allocation of shareable tasks to the
assistance system (cf. §7.1, p.122). This ability of the schedule module was in fact the KHC of the assistance
system that is able to predict human workload through the model of them that it builds (Jean-Michel Hoc &
Lemoine, 1998b).
The evaluation of situation awareness is also a useful tool to state how participants perceive and understand a process or an environment, and come to a decision. Several methods exist. Endsley proposed a method associated with her three-level model of SA (Endsley, 1995a)(cf. §2.3.1, p.33). The method, known as SAGAT (Situation Awareness Global Assessment Technique), used in simulated environments, consists in freezing the situation randomly, blanking the displays and asking participants relevant questions on their understanding of the current situation. This method compares participants' perception, understanding and projection to those of a Subject Matter Expert (SME) who also follows the same procedure, but without blanked displays. Endsley validated SAGAT experimentally in a set of air-to-air engagements, played on a flight simulator. Indeed, the questions are so dependent on the situation that participants are unable to answer if they are not aware of that situation. SAGAT has acquired a status as a reference method. However, its main limitation lies in the freezing of the simulation, which prevents its use in real applications. During experiments conducted in robotics (cf. §7.4.3, p.140), we used an adapted method to evaluate participants' situation awareness. Among the tasks they had to complete, participants had to provide information about their situation awareness during the experiment. However, reconnaissance was one of the tasks they had to perform, so that was not an additional task requested only for data records. Fig. 41 provides an example of the electronic fulfilment of an interface displaying available information (walls, robots position and trajectory). Participants had to add information about what they perceive in this environment (victims and different types of signs).

<figure><img src="image_39.png" /><figcaption>Fig. 41 Participants' situation awareness during experiments conducted about human-robots cooperation (L. Habib, Pacaux-Lemoine, & Millot, 2017a)</figcaption></figure>

Simulated environments allow testing assistance systems in safe and controlled conditions. Such experiments are required before conducted tests in real world. Nevertheless, real world tests involve many constraints presented in the following sections.

### 4.3.2.2 Online data records in a real world

Experiments conducted in the real world are much more complex and much less controllable than in simulated environments, especially when the focus is on human-machine cooperation. Two experiments conducted in car driving and in robotics brought our attention to such difficulties. The first one deals with experiments in car driving to evaluate impacts of dynamic and adapted road signs on car drivers' behavior (cf. §7.2.2, p.127). The second one deals with human-robot cooperation in reconnaissance missions (cf. §7.4.2, p.137).
First at all, such types of experiments involve requesting various agreements from different institutes, especially to guaranty the safety of participants and to respect existing rules in the domain. These procedures can take several months, so experimenters must anticipate this.

In the first experiment about the study of car drivers' behavior, experimenters designed assistance systems, as well as systems to record data. Fig. 42 presents the set-up of experiments to evaluate the impact of two dynamic road signs, which repeat previous road signs when the speed of the car does not decrease on the second measurement (reminder) and on the third measurement (slow down). Experimenters recorded speeds of each car during several half days, as well as two video recording of each car. Video recordings supported the analysis of drivers' behavior, such as the position of the car in the lane and the use of the brake pedal (brake lights). The measurement systems were not visible. The objective was to prevent their detection by drivers who may make connection with policy control. RTMaps software supports synchronization between all data as well as data records (cf. §4.3.4, p.83, Fig. 54), (Gruyer, Royère, Du Lac, Michel, & Blosseville, 2006). Experiments observed about 1000 vehicles per day. Therefore, the data were more quantitative than qualitative. However, we were able to stop some drivers after traveling across the experimental road, and for a portion of the drivers, we were able to synchronize quantitative and qualitative data (Auberlet et al., 2012; Bordel et al., 2014).

<figure><img src="image_40.png" /><figcaption>Fig. 42 Data records during experiments conducted on real roads</figcaption></figure>

In the second experiment conducted within the framework of the TAROT project (cf. §7.4.2, p.137), participants were professional teleoperators of the French army. They had to supervise and/or control a robot moving in an environment with many obstacles like stones, bushes and holes in the ground. This environment, the ruin of an abandoned factory, increased the attention of participants who knew that robot might not be able to detect some of these obstacles. Moreover, unlike controlled simulated environments, technical problems occurred like communication disturbances or interruptions. Even if such problems can also occur during real missions, these uncontrolled events prevented repetitive scenarios from being conducted to extract statistically significant results. Therefore, due to the difficulty and the risks of completing the tasks with an expensive prototype robot, experimenters remained in contact with participants by radio. Participants regularly checked their decision making with experimenters. Even if such an experimental method distorted results, verbal reports were rich regarding the understandability of the robot and its cooperativeness. Unlike the experiments presented previously, results were more qualitative than quantitative, and more subjective than objective.