# Understanding perceptual-gestural knowledge in TEL systems with eye-tracking

Ben-Manson Toussaint, Vanda Luengo, Francis Jambon

► To cite this version:

Ben-Manson Toussaint, Vanda Luengo, Francis Jambon. Understanding perceptual-gestural knowledge in TEL systems with eye-tracking. Revista Română de Interacțiune Om-Calculator, 2015, 8 (4), pp.321-340. hal-01372937

HAL Id: hal-01372937

https://hal.science/hal-01372937v1

Submitted on 22 Aug 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Analyze eye-tracker traces to understand perceptual-gestural knowledge in TEL systems

Ben-Manson Toussaint

Ecole Sup. d'Infotronique d'Haïti
Université de Grenoble Alpes

29, 2e ruelle Nazon, Port-au-
Prince, Haiti

ben-manson.toussaint@imag.fr

Vanda Luengo

Université Pierre et Marie Curie

Site Jussieu

3, rue des Fossés St Bernard
75252 Paris

vanda.luengo@lip6.fr

Francis Jambon

Université de Grenoble Alpes

LIG - Bat. C, BP 53, 38041

Grenoble cedex 9, France
francis.jambon@imag.fr

## ABSTRACT

This paper presents our methodology to capture and model multimodal interactions in Intelligent Tutoring Systems (ITS). We are specifically interested in perceptual-gestural interactions combining perceptions, gestures and other type of actions. Traces of such interactions are multisource and heterogeneous. The challenge is to foster their representation into sequences that render their multimodal nature. In this work, we want to show how the proposed representation yields the analysis of visual perceptions influence on learners' performance. Our case study is the ITS TELEOS, a simulation-based Intelligent Tutoring System dedicated to percutaneous orthopedic surgery.

## Author Keywords

Intelligent Tutoring Systems; Perceptual-gestural knowledge, Educational Data Mining

## ACM Classification Keywords

H.5.m. Information interfaces and presentation (e.g., HCI); Miscellaneous.

## General Terms

Human Factors; Design; Measurement.

## INTRODUCTION

Knowledge is considered as perceptual-gestural when it can be described as a combination of different type of knowledge, specifically: theoretical knowledge, perceptual-knowledge and gestural knowledge. It is underlined in Intelligent Tutoring Systems, by interactions involving actions and/or gestures along with perceptions. Perceptions are used as controls for deciding on these actions/gestures execution or validation [7]. We assume that they provide useful insights on the information gathered by learners to support their decisions and thus, are non-negligible in a didactic point of view.

However, this type of knowledge is empirical and often tacit. As a consequence, it is hard to capture and model. In fact, capturing perceptual-gestural knowledge in a learning environment requires the use of complementary sensing devices that generate heterogeneous traces. In this study, we are specifically interested in learner's visualizations as support to executed actions and gestures.

Our case study is TELEOS [7], a simulation-based Intelligent Tutoring System dedicated to percutaneous

orthopedic surgery. Knowledge involved in this domain is perceptual-gestural [1, 8]. In this type of surgery, visual analyzes require the perfect mental coordination of 2D images (X-Rays) and 3D objects (X-Rays unit, patient's body, surgical tools) to insure the safe trajectory of surgical tools through the targeted anatomical area. The conducted experiment put the focus on the influence of learners' behavior related to visual analysis on their performance. Also, we propose a second case study to analyze the replicability of our approach.

The rest of the paper is organized as follows. The 2nd section presents related works on capturing and analyzing perceptions in Intelligent Tutoring Systems; the 3rd section describes the methodology to capture multisource traces in our case study; the 4th section presents the our proposition to formalize these traces into perceptual-gestural sequences; the 5th presents the conducted experiment and results; and the 6th section draws our conclusion and perspectives.

## RELATED WORKS

The literature reports many prominent works on Intelligent Tutoring Systems dedicated to domains where perceptual-gestural knowledge is involved. We can mention ITSs that have been proposed for training helicopters [10] and planes [11] piloting as well as car driving [16, 17]. As one of the most recent related researches, we can also cite CanadarmTutor that was designed to train astronauts of the International Space Station for handling an articulated robotic arm [4]. However, the emphasis is generally carried in these works on actions and gestures and not on the perceptions accompanying these latter. In CanadarmTutor, the manipulation of the robotic arm from one configuration to another is guided by cameras through the operation scenes. Visual perceptions that are likely in play for this guidance would be worth further analysis. Other works has been conducted on the analysis of perceptions in learning contexts. For example, visual perceptions are captured and analyzed to deduce learners' cognitive abilities [13] or their metacognitive skills in exploratory learning [2]. Some researchers would rather use collected perceptual information for measuring the learners' mental workload or cognitive effort [6], or for inferring their behavior in the learning process [3, 8]. In other studies, sensing devices are used for capturing postures, facial expressions and body language as emotional signals [13]. For our part, we believe that perceptions denote knowledge states along with actions
they are related to and, therefore, should be analyzed from an epistemic point of view. They can bring more precision to generated pedagogical feedback as experts strongly underline the importance of verifying specific anatomic points on the X-Rays to support decision or validation of surgical gesture [1]. The aim is to point out the benefits from studying perceptual-gestural knowledge up on its original multimodal characteristics. To realize this, we need first to foster the consistent representation of perceptions-related behaviors and actions/gestures into perceptual-gestural sequences.

# CAPTURING VISUAL INTERACTIONS

## Recording visual traces

The simulation interface of TELEOS is composed of sections that represent the main artefacts of a percutaneous operating room. Namely, as illustrated in figure 1.a, it includes a 3D model section where the patient's model is displayed; the current and previous X-rays sections and the settings panel that embeds three settings sub-sections: the fluoroscope settings panel; the cutaneous marks panel and the trocar manipulation panel. The “fluoroscope” is the unit used to generate X-rays throughout the operation; the “trocar” is the surgical tool used to reach the targeted vertebra; the cutaneous marks are lines drawn by the

surgeon on the patient's skin to spot the insertion point of the surgical tools. These sections represent the areas of interest (AOI) of the interface, i.e., areas that are recorded throughout the simulation session when they are fixed by the user. The AOIs associated to the X-rays embed the points of interest of the targeted vertebra, i.e., specific parts of the vertebra that should be analyzed on X-rays to support decisions on surgical actions and gestures. These points of interest have been determined by expert surgeons for each clinical case and integrated manually as annotations to the patients' models [7]. The simulation environment is dynamic. As a consequence, the occurrences of the above-mentioned AOIs are also dynamic. In fact they can move, appear and disappear from the interface based on learners' activity and interaction with the system.

In other words, the presence of an AOI at a given location is neither stable nor predictable because it depends on the co-evolution of the activity the learner and the state of the simulation. As an illustration in TELEOS, the position of the points of interest that mark the pedicles of a vertebra, changes with respect to the angle of capture of the X-rays. For addressing this challenge, a specific tool has been implemented to capture learners behaviors related to visual perceptions in a dynamic context [5].

<figure><img src="image_2.png" /><figcaption>Figure 1. a) TELEOS simulation interface. b) Visual path traced by the scanpath analyzer</figcaption></figure>

<figure><img src="image_3.png" /><figcaption>Figure 2. Identification and annotation process of vertebra with points of interest</figcaption></figure>
## Categorizing visual perceptions

Perceptions should be differentiated based on the cognitive efforts they demand or the intent they underlie. In fact, some perceptions intend a precise analysis of the environment whereas others are simple information gathering on the environment state. Other parameters can also help at differentiating learners' perceptions and infer their intent, resolution strategy or their profile. Considering visual perceptions, those parameters are possibly the fixations duration, their frequency, the most gazed areas and points or a combination of all those.

We propose that visual perceptions should be considered into two different categories: (1) visual perceptions of verification/validation type and (2) visual perceptions of decision type.

### Visual perceptions of verification type

This type of perception underlies cognitive activity that aim at analyzing the environment by putting into play precise knowledge elements. They target specific points of the environment that reflect the consequences of executed actions or gestures. More than a simple information gathering, their role is to verify and validate those latter. For instance, in vertebroplasty, visual analysis of the position of the spinous of a vertebra as a landmark to decide on the correctness of its centering, underlie perceptions of verification and validation.

### Les perceptions visuelles de décision

This type of perceptions are less precise than perceptions of verification/validation type. Their role is limited to general information gathering on some specific artefacts of the environment. Generally, those artefacts are related to the tools that are used to act on the environment. As an illustration from our case study, we can cite the controller used for handling the X-Ray unit (Cf. Setting panels in Figure 1.a).

## Heterogeneity of recorded traces

Visual traces are recorded in TELEOS along with punctual actions executed on the simulation interface and gestures executed with a haptic arm that simulates the trocar. Traces from these three sources are recorded independently. They are heterogeneous in their content type and format, and their time granularities [14].

## FORMALIZATION OF PERCEPTUAL-GESTURAL SEQUENCES

Learner's interactions involved in a simulation session are captured into three different modalities. From this point, the challenge is to link those latter into sequences that reflect properly all aspects of underlying knowledge element. These sequences are referred to as perceptual-gestural sequences. We describe their representation in the following sections.

### Characterizing perceptual-gestural sequence

To represent the different modalities with their sequentiality, we consider each element of the sequence as

an item. Each group of items with similar time of occurrence is considered as an itemset. A sequence is composed of several itemsets.

The formal definition of a perceptual-gestural sequences is as follows:

$$ S = < (A_i, i=1..p [a_{ij, j=1..q}]; | G_i, i=1..r [g_{ij, j=1..s}]; [P_k, k=1..v [q_{kl, l=1..w}]]); (P_k, k=1..v [q_{kl, l=1..w}]) > \quad (1) $$

The brackets "(" and ")" define the itemsets in the sequence. There is no restriction to represent items from different categories in the same itemset if their occurrences are simultaneous. For example, in the case of a joint connection between the terms of the sequence, an action can be represented in the same itemset a viewing if the action was performed simultaneously with the visual information.

* $A_i = A_1 ... A_p$ is the set of actions; $p \ge 1$

* if $r >= 1$ then $p >= 0$
  * else, $p >= 1$

* $G_i = G_1 ... G_r$ : is the set of gestures

* if $p >= 1$ then $r >= 0$
  * else, $r >= 1$

* $|A \cup G| \ge 1$: A perceptual-gestural sequence comprises at least one action or gesture.

* $a_{ij} = a_{i1} ... a_{iq}$ : set of actions parameters; $q >= 0$

* $g_{ij} = g_{i1} ... g_{is}$ : set of gestures parameters; $s >= 0$

* $P_k = P_1 ... P_v$ : set of perceptions; $v \ge 1$

* $q_{kl} = q_{k1} ... q_{kw}$ : set of perceptions parameters; $w \ge 0$.

Figure 3 gives an example of perceptual-gestural sequence generated with traces from our case study.

<figure><img src="image_4.png" /><figcaption>Figure 3. Example of perceptual-gestural sequence</figcaption></figure>

The perceptual-gestural sequence illustrated in figure 3 represents the execution of the action "Impacter_Trocart". It reports the actions of putting the surgical tool in contact with the targeted vertebra. The gesture specifies that the trocar has been simultaneously inserted (trocartarametronterieur), while being moved to the right side of the patient (trocartTranslation_droite) and in the direction of the lower body of the patient (trocartTranslation_caudale). The visual perceptions parameters report the areas of interest of the interface that has been gazed. For instance, O_outil_vue3D refers to the verification of the position of the trocar on 3D model of the interface. O_manipReglage_1 refers to the visualization of the setting panel of the fluoroscope.

### Characterizing enriched perceptual-gestural sequences

In the perspective of increasing the precision of learners' interactions reported in a perceptual-gestural sequence, we
propose to enrich it with information from the system. Specifically, we propose to take into account the “reactions” of the learning environment related to learners’ interactions. Typically, we consider two types of interactions from the system: (1) the simulation states and (2) the automatic evaluations based on expert rules.

The simulation states denote the current positions of the different artefacts represented in the simulation interface. The automatic evaluations based on expert rules refer to the evaluations of learners’ actions produced by the diagnosis module of the ITS. These evaluations are based on expert rules provided by experts of the domain and are termed as “situational variables” [9].

The integration of this information produce what we call “enriched perceptual-gestural sequences”. These sequences are formally defined as follows:

$$S_g: < ([f_i], S_i [T_{ij}, j=1..η])_{i=1..η} (V_q=1..v [v_r=1..w])> \quad (2)$$

As presented in (2), an enriched perceptual-gestural sequence contains one or several perceptual-gestural sequences. These latter are characterized by information related to simulation states and annotations based on automatic evaluations from expert rules (situational variables). An evaluation can possibly assess several sequences:

* $f_i$ represents the temporal tag stating the order of the sequence $S_i$ in the enriched sequence;

* $S_i$ is the set of perceptual-gestural sequences composing the enriched sequence

* $T_{ij}$ is the set of simulation states recorded in a sequence $S_i$

* $V_q$ is the set of situational variables in the enriched sequence

* $v_r$: values of the situational variables

Figure 4 illustrates the representation described above by an example from our case study.

<figure><img src="image_5.png" /><figcaption>Figure 4. Example of enriched perceptual-gestural sequence.</figcaption></figure>

The sequence in figure y represents an action of taking an X-Ray followed by the marking of cutaneous marks. Cutaneous marks are used to spot the entrance point of the trocar on the patient’s skin. The item on the simulation state, *AmpliFace_inclinaison_craniale*, specifies that the X-

Ray unit is inclined toward the patient’s head. The reported evaluations inform on the correctness of the taken X-Ray with corresponding expert rules. For example, *RF_centrageVertebre-correct* reports that the centering of the targeted vertebra on the front radio is correct. *RF_disquesVisibles-incorrect*, specifies that the visibility of spinal discs is incorrect; and *RF_synttriePedicEpineuse-incorrect*, that the symmetry of the spinous with respect to the vertebra pedicle, is incorrect.

## EXPERIMENT

We want to determine in which extent our proposition for the representation of perceptual-gestural sequences facilitates the analysis of learners’ performance. More specifically, in the scope of this study, we are interested in determining how the proposed representation reveals the influence of visual perceptions on learners’ errors during a simulation.

<table><caption>Table 1. Data characteristics</caption><thead><tr><th>Profiles</th><th>Session</th><th>#Enriched p.-g. seq.</th><th>#Fixations</th><th>#Incorrect SV</th><th>#Validation errors</th><th>#Corrective actions</th></tr></thead><tbody><tr><td>Intern</td><td>S01</td><td>113</td><td>2033</td><td>750</td><td>9</td><td>11</td></tr><tr><td>Intern</td><td>S02</td><td>37</td><td>885</td><td>178</td><td>4</td><td>4</td></tr><tr><td>Intern</td><td>S03</td><td>33</td><td>690</td><td>208</td><td>3</td><td>5</td></tr><tr><td>Intern</td><td>S04</td><td>128</td><td>2482</td><td>644</td><td>10</td><td>39</td></tr><tr><td>Intern</td><td>S05</td><td>41</td><td>858</td><td>174</td><td>6</td><td>10</td></tr><tr><td>Expert</td><td>S06</td><td>59</td><td>1452</td><td>249</td><td>4</td><td>31</td></tr><tr><td>Expert</td><td>S07</td><td>47</td><td>1040</td><td>239</td><td>5</td><td>9</td></tr><tr><td>Intern</td><td>S08</td><td>117</td><td>2514</td><td>644</td><td>20</td><td>36</td></tr><tr><td>Intern</td><td>S09</td><td>41</td><td>869</td><td>193</td><td>4</td><td>22</td></tr></tbody></table>

For this purpose, we consider the following parameters:

* the number of erroneous validation of actions

* the number of corrective actions applied on these validation errors

* the number and type of visual perceptions associated to these corrective actions

* and the number of situational variables reported as incorrect

The traces used for this experiment were recorded from 9 simulation sessions of vertebroplasty performed by 5 interns and 1 expert surgeon of the University Hospital of Grenoble. The proposed simulation exercises consisted of treating a fracture of the 11th and/or the 12th thoracic vertebra. Each session lasts around one hour. The interns have never used the simulator before but have already assisted to at least one vertebroplasty operation in real life. We integrated an expert in the group as to define a reference scope for the performance of a simulated vertebroplasty. Table 1 presents the characteristics of collected and treated data. The enriched perceptual-gestural sequences have been generated from collected raw traces with the framework PeTRA [14].
<figure><img src="image_6.png" /><figcaption>Figure 5. a) Histogram of average incorrect situational variables, average visual perceptions and number of validation errors per session. b) Histogram of corrective actions and average associated visual perceptions per session. c) Histogram of average verification perceptions and average decision perceptions.</figcaption></figure>

## Analyzing Influence of Visual Perceptions on Learners' Performance

The number of sequences vary greatly from one session to another. Thus, we use the average number of visualizations per sequence that better reflects the trend of visual analysis. The same applies for situational variables reported as incorrect.

The graph of figure 4.a summarizes the distribution of visual perceptions, incorrect situational variables and validation errors for each session. Pearson index indicates strong negative correlation (-0.62) between visual perceptions and incorrect situational variables. On the other hand, correlation between visual perceptions taken as a whole and validation errors is rather moderate (-0.31). However, strong negative correlation is noticed between visual perceptions of verification type and validation errors (-0.53). In fact, the session with the highest rate of perceptions (24.6) reports 19% fewer incorrect situational variables than the others, in average. The same link can be observed between visual analysis and validation errors for all the sessions, except for S08. This can be explained by the fact that the subject performed few corrective actions and visual analysis to support these actions. In fact, in the graph b of figure 4, we can notice that this session has one of the lowest averages of corrective sequences (1.7) along with the lowest average of visual perceptions (15.5) associated to these sequences. As a comparison, session S02 reported the lowest average of corrective actions (1.0). See Fig. 4.b) but numerous visual analyzes (20.5) for supporting validation decisions and consequently limiting the number of errors (4. Cf. Figure 4.a). Moreover, it can be seen in Fig. 4.c that few visual analysis in S08 are of verification type (7.7 against 13.8 of decision perceptions). S09 was performed by the same intern.

Conversely, less validation errors and fewer incorrect situational variables were observed even with approximately the same rate of visual perceptions. This is the consequence of the reversal of the amount of visual perceptions and the execution of more corrective actions (Cf. Figure 4.b).

## Genericity: Proof of concept

We applied our model for the representation of multimodal interactions involving visual behavior in a new domain, aviation, and a new simulation environment PILOTE2.

<figure><img src="image_7.png" /><figcaption>Figure 6. Visual behavior captured on flight simulation environment.</figcaption></figure>

Actions executed by pilot are supported by visual verification of the state of the aircraft. The tools states of the aircraft are displayed on the indicators of the dashboard. Figure 6 shows the visual path of the pilot on the dashboard (in blue) as it is captured during a flight simulation. Each visualization, is also reported, with respect to their time of occurrence, on the monitoring screen (circled in red). The reported information sets the name of gazed areas of interest and duration of each visualization.

We showed that our model for the representation of perceptual-gestural sequences fosters the representation of pilot visual behavior with respect to executed actions during flight simulations [15]. Further, as illustrated in Figure 7, we could link, in enriched perceptual-gestural sequences, pilot's visualizations to errors and successes reported by the system.

<figure><img src="image_8.png" /><figcaption>Figure 7. Enriched perceptual-gestural sequence generated from flight simulation traces.</figcaption></figure>
Acronyms such as AH, RPM, ASI, etc. represent the indicators gazed by the pilot on the dashboard. To illustrate RPM stands for Rotation Per Minute indicating the rotation speed of the aircraft engine and the current power that it produces.

## CONCLUSION AND FUTURE WORKS

We presented in this paper our proposition for addressing the challenge of processing multi-source heterogeneous traces from ITSs dedicated to domains involving perceptual-gestural knowledge. The aim is to consistently connect perceptions to actions and gestures they support as to foster the consideration of all aspects of involved knowledge. We demonstrated the efficiency of our proposition for processing traces recorded on TELEOS, a simulation-based Intelligent Tutoring System (ITS) dedicated to percutaneous orthopedic surgery. We showed that our proposition for the formalization of perceptual-gestural sequences fosters the analysis of the influence of visual perceptions on interns' performance in simulation sessions of vertebroplasty. We also provided proof of concept of the genericity of our proposition based on traces from flight simulation sessions.

We plan to go further by extending our analyses to the measure of the effective gain from taking into account perceptual aspect of multimodal knowledge compared to treatment that discard either facet of this type of knowledge.

## REFERENCES

1. Ceaux, E., Vadcard, L., Dubois, M., & Luengo, V.: Designing a learning environment in percutaneous surgery: models of knowledge, gesture and learning situations. Paper presented at the EARLI Symposium "Simulation-Based Learning: Analyzing and Fostering Complex Skills in the Context of Medical Education", Amsterdam (2009)

2. Conati, C., Merten, C.: Eye-tracking for user modeling in exploratory learning environments: An empirical evaluation. Knowl.-Based Syst, Vol. 20(6) (2007) 557-574

3. D'Mello, S. Olney, A., Williams, C., Hays, P.: Gaze tutor: A gaze-reactive intelligent tutoring system. *Int. J. Hum.-Comput. Stud.*, Vol. 70(5) (2012) 377-398

4. Fournier-Viger, P., Nkambou, R., Mayers, A., Mephu Nguifo, E., Faghihi, U. A Hybrid Expertise Model to Support Tutoring Services in Robotic Arm Manipulations. In Proceedings of the 10th Mexican International Conference on Artificial Intelligence. LNAI 7094, Springer, (2011) 478-489

5. Jambon, F., Luengo, V.: Analyse oculométrique « on-line » avec zones d'intérêt dynamiques : application aux environnements d'apprentissage sur simulateur. In Actes de la Conférence Ergo'IHM sur les Nouvelles Interactions, Créativité et Usages, Biarritz France (2012)

6. Lach, P.: Intelligent Tutoring Systems: Measuring Student's Effort During Assessment. In: Zaïane, O.R. and Zilles, S. (eds.): Advances in Artificial Intelligence. Lecture Notes in Computer Science, Vol. 7884. Springer, Berlin Heidelberg (2013) 346-351

7. Luengo, V., Larcher, A., Tonetti, J.: Design and implementation of a visual and haptic simulator in a platform for a TEL system in percutaneous orthopedic surgery. In

Medecine Meets Virtual Reality 18. (eds.): Westwood J.D., Vestwood, S.W. (2011) 324-328

8. Mathews, M., Mitrovic, A., Lin, B., Holland, J., Churcher, N.: Do Your Eyes Give It Away? Using Eye Tracking Data to Understand Students' Attitudes towards Open Student Model Representations. In: Cerri, S.A., Clancey, W.J., Papadourakis, G., Panourgia, K. (eds.): Intelligent Tutoring Systems. Lecture Notes in Computer Science, Vol. 7315. Springer, Berlin Heidelberg (2012) 422-427

9. Minh Chieu, V., Luengo, V., Vadcard, L.: Student Modeling in Orthopedic Surgery Training: Exploiting Symbiosis between Temporal Bayesian Networks and Fine-grained Didactical Analysis. In International Journal of Artificial Intelligence in Education, 2010, Vol. 20(3). 33 pages

10. Mulgund, S.S., Asdigha, M., Zacharias, G. L., Ma, C., Krishnakumar, K., Dohme, J.A., Al, R.: Intelligent Tutoring System for Simulator-Based Helicopter Flight Training. Flight Simulation Technologies Conference. American Institute of Aeronautics and Astronautics Baltimore MD U.S.A. (1995)

11. Remolina, E., Ramachandran, S., Fu, D., Stottler, R., Howse, W.R.: Intelligent Simulation-Based Tutor for Flight Training. In: Interservice/Industry Training, Simulation, and Education Conference. (2004) 1-13

12. Ríos, H.V., Solís, A.L., Aguirre, E., Guerrero, L., Peña, J., Santamaría, A.: Facial Expression Recognition and Modeling for Virtual Intelligent Tutoring Systems. In: Cairó, O., Sucar, L.E., Cantu, F.J. (eds.): Advances in Artificial Intelligence. Lecture Notes in Computer Science, Vol. 1793. Springer, Berlin Heidelberg (2000) 115-126

13. Steichen, B., Carenini, G., Conati, C.: User-adaptive information visualization: using eye gaze data to infer visualization tasks and user cognitive abilities. In: Proceedings of the 2013 Int. Conf. on Intelligent User Interfaces. ACM, New York NY USA (2013) 317-328.

14. Toussaint, B.M., Luengo, V., Jambon, F. and Tonetti, J.: From Heterogeneous Multisource Traces to Perceptual-Gestural Sequences: the PeTra Treatment Approach. In: Conati, C., Heffernan, N., Mitrovic, A., Verdejo, M. F. (eds), Proceedings of the 17th International Conference on Artificial Intelligence in Education (AIED 2015), Madrid, Spain. LNCS, vol. 9112, pp. 480-491. Springer, Heidelberg (2015)

15. Toussaint, B.M.: Apprentissage automatique à partir de traces multi-sources hétérogènes pour la modélisation de connaissances perceptivo-gestuelles, Université Grenoble Alpe, PhD Thesis, Chapter 10.

16. Weevers, I., Kuipers, J., Brugman, A.O., Zwiers, J., van Dijk, E.M.A.G., Nijholt, A.: The Virtual Driving Instructor, Creating Awareness in a Multi-Agent System. In: Xiang, Y., Chaib-Draa, B. (eds.): Proceedings of the 16th Canadian society for computational studies of intelligence conference on Advances in artificial intelligence. Springer-Verlag, Berlin Heidelberg (2003) 596-602.

17. de Winter, J.C.F., de Groot, S., Dankelman, J., Wieringa, P.A., van Paassen, M.M., Mulder, M.: Advancing simulation-based driver training: lessons learned and future perspectives. In *Proceedings of the 10th international conference on Human computer interaction with mobile devices and services*. ACM, New York NY USA (2008) 459-464.