# Characterization of collaboration in a virtual environment with gaze and speech signals

Aurélien Léchappé, Aurélien Milliat, Cédric Fleury, Mathieu Chollet, Cédric Dumas

► To cite this version:

Aurélien Léchappé, Aurélien Milliat, Cédric Fleury, Mathieu Chollet, Cédric Dumas. Characterization of collaboration in a virtual environment with gaze and speech signals. ICMI '23: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION, Oct 2023, Paris France, France. ACM, pp.21-25, 2023, 10.1145/3610661.3617149 . hal-04327816

HAL Id: hal-04327816

https://hal.science/hal-04327816v1

Submitted on 5 Jun 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

University
of Glasgow

Léchappé, A., Milliat, A., Fleury, C., Chollet, M. and Dumas, C. (2023)
Characterization of Collaboration in a Virtual Environment with Gaze and Speech
Signal. In: 25th ACM International Conference on Multimodal Interaction (ICMI
2023), Paris, France, 9-13 October 2023, pp. 21-25. ISBN 9798400703218.

There may be differences between this version and the published version. You are
advised to consult the publisher's version if you wish to cite from it.

© 2023 Copyright held by the owner/author(s). This is the author's version of the
work. It is posted here for your personal use. Not for redistribution. The definitive
Version of Record was published in ICMI '23 Companion: Companion Publication
of the 25th International Conference on Multimodal Interaction.

<https://doi.org/10.1145/3610661.3617149>

<http://eprints.gla.ac.uk/305084/

Deposited on: 14 September 2023
# Characterization of collaboration in a virtual environment with gaze and speech signals

Aurélien Léchappé

IMT Atlantique, LS2N, Lab-STICC
Nantes, France

aurelien.lechappe@imt-atlantique.fr

Aurélien Milliat

IMT Atlantique, LS2N
Nantes, France

aurelien.milliat@imt-atlantique.fr

Cédric Fleury

IMT Atlantique, Lab-STICC
Brest, France

cedric.fleury@imt-atlantique.fr

Mathieu Chollet

School of Computing Science,
University of Glasgow
Glasgow, Scotland

mathieu.chollet@glasgow.ac.uk

Cédric Dumas

IMT Atlantique, LS2N
Nantes, France

cedric.dumas@imt-atlantique.fr

## ABSTRACT

Collaboration during the completion of complex tasks requires synchronization and effective communication within teams. In this paper, we study multimodal collaboration metrics with the intention of designing collaboration support systems that enable the evaluation of collaboration quality. The goal is to provide real-time feedback and prevent the emergence of critical situations due to poor collaboration. We use a collaborative virtual environment to control the situation and the effects of the environment. As a first step to measure collaboration, we present a work in progress that aims to test the effectiveness of verbal and gaze measurements as an indicator of collaboration quality. To do so, an assembly task was performed by twelve dyads in our virtual environment. A tool was designed to process activity inputs in real time. Our findings show that verbal and gaze metrics can provide feedback on collaboration in virtual environment. Using these measures together with others would provide more accurate feedback on collaboration. Lastly, suggestions for improving the tool and an interest for other collaboration indicators are raised.

## CCS CONCEPTS

* Human-centered computing → Collaborative interaction; Virtual reality.

## KEYWORDS

Collaboration, Virtual reality, Collaboration cues, Joint visual attention, Turn-taking, Overlapped Speech

## 1 INTRODUCTION

In some situations, such as in the medical field, a decrease in the quality of collaboration can lead to adverse events, sometimes with severe consequences for patients or task outcomes [13]. One possible solution to reduce the emergence of these types of events is to provide real-time feedback on the quality of collaboration within the group. This would allow operators to be informed whether their current collaboration or situation awareness level is optimal or not. Providing such feedback may make it possible to predict the emergence of incidents, to detect misunderstandings or coordination problems, and to act directly on the situation in a timely manner. To this end, we need modeling the collaboration over time.

In this paper, we propose a tool in which we use indicators to report on the collaboration quality. We use a virtual reality collaborative task (Figure 1) which allows us to collect precise behavioural data (e.g. eye-tracking data) but more importantly, enables fine control over the collaborative situation and to deploy a variety of modifications to the environment in response to users' actions and collaborative states. In this paper, our first step is to validate the use of audio and eye-tracking as real-time indicators of the level of collaboration.

<figure><img src="image_3.png" /><figcaption>Figure 1: Virtual environment and collaborative task</figcaption></figure>

## 2 THEORETICAL BACKGROUND

Collaboration occurs when there is symmetry in the interactions between two or more people working together to achieve a common goal [7]. Prior research on collaboration has focused on developing models that explain collaboration construction and the different elements that influence it during a collaborative activity. The emergence of increasingly efficient data collection tools has made it possible to simplify research on the detection of collaboration processes and behaviors [17]. This has contributed to the increase in research on modeling and detecting collaboration using multimodal data over the past decade. Schneider et al. [17] present a literature review about uses of multimodal data to capture different aspects of collaboration and called this approach Multimodal Collaboration Analytics (MMCA). Praharaj et al. [14] propose a literature review on modeling co-located collaboration.

Praharaj et al. [14] provide an inventory of different indicators of collaboration quality. Based mainly on Meier et al.'s [12] collaboration dimensions, the authors propose high-level indices derived from low-level indicators. These indices are synchrony, equality, individual accountability, intra-individual variability, information sharing, mutual understanding, and reciprocal interaction. They
are derived from low-level indicators such as the total speaking time of each team member, postures, gestures, looks, position in space, the way individuals answer a question or defend a point of view, or the similarity between group members in changes in pitch and amplitude of speech.

In a later paper, Praharaj et al. [15] propose an overview of collaboration indicators from audio. Still, based on the Meier dimensions, they propose classifying the indicators in different parameters such as dominance, rapport, expertise, active participation, roles, and knowledge co-construction.

**Dominance** can be represented by the total speaking time of each collaborator during the task. High participation equality within the group means less dominance and then would be related to better collaboration quality [3, 9]. A good **rapport**, i.e., a feeling of closeness or friendliness between people [10], can improve collaboration. Lubold & Pon-barry [10] shows that a high level of synchrony in pitch variation is positively correlated with rapport. A high frequency of turn-taking and an overlap of speech would be indicators of higher **active participation** and, therefore, a good collaboration quality [8].

Schneider et al. [17] report collaboration measures classified according to the theoretical framework of Dillenbourg et al. [7]. Like [14], Schneider et al. [17] propose verbal indicators, such as participation in discussions, verbal coherence (similarity of discourse, equality of contributions), or speaking time, as relevant indicators of collaboration quality. These authors also mention episodes of physiological synchrony as an indicator of increased investment of individuals in the task, and thus as an indicator of good collaboration [6]. Finally, the authors mention that joint visual attention (JVA), i.e., attention to the same object or area by two or more collaborators [16], is a measure that can be used to predict the quality of collaboration [18]. Indeed, JVA should be a way to maintain **mutual understanding** and awareness of the environment and what other members are doing.

Admoni & Scassellati [1] present 4 type of social eye gaze that can appear during human interaction. Mutual gaze, defined as the moment where each individual look explicitly at each other; Referrential gaze, when individual gaze is directed at an object. This type of gaze can be associated with verbal references to a specific object; Gaze aversion, when gaze averts from the main direction of the gaze and Joint attention gaze, corresponding to JVA. According to [1], JVA can begin with a mutual gaze to establish attention.

Praharaj [15] chose to concentrate his application on the use of audio cues to evaluate collaboration, using a face-to-face board game. It does not attempt to evaluate collaboration in real-time. Schneider et al. review's [17], list studies with a large diversity in the task context and measurement method. The authors raise the question of the transferability of multimodal analysis of collaboration to less frequently studied domains.

Our contribution to the past studies consists of applying and testing the efficiency of such multimodal indicators for **assessing a collaborative situation in real-time in a virtual reality environment**.

We have selected 4 indicators regarding the following criteria: the relevance according to the task; the representation of collaboration dimensions; and the technical feasibility. The selected indicators are the speaking time distribution between collaborators, turn-taking

with overlap, presence of joint visual attention, and mutual gaze. We present a tool providing a real-time computing and visualization of these metrics. As a first step, our objective is to validate the relevance of the chosen indicators to attest to a good or bad level of collaboration in our context. This will allow us to take a first step towards detecting in real time the moments when collaboration is not optimal and to intervene in the situation.

## 3 METHOD

Based on the literature, we formulate the following hypotheses:

H1 Good collaboration quality should be associated with equal speaking time distribution among team members [3, 9].

H2 Good collaboration quality should be associated with a higher frequency of speaking turns with overlap compared to low collaboration quality [8].

H3 Good collaboration quality should be associated with greater joint visual attention compared to low collaboration quality [18].

H4 Higher frequency of joint visual attention should be associated with higher frequency of mutual gaze.

H5 Good collaboration quality should be associated with more mutual gaze compared to low collaboration quality.

### 3.1 Participants

Thirty participants in 15 groups participated in this research. The participants were students and did not receive any compensation. Due to audio recording problems for 3 sessions, the final sample contained 24 participants for 12 groups (20M, 4F, μ age = 21.66, SD = 1.40). Each pair of participants knew each other. All pairs performed the task under the same conditions. The objective was to relate the different indicators to their collaboration level. The different groups are distinguished based on third-party ratings of how they collaborated during the task.

### 3.2 Materials

We used HTC Vive Pro eyes VR headsets to immerse both participants of each pair in a virtual environment, which is 1:1 copy of the room in which they were physically co-located. Participants interact with a raycast technique using their VR controllers. Gaze direction was calculated using Tobii eye-trackers integrated into the headsets, and audio was collected using headset microphones. 4 computers were used: one ran a Unity server and a data collection server, two others ran Unity clients for the headsets, and the last one collected data and video streams from a Kinect Azure.

### 3.3 Task and procedure

The task used in this study is a virtual reality puzzle-solving task. Participants must collaborate to retrieve pieces scattered throughout the environment and complete a puzzle according to a reference figure (right Figure 1). By default, all pieces are white, and they take the participant's color (blue and green) when placed on the puzzle. For the cyan color (mix between blue and green), both participants have to select the same piece. Therefore, they cannot succeed individually and must collaborate to complete the task. No constraints were imposed on how and when participants retrieved the puzzle pieces in the virtual environment.
Before starting, each participant is trained with an individual tutorial to become acquainted with the controls, environment, and task goal. When ready, both participants press the START button and the pieces appear. They can then begin the task. When they feel they have solved the puzzle, participants press a validation button on the right of the figure, which gives different audio and visual feedback depending on whether the figure has been completed correctly or not. The experimental protocol of our study was approved by a local ethical committee.

## 3.4 Data collected

Regarding quantitative data, our system recorded interactions between the participants and the environment (selection, gaze, hover of pieces), as well as between the participants themselves (gaze, hover, simultaneous selection, gaze at the other participant). For gaze data, Tobii API provides us the information each time a participant looks at an object or participant. The participants' position in the environment, puzzle completion time, video and audio were also recorded.

All these data are collected, synchronized and recorded with the MS /psi platform [5]. This platform allows the collection of synchronized multimodal data. It also allows real-time visualization of the collected data and the possibility of replaying the recorded sessions afterward, as if the session was played in real time. This allows us to test our algorithms by calculating different indicators of the collaboration, and to simulate and visualize a real-time calculation during the collaboration situation.

## 3.5 Collaboration coding

To have a ground truth on the collaboration quality during the task, all recorded sessions were segmented into 20-second windows. Four experts with a background in Computer-Supported Cooperative Work annotated the segments following the coding scheme of Bassiou et al. [4] and using 3 collaboration types commonly studied in VR. It considers the degree of investment and how individuals perform the task collaboratively. Bassiou et al. [4] used 30-second windows to segment their activity, but 20-second windows were more adapted to our task according to the mean total task duration (2.27 min). The last segment of each session was discarded if its length was shorter than 15 seconds. Within the 12 sessions, a total of 86 segments was scored for a total duration of 28.56 minutes.

To code segments, the experts had to assign one of the three following annotations to each segment:

* **Does not collaborate** (1): No participant contributes collectively to the task. They are passive or work independently.

* **Follows the leader** (2): One participant takes the lead and only calls on the other when necessary.

* **Active collaboration** (3): Both participants work together and actively contribute to solving the task.

The Interclass correlation coefficient (ICC) of the annotators is 0.53. Once the segments were annotated, we chose to classify them according to the coding scheme. Segments for which at least 3 annotators agreed were classified in these categories. A 4th category, named "Disagreement segments", was created for the segments for which less than 3 annotators agreed. This gives us 12 segments in the "No collaboration" category (1) (all agreed: 4; Three agreed: 8),

10 segments in the "Collaboration with leadership" category (2) (all agreed: 2; Three agreed: 8), 37 segments for the "Active collaboration" category (3) (all agreed: 13; Three agreed: 24) and 27 for the "Disagreement segments" category (4).

# 4 COMPUTING COLLABORATION INDICES

For the audio data, environmental noise and irrelevant sounds were removed to retain only actual speech acts. Speech data for each collaborator was generated using the psi platform's voice activity detection and Azure Cognitive Services for speech-to-text.

Gaze data counts events when a participant looks at a task-related object or another participant for more than 200 ms, referencing [2], which indicate that the threshold for gaze events is typically between 200 and 400 ms.

After data filtering, we developed an algorithm for real-time calculation of collaboration indicators. We tested our algorithm through offline session replays. Indicators are updated during the session and reported every 20 seconds, including the summed occurrences of Active participation, JVA, and Mutual gazes.

**Speaking time distribution (STD)** is calculated by dividing the talking time of each participant by the total speaking time of the participants in the time window. Then, the absolute difference between the two STD was calculated to obtain an index ranging from 0 to 1. The closer the index is to 0, more the STD is equitable.

**Active participation** was calculated using turn-taking and overlap features, following the approach of [8]. We distinguish turn-taking patterns without overlap from those with overlap.

A *turn-taking with overlap* is counted when user A speaks, user B starts to speak and user A stops speaking before user B. In this case, user B successfully interrupted and took the speaking turn. A *turn-taking without overlap* is counted when user B starts to speak within 3 seconds [11] after user A has finished speaking. An *overlap* is counted when user B speaks at the same time as user A but finishes speaking before A has finished speaking. In this case, user B did not interrupt user A, there is no change in the speaking turn, and we only count one overlap.

JVA was computed considering a 4-second time window during which one user's gaze at a particular object and another user's gaze at the same object [16]. In our experiment, all gaze events performed are recorded. When a gaze event of user A is received, we see if in the last 2 seconds, user B looks at the same object. If so, it is considered that the users had a JVA on this object.

Mutual gazes are counted when users look at each other.

# 5 PRELIMINARY RESULTS

<table><caption>Table 1: Collaboration categories</caption><thead><tr><td>Category</td><td>Name</td></tr></thead><tbody><tr><td>1</td><td>No Collaboration</td></tr><tr><td>2</td><td>Collaboration with leadership</td></tr><tr><td>3</td><td>Active collaboration</td></tr></tbody></table>

## 5.1 Agreed segments

Preliminary results are performed using Spearman's rank order correlation to assess the relationship between collaboration category
(Table 1) and all the indicators measured. There was a moderate correlation between collaboration categories and turn-taking with overlap, *rs*(59) = .469, *p* < .001, a weak negative correlation between collaboration categories and speaking time distribution, *rs*(59) = -.377, *p* < .003 and a trend for a weak correlation between collaboration categories and joint visual attention frequency, *rs*(59) = .245, *p* = .061. On all the segments with annotators agreement, no mutual gaze events were observed. Further pair-wise analysis for turn-taking with overlap show a moderate correlation between collaboration category 1 and 2, *rs*(22) = .566, *p* < .006, 1 and 3, *rs*(49) = .545, *p* < .001, but no correlation between 2 and 3. Pair-wise analysis of JVA shows a strong correlation between 1 and 2, *rs*(22) = .68, *p* < .001, a moderate correlation between 1 and 3, *rs*(49) = .473, *p* < .001 and a trend for a weak negative correlation between 2 and 3, *rs*(47) = -.285, *p* = .052. Lastly, pair-wise analysis for speaking time distribution shows a weak negative correlation between category 1 and 3, *rs*(49) = -.326, *p* < .022, 2 and 3, *rs*(47) = -.364, *p* < .012 and no correlation between 1 and 2.

## 5.2 Disagreement segments

On the 27 segments with annotator disagreements, 11 had 3 different ratings and 16 had 2 among the 4 experts. For these: in 2 segments, annotators rated 1 and 2 (7.4%); 3 were rated 1 and 3 (11.1%) and 11 were rated 2 and 3 (40.7%).

# 6 DISCUSSION

## 6.1 Results and collaboration categories

Preliminary findings support H1, showing better STD equality in the “Active collaboration” category (3) than in "No collaboration" (1) or "Leadership" (2) categories, but no significant differences between 1 and 2 were found. Results for turn-taking with overlap and JVA are consistent with H2 and H3. The lack of differences between categories 2 and 3 raises questions. Compared to the "Leadership" category, the "Active collaboration" category relates to a situation where the collaborators actively participate **without dominance**. It suggests our indicators might not be sufficient or adequate to distinguish effectively these collaboration types. The fact that 40% of the disagreement segments concerned segment when experts rated 2 and 3, confirms the difficulty to distinguish these types of collaboration. It could be relevant to combine indicators or add other elements to discriminate these two states of collaboration. With regard to the JVA, the trend observed between the “Leadership” and “Active” collaboration categories is interesting. It could be explained by the fact that in a collaboration with leadership, the non-dominant collaborators would be more prone to have a focused attention on the same objects as the dominant collaborator.

Furthermore, the absence of mutual gaze in our task means H4 and H5 couldn't be validated. The realism of the avatars in the virtual environment, representing only the head, hands, and orientation, may not encourage gaze. Real-life collaborators can gain information from facial expressions and gaze, unlike here. Plus, the task's nature required users to focus on puzzle pieces, with speech possibly being enough for synchronization. This might be why mutual gaze wasn't a significant indicator in our context.

## 6.2 Limits and future studies

While data in this study is limited, it yields guidelines for future research. Over a 20-second collaboration, turn-taking and JVA detect active from no collaboration, but not from leadership collaboration. However, the speaking-time distribution index differentiates active collaboration from both leadership and no collaboration segments. Combining turn-taking with overlap frequency, JVA events, and the STD index might discriminate active from leadership collaboration. High turn-taking with overlap frequency and JVA combined with unequal STD might suggest leadership collaboration, while equal STD indicates active collaboration.

More research with larger samples and equally distributed groups is needed to confirm these findings. In our method, only segments with 3 or more annotators agreeing were chosen, to evaluate the relevance of the indicators on more stereotyped collaboration scenarios. The next step is analyzing complex collaboration segments (where annotators did not agree) to identify items that may be ambiguous for collaboration assessment, i.e., items for which analysis in combination with other indicators might be relevant to determine their implication in collaboration and identify how to interpret them. The relevance of the time window and coding scheme chosen could also be discussed given the type of collaboration and the complexity of the collaborative behaviors they allow to measure.

Future research should include more multimodal features to capture collaboration levels' nuances. Analyzing speech can offer insights into content exchanges (e.g., dialogue related to spatial references, task-related content, and information non-relative to the task) and might be effectively paired with JVA data. Considering audio features, physical activity, and physiological signals can further assess synchrony between participants. Because our study focus on dyadic interactions, future work should explore collaboration in larger groups.

# 7 CONCLUSION

Based on recent studies [14, 17], this ongoing research offers an initial approach to give a status of some collaboration indicators and have a real-time feedback on the current level of collaboration between two users during an activity. Our exploratory study revealed that both the speaking time distribution and the number of turn-taking with overlap and JVA can differentiate between active collaboration and no collaboration. However, they are insufficient for discerning more complex collaborative situations. In a future study, we plan to combine these indicators and introduce additional measures for a clearer representation of collaboration quality.

# ACKNOWLEDGMENTS

This work was conducted as part of a thesis, supported by ANR Missives and a grant from the Pays de la Loire region.
REFERENCES

[1] Henny Admoni and Brian Scassellati. 2017. Social Eye Gaze in Human-Robot Interaction: A Review. *Journal of Human-Robot Interaction* 6, 1 (March 2017), 25. https://doi.org/10.5898/JHRL.6.1.Admoni

[2] Najood AlGhamdi and Wadee Alhalabi. 2019. Fixation Detection with Ray-casting in Immersive Virtual Reality. *International Journal of Advanced Computer Science and Applications* 10 (Jan. 2019). https://doi.org/10.14569/IJACSA.2019.0100710

[3] Khaled Bachour, Frederic Kaplan, and Pierre Dillenbourg. 2010. An Interactive Table for Supporting Participation Balance in Face-to-Face Collaborative Learning. *IEEE Transactions on Learning Technologies* 3, 3 (July 2010), 203–213. https://doi.org/10.1109/TLT.2010.18 Conference Name: IEEE Transactions on Learning Technologies.

[4] Nikoletta Bassiou, Andreas Tsiartas, Jennifer Smith, Harry Bratt, Colleen Richey, Elizabeth Shriberg, Cynthia D'Angelo, and Nonye Alozie. 2016. Privacy-Preserving Speech Analytics for Automatic Assessment of Student Collaboration. In *Interspeech 2016*. ISCA, 888–892. https://doi.org/10.21437/Interspeech.2016-1569

[5] Dan Bohus, Sean Andrist, Ashley Feniello, Nick Saw, Mihai Jalobeanu, Patrick Sweeney, Anne Loomis Thompson, and Eric Horvitz. 2021. Platform for Situated Intelligence. https://doi.org/10.48550/arXiv.2103.15975 arXiv:2103.15975 [cs].

[6] Yong Dich, Joseph Reilly, and Bertrand Schneider. 2018. Using Physiological Synchrony as an Indicator of Collaboration Quality, Task Performance and Learning. In *Artificial Intelligence in Education (Lecture Notes in Computer Science)*. Springer International Publishing, Cham, 98–110. https://doi.org/10.1007/978-3-319-93843-1_8

[7] Pierre Dillenbourg. 1999. What do you mean by collaborative learning?

[8] Jaebok Kim, Khiet P. Truong, Vicky Charisi, Cristina Zaga, Manja Lohse, Dirk Heylen, and Vanessa Evers. 2015. Vocal turn-taking patterns in groups of children performing collaborative tasks: an exploratory study. In *Interspeech 2015*. ISCA, 1645–1649. https://doi.org/10.21437/Interspeech.2015-380

[9] Taemie Kim, Agnes Chang, Lindsey Holland, and Alex Sandy Pentland. 2008. Meeting mediator: enhancing group collaborationusing sociometric feedback. In *Proceedings of the 2008 ACM conference on Computer supported cooperative work*. ACM, San Diego CA USA, 457–466. https://doi.org/10.1145/1460563.1460636

[10] Nichola Lubold and Heather Pon-Barry. 2014. *Acoustic-Prosodic Entrainment and Rapport in Collaborative Learning Dialogues*. https://doi.org/10.1145/2666633.

2666635

[11] Margaret L. McLaughlin and Michael J. Cody. 1982. Awkward silences: Behavioral antecedents and consequences of the conversational lapse. *Human Communication Research* 8 (1982), 299–316. https://doi.org/10.1111/j.1468-2958.1982.tb00669.x Place: United Kingdom Publisher: Oxford University Press.

[12] Anne Meier, Hans Spada, and Nikol Rummel. 2007. A rating scheme for assessing the quality of computer-supported collaboration processes. *International Journal of Computer-Supported Collaborative Learning* 2, 1 (March 2007), 63–86. https://doi.org/10.1007/s11412-006-9005-x

[13] Jason C. Pradarelli, Steven Yule, and Douglas S. Smink. 2020. Evaluating Non-Technical Skills in Surgery. In *Health Services Research*, Justin B. Dimick and Carrie C. Lubitz (Eds.). Springer International Publishing, Cham, 125–135. https://doi.org/10.1007/978-3-030-28357-5_12

[14] Sambit Praharaj, Maren Scheffel, Hendrik Drachsler, and Marcus Specht. 2021. Literature Review on Co-Located Collaboration Modeling Using Multimodal Learning Analytics—Can We Go the Whole Nine Yards? *IEEE Transactions on Learning Technologies* 14, 3 (June 2021), 367–385. https://doi.org/10.1109/TLT.2021.3097766

[15] Sambit Praharaj, Maren Scheffel, Marcel Schmitz, Marcus Specht, and Hendrik Drachsler. 2022. Towards Collaborative Convergence: Quantifying Collaboration Quality with Automated Co-located Collaboration Analytics. In *LAK22: 12th International Learning Analytics and Knowledge Conference*. ACM, Online USA, 358–369. https://doi.org/10.1145/3506860.3506922

[16] Daniel C. Richardson and Rick Dale. 2005. Looking To Understand: The Coupling Between Speakers' and Listeners' Eye Movements and Its Relationship to Discourse Comprehension. *Cognitive Science* 29, 6 (Nov. 2005), 1045–1060. https://doi.org/10.1207/s15516709cog0000_29

[17] Bertrand Schneider, Gahyun Sung, Edwin Chng, and Stephanie Yang. 2021. How Can High-Frequency Sensors Capture Collaboration? A Review of the Empirical Links between Multimodal Metrics and Collaborative Constructs. *Sensors* 21, 24 (Dec. 2021), 8185. https://doi.org/10.3390/s21248185

[18] Bertrand Schneider, Marcelo Worsley, and Roberto Martinez-Maldonado. 2021. Gesture and Gaze: Multimodal Data in Dyadic Interactions. In *International Handbook of Computer-Supported Collaborative Learning*, Ulrike Cress, Carolyn Rosé, Alyssa Friend Wise, and Jun Oshima (Eds.). Springer International Publishing, Cham, 625–641. https://doi.org/10.1007/978-3-030-65291-3_34