# Measuring Cues of Leadership, Cohesion, and Fluidity in Joint Full-Body Movement to Support Embodied Interaction Design: A Pilot Study

Giorgio Gnecco, Antonio Camurri, Cora Gasparotti, Eleonora Ceccaldi, Gualtiero Volpe, Benoît Bardy, Marta Bieńkiewicz, Stefan Janaqi

<figure><img src="image_1.png" /></figure>

## To cite this version:

Giorgio Gnecco, Antonio Camurri, Cora Gasparotti, Eleonora Ceccaldi, Gualtiero Volpe, et al. Measuring Cues of Leadership, Cohesion, and Fluidity in Joint Full-Body Movement to Support Embodied Interaction Design: A Pilot Study. Human Behavior and Emerging Technologies, 2024, 2024 (1), pp.1636854. 10.1155/2024/1636854. hal-04623298

HAL Id: hal-04623298

https://imt-mines-ales.hal.science/hal-04623298

Submitted on 25 Jun 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Measuring Cues of Leadership, Cohesion, and Fluidity in Joint Full-Body Movement to Support Embodied Interaction Design: A Pilot Study

Giorgio Gnecco, Antonio Camurri, Cora Gasparotti, Eleonora Ceccaldi, Gualtiero Volpe, Benoît Bardy, Marta Bieńkiewicz, Stefan Janaqi

<figure><img src="image_1.png" /></figure>

## To cite this version:

Giorgio Gnecco, Antonio Camurri, Cora Gasparotti, Eleonora Ceccaldi, Gualtiero Volpe, et al. Measuring Cues of Leadership, Cohesion, and Fluidity in Joint Full-Body Movement to Support Embodied Interaction Design: A Pilot Study. Human Behavior and Emerging Technologies, 2024, 2024 (1), pp.1636854. 10.1155/2024/1636854. hal-04623298

HAL Id: hal-04623298

https://imt-mines-ales.hal.science/hal-04623298v1

Submitted on 25 Jun 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
WILEY

Research Article

# Measuring Cues of Leadership, Cohesion, and Fluidity in Joint Full-Body Movement to Support Embodied Interaction Design: A Pilot Study

Giorgio Gnecco<sup>1</sup>,<sup>2</sup> Antonio Camurri<sup>1</sup>,<sup>2</sup> Cora Gasparotti<sup>1</sup>,<sup>2</sup> Eleonora Ceccaldi<sup>1</sup>,<sup>2</sup> Gualtiero Volpe<sup>1</sup>,<sup>2</sup> Benoît Bardy<sup>1</sup>,<sup>3</sup> Marta Bieńkiewicz<sup>1</sup>,<sup>3</sup> and Stefan Janaqi<sup>1</sup>,<sup>4</sup>

Correspondence should be addressed to Giorgio Gnecco; giorgio.gnecco@imtlucca.it

Received 16 April 2024; Revised 20 May 2024; Accepted 4 June 2024

Academic Editor: Amine Moussa

Copyright © 2024 Giorgio Gnecco et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Innovative applications of human movement analysis, for example, for mitigating/slowing down certain pathological conditions, have recently emerged from the modeling and automated measurement of full-body expressive midlevel individual and group movement qualities, at a higher complexity level than movement qualities derived directly from physical signals, still not characterizing any gesture in a specific way. More in general, the availability of automated analysis techniques of midlevel expressive movement qualities can contribute to interaction design incorporating body-based performance practices inspired by artistic theories in dance and music. This work investigates how such practices and techniques can support embodied interaction design by enabling automated measuring of cues of leadership, cohesion, and fluidity in full-body movement in group settings. In particular, the dance-inspired scientific approach, the data collection protocol, and the analysis techniques adopted for assessing movement qualities connected to leadership and cohesion within the group and fluidity of the dancers’ full-body movement are described. Finally, future developments of this research are outlined.

**Keywords:** embodied movement-based interaction design; group scenario; midlevel expressive full-body movement qualities; multimodal data acquisition

## 1. Introduction

Emerging technologies are pushing a disruptive change of paradigm in the design of interactive digital tools and services, involving the users to engage in full-body, physical, artistic, and emotional experiences. The recently proposed “somatic turn” within the interaction design and human-computer interaction (HCI) communities [1–3], as well as art-inspired approaches to embodied interaction design (see, e.g., [4, 5]), bears witness of these methodological as well as technological changes of paradigm. In this scenario, the investigation of automated measurement techniques of mid- and high-level movement qualities—which are at

higher levels of complexity than low-level qualities, derived directly from physical signals—is leading to advances in embodied interaction design and to innovative applications of human movement analysis: for example, in mitigating or slowing down certain pathological conditions, with a particular focus on degenerative ones [6, 7].

Research on the development of computational methods for the analysis of human movement benefits from a fruitful interaction between various disciplines, such as biomechanics, neuroscience, experimental psychology, and arts and humanities [8]. In [9], an embodied approach and interactive system were proposed to support physical exercises at the hospital for people with Parkinson’s disease, based on

<footnote n="1"><sup>1</sup>AXES Research Unit, IMT School for Advanced Studies, Lucca, Italy</footnote>

<footnote n="2"><sup>2</sup>DIBRIS, University of Genoa, Genoa, Italy</footnote>

<footnote n="3"><sup>3</sup>EuroMov Digital Health in Motion, University of Montpellier, Montpellier, France</footnote>

<footnote n="4"><sup>4</sup>EuroMov Digital Health in Motion, IMT Mines Alès, Alès, France</footnote>
WILEY

Research Article

# Measuring Cues of Leadership, Cohesion, and Fluidity in Joint Full-Body Movement to Support Embodied Interaction Design: A Pilot Study

Giorgio Gnecco<sup>1</sup>,<sup>2</sup> Antonio Camurri<sup>1</sup>,<sup>2</sup> Cora Gasparotti<sup>1</sup>,<sup>2</sup> Eleonora Ceccaldi<sup>1</sup>,<sup>2</sup> Gualtiero Volpe<sup>1</sup>,<sup>2</sup> Benoît Bardy<sup>1</sup>,<sup>3</sup> Marta Bieńkiewicz<sup>1</sup>,<sup>3</sup> and Stefan Janaqi<sup>1</sup>,<sup>4</sup>

Correspondence should be addressed to Giorgio Gnecco; giorgio.gnecco@imtlucca.it

Received 16 April 2024; Revised 20 May 2024; Accepted 4 June 2024

Academic Editor: Amine Moussa

Copyright © 2024 Giorgio Gnecco et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Innovative applications of human movement analysis, for example, for mitigating/slowing down certain pathological conditions, have recently emerged from the modeling and automated measurement of full-body expressive midlevel individual and group movement qualities, at a higher complexity level than movement qualities derived directly from physical signals, still not characterizing any gesture in a specific way. More in general, the availability of automated analysis techniques of midlevel expressive movement qualities can contribute to interaction design incorporating body-based performance practices inspired by artistic theories in dance and music. This work investigates how such practices and techniques can support embodied interaction design by enabling automated measuring of cues of leadership, cohesion, and fluidity in full-body movement in group settings. In particular, the dance-inspired scientific approach, the data collection protocol, and the analysis techniques adopted for assessing movement qualities connected to leadership and cohesion within the group and fluidity of the dancers' full-body movement are described. Finally, future developments of this research are outlined.

**Keywords:** embodied movement-based interaction design; group scenario; midlevel expressive full-body movement qualities; multimodal data acquisition

## 1. Introduction

Emerging technologies are pushing a disruptive change of paradigm in the design of interactive digital tools and services, involving the users to engage in full-body, physical, artistic, and emotional experiences. The recently proposed "somatic turn" within the interaction design and human-computer interaction (HCI) communities [1-3], as well as art-inspired approaches to embodied interaction design (see, e.g., [4, 5]), bears witness of these methodological as well as technological changes of paradigm. In this scenario, the investigation of automated measurement techniques of mid- and high-level movement qualities—which are at

higher levels of complexity than low-level qualities, derived directly from physical signals—is leading to advances in embodied interaction design and to innovative applications of human movement analysis: for example, in mitigating or slowing down certain pathological conditions, with a particular focus on degenerative ones [6, 7].

Research on the development of computational methods for the analysis of human movement benefits from a fruitful interaction between various disciplines, such as biomechanics, neuroscience, experimental psychology, and arts and humanities [8]. In [9], an embodied approach and interactive system were proposed to support physical exercises at the hospital for people with Parkinson's disease, based on

<footnote n="1"><sup>1</sup>AXES Research Unit, IMT School for Advanced Studies, Lucca, Italy</footnote>

<footnote n="2"><sup>2</sup>DIBRIS, University of Genoa, Genoa, Italy</footnote>

<footnote n="3"><sup>3</sup>EuroMov Digital Health in Motion, University of Montpellier, Montpellier, France</footnote>

<footnote n="4"><sup>4</sup>EuroMov Digital Health in Motion, IMT Mines Alès, Alès, France</footnote>
the “aesthetic resonance” paradigm, according to which the full-body movement of a patient causes the emergence and real-time manipulation of interactive soundscapes and visual art feedback. In more detail, the system proposed in [9] measures the fluidity of the full-body movement of the patient: in case a high degree of fluidity is detected, the patient is enabled to “paint” using her own body as a “brush” on video projections in the surrounding white walls, and at the same time to create and develop naturalistic, layered soundscapes (e.g., a forest, the sea). In this way, physical rehabilitation exercises become engaging experiences, enabling the patient to interactively create and mold aesthetically resonant visual and auditory content. Such experience incentivizes the patient to overcome bradykinesia in Parkinson’s disease as the audio-visual feedback is driven by fluid movements of the participant.

A similar approach was adopted in the exergames designed for supporting the cognitive and motor rehabilitation of children in the ARIEL-Augmented Rehabilitation Joint Lab between Casa Paganini InfoMus-University of Genoa and the Giannina Gaslini Children Hospital.

In the European Horizon 2020 ICT DANCE project, the same group at the University of Genoa collaborated with choreographer Virgilio Sieni in a series of public events involving over 120 citizens (nondancers) participating in the (re)discovery of their own full-body expressive qualities [10]. In DANCE, the real-time measures and sonifications of *fragility* and *lightness* midlevel expressive qualities of full-body movement of one of the participants (the leader) nudge the other participants to entrain and express the same qualities in their full-body joint movements. Such midlevel qualities are at a higher complexity level than movement qualities derived directly from physical signals (e.g., raw data captured by sensors); still, they are not at such a high level to characterize any gesture in a specific way (i.e., their knowledge, without the additional one of other suitable features, cannot make such a gesture distinguishable from every other gesture).

In [11], interactive sonification of human movement qualities was proposed in rehabilitation from chronic pain disease: the interactive auditory stimuli have the role of enhancing fluid and safe movements and “distracting” the patient from the attention to chronic pain. Interactive sonification of human movement qualities was also developed in the EU Horizon 2020 ICT WHOLODANCE project (http://www.wholodance.eu/), grounded on cross-modal correspondences [12] to support the learning process of full-body movement and actions in sport, dance, music, and other scenarios [13].

In the EU FET PROACTIVE EnTimeMent project, a computational model for the automated detection of the perceived origin of movement (i.e., of the part of the body where movement appears to originate from the perspective of an external observer) was developed [14]. The case of dance movements was considered in the research performed, as a testbed for the possible future application of the developed system to motor and cognitive rehabilitation, for example, after a stroke [15].

The examples above are grounded on an original approach for the modeling of full-body multimodal gestural

expressiveness in individuals as well as in small groups of people. The approach is based on a cross-modal correspondence of movement qualities to visual and sonic stimuli [9], resulting in the “slow mood and aesthetic resonance” paradigm [16], which integrates aesthetic resonance with the rediscovery—by means of interactive technology—of the long time necessary for a deeper, social, embodied experience of artistic content. Originally developed for the interaction with an environment that stimulates active listening, this conceptual framework, including multitemporal scales and representational structures for affect processing, expressiveness, interpersonal synchronization, and entrainment, is structured into several layers, ranging from the acquirement of physical signals to the definition of both mid- and high-level qualities of full-body individual as well as small group movement, based on those signals.

In this scenario, the present work investigates, in the form of a pilot study, techniques for the automated measurement of midlevel movement qualities in a small group. To this aim, we designed a scientific experiment inspired by common technical exercises and performance practice in dance improvisation. Specifically, in our protocol, full-body expressive motion sequences inspired by typical exercises performed by dancers during a joint improvisation aiming at improving their group cohesion are taken into account, in which subtle behavioral changes are introduced endogeneously by the participants. We propose several ways to measure and track automatically midlevel movement qualities that either directly provide information about the motoric behavior of the group (fluidity in our case) or that are cues of functional roles (such as leadership) or emergent states (such as cohesion). Automatic tracking of such qualities can help identify differences between the group behavior and that of its members, and particularly the heterogeneity in the individual movement skills of the group members (connected with their soft skills, see [17, 18]), with possible applications related to the aesthetic resonance paradigm but also in areas different from performing arts (i.e., in fatigue detection, rehabilitation programs, support for hiring decisions). Of particular interest is the case of frailty detection in elderly people [19], which can inspire novel applications of artificial intelligence to healthcare [20]. Finally, the tracking of leadership can be a means to extend the definition of the perceived origin of movement [14] from the individual to the group level.

In the following sections, details about the art-inspired scientific protocol of the experiment and the data collected are described. Then, the results of some preliminary analyses performed on a subset of such data, based on the proposed definitions of midlevel movement qualities, are presented.

## 2. Improvised Group Movement Scenario—Dance-Inspired Embodied Interaction Design

Our scientific approach is inspired by the typical exercises usually performed by dancers before dancing together. We analyzed several exercises and performance practices
the “aesthetic resonance” paradigm, according to which the full-body movement of a patient causes the emergence and real-time manipulation of interactive soundscapes and visual art feedback. In more detail, the system proposed in [9] measures the fluidity of the full-body movement of the patient: in case a high degree of fluidity is detected, the patient is enabled to “paint” using her own body as a “brush” on video projections in the surrounding white walls, and at the same time to create and develop naturalistic, layered soundscapes (e.g., a forest, the sea). In this way, physical rehabilitation exercises become engaging experiences, enabling the patient to interactively create and mold aesthetically resonant visual and auditory content. Such experience incentivizes the patient to overcome bradykinesia in Parkinson’s disease as the audio-visual feedback is driven by fluid movements of the participant.

A similar approach was adopted in the exergames designed for supporting the cognitive and motor rehabilitation of children in the ARIEL-Augmented Rehabilitation Joint Lab between Casa Paganini InfoMus-University of Genoa and the Giannina Gaslini Children Hospital.

In the European Horizon 2020 ICT DANCE project, the same group at the University of Genoa collaborated with choreographer Virgilio Sieni in a series of public events involving over 120 citizens (nondancers) participating in the (re)discovery of their own full-body expressive qualities [10]. In DANCE, the real-time measures and sonifications of fragility and lightness midlevel expressive qualities of full-body movement of one of the participants (the leader) nudge the other participants to entrain and express the same qualities in their full-body joint movements. Such midlevel qualities are at a higher complexity level than movement qualities derived directly from physical signals (e.g., raw data captured by sensors); still, they are not at such a high level to characterize any gesture in a specific way (i.e., their knowledge, without the additional one of other suitable features, cannot make such a gesture distinguishable from every other gesture).

In [11], interactive sonification of human movement qualities was proposed in rehabilitation from chronic pain disease: the interactive auditory stimuli have the role of enhancing fluid and safe movements and “distracting” the patient from the attention to chronic pain. Interactive sonification of human movement qualities was also developed in the EU Horizon 2020 ICT WHOLODANCE project (http://www.wholodance.eu/), grounded on cross-modal correspondences [12] to support the learning process of full-body movement and actions in sport, dance, music, and other scenarios [13].

In the EU FET PROACTIVE EnTimeMent project, a computational model for the automated detection of the perceived origin of movement (i.e., of the part of the body where movement appears to originate from the perspective of an external observer) was developed [14]. The case of dance movements was considered in the research performed, as a testbed for the possible future application of the developed system to motor and cognitive rehabilitation, for example, after a stroke [15].

The examples above are grounded on an original approach for the modeling of full-body multimodal gestural

expressiveness in individuals as well as in small groups of people. The approach is based on a cross-modal correspondence of movement qualities to visual and sonic stimuli [9], resulting in the “slow mood and aesthetic resonance” paradigm [16], which integrates aesthetic resonance with the rediscovery—by means of interactive technology—of the long time necessary for a deeper, social, embodied experience of artistic content. Originally developed for the interaction with an environment that stimulates active listening, this conceptual framework, including multitemporal scales and representational structures for affect processing, expressiveness, interpersonal synchronization, and entrainment, is structured into several layers, ranging from the acquirement of physical signals to the definition of both mid- and high-level qualities of full-body individual as well as small group movement, based on those signals.

In this scenario, the present work investigates, in the form of a pilot study, techniques for the automated measurement of midlevel movement qualities in a small group. To this aim, we designed a scientific experiment inspired by common technical exercises and performance practice in dance improvisation. Specifically, in our protocol, full-body expressive motion sequences inspired by typical exercises performed by dancers during a joint improvisation aiming at improving their group cohesion are taken into account, in which subtle behavioral changes are introduced endogeneously by the participants. We propose several ways to measure and track automatically midlevel movement qualities that either directly provide information about the motoric behavior of the group (fluidity in our case) or that are cues of functional roles (such as leadership) or emergent states (such as cohesion). Automatic tracking of such qualities can help identify differences between the group behavior and that of its members, and particularly the heterogeneity in the individual movement skills of the group members (connected with their soft skills, see [17, 18]), with possible applications related to the aesthetic resonance paradigm but also in areas different from performing arts (i.e., in fatigue detection, rehabilitation programs, support for hiring decisions). Of particular interest is the case of frailty detection in elderly people [19], which can inspire novel applications of artificial intelligence to healthcare [20]. Finally, the tracking of leadership can be a means to extend the definition of the perceived origin of movement [14] from the individual to the group level.

In the following sections, details about the art-inspired scientific protocol of the experiment and the data collected are described. Then, the results of some preliminary analyses performed on a subset of such data, based on the proposed definitions of midlevel movement qualities, are presented.

## 2. Improvised Group Movement Scenario—Dance-Inspired Embodied Interaction Design

Our scientific approach is inspired by the typical exercises usually performed by dancers before dancing together. We analyzed several exercises and performance practices
typically adopted in contemporary dance by professional
dancers to increase their joint performance and raise kin-
esthetic sensitivity and attunement among dancers. Our
team includes a professional choreographer and dancer,
with whom we individuated, also with the help of tech-
niques of embodied sketching (e.g., by performing first-
person full-body joint sessions of nondancers with the
guide of the dancer), the specific exercise to be performed,
then refined/adapted its details for our objectives. This
work led to the development of a protocol for measuring
midlevel full-body movement features in small groups of
nondancers (e.g., synchronization), which are cues of
group phenomena, for example, cohesion and leadership.
The exercise emerging from our iterative embodied inter-
action design process is described in the following.

Three participants are instructed to move in a triangle
formation at a personal space distance [21] from each other
(i.e., their hands may touch if they keep their arms straight
open to each other); this triangle formation must remain
unchanged as much as possible during the exercise. More-
over, participants are allowed to walk and move in every
direction, including backward and lateral steps. The choice
of three as the number of participants is made with the aims
of simplifying the task for the nonexpert participants, reduc-
ing the number of participants, and simplifying the data
acquisition protocol.

One member of the group is initially assigned the role of
leader, and the two others the role of followers. The leader
freely decides the directions and the speed of walk in a free
space of around 10m x 10m and can propose movement
cells to the rest of the group (e.g., arm movements) that each
member must immediately imitate. When the leader inten-
tionally rotates and turns directed to one of the two fol-
lowers, the leadership migrates to the selected follower.
The triangle formation must remain unchanged, but now
with a new leader that decides the direction and movements
of the formation. If the group stops, the leader remains
unchanged from the one there was before the stop, unless
she herself, following the same protocol as above, gives the
leadership to someone else before restarting.

The pilot study was approved by the Ethical Committee
of the University of Genoa (where the data was collected, at
the Casa Paganini InfoMus Research Center). All the partici-
pants declared, when filling in a questionnaire, that they were
older than 18 and that they understood that their data would
be collected pseudoanonymously for research purposes. Data
was processed and analyzed by cross-disciplinary teams of
movement and artificial intelligence scientists at Casa Paga-
nini InfoMus-University of Genoa, EuroMov-University of
Montpellier and IMT Mines Alès, and IMT Lucca, sup-
ported by the partnership with the software company Qua-
lisys, a leading provider of precision Motion Capture
(MoCap) and 3D positioning tracking systems.

Capture of movement qualities was performed in a
multimodal environment operating at different temporal
scales (milliseconds for electromyographic activity, seconds
for MoCap, and minutes for breathing). More specifically,
data was collected in November 2021, using the novel Qua-
lisys markerless MoCap system (https://www.qualisys.com/

features/markerless-motion-capture/), with a frame rate of
50 frames per second. Additionally, microphones were
placed on the participants' headsets with the aim to capture
their breathing signals, whereas Trigno EKG biofeedback
wireless sensors (https://delsys.com/trigno-ekg-biofeedback/)
were used to provide information about their heart rates
and their variability (see Figure 1 for an example).

The collected data for this pilot study refer to four
groups of three participants each (mixed gender, healthy
young adults, age: mean = 22.9 years, std = 1.3 years). All
the participants were novice dancers. This choice was made
as they were representative of a larger population than
expert dancers, having in mind a future extension of the
analysis of midlevel full-body expressive features to various
categories of possible users (e.g., both healthy and frail older
people). The aim of data collection was to explore the emergence
of leadership, the unfolding of behavioral synchronization
(here investigated as a cue for cohesion) of the participants,
and the analysis of the fluidity of their movements.

Leadership roles were designed to be exchanged among
participants fluently and endogenously. Participants were
prompted to look at the participant they would like to give
the leadership to. The duration of each trial was set to 10
min, with the assumption that this was a sufficiently long
duration to bond the group and to study the evolution of
the features that may display its underlying cohesion, that
is, the synchronicity of the movements of its members, as
well as the fluency of their behavior, respiration, and heart
rate. The idea was that the participants would gradually
learn to better predict the movement of the leader and of
other agents ("learn the task"), then move accordingly
together, more in synchronization and more fluidly at the
end of the trial in comparison to its beginning, molding into
a single "organism."

Participants were instructed with a short video recording
describing the purpose of the task, the rules of the task and
of communication allowed during it (e.g., vocal expressions
were forbidden, and the only form of communication
allowed was through gaze and movement intentions, aimed
to give or take leadership). After the recordings, participants
were asked by a researcher at Casa Paganini to fill in a ques-
tionnaire asking for scores regarding the sense of leadership
perceived during the task, based on the work developed
by [22].

Movement analysis was performed based on the two
horizontal coordinates of the centers of mass of the partici-
pants' heads, being their vertical coordinates almost constant
(i.e., approximately equal to their heights). Figure 2(a)
reports an example of data acquisition, whereas Figures 2(b),
2(c), and 2(d) illustrate three successive frames acquired dur-
ing one execution of the task.

3. Measurement of Cues of Leadership, Cohesion, and Fluidity for a Group of Three People

First, each recording was divided into 10 different periods of
equal duration (1 minute each). In each period, for every
<figure><img src="image_3.png" /><figcaption>FIGURE 1: Example of heart rate data for one participant (the R peak refers to the maximum amplitude in the so-called R wave, which is recorded in an electrocardiogram).</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>FIGURE 2: Three participants performing the task addressed in this work. (a) Reconstruction of the three skeletons, based on the Qualisys markerless MoCap system. (b–d) Three successive frames (at seconds 9, 11, and 13) showing an example of execution of the task, for the first group.</figcaption></figure>

time instant $t$, a triangle $T(t)$ was defined, taking the positions of the centers of mass of the heads of the three members of the group (see Figure 3) to demark the triangle corners. The choice of the heads as the body parts to track was also motivated by the rare occurrence of occlusions, as compared to other parts of the body.

Then, several metrics were defined as cues of leadership, cohesion, and fluidity within the group.

**3.1. Leadership.** For each time instant, the direction of movement of the group was given by the velocity vector of the centroid of triangle $T(t)$, going from that point to the
<figure><img src="image_3.png" /><figcaption>FIGURE 1: Example of heart rate data for one participant (the R peak refers to the maximum amplitude in the so-called R wave, which is recorded in an electrocardiogram).</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>FIGURE 2: Three participants performing the task addressed in this work. (a) Reconstruction of the three skeletons, based on the Qualisys markerless MoCap system. (b–d) Three successive frames (at seconds 9, 11, and 13) showing an example of execution of the task, for the first group.</figcaption></figure>

time instant $t$, a triangle $T(t)$ was defined, taking the positions of the centers of mass of the heads of the three members of the group (see Figure 3) to demark the triangle corners. The choice of the heads as the body parts to track was also motivated by the rare occurrence of occlusions, as compared to other parts of the body.

Then, several metrics were defined as cues of leadership, cohesion, and fluidity within the group.

**3.1. Leadership.** For each time instant, the direction of movement of the group was given by the velocity vector of the centroid of triangle $T(t)$, going from that point to the
<figure><img src="image_5.png" /><figcaption>FIGURE 3: (a) Head movements in the XY plane and (b) their trajectories, for group 1.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>FIGURE 4: Calculation of leadership for (a) two successive frames and (b) time evolution of leadership, for group 1. In the first subfigure, the leader is Person 2. The second subfigure illustrates how the leader is detected starting from LeadMetric(t). In this subfigure, the leader is Person 1 approximately from second 4 to second 28.</figcaption></figure>

centroid of triangle $T(t + 1)$. Each centroid was found as the center of mass of the system of three points, by assigning equal mass to each of them. Then, because of the specific task we addressed, we say that the leader was neatly identified as the member of the group whose relative position vector with respect to the centroid formed the smallest angle with the velocity vector of that centroid (i.e., with the “red” arrow in Figure 4, as a positive rescaling of the vector did not change the angle), when the following additional conditions held: this angle was smaller than 90°; the corresponding angles associated with the other members of the group were larger than 90°. For instance, in the case of the first subfigure in Figure 4, the leader was Person 2. Otherwise, when these conditions were not met, we say that the leader was not neatly identified. In practice, to find the leader, the cosine of the angle between the two vectors above (LeadMetric(t)) was computed for every member of the group, and then, it was compared with the threshold zero to check the two additional conditions.

Other metrics related to leadership were, for each period, leadership duration and number of taking turns in leadership (see the second subfigure in Figure 4). These were evaluated by analyzing the time evolution of leadership. The idea is that, as a group got more synchronized and cohesive, participants were expected to achieve balanced numbers and durations of leadership periods.

### 3.2. Cohesion.
Group motor synchronization, here investigated as a cue for cohesion, was operationalized as moving “smoothly” by defining a suitable metric, based on the rate of deformation of the triangular shape associated with the set of agents. The feature was defined only when a leader was neatly identified and did not change between two successive frames. Given any two successive time instants $t$ and $t + 1$ characterized by the presence of a neatly identified and common leader, the “work” $W(t)$ needed to deform the triangle $T(t)$ into the triangle $T(t + 1)$ was evaluated according to the following procedure. First, $T(t)$ was translated to $T(t + 1)$, preserving the leader (i.e., making the two leader’s positions overlap after the translation). Then, the deformation of the triangle was measured by the sum of the lengths of the two “red” segments in Figure 5.

When the deformation was equal to 0, the triangle $T(t)$ was only translated, and the group was in perfect synchronization, which suggests that the group was cohesive. When the deformation was “big,” then synchronization was low. To model this condition, the “work” $W(t)$ was transformed to synchronization by the formula $SYN(t) = \exp(-aW(t))$ (see Figure 6). The value of $a$ was selected in such a way that $\exp(-a \cdot \text{mean}(W)) = 0.5$, that is, as $a = \ln(2)/\text{mean}(W)$. In this way, as being a positive, the resulting index ranged from 0 to 1.
<figure><img src="image_5.png" /><figcaption>FIGURE 3: (a) Head movements in the XY plane and (b) their trajectories, for group 1.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>FIGURE 4: Calculation of leadership for (a) two successive frames and (b) time evolution of leadership, for group 1. In the first subfigure, the leader is Person 2. The second subfigure illustrates how the leader is detected starting from LeadMetric(t). In this subfigure, the leader is Person 1 approximately from second 4 to second 28.</figcaption></figure>

centroid of triangle $T(t + 1)$. Each centroid was found as the center of mass of the system of three points, by assigning equal mass to each of them. Then, because of the specific task we addressed, we say that the leader was neatly identified as the member of the group whose relative position vector with respect to the centroid formed the smallest angle with the velocity vector of that centroid (i.e., with the “red” arrow in Figure 4, as a positive rescaling of the vector did not change the angle), when the following additional conditions held: this angle was smaller than 90°; the corresponding angles associated with the other members of the group were larger than 90°. For instance, in the case of the first subfigure in Figure 4, the leader was Person 2. Otherwise, when these conditions were not met, we say that the leader was not neatly identified. In practice, to find the leader, the cosine of the angle between the two vectors above (LeadMetric(t)) was computed for every member of the group, and then, it was compared with the threshold zero to check the two additional conditions.

Other metrics related to leadership were, for each period, leadership duration and number of taking turns in leadership (see the second subfigure in Figure 4). These were evaluated by analyzing the time evolution of leadership. The idea is that, as a group got more synchronized and cohesive, participants were expected to achieve balanced numbers and durations of leadership periods.

### 3.2. Cohesion.
Group motor synchronization, here investigated as a cue for cohesion, was operationalized as moving “smoothly” by defining a suitable metric, based on the rate of deformation of the triangular shape associated with the set of agents. The feature was defined only when a leader was neatly identified and did not change between two successive frames. Given any two successive time instants $t$ and $t + 1$ characterized by the presence of a neatly identified and common leader, the “work” $W(t)$ needed to deform the triangle $T(t)$ into the triangle $T(t + 1)$ was evaluated according to the following procedure. First, $T(t)$ was translated to $T(t + 1)$, preserving the leader (i.e., making the two leader’s positions overlap after the translation). Then, the deformation of the triangle was measured by the sum of the lengths of the two “red” segments in Figure 5.

When the deformation was equal to 0, the triangle $T(t)$ was only translated, and the group was in perfect synchronization, which suggests that the group was cohesive. When the deformation was “big,” then synchronization was low. To model this condition, the “work” $W(t)$ was transformed to synchronization by the formula $SYN(t) = \exp(-aW(t))$ (see Figure 6). The value of $a$ was selected in such a way that $\exp(-a \cdot \text{mean}(W)) = 0.5$, that is, as $a = \ln(2)/\text{mean}(W)$. In this way, as being a positive, the resulting index ranged from 0 to 1.
<figure><img src="image_7.png" /><figcaption>FIGURE 5: The “work” needed to deform the triangle $T(t)$ into the triangle $T(t+1)$ is the sum of the lengths of the two “red” segments. In the figure, the leader is Person 2.</figcaption></figure>

### 3.3. Fluidity.
When at movement (contrary to the static case, which means perfect synchronization and no fluidity at all), the movement of a group is taken as fluid when the norm $V(t)$ of the velocity of the centroid of the triangle $T(t)$ does not change a lot in a time window of duration $2\tau$, centered at time $t$. This idea is captured by the distribution of the norms of such velocities around their mean value. When the standard deviation is little, this means that the movement stays fluid. Then, the fluidity index was defined as $Fluid(t) = \exp(-\text{std}(V(t-\tau): V(t+\tau)))$: see again Figure 6 for an example of the calculation. In this way, high fluidity corresponds to a value of the fluidity index nearly equal to 1. In the figure, $\tau$ is equal to 1.5 s, corresponding with 75 frames. This choice is made with the aim of finding a suitable trade-off between a sufficiently high temporal resolution (which increases by reducing the duration of the time window) and a sufficiently high reliability of the estimate of the standard deviation (which improves by increasing the duration of the time window).

### 3.4. Time Evolution of the Average Indices for Period.
The evolution of the average for each of the computed cues of leadership, cohesion, and fluidity over the periods is reported in Figure 7, for group 1.

### 3.5. Analysis of the Answers to the Leadership Questionnaire.
This section briefly describes the analysis performed on the answers to the leadership questionnaire. After the session, each player (identified by a color, namely, red, yellow, or magenta) was asked to evaluate 5 statements aimed at assessing the degree of leadership of each team player (including themselves). The statements (also considered in [22]) were the following:

S1. This player decided what to do and how to do it.

S2. This player proposed her ideas to the group.

S3. This player acted as a guide for the group.

S4. This player provided indications to the group.

S5. This player gave the group its goals.

Participants were asked to provide their ratings on a scale from 1 to 6. Then, the following means were computed, giving the same weight (1/5) to the rating of each statement:

- Mean rating of the participant herself as a leader (self).

- Mean rating of the participant as a leader as assessed by the other participants (others).

- Mean rating of the participant as a leader as assessed by the other participants and by the participant herself (overall).

Results for one group are reported in Table 1.

Overall, the scores suggest an equal distribution of leadership across participants during the whole session.

## 4. Possible Extensions

This work describes an approach of embodied interaction design and a pilot study, to support the design of interactive full-body applications involving small groups of people. The protocol was inspired by dance performance practices and is grounded on midlevel movement qualities that either characterize the motoric behavior of the group (fluidity) or are cues of functional roles (leadership) or emergent states (cohesion) in a small group of people performing everyday movements. We presented preliminary results of analyses performed on a subset of the acquired data, based on the proposed definitions of these qualities.

The following are possible developments of this research:

- By extending this pilot study to groups of people with different movement capabilities, one could investigate, for example, the influence an expert dancer exerts on novice dancers by propagating her own movement patterns (individual motor signatures (IMSs) [23]), for stimulating the emergence of a group movement pattern (group motor signature (GMS) [24]). A particularly relevant aspect would concern the evaluation of the time needed for the emergence of the GMS. This study could also involve the analysis of the midlevel qualities considered in the present work.

- Variations of the definition of fluidity and inclusion of additional cues of leadership and cohesion could be considered, for example, based on a larger group size and/or a larger number of features to capture additional kinds of movements (for instance, rotations and arm movements).

- A deeper and multimodal analysis of the available data could be performed, relating physical signals measured by means of different data acquisition techniques and characterized by distinct time scales [25, 26].

- A dataset with an increased degree of multimodality could be acquired, with the aim of assessing, for example, the influence of different background soundscapes (e.g., intrusive/annoying soundscape vs. naturalistic/ pleasant soundscape) on the selected midlevel movement features, and their relationship with soundscape-induced synchronization [27].

- Suitable cooperative game-theoretical models could be developed, based on movement features, to evaluate the importance of each member of the group to determine movement fluidity, degree of cohesion, and strength of leadership. Possible measures of importance are the Shapley value and its generalizations
<figure><img src="image_7.png" /><figcaption>FIGURE 5: The “work” needed to deform the triangle $T(t)$ into the triangle $T(t+1)$ is the sum of the lengths of the two “red” segments. In the figure, the leader is Person 2.</figcaption></figure>

### 3.3. Fluidity.
When at movement (contrary to the static case, which means perfect synchronization and no fluidity at all), the movement of a group is taken as fluid when the norm $V(t)$ of the velocity of the centroid of the triangle $T(t)$ does not change a lot in a time window of duration $2\tau$, centered at time $t$. This idea is captured by the distribution of the norms of such velocities around their mean value. When the standard deviation is little, this means that the movement stays fluid. Then, the fluidity index was defined as $Fluid(t) = \exp(-\text{std}(V(t-\tau): V(t+\tau)))$: see again Figure 6 for an example of the calculation. In this way, high fluidity corresponds to a value of the fluidity index nearly equal to 1. In the figure, $\tau$ is equal to 1.5 s, corresponding with 75 frames. This choice is made with the aim of finding a suitable trade-off between a sufficiently high temporal resolution (which increases by reducing the duration of the time window) and a sufficiently high reliability of the estimate of the standard deviation (which improves by increasing the duration of the time window).

### 3.4. Time Evolution of the Average Indices for Period.
The evolution of the average for each of the computed cues of leadership, cohesion, and fluidity over the periods is reported in Figure 7, for group 1.

### 3.5. Analysis of the Answers to the Leadership Questionnaire.
This section briefly describes the analysis performed on the answers to the leadership questionnaire. After the session, each player (identified by a color, namely, red, yellow, or magenta) was asked to evaluate 5 statements aimed at assessing the degree of leadership of each team player (including themselves). The statements (also considered in [22]) were the following:

S1. This player decided what to do and how to do it.

S2. This player proposed her ideas to the group.

S3. This player acted as a guide for the group.

S4. This player provided indications to the group.

S5. This player gave the group its goals.

Participants were asked to provide their ratings on a scale from 1 to 6. Then, the following means were computed, giving the same weight (1/5) to the rating of each statement:

- Mean rating of the participant herself as a leader (self).

- Mean rating of the participant as a leader as assessed by the other participants (others).

- Mean rating of the participant as a leader as assessed by the other participants and by the participant herself (overall).

Results for one group are reported in Table 1.

Overall, the scores suggest an equal distribution of leadership across participants during the whole session.

## 4. Possible Extensions

This work describes an approach of embodied interaction design and a pilot study, to support the design of interactive full-body applications involving small groups of people. The protocol was inspired by dance performance practices and is grounded on midlevel movement qualities that either characterize the motoric behavior of the group (fluidity) or are cues of functional roles (leadership) or emergent states (cohesion) in a small group of people performing everyday movements. We presented preliminary results of analyses performed on a subset of the acquired data, based on the proposed definitions of these qualities.

The following are possible developments of this research:

- By extending this pilot study to groups of people with different movement capabilities, one could investigate, for example, the influence an expert dancer exerts on novice dancers by propagating her own movement patterns (individual motor signatures (IMSs) [23]), for stimulating the emergence of a group movement pattern (group motor signature (GMS) [24]). A particularly relevant aspect would concern the evaluation of the time needed for the emergence of the GMS. This study could also involve the analysis of the midlevel qualities considered in the present work.

- Variations of the definition of fluidity and inclusion of additional cues of leadership and cohesion could be considered, for example, based on a larger group size and/or a larger number of features to capture additional kinds of movements (for instance, rotations and arm movements).

- A deeper and multimodal analysis of the available data could be performed, relating physical signals measured by means of different data acquisition techniques and characterized by distinct time scales [25, 26].

- A dataset with an increased degree of multimodality could be acquired, with the aim of assessing, for example, the influence of different background soundscapes (e.g., intrusive/annoying soundscape vs. naturalistic/ pleasant soundscape) on the selected midlevel movement features, and their relationship with soundscape-induced synchronization [27].

- Suitable cooperative game-theoretical models could be developed, based on movement features, to evaluate the importance of each member of the group to determine movement fluidity, degree of cohesion, and strength of leadership. Possible measures of importance are the Shapley value and its generalizations
<figure><img src="image_8.png" /><figcaption>FIGURE 6: Work for deformation, synchronization, and fluidity index, for group 1.</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>FIGURE 7: Evolution of the average for each of the computed cues of leadership, cohesion, and fluidity over periods, for group 1.</figcaption></figure>
<figure><img src="image_8.png" /><figcaption>FIGURE 6: Work for deformation, synchronization, and fluidity index, for group 1.</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>FIGURE 7: Evolution of the average for each of the computed cues of leadership, cohesion, and fluidity over periods, for group 1.</figcaption></figure>
<table><caption>TABLE 1: Results of the analysis of the answers to the leadership questionnaire, for group 1.</caption><thead><tr><th>Mean rating</th><th>Red player (P1)</th><th>Yellow player (P2)</th><th>Magenta player (P3)</th></tr></thead><tbody><tr><td>Self</td><td>5.2</td><td>5.6</td><td>5.0</td></tr><tr><td>Others</td><td>5.4</td><td>5.1</td><td>5.6</td></tr><tr><td>Overall</td><td>5.3</td><td>5.3</td><td>5.4</td></tr></tbody></table>

[28, 29], which were recently applied in various contexts such as the analysis of human movement [14], human-robot interaction [30], sports team performance [31, 32], transportation networks [33], and international trade [34]. Beside MoCap, eye-tracking techniques [35] could be used to define movement-related features (e.g., gaze direction) for novel such cooperative game-theoretical models.

- Additional features could be included in the analysis, starting from the directions of the heads of the group members: this would provide information about who was seeing whom in a frame, taking such directions are proxies of gaze direction (alternatively, one could directly use eye-tracking techniques). On the one hand, this would allow one to detect a possible error in the neat identification of the leader based on the approach delineated in this work, for example, if in a frame such a person turned out to be not seen by another group member. On the other hand, verifying the occurrence of a small fraction of frames (compared to the total number of frames) containing this kind of error would constitute a reasonableness check of the proposed approach to identify the leader. Moreover, a ground truth for leadership (at least at a high level of the analysis) could be provided by a further study of the answers to the leadership questionnaire.

Finally, an alternative analysis of leadership is reported in the Appendix, based on block-wise Granger causality [36], i.e., loosely speaking, on a measure of how the leader anticipate her followers with her movements. Making a comparison with the analysis proposed in Section 3.1 could allow one to detect frames in which leadership is particularly “strong”.

# Appendix

## Alternative Measurement of Leadership Based on Block-Wise Granger Causality

A second way of analyzing leadership was considered, based on block-wise Granger causality for vector-valued time series (in discrete time). Loosely speaking, a vector-valued time series $X(t)$ “block-wise Granger causes” a second vector-valued time series $Y(t)$ if the prediction of the current value of $Y(t)$ based on a suitable regression model and on past values of $Y(t)$ is improved significantly by the inclusion in that model of past values of $X(t)$, together with the inclusion of past values of $Y(t)$ [36]. In other words, to assess

block-wise Granger causality, one needs to check that the coefficients in the latter model, associated with past values of $X(t)$, are significantly different from 0. This occurs when a suitable block-wise Granger causality test (e.g., chi-square) is passed, which is implemented, for instance, by the “gctest” MATLAB function. Classical Granger causality [37] is obtained when $X(t)$ and $Y(t)$ are replaced by univariate time series $x(t)$ and $y(t)$. In the context of movement analysis, classical Granger causality was applied by [38] to investigate musicians’ movements in an orchestra. To conclude, it is worth mentioning that classical Granger causality and its generalizations can also be defined starting from more sophisticated regression models, such as multivariate Gaussian process regression [39–41].

In the present application of block-wise Granger causality to movement analysis, each of the vectors $X(t)$ and $Y(t)$ was replaced by the 2D relative position vector (with respect to the centroid of the triangle $T(t)$) of one distinct individual of the group. The relative position was chosen (in place of the absolute position) to enforce, as much as possible, the stationarity of the time series in each time window of the analysis (details on the time windows are reported in the next paragraph). All the time series were subsampled, making any two successive frames be separated by 0.4 s. This was motivated by the relatively high time scale needed for prediction improvement in the successive frame, due to the inclusion of features of the candidate leader in the regression model. We say that the leader was neatly identified (using the block-wise Granger causality approach) as the unique person whose 2D relative position vector time series passed the largest number (two or one) of block-wise Granger causality tests—the first one with respect to the 2D relative position vector time series of one of the other 2 persons, and the second one with respect to the 2D relative position vector time series of the remaining person—in case each of the latter 2 vector time series passed a strictly smaller number (one or zero) of similarly defined block-wise Granger causality tests. Otherwise, when the conditions above were not met, we say that the leader was not neatly identified. The reason for which it was not required for the leader to always pass both block-wise Granger causality tests is that, in some frames, one of the participants was not moving, behaving as the center of rotation for a rotational movement guided, indeed, by the leader.

The details of the block-wise Granger causality tests performed are as follows. For each pair of persons, a stationary VAR(1) model (i.e., a stationary vector auto regressive model with $p = 1$ time lags) was estimated for the whole vector of four associated scalar time series (the 2D relative position components for each person). Then, starting from this model, the block-wise Granger causality test was performed, specifying as “causes” the first 2D time series (associated with one person), and as “effects” the second 2D time series (associated with the second person). A significance level $\alpha = 0.05$ was used for each test. Its value was justified by the desire to reject the null hypothesis of no block-wise Granger causality only in case of quite strong evidence against it. In each repetition, all the various time series were evaluated on a time window of size $W = 20$ s. Consecutive
<table><caption>TABLE 1: Results of the analysis of the answers to the leadership questionnaire, for group 1.</caption><thead><tr><th>Mean rating</th><th>Red player (P1)</th><th>Yellow player (P2)</th><th>Magenta player (P3)</th></tr></thead><tbody><tr><td>Self</td><td>5.2</td><td>5.6</td><td>5.0</td></tr><tr><td>Others</td><td>5.4</td><td>5.1</td><td>5.6</td></tr><tr><td>Overall</td><td>5.3</td><td>5.3</td><td>5.4</td></tr></tbody></table>

[28, 29], which were recently applied in various contexts such as the analysis of human movement [14], human-robot interaction [30], sports team performance [31, 32], transportation networks [33], and international trade [34]. Beside MoCap, eye-tracking techniques [35] could be used to define movement-related features (e.g., gaze direction) for novel such cooperative game-theoretical models.

- Additional features could be included in the analysis, starting from the directions of the heads of the group members: this would provide information about who was seeing whom in a frame, taking such directions are proxies of gaze direction (alternatively, one could directly use eye-tracking techniques). On the one hand, this would allow one to detect a possible error in the neat identification of the leader based on the approach delineated in this work, for example, if in a frame such a person turned out to be not seen by another group member. On the other hand, verifying the occurrence of a small fraction of frames (compared to the total number of frames) containing this kind of error would constitute a reasonableness check of the proposed approach to identify the leader. Moreover, a ground truth for leadership (at least at a high level of the analysis) could be provided by a further study of the answers to the leadership questionnaire.

Finally, an alternative analysis of leadership is reported in the Appendix, based on block-wise Granger causality [36], i.e., loosely speaking, on a measure of how the leader anticipate her followers with her movements. Making a comparison with the analysis proposed in Section 3.1 could allow one to detect frames in which leadership is particularly “strong”.

# Appendix

## Alternative Measurement of Leadership Based on Block-Wise Granger Causality

A second way of analyzing leadership was considered, based on block-wise Granger causality for vector-valued time series (in discrete time). Loosely speaking, a vector-valued time series $X(t)$ “block-wise Granger causes” a second vector-valued time series $Y(t)$ if the prediction of the current value of $Y(t)$ based on a suitable regression model and on past values of $Y(t)$ is improved significantly by the inclusion in that model of past values of $X(t)$, together with the inclusion of past values of $Y(t)$ [36]. In other words, to assess

block-wise Granger causality, one needs to check that the coefficients in the latter model, associated with past values of $X(t)$, are significantly different from 0. This occurs when a suitable block-wise Granger causality test (e.g., chi-square) is passed, which is implemented, for instance, by the “gctest” MATLAB function. Classical Granger causality [37] is obtained when $X(t)$ and $Y(t)$ are replaced by univariate time series $x(t)$ and $y(t)$. In the context of movement analysis, classical Granger causality was applied by [38] to investigate musicians’ movements in an orchestra. To conclude, it is worth mentioning that classical Granger causality and its generalizations can also be defined starting from more sophisticated regression models, such as multivariate Gaussian process regression [39–41].

In the present application of block-wise Granger causality to movement analysis, each of the vectors $X(t)$ and $Y(t)$ was replaced by the 2D relative position vector (with respect to the centroid of the triangle $T(t)$) of one distinct individual of the group. The relative position was chosen (in place of the absolute position) to enforce, as much as possible, the stationarity of the time series in each time window of the analysis (details on the time windows are reported in the next paragraph). All the time series were subsampled, making any two successive frames be separated by 0.4 s. This was motivated by the relatively high time scale needed for prediction improvement in the successive frame, due to the inclusion of features of the candidate leader in the regression model. We say that the leader was neatly identified (using the block-wise Granger causality approach) as the unique person whose 2D relative position vector time series passed the largest number (two or one) of block-wise Granger causality tests—the first one with respect to the 2D relative position vector time series of one of the other 2 persons, and the second one with respect to the 2D relative position vector time series of the remaining person—in case each of the latter 2 vector time series passed a strictly smaller number (one or zero) of similarly defined block-wise Granger causality tests. Otherwise, when the conditions above were not met, we say that the leader was not neatly identified. The reason for which it was not required for the leader to always pass both block-wise Granger causality tests is that, in some frames, one of the participants was not moving, behaving as the center of rotation for a rotational movement guided, indeed, by the leader.

The details of the block-wise Granger causality tests performed are as follows. For each pair of persons, a stationary VAR(1) model (i.e., a stationary vector auto regressive model with $p = 1$ time lags) was estimated for the whole vector of four associated scalar time series (the 2D relative position components for each person). Then, starting from this model, the block-wise Granger causality test was performed, specifying as “causes” the first 2D time series (associated with one person), and as “effects” the second 2D time series (associated with the second person). A significance level $\alpha = 0.05$ was used for each test. Its value was justified by the desire to reject the null hypothesis of no block-wise Granger causality only in case of quite strong evidence against it. In each repetition, all the various time series were evaluated on a time window of size $W = 20$ s. Consecutive
<figure><img src="image_10.png" /><figcaption>FIGURE 8: Evolution of the average block-wise Granger causality-based assessment of leadership over periods, for group 1.</figcaption></figure>

time windows were shifted by 1 s, resulting in an overlap of 19 s. For each block-wise Granger causality analysis, the first observation was used as a pre-sample for the VAR(1) model, whereas the other 24 observations formed the set of response data used to estimate its $4 + 16p = 20$ parameters. The total number of observations was large enough to obtain noninvertible matrices when computing their estimates. Finally, the fraction of time windows in which each person in the group was neatly identified as the leader was calculated.

Preliminary results of the block-wise Granger causality analysis for one group of people are reported in Figure 8. Such results are partially in agreement with those obtained in the first subfigure of Figure 7. Discrepancies could be explained by the different way of measuring leadership (e.g., evaluated on a time window vs. computed frame by frame). Finally, in the block-wise Granger causality approach, the large fraction of time windows with no leader neatly identified could be explained by the presence of a transition of leadership within the same time window.

## Data Availability Statement

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

## Conflicts of Interest

The authors declare no conflicts of interest.

## Author Contributions

The authors contributed equally to the preparation of the manuscript.

## Funding

The work was supported by the FET PROACTIVE project "EnTimeMent," funded by the European Union (project no. 824160); the Galileo 2021 project "Automatic Movement Analysis Techniques for Applications in Cognitive/Motor Rehabilitation," funded by the Italo-French University/ French-Italian University (UIF/UFI) (project no. G21_89); the PRIN 2020 project "MAHATMA – Multiscale Analysis of Human and Artificial Trajectories: Models and Applications," funded by the European Union–Next Generation EU program (CUP: D53D23008790006); the PRIN PNRR 2022 project "MOTUS – Automated Analysis and Prediction of Human Movement Qualities," funded by the European Union–Next Generation EU program (CUP: D53D23017470001); and the ICT STARTS project "ReSilence," funded by the European Union's Horizon Europe research and innovation programme (grant agreement no. 101070278).

## Acknowledgments

We thank Fredrik Müller, Erwin Schoonderwaldt, Daniel Bassett, and Sten Remmelg from Qualisys for their precious support.

## References

[1] K. Höök, *Designing with the Body: Somaesthetic Interaction Design*, MIT Press, Cambridge, United States of America, 2018.

[2] L. Loke and T. Schiphorst, "The somatic turn in human-computer interaction," *Interactions*, vol. 25, no. 5, pp. 54-5863, 2018.

[3] T. Schiphorst, L. Loke, and K. Höök, "Designing for sensory appreciation: cultivating somatic approaches to experience design," in *Extended Abstracts of the 2020 CHI conference on human factors in computing systems (CHI EA'20)*, pp. 1-4, Honolulu, United States of America, 2020.

[4] E. F. Duarte and M. C. C. Baranauskas, "Revisiting interactive art from an interaction design perspective: opening a research agenda," in *Proceedings of the 17th Brazilian Symposium on Human Factors in Computing Systems (IHC'18)*, pp. 1-10, Belém, Brazil, 2018.

[5] W. J. Mitchell, A. S. Inouye, and M. S. Blumenthal, *Beyond Productivity: Information Technology, Innovation, and Creativity*, The National Academies Press, Washington, United States of America, 2003.

[6] A. Cicerchia, A. Rossi Ghiglione, and C. Seia, *Welfare Culture*, Atlante della cultura Treccani, Rome, Italy, 2020.

[7] C. Buckley, L. Alcock, R. McArdle et al., "The role of movement analysis in diagnosing and monitoring neurodegenerative conditions: insights from gait and postural control," *Brain Sciences*, vol. 9, no. 2, p. 21, 2019.

[8] N. Chatzistefani and J. Heron, "A pedagogy of movement: interdisciplinary approaches to human motion," *Exchanges: The Interdisciplinary Research Journal*, vol. 5, no. 1, pp. 98-105, 2017.

[9] A. Camurri, B. Mazzarino, G. Volpe, P. Morasso, and C. Re, "Application of multimedia techniques in the physical
<figure><img src="image_10.png" /><figcaption>FIGURE 8: Evolution of the average block-wise Granger causality-based assessment of leadership over periods, for group 1.</figcaption></figure>

time windows were shifted by 1 s, resulting in an overlap of 19 s. For each block-wise Granger causality analysis, the first observation was used as a pre-sample for the VAR(1) model, whereas the other 24 observations formed the set of response data used to estimate its $4 + 16p = 20$ parameters. The total number of observations was large enough to obtain noninvertible matrices when computing their estimates. Finally, the fraction of time windows in which each person in the group was neatly identified as the leader was calculated.

Preliminary results of the block-wise Granger causality analysis for one group of people are reported in Figure 8. Such results are partially in agreement with those obtained in the first subfigure of Figure 7. Discrepancies could be explained by the different way of measuring leadership (e.g., evaluated on a time window vs. computed frame by frame). Finally, in the block-wise Granger causality approach, the large fraction of time windows with no leader neatly identified could be explained by the presence of a transition of leadership within the same time window.

## Data Availability Statement

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

## Conflicts of Interest

The authors declare no conflicts of interest.

## Author Contributions

The authors contributed equally to the preparation of the manuscript.

## Funding

The work was supported by the FET PROACTIVE project "EnTimeMent," funded by the European Union (project no. 824160); the Galileo 2021 project "Automatic Movement Analysis Techniques for Applications in Cognitive/Motor Rehabilitation," funded by the Italo-French University/ French-Italian University (UIF/UFI) (project no. G21_89); the PRIN 2020 project "MAHATMA - Multiscale Analysis of Human and Artificial Trajectories: Models and Applications," funded by the European Union-Next Generation EU program (CUP: D53D23008790006); the PRIN PNRR 2022 project "MOTUS - Automated Analysis and Prediction of Human Movement Qualities," funded by the European Union-Next Generation EU program (CUP: D53D23017470001); and the ICT STARTS project "ReSilence," funded by the European Union's Horizon Europe research and innovation programme (grant agreement no. 101070278).

## Acknowledgments

We thank Fredrik Müller, Erwin Schoonderwaldt, Daniel Bassett, and Sten Remmelg from Qualisys for their precious support.

## References

[1] K. Höök, *Designing with the Body: Somaesthetic Interaction Design*, MIT Press, Cambridge, United States of America, 2018.

[2] L. Loke and T. Schiphorst, "The somatic turn in human-computer interaction," *Interactions*, vol. 25, no. 5, pp. 54-5863, 2018.

[3] T. Schiphorst, L. Loke, and K. Höök, "Designing for sensory appreciation: cultivating somatic approaches to experience design," in *Extended Abstracts of the 2020 CHI conference on human factors in computing systems (CHI EA'20)*, pp. 1-4, Honolulu, United States of America, 2020.

[4] E. F. Duarte and M. C. C. Baranauskas, "Revisiting interactive art from an interaction design perspective: opening a research agenda," in *Proceedings of the 17th Brazilian Symposium on Human Factors in Computing Systems (IHC'18)*, pp. 1-10, Belém, Brazil, 2018.

[5] W. J. Mitchell, A. S. Inouye, and M. S. Blumenthal, *Beyond Productivity: Information Technology, Innovation, and Creativity*, The National Academies Press, Washington, United States of America, 2003.

[6] A. Cicerchia, A. Rossi Ghiglione, and C. Seia, *Welfare Culture*, Atlante della cultura Treccani, Rome, Italy, 2020.

[7] C. Buckley, L. Alcock, R. McArdle et al., "The role of movement analysis in diagnosing and monitoring neurodegenerative conditions: insights from gait and postural control," *Brain Sciences*, vol. 9, no. 2, p. 21, 2019.

[8] N. Chatzistefani and J. Heron, "A pedagogy of movement: interdisciplinary approaches to human motion," *Exchanges: The Interdisciplinary Research Journal*, vol. 5, no. 1, pp. 98-105, 2017.

[9] A. Camurri, B. Mazzarino, G. Volpe, P. Morasso, and C. Re, "Application of multimedia techniques in the physical
rehabilitation of Parkinson's patients," *The Journal of Visualization and Computer Animation*, vol. 14, no. 5, pp. 269-278, 2003.

[10] R. Niewiadomski, M. Mancini, A. Cera, S. Piana, C. Canepa, and A. Camurri, "Does embodied training improve the recognition of mid-level expressive movement qualities sonification?," *Journal on Multimodal User Interfaces*, vol. 13, no. 3, pp. 191-203, 2019.

[11] A. Singh, S. Piana, D. Pollarolo et al., "Go-with-the-flow: tracking, analysis and sonification of movement and breathing to build confidence in activity despite chronic pain," *Human-Computer Interaction*, vol. 31, no. 3-4, pp. 335-383, 2016.

[12] C. Spence, "Crossmodal correspondences: a tutorial review," *Attention, Perception, & Psychophysics*, vol. 73, no. 4, pp. 971-995, 2011.

[13] N. Virtanen, K. Tiippana, M. Tervaniemi, H. Poikonen, E. Anttila, and K. Kaseva, "Exploring body consciousness of dancers, athletes, and lightly physically active," *Scientific Reports*, vol. 12, p. 8353, 2022.

[14] K. Kolykhalova, G. Gneco, M. Sanguineti, G. Volpe, and A. Camurri, "Automated analysis of the origin of movement: an approach based on cooperative games on graphs," *IEEE Transactions on Human-Machine Systems*, vol. 50, no. 6, pp. 550-560, 2020.

[15] K. K. A. Bakhti, D. Mottet, N. Schweighofer, J. Froger, and I. Laffont, "Proximal arm non-use when reaching after a stroke," *Journal of Neuro Engineering and Rehabilitation*, vol. 15, 2018.

[16] A. Cera, C. Canepa, N. Ferrari et al., "Interactive sonification of expressive gesture: the DanzArTe-emotion wellbeing Technology project," in *Proceedings of the XXIII Colloquium on Musical Informatics (XXIII CIM)*, pp. 79-86, Ancona, Italy, 2022.

[17] D. R. Laker and J. L. Powell, "The differences between hard and soft skills and their relative impact on training transfer," *Human Resource Development Quarterly*, vol. 22, no. 1, pp. 111-122, 2011.

[18] D. J. Deming, "The growing importance of social skills in the labor market," *Quarterly Journal of Economics*, vol. 123, no. 4, pp. 1593-1640, 2017.

[19] D. L. Vetrano, A. Zucchelli, G. Onder et al., "Frailty detection among primary care older patients through the primary care frailty index (PC-FI)," *Scientific Reports*, vol. 13, p. 3543, 2023.

[20] G. Dicuonzo, F. Donofrio, A. Fusco, and M. Shini, "Healthcare system: moving forward with artificial intelligence," *TechnoVation*, vol. 120, article 102510, 2023.

[21] R. Laban and L. Ullmann, *The Mastery of Movement*, Macdonald & Evans, London, United Kingdom, 1971.

[22] L. Maman, E. Ceccaldi, N. Lehmann-Willenbrock et al., "Game-on: a multimodal dataset for cohesion and group analysis," *IEEE Access*, vol. 8, pp. 124185-124203, 2020.

[23] P. Słowiński, C. Zhai, F. Alderisio et al., "Dynamic similarity promotes interpersonal coordination in joint action," *Journal of the Royal Society, Interface*, vol. 13, no. 116, article 20151093, 2016.

[24] M. Bieńkiewicz, A. P. Smykovskyi, T. Olugbade et al., "Bridging the gap between emotion and joint action," *Neuroscience & Biobehavioral Reviews*, vol. 131, pp. 806-833, 2021.

[25] K. M. Newell, Y.-T. Liu, and G. Mayer-Kress, "Time scales in motor learning and development," *Psychological Review*, vol. 108, no. 1, pp. 57-82, 2001.

[26] R. Su, S. Dodge, and K. G. Goulias, "Understanding the impact of temporal scale on human movement analytics," *Journal on Geographical Systems*, vol. 24, no. 3, pp. 353-388, 2022.

[27] S. Ikeda, T. Nozawa, R. Yokoyama et al., "Steady beat sound facilitates both coordinated group walking and inter-subject neural synchrony," *Frontiers in Human Neuroscience*, vol. 11, p. 147, 2017.

[28] L. S. Shapley, "A value for N-person games," *Annals of Mathematical Studies*, vol. 28, pp. 307-317, 1953.

[29] A. Nowak and T. Radzik, "The Shapley value for N-person games in generalized characteristic function form," *Games and Economic Behavior*, vol. 6, no. 1, pp. 150-161, 1994.

[30] M. Hulcelle, G. Varni, N. Rollet, and C. Clavel, "Computational multimodal models of users' interactional trust in multiparty human-robot interaction," in *Proceedings of the International Workshops and Challenges of the 26th International Conference on Pattern Recognition (ICPR 2022)*, vol. 1364, pp. 225-239, Montréal, Canada, 2023.

[31] B. R. Auer and T. Hiller, "On the evaluation of soccer players: a comparison of a new game-theoretical approach to classic performance measures," *Applied Economics Letters*, vol. 22, no. 14, pp. 1100-1107, 2015.

[32] T. Yan, C. Kroer, and A. Peysakhovich, "Evaluating and rewarding teamwork using cooperative game abstractions," *Advances in Neural Information Processing Systems*, vol. 33, pp. 6925-6935, 2020.

[33] S. Béal, A. Navarro-Ramos, E. Rémila, and P. Solal, "Sharing the cost of hazardous transportation networks and the priority Shapley value for multi-choice games," *Annals of Operations Research*, pp. 1-45, 2023.

[34] R. A. Agnew, "Shapley value and international trade," *Theoretical Economics Letters*, vol. 13, no. 3, pp. 391-396, 2023.

[35] S. A. Byrne, A. P. F. Reynolds, C. Biliotti, F. J. Bargagli-Stoffi, L. Polonio, and M. Riccaboni, "Predicting choice behaviour in economic games using gaze data encoded as scanpath images," *Scientific Reports*, vol. 13, no. 1, p. 4722, 2023.

[36] A. Shojaie and E. B. Fox, "Granger causality: a review and recent advances," *Annual Review of Statistics and Its Application*, vol. 9, no. 1, pp. 289-319, 2022.

[37] C. W. J. Granger, "Investigating causal relations by econometric models and cross-spectral methods," *Econometrica*, vol. 37, no. 3, pp. 424-438, 1969.

[38] A. D'Ausilio, L. Badino, Y. Li et al., "Leadership in orchestra emerges from the causal relationships of movement kinematics," *PLoS One*, vol. 7, no. 5, article e35757, 2012.

[39] P. O. Amblard, O. J. J. Michel, C. Richard, and P. Honeine, "A Gaussian process regression approach for testing granger causality between time series data," in *Proceedings of the 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)*, pp. 3357-3360, Kyoto, Japan, 2012.

[40] A. B. Zaremba and G. W. Peters, "Statistical causality for multivariate nonlinear time series via Gaussian process models," *Methodology and Computing in Applied Probability*, vol. 24, no. 4, pp. 2587-2632, 2022.

[41] Z. Chen, J. Fan, and K. Wang, "Multivariate Gaussian processes: definitions, examples and applications," *Metron*, vol. 81, no. 2, pp. 181-191, 2023.
rehabilitation of Parkinson's patients," *The Journal of Visualization and Computer Animation*, vol. 14, no. 5, pp. 269-278, 2003.

[10] R. Niewiadomski, M. Mancini, A. Cera, S. Piana, C. Canepa, and A. Camurri, "Does embodied training improve the recognition of mid-level expressive movement qualities sonification?," *Journal on Multimodal User Interfaces*, vol. 13, no. 3, pp. 191-203, 2019.

[11] A. Singh, S. Piana, D. Pollarolo et al., "Go-with-the-flow: tracking, analysis and sonification of movement and breathing to build confidence in activity despite chronic pain," *Human-Computer Interaction*, vol. 31, no. 3-4, pp. 335-383, 2016.

[12] C. Spence, "Crossmodal correspondences: a tutorial review," *Attention, Perception, & Psychophysics*, vol. 73, no. 4, pp. 971-995, 2011.

[13] N. Virtanen, K. Tiippana, M. Tervaniemi, H. Poikonen, E. Anttila, and K. Kaseva, "Exploring body consciousness of dancers, athletes, and lightly physically active," *Scientific Reports*, vol. 12, p. 8353, 2022.

[14] K. Kolykhalova, G. Gneco, M. Sanguineti, G. Volpe, and A. Camurri, "Automated analysis of the origin of movement: an approach based on cooperative games on graphs," *IEEE Transactions on Human-Machine Systems*, vol. 50, no. 6, pp. 550-560, 2020.

[15] K. K. A. Bakhti, D. Mottet, N. Schweighofer, J. Froger, and I. Laffont, "Proximal arm non-use when reaching after a stroke," *Journal of Neuro Engineering and Rehabilitation*, vol. 15, 2018.

[16] A. Cera, C. Canepa, N. Ferrari et al., "Interactive sonification of expressive gesture: the DanzArTe-emotion wellbeing Technology project," in *Proceedings of the XXIII Colloquium on Musical Informatics (XXIII CIM)*, pp. 79-86, Ancona, Italy, 2022.

[17] D. R. Laker and J. L. Powell, "The differences between hard and soft skills and their relative impact on training transfer," *Human Resource Development Quarterly*, vol. 22, no. 1, pp. 111-122, 2011.

[18] D. J. Deming, "The growing importance of social skills in the labor market," *Quarterly Journal of Economics*, vol. 123, no. 4, pp. 1593-1640, 2017.

[19] D. L. Vetrano, A. Zucchelli, G. Onder et al., "Frailty detection among primary care older patients through the primary care frailty index (PC-FI)," *Scientific Reports*, vol. 13, p. 3543, 2023.

[20] G. Dicuonzo, F. Donofrio, A. Fusco, and M. Shini, "Healthcare system: moving forward with artificial intelligence," *TechnoVation*, vol. 120, article 102510, 2023.

[21] R. Laban and L. Ullmann, *The Mastery of Movement*, Macdonald & Evans, London, United Kingdom, 1971.

[22] L. Maman, E. Ceccaldi, N. Lehmann-Willenbrock et al., "Game-on: a multimodal dataset for cohesion and group analysis," *IEEE Access*, vol. 8, pp. 124185-124203, 2020.

[23] P. Słowiński, C. Zhai, F. Alderisio et al., "Dynamic similarity promotes interpersonal coordination in joint action," *Journal of the Royal Society, Interface*, vol. 13, no. 116, article 20151093, 2016.

[24] M. Bieńkiewicz, A. P. Smykovskyi, T. Olugbade et al., "Bridging the gap between emotion and joint action," *Neuroscience & Biobehavioral Reviews*, vol. 131, pp. 806-833, 2021.

[25] K. M. Newell, Y.-T. Liu, and G. Mayer-Kress, "Time scales in motor learning and development," *Psychological Review*, vol. 108, no. 1, pp. 57-82, 2001.

[26] R. Su, S. Dodge, and K. G. Goulias, "Understanding the impact of temporal scale on human movement analytics," *Journal on Geographical Systems*, vol. 24, no. 3, pp. 353-388, 2022.

[27] S. Ikeda, T. Nozawa, R. Yokoyama et al., "Steady beat sound facilitates both coordinated group walking and inter-subject neural synchrony," *Frontiers in Human Neuroscience*, vol. 11, p. 147, 2017.

[28] L. S. Shapley, "A value for N-person games," *Annals of Mathematical Studies*, vol. 28, pp. 307-317, 1953.

[29] A. Nowak and T. Radzik, "The Shapley value for N-person games in generalized characteristic function form," *Games and Economic Behavior*, vol. 6, no. 1, pp. 150-161, 1994.

[30] M. Hulcelle, G. Varni, N. Rollet, and C. Clavel, "Computational multimodal models of users' interactional trust in multiparty human-robot interaction," in *Proceedings of the International Workshops and Challenges of the 26th International Conference on Pattern Recognition (ICPR 2022)*, vol. 1364, pp. 225-239, Montréal, Canada, 2023.

[31] B. R. Auer and T. Hiller, "On the evaluation of soccer players: a comparison of a new game-theoretical approach to classic performance measures," *Applied Economics Letters*, vol. 22, no. 14, pp. 1100-1107, 2015.

[32] T. Yan, C. Kroer, and A. Peysakhovich, "Evaluating and rewarding teamwork using cooperative game abstractions," *Advances in Neural Information Processing Systems*, vol. 33, pp. 6925-6935, 2020.

[33] S. Béal, A. Navarro-Ramos, E. Rémila, and P. Solal, "Sharing the cost of hazardous transportation networks and the priority Shapley value for multi-choice games," *Annals of Operations Research*, pp. 1-45, 2023.

[34] R. A. Agnew, "Shapley value and international trade," *Theoretical Economics Letters*, vol. 13, no. 3, pp. 391-396, 2023.

[35] S. A. Byrne, A. P. F. Reynolds, C. Biliotti, F. J. Bargagli-Stoffi, L. Polonio, and M. Riccaboni, "Predicting choice behaviour in economic games using gaze data encoded as scanpath images," *Scientific Reports*, vol. 13, no. 1, p. 4722, 2023.

[36] A. Shojaie and E. B. Fox, "Granger causality: a review and recent advances," *Annual Review of Statistics and Its Application*, vol. 9, no. 1, pp. 289-319, 2022.

[37] C. W. J. Granger, "Investigating causal relations by econometric models and cross-spectral methods," *Econometrica*, vol. 37, no. 3, pp. 424-438, 1969.

[38] A. D'Ausilio, L. Badino, Y. Li et al., "Leadership in orchestra emerges from the causal relationships of movement kinematics," *PLoS One*, vol. 7, no. 5, article e35757, 2012.

[39] P. O. Amblard, O. J. J. Michel, C. Richard, and P. Honeine, "A Gaussian process regression approach for testing granger causality between time series data," in *Proceedings of the 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)*, pp. 3357-3360, Kyoto, Japan, 2012.

[40] A. B. Zaremba and G. W. Peters, "Statistical causality for multivariate nonlinear time series via Gaussian process models," *Methodology and Computing in Applied Probability*, vol. 24, no. 4, pp. 2587-2632, 2022.

[41] Z. Chen, J. Fan, and K. Wang, "Multivariate Gaussian processes: definitions, examples and applications," *Metron*, vol. 81, no. 2, pp. 181-191, 2023.