Physiological coupling in multi-person collaborative tasks: an intrinsic multi-scale synchrony approach
Creators: name: Hemakom, Apit
Publisher: name: Imperial College London
Year: 2017

description: Expert human performance in team scenarios (choir singing, surgical team) critically depends on the degree of collaboration between the individuals involved, and is reflected in elevated synchrony levels of their neuronal and physiological responses. Objective and physically meaningful synchrony metrics, however, are only just emerging, but are desperately needed in cognitive and behavioural science, education, robotics and gaming. Motivated by the multi-scale nature of human physiological and neuronal responses, such as the standard frequency bands in the electroencephalogram (EEG), this thesis sets out to explore and extend the so-called intrinsic multi-scale analysis framework in order to derive robust and physically meaningful metrics for the quantification of physiological and neuronal inter-component dependence across the scales of interest. In this way, standard data-association measures, such as correlation, coherence and phase synchrony, are equipped with the ability to operate at the intrinsic scale level, for example, through the noise-assisted multivariate empirical mode decomposition (NA-MEMD) algorithm. This thesis introduces three new intrinsic data-association measures for quantifying inter-individual physiological and neuronal couplings: (i) intrinsic coherence (ICoh), (ii) intrinsic synchrosqueezing coherence (ISC), and (iii) nested intrinsic phase synchrony (N-IPS). These measures are employed to robustly and accurately quantify physiological (respiration and heart rate variability) and neuronal (through electroencephalography) couplings between participants in several notoriously noisy and prone to artefacts real-world collective tasks, such as choir singing, multi-player gaming and performing a surgical procedure. The radically enhanced scale discrimination ability of the proposed approach is demonstrated over so-called hyperscanning (brain-to-brain communication) experiments using the next generation ear-EEG device for simultaneously recording neuronal function from multiple brains. The various EEG synchrony levels are evaluated against standard EEG in cooperative multi-player games, termed Bar Balancing and Ninjas, developed specifically for the work in this thesis to induce neuronal coupling between participants during the experiments. The ISC algorithm is shown to be capable of identifying inter-person neuronal coupling despite the detrimental effects from a variety of artefacts arising from the background brain activity and the sensing system, highly complex and nonlinear systems. Finally, to deal with power imbalances between physiological data channels (e.g. due to different electrode impedances), which is detrimental to the analysis, a robust extension of the adaptive, data-driven algorithm for the analysis of nonlinear and non-stationary multivariate signals, termed multivariate empirical mode decomposition (MEMD), is proposed and is referred to as adaptive-projection intrinsically transformed MEMD (APIT-MEMD). The APIT-MEMD algorithm in the intrinsic multi-scale analysis framework is validated on cooperative steady-state visual evoked potential (SSVEP) and P300-based brain-computer interface (BCI). The proposed data-association measures, together with the APIT-MEMD algorithm, offer a new avenue for the analysis of coupled physiological and neuronal signals, and are the underpinning technology for collaborative research and applications, including performance science and eSports.