Measuring Affective Sharing
between Two People by EEGHyperscanning
Taiki Kinoshitakinoshita.taiki.kk1@is.naist.jp
Nara Institute of Science and Technology
Ikoma, Nara, Japan

Hiroki Tanaka
Nara Institute of Science and Technology
Ikoma, Nara, Japan

Koichiro Yoshino
Nara Institute of Science and Technology
Ikoma, Nara, Japan

Satoshi Nakamura
Nara Institute of Science and Technology
Ikoma, Nara, Japan

ABSTRACT
Empathy plays an important role in human social interaction such as promoting stronger relationships
and collaboration. In this study, we measured affective sharing, one of the main components of empathy, from EEG signals. To elicit affective sharing, we conducted an experiment in which participants
communicated using facial expressions of joy, sadness, and neutrality. EEG signals were simultaneously recorded from both participants during the experiment. The result showed the correlations of
the EEG powers were significantly higher under the joy and sadness conditions in the alpha-mu band.
This result demonstrates that it is possible to measure affective sharing in response to emotional faces
from the correlation of EEG powers. To our knowledge, this is the first EEG hyperscanning study thatinvestigates affective sharing in response to emotional faces.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee
provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the fullcitation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstractingwith credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee. Request permissions from permissions@acm.org.
ICMI ’19, October 14–18, 2019, Suzhou, China© 2019 Association for Computing Machinery.
ACM ISBN 978-1-4503-6860-5/19/10. . . $15.00
https://doi.org/10.1145/3351529.3360657

Measuring Affective Sharing between Two People by EEG Hyperscanning

ICMI ’19, October 14–18, 2019, Suzhou, China

CCS CONCEPTS• Computing methodologies → Cognitive science.
KEYWORDS
empathy, affective sharing, electroencephalogram, facial expression
ACM Reference Format:
Taiki Kinoshita, Hiroki Tanaka, Koichiro Yoshino, and Satoshi Nakamura. 2019. Measuring Affective Sharing
between Two People by EEG Hyperscanning. In 2019 International Conference on Multimodal Interaction (ICMI ’19),
October 14–18, 2019, Suzhou, China. ACM, New York, NY, USA, 6 pages. https://doi.org/10.1145/3351529.3360657
Figure 1: Photograph of experimental setting. The target’s seat was placed on theright side and the observer’s on the left.
Seats were separated by a partition. A web
camera was installed on the target’s monitor.
TargetObserver

Trial 1
Trial 2
Trial 3
Trial 4
Trial 5
Trial 620 s 20 s 20 s 20 s
24 s 22 s 18 s 22 s 18 s

Figure 2: Outline of the experiment. Eachcolor indicates the kind of emotions instructed to a target. The order of emotions
was determined by the participant before
starting the experiment. The grey boxes
indicate the rest. The black boxes placedat the end of each trial indicate the SAM.

INTRODUCTION
Empathy plays an important role in human social interaction. For example, It promotes stronger
relationships and collaboration [11]. In much of the research, empathy is evaluated by subjective
evaluation [19]. However, It has been argued that subjective evaluation of empathy has less thanideal reliability [3] and low consistency among participants [6][14]. Therefore, the necessity ofpsychophysiological measurement of empathy has been pointed out [20].
Empathy involves bottom-up information processing and top-down information processing [23].
Here, affective sharing is the main bottom-up information process [23]. Affective sharing is an affective
response to another person that often, but not always, entails sharing that person’s emotional state[5]. It is known that affective sharing relies on inter-brain synchronization [5]. A previous study
measured the inter-brain synchronization during observation of emotional faces by functional magnetic resonance imaging (fMRI) hyperscanning [1]. In this case, hyperscanning was the simultaneousrecording of cerebral data from the two participants [16]. However, as fMRI requires a large-scale
device, it is considered difficult to use in a real environment, though necessity of measuring empathy
in interactive setting was pointed out [21].
Electroencephalography (EEG) is a representative method for measuring brain activity and has
the advantage of being usable in a real environment. A previous research measured the inter-brain
synchronization during observation of others in pain by EEG hyperscanning [8]. Although a facial
expression play crucial role in social interaction [7], there is no EEG hyperscanning study thatinvestigates affective sharing in response to emotional faces. In this study, we measure affective
sharing in response to emotional faces by EEG hyperscanning.

Measuring Affective Sharing between Two People by EEG Hyperscanning

METHODS9sadnessjoyneutralTarget's emotions

8
7

Experimental design

6
5
4
3
2

r = 0.778

11

2

3 4 5 6 7 8Perceived emotions

9

Figure 3: SAM values of target’s emotions
and perceived emotions. The number on
the lower right side is the polychoric correlation. Colors of plots indicate the kindof emotions instructed to targets. In the
scatter plot, random noise was added toscores to avoid overlap of dots.

98Target's emotions

ICMI ’19, October 14–18, 2019, Suzhou, China

7
6
5

The participants were six females (three pairs). The individuals in each pair were friends. As females
are typically considered more empathic than males [4], we only recruited female participants. The
average age of participants was 24.5 years and the standard deviation was 1.8. The experiment wasapproved by the ethics committee of the Nara Institute of Science Technology, and written informed
consent was obtained from all participants.
Our experiment was designed in the same manner as the previous fMRI study [1]. A photograph of
our experimental setting is shown in Figure 1. The outline of the experiment is shown in Figure 2. We
assigned one of the participant the role of “target" and the other of “observer". The participant given
the role of “target" was instructed using a monitor during the experiment. When the word “rest" wasdisplayed on the target’s monitor, the target was relaxed. When “joy" or “sadness" was displayed, thetarget indulged herself in emotional situations and expressed her emotion with her facial expression.
When “neutral" was displayed, the target tried to be as emotionless as possible. At the end of eachtrial, the target reported the valence of the emotion that she had expressed in nine levels using the
Self-Assessment Manikin (SAM) [2]. We call this score “target’s emotion". A higher SAM value meansa more negative emotion and a lower value means a more positive one.
For the participant given the role of “observer," the facial expression of the target was presented inreal time. The observer watched the target’s facial expression and tried to feel with her. At the endof each trial, the observer reported the valence of two kinds of emotions using SAM. The first one,
“perceived emotion," was the emotion that the observer perceived from the target’s facial expression.
The second one, “observer’s emotion," was the emotion of the observer that was elicited by looking at
the target’s facial expression. Six trials were conducted in one experiment. To obtain enough data, theexperiment was repeated three times on different days. We used two 30-channel EEG caps (Quick-30
manufactured by Cognionics) to record EEG signals. To investigate muscular artifacts, we recorded
electromyographic (EMG) signals from corrugator supercilii and zygomaticus muscle.

4

EEG signal analysis

32

r = 0.84

11

2 3 4 5 6 7 8
Observer's emotions

9

Figure 4: SAM values of target’s emotions
and observer’s emotions.

The pre-processing consisted of the following seven steps. All steps were performed with EEGLAB. (1)The sampling frequency was downsampled to 200 Hz. (2) EEG signals were band-pass filtered in therange of 0.3 to 45 Hz. (3) Onsets were set as the points where the emotional word was displayed on thetarget’s monitor. Then, EEG signals were segmented from 2 seconds to 20 seconds. Data from the first2 seconds were excluded from analysis because of artifacts. (4) Bad epochs were rejected by visualinspection. (5) Independent component analysis (ICA) was performed. (6) To deal with EMG artifacts,
we rejected ICA components that had a cross correlation with EMG signals of more than 0.1. Next,

Measuring Affective Sharing between Two People by EEG Hyperscanning

Joy

Sadness

TargetTargetICMI ’19, October 14–18, 2019, Suzhou, China

ICA components were automatically rejected by ADJUST [15] and by additional visual inspection. (7)Six electrodes T7, T8, CP5, CP6, P7, and P8 were rejected due to muscular artifacts.
In previous studies, inter-brain connectivity has been measured by correlation of EEG powers[12][13] or phase synchronization of EEG signals [10][8]. We calculated the former because bothpresentation of emotional faces and emotion elicitation are known to affect EEG powers [9].
The EEG powers were calculated by discrete Hilbert methods for specific frequency bands: theta(4-7Hz), alpha-mu (8-12Hz), beta (13-30Hz), and gamma (31-40Hz). Then the EEG data were segmented
into 1 second windows. The pearson correlation coefficient was calculated for each window over every
possible combination of target’s and observer’s electrodes. Next, we compared the correlations of
the high affective sharing condition (joy or sadness) and low affective sharing condition (neutral) by
a nonparametric permutation test. This statistical testing was performed in the same manner as aprevious research [10].
RESULTS

ObserverObserverFigure 5: The significant (p < 0.05, nonparametric permutation test, corrected
for multiple comparisons) connectivities
in the alpha-mu band. AF3 and FC5 wereused for recording EMG signals and someelectrodes were rejected due artifacts. So,
they are not shown in the figure.

Figure 3 and Figure 4 show the results of analysis on SAM. As indicated in the scatter plots, the SAM
scores reported by the targets and observers were highly correlated. Therefore, we consider target’s
emotions were successfully perceived by observers and successfully induced affective sharing.
Figure 5 shows the result of connectivity estimation. The correlations that were significantly higher
in the high affective sharing condition were found in the alpha-mu band. Regarding the connectivity
patterns, the target’s parietal and central regions were connected with the observer’s frontal andcentral regions in both joy and sadness conditions. We consider this result is reasonable because thealpha-mu band is associated with the mirror neuron system that underlies empathy [18][22] andwith empathic response for emotional faces [17].
CONCLUSION
In this study, we measured affective sharing in response to emotional faces by EEG hyperscanning.
The results showed that correlations of EEG powers were significantly higher under the high affective
sharing condition compared to the low affective sharing condition. These results demonstrate thepossibility of measuring affective sharing in response to emotional faces by EEG hyperscanning.
In the future work, we would like to conduct the experiment on more participants including
unfamiliar participant pairs to generalize the results. Furthermore, the result of connectivity patterns
needs further discussion because different patterns were found between sadness and joy condition.
This difference might be caused by the activation of visual cortex in the occipital region [7].
ACKNOWLEDGMENTS
Part of this work was supported by JSPS KAKENHI Grant Numbers JP17H06101, JP18K11437.

Measuring Affective Sharing between Two People by EEG Hyperscanning

ICMI ’19, October 14–18, 2019, Suzhou, China

REFERENCES[1] Silke Anders, Jakob Heinzle, Nikolaus Weiskopf, Thomas Ethofer, and John Dylan Haynes. 2011. Flow of affective
information between communicating brains. NeuroImage 54, 1 (2011), 439–446. https://doi.org/10.1016/j.neuroimage.2010.
07.004[2] Margaret M. Bradley and Peter J. Lang. 1994. Measuring emotion: The self-assessment manikin and the semantic
differential. Journal of Behavior Therapy and Experimental Psychiatry 25, 1 (1994), 49–59. https://doi.org/10.1016/00057916(94)90063-9 arXiv:0005-7916(93)E0016-Z[3] B. E. Chlopan, M. L. McCain, J. L. Carbonell, and R. L. Hagen. 1985. Empathy: Review of available measures. Journal of
Personality and Social Psychology 48, 3 (1985), 635–653.
[4] Mark H Davis. 1980. A multidimensional approach to individual differences in empathy. JSAS Catalog of Selected
Documents in Psychology 10 (1980), 85.
[5] Jean Decety and Philip L Jackson. 2006. A Social Neuroscience Perspective on Empathy. Current Directions in Psychological
Science 15, 2 (2006), 54–58. https://doi.org/10.1111/j.0963-7214.2006.00406.x[6] Nancy Eisenberg and Randy Lennon. 1983. Sex differences in empathy and related capacities. Psychological Bulletin 94, 1(1983), 100–131. https://doi.org/10.1037/0033-2909.94.1.100[7] Paolo Fusar-Poli, Anna Placentino, Francesco Carletti, Paola Landi, Paul Allen, Simon Surguladze, Francesco Benedetti,
Marta Abbamonte, Roberto Gasparotti, Francesco Barale, Jorge Perez, Philip McGuire, and Pierluigi Politi. 2009. Functionalatlas of emotional faces processing: a voxel-based meta-analysis of 105 functional magnetic resonance imaging studies.
Journal of psychiatry&neuroscience 34, 6 (2009), 418–432.
[8] Pavel Goldstein, Irit Weissman-Fogel, Guillaume Dumas, and Simone G. Shamay-Tsoory. 2018. Brain-to-brain coupling
during handholding is associated with pain reduction. Proceedings of the National Academy of Sciences 115, 11 (2018),
E2528–E2537. https://doi.org/10.1073/pnas.1703643115[9] Bahar Güntekin and Erol BaÅ§ar. 2014. A review of brain oscillations in perception of faces and emotional pictures.
Neuropsychologia 58 (2014), 33–51. https://doi.org/10.1016/j.neuropsychologia.2014.03.014[10] Ilana Heintz, Mary Beckman, Eric Fosler-Lussier, and Lucie Ménard. 2010. Inter-Brain Synchronization during Social
Interaction. PLoS ONE 5, 8 (2010), e12166. https://doi.org/10.1371/journal.pone.0012166[11] Martin L Hoffman. 2001. Empathy and moral development: Implications for caring and justice. Cambridge University Press,
Cambridge.
[12] Masahiro Kawasaki, Yohei Yamada, Yosuke Ushiku, Eri Miyauchi, and Yoko Yamaguchi. 2013. Inter-brain synchronization
during coordination of speech rhythm in human-to-human social interaction. Scientific Reports 3 (apr 2013), 1692.
https://doi.org/10.1038/srep01692http://10.0.4.14/srep01692[13] Sivan Kinreich, Amir Djalovski, Lior Kraus, Yoram Louzoun, and Ruth Feldman. 2017. Brain-to-Brain Synchrony during
Naturalistic Social Interactions. Scientific Reports 7, 1 (2017), 17060. https://doi.org/10.1038/s41598-017-17339-5[14] R. W. Levenson and A. M. Ruef. 1992. Empathy: A physiological substrate. Journal of Personality and Social Psychology 63,
2 (1992), 234–246.
[15] Andrea Mognon, Jorge Jovicich, Lorenzo Bruzzone, and Marco Buiatti. 2011. ADJUST: An automatic EEG artifact detector
based on the joint use of spatial and temporal features. Psychophysiology 48, 2 (2011), 229–240.
[16] P. Read Montague, Gregory S. Berns, Jonathan D. Cohen, Samuel M. McClure, Giuseppe Pagnoni, Mukesh Dhamala,Michael C. Wiest, Igor Karpov, Richard D. King, Nathan Apple, and Ronald E. Fisher. 2002. Hyperscanning: SimultaneousfMRI during linked social interactions. NeuroImage 16, 4 (2002), 1159–1164. https://doi.org/10.1006/nimg.2002.1150[17] Adrienne Moore, Irina Gorodnitsky, and Jaime Pineda. 2012. EEG mu component responses to viewing emotional faces.
Behavioural Brain Research 226, 1 (2012), 309–316. https://doi.org/10.1016/j.bbr.2011.07.048

Measuring Affective Sharing between Two People by EEG Hyperscanning

ICMI ’19, October 14–18, 2019, Suzhou, China[18] S.D. Muthukumaraswamy and B.W. Johnson. 2003. Changes in rolandic mu rhythm during observation of a precision grip.
Psychophysiology 41, 1 (2003), 152–156.
[19] David L. Neumann, Raymond C.K. Chan, Gregory J. Boyle, Yi Wang, and H. Rae Westbury. 2015. Measures of Empathy:
Self-Report, Behavioral, and Neuroscientific Approaches. In Measures of Personality and Social Psychological Constructs.
Academic Press, 257–289. https://doi.org/10.1016/B978-0-12-386915-9.00010-3[20] David L Neumann and H Rae Westbury. 2011. THE PSYCHOPHYSIOLOGICAL MEASUREMENT OF EMPATHY. In
Psychology of Empathy. 1–24.
[21] Ana Paiva, Iolanda Leite, Hana Boukricha, and Ipke Wachsmuth. 2017. Empathy in Virtual Agents and Robots. ACM
Transactions on Interactive Intelligent Systems 7, 3 (2017), 11:1–11:40. https://doi.org/10.1145/2912150[22] Ryszard Praszkier. 2016. Empathy, mirror neurons and SYNC. Mind and Society 15, 1 (2016), 1–25. https://doi.org/10.1007/
s11299-014-0160-x[23] Béatrice Tousignant, Fanny Eugène, and Philip L. Jackson. 2017. A developmental perspective on the neural bases ofhuman empathy. Infant Behavior and Development 48 (2017), 5–12. https://doi.org/10.1016/j.infbeh.2015.11.006