ICMI2019 Late Breaking Results

Measuring Affective Sharing between Two People by EEG Hyperscanning
Taiki Kinoshita, Hiroki Tanaka, Koichiro Yoshino, and Satoshi Nakamura
Nara Institute of Science and Technology, Japankinoshita.taiki.kk1@is.naist.jp

1. Introduction• Empathy plays an important role in social interaction
e.g., promotes stronger relationships and collaboration• Subjective measurement of empathy has less reliability• Evaluated with questionnaires• The accuracy is limited by human perception errors
Necessity of robust objective measurement
e.g., Electroencephalography (EEG)

3. Recording and Analysis of EEG data
3.1 Recording of EEG data
EEG cap: CGX Quick-30
Recording of facial electromyography (EMG) for preprocessing
AF3FC5Corrugator supercilii muscle
Zygomatic muscle• Many definitions of empathy involve affective sharing• Unconsciously resonating with another person’s affect• Inter-brain synchronization• Overview

1. Band pass filter
theta (4-7Hz), alpha-mu (8-12Hz),
beta (13-30Hz), and gamma (31-40Hz)
2. Hilbert transformInduced Emotion
SimultaneousEEG recording

This research:
Facial expression
Previous research [1]:Target in pain

Target

1. Calculation of EEG power

Observer2. Calculation of the Pearson correlation coefficient
TargetMeasurement ofaffective sharing

TargetObserver12161718 Time(sec)PowerEmotion

3.2 Analysis of EEG data

[1] Goldstein, P., Weissman-Fogel, I., Dumas, G., & Shamay-Tsoory, S. G. (2018).
Brain-to-brain coupling during handholding is associated with pain reduction.
Proceedings of the National Academy of Sciences of the United States of America, 115(11), E2528–E2537.

cor1

2. Experiment

cor17

cor2

ObserverPowerTarget<latexit sha1_base64="weg33g55cFIJA1iAe3P9aPN1tHg=">AAACjnichVE7SwNBEJ6cr/hM1EawCYaI1TGbiyYIYkCLlL7ygCjh7tzEI/fi7hKIh3/AzkowlYKF+AusbfwDFv4EsVSwsXDuEhELdZbdnf1mvtlvGMXWNddDfIoIA4NDwyPR0bHxicmpWHx6puRaLUflRdXSLaeiyC7XNZMXPc3TecV2uGwoOi8rzY0gXm5zx9Usc8/r2PzAkBumVtdU2SOorFpOzWcntXgSxWxmJSdJCRSl5ewyywUOYxJLJ5iIoSWhb1tW/A724RAsUKEFBnAwwSNfBxlcWlVggGATdgA+YQ55WhjncAJjxG1RFqcMmdAmnQ16VfuoSe+gphuyVfpFp+0QMwEpfMQbfMUHvMVn/Pi1lh/WCLR06FZ6XG7XYqdzu+//sgy6PTj6Zv2p2YM65EKtGmm3QyToQu3x28fnr7urOyl/Ea/whfRf4hPeUwdm+0293uY73T/1OLRk6iIY0tckEr87pbTIJDG9nUnmN/vjisI8LMASzSQLeSjAFhTDP87gArpCXFgR1oT1XqoQ6XNm4YcJhU/85ZQo</latexit><latexit sha1_base64="ewDwhGYx66vS3V3g9oaV9uFEewM=">AAACjnichVG7SsRQEJ2Nr/UdtRFsgotiFSbJvhBEQYstdXVdQWVJ4l0NmxdJdmEN+wN2VoJWChbiF1jb+AMWfoJYKthYOMlGxEKdy7137pk5c88wmmsafoD4lOJ6evv6B9KDQ8Mjo2Pj/MTktu80PZ1VdMd0vB1N9Zlp2KwSGIHJdlyPqZZmsqrWWI3i1RbzfMOxt4K2y/Yt9dA26oauBgRVdcerhXKnxmdQLGTzRUURUFRyhZxUjBxJUiRZkESMLQOJrTv8HezBATigQxMsYGBDQL4JKvi0dkECBJewfQgJ88gz4jiDDgwRt0lZjDJUQht0HtJrN0Ftekc1/Zit0y8mbY+YAszhI97gKz7gLT7jx6+1wrhGpKVNt9blMrc2fjK9+f4vy6I7gKNv1p+aA6hDMdZqkHY3RqIu9C6/dXz2urlYngvn8QpfSP8lPuE9dWC33vTrDVa++FOPR0ulLqIhfU1C+N3ZlkVJEeWNbGZlLRlXGmZgFhZoJgVYgRKsQyX+4xTO4YLjuTy3xC13U7lUwpmCH8aVPgH/CpQp</latexit><latexit sha1_base64="GfPKcwL1tzCtyi5QyaZxu03F4DQ=">AAACj3ichVE9T8JQFL3UL8Qv0MXEhUgwTs0tBUscDIkOuoGIkCghbX1gY2mbtpBgwx9wdHHQRRMH4y9wdvEPOPATjKMmLg7eFoxxQO9L37vvvHtuz81RLF1zXMReiBsZHRufCE9GpqZnZueisfl9x2zZKiuppm7aFUV2mK4ZrORqrs4qls3kpqKzsnKy6b+X28x2NNPYczsWqzblhqHVNVV2Caqopl3zBKlbiyaQl9JrWVGMIy9mpIyQ9RNBEIVUXOAxiAQMIm9GH+AQjsAEFVrQBAYGuJTrIIND6wAEQLAIq4JHmE2ZFrwz6EKEuC2qYlQhE3pCe4NuBwPUoLvf0wnYKv1Fp88mZhyS+Ix3+IZPeI8v+Dm0lxf08LV06FT6XGbV5s4Wix//spp0unD8w/pTswt1yAZaNdJuBYg/hdrnt08v3orru0lvBW/wlfRfYw8faQKj/a7eFtju5Z96bFoyTeGb9O1EfHiyn+IFkU8V0onc1sCuMCzBMqySJxLkYBvyUAq8OIdLuOJinMRtcLl+KRcacBbgV3A7X5rGlGk=</latexit>

cor18<latexit sha1_base64="kmrOQs3wCpqwol4hcyGoO7trn4w=">AAACj3ichVE7S8NQFD7GV3221UVwCRbFKZw0aoKDFHTQzVdrQUtJ4lVD8yJJCzX0Dzi6OOii4CD+AmcX/4CDP0EcK7g4eJJGxKF6Lrn33O+e7+Q7fJprGn6A+NLD9fb1DwymhoZHRsfG05nsRMl36p7OirpjOl5ZU31mGjYrBkZgsrLrMdXSTLan1Vaj970G83zDsXeDpssqlnpsG0eGrgYElXXHq4ai0qpmcijIC0uKJPEoSIvyoqhEiShKYp4XBYwjB0lsOpkHOIBDcECHOljAwIaAchNU8GntgwgILmEVCAnzKDPidwYtGCZunaoYVaiE1mg/ptt+gtp0j3r6MVunv5j0ecTkYRaf8Q7b+IT3+IqfXXuFcY9IS5NOrcNlbjV9NrXz8S/LojOAkx/Wn5oDOAIl1mqQdjdGoin0Dr9xetHeWd6eDefwBt9I/zW+4CNNYDfe9dsttn35px6PlkpTRCZ9O8F3T0p5QZSE/NZCrrCW2JWCaZiBefJEhgKswyYUYy/O4RKuuCwncytcoVPK9SScSfgV3MYXnOuUag==</latexit>

Observer cort : The correlation between target’s andobserver’s EEG power at the time window t

2.1 Purpose<latexit sha1_base64="2Sardqw6JOLurLKWNbRsUNPnGZM=">AAACjnichVE7SwNBEJ6cr/hM1EawCYaI1TGXywtBDGiRUqN5QJRwd27ikcvdcbcJxJA/YGclmErBQvwF1jb+AQt/glgq2Fg4d4mIhTrL7s5+M9/sN4xqG7rLEZ8Cwsjo2PhEcHJqemZ2LhSeXyi6VsvRWEGzDMspq4rLDN1kBa5zg5VthylN1WAltbHlxUtt5ri6Ze7zjs0Om0rd1Gu6pnCCSprlVLu8Vw1HUUwnUhlZjqAoJ9NJKeM5kiRL8Ygkom9RGNqOFb6DAzgCCzRoQRMYmMDJN0ABl1YFJECwCTuELmEOebofZ9CDKeK2KItRhkJog846vSpD1KS3V9P12Rr9YtB2iBmBGD7iDb7iA97iM378Wqvr1/C0dOhWB1xmV0OnS3vv/7KadHM4/mb9qZlDDTK+Vp202z7idaEN+O2T89e99Xysu4pX+EL6L/EJ76kDs/2mXe+yfP9PPQ4thbrwhvQ1icjvTjEuSrIY301Es9vDcQVhGVZgjWaShizkYAcK/h9ncAF9ISykhA1hc5AqBIacRfhhQu4TjKOUaw==</latexit>

Recording EEG data of high and low affective sharing condition

Iteration over every possible combination of electrodes

2.2 Participants

3. Statistical testing (Cluster-level permutation test)• Three pairs (Six people)• The individuals in each pairs are friends• Females

High affective sharing condition (joy or sadness) vslow affective sharing condition (neutral)

2.3 Experimental design [2]4. Results and Discussion
Single printed word(joy, sadness, or neutral)TargetVideo oftarget’s facial expression

Simultaneous
EEG recording

Observer2.4 Self report of the emotions
using Self assessment manikin (SAM)
Targets reported the valence scores of emotions:• she felt by imagining emotional situationObservers reported the valence scores of emotions:• perceived from target’s facial expression• induced by observing target’s facial expression

4.1 SAM
Target’s Emotions

Emotionalsituation

987654321r = 0.778

987
Joy
6Neutral 5Sadness 432
1r = 0.840

1 2 3 4 5 6 7 8 9
1 2 3 4 5 6 7 8 9Perceived Emotions
Induced Emotions
Target’s emotions were successfully perceived by observers
and induced observer’s emotion

4.2 EEG
Target

JoyObserver

TargetSadness

Observer2.5 ProcedureJoy
NeutralSadness
Target’s Procedure

Rest + Count down of 3s
SAM
Observer’s Procedure

Trial 1
Trial 2
Trial 3
Trial 4
Trial 5
Trial 620 s 20 s 20 s 20 s
24 s 22 s 18 s 22 s 18 s

• Significant connectivity was found in the alpha-mu band
• The alpha-mu band is known to associated withmirror neuron system and viewing emotional faces• The possibility of measuring affective sharing

5. Conclusion
Video of target’s
facial expression[2] Anders, S., Heinzle, J., Weiskopf, N., Ethofer, T., & Haynes, J. D. (2011).
Flow of affective information between communicating brains. NeuroImage, 54(1), 439–446.

• Purpose: Measuring affective sharing by EEG hyperscanning• Results: Significant inter-brain connectivity in the alpha-mu band
• Future works: Discussion on connectivity pattern,Experiments on more pairs