  

Detecting Cognitive Load during Simultaneous Interpretation using
EEG Phase Synchronization#*)   ( ' $ "   

2019/12/14

&!%



2019@Haruko Yagura AHC-Lab, IS, NAIST0

Ø # # Ø  Ø 2-3!15 
2019/12/14

Ø 話者と通訳者が交互に話す
Ø 工場視察や表敬訪問などØ  "2019 @Haruko Yagura AHC-Lab, IS, NAIST

1



+[Daniel+, 2011]ya],

Ø Extreme multitasking;speaking, listening,memory, translation[d2,m01a+mC2019/12/14

Cognitive loadm

2019 @Haruko Yagura AHC-Lab, IS, NAIST

simultaneous interpreter2Ø Ø  

2019/12/14

2019 @Haruko Yagura AHC-Lab, IS, NAIST

3 

(A: 
$*&"')


B: #%A +B↑#%Ø  !
[Cowan, 2005] [Mizuno, 2015]2019/12/142019 @Haruko Yagura AHC-Lab, IS, NAIST

4]hVocVkoblVfK ocHemK]

6658B

76-89+7h6H][ P2019/12/14(7koblM
8E
7I

P Hs61v
4 VVinKa6)7z
B,7R P
61 E
B+7240r
RP

2019 @Haruko Yagura AHC-Lab, IS, NAISTPH

5

[Koshkin+, 2018]

A: load-low

P1(40-60ms)
N1(120-160ms)

B: load-medium

C: load-highØ 0Ø  1) ]22)
0 L10] L20
Ø  
• a S
P 40 2 ]]Ø EEG
•P

B: Grand average ERPs for L1(Russian) and L2(English)

P1

4N1


onset

ERP

EEGi4ms(ERPs)•ERPERPl
2019/12/14[Stl→N1/P1lP

2filter-outl[P)P

01]r,+Ø 1) N1 P12)2019 @Haruko Yagura AHC-Lab, IS, NAISTbb6



Ø 32&*/→Ø4+ K#D;?2RQ  B9C4Ø )05  -+

4+N%FK#
N%F

D
N%<;?
N%<2R
N%<2&Q



N%F

*Ø # !
'6 .
4%7
.41(N%<!.4%
 $*"
.41,
N%<B9C
N%E

Ø '"@,A3:J750ms >1H8Ø  >1 SL EEG>1(ERP:N1-6-I70100msM!,
P16-I7050msM!)=.P$5O*
2019/12/14G(/)&

2019 @Haruko Yagura AHC-Lab, IS, NAIST

7ASSR)
nAuditory Steady-State ResponseØ \D=4/,N=4]S9&#_;T_ZXY
Ø *&JY3#H K[%Y3-B I7Y YEY
3 T(H#_#Ø QD=4\D!Q_ZXYT
Ø V`?>12@b)  :U
.AL K+R5<"c8C
%aØ 40HzASSR G/FbREMWO^'[$ [Tiitinen+1993 ][Griskova+2011][Roth+,2013]2019/12/14P+60[](ASSR
2006][,A:CF-1000Hz, MF-40Hz SAM
B:CF-1000Hz, MF-80Hz SAM
C:1000Hz, tone-pip(CF:-carrier frequency, MF: :-modulation frequency)

2019 @Haruko Yagura AHC-Lab, IS, NAIST

8ASSR ITC     [Yokota+,2017]

n40Hz-ASSRD>  4;%?.+$7(ITCInter-trial coherence)FE
6$<8A@"Ø ITC74;%-,--51 4;%-:- -1
Ø69 0n-back-N-backØ /*2(n-back-(n=1 8(, n =2 8,)
4$ 40Hz ASSR#=& 3%
Ø !340HzASSR;61' ITC."*47

Ø '5/8+  *<ITC.
):!ITC.: n = 2 8, < n =1 8
(, p<0.05
Ø 9CB+5#&A <8A)0=

2019/12/142<82019 @Haruko Yagura AHC-Lab, IS, NAIST[Yokota+, 2015, 2017]

9

The value of ITC

ITC 



2019/12/14   2019 @Haruko Yagura AHC-Lab, IS, NAIST

The value of value10Ø   ASSRØ  

2019/12/14

2019 @Haruko Yagura AHC-Lab, IS, NAIST11  
40Hz-ASSRITC 0,AS7Q' 40HzB2J=F6

ASSR-U1  
 G.ASG.C/:F&ASSR9+?5 <N3LV I>M!8($E;H0,AS7Q
4WOP

 0*8(E; 8($E;TRD

2019/12/14@#)%"

2019 @Haruko Yagura AHC-Lab, IS, NAISTK?5120(• )&1$+#.,*


'/
• 40Hz-ASSR2-•

ITC"

3

ITC".! < .2019/12/14%

2019 @Haruko Yagura AHC-Lab, IS, NAIST

13Ø  Ø ASSRØ 

2019/12/14

2019 @Haruko Yagura AHC-Lab, IS, NAIST

14n 10 ;KP;8=H1FV5
range; 46-710mean= 55.10SD = 2.54
C!"H1FV+.MSR TS!"H1FV+.M<15M#:
D!"O
D@%TFV5N-&4)6

C2VQV+./GM#:

Ø 3.?JX>BA(YW$%&9L 
7U" ""I

2019/12/14E',*&2019 @Haruko Yagura AHC-Lab, IS, NAIST

15

40Hz-ASSRAuditory steady-state responseü52

12 22 9 8)
,5 05 0 :212 29 . 7.
(2019/12/14H

228 2, 08 07:1
15
9 5

2019 @Haruko Yagura AHC-Lab, IS, NAIST

9 8 :422 2 2116: 40Hz-ASSR

n  NHK ASSR ")3'0&• ")
601

•



•   #-(2 5  /*$,%

 #-(2+

2019/12/14.!

2019 @Haruko Yagura AHC-Lab, IS, NAIST

17


n @,<-6B2'>→

n @ 72'>→>

<6B•

 → 
 Mizuno, 2015,Matsubara, 2012)• 6B =:;3  2'$8.01 )9>6B→>A+
• "2' ?$ &.*60? 4/C 8.

2019/12/145#(%!



2019 @Haruko Yagura AHC-Lab, IS, NAIST

18

(EEG)
nbaePQØØ

3 C03

cdHZf-E

D : D 30 0G0; :0 G
10 3 0 5 :3GF
3 :3

2019/12/14

X2019 @Haruko Yagura AHC-Lab, IS, NAIST

19

n

ITC  
ITC
t

3T

Ø

4(((C,Ø3z I(4ITC
Ø13Ø where f is a frequency, ch is thechannel number, fØ k is phases of frequency f andelectrode ch, k is a trial number, and K
is the number of trials. ITCs werecalculated forØ each news over the three central
electrodes (C3, Cz, and C4). ITCs
were then averaged over theelectrodes[Griskova+2011]2019/12/14

2019 @Haruko Yagura AHC-Lab, IS, NAIST

20The value of ITC



2019/12/14

Cognitive
load is lownear 1)

Cognitive
load isHigh(near0) 

The value of value

ITC

[koshikin+,2018][Yokota+,2015, 2017]

2019 @Haruko Yagura AHC-Lab, IS, NAIST21Ø Ø

Ø 

2019/12/14 2019 @Haruko Yagura AHC-Lab, IS, NAIST22

ITCShadowingnTR (translation:)nSW(Shadowing:)Ø ITC TR > SW (p<0.001)• TR; mean = 0.55sd = 0.22SW; mean=0.48sd = 0.20
•(ITC: < 2019/12/14) 2019 @Haruko Yagura AHC-Lab, IS, NAIST23





n dGiecgl
0 99 9n+•zv0 23 9324 o

Sz 2l324 ono

2019/12/14surp[E( ,)) 1zH

•



9(1dGi
24TNT

32

22kTE

hbG
s

2019 @Haruko Yagura AHC-Lab, IS, NAISTad ]EoSt24TRn,[,,33 3424n+ In+,

2019/12/14

arn]

e

ITC



WI S+ RS] + RSb+01 39,,it]
ar++
ATCTC
RSIAIAA

2019 @Haruko Yagura AHC-Lab, IS, NAISTT,]
925

nTn

2019/12/14

IC2019 @Haruko Yagura AHC-Lab, IS, NAIST

26

n

TI

ASAnn

T

0A

2019/12/14

Hz IWz I-z

A
ASTR

IAI

IC

4
AS

2019 @Haruko Yagura AHC-Lab, IS, NAISTI-zI27


•

5MP HLOMS - 7 BMS LC - 9PP C B H V DP HLF
KMCDJ ME PHKRJ LDMRP HL DONOD HLF, -L DON P RC :JM MLD SMJND•M MLC 8 ORPD V: PD BM DODLBD ME RCH MO P D C P D ODPNMLPDODEJDB P D KMRL ME BMFLH HSD TMO JM C HL KMCHEHDC L A BP
8DROMPBHDLBD ODPD OB SMJNN•M ML- 7H KM M LC 8 ORPD V1P HK HML ME RK LTMO JM C EOMK D RCH MO P D C P D ODPNMLPD ODBMOCDC SH
TD O AJD
DJDB OMDLBDN JMFO N P P DK CROHLF T J HLF 2OML HDOP HL RK LLDROMPBHDLBD SMJND DEEMO PLM• 7AOH
3RKN OHDP 7 DOADO 1 6HDADL J 4 .HLCDO 4 7 LF J R
LC - 0DP H V8DRO J DEEDB P ME BMFLH HSD BML OMJ JM C ML RCH MO PDJDB HSD
DL HML 8DR OMNP B MJMFH SMJNN

2019/12/14

Y2019 @Haruko Yagura AHC-Lab, IS, NAIST28



 

2019/12/14



2019 @Haruko Yagura AHC-Lab, IS, NAIST29



2019/12/14

2019 @Haruko Yagura AHC-Lab, IS, NAIST30

ITC

F
ch:
 : fch k :   K:  [Yokota+,2017]2019/12/14

2019 @Haruko Yagura AHC-Lab, IS, NAIST31