Eye movement control in word recognition

Koji Miwa
Nagoya University (Japan)
ICPEAL 17 – CLDC 9
10/19/2018, National Taiwan UniversityOutline
1

Lexical decision with eye-tracking

2

How many fixations are needed?

3

Are the eye movements informative?

4

What phenomena are particularly interesting at the first fixation?

5

What does LD with eye-tracking tell us about morphological processing?

2

1
Lexical decision
with eye-tracking

There are eye movements in reading
There are eye movements in reading (Rayner, 1998).

4

Word recognition

5

Lexical decision

556 ms6

Lexical decision

556 ms7

Lexical decision with eye-tracking

8Lexical decision with eye-tracking
Miwa, Libben, Dijkstra, & Baayen (2014)

Miwa, Dijkstra, Bolger, & Baayen (2014)

Participants: Japanese readersParticipants: English monolinguals

Item: Japanese two-kanji compounds (large font)Japanese two-kanji compounds (small font)Japanese-English bilinguals
Item: English simple words

Miwa, Libben, & Ikemoto (2017)

Miwa & Dijkstra (2017)

Participants: Japanese readersParticipants: Japanese readers

Item: Japanese three-kanji compounds

Item: Japanese compounds (vertical alignment)

Taylor, Miwa, & Mukai (in prep.)Miwa, Borgwaldt, Dijkstra, & Baayen (in prep.)

Participants: Japanese-English bilinguals

Participants: German-English bilinguals

Item: Japanese katakana words

Item: English compounds

English monomorphemic words

9Lexical decision with eye-tracking
Miwa, Libben, Dijkstra, & Baayen (2014)
L1

L1

Miwa, Dijkstra, Bolger, & Baayen (2014)

Miwa, Libben, & Ikemoto (2017)

Miwa & Dijkstra (2017)
L1

L1

Taylor, Miwa, & Mukai (in prep.)
L2

L1

L2

L1

Miwa, Borgwaldt, Dijkstra, & Baayen (in prep.)
L210Lexical decision with eye-tracking
I have conducted lexical decision experiments with English andJapanese words, testing both native and non-native speakers.
So far, I have collected over 40,000 lexical decision responses andeye movements in those trials.
I present what I learned about eye movement control in word recognition.

11

2
How many fixations
are needed?

How many fixations are needed?

5000

15000

93.8%

0Frequency
Numberof trials

25000

Words are read with multiple fixations almost all the time.

1

2

3

4

5

6

Fixation countFixationcount13

How many fixations are needed?

1234562345Fixation countFixationcount6523456Fixation countFixation
count

79.2%

1500

3000

23456Fixation countFixationcount

L297.6%

001

2000

16500

Frequency
Number of trials

2500
1500

90.1%

024L1500

Frequency
Number of trials

2500
1500
500
013Fixation countFixation
count

L1

89.9%

Number of trials

01

Fixation countFixationcount

L1

Frequency

1500
1000
06300052000

492.5%

1000

3Frequency
Number of trials

2Fixation countFixationcountFrequency
Number of trials

Frequency

Number of trials

0199.0%

500

2000
1000

Number of trials

99.0%

500

Frequency

4000
3000
2000
1000
0Frequency
Number of trials

92.0%

L1

L21000

L2L1123456

Fixation countFixationcount123456

Fixation countFixation count14

3

Are the eyemovements
informative?

Are the eye movements informative?
If the eye scans a word merely like a scanner, then the fixation durations arestrictly co-determined by word length.
In this case, eye movements are not informative for psycholinguistic research.

Short word

12Long word

1

23Time16

Are the eye movements informative?

If all linguistic processes are done at the first fixation, late fixation durations
reflect non-linguistic processes.
In this case, eye movements are not informative for psycholinguistic research.

Familiar word
Unfamiliar word

1

2

1

2Time

17

Are the eye movements informative?
If the first fixation is merely a corrective fixation, only the late fixation
durations reflect linguistic processes.
In this case, eye movements are not informative for psycholinguistic research.

Familiar word

1

Unfamiliar word

1

2

2Time

18

The eye movements are informative.
Miwa & Dijkstra (2017)
L1

12p = .001

ns
2

1

L1

time
Taylor, Miwa, & Mukai (in preparation)
L1

L2

1

2

ns

p < .001
2

1time

19

not arise at the first fixation.

5.85

5.85
●

5.80

5.75

5.70

●

logFirstFixationDuration

Native speaker advantage does

1stlogFirstFixationDuration
fix. duration

The eye movements are informative

5.65

ns

5.80
●

5.75

●

5.70

5.65
EnglishThere is no difference between

Japanese

NNS

Language
Language

NSPopulation
Population

6.1

6.1

6.0

ns

5.9
5.8

●

●

5.7
5.6
5.5

logLateFixationDuration

processing stage.

LatelogLateFixationDuration
fix. duration

Japanese and English at the late
6.0

●

5.9
5.8
5.7
5.6

●

5.5
EnglishJapanese

Language

Language

NNSNSPopulation

Population20

RTs and fixation durations
4.5

5.0

5.5

6.0

6.5

7.0

RTs correlate with late fixation

0.14−3.0 −2.5 −2.0 −1.5 −1.0 −0.5

invRT
RT

0.77

durations strongly.
7.0
6.5
6.0

745

6

logLateFixationDurationLate
fix dur.

3negatively.

4.5

5.0

fixation durations correlate

−0.37

5.5

First fixation durations and late

1 fix dur.

stlogFirstFixationDuration−3.0 −2.5 −2.0 −1.5 −1.0 −0.53456721

RTs and fixation durations
1st fixation duration (log)logFirstFixationDuration

When the 1st fixation isshort, late fixation durations
co-determine RTs more.

invRT−2−3−4−5

RTinvRT(inverse)−1

trade-off.

logFirstFixationDuration
6.6
4.6
6.6
7.3
5.3
7.3

3

4

600

There seems to be a

4.6
5.3
6−1−2−3−4−5−6567

LatelogLateFixationDuration
fixation duration (log)822

4What phenomena areparticularly interestingat the first fixation?

Early effects of whole word frequency

In ALL experiments, we observed a whole word frequency effectalready at the first fixation. It is always small but always significant.

−2−10

1

2

3

logTrimorphCompFreqKotonoha

Compound frequency

250

270

290

- 56 ms

230

Late
fix duration
Late fixation
duration (ms)

280

300

320

- 11 ms

260

1stfix duration
1st fixation
duration (ms)

L1

−2−10

1

23

logTrimorphCompFreqKotonoha

Compound frequency

24

Facilitatory effects of visual complexity
Visual complexity usually elicit processing cost.

−50510FirstCharacterStrokes
1st characterstrokes

440
360

400- 51 ms

320

1st fixation duration (ms)

440
360

400

108 ms320

L1

1st 1stfixfixation
duration
duration(ms)(ms)

However, visual complexity at unattended location facilitates processing.

−50510

2ndSecondCharacterStrokes
character strokes

25440400
360

10−55500
450
400
350

10−5−50510

FirstCharacterStrokes
1st char.
strokes

0510-18 ms

280
280

300
300

320
320

340
340

SecondCharacterStrokes

1st fixationfixation duration
duration (ms)(ms)
1st340
320
300
280

L1

1st1stFix
Duration
fixation
duration (ms)FirstCharacterStrokes

68 ms- 81 ms−5−50055

1010

SecondCharacterStrokes
2nd SecondCharacterStrokes
char strokes-11 ms

340

0

10

320

−55

SecondCharacterStrokes

1st fixation duration (ms)

400

450

500

151 ms350

L1

1st1stFix
Duration
fixation duration (ms)FirstCharacterStrokes

0300

5280

0

1st fixation duration (ms)−5- 51 ms

320

1st fixation duration (ms)

360

400

440

108 ms320

L1

1st1stFix
Duration
fixation duration (ms)Facilitatory effects of visual complexity−50510rd char strokes
3ThirdCharacterStrokes

26

5
What does LD witheye-tracking tell us
about morphologicalprocessing?

(e.g., trimorphemic words
are read with 2 fixations).

310
300

strictly by morphemes

L1290

fixation duration (ms)
1st 1stFixDuration

The eye is not guided

320

Morphological processing−2−1012

3

logTrimorphCompFreqKotonoha−4−3−2−1

0

1

logFirstCharFreqKotonoha

2

1st char. freq.

320
310305

ns290

295

300

1st fixation duration (ms)

315

320
315
310
305
295

300

1st fixation duration (ms)

315
310
305
300
295

ns

290

processes.

290

morphological

1st fixation duration (ms)movements reflect

1st Fix Duration

However, eye

320

Compound freq.

−4−3−2−10

1logSecondCharFreqKotonoha

2

2nd char. freq.

−3−2−101logThirdCharFreqKotonoha

2

3rd char. freq. 28

character and the compound units
have been observed consistently.

400

L1360

Frequency effects for the left

320

1st1stFix
Duration
fixationduration (ms)Morphological processing−4−202WholeWordFrequencyKotonoha−3 −2 −10

1FirstCharacterFreq

2

1st char. freq.

400
360
320

1st fixation duration (ms)

400
360
320

1st1stFixDuration
fixation
duration (ms)Compound freq.

−4−201

2

SecondCharacterFreq

2nd char. freq.

29character and compound units havebeen observed consistently.

440

L1400

Frequency effects for the left

360

st fixation
duration (ms)
11stFix Duration

Morphological processing−4−20

2−3 −2 −10

1FirstCharacterFreq

2

1st char. freq.

440
400
360

1st fixation duration (ms)440400360

1st1stFix
Duration
fixation
duration (ms)WholeWordFrequencyKotonoha
Compoundfreq.

−4

−20

1

2

SecondCharacterFreq

2nd char. freq.

30Take-home messages
1

Lexicaldecision control
with eye-tracking“Eyemovement
in word recognition” makes sense.

2

Isolatedwords
are readwith two fixations.
How many
fixations
are needed?

3RTswith thelatereflect?
fixation durations.
Whatcorrelate
do response
times

4

Early
effects of word
frequency andfacilitatoryof visual complexity were found.
What phenomena
are particularlyinterestingat theeffects
first fixation?

5Eyemorphologicaldecomposition.
Whatmovements
does LD withreflect
eye-tracking
tell us aboutmorphological processing?
31References
Miwa, K., & Dijkstra, T. (2017). Lexical processes in the recognition of Japanese horizontal and vertical compounds. Reading andWriting: An Interdisciplinary Journal, 30, 791-812.
Miwa, K., Dijkstra, T., Bolger, P., & Baayen, R. H. (2014). Reading English with Japanese in mind: Effects of frequency, phonology,
and meaning in different-script bilinguals. Bilingualism: Language and Cognition,17, 445-463.
Miwa, K., Libben, G., Dijkstra, T., & Baayen, R. H. (2014). The time-course of lexical activation in Japanese morphographic word
recognition: Evidence for a character-driven processing model. Quarterly Journal of Experimental Psychology, 67, 79-113.
Miwa, K., Libben, G., & Ikemoto, Y. (2017). Visual trimorphemic compound recognition in a morphographic script. Language,Cognition and Neuroscience, 32, 1-20.
Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124, 372422.
Taylor, J., Mukai, Y., & Miwa, K. (in preparation). Comparing the time-courses of lexical processes in L1 and L2 word

recognition: A lexical decision eye tracking study with Japanese-English bilinguals32Thank you
Another presentation on lexical decision with eye-tracking
October 20, 16:30-17:40
Jamie Taylor, Yoichi Mukai and Koji Miwa
Comparing the time-courses of lexical processes in L1 and L2 word recognition:
A lexical decision eye tracking study with Japanese-English bilinguals

Acknowledgment
The PowerPoint template was made by Jun Akizaki The Power of PowerPoint | thepopp.com

How many fixations are needed?

3.23.0
2.8
2.6
2.2

2.4

Fixation counts

3.0
2.8
2.6
2.4
2.2

Fixation counts

3.2

Fewer fixations are needed when reading L1 words, compared to L2 words.

NNSNS
Population

EnglishJapanese
Language34

LME and GAMM
logFirstFixationDuration
4.6
5.3
6

6.6
7.3

00−2
invRT−2−3−4

67

logLateFixationDuration

8

rat
io
DutionDurati
onixa

tion

ixa

tion 7

5.0

partial effect

5tion

st F

4

5.5

5eFi
xation 6Du
raLat
eF
ixa

Fir

3

6.0

FirstF

4logLat−6nrat
io−6Du−57.0
6.5

n−4loginvRT

se TimeRespon−135