A picture is worth two thousand words:
Visual complexity in morphographic word recognition
Koji Miwa 1, Peter Hendrix 1, David Allen 2, Ton Dijkstra 3, & Harald Baayen 1 4
1 Eberhard Karls Universität Tübingen, 2 Ochanomizu University, 3 Radboud University Nijmegen, 4 University of Alberta

Aims and backgroundExperiment 2 (Complexity rating)

Background• Although studies on morphographic word processing usuallyMethod• 20 native and 21 non-native speakers of Japanese rated

control for character stroke counts (e.g., 鯨 ‘whale’ has 19strokes), we do not know much about the visual complexity.
Research questions• Are stroke counts enough to capture visual complexity ofvisual complexity of 300 randomly selected kanji words,using a 9-point scale. Rating scores were analyzed.
Results• Only native speakers rated Prototypical words less complex.
• JPEG Size could replace Strokes, but Strokes was better.

Japanese kanji words? Doesn’t JPEG size work better?
• What is the nature of the stroke effect? Linear or non-linear?
• It is possible to obtain an visual essence of kanji words (i.e., aprototypical kanji word) through an averaging technique?

(1) Strokes, (2) Prototypicality, (3) JPEG Size,(4) Orthographic Levenshtein Distance, (5) Distance from the2Experiment 1 (Picture averaging)−4−200 2 24

Time
CharacterFreq−3−3 −1−1

11JPEG
JPEGsize
size

NonJapaneseJapanese

NonJapanese
2−402

4

Time

Experiment 3 (Progressive demasking)• It is known that “visual essence” of faces can be obtained
through averaging (Galton, 1878; Jenkins & Burton, 2008).
(a) Picture averaging
Method• We applied this technique to 1945 Japanese kanji words. Wealso asked 111 native speakers of Japanese to guess what
the averaged kanji would look like, which were then averaged.

Averaged kanji in Mincho font

Method• 20 native speakers of Japanese identified, as quickly as(b) Wanted persons(d) Kanjis in Gothic font(b) Wanted persons

possible, 300 kanji words that were progressively unmasked
RTs were analyzed
Results• Strokes had a linear effect, and it could not be replaced with(e) Kanjis in a Bitmap font
JPEG Size.
(1) Strokes, (2) Prototypicality, (3) JPEG Size, (4)Orthographic Levenshtein Distance, (5) Distance from the
Averaged Image, (6) Number of Character Constituents(c) Kanjis in Mincho font(d) Kanjis in Gothic font(d) Kanjis in Gothic font(d) Kanjis in Gothic font(e) Kanjis in a Bitmap font

Averaged kanji in Gothic font(c) Kanjis in Mincho font(e) Kanjis in a Bitmap font(c) Kanjis in Mincho font(e) Kanjis in a Bitmap fontMentally averaged kanji

Prototypical kanji(e.g., 鯨 ‘whale’)

Experiment 4 (Eye-tracking lexical decision)
Method• Data analyzed in Miwa et al. (2014) were reanalyzed.

• The following visual complexity variables were studied:(1) Strokes, (2) Prototypicality, (3) JPEG Size,(4) Orthographic Levenshtein Distance, (5) Distance from theAveraged Image, (6) Number of Character Constituents
AMLaP 2016 (Bilbao, Spain)• Strokes was a better predictor than JPEG Size.
• Prototypical words received longer fixations.
(1) Strokes, (2) Prototypicality,(3) JPEG Size, (4) Orthographic
Levenshtein Distance, (5) Distance fromthe Averaged Image, (6) Number ofCharacter Constituents

0.0recognition of logographic words (Hsiao & Cottrell, 2009).
• Native speakers could not correctly guess the visual essence.

−0.4

decomposition, with more information on the left (i.e.,Prototypicality), which also account for the left-side bias in• 21 Japanese made lexicality judgment with 708 two-characterwords and 708 nonwords, during which their eye movements
were recorded. 1st subgaze durations were analyzed.
Results

1st fixation duration

Results• The “visual essence” hints tripartite (but not bipartite)−2

1 2−11JPEG size

02200
−2−2−4−400 22 44TimeTime−1−3JPEGsize
Strokes−4−4−3Complexity rating score

11 22−1−1−3−3−3−2 −1 0 1 2Complexityrating
ratingscore
score
Complexity

2 2−4 −2−2 0 0−4Complexityrating
ratingscore
score
Complexity

20−2Complexity rating score−40Complexityrating
ratingscore
score
Complexity

12 2−4−3 −2 −1 0

22Japanese − NonJapanese JapaneseJapanese− NonJapanese−2(c) Kanjis in Mincho font00
Strokes
Strokes

CharacterFreq(b) Wanted persons(a) Picture averaging
JapanesePictureprocess (b) Wantedpersons wanted persons(a) averaging
Picture averaging

22−2−2NonJapanese−2• The effects of Strokes are not always linear.
• For Japanese kanji words, the left and middle parts of thecharacter region are more likely to contain visual information.
• The average-based prototypicality affects word recognition.

(a) Picture averaging

00

2NonJapaneseJapanese−40Strokes

Japanese
NonJapaneseComplexity rating score−2

NonJapaneseJapanese−4 −2−2−4Complexityrating
ratingscore
score
Complexity

20−2• Strokes was a better predictor than JPEG size, but it was notenough to fully account for visual complexity of kanji words.

−4Complexity rating scoreJapanese

Summary of the findings

Complexityrating
ratingscore
score
Complexity

Averaged Image, (6) Number of Character Constituents

02Strokes

The study was supported by the Killam Trusts.