Morphology and Picture Relatedness
Koji Miwa, Sally Rice, & Gary Libben - University of Alberta
Experiment 1
Picture Comparison

Background

Conceptual
Representation

To test the cross-language applicabilityof picture comparison tasks

Moderate word activation
花火
花火 ‘firework’

花束
花束 ‘bouquet’Aim

Abstract Image
火火‘fire’

束束‘bundle’花花‘flower’VisualForms

Lemma
Representation

nounnounnoun

Word Level
花火
花火

noun noun

Concrete
Image

Word Level

花束
花束/hanabi//hanataba/Morpheme Level

Morpheme Level/hana/ (KUN)/ka/ (ON)花花

 20 pairs of pictures share a left constituent
in Japanese translation equivalents,
but not in English.
 20 pairs of pictures share a right constituent
in Japanese translation equivalents, but not in English.
 20 pairs of control pictures were unrelated.
Three Types of Stimuli used
火火

which holds that the languages people are normally
exposed to influence their cognition of reality, has
motivated several lines of research in psychology andlinguistics, but the truth is not clear, due to methodologicallimitations (Lucy, 1997).

Morphology, especially compound representations in
the mental lexicon has motivated intensive
psycholinguistics research for the last 30 years.

Hypothesis 1:

noun nounNO

Hypothesis 2:
The degree of word activations in the mind influences
the picture relatedness ratings.

/hanataba/
VisualPerception

Morpheme Level

Morpheme Level/ka/ (ON)/hana/ (KUN)
花花

Absolute
Reality/taba/ (KUN)/hi/ (KUN)

束
束/soku/ (ON)花火
花火 ‘firework’

To test how
maximum word
activations in themind influence
picture comparison.

花束
花束 ‘bouquet’
Abstract Image

Conceptual
Representation

火火‘fire’

束束‘bundle’花花‘flower’

VisualForms

Lemma
Representation

Phonological Representation

nounnoun

Concrete
Imagenoun nounnoun

Word Level
花火
花火

Word Level

花束
花束/hanabi//hanataba/
VisualPerception

Morpheme Level

Morpheme Level/hana/ (KUN)/ka/ (ON)花
花
Absolute
Reality/taba/ (KUN)
火火/hi/ (KUN)

束束/soku/ (ON)

Orthographic Representation

Phonological Representation

Stimuli The Japanese and English translated wordequivalents of the picture stimuli used inExperiment 1 and Experiment 2Participants 20 native speakers of Japanese 20 native speakers of Englishwho did not participate in Experiment 1

Participants
 The same as in Experiment 2

ProcedureProcedure The same as in Experiment 1, except thatparticipants named pictures before making
picture relatedness judgments

 The same as in Experiment 1, except thatword stimuli were used, instead of pictures
The screen shots and procedure in Experiment 3

The screen shot and procedure in Experiment 2(Picture presentation → judgment)(Picture presentation → naming → judgment)

Results and Discussion(word presentation → judgment)

2.497

2Stimulus Types

 The two groups’ rating pattern became
significantly different (P < 0.0001).

The two groups’ relatedness ratings

The trend of ratings on control stimuli

The two groups’ relatedness ratings

Native Speakers of English

Native Speakers of Japanese

JPN Rating (None)

ENGL Rating (None)

Native Speakers of Japanese

Native Speakers of English

Native Speakers of English

Native Speakers of Japanese

Native Speakers of English

998

376
58

F(2,76)=3.289, P=0.05
5.174

4.837

4.291 4.422

432.2342.753

211None The two groups’ rating pattern became
different (P < 0.05).

Native Speakers of Japanese

2ModifierResults

Relatedness Ratings

3.058

3Results

0102030
Trials405060

A task effect observed

765432HeadModifierNone01020Stimulus Types

30Trials

405060

The trend of ratings on control stimuli

Native Speakers of Japanese
Native Speakers of English
987
65F(2,76)=11.375, P=0.0001
5.567
4.923

4.739 4.461

42.694

3

JPN Rating (None )ENGL Rating (None)

Nativ e Spe akers of Japanese

Native Speake rs of English

92.06528The trend from Experiment
2 remains unchanged.

765432111

HeadModifier

None

Stimulus Types

01020304050

TrialsReferences

General Discussion

Hypothesis 2:

ConcreteImage

Word Level/hanabi/ The same as in Experiment 1

 Pairs of pictures were presented on a computer screen
simultaneously by the PsyScope.
The screen shot and procedure in Experiment 1
 The participants were told to judge
the similarity between the twopresented pictures and click anappropriate point on a 9-point scale.

HeadYES consistent with Baayen & Boroditsky (in press)

nounStimuli

Procedure1

There is morphological effect on picture relatedness.

noun

花束花束火火 20 native speakers of Japanese 20 native speakers of English

Relatedness Ratin gs

Does structural relatedness in morphology leads toperceptual similarity (picture relatedness) ?

noun

Word Level
花火
花火

Orthographic Representation

Participants

Relatedn ess Ratings

Research Question:

VisualForms/soku/ (ON)Phonological Representation

 Japanese speakers rated higher on target stimuli but also showed
higher ratings on control stimuli.
 Difference on control stimuli could be attributed to inherent cultural
The two groups’ relatedness ratings
The trend of ratings on control stimuli
difference, rather thanNative Speakers of Japanesetask effects.
Native Speakers of English
99 The two groups showed
88F(2,76)=0.077, P=0.93
7
No task effect7
identical rating patterns
66
5.171
4.539
54.3295(p=0.93).
3.739
44

Research Scheme

束束‘bundle’花花‘flower’Aim

Relatedness Ratings

The Linguistic Relativity Hypothesis (LRH),

火火‘fire’

Lemma
Representation

AbsoluteReality/taba/ (KUN)/hi/ (KUN)

束束Orthographic Representation

Maximum word activation

花束花束 ‘bouquet’
Abstract Image

ConceptualRepresentation

To test if word
activations in themind influence
picture comparison.

VisualPerception

Stimuli

Baayen & Boroditsky (in press) revealed thatmorphological structural relatedness in Dutch(e.g. vrachtwagen & kinderwagen) influenced Dutch
speakers’ perception and conceptualization of reality bymeans of picture comparison tasks.

Carrying more meaning, head constituents
have more influence on picture relatedness
than modifier constituents.

花火
花火 ‘firework’R elated ness Ratingse.g.
fireworks (English)
hanabi ‘flower-fire’(Japanese)yànhuŎ ‘flame-fire’(Mandarin Chinese)

Absence of deliberate word activation

AimRelated ness Ratin gs

Different languages use different words to express thesame concept and thus have different sharedconstituency patterns.

Experiment 3
Word Image Comparison

Experiment 2
Picture Comparison & Naming Contrary to Baayen and Boroditsky (in press) study, shared constituency phenomenon did not influence thetarget group’s picture relatedness ratings significantly in the picture comparison paradigm.
 Picture Perception does not lead to automatic word activations in the mental lexicon (or very weak activation).
 Picture comparison could influence language performance (Ikeda, 1994), but not vice versa.
Influence of language on picture relatedness is conditional
 Influence of shared constituency on picture relatedness ratings are positively correlated with the degree ofword activation in the mind.

Baayen, R. H. & Boroditsky, L. (in press). Lexical effects on picture comparison,Elsevier Science.
Cohen, J. D., MacWhinney, B., Flatt, M., & Provost, J. (1993). Psyscope: A interactive
graphic system for designing and controlling experiments in psychology laboratory
using macintosh computers. Behavioral Methods, Research, Instruments, andComputers, 25, 257-271.
Freeimages.co.uk. (n.d.). Retrieved October 6, 2006, from http://www.freeimages.co.uk/
Ikeda, S. (1994). A study on the process of translating English and Japanesewords: An examination of interference and facilitation from pictures.
The Japanese Journal Psychology, 65, 121-129.
Lucy, J. (1997). Linguistic relativity. Annual Review of Anthropology, 26, 291-312.
Tamaoka, K. & Hatsuzuka, M. (1998). The effects of morphological semantics on theprocessing of Japanese two-kanji compound words. Reading and Writing:
An interdisciplinary Journal, 10, 293-322.

This research was supported by a Major Collaborative Research Initiative Grant (#412-2001-1009) from the Social Sciences and Humanities
Research Council of Canada to Gary Libben (Director), Gonia Jarema, Eva Kehayia, Bruce Derwing, and Lori Buchanan (Co-investigators).
The authors would like to extend thanks to Dr. Bernard Linsky, who supported the follow-up project through the Roger Smith Undergraduate
Researcher’s Award. Without his support, the present research would have never been developed to the current extent.

60