Lexical Activations in Picture Comparison: A Cross-Linguistic Approachto the Relation between Language and Thought in the Mental Lexicon
Koji Miwa*, Gary Libben, Sally Rice, R. Harald Baayen
Department of Linguistics, University of Alberta, Edmonton, T6G2E7, Canada* Correspondence should be addressed to kmiwa@ualberta.ca

Abstract
Given that word translation equivalents in different
languages can give rise to different word structures
which can cast different shades of meaning, thisstudy investigates whether such cross-linguistic
differences influence how speakers of different
languages compare two objects. Picture comparisontasks revealed that speakers of Japanese andEnglish utilize distinct cognitive processes when
asked to evaluate how similar two objects are,influenced by various lexical properties of the wordtranslation equivalents of the names for the objects
in each language. This result provides partial
support for the Linguistic Relativity Hypothesis,
which holds that the language speakers are exposed
to influences their conception of reality, ahypothesis which allows us to explore empiricallythe relationship between language and thought inthe mental lexicon.

The Linguistic Relativity Hypothesis, which holds
that the specific language that people are normally
exposed to influences their conceptualization of theworld (Whorf, 1956), has attracted cognitive
scientists for the last half century mainly as ameans to study what thought is and how strongly
non-linguistic cognition is influenced by language(Boroditsky, 2003; Pinker, 1994; Vygotsky, 1986).
From a psycholinguistic perspective, this study
explores what lexical properties contribute to apicture comparison task and discusses how thetraditional dichotomy between language andthought can be interpreted within the mentallexicon. With regard to language processing tasks,
the fundamental nature of the mental lexicon is tomaximize opportunities by redundant activations oflexical items (Libben, 2006). However,psycholinguists have not tended to focus onwhether the mental lexicon is involved in nonlinguistic tasks as well and, if so, which propertiesof lexical items actually contribute to nonlinguistic cognitive processing.

Different languages use different words toexpress the same concepts, and cross-language
differences in word structure might reflect slightbut perhaps significant differences in meaning.
Many studies on language and thought suggest thatdifferent language groups behave differently invarious non-linguistic tasks (Winawer et al., 2007;
Baayen & Boroditsky, 2004; Majid et al., 2004;
Lucy, 1992). Following this line of research, thepresent study investigated whether a task tocompare and rate the degree of similarity between
two pictorial objects involves language-specificcognitive processing. Baayen and Boroditsky(2004) conducted a picture comparison task to
explore whether language-specific morphologicalstructures in Dutch and English (e.g. the related
Dutch vrachtwagen and kinderwagen compared totheir morphologically unrelated English translationequivalents truck and stroller) influences how
people perceive the same line-drawn objects andwhat lexical properties contribute to this picturecomparison process. Their study revealed that thedegree of judged picture relatedness is influenced
by the morphological structure of the words used
to describe the drawings in the two languages.

Picture Comparison TaskTo investigate the cross-linguistic applicability of
their research, the present study compared thebehaviour of Japanese and English speakers inpicture comparison tasks. The primary questions
addressed in this study were (1) whether
differences in conceptual familiarity enhanced
perceived picture similarity and (2) whether
pictorial objects that have single word
representations give rise to greater picture
similarity ratings.

Figure 1: Significant predictors and interactions in picture comparison tasks
Note: The bottom two graphs show the effects of English names for the objects depicted: Whether they contain one(=FALSE) or two words (=TRUE). The names were always two-character compounds in Japanese. Monomorphemic
words were grouped together with non-spaced compounds.

MethodParticipants Twenty native speakers of Japaneseand twenty native speakers of English wererecruited at the University of Alberta and inneighboring cities. Each participant wascompensated at a rate of $10 Canadian for a thirtyminute session.
Materials Sixty pairs of photographs of objects
were prepared. All the depicted objects weredescribable as two-character compounds inJapanese, and forty of the pairs shared aconstituent in Japanese, but not in their English
translation equivalents. Among these linguisticallyrelated pairs, twenty object pairs shared the firstmodifier constituent of compounds in Japanese,but not in English (shared modifier condition: e.g.
hana-bi ‘firework’ and hana-taba ‘bouquet’).
Another twenty object pairs shared the second
head constituent in the Japanese compounds but

were not morphologically related in English at all(e.g. senro ‘railroad’ and mero ‘maze’). The
remaining twenty object pairs shared no linguistic
element in either Japanese or English.
Procedure In each trial, a pair of two pictorial
objects was presented side-by-side on a computer
screen with a nine-point relatedness scale
underneath. Participants were asked to compare
the simultaneously presented picture pairs anddecide how similar the two objects were byclicking an appropriate point on the relatedness
scale (from ○△ = very different to ○○ = verysimilar).

Results
In a mixed effect linear regression analysis, as
shown in Figure 1, four predictors were found tocontribute significantly to the relatedness ratings.
LSA word similarity (co-occurrence valuesbetween the left and right pictures’ English wordequivalents) had the same facilitatory effect on
both groups (p=0.0002). This measure taps into
general shared conceptual similarity acrossJapanese and English speakers. Japanesecompound entropy (Shannon’s entropy calculated
for the probability distribution of frequency countsof the left and right pictures’ word translationequivalents) was a significant facilitatory predictor
for the Japanese group, but not for the English
group (p=0.0001). This suggests that languagespecific familiarity with two-character compounds
codetermined the ratings. English compound
spacing (whether English word translationequivalents have a space between the twoconstituents of a compound or not or whether theobject pairs were lexicalized by simplex words)
interacted significantly with groups (p=0.02), andpictorial objects that have single word
representations gave rise to greater picture
similarity ratings.

Discussion
During the supposedly non-linguistic picturecomparison tasks, certain lexical properties
contributed to speakers’ judgments of picture
relatedness. The effects of frequency-based
predictors, English LSA word similarity andJapanese compound entropy, were large. Suchrobust involvement of frequency-based predictors
in a non-linguistic task reassures us that the locus
of word frequency effects is in thesemantic/conceptual domain, rather than just theperceptual domain, and that these effects reflect
the degree of entrenchment of concepts asconceptual familiarity (Baayen, Feldman, &Schreuder, 2006). Unlike the effects of LSA word
similarity and Japanese compound entropy, theeffect of orthographic spacing in the compound
was tiny. Nevertheless, all were significant in a
linear mixed effect model with subject and picture
pair as crossed random effects. That the effect wassmall was expected, since the speakers in the twogroups had many more commonalities thandifferences. In line with previous studies, thepresent study saw subtle differences reflecting
how language experience affects non-linguisticcognition. The study thus provides some support to
the Linguistic Relativity Hypothesis when therelationship between language and thought isscrutinized across items in speakers of different
languages’ mental lexicons.

Acknowledgments
This study was supported by the Social Sciencesand Humanities Research Council of Canada as aMajor Collaborative Research Initiative Grant(#412-2001-1009) to Gary Libben (Director),Gonia Jarema, Eva Kehayia, Bruce Derwing, andLori Buchanan (Co-investigators).

References
Baayen, R. H., & Boroditsky, L. (2004, October)
Effects of lexical structure on picturecomparison. Presented at Conceptual Structure,
Discourse, & Language. Edmonton, Canada.
Baayen, R. H., Feldman, L. B., & Schreuder, R. S.
(2006) Morphological influences on therecognition of monosyllabic monomorphemic
words. Journal of Memory and Language, 55.
290-313.
Boroditsky, L. (2003) Linguistic Relativity. In L.
Nadel (Ed.), Encyclopedia of Cognitive Science.
London: MacMillan.
Libben, G. (2006) Why study compoundprocessing? An overview of the issues. In G.
Libben & G. Jarema (Eds.), The Representation
and Processing of Compound words (pp. 1-22).
New York: Oxford University Press.
Lucy, J. A. (1992) Grammatical Categories andCognition: A Case Study of the Linguistic
Relativity Hypothesis. New York: CUP.
Majid, A., Bowerman, M., Kita, S., Haun, B. M.
D., & Levinson, C. S. (2004) Can language
restructure cognition? The case for space.
TRENDS in Cognitive Sciences, 8 (3), 108-114.
Pinker, S. (1994) The Language Instinct. New
York: William Morrow.
Vygotsky, L. (1986) Thought and Language. (E,Haufmann & G. Vakar. Trans.). Cambridge,
MA: MIT Press. (Original work published 1934)
Whorf, B. L. (1956) Science and linguistics. In J.
B. Carroll (Ed.), Language, Thought andReality: Selected Writings of Benjamin Lee
Whorf. Cambridge, MA: MIT Press.
Winawer, J., Witthoft, N., Frank, M. C., Wu, L.,
Wade, A. R., & Boroditsky, L. (2007) Russian
blues reveal effects of language on color
discrimination. PNAS (pp. 7780-7785).