Ostarek, M., & Huettig, F. (2019). Six challenges for embodiment research. *Current Directions in Psychological Science*, 28, 593–599.
https://doi.org/10.1177/0963721419866441

Paivio, A., Yuille, J. C., & Madigan, S. A. (1968). Concreteness, imagery, and meaningfulness values for 925 nouns. *Journal of Experimental Psychology*, 76, 1–25.
https://doi.org/10.1037/h0025327

Pecher, D. (2018). Curb your embodiment. *Topics in Cognitive Science*, 10, 501–517.
https://doi.org/10.1111/tops.12311

Perea, M., & Rosa, E. (2002). The effects of associative and semantic priming in the lexical decision task. *Psychological Research*, 66, 180–194. https://doi.org/10.1007/s00426-002-0086-5

Plaut, D. C. (1995). Semantic and associative priming in a distributed attractor network. In Cognitive Science Society (Ed.), *Proceedings of the Seventeenth Annual Conference of the Cognitive Science Society* (pp. 37–42). Hillsdale, NJ: Erlbaum

Plaut, D. C., & Booth, J. R. (2000). Individual and developmental differences in semantic priming: Empirical and computational support for a single-mechanism account of lexical processing. *Psychological Review*, 107, 786–823. https://doi.org/10.1037/0033-295X.107.4.786

Pulvermüller, F. (2013). How neurons make meaning: Brain mechanisms for embodied and abstract-symbolic semantics. *Trends in Cognitive Sciences*, 17, 458–470.
https://doi.org/10.1016/j.tics.2013.06.004

Pulvermüller, F., Shtyrov, Y., & Ilmoniemi, R. (2005). Brain signatures of meaning access in action word recognition. *Journal of Cognitive Neuroscience*, 17, 884–892.
https://doi.org/10.1162/0898929054021111

Qualtrics (2020) Qualtrics.com. Available at: http://www.qualtrics.com/

Recchia, G., & Jones, M. N. (2012). The semantic richness of abstract concepts. *Frontiers in Human Neuroscience*, 6. https://doi.org/10.3389/fnhum.2012.00315

Roelofs, A. (1997). The WEAVER model of word-form encoding in speech production. *Cognition*, 64, 249–284. https://doi.org/10.1016/S0010-0277(97)00027-9

Rogers, T. T., & McClelland, J. L. (2004). *Semantic cognition: A parallel distributed processing approach*. Cambridge, MA: MIT Press.

Sánchez-Casas, R., Ferré, P., García-Albea, J. E., & Guasch, M. (2006). The nature of semantic priming: Effects of the degree of semantic similarity between primes and targets in Spanish. *European Journal of Cognitive Psychology*, 18, 161–184.
https://doi.org/10.1080/09541440500183830

Schwanenflugel, P. J., Harnishfeger, K. K., & Stowe, R. W. (1988). Context availability and lexical decisions for abstract and concrete words. *Journal of Memory and Language*, 27, 499–520. https://doi.org/10.1016/0749-596X(88)90022-8

Smith, E. E., Shoben, E. J., & Rips, L. J. (1974). Structure and process in semantic memory: A featural model for semantic decisions. *Psychological Review*, 81, 214–241.
https://doi.org/10.1037/h0036351

Sperber, A. D., Devellis, R. F., & Boehlecke, B. (1994). Cross-cultural translation. *Journal of Cross-Cultural Psychology*, 25, 501–524. https://doi.org/10.1177/0022022194254006

Vigliocco, G., Meteyard, L., Andrews, M., & Kousta, S. (2009). Toward a theory of semantic representation. *Language and Cognition*, 1, 219–247.
https://doi.org/10.1515/langcog.2009.011

Vigliocco, G., Vinson, D. P., Lewis, W., & Garrett, M. F. (2004). Representing the meanings of object and action words: The featural and unitary semantic space hypothesis. *Cognitive Psychology*, 48, 422–488. https://doi.org/10.1016/j.cogpsych.2003.09.001
Vigliocco, G., & Vinson, D. P. (2007). Semantic representation. In G. Gaskell & G. Altmann (Eds.), *The Oxford handbook of psycholinguistics* (pp. 195–215). Oxford, England: Oxford University Press. https://doi.org/10.1093/oxfordhb/9780198568971.013.0012

Villani, C., Lugli, L., Liuzza, M. T., & Borghi, A. M. (2019). Varieties of abstract concepts and their multiple dimensions. *Language and Cognition, 11*, 403–430.
https://doi.org/10.1017/langcog.2019.23

Vinson, D. P., & Vigliocco, G. (2008). Semantic feature production norms for a large set of objects and events. *Behavior Research Methods, 40*, 183–190.
https://doi.org/10.3758/BRM.40.1.183

Vivas, J., Vivas, L., Comesaña, A., Coni, A. G., & Vorano, A. (2017). Spanish semantic feature production norms for 400 concrete concepts. *Behavior Research Methods, 49*, 1095–1106. https://doi.org/10.3758/s13428-016-0777-2

Wiemer-Hastings, K., & Xu, X. (2005). Content differences for abstract and concrete concepts. *Cognitive Science, 29*, 719–736.
https://doi.org/10.1207/s15516709cog0000_33

Wang, H. & Song, M. (2011). Ckmeans.1d.dp: optimal k-means clustering in one dimension by dynamic programming. *R Journal, 3*, 29–33.

Wu, L., & Barsalou, L. W. (2009). Perceptual simulation in conceptual combination: Evidence from property generation. *Acta Psychologica, 132*, 173–189.
https://doi.org/10.1016/j.actpsy.2009.02.002

Yap, M. J., & Balota, D. A. (2009). Visual word recognition of multisyllabic words. *Journal of Memory and Language, 60*, 502–529. https://doi.org/10.1016/j.jml.2009.02.001

Yarkoni, T., Balota, D., & Yap, M. (2008). Moving beyond Coltheart's N: A new measure of orthographic similarity. *Psychonomic Bulletin and Review, 15*, 971–979.
https://doi.org/10.3758/PBR.15.5.971

Zwaan, R. A. (2004). The immersed experiencer: Toward an embodied theory of language comprehension. *Psychology of Learning and Motivation - Advances in Research and Theory, 44*, 35–62. https://doi.org/10.1016/S0079-7421(03)44002-4
The previous chapter was intended to bridge the gap in the French literature as regards to the
availability of semantically similar pairs for abstract concepts. As such, it proposed the first
French database composed of abstract semantically similar pairs of concepts. The database
also included a concreteness variable in order to answer the suggestion of Crutch and
Warrington (2011) about a difference in the organisation of the mental lexicon in terms of
semantic similarity and verbal association for concrete and abstract concepts respectively. In
addition, previous findings have established the need to systematically differentiate between
these two major types of relationships (McRae & Boisvert, 1998). Ferrand and New (2003)
have applied this strict distinction between semantic similarity and verbal association by using
pairs of concepts that were semantically similar but not verbally associated and obtained a
pure semantic effect in a priming experiment. These results illustrate the need to have access
to controlled stimuli and to provide the ability to researchers to check the link between chosen
materials. In order to address such a need, the next chapter introduces a database for word
associations between concepts using the same words as in the previous database. Users will be
able to craft materials that is controlled on both the semantically similar and associative
dimensions.

The amount of data generated by the word association task allowed for the use of
techniques borrowed from distributional statistics from which a vocabulary growth curve was
plotted. Correlation analyses revealed that cue words presented to the participants elicited
response words of a similar level of concreteness meaning that concepts are organised in the
mental lexicon according to a gradient of concreteness. Analyses from associative strength
have shown that concrete concepts elicited stronger associations compared to abstract
concepts. Finally, association frequencies were computed to predict semantic processing and
showed that the more salient a word is in the mental lexicon, the less time it takes to be
processed. The entire database is available on OSF.
# Chapter 3. Word associations norms for 1,100 French words
with varying levels of concreteness

Dounia Lakhzoum, Marie Izaute, and Ludovic Ferrand

**Chapter 3 was submitted as:** Lakhzoum, D., Izaute, M., & Ferrand, L. (submitted). Word associations norms for 1,100 French words with varying levels of concreteness. *Journal of Psycholinguistic Research*.

Supplementary material: https://osf.io/dhuqs/

Université Clermont Auvergne, CNRS, LAPSCO, F-63001 Clermont-Ferrand, France

**Corresponding author:** Dounia Lakhzoum or Ludovic Ferrand, Université Clermont Auvergne, CNRS LAPSCO, 34 avenue Carnot TSA 60401-63001 Clermont-Ferrand Cedex 1, France

Phone: + 33 4 73 40 62 57 – Fax: + 33 4 73 40 64 82

Email: dounia.lakhzoum@uca.fr ; ludovic.ferrand@uca.fr

**Author's note**

This research was financed by the French government IDEX-ISITE initiative 16-IDEX-0001 (CAP 20-25).

## ACKNOWLEDGMENT

This work was sponsored by a public grant overseen by the French National Research Agency as part of the "Investissements d'Avenir" through the IDEX-ISITE initiative CAP 20-25 (ANR-16-IDEX-0001).
**ABSTRACT**

The organization of concepts in the mental lexicon is a widely studied phenomenon in experimental psychology. For instance, several studies have shown that whereas concrete concepts are organized according to semantic similarity, abstract concepts are organized according to verbal association. However, these results are not systematically replicated, mainly because there is no normative database for studying the phenomenon. This is particularly true for the French language. To that end, we compiled a French word association database of 1100 cues with varying levels of concreteness from abstract to concrete concepts.
First results revealed stronger association strength for concrete concepts than abstract concepts, confirming the existence of different organisation patterns, with concrete concepts more closely connected to a small number of contexts while abstract concepts are more loosely connected to a high number of varied contexts. Additional results revealed that cues and responses were organised according to a concreteness and perceptual strength gradient.
These results provide insights into the rules governing organisation of the mental lexicon. The full data are available in the supplementary materials as well as on OSF (https://osf.io/dhuqs/).

**Keywords:** Word association norms, Concreteness, Normative ratings, French words.
# INTRODUCTION

Psycholinguistic experiments rely on normative stimuli. Two types of norms have a long tradition in the field: semantic similarity and word association norms. Semantic similarity refers to the overlap between the defining features of concepts (for example <bee> and <wasp> are semantically similar). In the case of word associations, two words are considered associates when they frequently occur together across language (for example <bee> and <honey> are verbal associates; McNamara, 1992; Plaut, 1995; Ferrand & Alario, 1998).

## Semantic and associative priming

The development of datasets addresses the crucial need to access carefully crafted material when studying cognitive phenomena such as free recall (Nelson, McEvoy & Schreiber, 2004), age-related lexical access (Taconnat et al., 2009), metamemory (Eakin, 2005; Bugaiska, Morson, Moulin & Souchay, 2011; West & Mulligan, 2019), or clinical pathologies such as schizophrenia (Kircher, Whitney, Krings, Huber & Weis, 2008) and semantic refractory access dysphasia (Crutch & Warrington, 2005).

For language study in particular, the importance of strictly differentiating between associative and semantic relations when studying the mental lexicon has long been established (McRae & Boisvert, 1998; Perea & Rosa, 2002; Thompson-Schill, Kurtz, & Gabrieli, 1998; Lucas, 2000; Hutchison, 2003; Ferrand & New, 2003). The semantic priming effect is one of the most robust effects for studying the mental lexicon. Historically, at least two types of major lexical relations have been used in psycholinguistic experiments, associative and semantic relations, occasionally with some overlaps between relation types (for example <cat> and <dog> are semantically similar but also verbal associates). As a result, several discrepancies have been reported in the literature as regards semantic and associative-priming effects (see, for example, Thompson-Schill et al., 1998, but also Hutchison, 2003). McRae
and Boisvert (1998) suggested we use semantically similar pairs that are not associates (for example <whale> and <dolphin>). On the basis of this suggestion, Ferrand and New (2003) obtained a pure semantic priming effect by using stimuli that were semantically similar while controlling for verbal association (e.g., <dolphin> and <whale>). Conversely, they were able to obtain a pure associative priming effect by using strongly associated stimuli with no semantic similarity (e.g., <spider> and <web>). Hence, researchers rely heavily on word association norms either to select associated word pairs or to check that semantically similar pairs are not also verbally associated. This robustness of semantic and associative priming reported in the literature reflects how the mental lexicon is organised. More recently, the debate on the nature of what is measured by associations and similarity has evolved to include distributional semantic models derived from co-occurrences compared to word association models (Schloss & Li, 2016). Recent evidence has shown that word associations collected from human participants usually outperform text models (De Deyne, Verheyen & Storms, 2016; Van Rensbergen et al., 2016; Vankrunkelsven et al., 2018). This addition to the debate also served to re-establish the place of word associations as a way of studying psychological meaning rather than as just a tool for creating only semantically similar pairs. Recent studies have shown, for instance, how it is possible to compute cosine similarity based on word-association data (see De Deyne, Navarro, Perfors, Brysbaert & Storms, 2019 for an example).

With the introduction of algorithms based on distributional statistics, the frontier between associative strength and semantic similarity is no longer needed. To focus exclusively on the two aforementioned modes of organisation of the mental lexicon would not provide as full a picture, however, as recent studies on concreteness have shown.

## Concreteness and organisation of the mental lexicon

Recent studies have shown that concreteness must be taken into account in addition to association strength and semantic similarity in order to have a fuller picture of how the mental
lexicon is organised. Crutch and colleagues (2005; Crutch, Connell & Warrington, 2009; Crutch and Warrington, 2010) reported a case study of a patient with symptoms of semantic refractory access dysphasia. They found that, for this patient, concrete concepts elicited only a semantic priming effect and no associative priming effect, whereas abstract concepts elicited the opposite pattern. This finding suggests that while concrete concepts are organised according to semantic similarity, abstract concepts are organized according to verbal association. Several studies tried to replicate this pattern with discrepancies (e.g., Hamilton & Coslett, 2008; Duñabeitia, Avilés, Afonso, Scheepers & Carreiras, 2009; Geng & Schnur, 2015). Crutch and Jackson (2011) suggested the disparities might be due to how the concreteness variable was operationalised. In two semantic odd-one-out judgement and matching-to-sample paradigms, they used triplets of low, middle and high levels of concreteness. Results showed that the semantic effect increased with concreteness while the associative effect decreased. Furthermore, they suggested that concreteness be used as a graded variable rather than a binary one, especially when studying its effect on the organisation of semantic representation.

## Accounts of semantic representation

In recent years, two main accounts of the nature of semantic representation have been debated, namely the distributional and embodiment accounts. They differ in terms of the information used to represent meaning. While distributional semantics relies on symbolic and linguistic features, embodiment relies on modality-specific information.

According to the distributional hypothesis, words occurring in similar contexts have similar meanings (Harris, 1954). The meaning of both abstract and concrete concepts can therefore be derived from statistical co-occurrences across large corpora. Hence, this account makes great use of variables, such as semantic and relatedness variables, that describe associations between words (see Louwerse & Jeuniaux, 2010; Andrews, Frank & Vigliocco,
2014; Borghi, Binkofski, Castelfranchi, Cimatti, Scorolli & Tummolini, 2017). It is an account
which has come under attack in recent years, however, because it fails to explain how concepts
are grounded in world referents.

The embodied account, on the other hand, defines semantic representation as being
grounded in perceptual and motor states (Barsalou, 1999; Glenberg, 1997; Zwaan, 2004; Kiefer
& Pulvermüller, 2012; Meteyard, Cuadrado, Bahrami & Vigliocco, 2012). Studies of this
account usually show evidence of behavioural and brain activation of modality-specific
processes, which makes it better suited to explain the processing of concrete concepts over
abstract concepts. Recent hypotheses have been formulated to account for the processing of
abstract concepts within the embodied framework, with studies showing that abstract concepts
are grounded in social and introspective aspects of situations (Barsalou & Wiemer-Hastings,
2005), emotional features (Kousta, Vigliocco, Vinson, Andrews & Del Campo, 2011; Lenci,
Lebani & Passaro, 2018), information about events, and thematic roles (Ferretti, McRae &
Hatherell, 2001).

To capture relations other than word associations for both abstract and concrete concepts
new variables have emerged. For example, the perceptual strength variable developed by
Chedid, Brambati, Bedetti, Rey, Wilson & Vallet (2019) aims to yield information about
perceptual experiences. It represents the extent to which a concept relies on sensorimotor traits
for its representation. Accordingly, their dataset comprises perceptual strength norms for more
than 3 000 nouns. It contributes to the study of relations between word associations and
perceptual modalities.

Word associations and normative datasets

In a word association task, participants are presented with a word (the cue) and asked
to generate the first word that comes to mind (the response). The task is particularly effective
and powerful because it demands barely any cognitive cost and is easily gamified, and yet it
can unveil shared mental representations, language patterns and mechanisms of meaning
extraction (Prior & Bentin, 2008; Mollin, 2009, De Deyne & Storms, 2008; De Deyne et al.,
2019). Word association norms have mostly been collected in English. The *Edinburgh
Associative Thesaurus* (Kiss, Armstrong, Milroy & Piper, 1972), with more than 8 000 cues,
is one of the most commonly used. More recently, Moss and Older (1996) introduced the
*Birkbeck norms* with over 2 600 cues in British English. The more recent set of *University of
South Florida norms* (Nelson et al., 2004) introduced free association data for 72 000 word
pairs and is the most widely used, with more than 2 000 citations from psycholinguistics to
computational linguistics. Even more recently, De Deyne et al. (2019) launched the *Small
World of Words* project (SWOW-EN), an ongoing megastudy which aims to capture most
word associations. It consists of over 12 000 cue words, each of which has elicited 300
responses from half a million participants at the time of writing this paper, making this dataset
the most comprehensive one to date. This particular project followed a similar one by De
Deyne, Navarro & Storms (2013) which consisted of 16 000 cues in Dutch (SWOW-NL).

Other languages also have word association datasets. The number of cues is still significant,
albeit reduced, and the datasets are still highly valuable for researchers studying these
languages, such as Korean (Jung, Na & Akama, 2010), Japanese (Joyce, 2005) and Chinese
(Kwong, 2013). In Spanish, Fernández, Díez, Alonso & Beato (2004) introduced free-
association norms for 247 frequent words, and, more recently, Barron-Martinez and Arias-
Trejo (2014) presented the *Spanish Word Association Norms* for 234 concrete nouns in
Mexican Spanish. In French, Ferrand and Alario (1998) introduced word associations for 366
concrete nouns. They were quickly followed by a new dataset of 260 abstract words (Ferrand,
2001). Later on, Duscherer and Mounoud (2006) collected free association data for 151
French action verbs, and, more recently, Bonin, Méot, Ferrand & Bugaïska (2013) collected
word associations for 520 concrete words. As evidenced by all these datasets, the history of
the collection of word associations in French is rather spread out in time, making the present
study the first for nearly a decade and the one involving the largest number of cues from the
largest number of participants for the French language.

The present study

The present study introduces a dataset comprising word association norms for 1 100
cues obtained from French participants. The method used allowed only one response per
word, whereas other studies such as De Deyne et al., (2019) used several. Nelson, McEvoy
and Dennis (2000) reported a high reliability coefficient computed from the correlation
between the probability of a single-response task and that of a first-response when two
responses were collected. This high level of reliability suggests the discrete responses
collected for the present dataset are sufficient to capture stable association strength. The cues
on the concreteness spectrum varied, and we added a measure of cue word concreteness
collected in a previous study (Lakhzoum et al., 2021) which will allow researchers who study
the organisation of the mental lexicon to select abstract and concrete concepts for cues. To
provide a dataset of experimental stimuli according to the significant lexical variables and
lexical latencies previously discussed, we have combined our list of words with existing
databases such as the French Lexicon Project (FLP, Ferrand et al., 2010), Lexique (New et al.,
2004), Megalex (Ferrand et al., 2018), and Wordlex (Gimenes & New, 2016). In response to
Crutch and colleagues' observations, our dataset includes the levels of concreteness for the
cues previously collected by Lakhzoum, Izaute and Ferrand (2021). With the exception of
Ferrand et al., 2001, to the best of our knowledge this is the first time a deliberate effort has
been made to include abstract concepts in a French word association dataset.
# METHOD

## Participants

The word association task was presented as an online questionnaire. All the participants were native French-speakers aged between 18 and 45 years old. We collected data from 1200 participants (627 women; $M_{age} = 29.30$; SD = 7.48). Participants volunteered in response to an announcement posted on Facebook group walls and received no payment. The study was approved by the Research Ethics Committee of Clermont Auvergne University.

## Stimuli

The stimuli were composed of the isolated words used to create the materials in Lakhzoum, Izaute and Ferrand (2021), which were originally based on Coltheart's (1981) concreteness norms so as to ensure that varying levels of concreteness were included, with abstract words *monarchy* and *joy* as well as concrete words such as *dress* and *bee*. The words had been back-translated and used to form semantically similar pairs. For the present study, the pairs were dissociated; cue and target were separated in different lists so that participants would not see two words that shared semantic features to avoid influencing their responses. To that end, and to keep the task short, since participants' focus in online tasks tends to be less than in non-online tasks, we divided the material into 11 lists of 100 words each.

## Procedure

The stimuli were divided into 11 lists of 100 isolated cues. Each participant was randomly assigned to only one of the lists. The experiment was conducted online using Qualtrics software (2020). The experiment interface was designed to allow participants to complete the task on either a computer or smartphone. Their task was to read the word that appeared on the screen and to answer in the space below with the first word that came to mind as quickly as possible. The cue words appeared on the screen one after another. They were replaced as soon as the participant had answered. To avoid unacceptable responses, when
participants could not think of an answer they were given the opportunity of leaving a blank
and moving onto the next cue. The cues were presented in the middle of the screen in Arial 12
font against a white background. No training was given before the tasks started. The study was
self-paced, with no time limit for either the stimulus presentation or the participant's answer.
The task took about 15 minutes to complete.

## RESULTS

### Data Pre-processing

Before performing any analysis, the dataset was cleaned to remove any responses that failed to meet our criteria for inclusion. Symbols and punctuation such as tags, quotes, final punctuation and double spaces were removed. We also removed responses that seemed to indicate participants did not know what to answer. For instance, they sometimes entered “rien”, “aucune réponse” or “aucun mot” in French (nothing, no response and no word), indicating that no response came to mind, instead of leaving a blank as the procedure allowed them to do. Finally, responses that were phrases intended to describe what the cue brought to mind were removed. This concerned 523 responses. The responses were then spell-checked and lemmatized by cross-referencing the dataset with Lexique (New et al., 2004). By the end of the process, 2.5% of the data had been removed. Analyses were performed on the remaining 89 707 unique responses or tokens.

### Distributional statistics

A first exploration of the dataset focused on its distributional characteristics. When all responses were aggregated, there were 10 604 distinct words or *types*, 4 233 of which appeared only once. The responses occurring only once are known as *hapax legomena* responses. In accordance with De Deyne et al. (2013, 2018, but see Nelson et al., 2004), we decided to keep these responses because they reflect the long tail of the frequency spectrum. When sorting the
responses according to cue median concreteness, the hapax legomena were equally distributed
at 53% for the responses elicited by abstract cues and 55% for the responses elicited by concrete
cues. These aspects of quantitative linguistics can be a good tool for capturing common
linguistic phenomena across datasets of varying sizes. For instance, Figure 1 is a frequency
spectrum of the distribution of the number of types ($V_m$) as a function of frequency class ($m$).

<figure><img src="image_15.png" /><figcaption>**Figure 1.** Frequency spectrum on a logarithmic scale representing the number of types (Vm) as a function of frequency class (m). The class corresponding to m = 1 corresponds to the hapax legomena.</figcaption></figure>

Figure 1 shows that there are more than 4 000 words in frequency class *V*₁, which represents the *hapax legomena*, but very few words in frequency class *V*₅₀. This pattern, characterized by few exemplars in low frequency classes and many exemplars in high frequency classes and is consistently found across corpora and languages (Evert & Baroni, 2005). The frequency spectrum indicates that the process for the present word association task is productive, meaning that if we were to sample more responses from participants the number of types would also increase. This phenomenon is best captured by Herdan's law (1964), which describes a bounded exponential relation between the number of types and corpus size. The process can be illustrated by a vocabulary growth curve, which plots how the number of distinct cue-response pairs evolves as the number of collected responses increases. The *V* and *Vm* estimates of the
frequency spectrum can be extrapolated using a finite Zipf-Mandelbrot model<fnref n="4" /> (fZM, Evert & Baroni, 2007; Baroni & Evert, 2006) to obtain estimates of vocabulary growth at arbitrary sample sizes (see Figure 2). It is customary to generate a vocabulary growth curve for about twice the size of the observed sample (here N = 180,000).

<figure><img src="image_16.png" /><figcaption>**Figure 2.** Vocabulary Growth Curve (VGC) representing the increase of cue-response pairs as new responses are collected by comparing the VGC of our dataset (grey curve) to an empirical VGC obtained from the fZM model red line). The vertical lines indicate the total number of observed tokens for our sample.</figcaption></figure>

The vocabulary growth curve compares the empirical growth obtained from the fZM model with the observed growth in our dataset. For the present word-association task, the curves express how the number of types (here, distinct cue-response pairs) increases as a function of the number of tokens (here, the responses collected). According to Figure 2, the observed and extrapolated curves almost completely overlap, providing a clear visual assessment of the model’s goodness of fit for items sampled from our dataset.

<footnote n="4">⁴ The finite ZM model was preferred to the ZM model because the latter is founded on the unlikely assumption of an infinite vocabulary size (Evert, 2004).</footnote>
## Cue Descriptive statistics

General statistics were first computed for the cue words. The mean concreteness variable reported was taken from the Lakhzoum et al. (2021) concreteness study. The general statistics for the lexical variables presented in Table 1 were obtained by cross-referencing the present cue words with the Lexique (New et al., 2004), FLP (Ferrand et al., 2010), MEGALEX (Ferrand et al., 2018) and Wordlex (Gimenes & New, 2016) databases. The concreteness statistics indicate that the present dataset contains cue words of varying levels of concreteness ranging from 2.10 to 6.58 on a 7-point scale allowing users to study responses elicited by both abstract and concrete concepts. Separating very concrete cue words (M = 3.12; SD = 0.44) from very abstract ones (M = 5.52; SD = 0.92) based on the median revealed a significant effect of concreteness [t(910) = 56.71; p < 0.0001]. This will allow concreteness to be used as either a continuous or categorical variable by researchers wishing to study the effect of variation in concreteness of the cues that elicited the associated responses.

<table><caption>**Table 1.** Descriptive and behavioural data for cue words.</caption><thead><tr><th></th><th>min</th><th>max</th><th>M</th><th>SD</th></tr></thead><tbody><tr><td>Mean Concreteness</td><td>2.10</td><td>6.58</td><td>5.17</td><td>1.57</td></tr><tr><td>Movie Frequency</td><td>0.00</td><td>2752.00</td><td>440.15</td><td>136.47</td></tr><tr><td>Book Frequency</td><td>0.00</td><td>4696.15</td><td>44.92</td><td>172.65</td></tr><tr><td>Blog Frequency</td><td>0.06</td><td>3095.93</td><td>61.24</td><td>193.06</td></tr><tr><td>Twitter Frequency</td><td>0.00</td><td>4070.02</td><td>55.45</td><td>232.49</td></tr><tr><td>Newspaper Frequency</td><td>0.00</td><td>3782.26</td><td>57.49</td><td>200.59</td></tr><tr><td>N-letters</td><td>3.00</td><td>14.00</td><td>7.00</td><td>2.02</td></tr><tr><td>N-orthographic Neighbours</td><td>0.00</td><td>23.00</td><td>2.42</td><td>3.60</td></tr><tr><td>OLD20</td><td>1.00</td><td>5.40</td><td>2.09</td><td>0.62</td></tr><tr><td>Reaction Times (FLP)</td><td>515.54</td><td>1005.15</td><td>658.41</td><td>65.67</td></tr><tr><td>Reaction Times (Megalex)</td><td>473.48</td><td>747.84</td><td>553.05</td><td>44.18</td></tr></tbody></table>

Note. OLD20: orthographic Levenshtein distance (Yarkoni, Balota, & Yap, 2008); Movie subtitles frequency (from Lexique; New et al., 2004) and Mean concreteness (from Lakhzoum et al., 2021).

A more extensive description of the cues can be found in Lakhzoum et al., (2021).
## Cue concreteness and Response frequency analyses

General statistics were then computed for the responses following a similar cross-referencing process. Table 2 shows the aggregated statistics for all the responses

<table><caption>**Table 2.** Mean frequency and reaction times for response words</caption><thead><tr><th></th><th><em>M</em></th><th><em>SD</em></th></tr></thead><tbody><tr><td>Movie Subtitles Frequency</td><td>29.85</td><td>349.18</td></tr><tr><td>Book Frequency</td><td>33.38</td><td>450.23</td></tr><tr><td>Blog Frequency</td><td>57.59</td><td>746.03</td></tr><tr><td>Twitter Frequency</td><td>55.60</td><td>646.61</td></tr><tr><td>Newspaper Frequency</td><td>55.26</td><td>797.73</td></tr><tr><td>Reaction Times (FLP)</td><td>690.32</td><td>79.01</td></tr><tr><td>Reaction Times (Megalex)</td><td>574.11</td><td>53.04</td></tr></tbody></table>

With respect to the responses, frequency and reaction time data were our main interest. Additional analyses compared the frequency and reaction time data for responses elicited by abstract and concrete cue words. They revealed an effect of movie subtitle frequencies with more frequent response words corresponding to abstract cue words ($M = 87.44; SD = 462.69$) than more concrete cue words ($M = 72.40; SD = 418.31$) – t(37146) = 3.36; p < 0.001. The focus on movie subtitle frequencies was due mainly to the fact that they are commonly used, particularly when it comes to norming stimuli in language experiments. An additional analysis revealed an effect of reaction times, with shorter latencies for words elicited by concrete cue words ($M = 647.00; SD = 64.85$) than abstract cue words ($M = 651.12; SD = 65.72$) – t(34184) = 5.83; p < 0.001. Taken together, the analyses revealed that abstract cue words elicited more frequent responses but with longer latencies (see Table 3).
<table><caption>**Table 3.** Response frequencies and latencies according to cue concreteness</caption><thead><tr><th></th><th>abstract cues</th><th>concrete cues</th></tr></thead><tbody><tr><th scope="row">response frequencies</th><td>87.44 (462.69)</td><td>72.40 (418.31)</td></tr><tr><th scope="row">response latencies</th><td>651.12 (65.72)</td><td>647.00 (64.85)</td></tr></tbody></table>

Note. Mean (SD). All reported means are statistically significant (p < 0.0001)

### Cue-response organisation according to concreteness and perceptual gradients

An additional study was designed to establish whether cue words prompted responses of similar concreteness levels. Using the same protocol as Lakhzoum et al. (2021), we conducted an additional concreteness study where the responses from the 150 most strongly associated pairs were submitted to an independent group of 50 French native-speakers ($M_{age} = 26.63; SD = 7.64$). The collected concreteness levels for both cues and responses are presented in Table A1. Results showed a strong correlation between cue and response concreteness levels ($r = .73$) – $t(147) = 13.00; p < 0.001$. An additional independent t-test showed no statistical difference between cue and response concreteness ($t(292) = 0.64; p = 0.52 ns$). Taken together, these results show that concrete cue words tend to elicit concrete responses and abstract cue words tend to elicit abstract responses. Aggregating across all responses, cue-response pairs elicited by concrete cues were more strongly associated ($M = 3.10; SD = 5.60$) than pairs elicited by abstract cues ($M = 2.45; SD = 3.97$) – $t(35861) = 13.00; p < 0.001$.

Finally, we considered the effect of the modality-specific perceptual strength variable developed by Chedid et al. (2019). A comparison of perceptual strength between cues and responses gave an overall moderate correlation ($r = .41$) – $t(6466) = 35.70; p < 0.001$.
However, when only pairs that are verbally associated (association strength ≥ .10) are considered, the correlation increases considerably (r = .67) – t(359) = 17.00; p < 0.001, signifying that participants tend to produce responses of similar perceptual strength to the cues that elicited said responses. This result is similar to the one obtained with the concreteness variable, suggesting modality-specific variables can also capture patterns of organisation of the mental lexicon. We then estimated the ability of perceptual strength to predict semantic latencies, obtaining a moderate negative correlation (r = -.20) – t(1822) = 8.90; p < 0.001.

## Use of association frequencies to predict semantic processing

We verified that response frequency – i.e., the number of times word *j* is given as a response to cue word *i* – is correlated to the association strength between cues and responses. Results showed a strong correlation between frequency and strength (r = .98) – t(39911) = 1032; p < 0.001. Response frequency offers up additional information to strength insofar as it emphasises the lexical centrality or saliency of a response, and thus of a word, in the mental lexicon. It is therefore often used to estimate other measures of semantic relatedness, such as reaction latencies (see De Deyne et al., 2019). The comparison between response lexical centrality and lexical latencies from MEGALEX (Ferrand et al., 2018) showed a substantial negative correlation (r = -.55)<fnref n="5" /> – t(5722) = 47.70, p < 0.001) which means that the more salient the less time it takes to be processed.

## DISCUSSION

The present study aimed to produce French word-association norms with a focus on abstract concepts. The task yielded a sizeable number of responses that allowed for an exploration of the dataset's distributional characteristics. Frequency spectrum analyses

<footnote n="5">⁵ De Deyne et al. (2018) reported a correlation of -.59 based on the SWOW dataset.</footnote>
revealed that the dataset displayed the same commonly found pattern across language and corpora (Evert & Baroni, 2005). In addition, the vocabulary growth curves highlighted the productivity of the task, meaning that with continuous response collection, the number of types (or unique responses) would exponentially increase, an illustration of the fact that new words are constantly coined (Baayen, 2001). Taken together, the frequency spectrum and vocabulary growth curve reflect the productivity of the human language and the fact that new word types are continuously produced, be it in the context of a word-association task or as a broader linguistic phenomenon.

We reported the previously collected concreteness variable for the cue words, which allowed for a comparison of response statistics between words prompted by an abstract cue or a concrete cue. The presence of the concreteness variable in this dataset, even if it is reserved for the cue words, is of the utmost importance when considering the previously reported discrepancies in the literature as to the role of concreteness in the organisation of the mental lexicon (see Crutch & Warrington, 2010 for example). The additional study used to collect response concreteness revealed a high correlation between cue and response concreteness and no significant difference in concreteness levels between the two words of a pair. These important results shed more light on how the mental lexicon is organised according to abstract and concrete concepts, in that they suggest abstract concepts are closely coded in the semantic network if an abstract cue tends to prompt an abstract response, but, in addition, they also allow for a deeper exploration of the link between concreteness and frequency. For example, the results showed that, in comparison with concrete cues, abstract cues prompted responses that are more frequently distributed across movie subtitles. At first sight this result might seem to be a rather counterintuitive, insofar as concrete concepts appear to be more common in language use. In actual fact, the vast majority of meaningful words used in everyday language tend to be more abstract, as illustrated in particular by the regular use of abstract
verbs such as *think*, *feel*, and *want* to express volition and states of mind. According to the British National Corpus (Leech, Garside & Bryant, 1994), 72 % of the noun or verb tokens in British English are more abstract than the noun *war*. Nevertheless, given that previous language norms and studies have relied heavily on concrete concepts, it is not surprising these results seem counterintuitive when in fact they provide more of an incentive to further the study of abstract concepts.

The introduction of a modality-specific variable such as perceptual strength is further testament to the intricate nature of abstract concepts. Results showed that cue words elicited responses of a similar level of not only concreteness but also perceptual strength. This result is important in light of the seeming chiasm between the distributional and grounded accounts of concept representation. The suggestion is that the organisation of the mental lexicon follows statistical patterns that can be captured by variables that are historically distributional, such as semantic similarity, frequency and relatedness, and by variables that are closer to the grounded end of the spectrum, such as concreteness and perceptual strength. Our analyses in respect of the latter were constrained, however, by the limited number of items between our cue-response pairs and the items present in Chedid and colleagues' database. This is a further indication of the need to develop larger normative datasets based on these modality-specific variables and with an emphasis on abstract concepts.

The aim of this dataset was to fill a gap in the French literature as regards word association norms by introducing 1 100 cues of varying levels of concreteness that elicited more than 89k responses from over a 1 000 participants. To the best of our knowledge, this is the first dataset since Bonin et al., (2013), nearly a decade ago, and also the most recent since Ferrand et al., (2001), two decades ago, to attempt to focus on abstract concepts as well. We anticipate that it will be very useful for researchers working on language- and memory-related tasks, especially those wishing to control for association when working on semantic similarity
or on the impact of concreteness in the organisation of the mental lexicon. The correlation
between lexical centrality and word latencies guarantees the validity of the dataset and its
ability to predict semantic processing.

Open Practices Statement

In line with an open data policy, all data discussed in this article are freely available on
our web site on the Open Science Framework web site (https://osf.io/dhuqs/).

REFERENCES

Andrews, M., Frank, S., & Vigliocco, G. (2014). Reconciling embodied and distributional accounts of meaning in language. *Topics in Cognitive Science, **6***, 359–370.
https://doi.org/10.1111/tops.12096

Baayen, R. H. (2001). *Word frequency distributions* (Vol. 18). Springer Science & Business Media.

Baroni, M., & Evert, S. (2006). The zipfR package for lexical statistics: A tutorial introduction. http://zipfr.r-forge.r-project.org.

Barrón-Martínez, J. B., & Arias-Trejo, N. (2014). Word association norms in Mexican

Spanish. *The Spanish Journal of Psychology, **17***, 1–13. https://doi.org/10.1017/sjp.2014.91.

Barsalou, L. W. (1999). Perceptual symbol systems. *Behavioral and Brain Sciences, **22***, 577–660. https://doi.org/10.1017/S0140525X99002149

Barsalou, L. W., & Wiemer-Hastings, K. (2005). Situating abstract concepts. In *Grounding
Cognition: The Role of Perception and Action in Memory, Language, and Thinking* (pp.
129–163). Cambridge University Press.
https://doi.org/10.1017/CBO9780511499968.007

Bonin, P., Méot, A., Ferrand, L., & Bugaïska, A. (2013). Normes d'associations verbales pour
520 mots concrets et étude de leurs relations avec d'autres variables psycholinguistiques.
*L'Année Psychologique, **113***, 63–92.

Borghi, A. M., Binkofski, F., Castelfranchi, C., Cimatti, F., Scorolli, C., & Tummolini, L. (2017). The challenge of abstract concepts. *Psychological Bulletin, **143***, 263–292.
https://doi.org/10.1037/bul00000089

Bugaiska, A., Morson, S., Moulin, C. J. A., & Souchay, C. (2011). Metamemory, recollection and familiarity in Alzheimer's disease. *Revue Neurologique, **167***, 3–13.
https://doi.org/10.1016/j.neurol.2010.03.001

Chedid, G., Brambati, S. M., Bedetti, C., Rey, A. E., Wilson, M. A., & Vallet, G. T. (2019). Visual and auditory perceptual strength norms for 3,596 French nouns and their relationship with other psycholinguistic variables. *Behavior Research Methods, **51***, 2094–2105. https://doi.org/10.3758/s13428-019-01254-w

Coltheart, M. (1981). The MRC Psycholinguistic Database. *The Quarterly Journal of*
*Experimental Psychology Section A*, 33, 497–505.

https://doi.org/10.1080/14640748108400805

Crutch, S. J., Connell, S., & Warrington, E. K. (2009). The different representational frameworks underpinning abstract and concrete knowledge: Evidence from odd-one-out judgements. *Quarterly Journal of Experimental Psychology*, 62, 1377–1390.

https://doi.org/10.1080/17470210802483834

Crutch, S. J., & Jackson, E. C. (2011). Contrasting graded effects of semantic similarity and association across the concreteness spectrum. *Quarterly Journal of Experimental Psychology*, 64, 1388–1408. https://doi.org/10.1080/17470218.2010.543285

Crutch, S. J., & Warrington, E. K. (2005). Abstract and concrete concepts have structurally different representational frameworks. *Brain*, 128, 615–627.

https://doi.org/10.1093/brain/awh349

Crutch, S. J., & Warrington, E. K. (2010). The differential dependence of abstract and concrete words upon associative and similarity-based information: Complementary semantic interference and facilitation effects. *Cognitive Neuropsychology*, 27, 46–71.

https://doi.org/10.1080/02643294.2010.491359

De Deyne, S., Navarro, D. J., Perfors, A., Brysbaert, M., & Storms, G. (2019). Measuring the associative structure of English: The “Small World of Words” norms for word association. *Behavior Research Methods*, 51, 987–1006.

De Deyne, S., Navarro, D. J., & Storms, G. (2013). Better explanations of lexical and semantic cognition using networks derived from continued rather than single-word associations. *Behavior Research Methods*, 45, 480–498. https://doi.org/10.3758/s13428-012-0260-7

De Deyne, S., & Storms, G. (2008). Word associations: Network and semantic properties. *Behavior Research Methods*, 40, 213–231. https://doi.org/10.3758/BRM.40.1.213

Deyne, S. De, Verheyen, S., & Storms, G. (2016). Structure and organization of the mental lexicon: A network approach derived from syntactic dependency relations and word associations. *Understanding Complex Systems*, 99, 47–79. https://doi.org/10.1007/978-3-662-47238-5_3

Duñabeitia, J. A., Avilés, A., Afonso, O., Scheepers, C., & Carreiras, M. (2009). Qualitative differences in the representation of abstract versus concrete words: Evidence from the visual-world paradigm. *Cognition*, 110, 284–292.

https://doi.org/10.1016/j.cognition.2008.11.012

Duscherer, K., & Mounoud, P. (2006). Normes d'associations verbales pour 151 verbes d'action. *L'Année Psychologique*, 106, 397–413.

Eakin, D. K. (2005). Illusions of knowing: Metamemory and memory under conditions of retroactive interference. *Journal of Memory and Language*, 52, 526–534.

https://doi.org/10.1016/j.jml.2005.01.009

Evert, S., & Baroni, M. (2005). Testing the extrapolation quality of word frequency models. In *Proceedings of Corpus Linguistics*.

Evert, S., & Baroni, M. (2007). zipfR: Word frequency distributions in R. In *Proceedings of the 45th annual meeting of the ACL on interactive poster and demonstration sessions*
(pp. 29–32).

Fernández, Á., Díez, E., Alonso, M. Á., & Beato, M. S. (2004). Free-association norms for the Spanish names of the Snodgrass and Vanderwart pictures. *Behavior Research Methods, Instruments, & Computers, 36*, 577–583.

Ferrand, L. (2001). Normes d'associations verbales pour 260 mots « abstraits ». L'Année Psychologique, 101, 683–721. https://doi.org/10.3406/psy.2001.29575

Ferrand, L., & Alario, F. X. (1998). Word association norms for 366 names of objects. L'Année Psychologique, 98, 659–709. https://doi.org/10.3406/psy.1998.28564

Ferrand, L., Méot, A., Spinelli, E., New, B., Pallier, C., Bonin, P., Dufau, S., Mathôt, S., & Grainger, J. (2018). MEGALEX: A megastudy of visual and auditory word recognition. *Behavior Research Methods, 50*, 1285–1307. https://doi.org/10.3758/s13428-017-0943-1

Ferrand, L., & New, B. (2003). Semantic and associative priming in the mental lexicon. *Mental Lexicon: Some Words to Talk about Words*, 25–43.

Ferrand, L., New, B., Brysbaert, M., Keuleers, E., Bonin, P., Méot, A., Augustinova, M., & Pallier, C. (2010). The French lexicon project: Lexical decision data for 38,840 French words and 38,840 pseudo words. *Behavior Research Methods, 42*, 488–496.
https://doi.org/10.3758/BRM.42.2.488

Ferretti, T. R., McRae, K., & Hatherell, A. (2001). Integrating Verbs, Situation Schemas, and Thematic Role Concepts. *Journal of Memory and Language, 44*, 516–547.
https://doi.org/10.1006/jmla.2000.2728

Geng, J., & Schnur, T. T. (2015). The representation of concrete and abstract concepts: Categorical versus associative relationships. *Journal of Experimental Psychology: Learning Memory and Cognition, 41*, 22–41. https://doi.org/10.1037/a0037430

Gimenes, M., & New, B. (2016). Worldlex: Twitter and blog word frequencies for 66 languages. *Behavior Research Methods, 48*, 963–972.

Glenberg, A. M. (1997). What memory is for. *Behavioral and Brain Sciences, 20*, 1–19.

Hamilton, A. C., & Coslett, H. B. (2008). Refractory access disorders and the organization of concrete and abstract semantics: Do they differ? *Neurocase, 14*, 131–140.
https://doi.org/10.1080/13554790802032218

Harris, Z. (1954). Distributional hypothesis. *Word World, 10*, 146–162.

Herdan, G. (1964). *Quantitative Linguistics*. London: Buttersworths.

Joyce, T. (2005). Constructing a large-scale database of Japanese word associations. *Glottometrics, 10*, 82–99.

Jung, J., Li, N., & Akama, H. (2010). Network analysis of Korean word associations. *Proceedings of the NAACL HLT 2010 First Workshop on Computational Neurolinguistics, 27–35*. http://www.aclweb.org/anthology/W/W10/W10-0604

Kiefer, M., & Pulvermüller, F. (2012). Conceptual representations in mind and brain: Theoretical developments, current evidence and future directions. *Cortex, 48*, 805–825.
https://doi.org/10.1016/j.cortex.2011.04.006

Kircher, T., Whitney, C., Krings, T., Huber, W., & Weis, S. (2008). Hippocampal dysfunction during free word association in male patients with schizophrenia. *Schizophrenia Research, 101*, 242–255. https://doi.org/10.1016/j.schres.2008.02.003
Kiss, G. R., Armstrong, C., Milroy, R., & Piper, J. (1972). The Edinburgh Associative
Thesaurus. *The Edinburgh Associative Thesaurus*. https://w3id.
org/associations/eat.nt.gz.

Kousta, S. T., Vigliocco, G., Vinson, D. P., Andrews, M., & Del Campo, E. (2011). The representation of abstract words: Why emotion matters. *Journal of Experimental Psychology: General, 140*, 14–34. https://doi.org/10.1037/a0021446

Kwong, O. Y. (2013). Exploring the Chinese mental lexicon with word association norms.
*27th Pacific Asia Conference on Language, Information, and Computation, PACLIC 27*,
153–162.

Lakhzoum, D., Izaute, M., & Ferrand, L. (2021). Semantic similarity and associated abstractness norms for 630 French word pairs. *Behavior Research Methods, 53*, 1166–1178.

Leech, G., Garside, R., & Bryant, M. (1994). CLAWS4: The tagging of the British National Corpus. *Proceedings of the 15th International Conference on Computational Linguistics*, 622–628.

Lenci, A., Lebani, G. E., & Passaro, L. C. (2018). The emotions of abstract words: A distributional semantic analysis. *Topics in Cognitive Science, 10*, 550–572.
https://doi.org/10.1111/tops.12335

Louwerse, M. M., & Jeuniaux, P. (2010). The linguistic and embodied nature of conceptual processing. *Cognition, 114*, 96–104. https://doi.org/10.1016/j.cognition.2009.09.002

McNamara, T. P. (1992). Theories of priming: I. Associative distance and lag. *Journal of Experimental Psychology: Learning, Memory, and Cognition, 18*, 1173–1190.
https://doi.org/10.1037/0278-7393.18.6.1173

McRae, K., & Boisvert, S. (1998). Automatic semantic similarity priming. *Journal of Experimental Psychology: Learning Memory and Cognition, 24*, 558–572.
https://doi.org/10.1037/0278-7393.24.3.558

Meteyard, L., Cuadrado, S. R., Bahrami, B., & Vigliocco, G. (2012). Coming of age: A review of embodiment and the neuroscience of semantics. *Cortex, 48*, 788–804.
https://doi.org/10.1016/j.cortex.2010.11.002

Mollin, S. (2009). Combining corpus linguistic and psychological data on word co-occurrences: Corpus collocates versus word associations. *Corpus Linguistics and Linguistic Theory, 5*, 175–200. https://doi.org/10.1515/CLLT.2009.008

Moss, H., Older, L., & Older, L. (1996). *Birkbeck word association norms*. Psychology Press.

Nelson, D. L., McEvoy, C. L., & Dennis, S. (2000). What is free association and what does it measure? *Memory and Cognition, 28*, 887–899.

Nelson, D. L., McEvoy, C. L., & Schreiber, T. A. (2004). The University of South Florida free association, rhyme, and word fragment norms. *Behavior Research Methods, Instruments, and Computers*. Psychonomic Society Inc.
https://doi.org/10.3758/BF03195588

New, B., Pallier, C., Brysbaert, M., & Ferrand, L. (2004). Lexique 2: A new French lexical database. *Behavior Research Methods, Instruments, and Computers*. Psychonomic Society Inc. https://doi.org/10.3758/BF03195598
Plaut, D. C., & Erlbaum Associates, L. (1995). *Semantic and Associative Priming in a Distributed Attractor Network*.

Prior, A., & Bentin, S. (2008). Word associations are formed incidentally during sentential semantic integration. *Acta Psychologica*, 127, 57–71.

Qualtrics (2020). Qualtrics.com. http://www.qualtrics.com/

Schloss, B., & Li, P. (2016). Disentangling narrow and coarse semantic networks in the brain: The role of computational models of word meaning. *Behavior Research Methods*, 49, 1582–1596. https://doi.org/10.3758/s13428-016-0807-0

Taconnat, L., Raz, N., Toczé, C., Bouazzaoui, B., Sauzéon, H., Fay, S., & Isingrini, M. (2009). Ageing and organisation strategies in free recall: The role of cognitive flexibility. *European Journal of Cognitive Psychology*, 21, 347–365. https://doi.org/10.1080/09541440802296413

Thompson-Schill, S. L., Kurtz, K. J., & Gabrieli, J. D. E. (1998). Effects of semantic and associative relatedness on automatic priming. *Journal of Memory and Language*, 38, 440–458.

West, J. T., & Mulligan, N. W. (2019). Prospective Metamemory, Like Retrospective Metamemory, Exhibits Underconfidence With Practice. *Journal of Experimental Psychology: Learning Memory and Cognition*, 45. https://doi.org/10.1037/xlm0000708

Yarkoni, T., Balota, D., & Yap, M. (2008). Moving beyond Coltheart's N: A new measure of orthographic similarity. *Psychonomic Bulletin and Review*, 15, 971–979. https://doi.org/10.3758/PBR.15.5.971

Zwaan, R. A. (2004). The Immersed Experiencer: Toward an embodied theory of language comprehension. *Psychology of Learning and Motivation - Advances in Research and Theory*, 44, 35–62. https://doi.org/10.1016/S0079-7421(03)44002-4
APPENDIX

<table><caption>**Table A1.** First 150 most associated pairs with cue and response mean concreteness</caption><thead><tr><th>cue</th><th>translation</th><th>response</th><th>translation</th><th>%strength</th><th>cue_concreteness</th><th>response_concreteness</th></tr></thead><tbody><tr><td>crainte</td><td>fright</td><td>peur</td><td>fear</td><td>85.71</td><td>2.95</td><td>2.37</td></tr><tr><td>omelette</td><td>omelet</td><td>œuf</td><td>egg</td><td>82.05</td><td>6.67</td><td>6.61</td></tr><tr><td>chorégraphie</td><td>choregraphy</td><td>danse</td><td>dance</td><td>79.49</td><td>5.12</td><td>4.71</td></tr><tr><td>poivre</td><td>pepper</td><td>sel</td><td>salt</td><td>78.57</td><td>6.55</td><td>6.10</td></tr><tr><td>croquis</td><td>sketch</td><td>dessin</td><td>drawing</td><td>74.39</td><td>5.46</td><td>5.57</td></tr><tr><td>ruche</td><td>hive</td><td>abeille</td><td>bee</td><td>72.73</td><td>6.43</td><td>6.67</td></tr><tr><td>brasier</td><td>inferno</td><td>feu</td><td>fire</td><td>72.37</td><td>5.75</td><td>5.76</td></tr><tr><td>question</td><td>question</td><td>réponse</td><td>answer</td><td>72.00</td><td>3.93</td><td>2.94</td></tr><tr><td>homme</td><td>man</td><td>femme</td><td>woman</td><td>71.88</td><td>6.26</td><td>5.78</td></tr><tr><td>selle</td><td>saddle</td><td>cheval</td><td>horse</td><td>71.43</td><td>6.46</td><td>6.63</td></tr><tr><td>luge</td><td>sledge</td><td>neige</td><td>snow</td><td>70.42</td><td>6.58</td><td>6.29</td></tr><tr><td>primate</td><td>primate</td><td>singe</td><td>monkey</td><td>70.27</td><td>5.36</td><td>6.43</td></tr><tr><td>soucoupe</td><td>saucer</td><td>volant</td><td>flying</td><td>69.77</td><td>5.90</td><td>5.61</td></tr><tr><td>égaré</td><td>misplaced</td><td>perdre</td><td>loose</td><td>69.57</td><td>3.49</td><td>2.86</td></tr><tr><td>prévision</td><td>forecast</td><td>météo</td><td>weather</td><td>69.57</td><td>2.74</td><td>3.88</td></tr><tr><td>avalanche</td><td>avalance</td><td>neige</td><td>snow</td><td>69.14</td><td>6.17</td><td>6.29</td></tr><tr><td>aéroport</td><td>airport</td><td>avion</td><td>plane</td><td>67.95</td><td>6.24</td><td>6.63</td></tr><tr><td>nid</td><td>nest</td><td>oiseau</td><td>bird</td><td>67.86</td><td>6.31</td><td>6.69</td></tr><tr><td>effroi</td><td>dread</td><td>peur</td><td>fear</td><td>66.67</td><td>3.16</td><td>2.37</td></tr><tr><td>robinet</td><td>faucet</td><td>eau</td><td>water</td><td>65.52</td><td>6.61</td><td>6.22</td></tr><tr><td>tendance</td><td>tendency</td><td>mode</td><td>fashion</td><td>64.84</td><td>2.57</td><td>2.82</td></tr><tr><td>rouge-gorge</td><td>robin</td><td>oiseau</td><td>bird</td><td>64.71</td><td>6.58</td><td>6.69</td></tr><tr><td>cacao</td><td>cocoa</td><td>chocolat</td><td>chocolate</td><td>64.37</td><td>6.23</td><td>6.59</td></tr><tr><td>ancre</td><td>anchor</td><td>bateau</td><td>boat</td><td>62.35</td><td>6.14</td><td>6.63</td></tr><tr><td>chlore</td><td>chlorine</td><td>piscine</td><td>pool</td><td>62.12</td><td>5.41</td><td>6.49</td></tr><tr><td>bibliothèque</td><td>library</td><td>livre</td><td>book</td><td>61.97</td><td>6.48</td><td>6.63</td></tr><tr><td>baril</td><td>barrel</td><td>pétrole</td><td>oil</td><td>61.33</td><td>6.34</td><td>5.88</td></tr><tr><td>périr</td><td>perish</td><td>mourir</td><td>die</td><td>61.29</td><td>4.11</td><td>3.51</td></tr><tr><td>riche</td><td>wealthy</td><td>pauvre</td><td>poor</td><td>61.29</td><td>3.48</td><td>2.94</td></tr><tr><td>flamme</td><td>flame</td><td>feu</td><td>fire</td><td>61.11</td><td>5.94</td><td>5.76</td></tr><tr><td>réponse</td><td>answer</td><td>question</td><td>question</td><td>60.53</td><td>3.92</td><td>3.00</td></tr><tr><td>cliché</td><td>stereotype</td><td>photo</td><td>picture</td><td>60.44</td><td>3.71</td><td>5.73</td></tr><tr><td>bravoure</td><td>bravery</td><td>courage</td><td>courage</td><td>60.26</td><td>2.97</td><td>2.69</td></tr><tr><td>cercle</td><td>circle</td><td>rond</td><td>round</td><td>60.24</td><td>5.21</td><td>4.82</td></tr><tr><td>ampoule</td><td>bulb</td><td>lumière</td><td>light</td><td>59.55</td><td>6.78</td><td>4.76</td></tr><tr><td>gare</td><td>train station</td><td>train</td><td>train</td><td>59.46</td><td>6.19</td><td>6.55</td></tr><tr><td>frère</td><td>brother</td><td>sœur</td><td>sister</td><td>59.21</td><td>5.94</td><td>5.67</td></tr><tr><td>roi</td><td>king</td><td>reine</td><td>queen</td><td>59.15</td><td>5.59</td><td>5.31</td></tr><tr><td>oncle</td><td>uncle</td><td>tante</td><td>aunt</td><td>58.82</td><td>6.46</td><td>5.75</td></tr></tbody></table>
<table>
  <tr>
    <td>pacifique</td>
    <td>peaceful</td>
    <td>océan</td>
    <td>ocean</td>
    <td>58.75</td>
    <td>3.64</td>
    <td>5.82</td>
  </tr>
  <tr>
    <td>bien</td>
    <td>good</td>
    <td>mal</td>
    <td>bad</td>
    <td>58.11</td>
    <td>2.60</td>
    <td>2.22</td>
  </tr>
  <tr>
    <td>décret</td>
    <td>decree</td>
    <td>loi</td>
    <td>law</td>
    <td>57.97</td>
    <td>4.26</td>
    <td>3.04</td>
  </tr>
  <tr>
    <td>tablier</td>
    <td>apron</td>
    <td>cuisine</td>
    <td>kitchen</td>
    <td>57.69</td>
    <td>6.53</td>
    <td>5.73</td>
  </tr>
  <tr>
    <td>tonneau</td>
    <td>cask</td>
    <td>vin</td>
    <td>wine</td>
    <td>57.69</td>
    <td>6.63</td>
    <td>6.35</td>
  </tr>
  <tr>
    <td>vie</td>
    <td>life</td>
    <td>mort</td>
    <td>death</td>
    <td>57.69</td>
    <td>3.74</td>
    <td>3.45</td>
  </tr>
  <tr>
    <td>femme</td>
    <td>woman</td>
    <td>homme</td>
    <td>man</td>
    <td>57.33</td>
    <td>5.86</td>
    <td>5.57</td>
  </tr>
  <tr>
    <td>grand</td>
    <td>large</td>
    <td>petit</td>
    <td>small</td>
    <td>57.14</td>
    <td>4.59</td>
    <td>3.88</td>
  </tr>
  <tr>
    <td>soulier</td>
    <td>shoe</td>
    <td>chaussure</td>
    <td>shoe</td>
    <td>56.99</td>
    <td>6.69</td>
    <td>6.45</td>
  </tr>
  <tr>
    <td>réfléchir</td>
    <td>reflect</td>
    <td>penser</td>
    <td>think</td>
    <td>56.63</td>
    <td>3.62</td>
    <td>2.82</td>
  </tr>
  <tr>
    <td>moineau</td>
    <td>sparrow</td>
    <td>oiseau</td>
    <td>bird</td>
    <td>56.47</td>
    <td>6.48</td>
    <td>6.69</td>
  </tr>
  <tr>
    <td>célébration</td>
    <td>celebration</td>
    <td>fête</td>
    <td>party</td>
    <td>55.95</td>
    <td>3.97</td>
    <td>4.06</td>
  </tr>
  <tr>
    <td>égalité</td>
    <td>equality</td>
    <td>fraternité</td>
    <td>fraternity</td>
    <td>55.95</td>
    <td>3.05</td>
    <td>2.49</td>
  </tr>
  <tr>
    <td>étincelle</td>
    <td>sparke</td>
    <td>feu</td>
    <td>fire</td>
    <td>55.26</td>
    <td>5.58</td>
    <td>5.76</td>
  </tr>
  <tr>
    <td>après</td>
    <td>after</td>
    <td>avant</td>
    <td>before</td>
    <td>55.13</td>
    <td>3.77</td>
    <td>2.45</td>
  </tr>
  <tr>
    <td>terrier</td>
    <td>burrow</td>
    <td>lapin</td>
    <td>rabbit</td>
    <td>54.67</td>
    <td>6.43</td>
    <td>6.65</td>
  </tr>
  <tr>
    <td>monarchie</td>
    <td>monarchy</td>
    <td>roi</td>
    <td>king</td>
    <td>53.73</td>
    <td>4.04</td>
    <td>5.43</td>
  </tr>
  <tr>
    <td>berceau</td>
    <td>crib</td>
    <td>bébé</td>
    <td>baby</td>
    <td>53.41</td>
    <td>6.60</td>
    <td>6.20</td>
  </tr>
  <tr>
    <td>anatomie</td>
    <td>anatomy</td>
    <td>corps</td>
    <td>body</td>
    <td>53.09</td>
    <td>5.82</td>
    <td>5.96</td>
  </tr>
  <tr>
    <td>sapin</td>
    <td>fir</td>
    <td>noël</td>
    <td>christmas</td>
    <td>52.63</td>
    <td>6.76</td>
    <td>3.88</td>
  </tr>
  <tr>
    <td>trébucher</td>
    <td>stumble</td>
    <td>tomber</td>
    <td>fall</td>
    <td>52.63</td>
    <td>5.26</td>
    <td>4.12</td>
  </tr>
  <tr>
    <td>capture</td>
    <td>capture</td>
    <td>écran</td>
    <td>screen</td>
    <td>52.44</td>
    <td>3.75</td>
    <td>6.41</td>
  </tr>
  <tr>
    <td>rusé</td>
    <td>cunning</td>
    <td>renard</td>
    <td>fox</td>
    <td>52.05</td>
    <td>3.27</td>
    <td>6.73</td>
  </tr>
  <tr>
    <td>lait</td>
    <td>milk</td>
    <td>vache</td>
    <td>cow</td>
    <td>51.81</td>
    <td>6.48</td>
    <td>6.67</td>
  </tr>
  <tr>
    <td>astronomie</td>
    <td>astronomy</td>
    <td>étoile</td>
    <td>stars</td>
    <td>51.76</td>
    <td>4.59</td>
    <td>5.59</td>
  </tr>
  <tr>
    <td>émissaire</td>
    <td>emissary</td>
    <td>bouc</td>
    <td>goat</td>
    <td>51.47</td>
    <td>4.33</td>
    <td>6.29</td>
  </tr>
  <tr>
    <td>abdomen</td>
    <td>abdomen</td>
    <td>ventre</td>
    <td>belly</td>
    <td>51.32</td>
    <td>6.53</td>
    <td>6.14</td>
  </tr>
  <tr>
    <td>privé</td>
    <td>private</td>
    <td>public</td>
    <td>public</td>
    <td>51.28</td>
    <td>3.31</td>
    <td>3.84</td>
  </tr>
  <tr>
    <td>bulle</td>
    <td>buble</td>
    <td>savon</td>
    <td>soap</td>
    <td>51.16</td>
    <td>5.71</td>
    <td>6.43</td>
  </tr>
  <tr>
    <td>flore</td>
    <td>flora</td>
    <td>faune</td>
    <td>fauna</td>
    <td>50.77</td>
    <td>5.24</td>
    <td>4.47</td>
  </tr>
  <tr>
    <td>existence</td>
    <td>existence</td>
    <td>vie</td>
    <td>life</td>
    <td>50.55</td>
    <td>3.52</td>
    <td>3.14</td>
  </tr>
  <tr>
    <td>palpitation</td>
    <td>palpitation</td>
    <td>coeur</td>
    <td>fever</td>
    <td>50.53</td>
    <td>4.87</td>
    <td>5.47</td>
  </tr>
  <tr>
    <td>album</td>
    <td>album</td>
    <td>photo</td>
    <td>photo</td>
    <td>50.00</td>
    <td>6.23</td>
    <td>5.73</td>
  </tr>
  <tr>
    <td>tentative</td>
    <td>attempt</td>
    <td>essai</td>
    <td>test</td>
    <td>50.00</td>
    <td>3.37</td>
    <td>3.35</td>
  </tr>
  <tr>
    <td>navire</td>
    <td>ship</td>
    <td>bateau</td>
    <td>boat</td>
    <td>49.44</td>
    <td>6.51</td>
    <td>6.63</td>
  </tr>
  <tr>
    <td>problème</td>
    <td>problem</td>
    <td>solution</td>
    <td>solution</td>
    <td>49.41</td>
    <td>3.47</td>
    <td>3.10</td>
  </tr>
  <tr>
    <td>tasse</td>
    <td>cup</td>
    <td>café</td>
    <td>coffee</td>
    <td>49.37</td>
    <td>6.68</td>
    <td>6.43</td>
  </tr>
  <tr>
    <td>source</td>
    <td>source</td>
    <td>eau</td>
    <td>water</td>
    <td>49.30</td>
    <td>4.20</td>
    <td>6.22</td>
  </tr>
  <tr>
    <td>liquide</td>
    <td>liquide</td>
    <td>eau</td>
    <td>water</td>
    <td>49.21</td>
    <td>5.34</td>
    <td>6.22</td>
  </tr>
  <tr>
    <td>torchon</td>
    <td>napkin</td>
    <td>serviette</td>
    <td>towel</td>
    <td>48.84</td>
    <td>6.41</td>
    <td>6.47</td>
  </tr>
  <tr>
    <td>panique</td>
    <td>panic</td>
    <td>peur</td>
    <td>fear</td>
    <td>48.78</td>
    <td>3.49</td>
    <td>2.37</td>
  </tr>
  <tr>
    <td>brise</td>
    <td>breeze</td>
    <td>vent</td>
    <td>wind</td>
    <td>48.65</td>
    <td>4.51</td>
    <td>4.92</td>
  </tr>
  <tr>
    <td>rectangle</td>
    <td>rectangle</td>
    <td>carré</td>
    <td>square</td>
    <td>48.61</td>
    <td>6.56</td>
    <td>3.55</td>
  </tr>
  <tr>
    <td>pauvre</td>
    <td>poor</td>
    <td>riche</td>
    <td>wealthy</td>
    <td>48.53</td>
    <td>3.83</td>
    <td>3.25</td>
  </tr>
</table>
<table>
  <tr>
    <td>Pâque</td>
    <td>Easter</td>
    <td>œuf</td>
    <td>egg</td>
    <td>48.44</td>
    <td>3.97</td>
    <td>6.61</td>
  </tr>
  <tr>
    <td>bizarre</td>
    <td>weird</td>
    <td>étrange</td>
    <td>strange</td>
    <td>48.24</td>
    <td>2.50</td>
    <td>2.27</td>
  </tr>
  <tr>
    <td>épave</td>
    <td>wreck</td>
    <td>bateau</td>
    <td>boat</td>
    <td>47.89</td>
    <td>5.80</td>
    <td>6.63</td>
  </tr>
  <tr>
    <td>dévorer</td>
    <td>devour</td>
    <td>manger</td>
    <td>eat</td>
    <td>47.76</td>
    <td>4.20</td>
    <td>4.82</td>
  </tr>
  <tr>
    <td>démence</td>
    <td>dementia</td>
    <td>folie</td>
    <td>madness</td>
    <td>47.56</td>
    <td>2.87</td>
    <td>2.10</td>
  </tr>
  <tr>
    <td>étrange</td>
    <td>strange</td>
    <td>bizarre</td>
    <td>weird</td>
    <td>47.52</td>
    <td>2.49</td>
    <td>2.02</td>
  </tr>
  <tr>
    <td>bouée</td>
    <td>buoy</td>
    <td>sauvetage</td>
    <td>rescue</td>
    <td>47.50</td>
    <td>6.51</td>
    <td>3.82</td>
  </tr>
  <tr>
    <td>forêt</td>
    <td>forest</td>
    <td>arbre</td>
    <td>tree</td>
    <td>47.44</td>
    <td>6.00</td>
    <td>6.67</td>
  </tr>
  <tr>
    <td>chouette</td>
    <td>owl</td>
    <td>hibou</td>
    <td>owl</td>
    <td>47.37</td>
    <td>5.97</td>
    <td>6.49</td>
  </tr>
  <tr>
    <td>aisé</td>
    <td>convenient</td>
    <td>riche</td>
    <td>wealthy</td>
    <td>47.13</td>
    <td>3.26</td>
    <td>3.25</td>
  </tr>
  <tr>
    <td>solution</td>
    <td>solution</td>
    <td>problème</td>
    <td>problem</td>
    <td>46.99</td>
    <td>3.91</td>
    <td>2.80</td>
  </tr>
  <tr>
    <td>coïncidence</td>
    <td>coincidence</td>
    <td>hasard</td>
    <td>chance</td>
    <td>46.91</td>
    <td>2.77</td>
    <td>2.27</td>
  </tr>
  <tr>
    <td>hibou</td>
    <td>owl</td>
    <td>chouette</td>
    <td>owl</td>
    <td>46.48</td>
    <td>6.78</td>
    <td>5.53</td>
  </tr>
  <tr>
    <td>pinceau</td>
    <td>brush</td>
    <td>peinture</td>
    <td>paint</td>
    <td>46.39</td>
    <td>6.56</td>
    <td>5.92</td>
  </tr>
  <tr>
    <td>arrivée</td>
    <td>arrival</td>
    <td>départ</td>
    <td>departure</td>
    <td>46.38</td>
    <td>4.07</td>
    <td>3.51</td>
  </tr>
  <tr>
    <td>soi</td>
    <td>self</td>
    <td>moi</td>
    <td>me</td>
    <td>46.34</td>
    <td>2.84</td>
    <td>4.12</td>
  </tr>
  <tr>
    <td>avoir</td>
    <td>to have</td>
    <td>être</td>
    <td>to be</td>
    <td>46.25</td>
    <td>3.49</td>
    <td>2.08</td>
  </tr>
  <tr>
    <td>entrée</td>
    <td>entrance</td>
    <td>sortie</td>
    <td>exit</td>
    <td>45.65</td>
    <td>4.56</td>
    <td>3.67</td>
  </tr>
  <tr>
    <td>geler</td>
    <td>congeal</td>
    <td>froid</td>
    <td>cold</td>
    <td>45.65</td>
    <td>5.05</td>
    <td>3.80</td>
  </tr>
  <tr>
    <td>timbre</td>
    <td>stamp</td>
    <td>poste</td>
    <td>post office</td>
    <td>45.59</td>
    <td>6.35</td>
    <td>4.80</td>
  </tr>
  <tr>
    <td>planète</td>
    <td>planet</td>
    <td>terre</td>
    <td>earth</td>
    <td>45.56</td>
    <td>5.68</td>
    <td>5.98</td>
  </tr>
  <tr>
    <td>célébrité</td>
    <td>fame</td>
    <td>star</td>
    <td>star</td>
    <td>45.35</td>
    <td>3.33</td>
    <td>4.06</td>
  </tr>
  <tr>
    <td>bon</td>
    <td>good</td>
    <td>mauvais</td>
    <td>bad</td>
    <td>45.26</td>
    <td>3.13</td>
    <td>2.51</td>
  </tr>
  <tr>
    <td>monter</td>
    <td>go up</td>
    <td>descendre</td>
    <td>come down</td>
    <td>45.24</td>
    <td>4.50</td>
    <td>3.82</td>
  </tr>
  <tr>
    <td>soir</td>
    <td>evening</td>
    <td>nuit</td>
    <td>night</td>
    <td>45.00</td>
    <td>4.84</td>
    <td>2.63</td>
  </tr>
  <tr>
    <td>auteur</td>
    <td>author</td>
    <td>livre</td>
    <td>book</td>
    <td>44.93</td>
    <td>5.54</td>
    <td>6.63</td>
  </tr>
  <tr>
    <td>poignée</td>
    <td>handle</td>
    <td>porte</td>
    <td>door</td>
    <td>44.59</td>
    <td>5.95</td>
    <td>6.47</td>
  </tr>
  <tr>
    <td>vache</td>
    <td>cow</td>
    <td>lait</td>
    <td>milk</td>
    <td>44.58</td>
    <td>6.89</td>
    <td>6.45</td>
  </tr>
  <tr>
    <td>budget</td>
    <td>budget</td>
    <td>argent</td>
    <td>money</td>
    <td>44.44</td>
    <td>5.03</td>
    <td>5.24</td>
  </tr>
  <tr>
    <td>finance</td>
    <td>finance</td>
    <td>argent</td>
    <td>money</td>
    <td>44.30</td>
    <td>3.45</td>
    <td>5.24</td>
  </tr>
  <tr>
    <td>cage</td>
    <td>cage</td>
    <td>oiseau</td>
    <td>bird</td>
    <td>44.29</td>
    <td>6.31</td>
    <td>6.69</td>
  </tr>
  <tr>
    <td>collège</td>
    <td>middle school</td>
    <td>lycée</td>
    <td>high school</td>
    <td>44.29</td>
    <td>6.17</td>
    <td>5.71</td>
  </tr>
  <tr>
    <td>divertir</td>
    <td>entertain</td>
    <td>amuser</td>
    <td>amuse</td>
    <td>43.94</td>
    <td>3.35</td>
    <td>2.88</td>
  </tr>
  <tr>
    <td>compte</td>
    <td>account</td>
    <td>banque</td>
    <td>bank</td>
    <td>43.90</td>
    <td>4.44</td>
    <td>5.47</td>
  </tr>
  <tr>
    <td>individu</td>
    <td>individual</td>
    <td>personne</td>
    <td>person</td>
    <td>43.90</td>
    <td>5.10</td>
    <td>4.71</td>
  </tr>
  <tr>
    <td>mauvais</td>
    <td>bad</td>
    <td>bon</td>
    <td>good</td>
    <td>43.84</td>
    <td>3.26</td>
    <td>2.71</td>
  </tr>
  <tr>
    <td>mélancolie</td>
    <td>melancoly</td>
    <td>tristesse</td>
    <td>sadness</td>
    <td>43.84</td>
    <td>2.69</td>
    <td>2.61</td>
  </tr>
  <tr>
    <td>béton</td>
    <td>concrete</td>
    <td>armer</td>
    <td>reinforced</td>
    <td>43.68</td>
    <td>6.52</td>
    <td>3.92</td>
  </tr>
  <tr>
    <td>propre</td>
    <td>clean</td>
    <td>sale</td>
    <td>dirty</td>
    <td>43.62</td>
    <td>3.98</td>
    <td>3.73</td>
  </tr>
  <tr>
    <td>cartable</td>
    <td>schoolbag</td>
    <td>école</td>
    <td>school</td>
    <td>43.59</td>
    <td>6.78</td>
    <td>5.96</td>
  </tr>
  <tr>
    <td>pyramide</td>
    <td>pyramid</td>
    <td>Egypte</td>
    <td>Egypt</td>
    <td>43.53</td>
    <td>6.31</td>
    <td>5.31</td>
  </tr>
  <tr>
    <td>avenir</td>
    <td>future</td>
    <td>futur</td>
    <td>future</td>
    <td>43.48</td>
    <td>3.04</td>
    <td>2.27</td>
  </tr>
  <tr>
    <td>plage</td>
    <td>beach</td>
    <td>sable</td>
    <td>sand</td>
    <td>43.43</td>
    <td>6.09</td>
    <td>6.49</td>
  </tr>
  <tr>
    <td>velouté</td>
    <td>chowder</td>
    <td>soupe</td>
    <td>soup</td>
    <td>43.43</td>
    <td>5.35</td>
    <td>6.33</td>
  </tr>
</table>
<table><tr><td>faiblesse</td><td>weakness</td><td>force</td><td>strength</td><td>43.42</td><td>2.72</td><td>3.04</td></tr><tr><td>neveu</td><td>nephew</td><td>nièce</td><td>niece</td><td>43.18</td><td>6.20</td><td>5.67</td></tr><tr><td>pays</td><td>country</td><td>France</td><td>France</td><td>42.86</td><td>5.66</td><td>4.94</td></tr><tr><td>maximum</td><td>maximum</td><td>minimum</td><td>minimum</td><td>42.71</td><td>3.04</td><td>2.96</td></tr><tr><td>mal</td><td>bad</td><td>bien</td><td>good</td><td>42.47</td><td>3.14</td><td>2.53</td></tr><tr><td>ricaner</td><td>giggle</td><td>rire</td><td>laugh</td><td>42.47</td><td>4.51</td><td>3.90</td></tr><tr><td>berger</td><td>shepherd</td><td>mouton</td><td>sheep</td><td>42.45</td><td>6.10</td><td>6.65</td></tr><tr><td>aérien</td><td>aerial</td><td>avion</td><td>plane</td><td>42.19</td><td>4.21</td><td>6.63</td></tr><tr><td>action</td><td>action</td><td>réaction</td><td>reaction</td><td>42.17</td><td>4.09</td><td>2.86</td></tr><tr><td>poignard</td><td>dagger</td><td>couteau</td><td>knife</td><td>42.17</td><td>6.39</td><td>6.61</td></tr><tr><td>lit</td><td>bed</td><td>dormir</td><td>sleep</td><td>41.98</td><td>6.84</td><td>4.37</td></tr><tr><td>adulte</td><td>adult</td><td>enfant</td><td>child</td><td>41.94</td><td>4.40</td><td>5.86</td></tr><tr><td>émeraude</td><td>emerald</td><td>vert</td><td>green</td><td>41.94</td><td>6.21</td><td>4.37</td></tr><tr><td>gravir</td><td>climb</td><td>monter</td><td>ascend</td><td>41.94</td><td>4.94</td><td>3.90</td></tr><tr><td>courgette</td><td>zuchini</td><td>légume</td><td>vegetable</td><td>41.46</td><td>6.76</td><td>6.12</td></tr><tr><td>abeille</td><td>bee</td><td>miel</td><td>honey</td><td>41.38</td><td>6.64</td><td>6.33</td></tr><tr><td>printemps</td><td>spring</td><td>été</td><td>summer</td><td>41.38</td><td>4.38</td><td>3.80</td></tr><tr><td>orner</td><td>adorn</td><td>décorer</td><td>decorate</td><td>41.05</td><td>4.61</td><td>3.82</td></tr><tr><td>horreur</td><td>horror</td><td>film</td><td>movie</td><td>41.03</td><td>3.00</td><td>5.69</td></tr><tr><td>célébrer</td><td>showcase</td><td>fêter</td><td>celebrate</td><td>40.82</td><td>3.89</td><td>3.75</td></tr><tr><td>part</td><td>piece</td><td>gâteau</td><td>cake</td><td>40.74</td><td>3.82</td><td>6.31</td></tr><tr><td>colonie</td><td>colony</td><td>vacance</td><td>vacation</td><td>40.63</td><td>4.30</td><td>3.73</td></tr></table>
The aim of the previous chapter was twofold. Firstly, we wanted to provide an update to the availability of word association for French abstract concepts because the most recent database was introduced by Ferrand et al. (2001). Secondly, we developed this database in order to provide control for researchers over the crafting of materials when studying processes related to semantic similarity and verbal association. The implementation of a word association task produced a large amount of data which allowed for the use of methods borrowed from distributional statistics. Main results have shown that concepts are organised in the mental lexicon according to a gradient of concreteness and that concrete concepts elicit stronger associations compared to abstract concepts.

The large amount of data generated by the word association task allowed for the application of semantic network analyses borrowed from the mathematical graph theory. A key advantage of cognitive network science is that it renders possible the exploration of large-scale semantic phenomena from a structural level. The next chapter introduces the first semantic network built from French association data and the first to compare topological parameters for concrete and abstract concepts. Results have shown that the mental lexicon is structured according to a small-world pattern characterised by a sparse density, a short average path length between nodes and a high clustering coefficient.

Comparison analyses between the networks for concrete and abstract networks respectively have shown that concrete concepts are organised in denser communities compared to abstract concepts. Additional analyses have shown that the concrete word nodes are more influential in the network due to their position and patterns of connectivity. This means that concrete nodes have a higher ability to spread information in the network compared to abstract nodes. We relate these characteristics to the assumptions of Paivio (1971) according to which concrete concepts are strongly connected to a limited number of
contexts while abstract concepts are more loosely connected to a great number of contexts.

Semantic network analyses have confirmed these assumptions on a larger scale based on the structural parameters of the mental lexicon.
# Chapter 4. Semantic network analysis of abstract and concrete word associations.

Dounia Lakhzoum, Marie Izaute, and Ludovic Ferrand

**Chapter 4 was submitted as a preprint under the reference:**

Lakhzoum, D., Izaute, M., & Ferrand, L. (2021). Semantic network analysis of abstract and concrete word associations. https://doi.org/10.31234/osf.io/epk47

Supplementary material: https://osf.io/4w69k/

Université Clermont Auvergne, CNRS, LAPSCO, F-63001 Clermont-Ferrand, France

**Corresponding author:** Dounia Lakhzoum or Ludovic Ferrand, Université Clermont Auvergne, CNRS LAPSCO, 34 avenue Carnot TSA 60401-63001 Clermont-Ferrand Cedex 1, France

Email: dounia.lakhzoum@ext.uca.fr

**Author's note**

This research was financed by the French government IDEX-ISITE initiative 16-IDEX-0001 (CAP 20-25).

**ACKNOWLEDGMENT**

This work was sponsored by a public grant overseen by the French National Research Agency as part of the "Investissements d'Avenir" through the IDEX-ISITE initiative CAP 20-25 (ANR-16-IDEX-0001).
ABSTRACT

In recent years, a new interest for the use of graph-theory based networks has emerged within the field of cognitive science. This has played a key role in mining the large amount of data generated by word association norms. In the present work, we applied semantic network analyses to explore norms of French word associations for concrete and abstract concepts (Lakhzoum et al., 2021). Graph analyses have shown that the network exhibits high clustering coefficient, sparse density, and small average shortest path length for both the concrete and abstract networks. These characteristics are consistent with a small-world structure.

Comparisons between local node statistics and global structural topology showed that abstract and concrete concepts present a similar local connectivity but different overall patterns of structural organisation with concrete concepts presenting an organisation in densely connected communities compared to abstract concepts. These patterns confirm previously acquired knowledge about the dichotomy of abstract and concrete concepts on a larger scale.

To the best of our knowledge, this is the first attempt to confirm the generalisability of these properties to the French language and with an emphasis on abstract and concrete concepts.
# INTRODUCTION

In recent years, a new interest has emerged within the field of cognitive science for mathematical models such as graph-theory based science networks (Baronchelli, Ferrer-i-Cancho, Pastor-Satorras, Chater, & Christiansen, 2013; see Siew, Wulff, Beckage, & Kenett, 2019 for a review). More specifically, the use of semantic network analysis on word association offers a powerful computational technique for the modelling of structures like the mental lexicon (De Deyne & Storms, 2008; De Deyne, Verheyen & Storms, 2016; De Deyne, Navarro, Perfors, Brysbaert & Storms, 2019; Stella, Beckage, Brede, & De Domenico, 2018). Models have shown that word associations are organised according to the same small-world structure found in many natural phenomena (Steyvers & Tenenbaum, 2005).

## Network science and characteristics

A network is built as a graph that consists of nodes linked by edges or arcs. In a network based on association data, the nodes correspond to words connected by an undirected edge or a directed arc. A directed network provides additional information about the direction of the association from cue to response words. The arcs and edges can have weights to represent the frequency of association. Adjacent nodes are directly connected by an arc or edge whereas nonadjacent nodes are connected by a path. Based on this macroscopic structure, it is possible to compute network statistics of interest. The macrostructure of the network can be described in terms of its diameter ($D$), which gives the largest path length between two nodes. The average shortest path length (ASPL) that links two nodes is also of interest. It refers to the average minimum number of steps from one node to another. Nodes are characterised by the degree corresponding to their number of connections. In a directed network, each node has an in-degree ($k^{in}$) that is the number of incoming arcs and an out-degree ($k^{out}$) that is the number of outgoing arcs. An undirected network only has degree $k$ corresponding to the number of
edges of a node. The average degree $<k>$ of the network can provide valuable information about the connectivity of the network itself but also about nodes centrality within the network. The clustering coefficient (CC) constitutes such a measure of centrality. It is the probability that two nodes are interconnected knowing they both are connected to a common neighbour. Finally, the density ($d$) is defined as a ratio of the number of edges in the network to the number of all possible edges (Steyvers & Tenenbaum, 2005; De Deyne & Storms, 2008; Csardi & Nepusz, 2006; Kolaczyk & Csardi, 2014; see Table 1 for a summary of all network statistics described above).

<table><caption>**Table 4.** Description of network characteristics used throughout the paper.</caption><thead><tr><td>network characteristics</td><td>Description</td></tr></thead><tbody><tr><td>D</td><td>the diameter of the network</td></tr><tr><td>ASLP</td><td>the average length of shortest path</td></tr><tr><td>CC</td><td>the clustering coefficient</td></tr><tr><td>&lt;k&gt;</td><td>average degree</td></tr><tr><td>k, k<sup>in</sup>, k<sup>out</sup></td><td>the degree, in-degree, and out-degree</td></tr><tr><td>d</td><td>density of the network</td></tr><tr><td>S</td><td>Smallworldness</td></tr></tbody></table>

Based on these characteristics, some networks exhibit small-world properties when compared to similar random networks (Milgram, 1967; Watts & Strogatz, 1998; Steyvers & Tenenbaum, 2005). Humphries and Gurney (2008) introduced a quantitative measure called smallworldness ($S$) aimed to assess the small-world properties of a network. This measure corresponds to the ratio of the clustering coefficient (CC) to the average shortest path length (ASPL). The network is considered smallworded when $S$ is higher

Based on these characteristics, some networks exhibit small-world properties when compared to similar random networks (Milgram, 1967; Watts & Strogatz, 1998; Steyvers & Tenenbaum, 2005). Humphries and Gurney (2008) introduced a quantitative measure called smallworldness ($S$) aimed to assess the small-world properties of a network. This measure
corresponds to the ratio of the clustering coefficient (CC) to the average shortest path length (ASPL). The network is considered smallworded when *S* is higher than 3 (Humphries & Gurney, 2008). A small-world network when compared to a random network of equal size and number of edges would have a lower average shortest path length but a higher clustering coefficient (Erdös & Rényi, 1960; Watts & Strogatz, 1998; Steyvers & Tenenbaum, 2005; De Deyne & Storms, 2008). Smallworld networks represent a vast collection of models that can be found in multiple real-world phenomena.

## Small-world structures and semantic networks

Milgram (1967), in an experiment on social networks tested the principle according to which people are on average separated by only six social connections. He asked a group of participants (origin node) to send a letter to a stockbroker (destination node) using their own social connections. Results suggested a small degree of separation between any two people living in the United States at the same time. According to Milgram (1967), this suggested an organisation of social connections according to a small-world structure.

Other studies later confirmed these small-world properties in social networks (e.g., Watts & Strogatz, 1998; Kitsak et al., 2010), epidemiology of infectious diseases (e.g., Keeling & Rohani, 2008), spread of computer viruses (e.g., Pastor-Satorras & Vespignani, 2001), protein interaction (e.g., Bork et al., 2004), gene expression (e.g., van Noort, Snel & Huynen, 2004), neural (e.g., Basset & Bullmore, 2017) and semantic networks (e.g., Steyvers & Tenenbaum, 2005).

The use of semantic networks for the study of language corpora has a long history as evidenced by the development of influential computational models such as LSA (Latent Semantic Analysis, Landauer & Dumais, 1997). It models the assumption that words that co-occur in similar contexts share the same meaning (Firth, 1968). This corpus approach relies
on the mining of large text corpora that provide a good representation of the linguistic
environment from which language is acquired and processed (De Deyne et al. 2016). The use
of co-occurrence to define meaning is limited however because it does not distinguish
between similarity-based co-occurrences and purely verbal associates as it is the case for
psycholinguistic models of semantics. The use of word associations collected from human
participants can address this limitation by providing large-scale networks to study the mental
lexicon from data that are psychologically plausible.

The word association task is a simple yet powerful tool that can generate a large
amount of data to gain insights into the organization of the mental lexicon. Traditionally,
these data are used to create word association norms that are applied in factorial designs to
study varied cognitive phenomena such as visual word recognition, memory recall, and
semantic and lexical decision tasks (Balota, Cortese, Sergent-Marshall, Spieler & Yap, 2004;
Hutchison, 2003; McNamara, 2005). The modelling of these data using network science not
only is interesting because it provides psychologically plausible data or the ability to explore
semantic memory on a larger scale than what is possible to achieve with factorial designs but
is also theoretically sound. This technique is embedded in the seminal work of Collins and
Loftus (1975) who established the network structure of the mental lexicon. In addition to the
structural properties, semantic networks have the ability to model functional properties such
as the spreading activation process that takes place when a node is activated in the network
and all nodes that are connected to it are activated in turn. It is also possible to consider the
weights and paths length in a directed network to model activation decay as a function of time
and distance (Collins & Loftus, 1975; Den Heyer, & Briand, 1986; Kenett, Levi, Anaki &
Faust, 2017).

Many studies have investigated the above-mentioned characteristics of semantic
networks (Motter, Moura, Lai & Dasgupta, 2002; Sigman & Cecchi 2002; Steyvers &
Tenenbaum 2005; Bales & Johnson 2006; Borge-Holthoefer & Arenas 2010; Choudhury & Mukherjee 2009; Fukś & Krzemiński 2009; Veremyev, Semenov, Pasiliao & Boginski, 2019).

Steyvers and Tenenbaum (2005) analysed semantic networks based on behavioural word associations and large text corpora. They showed that they all have a small-world structure characterized by sparse connectivity, short average path lengths, and a strong clustering coefficient. These small-world properties reflect an organisation of the lexicon in highly connected hubs that allow efficient information distribution and contributes to the robustness of the structure as a whole (Borge-Holthoefer, Moreno, & Arenas, 2012; De Deyne et al. 2016). It also provides insights into the network growth mechanisms as newly acquired concepts join tightly connected hubs over time (Steyvers, and Tenenbaum, 2005). These results were later replicated in Dutch (De Deyne et al., 2008), German and Spanish (Borge-Holthoefer & Arenas, 2010), Hebrew (Kenett et al., 2017) and most recently in Persian (Karimkhani et al., 2021) showing that these properties still hold for Semitic languages.

**The present work**

In the present work, we applied techniques of semantic network analyses on behavioural abstract and concrete word associations previously collected from French native speakers (Lakhzoum et al., submitted). The first aim of this work is to replicate the network analyses and small-world properties found by Steyvers and Tenenbaum (2005) and generalise their results to the French language. A second aim is to explore potential differences in network properties when separating abstract and concrete word associations in two distinct networks. This is, to the best of our knowledge, the first attempt to study the characteristics of abstract and concrete networks and the first attempt to generalise small world properties to the French language.
# METHOD

## Material and procedures

A detailed description of the French word association data is provided in Lakhzoum et al., (submitted). The word associations were based on 1100 cues with varying levels of concreteness from very abstract to very concrete on a 7-points scale. They were collected in an online experiment among 1200 French native speakers. This resulted in a dataset of 92000 responses. Following the procedure of De Deyne et al. (2019), only the responses suggested by at least two participants were kept. For the remaining data, a weighted adjacency matrix was computed. The cue words and associated responses were used to create the nodes in the networks. The links between cue and responses were used as the directed arcs and the association strength values were represented in the weights of the arcs. The whole network was later divided into the concrete and abstract networks based on the median level of concreteness of the cues.

# RESULTS

Semantic network analyses do not rely on techniques of hypothesis testing, as it is the case for factorial designs (Moreno & Neville, 2013). The insights about the data are drawn from comparisons of directed and undirected networks with random networks of comparable size, density and mean connectivity <k>. More specifically, the benchmark random networks used to estimate smallworldness (ASPL and CC parameters) were generated with the same size and mean connectivity that provided the parameters ASPLrandom and CCrandom (see Table 1). Most of the literature on semantic networks relies on unweighted, undirected networks. However, considering the present application in word association data, the direction of the edges and the frequency of the association are of importance. The following analyses are based on directed and undirected weighted graphs. The variable *n* refers to the number of
nodes corresponding to the word association dataset (Lakhzoum et al., submitted). The whole
network corresponds to the graph representation of the entire dataset (see Figure 1 for a
representation of the whole network; see Figure 2 for a more focused visualisation of the
network)

<figure><img src="image_17.png" /><figcaption>**Figure 1.** Visual representation of the whole network. Each point represents a word. The colour gradient from grey to red represents the distance between connected nodes the greyer the farther and the redder the closer (built with the software Gephi; Bastian, Heymann & Jacomy, 2009).</figcaption></figure>
<figure><img src="image_18.png" /><figcaption>**Figure 2.** Visualisation of a portion of the network with a representation of the nodes degree were the larger and the darker green a node is and the more connection it has. The node with the most number of connection on this portion is the French concept “amour” or love in English. One of the nodes with the smallest degree is the French concept “ennui” or boredom in English (built with the software Gephi; Bastian, Heymann & Jacomy, 2009).</figcaption></figure>

Based on our previous study, we were able to determine that concrete and abstract cue words tend to elicit responses of the same level of concreteness (Lakhzoum et al., submitted) which justified our choice to explore both levels in separate networks. The concrete and abstract networks correspond to the datasets for the concrete and abstract cue words respectively.

The first parameter of interest is the diameter (D) of the network that defines the maximum distance between two nodes. It provides information about the macrostructure of the network and an estimation of the overall connectivity as it relates to the density parameter (d). For the whole network, the diameter shows that the most distant nodes are separated by
20 (directed) and 9 (undirected) connections. In the concrete network, the most distant nodes are separated by 24 (directed) and 9 (undirected) connections. Finally, in the abstract network, the most distant nodes are separated by 19 (directed) and 10 (undirected) connections. These diameters are overall higher than the ones found in the Leven dataset (De Deyne et al., 2008) which is explained by the higher density of their network.

<table><caption>**Table 2.** Network statistics for the directed and undirected associative networks for the whole, concrete and abstract datasets.</caption><thead><tr><th rowspan="2">variables</th><th colspan="2">Whole network</th><th colspan="2">Concrete network</th><th colspan="2">Abstract network</th></tr><tr><th>directed</th><th>undirected</th><th>directed</th><th>undirected</th><th>directed</th><th>undirected</th></tr></thead><tbody><tr><td>n</td><td colspan="2">4,758</td><td colspan="2">2,961</td><td colspan="2">2,827</td></tr><tr><td>D</td><td>20</td><td>9</td><td>24</td><td>9</td><td>19</td><td>10</td></tr><tr><td>d</td><td colspan="2">5.10<sup>-3</sup></td><td colspan="2">6.10<sup>-3</sup></td><td colspan="2">7.10<sup>-3</sup></td></tr><tr><td>ASPL</td><td>7.07</td><td>4.90</td><td>8.46</td><td>5.13</td><td>7.20</td><td>5.00</td></tr><tr><td>ASPL<sub>random</sub></td><td>10.00</td><td>7.05</td><td>10.21</td><td>6.63</td><td>10.13</td><td>6.58</td></tr><tr><td>k<sub>in/out</sub></td><td>2.46</td><td>-</td><td>1.97</td><td>-</td><td>2.07</td><td>-</td></tr><tr><td>&lt;k&gt;</td><td>4.92</td><td>4.92</td><td>3.94</td><td>3.94</td><td>4.13</td><td>4.13</td></tr><tr><td>CC</td><td>0.15</td><td>0.07</td><td>0.15</td><td>0.07</td><td>0.14</td><td>0.06</td></tr><tr><td>CC<sub>random</sub></td><td>0.0005</td><td>0.0005</td><td>0.001</td><td>0.0008</td><td>0.0007</td><td>0.001</td></tr><tr><td>S</td><td colspan="2">682</td><td colspan="2">925</td><td colspan="2">657</td></tr><tr><td>Q</td><td colspan="2">0.62</td><td colspan="2">0.70</td><td colspan="2">0.64</td></tr></tbody></table>

Note. n = the number of nodes; D = the diameter of the network; <k> = the average number of connections; ASPL = the average shortest path length; CC = clustering coefficient; ASPL<sub>random</sub> = the average shortest path length with random graph of same size and density; CC<sub>random</sub> = the clustering coefficient for a random graph of same size and density.

The density of the network serves two purposes. Firstly, it allows to create a comparable random network to provide a benchmark for small-world analyses. Secondly, it serves to check the sparseness of the network. The sparseness of the whole network is shown by a very low density of 5.10<sup>-3</sup>. This means that about 0.05% of all possible associations between words are represented. When comparing the density of the concrete and abstract
networks, the concrete network presents an even lower density, which is in accordance with its higher diameter. This extreme sparseness can be explained by the variety of the dataset in terms of levels of concreteness of the cues. We expect that the concrete and abstract data share very few connections and belong to separate clusters. The sparseness of these networks -very low density- is the first mark of their small-world structure.

Based on the average shortest path length (ASPL), there was an average of 7.07 (directed) and 4.90 (undirected) steps from one node to another in the whole network. This average number of steps was higher in the concrete network (8.46 for the directed and 5.13 for the undirected) compared to the abstract network (7.20 for the directed and 5.00 for the undirected). This pattern is in accordance with the diameter. A series of random networks were generated with comparable size *n* and mean connectivity <*k*> to serve as benchmark for smallworldness estimation. All networks had the same density as their corresponding random benchmarks. According to Table 2, the ASPL statistics were consistently smaller for the actual networks compared to the randomly generated ones. A small ASPL is a fundamental characteristic of a small-world structure.

The clustering coefficient is a measure of the probability that associates of a word are also connected to each other. This parameter is evaluated against a comparable random network. Table 2 shows that associates of a word in the directed networks were also connected to each other about 15% of the times for the whole and concrete networks and 14% of the times for the abstract network. For the undirected networks, this probability dropped to approximately 6-7 % of the time. The clustering coefficients of the whole, concrete and abstract networks were very high compared to the ones found for the benchmark random networks that ranged from 0.05 to 0.1% probability to find a link between two nodes that have a neighbour in common. The clustering coefficient is among the most important indicator of a
small-world structure. It suggests that when additional nodes are introduced in the network, they join a tightly connected hub, which insures the stability of the entire network.

Taken together, the sparseness, average short path lengths and clustering coefficient show that the present networks exhibit the same small-world properties found by Steyvers and Tenenbaum (2005). A final measure established by Humphries and Gurney (2008) was computed for each network and their corresponding random networks to evaluate their smallworldness. Based on this parameter, Humphries and Gurney (2008) indicated that an S measure > 3 indicates a small-world structure. All the networks presented a very high S measure.

Finally, the modularity measure (Q) introduced by Newman (2006) indicates the extent to which a network is composed of sub-communities. Communities are non-overlapping groups of nodes with a high number of within community connections and a low number of between-community connections. The concrete network is more modular (Q = 0.70) compared to the abstract network (Q = 0.64).

Network analysis is rooted in the seminal work of Collins and Loftus (1975) who described the spreading of activation according to which the activation of concepts in memory spreads and activates other closely related concepts. This process is fundamental in cognitive science for its ability to explain many phenomena related to the mental lexicon (see Siew et al., 2019 for a review). Given the structural differences we have found in the concrete and abstract networks, it is fair to assume that the two types of concepts would exhibit different patterns of spreading scores. In the same way that spreading activation mimics the diffusion of discrete events such as the spread of disease among individuals, it is possible to identify initial “infected” nodes and their ability to spread the discrete event which is modelled (Siew et al., 2019). Salavaty, Ramialison and Currie (2020) have proposed a computational method
to identify the most influential nodes in a network that have a higher spreading scores for each node. Spreading scores are an indication of the capacity for a node to spread information within the network. The algorithm was used for each abstract and concrete network and generated a vector of spreading scores for each node. Comparison of spreading scores for abstract and concrete nodes respectively showed that abstract nodes (M=2.56; SD=4.75) have a lower spreading score compared to concrete nodes (M=3.72; SD = 6.57) – t(5390) = 7.70; p < 0.001. These results suggest that the abstract nodes in the network are less influential and have a lesser ability to spread information within the network compared to concrete nodes.

DISCUSSION

In the present work, the data obtained from word associations was represented as
several networks using graph-theory based principles. This aimed at extracting structural and
organisational patterns of the mental lexicon that would not be possible to gain from the raw
dataset (De Deyne & Storms, 2008; Csardi & Nepusz, 2006; Kolaczyk & Csardi, 2014). The
whole network represented the entire dataset. Next, based on a previously established pattern
of cues eliciting responses of the same level of concreteness (Lakhzoum et al., 2021), the
whole network was divided into a concrete and an abstract network by using the median
concreteness level to separate them. A first series of analyses used random networks of a
similar density and connectivity as benchmark (Lerner, Ogrocki, & Thomas, 2009).
Comparisons between the present networks and comparable random ones showed that the
networks exhibited a small ASPL and a high clustering coefficient characteristic of a small-
world structure (Watts & Strogatz, 1998; Steyvers & Tenenbaum, 2005; De Deyne & Storms,
2008). The small world structure provides valuable information not only about the structural
characteristics of the networks but also about the way the networks will expand with the
addition of new nodes. In a small-world network, when new nodes are included they join an
already established cluster of existing nodes. This means that the network does not expand
proportionally with the addition of new nodes. This small-world effect insures that networks
present a highly robust architecture that is resistant to change and disturbances (Peng, Tan,
Wu & Holme, 2016).

When comparing the topological properties of the concrete and abstract networks, the
concrete networks seems more spread out (higher ASPL), more connected (higher S
measure), more modular (higher Q measure) than the abstract network. This pattern of
densely connected but loosely interconnected nodes for the concrete network concurs with the
pattern found in association strength with concrete cues eliciting stronger associates compared
to the abstract ones. It also concurs with the dichotomy established by Paivio (Paivio, Yuille
& Madigan, 1968; but see Della Rosa, Catricalà, Vigliocco & Cappa, 2010) between concrete
and abstract concepts with concrete concepts being more highly connected to a smaller
number of contexts while abstract concepts are more loosely connected to a higher number of
contexts. This pattern of abstract concepts was confirmed in previous studies on feature-
listing norms (Recchia & Jones, 2012) and now on a larger scale as evidence by a smaller
modularity measure in the abstract network.

Taken together with previous findings, these local node statistics suggest that abstract
and concrete concepts do not differ in their pattern of connectivity but rather on the strength
of these connections with concrete concepts presenting stronger association compared to
abstract concepts (Lakhzoum et al., submitted). This pattern of weaker connection for abstract
nodes compared to concrete nodes is further reflected in their respective spreading scores with
abstract nodes presenting a lesser ability to spread information in the network compared to
concrete nodes. These scores are reflective of the position of the nodes in the network and
influenced by clustering and modularity as well as the weights of their connections. This
suggests that abstract concepts, due to their more diffuse pattern of organisation and weaker
connections given by associative strength from the word association data have a lesser ability
to spread semantic activation in the mental lexicon. This is in accordance with previous findings according to which abstract concepts present a processing disadvantage compared to concrete concepts as evidenced by the longer response times they elicit (see Borghi, Binkofski, Castelfranchi, Cimatti, Scorolli & Tummolini, 2017).

Taken together, these characteristics suggest that the local organisation of abstract and concrete concepts is similar while their structural position within the network shows a different pattern with concrete concepts presenting a more modular pattern compared to abstract concepts.

## FUTURE WORK

The present work represents a first attempt to study the structural dichotomy in the organisation of concrete and abstract concepts in the mental lexicon. A future project will have to replicate the same analyses using an English word association dataset such as the SWOW-EN database of De Deyne and colleagues (2019). This will allow for a replication on a higher scale because the database contains 12.000 cues for more than 4 million responses. Realising the same network in English will also provide a baseline comparison for the research community and the opportunity to compare between English and other languages. By cross-referencing with concreteness ratings norms (Brysbaert, Warriner & Kuperman, 2014), it will be possible to understand the same structural organisation of abstract and concrete concepts on a larger scale.

Additionally, it must be kept in mind that the present network representation is not a neural network and does not yield any insights into the mapping of abstract and concrete concepts representation in the human brain. Network science can be applied to study brain connectivity based on the same techniques and parameters used in the present work (Siew et al., 2019 for a review). Based on recording of brain activity during a word association task for
abstract and concrete concepts, it will be possible to map the cognitive patterns obtained in
the present work to the neural substrates that generated said cognitive patterns.

REFERENCES

Baronchelli, A., Ferrer-i-Cancho, R., Pastor-Satorras, R., Chater, N., & Christiansen, M. H. (2013). Networks in cognitive science. *Trends in Cognitive Sciences, 17*, 348–360.
https://doi.org/10.1016/j.tics.2013.04.010

Bastian, M., Heymann, S., & Jacomy, M. (2009). Gephi: an open source software for exploring and manipulating networks. In *Third international AAAI conference on weblogs and social media*.

Bassett, D. S., & Bullmore, E. T. (2017). Small-World Brain Networks Revisited. *Neuroscientist*, 23, 499–516. https://doi.org/10.1177/1073858416667720

Borge-Holthoefer, J., & Arenas, A. (2010). Semantic networks: Structure and dynamics. *Entropy*, 12, 1264–1302. https://doi.org/10.3390/e12051264

Borge-Holthoefer, J., Moreno, Y., & Arenas, A. (2012). Topological versus dynamical robustness in a lexical network. *International Journal of Bifurcation and Chaos*, 22, 1–9.
https://doi.org/10.1142/S021812741250157X

Borghi, A. M., Binkofski, F., Castelfranchi, C., Cimatti, F., Scorolli, C., & Tummolini, L. (2017). The challenge of abstract concepts. *Psychological Bulletin*, 143, 263–292.
https://doi.org/10.1037/bul00000089

Bork, P., Jensen, L. J., Von Mering, C., Ramani, A. K., Lee, I., & Marcotte, E. M. (2004). Protein interaction networks from yeast to human. *Current Opinion in Structural Biology*, 14, 292–299.
https://doi.org/10.1016/j.SBI.2004.05.003

Brysbaert, M., Warriner, A. B., & Kuperman, V. (2014). Concreteness ratings for 40 thousand English word lemmas. *Behavior Research Methods*, 46, 904–911.

Choudhury, M., & Mukherjee, A. (2009). The Structure and Dynamics of Linguistic Networks. *Modeling and Simulation in Science, Engineering and Technology*, 42, 145–166.
https://doi.org/10.1007/978-0-8176-4751-3_9

Collins, A. M., & Loftus, E. F. (1975). A spreading-activation theory of semantic processing. *Psychological Review*, 82, 407–428.
https://doi.org/10.1037/0033-295X.82.6.407

Csardi, G., & Nepusz, T. (2006). The igraph software package for complex network research. *InterJournal Complex Systems*, 1695. Retrieved from http://igraph.sf.net

De Deyne, S., Navarro, D. J., Perfors, A., Brysbaert, M., & Storms, G. (2019). The “Small World of Words” English word association norms for over 12,000 cue words. *Behavior Research Methods*, 51, 987–1006.
https://doi.org/10.3758/s13428-018-1115-7

De Deyne, S., & Storms, G. (2008). Word associations: Network and semantic properties. *Behavior Research Methods*, 40, 213–231.
https://doi.org/10.3758/BRM.40.1.213

Della Rosa, P. A., Catricalà, E., Vigliocco, G., & Cappa, S. F. (2010). Beyond the abstract-concrete dichotomy: Mode of acquisition, concreteness, imageability, familiarity, age of acquisition, context availability, and abstractness norms for a set of 417 Italian words. *Behavior Research Methods*, 42, 1042–1048.
https://doi.org/10.3758/BRM.42.4.1042

den Heyer, K., & Briand, K. (1986). Priming Single Digit Numbers: Automatic Spreading Activation Dissipates as a Function of Semantic Distance. *The American Journal of Psychology*, 99, 315–340.
https://doi.org/10.2307/1422488

Deyne, S. De, Verheyen, S., & Storms, G. (2016). Structure and Organization of the Mental Lexicon: A Network Approach Derived from Syntactic Dependency Relations and Word
Associations. Understanding Complex Systems, 99, 47-79. https://doi.org/10.1007/978-3-662-47238-5_3

Erdos, P., & Rényi, A. (1960). On the evolution of random graphs. *Publications of the Mathematical Institute of the Hungarian Academy of Sciences*, 5, 17-60.

Fukś, H., & Krzemiński, M. (2009). Topological structure of dictionary graphs. *Journal of Physics A: Mathematical and Theoretical, 42*, 375101. https://doi.org/10.1088/1751-8113/42/37/375101

Humphries, M. D., & Gurney, K. (2008). Network 'Small-World-Ness': A Quantitative Method for Determining Canonical Network Equivalence. *PLOS ONE*, 3, e0002051. https://doi.org/10.1371/JOURNAL.PONE.0002051

Hutchison, K. A. (2003). Is semantic priming due to association strength or feature overlap? A microanalytic review. *Psychonomic Bulletin & Review*, 10, 785-813. https://doi.org/10.3758/BF03196544

Karimkhani, F., Rahmani, H., Zare, A., Sahebnassagh, R., & Aghakasiri, K. (2021). Tarvajeh: Word Association Norms for Persian Words. *Journal of Psycholinguistic Research*, 50, 863-882. https://doi.org/10.1007/s10936-020-09751-2

Keeling, M., & Rohani, P. (2011). *Modeling infectious diseases in humans and animals* Princeton: New York University Press. https://doi.org/10.1515/9781400841035

Kenett, Y. N., Levi, E., Anaki, D., & Faust, M. (2017). The semantic distance task: Quantifying semantic distance with semantic network path length. *Journal of Experimental Psychology: Learning Memory and Cognition*, 43, 1470-1489. https://doi.org/10.1037/XLM0000391

Kitsak, M., Gallos, L. K., Havlin, S., Liljeros, F., Muchnik, L., Stanley, H. E., & Makse, H. A. (2010). Identification of influential spreaders in complex networks. *Nature Physics* 2010 6:11, 6, 888-893. https://doi.org/10.1038/nphys1746

Kolaczyk, E. D., & Csárdi, G. (2014). *Statistical Analysis of Network Data with R*. New York: Springer. Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-44129-6

Landauer, T. K., & Dumais, S. T. (1997). A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. *Psychological Review*, 104, 211-240. https://doi.org/https://doi.org/10.1037/0033-295X.104.2.211

Lerner, A. J., Ogrocki, P. K., & Thomas, P. J. (2009). Network graph analysis of category fluency testing. *Cognitive and Behavioral Neurology*, 22, 45-52. https://doi.org/10.1097/WNN.0B013E318192CCAF

M, S., & JB, T. (2005). The large-scale structure of semantic networks: statistical analyses and a model of semantic growth. *Cognitive Science*, 29, 41-78. https://doi.org/10.1207/S15516709COG2901_3

McNamara, T. P. (2005). *Semantic priming: Perspectives from memory and word recognition. Semantic priming: Perspectives from memory and word recognition*. New York, NY, US: Psychology Press. https://doi.org/10.4324/9780203338001

Milgram, S. (1967). The small world problem. *Psychology Today*, 2, 60-67.

Moreno, S., & Neville, J. (2013). Network Hypothesis Testing Using Mixed Kronecker Product Graph Models. In *2013 IEEE 13th International Conference on Data Mining* (pp. 1163-1168). https://doi.org/10.1109/ICDM.2013.165

Motter, A., Moura, A., Lai, Y.-C., & Dasgupta, P. (2002). Topology of the conceptual network of language. *Physical Review. E, Statistical, Nonlinear, and Soft Matter Physics*, 65, 65102. https://doi.org/10.1103/PhysRevE.65.065102
Newman, M. E. J. (2006). Modularity and community structure in networks. *Proceedings of the National Academy of Sciences of the United States of America*, 103, 8577-8582.
https://doi.org/10.1073/pnas.0601602103

Paivio, A., Yuille, J. C., & Madigan, S. A. (1968). Concreteness, imagery, and meaningfulness values for 925 nouns. *Journal of Experimental Psychology*, 76, 1-25.
https://doi.org/10.1037/h0025327

Pastor-Satorras, R., & Vespignani, A. (2001). Epidemic spreading in scale-free networks. *Physical Review Letters*, 86, 3200-3203. https://doi.org/10.1103/PhysRevLett.86.3200

Peng, G. S., Tan, S. Y., Wu, J., & Holme, P. (2016). Trade-offs between robustness and small-world effect in complex networks. *Scientific Reports*, 6, 1-10.
https://doi.org/10.1038/srep37317

Recchia, G., & Jones, M. N. (2012). The semantic richness of abstract concepts. *Frontiers in Human Neuroscience*, 6, 315. https://doi.org/10.3389/fnhum.2012.00315

Salavaty, A., Ramialison, M., & Currie, P. D. (2020). Integrated Value of Influence: An Integrative Method for the Identification of the Most Influential Nodes within Networks. *Patterns*, 1, 100052. https://doi.org/10.1016/j.patter.2020.100052

Siew, C. S. Q., Wulff, D. U., Beckage, N. M., & Kenett, Y. N. (2019). Cognitive Network Science: A Review of Research on Cognition through the Lens of Network Representations, Processes, and Dynamics. *Complexity*, 2019, 2108423.
https://doi.org/10.1155/2019/2108423

Sigman, M., & Cecchi, G. A. (2002). Global organization of the Wordnet lexicon. *Proceedings of the National Academy of Sciences*, 99(3), 1742-1747.
https://doi.org/10.1073/PNAS.022341799

Stella, M., Beckage, N. M., Brede, M., & De Domenico, M. (2018). Multiplex model of mental lexicon reveals explosive learning in humans. *Scientific Reports*, 8, 1-11.
https://doi.org/10.1038/s41598-018-20730-5

van Noort, V., Snel, B., & Huynen, M. A. (2004). The yeast coexpression network has a small-world, scale-free architecture and can be explained by a simple model. *EMBO Reports*, 5, 280-284. https://doi.org/10.1038/sj.embor.7400090

Veremyev, A., Semenov, A., Pasiliao, E. L., & Boginski, V. (2019). Graph-based exploration and clustering analysis of semantic spaces. *Applied Network Science*, 4(1).
https://doi.org/10.1007/s41109-019-0228-y

Watts, D. J., & Strogatz, S. H. (1998). Collective dynamics of ‘small-world’ networks. *Nature*, 393, 440-442. https://doi.org/10.1038/30918

Siew, C. S. Q., Wulff, D. U., Beckage, N. M., & Kenett, Y. N. (2019). Cognitive Network Science: A Review of Research on Cognition through the Lens of Network Representations, Processes, and Dynamics. *Complexity*, 2019, 2108423.
https://doi.org/10.1155/2019/2108423
The previous chapter was an application of the mathematical graph-theory on French
word association data for concrete and abstract networks. Graph analyses have extended
previous findings as regards to the small-world structure of the mental lexicon characterised
by a sparse density, a short average path length and a high clustering coefficient to the French
language. Comparison of the concrete and abstract networks have shown that abstract
concepts are organised in a more diffuse pattern of connectivity compared to concrete
concepts which confirms our findings about the weaker associative strength exhibited by
abstract concepts compared to concrete concepts on a larger scale and on a structural level.
Additional comparison of spreading scores for abstract and concrete nodes have shown that
concrete concepts are more influential in the network compared to abstract concepts meaning
they have a higher ability to spread information in the network. These results are providing
some insights as regards to the apparent paucity in semantic content of abstract concepts
compared to concrete concepts and the processing advantage of concrete concepts over
abstract concepts (Paivio et al., 1986, but see Borghi et al., 2017 for a discussion).

More recent studies have explored the richness of abstract concepts and have found
that what abstract concepts lack in physical features they gain in other types of features such
as emotion, social, introspective and metacognitive (see for example Recchia & Jones, 2012).
The next chapter builds on the work of McRae et al. (2018) who used a picture-word
paradigm and provided evidence for the extraction of situational features in abstract concept
processing. In contrast, Kuipers et al. (2018) showed that abstract concepts can be activated
by abstract pictures, which lack such tangible situational features. In a series of experiments,
we compared the role of situational and intangible features in the processing of abstract
concepts. Results have shown that even when compared with situational picture primes,
extraction mechanisms still occurred in the case of abstract pictures. We interpreted these
results as further evidence of the richness of abstract concepts which can be activated from
features other than lexical or situational, namely also abstract and intangible.
# Chapter 5. Intangible features extraction in the processing of abstract concepts: Evidence from picture-word priming

Dounia Lakhzoum¹*, Marie Izaute¹, and Ludovic Ferrand¹*

¹Université Clermont Auvergne, CNRS, LAPSCO, Clermont-Ferrand, France

**Chapter 5 was published under the reference** Lakhzoum, D., Izaute, M., & Ferrand, L. (2021). Intangible features extraction in the processing of abstract concepts: Evidence from picture-word priming. *Plos One*, 16, e0251448.

Supplementary material: https://osf.io/483tu/

*Corresponding authors: Dounia Lakhzoum or Ludovic Ferrand, Université Clermont

E-mail: [dounia.lakhzoum@uca.fr](mailto:dounia.lakhzoum@uca.fr) (DL); [ludovic.ferrand@uca.fr](mailto:ludovic.ferrand@uca.fr) (LF)
ABSTRACT

Over the last decade, hypotheses ranging from linguistic symbol processing to embodiment have been formulated to account for the content and mechanisms responsible for the representation of abstract concepts. Results of recent studies have suggested that abstract concepts, just like concrete ones, can benefit from knowledge of real-world situational context, but that they can also be processed based on abstract pictures devoid of such situational features. This paper presents two semantic priming experiments to explore such mechanisms further. The first experiment replicates Kuipers, Jones, and Thierry (2018) in a cross-linguistic setting which shows that abstract concepts can be processed from abstract pictures devoid of tangible features. In the second experiment, we studied extraction mechanisms that come into play when participants are presented with abstract and concrete pictures that provide situational information to illustrate target abstract concepts. We expected this facilitatory effect to be limited to concrete picture primes. Our data analysed with both Bayesian and Frequentist tests showed however that even when presented with tangible situational information, the extraction of features still occurred for abstract pictures. We discuss the implications of this with respect to future avenues for studying the processing of abstract concepts.
# INTRODUCTION

Abstract concepts have long been a riddle for the brain and cognitive sciences. The past decade saw a greatly enhanced research effort as regards abstract concepts representation (e.g., [1]; see [2] for a review). In spite of such efforts, challenging questions remain as to how abstract concepts convey meaning and the type of information in which their representation is encoded.

## Dimensions of abstract concepts

With abstract concepts, there is no one-to-one matching between the concept-word and a single referent. Usually they refer to complex situations involving multiple objects and entities [3–5]. Until recently, their definition was formulated primarily in opposition to concrete concepts, therefore constraining the scope of the debate because the dichotomy between abstract and concrete concepts is not clear-cut [6]. For a long time, the most commonly used dimension was imageability, with concrete concepts referring to highly imageable entities that can be perceived through the senses and abstract concepts having low imageability. This dimension is the direct consequence of Paivio’s Dual Coding Theory (DCT, [7]) according to which concrete concepts trigger processing based on two informational systems, one visual, the other verbal, whereas abstract concepts are processed only in the verbal system. More specifically, imageability is the ability of “words [to] arouse sensory experience” (Paivio et al., 1968, p.4; see also [8]). In that sense, it is implicitly equated with abstractness, so much so, in fact, that abstract concepts stimuli were often chosen among words with low imageability ratings.

A closely related dimension used to distinguish abstract concepts comes from the Context Availability Theory (CAT, [9]) according to which concrete concepts refer to a set number of contexts whereas abstract concepts are loosely connected to a variety of contexts.
The argument here is that the availability of a strong network of highly defined features provides concrete concepts with a processing advantage over abstract concepts. This distinction perpetuates the inaccurate view that abstract concepts are poor in features. It has been somewhat invalidated, however, by recent evidence showing that although abstractness is highly correlated with imageability, the two dimensions are not equivalent. Kousta, Vigliocco, Vinson, Andrews, and Del Campo (2011) found a processing advantage in favour of abstract concepts once the imageability and context variables were controlled, a finding which intensified the perceived need to uncover more dimensions to define abstract concepts [10]. To that end, Villani et al., (2019) introduced a novel database of 15 dimensions clustered around sensorimotor, linguistic, inner, social states and hand and mouth effectors [5]. The database offers researchers the ability to select abstract stimuli along more defined dimensions and allows for better-controlled experiments.

The study of abstract concepts is benefiting from the greater emphasis placed on their variety, dimensions and richness. Not only has this intensified interest been concerned with defining their dimensional content, it has also produced theoretical accounts to explain how they are processed. Several theories on a spectrum ranging from amodal to grounded assumptions have been formulated to answer questions about processing mechanisms for abstract concepts.

**Theoretical accounts in the study of abstract concepts**

At the amodal end of the spectrum, traditional accounts of semantics use symbolic representations to define the meaning of concepts [11–14]. In recent years, this account has been integrated in models of distributional semantics able to extract meaning from statistical distribution across corpora (see [15] for a review of these models). Given that word co-occurrences can be computed for concrete and abstract concepts alike, this account finds no
issue with explaining the representation of abstract concepts. However, use of amodal
linguistic symbols as a proxy for representing meaning has been under fire for its lack of
grounding in modality-specific brain areas [16–20].

At the grounded end of the spectrum, the meaning of concepts is derived from
perceptual and motor states [21–23]. For instance, Pulvermüller, Shtyrov, and Ilmoniemi
(2005) used MEG techniques to show that brain areas responsible for motor actions of the
face or leg are activated when action words such as *lick* or *kick* are processed [24]. Such
evidence is challenged, however, when faced with abstract concepts such as *truth* or *freedom*,
as they do not refer directly to physical features of the world. That explains why the embodied
account had to be extended to include hypotheses for the grounding of abstract concepts.

Several studies have investigated the representation of abstract concepts through
feature-listing paradigms and have demonstrated that abstract concepts also benefit from some
form of representation in modality-specific. For example, abstract concepts activate social and
 introspective aspects of situations [25,26], emotional features [27], event information and
thematic roles [28]. More generally, these studies unveiled representation mechanisms for
abstract concepts that place greater emphasis on the context in which they are used (e.g., [29–
31]). For instance, Wilson-Mendenhall, Martin, Simmons and Barsalou (2013) performed a
task requiring deep conceptual processing of abstract concepts such as *convince [32]*. They
observed neural activation patterns of non-linguistic brain areas associated with mentalizing
and social cognition. In a more recent study, Harpainter et al. (2020) studied brain activation
patterns for visual and motor abstract concepts such as *beauty* and *fight [33]*. In line with the
grounding of representation framework, their results suggest some categories of abstract
concepts benefit from a similar grounding mechanism to concrete concepts. Middle-ground
theories, however, consider that both amodal and grounded content contribute to the
representation of abstract concepts. For instance, Dove's representation pluralism ([34–36] see also [2] and [37] for a review) represents just such a hybrid approach, according to which abstract concepts activate both linguistic and modality-specific features.

**Picture priming paradigms in the study of abstract concepts**

The representation of abstract concepts in situations and contexts has been further investigated in recent years [24, 28, 29, 31]. More recently, McRae, Nedjadrasul, Pau, Pui-Hei Lo and King (2018) presented abstract target words primed by pictures depicting scenes either related or unrelated to the target words [38]. For example, the concept *discipline* was primed by the picture of students lined up. When target words were preceded by related pictures, lexical decision latencies were shorter than when unrelated pictures were shown. Such results support the assumption of extraction of situational features in abstract concept processing.

In another recent study, Kuipers, Jones and Thierry (2018) used a similar priming paradigm, but with abstract picture primes, which lack tangible situational features [39]. The results showed shorter manual latencies and a smaller N400 amplitude in the EEG data for related abstract picture-word pairs than unrelated ones. The authors inferred that the abstract pictures conveyed the same meaning as the abstract words. The fact that abstract images devoid of situational context elicited an activation pattern similar to the abstract concepts they primed suggests the need for further investigation of the mechanisms involved in abstract concept processing.

**The present study**

After norming the abstract picture-word pairs from Kuipers et al. (2018) with French participants, we replicated their study in French in a first experiment to make sure their findings remained valid in a cross-linguistic setting. In a second experiment, we expanded on
their study by comparing abstract picture primes to concrete ones within the same experiment [39]. For the first experiment, we expected to find the same facilitatory effect as in English, with shorter manual latencies for related abstract picture-word pairs than for unrelated ones.

In a second experiment, we compared concrete picture primes allowing for situational grounding to abstract picture primes composed of intangible features. For this second experiment, in which concrete and abstract picture primes were presented together we also expected shorter latencies for related picture-word pairs than for unrelated ones in the concrete picture condition. However, and in line with the previously discussed role of situational features in the representation of abstract concepts, we expected this facilitatory effect to be limited to concrete picture primes, insofar as they allow for a more tangible form of situational or contextual information, and to disappear in respect of abstract picture primes (see [38]). Tangible features, as opposed to intangible ones, should allow for easier processing [40]. We expected that when presented with both types of features the participants would develop a strategy by allocating more resources to extracting tangible features, to the detriment of intangible ones.

# Experiment 1

## MATERIALS and METHODS

### Participants

The same number of participants were tested in Experiment 1 as were tested by Kuipers et al. (2018) [39]. Twenty native French speakers from Université Clermont Auvergne, France, took part in this experiment (4 males; M<sub>age</sub> = 24, SD = 3). All participants were right-handed with corrected-to-normal vision. They all gave their informed written consent before taking part in the study. They were rewarded for their participation with a 10€
gift card. The study was approved by the local ethics committee (Comité d'éthique de la
Recherche IRB-UCA).

Materials

The material used in Experiment 1 was crafted on the basis of the norming study described hereafter. For Experiment 1, the purpose was to create a set of semantically related and unrelated picture-word combinations based on the stimuli used in [39]. We obtained the abstract images and words collected for their study by contacting the authors. The materials consisted of 100 abstract images, each paired with one related and one unrelated abstract word. We translated the words into French and submitted the picture-word combinations to a group of 166 French participants (56 men, M<sub>age</sub> = 21.7, SD = 4.6) using Qualtrics (2020) in a preliminary study. As in Kuipers et al. (2018), participants were asked, “How strongly does the word below match the above picture?” (on a scale of 0 to 10) [39]. We kept only the 71 picture-word combinations that elicited the highest scores for the related conditions, and those that elicited the lowest scores for the unrelated conditions (M<sub>Related</sub> = 6.5, SD = 1.0; M<sub>Non_Related</sub> = 2.2, SD = 1.1; t(70) = 21.6, p < 0.001, d = 2.6, 95CI [2.1-3.0], see Figure 1 for an example of combination).

Procedure

Experiment 1 used exactly the same procedure as Kuipers et al. (2018), except that we recorded only reaction times, and not EEG. We presented abstract image-word combinations that were either semantically related or unrelated (see Figure 1 for example stimuli) [39]. Both the related and unrelated modalities were presented in a within-subjects design. Stimuli were presented using E-prime software v2.1 on a 15-inch Dell PC colour monitor with a refresh rate of 60 ms and connected to an AZERTY keyboard. Participants were seated at a distance
of 50 cm from the monitor. Each trial began with a fixation cross, displayed in the middle of the screen for 300ms followed by an abstract picture that remained on the screen for 5000ms. The fixation cross was then replaced with an abstract target word, which stayed on the screen until the participant answered, or until 5000ms had elapsed. The participant's task was to indicate, using the D and K keys on the AZERTY keyboard, whether the target word was semantically related or unrelated to the previous picture. Response latencies were recorded, as well as accuracy. The experiment took approximately 20 minutes to complete.

# RESULTS

## Reaction times

Latencies > 3 SDs above or below each participant's mean latencies for each condition were excluded from the analyses (less than 2% of the data). Mean correct latencies were analysed by means of a one-sided paired-samples t-test testing the same directional hypothesis as Kuipers et al., (2018). We conducted analyses within a Bayesian framework relying on prior specification of theoretical knowledge which includes effect sizes and the direction of hypotheses [33], see also [41]. The analysis revealed an effect of Target Word Type with shorter latencies for the related pairs than the unrelated ones ($M_{Related} = 1411$ ms; SD = 429; $M_{Non_Related} = 1529$ ms; SD = 464; Mean Difference = 118 ms; 95CI [21, 214]; $t(19) = 2.55$, $p = 0.01$, Cohen's d = 0.57, see Figure 2). We conducted a Bayesian paired-samples t-test using JASP [42]. For the informed prior we used Oosterwijk's recommendation (t-distribution with location 0.350, scale 0.102, and 3 degrees of freedom), because it is considered a good prior for small-to-medium effect sizes (see [43]). This analysis showed a Bayes Factor of 10.93 (median = 0.39; 95CI [0.19-0.67]). This means the results are about 11 times more likely under the alternative hypothesis compared to the null. Taken together, the Bayesian analyses
show substantial to strong support for the finding that abstract images can elicit a semantic priming effect [44].

Yet, this effect is somewhat surprising and rather counter-intuitive with respect to the role of situational grounding in concept representation. We therefore conducted supplementary analyses in a Bayesian replication framework to test for the robustness of such an effect. To that end, we applied a Replication Test in Rstudio [45] (ReplicationBF package [46]) which can be used to assess the success or failure of a replication attempt by testing whether the effect identified in previous studies is present or absent in the replication attempt [46], whether the effect size in the replication is equal to that found in the original study [47], and whether the effect is present when the data are pooled in a meta-analysis [48]. The replication test relies on sample sizes and t-values and requires no specification of a prior distribution, thus escaping the influence of a researcher's choice of priors, but it uses the posterior distribution of the original study as a prior for the replication attempt ([49,50] but see [51]). The results of the replication tests are reported in the supplementary analyses. For the equality-of-effect size Bayes factor, support for the null hypothesis, according to which there is no difference between effect sizes, is indicative of a successful replication. The tests provided substantial support for the equality of effect sizes (Equality B01 ≈ 5; see Table A.1), with the fixed-effect meta-analysis Bayes factor test showing strong support for the presence of an overall effect (Meta B10 ≈ 30). Finally, the new Bayes factor for replication shown in Figure A.1 indicates that the results are about 12.5 times more likely under the proponent's hypothesis that the effect is consistent with the one found in the original study as opposed to the sceptic's hypothesis that the effect is spurious [46].
## Errors

A paired-samples t-test showed no effect of Target Word Types for the errors ($t(19) = 0.14; p = 0.90 ns$).

## DISCUSSION

The aim of Experiment 1 was first to extend the replicability of Kuipers et al. (2018) to another language before considering expanding it in a follow-up experiment [39]. As in the study by Kuipers and colleagues, we obtained shorter latencies for related combinations than for unrelated ones. They interpreted their results in terms of meaning processing and concluded that it was possible that abstract pictures can convey the same meaning as lexical abstract concepts. Given the counter-intuitive results obtained by Kuipers et al. (2018), especially given the important role of situational information, we decided to conduct Bayesian analyses [46], the results of which provided strong evidence in favour of Kuipers and colleagues' alternative hypothesis, according to which abstract pictures can convey the same meaning as abstract concepts. Furthermore, we conducted a supplementary replication test to ensure our attempt at replication was successful. We demonstrated therefore that the effect of picture-word priming using abstract pictures could be replicated in a cross-linguistic setting using different abstract images. Despite our analyses confirming the evidence uncovered by Kuipers and colleagues we do not rule out other possible interpretations (see General Discussion). Having confirmed the replicability of a study using such stimuli, we are now keen to expand Experiment 1 in a paradigm that would allow abstract concepts to be extracted from situational pictures. Experiment 2 aimed to implement such a paradigm. To that end, we kept the abstract priming pictures from Experiment 1 and added a condition for which participants were presented with concrete priming pictures that provided situational information.
# Experiment 2

# MATERIALS and METHODS

## Participants

An independent group of 90 students from Université Clermont Auvergne who met the same criteria as for Experiment 1 took part in Experiment 2 (18 males; $M_{age} = 20.2$; SD = 2.86). They all gave their informed written consent before taking part in the study. The study was approved by the local ethics committee (Comité d'éthique de la Recherche IRB-UCA).

## Materials

The materials used for Experiment 2 were the result of a second norming study. The purpose of the norming study for Experiment 2 was to find corresponding concrete images for each picture-word combination in Experiment 1. To that end, we searched online databases for images likely to elicit the same meaning as the abstract words. For each word, we selected two potential images which we then submitted to an independent group of 146 French participants (38 men, $M_{age} = 22.4$, SD = 5.6) using Qualtrics. Participants were asked, "How strongly does the word below match the above picture?" (on a scale of 0 to 10). Picture-word combinations that obtained a low score (< 5 on a scale of 0 to 10) for both image options were discarded. The remaining 56 stimuli combinations (abstract/concrete image-related/unrelated word) were used in Experiment 2 (see Figure 1 for an example of stimuli).

## Procedure

We used a similar procedure as for Experiment 1, but expanded it with 2 within-subjects variables: Priming Image Type (abstract vs. concrete) and Target Word Type (related
vs. unrelated). The stimuli were divided in a Latin square with 4 modalities: abstract prime images, concrete prime images, semantically-related target words and unrelated target words.

Following this Latin-square design, the pictures were divided into two lists so that each picture presented in the related condition to participant 1 was also presented to participant 2 in the unrelated condition. Participants were exposed to all variables but saw only one of the 4 possible types of picture-word combination for each stimulus (see Figure 1 for an illustration of the trial procedure with an example of stimuli).

<figure><img src="image_19.png" /><figcaption>**Figure 1.** Trial procedure for the concept royalty ("royauté" in French).</figcaption></figure>

Note. For this concept, each participant was exposed to either the abstract or the concrete image prime followed by the related or unrelated target word. The trial procedure for Experiment 1 consisted of only the abstract images paired with related or unrelated words, whereas Experiment 2 also had
concrete images in addition to the abstract pictures. The abstract image stimulus is a fractal created by Sven Geier (http://www.sgeier.net/fractals/fractals/07/The%20Road%20Ahead.jpg) and the concrete image stimulus is a coronation picture credited to the National Film Board of Canada. Photothèque. Library and Archives Canada, PA-196667; CC BY 2.0; https://tinyurl.com/pxfvv8p7).

# RESULTS

## Reaction times

Latencies > 3 SDs above or below each participant's mean latencies for each condition were excluded from the analyses (i.e. ~2% of the total data). Data from 3 participants were discarded based on z-scores that showed very slow RTs in all conditions (z-scores > 2.99). The following analyses are based on the data from 87 participants.

Mean correct latencies are presented in Figure 2. Mean correct latencies were analysed with a 2(Priming Image Type = Concrete vs. Abstract) * 2(Target Word Type = Related vs. Unrelated) repeated measures ANOVA.

This analysis revealed a main effect of Priming Image Type with shorter latencies for concrete images compared to abstract images (respectively, $M_{Concrete_Image} = 1183$ ms; SD = 357; $M_{Abstract_Image} = 1338$ ms; SD = 383; Mean Difference = 116 ms; 95CI [80, 152]; $F(1, 86) = 40.65, p < 0.001, \eta_p^2 = 0.32$). The results also showed a main effect of Target Word Type with shorter latencies for the related targets compared to unrelated targets ($M_{Related} = 1144$ ms; SD = 346; $M_{Non_Related} = 1300$ ms; SD = 372; Mean Difference = 193 ms; CI [145, 241]; $F(1, 86) = 63.74, p < 0.001, \eta_p^2 = 0.43$). The results showed an interaction effect between Priming Image Type and the Target Word Type ($F(1, 86) = 70.48, p < 0.001, \eta_p^2 = 0.45$). Contrast analyses were conducted to investigate this interaction further. There was a significant effect of word type within each level of image type, with shorter latencies for the related targets compared to unrelated targets for the abstract images ($M_{Abstract_Related} = 1265$; SE = 40; 95CI
[1185, 1344]; M<sub>Abstract_NonRelated</sub> = 1334; SE = 40; CI [1255, 1414]; t(86) = 2.33; p = 0.011; d = 0.3; CI [0.04, 0.5]) and concrete images (M<sub>Concrete_Related</sub> = 1025; SE = 34; CI[957, 1093]; M<sub>Concrete_NonRelated</sub> = 1341; SE = 42; CI [1258, 1426]; t(86) = 12.00; p < 0.001; d = 1.3; CI [0.99, 1.56], see Figure 2). The semantic facilitatory effect was therefore present for both types of prime stimuli.

<figure><img src="image_20.png" /><figcaption>**Figure 2.** Mean correct latencies (in ms with standard error bars) for Type of Image Prime and Type of Target Word (R = related pairs; NR = non-related pairs; **p** = 0.01; ***p** < 0.001) in both Experiments 1 and 2.</figcaption></figure>

A Bayesian repeated measures ANOVA compared four models to the null model. We kept the default JAPS prior for fixed effects (r scale prior width = 0.5; [47,48]). Based on Bayes Factors, the model including only the types of priming images was 5770 times more likely than the null model. There is therefore strong evidence for the type of prime model. There was also strong evidence for the model including only the type of word target (BF = 1.902x10<sup>13</sup>), for the model including both main effects (BF = 2.990x10<sup>18</sup>), and for the interaction model (BF = 2.551x10<sup>25</sup>). Finally, we compared the main effects model to the
interaction model (2.551×10²⁵/ 2.990×10¹⁸ = 8.533 x10⁶; based on the Baws Factor suggested by Mathôt, 2017 [52]) and obtained strong evidence for the interaction model.

We used a Bayesian paired-samples t-test to explore this interaction, with an informed prior following Oosterwijk's recommendation. The analysis showed a Bayes Factor of 7.67 (median = 0.30; CI [0.13, 0.45]) in favour of the alternative hypothesis for the difference between related and unrelated target words in the abstract picture primes condition, and a Bayes Factor of 4.42×10¹⁵ (median = 1.19; CI [0.89, 1.48]) in favour of the alternative hypothesis for the difference between related and unrelated target words in the concrete picture primes. These Bayesian analyses concur with the ones obtained from traditional paired-samples t-test and confirm substantial evidence for the effect of the abstract picture primes and extreme evidence for the concrete ones.

## Errors

Given the difficulty of the task, especially concerning the abstract image condition, we expected larger rates of errors overall, but more specifically for the abstract image primes.

Indeed, a repeated measures ANOVA revealed a main effect of Priming Image Type with a higher percentage of errors for the abstract images compared to the concrete images

(M<sub>Abstract_Image</sub> = 29%; SD = 16.5; M<sub>Concrete_Image</sub> = 12%; SD = 10; F(1,87) = 189.87, p < 0.001, η<sub>p</sub><sup>2</sup> = 0.69). As for Experiment 1, the results showed no main effect of Target Word Type.

## DISCUSSION

Experiment 2 expanded on Experiment 1 by comparing abstract to concrete pictures priming abstract lexical concepts. We opted for a semantic, as opposed to a lexical decision task given that Recchia and Jones (2012) suggested the latter did not yield deep semantic
processing and therefore accounted for the discrepancies between results in the abstract concept processing literature [53]. Recognising this might be a concern, however, we checked for such surface features based on analyses of emotional valence in an additional study. We presented the pictures to an independent pool of 49 participants online and asked them to rate the emotional valence of the pictures on a scale of 0-10. We then used this valence variable in a *post-hoc* analysis and found it failed to explain the variance in latencies for semantic priming (based on a repeated measures ANOVA (F(2, 48) = 1.76; p = .19 ns)). Based on this analysis we can rule out a shallow picture-word association based only on surface physical features.

This facilitatory effect was significantly stronger for concrete pictures than for abstract ones. However, it was weaker in relation to Experiment 1, although it did not disappear for abstract pictures.

This result expands on those obtained by Kuipers et al. (2018), showing that abstract concepts can be processed on the basis of abstract and concrete pictures alike and more broadly from tangible and intangible features. The Bayesian models provide decisive evidence in favour of the alternative hypothesis of semantic priming. The error rates obtained were quite high compared to traditional effects in the semantic priming literature. We were not surprised by these rates, however, as the task was more difficult in comparison to traditional priming studies. We asked Kuipers et al. (2018) for the error rates for their study (27% error rates for the related and 31% for the unrelated items), which confirmed our intuition that such stimuli would elicit higher error rates compared to more traditional semantic priming studies.
GENERAL DISCUSSION

The aim of the present studies was to investigate conceptual processing mechanisms
based on picture-word combinations in terms of the role of situational features compared to
intangible abstract features.

In the first experiment and in line with Kuipers and colleagues, we investigated conceptual processing mechanisms in relation to abstract pictures. The results showed participants were able to process concepts from abstract pictures devoid of tangible features (therefore replicating Kuipers et al., 2018 results and extending them to the French language). To the best of our knowledge, Kuipers et al. (2018) were the first to provide evidence that abstract pictures can activate the meaning of abstract concepts. The data of Experiment 1 which was analysed using a Bayesian replication framework, corroborated their findings. This fact alone broadens the scope of conceptual representation beyond the usual debate of linguistic vs. situational features (see for e.g., [14,25,34]). It also goes to show that abstract concepts are even richer and activate features beyond their linguistic or situational components [53].

In a second experiment, we gave participants an opportunity to extract situational
features (see also [38]). Results showed a stronger facilitatory effect for concrete picture-
words pairs compared to abstract ones. Participants relied more heavily on the priming
pictures with distinguishable features. Our results support previous findings according to
which some types of abstract concepts are grounded in situations and events [25,32,40].

From the evidence from the two experiments, it appears that it is possible to represent
abstract concepts based on abstract and intangible pictures as well as on concrete and tangible
ones. McRae et al. (2018) showed that in a picture-priming paradigm similar to our “concrete
images” condition, participants could process the meaning of abstract concepts derived from
pictures depicting scenes. Kuipers et al. (2018) showed how this processing could also be
derived from abstract pictures that differ from the stimuli used by McRae and colleagues
because they are devoid of tangible features. In the present study, and based on the
assumption that it is costlier to extract intangible features when tangible ones are available,
we showed that extraction mechanisms still occurred in the case of abstract pictures.

The immediate interpretation of these results refers to how Kuipers and colleagues and McRae and colleagues interpreted meaning processing based on tangible and intangible features. This initial interpretation infers that there is a semantic level of processing and implies a featural view of semantic representation according to which concepts can be broken up into a set of defining features that represent their meaning (see [54] for an account of the featural view). If confirmed, this assumption would mean such features can be other than lexical or situational, namely also abstract and intangible.

For instance, Bolognesi and Vernillo (2019) proposed ‘Abstraction by Metonymy’ as a
novel grounding mechanism for abstract concepts in the pictorial mode, where they use
verbo-pictorial metaphors to investigate people’s ability to illustrate abstract concepts [55].
According to their hypothesis, this abstraction process allows for an inferential mechanism
that moves from concretely depicted entities to more abstract ones. In the present studies, the
reverse mechanism seems to have occurred. It might be that in the abstract image condition,
participants extracted intangible features and inferred a more tangible representation of the
concept. Therefore, these results could be construed as complementing the Abstraction by
Metonymy.

In conclusion, by directly comparing the two types of features for representation, we
were able to show that abstract concepts can be processed based on situational features and
abstract ones. For us, these conclusions signal the need to explore further the mechanisms of
concept representation and abstraction.

Acknowledgments

This research conducted within the FACTOLAB framework has been sponsored by: Michelin
tyre manufacturer, the French government research programs "Investissements d'Avenir," the
IDEX-ISITE initiative 16-IDEX-0001 (CAP 20-25), and the IMobS3 Laboratory of
Excellence (ANR-10-LABX-16-01).

This work received technical support from the CNRS and UCA of Clermont-Ferrand. The
authors would like to thank Gillian Wakenhut for reviewing and editing the language of the
manuscript. The authors would also like to thank Jan R. Kuipers for giving us access to his
data and stimuli.

REFERENCES

1. Lakhzoum D, Izaute M, Ferrand L. Semantic similarity and associated abstractness norms for 630 French word pairs. doi:10.3758/s13428-020-01488-z

2. Borghi AM, Binkofski F, Castelfranchi C, Cimatti F, Scorolli C, Tummolini L. The challenge of abstract concepts. Psychol Bull. 2017;143: 263–292. doi:10.1037/bul0000089

3. Barsalou LW. Abstraction in perceptual symbol systems. Philosophical Transactions of the Royal Society B: Biological Sciences. Royal Society; 2003. pp. 1177–1187. doi:10.1098/rstb.2003.1319

4. Pulvermüller F. Neural reuse of action perception circuits for language, concepts and communication. Progress in Neurobiology. Elsevier Ltd; 2018. pp. 1–44. doi:10.1016/j.pneurobio.2017.07.001

5. Villani C, Lugli L, Liuzza MT, Borghi AM. Varieties of abstract concepts and their multiple dimensions. Lang Cogn. 2019;11: 403–430. doi:10.1017/langcog.2019.23

6. Della Rosa PA, Catricalà E, Vigliocco G, Cappa SF. Beyond the abstract-concrete dichotomy: Mode of acquisition, concreteness, imageability, familiarity, age of acquisition, context availability, and abstractness norms for a set of 417 Italian words. Behav Res Methods. 2010;42: 1042–1048. doi:10.3758/BRM.42.4.1042

7. Paivio A, Yuille JC, Madigan SA. Concreteness, imagery, and meaningfulness values for 925 nouns. J Exp Psychol. 1968;76: 1–25. doi:10.1037/h0025327

8. Dellantonio S, Job R, Mulatti C. Imageability: Now you see it again (albeit in a different form). Frontiers in Psychology. Frontiers Media SA; 2014. p. 991. doi:10.3389/fpsyg.2014.00279

9. Schwanenflugel PJ, Harnishfeger KK, Stowe RW. Context availability and lexical
decisions for abstract and concrete words. J Mem Lang. 1988;27: 499–520.
doi:10.1016/0749-596X(88)90022-8

10. Kousta ST, Vigliocco G, Vinson DP, Andrews M, Del Campo E. The Representation of Abstract Words: Why Emotion Matters. J Exp Psychol Gen. 2011;140: 14–34.
doi:10.1037/a0021446

11. Collins AM, Loftus EF. A spreading-activation theory of semantic processing. Psychol Rev. 1975;82: 407–428. doi:10.1037/0033-295X.82.6.407

12. Pylyshyn ZW. Cognitive representation and the process-architecture distinction: Cognitive representation and the process-architecture distinction. Behavioral and Brain Sciences. Cambridge University Press; 1980. pp. 154–169.
doi:10.1017/S0140525X00002302

13. Fodor JA, M.I.T. Press. The mind doesn't work that way : the scope and limits of computational psychology. MIT Press; 2001.

14. Mahon BZ, Caramazza A. Concepts and Categories: A Cognitive Neuropsychological Perspective. Annu Rev Psychol. 2009;60: 27–51.
doi:10.1146/annurev.psych.60.110707.163532

15. Andrews M, Frank S, Vigliocco G. Reconciling Embodied and Distributional Accounts of Meaning in Language. Top Cogn Sci. 2014;6: 359–370. doi:10.1111/tops.12096

16. Gallese V, Lakoff G. The brain's concepts: The role of the sensory-motor system in conceptual knowledge. Cognitive Neuropsychology. 2005. pp. 455–479.
doi:10.1080/02643290442000310

17. Pulvermüller F. Brain mechanisms linking language and action. Nature Reviews Neuroscience. Nature Publishing Group; 2005. pp. 576–582. doi:10.1038/nrn1706

18. Barsalou LW. Grounded Cognition. Annu Rev Psychol. 2008;59: 617–645.
doi:10.1146/annurev.psych.59.103006.093639

19. Borghi AM, Cimatti F. Embodied cognition and beyond: Acting and sensing the body. Neuropsychologia. 2010;48: 763–773. doi:10.1016/j.neuropsychologia.2009.10.029

20. Meteyard L, Cuadrado SR, Bahrami B, Vigliocco G. Coming of age: A review of embodiment and the neuroscience of semantics. Cortex. 2012;48: 788–804.
doi:10.1016/j.cortex.2010.11.002

21. Barsalou LW. Perceptual symbol systems. Behavioral and Brain Sciences. 1999. pp.
577–609. doi:10.1017/S0140525X99002149

22. Glenberg AM. Mental models, space, and embodied cognition. Creative thought: An investigation of conceptual structures and processes. American Psychological Association; 2004. pp. 495–522. doi:10.1037/10227-018

23. Zwaan RA. The Immersed Experiencer: Toward An Embodied Theory Of Language Comprehension. Psychol Learn Motiv - Adv Res Theory. 2004;44: 35–62.
doi:10.1016/S0079-7421(03)44002-4

24. Pulvermüller F, Shtyrov Y, Ilmoniemi R. Brain signatures of meaning access in action word recognition. J Cogn Neurosci. 2005;17: 884–892.
doi:10.1162/0898929054021111

25. Barsalou LW, Wiemer-Hastings K. Situating abstract concepts. Grounding Cognition: The Role of Perception and Action in Memory, Language, and Thinking. Cambridge University Press; 2005. pp. 129–163. doi:10.1017/CBO9780511499968.007

26. Borghi AM, Barca L, Binkofski F, Castelfranchi C, Pezzulo G, Tummolini L. Words as social tools: Language, sociality and inner grounding in abstract concepts. Phys Life Rev. 2018;29: 120–153. doi:10.1016/j.plrev.2018.12.001

27. Kousta ST, Vigliocco G, Vinson DP, Andrews M, Del Campo E. The Representation of Abstract Words: Why Emotion Matters. J Exp Psychol Gen. 2011;140: 14–34.
doi:10.1037/a0021446
28. Ferretti TR, McRae K, Hatherell A. Integrating Verbs, Situation Schemas, and Thematic Role Concepts. J Mem Lang. 2001;44: 516-547. doi:10.1006/jmla.2000.2728

29. Barsalou LW. Perceptions of perceptual symbols. Behavioral and Brain Sciences. 1999. pp. 637-660. doi:10.1017/S0140525X99532147

30. Barsalou LW. Situated simulation in the human conceptual system. Language and Cognitive Processes. Psychology Press Ltd; 2003. pp. 513-562. doi:10.1080/01690960344000026

31. Wiemer-Hastings K, Xu X. Content differences for abstract and concrete concepts. Cogn Sci. 2005;29: 719-736. doi:10.1207/s15516709cog0000_33

32. Wilson-Mendenhall CD, Simmons WK, Martin A, Barsalou LW. Contextual processing of abstract concepts reveals neural representations of nonlinguistic semantic content. J Cogn Neurosci. 2013;25: 920-935. doi:10.1162/jocn_a_00361

33. Harpaintner M, Sim EJ, Trumpp NM, Ulrich M, Kiefer M. The grounding of abstract concepts in the motor and visual system: An fMRI study. Cortex. 2020;124: 1-22. doi:10.1016/j.cortex.2019.10.014

34. Dove G. Beyond perceptual symbols: A call for representational pluralism. Cognition. 2009;110: 412-431. doi:10.1016/j.cognition.2008.11.016

35. Dove G. On the need for Embodied and Dis-Embodied Cognition. Front Psychol. 2011;1: 242. doi:10.3389/fpsyg.2010.00242

36. Dove G. Thinking in Words: Language as an Embodied Medium of Thought. Top Cogn Sci. 2014;6: 371-389. doi:10.1111/tops.12102

37. Pecher D. Curb Your Embodiment. Top Cogn Sci. 2018;10: 501-517. doi:10.1111/tops.12311

38. McRae K, Nedjadrasul D, Pau R, Lo BPH, King L. Abstract Concepts and Pictures of Real-World Situations Activate One Another. Top Cogn Sci. 2018;10: 518-532. doi:10.1111/tops.12328

39. Kuipers JR, Jones MW, Thierry G. Abstract images and words can convey the same meaning. Sci Rep. 2018;8: 1-6. doi:10.1038/s41598-018-25441-5

40. McRae K, Nedjadrasul D, Pau R, Lo BP-H, King L. Abstract Concepts and Pictures of Real-World Situations Activate One Another. Top Cogn Sci. 2018 [cited 19 Mar 2018]. doi:10.1111/tops.12328

41. Lee MD, Vanpaemel W. Determining informative priors for cognitive models. Psychon Bull Rev. 2018;25: 114-127. doi:10.3758/s13423-017-1238-3

42. JASP - A Fresh Way to Do Statistics. [cited 20 Oct 2020]. Available: https://jasp-stats.org/

43. Gronau QF, Ly A, Wagenmakers E-J. Informed Bayesian t-Tests. Am Stat. 2019; 1-14. doi:10.1080/00031305.2018.1562983

44. Ly A, Verhagen J, Wagenmakers EJ. Harold Jeffreys's default Bayes factor hypothesis tests: Explanation, extension, and application in psychology. J Math Psychol. 2016;72: 19-32. doi:10.1016/j.jmp.2015.06.004

45. RStudio | Open source & professional software for data science teams - RStudio. [cited 20 Oct 2020]. Available: https://rstudio.com/

46. Verhagen J, Wagenmakers E-J. Bayesian Tests to Quantify the Result of a Replication Attempt. 2014 [cited 10 Mar 2020]. doi:10.1037/a0036731

47. Bayarri MJ, Mayoral AM. Bayesian Design of “Successful” Replications. The American Statistician. Taylor & Francis, Ltd. American Statistical Association; pp. 207-214. doi:10.2307/3087300

48. Rouder JN, Morey RD. A Bayes factor meta-analysis of Bem's ESP claim. Psychon Bull Rev. 2011;18: 682-689. doi:10.3758/s13423-011-0088-7

49. Kruschke JK. Bayesian data analysis. Wiley Interdiscip Rev Cogn Sci. 2010;1: 658-
676. doi:10.1002/wcs.72

50. Liu CC, Aitkin M. Bayes factors: Prior sensitivity and model generalizability. J Math Psychol. 2008;52: 362–375. doi:10.1016/j.jmp.2008.03.002

51. Vanpaemel W. Prior sensitivity in theory testing: An apologia for the Bayes factor. J Math Psychol. 2010;54: 491–498. doi:10.1016/j.jmp.2010.07.003

52. Mathôt S. Mathôt S. (2017). Bayes Like a Baws: Interpreting Bayesian Repeated Measures in JASP. [Google Scholar] - Recherche Google. [cited 20 Oct 2020].
Available:
https://www.google.com/search?q=Mathôt+S.+(2017).+Bayes+Like+a+Baws%3A+Int
erpreting+Bayesian+Repeated+Measures+in+JASP.+%5BGoogle+Scholar%5D&ie=ut
f-8&oe=utf-8&aq=t

53. Recchia G, Jones MN. The semantic richness of abstract concepts. Front Hum Neurosci. 2012;6. doi:10.3389/fnhum.2012.00315

54. Smith EE, Shoben EJ, Rips LJ. Structure and process in semantic memory: A featural model for semantic decisions. Psychol Rev. 1974;81: 214–241. doi:10.1037/h0036351

55. Bolognesi M, Vernillo P. How abstract concepts emerge from metaphorical images: The metonymic way. Lang Commun. 2019;69: 26–41.
doi:10.1016/j.langcom.2019.05.003
**Appendices: supplementary analyses**

<table><caption>**Table A.5.** Results of the Bayesian Replication Test.</caption><thead><tr><th></th><th>BF<sub>r0</sub></th><th>Equality B<sub>01</sub></th><th>Meta B<sub>10</sub></th></tr></thead><tbody><tr><th>Original</th><td>N/A</td><td>N/A</td><td></td></tr><tr><th>Replication</th><td>12.5</td><td>5.0</td><td>30.3</td></tr></tbody></table>

Note. “Rep $B_{r0}$” is the new Bayes factor test for replication; “Equality $B_{01}$” is the equality-of-effect-size Bayes factor test; and “Meta $B_{10}$” is the fixed-effect meta-analysis Bayes factor test.

<figure><img src="image_21.png" /><figcaption>**Figure A.1.** Results of the Bayes factor replication test applied to our replication attempt based on Kuipers et al. (2018).</figcaption></figure>

Note. The dotted line represents the posterior from the original study, which was used as the prior for the effect sizes in the replication tests. The solid line represents the posterior distributions once data from the present replication attempt have been taken into account. The grey dots indicate the ordinates of this prior and posterior for the null hypothesis according to which the effect size is zero. The ratio between these two ordinates gives the result of the replication test [46].
The previous chapter introduced a series of experiments which compared the role of
situational information with intangible features in the processing of abstract concepts. Results
have shown that even when compared with situational picture primes, extraction mechanisms
still occurred in the case of abstract pictures. We interpreted these results as further evidence
of the richness of abstract concepts which can be composed of features other than lexical or
situational, namely also abstract and intangible. These intangible features could be construed
as evidence of abstracted representations from varied exemplars of concepts becoming
generalised statistical traces in long-term memory (see for example Barsalou, 2003). In this
context, we draw the distinction between two dimensions of abstraction, one referring to a
form of content, the other to a mechanism. The first dimension is a horizontal abstraction that
refers to the type abstract concepts such as ‘justice’. The abstraction in this case refers to the
featural content of the concept. The second dimension is vertical and refers to a mechanism of
generalisation through abstraction in concept learning tasks for instance. As such, abstraction
refers to the process of extracting features across multiple instances of a concept so that
common features can be generalised to novel instances. The question of concept processing is
closely related to the latter mechanisms of abstraction and generalisation.

In the next chapter, we introduce a database of novel abstract concepts in a series of
experiments to investigate the effect of similarity and contextual diversity on the ability to
generalise the meaning of abstract concepts to all instances. An important question that we
aim to address is what would be the optimal set of training exemplars to learn a concept so
that it generalises to new exemplars. In the case of similarity-based generalisation,
participants are expected to better generalise the meaning of concepts when exposed to similar
exemplars. In contrast, contextual diversity assumes a generalisation of meaning to novel
instances based on exposure to contextually diverse exemplars. Results have shown that
exposure to contextually diverse exemplars enhanced performance in the testing phase

compared to similarity-based exposure.
# Chapter 6. Generalisation mechanisms in the processing of abstract concepts

**Chapter 6 is in preparation as Lakhzoum et al. (in preparation). Generalisation mechanisms in the processing of abstract concepts.**

Supplementary material: https://osf.io/svm2p/

## INTRODUCTION

The question of how concepts are represented is impossible to treat without addressing mechanisms of *abstraction* and *generalisation*. Abstraction refers to the process of extracting features across multiple instances of a concept so that common features are given more weight in the representation of meaning compared to idiosyncratic features (see Murphy, 2002; Glenberg, 2006; Altmann, 2017; Yee, 2019). The resulting abstracted representations can be generalised to all instances of a concept that become aggregated under a common *categorisation* label (Thorndike, 1924; Thorndike & Woodworth, 1901, see also Gardner, 1987; Goldstone, Kersten, Carvalho & 2018). This ability to extract meaningful representations from a few exemplars and generalise to novel situations is the hallmark of human cognition. During the last decade, this feat has been emphasised especially when comparing human performances to those of computational models, which are constantly outperformed due to their failure to implement efficient rules of generalisation (see Cambria, & White, 2014 for a review). Since the early 20th century, this particular cognitive ability has been the object of much work in the concept learning literature focused on abstraction mechanisms. Yet, despite more than a century worth of research, the cognitive principles that give rise to such mechanisms are still poorly understood. In order to elucidate these
principles, two major questions need to be answered: (1) What sort of features are acquired during the learning of concepts through exposure to a set of exemplars? (2) How are these features generalised to novel instances? As Murphy (2013) observed, the literature on concept learning is still dominated by the debate opposing prototype and exemplar models of abstraction as it has been for decades. This debate has been fuelled by many experiments and models that simulate the categorisation of concepts as the result of an abstraction process.

## The prototype and exemplar accounts

According to the prototype theory of Rosch (1975; Rosch & Mervis, 1975; see also Barsalou, 1990), mental representations require a summary description of fundamental properties that form a category. The categorisation of concepts is determined by their similarity with the most prototypical member or ideal example of the category. In a series of priming experiments, Rosch showed a facilitation effect for members of categories that had the most properties in common with other members of the category and the least properties in common with other categories (Posner & Keele, 1968; Hampton, 1979; Rosch & Mervis, 1975; Smith & Medin, 1981; Murphy, 2002; 2016).

Exemplar models are an alternative theory according to which prototypes are not necessary. According to exemplar theories, categories are represented by memories of instances and situations where the concept applies (Medin & Schaffer, 1978). Generalisation based on exemplars offers a better explanation compared to prototypes especially for poorly defined categories (Blair & Homa, 2001; Medin & Schwanenflugel, 1981; Murphy, 2002; 2005). The probability of generalising a category label from encountering a set of examples is a function of the similarity between the examples and each member of the category (Medin & Schaffer, 1978; Nosofsky, 1986). A child that would encounter three instances of a bird and recognise it as such would have formed three “bird” exemplars that can be generalised to all instances and serve for classification (Barsalou, Huttenlocher, & Lamberts, 1998; Nosofsky,
1988). One of the most influential models of the exemplar account is the GCM (generalised categorisation model) of Nosofsky (1986). In the GCM, similarity is modelled according to a multidimensional scaling (MDS; see Kruskal, 1984; Shepard, 1958; 1987) that represent exemplars as points in a multidimensional psychological space. Similarity is a function of the distance between exemplar points: the closer they are, the more similar. In recent years, Vanpaemel and Storms (2008) proposed the VAM (varying abstraction model) as a middle-ground position that combined both the rich amount of stored informational features from the exemplar account and the cognitive economy of the prototype account. The model is rooted in the assumption that prototype and exemplar exist on a continuum where exemplars represent minimal abstraction whereas prototypes represent maximal abstraction.

This notion of a dichotomy between exemplar account and abstraction process has been challenged by recent models and experiments which suggest that both assumptions of storing exemplars and abstraction process are not contradictory. For instance, studies of concept learning in children have shown that they develop more abstract representation as their language grows until they are able to generalise across common features of meaning (Savage, Lieven, Theakston & Tomasello, 2006; Rowland, Chang, Ambridge, Pine & Lieven, 2012). In addition, hybrid models posit that abstracted representations are composed of stored exemplars (Abbot-Smith & Tomasello, 2006; Goldberg, 2006; Langacker, 1988).

Most of the literature on concept learning and abstraction has been dedicated to learning generic concepts such as lines, triangles, faces that are easier to generate, to norm and to diversify along chosen dimensions which provides the researchers with an opportunity to simulate the underlying mechanisms in a computational model such as GCM or VAM for instance. The combination of word learning experiments and concept learning can provide a powerful framework for studying how semantic concepts are abstracted and generalised. Most of the literature points to seemingly opposite assumptions to explain these mechanisms
namely similarity-based generalisation and contextual diversity. One view proposes that generalisation happens when the set of training exemplars of a category are relatively similar; the other view proposes that generalisation happens when the set of training exemplars is relatively diverse.

# The role of similarity and contextual diversity on abstraction and generalisation

## Similarity-based generalisation

Similarity is one of the core tenets of cognitive processes (Medin, Goldstone & Gentner, 1993; Goldstone, 1994; Goldstone & Son, 2012). The seminal work of Gentner (1987) established similarity as a key parameter of a successful generalisation process. According to this account, similarity refers to a mental experience sparked by a current situation similar to a prior experience stored in memory. Gentner, Loewenstein and Hung (2007) tested this similarity principle in children's ability to name parts of objects and found that children showed a facilitated process of structure-mapping when presented with set of similar exemplars. Structure-mapping refers to a process of finding commonalities between conceptual representations based on relational similarities through structural alignment (Gentner, 2003; Gentner & Markman, 2007). Gentner has shown that this alignment can strengthen the ability to generalised extracted common relational features to novel objects and situations (Gentner & Namy, 2016; Gentner, 2010; Gentner, Anggoro, & Klibanoff, 2011; Gentner, Loewenstein, & Hung, 2007; Kotovsky & Gentner, 1996).

The benefit of similarity in generalisation depends on the weight attributed to shared underlying structure at the expense of variations in superficial features (Braithwaite & Goldstone, 2015; Hammer, Bar-Hillel, Hertz, Weinshall, & Hochstein, 2008). Other accounts have shown that a high degree of similarity between stored exemplars can hinder the ability to generalise to new instances of a concept. This can be explained by a failure to recall relevant examples particularly when stored exemplars and the novel instance differ in surface features
(Gick & Holyoak, 1980; Keane, 1988; Holyoak & Koh, 1987; Simon & Hayes, 1976; Weisberg, DiCamillo, & Phillips, 1978; Gentner, Loewenstein & Thompson, 2003; Gentner & Bowdle, 2008; Braithwaite & Goldstone, 2015). Studies have shown that exposure to exemplars presenting a diversity of superficial features can emphasise common underlying structures, thereby promoting generalisation (Gomez, 2002; Onnis, Monaghan, Christiansen, & Chater, 2004; Perry, Samuelson, Malloy and Schiffer, 2010; Rost & McMurray, 2010). In recent years, many studies have reported the importance of contextual diversity on the representation and abstraction of semantic concepts (see Jones, Dye, & Johns, 2017 for a review).

## Contextual diversity and generalisation

The importance of diversity for generalisation stems from the work of Posner and Keele (1968) on the classification of abstract patterns. They have shown that when learning a set of abstract patterns, the ability to classify new instances correctly increases with the variability of memorised patterns. More broadly, the hypothesis of generalisation from stored exemplars to novel instances assumes a presentation of common core features between exemplars while idiosyncratic surface features may vary (Belenky & Schalk, 2014; Day & Goldstone, 2012; Gentner, 1983; Gick & Holyoak, 1983). In this way, stable concept representations rely on a common core of overlapping features while varied superficial surface features become anecdotic information. This assumption also insures the flexibility of conceptual representations as learners become able to overlook such anecdotic features as noise and recognise the core meaning of a concept (see Braithwaite & Goldstone, 2015).

Context generalisation studies have shown that children exposed to multiple exemplars of novel words across varied contexts can better abstract over training examples and generalise meaning to novel test exemplars (Vlach & Sandhofer, 2011; Goldenberg & Sandhofer, 2013; Twomey & Westermann, 2018, Sandhofer & Schonberg, 2020). Smith and
Handy (2014) examined the decontextualization of memories in adult learners and found that acquisition trials presented in varied situations resulted in memory retrieval independently of context. Braithwaite and Goldstone (2015) built on the assumptions that learning abstract concepts through concrete examples could inhibit abilities to generalise to novel instances. In a series of experiments, they showed that varying superficial features of exemplars resulted in better transfer abilities from learned examples to new situations.

Finally, Pagan and Nation (2019) examined the effect of contextual diversity in a word learning experiment where participants had to learn the meaning of low-frequency words through exposure to varied or similar situations. They found that although words presented in diverse contexts were acquired more slowly, participants had more facility answering inferential questions about word meaning compared to similar contexts which they interpreted as better performances of generalisation.

Taken together, the literature centered on similarity and contextual diversity provides inconsistent evidence as regards to the underlying mechanisms involved in abstraction and generalisation in concept learning. Most of the literature is centered on the use of generic concepts such as faces, triangles and dot patterns. Consequently, there is little evidence as regards to the generalisation mechanisms of semantic concepts. This is even truer for abstract concepts such as *fantasy* or *justice*.

## Application to the generalisation of abstract concepts

The variety and complexity of abstract concepts makes them difficult to classify in categories with clear-cut boundaries. This complexity represents even more of a challenge mainly due to the fact that the literature on abstract concepts is rather young and the majority of studies and models have been dedicated to their representation and grounding (see Pecher, 2011; Borghi et al., 2017; Pecher, 2011, 2018 for reviews). Studies have shown that the
preferred mechanism of abstraction depends on the structure of the category. For instance, Minda and Smith (2001) have shown that the prototype account provides a better explanation for large categories. In contrast, Feldman (2003) suggests that the exemplar account provides a better explanation for complex categories. Instead of categories, the literature on abstract concepts representation has established some domains which are number concepts, morals, aesthetics, social and cultural dimensions, as well as emotion that can be grounded in dimensions such as sociality, language, interoception, perception-action, and metacognition (see Barsalou, 2003; Borghi, Barca, Binsofsky & Tummolini, 2018 for a review of these domains and dimensions of abstract concepts). As a result, little is known about how abstract concepts are generalised to all instances of a concept. Given this complexity and lack of clear categorisation for abstract concepts, we suggest that the exemplar end of the exemplar-prototype continuum would provide a better explanation for their mechanisms of generalisation than a prototype view would (see Vanpaemel & Storms, 2008). According to Paivio's Dual Coding Theory (DCT, 1968, 1971), abstract concepts are more loosely related to a multitude of contexts compared to concrete concepts. Taken together the exemplar view and the DCT would suggest that the generalisation of abstract concepts could benefit from exposure to multiple exemplars in diverse contexts.

**The present work**

The combination of word learning experiments and concept learning models provides a powerful framework for studying how semantic concepts can be generalised to all instances. While this framework has been applied successfully to studying the abstraction mechanisms of concrete objects, very little is known about how abstract concepts are abstracted and generalised. The present project was intended as an attempt to bridge this gap in the literature.

The literature on conceptual representation tends to consider the keywords “concept” and “word” as interchangeable on principles (Murphy, 2002). This is not strictly true however
and a distinction does exist between the two notion (Murphy, 2002). A conceptual view based
on a strict one-to-one mapping between words and concepts would mean that for each mental
representation there is one and only one word and that each word of the mental lexicon is
associated with a single mental representation. This view of one-to-one mapping between
concepts and words is oversimplified and impractical as it cannot account for the complexity
of abstract concepts. On the theoretical and empirical levels, this distinction is of little
importance to the ability of researcher to design and conduct experiments. It becomes
important in the context of the present project however, which takes advantage of this lack of
one-to-one mapping between words and concepts to introduce the NewAbstractConcepts
database. This database was motivated by the difficulty to craft experimental stimuli for word
learning experiments in adults. In contrast with children, adults present a vast knowledge of
concepts. Introducing concepts that have no word referent provides some control of
participants' background knowledge compared to previous studies that have used existing
low-frequency words (see for example Pagan & Nation, 2019).

In the present project, we introduce a database of 42 novel abstract concepts which
describe situations that are commonly encountered in an everyday setting but for which no
word label exists. Each abstract concept is composed of 9 exemplars of scenarios (3 sets of
scenarios composed of 3 exemplars each). The database was presented to participants who
rated the similarity between exemplars. On the basis of these similarity judgments, a
psychological space similar to GCM model of Nosofsky (1986, 1992, 2014) was built. This
attempt aimed to establish a proof of concept for the use of semantic concepts composed of
linguistic exemplars in computational models such as GCM that are usually applied to generic
concepts. The database was then used in a word learning experiment for which participants
were exposed to a series of exemplars that were either similar or dissimilar to each other.
# EXPERIMENT 1a: selection of the NewAbstractConcepts database

To create the database, samples were selected from the book *That should be a word* by author Lizzie Skurnick (Skurnick, 2015). The book is a collection of fake words and their definition that correspond to situations and feelings that people can relate to on an everyday basis but that do not have words to label them. For instance, one such word is JOUSKA coined by Skurnick (2015) and which described *A hypothetical conversation that you compulsively play in your head*. The samples were presented to participants in Experiment 1a in order to ensure that people could relate to the concepts.

## METHOD

### Participants

Eighty psychology students took part in this experiment (6 males; $M_{age} = 20.55, SD = 4.99$) presented online using Qualtrics (2020). They all gave their informed consent before taking part in the study. They were awarded course credits for their participation.

### Materials and Procedure

The material consisted of 76 concepts with description selected from the book *That should be a word* (Skurnick, 2015). The fake word labels composed by the author such as *Jouska*, *Sonder* or *Gistake* were not presented to the participants as they had not been normed and could influence the perception of the concepts. Each concept was presented as a description of a situation or feeling (e.g., A hypothetical conversation that you compulsively play in your head; The realisation that each passer-by has a life as vivid and complex as your own; Using incorrect words that still get the point across).

For each concept, participants were asked to indicate in a forced multiple-choice questionnaire whether they had experienced the described situation, or had not experienced but could image the situation, or if they could neither relate to nor imagine the situation. They
were then asked to provide an example where the described concept could apply in a free text entry question to ensure that they understood the concept. Participants were randomly presented with 20 out of the 76 concepts. The concepts were presented one-by-one on the screen. The study was self-paced and could be completed in about 30 minutes.

## RESULTS and DISCUSSION

We aggregated the percentage of response from all the participants for each choice. According to Figure 1, 80% of responses indicated that the concept had been experienced or imagined by the participant. Only 20% of responses indicated that the participant could not relate at all to the concept.

<figure><img src="image_22.png" /><figcaption>**Figure 1.** Percentage of response for each choice.</figcaption></figure>

On the basis of this first analysis, the concepts that were not understood by participants were removed. The concepts for which at least 25% of the responses indicated that the participant could not relate were removed. As a result, 26 concepts out of 76 such as *The art of saying "no" by seeming overly pleased* were removed. In contrast, 7 concepts were judged as relatable or understood by all participants such as *A hypothetical conversation that you compulsively play out in your head* or *The desire to care less about things*. After removing
concepts that were the least understood or relatable, the remaining 50 concepts were used to generate the database.

## EXPERIMENTS 1b and 1c: generating the NewAbstractConcepts database

For each selected concept, 3 different scenarios where the concept could apply were generated. Each scenario was then described by 3 different exemplars of statements that were similar to each other. This process resulted in 9 different exemplars to describe each concept. For instance, the concept SONDER described as *The realization that each passer-by has a life as vivid and complex as your own* was described by 3 scenarios each of them, illustrated exemplars presented in Table 1. For the concept SONDER, the first scenario described a stranger on a train; scenario 2 described a colleague who speaks a different language and finally scenario 3 described a teacher in a private setting.

<table><caption>**Table 1.** Examples of scenarios and exemplars for the concept SONDER.</caption><thead><tr><td>Scenarios</td><td>Exemplars</td><td>Descriptive statements</td></tr></thead><tbody><tr><td>1</td><td>A</td><td>On the train, he looked at the woman in the opposite seat as she opened her laptop and wondered what type of work she might do.</td></tr><tr><td>1</td><td>B</td><td>The man on the train was looking at sheet music and humming to himself and she thought he might be a musician travelling all over the country to give concerts.</td></tr><tr><td>1</td><td>C</td><td>She kept thinking that the woman sitting across the aisle on the train might be a scientist because she was studying papers with complicated looking graphs.</td></tr><tr><td>2</td><td>A</td><td>Listening to her colleague answer the phone in his mother tongue, she suddenly realized that he grew up in a tropical country.</td></tr><tr><td>2</td><td>B</td><td>She heard the office manager talk to his children in a foreign language and started daydreaming about how their upbringing might have differed from her own.</td></tr><tr><td>2</td><td>C</td><td>When he heard the team manager answer her phone in fluent Japanese, he imagined what it was like growing up in the Japanese culture.</td></tr><tr><td>3</td><td>A</td><td>It was strange to see the math teacher in the dairy section but he realized that the teacher must have a family too.</td></tr><tr><td>3</td><td>B</td><td>At the concert he became aware of the coach's private life when he saw him dressed for a party rather than in his usual sports outfit.</td></tr><tr><td>3</td><td>C</td><td>When the history teacher proudly told them he ran the New York City marathon, they knew he must have spent a lot of his free time training.</td></tr></tbody></table>
The triplets of statements generated for each scenario were made to provide a within-scenario similarity. The exemplars were then reorganised by mixing the scenarios to generate dissimilar combinations as shown in Table 2.

<table><caption>**Table 2.** All combinations for the concept SONDER</caption><thead><tr><th></th><th>Version 1</th><th>Version 2</th><th>Version 3</th></tr></thead><tbody><tr><td>similar</td><td>1A/1B/1C</td><td>2A/2B/2C</td><td>3A/3B/3C</td></tr><tr><td>dissimilar</td><td>1A/2C/3B</td><td>1B/2B/3C</td><td>1C/2A/3A</td></tr></tbody></table>

The within-scenarios similarity and between-scenarios dissimilarity were tested in Experiment 1b using a paradigm of paired similarity judgments.

# METHOD

## Participants

Seventy-one British native speakers were recruited online using the Prolific platform to take part in this experiment (32 males; $M_{age} = 27.01$, SD = 5.13) presented online using Qualtrics (2020). They all gave their informed consent before taking part in the study. They received £2.50 for their participation.

## Materials and Procedure

The material was composed of the exemplars from 50 concepts organised by pairs that were either similar to each other or dissimilar. For instance, for the concept SONDER, the pairs were organised as described in Table 3 below so as to obtain 9 pairs in each similar or dissimilar condition.
<table><caption>**Table 3.** Example of similar and dissimilar pairs for the concept SONDER</caption><thead><tr><th>Similar pairs</th><th>Dissimilar pairs</th></tr></thead><tbody><tr><td>1A-1B</td><td>1A-2C</td></tr><tr><td>1A-1C</td><td>1A-3B</td></tr><tr><td>1B-1C</td><td>2C-3B</td></tr><tr><td>2A-2B</td><td>1B-2B</td></tr><tr><td>2A-2C</td><td>1B-3C</td></tr><tr><td>2B-2C</td><td>2B-3C</td></tr><tr><td>3A-3B</td><td>1C-2A</td></tr><tr><td>3A-3B</td><td>1C-3A</td></tr><tr><td>3B-3C</td><td>2A-3A</td></tr></tbody></table>

The material was composed of 900 exemplars that were divided into 16 blocks of 6 or 7 concepts for a total of 54 to 63 pairs per block (due to some blocks having 1 additional concept with 9 more pairs). The concept labels such as SONDER were never shown to the participants. The material was divided so that each participant saw only 1 condition for each concept either similar or dissimilar. Each participant was randomly assigned to only one block. The order of the pairs within a block was also randomised. For each pair, participants were asked to rate on a scale from 0 to 100 to what extent the two statements describe similar situations. Pairs of exemplars appeared one by one on the screen. The study was self-paced and took about 20 minutes to complete.

## RESULTS and DISCUSSION

The aim of this similarity judgment study was twofold. Firstly, it was designed to ensure that the distance between pairs could reflect the similar and dissimilar conditions meaning that the similar and dissimilar pairs constructed by the researchers were also judged similar and dissimilar by an external pool of participants. Secondly, the similarity judgment were embedded in a psychological space that reflected Euclidian distances between exemplars (see Shepard, 1991 for a review).
For the first objective to adjust the database only the concepts that elicited the highest similarity ratings for the similar pairs and the lowest ratings for the dissimilar pairs were kept. Mean similarity between pairs were analysed to build the similar and dissimilar conditions for the whole concepts. For the entire dataset, mean similarity ratings for similar and dissimilar combinations were compared by means of a paired-samples t-test. The analysis revealed that the constructed similar pairs elicited higher similarity ratings compared to the dissimilar pairs ($M_{Similar} = 70.08; SD = 12.28; M_{Dissimilar} = 44.74; SD = 6.61; Mean Difference = 25.29; 95CI [24, 27]; t(449) = 29.00, p < 0.001, Cohen's d = 1.32). These results were satisfactory as they validated the pairs generated based on the researcher's intuition. Although this analysis provided satisfactory results, a closer examination on a qualitative level of each concept revealed that some combinations presented a reversed pattern with dissimilar pairs showing a higher similarity rating compared to similar pairs. According to this qualitative examination, 25 concepts presented a reversed pattern for some combinations. The reversed pairs were reworked so as to increase within-scenario similarity compared to between-scenario dissimilarity. Although, sometimes only 1 pair from a concept was problematic, the 25 concepts were re-tested in their entirety so as to ensure the validity of the combinations after modifications were applied. The new version of the concepts was presented in Experiment 1c to a new pool of participants.

# METHOD

## Participants

One hundred and twelve British native speakers were recruited online using the Prolific platform to take part in this experiment (34 males; $M_{age} = 30.01, SD = 6.63$) presented online using Qualtrics (2020). They all gave their informed consent before taking part in the study. They received £2.50 for their participation.
**Materials and Procedure**

The material was composed of the exemplars from 25 modified concepts organised by
pairs that were either similar to each other or dissimilar. The procedure was the same as
Experiment 1b.

**RESULTS and DISCUSSION**

A first qualitative exploration of the data revealed that 8 concepts still presented a reversed pattern of similarity with some dissimilar pairs showing a higher similarity rating compared to dissimilar pairs. The rest of the analyses is based on the remaining 42 concepts.
As shown in Figure 2, the analysis revealed that the constructed similar pairs elicited higher similarity ratings compared to the dissimilar pairs ($M_{similar} = 72.80; SD = 10.10; M_{Dissimilar} = 40.08; SD = 12.13; Mean Difference = 32.72; 95CI [31, 34]; t(383) = 49.63, p < 0.0001, Cohen's d = 2.53).

<figure><img src="image_23.png" /><figcaption>**Figure 2.** Mean ratings for the similar and dissimilar conditions in Experiment 1c (* p < 0.001).</figcaption></figure>
These results showed an improvement of all the statistics compared to Experiment 1b. The mean rating for the similar condition was higher while the mean rating for the dissimilar condition was lower compared to the same statistics obtained from Experiment 1b. Overall mean difference between the similar and dissimilar conditions was higher compared to Experiment 1b ($M_{diff\_1b} = 25.29; M_{diff\_1c} = 32.72$). The consistency of the similarity pattern for pairs in the similar and dissimilar condition was established upon visual inspection of the concepts and was statistically confirmed by the increase in the effect size from Experiment 1b to 1c (Cohen's d 1b = 1.32; Cohen's d 1c = 2.53). Taken together, these statistics indicate that the database is composed of a robust combination of stimuli. These similarity ratings were then used to build the psychological space as a graphical representation of the Euclidian distance between similar and dissimilar dimension for each exemplar.

According to the GCM model, stimuli can be represented as points in a multidimensional psychological space (Nosofsky, 1986, 1992, 2011, 2014). This geometric representation of exemplars is based on a multidimensional space (MDS) algorithm developed by Kruskal (1964; see also Shepard, 1987). The MDS represents the pairwise similarities obtained from a similarity rating task. For instance, when considering the concept SONDER, the coordinates are given by the similarity rating of exemplar 1A in the similar condition and in the dissimilar condition (see Figure 3). The similar and dissimilar conditions represent the 2 dimensions of the psychological space which reflects the stored exemplars in memory that are the basis for comparison in a categorisation task. In the present case, let $x_{sonder} = (x_{sonder_s; 1}; x_{sonder_s; 2})$ be the coordinate locations for concept SONDER in the 2-D psychological space with $x_{sonder_s}$ the exemplars' mean similarity rating in the similar dimension and $x_{sonder_d}$ the exemplars' mean similarity rating in the dissimilar dimension.
<figure><img src="image_24.png" /><figcaption>**Figure 3.** 2-dimensional graphical representation of the psychological space for concept SONDER. Each point represents an exemplar with coordinates *x*<sub>sonder s</sub> as the exemplar's mean similarity rating in the similar dimension and *x*<sub>sonder d</sub> as the exemplar's mean similarity rating in the dissimilar dimension.</figcaption></figure>

This psychological space is a graphical representation of the exemplars that are stored
in memory. Each concept was described by 9 exemplars that could be combined so as to have
similar or dissimilar combinations. In a second experiment, the NewAbstractConcepts
database was used to test the ability to generalise the meaning of concepts learned through
exposure to either similar or dissimilar combinations to a novel test exemplar. Studies have
produced incoherent results as regards to the mechanisms involved in the generalisation of
meaning to novel situations. The literature has suggested two opposite mechanisms that are
similarity-based generalisation and contextual diversity. According to similarity accounts,
generalisation mechanisms are dependent on commonalities between conceptual
representations. In contrast, contextual diversity posits that a diversity of superficial features
can emphasise common underlying structures, thereby promoting generalisation. Experiment
2 aimed at testing these two hypotheses using abstract concepts that have no word referent but
are described by exemplars either similar to each other or contextually diverse.
# EXPERIMENT 2

## METHOD

### Participants

Seventy-two British native speakers were recruited online using the Prolific platform to take part in this experiment (16 males; $M_{age} = 30.29$; SD = 8.29) presented online using Qualtrics (2020). They all gave their informed consent before taking part in the study. They received £2.50 for their participation.

### Materials and Design

The material was composed of 42 NewAbstractConcepts from the database developed in the previous Experiment 1. For each concept 9 exemplars were constructed following 3 different scenarios. The triplet of exemplars belonging to one scenario were constructed as similar to each other. The dissimilar triplets of exemplars were obtained from reordering the scenarios so as to create combinations of exemplars that are different from one another. The concepts were then divided in 2 lists of 3 blocks each. Each block was composed of 3 exemplars either similar or dissimilar. Participants were randomly assigned to a single block following a Latin Square design meaning that each participant was presented with 1 triplet of similar or dissimilar exemplars from 1 scenario for each concept (see Table 4). Participants were all presented with the similar and dissimilar conditions.