When Language Models Fall in Love: Animacy Processing in Transformer Language Models
Authors: Michael Hanna
Yonatan Belinkov
Sandro Pezzelle
Venue: EMNLP 2023 Main (2023)
Keywords: animacy
language models
selectional constraints
semantics
discourse context
TL;DR: Language models process animacy in a surprisingly humanlike fashion, even in challenging scenarios.

Abstract: Animacy—whether an entity is alive and sentient—is fundamental to cognitive processing, impacting areas such as memory, vision, and language. However, animacy is not always expressed directly in language: in English it often manifests indirectly, in the form of selectional constraints on verbs and adjectives. This poses a potential issue for transformer language models (LMs): they often train only on text, and thus lack access to extralinguistic information from which humans learn about animacy. We ask: how does this impact LMs' animacy processing—do they still behave as humans do? We answer this question using open-source LMs. Like previous studies, we find that LMs behave much like humans when presented with entities whose animacy is typical. However, we also show that even when presented with stories about atypically animate entities, such as a peanut in love, LMs adapt: they treat these entities as animate, though they do not adapt as well as humans. Even when the context indicating atypical animacy is very short, LMs pick up on subtle clues and change their behavior. We conclude that despite the limited signal through which LMs can learn about animacy, they are indeed sensitive to the relevant lexical semantic nuances available in English.

When Language Models Fall in Love:
Animacy Processing in Transformer Language Models
Michael Hanna
Yonatan Belinkov
Sandro Pezzelle
ILLC
Technion Israel
ILLC
University of Amsterdam m. w. haana@unaa.nl
belinkov@technion. ac.
University of Amsterdam s. s.pezzelle@uva nl
Abstract
imacy distinctions in English often take the form of selectional constraints that limit the use of certain verbs or adjectives with in/animate entities. For ex- ample, only animate entities can walk or think. So, while animacy is rich distinction at the cognitive level, at the linguistic level, its signal can be muted. Today's pre-trained transformer language mod- els (LMs), however, are trained only on linguistic input. If they are to learn to process animacy, they must thus do so only from its downstream effects in text, unlike humans, who use visual and physical stimuli. We therefore ask: do such LMs respond to animacy in language humans do?
Anhnacy-whether entity is alive and sen- tient-is is fundamental to cognitive processing, impacting areas such memory, vision, and language. However, animacy is not always expressed directly in language: in English often manifests indirectly, in the form of se- lectional constraints on verbs and adjectives. This poses a potential issue for transformer lan- guage models (L.ss): they often train only text, and thus lack access to extralinguistic in- formation from which humans learn about an- imacy. We ask: how does this impact LMs' animacy processing-do they still behave as humans do? We answer this question using open-source LMs. Like previous studies, we find that LMS behave much humans when presented with entities whose animacy is typ- ical. However, we also show that even when presented with stories about atypically animate entities, such peanut in love, LMs adapt: they treat these entities as animate, though they do not adapt as well as humans. Even when the context indicating atypical animacy is very short, LMs pick up on subtle clues and change their behavior. We conclude that despite the limited signal through which LMs can learn about animacy, they are indeed sensitive the relevant lexical semantic nuances available in English.
We answer this by treating LMs as psycholin- guistic test subjects, probing how they react to vio- lations of animacy-related selectional constraints. Like prior work (Warstadt et al., 2020; Kauf et al., 2022), we first study LMs' responses in scenarios involving typical animacy. such situations, ani- macy is a simple mapping between an object (e.g. a peanut) and its usual animacy (inanimate). We find that like humans, LMs generally prefer sentences that respect animacy-related selectional constraints, assigning higher probabilities to such sentences.
Unlike prior work, we also study atypical ani- macy (Coll Ardanuy et al., 2020), where a typically inanimate object becomes animate. We draw on Nieuwland and van Berkum (2006), which mea- sured human N400 responses in scenarios with atypically animate entities like a peanut in love. We compare LM surprisal to human N400 brain responses and find that like humans, LMs are ini- tially surprised encounter entities like a peanut in love, but quickly adapt, becoming less surprised. Stronger LMs are more able to replicate the large magnitude of human N400 reduction.
1 Introduction
Animacy plays a significant role in cognitive pro- cessing, as evidenced by the fact that animate enti- ties are easier to remember and prioritized in visual processing (Nairne et al., 2013; New et al., 2007; Bugaiska et al., 2019). It is so important that even young children can distinguish animate and inani- mate entities (Rakison and Poulin-Dubuin 2001), and these are processed in distinct domain-specifee brain regions (Caramazza and Shelton, 1998).
Given LMs' success adapting atypical an- imacy with long context, we test them on short sentences about atypically animate entities, and measure the extent to which their outputs reflect this atypical animacy. We find that even with lim-
Animacy distinctions also manifest in language; however these distinctions may appear indirectly. While some languages explicitly mark animacy, an-

---

ited context, LMs adapt their output distribution, treating the entity as animate. We conclude that, despite training without the modalities that hu- mans use to learn about animacy, LMs respond to shifting animacy in a surprisingly human-like way. Code for our experiments is available at wttptttw....///aammmmmmmmssiiii iiiimss-in-oooo love.
2016; Warstadt et al., 2020; Sinclair al., 2022). Other work compares LMs to humans by using surprisal-the negative log probability of string- to estimate measures of cognitive effort during text processing. LM surprisal is versatile, and well- correlated with reading times, eye-tracking fixa- tions, and EEG responses (Smith and Levy, 2013; Aurnhammer and Frank, 2018; Michaelov and Bergen, 2020); moreover, surprisal from stronger LMs provides better predictive power (Goodkind and Bicknell, 2018; Wilcox et al., 2020).
2
Related Work
2.1 Animacy in Language
Animacy in cognition is often framed as a gradient phenomenon (de Swart and de Hoop, 2018). In tripartite hier-
We use LM surprisal to predict the N400 brain response, which is elevated at semantically unusual content, like animacy-related relltted selectional constraint violations. Studies have found that word's sur- prisal correlates with human N400 response thereto (Frank al., 2013, 2015; Michaelov al., 2022); transformer LMs are the state of the art for this (Merkx and Frank, 2021; Michaelov et al., 2021).
language, this often simplifies archy (humans animals objects) or a binary (humans & animals > objects); entities are distin- guished synactically or morphologically by their position therein (Comrie, 1989).
Animacy exists both the type level (e.g. dogs rocks) and the token level (e.g. specific rock in story might be animate, though rocks are typically not). Moreover, linguistic animacy is based not only on biology, but also on the speaker's s closeness and empathy with the entity in question (Kuno and Kaburaki, 1977); thus a speaker might treat their dog as more animate than an unknown dog.
2.3 Animacy Detection
We note that our interest in the processing of atypical animacy parallels similar developments in the NLP task of animacy detection: determining whether given entity is animate. While many ani- macy detection studies originally considered only typical animacy (Orasan and Evans, 2007; Bow- man and Chopra, 2012), later animacy detection work has recognized that entities' animacy may not always be typical, and may change over the course of a narrative (Karsdorp et al., 2015; Jahan et al., 2018). Particularly relevant for the present study, Coll Ardanuy et al. (2020) combine LMs and atyp- ical animacy by using BERT for atypical animacy detection. Although this approach uses LMs tool label animacy, rather than studying how LMs process animacy, we share their interest in the atypical edge cases of animacy.
The precise effects of animacy in language vary cross-linguistically. from explicit animacy marking to more indirect effeets as in English. The latter include not only strict animacy-based selectional constraints (Caplan et al., 1994), but also more sub- tle grammatical influences (Rosenbach, 2008; Bres- nan and Hay, 2008). For example, animate entities are more often mentioned first in sentence, even if doing so produces less common constructions, such as the passive (Ferreira, 1994).
Here, we focus on the human / inanimate ob- ject dichotomy, and the mhimacy-basedd selectional constraints thereby imposed; this strong contrast should produce easier-to-measure effects in LMs.
2.4 Animacy in LMs
How neural models capture animacy has long inter- ested cognitive scientists; Elman (1990) trained a simple neural LM on artificial language data, and found that its representations of animate and inani- mate entities formed distinct clusters. More recent work has assessed the animacy-processing capabil- ities of modern LMs, mostly focusing on typical animacy. Animacy is one area tested by BLiMP (Warstadt et al., 2020), which we revisit in Sec- tion 4. Kauf et al. (2022) investigate animacy as part of LMs' generalized event knowledge; they also find that LMs are sensitive to (typical) ani-
2.2
LMs as Test Subjects
We study the behavior of LMs by treating them as psycholinguistic test subjects, a popular approach. One such line of work analyzes LMs by using the probability they assign to a sentence as proxy for acceptability judgments. Generally, such studies provide pairs of sentences, one acceptable and one not; LMs must assign the more plausible sentence a higher probability. This method has been used to study LMs' processing of negation, subject-verb agreement, and more (Ettinger, 2020; Linzen et al.,

---

macy pertains to selectional constraints.
Acc?
Sentence
We move beyond typical animacy to atypical animacy by using LM surprisal to replicate Nieuw- land and van Berkum's 's (2006) studies on human N400 response to atypical animacy. Contempora- neous work (Michaelov et al., 2023) replicates one of these experiments in the original Dutch. In con- trast, we replicate all experiments from Nieuwland and van Berkum (and Boudewyn et al. (2019)). These highlight situations which models can capture general trends, but fail to capture low-level nuances. Moreover, by studying a diverse set of English LMs, we can identify how LMs' strength affects their predictive power.
T
Yes
Naomi had cleaned fork.
T
No
That book had cleaned fork.
P
Yes
Lisa was kissed by the boys.
P
No
Lisa was kissed by the blouses.
Table BLiMP examples: we provide one example each from the Transitive and Passive datasets. Each is minimal pair of sentences: one Acceptable and one not.
Accuracy
Performance on BLIMP Animacy Datasets
Accuracy
Areumay
0.9
3 Models
We experiment with these models: GPT-2 small, medium, large, and XL (Radford al., 2019); OPT 2.7B, 6.7B, and 13B (Zhang et al., 2022); and LLaMA 7B, 13B and 30B (Touvron et al., 2023). We use autoregressive LMs, as we need com- pute probabilities for whole sentences. Morcover, we choose open-source models, to make our work replicable. We provide implementation details in Appendix A.
0.5
GFT2 5mall
69T-2
GPT22
GRT2
OPT
OFT
CPT
LLaMA
LLaMA
LUMMA
Medium
Lange
Model
Figure I: Model accuracy on BLiMP. Models match humans in the transitive, but not passive setting.
4 Typical Animacy
the acceptable sentence of the given minimal pair. Random performance for both datasets is 50%.
We test models' responses animacy in situations where the animacy of given token, or instance of entity, aligns with the animacy of its type more generally (e.g. cats are animate; rocks are not).
Models attain high performance in both scenar- ios. On transitive examples, they reach over 80% accuracy; some models prefer the sentence that respects animacy constraints more often than hu- mans do (>87%). In the passive scenario, the gap between models (80%) and humans (86%) is wider. This difference between the transitive and pas- sive may be due more setup differences than distinct animacy processing in the two scenar- ios. In the passive case, the target word is always in the last position, so model performance is deter- mined only by the target's probability. In contrast, the target word is not the final token in the transi- tive case, so model success is determined by the probability of longer string.
Experiment We test the models in Section on the animate-transiiee datasets of the BLiMP benchmark (Warstadt et al., 2020). Each dataset contains 1,000 minimal pairs of synthetic English sentences that differ only by one or two words (Table 1). By construction, one sentence respects animacy constraints; the other vi- olates them. We evaluate models these datasets by computing the probability it assigns each sentence of each minimal pair. A model gets an example correct assigns higher probability to the sentence that respects the animacy constraint. We compute model accuracy over each dataset.
and mnimate-passive
Discussion Our results indicate that models re- spect animacy constraints in typical scenarios: they match human performance on the transitive dataset, and are close behind on the passive. However, this test cannot distinguish between a model that truly understands animacy, and one that just associates words (types) with other words that reflect that word's typical animacy. For example, the model might simply associate a word like "shoe" with
Results Figure displays results for each model. It also includes human baselines, reported directly from Warstadt et al. (2020), which indicate the proportion of examples where annotators preferred
'The names of OPT and LLaMA models indicate (approxi- mate) parameter counts; the GPT-2 models have 1 17M, 345M, 762M, and 1,5B parameters respectivoly.

---

verbs that take inanimate objects, without under- standing that the inanimacy of an individual shoe is what prohibits its use with animate-selecting verbs.
A nurse was talking to the sailor/oar [1] who had been violent boating accident. The sailor/oar cried for long time over the storm that had raged over the lake for hours. The nurse consoled the sailor/oar [3], saying that he would soon be well again. The sailor/oar complained of a had headache that would not go away. The nurse gave the sallor/oar [5] large dose of aspirin. The sailor/oar thanked her and fell asleep.
To solve this problem, our analysis must move beyond type-level animacy, and test models' pro- cessing of animacy at the token level. We thus test models' responses to entities whose token-level ani- macy is atypical, distinet from their usual type-level animacy. If models process these entities accord- ing to their type-level animacy, their understanding of animacy is rather shallow. In contrast, models that process entities according their token-level animacy may better understand animacy in full.
Figure 2: Story from Nieuwland and van Berkum, repe- tition experiment (translated and edited). Times when N400 responses were recorded are numbered, in bold.
5.1 Repetition Experiment
Original Study In Nicuwland and van Berkum's 's first experiment, participants listened to Dutch sto- ries that contained either a typical, animate entity or an inanimate entity behaving as if it were ani- mate (Figure 2). Participants N400 responses were measured at the 1st, 3rd, and 5th mentions of the entity (in Figure 2, either oar or sailor, in bold). Nieuwland and van Berkum found that partic- ipants had a moderate N400 response to the first mention of typically animate entity, and a low response subsequent mentions. In contrast, par- ticipants initially had high N400 response to the atypically animate entity. However, by the 3rd and 5th mentions thereof, their N400 responses were so low as to be statistically indistinguishable from the responses to mentions of the animate entity in the same contexts. Thus, while humans were initially surprised by the atypically animate entity, they quickly adapted to the situation, and found it no more surprising than typically animate entities.
5 Atypical Animacy
In this section, we attempt to determine if LMs can capture animacy not only at the type-level, but also at the token-level. We do so by comparing model and human responses in cases of atypical animacy. where entities' canonical type-level animacy and their actual token-level animacy differ.
For human data, we turn to two similar studies- Nicuwland and van Berkum (2006) and Boudewyn et al. (2019) relied on the N400, a brain re- sponse measured via EEG that is elevated when pro- cessing semantically anomalous input. Both stud- ies measured participants' N400 responses while they read stories where typically inanimate en- tity acted animate (Figure 2), similar in tone and content to cartoon, or fairy-tale. Both found that while participants were initially surprised by the atypically animate entity, they quickly adapted, yielding low N400 responses to the entity.
Our Experiment We model N400 responses with LM surprisal, as discussed in Section 2.2. For each of the 60 examples, we measure the sur- prisal of the animate and inanimate entity given the context at each timestep. For example, to model the inanimate N400 response at T1 in the exam- ple from Figure 2 given model pa. compute log2 Po (oar)A A nurse was talking to the). Then, we compute the mean surprisal of examples con- taining animate and inanimate entities separately.
We ask the same true of pre-trained LMs: can they adapt to entities that are animate at the token-level, despite being typically inanimate? Or is their processing of animacy limited to a simple type-level understanding? To answer this question, we replicate these studies with pre-trained LMs, using their surprisal model N400 responses.
Since the original stimuli are in Dutch, we trans- late them to English, to make them compatible with the English LMs.3 3 We do so using DeepL:* 4 transla-
Nieuwland and van Berkum's repetition experiment; their con- text experiment; and Boudewyn et al.'s adapta- tion experiment. For each, we first explain the original study. Then, we explain how we adapt the experiment for LMs. Finally, we report our results and compare them the original study's 's results.
We replicate three experiments:
II Appendix C, we replicate Boudewyn al.' 's English version of Nieuwland and van Berkum's 's context experiment with LMs; results are identical to those of Section 5.2.
'W perform these experiments in Dutch in Appendix B. Dutch results are compurable to English results. "tt pottps:/ / /www. deepl. com/ trasslator

---

Surprisal [Dits]
Repetition Experiment Surprisals
A girl sat next to a diamond who was always doing strange things. The diamond told her that he liked to eat erasers. The girl ignored the diamond and his stories. Then the diamond said he also liked to sing songs. The diamond was quite foolish/valuable but secretly also very funny. That's why she always sat next to him.
Inanimate Surprieal
Antmate Surprtal
10
Figure 5: Story from Nieuwland and Berkum, con- text experiment (translated and edited). N400 responses were recorded the words in bold.
It. 13 1sla
13
15th
23 GPT2
15
13
15
13 OPT
15
15
15
15
GPT-2
GPT2
GPT2
OPT 2.78
OPT
LLAHA
LLaNA
LLaMA
are adapting to the extent that humans are. Since Nieuwland and Berkum found that human N400 responses in the two conditions were statisti- cally indistinguishable by T3, we test if the same true for model surprisal. We use the Wilcoxon signed-rank test for non-normally distributed data (Wilcoxon, 1945) to determine if model surprisals for animate and inanimate entities are distinct at each timestep. We find (Figure 4) that like humans, LMs have statistically significant difference be- tween animate and inanimate surprisals at TI. How- ever, while there was no difference in humans at T3, there are differences (p < 0.01) most models; only the largest exhibit none. At T5, differences disappear in yet more large models. While models can generally approximate trends in human N400 responses to atypical animacy, only the largest and most powerful fully replicate human adaptation.
Small
Mod
Large
XL
6.78
138
76
158
308
Model and Timestop
Figure 3: Mean repetition experiment surprisal. Inan- imate surprisal initially higher, but both surprisals decrease rapidly after TI, becoming near identical.
0.00
GPT-2 Medium
0.00
0.00
0.00
0.8
GPT-2 Large
0.00
0.00
0.01
0.6
p-vauue
Model
OPT-2.7B
0.00
0.00
0.14
7B
0.00
0.06
0.67
0.4
OPT-138
0.04
LLaMA-78
0.14
0.93
0.2
LLaMA 13B
0.00
0.04
0.75
LLaMA-308
0.00
0.16
0.33
5
Timestep
Figure 4: Stat. significance of the difference between animate inanimate surprisal, by model timestep
Overall, pre-trained LMs seem able to mimic human-like adaptation to atypically animate enti- ties. It is tempting to conclude that they have a human-like understanding of animacy, that works at the token rather than the type level. However, it is equally possible that their decreased surprisal is due to repetition, rather than deeper understand- ing of animacy. Transformer LMs even have low-level emergent structure, induction heads, ded- icated such copy-pasting (Olsson et al., 2022). Fortunately, Nieuwland and van Berkum shared this concern: humans might generate lower N400 responses only because they had seen the atypically animate token before. Thus, we also replicate their context experiment, which avoids this issue.
tions were checked by a native Dutch speaker. We then manually post-edited each stimulus to ensure it matched the cartoon-like tone and content of the original, and contained inanimate characters that violated typical animacy constraints in the 1st, 3rd, and 5th sentences of the stories. Because we pre- serve the relevant aspects of the stimuli we expect the trends in N400 responses be the same.
Results All models capture broad trends in hu- man N400 responses well (Figure 3). At TI, mod- els are very surprised by the inanimate entity, and only moderately surprised by the animate entity. At later timesteps, however, both entities" surprisals drop precipitously, to similar levels: models adapt both entities quickly, just like humans do. Still, the raw results do not prove that models
5.2 Context Experiment
Nieuwland and van Berkum's context experiment showed that participants' low N400 responses did not stem from lexical repetition.
Original Study As in the prior experiment, par- ticipants read 60 Dutch stories containing an atyp- ically animate entity; at the end of each story, the entity was described using an adjective that was
We also edited stories for fluency, and to convert Dutch cultural references to Anglophone counterparts. The trans- lated English stimuli can he found at ht https: //tp::///ithub thub com/ hannamw/ -as-lminooo love

---

Suprisa (bits)
Context Experiment Surprisals
A lucky fellow/peanut had a big smile [1] on his face. The fellow/peanut was amazed [2] about his good fortune. Just now he had won the jackpot of two million dollars. The fellow/peanut was elated/salted and who could blame him.
20.0
Animate Surprisal
Antimate Bastline
17.5
Insn Surpereal
Insnimate Baoeline
12.5
Story from Boudewyn al. (2019). N400 responses were recorded at the words in bold.
Figure 7:
10.0
Surprisal (bits)
Adaptation Experiment Surprisals
Surprisal
Surprisal
2.
5.
Small Medium Lange
2.78
6.78
135
78
138
308
10
Model
Figure 6: Context experiment surprisals. With context, the animate adjective is much less surprising; in the contextless baseline condition, this is reversed.
12
13
17
12 GPT2 Large
12
13
17
13
17 OPT 0.77
11
12
12
12
K3
12
GPT2 Small
GPT2
GPT22
ORT
OPT 1:30
LLaMA
LLMMA
LLaMA
138
300
Model and Timestep
Figure 8: Adaptation experiment surprisals. Inanimate surprisal starts higher; the gap shrinks for larger LMs.
either coppext-approprrite (and generally used for
animate entities) or context-inappropriate (but typi- cal for the inanimate entity; Figure 5). The N400 response was measured the adjective the end the story (foolish or valuable). N400 responses for the context-appropriate animate adjective were far lower than those to the appiry-aproopriate inani- mate adjective, showing that the first experiment's s effects were not caused by lexical repetition.
prior experiments. Like the first, it captures adapta- tion over time; like the second, it avoids the poten- tial issues of repetition.
Original Study Boudewyn et al.' 's adaptation periment parallels the two previous experiments. Participants listened to 120 English-language sto- ries containing either a typically or atypically an- imate entity (Figure 7). Participants' N400 re- sponses were measured at the first content verb of the first two sentences of each story. These verbs signal that their subject is (perhaps atypically) an- imate, although they are notably not the same in each sentence. Findings mirrored those of Nieuw- land and van Berkum: there was a sharp drop in N400 response at between the two timesteps in the inanimate scenario.
Our Experiment We calculate the surprisal at the animate and inanimate adjective for each of the 60 stories. We also compute baseline surprisals, the surprisal of the inanimate adjective without the entire story context, to show they are indeed high: e.g. - log2 pe [foolish]The diamond was quite).
Results For models, mean surprisal of the animate adjective is much lower than that of the inanimate adjective (Figure 6). This is significant in all cases (p < 0.01; Wilcoxon signed-rank test). Moreover, the animate adjective is assigned higher probability in almost all over 90% for large models. This mirrors the human trend: the N400 re- sponse to the context-appropriate animate adjective was much lower than the response to the entity- appropriate inanimate adjective. In the contextless baseline situation, the inanimate adjective receives a much lower surprisal than the animate adjective. Like humans, models use context and overcome their lexical knowledge regarding the traits can ap- ply to animate and inanimate entities.
Our Experiment For each of the 120 stories, we calculate the surprisal at the two critical verbs, in the animate and inanimate case.
The results of the adaptation experiment (Figure 8) might appear starkly different from those of the repetition experiment. As before, surprisal at the critical word drops in the inanimate case, though only slightly. But in the animate case, sur- prisal remains constant or increases.
Results
These results are in fact consistent with car- lier findings. The inanimate surprisal drop indicates that the short context sufficed convince models of the entity's s animacy. Moreover, the fact that sur- prisal does not drop in the animate case suggests that models are reacting specifically to the contex-
5.3 Adaptation Experiment
We now replicate Boudewyn et al. 's adaptation ex- periment, which combines the strengths of both

---

Dataset We craft a set of short incomplete sen- tences that describe an atypically animate entity (Table 2). The sentences are designed to elicit a critical next word- an adjective or a verb- indicates if the LM treats the entity as animate. For examplc, if an LM continues "The boat snored and started to" with the verb "dream" this indicates that the boat is animate; continuing with "sink" does not. Unlike in prior experiments, these sentences provide only one clue indicating atypical animacy. tual cues that the inanimate entity is animate, as opposed to the context more generally. The im- portance of model size is also consistent: stronger models have a smaller gap between animate and inanimate surprisals at T2. Although this gap is still significant for current models, trends indicate that stronger models may eliminate it.
5.4 Discussion
Across experiments, models replicate broad trends in human N400 responses. Stronger models repli- cate human results better, with lower surprisals at inanimate entities. Do they thus process animacy more like humans? We caution that low surprisals are not always desirable for cognitive modeling, as surprisal from LMs underestimate processing difficulty in terms of reading time (van Schijndel and Linzen, 2021; Arehalli et al., 2022; Oh and Schuler, 2023); this may be because even relatively small LMs can predict next words as well as hu- mans (Goldstein et al., 2022). Stronger LMs may only be better models of animacy processing in this situation because lower surprisals are desirable.
We create this dataset by defining prompts and filling them with nouns and verbs we sample from predefined set. We sample from 181 nouns that humans rated as not very animate, but highly con- crete; non-concrete inanimate nouns (e.g. "fear") cannot become animate, except metaphorically. We use concreteness ratings from Wilson (1988), and animacy ratings from VanArsdall and Blunt (2022). For the verbs, we use a manually-fittered set of 191 verbs that imply their subject is animate, from Ji and Liang (2018) 8). Each verb implies that its sub- ject is animate for either psychological or physical reasons; e.g. think is psychological while walk is physical. Each verb co-occurs with human subjects at a high, high-mid, or mid frequency. We create a dataset of 10,000 items (Table 2) by sampling prompts, nouns, and verbs.
Regardless of the effects of model size, another question remains: do these positive results indicate that these LMs understand animacy? We cannot be certain: there exist mechanisms by which trans- former LMs could perform well without any deep understanding of animacy. In the repetition exper- iment, models could use copying mechanism to reduce their surprisal at repeated entities. In the context and even adaptation experiment, models could rely on the context, while ignoring the inani- mate entity. This is real concern: Michaelov et al. (2023) construct (simple) model that does this.
We run all LMs on the dataset. For each example, we evaluate whether the LM treats the entity in that example as animate by compar- ing the LM's output distribution reference dis- tributions. If our original sentence is O =TTh chair spoke and began to", our inanimate reference is I =TTh chair began to", while our animate reference A "The [human] began to", with a human entity randomly sampled from person, man, woman, boy, girl, and child. We indicate via DxL(Al the divergence between the next-word distributions given A as context, and given O context, with other KL divergences defined analo- gously. We focus in particular on DK (xA())))) animacy divergence; lower animacy divergence implies a more "animate" distribution.
Experiment
In both cases, context is the complicating factor: LMS might exploit shallow context cues to sim- ulate animacy processing effects, without having any real internal model of animacy. To investigate this question further, we study LMs' reactions to atypically animate entities in low-context setting.
6 Low-Context Atypical Animacy
The previous experiments have shown that LMs can adapt in scenarios with atypically animate enti- ties; however, LMs could have exploited shallow context features do so, without any specific un- derstanding or representation of animacy. We now investigate the extent to which LMs can leverage cues in the context by testing their behavior on very short sentences exhibiting atypical animacy.
Quantitative Analysis Figure 9 shows KL diver- gences between the atypically animate sentence (O) and the reference distributions (A/1). For all models, the divergence between the inanimate
"A list of prompts, nouns and verbs is in Appendix F. Full dataset is available at https: //githubbbbb... con/ com/hannamw/ Ims in- For further experiments that vary properties of this dataset, see Appendix D.

---

Sentence
Rank
#1
#2
#3
#4
#5
The kilt commented and started to The cart noticed and very The dime retired and was very The telephone sighed and began
1
walk
get
move
laugh
say
3
angry
ups
excited interested popular
9994
rare
valuable
scar bu
collect
10000
ring
speak be
Table Dataset examples and their top-5 continuations (sometimes partial words). The example at rank has the rith lowest animacy divergence (of 10,000). Low-divergence examples have animate continuations; high-divergence ones are stereotypical and inanimate. The top two examples use psychological verbs; the bottom two, physical.
Mean KL Divergences by Model
vergences than those that elicit adjectives.
We then analyze the effects of verbs and nouns on the sentence. For each verb or noun, we calcu- late the mean animacy divergence of sentences con- taining it. We observe a wide spread per verb and noun (Figure 10, D and E), suggesting that they both impact model behavior. We then study the factors affecting individual nouns' and verbs' diver- gences. We find that psychologically animate verbs have somewhat lower divergence than physically animate verbs; psychologically animate verbs pro- duce more animate behavior from LMs (p < 0.01, T-test; Figure 10 B). Sentences with verbs that had higher co-occurrence with humans had a lower di- vergence than those with a lower co-occurrence (p < 0.01; Figure 10 C), though the effect size is rather small. For the nouns, however, neither ani- macy concreteness explains trends in animacy divergence.
Smol
Nedin
1age
XI
3.8
677
138
78
130
30
Model
Figure 9: KI. Divergence between atypically animate (O) and animate (A) inanimate (I) references. Error bars (95% CT) marked, but extremely small. The lower the bar, the more similar the distributions.
and animate references (purple) is the highest. Al- though only the added verb separates the atypi- cally animate sentence from the inanimate refer- ence, this leads its divergence with the animate reference (red) to be consistently lower; that is, adding the verb significantly increased the distri- bution's s animacy. For OPT models, the animacy divergence is the lowest of their three divergences: the atypically animate distribution is even more similar to the animate distribution (red) than inani- mate distribution (blue). This trend holds true for LLaMA models well, though only weakly. Still, all model behavior clearly shifts with the addition of the animacy-implyin verb.
Qualitative Analysis We also qualitatively ver- ify that sentences with low animacy divergence have more animate continuations than those with high divergence. We sort examples by divergence and examine their top-5 continuations, focusing on examples at the top and bottom of the list.
Table 2 shows that animacy divergence aligns well with the qualitative animacy of continuations. Low Low-divergence examples have entities that laugh, or angry. High-divergence uations stereotypical for the inanimate entity: collectable dime becomes rare, or a telephone be- gins to ring.
have contin-
To understand the cause of this shift, we analyze the data with respect to the known factors that vary between our prompts, discern which affected model behavior. Results were similar across mod- els, so we display results for one model, LLaMA-7B.
Results show that LMs can adapt to atypical animacy even given limited cues: one verb, rather than entire story. They also suggest that LMs do require long context adapt; how- ever, the factors that regulate this adaptation are complex. Some, like the choice of prompt, bear no clear relation to animacy; others, like the nouns,
Discussion
We first analyze the effect of prompts, focus- ing the difference between those that elicit verbs ("and began to...") and those that elicit adjectives ("and was very...").. We find (Figure 10, A) that berb-eliciting prompts produce lower animacy di-
Significant for all three groups (F-test), and pairwise.

---

Animacy Divergence Distribution by type. Verb Category, and Verb-ifumnn Co-octurrencc
Mean Animacy Divergance per Webb and Noun
A
sramst
we'k catepary plysical psychological
C
vab. /recurccy HIGH
1.5
VERU ABl
HIGH HDD
0.00
ND
10-
0.05
E
0.00
2 0.05
2
9.4
0.5
0.6
0.7
0.8
0.9
100
Animacy Divergence (1
Animacy Divergence (1e-4)
Figure 10: Left: Distribution of animacy divergences by prompt type, verb category, and verb-human co-occurrence.
Right: Distribution of mean animacy divergences per-verb and per-noun. Each point is one verb (or noun).
show no clear pattern in how they affect model re- sponses. Still, some interpretable factors exist. Psy- chological verbs may induce more animate continu- ations because they indicate animacy more strongly. While an inanimate object might metaphorically en- gage in physical activities like dance, they seldom marry or volunteer. These psychological words may thus serve as stronger signals animacy.
land and van Berkum's stimuli to English, but do not test them again on native English speakers; the N400 responses to Dutch data could differ from N400 responses English data, though we assume they will be similar. Similarly, studying human responses to our low-context atypical ani- macy stimuli (Section 6) would better inform our analysis LM performance.
7 Conclusions
Ethies Statement
Although animacy manifests only indirectly in English- it is not morphologically marked- pre- trained LMs demonstrate relatively good animacy processing abilities. They both respect typical an- imacy and adapt to atypical animacy at close-to- human levels, although differences remain. They also demonstrate some ability to adapt atypi- cal animacy even when indicated by a very short context, LM adaptation still lags behind that of hu- mans, but large models increasingly shrink the gap. We conclude that in the scenarios we test, LMs respond to animacy like humans do; however, our behavioral methodology can yield no conclusions about how models achieve this. Related work on world models (Li et al., 2021, 2023) suggests that by using causal techniques to search for internal structure in models, future work could not only demonstrate that LMs respond well to animacy, but also explain how they do so.
This work presents only minor ethical concerns. A particular concern is one of bias and stereotypes; the original stories Nieuwland and van Berkum (2006) do contain stcreotypes. We attempt to soften these in our translations, but some stereotypes are still present in the translated material.
More generally, LMs such as those analyzed must used with caution. Although such models achieve high performance on language-based tasks. this performance does not necessarily stem from genuine linguistic understanding. Morcover, mod- els can not only perpetuate harmful biases present in their training data, but also create misleading false output.
A Acknowledgements
The authors thank the members of the Dialogue Modelling Group, particularly Joris Baan and Raquel Fernández, as well Marianne de Heer Kloots, for their helpful feedback. They also thank the authors of the original human studies, in par- ticular Mante Nieuwland and Megan Boudewyn, who allowed us to translate use their stimuli. Fi- nally, the authors thank participants of the ELLIS, Al4media, and AIDA Symposium on Large Lan- guage and Foundation Models in Amsterdam for their insightful comments. This work was sup- ported by the ISRAEL SCIENCE FOUNDATION (grant No. 448/20), Open Philanthropy, and an Azricli Foundation Early Career Faculty Fellow- ship.
Limitations
this study, we use primarily behavioral experi- ments. These are suitable for comparing models to human data, but do not reveal the causal mecha- nisms by which LMs process animacy. In order to discover these, it would be more appropriate to use causal interventions similar techniques, which we do not explore.
Considering the limitations of behavioral tech- niques, this study is still limited by the fact that did not collect human data. We translate Nieuw-

---

References
International Committee on Computational Linguis- tics.
Suhas Arehalli, Brian Dillon, and Tal Linzen. 2022. Syntactic surprisal from neural models predicts. but underestimates, human processing difficulty from syntactic ambiguities. In Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL), pages 301-313, Abu Dhabi, United Arab Emirates (Hybrid). Association for Com- putational Linguisties.
Bernard Comrie. 1989. Language universals and lin- guistie typology: Syntax and morphology. University of Chicago press.
Peter de Swart and Helen de Hoop. 2018. Shifting animacy. Theoretical Linguistics. 44(1-2):1-23
Christoph Aurhaamerr and Stefan L Frank. 2018. Comparing gated and simple recurrent neural net- work architectures models of human sentence pro- cessing.
Wietse de Vries and Malvina Nissim. 2021. As good as new, how to successfully recycle English GPT-2 to make models for other languages. In Findings of the Association for Computational Linguistics: ACL- IJCNLP 2021, pages 836-846, Online. Association for Computational Linguistics.
Collin F. Baker, Charles J. Fillmore, and John B. Lowe. 1998. The Berkeley FrameNet project. In 36th An- nual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Volume 1. pages 86-90, Montreal, Quebee, Canada. Association for Compu- tational Linguistics.
Jeffrey L. Elman. 1990. Finding structure in time. Cog- nitive Science, 14(2):179-211
Allyson Ettinger. 2020. What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models. Transactions of the Association for Computational Linguistics, 8:34-48.
Megan Boudewyn, Adam Blalock, Debra Long, and Tamara Swaab. 2019. Adaptation to animacy vio- lations during listening comprehension. Cognitive, Affective, & Behavioral Neuroscience, 19.
Ferreira. 1994. Choice of passive voice is affected by verb type and animacy. Journal of Memory and Language, 33(6):715-7338
Samuel R. Bowman and Harshit Chopra. 2012. Au- tomatic Animacy classification. In Proceedings of the NAACL HLT 2012 Student Research Workshop, pages 7-10, Montréal, Canada. Association for Com- putational Linguistics.
Stefan L. Frank, Leun J. Otten, Giulia Galli, and Gabriella Vigliocco. 2013. Word surprisal predicts n400 amplitude during reading. In Proceedings of the S1st Annual Meeting of the Association for Compu- tational Linguistics (Volume 2: Short Papers), pages 878-883 Sofia, Bulgaria. Association for Computa- tional Linguistics.
Joan Bresnan and Jennifer Hay. 2008 Gradient gram- mar: An effect of animacy on the syntax of give in new zealand and american english. Lingua, 118(2):245-259, Animacy, Argument Structure, and Argument Encoding.
Stefan L. Frank, Leun J. Otten, Giulia Galli, and Gabriella Vigliocco. 2015. The ERP response to the amount of information conveyed by words in sentences. Brain and Language, 140:1-11.
Aurélia Bugaiska, Laurent Grégoire, Anna-Malika Cam- blats. Margaux Gelin, Alain Méot, and Patrick Bonin. 2019. Animacy and attentional processes: Evi- dence from the stroop task. Quarterly Journal of Experimental Psychology, 72(4):882-82) 889. PMID: 297 16460.
Dirk Goldhahn, Thomas Eckart, and Uwe Quasthoff. 2012. Building large monolingual dictionaries the Leipzig corpora collection: From 100 to 200 lan- guages. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (I,REC' pages 759-765, Istanbul, Turkey. Euro- pean Language Resources Association (ELRA).
David Caplan, Nancy Hildehrandt, and Gloria S. Wa- ters. 1994. Interaction of verb selectional restrictions, noun animacy and syntactic form in sentence process- ing. Language and Cognitive Processes, 94):5499 585.
Ariel Goldstein, Zaid Zada, Eliav Buchnik, Mariano Schain, Amy Price, Bobbi Aubrey, Samuel A. Nas- tase, Amir Feder, Dotan Emanuel, Alon Cohen, Aren Jansen, Harshvardhan Gazula, Gina Choe, Aditi Rao, Catherine Kim, Colton Casto, Lora Fanda, Werner Doyle, Daniel Friedman, Patricia Dugan, Lucia Mel- loni, Roi Reichart, Sasha Devore, Adeen Flinker, Liat Hasenfratz, Omer Levy, Avinatan Hassidim, Michael Brenner, Yossi Matias, Kenneth A. Norman, Orrin Devinsky, and Hasson Uri. 2022. Shared computa- tional principles for language processing in humans and deep language models. Nature Neuroscience, 25:369-380.
Alfonso Caramazza and Jennifer R. Shelton. 1998. Domain-specife knowledge systems in the brain: The caimate-inaniman distinction. J. Cognitive Neu- roscience, 10(1):1334.
Mariona Coll Ardanuy, Federico Nanni, Kaspar Beelen, Kasra Hosseini, Ruth Ahnert, Jon Lawrence, Kather- ine McDonough, Giorgia Tolfo, Daniel CS Wilson, and Barbara McGillivray. 2020. Living machines: A study of atypical animacy. In Proceedings of the 28th International Conference on Computational Linguis- fics, pages 4545, Barcelona, Spain (Online).

---

Adam Goodkind and Klinton Bicknell. 2018. Predictive power of word surprisal for reading times is a linear function of language model quality. In Proceedings of the 8th Workshop on Cognitive Modeling and Com- putational Linguistics (CMCL 2018), pages 10-18, Salt Lake City, Utah. Association for Computational Linguisties.
Danny Merkx and Stefan L. Frank. 2021. Human sen- tence processing: Recurrence or attention? In Pro- ceedings of the Workshop on Cognifive Modeling and Computational Linguisstii pages 12-22, Online. Association for Computational Linguistics.
James Michaelov and Benjamin Bergen. 2020. How well does surprisal explain n400 amplitude under dif- ferent experimental conditions? In Proceedings of the 24th Conference on Computational Natural Lan- guage Learning, pages 652 663, Online. Association for Computational Linguistics.
Yeb Havinga. 2021. GPT-2-medium- Dutchh.
Labiba Jahan, Geeticka Chauhan, and Mark Finlayson. 2018. new approach to Animacy detection. In Proceedings of the 27th International Conference on Computational Linguisties, pages 1-12, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
James A. Michaclov, Megan D. Bardolph, Seana Coul- son, and Benjamin K. Bergen. 2021. Different kinds of cognitive plausibility: why are transformers better than RNNs at predicting N400 amplitude? In Pro- ceedings of the 43rd Annual Meeting of the Cognitive Science Society, pages 300-306.
Jie Ji and Maocheng Liang. 2018, An animacy hierarchy within inanimate nouns: English corpus evidence from a prototypical perspective. Lingua, 205:7 1-89.
James A. Michaelov, Seana Coulson, and Benjamin K. Bergen. 2022. So cloze yet far: N400 amplitude is better predicted by distributional information than human predictability judgements, IEEE Transactions Cognitive and Developmental Systems, pages 1-1.
Folgert Karsdorp, Marten van der Meulen, Theo Meder, and Antal van den Bosch. 2015. Animacy Detec- tion in Stories. In 6th Workshop on Computational Models of Narrative (CMN 2015), volume 45 of Ope- nAccess Series in Informaties (OASlcs), pages 82- 97, Dagstuhl, Germany. Schloss Dagstuhl-Leibniz- Zentrum fuer Informatik.
James A. Michaelov, Seana Coulson, and Benjamin K. Bergen. 2023. Can peanuts fall in love with distribu- tional semantics? To appear in the Proceedings of the 45th Annual Meeting of the Cognitive Science Society (Sydney, Australia; 2023).
Carina Kauf, Anna A. Ivanova, Giulia Rambelli, Em- manuele Chersomi. Jingyuan S. She, Zawad Chowd- hury, Evelina Fedorenko, and Alessandro Lenci. 2022. Event knowledge in large language models; the gap between the impossible and the unlikely.
James S. Nairne, Joshua E. VanArsdall, Josefa N. S. Pandeirada, Mindi Cogdill, and James M. LeBreton. 2013. Adaptive memory: The mnemonic value of animacy. Psychological Science, 24(10):209--210
Karin Kipper, Hoa Trang Dang, and Martha Palmer. 2000. Class-based construction of a verb lexicon. In Proceedings of the Seventeenth National Confer- ence on Artificial Intelligence and Twelfth Confer- ence on Innovative Applications of Artificial Intelli- gence, page 691-696, AAAI Press.
Joshua New, Leda Cosmides, and John Tooby. 2007. Category-specific attention for animals reflects ances- tral priorities, not expertise. Proceedings of the Na- tional Academy of Sciences, 104(42):16998-16603
Mante S.
Nieuwland and Jos J. A. van Berkum. 2006. When peanuts fall in love: N400 evidence for the power of discourse. J. Cognitive Neuroscience, 18(7):1098-111
Susumu Kuno and Etsuko Kaburaki. 1977. Empathy and syntax. Linguistic inquiry, pages 627-672.
Belinda 7. Li, Maxwell Nye, and Jacob Andreas. 2021. Implicit representations of meaning in neural lan- guage models. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1181--887, Online. Association for Computational Linguistics.
Byung-Dof, Oh and William Schuler. 2023. Why Does Surprisal From Larger Transformer-Basee Language Models Provide a Poorer Fit to Human Reading Times? Transactions of the Association for Com- putational Linguistics, 1 11:336-350.
Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen. Tom Con- erly, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds Danny Hernandez, Scott Johnston, Andy Jones, Jack- son Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah, 2022. In-context learning and induction heads. Transformer Circuits Thread.
Kenneth Li, Aspen K Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. 2023. Emergent world representations: Exploring a sequence model trained on a synthetic task. In The Eleventh International Conference on Learning Representations.
Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. 2016. Assessing the ability of LSTMs to learn syntax- sensitive dependencies. Transactions of the Associa- tion for Computational Linguistics, 4:521-535.
Constantin Orasan and Richard Evans. 2007. NP ani- macy identification for anaphora resolution. J. Artif. Int. Res., 2911)799-

---

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf. Edward Yang, Zachary VeVito, Martin Raison, Alykhan Te- jani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang- Junjie Bai, and Soumith Chintala. 2019. Pytorch: An imperative style, high- aehh-erformance deep learning library. In H. Wallach, H. Larochelle, A. Beygelz- imer, F. d'Alché-Bue, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32, pages 8024-8035. Curran Associates, Inc.
Frank Wilcoxon. 1945. Individual comparisons by rank- ing methods. Biometrics Bulletin, 1(6):80-88 83.
Michael Wilson. 1988. MRC psycholinguistic database: Machine- dictionary. version 2.00. Behavior Research Methods, Instruments, & Computers, 20:6-
10. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- ric Cistac, Tim Rault, Remi Louf, Morgan Funtow- icz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jermite. Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush, 2020. Trans- formers: State-of of the-hhtt natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38 45, Online. Association for Computational Linguistics.
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners.
David Rakison and Diane Poulin-Dubois 2001. Devel- opmental origin of the animate-inanimate distinction. Psychological Bulletin, 127:209-228.
Anette Rosenbach. 2008.
Animacy and grammatical Poriation-fidding from english genitive variation. Lingua, 1 118(2)151-171 Animacy, Argument Struc- ture, and Argument Encoding.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher De- wan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mi- haylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. 2022. OPT: Open pre- trained transformer language models.
Arabella Sinclair, Jaap Jumelet, Willem Zuidema, and Raquel Fernández. 2022. Structural persistence in language models: Priming as a window into abstract language representations. Transactions of the Associ- ation for Computational Linguistics. 10:1031-1050.
A Implementation Details
We implement all experiments in PyTorch (Paszke et al., 2019). For all models but LLaMA, we use the implementationss and weights publicly available via the HuggingFace Transformers library (Wolf et al., 2020); for LLaMA, weights are only avail- able upon request via a form at https://gi thub. com/ htmm/aacebookreearcchh. hwwpcchhhllaaaaaaaaaa 1lama. We run models using an Nvidia A100 40GB GPU (or multiple when necessary, as for LLaMA 30B). The runtime of these experiments should not exceed 24 hours, even when run serially.
Nathaniel J. Smith and Roger Levy. 2013. The effect of word predictability reading time is logarithmic. Cognition, 128(3):302-319
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.1 2302.1197.
Marten van Schijndel and Tal Linzen. 2021. Single- stage prediction models do not explain the magnitude of syntactic disambiguation difficulty. Cognitive Sci- ence, e5 1 1988..
B N400 Results for Dutch Data
The methods for this set of experiments mostly identical to those of the English experiments. How- ever, instead of English LMs, we use Dutch LMs. We consider GPT-2 Small trained from scratch on Dutch, and GPT-2 Medium trained in English, with fine-tuned Dutch word embeddings (de Vries and Nissim, 2021), and GPT-2 Medium and Large trained from scratch on Dutch (Havinga, 2021). These represent the best Dutch autoregressive LMs available.
Joshua F. Arsall and Janell R. Blunt. 2022. Analyz- ing the structure of animacy: Exploring relationships among six new animacy and 15 existing normative dimensions for 1.200 concrete nouns. Memory & Cognition, 50:997-1012.
Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mo- hananey, Wei Peng. Sheng-Fu Wang, and Samuel R. Bowman. 2020. BLiMP: The benchmark of linguis- tic minimal pairs for English. Transactions of the Association for Computational Linguistics, 8:377- 392.
Ethan Gotlieb Wilcox, Jon Gauthier, Jennifer Hu, Peng Qian, and Roger Levy. 2020. On the predictive power of neural language models for human real-time com- prehension behavior. In Proceedings of the 42nd Annual Meeting of the Cognitive Science Society.
B.1 Results: Repetition Experiment
The results of the repetition experiment in Dutch (Figure 11) are rather similar to those in As before, the surprisal starts out high for both
English.

---

lucky peanut had big smile on his face. The peanut was amazed about his good fortune. Just now he had won the jackpot of two million dollars. The peanut was clated/salted and who could hlame him.
Dutch Repetition Experiment Surprisals
Inarimate Surprise
Surponal
20
Surpitaal (bits)
Figure 13: Story from Boudewyn et al. (2019), context experiment. N400 responses were recorded the words in bold.
Surprisal (bits) English Context Experiment Surprisals
(Emboudd riga)
20.0
Model and Timestep
17.5
Figure II: Repetition experiment surprisals. Surprisal drops rapidly after TI, with inanimate surprisal drawing close to animate surprisal.
125
10.0
Dutch Context Experiment
Animate Surpehal
Intrimate Surprial
Inanimate Surteiou
20.0
Animte Surpriest
25
Animate Bascline
Inarimato Bascline
Inanmmate Boselne
Anmate Baseline
CPE2
GP12
GPT-2
OFT
DPT
OFT
LLaMA
ULMMA
ILAMA
(bit)) (bits)
12.5
Figure 14: Surprisals for animate and inanimate adjec- tives in the normal and baseline condition
10.0
sured participants' N400 responses to adjectives at the end of each story (Figure 13). Each adjec- tive was either typical for animate entities (and thus context-approprrite) or typical for the inanimate en- tity in question (but context- context-inappropriate)) N400 responses were much lower in the former case than in the latter.
2.5
0.0
GPT-2 Small
Modium
Medium
Figure Context experiment surprisals. With context, the animate adjective is much less surprising; in the contextless baseline condition, this is reversed.
animate and inanimate entities (though higher for the latter). The difference between these two less pronounced than in English. The surprisal drops rapidly at T3, and again at T5. Unlike in English, there are no strong model-wise trends, whereby stronger models have lower difference in surprisals. And in all cases, the difference between the conditions T5 is statistically significant.
Experiment As in the earlier context experiment (Section 5.2), for each of the 120 stories, cal- culate the surprisal at the animate and inanimate adjective. We also compute baseline surprisals, de- fined as the surprisal of the critical adjective given only the sentence containing it as context.
The results of the English context ex- periment (Figure 14) are like those of the earlier context experiment (Section 5.2). Like before, sur- prisal at the animate adjective is much lower than surprisal at the inanimate adjective; in the baseline condition, this trend is reversed.
Results
B.2 Results: Context Experiment
Again, the results of the Dutch experiment (Fig- ure 12) are much like the English results. Surprisals animate adjectives
much lower than those inanimate adjectives; however, in the baseline con- dition, which lacks context, the trend is reversed.
D Further Low-Context Experiments
Boudewyn et al.: English Context Experiment
In the following experiments, we made changes our and experimental setup, in order ensure that our findings were not a result of any idiosynerasies of our dataset's 's construction.
Original Study Much like in Nieuwland and van Berkum (2006), Boudewyn et al. (2019) also mea-

---

D.1 Larger Human Entity Sample Pool Experiment
xun Mean Divergences by Model (Large Sample Pool)
PadtAlio
our original experiment, our human reference was A =TThe [human] began to", where [human] was sampled from a very limited pool: person, man, woman, boy, girl, and child. The pool was limited for two reasons. First, we did not want to introduce another axis of variation across examples.
KL Divegetce
Second, and more importantly, we wanted to create a generic "human-like action distribution. Some nouns (e.g- a musician, or thief) are ani- mate humans, but their distributions over next ac- tions are probably skewed in ways that are not representative of animate entities in general; musi- cians are likely to play music, and thieves, to steal. Thus, their next-token distributions might have high KL-ddivereence with those of atypically ani- mate objects for reasons unrelated to the object's perceived animacy. However, we still wanted to ensure that our findings are not reliant on this small pool of humans. To verify this, we constructed a larger pool of human entities to sample from.
GPT-2
GP1-2
GP1-2
QP1
UaKA
LLAMA
Model
Figure 15: KL Divergence between atypically animate (O) and animate (A) / inanimate (I) references, where the animate entity lower the bar, the similar the distributions.
drawn from larger pool. The
distribution. However, the animacy divergence is somewhat higher for the LLaMA models.
D.2 Matched Frequency Human Entity Experiment
In this experiment, we ensure that differences be- tween the frequency of the sampled human entity and that the inanimate entity do not undermine our experimental setup. Using the same frequency data before, we matched each inanimate entity in our pool to the (manually verified, valid) hu- man entity with the most similar frequency. We excluded inanimate object ("well") because its frequency was confounded by the very common adverb that shares its form. We were able to find a good human match for each inanimate object: the object frequency-to-hure humaann frequency ratio ranged from 0.92 to 1.09; this is near 1, the ideal ratio.
To construct this pool, we first added generic human entities (man, woman, person, boy, girl, child, teenager) and family relations (mother, fa- ther, grandfather, grandmother, wife, husband, grandchild, granddaughter, grandson, aunt, un- cle, niece, nephew, cousin). Then, we added more entities, starting from list of job titles, which are all naturally human nouns. We then used word frequency data from million words of Wikipedia (Goldhahn et al., 2012), filtering out any job titles that are not found that text. We then manually added valid job titles to our pool in order of de- scending frequency, until our expanded pool num- bered 100 entries: 21 generic human entities and 79 professions. We could have increased the pool size by including more jobs, but the job title pool noisy, and needed manual filtering weed out nonsensical jobs, and jobs that sound too much like inanimate objects.
Then, we conducted our experiment again as in Section 6, using this larger pool. The results of this experiment (Figure 16) strikingly similar those of the previous experiment (Figure 15). We take this to suggest that frequencies are indeed not very important to the phenomenon we observe.
Then, we conducted our experiment again as in Section 6, using this larger pool. The results of this experiment (Figure 15) are rather similar that of our original experiment. In all cases the output distribution po is clearly more animate than Pr: the OPT models, the Po is more like the animate (p red) than inanimate (pr, blue)
D.3 Cataphoric Prompt Experiment
In our original experiment, we test sentences of the form O "TThe chair spoke and began to".. One potential concern is that LMs might just look at "spoke and began to"., which implies ani- mate continuation, thus overcoming the inanimacy of chair. We can test this by using a cataphoric prompt, where a referring pronoun comes before the object, e.g. O' -'After it spoke, the chair began to". In such a prompt, there now exists the 4-gram
"The job titles originate from https://githu/ http:///gituuuccooo// meiidel/jabb htt/:/bbbbileeeee www.ttt/tteeeeaattteseee which collects job ti- tles from a variety of sources.

---

-00-
Divergences (Matched Frequency)
E Low-context Animacy: Evaluation
We first tried to classify potential next tokens as ani- mate, inanimate, or neither; we could then compute the probability assigned to cach group. However, classifying all next tokens was noisy: resources like FrameNet or VerbNet (Baker et al., 1998; Kipper al., 2000), it was infeasible to deter- mine if verb implied that its subject was animate.
PaulAlii
with
F Low-Context Animacy Dataset Details
UP122
GPI-
GPT1-2
GP1-4
091
QP1
+400
LaakA
LLKKA
This section contains lists of the prompts, nouns, and verbs used in constructing this dataset. The full dataset, and all variants thereof, be found at https://ge thub. com com/hannamw/lms-- love.
Model
Figure 16: KL Divergence between atypically animate (O) and animate (A) / inanimate (I) references, where the animate entity A is matched frequency with the inanimate entity in / and O. The lower the bar, the more similar the distributions.
F.1 Prompts
The [noun] [verb] and began to
The [noun] [verb] and started to
Divergences (Cataphor Prompt)
The [noun] [verb] and was very
The [noun] [verb] and became very
F.2 Nouns
accordion, ambulance, amplifier. appliance, arrow. automobile, axe, bagpipe, balloon, bandage, ban- ner, barrel, basket, bin, biscuit, blanket, blossom, blouse, boat, bomb, book, bottle, bouquet, bra, bracelet, bread, brush, bubble, bucket, buckle, bul- let, button, cake, camera, candle, candy, cane, non, canoe, cape, cart, casket, chisel, chocolate, clarinet, clock, clothing, coat, cocktail, coffin, coin, collar, corpse, dagger, dart, desk, dime, dress, en- gine, envelope, ferry, fiddle, firewood, flask, football, fruit, furniture, glass, glove, goblet, gown, hailstone, hairpin, hammer, harp, hat, helmet, hose, jar, keg, kilt, knife, lamp, lantern, lens, limou- sine, mallet, map, mattress, medallion, microscope. mirror, missile, moccasin, nail, napkin, necklace, needle, nickel, nightgown, oar, ornament, oven, overcoat, pants, pearl, pencil, pendulum, penny, phone, photograph. piano, pie, pillow, pipe, plank, pot, propeller, prune, purse, quilt, radio, record, refrigerator, ribbon, rifle, ring, rope, rug, sandal, satchel, saxophone, scissors, scroll, shawl, shield, shirt, shoe, ski, skull, sleigh, sock, sofa, spoon, statue, steak, stove, submarine, sword, tablespoon, telephone, telescope, thermometer, thorn, thread, ticket, tie, timepiece, tractor, tray, tripod, trombone. truck, trumpet, tube, tweezers, twig, typewriter, umbrella, van, vase, vehicle, vest, violin, wallet, wheel, whistle, wig, yacht, zipper,
GPT-2
CFT-33
GPT-2
OCT
OFT
DPT
11000 LLAMA
L13MA
Model
Figure 17: KL Divergence between atypically animate (O) and animate (A) inanimate (/) references, using the cataphoric prompt. The lower the har, the more similar the distributions.
flute,
"the chair began to",, which should make this task a little harder for LMs. Using the less-animate / inanimate pronoun "it" to refer the atypically animate object also makes this more challenging.
Then, we conducted our experiment again as in Section 6, using the original small pool of human entities. The results of this experiment (Figure 17) suggest that this setting indeed harder for LMs; no longer any LMs adapt such that divergence between Po and PA (red) is less than that between PO and Py (blue). However, there still increase in animacy compared to the case where the prompt does not hint at atypical animacy (purple).

---

F33
Verbs
F.3.1 Physical
High Co-Occurrence With Human Subjects stammer, grimace, mumble, drawl, frown, gasp, yell, nod, smile, laugh, shrug, sob, grin, kneel, wince, whisper, sigh, giggle, squint, murmur, doze, fiddle, gesture, mutter, faint, gulp, flinch, chuckle, drink, weep, stare, grunt, listen, watch, fumble, shiver, pace, lean, blush, shout, gaze, walk, sit, sleep, dine, pant, glare, clap, stumble, snore, shave, wave, omit, sniff, piss, cough, wail, grumble, breathe, snort, spit, eat, duck, die, swallow, growl, blink, inhale, bellow, starve, crouch, yawn, step, squat
High-Mid Co-Occurrence With Human Sub- jects pounce, scream, flee, shudder, wander, shriek, stagger, wink, sing, whistle, jog, limp, hiss, trot, jump, bathe, dance, paint, ramble, shower, drown, recover, pack, sweat, bow, flush, crawl
Mid Co-Occurrence With Human Subjects bark, swim, bleed, howl
F.3.2 Psychological
High Co-Occurrence With Human Subjects think, know, wonder, remember, guess, exclaim, retort, marry, notice, understand, hurry, pray, medi- tate, swear, forget, enquire, realise, confess, apolo- gise, hesitate, suspect, reply, talk, sneer, cry, moan, ponder, revel, learn, scowl, retire, snarl, groan, speak, complain, beg, wait, preach, grieve, read, plead, volunteer, answer, curse, choose, panic, chant, cheat, salute, emigrate, protest, visit, lament, misunderstand
dream,
High-Mid Co-Occurrence With Human Sub- jects consent, graduate, disagree, steal, mourn, study, argue, search, insist, practise, interrupt, obey, comment, concede, fight, upplaud, enlist, worry, teach, train, agree, struggle, rush, evacuate, object, pay. pursue, hasten
Mid Co-Occurrence With Human Subjects
vote, invest, register


Reviews:
[{"review_id": "FrkO5qCjLv", "reviewer": "Reviewer_jHyH", "summary": "", "questions": "\nA: Can you slightly lower the tone of the conclusion?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review! We’re happy to hear you found the paper enjoyable, interesting, and clear. We agree that the conclusion could be toned down—while LMs performed well on our experiments, these necessarily represent only a limited selection of the many ways in which animacy can manifest in language. We will revise the paper to make this nuance clearer.\n\nWe’ll also take a closer look at the colors chosen for our graphs, and will consider adding symbols that make them more legible when printed in black and white. We will also add the requested details, with respect to e.g. a/typical animacy.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "\nI enjoyed reading the paper. The topic is interesting, and the writing is good.\nEach experiment is described clearly and contains relevant and interesting discussions. \n\nI liked the use of existing tests from the literature and the comparison with human results.", "reasons_to_reject": "\nDespite the findings and results of the experiments, the conclusion seems to me to be formulated in a slightly too strong way.\n\nI would formulate this argument (\"LMs generally respond to animacy like humans\") with more caution, emphasizing that it holds in the experiments performed and not as a general conclusion (after all, there could possibly be other experiments in which a different behavior might emerge, right?)", "questions_to_authors": "\nA: Can you slightly lower the tone of the conclusion?", "paper_topic_and_main_contributions": "When Language Models Fall in Love: Animacy Processing in Transformer Language Models\n\nThis paper uses a series of experiments in order to compare human and LMs reactions to animacy processing.\nThe authors used existing experiment designs for humans (with recorded results) to test LMs.\nThe authors used existing datasets and translated them from Dutch to English.\nThe authors also added their own experiments.\n\nFinally, the authors conclude that according to the experiments, LMs generally respond to animacy like humans.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "\nYou might want to choose colors that will also suit the color blind (or for printing in black and white)\n\nline 104: \"not only not only\"\n\nIt would have been helpful if there had been more detail in the background about the tests themselves - what are they testing? when humans use them? what are their motivations? what was their purpose? and little clearer/detailed explanations in the introduction regarding what is atypical/typical animacy"}}, {"review_id": "iWWQ1UCc4K", "reviewer": "Reviewer_9t4f", "summary": "", "questions": "* While the focus in this paper is clearly English, given that the original study was performed in Dutch, it might be interesting to know a bit more about animacy processing in other languages?\n\n* What about words that are ambiguous at the type level, e.g. metonyms like \"The White House\"? Would be interesting to compare their processing to the clear-cut cases used in this study.\n\n* Who performed the psycholinguistic experiments under \"Adaptation Experiment\"? This is unclear from the text, as it can be read as being the authors and not Nieuwland and van Berkum. Please clarify.\n\n* For the low-context scenario, it seems unfortunate that the human entity for the A reference is randomly sampled, rather that being matched in terms of frequency with the inanimate noun. Did you consider this? Could frequency be influencing the processing here? \n", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review! We’re glad that you found the interdisciplinary aspects of our work to be clearly presented. We answer your questions below, point by point.\n\n> the background section is missing previous work on animacy detection, which, while somewhat different from the current experimental setup should still be mentioned (see below)\n\nIt’s true that our related work section omits animacy detection; this was cut in favor of discussing animacy and behavioral interpretability. If accepted, we’d be happy to use the extra page given to camera-ready papers to include the papers you’ve cited.\n\n> Who performed the psycholinguistic experiments under \"Adaptation Experiment\"? This is unclear from the text, as it can be read as being the authors and not Nieuwland and van Berkum. Please clarify.\n\nThe original human adaptation experiment comes from Boudewyn et al. (2019), while the two prior experiments (repetition and context, in Sections 5.1 and 5.2) come from Nieuwland and van Berkum (2006). We will revise our paper to make this even more explicit!\n\n> While the focus in this paper is clearly English, given that the original study was performed in Dutch, it might be interesting to know a bit more about animacy processing in other languages?\n\nWe look into Dutch models in Appendix B. Trends for Dutch models are quite similar to trends for English models: they show a large reduction in surprisal at the 3rd timestep (repetition experiment, 5.1/B.1), and prefer the animate over inanimate adjective (context experiment, 5.2/B.2). In general, higher-performing Dutch LMs showed larger reductions in surprisal, and thus more human-like performance. However, no Dutch LMs were able to replicate phenomena such as the lack of difference between animate / inanimate surprises in the repetition experiment. We attribute this to the lack of truly high-performing Dutch autoregressive LMs, given that better models yielded more human-like surprisals. \n\nWith respect to human animacy processing, the similarity between the findings of Nieuwland and van Berkum (Dutch) and Boudewyn et al. (English) suggests that in this case, humans also process atypical animacy in a similar way across the two languages.\n\n> What about words that are ambiguous at the type level, e.g. metonyms like \"The White House\"? Would be interesting to compare their processing to the clear-cut cases used in this study.\n\nThis is interesting! Intuitively, it seems like the degree to which such a word is treated as animate should be related to the frequency with which it is used in its metonymic sense. It would be valuable to investigate if models exhibit this sort of animacy gradient. This question deserves a thorough investigation which is outside the scope of our current work; however, we would love to see further exploration of this topic, and hope that future work can build on ours to answer it.\n\n> For the low-context scenario, it seems unfortunate that the human entity for the A reference is randomly sampled, rather that being matched in terms of frequency with the inanimate noun. Did you consider this? Could frequency be influencing the processing here?\n\nWe generally don’t think that frequency is influencing processing in this scenario. If you were suggesting that, for each inanimate entity, the corresponding human entity be set to a human entity with the same frequency, this seems feasible. However, we don’t think the frequency of the human entity, relative to that of the inanimate noun, should affect our results. The human and inanimate entities never appear in the same sentence; we only compare their distributions. So, any frequency effects would be somewhat indirect.\n\nWe run such experiments just to be sure, and find that the results in the matched and unmatched cases are the same. Our procedure was as follows: we constructed a list of humans from a list of professions, as well as a handcrafted list of common family relations. Then, we filtered out those humans that didn’t appear in our frequency corpus. We excluded one inanimate object (well) because its frequency was confounded by the very common adverb that shares its form. We were able to find a good human match for each object—the object freq.-to-human freq. ratio ranged from 0.92 to 1.09, near 1, which is the ideal ratio. \n\nWe then ran the low-context experiments in both the matched case, where each object was compared with its paired noun, and the unmatched case, where each object was paired with a randomly sampled noun. Note that the pool from which samples were taken in the unmatched case, contains only those human entities that were matched with an object in the matched case. Thus, the overall pool of human entities is the same; only the matching differs. The results of the two experiments are the same, and align with our original experimental results.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper is very well-written and clear, motivating each study well and guiding the reader through the results in an admirable manner. \n- The paper further navigates its interdisciplinary setting very well, by explaining the psycholinguistic studies and describing in enough detail its adaptation to LM evaluation. \n- The paper makes several contribution both in terms of datasets (which they will release) and experimental results.", "reasons_to_reject": "There are only a few minor issues: \n- the background section is missing previous work on animacy detection, which, while somewhat different from the current experimental setup should still be mentioned (see below)\n- it is unclear exactly who performed the final psycholinguistic experiment, might just be a missing reference, but should be easy enough to clarify by the authors", "questions_to_authors": "* While the focus in this paper is clearly English, given that the original study was performed in Dutch, it might be interesting to know a bit more about animacy processing in other languages?\n\n* What about words that are ambiguous at the type level, e.g. metonyms like \"The White House\"? Would be interesting to compare their processing to the clear-cut cases used in this study.\n\n* Who performed the psycholinguistic experiments under \"Adaptation Experiment\"? This is unclear from the text, as it can be read as being the authors and not Nieuwland and van Berkum. Please clarify.\n\n* For the low-context scenario, it seems unfortunate that the human entity for the A reference is randomly sampled, rather that being matched in terms of frequency with the inanimate noun. Did you consider this? Could frequency be influencing the processing here? \n", "paper_topic_and_main_contributions": "The paper presents a study of animacy processing in large language models, comparing it to human processing as evidenced by psycholinguistic studies. The paper analyzes animacy processing in openly available datasets, such as BLIMP, however, the main contribution is found in the adaptation of several Dutch psycholinguistic studies to evaluation by English LMs.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "Previous work on animacy detection, e.g. \n\n- Ardanuy et al (2020) who focus on atypical animacy\n- Jahan et al (2018)", "Typos_Grammar_Style_and_Presentation_Improvements": "not only not only -> not only"}, "extra_text": {}}, {"review_id": "DS71UnLbE6", "reviewer": "Reviewer_w2ux", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your thoughtful review! We’ve answered your questions below.\n\n> First, I'm not sure why the expression of animacy in text is called 'muted' (in the introduction) and 'indirect' (in the conclusion). Animacy is a major feature in causatives, for example, so I was not surprised to see it is learnable from data.\n\nOur main motivation for describing animacy as “indirect” in English is that it isn’t explicitly morphologically marked, in contrast with other phenomena (e.g. number), or animacy in other languages (e.g. Slavic languages). We will clarify this in the introduction!\n\n> Second, examples like 'The boat snored and started to _' in the third experiment might be confounded by the recency of the conjoined animate verb (in this case, 'snored'). I wonder if this could be mitigated by using cataphoric stimuli like 'After it snored, the boat started to _' that place the inanimate noun closer to the mask.\n\nThis is an interesting question! We performed preliminary tests on this new prompt and found that our results still hold: animacy divergence is still reduced, compared to divergence between the animate and inanimate distributions. The effect size is smaller, but overall trends, such as stronger language models having generally lower animacy divergence are the same. We found the same results for the prompt “After snoring, the boat started to”, which we tested to avoid the use of “it”, which could have suggested a lower level of animacy than “he” or “she”, and cued LMs towards an inanimate continuation. We’ll add these new results to the paper!\n\n> Finally, no significance testing was reported for results in Section 4.\n\nThis is true: we did not report significance tests. If you are referring to significance tests between model and human performance, this is due to a lack in human data. BLiMP’s human annotations are sparse: for each dataset (passive and transitive), 5 examples are annotated, with 20 human judgments each. So, paired statistical tests are impossible, and even the human accuracy metric is based on a very small number of examples, making it less than reliable. For this reason, we forgo statistical tests, and only broadly check that model and human performances are similar, before moving on to the main contributions of our paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper considers an interesting semantic question and uses what seems to me to be a reasonable experimental design to evaluate it.", "reasons_to_reject": "I only have a few minor issues.\n\nFirst, I'm not sure why the expression of animacy in text is called 'muted' (in the introduction) and 'indirect' (in the conclusion).  Animacy is a major feature in causatives, for example, so I was not surprised to see it is learnable from data.\n\nSecond, examples like 'The boat snored and started to _' in the third experiment might be confounded by the recency of the conjoined animate verb (in this case, 'snored').  I wonder if this could be mitigated by using cataphoric stimuli like 'After it snored, the boat started to _' that place the inanimate noun closer to the mask.\n\nFinally, no significance testing was reported for results in Section 4.\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper describes experiments showing large language models seem to be able to detect animacy in text, even with very little information.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "* The word 'entity' is repeated in Section 5.3.\n* In Section 6, I had trouble finding the antecedent of 'they' in the sentence beginning 'We find that psychologically animate verbs ...'.\n"}, "extra_text": {}}, {"review_id": "Amkv4PHsEN", "reviewer": "Reviewer_QnBL", "summary": "", "questions": "Line 304: What does it mean to make stimuli cartoon-like?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your careful review! We’re happy to hear you found our work novel, and our presentation clear. Below, we’ve answered your two main points.\n\n> The significance testing in Section 5.1 (described starting on line 321) seems sort of backwards. The real goal is to see if there is evidence that animate and inanimate surprisal are the same -- this would show that LMs are doing something special -- but the Wilcoxon test sets this as the null hypothesis. A negative result from a significance test does not support the null hypothesis. AFAIK, the proper kind of test to do here would be an equivalence test, where the alternative hypothesis would be that the surprisals match and the null hypothesis would be that they differ. The same concern applies to Section 5.3.\n\nThis is an interesting point. In principle, you are correct: the Wilcoxon test can only (fail to) reject the null hypothesis of equivalence; an equivalence test is needed to show that the two conditions yield the same result. However, the original human studies do not use equivalence tests. Instead, they too fail to reject the null hypothesis, concluding there is no statistically significant difference between the two conditions. \n\nOur study builds on the originals by comparing LM results to those of humans. Lacking results of equivalence tests run on human data, we would have nothing to compare LM equivalence test results to. For this reason, we did not perform such tests.\n\nWe could perform equivalence tests anyway, but we run into a practical problem. If we take the two one-sided tests (TOST) approach to equivalence testing, we ought to set bounds on the difference between our two conditions. Ideally, these bounds would reflect the bounds used by the corresponding tests on human data, but we have none. Alternatively, we should set our bounds such that any difference falling within the bounds is not meaningful, but what this means in terms of surprisal is not obvious. We could get around this by using something like Cohen’s d to determine the magnitude of surprisal difference that constitutes a meaningful effect size; however, this is complicated by the fact that Cohen’s d relies on a normality assumption, which our data violate.\n\nIn the interest of exploring equivalence tests, we performed equivalence tests (TOST, Wilcoxon signed-rank), and found that equivalence holds for some reasonable bounds, chosen to exclude small effect sizes as measured by Cohen’s d. However, the results always depend on the precise bounds chosen. We thus do not want to make any claims about equivalence, or how this might speak to LMs’ processing of animacy with respect to humans’.\n\nThis was a well explained and subtle point, so we thank you for raising it. If you have a clearer idea of how to choose bounds, or how to link equivalence test results to the prior human results, please let us know!\n\n> Line 304: What does it mean to make stimuli cartoon-like?\n\nCartoon-like stimuli is a phrase used by Nieuwland and van Berkum (2006). It refers to the fact that the original stimuli were humorous stories about concrete inanimate objects that are alive, as might often happen in a cartoon; we aimed to maintain these qualities. We’ll revise the paper to clarify this.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This paper addresses a fresh topic that is interesting from both the vantage point of cognitive modeling and BERTology. The experiments are presented very clearly, and do well at anticipating potential pitfalls, e.g. that LMs might show decreased surprisal merely due to lexical repetition. The overall finding that LMs respond in humanlike ways to atypical animacy seems like a novel, non-trivial result.", "reasons_to_reject": "The significance testing in Section 5.1 (described starting on line 321) seems sort of backwards. The real goal is to see if there is evidence that animate and inanimate surprisal are the same -- this would show that LMs are doing something special -- but the Wilcoxon test sets this as the null hypothesis. A negative result from a significance test does not support the null hypothesis. AFAIK, the proper kind of test to do here would be an equivalence test, where the alternative hypothesis would be that the surprisals match and the null hypothesis would be that they differ. The same concern applies to Section 5.3.", "questions_to_authors": "Line 304: What does it mean to make stimuli cartoon-like?", "paper_topic_and_main_contributions": "This paper explores how transformer models process animacy. While previous work has shown that LMs are sensitive to type-level animacy, this work looks at cases with atypical token-level animacy, e.g. an oar that is treated as animate. Previous EEG data shows that humans are able to quickly adapt to these cases of atypical animacy, and this work shows that LMs also adapt quickly in similar cases.", "excitement": 5, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a study of animacy processing in LLMs, analogizing it to human processing as evidenced by psycholinguistic studies. The paper analyzes animacy processing in openly available datasets, such as BLIMP. However, the main contribution is adapting several Dutch psycholinguistic studies to evaluation by English LMs.

The reviewers unanimously gave very positive feedback, which I summarise in the following lines:

** Reasons To Accept: **

- The experiments are described clearly and contain relevant and interesting discussions.
- The paper further navigates its interdisciplinary setting very well by describing the psycholinguistic investigations and describing in enough detail its adaptation to LM evaluation.

** Reasons To Reject: **

- The background section is missing previous work on animacy detection.
- Section 5 as mentioned by the third reviewer, should be refined to have a very competitive paper.
Decision: Accept-Main