Cognition 178 (2018) 147–161

Contents lists available at ScienceDirect

Cognition
journal homepage: www.elsevier.com/locate/cognit

Original Articles

Extremely costly intensiﬁers are stronger than quite costly ones⁎

T

Erin D. Bennett , Noah D. Goodman
Department of Psychology, Stanford University, 450 Serra Mall, Building 420, Stanford, CA 94305-2130, USA

A R T I C LE I N FO

A B S T R A C T

Keywords:
Intensiﬁers
Degree adverbs
Scalar adjectives
Pragmatics
M-implicature

We show that the wide range in strengths of intensifying degree adverbs (e.g. very and extremely) can be partly
explained by pragmatic inference based on diﬀering cost, rather than diﬀering semantics. The pragmatic theory
predicts a linear relationship between the meaning of intensiﬁers and their length and log-frequency. We ﬁrst
test this prediction in three studies, using two diﬀerent dependent measures, ﬁnding that higher utterance cost(i.e. higher word length or surprisal) does predict stronger meanings. In two additional studies we conﬁrm thatthe relationship between length and meaning is present even for novel words. We discuss the implications foradverbial meaning and the more general question of how extensive non-arbitrary form-meaning association may
be in language.

1. Introduction
How do diﬀerent words get their meanings? For instance, why is an“extremely good paper” better than a “quite good paper”? The traditional answer (De Saussure, 1916) is that diﬀerent meanings have been
arbitrarily and conventionally assigned to the diﬀerent word forms.
This view has been challenged by a number of examples in which word
meaning appears to be non-arbitrarily related to properties of the word.
In some cases, the phonetic form of a word is systematically related toits meaning, for example rounded vowels and voiced consonants tend torefer to round objects (Davis, 1961; Holland & Wertheimer, 1964;
Khler, 1970; Ramachandran & Hubbard, 2001). In other cases, orthographic form is diagnostic of meaning, for example, speakers of Hebrew
who have never seen Chinese characters are nonetheless above chanceat matching them to their corresponding Hebrew words (Koriat & Levy,
1979). Similarly, the length of words predicts aspects of their meanings:
across languages longer words refer to more complex meanings (Lewis,2016). Open questions remain about the systematic factors that caninﬂuence meaning and the source of these eﬀects.
In this paper, we explore adjectival intensiﬁers,1 like extremely and quite,
as a case study in which to empirically explore the relationship of meaning
to factors like word form and distribution of usage. Intensiﬁers form a goodcase study because they are amenable to simple quantitative measures of⁎meaning: Many adjectives correspond to concrete numeric scales, and an
intensiﬁer’s strength can be measured as the numeric extent to which it
shifts the interpretation of such a scalar adjective. Intensiﬁers are of interest
because theoretical considerations, which we lay out below, suggest a relationship between intensiﬁer meaning and their communicative cost (i.e.
frequency and length). This account of intensiﬁer meaning adds to a
growing body of literature exploring how principles of recursive, rational
communication shape language interpretation (e.g. Bergen, Levy, &Goodman, 2014; Franke, 2011; Frank & Goodman, 2012; Goodman &Stuhlmüller, 2013; Grice, 1975; Kao, Wu, Bergen, & Goodman, 2014b;
Russell, 2012).
In the next section, we discuss a minimal semantics for intensiﬁers,
building oﬀ of previous work on scalar adjectives. We show how
pragmatic eﬀects predict systematic variation in the meanings of intensiﬁers: the meanings of intensiﬁers are expected to be inﬂuenced bytheir form (in length) and their distribution (frequency) of usage. We
formalize this semantics in Appendix A, and derive the prediction thatthe interpreted strength of an intensiﬁed phrase should be linearly related to communicative cost (i.e. length and frequency) of that phrase.
The impact of word length is reminiscent of the results of Lewis (2016),
who studied noun categories. While word frequency is known to have
major eﬀects on sentence processing (e.g. Levy, 2008), the predictionthat frequency should aﬀect meaning is more surprising.

Corresponding author.
E-mail addresses: erindb@stanford.edu (E.D. Bennett), ngoodman@stanford.edu (N.D. Goodman).
1
Intensiﬁers are adverbs that modify scalar adjectives so that the interpretation of the intensiﬁed adjective phrase is more extreme than the interpretation of the bare adjective phrase.
The word “intensiﬁer” is often used to denote the full range of degree adverbs, be they “ampliﬁers”, or “downtoners” (Quirk, Greenbaum, Leech, & Svartvik, 1985). The “intensiﬁers” weare looking at in this paper are, according to this typology, “ampliﬁers” because they increase (rather than decrease) the threshold associated with a gradable predicate. This typology also
distinguishes between two diﬀerent kinds of ampliﬁers: those that increase an adjective maximally (e.g. completely and utterly) and those that merely increase (e.g. greatly and terribly). We
do not make this distinction. The word “intensiﬁer” is sometimes used for a completely diﬀerent linguistic phenomenon, where a reﬂexive is used for emphasis, e.g. “The king himself
gave the command,” which we do not analyze in this paper.
https://doi.org/10.1016/j.cognition.2018.05.011
Received 24 August 2015; Received in revised form 10 May 2018; Accepted 17 May 2018
0010-0277/ © 2018 Elsevier B.V. All rights reserved.

Cognition 178 (2018) 147–161

E.D. Bennett, N.D. Goodman

We conﬁrm, in our ﬁrst series of studies (Studies 1a, 1b, and 2), that
English intensiﬁers in adjective phrases are indeed interpreted as muchstronger for less frequent intensiﬁers. This holds in quantitative judgments of meaning and in forced comparisons, and across a number ofadjectival dimensions. With the more sensitive dependent measure of
Study 2, we also ﬁnd an additional eﬀect of length above and beyond
surprisal. In our second set of studies (Studies 3 and 4), we replicate thisﬁnding, and extend it to novel intensiﬁers, showing that length is asigniﬁcant predictor of the strength of an intensiﬁer’s meaning even in
the absence of any conventional meaning. We conclude with a discussion of diﬀerent interpretations of these phenomena and future directions.

2.1. Intensiﬁcation as an M-implicature
We explore the idea that an adjective phrase with an intensifying degree
adverb derives much of its meaning from a M(arkedness)-implicature(Levinson, 2000): more marked (costly to utter) versions of an adjective
phrase will be interpreted as implicating higher values (e.g. in case of theadjective expensive, higher prices). Given two possible utterances a speakercould say to communicate the same meaning, a speaker will usually choose
the less costly utterance. If the speaker instead chooses a more costly utterance (e.g. “I got the car to start” as opposed to “I started the car”), they
may be doing so in order to communicate something more distinct, intense,
or unusual (e.g. “I got the car to start, but it was unusually diﬃcult”). In
other words, the marked form corresponds to the marked meaning. If scalaradjectives include a free threshold variable inferred from context, then the
speaker’s use of a longer, intensiﬁed adjective phrase could lead the listenerto infer that the threshold for this adjective phrase is unusually extreme
relative to other, less costly phrases that the speaker could have used.
To realize such an M-implicature, we suggest extending (Lassiter &Goodman, 2013)’s probabilistic model of scalar adjective interpretationslightly. We assume that each time a scalar adjective is used, in each phrase,
it introduces a free threshold variable—a new token threshold is inferred foreach access of the lexical entry of the adjective. The set of thresholds, for theactual sentence and all alternative sentences, is then established by apragmatic inference that takes into account the diﬀering costs of the sentences. The intensiﬁers themselves do not contribute to the semantics but
increase the cost of the utterance, thus aﬀecting pragmatic inferences. Thismodel is described in detail in Appendix A. As in previous RSA models that
include utterances with similar semantics but diﬀerent costs (Bergen et al.,2014; Bergen, Goodman, & Levy, 2012), we ﬁnd an M-implicature, suchthat more costly intensiﬁers result in stronger adjective phrases. As illustrated in Appendix A this relationship is expected to be approximately
linear, resulting in a straightforward quantitative hypothesis that we evaluate against empirical data in our studies.
We view this model as an illustrative caricature of intensiﬁer
meaning: In this model intensiﬁers contribute nothing to the literal,compositional semantics. Yet, pragmatic interpretation yields a spectrum of eﬀective meanings for the intensiﬁers, determined by theirrelative usage costs. This predicts an empirically testable systematic
variation in meaning as a function of cost. It is very likely that themeaning of individual intensiﬁers includes idiosyncratic, conventional
aspects in addition to these systematic factors. This would be expected
to show up as residual variation not predicted by cost, but would notnullify the hypothesized relationship between cost and meaning. Thisaccount applies straightforwardly only to intensifying degree adverbs;“de-intensifying” adverbs that eﬀectively lower the threshold will require further work to explain.

2. The semantics of intensifying degree adverbs
Our paper focuses on intensifying degree adverbs applied to scalar adjectives.2 Scalar adjectives have been described as having a threshold semantics (Kennedy, 2007), where, for example, expensive means “having a
price greater than θ ” and θ is a semantic variable inferred from context (e.g.,$100). Above the threshold degree θ , the adjective is true of an object, andbelow, the adjective is false. Lassiter and Goodman (2013) build on the Rational Speech Acts (RSA) framework (Frank & Goodman, 2012; Goodman &Stuhlmüller, 2013) to give a formal, probabilistic model of how this threshold
might be established by pragmatic inference that takes into account statisticalbackground knowledge (such as the distribution of prices for objects). We
return to this model below and present a full model in Appendix A.
Previous researchers have proposed that adjective phrases modiﬁedby intensiﬁers have the same semantics as unmodiﬁed adjective
phrases, except with new, higher thresholds (Kennedy & McNally,
2005; Klein, 1980; Wheeler, 1972). That is, some threshold, inferredfrom context, exists above which objects are expensive and below which
they are not, and the intensiﬁer very determines a new, higher thresholdfor the adjective phrase very expensive. These researchers suggest thatthe intensiﬁed thresholds are determined by ﬁrst collecting the set ofobjects in the comparison class for which the bare adjective is true, andthen using that as the comparison class to infer a new threshold, i.e.
very expensive laptop means “expensive for an expensive laptop”. This
analysis results in the expected intensiﬁcation of adjectives (“expensive
for an expensive laptop” has a higher threshold for being true thansimply “expensive for a laptop”) and is appropriately sensitive to different domains (e.g. the absolute diﬀerence in price between thresholds
for expensive and very expensive is much higher in the context of “Thatspace station is very expensive,” than in the context of “That coﬀee isvery expensive.”). However, this proposal does not distinguish between
the graded strengths of diﬀerent intensiﬁers, for example, very expensive
and phenomenally expensive.
Intuition suggests that diﬀerent intensiﬁers do have diﬀerent
strengths (e.g. outrageously seems stronger than quite), and we provide
further evidence of this in our studies, where participants interpret andcompare diﬀerent intensiﬁers. It could be that the degree of strength ofdiﬀerent intensiﬁers is conventionally speciﬁed by the lexicon. But thesemantics must then specify how these entries aﬀect the very ﬂexible
threshold of the relevant adjective. In addition, the multitude of intensiﬁers (Bolinger, 1972) and their apparent productivity3 suggest amore parsimonious solution would be welcome. That is, having alexically determined meaning for each diﬀerent intensiﬁer mightoverlook the similarity among words of this class. In the account that
follows, we build minimally on existing models of adjective interpretation and rational communication to articulate a model of intensiﬁed adjective phrase interpretation.

2.2. Factors aﬀecting utterance cost
We have identiﬁed an intensiﬁer’s cost as a potentially critical determiner of its interpreted meaning. To connect this prediction to empirical facts, we still must specify (at least a subset of) the factors we
expect to impact cost. The most natural notion of cost is the eﬀort a
speaker incurs to produce an utterance. This could include cognitiveeﬀort to access lexical items from memory, articulatory eﬀort to produce the sound forms, and other such direct costs. Speakers might alsoseek to minimize comprehension cost for their listeners, resulting inother contributions to cost. For the purposes of this paper, we restrict
ourselves to the most obvious contributors to production cost and use
proxies that are straightforward to quantify: length (longer utterances
are more costly)4 and frequency (rarer intensiﬁers are harder to retrieve

2
Some of these intensiﬁers can also apply to verbal and nominal predicates, and different restrictions apply for diﬀerent intensiﬁers, e.g. I truly like carrots is an acceptableutterance, whereas I very like carrots is not. See Bolinger (1972) for a discussion.
3
For example, altitudinously expensive is not in common usage, but one can easily interpret altitudinously as a novel intensiﬁer.

4
We measure length in number of syllables, although length in characters (whichmight be a more relevant source of utterance cost in a written format) has similar predictive power to syllable length in all of our analyses.

148

Cognition 178 (2018) 147–161

E.D. Bennett, N.D. Goodman

from memory in production and therefore more costly). In a number ofdiﬀerent tasks, lexical frequency aﬀects diﬃculty in an approximately
logarithmic way. For instance, word recognition time (McCusker, 1977)
and reading time in context (Smith & Levy, 2013) are both logarithmic
in frequency. We thus use the log-frequency (whose negative is also
called surprisal) as the quantitative contribution to cost.
Our model predicts a linear contribution of longer and higher surprisal
intensiﬁers to the meaning of an adjective phrase (see Appendix A for moredetail). This leaves open the relative importance of length and surprisal (aswell as other factors that might enter into cost), which we explore in ourstudies. For interpreting the results of these studies, we use mixed linear
models, since further quantitative comparison between the pragmatic account and the data would simply be overﬁtting.

Fig. 1. Screenshot from Study 1a target question.
Table 1
Intensiﬁers from Study 1a, number of occurrences in Google Web 1T 5grams
corpus, and length in syllables.
Ngram
surpassinglycolossally
terriﬁcallyfrightfully
astoundingly
phenomenallyuncommonly
outrageously
fantasticallymightilysupremely
insanely
strikingly
acutelyawfully
decidedly
excessively
extraordinarily
exceedingly
intensely
markedlyamazingly
radically
unusually
remarkably
terribly
exceptionallydesperately
utterly
notably
incredibly
seriouslytruly
signiﬁcantly
totally
extremely
particularlyquite
especiallyvery

3. Utterance cost predicts intensiﬁer strength
The proposal detailed above predicts an association between measures of cost and strength of interpretations. In our ﬁrst series of studies,detailed in this section, we tested whether our measures of communicative cost can in fact predict intensiﬁer strength.
We used two measures of intensiﬁer strength. Our ﬁrst measure ofintensiﬁer strength (used in Studies 1a and 1b) was asking participants
for a numeric interpretation of intensiﬁed adjective phrases. Our second
measure (used in Study 2) was asking participants to rank the strengthof adjective phrases that diﬀered only in their intensiﬁer. The ﬁrst
measure allowed us to compare our full set of intensiﬁers to one another on a numeric scale. The second allowed us to test our hypothesis
on a wider range of adjectives at once, some of which (e.g. beautiful)correspond to more abstract, non-numeric scales.
3.1. Study 1a
In Study 1a, we tested the qualitative prediction that as the communicative cost of an intensiﬁer increases, so will the numeric interpretation of the adjective phrase it is part of. We tested this prediction
by eliciting free-response price estimates from participants for phrases
such as very expensive watch and determining whether the prices participants responded with were correlated with independent measures ofcommunicative cost.
3.1.1. Methods
Participants. 30 participants with US IP addresses were recruited
through Amazon’s Mechanical Turk and paid $0.40 for their participation. 1 participant was excluded from the analysis for admitting thatthey did not think they followed the instructions in a post-experiment
survey and another for not being a native speaker of English.
Items. The sentences in the study included intensiﬁers paired withthe adjective expensive (Fig. 1). There were three categories of objects(laptop, watch, and coﬀee maker) and 40 intensiﬁers (see Table 1).
The intensiﬁers in our study were collected from word lists online
and searching thesauri for more intensiﬁers. We chose intensiﬁers thathave a wide range of frequencies and excluded intensiﬁers that areeither more commonly used to signal aﬀect than to signal degree (e.g.
depressingly expensive might indicate a degree, but it mainly indicatesaﬀect) or are ambiguous between other parts of speech (e.g. super canbe used as an intensiﬁer, as in “super expensive”, but it can also be used
as an exclamation, as in “Super!”).
We chose the domain of price for Study 1a and used only the adjective expensive. Because price constitutes a quantitative scale withstandard units (dollars for our US participants), this allowed us toquantitatively measure the relative strengths of diﬀerent intensiﬁers.
Procedure. We asked participants to estimate the prices of diﬀerent
objects based on diﬀerent descriptions of those objects.
Each participant gave price judgments for every intensiﬁer-category
pairing in a randomized order (diﬀerent for diﬀerent participants), for a
total of 120 price judgments per participant.

Frequency

Length11,156
11,16762,292
65,38973,041
120,769
135,747
240,010
250,989
252,135
296,134
359,644
480,417493,931651,519
817,806
877,280
900,456
977,435
1,084,765
1,213,704
1,384,225
1,414,254
1,583,939
1,902,493
1,906,059
2,054,231
2,139,968
2,507,480
3,141,835
4,416,030
12,570,333
19,778,608
19,939,125
20,950,052
21,862,963
41,066,217
55,269,390
55,397,873
292,897,993

4
4
4
3
4
5
4
4
4
3
3
3
3
3
3
4
4
6
4
3
3
4
3
4
4
3
5
3
3
3
4
4
2
5
3
3
5
1
4
2

The only allowable characters in responses were the digits 0–9 and(optionally) one decimal point (.) followed by two digits. All other
responses were immediately rejected. Participants were prevented
from continuing until they provided a valid numeric response for eachtrial.
3.1.2. Corpus methods
Table 1 shows word frequency and length in syllables for the intensiﬁers used in Study 1a. The frequencies were collected from theGoogle Web 1T 5-grams database (Brants & Franz, 2006).5 In the analysis below we use word length and word surprisal (negative log-frequency) as proxies for a word’s cost, as motivated above. The syllable
lengths of our intensiﬁers and the surprisals were correlated (r = 0.26).
This correlation makes it somewhat diﬃcult to determine the eﬀect of
5
We also ran the same analyses on frequency information collected from the Google
Books American Ngrams Corpus (Michel et al., 2011) and found similar results.

149

Cognition 178 (2018) 147–161

E.D. Bennett, N.D. Goodman

one measure of cost above and beyond the other. In our ﬁrst series ofstudies, we focus on the primary eﬀect of surprisal, since we have morerange in surprisal across intensiﬁers than in length and since we manipulate length in syllables for our next series of studies. However, we
model both measures of communicative cost in our analyses: We include both predictors in regressions and report likelihood-ratio tests
between the full model and simpler models.
3.1.3. Analysis
Prices that participants give obviously vary systematically with theobject (expensive laptops tend to be more expensive than expensive
coﬀee makers). Responses are also likely sensitive to variation acrossparticipants due to their diﬀerent beliefs about likely prices. Because wehave few objects, we are unable to model the variation due to object,
but because we have many intensiﬁers (they are fully crossed with
objects), normalizing is fairly eﬀective at converting all objects to thesame scale. We are not theoretically interested in the variation due toobjects, and so in order to compare intensiﬁers across these objects, weﬁrst normalized log-transformed prices within participant and object.
We used the logarithm of participants’ price estimates because of evidence that people’s representation of numbers, including prices, islogarithmic (Fechner, 1860).6
We ran a linear mixed eﬀects regression to predict scaled priceestimates. We included centered ﬁxed eﬀects of length and surprisal.
To model random eﬀects due to participant, we only included
random slopes, since the normalization gives each participant anintercept of 0. We also included a random intercept for intensiﬁer tomodel any idiosyncratic meaning there might be to a particular intensiﬁer beyond communicative cost. The predicted mean yij for the
ith intensiﬁer and the jth participant under this model is shown in Eq.
(1), where fi represents the surprisal of the ith intensiﬁer and li represents its length.

yij = β0 + U0i + (U1 j + β1) fi + (U2 j + β2) li

Fig. 2. Results of Study 1a. As surprisal increases, participants’ scaled free response prices increase.

suggesting that both might separately contribute to the cost of
the utterance. When length was residualized with respect to suprisal,
both the residuals (b = 0.106, t (38.9) = 3.411, p = 0.002 ) and length(b = 0.202, t (40.9) = 2.693, p = 0.010 ) were signiﬁcant predictors ofscaled ratings. However, when surprisal was residualized with respect tolength, surprisal (b = 0.12, t (39.9) = 3.985, p < 0.0005) but not length
residuals ( p = 0.0936) was a signiﬁcant predictor of scaled ratings. This
discrepancy is likely a due to the fact that, since length in syllables is adiscrete value taking one of 6 values, whereas surprisal is continuous,
length can be more informatively predicted from surprisal (diﬀerent surprisals can map onto approximately the same length) than surprisal can bepredicted from length (the same length cannot map onto diﬀerent surprisals).
Overall, we conﬁrmed our main prediction that intensiﬁers thatare less frequent (and therefore are more costly to communicate) also
tend to be interpreted as having stronger meanings, at least whenused to modify the adjective expensive. We found inconclusive
results for our secondary prediction that length (another factor incommunicative cost) has any eﬀect beyond that of surprisal. This
ambiguity motivated a more sensitive dependent measure in Studies
2 and 4.

(1)

As previously noted, surprisal and length in syllables are correlated
in this set of intensiﬁers (r = 0.26). We focus on surprisal as our primaryeﬀect, but we are also interested in the independent contribution oflength in syllables. To address the independent eﬀects of these variables, we ran model comparisons using log likelihood, leaving out information about one of the predictors. We also ran two additional
mixed eﬀects regressions: one in which the suprisal is ﬁrst residualizedagainst syllables (using ordinary linear regression), and one wherelength is residualized against surprisal.

3.2. Study 1b
Our initial selection of intensiﬁers was somewhat haphazard, being
chosen partly to give the best (intuitive) chance of observing an eﬀect.
To show that there was no implicit bias induced by this selection, we
next replicated using a revised, and more systematic, set of intensiﬁers.
We additionally took this opportunity to increase the sample size,
guided by a power analysis based on the results of Study 1a.

3.1.4. Results
If the meaning of an intensiﬁer is stronger for higher cost intensiﬁers, we would expect to ﬁnd that as surprisal increases and lengthin syllables increases, the prices participants give will also increase. We
ﬁnd that this is the case for surprisal, but do not show a signiﬁcanteﬀect of syllable length beyond the eﬀect of surprisal. Our results areshown in Fig. 2, in a way that highlights the surprisal predictor. We
present full regression output in Appendix B.
We ﬁnd a signiﬁcant eﬀect of surprisal (b = 0.106, t (38.9) = 3.411, p = 0.0015)
such that less frequent words tend to be associated with higher price estimates. In this regression, we did not a signiﬁcant eﬀect of syllable length( p = 0.0936) above and beyond surprisal.
Because surprisal and syllable-length are correlated, in addition to theanalysis reported here we also used likelihood ratio tests to compare thefull model to models with only one of the two predictors. These tests showthat length in syllables alone account for the data less well than the fullmodel ( χ 2 (8) = 17.055, p < 0.0005) and that surprisal alone also accounts
for the data less well than the full model ( χ 2 (8) = 13.697, p < 0.0005),

3.2.1. Methods
Participants. We wanted enough participants for a power level of
0.8 for the principal eﬀect of surprisal. Power analyses for mixed
eﬀects models with continuous predictors are not analyticallystraightforward, and so we approximated the number of participants
necessary for our desired power by bootstrapping. We used the datafrom our original Study 1a to simulate alternative possible datasetswith diﬀerent numbers of intensiﬁers and participants. For each of
100 iterations, we sampled with replacement from Study 1a a set of P
participants and a set of I intensiﬁers, varying P and I. We created aresampled dataset where we combined the resampled participants
with the resampled intensiﬁers and collected the corresponding responses for each pair. For each resampled dataset, we ran the regression model from Study 1a. We computed the proportion of runs

6
I.e. the perceptual distance between two prices the same dollar amount apart is more
for small numbers (e.g. $3 and $6) and less for large numbers (e.g. $1543 and $1546).

150

Cognition 178 (2018) 147–161

E.D. Bennett, N.D. Goodman

in which the surprisal term was signiﬁcant and interpreted this as thepower of such a study.
We found that our original Study 1a was somewhat underpoweredby this metric (statistical power was near 0.70 for the surprisal regressor, our principal eﬀect of interest). For the replication, we determined that with the 71 intensiﬁers we collected from grammars ofEnglish (described below), 50 was the minimum number of subjects for
power at the 0.8 level for the surprisal term. We doubled this amountfor a goal of 100 participants for the replication. (Despite having sufﬁcient power for the surprisal term, our bootstrapped power analysis
suggested that even with 100 participants, the power for the lengthterm in the model is only 0.29. We address this instead, with a direct
manipulation of length, in experiments 4 and 5.) Due to a slight error indata collection, we actually recruited 108 participants.
Following this power analysis, 108 participants with US IP addresses were recruited through Amazon’s Mechanical Turk and paid
$1.00 for their participation. 1 participant was excluded from theanalysis for admitting that they did not think they followed the instructions in a post-experiment survey.
Items. For Study 1b, we again used expensive as the adjective for theintensiﬁers to modify, and we again collected responses as prices indollars. We used the same objects (laptop, watch, and coﬀee maker) as inStudy 1a.
As we explain above, in Study 1a and the remaining studies detailed in this paper, our choice of the set of intensiﬁers to include wassomewhat arbitrary and collected after formation of our hypothesis.
To address the concern that this might bias our results, we followed amore systematic procedure for generating a list of intensiﬁers for our
replication Study 1b. We sought out English grammars that listed
intensiﬁers. Since no single source contained the number of intensiﬁers desired for suﬃcient power in Study 1b, our process forcollecting intensiﬁers in this replication was to combine word lists
from multiple grammars of English. We ﬁrst found 12 grammars ofEnglish that mentioned one of the following terms: “intensiﬁers”,“adverbs of degree”, or “ampliﬁers” (Aarts, Chalker, & Weiner, 2014;
Declerck, 1991; Douglas & Broussard, 2000; Garner, 2016; Givn,
1993; Greenbaum, 1996; Huddleston & Pullum, 2002; Huddleston,
1984; Nelson, 2010; Quirk, 1972; Greenbaum & Quirk, 1990; Van
Gelderen, 2010). Most of these grammars mentioned examples ofsuch words, and many contained lists of them. The average length ofsuch a list or collection was approximately 21 words. We collected anaggregate list of all words that occurred in an “intensiﬁers”, “adverbs
of degree” or “ampliﬁers” list in at least one grammar. Some
“downtoners”/“diminishers” were mixed into some of these lists (e.g.
slightly, barely). Some other intensiﬁers cannot occur felicitously as
simple pre-modiﬁers (e.g. a lot, indeed) and therefore would not ﬁt in
the same syntactic frame as the other intensiﬁers. Other intensiﬁers
were marked as occurring exclusively in British English (e.g. bloody,jolly), and since our participants were restricted to US IP addresses,
we did not include those words in our study. In addition, some lists
included comparative degree adverbs like more. We excluded
downtowners, intensiﬁers that do not pre-modify, comparatives, andexclusively British English intensiﬁers. This resulted in a total of 71
unique intensiﬁers. Of these, only 19 had been in Study 1a. 21 words
that appeared in our previous experiment did not appear in a list in
any of the English grammars, including insanely, wildly, exceptionally,and frightfully. Surprisal and length in syllables were even morecorrelated in this new set of intensiﬁers (r = 0.63). The full list ofintensiﬁers included in Study 1b is in Table 2.
Because replicating Study 1a perfectly with this new set of intensiﬁers would require each participant to answer 213 very similar
questions and would likely take at least 30 min (the higher end of task
lengths on Amazon’s Mechanical Turk), we opted for a “replication

Table 2
Intensiﬁers from Study1b, number of occurrences in Google Web 1T 5grams
corpus, and length in syllables.
Intensiﬁer
dreadfully
fantasticallysupremely
suspiciously
strikingly
noticeably
awfully
unbelievably
downright
excessively
extraordinarily
exceedingly
tremendously
enormously
immensely
hugelyintensely
profoundly
inﬁnitely
amazingly
unusuallyoutright
wonderfully
remarkably
terribly
sharplyutterly
positively
extensively
mightysurprisingly
altogether
purelywhollyincredibly
badly
considerably
suﬃciently
good andthoroughly
damn
deeplyperfectly
greatlylargely
very muchstrongly
entirelyplain
absolutely
truly
totally
extremelydeadcompletely
highlyextra
easilyfully
prettysimplyquite
ratherreal
really
too
way
very
well
most
so

151

Freq

Syll

147,917
250,989
296,134
398,581
480,417632,679
651,519686,210876,782
877,280
900,456
977,435
989,532
1,011,751
1,061,341
1,074,430
1,084,765
1,172,521
1,226,005
1,384,225
1,583,939
1,662,351
1,776,763
1,902,493
1,906,059
2,377,367
2,507,480
3,225,521
3,447,083
3,492,518
3,554,188
3,683,374
4,201,779
4,308,225
4,416,030
4,808,245
4,834,700
5,059,075
5,671,809
6,167,601
6,930,185
7,242,890
10,031,907
11,337,773
11,379,702
11,415,215
13,931,652
14,720,396
15,433,319
16,064,235
19,778,608
20,950,052
21,862,963
28,609,410
32,310,795
36,460,329
36,838,459
39,241,261
41,415,591
43,623,658
50,172,762
55,269,390
66,341,863
144,660,526
148,918,637
159,399,185
268,084,494
292,897,993
301,853,777
324,420,476
518,878,130

3
4
3
4
3
4
3
5
2
4
6
4
4
4
3
2
3
3
4
4
4
2
3
4
3
2
3
4
4
2
4
4
2
2
4
2
5
4
2
3
1
2
3
2
2
3
2
4
1
4
2
3
3
1
3
2
2
3
2
2
2
1
2
1
2
1
1
2
1
1
1

Cognition 178 (2018) 147–161

E.D. Bennett, N.D. Goodman

Fig. 3. Results of Study 1b. As surprisal increases, participants’ scaled free response prices increase.
Fig. 4. Screenshot from Study 2 target question.

design” for our replication Study 1b (following Judd, Westfall, & Kenny,2017). We randomly spit the full set of intensiﬁers into 3 replication
sets and varied the set of intensiﬁers between participants. Within each
replication set, the subset of intensiﬁers was fully crossed with objects.
Our ﬁnal simulations for the bootstrapped power analysis detailed
above included this replication design in computing power.
Procedure. The procedure for Study 1b was identical to that of Study
1a.

occurs.7 To explore this, we extend our results to four additional adjectival scales.
3.3.1. Methods
Participants. 30 participants with US IP addresses were recruited
through Amazon’s Mechanical Turk and paid $0.40 for participation. 2
participants were excluded from the analysis for admitting that they did
not think they followed the instructions in a post-experiment survey.
Items. The full set of intensiﬁers in Study 2 is identical to that ofStudy 1a. For Study 2, we used a ranking dependent measure: asking
participants to sort a set of adjective phrases according to strength.
Because arranging these phrases required participants to be aware of
the full set of adjective phrases and access all of them on the same
computer screen (which might vary in size for diﬀerent participants),
not all of our 40 intensiﬁers could eﬀectively be presented at once. To
make the task easier for participants and to extend to more adjective
scales, we divided the full set of intensiﬁers into 4 smaller sets, maintaining a range of syllable lengths and surprisal across the intensiﬁers ineach smaller set.
With the ranking dependent measure, we no longer need the adjectives to correspond to a standard numeric scale. We therefore included 4 adjectives: old, expensive, beautiful, and tall.
Procedure. For each trial of Study 2, we asked participants to order aset of 10 adjective phrases according to strength of meaning. Adjective
phrases were randomly ordered in a list on the left. Participants wereasked to move the adjective phrases from the left to the right side of thescreen, reordering the phrases from the “lowest” to the “highest” degree(Fig. 4). Each adjective phrase in a trial contained the same adjective,
modiﬁed with diﬀerent intensiﬁers. Each participant saw 4 intensiﬁer
sets, each paired with one of the four adjectives. The pairings of intensiﬁer-sets to adjectives varied between participants. Participants
completed 4 such trials, ranking intensiﬁers for all 4 adjectives and all 4
intensiﬁer sets.

3.2.2. Analysis
As in Study 1a, we normalized log-transformed prices within participant and object and then ran a linear mixed eﬀects regression withcentered ﬁxed eﬀects of length and surprisal, a random slope for participant, and a random intercept for intensiﬁer.
3.2.3. Results
Our results are shown in Fig. 3, with full output in Appendix B.
As in Study 1a, we found a signiﬁcant main eﬀect of surprisal(b = 0.107, t (70.1) = 3.255, p = 0.0017 ) such that less frequent words
tend to be associated with higher price estimates. We again did not ﬁnd
a signiﬁcant main eﬀect of syllable length ( p = 0.3106) above and beyond surprisal.
In likelihood ratio tests, we found that length alone accounts for thedata less well than the full model ( χ 2 (8) = 30.318, p < 0.0005) and thatsurprisal alone also accounts for the data less well than the fullmodel ( χ 2 (8) = 11.571, p = 0.009). When residualizing length withrespect to surprisal, we found a signiﬁcant eﬀect of surprisal(b = 0.128, t (74.9) = 4.96, p < 0.0005) but no signiﬁcant eﬀect oflength residuals ( p = 0.3106). When residualizing surprisal withrespect to length, we again found signiﬁcant eﬀects of both
length (b = 0.166, t (74.1) = 3.93, p = 0.0002 ) and surprisal residuals(b = 0.107, t (70.1) = 3.255, p = 0.0017 ).
This replicates the results of Study 1a, and extends them to a larger,
more systematic set of intensiﬁers.
3.3. Study 2

3.3.2. Analysis
We ran a rank-ordered logit model (Beggs, Cardell, & Hausman,
1981; Hausman & Ruud, 1987) with alternative-speciﬁc variables forsurprisal and for length in syllables. This model did not include an intercept. To test whether we ﬁnd an interaction between intensiﬁer
strength and the adjective it modiﬁes, we ran a second rank-ordered

For Study 2 we used a forced-ranking dependent measure, which
provides a more sensitive measure of the diﬀerences in degrees between
similar adjective phrases. This sensitivity makes Study 2 a better test of
the independent eﬀects of length and surprisal. This dependent measure, because it does not depend on a numeric scale, also allows us toconsider additional adjectival scales.
The M-implicature account described above implies that there is nosemantic interaction between the intensiﬁer and the adjective it is applied to. Instead an intensiﬁer should contribute similar cost, andtherefore meaning, to the diﬀerent adjective phrases in which it

7
If the bigram frequency of the modiﬁed adjective phrase (very expensive) deviatedfrom that expected based on independent word frequencies our frequency-based costaccount would predict an interactive eﬀect on meaning. This would likely be a relatively
small eﬀect, and the relevant bigrams were too sparse in our corpora to pursue.

152

Cognition 178 (2018) 147–161

E.D. Bennett, N.D. Goodman

Fig. 5. Results of Study 2. As surprisal and length in syllables increase, participants’ rankings increased.

is, more extreme intensiﬁers naturally refer to less probable eventsand properties in the world, and therefore might be used less frequently.8 Syllable length in turn can depend on the frequency,
simplicity, or predictability of a word (Lewis, 2016; Mahowald,
Fedorenko, Piantadosi, & Gibson, 2013; Zipf, 1935), either because
words that are frequently used get shortened over time (Kanwal,
Smith, Culbertson, & Kirby, 2016) or perhaps because words thatrefer to simpler or more common concepts enter the lexicon sooner(when more short word forms remain to be assigned meanings). It istherefore possible that neither of these measures of cost have causal
inﬂuence on the meanings of intensiﬁers within a particular communicative act.
To more directly address the question of whether utterance costcauses people to interpret an intensiﬁer as stronger, we ran Studies 3
and 4, where we directly manipulated one of our measures of cost—length—in novel intensiﬁers which have no conventional meaning associated to them.

logit model with additional alternative-speciﬁc variables for the interaction between the adjective being modiﬁed and both surprisal andlength.
3.3.3. Results
Our results for Study 2 are shown in Fig. 5, with full output inAppendix B. In our ﬁrst model, with surprisal and length as the only
predictors in question, we again found a signiﬁcant eﬀect of surprisal(b = 0.164, t = 10.128, p < 0.0005 ). We also ﬁnd a signiﬁcant eﬀect ofsyllable length (b = 0.243, t = 5.888, p < 0.0005).
In likelihood ratio tests, we again found that length alone accounts
for the data less well than the full model ( χ 2 (1) = 108.085, p < 0.0005)
and that surprisal alone also accounts for the data less well than
the full model ( χ 2 (1) = 34.694, p < 0.0005). When residualizing
length with respect to surprisal, we again found a signiﬁcanteﬀect of surprisal (b = 0.19, t = 11.663, p < 0.0005). In addition,with this more sensitive ranking measure, we found a signiﬁcanteﬀect of length residuals (b = 0.243, t = 5.888, p < 0.0005). When residualizing surprisal with respect to length, we found signiﬁcant eﬀects
of length (b = 0.35, t = 8.459, p < 0.0005) and surprisal residuals(b = 0.164, t = 10.128, p < 0.0005 ).
In the second model, with adjective interactions, we again foundsigniﬁcant eﬀects of surprisal (b = 0.074, t = 2.482, p = 0.0131) andlength (b = 0.318, t = 3.658, p = 0.0003), and also found a signiﬁcantinteraction between surprisal and the adjective being modiﬁed (estimates for tall and expensive were higher than for beautiful, while old was
not signiﬁcantly diﬀerent from beautiful). This suggests that contextspeciﬁc surprisal (e.g. bigram frequencies) might aﬀect the utterance
cost, or that factors of utterance cost might have diﬀerent eﬀects fordiﬀerent adjectives.
In other words, we again found that participants assign stronger
interpretations to intensiﬁers with higher surprisals and/or higher syllable lengths, extending now across four diﬀerent adjectival scales. In
addition, we found interactions between modiﬁer and adjective (orperhaps scale).

4. Novel intensiﬁer length increases strength of interpretation
Although the meanings of our existing English intensiﬁers could
have inﬂuenced their lengths and frequencies over time, novel intensiﬁers have no meaning already associated with them. Therefore, if
we found a relationship between the length of a novel intensiﬁer and its
interpreted meaning, we would have evidence that length can causallyinﬂuence meaning. In the following two experiments, we directly manipulate the lengths of novel intensiﬁers and show that longer novel
intensiﬁers are interpreted as having stronger meanings.
4.1. Study 3
In Study 3 we show that longer novel intensiﬁers are interpreted
as having stronger meanings, using our dependent measure from
Study 1a.
4.1.1. Method
Participants. 30 participants with US IP addresses were recruited
through Amazon’s Mechanical Turk and paid $0.80 for their participation. 2 participants were excluded from the analysis for admittingthat they did not think they followed the instructions in a post-experiment survey and 1 for being a non-native English speaker.

3.4. Discussion
In Studies 1a and 1b, we found evidence that an intensiﬁer’s surprisal is a signiﬁcant predictor of strength of its meaning. In Study 2, weused a more sensitive dependent measure and showed that length insyllables is also a signiﬁcant predictor of strength of meaning. Thesestudies taken together provide evidence that intensiﬁer meanings depend systematically on the length and frequency of their word forms.
Since this is a correlational study, such a relationship does not
conﬁrm that an intensiﬁer’s cost causes it to have a given meaning.
In particular, we can explain the correlation with rarity if thestrength of an intensiﬁer’s meaning causes it to be rarely used. That

8
However, note that the frequency with which things occur in the world does not map
directly on to how often those things are talked about. Although it seems reasonable tosuspect that word frequencies reﬂect the probabilities of the real-world concepts theydescribe, it might also be the case that improbable things are more likely to be commented on, and so to a certain extent the frequencies of words that describe rare concepts
will be inﬂated.

153

Cognition 178 (2018) 147–161

E.D. Bennett, N.D. Goodman

Items. In Study 1a, we rescaled prices within participants andobjects. In order to do the same normalization for novel intensiﬁers,
we chose 9 ﬁller intensiﬁers to include in Study 3. We chose a set ofﬁller intensiﬁers to cover a wide range of surprisals and syllable
lengths: colossally, phenomenally, mightily, extraordinarily, amazingly,
terribly, notably, signiﬁcantly, quite. Other than covering a range ofprices, the particular choice of ﬁllers should not aﬀect our analysis of
the novel intensiﬁers. Each novel intensiﬁer was presented in thesame context of 9 ﬁller intensiﬁers. Eliciting ratings for existing
English intensiﬁers along with novel intensiﬁers allowed us to again
rescale and normalize responses within participants and items, placing all responses for the novel intensiﬁers on the same scale. It also
allowed us to somewhat obscure our use of fabricated words, andthus decrease task demand.
We varied the novel intensiﬁer between participants from a set of 6
novel intensiﬁers, three of which were relatively short (lopusly, ratumly,
and bugornly) and three of which shared the same “root” but were two
CVCV syllables longer (fepolopusly, gaburatumly, and tupabugornly).
These items were taken from previous studies on complexity bias
(Lewis, 2016) and modiﬁed by adding a ﬁnal -ly suﬃx.
Procedure. The procedure for Study 3 was identical to that of Study
1a, except that we included only a subset of the intensiﬁers from Study
1a and each participant also saw one novel intensiﬁer, randomly mixed
in with the rest.
Participants again estimated prices for objects of three diﬀerent
categories paired with all of the intensiﬁers. The order of the questions
was randomized between and within participants.

Fig. 7. In Study 3, we replicated our ﬁndings from Study 1a.

4.1.3. Results
In our analysis of novel intensiﬁers, we found a signiﬁcant eﬀect oflength condition (b = 0.538, t = 2.757, p = 0.0002 ), indicating thatpeople interpret a longer intensiﬁer as stronger, even for novel intensiﬁers with no conventional meaning (Fig. 6).
Replicating our ﬁndings from Study 1a, we found signiﬁcant maineﬀects of surprisal (b = 0.14, t (7.2) = 2.49, p = 0.0407 ) but no signiﬁcant eﬀect of syllable length ( p = 0.0846) (Fig. 7). In likelihood ratio
test, we found that the full model was a better ﬁt than the model withsurprisal only ( χ 2 (8) = 8.296, p = 0.0403) and the model with length
only ( χ 2 (8) = 39.194, p < 0.0005). Residualizing each regressor withrespect to one another, we again found qualitatively similar eﬀects toStudies 1a and 1b, with a signiﬁcant eﬀect of surprisal when residualized by length (b = 0.14, t (7.2) = 2.49, p = 0.0407 ) and no signiﬁcant eﬀect of length when residualized by surprisal ( p = 0.085).
We observe that, numerically, average responses for ratumly were much
lower than all the other intensiﬁers used in Study 3. In a post hoc regression
with ﬁxed eﬀects for the novel adverb roots (ratum/other = 2/−1, lopus/
bugorn = 1/−1), we found a signiﬁcant diﬀerence between ratum and theother roots (b = −0.56, t (23) = −2.60, p = 0.0160 ) and no signiﬁcant difference between the remaining roots (p = 0.759). The very low rankings forthe root ratum suggest possible additional eﬀects of form that we have notcaptured with length in syllables alone.
Full results for Study 3 are presented in Appendix B.

4.1.2. Analysis
In Study 3, to study the eﬀect of the length of a novel intensiﬁer on
its interpretation, we ran a linear mixed eﬀects model on only the novel
intensiﬁers, with length (“long” = 1 or “short” = −1) as a ﬁxed eﬀect,
a random slope for participant, and random intercepts for the three
diﬀerent “roots”.
We included as ﬁller a subset of the intensiﬁers we tested in Study
1a, and so we again replicated our ﬁndings from Study 1a. As in Study
1a, we ran a linear mixed eﬀects regression with centered ﬁxed eﬀects
of length and surprisal and a random slope for participant, and arandom intercept for intensiﬁer. To see the eﬀect of the two measures ofcommunicative cost separately, we compared the full model to a model
without that measure as a regressor using a likelihood ratio test and ran
versions of the model where we ﬁrst residualized the measures withrespect to one another.

4.2. Study 4
In Study 4 we again show that longer novel intensiﬁers are interpreted as having stronger meanings, this time for an additional adjective, using our dependent measure from Study 2.
4.2.1. Methods
Participants. 60 participants with US IP addresses were recruited
through Amazon’s Mechanical Turk and paid $0.16 for their participation. 3 participants were excluded from the analysis for admittingthat they did not think they followed the instructions in a post-experiment survey.
Items. In Study 4, each participant saw exactly one of two adjectives(expensive or tall, varied between participants) with the set of intensiﬁers from Study 3. This set included 9 context/ﬁller words and one
novel intensiﬁer, which we varied between participants.

Fig. 6. In Study 3, we found a signiﬁcant eﬀect of length for all novel intensiﬁers.

154

Cognition 178 (2018) 147–161

E.D. Bennett, N.D. Goodman

4.3. Discussion
Overall, in Studies 3 and 4 we found that word length in syllables isa signiﬁcant predictor of interpretation strength for novel intensiﬁers.
These novel intensiﬁers have no established meaning, so the relationship between their length and strength cannot be a direct consequenceof the lexicon becoming more eﬃcient over time. This result is consistent with the hypothesis that participants are inferring the meaningsof the novel intensiﬁers pragmatically, as in the M-implicature accountsketched above.
Alternatively, it could be that participants have learned a generalrelationship between length and meaning of intensiﬁers in English, andare utilizing this meta-linguistic knowledge to interpret the new words
they encounter.
This meta-linguistic hypothesis is less parsimonious than the pragmatic hypothesis, since the pragmatic hypothesis relies only on mechanisms (M-implicature) that we know to be involved in other examples of language understanding (e.g. as in the “I got the car to start”
example above). It builds on previous formal models of adjective
meaning and informal models of intensiﬁer meaning, generalizing to awide range of intensiﬁers.
Whether due to a process of M-implicature or meta-linguisticknowledge, these results demonstrate that the relationship between
word cost and meaning is not merely a static, gradual result of language
evolution—interpreted meaning of intensiﬁers depends on length in an
active, dynamic way.

Fig. 8. In Study 4, we replicated our ﬁnding from Experiment 2: longer and lessfrequent intensiﬁers are ranked higher than shorter and more frequent ones.

Procedure. Except for the narrower set of items, noted above, Study
4 was identical to Study 2.
As in Study 2, adjective phrases for each intensiﬁer-adjective
pairing were initialized on the left in a random order.
4.2.2. Analysis
For the novel intensiﬁers, we ran a cumulative logit model on the
rankings (relative to the ﬁller intensiﬁers) that participants gave to thenovel intensiﬁer.
As in Study 3, we also ran such a model with regressors for the word
stem (or “root”).
With our ﬁller intensiﬁers for Study 4 we again ran a ranked order
logit model (re-ranking to ignore novel intensiﬁers) to conﬁrm eﬀects ofsyllable length and surprisal.

5. General discussion
Motivated by a recent probabilistic model of scalar adjectives(Lassiter & Goodman, 2013), we argued that adjectival intensiﬁers
could gain aspects of their meaning through a systematic pragmatic
inference, even in the absence of conventional literal meaning. Our Mimplicature model (presented in further detail in Appendix A) predicted
a linear relationship between the intensity of an intensiﬁer and its
communicative cost, measured here in terms of length and negative logfrequency, and we explore this linear relationship using regression
models.
In ﬁve experiments we provided evidence that intensiﬁer meanings do depend systematically on the length and frequency of occurence of those word forms and that this relationship holds even for
novel words.
While it is unlikely that this accounts for all intensiﬁer meaning, ourresults do suggest that a major portion of meaning comes not fromarbitrary, conventional association of signal to sign (De Saussure,1916), but from features of the word’s form and distribution, together
with inferential processes of listeners.
For the semantics of adverbial modiﬁers, we have shown how
pragmatic mechanisms could be central in establishing ﬂexible contributions to sentence meaning. We have extended previous proposals
that degree adverbs transform or create new threshold variables, providing a concrete mechanism for interpreting an arbitrary degree adverb in an arbitrary context.
This mechanism for linking a parsimonious semantics to interpretation via pragmatic inference follows naturally and straightforwardly from an understanding of rational social agents engaging incommunication. With the minimal assumption that each use of ascalar adjective has a diﬀerent threshold, we have been able to extend a model of adjective interpretation to account for a major source

4.2.3. Results
In the cumulative logit model, we found a signiﬁcant eﬀect of length
condition (b = 0.7039, z = −2.79, p = 0.0054 ).
When we included a regressor for the root of the novel intensiﬁer,
we did not ﬁnd a signiﬁcant diﬀerence between participants’ relative
rankings for ratum and other roots ( p = 0.110 ) or between the other tworoots ( p = 0.304 ).
Rankings for novel intensiﬁers had much higher variance thanrankings for attested intensiﬁers, since we had many fewer rankings forthe novel intensiﬁers (which varied between participants) than for theattested ones (which every participant saw once). The novel intensiﬁer
ratumly was again on average ranked as less strong than any other intensiﬁer, but the highest-ranked novel intensiﬁers (gaburatumly andtupabugornly) were on average ranked below the highest-ranked attested intensiﬁers (colossally, phenomenally, extraordinarily, and amazingly).
For the ﬁller items, we replicated our ﬁndings from Study 2, showing
signiﬁcant eﬀects of both surprisal (b = 0.447, t = 13.879, p < 0.0005) andsyllable length (b = 0.581, t = 10.738, p < 0.0005 ) on the order in the listthat participants chose for the intensiﬁers (see Fig. 8). In likelihood ratio tests,
we again found that length alone ( χ 2 (1) = 231.574, p < 0.0005) or surprisal
alone ( χ 2 (1) = 130.33, p < 0.0005) accounted for the data less well than thefull model. As in Study 2, residualized surprisal was signiﬁcant(b = 0.447, t = 13.879, p < 0.0005) and residualized length was also signiﬁcant (b = 0.581, t = 10.738, p < 0.0005). This replication is perhaps unsurprising, since we chose intensiﬁers to cover the full range of surprisal andsyllable lengths.
Full results for Study 4 are presented in Appendix B.

155

Cognition 178 (2018) 147–161

intensiﬁer

E.D. Bennett, N.D. Goodman
inﬁnitely
unbelievably
astoundingly
supremely
tremendously
insanelyoutrageously
enormouslycolossally
extraordinarily
extremely
phenomenallyimmensely
exceedingly
profoundly
excessively
dreadfullyincredibly
frightfully
hugelymostfantasticallytupabugornly
amazingly
exceptionallyremarkably
fepolopusly
intensely
absolutely
outright
strikingly
gaburatumly
extra
highlyextensively
terriblydeadterriﬁcallysurpassingly
deeplyradicallyutterlysuspiciously
damnunusuallybugornly
very much
truly
badly
sharply
mightilyawfullytoo
greatlythoroughly
downright
mightypurelydesperately
lopusly
entirely
positively
signiﬁcantly
reallycompletely
whollyway
considerably
surprisingly
seriouslylargely
totallyvery
fully
wonderfullysostrongly
altogether
realmarkedly
uncommonly
notably
acutely
good and
perfectly
quite
well
especiallyparticularly
decidedly
plain
noticeably
sufﬁciently
rathersimplyratumlyeasilypretty(Qing, Goodman, & Lassiter, 2016)—which can be explained
within the basic RSA framework, each with minimal, compatible
assumptions. The precise speciﬁcation of this model is laid out inmore detail in Appendix A.
Our proposal and our experiments suggest that even a veryminimal semantics for intensifying degree adverbs can be plausible
and productive. We have implemented and described one version
of such a semantics, but other versions might exist with diﬀerent
methods of generating the threshold variable for an adjective
phrase.
We use surprisal and length in syllables as measures of communicative cost, but there may be other systematic features of intensiﬁers that contribute to strength. In particular, many intensiﬁers
seem to be derived from adverbs having to do with emotion, and thevalence and/or arousal of these root emotions might inﬂuence thestrength of an intensiﬁer or its aﬃnity to co-occur with some adjective types rather than others. This might be especially true
for intensiﬁers that are still making the change from manneradverb to intensiﬁer (e.g. terribly once only carried the qualitative
meaning of “bad and frightening”, but now almost exclusively means
simply “a lot”).
As an illustration of the variance explained by our surprisal measureof communicative cost, we ﬁrst averaged across participants and adjectives to get an average scaled response for each intensiﬁer. We thencomputed the squared correlation between those values and the surprisal predictor. We ﬁnd that surprisal can explain around 30% of thevariance for attested intensiﬁers (R2 = 0.293 for Study 1a, R2 = 0.273 forStudy 1b, and R2 = 0.358 for Study 2).9
In investigating our proposal of how intensiﬁers get their meanings, we have collected many judgements about the relative strengths
of intensiﬁers across several adjective scales and dependent measures. As a summary of these judgements, we have aggregated acrossthe diﬀerent dependent measures in our studies and displayed themin a common scale in Fig. 9. For each study, we computed theaverage value of the dependent measure (normalized log-transformed price for Studies 1a, 1b and 3 and height in list for Studies 2
and 4) for each intensiﬁer in that study. We then z-scored within the
study, so that our diﬀerent operationalizations of “strength” in thediﬀerent study designs could be reasonably compared together.
Given this rescaling, we can compare the strengths of these intensiﬁers to one another, including the novel intensiﬁers from Studies 3 and 4. We see some consistency across experiments and showthat our choices of novel intensiﬁers spanned the full range of natural intensiﬁer strengths.
For the broader question of form-meaning mapping, we havesuggested a source of non-arbitrary association based on bothproperties of the word form and of its distribution. The eﬀect of aword’s distribution on its interpretation has potentially interesting
implications for language change. If the distribution of a particular
grammatical category of word (e.g. intensiﬁers) inﬂuences its
meaning and the meaning of a word in turn inﬂuences its distribution, this would result in an unstable lexicon for this grammaticalcategory. This suggests a mechanism by which overused words might
become stale, and would predict the rapid creation of new, unusual
intensiﬁers. This process indeed seems to be evident in the history ofEnglish (Bolinger, 1972).
We have argued that form-to-meaning mapping can come about
through the inferences known to support pragmatic interpretation.
Seen another way, the basic assumption that people are actively
trying to communicate with each other—each reasoning about
what the interlocutor means—requires non-arbitrary relationships

study
1a
1b
2
3
4

−3

−2

−1

0

1

2

intensity

Fig. 9. Intensities (each dependent measure, z-scored) of all intensiﬁers acrossall 5 studies. Novel intensiﬁers are shown in bold, standard English intensiﬁers
in plain font.

of intensiﬁer meaning. This model adds to an abundance of linguistic
phenomena—e.g. aspects of metaphor (Kao, Bergen, & Goodman,2014a), prosody (Bergen & Goodman, 2015), politeness (Yoon,
Tessler, Goodman, & Frank, 2017), and presupposition projection

9
For intuition this means that one nat diﬀerence in surprisal predicts 0.128 normalized
units of cost diﬀerence. In the case of laptops, for example, one nat of surprisal would
predict approximately a 9% increase in the interpreted price.

156

Cognition 178 (2018) 147–161

E.D. Bennett, N.D. Goodman

Acknowledgments

between a variety of factors and eﬀective meaning. Some systematic
aspects of meaning follow directly from the principles of language
understanding.

This work was supported by a James S. McDonnell Foundation
Scholar Award to NDG and ONR grant N00014-13-1-0788.

Appendix A. Pragmatic model
Lassiter and Goodman (2013)’s model of scalar adjectives belongs to the family of Rational Speech Act (RSA) models in which speaker andlistener communicate by reasoning about each other’s goals and inferences (Franke, 2011; Frank & Goodman, 2012; Goodman & Stuhlmüller, 2013;
Russell, 2012). These models have been shown to account for a number of key phenomena in pragmatics. The adjective model accounts for
uncertainty about the adjectival threshold by including a semantic variable, which the pragmatic listener infers at the same time that she infers the
speaker’s intended meaning.
RSA models begin with a literal listener, which captures the semantic denotation of sentences. We assume adjective phrases with the same scale
and polarity have the same denotation. For example, expensive, very expensive and phenomenally expensive all denote: λx . price(x ) > θi . However,→every adjective phrase has its own threshold variable θi , 10 together notated θ , allowing their meanings to diﬀer. Given an utterance ui (e.g. an
expensive laptop or a very expensive laptop) and a set of thresholds, a literal listener L0 will use Bayesian inference to update his prior beliefs P (d ) about
the degree d (e.g. the laptop’s price) given that the degree is greater than the threshold for that utterance.

PL0 (d|ui,θi ) ∝ P (d )·δd > θiA speaker with the goal of communicating some actual degree d assigns a utility  (ui |d ) to each utterance such that he prefers utterances which
will inform the literal listener, but avoids utterance cost, C (ui ) :→ (ui |d, θ ) = ln(PL0 (d|ui,θi ))−C (ui )Given a set of alternative utterances (e.g. the speaker might be choosing between saying very expensive as opposed to expensive or extremely
expensive, or saying nothing at all), the speaker S1 will choose utterances according to a softmax decision rule (Sutton & Barto, 2011) with optimality
parameter λ , so that:→→
PS1 (ui |d, θ ) ∝ e λ (ui | d, θ )A pragmatic listener L1 uses the prior probability, P (d ) , of diﬀerent degrees, along with knowledge of the cost of each utterance, in order to guessboth the thresholds for each utterance and which degree the speaker intended to communicate11:→→
PL1 (d, θ |ui ) ∝ P (d)·PS1 (ui |d, θ )We simulated such a model with three alternative adjective phrases (i.e. three intensiﬁers) with costs of 1,5, and 10. We also included a nullutterance, with trivial meaning (always true) and cost of 0. The prior distribution of degrees along this adjective’s scale (which we will discuss as“prices” for concreteness and consistency with our Experiment 1) was a gaussian peaked at 0. We used an optimality parameter of λ = 5 in our
simulation.
Though the literal semantics are identical (but permitting diﬀerent threshold parameters), the diﬀerent phrases received diﬀerent interpretations:the more costly intensiﬁers corresponded to less probable, more extreme prices (Fig. A.1). This can be seen as an M-implicature: more costly
intensiﬁers are assigned stronger, less probable, meanings. The model therefore predicts an association between intensiﬁer meaning and utterance
cost (see Bergen et al. (2014) for other M-implicature models within the RSA framework).

L1(price | utterance)

0.6

cost
prior
1
5
10

0.4

0.2

0.0
−2

−1

0

1

2

standardized "prices"Fig. A.1. Modeling intensiﬁers as M-implicature: more costly intensiﬁers correspond to more extreme meanings.

To assess the quantitative relationship between cost and meaning, we ran a second simulation, identical to the ﬁrst except using 6 diﬀerent
utterance costs (or “intensiﬁers”). The quantitative form predicted by the model is a approximately linear (Fig. A.2). It is this simple prediction that
we explore in the main text.

10
Other versions of this model could easily be imagined in which the threshold for an adjective phrase is determined by the basic threshold for the adjective and some transformationon that threshold caused by the intensiﬁer. For example, Cliﬀ (1959) argues that intensifying adverbs operate on adjectives meanings through scalar multiplication. If, as in multiplication, the transformation is regular, with a single parameter needing to be inferred for each intensiﬁer, and if the values of these parameters are inferred for each adjective phrase,then such a model would be functionally equivalent to the one we describe here.
11
We assume a uniform prior on thresholds θi .

157

Cognition 178 (2018) 147–161

E.D. Bennett, N.D. Goodman

Fig. A.2. Model prediction of expected price as cost of intensiﬁer increases, based on intensiﬁers evenly spaced in cost. The relationship is approximately linear.

Appendix B. Regression results
In Table B.1, we present results of linear mixed-eﬀects models of scaled responses in Study 1a as a function of length in syllables and surprisal.
Model 1 includes raw predictors. In Model 2, we replace the length predictor by the residuals when length is modeled as a linear function of surprisal.
In Model 3, we replace the surprisal predictor by the residuals when surprisal is modeled as a linear function of length. Finally, we present results oflikelihood ratio tests when each of the predictors is removed from the full model.
Table B.1
Study 1a Regression results.
EstimateseModel 1: Colinear predictors
0.077
0.031

t

χ2

p

1.72
3.41

–
–

0.094
0.002

LengthSurprisal

0.133
0.106

Lengthresid
Surprisal

0.133
0.120

Model 2: Length, residualized by surprisal
0.077
1.72
0.030
3.98

–
–

0.094
<0.001

LengthSurprisalresid

0.202
0.106

Model 3: Surprisal, residualized by length
0.075
2.69
0.031
3.41

–
–

0.010
0.002

13.70
17.05

0.003
0.001

Model comparison: predictors removed from Model 1
LengthSurprisal

In Table B.2, we present results of linear mixed-eﬀects models of scaled responses in Study 1b as a function of length in syllables and surprisal.
Model 1 includes raw predictors. In Model 2, we replace the length predictor by the residuals when length is modeled as a linear function of surprisal.
In Model 3, we replace the surprisal predictor by the residuals when surprisal is modeled as a linear function of length. Finally, we present results oflikelihood ratio tests when each of the predictors is removed from the full model.
Table B.2
Study 1b Regression results.
EstimateseModel 1: Colinear predictors
0.054
0.033

t

χ2

p

1.02
3.26

–
–

0.311
0.002

LengthSurprisal

0.055
0.107

Lengthresid
Surprisal

0.055
0.128

Model 2: Length, residualized by surprisal
0.054
1.02
0.026
4.96

–
–

0.311
<0.001

LengthSurprisalresid

0.166
0.107

Model 3: Surprisal, residualized by length
0.042
3.93
0.033
3.26

–
–

<0.001
0.002

11.57
30.32

0.009
< 0.001

Model comparison: predictors removed from Model 1
LengthSurprisal

In Table B.3, we present results of linear mixed-eﬀects models of rankings in Study 2 as a function of length in syllables and surprisal. Model 1
includes raw predictors. In Model 2, we replace the length predictor by the residuals when length is modeled as a linear function of surprisal. In
Model 3, we replace the surprisal predictor by the residuals when surprisal is modeled as a linear function of length. In Model 4, we add interactions
for adjectives. Finally, we present results of likelihood ratio tests when each of the length and surprisal predictors are removed from Model 1.
158

Cognition 178 (2018) 147–161

E.D. Bennett, N.D. Goodman

Table B.3
Study 2 Regression results.
EstimateseModel 1: Colinear predictors
0.016
0.041

t

χ2

p

10.13
5.89

–
–

<0.001<0.001

Surprisal
Length

0.164
0.243

Surprisal
Lengthresid

0.190
0.243

Model 2: Length, residualized by surprisal
0.016
11.66
0.041
5.89

–
–

<0.001<0.001

LengthSurprisalresid

0.350
0.164

Model 3: Surprisal, residualized by length
0.041
8.46
0.016
10.13

–
–

<0.001<0.001

Surprisal
LengthSurprisal:Adj
Adj = Tall
Adj = Expensive
Adj = Old
Length:Adj
Adj = Tall
Adj = Expensive
Adj = Old

0.074
0.318

Model 4: With adjective interaction
0.030
2.48
0.087
3.66

–
–
–
–
–
–
–
–
–
–

0.013
<0.001

34.69
108.08< 0.001< 0.001

0.188
0.170
0.037

0.045
0.047
0.043

4.14
3.66
0.85

−0.103−0.100−0.092

0.121
0.119
0.119

−0.86
−0.84
−0.78

<0.001<0.001
0.393
0.393
0.400
0.438

Model comparison: predictors removed from Model 1
LengthSurprisal

In Table B.4, we present results of linear mixed-eﬀects models of scaled responses in Study 3’s replication portion as a function of length insyllables and surprisal. Model 1 includes raw predictors. In Model 2, we replace the length predictor by the residuals when length is modeled as alinear function of surprisal. In Model 3, we replace the surprisal predictor by the residuals when surprisal is modeled as a linear function of length.
Finally, we present results of likelihood ratio tests when each of the predictors is removed from the full model.
In Table B.5, we present results of regressions predicting scaled responses for novel intensiﬁers in Study 3.
In Table B.6, we present results of regression predicting rankings for novel intensiﬁers in Study 4.

Table B.4
Study 3 Regression results.
EstimateseModel 1: Colinear predictors
0.095
0.056

t

χ2

p

2.05
2.49

–
–

0.085
0.041

LengthSurprisal

0.194
0.140

Lengthresid
Surprisal

0.194
0.182

Model 2: Length, residualized by surprisal
0.095
2.05
0.052
3.47

–
–

0.085
0.010

LengthSurprisalresid

0.286
0.140

Model 3: Surprisal, residualized by length
0.088
3.24
0.056
2.49

–
–

0.016
0.041

8.30
39.19

0.040
< 0.001

Model comparison: predictors removed from Model 1
LengthSurprisal

Table B.5
Study 3 Replication portion regression results.
Estimatese(Intercept)
Length

0.110
0.538

Study 3 (Novel intensiﬁers) regression results
Model 1: Short/long novel adverbs
0.366
0.195

(Intercept)
LengthRatum/other
Lopus/Bugorn

0.100
0.540
−0.371
0.073

Model 2: Length and root type
0.197
0.195
0.143
0.235

159

t

p

0.299
2.757

0.768
0.011

0.510
2.772
−2.601
0.311

0.615
0.011
0.016
0.759

Cognition 178 (2018) 147–161

E.D. Bennett, N.D. Goodman

Table B.6
Study 4 Regression results.
Estimate

LengthLengthRatum/other
Lopus/Bugorn

se

z

p

0.70

Study 4 (Novel intensiﬁers) regression results
Model 1: Short/long novel adverbs
0.253

2.79

0.005

0.757
−0.047−0.518Model 2: Length and root type
0.263
0.172
0.324

2.88
−0.27
−1.60

0.004
0.786
0.110

In Table B.7, we present results of linear mixed-eﬀects models of rankings in Study 4’s replication portion as a function of length in syllables andsurprisal. Model 1 includes raw predictors. In Model 2, we replace the length predictor by the residuals when length is modeled as a linear function of
surprisal. In Model 3, we replace the surprisal predictor by the residuals when surprisal is modeled as a linear function of length. Finally, we present
results of likelihood ratio tests when each of the predictors is removed from the full model.

Table B.7
Study 4 Replication portion regression results.
Estimate

se

t

χ2

p

–
–

<0.001<0.001

Surprisal
Length

0.447
0.581

Study 4 (Replication portion) regression results
Model 1: Colinear predictors
0.032
13.88
0.054
10.74

Surprisal
Lengthresid

0.573
0.581

Model 2: Length, residualized by surprisal
0.037
15.54
0.054
10.74

–
–

<0.001<0.001

LengthSurprisalresid

0.875
0.447

Model 3: Surprisal, residualized by length
0.063
13.94
0.032
13.88

–
–

<0.001<0.001130.33231.57< 0.001< 0.001

Model comparison: predictors removed from Model 1
LengthSurprisal

Appendix C. Supplementary material
Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.cognition.2018.05.011.

language understanding as social cognition. Topics in Cognitive Science, 5(1), 173–184.
Greenbaum, S. (1996). The Oxford English Grammar, Vol. 652. Oxford: Oxford University
Press.
Grice, H. P. (1975). Logic and Conversation, 1975, 41–58.
Hausman, J. A., & Ruud, P. A. (1987). Specifying and testing econometric models forrank-ordered data. Journal of Econometrics, 34(1), 83–104.
Holland, M. K., & Wertheimer, M. (1964). Some physiognomic aspects of naming, or,maluma and takete revisited. Perceptual and Motor Skills, 19(1), 111–117.
Huddleston, R. (1984). Introduction to the grammar of English. Cambridge University Press.
Huddleston, R., & Pullum, G. K. (2002). The Cambridge grammar of the English language.
Cambridge, 14, 199–212.
Judd, C. M., Westfall, J., & Kenny, D. A. (2017). Experiments with more than one random
factor: Designs, analytic models, and statistical power. Annual Review of Psychology,
68, 601–625.
Kanwal, J., Smith, K., Culbertson, J., & Kirby, S. (2016). The evolution of Zipfs law ofabbreviation. The evolution of language: Proceedings of the 11th international conference(EVOLANG11).
Kao, J. T., Bergen, L., & Goodman, N. D. (2014a). Formalizing the pragmatics of metaphor
understanding. Proceedings of the thirty-sixth annual conference of the Cognitive Science
Society.
Kao, J. T., Wu, J. Y., Bergen, L., & Goodman, N. D. (2014b). Nonliteral understanding of
number words. Proceedings of the National Academy of Sciences, 111(33),12002–12007.
Kennedy, C. (2007). Vagueness and grammar: The semantics of relative and absolute
gradable adjectives. Linguistics and Philosophy, 30(1), 1–45.
Kennedy, C., & McNally, L. (2005). Scale structure, degree modiﬁcation, and the semantics of gradable predicates. Language, 345–381.
Klein, E. (1980). A semantics for positive and comparative adjectives. Linguistics andPhilosophy, 4(1), 1–45.
Koriat, A., & Levy, I. (1979). Figural symbolism in Chinese ideographs. Journal of
Psycholinguistic Research, 8(4), 353–365.
Khler, W. (1970). Gestalt psychology: An introduction to new concepts in modern psychology.

References
Aarts, B., Chalker, S., & Weiner, E. (2014). The Oxford dictionary of English grammar.
Oxford University Press.
Beggs, S., Cardell, S., & Hausman, J. (1981). Assessing the potential demand for electriccars. Journal of Econometrics, 17(1), 1–19.
Bergen, L., Goodman, N., & Levy, R. (2012). That’s what she (could have) said: Howalternative utterances aﬀect language use. CogSci.
Bergen, L., & Goodman, N. D. (2015). The strategic use of noise in pragmatic reasoning.
Topics in Cognitive Science, 7(2), 336–350.
Bergen, L., Levy, R., & Goodman, N. D. (2014). Pragmatic reasoning through semantic
inference. Semantics and Pragmatics.
Bolinger, D. (1972). Degree words, Vol. 53. Walter de Gruyter.
Brants, T., & Franz, A. (2006). Web 1t 5-gram Version 1.
Cliﬀ, N. (1959). Adverbs as multipliers. Psychological Review, 66(1), 27.
Davis, R. (1961). The ﬁtness of names to drawings. A cross-cultural study in Tanganyika.
British Journal of Psychology, 52(3), 259–268.
De Saussure, F. (1916). Nature of the linguistic sign. Course in General Linguistics, 65–70.
Declerck, R. (1991). Comprehensive descriptive grammar of English. A. Kaitakusha.
Douglas, D., & Broussard, K. M. (2000). Longman grammar of spoken and written English.
TESOL Quarterly, 34(4), 787–788.
Fechner, G. (1860). Elements of psychophysics (H. Adler, Trans.)Brietkoph & Hrtel 1.
Frank, M. C., & Goodman, N. D. (2012). Predicting pragmatic reasoning in language
games. Science, 336(6084), 998.
Franke, M. (2011). Quantity implicatures, exhaustive interpretation, and rational conversation. Semantics and Pragmatics, 4, 1.
Garner, B. A. (2016). The Chicago guide to grammar, usage, and punctuation. SPRINGBOOKS, 16, 3.
Givn, T. (1993). English grammar: A function-based introduction, Vol. 2. John Benjamins
Publishing.
Goodman, N. D., & Stuhlmüller, A. (2013). Knowledge and implicature: Modeling160

Cognition 178 (2018) 147–161

E.D. Bennett, N.D. Goodman

Society.
Quirk, R. (1972). A grammar of contemporary English. Longman Group.
Quirk, R., Greenbaum, S., Leech, G., & Svartvik, J. (1985). A comprehensive grammar of the
English language. London and New York: Longman.
Greenbaum, S., & Quirk, R. (1990). A student’s grammar of the English language. Pearson
Education India.
Ramachandran, V. S., & Hubbard, E. M. (2001). Synaesthesia—A window into perception,
thought and language. Journal of Consciousness Studies, 8(12), 3–34.
Russell, B. (2012). Probabilistic reasoning and the computation of scalar implicatures (Ph.D.
thesis)Citeseer.
Smith, N. J., & Levy, R. (2013). The eﬀect of word predictability on reading time islogarithmic. Cognition, 128(3), 302–319.
Sutton, R. S., & Barto, A. G. (2011). Reinforcement learning: An introduction. Cambridge
Univ Press.
Van Gelderen, E. (2010). An introduction to the grammar of English: Revised edition. John
Benjamins Publishing.
Wheeler, S. C. (1972). Attributives and their modiﬁers. Nos, 310–334.
Yoon, E. J., Tessler, M. H., Goodman, N. D., & Frank, M. C. (2017). I won’t lie, it wasn’t
amazing: Modeling polite indirect speech. Proceedings of the thirty-ninth annual conference of the Cognitive Science Society.
Zipf, G. K. (1935). The psycho-biology of language.

WW Norton & Company.
Lassiter, D., & Goodman, N. D. (2013). Context, scale structure, and statistics in the interpretation of positive-form adjectives. Semantics and linguistic theory: Vol. 23, (pp.
587–610).
Levinson, S. C. (2000). Presumptive meanings: The theory of generalized conversational implicature. MIT Press.
Levy, R. (2008). Expectation-based syntactic comprehension. Cognition, 106(3),1126–1177.
Lewis, M. L. (2016). Conceptual complexity and the evolution of the lexicon (Ph.D. thesis)
Stanford University.
Mahowald, K., Fedorenko, E., Piantadosi, S. T., & Gibson, E. (2013). Info/information
theory: Speakers choose shorter words in predictive contexts. Cognition, 126(2),313–318.
McCusker, L. (1977). Some determinants of word recognition: Frequency. 24th Annual
convention of the southwestern psychological association, Fort Worth, TX.
Michel, J.-B., Shen, Y. K., Aiden, A. P., Veres, A., Gray, M. K., Pickett, J. P., ... Aiden, E. L.
(2011). Quantitative analysis of culture using millions of digitized books. Science,
331(6014), 176–182.
Nelson, G. (2010). English: An essential grammar. Routledge.
Qing, C., Goodman, N. D., & Lassiter, D. (2016). A rational speech-act model of projective
content. Proceedings of the thirty-eighth annual conference of the Cognitive Science161