Nathaniel J. Smith, U. Edinburgh<nathaniel.smith@ed.ac.uk>
Noah D. Goodman, Stanford<ngoodman@stanford.edu>
Michael C. Frank, Stanford<mcfrank@stanford.edu>

Learning and using language via recursive
pragmatic reasoning about other agents]'s

Listener

In existing pragmatics models:
a communicative goal, requiring recursive reasoning

Word
Base case: L0, the literal listener
who has arbitrary behavior encoded as
literal meanings in the literal lexicon

LexiconReferentWord[2, 3]

A fundamental paradox for social learning
Word meaning, in the latent variable sense, does not appear in either Bayesian pragmatics models or
the real world's actual generative process. Without uncertainty about a latent variable, learning isimpossible. Yet humans both learn word meanings and perform pragmatic reasoning. How can wereconcile these approaches?

Make L0 or S1 uncertain about the literal lexicon?
But then actual speaker and listener must both marginalize out this uncertainty, so no-one's behavior
is sensitive to the actual literal lexicon, so there is no data to learn it.
Make each recursive agent maintain uncertainty about their model of the next agent down?
Arguably correct in theory, but would require actual speaker and listener to learn hyper-hyper-...hyper-distributions, which is probably impossible even in principle due to data sparsity.

Each agent assumes:(a) There is a speci�c, "conventional" literal lexicon that everyone is supposed to be using,(b) and everyone else knows what this lexicon is, and believes that I know it as well,(c) but in fact I don't know it, and have to do my best to fake it.
Assumption (a) explains why naive language users will argue – falsely! – that words have objective
meanings (the "lexicographer's illusion"). Assumption (b) means that data is available, but avoids theexplosion of hypern-distributions (and is uncomfortably familiar). Assumption (c) means there's
something to learn. When combined with standard Bayesian techniques, these assumptions give ade�nition of what a "convention" is, and provide a mechanism for learning, using, and creating them.

Our model

Learning: uses Ln-1 as generative model

Learning: uses Sn-2 as generative model

References:[1] M. C. Frank, N. D. Goodman, and J. B. Tenenbaum. Using speakers’ referential intentions to model early crosssituational word learning. Psychological Science, 20:578–585, 2009.
[2] M. C. Frank and N. D. Goodman. Predicting pragmatic reasoning in language games. Science,336(6084):998–998, 2012.
[3] L. Bergen, N. D. Goodman, and R. Levy. That’s what she (could have) said: How alternative utterances
affect language use. In Proceedings of the 34th Annual Conference of the Cognitive Science Society, 2012.

Click on the dax.

Using a vague word implies that aspeci�c word does not apply.

Click on the face with glasses.

Click on the bowl with some red balls.
Cheap utterances refer to common objects/situations;
expensive utterances refer to unusual objects/situations.

Horn implicature
Pretend youlive in aworld withthese objects:

Now dothis task:

We set n = 3, λ = 3, andperform inference byimportance sampling oflexicons from a Dirichletprior.

wub.

Click the�ustergubbet.

I (started the car / got the car started).

Learning scalar quanti�ers like "some" and "all"Training data: pragmatically strengthened uses
"some""all"

Learning + Pragmatics

Our solution: Assume conventionality + knowledgeable peers

Convention-based listener L:
Acts like Ln-1 + uncertainty about L0:

In some situations children select the correct object but
then do not retain this mapping. Our model suggests an
intriguing explanation: on single exposures, it selects thenovel object but hypothesizes dax is a vague/genericterm being strengthened by a speci�city implicature.

Black Bart (killed the sheriff / caused the sheriff to die).

Several obvious solutions don't work

Convention-based speaker S:
Acts like Sn + uncertainty about L0:

Disambiguation without learning

Speci�city implicature

Word[1]

Both humans and our model can learn words given only
ambiguous input of this kind.

Marginal literal belief"some"
0.78
0.22

Dirichlet-multinomial: 0.83

0.17

Emergence of novel & ef�cient conventions in interaction
When humans interact, novel and task-adapted communicative systems emerge. We
simulate interactions between agents who begin with uniform priors over lexicons.
Mean communicative
success over time

1.00.80.6
0.4

2x2 uniform priorHorn implicature

0.20.00102030

Dialogue turn40

Performance improves asagents converge on ashared lexicon

Example runs
Simple naming game
Horn implicature game

1.0

0.5

0.0

12

3

4

5

6

78

Dialogue turn

Better run

Worse runwords

Referent

Learning new words using old words

Disambiguating newwords using old words

P(L understands S)

Lexicon

Listener
Given the speakerproduced this word,
what do they desire
me to believe?

Pragmatics

ReferentSpeakerSpeaker
What word willproduce thedesired belief
in my listener?

Learningred

Our problem: What is a word's meaning?
In existinglearning models:a latent variable

Phenomena

Perform once,or repeat overmultiple
iterations

Word usage

Speakerelodm lur d e
O omel]'s[ 3 odmel]'s[ 2 odm[1Our domain: We simulate
language usage in simple
referential games

910

12

Better run

34

5

6

7

8

9

10

Dialogue turnWorse runobjects

Bias towards
sparse lexicons

Bias towards "Horncompatible" lexicons;
implicature shifts tobecome literal meaning→ Biases from "irrational" social anxiety assumption systematically drive thecommunicative system towards greater ef�ciency, and in the long run may leave
their mark on structure of languages themselves.