Regularized Conventions: Equilibrium Computation as a Model of Pragmatic Reasoning (2024)

Abstract: We present a game-theoretic model of pragmatics that we call ReCo (for Regularized Conventions). This model formulates pragmatic communication as a game in which players are rewarded for communicating successfully and penalized for deviating from a shared, “default” semantics. As a result, players assign utterances context-dependent meanings that jointly optimize communicative success and naturalness with respect to speakers’ and listeners’ background knowledge of language. By using established game-theoretic tools to compute equilibrium strategies for this game, we obtain principled pragmatic language generation procedures with formal guarantees of communicative success. Across several datasets capturing real and idealized human judgments about pragmatic implicature, ReCo matches, or slightly improves upon, predictions made by Iterated Best Response and Rational Speech Acts models of language understanding.

## introduction meaning in language is fluid and context-sensitive: speakers can use the word blue to pick out a color that in other contexts would be described as purple, or identify a friend as the one with glasses in a room in which everyone is wearing glasses (figure 1 ). such context-dependent meanings can arise as conventions among language users communicating repeatedly to solve a shared task @xcite . but remarkably, they can also arise without any interaction at all, among language users who share only common knowledge of words' default meanings @xcite . what makes this kind of context-dependent pragmatic language use possible? almost all existing computational models of pragmatics are implemented as recursive reasoning procedures, in which listeners interpret utterances by reasoning about the intentions of less-sophisticated speakers @xcite . these models have been successful at explaining a number of aspects of pragmatics. but they can be challenging to fit to real data: because they specify speaker and listener behavior procedurally, rather than in terms of a shared objective, recursive reasoning models can be highly sensitive to implementation-level details (e.g. the number of "levels" of reasoning). we present an alternative model of pragmatic understanding based on equilibrium search rather than recursive reasoning. in this model (which we call regularized conventions, or reco), speakers and listeners solve communicative tasks like figure 1 : the reco model. to communicate (or resolve) an intended meaning from a set of possibilities (a), language users search for distributions over utterances and interpretations that are close to some "default semantics" (b) and close to a (game-theoretically) optimal signaling convention (d). the resulting "regularized conventions" (c) predict human judgments on a variety of pragmatic implicature tasks. reco is by no means the first application of game-theoretic tools to model pragmatic language understanding @xcite @xcite -in fact, many recursive reasoning models (e.g. franke, 2009a) also have a game-theoretic interpretation. but by leveraging recently developed algorithmic tools for computing regularized equilibria of games, reco can efficiently learn models of pragmatic communication from data, while providing formal guarantees about communicative success and deviation from default semantics. the algorithms that compute these equilibria turn out to have a very similar structure to some probabilistic recursive reasoning methods (e.g. @xcite , offering a bridge between procedural characterizations of pragmatic reasoning and reco's optimality-based characterization. most importantly, reco gives a good fit to human data: on classic exemplars of pragmatic implicature, reference tasks eliciting graded human judgments, and tasks featuring perceptually complex meaning spaces, its predictions match (and sometimes modestly outperform) standard recursive reasoning models. these results show that game-theoretic approaches offer a viable foundation for expressive, learned models of pragmatic communication, and highlight the usefulness of the modern game-theoretic toolkit in more general systems for language production and comprehension. ## background and preliminaries consider again the example in figure 1 . we wish to understand the process by which a speaker might use blue to refer to the second color in the second row, and by which a listener might resolve it correctly. ## our approach: pragmatic inference as regularized equilibrium search the key idea underlying reco is to use regularized equilibrium concepts to describe pragmatic communication, by modeling listeners and speakers as directly optimizing both communicative success and adherence to existing linguistic conventions. as noted in section 2.2, simply searching for highutility equilibria of signaling games is unlikely to predict the behavior of human language users, or result in successful communication with new interlocutors: instead, we must guide inference toward policies that look like natural language. in reco, we do so by optimizing utilities of the following form: here τ s and τ l represent the speaker's and lis-tener's prior knowledge of language (independent of any specific communicative goal or context). we refer to these policies as the default semantics in the language used for communication. they play a similar role to the literal semantics used by rsa and other iterated response models. but here, we need not assume that they correspond specifically to literal semantics-instead, they model agents' prior expectations about how utterances are likely to be produced and interpreted in general by pragmatic language users. the regularization parameters λ s and λ l control the tradeoff between optimizing for communicative success and proximity to default semantics τ s , τ l . when the value of λ i is large, an agent @xmath0, l} will consider only policies extremely close to τ i ; conversely, when λ i is close to zero, the agent will not be penalized for adopting semantics that differ significantly from τ i . ## two model problems: q-implicature and m-implicature we begin with two simple, widely studied "model problems" in pragmatics: quantity implicature and manner implicature. the experiments in this section aim to demonstrate that reco makes predictions that agree qualitatively with key motivating examples in theories of pragmatics. ## probabilistic human judgments we next study a family of four reference tasks introduced by frank (2016), which we refer to as simple , complex , twins and oddman . we refer readers to the original work for the default meanings that define each of these tasks. frank gathered graded human judgments about the probability that particular utterances might carry particular meanings. as reco, like rsa-family models, captures probabilistic associations between utterances and meanings, we evaluate its predictions by measuring their correlation between human judgments. specifically, for each task (and all tasks jointly), we compute the correlation between p(meaning | utterance) predicted by the model, and the average p(meaning | utterance) predicted by humans (with one data point for each (meaning, utterance, context) triple). we refer the reader to @xcite for more details about the experimental setup. comparisons between reco, rsa, br speaker (i.e., best-response to a literal speaker) and rd-rsa @xcite are shown in ## complex referents and utterances our final experiments focus on colors in context ( cic ), a dataset of color reference tasks like the one in figure 1 featuring a more complex space of meanings and a larger space of utterances. another example from the dataset (introduced by @xcite is given in ## conclusion our work can be interpreted as a response to the observation by @xcite that "it is not so clear whether the solution concept of a nash equilibrium (or strengthenings thereof) is really appropriate to model the action of rational agents in one-shot @xcite games." equilibriumfinding models of language understanding have been widely studied in the context of iterated communication and language evolution, where there is a clear mechanism by which groups of language users might collectively arrive at one of many game-theoretically optimal equilibria @xcite @xcite . these studies characterize the stability of population-level conventions in game-theoretic terms. however, as noted in @xcite , such approaches have historically struggled to explain how language understanding occurs in one-shot settings. reco, by way of "regularized equilibrium" concepts, offers an explanation for these behaviors, and in doing so bridges the gap between past evolutionary work and the one-shot inference problems that are of special interest in pragmatics. looking ahead, reco can be used as a platform for studying related problems in context-dependent, multi-party communication. for example, it might be possible to study iterated conventions @xcite , established over multiple rounds of communication, by updating the default semantics τ to the equilibrium policy at the previous round. while our experiments here have focused on singleturn interactions, tools for solving extensive-form games might similarly be used to model communicative strategies that play out over multiple turns of dialog. more generally, we hope these results highlight the effectiveness of game theoretic tools for understanding and enriching models of pragmatic language production and comprehension. using the adam optimizer with learning rate 10 -4 for 25 epochs. we trained the models using pytorch @xcite and huggingface @xcite libraries. these models were trained using a single v100 gpu for 3-4 hours. all other experiments were performed on an 8-core intel cpus and m2 macbook pro. for experiments in section 5, reco was run with 10 seeds and the run with the highest sum of regularized utilities of the speaker and listener was used. the parameter η was set to 0.1 in all our experiments.