Towards Pragmatic Production Strategies for Natural Language Generation Tasks (2022)

Abstract: This position paper proposes a conceptual framework for the design of Natural Language Generation (NLG) systems that follow efficient and effective production strategies in order to achieve complex communicative goals. In this general framework, efficiency is characterised as the parsimonious regulation of production and comprehension costs while effectiveness is measured with respect to task-oriented and contextually grounded communicative goals. We provide concrete suggestions for the estimation of goals, costs, and utility via modern statistical methods, demonstrating applications of our framework to the classic pragmatic task of visually grounded referential games and to abstractive text summarisation, two popular generation tasks with real-world applications. In sum, we advocate for the development of NLG systems that learn to make pragmatic production decisions from experience, by reasoning about goals, costs, and utility in a human-like way.

## introduction novelists choose the right words to keep readers engaged and enthused, good journalists can convey facts clearly and convincingly, while poets may want to surprise the reader. teachers adapt their explanations to the level of their students, and the language of parents changes with the proficiency of their children, with the same objects described first using simplified funny expressions ('moo moo') and then more informative and discriminative names @xcite . using language to communicate successfully requires effort. on the side of the language producer, it is first of all effortful to come up with words that truthfully correspond to one's communicative intent. then, words must be actually produced, e.g. said out loud or typed on a keyboard. at the same time, the producer has to take into consideration whether the comprehender-for whom, too, linguistic communication is costly-will be able to infer the original intent. comprehenders make efforts to pay attention to the utterance they are being addressed with, to interpret it, and to infer their interlocutor's communicative intent. fortunately, these efforts are often not in vain. they allow people to exchange knowledge, ideas, plans, and to achieve goals. this paper introduces a conceptual framework for natural language generation (nlg) in variably complex communicative scenarios, which relies on three main notions: communicative goals, production and comprehension costs, and utility. we define these notions formally and then, in two case studies, we provide suggestions for their operationalisation in classic nlg tasks. in sum, we model humans as decision makers striving for efficient and effective communication, and argue that human-like linguistic behaviour emerges as a result of reasoning about goals, costs, and utility. learning to navigate the complex decision space defined by these notions is still an open problem: we discuss possible promising directions. ## doing things with words communication always comes with a goal: speakers use words to change the state of the world. in this section, we give a characterisation of communicative goals, discuss the types of effort (or costs) necessary to achieve goals, and describe the rewards associated with successful communication. ## case study 1: reference games in this section, we demonstrate how to use our framework to conceptualise a communication scenario that corresponds to a classic nlg task, referring expression generation @xcite . we will also provide concrete examples of how to model the costs and utility described in section 2. in a reference game, the goal is for participants to produce descriptions that allow comprehenders to identify the correct referent out of a set of candidates. these games have been extensively used in psycholinguistics to study human strategies for effective reference @xcite @xcite . for our case study, we use a visually grounded reference game with two participants, a speaker s and a listener a. the speaker produces referring utterances r such as "a boy cutting a cake" and the listener needs to identify the target image i * among a set of similar images v , the visual context (see, e.g., @xcite . the initial state of the world is one where the speaker is aware of the target referent while the listener has no information about it. we can express such a state of the world as @xmath0, p s , p a ), i.e. in terms of the speaker and listener's probability distributions p s and p a over candidate images v before anything is uttered (@xmath1, the empty string): foot_2 p s (i|v ) : p s (@xmath2 note that p s is never observable by a, and for this scenario to be realistic, p a should also not be observable by s. the communicative goal g s is a transformation of w into w * , a state of the world in which a identifies i * as the target referent: how can the costs associated with reaching this state of the world using utterance r be estimated? a computer vision model may be used to encode the communicative goal w * = (v, p s , p ′ a ) into a mental representation. this model receives as input the visual context v and information about the target image p s and yields a mental (abstract) representation @xmath3. if this is, e.g., a model that produces image segmentations, the encoding effort c e can be quantified as the uncertainty of the model over its segmentation decisions, as the number of output image segments, or, if the segments form a scene graph, as a measure of the graph complexity. the encoding e may then be fed to an nlg model r s which realises it into an utterance @xmath4. the realisation cost c r can be computed as the utterance length, the depth of the syntactic tree corresponding to the utterance, or as a function of the distribution of vocabulary ranks for the sampled utterance tokens. next, r is received by the listener, who processes it into a reconstruction of the original mental representation: e ′ = p a (r). this can be achieved using a neural language model, the processing cost c p being calculated as the model's cumulative surprisal (the sum of the per-word information content). from e ′ the listener decodes a state of the world w ′ . the decoding system may be one that measures the similarity of e ′ to candidate image embeddings and outputs a probability distribution over v . the decoding cost c d can be estimated as the entropy reduction with respect to the prior probability p a (i|v ) (the information gain), or as the increase in the target image's probability. communication is successful if p ′ a (@xmath5, r) = 1 (see eq. 5); in practice the condition is often relaxed to: in a simplified reference game where p a is observable by s, the speaker's positive utility u s can be simply modelled as log p ′ a (i * |v, r)log p a (i * |v ). in a more realistic scenario, either the speaker entertains a mental model of p a and uses it to compute utility, or the listener must in turn execute a bit of behaviour to communicate the state of p ′ a , for example by selecting an image through a simple decision rule (e.g., arg max p ′ a ). u s can then be modelled as a binary reward based on the listener's behaviour: 1 for a correct guess, 0 for an incorrect one. recall that u s is not only a function of positive cognitive effects. it is also inversely proportional to the costs c e , c r , c p , and c d . ## case study 2: text summarisation with our second case study, we demonstrate the generality of our framework by applying it to text summarisation, a widely studied nlg (and nlu) task with a large range of practical applications. when people summarise a text, they produce a concise and meaning-preserving version of that text with the goal of conveying to the audience the text's most important ideas. in nlp, texts have been typically summarised either via extraction of their most significant sentences @xcite or by the generation of fewer, new sentences @xcite . here, we look at the second case, often referred to as abstractive summarisation, where a summariser s produces an utterance r made up of one or multiple sentences to succinctly report the main content of a text t to an audience a. the initial state of the world is one where the summariser knows the content of t while the audience has no information about it. summaries can have multiple communicative goals-sometimes simultaneously-roughly corresponding to practical goals of nlp summarisation systems. for example, the communicative goal g s of a summary can be a transformation of the state of the world into one in which a knows the general topic of t and is interested in reading t. this setup roughly corresponds to headline generation, a classic abstractive summarisation task. if the practical goal of the summary, instead, is to make the audience aware of the main facts reported in a text, the communicative goal g s is a transformation of the state of the world into one in which those facts are part of a's knowledge. this is the goal, for example, of summaries of financial, legal, or medical reports. we now look at this second case, providing examples of how to model communicative goals, costs, and utility. a hierarchical language model with explicit attention over multiple sentences can be used to encode the document into a mental representation e. the encoding cost c e can be quantified as the entropy of the attention distribution-the rationale being that it is harder to condense the information in a document in which each sentence contains salient details. the encoding e may then be fed to a generation model r s which realises it into an utterance @xmath6. the realisation cost c r can be computed as the utterance length or as a function of the predicted tokens' probabilities. the summary r is received by the audience, for example via a neural language model pretrained on summaries, which processes it into a reconstruction of the original mental representation: e ′ = p a (r). the processing cost c p can be calculated as the model's cumulative surprisal. from e ′ , the audience decodes a new state of the world, one where it can hopefully answer factual questions about the target document correctly. the decoding system can be a question answering model (which can be as simple as a table-lookup and as complex as a response generation model) and the decoding cost c d can be estimated as the system's reduction in uncertainty in answering a set of questions designed to probe understanding of the main content of the documentformulated, e.g., as key-value queries or using natural language. the speaker's utility u s can be modelled as the accuracy of the audience in answering questions about the content of the document. ## pragmatic production strategies language producers are thought to balance their own production costs and their audience's comprehension costs in a way that minimises joint collaborative effort @xcite while attempting to gain utility from successful communication. nevertheless, most modern nlg systems, whose aim is arguably to reproduce the communicative behaviour of human language users, do not take into consideration the costs and utility for which humans are constantly optimising. as a major example, gpt-3 @xcite , one of the best foundation models currently available for nlg, conflates all costs into a single next-word probability value. to generate words from this model, typically, nextword probabilities are passed to a decoding algorithm such as beam search or nucleus sampling @xcite . this algorithm can be seen as a way to search through the space of possible utterances by following a simple utility-maximising decision rule, with higher probability utterances having higher utility. future work should investigate decision making rules that take into account production and comprehension costs more explicitly, connecting them to the goal of the linguistic interaction. the rational speech act model (rsa; frank and goodman, 2012) is a compelling solution: it was shown to optimise the trade-off between expected utility and communicative effort and it is related to rate-distortion theory @xcite , the branch of information theory that studies the effect of limited transmission resources on communicative success @xcite . its application to simple reference games has indeed demonstrated that richer decision making routines, grounded in listeners' actions and beliefs, result in human-like pragmatic behaviour @xcite . bounded rationality @xcite , which models optimal decision making under constrained cognitive resources, is a strong alternative to rsa theory but there is so far only limited evidence that it can be used to characterise language production choices @xcite . a third, more practically oriented solution, are utility-based decoding algorithms-e.g., minimum bayes risk @xcite decodingwhich have been successfully used to weigh in utilities and costs when selecting utterances for nlg tasks @xcite . modelling and artificially reproducing human communicative behaviour requires advanced decision making algorithms that are able to learn from experience efficient and effective strategies for weighing costs and utility. the learned strategies should apply both to individual utterances and to sequences of utterances: this will allow successful multi-turn planning of communicative subgoals and strategies. reinforcement learning (rl) can naturally interact with notions of cost and utility (these can be used as learning signal for rl models, or they can be inferred by rl models from observations of human behaviour) and it has been used in combination with rsa and bounded rationality; it thus appears to be a promising avenue for the strategy learning problem. independent of the choice of language modelwhich is an important open question-we believe that our conceptual framework can account for a variety of human behavioural patterns of communication as described in pragmatics, the field of linguistics which studies the aspects of language use that involve reasoning about context, goals and beliefs. let us take as an emblematic example grice's four maxims of conversation @xcite . the maxim of quantity, which states that speakers should make their contribution as informative as required for the current purposes of the exchange, can be understood as the optimisation of realisation and processing costs, c r and c p , while ensuring that the distance from the communicative goal is reduced. the maxim of quality, which is about making truthful contributions, can be thought of as the result of minimising decoding cost c d and maximising the probability of achieving the communicative goal. the maxim of relation, stating that speakers should provide information that is relevant to the exchange, can be seen as a way to ensure that production and comprehension costs are always balanced by gains in positive utility. finally, the maxim of manner states that speakers should avoid obscurity of expression, ambiguity, and strive for brief and orderly contributions. this can be easily understood as the optimisation of realisation and processing cost, c r and c p , given fixed encoding and decoding costs c e and c d . ## conclusion we have presented a conceptual framework for natural language generation that relies on three central notions: communicative goals, production and comprehension costs, and utility optimisation. we have defined these notions formally and demonstrated their application to two realistic communication scenarios, providing examples for the modelling of goals, costs, and utility with modern method of statistical learning. we have further argued for our framework's ability to account for a variety of pragmatic patterns of communicative behaviour, highlighting the importance of the development of new complex decision making algorithms that learn to reproduce human-like production strategies from experience.