Learning to Mediate Disparities Towards Pragmatic Communication (2022)

Abstract: Human communication is a collaborative process. Speakers, on top of conveying their own intent, adjust the content and language expressions by taking the listeners into account, including their knowledge background, personalities, and physical capabilities. Towards building AI agents with similar abilities in language communication, we propose a novel rational reasoning framework, Pragmatic Rational Speaker (PRS), where the speaker attempts to learn the speaker-listener disparity and adjust the speech accordingly, by adding a light-weighted disparity adjustment layer into working memory on top of speaker’s long-term memory system. By fixing the long-term memory, the PRS only needs to update its working memory to learn and adapt to different types of listeners. To validate our framework, we create a dataset that simulates different types of speaker-listener disparities in the context of referential games. Our empirical results demonstrate that the PRS is able to shift its output towards the language that listeners are able to understand, significantly improve the collaborative task outcome, and learn the disparity more efficiently than joint training.

## introduction in human communication, speakers often adjust their language production by taking into consideration listeners' personality, background knowledge, perceptual or physical capabilities etc @xcite . recent years have seen an increasing amount of work that explores pragmatic reasoning based on rational speech act (rsa) @xcite @xcite , multi-agent emergent communication framework @xcite , and theory of mind in communication @xcite . however, except for @xcite , figure 1 : task: given two images, the speaker generates a description for the target image and asks the listener to pick out the image described. both players win if the listener picks the correct one. in this example, a literal speaker could generate multiple captions that suit the target, such as the one above, whereas a rational speaker limits the description to the unique features of the target (e.g. pizza). if the listener only understands the hypernym of food (disparity), a pragmatic rational speaker would learn the disparity and use food instead of pizza to help the listener understand. to address this limitation, this paper extends the rational speech act (rsa) (frank and goodman, 2012) model towards rational agents learning to adapt behaviors based on their experience with the listener. the design choice of our model is inspired by the human cognitive system @xcite where a limited capacity working memory is built on top of the long-term memory to adjust the output to be task and environment specific. each communication is a modification on the long-term memory @xcite with situation-specific factors. in our framework, we fix the long-term memory which captures lan-guage structure for communication, and introduce a light-weighted working memory @xcite for the pragmatic rational speaker to modify and accommodate two goals: 1) a task goal which retrieves relevant information from the long-term memory and accomplish the task, and 2) a disparity goal which learns and adjusts the conversation to accommodate the listener's disparity through reinforcement learning. we separate each component as they are independent of each other in utility, and can be easily switched and adapted for new tasks and new environment. different from previous works which only demonstrate how learned models affect task performance (e.g. @xcite @xcite ), one of our goals is to also provide transparency on what models have indeed learned towards the end goal. it's well established that end-to-end neural models can often take advantage of spurious data bias to gain end performance. models that only report end measure without showing their internal works would not be sufficient to tell the whole story about model's abilities. to serve this goal, we situated our investigation in the context of a referential game foot_0 as shown in figure 1 . we carefully curated a dataset to simulate two types of disparity: knowledge disparity and perceptual disparity. our empirical results demonstrate that our model is able to significantly improve the collaborative game performance by shifting communication towards the language that the listeners with disparities are able to understand. in addition, our results show that separating working memory from long-term memory leads to faster learning and better performance than the previous model which conducted joint end-to-end learning. our contributions are the following. 1) following human cognition, we demonstrate the benefits of separating working memory from the long-term memory, compared to end-to-end joint training. 2) we propose a new dataset to simulate multiple distinct types of disparities, and demonstrate the pragmatic adaptability of our model. 3) instead of focusing on mere end task performance, we show model's strong language shift ability to accommodate listener's disparities. the dataset and code are available through https://github.com/sled-group/ pragmatic-rational-speaker to facilitate future work on pragmatics and theory of mind in language interpretation and generation. ## related work it has been studied @xcite @xcite in psychology that human speakers adjust the way how we speak for successful communication after learning the listener's disparity. some recent work @xcite @xcite attempt to address similar questions. we build our model upon the following two concepts. ## dataset there are many levels of disparities during verbal communication @xcite , including phonetic, lexical, grammatical, semantic representations, etc. in our work, we assembled two datasets, and challenge the speaker model to handle two types of disparities: 1) knowledge disparity, and 2) perceptual disparity. the knowledge disparity is simulated through the hypernym dataset, where the listener only understands the hypernym for all the objects (e.g. "food" instead of "pizza"), whereas the speaker understands both. this dataset challenges the speaker model at the lexical level to learn what listener's vocab limitation, and shift towards the words that they understand. the perceptual disparity is simulated through the limited visual dataset, where the listener has impaired vision or some objects were physically blocked from the eyesight. this dataset challenges the speaker to shift attention and pick the visible objects for the listener to describe. for control and demonstration purposes, we remove all the animalrelated objects and words from listener's training. these datasets are used to simulate listener's disparities and train the listener's model as described in section 4.2. the speaker's long term memory was trained with the original data which has full knowledge of the vocab and objects, but no idea what the listeners are or aren't capable of. detailed dataset components can be found in the appendix. we modified the abstract scenes @xcite dataset for our experiments. there are 10020 images, each including 3 ground truth captions, and a median of 6 to 7 objects. we assembled ∼35k pairs of images that differ @xmath0, ∼25k pairs that differ @xmath1, and together as the combined set. the image pairs were split into training, validation and testing by a ratio of 8:1:1. ## method given a pair of images m 0 , m 1 , the target image indicator @xmath2, 1}, and the listener's disparity d, the speaker generates a caption c for the target image m t , and the listener needs to pick out the correct target t given c. both receive a reward of +1 upon correct choice, and -1 otherwise. following the rsa model, as shown figure 2 , we start by building the literal speaker s 0 , gradually increase model structure and functionality with the vanilla rational speaker s 1 and the pragmatic rational speaker s d 1 . upon retrieving a list of candidate captions c from the long-term memory, the final goal for s d 1 is to output the best caption c in the working memory, that accommodates both 1) task goal: describes the unique features of the target image, and 2) disparity goal: learns and accommodates the listener's disparity. ## results and analysis we evaluate our models (s 0 , s 1 , s d 1 ) on the referential game (figure 1 ) ## conclusion and future work in this work, we present a novel framework based on the rational speech act framework for pragmatic communication that can adjust the conversation content according to listener's disparities by adding a light-weighted working memory on top of speaker's long-term memory. the pragmatic rational speaker significantly improved the collaborative game performance by shifting the conversation towards the language that the listeners are able to understand. the flexibility and training efficiency also makes it easy to be applied broadly. there are, however, several limitations that requires further investigation. first of all, despite recent progress, algorithms that connect language and the visual world are still limited. for example, caption generation, even in this simple setup, often does not faithfully capture what's been conveyed in the images. as our framework heavily relies on the quality of various models that bridge language and vision, e.g., as part of our long term memory, it's important to improve functionality and performance of these base models. we conducted our experiments in a relative simple and artificial environment with the purpose of easy control and demonstration. we emphasize on evaluating model's actual language ability of adjusting for the disparities on top of task performance. the next step would be to apply the framework to more realistic images and interactive environment. other than listener's knowledge background and perceptual capabilities, there are a lot of other reasons for language communication to be adjusted, such as the physical environment, relative positions, speaker's personalities, etc. studying how a rational agent can accommodate these disparities would require additional multimodal datasets and information processing methods. at the moment, the pragmatic rational speaker trains a new layer in working memory from scratch for each type of disparity. this could have backward influence on the long-term memory. in lifelong learning @xcite like humans, the working memory can shape their long-term memory. at the very least, the model could store each learned disparity adjustments for future encounter. this modification is left for future work. last but not least, instead of training for every single type of disparity to name, human learners have the ability of meta-learning and zero-shot transferring existing knowledge to a new category. future work on pragmatic reasoning should be easily adaptable to different disparities and situations.