Modeling Non-Cooperative Dialogue: Theoretical and Empirical Insights (2022)

Abstract: Investigating cooperativity of interlocutors is central in studying pragmatics of dialogue. Models of conversation that only assume cooperative agents fail to explain the dynamics of strategic conversations. Thus, we investigate the ability of agents to identify non-cooperative interlocutors while completing a concurrent visual-dialogue task. Within this novel setting, we study the optimality of communication strategies for achieving this multi-task objective. We use the tools of learning theory to develop a theoretical model for identifying non-cooperative interlocutors and apply this theory to analyze different communication strategies. We also introduce a corpus of non-cooperative conversations about images in the GuessWhat?! dataset proposed by De Vries et al. (2017). We use reinforcement learning to implement multiple communication strategies in this context and find that empirical results validate our theory.

## introduction a robust dialogue agent cannot always assume a cooperative conversational counterpart when deployed in the wild. even in goal-oriented settings, where the intent of an interlocutor may seem to be granted, bad actors and disinterested parties are free to interact with our dialogue systems. these non-cooperative interlocutors add harmful noise to data, which can elicit unexpected behaviors from our dialogue systems. thus, the need to study noncooperation increases daily as we build and deploy conversational systems which interact with peo-ple from different demographics, political views, and intents, continuously learning from the collected data. examples include amazon alexa, task-oriented systems that help patients recovering from injuries or can teach a person a new language, and systems that help predict deceptive behaviors in courtrooms. to effectively communicate in the presence of unwanted behaviors like bullying @xcite , systems need to understand users' strategic behaviors @xcite and be able to identify non-cooperative actions. designing agents that learn to identify non-cooperative interlocutors is challenging since it requires processing the context of the dialogue in addition to modeling the choices that interlocutors make under uncertainty-choices which typically affect their ability to complete tasks unrelated to identifying non-cooperation as well. in light of this, we ask: what communication strategies are effective for identifying non-cooperative interlocutors, while also achieving the goals of a distinct dialogue task? to answer this question, we appeal to a simple non-cooperative version of the visual dialogue game guess what?! @xcite . see figure 1 for an example. the game consists of a multi-round dialogue between two players: a question-player and an answer-player. both have access to the same image whereas only the answerplayer has access to an image-secret; that is, a table ). the answer-player, who may be cooperative or noncooperative, gives binary responses to the questionplayer's queries. in this example, the answer-player is non-cooperative and leads the question-player to an incorrect object (the orange). this is a real example produced by autonomous agents (described in section 5). particular goal-object for the question-player to recognize. the question-player's goal is to ask the answer-player questions which will reveal the secret. a cooperative answer-player then provides good answers to assist in this goal. in the original game, the answer-player is always cooperative. our modified game instead allows the answerplayer to be non-cooperative with some non-zero probability. unlike a cooperative answer-player, a non-cooperative answer-player will not necessarily act in assistance to the question-player, and instead, may attempt to reveal an incorrect secret or otherwise hinder information exchange. in experiments, the specific strategies we study are learned from human non-cooperative conversation. the question-player, importantly, does not know if answer-player is non-cooperative. at the end of the dialogue, the question-player's final objective is not only to identify the goal-object, but also to determine if the conversation takes place with a cooperative or non-cooperative answer-player. we propose a formal theoretical model for analyzing communication strategies in the described scenario. we frame the question-player's objective in terms of two distinct classification tasks and use tools from the theory of learning algorithms to analyze relationships between these tasks. our main theoretical result identifies circumstances where the question-player's performance in identifying non-cooperation correlates with performance in identifying the goal-object. building on this, we provide a mathematical definition of the efficacy of a non-cooperative player which is based on the conceptual idea that cooperation is necessary to make progress in dialogue. our analysis concludes that when the answer-player is effective in this sense, the question-player can gather useful information for both the object identification task and the non-cooperation identification task by selecting a communication strategy based only on the former objective. to test the assumptions of our theoretical model as well as the value of the aforementioned communication strategy in practice, we implement this strategy using reinforcement learning (rl). our experiments validate our theory. as compared to heuristically justified baselines, the communication strategy motivated by our theory yields consistently better results. to conduct this experiment, we have collected a novel corpus of non-cooperative guess what?! game instances which is publicly available. foot_0 throughout experimentation, we provide a qualitative and quantitative analysis of the non-cooperative strategies present in our corpus. these results, in particular, demonstrate that non-cooperative autonomous agents that utilize dialogue history can better deceive question-players. this contrasts the observation of @xcite that cooperative answer-players do not use this information. in total, our work is positioned at the intersection of two foci: detection of non-cooperative dialogue and modeling of non-cooperative dialogue. unlike many detection works, we consider detection in context of interaction. additionally, while many modeling works consider the intent of conversational agents and construct strategies for non-cooperative dialogue based on this, our strategies are motivated purely from a learning theoretic argument. as we are aware, a theoretical description similar to ours has not been given before. ## related works the view that conversation is not necessarily cooperative is not novel, but the argument can be made that it has lacked sufficient investigation in the dialogue literature @xcite . game theoretic investigations of non-cooperation are plentiful, perhaps beginning with work of @xcite . concepts from this space, such as the stochastic games introduced by @xcite , have been used to model dialogue @xcite when non-cooperation between parties is allowed. @xcite also consider a game-theoretic model of speech. in fact, even the dialogue game we consider in this text can be modeled through game-theoretic constructs; for example, a bayesian game @xcite . whereas game theory focuses primarily on analysis of strategies, studying non-cooperation in dialogue requires both the learning of strategies and the learning of utterance meaning. aptly, our use of the theory of learning algorithms (rather than game theory) is suited to handle both of these. while we are first to use learning theory, efforts to characterize non-cooperation in dialogue, learn noncooperative strategies in autonomous agents, and detect non-cooperation in dialogue are not absent from the literature @xcite @xcite . we discuss these topics in detail in the following. ## dataset in this section, we first describe our modified version of the guesswhat?! game. then, we describe the data acquisition process as well as the non-cooperative dataset used in this study. the dataset will be made publicly available upon publication. ## a theoretical model this section formally models the objectives of the question-player as two distinct learning tasks. we use results from the theory of learning algorithms to give a relationship between these tasks in thm 4.1. we then use thm 4.1 to analyze communication strategies in section 4.3. ## conclusion combining tools from learning theory, reinforcement learning, and supervised learning, we model partially non-cooperative communicative strategies in dialogue. understanding such strategies is essential when building robust agents capable of conversing with parties of varying intent. our theoretical and empirical findings suggest non-cooperative agents may sufficiently reveal themselves through their non-cooperative communicative behavior. although the dialogue game studied is simple, the results have ramifications for more complex dialogue systems. our theoretical results, in particular, are not limited in this sense and may apply to designing communication strategies in distinct contexts. as noted in section 4.1.1, the limited assumptions we make facilitate this. for example, classifying intents and asking the right clarification questions is crucial to decision making in dialogue @xcite @xcite . our theory is directly applicable to this setting and could be applied to inform learning objectives for any dialogue agent that asks clarification questions to make a classification. a real-world example of this is the online-banking setting studied by dhole (2020), in which the dialogue agent asks clarification questions to decide the type of account a user would like to open. if we suppose some users may be non-cooperative in this context, our theoretical setup is satisfied: there is some feature space (the dialogues), the label space of user-intents is finite, users are labeled with a binary indicator of cooperation, and the dialogue agent can control the distribution over which it learns by asking clarification questions. our theoretical results should apply to many similar dialogue systems that can ask clarification questions or other types of questions. the only stipulations are that the theoretical setup is satisfied (e.g., in the manner just shown) and that our proposed assumptions on the nature of non-cooperative dialogue still hold (i.e., see section 4.3, def. 4.2). to promote continued research, the collected corpus as well as our code are publicly available. 7 7 ethical considerations we have described a research prototype. the proposed dataset does not include sensitive or personal data. our human subject board approved our protocol. human subjects participated voluntarily and were compensated fairly for their time. the publicly available dataset is fully anonymized. the proposed architecture relies on pretrained models such as word or image embeddings so any harm or bias associated with these models may be present in our model. we believe general methods that propose to mitigate harms can resolve these issues.