Probing Neural Dialog Models for Conversational Understanding (2020)

Abstract: The predominant approach to open-domain dialog generation relies on end-to-end training of neural models on chat datasets. However, this approach provides little insight as to what these models learn (or do not learn) about engaging in dialog. In this study, we analyze the internal representations learned by neural open-domain dialog systems and evaluate the quality of these representations for learning basic conversational skills. Our results suggest that standard open-domain dialog systems struggle with answering questions, inferring contradiction, and determining the topic of conversation, among other tasks. We also find that the dyadic, turn-taking nature of dialog is not fully leveraged by these models. By exploring these limitations, we highlight the need for additional research into architectures and training methods that can better capture high-level information about dialog.

## introduction open-domain dialog systems often rely on neural models for language generation that are trained end-to-end on chat datasets. end-to-end training eliminates the need for hand-crafted features and task-specific modules (for example, for question answering or intent detection), while delivering promising results on a variety of language generation tasks including machine translation @xcite , abstractive summarization @xcite , and text simplification @xcite . however, current generative models for dialog suffer from several shortcomings that limit their usefulness in the real world. neural models can be opaque and difficult to interpret, posing barriers to their deployment in safety-critical applications such as mental health or customer service @xcite . end-to-end training provides little insight as to what these models learn about engaging in dialog. open-domain dialog systems also struggle to maintain basic conversations, frequently ignoring user input @xcite while generating irrelevant, repetitive, and contradictory responses @xcite @xcite . in light of these limitations, we aim to answer the following questions: (i) do neural dialog models effectively encode information about the conversation history? (ii) do neural dialog models learn basic conversational skills through end-to-end training? (iii) and to what extent do neural dialog models leverage the dyadic, turn-taking structure of dialog to learn these skills? to answer these questions, we propose a set of eight probing tasks to measure the conversational understanding of neural dialog models. our tasks include question classification, intent detection, natural language inference, and commonsense reasoning, which all require high-level understanding of language. we also carry out perturbation experiments designed to test if these models fully exploit dialog structure during training. these experiments entail breaking the dialog structure by training on shuffled conversations and measuring the effects on probing performance and perplexity. we experiment with both recurrent @xcite and transformer-based @xcite open-domain dialog models. we also analyze models with different sizes and initialization strategies, training small models from scratch and fine-tuning large pre-trained models on dialog data. thus, our study covers a variety of standard models and approaches for open-domain dialog generation. our analysis reveals three main insights: our code integrates with and extends parlai @xcite , a popular open-source platform for building dialog systems. we also publicly release all our code at https://github.com/ abdulsaleh/dialog-probing , hoping that probing will become a standard method for interpreting and analyzing open-domain dialog systems. ## related work evaluating and interpreting open-domain dialog models is notoriously challenging. multiple studies have shown that standard evaluation metrics such as perplexity and bleu scores @xcite correlate very weakly with human judgements of conversation quality @xcite @xcite . this has inspired multiple new approaches for evaluating dialog systems. one popular evaluation metric involves calculating the semantic similarity between the user input and generated response in high-dimensional embedding space @xcite @xcite @xcite . @xcite proposed calculating conversation metrics such as sentiment and coherence on self-play conversations generated by trained models. similarly, @xcite use neural classifiers to identify whether the modelgenerated responses entail or contradict user input in a natural language inference setting. to the best of our knowledge, all existing approaches for evaluating the performance of opendomain dialog systems only consider external model behavior in the sense that they analyze properties of the generated text. in this study, we explore internal representations instead, motivated by the fact that reasonable internal behavior is crucial for interpretability and is often a prerequisite for effective external behavior. outside of open-domain dialog, probing has been applied for analyzing natural language processing models in machine translation @xcite and visual question answering @xcite . probing is also commonly used for evaluating the quality of "universal" sentence representations which are trained once and used for a variety of applications @xcite (for example, infersent @xcite , skipthought @xcite , use @xcite ). along the same lines, natural language understanding benchmarks such as glue @xcite and superglue @xcite propose a set of diverse tasks for evaluating general linguistic knowledge. our analysis differs from previous work since it is focused on probing for conversational skills that are particularly relevant to dialog generation. with regard to perturbation experiments, @xcite found that standard dialog models are largely insensitive to perturbations of the input text. here we introduce an alternative set of perturbation experiments to similarly explore the extent to which dialog structure is being leveraged by these models. ## probing tasks the probing tasks selected for this study measure conversational understanding and skills relevant to dialog generation. some tasks are inspired by previous benchmarks @xcite , while others have not been explored before for probing. examples are listed in the supplemental material. trec: question answering is a key skill for effective dialog systems. a system that deflects user questions could seem inattentive or indifferent. in order to correctly respond to questions, a model needs to determine what type of information the question is requesting. we probe for question answering using the trec question classification dataset @xcite , which consists of questions labeled with their associated answer types. dialoguenli: any two turns in a conversation could entail each other (speakers agreeing, for example), or contradict each other (speakers disagreeing), or be unrelated (speakers changing topic of conversation). a dialog system should be sensitive to contradictions to avoid miscommunication and stay aligned with human preferences. we use the dialogue nli dataset @xcite , which consists of pairs of dialog turns with entailment, contradiction, and neutral labels to probe for natural language inference. the original dataset examines two utterances from the same speaker ("i go to college", "i am a student"), so we modify the second utterance to simulate a second speaker ("i go to college", "you are a student"). multiwoz: every utterance in a conversation can be considered as an action or a dialog act performed by the speaker. a speaker could be making a request, providing information, or simply greeting the system. multiwoz 2.1 @xcite is a dataset of multi-domain, goal-oriented conversations. human turns are labeled with dialog acts and the associated domains (hotel, restaurant, etc.), which we use to probe for natural language understanding. sgd: tracking user intent is also important for generating appropriate responses. the same intent is often active across multiple dialog turns since it takes more than one turn to book a hotel, for example. determining user intent requires reasoning over multiple turns in contrast to dialog acts which are turn-specific. to probe for this task, we use intent labels from the multi-domain, goaloriented schema-guided dialog dataset @xcite . wnli: endowing neural models with commonsense reasoning is an ongoing challenge in machine learning @xcite . we use the winograd nli dataset, a variant of the winograd schema challenge @xcite , provided in the glue benchmark @xcite to probe for commonsense reasoning. wnli is a sentence pair classification task where the goal is to identify whether the hypothesis correctly resolves the referent of an ambiguous pronoun in the premise. ## limitations some of our conclusions assume that probing performance is indicative of performance on the endtask of dialog generation. yet it could be the case that certain models learn high quality representations for probing but cannot effectively use them for generation, due to a weakness in the decoder for example. to address this limitation, future work could examine the relationship between probing performance and human judgements of conversation quality. @xcite argues more research on the causal relation between probing and end-task performance is required to address this limitation. however, it is reasonable to assume that capturing information about a certain probing task is a pre-requisite to utilizing information relevant to that task for generation. for example, a model that cannot identify user sentiment is unlikely to use information about user sentiment for generation. we also find that lower perplexity (better data fit) is correlated with better probing performance ( ## conclusion we use probing to shed light on the conversational understanding of neural dialog models. our findings suggest that standard neural dialog models suffer from many limitations. they do not effectively encode information about the conversation history, struggle to learn basic conversational skills, and fail to leverage the dyadic, turn-taking structure of dialog. these limitations are particularly severe for small models trained from scratch on dialog data but occasionally also affect large pre-trained models. addressing these limitations is an interesting direction of future work. models could be augmented with specific components or multi-task loss functions to support learning certain skills. future work can also explore the relationship between probing performance and human evaluation.