Robust Task-Oriented Dialogue Generation with Contrastive Pre-training and Adversarial Filtering (2022)

Abstract: Data artifacts incentivize machine learning models to learn non-transferable generalizations by taking advantage of shortcuts in the data, andthere is growing evidence that data artifacts play a role for the strong results that deep learning models achieve in recent natural language processing benchmarks.In this paper, we focus on task-oriented dialogue and investigate whether popular datasets such as MultiWOZ contain such data artifacts.We found that by only keeping frequent phrases in the trainingexamples, state-of-the-art models perform similarly compared to the variant trained with full data, suggesting they exploit these spurious correlationsto solve the task. Motivated by this, we propose a contrastive learning based framework to encourage the model to ignore these cues and focus on learning generalisable patterns. We also experiment with adversarial filtering to remove easy training instances so that the model would focus on learning from the harder instances. We conduct a number of generalization experiments — e.g., cross-domain/dataset and adversarial tests — to assess the robustness of our approach and found that it works exceptionally well.

## introduction task-oriented dialogue systems aim to help human accomplish certain tasks such as restaurant reservation or navigation via natural language utterances. recently, pre-trained language models @xcite @xcite achieve impressive results on dialogue response generation and knowledge base (kb) reasoning, two core components of dialogue systems. however, neural networks are found to be prone to learning data artifacts @xcite , i.e. superficial statistical patterns in the training data, and as such these results may not generalise to more challenging test cases, e.g., test data that is drawn from a different distribution to the training data. this issue has been documented in several natural language processing (nlp) tasks @xcite @xcite . for example, in natural language inference (nli), where the task is to determine whether one given sentence entails the other, the models trained on nli benchmark datasets are highly likely to assign a "contradiction" label if there exists a word not in the input sentences even if the true relation is "entailment", as not often co-occurs with the label "contradiction" in the training set. similar issues have also been observed in many other tasks such as commonsense reasoning @xcite , visual question answering @xcite , and argument reasoning @xcite . however, it's unclear whether such shortcuts exist in popular task-oriented dialogue datasets such as multiwoz @xcite , and whether existing dialogue models are genuinely learning the underlying task or exploiting biases foot_0 hidden in the data. to investigate this, we start by probing whether state-of-the-art dialogue models are discovering and exploiting spurious correlations on a popular task-oriented dialogue dataset. specifically, we measure two state-of-the-art dialogue models' performance under two different configurations: full input (original dialogue history, e.g., i need to find a moderately priced hotel) and partial input (dialogue history that contains only frequent phrases, e.g., i need to). preliminary experiments found that these models perform similarly under the two configurations, suggesting that these models have picked up these cues -frequent word patterns which are often not meaning bearing -to make predictions. this implies that these models did not learn transferable generalizations for the task, and will likely perform poorly on out-of-distribution test data, e.g., one that has a different distribution to the training data. to address this, we decompose task-oriented dialogue into two task: delexicalized response generation and kb reasoning (prediction of the right entities in the response), and explore methods to improve model robustness for the latter. using frequent phrases as the basis of dataset bias, we experiment with contrastive learning to encourage the model to ignore these phrases to focus on meaning bearing words. specifically, we pre-train our language model with a contrastive objective to encourage it to learn a similar representation for an original input (e.g., i need to find a moderately priced hotel) and its debiased pair (e.g., find a moderately priced hotel) before fine-tuning it for kb entity prediction. another source of bias comes from the data distribution @xcite . we found that the kb entity distribution in multiwoz can be highly skewed in certain contexts, e.g., if the dialogue context starts with i need to, the probability of the kb entity cambridge substantially exceeds chance level, which leads to inadequate learning of entities in the tail of the distribution. here we adapt an adversarial filtering algorithm @xcite to our task, which filters "easy samples" (i.e., samples in the head of the distribution) in the training data to create a more balanced data distribution so as to encourage the model to learn from the tail of the distribution. we conduct a systematic evaluation on the robustness of our method and four state-of-theart task-oriented dialogue systems under various out-of-distribution settings. experimental results demonstrate that our method substantially outperforms these benchmark systems. to summarize, our contributions are as follows: more recently, the field has shifted towards using largescale pre-trained language models such as bert @xcite and gpt-2 @xcite for task-oriented dialogue modeling due to their success on many nlp tasks @xcite @xcite . peng et al. (2020) and hosseini-asl et al. (2020) employed a gpt-2 based model jointly trained for belief state prediction and response generation in a multi-task fashion. wu et al. (2020) pre-train bert on multiple task-oriented dialogue datasets for response selection. spurious cues in nlp dataset deep neural networks have achieved tremendous progress on many nlp tasks @xcite @xcite @xcite with the emergence of large language models like bert @xcite and gpt-2 @xcite . however, many recent nlp studies @xcite @xcite @xcite have found that deep neural networks are prone to exploit spurious artifacts present in the data rather than learning the underlying task. in natural language inference (nli), @xcite found that certain linguistic phenomenon in the nli benchmark datasets correlate well with certain classes. for example, by only looking at the hypothesis, simple classifier models can perform as well as the model using full inputs (both hypothesis and premise). @xcite found that bert achieves a performance close to human on argument reasoning comprehension task (arct) with 77% accuracy (3% below human performance). however, they discover that the impressive performance is attributed to the exploitation of shortcuts in the dataset. @xcite analyze annotator bias on nlp datasets and found that a model that uses only annotator identifiers can achieve a similar performance to one that uses the full data. in commonsense reasoning, @xcite has performed a systematic investigation over four commonsense related tasks and found that most datasets experimented with are problematic with models are prone to leveraging the non-robust features in the inputs to make decisions and do not generalize well to the overall tasks intended to be conveyed by the commonsense reasoning tasks and datasets. inspired by these studies, our paper focus on eliminating spurious correlations in task-oriented dialogue systems. ## experiments to verify the effectiveness of our proposed debiasing approach, we conduct a comprehensive study comparing our model against a number of benchmark systems. our experiments include crossdomain/dataset generalization test, adversarial samples (created by distorting words and sentences), and utterances featuring unseen n-grams. ## conclusion in this work, we propose a contrastive learning framework to debias task-oriented dialogue models by encouraging models to ignore these frequent phrases to focus on semantic words in the input. we also adapt adversarial filtering to our task to further improve model robustness. we conduct a series of generalization experiments, testing our method and a number of state-of-the-art benchmarks. experimental results show that contrastive learning and adversarial filtering complement each other, and combining both produces the most robust dialogue model.