Neural Conversational QA : Learning to Reason vs Exploiting Patterns (2020)

Abstract: Neural Conversational QA tasks such as ShARC require systems to answer questions based on the contents of a given passage. On studying recent state-of-the-art models on the ShARC QA task, we found indications that the model(s) learn spurious clues/patterns in the data-set. Further, a heuristic-based program, built to exploit these patterns, had comparative performance to that of the neural models. In this paper we share our findings about the four types of patterns in the ShARC corpus and how the neural models exploit them. Motivated by the above findings, we create and share a modified data-set that has fewer spurious patterns than the original data-set, consequently allowing models to learn better.

## introduction sharc, a conversational qa task @xcite , requires a system to answer user questions based on rules expressed in natural language text. an example in figure 1 shows a user sharing some background information (referred to as scenario) and asking a question about continuing to pay for 'uk national insurance'. the rule text associated with this dialog exchange defines the policy that guides the conversation flow. at any turn in the conversation, a system may choose to respond with a final yes/no answer; ask a follow-up question to obtain more information from the user; or reply that the question is irrelevant to the context. several deep learning models such as bert-qa @xcite , e3 @xcite , and bison @xcite perform reasonably well on this task. however, our exploration of the sharc dataset indicates that there are multiple spurious patterns that could be exploited by neural models. we observe that the performance of the models mentioned above drops when they are tested on a perturbed dataset, suggesting that ## patterns in the sharc dataset this section describes the spurious patterns in the sharc dataset and presents a simple heuristic based program designed to exploit these patterns. pattern 1: last follow-up answer is the predicted answer: based on the asterisk ('*') as a separator between rule clauses, we found that 54.52% of the instances consist of a list of only-conjunctive or only-disjunctive clause conditions. consider a case where a rule consists of only conjunctive clause statements. if any single follow-up question, generated based on one of these clauses, is answered with a 'no' by a user, the answer to the user's question shall be 'no'. in this case, no follow-up questions need to be asked. thus, one often finds the sequence of follow-up answers in the dialog history as ('yes', . . . , 'yes', 'no'), for which the answer to the user question is a 'no'. similarly, corresponding to a case where the rule consists of only disjunctive clauses, the follow-up answer sequence is ('no', . . . , 'no', 'yes'), the answer to which is a 'yes'. this indicates a high correlation between the final answer and the last answer of the history. we found that 74.6% of the instances in the train set with a 'yes'/'no' answer have the same answer as that of the last follow-up question. although this is reflective of real-world conversations, a model can do a good job on this task by exploiting just this pattern. pattern 2: likelihood of asking a follow-up question decreases with number of turns: it is intuitive to expect that as the number of followup questions that have been asked increases, the likelihood of asking another follow-up question decreases (clauses are finite). appendix a.1, contains an empirical study on the training data, and demonstrates the decrease in the probability of asking a follow-up question with the increase in the number of turns in the dialog history. pattern 3: follow-up questions occur in the same sequence as the rule clauses in the passage: as discussed earlier, many of the rule clauses tend to be conjunctive/disjunctive. thus, the next follow-up question that one needs to ask is not unique, since one can always consider any of the statements in the clause that has not been considered so far. however, the ground truth data considers these clauses in sequential order to generate the follow-up questions. among all instances where a conjunctive/disjunctive clause can be discerned and have a follow-up question generated as a part of the ground truth, 62.8% satisfy the condition that the first clause that has not yet been asked is indeed the next follow-up question. we explain this pattern in detail and discuss how it affects computation of the bleu metric in appendix a.2. pattern 4: answer as 'irrelevant': amongst the train instances where user background information and dialog history is empty, 66.67% have the final answer as irrelevant. ## evaluation metrics in sharc the following metrics are used in the evaluation of the sharc task: our experiments with the dataset suggest two weaknesses in the evaluation of follow-up questions which we discuss below. 1. incomplete reference set: recall that pattern 3 suggests the existence of a sequential correlation between the rule clauses 1 in the rule text and the follow-up questions. this means that if an out of sequence follow-up question is generated by a model, then it is incorrectly penalized because the evaluation script expects the next follow-up question that would have occured in the original rule sequence. to mitigate this penalization, we create a list of alternative candidate references using the clauses in conjunctive-only or disjunctive-only instances in dataset. we make use of the standard implementation of bleu which supports multiple references @xcite . to generate multiple candidate references for the sharc dataset, we identify instances which have a follow-up question as the gold answer and the follow-up question seems to be based on one of the clause statements in the rule text. we then create alternative follow-up questions from each of the clause statements which have not been a part of any follow-up question in the history. please see appendix a.3 for details about the algorithm to generate these alternative follow-ups. these alternative follow-up questions constitute a set of candidate questions. in the rest of the paper, we refer to this bleu score computed using multiple references as multi-bleu, to distinguish it from the officially reported bleu metric. 2. improper penalization for bleu: as mentioned in the section 1, the official evaluation scripts 1 conjunctive-only / disjunctive-only clauses do not penalize bleu scores of a model if it does not predict a follow-up question, and rather predicts a final answer (yes/no or irrelevant). this is because it considers only those cases where both the ground-truth and the model predictions are followups. a hypothetical model which classifies only 1 test instance as follow-up and produces it perfectly can get a bleu score of 100 in this metric. we therefore update the evaluation script to penalize bleu score in such cases, and refer to this as bleu-p (bleu-penalized). this considers all instances where the ground truth is a follow-up question. we use the predicted response from the model as the answer to evaluate. this effectively counts a bleu of 0 in almost all cases for these instances. when we use multiple references for computing bleu, we refer to this as multi-bleu-p. ## evidence of patterns to prepare a perturbed dev set, we modify the official dev set by shuffling the dialog turns of the history of a conversation (we modify approximately 20% of the dev set. for more details, please refer to appendix a.5). we refer to this set as the "history-shuffled" dev set, a perturbation introduced by shuffling dialog history to dilute patterns 1 & 3. note that shuffling the dialog history introduces examples which are unlikely to occur in real conversations (eg. asking a follow-up question based on a set of conjunctive rule-clauses even though a user has already responded with a "no"). for model details, please refer to section 6. using the evaluation metrics (micro and macro accuracy for turn level classification; and multi-bleu and multi-bleu-p for answer generation accuracy), we report the performance of the top two ranked models foot_0 (e3 (zhong and zettlemoyer, 2019) and bison ( lawrence et al., 2019) ), the bert based model (bert-qa @xcite ) and our heuristic based model by training them on the official sharc training set and evaluating them on the original and history-shuffled dev sets. we use the code released by respective authors for e3 and bison. we also use the same hyperparameters as mentioned in the respective papers. weakness in offical bleu scores: in heuristic model and effect of patterns: tables 1 and 2 show that the heuristic-based model not only has comparable performance across all other models but also across both dev sets (original and history-shuffled). if we look at the first two columns (micro and macro accuracy), all models tested on the history-shuffled dev set report a drop in performance. the average drops in micro & macro accuracies are 8.16% and 5.3% respectively. while changes in performance are can be attributed to change in train and test distributions, the goal of this experiment is to demonstrate that all models are relying on a spurious pattern induced by the sequence of follow-up answers in dialog history. thus, it is interesting to note that a dilution of just 20% of the patterns leads to a sizeable drop in performance. ## modified sharc dataset in an attempt to mitigate the effects of the patterns listed in section 2 and to reduce the sensitivity of neural models, we create a modified version of the sharc dataset. for each occurence of an instance conforming to any of the patterns, we automatically construct alternatives where we choose to either replace the current instance with an alternative instance which does not exhibit the pattern; or retain the original instance. the alternative instances are generated using pattern-specific modifications. for example, we shuffle dialog history to reduce the effect of patterns 1 & 3 (for more details, please see appendices a.4 and a.5). we individually modify both the official train and dev datasets and refer to them as sharc-mod. ## recommendations & conclusion in this paper we demonstrate how a popular neural conversational qa dataset inadvertently encodes patterns. we would like to emphasize that the patterns found, by their very nature, are likely to occur in real world tasks but the same patterns can also cause neural models to learn poorly. we release a modified version of this dataset and also improve evaluation criteria that better reflects model performance. we conclude the paper with a few recommendations for the community. for dataset creators: patterns may exist in a real-world task and artificially introducing perturbations may be an easy way to help reduce their effects. this may result in 'unnatural' instances in the dataset but could help train better models. for model creators: (1) model probing and experimenting with perturbed inputs can give deep insights about how a model is reasoning (2) experimenting with adversarial inputs early on in the design process can help build better models.