A Mutual Information Maximization Approach for the Spurious Solution Problem in Weakly Supervised Question Answering (2021)

Abstract: Weakly supervised question answering usually has only the final answers as supervision signals while the correct solutions to derive the answers are not provided. This setting gives rise to the spurious solution problem: there may exist many spurious solutions that coincidentally derive the correct answer, but training on such solutions can hurt model performance (e.g., producing wrong solutions or answers). For example, for discrete reasoning tasks as on DROP, there may exist many equations to derive a numeric answer, and typically only one of them is correct. Previous learning methods mostly filter out spurious solutions with heuristics or using model confidence, but do not explicitly exploit the semantic correlations between a question and its solution. In this paper, to alleviate the spurious solution problem, we propose to explicitly exploit such semantic correlations by maximizing the mutual information between question-answer pairs and predicted solutions. Extensive experiments on four question answering datasets show that our method significantly outperforms previous learning methods in terms of task performance and is more effective in training models to produce correct solutions.

## introduction weakly supervised question answering is a common setting of question answering (qa) where only final answers are provided as supervision signals while the correct solutions to derive them are not. this setting simplifies data collection, but exposes model learning to the spurious solution problem: there may exist many spurious ways to derive the correct answer, and training a model with spurious solutions can hurt model performance (e.g., misleading the model to produce unreasonable solutions or wrong answers). as shown in fig 1 , figure 1 : examples from three weakly supervised qa tasks, i.e., multi-mention reading comprehension, discrete reasoning, and semantic parsing. spans in dark gray and green denote semantic correlations between a question and its solution, while spans in orange are spurious information and should not be used in a solution. for multi-mention reading comprehension, many mentions of an answer in the document(s) are irrelevant to the question; for discrete reasoning tasks or text2sql tasks, an answer can be produced by the equations or sql queries that do not correctly match the question in logic. some previous works heuristically selected one possible solution per question for training, e.g., the first answer span in the document @xcite @xcite ; some treated all possible solutions equally and maximized the sum of their likelihood (maximum marginal likelihood, or mml) @xcite @xcite ; many others selected solutions according to model confidence @xcite , i.e., the likelihood of the solutions being derived by the model. a drawback of these methods is that they do not explicitly consider the mutual semantic correlations between a question and its solution when selecting solutions for training. intuitively speaking, a question often contains vital clues about how to derive the answer, and a wrong solution together with its context often fails to align well with the question. take the discrete reasoning case in fig 1 as an example. to answer the question, we need to know the start year of the battle of powder river, which is answered by the first 1876; the second 1876 is irrelevant as it is the year of an event that happened during the battle. to exploit the semantic correlations between a question and its solution, we propose to maximize the mutual information between question-answer pairs and model-predicted solutions. as demonstrated by @xcite , for many qa tasks, it is feasible to precompute a modestly-sized, taskspecific set of possible solutions containing the correct one. therefore, we focus on handling the spurious solution problem under this circumstance. specifically, we pair a task-specific model with a question reconstructor and repeat the following training cycle (fig 2 ): (1) sample a solution from the solution set according to model confidence, train the question reconstructor to reconstruct the question from that solution, and then (2) train the task-specific model on the most likely solution according to the question reconstructor. during training, the question reconstructor guides the taskspecific model to predict those solutions consistent with the questions. for the question reconstructor, we devise an effective and unified way to encode solutions in different tasks, so that solutions with subtle differences (e.g., different spans with the same surface form) can be easily discriminated. our contributions are as follows: ## related work question answering has raised prevalent attention and has achieved great progress these years. a lot of challenging datasets have been constructed to advance models' reasoning abilities, such as (1) reading comprehension datasets with extractive answer spans @xcite , with free-form answers @xcite , for multi-hop reasoning @xcite , or for discrete reasoning over paragraphs @xcite , and (2) datasets for semantic parsing @xcite @xcite . under the weakly supervised setting, the specific solutions to derive the final answers (e.g., the correct location of an answer text, or the correct logic executing an answer) are not provided. this setting is worth exploration as it simplifies annotation and makes it easier to collect large-scale corpora. however, this setting introduces the spurious solution problem, and thus complicates model learning. most existing approaches for this learning challenge include heuristically selecting one possible solution per question for training @xcite @xcite , training on all possible solutions with mml @xcite @xcite , reinforcement learning @xcite , and hard em @xcite . all these approaches either use heuristics to select possibly reasonable solutions, rely on model architectures to bias towards correct solutions, or use model confidence to filter out spurious solutions in a soft or hard way. they do not explicitly exploit the semantic correlations between a question and its solution. most relevantly, @xcite focused on text2sql tasks; they modeled sql queries as the latent variables for question generation, and maximized the evidence lower bound of log likelihood of questions. a few works treated solution prediction and question generation as dual tasks and introduced dual learning losses to regularize learning under the fully-supervised or the semi-supervised setting @xcite @xcite . in dual learning, a model generates intermediate outputs (e.g., the task-specific model predicts solutions from a question) while the dual model gives feedback signals (e.g., the question reconstructor computes the likelihood of the question conditioned on predicted solutions). this method is featured in three aspects. first, both models need training on fully-annotated data so that they can produce reasonable intermediate outputs. second, the intermediate outputs can introduce noise during learning as they are sampled from models but not restricted to solutions with correct answer or valid questions. third, this method typically updates both models with reinforcement learning while the rewards provided by a dual model can be unstable or of high variance. by contrast, we focus on the spurious solution problem under the weakly supervised setting and propose a mutual information maximization approach. solutions used for training are restricted to those with correct answer. what's more, though a taskspecific model and a question reconstructor interact with each other, they do not use the likelihood from each other as rewards, which can stabilize learning. ## evaluation of solution prediction as solutions with correct answer can be spurious, we further analyzed the quality of predicted solutions. we randomly sampled 50 test examples from drop for which our method produced the correct answer, and found that our method also produced the correct solution for 92% of them. to investigate the effect of different learning methods on models' ability to produce correct solutions, we manually analyzed another 50 test samples for which hardem, hardem-thres, and our method produced the correct answer with different solutions. the percentage of samples for which our method produced the correct solution is 58%, much higher than that of hardem (10%) and hardemthres (30%). for experimental details, please refer to the appendix. ## case study fig 6 compares nerd predictions on four types of questions from drop when using different learning methods. an observation is that nerd using our method shows more comprehensive understanding of questions, e.g., in the arithmetic case, nerd using our method is aware of the two key elements in the question including the year when missionaries arrived in ayutthaya and the year when the seminary of saint joseph was built, while nerd using hardem-thres misses the first element. what's more, nerd using our method is more precise in locating relevant information, e.g., in the first sorting case, nerd with our method locates the second appearance of 2 whose contextual semantics matches the question, while nerd using hardem-thres locates the first appearance of 2 which is irrelevant. these two observations can be attributed to our mutual information maximization objective which biases a task-specific model towards those solutions that align well with the questions. however, we also observed that when there are multiple mentions of relevant information of the same type, nerd trained with hardem-thres or our method has difficulty in recalling them all, e.g., in the second sorting case, the correct solution should locate all four mentions of sebastian janikowski's field goals while nerd using either method locates only two of them. we conjecture that this is because the solution sets provided by chen et al. (2020) are noisy. for example, all precomputed solutions of sorting type for numeric answers involve up to two numbers from reference information, which makes it hard for a model to learn to sort more than two numbers. ## conclusion to alleviate the spurious solution problem in weakly supervised qa, we propose to explicitly exploit the semantic correlations between a question and its solution via mutual information maximization. during training, we pair a task-specific model with a question reconstructor which guides the task-specific model to predict solutions that are consistent with the questions. experiments on four qa datasets demonstrate the effectiveness of our learning method. as shown by automatic and manual analyses, models trained with our method are more resistant to spurious solutions during training, and are more precise in locating information that is relevant to the questions during inference, leading to higher accuracy of both answers and solutions.