Knowledge Transfer from Answer Ranking to Answer Generation (2022)

Abstract: Recent studies show that Question Answering (QA) based on Answer Sentence Selection (AS2) can be improved by generating an improved answer from the top-k ranked answer sentences (termed GenQA). This allows for synthesizing the information from multiple candidates into a concise, natural-sounding answer. However, creating large-scale supervised training data for GenQA models is very challenging. In this paper, we propose to train a GenQA model by transferring knowledge from a trained AS2 model, to overcome the aforementioned issue. First, we use an AS2 model to produce a ranking over answer candidates for a set of questions. Then, we use the top ranked candidate as the generation target, and the next k top ranked candidates as context for training a GenQA model. We also propose to use the AS2 model prediction scores for loss weighting and score-conditioned input/output shaping, to aid the knowledge transfer. Our evaluation on three public and one large industrial datasets demonstrates the superiority of our approach over the AS2 baseline, and GenQA trained using supervised data.

## introduction in recent times, extractive qa research can be categorized into two broad directions for the task of producing the final answer for a question: (i) answer sentence selection (as2), which, given a question and a set of answer-sentence candidates, selects sentences (e.g., retrieved by a search engine) that correctly answer the question; and (ii) machine reading (mr), e.g., @xcite , which, given a question and a reference text, involves finding an exact text span that answers the question. as2 models can perform more efficiently with large text databases (as they originated from the trec-qa track @xcite ), and there seems a renewed research interest in these models for applications to personal assistants, e.g., alexa @xcite @xcite . both approaches (as2 and mr) when applied for qa over unstructured web text, while effective, may have certain drawbacks. arbitrary web sentences may not contain all the information needed to answer a question, or may contain distracting extraneous information. moreover, they may have a particular sentiment or style that is not suited to qa, or be too structurally reliant on longer discourse context to serve as a standalone answer. in light of this, researchers have been exploring text generation systems for writing 'better' answers. for example, in mr, rag @xcite generates an answer from a set of documents selected by dense passage retrieval models. for as2 systems, research has focused on learning to summarize answers from relevant paragraphs @xcite , or to synthesize information from the top ranked candidates of an as2 system @xcite . the latter approach, termed as genqa, has shown improvements in terms of both answer accuracy and style suitability. a distinctive characteristic of genqa over a generation-based approach for mr is the length of the answer: the former uses an entire sentence as the target, while the latter in practice uses a short text (primarily targeting entity names). in this work, we focus on genqa as we are interested to generate complete answer sentences from precise information selected by as2 models. a challenge for training effective genqa models is the difficulty of obtaining large-scale, highquality training data. producing such data for genqa typically requires human annotators to read questions and paragraphs of relevant background information, and then author a self-contained, natural answer (typically a sentence). this fairly involved procedure highly diminishes the veloc-ity of annotation. existing datasets in research works either offer limited coverage of all domains, where genqa can be applied @xcite , or are too small to be used as supervised training data @xcite . generally, collecting a human-authored answer to a question when given a context is significantly more expensive compared to annotating the correctness of an extracted web sentence as an answer for the same question. consequently, there are a large number of annotated datasets @xcite @xcite available for the latter type, aimed at training answer sentence selection (as2) systems. in this work, we propose a training paradigm for transferring the knowledge learned by a discriminative as2 ranking model to train an answer generation qa system. towards this, we learn a genqa model using weak supervision provided by a trained as2 model on a unlabeled data set comprising of questions and answer candidates. specifically, for each question, the as2 model is used to rank a set of answer candidates without having any label of correctness/incorrectness for answering the question. the top ranked answer is used as the generation target for the genqa model, while the question along with the next k top-ranked answers are used as the input for the genqa model. we supplement the ranking order of answer candidates with the prediction confidence scores provided by the as2 model for each answer candidate. this is done by modifying our knowledge transfer strategy in two ways. first, we weight the loss of each training instance (question + context, comprised of k answer candidates) using the as2 model score of the top ranked answer, which is to be used as the genqa target. this allows the genqa model to selectively learn more from 'good' quality target answers in the weakly supervised training data (as2 models are calibrated to produce higher confidence scores for correct answers). however, this loss weighting only considers the score of the output target, and does not exploit the scores of the input candidates. to overcome this limitation, we discretize and label the as2 scores into l confidence buckets, add these bucket labels to the genqa vocabulary, and finally prepend the corresponding label to each answer candidate in the input and/or the output. this confidence bucket label provides the genqa model with an additional signal about the answer quality of each candidate as assigned by the as2 model. we show that both these techniques improve the qa accuracy, and can be combined to provide additional improvements. we empirically evaluate foot_0 our proposed knowledge transferring technique from as2 to genqa on three popular public datasets: ms-marco nlg @xcite , wikiqa @xcite , trec-qa @xcite ; and one large scale industrial qa dataset. our results show that the genqa model trained using our paradigm of weak supervision from an as2 model can surprisingly outperform both the as2 model that was used for knowledge transfer (teacher), as well as a genqa model trained on fully supervised data. on small datasets such as wikiqa and trec-qa, we show that as2 models trained even on small amounts of labeled data can be effectively used to weakly supervise a genqa model, which then can outperform its teacher in qa accuracy. additionally, on ms-marco nlg, where fully supervised genqa training data is available, we show that an initial round of training with our weakly supervised methods yields additional performance improvements compared to the standard supervised training of genqa. qualitatively, the answers generated by our model are often more directly related to the question being asked, and stylistically more natural-sounding and suitable as responses than answers from as2 models, despite being trained only on extracted sentences from the web. ## related work our work builds upon recent research in as2, answer generation for qa, and transfer learning. ## knowledge transfer: @xmath0 previous works on genqa require the use of labeled data for effectively training the genqa model. to reduce the need of expensive largescale training data for genqa, we propose a training paradigm that uses unlabeled data while being weakly-supervised by a discriminative as2 model (as shown in fig. 2 ). ## datasets and models for training and evaluating our knowledge transfer techniques (ws, lw, sc) described above, we categorize the data that we use for each experiment into the following four sources/types: • as2: labeled (q, a) pairs with correctness and incorrectness annotation for training m • transfer: unlabeled (q, a) pairs that are ranked by m, and used for knowledge transfer to m g • fine-tuning: labeled data (human written answers / answers with correctness labels) for finetuning m g , whenever available • evaluation: evaluation data for m and m g in section 5, we vary the sources of different types of the data described above, to demonstrate the robustness and generality of our knowledge transfer method. below, we provide details about the data sources we use, along with a summary of the underlying models. ## experiments and results we perform experiments in three data settings to evaluate different features of our method. on the ms-marco datasets, we show that weak supervision can augment strong models trained on indomain data. on wikiqa and trec, we show that weak supervision on large data improves over direct supervision on small data for this qa task. we also present an experiment on a very large industrial dataset to measure the contribution of each of our proposed techniques for using unlabeled training data at scale. ## conclusion in this paper, we have presented a novel approach for transferring knowledge from a discriminative as2 model to an answer generation model, only using unlabeled data. we use the ranking produced by the as2 model for training a genqa model using the top answer as the target output, and the next k top ranked answers along with the question as the input. we also propose input/output shaping and loss weighting techniques during knowledge transfer to improve the performance of genqa. our experimental results on three public and one large industrial datasets show that genqa models trained with knowledge transfer from as2 models achieve higher answering accuracy than both the as2 teacher and supervised genqa trained with in-domain data. we are releasing our code and trained models to support future research.