Unsupervised Domain Adaptation for Question Generation with D omain D ata Selection and Self-training (2022)

Abstract: Question generation (QG) approaches based on large neural models require (i) large-scale and (ii) high-quality training data. These two requirements pose difficulties for specific application domains where training data is expensive and difficult to obtain. The trained QG models’ effectiveness can degrade significantly when they are applied on a different domain due to domain shift. In this paper, we explore an unsupervised domain adaptation approach to combat the lack of training data and domain shift issue with domain data selection and self-training. We first present a novel answer-aware strategy for domain data selection to select data with the most similarity to a new domain. The selected data are then used as pseudo-in-domain data to retrain the QG model. We then present generation confidence guided self-training with two generation confidence modeling methods (i) generated questions’ perplexity and (ii) the fluency score. We test our approaches on three large public datasets with different domain similarities, using a transformer-based pre-trained QG model. The results show that our proposed approaches outperform the baselines, and show the viability of unsupervised domain adaptation with answer-aware data selection and self-training on the QG task.

## introduction natural language question generation (qg) aims to generate questions from given passages of text. it has been applied to a wide range of applications, such as question answering @xcite , conversational systems @xcite , and education @xcite . recently, pre-trained language models (lm) have advanced the state-of-the-art across a variety of natural language processing tasks @xcite . consequently, by modeling qg as a sequence-to-sequence task and finetuning on task-specific data, pre-trained lms have substantially advanced the state-of-art performance on qg @xcite . however, with billions of parameters, the performance of these deep neural models heavily relies on the quantity and quality of available training data. as the manual process of creating high-quality questions is expensive in terms of time and money, compared with abundant unlabeled data, the available data sources containing well-formed questions are insufficient, especially in the educational domain, where a lot of expertise is required to create questions geared towards human learning. to mitigate the lack of labeled training data, one solution is to pre-train models for qg on a data-abundant labeled domain (source domain) and transfer the learned knowledge to the unlabeled target domain, which is known as unsupervised domain adaptation (uda) @xcite . it is a common challenge in machine learning research to learn knowledge in one domain and apply it in other domains with good generalization performance. one obstacle is the domain shift @xcite between the source domain and the target domain, as illustrated in figure 1 , which violates the assumption that the training set and the test set are independent and identically distributed @xcite . this in turn limits the model's generalization and portability. to understand the effect of differences among domains on the performance of downstream qg tasks, fol-lowing previous research @xcite , we perform a preliminary cross-domain study. we first train the qg model on all domains separately and evaluate them across different domain test sets. as shown in in the second stage, we focus on self-training on the unlabeled target domain with the qg model trained in the first stage. the self-training approach is substantially hindered by noisy and low-quality generated pseudo labels. we first propose a normalization method to avoid re-enforcing poorly generated questions. we also explore using sentence perplexity and fluency scores to model the confidence of sequence generation. we filter pseudo labels with low sequence confidence during selftraining to prevent the model from being degraded by wrong or low-quality predictions. we conduct experiments across three domains, including the natural question dataset as the source domain, race as one target domain of education, and sciq as the target domain of science. our results show our proposed approach is effective even when the target domain is substantially different from the source domain and outperforms several baselines including latent dirichlet allocation (lda) @xcite , bert discriminator based data selection @xcite , and unsupervised gaussian mixture model(gmm) clustering on pre-trained language model features @xcite . ## background in this section, we first present a short review for uda and question generation, then we briefly discuss how our work is different from recent related research. ## formalization we now formulate the problem and present our notation. the data in the source domain with groundtruth questions are denoted as d @xmath0, q s )}, while unlabeled data in the target domain is d @xmath1; here, c is denoting the context (the passages, and answer spans used for generating questions). the question generation task is then to generate a sequence q that maximizes the conditional probability of the prediction p(q|c, θ): ## domain data selection not all data are required or even useful for domain adaptation. irrelevant data samples can add noise, and affect the learned model's performance and robustness towards cross-domain application considerably @xcite . a solution to reduce the impact of irrelevant data is domain data selection, i.e. to retrieve the most appropriate data from the source domain data given the target domain data. most proposed domain data selection approaches consider ranking training examples from d s according to a domain similarity measure and select the top-n examples that are closest to d t . we encode the context passage at the paragraph level with bert, and perform average pooling of the last layer hidden state of each token to create its vector representation. to show that this is a robust representation for mapping sentences to domains in an unsupervised, data-driven approach, we first visualize them with t-sne, as shown in figure 1 . we can observe the encoding vector representation with bert indeed can cluster data examples to domains. following the practice of aharoni and goldberg (2020), we then perform unsupervised clustering by fitting gaussian mixture models (gmms) to the vector context representations with k predefined clusters. we assign each cluster the domain class by measuring its purity (proportion of examples belonging to each domain). we use the euclidean distance @xcite of each example to cluster center as the measure of domain distance. figure 2 shows the distribution of nq dataset examples' distance to nq's, race's and sciq's domain center respectively. we sort source data examples based on their distance to the target domain center and select data examples with most domain similarity as the pseudo-in-domain data. ## self-training when training the qg model with pseudo-labels, it is natural to put more emphasis on the labels that the model is more confident about. an intuitive solution is to weigh each pseudo-token according to its estimated probability in order to avoid reenforcing poor predictions. thus, we propose the following normalized training objective for selftraining: we apply the qg model to generate questions on unlabeled target-domain data, which are then used as 'pseudo' gold labels for further training. the self-training approach is substantially hindered by noisy, low-quality labels. how to deal with noisy pseudo labels is crucial to the final uda effectiveness. classical pseudo label generation methods @xcite @xcite filter generated labels by their 'confidence' which is the predicted probability of the label in those classification tasks. how to represent confidence of sequence generation in pseudo-labeling is insufficiently explored. traditionally, confidence estimation has been defined as a task of assessing the quality of the whole sequence of words in the target sentence. therefore, we propose a question quality guided pseudo labeling method to address this problem, with two confidence metrics: (i) the sentence perplexity, and (ii) the bert-based fluency score. sentence perplexity the first metric is the perplexity of the generated questions. the generation with higher confidence should have lower perplexity. here, perplexity (ppl) is defined as follows: bert-based fluency score for our second metric, we use fluency as the question quality metric, which indicates whether the generation follows grammar rules and correct logic. the perplexity of a sentence under a well-trained language model usually serves as a good indicator of its fluency @xcite . we use a fine-tuned bert language model as evaluator. the fluency metric r f luency for question q is calculated as follows: r fluency (q) = exp (-1 t t @xmath2. ## experiments in this section, we describe the model and the training regime in more detail. ## conclusion we proposed an unsupervised domain adaptation approach for question generation. our approach includes an answer-type aware unsupervised domain data selection method and a sequence generation confidence guided self-training algorithm. we conduct experiments on three domains. we use the natural questions dataset as labeled source domain, race as target education domain and sciq as target science domain. our results suggest our approach is effective for this application settings. we find that it significantly improves domain adaptation performance of our qg model. in future work, we plan to expand our work to more domains and additional qg model types. zichao yang, zhiting hu, chris dyer, eric p xing, and taylor berg-kirkpatrick. 2018. unsupervised text style transfer using language models as discriminators. arxiv preprint arxiv:1805.11749.