Contrastive Multi-document Question Generation (2021)

Abstract: Multi-document question generation focuses on generating a question that covers the common aspect of multiple documents. Such a model is useful in generating clarifying options. However, a naive model trained only using the targeted (‘positive’) document set may generate too generic questions that cover a larger scope than delineated by the document set. To address this challenge, we introduce the contrastive learning strategy where given ‘positive’ and ‘negative’ sets of documents, we generate a question that is closely related to the ‘positive’ set but is far away from the ‘negative’ set. This setting allows generated questions to be more specific and related to the target document set. To generate such specific questions, we propose Multi-Source Coordinated Question Generator (MSCQG), a novel framework that includes a supervised learning (SL) stage and a reinforcement learning (RL) stage. In the SL stage, a single-document question generator is trained. In the RL stage, a coordinator model is trained to find optimal attention weights to align multiple single-document generators, by optimizing a reward designed to promote specificity of generated questions. We also develop an effective auxiliary objective, named Set-induced Contrastive Regularization (SCR) that improves the coordinator’s contrastive learning during the RL stage. We show that our model significantly outperforms several strong baselines, as measured by automatic metrics and human evaluation. The source repository is publicly available at ‘www.github.com/woonsangcho/contrast_qgen’.

## introduction user queries on web search engines can sometimes be vague. search engines may resolve this ambi- † work done when the author was an intern at microsoft research. positive set about: figure 1 : non-contrastive and contrastive method for multidocument question generation. left: noncontrastive modeling that takes input as a set of positive documents. however, model-generated questions from this method are rather generic and not specific to the input documents. right: contrastive modeling, which considers both positive and negative document sets, and learns to generate questions that are more grounded on the positive document set. this particular approach may involve three nontrivial phases: i) retrieval: gather the initial return documents by the search engine; ii) partition: partition the documents into semantically similar clusters in an unsupervised manner; iii) multidocument question generation: generate a clarification question by finding an "overlap" among documents in each cluster. in principle, the clarification questions should be specific to each cluster rather than generic and bland, otherwise it is counter to the objective of clarification @xcite . in this work, we focus on developing a multi-document question generator to generate cluster-specific questions in the iii) step. nevertheless, we believe our approach can be readily applied to multi-document text generation such as summarization @xcite and response generation @xcite . we address this challenge by leveraging contrastive learning. given a set of positive documents d + and a set of negative documents d -(where d -is yet semantically close to d + ), we propose a new strategy to generate a question that is semantically relevant to d + and far away from d -. ideally, the model would use both d + and d -to identify distinguishing features between the two sets and constrain the generation to be specific to d + . the similarity between the d + and d - makes the generation more challenging and forces the model to be as specific as possible in order to distinguish between the two sets. the comparison between the contrastive and non-contrastive multi-document question generation is illustrated in figure 1 . this task is particularly challenging because i) there does not exist direct supervised ground-truth multi-document question given positive and negative sets of documents. ii) the whole procedure involves multiple aspects including language understanding, inter-document information aggregation, coordinative planning and language generation. in theory, the generator can be trained to maximize the chance that the generated question specifically retrieves the given document cluster, using rl. however, the space of possible sequence is prohibitively large which results in large variance in rl @xcite . to effectively reduce the search space of rl, we employ a hybrid supervised learning (sl) and rl strategy. we also propose a novel reward-shaping auxiliary objec-tive, set-induced contrastive regularization (scr) (section 2), which heuristically drives the generation closer towards d + , by minimizing/maximizing the kl divergence between the hypothesis distribution and distributions induced by d + / d -. our contributions are summarized below: i) we develop a novel multi-source coordinated question generator (mscqg) model that is trained using a hybrid hierarchical generation scheme. the document-specific generator is fine-tuned from gpt-2 and the inter-document coordinator is trained using reinforcement learning. ii) we introduce set-induced contrastive regularization (scr), an auxiliary regularizer that pushes mscqg toward d + relative to d -while limiting the effect of d -in a principled manner. iii) empirical results show that our model is able to generate more grounded and specific questions, significantly outperforming existing baseline models in automatic measures and human evaluation. ## method overview: the overview of our model is illustrated in figure 2 . the model consists of two major components: i) the document-specific generator generates a question from a single document, and is fine-tuned from openai gpt-2. ii) the inter-document coordinator integrates multiple-document information from the documentspecific generator instances. the coordinator is trained using reinforcement learning after fixing the document-specific generator. during the rl training, at each generation time step, each (positive and negative) document will independently use the same generator trained from i) to predict the next token. the coordinator will learn to aggregate the probabilities to a consensus probability by maximizing a reward function. the reward function is designed to encourage the generated question to tie to the positive set and to be away from the negative set. the word newly generated from the consensus probability are concatenated to all documents as inputs for next time step. ## related work multi-source encoder-decoder: ensemble set induction mechanism @xcite has been widely applied to neural machine translation (nmt) tasks @xcite . @xcite introduced a new type of ensemble of nmt systems which take inputs as multiple sentences in different languages and output a translation into a single language. each nmt system is trained on a mono-lingual source to target language translation dataset. @xcite further developed the multi-source encoder-decoder framework for multi-lingual nmt systems, by learning to assign uneven attention weights, called expert combination weights. to handle multi-source input, we take a similar multi-source encoder-decoder approach figure 5 shows that the model learns to push the sequential generation semantics more toward d + by gradually penalizing d -. figure 6 shows that frequent and semantically less distinguishing words such as 'of' are encouraged even by d -, which empirically aligns with our intuition for tf-idf. for our coordinator model. for such multi-lingual translation tasks, the target translation is available. however, in our task of generating multi-document questions, the target does not exist which makes it more challenging, thus we train via rl, rather than supervised learning. ## conclusion we proposed a novel coordinator model that can generate questions that are more grounded on documents of interest. this coordinator model consists of transformer blocks, and is trained through reinforcement learning and an effective auxiliary: set-induced contrastive regularization. the rewards are derived from a publicly available state-of-the-art pre-trained ranker (section 2) to compute retrieval statistics among d + and d -. our novel contrastive regularization induces generations to be more specific to d + than to d -while limiting the effect of d -in a principled manner by accounting for their semantic similarity. we evaluate a generated question from each model by assessing how many of the input d + documents among a pool of relevant documents it can retrieve, based on the (document, question) ranker that is trained on the same wide-ranging do-main. for a comprehensive automatic evaluation of the models, retrieval statistics are computed from a larger pool of relevant documents gathered via bm25. experiment results show that our model significantly outperforms previous neural generation as well as strong retrieval baselines in both automatic and human metrics. given the promising comprehensive results of the proposed models and training approach, we can extend the framework with appropriate modifications and train via imitation learning algorithms, and this is left for future work. to find two 10-document sets d + and d -that are similar, we find a pair of questions that are semantically similar. however, computing pair-wise similarities among roughly 1 million questions is computationally intractable. therefore, we leverage another dataset: ms-marco-conversational search ¶ : an artificially constructed public dataset that simulate user search sequences. each data point or session is an artificial sequence of similar questions grounded on true user behavior. since many similar questions are grouped together, we can reduce the search space for finding pairs of similar questions. then we take pairs of high semantic similarity (≥ 0.7) yet not a paraphrase (≤ 0.85 following their classification criteria) using gen-encoder @xcite which two associated 10-document sets do not have overlaps, primarily for prototype evaluation convenience. for deployment models, one may choose to allow overlaps between two sets for more challenging learning. from the two similar 10-document sets, either one is set to positive d + or negative d -, yielding two data points for the our derived dataset. these pre-processing steps yield 346,215 data points, each of which contains a pair of positive and negative questions, and positive and negative 10-document sets. training mscqg on the entire dataset requires processing about 7 million marco documents. this is computationally intensive and takes about two days on 8 nvidia tesla v100 gpu cards for a single epoch. therefore, for building small research prototypes and ¶ https://github.com/microsoft/msmarcoconversational-search benchmarks, we will also release a subset of the data, that consists of 100k/10k/10k training, development, and evaluation data points.