Unsupervised Question Answering via Answer Diversifying (2022)

Abstract: Unsupervised question answering is an attractive task due to its independence on labeled data. Previous works usually make use of heuristic rules as well as pre-trained models to construct data and train QA models. However, most of these works regard named entity (NE) as the only answer type, which ignores the high diversity of answers in the real world. To tackle this problem, we propose a novel unsupervised method by diversifying answers, named DiverseQA. Specifically, the proposed method is composed of three modules: data construction, data augmentation and denoising filter. Firstly, the data construction module extends the extracted named entity into a longer sentence constituent as the new answer span to construct a QA dataset with diverse answers. Secondly, the data augmentation module adopts an answer-type dependent data augmentation process via adversarial training in the embedding level. Thirdly, the denoising filter module is designed to alleviate the noise in the constructed data. Extensive experiments show that the proposed method outperforms previous unsupervised models on five benchmark datasets, including SQuADv1.1, NewsQA, TriviaQA, BioASQ, and DuoRC. Besides, the proposed method shows strong performance in the few-shot learning setting.

## introduction extractive question answering (extractive qa) aims to provide answer spans extracted from the context to answer questions. it can improve the interaction between humans and machines in applications such as search engines and dialog systems. existing extractive qa methods can be divided into two categories: supervised qa and unsupervised qa. traditionally, for supervised qa @xcite , human-labeled context, questions and answers are given to train a qa model. since the construction cost of labeled data is too high for supervised qa, recently, researchers pay more attention to the unsupervised setting, where qa pairs are unavailable. for example, @xcite propose an unsupervised machine translation model to generate qa pairs. @xcite , @xcite and @xcite try to alleviate the overlap between the context and the generated question. however, most existing works regard named entities as the only answer type, which ignores the high diversity of answers in the real world. for instance, as shown in figure 1 , in squadv1.1 @xcite and newsqa @xcite ), the answer type of named entities only account for 52.4% and 41.1% respectively. to solve the problem, an intuitive way is to extract each type of answer spans independently in the raw text. yet, it leads to two critical problems. firstly, it's hard to determine the proportion of each answer type in the synthetic dataset. secondly, extracting answers without any guidance could probably generate trivial and noisy questionanswer pairs. to tackle the problem above, we propose diverseqa, a novel unsupervised qa method. specifically, the proposed method consists of three modules: data construction, data augmentation and denoising filter. firstly, the data construction mod-ule employs a simple answer-extending rule to construct a dataset with diverse answers. as shown in figure 2 , a named entity (ne) is extracted and extended into a longer answer span, which should be a constituent of the sentence (e.g. noun phrase, verb phrase). in this way, the proportion of each type of answers can be obtained entirely from the original text. besides, ne-based extension largely guarantees that extracted answers are meaningful. secondly, an answer-type dependent data augmentation module is proposed. concretely, an adjusting vector generator is designed to produce answertype enhanced qa pairs in the embedding level. then, a discriminator classifies the embeddings into the corresponding answer type while minimizing the kl divergence between the distribution and its prior to fool the discriminator in an adversarial way. thirdly, the denoising filter is applied to alleviate the noise in the synthetic qa pairs. extensive experiments on six benchmarks, including squadv1.1 @xcite , triviaqa @xcite , naturalquestions @xcite , newsqa @xcite , bioasq @xcite and duorc @xcite show that diverseqa outperforms previous unsupervised methods on five datasets and obtains comparable results on one dataset among unsupervised qa models. further analysis shows that diverseqa can largely improve the question-answering ability of the model on diverse answer types. in addition, our method shows strong performance in the few-shot learning setting. the contributions of our method are as follows: ## method to explore answer types beyond named entities, we propose an unsupervised method diverseqa, which can be divided into three modules, data augmentation, data construction and denoising filter. firstly, a simple but effective span extending rule is applied in the data construction module to do answer extraction and natural question generation to construct a synthetic qa dataset with diverse answers. secondly, an answer-type dependent data augmentation module via adversarial training is designed to produce high-quality augmented qa pairs. thirdly, a denoising filter is proposed to alleviate high noise in the dataset. in the next few subsections, we will introduce the three modules in detail. ## conclusion we propose diverseqa, an unsupervised qa method comprising a synthetic qa dataset with diverse answers, an answer-type-dependent data augmentation process via adversarial training, and a denoising filter to improve the performance of a qa model. our method reaches the state-of-the-art on five benchmarks and shows strong performance in the few-shot learning setting.