Improving Low-resource Question Answering by Augmenting Question Information (2023)

Abstract: In the era of large models, low-resource question-answering tasks lag, emphasizing the importance of data augmentation - a key research avenue in natural language processing. The main challenges include leveraging the large model’s internal knowledge for data augmentation, determining which QA data component - the question, passage, or answer - benefits most from augmentation, and retaining consistency in the augmented content without inducing excessive noise. To tackle these, we introduce PQQ, an innovative approach for question data augmentation consisting of Prompt Answer, Question Generation, and Question Filter. Our experiments reveal that ChatGPT underperforms on the experimental data, yet our PQQ method excels beyond existing augmentation strategies. Further, its universal applicability is validated through successful tests on high-resource QA tasks like SQUAD1.1 and TriviaQA.

## introduction low-resource issues pose a significant challenge within the field of question-answering (qa) tasks. these problems largely arise from data scarcity and lack of domain-specific training, which often results in underfitting. even though we are now in the era of large models, which have considerably advanced the field of natural language processing, these low-resource problems in qa tasks persist @xcite @xcite . consequently, addressing these lowresource issues has become an essential research focus, with data augmentation strategies emerging as a common and effective approach. however, current research in data augmentation presents several unanswered questions: how can the internal knowledge of large models be harnessed to enhance the quality of augmented data? which component of qa data -the question, passage, or answer -benefits most from augmentation? how can we strike a balance between maintaining the consistency of the augmented content and not introducing excessive noise? despite significant advancements in the field, many of these questions remain unanswered. for instance, the multi-modal qa benchmark, maqa @xcite , accentuates the need for data augmentation using negation examples in qa tasks but doesn't focus on leveraging the knowledge of large models. the fill-in-theblank qa framework, gotta @xcite , leverages generative, prompt-based augmentation methods to enhance learning. however, it does not identify the most beneficial segments of the qa data for improvement. lastly, the label-guided data augmentation framework promptda @xcite utilizes label semantics for data augmentation in few-shot learning, however, it overlooks the task of balancing content consistency with noise introduction. existing augmentation techniques, such as template-based question generation @xcite @xcite , and the broad application of seq2seq models in natural language processing @xcite , have made certain progress but also show similar limitations. to address these challenges, we propose a novel and straightforward question data augmentation method, termed pqq. firstly, we construct prompt templates to generate answers that are related to the original ones but expressed differently (prompt answer). then, a diverse set of questions is produced by a question generation model (question generation). lastly, using a question filter, we retain only those questions that are logically related to the original qa data as the augmented data (question filter). our experimental findings indicate a distinct underperformance of chatgpt on our experimental dataset, while our proposed methodology, pqq, exhibits a superior performance, outperforming contemporary mainstream data augmen- original passage question generation model original answer q 1 q k q 2 ,..., <passage, answer, @xmath0....... <passage, answer, new @xmath1....... discard step1: prompt answer step2: question generation step3: question filter augumented dataset [cls]question is : what areas did beyonce compete in she was growing up?[sep]passage is: ...beyoncé performed in various [mask]( singing and dancing) competitions as a child...[sep] talent vocal artistic prompted answers what is beyoncé's greatest asset? what is beyoncé's passion? what type of music does beyoncé perform? ## pqq apporach this section succinctly outlines the pqq data augmentation approach. this process, illustrated in figure 1 , consists of three main steps. ## dataset and experimental design in assessing our method's effectiveness on lowresource qa datasets, we chose policyqa and techqa, with 25,107 and 600 training instances respectively, focusing on privacy policies and technology (see appendix a.1). we conducted two comparative experiments, one applying traditional and sota data augmentation techniques in the qa domain, and another employing inference via a large-scale pre-training language model, chat-gpt (details in appendix a.2). to ascertain the impact of these different approaches, we adopted two evaluation metrics: exact match (em) and f1 scores (appendix a.3). we trained with bert-large and t5 as the prompt answer/question filter and question-generation models respectively. for the ultimate qa tasks, we opted for a finetuning approach using a bert-base model on qa data (training specifics in appendix a.4). we standardized the augmented data for both datasets to an additional 45,000 <passage, answer, @xmath2. appendix ?? provides more examples of the data after augmentation." ## experiment result in this section, we provide a comprehensive analysis of the results to thoroughly evaluate the performance of the proposed approach. ## conclusion our work presents a new data augmentation method for low-resource question-answering tasks. we first use prompt-based learning to gather various answers. then, a model generates diverse questions, and a filter retains only those relevant to the original qa data as augmented data. our approach excels in low-resource contexts and shows broad applicability in the general qa field. future work will employ large pre-trained models to enhance data quality.