Towards Diverse and Effective Question-Answer Pair Generation from Children Storybooks (2023)

Abstract: Recent advances in QA pair generation (QAG) have raised interest in applying this technique to the educational field. However, the diversity of QA types remains a challenge despite its contributions to comprehensive learning and assessment of children. In this paper, we propose a QAG framework that enhances QA type diversity by producing different interrogative sentences and implicit/explicit answers. Our framework comprises a QFS-based answer generator, an iterative QA generator, and a relevancy-aware ranker. The two generators aim to expand the number of candidates while covering various types. The ranker trained on the in-context negative samples clarifies the top-N outputs based on the ranking score. Extensive evaluations and detailed analyses demonstrate that our approach outperforms previous state-of-the-art results by significant margins, achieving improved diversity and quality. Our task-oriented processes are consistent with real-world demand, which highlights our system’s high applicability.

## introduction pedagogical studies over the years have demonstrated that asking questions about a given storybook nurtures insight and expands knowledge @xcite @xcite . hence, posing questions becomes a fundamental part of education to engage children and promote literacy @xcite @xcite . along with the remarkable strides in natural language processing, recent studies have actively explored question-answer pair generation (qag) systems that target education @xcite @xcite . as qag is a labor-intensive manual process, it benefits from automated production methods. furthermore, sustainable system update and utilization emphasize their high applicability @xcite . a challenge in educational qag is the diversity of generated qa pairs as well as their quality @xcite . exploiting various qa types facilitates comprehensive learning, as each question inquires information specific to its type and stimulates different brain activities in the answering process @xcite . controlling difficulty by adopting different types of questions or answers also enables a balanced assessment of the reading comprehension skills of children @xcite . consequently, actively using questions with various interrogative words and answers reflecting both implicitness and explicitness is important. yet, existing educational qag studies have rarely considered diversity. generated questions of existing models are extremely biased to the 'what' and 'who' type questions. answer extraction focuses on detecting spans within passages, resulting in an inability to create implicit answers that do not directly appear in the passage. to address the limitation, we propose an effective qag framework that enhances diversity and quality. our framework consists of a qfs-based answer generator, an iterative qa generator, and a relevancy-aware ranker. specifically, qfs-based answer generator adopts query-focused summarization (qfs)-based @xcite answer generation model (agm), with the aim of obtaining diverse and proper answer candidates. iterative qa generator is designed to increase question type variety by exploiting the interrogative word-indicated question generation model (qgm). we jointly execute this qgm with the question-answering model (qam) to adjust the final answers. relevancyaware ranker inspects quality to determine the final top-n outputs among the generated candidates. to grasp better pairs with high relevancy, the ranker is trained using in-context negative samples. the experimental results indicate that our framework outperforms the existing state-of-theart method by a large margin, with a gain of up to 0.435→0.503 on map@n with rouge-l f1 and 0.9077→0.9178 on map@n with bertscore @xcite . additional statistical and human evaluations with detailed analyses consistently show higher qa type diversity and quality compared to previous studies, which demonstrates the superiority of the proposed approach. the three modules of our framework are process-oriented, providing outputs from each, which is in line with the real-world demand investigated by @xcite . this highlights the high applicability to the education field in terms of human-ai collaboration. we summarize our contributions as follows: ## related works the question-answer pair generation (qag) task aims to automatically generate qa pairs based on the input text. in the early days, rule-based qag systems are dominant @xcite . with the advent of a deep learningbased paradigm, it was demonstrated for the first time by @xcite that a fully end-to-end qag system generates exceptionally good questions. accordingly, diverse studies have been conducted and developed @xcite @xcite . @xcite adopt an interrogative words-based approach to clarify the semantics of words from a passage, resulting in the generation of questions containing key information of the context. @xcite attempt to generate questions in an answer-agnostic manner by adapting the self-attention mechanism of transformers with a copying mechanism, placeholders, and contextual word embeddings. dong et al. (2022) propose a qag model for closedbook setting without access to external knowledge by modeling the semantic relationships between questions and answers at a contextual level and measuring the answerability of the generated questions. in recent times, several attempts have been made to automatically generate valid qa pairs for educational purposes. fairytaleqa proposed by xu et al. (2022) is a representative dataset in educational qag. education experts manually generated qa pairs suitable for learning and assessing children's reading comprehension skills. with the dataset, yao et al. (2022) present an educational qag system through a combination of three-step modules. zhao et al. ( 2022 ) deal with the highcognitive demand question generation based on three out of seven narrative elements in the fairy-taleqa. @xcite summarize qa pairs to given book chapters and provide them to the fine-tuned t5 @xcite ) models. however, these studies rarely consider diversity when performing qag. leveraging diverse qa types are important aspect in qag since using various interrogative words promotes different parts of the brain, which facilitates children's comprehensive learning @xcite . varying answer types is also a factor that contributes to a balanced assessment, as the difficulty can be controlled by adjusting whether the answer is revealed in the passage @xcite . the evidence emphasizes the importance of considering a variety of qa pairs from a broad perspective for the effectiveness of reading comprehension @xcite . ## method our qag framework comprises three task-oriented processes: a qfs-based answer generator, an iterative qa generator, and a relevancy-aware ranker. the main goal of the two generators is to expand qa pair candidates containing diverse question and answer types. the ranker aims to determine the final output by scoring qa pair candidates. the overall qag architecture of our framework is depicted in figure 1 . ## case study performance of multiple agm we investigate various methods to add a clue that can be a key element when constructing agm inputs. the clue is then fed into the bart large-based agm input along with the given passages, and the answer is predicted. a is the baseline where this learns to directly generate answers for given passages. in ds, one to three sentences in the passage closest to the question are retrieved. in ext-ret, a phrase or sentence closest to the question is retrieved from the external resource narrativeqa @xcite . ## conclusion in this paper, we proposed a qag framework for educational purposes featuring diverse and valid question and answer types. our framework is structured with three task-oriented processes, with a particular emphasis on expanding diverse and valid types of qa pair candidates in the generator, and selecting high-quality qa pairs in the ranker. we conducted extensive evaluations of generated qa pairs, including quantitative, qualitative, and statistical evaluations with detailed analyses, and observed that our system achieved remarkable performance. our framework has the potential to promote various cognitive activities in children learning by providing diverse and effective qa pairs for educational purposes. as our modularized task-oriented frameworks are tailored to real-world demand, we further expect the collaborative use of humans and ai.