Question: Does the reinforcement learning (RL) bring the complete and complex question to otherwise maximum likelihood estimation (MLE) optimized QG model?
Evidence: In natural language processing (NLP), question generation is considered to be an important yet challenging problem. Given a passage and answer as inputs to the model, the task is to generate a semantically coherent question for the given answer.
In the past, question generation has been tackled using rule-based approaches such as question templates BIBREF0 or utilizing named entity information and predictive argument structures of sentences BIBREF1. Recently, neural-based approaches have accomplished impressive results BIBREF2, BIBREF3, BIBREF4 for the task of question generation. The availability of large-scale machine reading comprehension datasets such as SQuAD BIBREF5, NewsQA BIBREF6, MSMARCO BIBREF7 etc. have facilitated research in question answering task. SQuAD BIBREF5 dataset itself has been the de facto choice for most of the previous works in question generation. However, 90% of the questions in SQuAD can be answered from a single sentence BIBREF8, hence former QG systems trained on SQuAD are not capable of distilling and utilizing information from multiple sentences. Recently released multi-hop datasets such as QAngaroo BIBREF9, ComplexWebQuestions BIBREF10 and HotPotQA BIBREF11 are more suitable for building QG systems that required to gather and utilize information across multiple documents as opposed to a single paragraph or sentence.
In multi-hop question answering, one has to reason over multiple relevant sentences from different paragraphs to answer a given question. We refer to these relevant sentences as supporting facts in the context. Hence, we frame Multi-hop question generation as the task of generating the question conditioned on the information gathered from reasoning over all the supporting facts across multiple paragraphs/documents. Since this task requires assembling and summarizing information from multiple relevant documents in contrast to a single sentence/paragraph, therefore, it is more challenging than the existing single-hop QG task. Further, the presence of irrelevant information makes it difficult to capture the supporting facts required for question generation. The explicit information about the supporting facts in the document is not often readily available, which makes the task more complex. In this work, we provide an alternative to get the supporting facts information from the document with the help of multi-task learning. Table TABREF1 gives sample examples from SQuAD and HotPotQA dataset. It is cleared from the example that the single-hop question is formed by focusing on a single sentence/document and answer, while in multi-hop question, multiple supporting facts from different documents and answer are accumulated to form the question.
Multi-hop QG has real-world applications in several domains, such as education, chatbots, etc. The questions generated from the multi-hop approach will inspire critical thinking in students by encouraging them to reason over the relationship between multiple sentences to answer correctly. Specifically, solving these questions requires higher-order cognitive-skills (e.g., applying, analyzing). Therefore, forming challenging questions is crucial for evaluating a student’s knowledge and stimulating self-learning. Similarly, in goal-oriented chatbots, multi-hop QG is an important skill for chatbots, e.g., in initiating conversations, asking and providing detailed information to the user by considering multiple sources of information. In contrast, in a single-hop QG, only single source of information is considered while generation.
In this paper, we propose to tackle Multi-hop QG problem in two stages. In the first stage, we learn supporting facts aware encoder representation to predict the supporting facts from the documents by jointly training with question generation and subsequently enforcing the utilization of these supporting facts. The former is achieved by sharing the encoder weights with an answer-aware supporting facts prediction network, trained jointly in a multi-task learning framework. The latter objective is formulated as a question-aware supporting facts prediction reward, which is optimized alongside supervised sequence loss. Additionally, we observe that multi-task framework offers substantial improvements in the performance of question generation and also avoid the inclusion of noisy sentences information in generated question, and reinforcement learning (RL) brings the complete and complex question to otherwise maximum likelihood estimation (MLE) optimized QG model.
Our main contributions in this work are: (i). We introduce the problem of multi-hop question generation and propose a multi-task training framework to condition the shared encoder with supporting facts information. (ii). We formulate a novel reward function, multihop-enhanced reward via question-aware supporting fact predictions to enforce the maximum utilization of supporting facts to generate a question; (iii). We introduce an automatic evaluation metric to measure the coverage of supporting facts in the generated question. (iv). Empirical results show that our proposed method outperforms the current state-of-the-art single-hop QG models over several automatic and human evaluation metrics on the HotPotQA dataset.
Additionally, we observe that multi-task framework offers substantial improvements in the performance of question generation and also avoid the inclusion of noisy sentences information in generated question, and reinforcement learning (RL) brings the complete and complex question to otherwise maximum likelihood estimation (MLE) optimized QG model.
Answer: Yes, it does.