Question Generation Based on Grammar Knowledge and Fine-grained Classification (2022)

Abstract: Question generation is the task of automatically generating questions based on given context and answers, and there are problems that the types of questions and answers do not match. In minority languages such as Tibetan, since the grammar rules are complex and the training data is small, the related research on question generation is still in its infancy. To solve the above problems, this paper constructs a question type classifier and a question generator. We perform fine-grained division of question types and integrate grammatical knowledge into question type classifiers to improve the accuracy of question types. Then, the types predicted by the question type classifier are fed into the question generator. Our model improves the accuracy of interrogative words in generated questions, and the BLEU-4 on SQuAD reaches 17.52, the BLEU-4 on HotpotQA reaches 19.31, the BLEU-4 on TibetanQA reaches 25.58.

## introduction question generation is an essential research direction in natural language generation, which aims to generate a grammatical question based on the given paragraph and answer, and the question can be answered by the given answer. recently, question generation has received extensive attention in academia and industry. it can automatically generate training data for question answering systems @xcite and reading comprehension tasks @xcite . it can also simulate user questions in the field of education @xcite , and guide the machine to ask questions actively in the field of dialogue @xcite , and strengthen human-computer interaction @xcite . in recent years, the research on question generation has achieved great success, but we analyze the questions generated by an advanced answeraware s2s model @xcite and find that some question types and answers did not match. paragraph ..., to ensure a space, in 1947, abc submitted five applications for television station licenses, ... answer 1947 gold in what year did abc submit licenses for 5 television stations ? generated what is the name of the space that was used to ensure a space on television station ? as shown in to solve the problem, some studies have proposed to use answer information to improve the interrogative words accuracy of generated questions @xcite @xcite . however, they did not consider the structural and grammatical features of the answers. some studies classify question types according to the category of interrogative words, and use the first word of the question as the question type. however, we analyze the squad @xcite and find that the first word of question is not always an interrogative word, and there is a mismatch between interrogative words and question types. for example, "in what year" is a question about time, but using the classification of interrogative word will classify it into "what" category. therefore, using interrogative words to classify questions will also introduce wrong information. to solve above problems, this paper constructs a question type classifier based on a pre-trained language model and a question generator. we integrate the grammatical knowledge of question into the question type classifier, and classify the type according to the inquiry object of the question, we apply this method to english and tibetan. the grammatical features of tibetan are different from those of english, for example, interrogative words in english often appear at the beginning of the ques-tions, but in tibetan, they will appear in multiple positions of questions, and tibetan interrogative words have ambiguity. tibetan is a phonetic writing script, which belongs to the consonant character type. it is divided into two parts: consonants and vowels. the grammatical rules in tibetan are relatively complex, but there are clear organizational forms and verb changes. tibetan is composed of syllables, a syllable containing one or up to seven characters @xcite , and syllables are separated by an intersyllable marker " ", which is a simple superscripted dot, for example, (what is tashi's major?). in this paper, we conduct experiments on english and tibetan datasets respectively, the experimental results show that the bleu-4 on squad is 17.52, which is 1.21 higher than the experimental result of zhou et al @xcite . bleu-4 on hotpotqa is 19.31, which is 3.9 higher than the result of xie et al @xcite . bleu-2 on tibetanqa is 35.07, which is 9.73 highter than the result of sun et al @xcite . the main contributions of this paper are as follows. ## related work question generation mainly adopts two methods, rule-based and neural network-based. the method based on rules and templates is to define some heuristic rules on the syntax tree to convert a sentence into a question @xcite @xcite , but the generated questions have limited diversity and poor portability. with the emergence of largescale and high-quality machine reading comprehension datasets, such as squad @xcite , marco @xcite , hotpotqa @xcite and triviaqa @xcite , neural network-based question generation has made great progress. existing works are mainly based on seq2seq architecture with attention mechanism and copy mechanism @xcite @xcite . to generate answer-related questions, the methods of using the answer position as input or encoding the answer are proposed @xcite @xcite . song et al. @xcite propose to use a multi-view matching method to calculate the similarity between the answer representation and the hidden layer of the paragraph in three ways, and finally the bleu-4 of this method reaches 13.98 on squad. wang et al. @xcite introduce a novel hidden answer pivot module to model the hidden answer information to generate questions. @xcite jointly model unstructured sentence and structured answer-relevant relation to generate questions. murakhovs' ka et al. @xcite train a unified qg model and generate different cognitive levels questions by controlling the answer type. to solve the two problems, the mismatch between the generated question words and answers, and the words copied by the model are far from and irrelevant to the answer. sun et al. @xcite propose an answer-focused and position-aware neural question generation model. the final experimental result bleu-4 reaches 15.64 on squad. to address the challenge of long text on question generation model based on seq2seq, tuan et al. @xcite propose to encode context information in the long text so that model can obtain more information to improve the performance of question generation. zhao et al. @xcite propose a paragraph-level question generation with maxout pointer and gated self-attention, which can effectively utilize paragraph-level context and outperform sentence-level context performance, and finally achieve bleu-4 of 16.38 on squad. to improve the answerability of the question and the relevance of the original text, xie et al. @xcite design three different reward discriminators to calculate reward scores and feed them back to the question generator to improve the fluency, relevance and answerability of generated questions, and finally bleu-4 reach 15.43 on the hotpotqa @xcite . the above methods improve the relevance between the generated question and the answer, but there is a problem that the generated lstm p a c p a p a p a bert/tibert encoding knowledges + knowledges embedding bert/tibert vector softmax linear token embedding segment embedding position embedding tag embedding input [cls] p [sep] a lexical embedding ask facts (what) ask people (who) ask methods (how) ask time (when) ask choice (which) ask place (where) ask reason (why) gengral questions (whether) others lstm p a c lstm p a c c lstm lstm lstm q: in what year was the victoria and albert museum founded ? a: 1852 question generator question type classifier type question type and the answer type do not match. to solve the problems, ma et al. @xcite propose to integrate sentence-level semantic matching and answer position inference. zhou et al. @xcite propose a unified model to predict the question type and to generate question. wang et al. @xcite ) propose use answer-aware initialization module to introduce document and answer to decoder, and design a semantic-rich fusion attention to support decoder. the finally bleu-4 reach 17.54 on hotpotqa. the above research are applications of question generation on english, there are few related research on tibetan. sun et al. @xcite construct a tibetan reading comprehension dataset named tibetanqa, and propose a tibetan question generation model based on a seq2seq. by introducing an attention mechanism to capture the key semantic information related to the answer and generate an answer-aware question. we follow the method and analyze the generated questions, and find that the accuracy of interrogative words in tibetan is only 26.03%, which seriously hinders the research on tibetan question generation. our method of integrating tibetan grammar knowledge can significantly improve accuracy of interrogative words in tibetan question generation. ## model description the model architecture of this paper is shown in figure 1 . the question type classifier is used to predict the type of target question, and the prediction will be integrated into question generator to guide question generation. ## case analysis to explore the impact of our model on question generation, ## conclusion to solve the problem of mismatch between question types and answers in question generation, this paper constructs a question type classifier and a question generator. we classify questions into finegrained classification and integrate grammar knowledge into question type classifier to improve the accuracy of question types, then, the prediction results of the classifier are fused into the question generator to improve the performance of question generation. to verify the effectiveness of our model, we perform an upper bound analysis on it, we integrate grammar knowledge into the question generator to provide accurate question types to guide the model to generate questions. the final experimental results show that the method proposed in this paper not only improves the accuracy of question words in the generated question, but also improves the quality of the generated question.