Generating Deep Questions with Commonsense Reasoning Ability from the Text by Disentangled Adversarial Inference
Year: 2023

Abstract:
This paper proposes a new task of commonsense question generation, which aims to yield deep-level and to-the-point questions from the text. Their answers need to reason over disjoint relevant contexts and external commonsense knowledge, such as encyclopedic facts and causality. The knowledge may not be explicitly mentioned in the text but is used by most humans for problem-shooting. Such complex reasoning with hidden contexts involves deep semantic understanding. Thus, this task has great application value, such as making high-quality quizzes in advanced exams. Due to the lack of modeling complexity, existing methods may produce shallow questions that can be answered by simple word matching. To address these challenges, we propose a new QG model by simultaneously considering asking contents, expressive ways, and answering complexity. We first retrieve text-related commonsense context. Then we disentangle the key factors that control questions in terms of reasoning content and verbalized way. Independence priors and constraints are imposed to facilitate disentanglement. We further develop a discriminator to promote the deep results by considering their answering complexity. Through adversarial inference, we learn the latent factors from data. By sampling the expressive factor from the data distributions, diverse questions can be yielded. Evaluations of two typical data sets show the effectiveness of our approach.

heading: 1 Introduction
text: Text-oriented question generation (QG) aims to endow machines with the ability to ask relevant and thought-provoking questions about the given text. This task can support a wide range of real-world applications, such as yielding quizzes from course materials for education (Qu et al., 2021), and generating questions as synthetic data to train a QA system (Wang et al., 2019). According to Bloom’s
∗Corresponding author.
taxonomy (Zhang et al., 2022), questions can be classified into different levels of cognitive ability. The simple ones involve only the shallow meaning of the text. For example, the question “What is the longest river in the world?” about the given text “The Nile is the longest river in the world” can be answered directly by matching. However, matching is far from a real understanding of the semantics (Ko et al., 2020). For example, in the field of education, simple questions are hard to fully evaluate students’ learning effects, especially in advanced exams. Thus, the deep questions that require semantic understanding and reasoning have attracted extensive attention. As shown in Fig.(1), the question asks about some kind of clothing. The answer needs to be deduced from multiple relevant but disjoint clues in the contexts, i.e., “traditional local costumes,” “mountain,” “Mount Fuji,” as well as implicit commonsense knowledge, such as Mount Fuji is a famous mountain in Japan, Kimono is the traditional local costume of Japanese, and Japanese kimono is a kind of robe clothing. Here, commonsense refers to the self-evident and unwritten knowledge shared by most humans, such as encyclopedism and causality. Although it does not appear in the text, it is hard to find the correct answer without it due to the incomplete context. Asking this kind of question requires a full understanding of commonsense and the ability to make
470
inferences. That is a key ingredient for general intelligence. Some works have studied how to answer such questions, represented by commonsense QA and multi-hop QA (Rajani et al., 2019), but less effort explores how to generate them. We thus propose a new QG task to fill this research gap.
Raising deep questions involves three fundamental processes: what to ask, how to ask, and how to answer. What to ask is to identify the answer and its relevant reasoning contents. Learning how to ask focuses on the language qualities, such as grammatical correctness and expressive diversity, since the question could be asked in various ways and each way needs to be fluent. Respectively, how to answer reflects the question’s complexity, shallow questions only need to match the text while deep reasoning ones require understanding the semantics in contexts with long-range dependencies and hidden commonsense knowledge. For these processes, traditional QG models have considerable defects. The rule-based method relies on handcrafted rules or transformation templates with a limited scale. That would restrict the coverage of results. Due to the neglect of indispensable answering feedback, the results are not guaranteed to be inferable and deep. On the other hand, the neural model mainly follows the sequence-to-sequence framework which is data-driven and labor-saving, but this monotonous mapping is hard to learn the one-to-many diversified generation. Besides, this method cannot cover the nuances of data by using a single vector to encode complex input features, especially when the training data is insufficient or has a long tail distribution. Spurious correlations and unexpected variances would easily mislead the single-factor model and deteriorate its robustness.
Motivated by the above observations, we propose a practical model for the new commonsense reasoning QG task. Concretely, we first leverage a knowledge-enhanced model to represent the text contexts, as well as relevant commonsense concepts and relations. We then learn the key factors related to the necessary ask contents and expressive ways. The first factor refers to the reasoning clues involved in asking deep questions, including entities and relations in the commonsense deductive context. Another encompasses other variations not covered by the content factor, like the verbalized styles and patterns. These factors can be sampled from the data manifold and used as conditions to generate results. This sampling-then-generate way
alleviates the difficulty of collecting real data at the lower ends of a distribution tail in order to learn diversified generation. All these unknown factors may be mutually interrelated. Simply assuming that they are independent would oversimplify the latent manifold, leading to unsatisfied results due to the incorrect preservation of the redundant noises. We thus propose to disentangle such factors explicitly to ensure their independence and prevent information leakage between them. To achieve this goal, we introduce two kinds of latent variables to characterize the factors and impose constraints to learn their disentangled representations. These variables are forced to obey two prior non-overlapping distributions, including an isotropic Gaussian for the expressive way and another conditional Gaussian mixture for the reasoning content. Each component can be viewed as a cluster of neural templates or prototypes, which can be used as a guide to control the detailed nuances of a generation process. To encourage the deep and inferable questions, we impose regularization on the distributions by considering the answering complexity, including whether the answer matches the question and involves multihop reasoning with implicit commonsense knowledge. Moreover, we design an adversarial inference mechanism to derive optimal distributions for the disentangled factors. To facilitate deployment, we further employ the prefix-tuning technique (Li and Liang, 2021) that can support inference with limited labeled data. Our model enables one-to-many generation by randomly sampling the expressive factor from the distributions to yield new reasoning questions. Experimental results on two popular data sets show the effectiveness of our approach.
The main contributions of this paper include,
• We are the first to study the task of commonsense reasoning question generation from text.
• We propose a new model for the commonsense reasoning QG task. By a latent space with disentangled priors, our model can grasp the key factors that control the reasoning content and expressive way. Based on the factors as generative conditions, we can yield new diverse results by sampling data distributions.
• We design a discriminator and learn it by adversarial inference. It can provide complexity feedback as a guide to regularize the generator. Extensive experiments are conducted to evaluate our model quantitatively and qualitatively.
The rest of this paper is organized as follows. Section 2 elaborates on the proposed method for the new commonsense QG task. Afterward, Section 3 presents experimental results. Section 4 reviews related works and Section 5 concludes the paper.

heading: 2 Approach
text: As shown in Fig.(2), we propose a new framework for this task. We first encode the text context and relevant commonsense knowledge. We then disentangle the key ask-related factors on reasoning content and expressive way. The answering feedback is also considered. By sampling the learned data manifold, we can derive factors as conditions to yield new questions. Next, we define some notations and then show the details of each component.

heading: 2.1 Notations and Problem Formulation
text: Given a passage c, the QG task aims to generate a valid question y corresponding to c and the answer a. The answering process involves deducing over a subset of disjoint supporting clues Py = {p1, · · · , pk} from c, that is, {y → p1 → · · · → pk → a}, where → represents entailment, pj is a necessary problem-solving clue which can be a sentence or entity in c, k is the number of clues. When k = 1, we call y a traditional shallow question whose answer can be found by one-step matching of the given text. Respectively, when k > 1, y is a deep question with k reasoning steps. In complex reasoning, some clues are not in c, but from the background knowledge outside c. That is, a cannot be derived by using only c, and we have to answer y by introducing commonsense knowledge to supplement the missing contexts. Our task aims to yield this kind of question with a commonsense multi-step reasoning requirement. Compared to existing QG tasks, our question needs a deeper understanding of the semantics in c. Moreover, it is necessary to simultaneously figure out the ask-
related contents, verbalized ways, and answering complexity. This task can be applied to many commercial scenarios like making quizzes for advanced exams. Since a question can be asked in many acceptable ways, where each y should be answered by a, we input a to indicate the asking direction.

heading: 2.2 Commonsense-enhanced Representation
text: Since asking deep questions involves understanding and reasoning the input text content, we need to derive a good semantic representation of the text. In particular, we first embed the context features in the input sample by looking up the pre-trained vectors in RoBERTa (Liu et al., 2019). The given text c and answer a are embedded as eac = ROBERTA([CLS], c, [SEP ], a, [SEP ]), where [CLS] and [SEP ] are special separator tokens. Similarly, question y is represented as ey = ROBERTA(y). Afterward, we retrieve the commonsense features related to the given text. We resort to the knowledge graphs (KG) which contain plentiful human-shared knowledge. The first KG we consider is ConceptNet (Li et al., 2016). It contains millions of factual knowledge like encyclopedic concepts and parent-child relations. Another is ATOMIC (Sap et al., 2019) with plentiful procedural knowledge like if-then causal events. Such KGs can help to fill the implicit commonsense gap in the context. Since the KGs have different structures, we adopt the work of Ma et al. (2019b) to elicit the relevant KG contents. In particular, we identify ConceptNet entities appearing in the text by phrase-based matching, and then collect the relevant ν-hop triples. Accordingly, we utilize a transformer called COMET (Bosselut et al., 2019) which is pretrained on ATOMIC to generate the event triples based on the text and pre-defined relation types. Nine reasoning types in COMET are employed. Based on the extracted and generated contents, we can obtain a commonsense augmented graph. We then employ graph convolutional net-
works (GCN) (Kipf and Welling, 2017) to encode the graph as ekgc = GCN(V,E), where V , E denotes the set of nodes and edges, respectively. The nodes are the concepts, entities in the text and KGs, and the edges represent their relations. To integrate context and commonsense features, we apply an MLP network with ReLU activation to fuse the vectors as u = MLP ([eac; e kg c ]), where [·; ·] is the concatenation operator.

heading: 2.3 Commonsense Reasoning QG Model
text: Traditional QG methods often learn an encoding vector of the input to decode the result. This single vector is insufficient to grasp the subtle structure of reasoning questions, and the one-to-one mapping is hard to capture diverse expressive ways. It is also difficult to find a suitable mapping for the rare cases at the distribution tail. We thus design a conditional generation framework that can disentangle multiple factors to finely model the reasoning contents and expressive patterns. The results can be easily inferred from a continuous data manifold, which has better generalization ability than learning the mapping of scattered points. That provides great flexibility to yield diverse results by adjusting the expression factors sampled from data distributions.
Conditional Generation: Our QG model yields the question based on the input of two latent variables. One is to characterize the reasoning contents related to what to ask, namely zu. Another is used to quantify the verbalized expressions of how to ask, i.e., ze. These variables can be learned from data by conducting approximate inference. Since the latent space allows invariance of distracting transformations, it is easier to discover elements of variations governing the data distribution. That helps to reason the data at an abstract level and find the key question-controlled factors. Our task can be formalized as an iterative word generative process based on a marginal distribution pθ(ŷ|ze, zu), where θ is the model parameters. ze can be sampled from a verbalized prior distribution, which helps to form the results expressed in various ways. To reduce the labeled data demand for training θ, we further employ the prefix-tuning technique that can freeze pre-trained vectors and learn only a few prompt parameters. The continuous prompt is designed as Mθ[i, :] = MLPθ([M′θ[i, :]; ze; zu]), where M′θ is a learnable matrix, MLP (·) is a multilayer network. Based on this prompt, we can produce the question word-by-word by Eq.(1), where
ŷ<t represents the outputted 1th to (t− 1)th words. pθ(ŷ|ze, zu) = ∏J t=1 pθ(ŷt|ŷ<t,Mθ[i, :]) (1)
To well capture abundant expressive patterns in the questions, we let ze obey the prior distribution pψ of factorized Gaussian N (ze;µye , λeI), where µye is the mean, and λe is the variance. Different from a standard normal distribution N (0, I), this allows us to associate its mean with the linguistic features Φ(y) from the question y by µye = WyΦ(y), where Wy is a project matrix and Φ(y) is the mean of question encodings. Considering the given text may contain multiple inquiry topics, the content latent zu is expected to be composed of K-independent components. Thus, we make zu follow Gaussian mixture distributions, i.e., ∑K k=1 pψ(Mk|u)N (zu;µ y uk , λuI), where Mk is a random variable to indicate the kth component. Disentangled Inference: To better learn the latent representation z, we introduce a series of constraints. First, the latent vector should be able to fully characterize the corresponding content. That can be quantified by maximizing mutual information (MI) (Cheng et al., 2020) of MI(ze, y) and MI(zu,u), where u is the commonsenseenhanced representation of the inputs c and a. To improve the model’s robustness, we impose disentangled constraints. The content vector is encouraged to encode disjoint information with the expression vector and vice versa. That can reduce redundancy and provide refined control over results. We seek to explicitly minimize the shared information of vectors by adding a divergence-based regularization of Maximum Mean Discrepancy (MMD) (Gretton et al., 2012), as MMD(p(ze|y), p(zu|u)). By aggregating the constraints, our generator’s objective of Eq.(1) can be reformulated as Eq.(2).
max ∫ pθ(ŷ|ze, zu)pψ(ze|y)pψ(zu|a, c)dzedzu
= max ∑n
i=1 [log p(ŷ|yi, ai, ci) +MI(zyi , yi) +MI(zui ,ui)−MMD(p(zyi |yi), p(zui |ui))]
(2) We then utilize the variational inference technique to solve it since direct optimization is intractable. A variational posterior qϕ(·) is introduced to approximate the prior distribution pψ(·). By maximizing the evidence lower bound (ELBO) of Eq.(2), we can derive an equivalent objective as Eq.(3).
maxEqϕ(ze,zu|y,u)[log pψ(ŷ, ze, zu|y,u) − log qϕ(ze, zu|y,u)] (3)
This ELBO can be decomposed into Eq.(4) by minimizing the reconstruction loss Lr of y given the
inputs c and a (encoded as u), and regularizing the approximate posterior qϕ(·) to be close to the prior pψ(·) by KL divergence, where Le and Lu are the divergence losses for latent ze and zu, respectively.
Lgenerator(ψ, ϕ, y, c, a) = Lr + Le + Lu Lr = Eqϕ(ze, zu|y,u)[log pψ(ŷ|ze, zu)] Le = DKL(qϕ(ze|ŷ, y)||pψ(ze|y)) Lu = DKL(qϕ(zu|ŷ, u)||pψ(zu|u)) (4)
Le is the loss related to the expression factor. Similar to the prior pψ(·), the posterior qϕ(·) is followed the factorized Gaussian, as N (ze;µye , diag(σ2ye)). By applying the reparameterization trick (Kingma and Welling, 2014), we can calculate the latent ze as µe + σe ⊙ ϵe, where ϵe is the Gaussian factor drawn from N (0, I), ⊙ is the element-wise product. Based on ze, Le can be calculated as Eq.(5).
Le = − 1λe ||ze − µ y e ||2 + logσ2ye (5)
Another loss Lu is relevant to the reasoning contents in passage c and answer a. Considering the contents may contain multiple inquiry topics, we characterize the posterior qϕ by Gaussian mixture distributions, and introduce K latent topic prototypes {tk}Kk=1. Each Gaussian component is promoted to be close to the prototype variational distribution. That can be achieved by making the component be N (zu;µyuk , diag(σ2u)). The K is preset, when the value is small, the content modeling is simple and coarse-grained. The reasoning aspects involved in the generated results will be less. When the K value is large, the convergence speed becomes slower. By tuning, we set K to 10 in the experiment. To encourage its mean corresponding to one kind of topic, we compute µyuk as Wttk, where tk is the centroid of a cluster k. Each cluster can be computed by the kmeans method. The probability of the input content belonging to the k prototype is parameterized as qϕ(Mk|u) = exp(−dist(zu,µyuk )/τ)∑ k′ exp(−dist(zu,µ y uk′ )/τ) , where τ is a temperature set to 1 normally, dist(·) is a Euclidean distance between the mean and the latent zu. In this way, we compute the loss Lu as Eq.(6)
Lu = K∑ k=1 qϕ(Mk|u)[− 12λu ||zu − µ p uk ||2] + logσ2u
(6) Adversarial Training: Unlike shallow question, complex one has an inherent reasoning structure. Based on traditional supervised training, the model is only required to have maximum likelihood with
the ground truth, but neglects to grasp this crucial structure. It may learn some trivial tricks to simply copy similar terms, leading to shallow results. Thus, it is necessary to inject the answering feedback into the generator for judging the rationality of results. Instead of using a discrete judged metric, we design a differentiable discriminator that can guide the generator optimization via policy gradient. It is trained to distinguish between real data examples and synthetic ones produced by the generator. The generator is then optimized for fooling the discriminator. By their adversarial game, the distribution of the generated examples moves towards the distribution of real data. That directs the generator to learn complex distributions and produce reasonable realistic questions. In particular, we use a QA model called UNICORN (Lourie et al., 2021) to capture the answerable feedback. It obtains stateof-the-art performance on solving commonsense reasoning questions. For each sample (c, a, ŷ), we compute dans = σ1(W1[eâ; ea]), where W is the weight, σ(·) is the logistic function, eâ is the answer predicted by UNICORN(c, ŷ), ea is an answer encoding. To ensure that the question is inferable, we thus leverage a typical matching-based QA model called gated-attention reader(GA) (Dhingra et al., 2017). We then compare its answer against the reasoning model UNICORN. When these two answers match, there is no need for reasoning. It is highly likely to be a simple but not deep question. We introduce a metric dcpx = σ2(W2[eâ1 ; eâ2 ]), where eâ1 and eâ2 are the answers predicted by UNICORN(c, ŷ) and GA(c, ŷ), respectively.
The discriminator is developed by integrating these aspects. For each sample x = (c, a, y), we can predict a reward as dδ(x) = γdans(x) + (1− γ)dcpx(x), where λ is a trade-off factor. This reward can be used as guidance to co-train the generator by reinforcement learning. The discriminator can be trained based on the supervised loss of human-written data. Considering such labeled data may not be sufficient, we use the model-generated samples as extra data to augment the training.
In the prediction phase, the input is a passage and an answer. Each test case can generate multiple questions with three steps. We first encode the input passage and answer, and then derive a latent content factor zu based on pψ(zu|u). Accordingly, we sample another verbalized factor ze from the prior pψ. Afterward, we feed them into the prefix encoder and decode question ŷ by pθ(·) in Eq.(1).

heading: 3 Evaluations
text: We extensively evaluated the effectiveness of our method with quantitative and qualitative analysis.

heading: 3.1 Data and Experimental Settings
text: Since QG is a complementary task of QA, we conducted experiments on two typical QA data sets that involved commonsense reasoning, including Cosmos QA (Huang et al., 2019) and MCScript 2.0 (Ostermann et al., 2018). These data sets were split as train/dev/test sets with the size of 25.6k/3k/7k and 14.2k/2.0k/3.6k samples, respectively. The samples mostly required context understanding and commonsense reasoning. They were more suitable than other data sets like CommonsenseQA (Talmor et al., 2019) which provided no text context, SQuAD (Rajpurkar et al., 2016) did not need multihop deduction, and LogiQA (Liu et al., 2020) with the general questions such as “Which one is true?” that can be yielded by rules. For each test case, our inputs included a passage and an answer to guide the asking direction. We employed three standard metrics in the field of text generation to evaluate the generative quality based on n-gram overlap with the ground truth, including BLEU-4 (Papineni et al., 2002), METEOR (Banerjee and Lavie, 2005), and ROUGE-L (Lin, 2004). In addition, we observed that the question involves fine-grained reasoning logic on the answering process. Even if a similar word is substituted, the questions may mismatch the answers, or become too shallow to be inferable. Thus, we utilized two distribution overlap metrics, i.e., QA-based Evaluation (QAE) (Zhang and Bansal, 2019b), and Reverse QAE (R-QAE) (Lee et al., 2020a) to measure diversity instead of using traditional similarity-based metrics. To compute QAE, we first trained a QA model on the generated data and then tested it on ground-truth data. The score is high when these two distributions match, which indicates the generated quality reaches human annotations. R-QAE was calculated by swapping the train and test data. Its value is lower when the generated data is more diverse than the ground truth. That is more suitable to evaluate our task by considering the answering process. Besides, the commonsense reasoning ability was evaluated by human evaluation. To avoid biases, we randomly sampled 500 test cases and rated the predictions by a crowdsourcing platform Figure-Eight 1 with five participants. It was a rating in terms of three
1https://appen.com/figure-eight-is-now-appen/
metrics, including valid syntax, relevance to input text, and commonsense deductibility of the answer. We averaged the cumulative scores of judgments as performance. The scores are between 1∼10, where 1 is the worst, 10 is the best. For the methods with multiple diverse results, we computed metrics for each prediction and reported the average scores.
Our model was implemented based on the PyTorch (Paszke et al., 2019) and ran on the 24 GB Nvidia RTX 3090 GPU for 18 hours. We leveraged the RoBERTa-large (355M parameters) model provided by HuggingFace library to initialize the word embeddings. We employed the transformer-based GPT-2 medium as the decoder. In the diversity evaluation, the metrics (i.e., QAE and R-QAE) were computed based on the UNICORN QA model. We trained for a maximum of 10, 000 steps and validated every 200 steps, with early stopping after one round of no improvement in validation loss. AdamW (Loshchilov and Hutter, 2019) was used as the optimizer, with a linear learning rate scheduler taking 5,000 warm-up steps. Gradients were clipped if their norm exceeds 1.0, and weight decay on all non-bias parameters was set to 0.01. In the prediction phase, the outputted candidate size was set to 3. The trade-off factor γ was tuned to 0.3.

heading: 3.2 Comparisons against State of the Arts
text: To evaluate the model persuasively, we utilized six baselines that performed well in the QG task, including (a) NQG++ (Zhou et al., 2017), a basic sequence-to-sequence model; (b) UniLM (Dong et al., 2019), a pre-trained language model that can fine-tune on KGs to incorporate commonsense context; (c) SGGDQ (Pan et al., 2020), a graph-based model which can produce results with multi-hop deduction ability by capturing the context dependency of the text; (d) HCVAE (Lee et al., 2020b), a VAEbased model that can yield results in several ways for one test case. (e) DAANet (Xiao et al., 2018), dual learning of QG and QA that mutually provided feedback to enhance each other simultaneously; (e) SemQG (Zhang and Bansal, 2019b), which trained QG by reinforcement learning with a QA-based reward. These baselines were open-source and we reimplemented them with the original settings.
Fig.(3) showed the comparison results in terms of three n-gram overlap metrics. Our model held the best performance against other baselines. As illustrated in Tab.(1), our model obtained high QAE but low R-QAE. That reflected the synthetic data
were closer to human annotations. As shown in Lee et al. (2020a), lower R-QAE means resultant data covers larger distributions. Although trivially invalid questions may also cause low R-QAE, a combination of high QAE and low R-QAE can indicate the diversity of our results. By a single encoded vector, NQG++ was difficult to cover the nuances of data. UniLM could encode commonsense but its reasoning ability was insufficient. The graph model SGGDQ was good at multi-hop samples, but its monotonous mapping framework is difficult to support one-to-many generation. Due to the lack of disentanglement, VAE-based model HCVAE would be affected by unexpected irreverent noises which will harm performance. All baselines neglected to consider the feedback of answering complexity. Without this crucial guidance, the performance would be deteriorated. DAANet and SemQG used the QA feedback, but the dual soft constraint of DAANet and the high variance of the reinforced SemQG were hard to ensure results’ consistency.
Moreover, we evaluated our model’s applicability in low-resource scenarios. We started to train it with the full training data and gradually halved the size. The results on 1/2 and 1/8 data size were presented in Tab.(2) and Tab.(3), respectively. We found that our performance decline
was smallest when training sets shrunk. That reflected our model had a good generalization ability to achieve greater outperformance by disentangling key question-controlled factors.

heading: 3.3 Ablation Studies
text: To better gain insight into the relative contributions of our QG’s components, we performed ablation studies on four parts, including (1) Ours-LM which replaced the commonsense-enhanced model with the raw PLM; (2) Ours-Disentangler that discarded the independence constraints with disentangled priors; (3) Ours-Prefix threw away the prefix tuning then trained the model on the full parameters; (4) Ours-Discriminator that abandoned the discriminator and learned with typical supervised loss.
As shown in Tab.(4), the ablation of all evaluated parts led to a performance drop, where some drops were more than 10%. We could infer that commonsense knowledge can help to supplement missing contexts implied in the text. Without this guidance, the results’ rationality will be harmed. When the prefix tuning module was discarded, the training
adequacy would be reduced with limited labeled data. Deleting a disentangled module would reduce the model’s robustness and controllability. Without the discriminator, there was inadequate to indicate that the results were deep and logically consistent.

heading: 3.4 Human Evaluations and Analysis
text: Furthermore, we conducted human evaluations to judge whether the results were deep and had highlevel answering skills like commonsense reasoning. We employed Randolph’s kappa for inter-rater reliability measurement. The kappa κ scores were 0.77, 0.65, and 0.75 for syntax, relevance, and deductibility, respectively, which indicated a good agreement. As presented in Fig.(4), our model significantly outperformed the baselines in terms of three metrics. That was consistent with the quantitative results in the previous section. The improvement in the deductibility metric was the largest. That indicated our results were to-the-point and valid, especially inferable, due to the simultaneous consideration of what to ask, how to ask, and how to answer.

heading: 3.5 Evaluations on the Trade-off Parameter
text: To examine the trade-off parameter (i.e., γ) in the discriminator dδ, we tuned it from [0, 1] with 0.1 as an interval. The performance change curve was plotted in Fig.(5). The best results were obtained at
around 0.3. The performance dropped dramatically when any parameter was close to 0 or 1. We could infer that all loss metrics were helpful, thereby training our model efficiently.

heading: 3.6 Case Studies and Discussions
text: We next conducted case studies to analyze the results of each method qualitatively. As exhibited in Fig.(6), our model could produce multiple commonsense questions. Contrastively, the sequential NQG++ yielded a shallow question that can be answered by directly matching the input text. The pretrained UniLM showed a bit of fluency and graphbased SGGDQ reflected a certain amount of reasoning. Their results were monotonous and cannot yield results in other acceptable expressive ways. HCVAE could produce diverse results which could not match the answers. The reinforced SemQG and dual model DAANet were answer-related, but their results’ deductibility was weak. These results further validated the effectiveness of our model. By analyzing our bad cases, the mistakes mainly came from temporal errors, e.g. “do” should be “did” at “Which country do Bob visit yesterday?” and special symbols errors, e.g. missing “’s.” These challenges would be studied in future work.

heading: 4 Related Works
text: Question Generation (QG) is a hot research topic that can support many valuable applications, including synthesizing training data for the questionanswering (QA) task (Duan et al., 2017), producing exercises on the textbook (Chen et al., 2018), and clarifying users’ needs for a dialog agent (Aliannejadi et al., 2019). Previous studies mainly focus on shallow questions (Wang et al., 2020a). They can be tackled by matching the text without demanding a real understanding of semantics (Yu et al., 2023). The researchers gradually pay attention to deep questions (Hua et al., 2020), such as multi-hop QG (Yu et al., 2020). However, these questions only involve the context that appears in the text without the need of understanding the commonsense knowledge. Asking questions with this background knowledge is indispensable for machine intelligence, but has been less explored. Thus, we propose a new QG task to fill this research gap.
Most of the earlier methods in the QG task were rule-based (Dhole and Manning, 2020). The handcrafted rules were labor-intensive with poor scalability (Zhang et al., 2022). To reduce labor costs, recent attempts turned to a data-driven neural model with better language flexibility (Dou and Peng, 2022). They learned direct mappings from input texts to questions by an encoder-decoder framework (Du et al., 2017). Considering the question would be asked in diverse ways (Shu et al., 2020), it was hard to support one-to-many generation based on a fixed encoded vector (Lachaux et al., 2020). Some studies proposed to enhance the generalization ability (Wang et al., 2021) by variational autoencoder (VAE) (Li et al., 2022a). It can learn an ask-related vector (Li et al., 2022b) which can be resampled to produce multiple questions (Wang et al., 2022) based on data distribution. However, one single vector was not sufficient to capture the complex and entangled asking features (Wang et al., 2020b). In contrast, we consider multiple factors and disentangle them to control the generation finely.
Deep questions require reasoning the knowledge both inside and outside the text (Zhang et al., 2021), including hidden commonsense context (Lv et al., 2020). To capture this context, we can resort to the knowledge graphs (KG) (Zhao et al., 2020) or pre-training models (Chen et al., 2020), such as BERT (Devlin et al., 2019), GPT-2 (Radford et al., 2019), and GPT-3 (Brown et al., 2020). The KG knowledge can be collected by matching (Ye et al.,
2022), and the pre-training one is often obtained by prompt learning (Gao et al., 2021). In addition, the depth of questions is mainly reflected in how to answer (Hu et al., 2017). There are often two ways to incorporate the answering feedback (Liu et al., 2022). One is reinforcement learning which views the answer as a reward (Bao et al., 2018). Since there is no prior guidance, the robustness of this method is weak (Bao et al., 2018). Another way is to use generative adversarial learning (GAN) to jointly train the QA and QG tasks (Sun et al., 2020). This method only judges the final answer but neglects to grasp the answering process, leading to the results’ lack of commonsense reasoning ability (Wu et al., 2022). Also, this discrete judge is non-differentiable (Jin et al., 2020), causing unstable training (Ma et al., 2019a). In contrast, our discriminator simultaneously consider the matched answer and its reasoning complexity, which can facilitate the training of deep question generator.

heading: 5 Conclusions
text: We have proposed a new commonsense reasoning QG task which aimed to generate valid and inferable questions about the given text. Unlike traditional QG tasks, our questions needed to deduce multiple clues in disjoint contexts, where not all clues were provided in the given text, and some required to resort to commonsense knowledge outside the text. Since understanding semantics is the prerequisite to asking high-quality questions, our complex QG task requires a higher level of machine intelligence. Due to the lack of modeling complexity, traditional methods often yield shallow results. To address the problem, we proposed a practical framework that can flexibly incorporate the asking contents, expressive ways, and answering complexity to yield deep results by disentangling adversarial inference. We first retrieved the commonsense knowledge related to the given text. We then disentangled the key question-controlled factors in terms of reasoning content and verbalized way based on the independency priors and constraints. To promote deep results, we further designed a discriminator to regularize the generator by providing the answering feedback. By adversarial inference, we can derive the factors and use them as conditions to decode questions. By sampling the expressive factor from the data distribution, diverse results can be produced. Experimental results on two typical data sets showed the effectiveness of our approach.

heading: Acknowledgments
text: This work is supported by the National Natural Science Foundation of China (62276279,U1911203, 62102463, 62276280, U2001211, U22B2060), Guangdong Basic and Applied Basic Research Foundation (2019B1515130001), Tencent WeChat Rhino-Bird Focused Research Program (WXG-FR2023-06), Natural Science Foundation of Guangdong Province of China (2022A1515011135), Key-Area Research and Development Program of Guangdong Province (2020B0101100001), Zhuhai Industry-University-Research Cooperation Project (2220004002549, ZH22017001210010PWC).
Limitations
Deep questions not only require an in-depth understanding of the semantics in the text, but also involve the formulation of questions with correct grammar, such as tense transformation, and special symbols adjustment. For this task, our model simultaneously capture the key factors on the reasoning content, expressive way, and answering complexity, aiming to make results valid, relevant and inferable. However, as mentioned in the case study section, our model has some bad cases with grammatical flaws. For example, “do” needs to be transformed to “did” when the given text is in the past tense. This requires linguistic knowledge on top of words. Learning to ask with the guidance of this abstract knowledge is not covered in this paper. One way to tackle this problem is to resort to post-processing with a grammar error corrector. In addition, the interpretability of latent variables and the robustness of the model are not explored in this paper. We will investigate them in future works.
Ethics Statement
The technology proposed in this paper can be used in many applications, such as in the fields of education, Q&A, and dialogue systems. For example, it can yield quizzes for exams, or provide reasonable clarification question to warm up the conversation. Unlike shallow matching-based questions, our deep questions require fully understanding the semantics inside and outside the text. That involves many high-level cognitive skills, including reasoning the incomplete contexts with hidden commonsense knowledge. That can better support the real applications such as advanced exams in TOEFL and SAT, since there are few or even no simple
questions. When excluding the misusage scenarios, there are usually no ethical issues with this technology. However, the questions can be generated as long as we input the text. It is possible to input some inappropriate content related to the topics of racial discrimination, war, and so on, resulting in some offensive questions. This problem can be addressed by limiting the topics of input contents.

heading: A Settings of All Evaluated Baselines
text: The pre-trained language model RoBERTa was used to initialize the word embeddings. The distribution overlap metrics (i.e., QAE and R-QAE) were computed by the UNICORN QA model.
Settings of NQG++: The hidden state size of the GRU was set to 512. The lexical and answer position features were embedded to 32-dimensional vectors. The dropout was used with a probability p = 0.5. Stanford CoreNLP v3.7.0 was utilized to annotate POS and NER tags in the sentences. During training, the model was initialized randomly by a Gaussian distribution with the Xavier scheme. A combination of Adam and simple SGD was used as the optimizer. For the Adam optimizer, the learning rate was set to 0.001 with two momentum parameters β1 = 0.9 and β2 = 0.999 respectively. ϵ was set to 10−8. The SGD optimizer was initialized with a learning rate of 0.5 and halved if the BLEU score on the development set drops for twelve consecutive tests. Gradient clipping with range [−5, 5] was utilized for both Adam and SGD phases. To speed up convergence, grid search was employed with the mini-batch size of 64. In the test phase, a beam search was used with a size of 12.
Settings of UniLM: The batch size was 32. The masking probability, learning rate and label smoothing rate were 0.7, 2e−5 and 0.1, respectively.
Settings of SGGDQ: It adopted a 1-layer GRU with hidden units of 512 dimensions. For the graph encoder, the node embedding size was set to 256, plus the POS and answer tag embeddings with 32 dimensions for each. The number of layers was set to 3 and the hidden state size was 256. Adam was
employed with a mini-batch size 32. The learning rate was initially set to 0.001, and adaptive learning rate decay was applied. Early stopping was utilized with a dropout rate of 0.3 for both the encoder and decoder and 0.1 for all attention mechanisms.
Settings of HCVAE: The hidden dimension of the Bi-LSTM was set to 300 for posterior and prior generation networks. The dimension of the encoder and the decoder was set to 450 and 900, respectively. The dimension of latent variable zx was set as 50, and zy was defined to be a 10-way categorical variable. The QA model was fine-tuned for 2 epochs. Adam optimizer was used with a batch size of 32 and the initial learning rate of 5 · 10−5 and 10−3 respectively. To prevent posterior collapse, the model multiplied 0.1 to the KL divergence terms of question and answer.
Settings of DAANet: The parameters were randomly initialized by the fan-avg strategy. Dropout was mainly applied to the encoding layer with a keep rate of 0.9. The coverage loss weight κ was 1.0. The gradient was clipped by restricting its ℓ2 − norm less than or equal to 5.0. Adam optimizer was adopted with a batch size of 16. The learning rate was increased from zero to 0.001 with an inverse exponential function and then fixed for the remainder of the training. During testing, autoregressive decoding was conducted separately for QA and QG. Decoding is terminated when the model encountered the first <END> or when the sequence contained more than 100 words.
Settings of SemQG: The WordPiece tokenizer was used to tokenize each word and extend the POS / NER tags to each word piece. A 2-layer LSTM-RNNs was employed for both the encoder and decoder with a hidden size of 600. Dropout with a probability of 0.3 was applied to the input of each LSTM-RNN layer. Adam was utilized as the optimizer with a learning rate of 0.001 for teacher forcing and 0.00001 for reinforcement learning. The batch size was set to 32. For stability, It was first pre-trained with teacher forcing until convergence, then fine-tuned with the mixed loss. Hyperparameters were tuned on the development set with γqpp = 0.99, γqap = 0.97, and n : m = 3 : 1. The beam search was employed with the size of 10 for decoding. The bigram and trigram repetition penalty was applied.

heading: B Human Evaluation Settings
text: The rated guideline was shown in Fig.(7).
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Limitations

heading: 3 A2. Did you discuss any potential risks of your work?
text: Ethics Statement

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Abstract and Section 1 Introduction
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 7 Did you use or create scientific artifacts? Left blank.
7 B1. Did you cite the creators of artifacts you used? Left blank.
7 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Left blank.
7 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Left blank.
7 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Left blank.
7 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Left blank.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Left blank.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
C 3 Did you run computational experiments?

heading: 3.1 Data and Experimental Settings

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? 3.1 Data and Experimental Settings

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? 3.1 Data and Experimental Settings

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? 3.2 Comparisons against State of the Arts

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Appendix A Settings of All Evaluated Baselines
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants?

heading: 3.4 Human Evaluations and Analysis

heading: 3 D1. Did you report the full text of instructions given to participants, including e.g., screenshots,
text: disclaimers of any risks to participants or annotators, etc.? Appendix B Human Evaluation Settings

heading: 3 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students)
text: and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Appendix B Human Evaluation Settings
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Not applicable. Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not applicable. Left blank.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Not applicable. Left blank.

heading: Future Work
text: We will investigate them in future works