Elaboration-Generating Commonsense Question Answering at Scale
Year: 2023

Abstract:
In question answering requiring common sense, language models (e.g., GPT-3) have been used to generate text expressing background knowledge that helps improve performance. Yet the cost of working with such models is very high; in this work, we finetune smaller language models to generate useful intermediate context, referred to here as elaborations. Our framework alternates between updating two language models—an elaboration generator and an answer predictor—allowing each to influence the other. Using less than 0.5% of the parameters of GPT-3, our model outperforms alternatives with similar sizes and closes the gap with GPT3 on four commonsense question answering benchmarks. Human evaluations show that the quality of the generated elaborations is high.1

heading: 1 Introduction
text: Commonsense question answering (QA; Talmor et al., 2019) provides benchmarks used to evaluate the extent to which NLP models—increasingly based on language models—can “understand” questions and reason about their answers. For example, consider the question in Figure 1: Gases released during the use of fossil fuels cause a what? A reasonably informed human could give the answer global warming, by reasoning that: Fossil fuel emissions are the main source of greenhouse gases. They cause global warming.
It is common to use LMs to predict answers directly for QA tasks (Devlin et al., 2019; Liu et al., 2019; Khashabi et al., 2020). On challenging datasets whose questions rely on unstated background knowledge (Talmor et al., 2021; Mihaylov et al., 2018; Khot et al., 2020), some recent works rely on external knowledge, e.g., Wikipedia or structured knowledge bases (Mihaylov and Frank,
1The source code is available at https://github. com/happywwy/Elabor/tree/main.
2018; Lin et al., 2019; Banerjee et al., 2019) for additional information that helps to answer the question. Such attempts are limited by the availability and coverage of the knowledge sources. Another line of study (Liu et al., 2022b; Paranjape et al., 2021; Shwartz et al., 2020) reveals that generating text that expresses additional background knowledge relevant to a question is beneficial for answer prediction. The ability to express such knowledge may promote model explainability by explicitly showing the reasoning process. However, expressing high-quality knowledge relies on massive (and thus, expensive) pretrained LMs, e.g., GPT-3 with 175B parameters (Brown et al., 2020).
In this work, we focus on a more practical setting and ask: Can smaller LMs, e.g., BART which is about 400× smaller than GPT-3, support reasoning and inference in an end-to-end manner? To this end, we propose a scalable framework, alternating ELABoration and answer predictOR (ELABOR), consisting of two interacting modules: an elaboration generator and an answer predictor. Here an elaboration refers to additional context describing some background knowledge about the question. Instead of generating elaborations independently, we propose a probabilistic framework that treats the elaboration as a latent variable and iteratively optimizes the elaboration generator after receiving feedback from the answer prediction. Specifically, for each question-answer pair (q, a), we decompose the distribution of the answer conditioned on the
1619
question P (a | q) into a distribution P (e | q) over a latent elaboration, modeled by the elaboration generator, and a likelihood distribution P (a | e, q) over the answer, modeled by the answer predictor. We alternately train the elaboration generator and the answer predictor so that each can benefit the other. Earlier work either pre-constructs elaborations e from external knowledge (Mihaylov and Frank, 2018) or learns P (e | q) solely based on annotations (Rajani et al., 2019); we learn the elaboration generator by distilling high-quality knowledge from GPT-3. We do this using a procedure inspired by hard Expectation-Maximization (Min et al., 2019). This involves refining and filtering elaborations informed by the answer predictor, as shown in Figure 1. ELABOR is thus capable of propagating information in both directions: from elaboration generator to answer predictor and vice versa.
We conduct experiments on four commonsense QA datasets: CommonsenseQA (Talmor et al., 2019), CommonsenseQA 2.0 (Talmor et al., 2021), Scientific Commonsense (Khot et al., 2020), and OpenBookQA (Mihaylov et al., 2018). Our experiments reveal that (1) alternating training with smaller LMs (e.g., BART, and GPT-2) narrows the gap between small models and GPT-3; (2) the ability to generate and reason with background elaborations indeed brings larger performance gains than direct inference on more challenging Commonsense QA datasets; (3) the alternating framework helps to filter irrelevant elaborations generated from GPT-3 and the learned elaboration generator can express information that helps to answer the question, as shown through human evaluations.

heading: 2 Modeling Answers and Elaborations
text: We focus on the task of commonsense question answering in the multiple-choice setting: we seek to identify the answer to a commonsense question among provided candidate choices. Importantly, we are not provided with additional elaboration that may be needed to do so. We formalize the setting and define the model in this section, and Section 3 details the training procedure.

heading: 2.1 Elaborations as a Latent Variable
text: We formalize commonsense QA in a probabilistic framework. Given a question q and its correct answer a, we seek to train a model that maximizes the probability of the correct answer P (a | q). Directly
predicting the answer can be be challenging when complex understanding is needed. Moreover, doing so renders the provenance of the answer unclear. To address both issues, we assume that the answer depends on some latent elaboration e ∈ E with E denoting a set of probable elaborations. With the latent variable, the training objective becomes
logP (a | q) = log ∑
e∈E P (e | q)P (a | e, q). (1)
Here, the first term in the summation, P (e | q), denotes the probability of an elaboration e conditioned on question q and is captured by the elaboration generator. The second term P (a | e, q) characterizes the distribution of the answer a conditioned on both the elaboration and the question and is captured by the answer predictor. The decomposition in Eq. 1 has also been adopted by Lewis et al. (2020b), taking retrieved knowledge as the hidden variable. Different from the retrieval setting, the generation distribution P (e | q) is intractable. We instead resort to hard EM and alternating optimization.

heading: 2.2 A Joint Model
text: The elaboration generator seeks to generate an elaboration sequence e given the question q as a prompt. We denote the conditional probability of an elaboration given a question by FE ; that is, using the notation from Eq. 1, we have P (e | q) = FE(e, q; Φ). We model the elaboration generator using a generative language model that computes the distribution of tokens at each generation step:
FE(e, q; Φ) = m∏
t=1
pGEN(et | q, e1, ..., et−1), (2)
where e = {e1, ..., em} denotes the generated elaboration sequence. In our experiment, we adopt two generation models—BART (Lewis et al., 2020a) and GPT-2 (Radford et al., 2019)—to model pGEN.
The answer predictor, denoted FA, aims to produce the probability of an answer sequence a given a question q and an elaboration e, i.e., P (a | e, q) = FA(a, e, q; Θ). Any language model could be adopted as the answer predictor. For generality, we select two commonly-used language models from two different paradigms, namely BERT (Devlin et al., 2019) as a masked language model and T5 (Raffel et al., 2020) as a generative language model. For T5, FA(a, e, q; Θ) is computed
for an answer sequence a = {a1, ..., an} using
FA(a, e, q; Θ) = n∏
t=1
pT5(at | e, q, a1, ..., at−1),
(3) with pT5 denoting the generation probability of token at using T5. For BERT, FA(a, e, q; Θ) is computed using a softmaxed linear layer over the representation of the [CLS] token:
FA(a, e, q; Θ) = softmax(Wh[CLS] + b) (4)
by giving “[CLS] elaboration [SEP] question [SEP] answer [SEP]” to BERT.

heading: 2.3 Inference
text: In the testing phase, for each question, we first use the trained elaboration generator FE to sample a set of elaborations Ẽ . For each ẽ ∈ Ẽ , we use the answer predictor FA with softmax to produce a normalized distribution over the candidate set. By running the answer predictor for each sampled elaboration, we take the maximum probability as the score for candidate ai which is then used to produce the final prediction:
a′ = argmaxai∈Amax ẽ∈Ẽ
expFA(a i,ẽ,q;Θ)
∑ aj∈A exp
FA(aj ,ẽ,q;Θ) (5)
with A denoting the set of candidate answers.

heading: 3 Alternating Elaboration and Answer Predictor (ELABOR)
text: Many existing retrieval or knowledge-based QA methods only optimize P (a | e, q), assuming e is given and fixed. Explanation-based methods, on the other hand, train P (e | q) separately using human-annotated explanations. Doing so poses two problems: (1) we need an annotated explanation corpus, and (2) the elaboration generator cannot be calibrated towards the answer.
In this work, we propose an approach that tackles both problems by jointly training the elaboration generator and the answer predictor in an alternating framework. Figure 2 illustrates the overall architecture for training. In each iteration, the elaboration generator FE learns to produce high-quality elaborations using feedback from the answer predictor (Section 3.1). The answer predictor FA then takes the generated elaborations as input to produce more reliable answers (Section 3.2). This strategy allows mutual interaction between the two components, propagating information in both directions.
To reduce the search space of possible elaborations, we propose to distill knowledge from the pretrained GPT-3 model in a selective way to learn a lightweight elaboration generator (Section 3.3).

heading: 3.1 An EM-Inspired Learner
text: Our goal is to optimize Eq. 1, rewritten below:
logP (a | q) = logEe∼P (e|q)[P (a | e, q)]. (6)
Directly optimizing the elaboration generator in this expression is difficult.2 Inspired by Qu et al. (2021), we adopt a hard EM framework to do so. The E-step first generates a set of elaborations related to the question and then selects “good” elaborations that help to predict the correct answer. The M-step maximizes the probability of generating these “good” elaborations.
E-Step. The E-step aims to identify a set of “good” elaborations from the posterior probability of an elaboration e after observing the correct answer a:
P (e | q, a) ∝ P (e | q)P (a | e, q) (7)
The posterior approximation on the right-hand-side of Eq. 7 aligns with the intuition that the elaboration could have higher probability if it is both relevant to the question (i.e., P (e | q)) and, when combined with the question, provides higher chance of predicting the correct answer (i.e., P (a | e, q)).
However, the intractable space of possible elaborations renders sampling from P (e | q)P (a | e, q)
2One popular option would be to adopt the REINFORCE algorithm (Williams, 1992) that updates FE(e, q; Φ) using differentiable policy gradient. However, this strategy involves searching in a huge symbolic space and can be unstable.
nontrivial. To alleviate this issue, we adopt two approximations. First, we use GPT-3 to produce more reliable distribution P (e | q), and thus rewriting Eq. 7 as P (e | q, a) ∝ PGPT-3(e | q)P (a | e, q). Second, we approximate the sampling process via a two-step sample-and-filter procedure. Specifically, we first sample a set of elaborations Ē from PGPT-3(e | q) which will be discussed in Section 3.3. Then, we filter Ē according to P (a | e, q). Specifically, for each ē ∈ Ē , we use the answer predictor3 to produce P (a | ē, q) = FA(a, ē, q). Then we select top-K elaborations from Ē to form E as the set of “good” elaborations. This operation allows the answer predictor to assist in learning how to select elaborations.
M-Step. With the selected context set E produced in the E-step, the M-step aims to maximize the probability of each elaboration e ∈ E to update the elaboration generator FE while keeping the answer predictor fixed:
max Φ logP (E | q) = max Φ
∑ e∈E logFE(e, q; Φ),
(8) given P (E | q) = ∏e∈E P (e|q). In this way, the elaboration generator learns to produce elaborations that are both relevant to the question and with a higher probability of predicting the correct answer. Eq. 8 could also be viewed as a kind of selective distillation, which instead of distilling all the sampled elaborations Ē from GPT-3, learns to filter out noisy elaborations before transferring knowledge to the elaboration generator.

heading: 3.2 Optimizing Answer Predictor
text: After updating the elaboration generator, the next step of the alternative training aims to update the answer predictor FA(a, e, q; Θ) while keeping the elaboration generator fixed. To achieve that, we approximate the objective of Eq. 6 to logP (a | ẽ, q) by sampling a set of elaborations ẽ ∈ Ẽ from the elaboration generator P (ẽ | q) = FE(ẽ, q; Φ). Then the objective becomes to maximize
logP (a | ẽ, q) = logFA(a, ẽ, q; Θ) (9)
for the correct answer a. The sampled elaboration ẽ from the elaboration generator acts as additional background and explanation for the question, which helps to learn a more reliable prediction
3We also study other filtering strategies as detailed in Section 4.4.
model to answer the question. The alternation between updating the answer predictor and the elaboration generator promotes mutual enhancement of each component. The entire training procedure of ELABOR can be found in Appendix A.1.

heading: 3.3 Distilling GPT-3
text: As discussed in the E-step, we use GPT-34 to sample possible elaborations to train our elaboration generator. Liu et al. (2022b) showed that, using a small number of prompts and a question, GPT-3 can generate useful knowledge to enhance answer prediction. Inspired by Hinton et al. (2015) and West et al. (2021), we adopt the idea of knowledge distillation to transfer knowledge from GPT3 (expensive to deploy at inference time) to our (cheaper) elaboration generator. We first use GPT-3 to generate a set of elaborations given some predefined prompts. Following Liu et al. (2022b), for each task, we design the prompt as a short instruction followed by five demonstrative examples and a new-question placeholder. By plugging each question into the placeholder, we can repeatedly sample an elaboration ē as the continuation of the prompt. This yields a set of candidate elaborations, Ē .
Here we use nucleus sampling (Holtzman et al., 2020) to sample each elaboration ē. For knowledge distillation, a naive strategy could be optimizing the elaboration generator by minimizing
D(PGPT-3, Ps) = Eē∼PGPT-3 [− logPs(ē | q)], with Ps denoting the student network, i.e., our elaboration generator. However, as shown in the experiments, GPT-3 is prone to generating noisy text sequences that may not be relevant to answer the question. This would lead to negative transfer. Our proposal in the E-step is a form of selective knowledge distillation (Kang et al., 2020) which filters elaborations generated from GPT-3 according to the answer score before optimizing our student model.

heading: 4 Experiments
text: In this section, we examine the question: Does jointly optimizing the elaboration generator with the answer predictor outperform approaches that merely retrieve knowledge from trained models, if at all? As a secondary objective, we also investigate the impact of the design choices in our approach, including the choice of the language model,
4We also tried more accessible models, e.g., GPT-J (6B), but observed much worse generation quality.
the need for distillation, the choice of elaboration filtering and the decoding strategy.

heading: 4.1 Data and Setup
text: We select four multiple-choice commonsense QA datasets involving commonsense concepts or scientific facts: (1) CommonsenseQA (CSQA; Talmor et al., 2019), (2) CommonsenseQA 2.0 (CSQA2,Talmor et al., 2021) (3) Scientific Commonsense (QASC, Khot et al., 2020), and (4) OpenBookQA (OBQA; Mihaylov et al., 2018). The elaboration generator is implemented using GPT2large (Radford et al., 2019) and BART-large (Lewis et al., 2020a). The answer predictor is implemented using T5-large (Raffel et al., 2020) and BERT-baseuncased (Devlin et al., 2019). We also experiment with more competitive and larger answer predictors, e.g., UnifiedQA-large/3b (Khashabi et al., 2020). We sample 20 elaborations from GPT-3, of which 3 are selected to form E . We sample 10 elaborations from our elaboration generator during both training and inference. Appendix A.2 has more details on the datasets and experiment settings.

heading: 4.2 Baselines
text: We organize the baselines into four groups: (1) Direct answer prediction without additional knowledge (vanilla). (2) Answer prediction with retrieved knowledge: COMET (Bosselut et al., 2019) is trained on the ATOMIC corpus (Sap et al., 2019) to automatically generate causes and effects of a question. Wikipedia follows Chen et al. (2017), which retrieves and ranks text spans in Wikipedia articles. (3) Fixed elaboration generator: selftalk
generates extra background knowledge based on some clarification questions (Shwartz et al., 2020). GPT-3 (Brown et al., 2020) samples 10 knowledge spans as continuations of the question using some demonstrative prompts. (4) Trained elaboration generator: scratch implements alternative training without distilling knowledge from GPT-3. pipeline first pretrains the generator using all the sequences generated from GPT-3, then finetunes the answer predictor. For fair comparisons, all four groups require training the answer predictor FA. The second and third groups additionally involve intermediate contexts which are kept fixed. The last group learns both an elaboration generator and an answer predictor. During inference, we pick the choice with maximum score across all the knowledge sequences or generations following Eq. 5.

heading: 4.3 Results
text: Table 1 shows the main experimental results. Here we use T5-large as the answer predictor for CSQA, CSQA2, QASC, and BERT for OBQA. These are chosen according to the best performances given. To account for more general scenarios, we first use T5 in an open-domain QA setting where no answer choices are given as input, and the target output is the gold answer tokens. We also experiment with other input/output formats for T5 as will be shown in Section 4.4. From Table 1, the advantage of additional knowledge or elaborations is more evident for CSQA2, QASC, and OBQA, compared with CSQA (which contains relatively simpler questions). This confirms the importance of reasoning for complex QA problems. GPT-3 demonstrates performance gains over other knowledge sources. Using less than 5% of the parameters of GPT-3, ELABOR outperforms GPT-3 on two datasets. It also clearly outperforms those models having similar computational cost (e.g., scratch, pipeline). The performance gain of ELABOR over pipeline demonstrates the advantage of our alternating framework. The scratch model on the other hand is prone to learning meaningless shortcuts, e.g., “The correct answer: I know I’m not sure but
whatever.”

heading: 4.4 Analysis
text: In subsequent experiments, we use the development set of each corpus to make evaluations because the test set is not publicly available. Elaboration Generator. Table 2 shows the effects of different LMs, specifically BART-large and GPT2-large, as elaboration generators. Both demonstrate consistent results across different training strategies (scratch, pipeline, ELABOR). In addition, GPT2-large slightly outperforms BART-large across all the experiments. The higher performance of GPT2-large could be credited to a larger parameter size (774M) compared to BART-large (406M). Another observation is that GPT2-large has more generation flexibility which appears to be less repetitive and cover more aspects relevant to the question, compared to BART-large. Answer Predictor. Table 3 reveals the effect of our framework on more competitive settings and larger answer predictors. We consider another input/output format for T5, referred to as T5-id, which takes both IDs (we use (A), (B), etc. as answer IDs) and tokens of the answer choices as input, and the ID for the gold answer as output. This was adopted in GenMC (Huang et al., 2022). Obviously, T5-id outperforms T5 under the open-domain setting (Table 1) by a large margin, and ELABOR shows clear gains over GenMC. A larger model, UnifiedQA-3b, brings huge improvements even for the vanilla model. Still, additional elaborations (GPT-3 or ELABOR) bring further improvements across all the datasets.
Elaboration Filtering. The first block (Elaboration filtering) of Table 4 shows the effect of different filtering criteria as discussed in the E-step of Section 3.1. We implement three other filtering strategies. The random option filters GPT3-
generated elaborations by randomly selecting 3 out of 20. The correct option selects all the elaborations that produce the correct answer when fed into the answer predictor. The pos-neg option computes the score difference between the correct answer and the average of incorrect answers, based on which 3 elaborations with highest scores are being selected. The pos option uses the answer predictor as adopted by ELABOR. Clearly, random selection produces inferior results among all the options, verifying the benefit of filtering high-quality elaborations for training the elaboration generator.
Elaboration Integration. The second block (Elaboration integration) of Table 4 investigates the effect of different elaboration integration methods during inference. Recall from Eq. 5 that ELABOR uses maximum pooling among all the generated elaborations Ẽ for final predictions. We are interested in how different inference strategies may affect the final performance. Specifically, instead of maximum pooling, we concatenate all the elaborations in Ẽ in a single sequence and feed it into the answer predictor (concatenate). This brings a clear performance drop on CSQA and QASC, probably due to the unexpected noise and the forgetting issue for long sequences. Another strategy is to formalize inference with a probabilistic view where each generated elaboration has a probability contributing to the final prediction via weighted aggregation (probability). To produce the probability, we apply a softmax layer on top of the output logit of each generated elaboration ẽ ∈ Ẽ . The last option is to compute the similarity between each elaboration and the question and use the most similar elaboration for final inference (similarity). We use sentence embeddings generated from sentence transformers (Reimers and Gurevych, 2019) with cosine similarity to select the optimal elaboration. As a result, maximum pooling outperforms other variations at most of the times.
Decoding Strategy. The last block (Elaboration generation) of Table 4 reflects how different decoding strategies inherent in the LMs may affect the final performance. We compare the results of greedy decoding (greedy) where each decoding step only selects the token with highest probability, beam search (beam) with size 10 at each decoding step and selecting top 10 sequences via nucleus sampling (sample) adopted in the proposed model ELABOR. Clearly, decoding via sampling produces the best results or comes very close.
Sensitivity Test. Figure 3 demonstrates the effects of changing (1) the number of filtered high-quality elaborations (K) from GPT-3 and (2) the size of set Ẽ corresponding to the total number of elaborations generated from the elaboration generator. The left plot demonstrates the performance increases when increasing K from 1 to 3, but decreases for K > 3. This pattern verifies that GPT-3 may generate elaborations that negatively affect the final performance. On the other hand, increasing the number of sampled elaborations from the elaboration generator (from 2 to 20) during both training and testing phases brings gradual improvements. This is as expected, given that sampling a diverse set of elaborations should add up to a wide coverage of relevant knowledge for the question.

heading: 4.5 Human Evaluation
text: To evaluate the quality of elaborations for question answering, we conduct two sets of human evaluations on QASC and CSQA2. For the first experiment, we investigate whether the filtered elaborations from GPT-3 are considered more helpful to answer the question compared to those that are not selected by the model. For the second experiment, we evaluate the quality of the generated elaborations. Some concrete examples of questions and generations can be found in Appendix A.3. The annotation task was carried out in Amazon Mechanical Turk. We restrict annotators to those located in English-speaking countries and who have at least 99% approval rate over more than 1000 tasks. The results are aggregated using majority vote among annotations from 3 workers. Our institution’s IRB approved the study. We paid workers an estimated US$15 per hour. Effect of Filtering. Recall that we use the answer predictor to filter elaborations generated from GPT-3 in the E-step. To demonstrate whether the filtering process is capable of removing noisy elaborations, we randomly sample 100 questions from
the training corpus of each of two datasets (QASC, CSQA2). For each instance, we present the crowd workers with a question, the correct answer, the GPT3-generated elaboration e that has the highest score P (a | e, q) (denoted SELECT), and an elaboration randomly sampled from the remaining ones that are discarded by the answer predictor (denoted DISCARD). The workers are then asked to evaluate the SELECT and DISCARD elaborations by choosing 1-out-of-3 choices: helpful (the elaboration adds useful information to answer the question), neutral (the elaboration has no influence on the problem), and harmful (the elaboration is misleading). To avoid annotation bias, we randomize the order of SELECT and DISCARD elaborations for each example. The results are shown in Figure 4. Among 100 examples for each dataset, the number of helpful elaborations annotated by the workers is considerably higher for the selected category than that of the discarded category. In contrast, the workers agree that the selected elaborations are less likely to be neutral or harmful compared to those that are discarded. The difference is even more evident on CSQA2. This verifies the necessity of using the answer predictor to filter noisy elaborations generated by GPT-3 before distilling the knowledge.
Elaboration Quality. In another experiment, we compare the quality of the elaboration generators from the pipeline setup, GPT-3 and our proposed model ELABOR. We select only one elaboration generated from each model that gives the highest score of the predicted answer during inference, which is actually adopted to produce the final prediction. Adapting from the metrics provided by Shwartz et al. (2020) and Liu et al. (2022b), given a piece of automatically-generated text, we pick three aspects: (1) Factuality evaluates whether the text is entirely correct (factual), partially correct (partial) or entirely incorrect (incorrect); (2) Rel-
evance evaluates whether the text is relevant or irrelevant to the topics discussed in the question; (3) Helpfulness evaluates whether the text provides useful information that helps answer the question (helpful), has no effect (neutral) or is misleading (harmful). The human evaluation results on 100 randomly sampled test examples from CSQA2 are shown in Figure 5. Clearly, ELABOR achieves better scores across all the three aspects, with the most evident improvement in terms of helpfulness. We additionally evaluate how humans benefit from those elaborations generated from our model. The detailed analysis is presented in Appendix A.4. Further analysis on how in general the generations from ELABOR and GPT-3 differ is shown in Appendix A.5.
Based on the annotations given by crowdsourced workers, we collect only those instances containing an elaboration generated by our model that is labeled as helpful by the workers. This results in 70 and 76 instances from the development set of QASC and CSQA2, respectively. We then compare the performance of ELABOR under three different settings: (1) No Elaboration only presents the question to the model during inference; (2) Random Elaboration additionally provides a generated elaboration randomly selected after removing the one labeled as helpful; (3) Helpful Elaboration contains the single elaboration that is labeled as helpful by workers. The results are shown in Table 5. As expected, our model with helpful elaborations outperforms the other two settings by a large
margin, aligning with our intuition that meaningful elaborations are beneficial to the task.

heading: 5 Related Work
text: Direct Inference. Given only natural-language commonsense questions, a straightforward solution is to directly use language models, either finetuned from the gold-annotated answers (Sakaguchi et al., 2021; Talmor et al., 2019; Khashabi et al., 2020; Talmor et al., 2021) or in an unsupervised setting (Trinh and Le, 2018; Petroni et al., 2019; Puri and Catanzaro, 2019; Yang et al., 2020; Jiang et al., 2020) that exploit knowledge already encoded in the pretrained parameters to perform inference. However, beyond the performance score, it is unclear how these models reach the final answer and whether they perform correct reasoning. It is also challenging to conduct direct inference without additional knowledge for complex queries.
Inference with External Knowledge. It has been shown that external knowledge such as knowledge bases or Wikipedia contains rich information that could assist inference. Knowledge bases, e.g., ConceptNet (Speer et al., 2017) or ATOMIC (Sap et al., 2019), contain relational knowledge that could be incorporated as additional inputs for commonsense QA (Mitra et al., 2019; Chang et al., 2020; Bian et al., 2021; Ma et al., 2021; Lv et al., 2020; Yasunaga et al., 2021). Large corpora are another knowledge source to retrieve question-related facts (Lin et al., 2017; Tandon et al., 2018; Banerjee et al., 2019; Joshi et al., 2020; Xiong et al., 2019; Lewis et al., 2020b). These knowledge-based approaches depend on the availability and coverage of the knowledge source, which usually depends on the problem domain.
Inference with Generation. To alleviate the dependence on external knowledge, recent trends advocate for automatic generation of additional knowledge related to the question via language models. One direction is to learn a generator to generate meaningful justifications for question answering via human-authored explanations (Camburu et al., 2018; Rajani et al., 2019; Latcinnik and Berant, 2020). Bosselut et al. (2021) adopted a pretrained commonsense generation model (Bosselut et al., 2019) to generate implications of the questions. These approaches, however, require goldannotated commonsense facts to train a good generator. Another direction explores zero-shot generations using pretrained language models. Shwartz
et al. (2020) introduced Selftalk, which elicits question clarifications using a few pre-defined templates. Paranjape et al. (2021) proposed contrastive prompts that compare candidate options for choosing the correct answer. Liu et al. (2022b) generated additional texts as continuations of each question by feeding demonstrative prompts to GPT-3. Another work (Liu et al., 2022a) used reinforcement learning to guide meaningful generations. Huang et al. (2022) recently proposed to generate clues, which are short phrases or single tokens similar to the gold answers, before answering the question. Different from existing approaches, we seek to learn an effective generation model jointly with the answer prediction to allow for mutual enhancement.

heading: 6 Conclusion
text: We propose a framework for commonsense QA problems that alternates between learning a meaningful, relatively lightweight elaboration generator and producing an answer from the question and automatically generated elaboration. These two steps are trained interactively, propagating signals to each other. We narrow the performance gap between small LMs and GPT-3, with the elaboration generator producing elaborations judged useful by humans, and matching the performance of the much more expensive GPT-3 model as an elaboration generator. One limitation of ELABOR is lack of exploration beyond GPT-3. We consider investigating this problem as our future work.
Limitations
Given the ability of ELABOR to generate free-text elaborations for commonsense question answering, we still observe some cases where the modelgenerated elaborations are not factually correct, or irrelevant to the question, distracting the answer predictor towards incorrect answers. This reflects a limitation of ELABOR on the controllability of its generations, which is also commonly discovered when using language models for text generation. We consider this as a possible future direction which aims at verifying the factuality and relevancy of model-generated texts before incorporating them for final inference or as a controlling mechanism during generation.
Ethics & Broader Impact
In this work, we only experiment with publicly available datasets. For human evaluation, we do not have access to or collect any personal information from our crowd-sourced workers, except that we only restrict participants to be located in English-speaking countries and have higher qualifications in terms of approval rate. As we work on language model generations, it is possible that the model could produce unintended toxic contents that impede its safe deployment (Gehman et al., 2020). We do not address this issue here but leave it to the field of controlled generation and language detoxicity.

heading: Acknowledgments
text: The authors appreciate helpful feedback from the anonymous reviewers. We thank Jiacheng Liu for helpful discussions, and the members of H2lab and ARK lab for their constructive feedback. This work was funded in part by the DARPA MCS program through NIWC Pacific (N66001-19-2- 4031), NSF IIS-2044660 and NSF III-2007398. It was also supported by International Postdoctoral Fellowship, Nanyang Technological University.

heading: A Appendix
text: A.1 Algorithm
The overall algorithm for training ELABOR is shown in Algorithm 1.
Algorithm 1 Training procedure of ELABOR. 1: Initialize: For each question q, use GPT-3 to sample a set
of knowledge Ē as continuations of q (Section 3.3). 2: for epoch= 1, ..., T do 3: for batch= 1, ..., N do 4: Optimize Eq. 6 by alternating between A and B: 5: A. Optimize elaboration generator FE to produce P (e|q) (Section 3.1) 6: for a question-answer pair (q, a) in batch do 7: E-Step: Select top-K elaborations E =
{e1, ..., eK} ⊆ Ē given scores produced from the answer predictor.
8: M-Step: Update the elaboration generator FE using Eq. 8 with E and q.
9: end for 10: B. Optimize answer predictor FA to produce P (a | e, q) (Section 3.2) 11: for a question-answer pair (q, a) in batch do 12: Sample a set of candidate elaborations Ẽ using FE trained in the previous step. 13: For each ẽ ∈ Ẽ , update the answer predictor FA by maximizing Eq. 9 given a and ẽ. 14: end for 15: end for 16: end for
A.2 Data & Experimental Setup
(1) CommonsenseQA (CSQA; Talmor et al., 2019) is created based on commonsense knowledge from various concepts in ConceptNet. Most of the questions require implicit background knowledge that is trivial to humans. The dataset consists of 12,247 examples (80%/10%/10% train/dev./test split), each of which is a 5-way multiple-choice selection problem. (2) CommonsenseQA 2.0 (CSQA2; Talmor et al., 2021) is a more challenging dataset collected
in an adversarial manner where a user is encouraged to create questions for which a well-trained ROBERTA model (Liu et al., 2019) fails to provide the correct answer. The dataset contains a total of 14,343 questions (9,282 train, 2,544 dev., 2,517 test) with binary answer choices (yes/no). (3) QASC (Khot et al., 2020) is a question answering dataset requiring compositions of multiple pieces of texts. It is collected from elementary and middleschool science questions. The dataset contains 9,980 questions (8,134 train, 926 dev., 920 test), each of which is followed by 8 different choices. Note that we do not use the gold-annotated background facts accompanied with the original data, in order to test the model’s ability to automatically elicit knowledge and reason. (4) OpenBookQA (OBQA; Mihaylov et al., 2018) is a collection of open book exams on elementary-level science facts. It contains a total of 5,957 questions (4,957 train, 500 dev., 500 test) with four candidate choices for each question. Similar to QASC, we also remove the gold-annotated science facts in the original release.
For experimental setup, we use GPT-3 (Brown et al., 2020) under few-shot prompting and with nucleus sampling p = 0.5 (Holtzman et al., 2020) to sample 20 elaborations for each question. We use the same prompts as those from Liu et al. (2022b) and provide them in Table 7. During alternative training, for each iteration, we use 100 instances to update the elaboration generator followed by the answer predictor. We adopt Adam optimizer with learning rate initialized at 10−5 for both components. The elaboration generator generates |Ẽ | = 10 elaborations during both training
and testing phases via nucleus sampling p = 0.95 and with temperature set as 0.7. We set K = 3 when forming the top-K elaboration set Ē during the E-step. For elaboration generation, GPT2large and BART-large has 774M and 406M parameters, respectively. For answer prediction, we use T5 with varying model sizes: 770M for T5large/UnifiedQA-large and 3B for UnifiedQA-3b.
A.3 Generations from ELABOR
We list some actual generations from ELABOR using the learned elaboration generator GPT2-large in Table 6. These examples are selected from those used for human evaluations. The listed elaboration for each question is the most confident elaboration that is used for final prediction.
A.4 Human Evaluation
We additionally evaluate how humans benefit from those elaborations generated from our model across 100 random-sampled development examples from QASC. For each example, we first present the workers with the question and ask them to choose only one answer from multiple choices. In another round, we provide both the question and the generated elaboration to the workers and collect their answers. The two rounds of experiments recruit non-overlapping annotators to ensure validity. As a result, 78 questions are correctly answered by workers without seeing extra elaborations. On the other hand, 81 questions are correctly answered when elaborations are provided. This shows our elaboration generator is still beneficial to humans even though commonsense QA appears to be much easier for humans than machines.
A.5 ELABOR vs. GPT-3 We select 50 examples from those used for human evaluation, half of which are correctly predicted by ELABOR but wrongly predicted by GPT-3 (denoted as D1). In the remaining 25 cases, the situation is the opposite (denoted as D2). Through manual inspection, we observe that in D1, ELABOR is often better off when the question is more general, e.g., “What is a simple mode of transportation?”. ELABOR can generate more specific information relevant to some answer choices and tends to speak more. For D2, ELABOR performs worse when the model overgenerates noisy information not related to the question context leading to wrong answers. For example, the question “What do choanocytes have to trap the particles?” causes ELABOR to generate “The particle is a virus. The choanocytes are part of the immune system. The antibodies that bind the virus and destroy it.” which does not answer the question.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: A1. Did you describe the limitations of your work?
Left blank.
A2. Did you discuss any potential risks of your work? Left blank.
A3. Do the abstract and introduction summarize the paper’s main claims? Left blank.
A4. Have you used AI writing assistants when working on this paper? Left blank.

heading: B Did you use or create scientific artifacts?
text: Left blank.
B1. Did you cite the creators of artifacts you used? Left blank.
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Left blank.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Left blank.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Left blank.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Left blank.
B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Left blank.

heading: C Did you run computational experiments?
text: Left blank.
C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Left blank.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Left blank.
C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Left blank.
C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Left blank.

heading: D Did you use human annotators (e.g., crowdworkers) or research with human participants?
text: Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Left blank.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Left blank.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Left blank.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Left blank.

heading: Future Work
text: We consider investigating this problem as our future work. Limitations
Given the ability of ELABOR to generate free-text elaborations for commonsense question answering, we still observe some cases where the modelgenerated elaborations are not factually correct, or irrelevant to the question, distracting the answer predictor towards incorrect answers. This reflects a limitation of ELABOR on the controllability of its generations, which is also commonly discovered when using language models for text generation. We consider this as a possible future direction which aims at verifying the factuality and relevancy of model-generated texts before incorporating them for final inference or as a controlling mechanism during generation