TSGP : Two-Stage Generative Prompting for Unsupervised Commonsense Question Answering (2022)

Abstract: Without training on labeled task data, unsupervised commonsense question answering seems challenging since it requires commonsense knowledge beyond the context of questions. Previous methods typically retrieved from traditional knowledge bases or used pre-trained language models (PrLMs) to generate fixed types of knowledge, which have poor generalization ability.In this paper, we aim to address the above limitation by leveraging the implicit knowledge stored in PrLMs and propose a two-stage prompt-based unsupervised commonsense question answering framework (TSGP). We first use knowledge generation prompts to generate the knowledge required for questions with unlimited types and possible candidate answers independent of specified choices. Then, we further utilize answer generation prompts to generate possible candidate answers independent of specified choices. Experimental results and analysis on three different commonsense reasoning tasks, CommonsenseQA, OpenBookQA, and SocialIQA, demonstrate that TSGP significantly improves the reasoning ability of language models in unsupervised settings.

## introduction commonsense question answering (csqa) requires systems to acquire different types of commonsense knowledge, consisting of widely known facts that humans use to reason and respond to everyday situations, but are challenging for machines @xcite . as illustrated in figure 1 , humans consider their own experience or education when answering questions, and their answers can vary from person to person. existing studies commonly focus on acquiring relevant knowledge by retrieving external knowledge bases and then fine-tuning pre-trained language models (prlms) in a supervised manner on task-specific data @xcite @xcite . recently, unsupervised commonsense question answering has attracted the attention of researchers because it does not rely on any labeled downstream task data and does not require fine-tuning of pre-trained models. unsupervised commonsense question answering presents two challenges: (i) eliciting the required commonsense knowledge from a pre-trained language model; and (ii) scoring the answers reasonably. correspondingly, previous works can be divided into two categories. one is to use preset templates to allow prlms to generate specific types of knowledge or to generate the meaning of questions @xcite . however, the type of knowledge generated by this method is limited, making it difficult to transfer to new task domains. another class of methods design scoring functions to rank answer choices, such as using cross-entropy or mutual information to score each answer choice directly or calculating the semantic similarity between choices and generated pseudo-answers @xcite . however, they are limited to modeling using the implicit parameters of prlms, without explicitly capturing question-related knowledge from prlms and using them to guide inference. ## method we focus on the unsupervised multiple-choice commonsense question answering task, which is formalized as follows: given a question q and a set of choices a, the model should choose the correct option â: â = argmax @xmath0. as shown in figure 2 , we aim to fully exploit the knowledge encoded in prlms and propose the tsgp framework, that mainly contain 3 parts: 1) knowledge generation, that using gpt to generate question-conditioned knowledge statements; 2) answer generation, that use a language model to generate pseudo-answers conditioned by the question and the generated knowledge; and 3) answer prediction, that compute the semantic scores of the generated pseudo-answers with each option, and vote for the most semantically relevant option. in this subsection, we will describe these in detail. ## experiment in this section, we first describe our experiments on three commonsense question answering datasets, followed by further analysis and case studies. ## conclusion in this paper, we propose a two-stage prompt-based unsupervised commonsense question answering framework that makes implicit intermediate reasoning steps explicit and generates possible candidate answers independent of specified choices. specifically, we design knowledge and answer generation prompts that can prompt the language model to flexibly generate commonsense knowledge required for a question and its multiple corresponding candidate answers. experiments show our method's effectiveness across multiple datasets, achieving state-of-the-art results on three commonsense reasoning tasks in an unsupervised setting. in addition, it does not require fine-tuning, does not depend on specific pre-trained models and tasks, and is flexible in transfer. we hope that our framework will facilitate future research in this field.