Open-ended Commonsense Reasoning with Unrestricted Answer Candidates (2023)

Abstract: Open-ended Commonsense Reasoning is defined as solving a commonsense question without providing 1) a short list of answer candidates and 2) a pre-defined answer scope. Conventional ways of formulating the commonsense question into a question-answering form or utilizing external knowledge to learn retrieval-based methods are less applicable in the open-ended setting due to an inherent challenge. Without pre-defining an answer scope or a few candidates, open-ended commonsense reasoning entails predicting answers by searching over an extremely large searching space. Moreover, most questions require implicit multi-hop reasoning, which presents even more challenges to our problem. In this work, we leverage pre-trained language models to iteratively retrieve reasoning paths on the external knowledge base, which does not require task-specific supervision. The reasoning paths can help to identify the most precise answer to the commonsense question. We conduct experiments on two commonsense benchmark datasets. Compared to other approaches, our proposed method achieves better performance both quantitatively and qualitatively.

## introduction current research on commonsense reasoning conventionally formulates the problem into a multiplechoice question answering (qa) format, where the best answer is expected to be chosen from a list of candidates for the given commonsense question. however, there are many practical and real-world scenarios where a small list of answer candidates or an answer scope (i.e., a relatively large set of concepts where the correct answer exists) are missing or not even provided (e.g., arbitrary questions asked in the search engine), which requires the intelligent system to understand the commonsense question rather than picking a correct answer from question context: what gives fish more speed? transformed prompt: t5-3b: in fact, they are the fastest of all living creatures on our planet. ## proposed method in this section, we first introduce the problem formulation, and then discuss the detailed framework of the proposed method, which can be divided into three components: 1) entity extraction and linking, 2) local knowledge graph expansion, and 3) training strategy and answer prediction. ## experiment we leverage roberta-large @xcite as our base plm. we empirically verify the performance of the proposed method against other methods on commonsense reasoning benchmark datasets under the open-ended setting. due to the space limit, more experiments, case studies, and implementation details can be found in appendix a.1. ## conclusion we present an off-the-shelf framework keep to predict answers for open-ended commonsense reasoning without requiring answer candidates and a pre-defined answer scope. by integrating the implicit knowledge stored in plms and the external knowledge base, keep retrieves relevant reasoning paths and extracts suitable answers for commonsense questions while maintaining both efficiency and efficacy. we believe this work poses a new direction to automated commonsense reasoning under the zero-shot and open-ended setting in the large language model era @xcite .