Differentiable Open-Ended Commonsense Reasoning (2021)

Abstract: Current commonsense reasoning research focuses on developing models that use commonsense knowledge to answer multiple-choice questions. However, systems designed to answer multiple-choice questions may not be useful in applications that do not provide a small list of candidate answers to choose from. As a step towards making commonsense reasoning research more realistic, we propose to study open-ended commonsense reasoning (OpenCSR) — the task of answering a commonsense question without any pre-defined choices — using as a resource only a corpus of commonsense facts written in natural language. OpenCSR is challenging due to a large decision space, and because many questions require implicit multi-hop reasoning. As an approach to OpenCSR, we propose DrFact, an efficient Differentiable model for multi-hop Reasoning over knowledge Facts. To evaluate OpenCSR methods, we adapt several popular commonsense reasoning benchmarks, and collect multiple new answers for each test question via crowd-sourcing. Experiments show that DrFact outperforms strong baseline methods by a large margin.

## introduction the conventional task setting for most current commonsense reasoning research is multiplechoice question answering (qa) -i.e., given a question and a small set of pre-defined answer choices, models are required to determine which of the candidate choices best answers the question. existing commonsense reasoning models usually work by scoring a question-candidate pair @xcite @xcite . hence, even an accurate multiple-choice carbon dioxide is the major greenhouse gas contributing to global warming . ## drfact: an efficient approach for differentiable reasoning over facts in this section we present drfact, a model for multi-hop reasoning over facts. more implementation details are in appendix b. ## conclusion we introduce and study a new task -open-ended commonsense reasoning (opencsr) -which is both realistic and challenging. we construct three opencsr versions of widely used datasets targeting commonsense reasoning with a novel crowdsourced collection of multiple answers, and evaluate a number of baseline methods for this task. we also present a novel method, drfact. dr-fact is a scalable multi-hop reasoning method that traverses a corpus (as a hypergraph) via a differentiable "fact-following" reasoning process, employing both a neural dense index of facts and sparse tensors of symbolic links between facts, using a combination of mips and sparse-matrix computation. drfact outperforms several strong baseline methods on our data, making a significant step towards adapting commonsense reasoning approaches to more practical applications. base on the multi-hop reasoning framework of drfact, we hope the work can benefit future research on neural-symbolic commonsense reasoning. naval research, the defense advanced research projects agency with award w911nf-19-20271, and nsf sma 18-29268. we thank all reviewers for their constructive feedback and comments.