SlowGuess/NAACL_2021_OCR huggingface dataset:
Bill Yuchen Lin $^{1*}$ , Haitian Sun $^{2}$ , Bhuwan Dhingra $^{2}$ , Manzil Zaheer $^{2}$ , Xiang Ren $^{1}$ , William W. Cohen $^{2}$
<sup>1</sup> University of Southern California
{yuchen.lin, xiangren}@usc.edu
{haitiansun,bdhingra,manzilzaheer,wcohen}@google.com
Current commonsense reasoning research focuses on developing models that use commonsense knowledge to answer multiple-choice questions. However, systems designed to answer multiple-choice questions may not be useful in applications that do not provide a small list of candidate answers to choose from. As a step towards making commonsense reasoning research more realistic and useful, we propose to study open-ended commonsense reasoning (OpenCSR) — the task of answering a commonsense question without any predefined choices — using as a resource only a knowledge corpus of commonsense facts written in natural language. OpenCSR is challenging due to a large decision space, and because many questions require implicit multi-hop reasoning. As an approach to OpenCSR, we propose DRFACT, an efficient Differentiable model for multi-hop Reasoning over knowledge Facts. To evaluate OpenCSR methods, we adapt three popular multiple-choice datasets, and collect multiple new answers to each test question via crowd-sourcing. Experiments show that DRFACT outperforms strong baseline methods by a large margin. $^1$
The conventional task setting for most current commonsense reasoning research is multiple-choice question answering (QA) — i.e., given a question and a small set of pre-defined answer choices, models are required to determine which of the candidate choices best answers the question. Existing commonsense reasoning models usually work by scoring a question-candidate pair (Lin et al., 2019; Lv et al., 2020; Feng et al., 2020). Hence, even an accurate multiple-choice
Q: What can help alleviate global warming?
Multiple-Choice CSR
(reason w/ question+choice)
Open-Ended CSR (reason w/ question)
(A) air cooler (B) fossil fuel
(C) renewable energy (D) water
a large text corpus of commonsense facts
Multi-Hop
carbon dioxide is the major greenhousegas contributing to global warming .
Figure 1: We study the task of open-ended commonsense reasoning (OpenCSR), where answer candidates are not provided (as in a multiple-choice setting). Given a question, a reasoner uses multi-hop reasoning over a knowledge corpus of facts, and outputs a ranked list of concepts from the corpus.
trees remove carbon dioxide from theatmosphere through photosynthesis .
Output: ..., renewable energy, tree, solar battery, ...
a ranked list of concepts as answers.
QA model cannot be directly applied in practical applications where answer candidates are not provided (e.g., answering a question asked on a search engine, or during conversation with a chat bot).
Because we seek to advance commonsense reasoning towards practical applications, we propose to study open-ended commonsense reasoning (OpenCSR), where answers are generated efficiently, rather than selected from a small list of candidates (see Figure 1). As a step toward this, here we explore a setting where the model produces a ranked list of answers from a large question-independent set of candidate concepts that are extracted offline from a corpus of common-sense facts written in natural language.
The OpenCSR task is inherently challenging. One problem is that for many questions, finding an answer requires reasoning over two or
* The work was mainly done during Bill Yuchen Lin's internship at Google Research.
1Our code and data are available at the project website - https://open-csr.github.io/. The human annotations were collected by the USC-INK group.
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4611-4625 June 6-11, 2021. ©2021 Association for Computational Linguistics
more natural-language facts from a corpus. In the multiple-choice QA setting, as the set of candidates is small, we can pair a question with an answer, and use the combination to retrieve relevant facts and then reason with them. In the open-ended setting, this is impractical: instead one needs to retrieve facts from the corpus using the question alone. In this respect, OpenCSR is similar to multi-hop factoid QA about named entities, e.g. as done for HotpotQA (Yang et al., 2018).
However, the underlying reasoning chains of most multi-hop factoid QA datasets are relatively clear and context-independent, and are thus easier to infer. Commonsense questions, in contrast, exhibit more variable types of reasoning, and the relationship between a question and the reasoning to answer the question is often unclear. (For example, a factoid question like "who starred in a movie directed by Bradley Cooper?" clearly suggests following a directed-by relationship and then a starred-in relationship, while the underlying reasoning chains of a question like "what can help alleviate global warming?" is relatively implicit from the question.) Furthermore, annotations are not available to identify which facts are needed in the latent reasoning chains that lead to an answer — the only supervision is a set of questions and their answers. We discuss the formulation of OpenCSR and its challenges further in Section 3.
As shown in Fig. 1, another challenge is that many commonsense questions require reasoning about facts that link several concepts together. E.g., the fact "trees remove carbon dioxide from the atmosphere through photosynthesis" cannot be easily decomposed into pairwise relationships between "trees", "carbon dioxide", "the atmosphere", and "photosynthesis", which makes it more difficult to store in a knowledge graph (KG). However, such facts have been collected as sentences in common-sense corpora, e.g., GenericsKB (Bhakthavatsalam et al., 2020). This motivates the question: how can we conduct multi-hop reasoning over such a knowledge corpus, similar to the way multi-hop reasoning methods traverse a KG? Moreover, can we achieve this in a differentiable way, to support end-to-end learning?
To address this question, we extend work by Seo et al. (2019) and Dhingra et al. (2020), and propose an efficient, differentiable multi-hop reasoning method for OpenCSR, named DRFACT (for Differentiable Reasoning over Facts). Specifically,
we formulate multi-hop reasoning over a corpus as an iterative process of differentiable fact-following operations over a hypergraph. We first encode all fact sentences within the corpus as dense vectors to form a neural fact index, such that a fast retrieval can be done via maximum inner product search (MIPS). This dense representation is supplemented by a sparse fact-to-fact matrix to store symbolic links between facts (i.e., a pair of facts are linked if they share common concepts). DR-FACT thus merges both neural and symbolic aspects of the relationships between facts to model reasoning in an end-to-end differentiable framework (Section 4).
To evaluate OpenCSR methods, we construct new OpenCSR datasets by adapting three existing multiple-choice QA datasets: QASC (Khot et al., 2020), OBQA (Mihaylov et al., 2018), and ARC (Clark et al., 2018). Note that unlike factoid questions that usually have a single correct answer, open-ended commonsense questions can have multiple correct answers. Thus, we collect a collection of new answers for each test question by crowd-sourcing human annotations. We compare with several strong baseline methods and show that our proposed DRFACT outperforms them by a large margin. Overall DRFACT gives an  $4.6\%$  absolute improvement in Hit@100 accuracy over DPR (Karpukhin et al., 2020), a state-of-the-art text retriever for QA, and  $3.2\%$  over DrKIT (Dhingra et al., 2020), a strong baseline for entity-centric multi-hop reasoning. With a relatively more expensive re-ranking module, the gap between DRFACT and others is even larger. (Sec. 5)
Commonsense Reasoning. Many recent commonsense-reasoning (CSR) methods focus on multiple-choice QA. For example, KagNet (Lin et al., 2019) and MHGRN (Feng et al., 2020) use an external commonsense knowledge graph as structural priors to individually score each choice. These methods, though powerful in determining the best choice for a multi-choice question, are less realistic for practical applications where answer candidates are typically not available. UnifiedQA (Khashabi et al., 2020) and other closed-book QA models (Roberts et al., 2020) generate answers to questions by fine-tuning a text-to-text transformer such as BART (Lewis et al., 2020a) or T5 (Raffel et al., 2020), but a
disadvantage of closed-book QA models is that they do not provide intermediate explanations for their answers, i.e., the supporting facts, which makes them less trustworthy in downstream applications. Although closed-book models exist that are augmented with an additional retrieval module (Lewis et al., 2020b), these models mainly work for single-hop reasoning.
QA over KGs or Text. A conventional source of commonsense knowledge is triple-based symbolic commonsense knowledge graphs (CSKGs) such as ConceptNet (Speer et al., 2017). However, the binary relations in CSKGs greatly limit the types of the knowledge that can be encoded. Here, instead of a KB, we use a corpus of generic sentences about commonsense facts, in particular GenericsKB (Bhakthavatsalam et al., 2020). The advantage of this approach is that text can represent more complex commonsense knowledge, including facts that relate three or more concepts. Formalized in this way, OpenCSR is a question answering task requiring (possibly) iterative retrieval, similar to other open-domain QA tasks (Chen et al., 2017) such as HotpotQA (Yang et al., 2018) and Natural Questions (Kwiatkowski et al., 2019). As noted above, however, the surface of commonsense questions in OpenCSR have fewer hints about kinds of multi-hop reasoning required to answer them than the factoid questions in open-domain QA, resulting in a particularly challenging reasoning problem (see Sec. 3).
Multi-Hop Reasoning. Many recent models for open-domain QA tackle multi-hop reasoning through iterative retrieval, e.g., GRAFT-Net (Sun et al., 2018), MUPPET (Feldman and El-Yaniv, 2019), PullNet (Sun et al., 2019), and GoldEn (Qi et al., 2019). These models, however, are not end-to-end differentiable and thus tend to have slower inference speed, which is a limitation shared by many other works using reading comprehension for multi-step QA (Das et al., 2019; Lee et al., 2019). As another approach, Neural Query Language (Cohen et al., 2020) designs differentiable multi-hop entity-following templates for reasoning over a compactly stored symbolic KG, but this KG is limited to binary relations between entities from an explicitly enumerated set.
DrKIT (Dhingra et al., 2020) is the most similar work to our DRFACT, as it also supports multi-hop reasoning over a corpus. Unlike DRFACT, DrKIT is designed for entity-centric reasoning. DrKIT
begins with an entity-linked corpus, and computes both sparse and dense indices of entity mentions (i.e., linked named-entity spans). DrKIT's fundamental reasoning operation is to "hop" from one weighted set of  $X$  entities to another, by 1) finding mentions of new entities  $x'$  that are related to some entity in  $X$ , guided by the indices, and then 2) aggregating these mentions to produce a new weighted set of entities. DrKIT's operations are differentiable, and by learning to construct appropriate queries to the indices, it can be trained to answer multi-hop entity-related questions.
Prior to our work DrKIT been applied only on factoid questions about named entities. In CSR, the concepts that drive reasoning are generally less precise than entities, harder to disambiguate in context, and are also much more densely connected, so it is unclear to what extent DrKIT would be effective. We present here novel results using DrKIT on OpenCSR tasks, and show experimentally that our new approach, DRFACT, improves over DrKIT. DRFACT mainly differs from DrKIT in that its reasoning process learns to "hop" from one fact to another, rather than from one entity to another, thus effectively using the full information from a fact for multi-hop reasoning.
3 Open-Ended Commonsense Reasoning
Task Formulation. We denote a corpus of knowledge facts as  $\mathcal{F}$ , and use  $\nu$  to denote a vocabulary of concepts; both are sets consisting of unique elements. A fact  $f_{i} \in \mathcal{F}$  is a sentence that describes generic commonsense knowledge, such as "trees remove carbon dioxide from the atmosphere through photosynthesis." A concept  $c_{j} \in \mathcal{V}$  is a noun or base noun phrase mentioned frequently in these facts (e.g., 'tree' and 'carbon dioxide'). Concepts are considered identical if their surface forms are the same (after lemmatization). Given only a question  $q$  (e.g., "what can help alleviate global warming?"), an open-ended commonsense reasoner is supposed to answer it by returning a weighted set of concepts, such as  $\{(a_{1} = 'renewable energy', w_{1}), (a_{2} = 'tree', w_{2}), \ldots\}$ , where  $w_{i} \in \mathbb{R}$  is the weight of the predicted concept  $a_{i} \in \mathcal{V}$ .
To learn interpretable, trustworthy reasoning models, it is expected that models can output intermediate results that justify the reasoning process - i.e., the supporting facts from  $\mathcal{F}$ . E.g., an explanation for 'tree' to be an answer to the ques
Figure 2: A motivating example of how DrFact works for OpenCSR. We model the knowledge corpus as a hypergraph consisting of concepts in  $\mathcal{V}$  as nodes and facts in  $\mathcal{F}$  as hyperedges. Then, we develop a differentiable reasoning method, DrFact, to perform multi-hop reasoning via fact-following operations (e.g.,  $f_{1} \rightarrow f_{2}$ ).
tion above can be the combination of two facts:  $f_{1}$  = "carbon dioxide is the major ..." and  $f_{2}$  = "trees remove ..." as shown in Figure 1.
Implicit Multi-Hop Structures. Commonsense questions (i.e., questions that need commonsense knowledge to reason) contrast with better-studied multi-hop factoid QA datasets, e.g., HotpotQA (Yang et al., 2018), which primarily focus on querying about evident relations between named entities. For example, an example multi-hop factoid question can be "which team does the player named 2015 Diamond Head Classic's MVP play for?" Its query structure is relatively clear and self-evident from the question itself: in this case the reasoning process can be decomposed into  $q_{1} =$  "the player named 2015 DHC's MVP" and  $q_{2} =$  "which team does  $q_{1}$ . answer play for".
The reasoning required to answer commonsense questions is usually more implicit and relatively unclear. Consider the previous example in Fig. 1,  $q =$  'what can help alleviate global warming?' can be decomposed by  $q_{1} =$  "what contributes to global warming" and  $q_{2} =$  "what removes  $q_{1}$ .answer from the atmosphere" - but many other decompositions are also plausible. In addition, unlike HotpotQA, we assume that we have no ground-truth justifications for training, which makes OpenCSR even more challenging.
4 DrFact: An Efficient Approach for Differentiable Reasoning over Facts
In this section we present DRFACT, a model for multi-hop reasoning over facts. More implementation details are in Appendix B.
In DRFACT, we propose to model reasoning as traversing a hypergraph, where each hyperedge corresponds to a fact in  $\mathcal{F}$ , and connects the concepts in  $\mathcal{V}$  that are mentioned in that fact. This is shown in Figure 2. Notice that a fact, as a hyperedge, connects multiple concepts that are mentioned, while the textual form of the fact maintains the contextual information of the original natural language statement, and hence we do not assume a fixed set of relations.
Given such a hypergraph, our open-ended reasoning model will traverse the hypergraph starting from the question (concepts) and finally arrive at a set of concept nodes by following multiple hyperedges (facts). A probabilistic view of this process over  $T$  hops is:
$$
P (c \mid q) = P (c \mid q, F _ {T}) \prod_ {t = 1} ^ {T} P (F _ {t} \mid q, F _ {t - 1}) P (F _ {0} \mid q)
$$
Intuitively, we want to model the distribution of a concept  $c \in \mathcal{V}$  being an answer to a question  $q$  as  $P(c \mid q)$ . This answering process can be seen as a process of multiple iterations of "fact-following," or moving from one fact to another based on shared concepts, and finally moving from facts to concepts. We use  $F_{t}$  to represent a weighted set of retrieved facts at the hop  $t$ , and  $F_{0}$  for the initial facts below. Then, given the question and the current retrieved facts, we iteratively retrieve the facts for the next hop. Finally, we score a concept using retrieved facts.
4.2 Pre-computed Indices
Dense Neural Fact Index  $D$ . We pre-train a bi-encoder architecture over BERT (Devlin et al.,
Figure 3: The overall workflow of DRFACT. We encode the hypergraph (Fig. 2) with a concept-to-fact sparse matrix  $E$  and a fact-to-fact sparse matrix  $S$ . The dense fact index  $D$  is pre-computed with a pre-trained bi-encoder. A weighed set of facts is represented as a sparse vector  $F$ . The workflow (left) of DRFACT starts mapping a question to a set of initial facts that have common concepts with it. Then, it recursively performs Fact-Follow operations (right) for computing  $F_{t}$  and  $A_{t}$ . Finally, it uses learnable hop-weights  $\alpha_{t}$  to aggregate the answers.
2019), which learns to maximize the score of facts that contain correct answers to a given question, following the steps of Karpukhin et al. (2020) (i.e., dense passage retrieval), so that we can use MIPS to do dense retrieval over the facts. After pre-training, we embed each fact in  $\mathcal{F}$  with a dense vector (using the [CLS] token representation). Hence  $D$  is a  $|\mathcal{F}| \times d$  dense matrix.
Sparse Fact-to-Fact Index  $S$ . We pre-compute the sparse links between facts by a set of connection rules, such as  $f_{i} \rightarrow f_{j}$  when  $f_{i}$  and  $f_{j}$  have at least one common concept and  $f_{j}$  introduces at least two more new concepts that are not in  $f_{i}$  (see Appendix B (2) for more). Hence  $S$  is a binary sparse tensor with the dense shape  $|\mathcal{F}| \times |\mathcal{F}|$ .
Sparse Index of Concept-to-Fact Links  $E$ . As shown in Figure 2, a concept can appear in multiple facts and a fact also usually mentions multiple concepts. We encode these co-occurrences between each fact and its mentioned concepts into a sparse matrix with the dense shape  $|\mathcal{V}| \times |\mathcal{F}|$  — i.e., the concept-to-fact index.
4.3 Differentiable Fact-Following Operation
The most important part in our framework is how to model the fact-following step in our formulation, i.e.,  $P(F_{t} \mid F_{t-1}, q)$ . For modeling the translation from a fact to another fact under the context of a question  $q$ , we propose an efficient approach with a differentiable operation that uses both neural embeddings of the facts and their symbolic connections in the hypergraph.
The symbolic connections between facts are represented by the very sparse fact-to-fact matrix
$S$ , which in our model is efficiently implemented with the tf.RaggedTensor construct of TensorFlow (Dhingra et al., 2020).  $S$  stores a precomputed dependency between pairs of facts,  $S_{ij}$ . Intuitively, if we can traverse from  $f_{i}$  to  $f_{j}$  these facts should mention some common concepts, and also the facts' semantics are related, so our  $S_{ij}$  will reflect this intuition. The fact embeddings computed by a pre-trained bi-encoder are in the dense index of fact vectors  $D$ , which contains rich semantic information about each fact, and helps measure the plausibility of a fact in the context of a given question.
The proposed fact-follow operation has two parallel sub-steps: 1) sparse retrieval and 2) dense retrieval. The sparse retrieval uses a fact-to-fact sparse matrix to obtain possible next-hop facts. We can compute  $F_{t}^{s} = F_{t - 1}S$  efficiently thanks to the ragged representation of sparse matrices.
For the neural dense retrieval, we use a maximum inner product search (MIPS) (Johnson et al., 2019; Guo et al., 2020) over the dense fact embedding index  $D$ :
$$
\mathbf {z} _ {\mathbf {t} - \mathbf {1}} = F _ {t - 1} D
$$
$$
\mathbf {h} _ {\mathbf {t} - \mathbf {1}} = g (\mathbf {z} _ {\mathbf {t} - \mathbf {1}}, \mathbf {q} _ {\mathbf {t}})
$$
$$
F _ {t} ^ {d} = \operatorname {M I P S} _ {K} (\mathbf {h} _ {\mathbf {t} - \mathbf {1}}, D)
$$
We first aggregate the dense vectors of the facts in  $F_{t - 1}$  into the dense vector  $\mathbf{z}_{\mathbf{t} - 1}$ , which is fed into a neural layer with the query embedding at the current step,  $\mathbf{q_t}$  (encoded by BERT), to create a query vector  $\mathbf{h}_{\mathbf{t} - 1}$ . Here  $g(\cdot)$  is an MLP that maps the concatenation of the two input vectors to
a dense output with the same dimensionality as the fact vectors, which we named to be fact-translating function. Finally, we retrieve the next-hop top-K facts  $F_{t}^{d}$  with the  $\mathrm{MIPS}_K$  operator.
To get the best of both symbolic and neural world, we use element-wise multiplication to combine the sparse and dense retrieved results:  $F_{t} = F_{t}^{s} \odot F_{t}^{d}$ . We summarize the fact-following operation with these differentiable steps:
$$
\begin{array}{l} F _ {t} = \text {F a c t - F o l l o w} \left(F _ {t - 1}, q\right) \tag {1} \\ = F _ {t - 1} S \odot \operatorname {M I P S} _ {K} (g \left(F _ {t - 1} D, \mathbf {q} _ {\mathbf {t}}\right), D) \\ \end{array}
$$
After each hop, we multiply  $F_{t}$  with a precomputed fact-to-concept matrix  $E$ , thus generating  $A_{t}$ , a set of concept predictions. To aggregate the concept scores, we take the maximum score among the facts that mention a concept  $c$ . Finally we take the weighted sum of the concept predictions at all hops as the final weighted concept sets  $A = \sum_{t=1}^{T} \alpha_{t} A_{t}$ , where  $\alpha_{t}$  is a learnable parameter. Please read Appendix B for more details.
Equation 1 defines a random-walk process on the hypergraph associated with the corpus. We found that performance was improved by making this a "lazy" random walk—in particular by augmenting  $F_{t}$  with the facts in  $F_{t-1}$  which have a weight higher than a threshold  $\tau$ :
$$
F _ {t} = \operatorname {F a c t - F o l l o w} \left(F _ {t - 1}, q\right) + \operatorname {F i l t e r} \left(F _ {t - 1}, \tau\right).
$$
We call this as self-following, which means that  $F_{t}$  contains highly-relevant facts for all distances  $t' < t$ , and thus improve models when there are variable numbers of "hops" for different questions.
Initial Facts. Note that the set of initial facts  $F_0$  is computed differently, as they are produced using the input question  $q$ , instead of a previous-hop  $F_{t-1}$ . We first use our pre-trained bi-encoder and the associated index  $D$  via MIPS query to find facts related to  $q$ , and then select from the retrieved set those facts that contain question concepts (i.e., concepts that are matched in the question text), using the concept-to-fact index  $E$ .
4.4 Auxiliary Learning with Distant Evidence
Intermediate evidence, i.e., supporting facts, is significant for guiding multi-hop reasoning models during training. In a weakly supervised setting, however, we usually do not have ground-truth annotations as they are expensive to obtain.
To get some noisy yet still helpful supporting facts, we use as distant supervision dense retrieval
based on the training questions. Specifically, we concatenate the question and the best candidate answer to build a query to our pre-trained index  $D$ , and then we divide the results into four groups depending on whether they contain question/answer concepts: 1) question-answer facts, 2) question-only facts, 3) answer-only facts, and 4) none-facts.