SlowGuess/NAACL_2022_OCR huggingface dataset:
Yueqing Sun, Qi Shi, Le Qi, Yu Zhang*
Research Center for Social Computing and Information Retrieval
{yqsun,qshi,lqi,zhangyu}  $@$  ir.hit.edu.cn
Existing KG-augmented models for commonsense question answering primarily focus on designing elaborate Graph Neural Networks (GNNs) to model knowledge graphs (KGs). However, they ignore (i) the effectively fusing and reasoning over question context representations and the KG representations, and (ii) automatically selecting relevant nodes from the noisy KGs during reasoning. In this paper, we propose a novel model, JointLK, which solves the above limitations through the joint reasoning of LM and GNN and the dynamic KGs pruning mechanism. Specifically, JointLK performs joint reasoning between LM and GNN through a novel dense bidirectional attention module, in which each question token attends on KG nodes and each KG node attends on question tokens, and the two modal representations fuse and update mutually by multi-step interactions. Then, the dynamic pruning module uses the attention weights generated by joint reasoning to prune irrelevant KG nodes recursively. We evaluate JointLK on the CommonsenseQA and OpenBookQA datasets, and demonstrate its improvements to the existing LM and  $\mathrm{LM + KG}$  models, as well as its capability to perform interpretable reasoning<sup>1</sup>.
Commonsense question answering (CSQA) requires systems to acquire different types of commonsense knowledge and reasoning skills, which is normal for humans, but challenging for machines (Talmor et al., 2019). Recently, large pre-trained language models (LMs) have achieved remarkable success in many QA tasks and appear to use implicit (factual) knowledge encoded in their model parameters during fine-tuning (Liu et al., 2019; Raffel et al., 2020). Nevertheless, commonsense
Figure 1: Our knowledge-augmented joint reasoning model framework with an example from Common-senseQA. The subgraph is retrieved from ConceptNet.
knowledge is self-evident to humans and is rarely expressed clearly in natural language (Gunning, 2018), which makes it difficult for LMs to learn commonsense knowledge from the pre-training text corpus alone.
An extensive research path is to elaborately design graph neural networks (GNNs) (Scarselli et al., 2008) to perform reasoning over explicit structural common sense knowledge from external knowledge bases (Vrandecic and Krötzsch, 2014; Speer et al., 2017). Related methods usually follow a retrieval-and-modeling paradigm. First, the knowledge subgraphs or paths related to a given question are retrieved by string matching or semantic similarity; such retrieved structured information indicates the relation between concepts or implies the process of multi-hop reasoning. Second, the retrieved subgraphs are modeled by a well-designed graph neural network module (Lin et al., 2019; Feng et al., 2020; Yasunaga et al., 2021) to perform reasoning over knowledge graphs.
However, these approaches have two main issues. First, the retrieved knowledge subgraph contains many noisy nodes. Whether through
$^{1}$ Our code is available at: https://github.com/Yueqing-Sun/JointLK
Human Language Technologies, pages 5049 - 5060
simple string matching or semantic matching, in order to retrieve sufficient relevant knowledge, noise knowledge graph nodes will inevitably be included (Lin et al., 2019; Yasunaga et al., 2021). Especially with the increase of hop count, the number of irrelevant nodes will expand dramatically, raising the burden of the model. As the example in Figure 1, some graph nodes such as "wood", "burn", and "gas", although related to some entities in the questions and choice, can mislead the global understanding of the question. Second, there are limited interactions between language representation and knowledge graph representation. Specifically, existing LM+KG methods (Lin et al., 2019; Feng et al., 2020) model question context and knowledge subgraphs in isolation by LMs and GNNs, and perform only one interaction in a shallow manner to fuse their representations at the output for prediction. We argue that the limited interaction between the two modalities is the main bottleneck that may prevent the model from understanding the complex question-knowledge relations necessary to answer the question correctly.
Based on the above consideration, we propose JointLK, a model that performs the fine-grained modal fusion and multi-layer joint reasoning between the language model and the knowledge graph (see Figure 2). Specifically, given a question and retrieved subgraphs, JointLK first obtain the representations of the two modalities by using an LM encoder and a GNN encoder respectively. Then we design a joint reasoning module to generate fine-grained bidirectional attention maps between each question token and each KG node to fuse the information from each modality to the other. Guided by the attention generated in the interaction process, the dynamic pruning module deletes irrelevant nodes to make the model reason along the correct knowledge path. Multiple JointLK layers are stacked to form a hierarchy that supports multi-step interactions and recursive pruning. In summary, our contributions are three-fold:
- We propose JointLK, a novel model that supports multi-step joint reasoning between LM and KG. It uses dense bidirectional attention to simultaneously update query-aware knowledge graph representation and knowledge-aware query representation, bridging the gap between the two information modalities.
- We design a dynamic graph pruning module that recursively removes irrelevant graph
nodes at each JointLK layer to ensure that the model reasons correctly with complete and appropriate evidence.
- Experimental results show that JointLK is superior to current LM+KG methods, and the refined evidence is interpretable. Furthermore, through the multi-layer fusion of these two modalities, JointLK exhibits strong performance over previous state-of-the-art LM+KG methods in performing complex reasoning, such as solving questions with negation and complex questions with more entities.
Commonsense question answering is challenging because the required commonsense knowledge is rarely given in the context of questions and answer choices or encoded in the parameters of pre-trained LMs. Therefore, many works obtain the required knowledge from external sources (e.g., KGs, corpus) to augment CSQA models. Due to the heterogeneity between structured knowledge and unstructured text questions, there are currently two main research methods. Some works (Lv et al., 2020; Bian et al., 2021; Xu et al., 2021) unify the two modalities during model input, such as transforming structured knowledge into plain text through templates or transforming question context into structured graphs. However, the original structural/textual information will inevitably be lost during the conversion process. Other works (Lin et al., 2019; Feng et al., 2020; Yan et al., 2021) use LM and GNN to model the two modalities separately, and perform shallow interactions in the latter model stage, such as attentive pooling or simple concatenation of the two modal representations. Although this method can retain the original information of question context and KGs, the limited interaction will affect the flow of information between the two modalities, so we mainly improve on this point.
Recently, QA-GNN (Yasunaga et al., 2021) explicitly views the QA context as an additional node, connects it and KG to form a joint graph, and mutually updates their representations through graph-based message passing. However, it pools the representation of the question context into a single node, which limits the updating of the text representation and fine-grained interaction between LM and GNN. Compared with prior works, we retain the individual structure of both modalities, consider fine
Figure 2: Overall architecture of our proposed JointLK model, which takes a query (question + choice) and a retrieved knowledge subgraph as input, and outputs a scalar that represents the plausibility score of this query. JointLK mainly consists of four modules the Query Encoder, the Graph Layer, the Joint Reasoning Module and the Dynamic Pruning Module, of which the latter three form a stack of N identical layers.
grained interaction between any token in question and any entity in KG through dense bidirectional attention, and perform multi-step joint reasoning by stacking several interaction layers. Furthermore, we gradually prune the KG size in each stacked model layer under the guidance of attention weights generated in the interactions, making the reasoning path transparent and interpretable.
In this section, we introduce the task definition (§ 3.1) and our JointLK model. The model framework is shown in Figure 2. JointLK takes the query and the retrieved knowledge subgraph as input, and outputs a real value as the correctness score of the answer. The model is mainly composed of four parts: query encoder, GNN layer, joint reasoning module and dynamic pruning module, of which the latter three form a stack of N identical layers. We use a pre-trained language model to learn the query representation (§ 3.2), and use the GNN layer to learn the graph representation (§ 3.3). The Joint Reasoning Module receives these two modalities' representations and then apply dense bidirectional attention to make information fusion and representation update for each token and node (§ 3.4). The LM-to-KG attention weights generated in reasoning represents the global importance of each node in the graph, so the dynamic pruning module prunes the graph layer by layer according to this
weights and finally retains the most relevant nodes (§ 3.5). After N layers of iteration, the query representation and the trimmed graph representation are used to predict the answer (§ 3.6).
The CSQA task in this paper is a multiple-choice problem with some answer choices. Given a commonsense question  $q$  and a set of answer choices  $\{a_1, a_2, \dots, a_n\}$ , our task is to measure the plausibility score between  $q$  and each answer choice  $a$  then select the answer with the highest plausibility score. In general, questions do not contain any reference to answer choices, so the external knowledge graph provides the necessary background knowledge. We extract from the external KG a subgraph  $g = (V, R)$  with the guidance of question and choice. Here  $V$  is a subset of entity nodes retrieved from the external KG.  $E \subseteq V \times R \times V$  is the set of edges that connect nodes in  $V$ , where  $R$  is a set of relations types. We describe the detailed extraction process in Appendix A.
3.2 Query Encoder
We follow baselines to use pre-trained language models to encode the query  $\{w_{i}\}_{i = 1}^{M}$  (question and choice) into a sequence of vectors  $\{q_i^0\}_{i = 1}^M$
$$
\{\widetilde {q} _ {1} ^ {0}, \dots , \widetilde {q} _ {M} ^ {0} \} = \mathbf {E n c} _ {\mathbf {L M}} \left(\left\{w _ {1}, \dots , w _ {M} \right\}\right) \tag {1}
$$
Here  $\{\tilde{q}_i^0\}_{i = 1}^M\in \mathbb{R}^T$  is the last hidden layer vector of each token in the query. Then we feed the representation of tokens into a non-linear layer so that the text representation space is aligned to the entity representation space:
$$
q _ {i} ^ {0} = \sigma \left(f _ {s} \left(\tilde {q} _ {i} ^ {0}\right)\right) \tag {2}
$$
where  $f_{s}:\mathbb{R}^{T}\to \mathbb{R}^{D}$  is a linear transformation, and  $\sigma$  is the activation function. The representations of tokens  $\mathbf{Q}^0 = \{q_i^0\}_{i = 1}^M\in \mathbb{R}^D$  will be provided to the joint reasoning module for further interaction with the graph entities representations.
3.3 GNN Layer
After obtaining token representations by the query encoder, we further model the subgraph to obtain entity representations. First, We use the BERT model with average pooling to get the initial representation for each entity  $\mathbf{X}^0 = \{x_i^0\}_{i=0}^{|V|} \in \mathbb{R}^D$ . Then, we apply GNN Layer to update node representation through iterative message passing between neighbors on the graph, while GNN is built on the RGAT (Wang et al., 2020a) and is a simplification of Yasunaga et al. (2021). For brevity, we formulate the entire computation in one layer as:
$$
\{\widetilde {x} _ {1} ^ {l}, \dots , \widetilde {x} _ {| V |} ^ {l} \} = \mathbf {G N N} \left(\left\{x _ {1} ^ {l - 1}, \dots , x _ {| V |} ^ {l - 1} \right\}\right) \tag {3}
$$
The output representation  $x_{i}^{l}$  is computed by
$$
\hat {\alpha} _ {j i} = \left(x _ {i} ^ {l - 1} W _ {q}\right) \left(x _ {j} ^ {l - 1} W _ {k} + r _ {j i}\right) ^ {T}, \tag {4}
$$
$$
\alpha_ {j i} = \operatorname {s o f t m a x} \left(\hat {\alpha} _ {j i} / \sqrt {D}\right), \tag {5}
$$
$$
\hat {x} _ {i} ^ {l - 1} = \sum_ {j \in N _ {i} \cup \{i \}} \alpha_ {j i} \left(x _ {j} ^ {l - 1} W _ {v} + r _ {j i}\right), \tag {6}
$$
$$
\widetilde {x} _ {i} ^ {l} = \operatorname {L a y e r N o r m} \left(x _ {i} ^ {l - 1} + \hat {x} _ {i} ^ {l - 1} W _ {o}\right) \tag {7}
$$
where matrices  $W_{q}, W_{k}, W_{v}, W_{o} \in \mathbb{R}^{D \times D}$  are trainable parameters,  $N_{i}$  is the neighbor of node  $i$ .  $r_{ji} = \psi(e_{ji}, u_{j}, u_{i})$  is the relation feature vector, where  $e_{ji}$  is a one-hot vector denoting the relation type of the edge  $(j, i)$  and  $u_{j}, u_{i}$  are one-hot vectors denoting the node types of  $j$  and  $i$ . The following joint reasoning module will further fuse  $\widetilde{x}_{i}^{l}$  and  $q_{i}^{l-1}$  to obtain their updated representations.
3.4 Joint Reasoning Module
To reduce the gap of query and knowledge graph features, we fuse them in the joint reasoning module by the dense bidirectional attention mechanism that connects two encoding layers of query and
knowledge graph and captures the fine-grained interplay between them.
The module takes the query and KG representations  $\mathbf{Q}$  and  $\mathbf{X}$  as inputs and then outputs their updated versions. We denote the inputs to the joint reasoning module in the l-st fusion layer by  $\mathbf{Q}^{l - 1} = \{q_i^{l - 1}\}_{i = 1}^M$  and  $\widetilde{\mathbf{X}}^l = \{\widetilde{x}_i^l\}_{i = 1}^{|V|}$ . Given  $q_{i}^{l - 1}$  and  $\widetilde{x}_i^l$ , an affinity matrix is first constructed via:
$$
S _ {i j} ^ {l} = W _ {S} ^ {T} \left[ q _ {i} ^ {l - 1}; \widetilde {x} _ {j} ^ {l}; q _ {i} ^ {l - 1} \circ \widetilde {x} _ {j} ^ {l} \right] \tag {8}
$$
where  $W_{S}^{T}$  is a learnable weight matrix,  $\circ$  is elementwise multiplication, [;] is vector concatenation across row. We normalize  $S_{ij}^{l}$  in row-wise to derive KG-to-LM attention maps on query tokens conditioned by each entity in KG as
$$
S _ {q _ {i}} ^ {l} = \operatorname {s o f t m a x} \left(S _ {i j}\right) \tag {9}
$$
and also normalize  $S_{ij}^{l}$  in column-wise to derive LM-to-KG attention maps on entities conditioned by each query token as
$$
S _ {x _ {j}} ^ {l} = \operatorname {s o f t m a x} \left(S _ {i j} ^ {T}\right) \tag {10}
$$
The attended representations are computed as follows:
$$
\hat {q} _ {i j} = q _ {i} ^ {l - 1} \otimes S _ {q _ {i}} ^ {l}, \hat {x} _ {i j} = \tilde {x} _ {j} ^ {l} \otimes S _ {x _ {j}} ^ {l} \tag {11}
$$
where  $\otimes$  represents matrix multiplication. The attended features are fused with the original features of the other modality by concatenation and then compressed to low-dimensional space by:
$$
q _ {i} ^ {l} = W _ {Q} \left[ q _ {i} ^ {l - 1}; \hat {x} _ {i j}; q _ {i} ^ {l - 1} \circ \hat {x} _ {i j}; q _ {i} ^ {l - 1} \circ \hat {q} _ {i j} \right], \tag {12}
$$
$$
\bar {x} _ {j} ^ {l} = W _ {X} \left[ \widetilde {x} _ {j} ^ {l}; \hat {q} _ {i j}; \widetilde {x} _ {j} ^ {l} \circ \hat {q} _ {i j}; \widetilde {x} _ {j} ^ {l} \circ \hat {x} _ {i j} \right] \tag {13}
$$
where  $W_{Q}, W_{X}$  are learnable weights. Then the updated query representation  $\mathbf{Q}^{l} = \{q_{i}^{l}\}_{i=1}^{M}$  will be input to the next  $l$ -th stacked JointLK layer of to continue participating in joint reasoning, and the updated KG representation  $\bar{\mathbf{X}}^{l} = \{\bar{x}_{i}^{l}\}_{i=1}^{|V|}$  will be input to the next module of the current JointLK layer for pruning.
3.5 Dynamic Pruning Module
In Equation 10, the LM-to-KG attention value implies the importance of different nodes in the subgraph for question answering. Inspired by SAG-Pool (Lee et al., 2019), under the guidance of query, we retain relevant nodes and cut out irrelevant nodes according to the LM-to-KG attention. Then,
We define a hyperparameter, the Retention ratio  $K \in (0,1]$ , which determines the number of nodes to be retained. We choose the top  $|K \cdot |V|$  nodes according to the value of LM-to-KG attention:
$$
i d x = \operatorname {t o p} - \operatorname {r a n k} (Z, \lceil K \cdot | V | \rceil), \tag {14}
$$
$$
Z _ {\text {m a s k}} = Z _ {\text {i d x}} \tag {15}
$$
where top-rank is a function that returns the index of top  $\lceil K\cdot |V| \rceil$  value,  $\cdot_{idx}$  is an indexing operation, and  $Z_{mask}$  is corresponding attention mask. Next, the subgraph is formed by pooling out the less essential entity nodes as:
$$
\mathbf {X} ^ {l} = \bar {\mathbf {X}} _ {i d x, \vdots} ^ {l} \odot Z _ {m a s k}, \tag {16}
$$
$$
\mathbf {A} ^ {l} = \bar {\mathbf {A}} _ {i d x, i d x} ^ {l}
$$
where  $\bar{\mathbf{X}}_{idx,:}^{l}$  is the row-wise indexed representation matrix of  $\bar{\mathbf{X}}^l$ ,  $\odot$  is the broadcasted elementwise product, and  $\bar{\mathbf{A}}_{idx,idx}^{l}$  is the row-wise and col-wise of indexed adjacency matrix.  $\mathbf{X}^{l} = (x_{1}^{l},x_{2}^{l},\ldots ,x_{|k|V|\lceil |}^{l}),\mathbf{A}^{l}$  and  $\lceil K\cdot |V|\rceil$  are the representation matrix, the adjacency matrix and the number of graph nodes in the next JointLK layer.
3.6 Answer Prediction
After N layers of iteration, we finally obtain the query representation  $\mathbf{Q}^N$  that fuses knowledge information and the graph representation  $\mathbf{X}^N$  that fuses question information. We compute the score of  $a$  being the correct answer as:
$$
p = (a | q) = \mathbf {M L P} ([ s; g ]) \tag {17}
$$
where  $s$  is the mean pooling of  $\mathbf{Q}^N$ , and  $g$  is the attention-based pooling of  $\mathbf{X}^N$ . We get the final probability by normalizing all question-choice pairs with softmax.
We evaluate our model on two typical commonsense question answering datasets CommonsenseQA (Talmor et al., 2019) and OpenBookQA (Mihaylov et al., 2018). CommonsenseQA is a 5-way multiple-choice question answering dataset that requires commonsense for reasoning and contains 12,102 questions. We experiment and report the accuracy on the in-house dev (IHdev) and test (IHtest) splits used by Lin et al. (2019), and report the accuracy of our final system on the official test set. OpenBookQA is a 4-way multiple choice
question answering dataset that requires reasoning with elementary science knowledge. It contains 5,957 questions along with an open book of scientific facts. We use the official data split.
Following previous work (Yasunaga et al., 2021), we use ConceptNet (Speer et al., 2017), a commonsense knowledge graph, as our structured knowledge source for both of the above tasks. Given each query, we follow the preprocessing steps described in Feng et al. (2020) to retrieve the subgraph from ConceptNet, and the max hop size is 3 (see Appendix A for the detail). We use cross-entropy loss and RAdam optimizer (Liu et al., 2020). In training, we set the maximum input sequence length to text encoders to 100, batch size to 128, and perform early stopping. We set the dimension  $(\mathrm{D} = 200)$  and number of layers  $(\mathrm{N} = 5)$  of our GNN module, with dropout rate 0.2 applied to each layer (Srivastava et al., 2014). We use separate learning rates for the LM encoder and the graph encoder. We choose the LM encoder learning rate from  $\{1 \times 10^{-5}, 2 \times 10^{-5}, 3 \times 10^{-5}\}$ , and choose the graph encoder learning rate from  $\{1 \times 10^{-3}, 2 \times 10^{-3}\}$ . Each model is trained using one GPU (Tesla_v100-sxm2-16gb), which takes 20 hours on average.
4.3 Compared Method
Although text corpus can provide complementary knowledge except for knowledge graphs, our model focuses on improving the use of KG and the joint reasoning between LM and KG, so we choose LM and  $\mathrm{LM + KG}$  as the comparison methods.
To investigate the role of KGs, we compare with the benchmark model RoBERTa-large (Liu et al., 2019) for CommonsenseQA, and compare with RoBERTa-large and AristoRoBERTa (Clark et al., 2020) for OpenBookQA. For LM+KG methods, they share a similar high-level framework with our methods, that is, LM is used as a text encoder, GNN or RN is used as a KG encoder, but the way of using knowledge or reasoning is different: (1) Relationship network (RN) (Santoro et al., 2017), (2) RGCN (Schlichtkrull et al., 2018), (3) GconAttn (Wang et al., 2019), (4)KagNet (Lin et al., 2019) and (5)MHGRN (Feng et al., 2020), (6) QA-GNN (Yasunaga et al., 2021). (1), (2) and (3) are the relational perception GNNs for KGs, and (4), (5) and (6) are further model paths in KGs. To be fair, we use the same LM for all comparison methods.
Methods	IHdev-Acc.(%)	IHtest-Acc.(%)
RoBERTa-large(w/o KG)	73.07 (±0.45)	68.69 (±0.56)
+ RGCN	72.69 (±0.19)	68.41 (±0.66)
+ GconAttn	71.61 (±0.39)	68.59 (±0.96)
+ KagNet	73.47 (±0.22)	69.01 (±0.76)
+ RN	74.57 (±0.91)	69.08 (±0.21)
+ MHGRN	74.45 (±0.10)	71.11 (±0.81)
+ QA-GNN	76.54 (±0.21)	73.41 (±0.92)
+ JointLK (Ours)	77.88 (±0.25)	74.43 (±0.83)
Table 1: Performance comparison on CommonsenseQA in-house split. We follow the data division method of Lin et al. (2019) and report the in-house Dev (IHdev) and Test (IHtest) accuracy(mean and standard deviation of four runs).
Methods	Test
RoBERTa (Liu et al., 2019)	72.1
Albert (Lan et al., 2020) (ensemble)	76.5
RoBERTa + FreeLB (Zhu et al., 2020) (ensemble)	73.1
RoBERTa + HyKAS (Ma et al., 2019)	73.2
RoBERTa + KE (ensemble)	73.3
RoBERTa + KEDGN (ensemble)	74.4
XLNet + GraphReason (Lv et al., 2020)	75.3
RoBERTa + MHGRN (Feng et al., 2020)	75.4
Albert + PG (Wang et al., 2020b)	75.6
RoBERTa + QA-GNN (Yasunaga et al., 2021)	76.1
RoBERTa + JointLK (Ours)	76.6
The results on CommonsenseQA in-house split dataset and official test dataset are shown in Table 1 and Table 2. The results on OpenBookQA test dataset and leaderboard are shown in Table 3 and Table 4. We can observe that JointLK performs best among all fine-tuned LMs and existing LM+KG models. On CommonsenseQA, our model's test performance improves by  $5.74\%$  over fine-tuned LMs and  $1.02\%$  over the prior best LM+KG model, QA-GNN. On OpenbookQA, our model's test performance improves by  $6.52\%$  over fine-tuned AristotleRoBERTa, and  $2.15\%$  over QA-GNN. Additionally, we also submit our best model to the leaderboards, and our JointLK (with the text encoder being RoBERTa-large) ranks first among comparable approaches. Compared with the previous best model MHGRN and QA-GNN, the boost over them suggests the effectiveness of our proposed joint reasoning between LM and KG and the dy
Table 2: Performance comparison on the Common-senseQA official leaderboard. Our model has achieved state-of-the-art under the setting of RoBERTa-large.
Methods	RoBERTa-large	AristoRoBERTa
Fine-tuned LMs (w/o KG)	64.80 (±2.37)	78.40 (±1.64)
+ RGCN	62.45 (±1.57)	74.60 (±2.53)
+ GconAttn	64.75 (±1.48)	71.80 (±1.21)
+ RN	65.20 (±1.18)	75.35 (±1.39)
+ MHGRN	66.85 (±1.19)	80.6
+ QA-GNN	67.80 (±2.75)	82.77 (±1.56)
+ JointLK (Ours)	70.34 (±0.75)	84.92 (±1.07)
Table 3: Test accuracy on OpenBookQA. Methods with AristoRoBERTa use the textual evidence by Clark et al. (2020) as an additional input to the QA context.
Methods	Test
Careful Selection (Banerjee et al., 2019)	72.0
AristoRoBERTa	77.8
KF + SIR (Banerjee and Baral, 2020)	80.0
AristoRoBERTa + PG (Wang et al., 2020b)	80.2
AristoRoBERTa + MHGRN (Feng et al., 2020)	80.6
ALBERT + KB	81.0
AristoRoBERTa + QA-GNN (Yasunaga et al., 2021)	82.8
T5* (Raffel et al., 2020)	83.2
UnifiedQA(11B)* (Khashabi et al., 2020)	87.2
AristoRoBERTa + JointLK (Ours)	85.6