SlowGuess/NAACL_2022_OCR huggingface dataset:
Yu Wang Vijay Srinivasan Hongxia Jin
Samsung Research America
{yu.wang1,v.srinivasan,hongxia.jin}@samsung.com
Knowledge based question answering (KBQA) is a complex task for natural language understanding. Many KBQA approaches have been proposed in recent years, and most of them are trained based on labeled reasoning path. This hinders the system's performance as many correct reasoning paths are not labeled as ground truth, and thus they cannot be learned. In this paper, we introduce a new concept of KBQA system which can leverage multiple reasoning paths' information and only requires labeled answer as supervision. We name it as Mutiple Reasoning Paths KBQA System (MRPQA). We conduct experiments on several benchmark datasets containing both single-hop simple questions as well as muti-hop complex questions, including WebQuestionSP (WQSP), ComplexWebQuestion-1.1 (CWQ), and PathQuestion-Large (PQL), and demonstrate strong performance.
Knowledge-based question answering (KBQA) is the task of finding answers to questions by processing a structured knowledge base  $\mathcal{K}\mathcal{B}$ . A  $\mathcal{K}\mathcal{B}$  consists of a set of entities  $\mathcal{E}$ , a set of relations  $\mathcal{R}$ , and a set of literals  $\mathcal{S}$ . A knowledge base fact is defined as  $(h, r, t)$ , where  $h \in \mathcal{E}$  is the head entity,  $t \in \mathcal{E} \cup \mathcal{S}$  is the tail entity/literal, and  $r \in \mathcal{R}$  is the directed relation between  $h$  and  $t$ . To answer a simple single-relation question (i.e. a 1-hop question) such as: "Who is the president of the United States?", a typical KBQA system first identifies the entity (i.e. United States) and the relation (i.e. "president") asked in the question, and then searches for the answer entity by matching the entity-relation tuple <United States, president,?> over  $\mathcal{K}\mathcal{B}$ .
While a single-hop question can be answered by searching a predicate relation in  $\kappa \mathcal{B}$ , it is much
harder to answer more complex multi-hop questions containing multiple entities and relations with constraints. For instance, for complex compositional questions, it is not easy to extract all the relations correctly together with their head and tail entities in the right order. For complex conjunction questions that requires a conjunction of multiple evidences, it is even more difficult to correctly extract all the reasoning paths included.
Most prior works on multi-hop KBQA focus on learning a single given ground truth reasoning path for each question, and outputting the most possible reasoning path during prediction (Zhou et al., 2018; Zhang et al., 2018; Yu et al., 2018; Lan et al., 2019). However, it is common that  $\mathcal{KB}$  has many alternative paths leading to the correct answer, of various reasoning qualities. These alternative reasoning paths are usually not provided as ground truth by the human annotators.
For example, Figure 1 shows 7 reasoning paths  $\mathbf{p}^n = e_0^n\rightarrow r_1^n\rightarrow e_1^n\rightarrow \dots \rightarrow e_{ans}^n$  ( $n = \{1,\ldots ,7\}$ ) leading to an answer set containing the correct answer "West Lafayette" for a given question "What city is home to the University that is known for Purdue Boilermakers men's basketball?", but only the reasoning path  $\mathbf{p}^1$  is labeled as the correct path in the dataset. A model trained with only  $\mathbf{p}^1$  as supervision is likely to miss other paths which are also valid. For example, it will probably map a similar question "What city is home to the stadium that is known for Los Angeles Lakers?" to path  $\mathbf{p}^1$ , but fail to associate it with  $\mathbf{p}^3$  or  $\mathbf{p}^4$ , because  $\mathbf{p}^3$  or  $\mathbf{p}^4$  contain different types of relations. However,  $\mathbf{p}^1$  is a wrong reasoning path for that test question.
As the example shown in Figure 1, there are four paths  $(\mathbf{p}^1,\mathbf{p}^2,\mathbf{p}^3,\mathbf{p}^4)$  pointing to the exact answer set containing only the answer entity, and thus can be treated as ground truth paths when training. Comparatively, reasoning paths  $\mathbf{p}^5$  and  $\mathbf{p}^6$  lead to a larger final entity set containing the correct answer
Human Language Technologies, pages 4007 - 4017
Q: What city is home to the University that is known for Purdue Boilermakers men's basketball?
Figure 1: One QA example with Multiple Reasoning Paths (MRPs) from COMPLEXWEBQUESTION-1.1. The blue color highlighted is the extracted topic entity. Each square represents an entity, and the arrows represent the relations. Reasoning path  $\mathbf{p}^1$  to  $\mathbf{p}^4$  are the correct ones containing meaningful reasoning paths to the final answer.  $\mathbf{p}^5$  and  $\mathbf{p}^6$  are the "second choice" paths that generate a larger final answer set containing some wrong entities.  $\mathbf{p}^7$  is the wrong one as its reasoning path is totally not interpretable and the answer set is huge.
"West Lafayette" but also other entities. These two paths can be considered as inferior to the top 4 paths; however, it is still worth including them in the training as a "second choice", as it is not difficult to extract the correct answer from final sets by additional post-processing. For example, a simple filter can be applied to filter out "United States of America" and "Indiana" from the predicted set, as they are not cities. Path  $\mathbf{p}^7$  is bad because it is not interpretable, in addition to the final answer set being exaggeratedly large with invalid answers. Hence, path  $\mathbf{p}^7$  should not be considered as a training path for this question. Unfortunately, it is not possible for any existing models to use multiple good/inferior paths, but not the bad ones, since current models are only trained with a single path for each question answer pair.
In this paper, we propose a novel multi-hop KBQA system, which can leverage the training information from multiple reasoning paths (MRPs)
without using any path annotations, we name it as Mutiliple Reasoning Paths KBQA System (MRP-QA). We model the reasoning path as a latent variable, and propose supporting training and prediction methods. The system can output diverse reasoning paths, and reward the "better" paths over the inferior ones by assigning "better" paths higher probabilities. Our method can be applied to most KBQA systems to predict the answer, and can be used with any model architecture. We achieve strong performance on three popular KBQA datasets. Experimental results show that our model performs especially well on multi-hop question, and in particular on complex questions that cannot be solved with a single reasoning path.
Our MRP-QA System does not need training paths annotation (only the question, and head and final entities), since it can sample the paths from the  $\kappa \mathcal{B}$  graph. This is of enormous practical importance, because in practice questions and answers are easy to collect (sometimes for free), but path annotation is very labor-intensive and expensive.
We first introduce some notations. For a given question  $q$  and its topic entity  $e_0$  (identified by entity linking tool), a reasoning path is a sequence in the form  $\mathbf{p} = (e_0, r_1, e_1, r_2, \dots, e_{T-1}, r_T)$  that points to the answer entity  $e_T = y$ . That is,  $\mathbf{p} \rightarrow e_T = y$ . Each step  $(e_{t-1}, r_t, e_t)$  is a valid fact in the knowledge base  $\mathcal{KB}$ . Our goal is to build a model that can use multiple paths  $\mathbf{p}$  to predict answer  $y$  given question  $q$  and topic entity  $e_0$ . In this section, we first present the design of our model architecture, and then explain the training and inference algorithms in detail.
2.1 MRP-QA Model Architecture
Figure 2 illustrates the architecture of our MRPQA model. We model path probabilities using recurrent neural network with gated recurrent units (GRU). At a timestep  $t$ , the input hidden representations of GRU unit and predicted relation are denoted by  $h_{t-1}$  and  $r_t$  respectively. The model relies on the attention mechanism (Bahdanau et al., 2015) to produce a question context vector  $c_t$ . Specifically, all the words  $w_0, w_1, \dots, w_{|q|-1}$  in the given question  $q$  are first sent to a fixed embedding layer to acquire word embeddings  $\varepsilon_w(w_0), \varepsilon_w(w_1), \dots, \varepsilon_w(w_{|q|-1})$ . Next we apply GRU to produce a temporary hidden state  $h_t' =$
Figure 2: An illustration of how our MRP-QA model works with a QA pair "What city is fc_schalke_04 in?" and Gelesenkirchen. The entity linker extracts fc_schalke_04 as the topic entity. We only show one possible paths here:  $r_1$  is Organization.headquarters and  $r_2$  is Mailing_address.city_town, our model can be used to output the probability of this given path. The symbol  $\oplus$  represents concatenation, and  $\otimes$  represents knowledge base lookup.
$GRU(h_{t-1}, r_{t-1})$ , and then apply a parameterized feed-forward neural network  $a$  to calculate the similarity score  $u_{tk} = a(h_t', \varepsilon_w(w_k))$  of two inputs  $h_t'$  and  $\varepsilon_w(w_k)$ , and then these scores are normalized into attention weights  $\alpha_{tk} = \frac{\exp(u_{tk})}{\sum_{0 \leq j \leq |q| - 1} \exp(u_{tj})}$ , which are used to produce the question context vector  $c_t = \sum_{0 \leq j \leq |q| - 1} \alpha_{tj} \varepsilon_w(w_j)$ . In this fashion, word embeddings are combined in different ways based on attention weights to show different reasoning focuses at each timestep.
The model then concatenates temporary hidden state  $h_t'$ , entity representation  $\varepsilon_e(e_{t-1})$ , and question context  $c_t$  together, and passes the concatenation through a linear transformation  $f$  with ReLU activation to obtain the hidden state  $h_t = ReLU(f([h_t'; \varepsilon_e(e_{t-1}); c_t]))$ . This process is recurrently done until the model predicts a stop symbol  $<eop>^1$ . Note that the vanilla RNN attention model only has  $h_t'$  and  $c_t$  when calculates  $h_t$ . We add entity representation into the calculation, since entity captures important information in the reasoning path.
2.2 Probabilities and Objective Function
The probability of predicting the  $k$ -th relation  $\gamma_{k}$  in  $\mathcal{R}$  at timestep  $t$  is:
$$
\begin{array}{l} p (r _ {t} = \gamma_ {k} | q, e _ {0}, r _ {1}, \dots , e _ {t - 1}) \\ = \frac {\exp <   h _ {t} , \varepsilon_ {r} (\gamma_ {k}) >}{\sum_ {j} \exp <   h _ {t} , \varepsilon_ {r} (\gamma_ {j}) >} \\ \end{array}
$$
where  $\varepsilon_r$  is the embedding function,  $<<$  is the dot product between two inputs.
Given the previous entity  $e_{t-1}$  and relation  $r_t$ , the next matched entity may not be unique when we query the knowledge base. For example, if  $e_{t-1} = \text{"united states" }$ , and  $r_t = \text{"president of" }$ , then the resulting entity has 45 possibilities. Since we do not have additional constraints, all of them are equally likely to be selected, and hence we define:
$$
\begin{array}{l} p \left(e _ {t} \mid e _ {t - 1}, r _ {t}\right) \tag {1} \\ = \left\{ \begin{array}{l l} 1 / M & \text {i f e t i s o n e o f t h e M m a t c h e d e n t i t i e s} \\ 0 & \text {i f e t i s n o t a m a t c h e d e n t i t y} \end{array} \right. \\ \end{array}
$$
Thus the probability of a path containing both entities and relations can be computed using the chain rule:
$$
p (\mathbf {p} | q)
$$
$$
= \prod_ {t = 1} ^ {T - 1} p \left(e _ {t} \mid e _ {t - 1}, r _ {t}\right) \prod_ {t = 1} ^ {T} p \left(r _ {t} \mid q, e _ {0}, r _ {1}, \dots , e _ {t - 1}\right) \tag {2}
$$
<sup>1</sup>This stop mechanism is the same as how it works in a vanilla RNN. Similarly, we also attach  $\langle \mathrm{sop} \rangle$  to the beginning of each sequence to denote the start state. We will omit these symbols in formulas for simplicity.
We assume that there are multiple valid paths  $\mathbf{p} \in \mathcal{P}$  that can lead to the correct answer  $y$  and they are not given by the annotator in the dataset. We treat these paths as hidden variables and we marginalize them out to compute the probability of getting the answer  $y$ :
$$
\begin{array}{l} p (y | q) \\ = \sum_ {\mathbf {p} \in \mathcal {P}} [ p (e _ {T (\mathbf {p})} = y | \mathbf {p}, q) p (\mathbf {p} | q) ] \\ = \sum_ {\mathbf {p} \in \mathcal {P}} \prod_ {t = 1} ^ {T (\mathbf {p})} \left[ p \left(e _ {t} \mid e _ {t - 1}, r _ {t}\right) p \left(r _ {t} \mid q, e _ {0}, r _ {1}, \dots , e _ {t - 1}\right) \right] \tag {3} \\ \end{array}
$$
where  $\mathcal{P}$  is a set of all valid paths leading to the answer  $y$ , and  $T(\mathbf{p})$  is the number of hops in the path  $\mathbf{p}$ .
To train our model, we would like to maximize the answer probability  $p(y|q)$  using only the given answer for each training instance. To make prediction on each test case, we would like to find the answer  $y$  with the highest probability.
It is a novel way that we define answer probability as in (3) in the KBQA task. Most of the existing methods assume the availability of a single ground truth path annotation and aim to maximize the probability of the given path (Zhou et al., 2018). As we will demonstrate later in the Section 3.3, considering multiple paths leads to a better model performance.
2.3 System Training
In order to train our MRP-QA model by maximizing the marginalized answer probability given in (3), it requires summing over all valid reasoning paths from the topic entity to the answer entity in knowledge base. Thus computing this objective exactly can be intractable. As shown in the early example, some reasoning paths  $(R_{5}, R_{6}, R_{7}$  in Figure 1) are not very helpful for training, thus should be either removed from training or assigned low probabilities. To achieve this goal, we first apply depth first search (DFS) algorithm with maximum 3 hops to get valid path candidates. The algorithm starts the traversal from the topic entity node, and ends at the answer entity node. All possible paths between the topic entity and the answer entity within 3 hops are extracted as candidates. We then set a threshold to remove paths which point to too many
entities at the last hop. To further filter out bad reasoning paths, we propose to dynamically choose reasoning paths deemed as most probable by the current model during training. The overall training procedure is summarized in Algorithm 1. Note that training with this algorithm does not require ground truth reasoning path label. Labeled reasoning path is a plus, but not necessary. If it is given, we can either include the ground truth paths in  $\mathcal{P}$ , or use them to initialize model training.
Algorithm 1: Training method for a MRP-QA System
Input :KBQA dataset
$$
\left(q ^ {n}, y ^ {n}, e _ {0} ^ {n}\right), n = 1, 2, \dots , N,
$$
Knowledge Base  $\kappa \mathcal{B}$
Threshold  $k_{1}$  and  $k_{2}$ .
Output:Trained model parameters
1 foreach instance  $(q^n,y^n,e_0^n)$  do
2 Use DFS algorithm to get a set of paths  $\mathcal{P}^n$  from  $e_0^n$  to  $y^n$ .
3 Remove from  $\mathcal{P}^n$  paths that point to more than  $k_{1}$  entities.
5 foreach batch do
6 | foreach  $(q^n, y^n, e_0^n)$  in the batch do
7 Get top  $k_{2}$  paths in  $\mathcal{P}$  sorted by  $p(\mathbf{p}|q)$  based on current model:
$$
\mathcal {P} ^ {n} = \left\{\mathbf {p} _ {1} ^ {n}, \dots , \mathbf {p} _ {k _ {2}} ^ {n} \right\}
$$
8 end
9 Update model parameters by maximizing
$$
\sum_ {\left(q ^ {n}, y ^ {n}, \epsilon_ {0} ^ {n}\right)} \log \sum_ {\mathbf {p} \in \widetilde {\mathcal {P}} ^ {n}} p \left(y ^ {n} \mid \mathbf {p}, q ^ {n}\right) P (\mathbf {p} \mid q ^ {n})
$$
10 end
2.4 Prediction
During the prediction, we aim to select the answer  $y$  with the highest marginalized probability  $p(y|q)$  as defined in (3). Similar to training, we need to approximate the sum with selected paths from  $\mathcal{P}$ . We use a modified beam search to find paths that have high probabilities. We add two constraints to standard beam search to only select the valid paths that match the knowledge base: (1) The first relation  $r_1$  should connect to the topic entity  $e_0$ . (2) Each triple  $(e_{t-1}, r_t, e_t)$  should match a fact in KB. Given the set of paths collected as above, we can then collect a set of candidate answers that these paths point to. For each answer  $y$ , we evaluate
its probability  $p(y|q)$  approximately using the collected paths, and among them we output the answer with the highest probability.
Additionally, we observe that it could be beneficial to de-emphasize the impact of the topic entity during prediction, as noted in (Li et al., 2016), which can improve inference performance by avoiding generating generic predictions and reducing overfitting. Specifically, instead of searching  $y^{*}$  that maximizes  $p(y|q)$ , we can find an answer that maximizes  $\frac{p(y|q)}{p(y|e_0)}$ , where  $p(y|e_0)$  is the probability of getting the answer  $y$  when the question only contains the topic entity word. Mathematically, one can show that this is equivalent to maximizing the point-wise conditional mutual information  $\mathrm{PMI}((y;q\backslash e_0)|e_0)$  between  $y$  and  $q\backslash e_0$  given  $e_0$  where  $q\backslash e_0$  stands for the question with the topic entity term removed. Further discussion can be found in Section 4.
3 Results and Analysis
We conduct experiments on 3 multi-hop KBQA datasets, WEBQUESTIONSP (WQSP) (Yih et al., 2015), COMPLEXWEBQUESTION-1.1 (CWQ) (Talmor and Berant, 2018), and PATHQUESTION-LARGE (PQL) (Zhou et al., 2018), and use the original train/dev/test split. WQSP is a dataset that has been widely used for relation extraction and end-to-end KBQA tasks, which contains 1 or 2 hops questions. CWQ dataset is designed to study complex questions by adding more constraints to questions in WEBQUESTIONSP. PQL is a small dataset used to study sequential questions. Its original release contains two subsets: PQL2H and PQL3H, which contains only 2-hop and 3-hop questions correspondingly. Chen et al. (2019) then combined these two subsets and renamed the unified dataset as  $\mathrm{PQL + }$ . All of the three datasets use Freebase (Google, 2013) as the supporting knowledge base. Table 1 contains statistics of these datasets.
#train	#valid	#test	max_hops	>1 path
WQSP	2677	297	1639	2	79.4%
CWQ	27639	3519	3531	6	83.4%
PQL2H	1275	159	160	2	12.5%
PQL3H	1649	206	207	3	45.2%
PQL+	2924	365	367	3	30.6%
Table 1: Statistics of datasets. To count the data percentage with more than one path, i.e.  $> 1$  path,we use graph search algorithm to calculate what percentage of QA pairs can be solved with MRPs.
For questions with multiple answers, we use each answer to construct a question-answer (QA) pair. For WQSP and CWQ, we build a subgraph in a similar way as in (Sun et al., 2018), in order to generate the entity and relation candidates. For PQL, the original paper provides a subgraph of the Freebase. We implement our model using TENSORFLOW-1.11.0 and choose S-MART (Yang and Chang, 2016) and AllenNLP (Gardner et al., 2017) as our entity linking tools. If multiple topic entities are extracted, we use each topic entity to construct a question-answer pair. We test three different graph embedding methods WORD2VEC (Mikolov et al., 2013), TRANSE (Bordes et al., 2013), and HOLE (Trouillon and Nickel, 2017), and decide to use TRANSE in our final experiment based on validation performance. The threshold  $k_{1}$  is set to be: 15 plus the number of answers in the ground truth answer set, and  $k_{2}$  is top  $50\%$ . We adopt the average F1 score and the set accuracy as our main evaluation metrics. It is worth noticing that: except our methods' results, all other experimental results are obtained from early published papers. Details of these models can be found from our referenced papers.
In Table 2 we compare our MRP-QA method to state-of-the-art models. All comparisons are divided into two groups based on different training supervisions. The upper block shows methods that are only trained with final answer as supervision, and the second block contains methods using extra annotations such as parsing results of the query. Experimental results show that our MRP-QA model performs better than all other methods on WQSP.
Also although NSM and  $NSM_{+h}$  (2nd best result) only relies on answers to train their model, it requires many prior knowledges, such as a big vocabulary to train word embeddings and graph embeddings, type label of the entity and of the relation, and pre-defined templates. The experiments from their papers show that these knowledge play a very important role in the system, e.g. F1 score drops from 69.0 to 60.7 by not using the pretrained embeddings for NSM.
In contrast, our model supports a training method that takes only raw QA pairs and the facts in knowledge base, and does not rely on any additional labels and pre-defined knowledge.
To further disentangle the contribution of differ
WQSP	CWQ
STAGG_SP (Yih et al., 2016)	71.7	-
HR-BiLSTM (Yu et al., 2017)	62.3	31.2
KBQA-GST (Lan et al., 2019)	67.9	36.5
KV-MemNN* (Miller et al., 2016)	38.6	-
STAGG_answer* (Yih et al., 2016)	66.8	-
NSM* (Liang et al., 2017)	69.0	-
GRAFT-Net* (Sun et al., 2018)	62.8	26.0
PullNet*(Sun et al., 2019)	68.1	47.2
TransferNet*(Shi et al., 2021)	71.4	48.6
NSM+_h*(He et al., 2021)	74.3	48.8
MRP-QA-marginal_prob*	74.9	49.9
Table 2: We report F1  $(\%)$  on WQSP and CWQ test sets. Methods labeled with  $*$  only require the final answer as the supervision, and they are directly comparable to our MRP-QA model. As references, We also report the performance of methods that requires extra supervisions in the first block.
Setting	Δ F1 (std)
MRP-QA – entity_in_RNN	-2.1 (0.21)
MRP-QA – marginal_prediction	-1.8 (0.32)
MRP-QA – inference_in_training	-3.4 (0.15)
MRP-QA – mutual_information	-1.8 (0.16)
Table 3: Feature ablation study on the dev set with a mean of 5 runs.
ent factors in our MRP-QA method, we present a feature ablation test on WQSP dataset shown in Table 3. The vanilla RNN structure only maintains a hidden state and the previous prediction in the loop. Here, we show the performance boost by considering entity features in KBQA task. Instead of using greedy algorithm or beam search to output the top prediction with the highest joint probability  $P(y,\mathbf{p})$ , we propose to make the prediction based on marginalized probability  $P(y)$ , which also improves the performance by  $1.8\%$ . In addition, we show the benefits of using inference during training (line 6 and 7 in algorithm 1) and mutual information objective (Section (2.4)). More discussions can be found in the Section 4.
3.3 Choices of paths
In the second set of experiment, we test our MRPQA model with different objective functions and compare their results correspondingly. The objective functions are as defined in Table 4, where the paths used for training are given in the last column. The detailed explanations are given as following:
Single ground truth path. When one reasoning path is given for each QA pair in addition to the answer, we can train the model to fit the
given path and answer by maximizing  $p(y, \mathbf{p} | q) = p(y | \mathbf{p}, q) p(\mathbf{p} | q)$ . This objective ignores the fact that MRPs could be valid for the same answer (see Figure 1) and pushes all the probability mass to the single given one.
Single random path. Many existing methods require a ground truth path for each question in order to train the model. When only the ground truth answer but no path is given to each question, one can randomly sample a path that leads to the given answer and treat the sampled path as ground truth for training.