SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Hyeong Kyu Choi *
Computer Sciences University of Wisconsin-Madison hyeongkyu.choi@wisc.edu
Jaewon Chu
Computer Science & Engineering Korea University allonsy07@korea.ac.kr
Seunghun Lee
Computer Science & Engineering Korea University 11sshh319@korea.ac.kr
Hyunwoo J. Kim†
Computer Science & Engineering Korea University hyunwoojkim@korea.ac.kr
Multi-hop Knowledge Graph Question Answering (KGQA) is a task that involves retrieving nodes from a knowledge graph (KG) to answer natural language questions. Recent GNN-based approaches formulate this task as a KG path searching problem, where messages are sequentially propagated from the seed node towards the answer nodes. However, these messages are past-oriented, and they do not consider the full KG context. To make matters worse, KG nodes often represent proper noun entities and are sometimes encrypted, being uninformative in selecting between paths. To address these problems, we propose Neural Tree Search (NuTrea), a tree search-based GNN model that incorporates the broader KG context. Our model adopts a message-passing scheme that probes the unreached subtree regions to boost the past-oriented embeddings. In addition, we introduce the Relation Frequency-Inverse Entity Frequency (RF-IEF) node embedding that considers the global KG context to better characterize ambiguous KG nodes. The general effectiveness of our approach is demonstrated through experiments on three major multi-hop KGQA benchmark datasets, and our extensive analyses further validate its expressiveness and robustness. Overall, NuTrea provides a powerful means to query the KG with complex natural language questions. Code is available at https://github.com/mlvlab/NuTrea.
The knowledge graph (KG) is a multi-relational data structure that defines entities in terms of their relationships. Given its enormous size and complexity, it has long been a challenge to properly query the KG via human languages [1-6]. A corresponding machine learning task is knowledge graph question answering (KGQA), which entails complex reasoning on the KG to retrieve the nodes that correctly answers the given natural language question. To resolve the task, several approaches focused on parsing the natural language to a KG-executable form [7-10], whereas others tried to process the KG so that answer nodes can be ranked and retrieved [11-14, 2]. Building on these works, there has been a recent stream of research focusing on answering more complex questions with intricate constraints, which demand multi-hop reasoning on the KG.
*Work done at Korea University.
Answering complex questions on the KG requires processing both the KG nodes (entities) and edges (relations). Recent studies have addressed multi-hop KGQA by aligning the question text with KG edges (relations), to identify the correct path from seed nodes (i.e., nodes that represent the question subjects) towards answer nodes. Many of these methods, however, gradually expand the search area outwards via message passing, whose trailing path information is aggregated onto the nodes, resulting in node embeddings that are past-oriented. Also, as many complex multi-hop KGQA questions require selecting nodes that satisfy specific conditions, subgraph-level (subtree-level) comparisons are necessary in distinguishing the correct path to the answer node. Furthermore, KG node entities often consist of uninformative proper nouns, and sometimes, for privacy concerns, they may be encrypted [15]. To address these issues, we propose Neural Tree Search (NuTrea), a graph neural network (GNN) model that adopts a tree search scheme to consider the broader KG contexts in searching for the path towards the answer nodes.
NuTrea leverages expressive message passing layers that propagate subtree-level messages to explicitly consider the complex question constraints in identifying the answer node. Each message passing layer consists of three steps, Expansion  $\rightarrow$  Backup  $\rightarrow$  Node Ranking, whose Backup step probes the unreached subtree regions to boost the past-oriented embeddings with future information. Moreover, we introduce the Relation Frequency-Inverse Entity Frequency (RF-IEF) node embedding, which takes advantage of the global KG statistics to better characterize the KG node entities. Overall, NuTrea provides a novel approach in addressing the challenges of querying the KG, by allowing it to have a broader view of the KG context in aligning it with human language questions. The general effectiveness of NuTrea is evaluated on three major multi-hop KGQA benchmark datasets: WebQuestionsSP [16], ComplexWebQuestions [17], and MetaQA [18].
Then, our contributions are threefold:
- We propose Neural Tree Search (NuTrea), an effective GNN model for multi-hop KGQA, which adopts a tree search scheme with an expressive message passing algorithm that refers to the future-oriented subtree contexts in searching paths towards the answer nodes.
- We introduce Relation Frequency-Inverse Entity Frequency (RF-IEF), a simple node embedding technique that effectively characterizes uninformative nodes using the global KG context.
- We achieve the state-of-the-art on multi-hop KGQA datasets, WebQuestionsSP and ComplexWebQuestions, among weakly supervised models that do not use ground-truth logical queries.
A knowledge graph (KG) is a type of a heterogeneous graph [19, 20]  $\mathcal{G} = (\mathcal{V},\mathcal{E})$ , whose edges in  $\mathcal{E}$  are each assigned to a relation type by the mapping function  $f:\mathcal{E}\to \mathcal{R}$ . The KGs contain structured information from commonsense knowledge [15, 21] to domain-specific knowledge [22], and the Knowledge Graph Question Answering (KGQA) task aims to answer natural language questions grounding on these KGs by selecting the set of answer nodes. Recent methods challenge on more complex questions that require multi-hop graph traversals to arrive at the correct answer node. Thus, the task is aliased as multi-hop KGQA or complex KGQA, which is generally discussed in terms of two mainstream approaches [23]: Semantic Parsing and Information Retrieval.
Semantic Parsing. The main idea of semantic parsing-based methods is to first parse natural language questions into a logical query. The logical query is then grounded on the given KG in an executable form. For example, [8] applies the semantic query graph generated by natural language questions. In replace of hand-crafted query templates, [7] introduced a framework that automatically learns the templates from the question-answer pairs. Also, [24] proposed a novel graph generation method for query structure prediction. Other methods take a case-based reasoning (CBR) approach, where previously seen questions are referenced to answer a complex question. Approaches like [9, 25, 26] use case-based reasoning by referring to similar questions or KG structures. Recently, [10] proposed a framework that jointly infers logical forms and direct answers to reduce semantic errors in the logical query. A vast majority of methods that take the semantic parsing approach utilize the ground-truth logical forms or query executions during training. Thus, these supervised methods are generally susceptible to incomplete KG settings, but have high explainability.
Information Retrieval. Information Retrieval-based methods focus on processing the KG to retrieve the answer nodes. The answer nodes are selected by ranking the subgraph nodes conditioned on the given natural language question. One of the former works [27] proposes an enhanced Key-Value
Memory neural network to answer more complex natural language questions. [28] and [29] extract answers from question-specific subgraphs generated with text corpora. To deal with incompleteness and sparsity of KG, [11] presents a KG embedding method to answer questions. Also, [12] handles this problem by executing query in the latent embedding space. In an effort to improve explainability of the information retrieval approach, [13] infers an adjacency matrix by learning the activation probability of each relation type. By adapting the teacher-student framework with the Neural State Machine (NSM), [14] made learning more stable and efficient. Furthermore, [30] utilized multi-task learning that jointly trains on the KG completion task and multi-hop KGQA, while [2] provides a novel link prediction framework. Recently, there has been a trend of borrowing the concept of logical queries from Semantic Parsing approaches, attempting to learn the logical instructions that guide the path search on the KG [2, 1]. Our NuTrea also builds on these approaches.
In recent studies, models that sequentially process the knowledge graph (KG) from the seed nodes have shown promising results on the KGQA task. Building upon this approach, we propose Neural Tree Search (NuTrea). NuTrea adopts a novel message passing scheme that propagates the broader subtree-level information between adjacent nodes, which provides more context in selecting nodes that satisfy the complex question constraints. Additionally, we introduce a node embedding technique called Relation Frequency-Inverse Entity Frequency (RF-IEF), which considers the global KG information when initializing node features. These methods allow for a richer representation of each node by leveraging the broader KG context in answering complex questions on the KG.
Here, we first define the problem settings for Neural Tree Search (NuTrea). The multi-hop KGQA task is primarily a natural language processing problem that receives a human language question  $x_{q}$  as input, and requires retrieving the set of nodes from  $\mathcal{G} = (\mathcal{V},\mathcal{E})$  that answer the question. Following the standard protocol in KGQA [11], the subject entities in  $x_{q}$  are given and assumed always to be mapped to a node in  $\mathcal{V}$  via entity-linking algorithms [31]. These nodes are called seed nodes, denoted  $\nu_{s}\in \mathcal{V}_{s}$ , which are used to extract a subgraph  $\mathcal{G}_q = (\mathcal{V}_q,\mathcal{E}_q)$  from  $\mathcal{G}$  so that  $\mathcal{V}_q$  is likely to contain answer nodes  $\nu_{a}\in \mathcal{V}_{a}$ . Then, the task reduces to a binary node classification problem of whether each node  $\nu \in \mathcal{V}_q$  satisfies  $\nu \in \mathcal{V}_a$ .
Following prior works [14, 1], we first compute different question representation vectors with a language model based module, called Instruction Generators (IG). We have two IG modules, each for the Expansion (section 3.2.1) and Backup (section 3.2.2) step, to compute
$$
\left\{\mathbf {q} _ {\exp} ^ {(i)} \right\} _ {i = 1} ^ {N} = \operatorname {I G} _ {\exp} \left(x _ {q}\right), \quad \left\{\mathbf {q} _ {\mathrm {b a k}} ^ {(j)} \right\} _ {j = 1} ^ {M} = \operatorname {I G} _ {\mathrm {b a k}} \left(x _ {q}\right), \tag {1}
$$
where we name  $\mathbf{q}_{\mathrm{exp}}^{(i)}, \mathbf{q}_{\mathrm{bak}}^{(j)} \in \mathbb{R}^D$  as the expansion instruction and backup instruction, respectively. Detailed description on the IG module is in the supplement. Then, the learnable edge (relation) type embeddings  $R \in \mathbb{R}^{|\mathcal{R}| \times D}$  are randomly initialized or computed with a pretrained language model. On the other hand, node embeddings  $H \in \mathbb{R}^{|\mathcal{V}_q| \times D}$  of  $\mathcal{V}_q$  are initialized using the edges in  $\mathcal{E}_q$  and their relation types by function  $\mathcal{H}$  as  $H = \mathcal{H}(\mathcal{E}_q, R)$  (e.g.,  $\mathcal{H} \equiv$  arithmetic mean of incident edge relations). This is because the node entities are often uninformative proper nouns or encrypted codes. Then, our NuTrea model  $\mathcal{F}$  function is defined as
$$
\hat {\mathbf {y}} = \mathcal {F} \left(\left\{\mathbf {q} _ {\exp} ^ {(i)} \right\} _ {i = 1} ^ {N}, \left\{\mathbf {q} _ {\mathrm {b a k}} ^ {(j)} \right\} _ {j = 1} ^ {M}, \mathcal {V} _ {s}; \mathcal {G} _ {q}, \boldsymbol {H}, \boldsymbol {R}\right), \tag {2}
$$
where  $\hat{\mathbf{y}}\in \mathbb{R}^{|\mathcal{V}_q|}$  is the predicted node score vector normalized across nodes in  $\mathcal{V}_q$ , whose groundtruth labels are  $\mathbf{y} = [\mathbb{1}(\nu \in \mathcal{V}_a)]_{\nu \in \mathcal{V}_q}\in \mathbb{B}^{|\mathcal{V}_q|}$ . Then, the model is optimized with the KL divergence loss between  $\hat{\mathbf{y}}$  and  $\mathbf{y}$ . In this work, we claim our contributions in  $\mathcal{F}$  (section 3.2) and  $\mathcal{H}$  (section 3.3).
3.2 Neural Tree Search  $(\mathcal{F})$
Our Neural Tree Search (NuTrea) model consists of multiple layers that each performs message passing in three consecutive steps: (1) Expansion, (2) Backup, and (3) Node ranking. The Expansion step propagates information outwards to expand the search tree, which is followed by the Backup step, where the depth-  $K$  subtree content is aggregated to each of their root nodes, to enhance the past-oriented messages from the Expansion step. Then, the nodes are scored based on how likely they answer the given question.
Q. What school did Martin Luther King Jr. receive his doctoral degree from?
$$
\begin{array}{l} \mathbf {q} _ {\text {e x p}} = [ \text {e d u c a t i o n , i n s t i t u t e ,} \dots ] \\ \mathbf {q} _ {\text {b a k}} = [ \text {d o c t o r , d e g r e e ,} \dots ] \end{array}
$$
1. RF-IEF Node Embedding
previous methods w/o Backup
2. Neural Tree Search
Figure 1: Neural Tree Search. Given a natural language question, a corresponding KG subgraph  $\mathcal{G}_q$  is extracted, and expansion instructions  $\mathbf{q}_{\mathrm{exp}}$  and backup instructions  $\mathbf{q}_{\mathrm{bak}}$  are computed with Instruction Generators (IG). The node embeddings of  $\mathcal{G}_q$  are first defined by our RF-IEF (section 3.3), which characterizes nodes based on the global KG information by suppressing prevalent relation types that hold less meaning. Then, in each NuTrea layer (section 3.2), messages are propagated outwards from the seed node with respect to  $\mathbf{q}_{\mathrm{exp}}$  (Expansion), which contain future-oriented subtree information conditioned by  $\mathbf{q}_{\mathrm{bak}}$  (Backup). Then, the nodes are scored by adding and normalizing the logits (Node Ranking), which are utilized in the subsequent layer. The figure describes the first layer of NuTrea whose Backup subtree depth  $K$  is 2, and  $\phi$  is the softmax function. Overall, NuTrea considers the broader KG contexts in distinguishing the correct paths, contrary to previous methods that do not incorporate the Backup step.
3.2.1 Expansion
Starting from the seed node  $\nu_{s} \in \mathcal{V}_{s}$ , a NuTrea layer first expands the search tree by sequentially propagating messages outwards to the adjacent nodes. The propagated messages  $\mathbf{f}_{uv}^{(i)}$  are conditioned on the expansion instructions  $\{\mathbf{q}_{\mathrm{exp}}^{(i)}\}_{i=1}^{N}$ , which are computed as
$$
\mathbf {f} _ {u v} ^ {(i)} = \operatorname {R e L U} \left(W _ {\mathbf {f}} \mathbf {r} _ {u v} \odot \mathbf {q} _ {\exp} ^ {(i)}\right), \tag {3}
$$
where  $i \in [1, N]$ ,  $\mathbf{W}_{\mathbf{f}} \in \mathbb{R}^{D \times D}$  is a learnable linear projection,  $\mathbf{r}_{uv} = \operatorname{row}(\mathbf{R}) \in \mathbb{R}^D$  is the relation type embedding of edge  $u \to v$ , and  $\odot$  is an element-wise product operator. Optionally, we use the relative position embedding  $\mathbf{e}_{uv} \in \mathbb{R}^D$  as
$$
\mathbf {f} _ {u v} ^ {(i)} = \operatorname {R e L U} \left(\left(W _ {\mathbf {f}} \mathbf {r} _ {u v} + \mathbf {e} _ {u v}\right) \odot \mathbf {q} _ {\exp} ^ {(i)}\right), \tag {4}
$$
where  $\mathbf{e}_{uv}$  is defined for each relation type. Then, for an edge  $u\rightarrow \nu$ $N$  types of messages are propagated, which are element-wise products between the edge relation and the  $N$  different question representation vectors. This operation highlights the edges that are relevant to the given question. Then, the messages are aggregated to a node  $\nu$  via an MLP aggregator, computed as
$$
\tilde {\mathbf {f}} _ {v} = \left\| _ {i = 1} ^ {N} \sum_ {u \in \mathcal {N} (v)} s _ {u} \mathbf {f} _ {u v} ^ {(i)} \right. \tag {5}
$$
$$
\mathbf {f} _ {\nu} = \operatorname {M L P} \left(\mathbf {h} _ {\nu} \| \tilde {\mathbf {f}} _ {\nu}\right),
$$
where  $\mathbf{h}_v = \mathrm{row}(\pmb {H})\in \mathbb{R}^D$  is the node embedding and  $s_u\in \mathbb{R}$  is the score value of a head node  $u$ . In the first layer, the seed nodes are the only head nodes, whose scores are initially set to 1. In subsequent layers, we use the updated node scores as  $s_u$ , whose computation will be introduced shortly in section 3.2.3. Notably, the nodes with score  $s_u = 0$ , which are typically nodes that are yet to be reached, do not pass any message to their neighbors.
3.2.2 Backup
After the Expansion step grows the search tree, the leaf nodes of the tree naturally contain the trailing path information from the seed nodes, which is past-oriented. To provide future context, we employ
the Backup step to aggregate contextual information from subtrees rooted at the nodes reached by previous NuTrea layers. We denote a subtree of depth  $K$  rooted at node  $\nu$  as  $\mathcal{T}_{\nu}^{K} = (\mathcal{V}_{\nu},\mathcal{E}_{\nu})\subset \mathcal{G}_{q} = (\mathcal{V}_{q},\mathcal{E}_{q})$ . Here,  $\mathcal{V}_{\nu} = \{u\mid \mathrm{SP}(u,\nu)\leq K\text{and} u\in \mathcal{V}_{q}\}$  and  $\mathcal{E}_{\nu} = \{(u_{1},u_{2})\mid (u_{1},u_{2})\in \mathcal{E}$  and  $u_{1},u_{2}\in \mathcal{V}_{\nu}\}$ , where  $\mathrm{SP}(u,\nu)$  is a function that returns the length of the shortest path between nodes  $u$  and  $\nu$ . For the Backup step, we consider only the edge set  $\mathcal{E}_{\nu}$  of  $\mathcal{T}_{\nu}^{K}$ . The reason behind this is that the edges (relation types) better represent the question context in guiding the search on the KG. Also, using both the node and edge sets may introduce computational redundancy, as the initial node features originate from the edge embeddings [14, 1] (See section 3.1).
To pool the constraint information from  $\mathcal{E}_v$ , we apply max-pooling conditioned on the question content. Specifically, we take a similar measure as the Expansion step by computing  $M$  types of messages conditioned on the backup instructions  $\{\mathbf{q}_{\mathrm{bak}}^{(j)}\}_{j = 1}^{M}$ .
$$
\mathbf {c} _ {u _ {1} u _ {2}} ^ {(j)} = \operatorname {R e L U} \left(\boldsymbol {W} _ {\mathbf {c}} \boldsymbol {r} _ {u _ {1} u _ {2}} \odot \mathbf {q} _ {\text {b a k}} ^ {(j)}\right), \tag {6}
$$
where  $j\in [1,M]$  and  $\pmb{W_{\mathbf{c}}}\in \mathbb{R}^{D\times D}$ . Then, we max-pool the messages as
$$
\mathbf {c} _ {v} ^ {(j)} = \operatorname {M A X - P O O L} \left(\left\{\mathbf {c} _ {u _ {1} u _ {2}} ^ {(j)} \mid (u _ {1}, u _ {2}) \in \mathcal {E} _ {v} \right\}\right), \tag {7}
$$
which represents the extent to which the local subtree context of  $\nu$  is relevant to the conditions and constraints in the question. Next, the information is aggregated with an MLP layer, and the node embedding  $\mathbf{h}_{\nu}$  is updated as
$$
\mathbf {c} _ {v} = \left\| _ {j = 1} ^ {M} \mathbf {c} _ {v} ^ {(j)} \right. \tag {8}
$$
$$
\mathbf {h} _ {\nu} := \operatorname {M L P} \left(\mathbf {f} _ {\nu} \| \mathbf {c} _ {\nu}\right),
$$
where  $\mathbf{f}_{\nu}$  refers to the propagated embeddings originating from Eq. (5). This serves as a correction of the original past-oriented message with respect to question constraints, providing the next NuTrea layer with rich local context.
3.2.3 Node Ranking
Finally, each node is scored and ranked based on the embeddings before and after the Backup step. For node  $\nu$ ,  $\mathbf{f}_{\nu}$  from Eq. (5) and  $\mathbf{h}_{\nu}$  from Eq. (8) are projected to the expansion-score  $s_{\nu}^{(e)}$  and backup-score  $s_{\nu}^{(b)}$ , respectively, as
$$
s _ {v} ^ {(e)} = \mathbf {f} _ {v} \cdot W _ {e} \quad s _ {v} ^ {(b)} = \mathbf {h} _ {v} \cdot W _ {b}, \tag {9}
$$
where  $W_{e}, W_{b} \in \mathbb{R}^{D}$ . The final node score is retrieved by adding the two scores. We use a context coefficient  $\lambda$  to control the effect of the Backup step, and apply softmax to normalize the scores as
$$
s _ {v} = \operatorname {S o f t m a x} \left(\left[ s _ {v} ^ {(e)} + \lambda \cdot s _ {v} ^ {(b)} \right]\right) _ {v \in \mathcal {V} _ {q}}, \tag {10}
$$
which is passed on to the next layer to be used for Eq. (5). Figure 1 provides a holistic view of our method, and pseudocode is in the supplement.
Overall, the message passing scheme of NuTrea resembles the algorithm of Monte Carlo Tree Search (MCTS): Selection  $\rightarrow$  Expansion  $\rightarrow$  Simulation  $\rightarrow$  Backup. The difference is that our method replaces the node 'Selection' and 'Simulation' steps with a soft GNN-based approach that rolls out subtrees and updates the nodes at once, rather than applying Monte Carlo sampling methods.
3.3 RF-IEF Node Embedding  $(\mathcal{H})$
Another notable challenge with KGs is embedding nodes. Many KG entities (nodes) are proper nouns that are not informative, and several KGQA datasets [16, 17] consist of encrypted entity names. Thus, given no proper node features, the burden is on the model layers to learn meaningful node embeddings. To alleviate this, we propose a novel node embedding method, Relation Frequency-Inverse Entity Frequency (RF-IEF), which grounds on the local and global topological information of the KG nodes.
Term frequency-inverse document frequency (TF-IDF) is one effective feature that characterizes a textual document [32-34]. The bag-of-words model computes the frequency of terms in a document and penalizes frequent but noninformative words, e.g., 'a', 'the', 'is', 'are', by the frequency of the
Figure 2: Expressiveness of NuTrea layers. In the input KG subgraph (left-most figure), let node 1 and 2 be the seed and answer node, respectively, where information  $d$  is critical in choosing node 2 as the answer. The shaded nodes indicate the regions that are sequentially reached by consecutive GNN layers. Compared to (a) previous methods [14, 1] that require 3 message passing steps for node 2 to access  $d$ , our (b) NuTrea layer requires only 1 step for node 2 to acquire information  $d$  to determine it as the answer.
term across documents. Motivated by the idea, we represent a node on a KG as a bag of relations. An entity node is characterized by the frequencies of rare (or informative) relations. Similar to TF-IDF, we define two functions: Relation Frequency (RF) and Inverse Entity Frequency (IEF). The RF function is defined for node  $\nu \in \mathcal{V}_q$  and relation type  $r\in \mathcal{R}$  as
$$
\operatorname {R F} (v, r) = \sum_ {e \in \mathcal {I} (v)} \mathbb {1} \{f (e) = r \}, \tag {11}
$$
where  $\mathcal{I}(\nu)$  is the set of incident edges of node  $\nu$ ,  $\mathbb{1}$  is an indicator function, and  $f: \mathcal{E} \to \mathcal{R}$  is a function that retrieves the relation type of an edge. Then, the output of the RF function is a matrix  $RF \in \mathbb{R}^{|\mathcal{V}_q| \times |\mathcal{R}|}$  that counts the occurrence of each relation type incident to each node in the KG subgraph. We used raw counts for relation frequency (RF) to reflect the local degree information of a node. On the other hand, the IEF function is defined as
$$
\operatorname {I E F} (r) = \log \frac {\left| \mathcal {V} _ {q} \right|}{1 + \operatorname {E F} (r)}, \tag {12}
$$
$$
\operatorname {E F} (r) = \sum_ {v \in \mathcal {V} _ {q}} \mathbb {1} \left\{\exists e \in \mathcal {I} (v) \text {s . t .} f (e) = r \right\}. \tag {13}
$$
EF counts the global frequency of nodes across KG subgraphs that have relation  $r$  within its incident edge set. With  $IEF \in \mathbb{R}^{|\mathcal{R}|}$ , the RF-IEF matrix  $F \in \mathbb{R}^{|\mathcal{V}_q| \times |\mathcal{R}|}$  is computed as
$$
\boldsymbol {F} = R F \operatorname {d i a g} (I E F), \tag {14}
$$
where  $\mathrm{diag}(IEF)\in \mathbb{R}^{|\mathcal{V}_q|\times |\mathcal{R}|}$  denotes a diagonal matrix constructed with the elements of  $IEF$ .