Title: NuTrea: Neural Tree Search for Context-guided Multi-hop KGQA
Venue: NeurIPS 2023
Accepted: yes
Keywords: Knowledge Graph Question Answering

Knowledge Graph

Graph Neural Networks
Abstract: Multi-hop Knowledge Graph Question Answering (KGQA) is a task that involves retrieving nodes from a knowledge graph (KG) to  answer natural language questions. Recent GNN-based approaches formulate this task as a KG path searching problem, where messages are sequentially propagated from the seed node towards the answer nodes. However, these messages are past-oriented, and they do not consider the full KG context. To make matters worse, KG nodes often represent pronoun entities and are sometimes encrypted, being uninformative in selecting between paths. To address these problems, we propose Neural Tree Search (NuTrea), a tree search-based GNN model that incorporates the broader KG context. Our model adopts a message-passing scheme that probes the unreached subtree regions to boost the past-oriented embeddings. In addition, we introduce the Relation Frequency-Inverse Entity Frequency (RF-IEF) node embedding that considers the global KG context to better characterize ambiguous KG nodes. The general effectiveness of our approach is demonstrated through experiments on three major multi-hop KGQA benchmark datasets, and our extensive analyses further validate its expressiveness and robustness. Overall, NuTrea provides a powerful means to query the KG with complex natural language questions. Code is available at https://github.com/mlvlab/NuTrea.

NuTrea: Neural Tree Search for Context-guided Multi-hop KGQA

Abstract

Multi-hop Knowledge Graph Question Answering (KGQA) is a task that involves retrieving nodes from a knowledge graph (KG) to answer natural language questions. Recent GNN-based approaches formulate this task as a KG path searching problem, where messages are sequentially propagated from the seed node towards the answer nodes. However, these messages are past-oriented, and they do not consider the full KG context. To make matters worse, KG nodes often represent proper noun entities and are sometimes encrypted, being uninformative in selecting between paths. To address these problems, we propose Neural Tree Search (NuTrea), a tree search-based GNN model that incorporates the broader KG context. Our model adopts a message-passing scheme that probes the unreached subtree regions to boost the past-oriented embeddings. In addition, we introduce the Relation Frequency-Inverse Entity Frequency (RF-IEF) node embedding that considers the global KG context to better characterize ambiguous KG nodes. The general effectiveness of our approach is demonstrated through experiments on three major multi-hop KGQA benchmark datasets, and our extensive analyses further validate its expressiveness and robustness. Overall, NuTrea provides a powerful means to query the KG with complex natural language questions. Code is available at https://github.com/mlvlab/NuTrea.

1 Introduction

The knowledge graph (KG) is a multi-relational data structure that defines entities in terms of their relationships. Given its enormous size and complexity, it has long been a challenge to properly query the KG via human languages [1-6]. A corresponding machine learning task is knowledge graph question answering (KGQA), which entails complex reasoning on the KG to retrieve the nodes that correctly answers the given natural language question. To resolve the task, several approaches focused on parsing the natural language to a KG-executable form [7-10], whereas others tried to process the KG so that answer nodes can be ranked and retrieved [11-14, 2]. Building on these works, there has been a recent stream of research focusing on answering more complex questions with intricate constraints, which demand multi-hop reasoning on the KG.

Answering complex questions on the KG requires processing both the KG nodes (entities) and edges (relations). Recent studies have addressed multi-hop KGQA by aligning the question text with KG edges (relations), to identify the correct path from seed nodes ( i.e ., nodes that represent the question subjects) towards answer nodes. Many of these methods, however, gradually expand the search area outwards via message passing, whose trailing path information is aggregated onto the nodes, resulting in node embeddings that are past-oriented. Also, as many complex multi-hop KGQA questions require selecting nodes that satisfy specific conditions, subgraph-level (subtreelevel) comparisons are necessary in distinguishing the correct path to the answer node. Furthermore, KG node entities often consist of uninformative proper nouns, and sometimes, for privacy concerns, they may be encrypted [15]. To address these issues, we propose N e u ral Tr ee S ea rch (NuTrea), a graph neural network (GNN) model that adopts a tree search scheme to consider the broader KG contexts in searching for the path towards the answer nodes.

NuTrea leverages expressive message passing layers that propagate subtree-level messages to explicitly consider the complex question constraints in identifying the answer node. Each message passing layer consists of three steps, Expansion → Backup → Node Ranking, whose Backup step probes the unreached subtree regions to boost the past-oriented embeddings with future information. Moreover, we introduce the Relation Frequency-Inverse Entity Frequency (RF-IEF) node embedding, which takes advantage of the global KG statistics to better characterize the KG node entities. Overall, NuTrea provides a novel approach in addressing the challenges of querying the KG, by allowing it to have a broader view of the KG context in aligning it with human language questions. The general effectiveness of NuTrea is evaluated on three major multi-hop KGQA benchmark datasets: WebQuestionsSP [16], ComplexWebQuestions [17], and MetaQA [18].

Then, our contributions are threefold:

We propose Neural Tree Search (NuTrea), an effective GNN model for multi-hop KGQA, which adopts a tree search scheme with an expressive message passing algorithm that refers to the future-oriented subtree contexts in searching paths towards the answer nodes.

We introduce Relation Frequency-Inverse Entity Frequency (RF-IEF), a simple node embedding technique that effectively characterizes uninformative nodes using the global KG context.

We achieve the state-of-the-art on multi-hop KGQA datasets, WebQuestionsSP and ComplexWebQuestions, among weakly supervised models that do not use ground-truth logical queries.

2 Related Works

A knowledge graph (KG) is a type of a heterogeneous graph [19, 20] G = (V , E) , whose edges in E are each assigned to a relation type by the mapping function 𝑓 : E → R . The KGs contain structured information from commonsense knowledge [15, 21] to domain-specific knowledge [22], and the Knowledge Graph Question Answering (KGQA) task aims to answer natural language questions grounding on these KGs by selecting the set of answer nodes. Recent methods challenge on more complex questions that require multi-hop graph traversals to arrive at the correct answer node. Thus, the task is aliased as multi-hop KGQA or complex KGQA, which is generally discussed in terms of two mainstream approaches [23]: Semantic Parsing and Information Retrieval.

Semantic Parsing. The main idea of semantic parsing-based methods is to first parse natural language questions into a logical query. The logical query is then grounded on the given KG in an executable form. For example, [8] applies the semantic query graph generated by natural language questions. In replace of hand-crafted query templates, [7] introduced a framework that automatically learns the templates from the question-answer pairs. Also, [24] proposed a novel graph generation method for query structure prediction. Other methods take a case-based reasoning (CBR) approach, where previously seen questions are referenced to answer a complex question. Approaches like [9, 25, 26] use case-based reasoning by referring to similar questions or KG structures. Recently, [10] proposed a framework that jointly infers logical forms and direct answers to reduce semantic errors in the logical query. A vast majority of methods that take the semantic parsing approach utilize the ground-truth logical forms or query executions during training. Thus, these supervised methods are generally susceptible to incomplete KG settings, but have high explainability.

Information Retrieval. Information Retrieval-based methods focus on processing the KG to retrieve the answer nodes. The answer nodes are selected by ranking the subgraph nodes conditioned on the given natural language question. One of the former works [27] proposes an enhanced Key-Value

Memory neural network to answer more complex natural language questions. [28] and [29] extract answers from question-specific subgraphs generated with text corpora. To deal with incompleteness and sparsity of KG, [11] presents a KG embedding method to answer questions. Also, [12] handles this problem by executing query in the latent embedding space. In an effort to improve explainability of the information retrieval approach, [13] infers an adjacency matrix by learning the activation probability of each relation type. By adapting the teacher-student framework with the Neural State Machine (NSM), [14] made learning more stable and efficient. Furthermore, [30] utilized multi-task learning that jointly trains on the KG completion task and multi-hop KGQA, while [2] provides a novel link prediction framework. Recently, there has been a trend of borrowing the concept of logical queries from Semantic Parsing approaches, attempting to learn the logical instructions that guide the path search on the KG [2, 1]. Our NuTrea also builds on these approaches.

3 Method

In recent studies, models that sequentially process the knowledge graph (KG) from the seed nodes have shown promising results on the KGQA task. Building upon this approach, we propose Neural Tree Search (NuTrea). NuTrea adopts a novel message passing scheme that propagates the broader subtree-level information between adjacent nodes, which provides more context in selecting nodes that satisfy the complex question constraints. Additionally, we introduce a node embedding technique called Relation Frequency-Inverse Entity Frequency (RF-IEF), which considers the global KG information when initializing node features. These methods allow for a richer representation of each node by leveraging the broader KG context in answering complex questions on the KG.

3.1 Problem Definition

Here, we first define the problem settings for Neural Tree Search (NuTrea). The multi-hop KGQA task is primarily a natural language processing problem that receives a human language question 𝑥 𝑞 as input, and requires retrieving the set of nodes from G = (V , E) that answer the question. Following the standard protocol in KGQA [11], the subject entities in 𝑥 𝑞 are given and assumed always to be mapped to a node in V via entity-linking algorithms [31]. These nodes are called seed nodes, denoted 𝑣 𝑠 ∈ V 𝑠 , which are used to extract a subgraph G 𝑞 = (V 𝑞 , E 𝑞 ) from G so that V 𝑞 is likely to contain answer nodes 𝑣 𝑎 ∈ V 𝑎 . Then, the task reduces to a binary node classification problem of whether each node 𝑣 ∈ V 𝑞 satisfies 𝑣 ∈ V 𝑎 .

Following prior works [14, 1], we first compute different question representation vectors with a language model based module, called Instruction Generators (IG). We have two IG modules, each for the Expansion (section 3.2.1) and Backup (section 3.2.2) step, to compute

where we name q ( 𝑖 ) exp , q ( 𝑗 ) bak ∈ R 𝐷 as the expansion instruction and backup instruction, respectively. Detailed description on the IG module is in the supplement. Then, the learnable edge (relation) type embeddings 𝑹 ∈ R | R | × 𝐷 are randomly initialized or computed with a pretrained language model. On the other hand, node embeddings 𝑯 ∈ R | V 𝑞 | × 𝐷 of V 𝑞 are initialized using the edges in E 𝑞 and their relation types by function H as 𝑯 = H(E 𝑞 , 𝑹 ) ( e.g ., H ≡ arithmetic mean of incident edge relations). This is because the node entities are often uninformative proper nouns or encrypted codes. Then, our NuTrea model F function is defined as

where ˆ y ∈ R | V 𝑞 | is the predicted node score vector normalized across nodes in V 𝑞 , whose groundtruth labels are y = [ ✶ ( 𝑣 ∈ V 𝑎 )] 𝑣 ∈V 𝑞 ∈ B | V 𝑞 | . Then, the model is optimized with the KL divergence loss between ˆ y and y . In this work, we claim our contributions in F (section 3.2) and H (section 3.3).

3.2 Neural Tree Search ( F )

Our Neural Tree Search (NuTrea) model consists of multiple layers that each performs message passing in three consecutive steps: (1) Expansion, (2) Backup, and (3) Node ranking. The Expansion step propagates information outwards to expand the search tree, which is followed by the Backup step, where the depth𝐾 subtree content is aggregated to each of their root nodes, to enhance the past-oriented messages from the Expansion step. Then, the nodes are scored based on how likely they answer the given question.

= [education, institute, ...]

receive his doctoral degree from?

IG

qbak = [doctor, degree, ..·]

1. RF-IEF Node Embedding

2. Neural Tree Search

related_to

deliver_speech

Expansion

1.0 education 1.0

related_to

at_place

education

O

related_to

1.0

at_place

0.8

degree

master

O

RF-IEF lo

hi

Backup

institute

O

previous methods w/o Backup

seednode

(qbak)

degree

-0.2

institute

1.0

0.3

Φ(1.0+ 0.8)=0.6

1.0 education

0.3

Node

education

answernode

0.2

Ranking

1.0

0.3

Expansion

Backup

Φ(1.0-0.2)=0.2

3.2.1 Expansion

Starting from the seed node 𝑣 𝑠 ∈ V 𝑠 , a NuTrea layer first expands the search tree by sequentially propagating messages outwards to the adjacent nodes. The propagated messages f ( 𝑖 ) 𝑢𝑣 are conditioned on the expansion instructions { q ( 𝑖 ) exp } 𝑁 𝑖 = 1 , which are computed as

where 𝑖 ∈ [ 1 , 𝑁 ] , 𝑾 f ∈ R 𝐷 × 𝐷 is a learnable linear projection, r 𝑢𝑣 = row ( 𝑹 ) ∈ R 𝐷 is the relation type embedding of edge 𝑢 → 𝑣 , and ⊙ is an element-wise product operator. Optionally, we use the relative position embedding e 𝑢𝑣 ∈ R 𝐷 as

where e 𝑢𝑣 is defined for each relation type. Then, for an edge 𝑢 → 𝑣 , 𝑁 types of messages are propagated, which are element-wise products between the edge relation and the 𝑁 different question representation vectors. This operation highlights the edges that are relevant to the given question. Then, the messages are aggregated to a node 𝑣 via an MLP aggregator, computed as

where h 𝑣 = row ( 𝑯 ) ∈ R 𝐷 is the node embedding and 𝑠 𝑢 ∈ R is the score value of a head node 𝑢 . In the first layer, the seed nodes are the only head nodes, whose scores are initially set to 1. In subsequent layers, we use the updated node scores as 𝑠 𝑢 , whose computation will be introduced shortly in section 3.2.3. Notably, the nodes with score 𝑠 𝑢 = 0 , which are typically nodes that are yet to be reached, do not pass any message to their neighbors.

3.2.2 Backup

After the Expansion step grows the search tree, the leaf nodes of the tree naturally contain the trailing path information from the seed nodes, which is past-oriented. To provide future context, we employ

the Backup step to aggregate contextual information from subtrees rooted at the nodes reached by previous NuTrea layers. We denote a subtree of depth 𝐾 rooted at node 𝑣 as T 𝐾 𝑣 = (V 𝑣 , E 𝑣 ) ⊂ G 𝑞 = (V 𝑞 , E 𝑞 ) . Here, V 𝑣 = { 𝑢 | SP ( 𝑢, 𝑣 ) ≤ 𝐾 and 𝑢 ∈ V 𝑞 } and E 𝑣 = {( 𝑢 1 , 𝑢 2 ) | ( 𝑢 1 , 𝑢 2 ) ∈ E and 𝑢 1 , 𝑢 2 ∈ V 𝑣 } , where SP ( 𝑢, 𝑣 ) is a function that returns the length of the shortest path between nodes 𝑢 and 𝑣 . For the Backup step, we consider only the edge set E 𝑣 of T 𝐾 𝑣 . The reason behind this is that the edges (relation types) better represent the question context in guiding the search on the KG. Also, using both the node and edge sets may introduce computational redundancy, as the initial node features originate from the edge embeddings [14, 1] (See section 3.1).

To pool the constraint information from E 𝑣 , we apply max-pooling conditioned on the question content. Specifically, we take a similar measure as the Expansion step by computing 𝑀 types of messages conditioned on the backup instructions { q ( 𝑗 ) bak } 𝑀 𝑗 = 1 .

where 𝑗 ∈ [ 1 , 𝑀 ] and 𝑾 c ∈ R 𝐷 × 𝐷 . Then, we max-pool the messages as

which represents the extent to which the local subtree context of 𝑣 is relevant to the conditions and constraints in the question. Next, the information is aggregated with an MLP layer, and the node embedding h 𝑣 is updated as

where f 𝑣 refers to the propagated embeddings originating from Eq. (5). This serves as a correction of the original past-oriented message with respect to question constraints, providing the next NuTrea layer with rich local context.

3.2.3 Node Ranking

Finally, each node is scored and ranked based on the embeddings before and after the Backup step. For node 𝑣 , f 𝑣 from Eq. (5) and h 𝑣 from Eq. (8) are projected to the expansion-score 𝑠 ( 𝑒 ) 𝑣 and backup-score 𝑠 ( 𝑏 ) 𝑣 , respectively, as

where 𝑊 𝑒 , 𝑊 𝑏 ∈ R 𝐷 . The final node score is retrieved by adding the two scores. We use a context coefficient 𝜆 to control the effect of the Backup step, and apply softmax to normalize the scores as

which is passed on to the next layer to be used for Eq. (5). Figure 1 provides a holistic view of our method, and pseudocode is in the supplement.

Overall, the message passing scheme of NuTrea resembles the algorithm of Monte Carlo Tree Search (MCTS): Selection → Expansion → Simulation → Backup. The difference is that our method replaces the node 'Selection' and 'Simulation' steps with a soft GNN-based approach that rolls out subtrees and updates the nodes at once, rather than applying Monte Carlo sampling methods.

3.3 RF-IEF Node Embedding ( H )

Another notable challenge with KGs is embedding nodes. Many KG entities (nodes) are proper nouns that are not informative, and several KGQA datasets [16, 17] consist of encrypted entity names. Thus, given no proper node features, the burden is on the model layers to learn meaningful node embeddings. To alleviate this, we propose a novel node embedding method, Relation Frequency-Inverse Entity Frequency (RF-IEF), which grounds on the local and global topological information of the KG nodes.

Term frequency-inverse document frequency (TF-IDF) is one effective feature that characterizes a textual document [32-34]. The bag-of-words model computes the frequency of terms in a document and penalizes frequent but noninformative words, e.g., 'a', 'the', 'is', 'are', by the frequency of the

{a}

{a}

{a,b,c}

{a,b,c}

{a}

①

{b}

{c}

{a,b}

{a,c}

{a,b}

{a,c}

{a,b,c,d}

a,b,c,e} {a, b, d}

{a,c,e}

2

3

{d}

{e}

{d}

{e}

{a,b,d}

{a,c,e}

{a,b,d}

{a,c,e}

{d}

{e}

<input>

<layer 1>

<layer 2>

<layer 3>

<layer 1>

(a)

(b)

term across documents. Motivated by the idea, we represent a node on a KG as a bag of relations. An entity node is characterized by the frequencies of rare (or informative) relations. Similar to TF-IDF, we define two functions: Relation Frequency (RF) and Inverse Entity Frequency (IEF). The RF function is defined for node 𝑣 ∈ V 𝑞 and relation type 𝑟 ∈ R as

where I( 𝑣 ) is the set of incident edges of node 𝑣 , 1 is an indicator function, and 𝑓 : E → R is a function that retrieves the relation type of an edge. Then, the output of the RF function is a matrix 𝑅𝐹 ∈ R | V 𝑞 | × | R | that counts the occurrence of each relation type incident to each node in the KG subgraph. We used raw counts for relation frequency (RF) to reflect the local degree information of a node. On the other hand, the IEF function is defined as

where

EF counts the global frequency of nodes across KG subgraphs that have relation 𝑟 within its incident edge set. With 𝐼𝐸𝐹 ∈ R | R | , the RF-IEF matrix 𝑭 ∈ R | V 𝑞 | × | R | is computed as

where diag ( 𝐼𝐸𝐹 ) ∈ R | V 𝑞 | × | R | denotes a diagonal matrix constructed with the elements of 𝐼𝐸𝐹 .

The RF-IEF matrix 𝑭 captures both the local and global KG structure and it can be further enhanced with the rich semantic information on the edges of KGs. Unlike entity nodes with uninformative text, e.g., proper nouns and encrypted entity names, edges generally are accompanied by linguistic descriptions (relation types). Hence, the relations are commonly embedded by a pre-trained language model in KGQA. Combining with the relation embeddings 𝑹 ∈ R | R | × 𝐷 , our final RF-IEF node embeddings 𝑯 is computed as

where 𝑯 ∈ R | V 𝑞 | × 𝐷 , and 𝑾 ℎ ∈ R 𝐷 × 𝐷 . The RF-IEF node embeddings can be viewed as the aggregated semantics of relations, which are represented by a language model, based on graph topology as well as the informativeness (or rareness) of relations. A row in 𝑯 is a node embedding vector h 𝑣 , which is used in Eq. (5) at the first NuTrea layer.

3.4 Discussion

Expressiveness of the NuTrea layer. While many previous multi-hop KGQA models simultaneously update all node embeddings, recent works [14, 2, 1] have shown the superiority of approaches that search paths on the KG. These methods gradually expand the searching area by sequentially

updating nodes closer to the seed node towards the answer nodes. Our model builds on the latter sequential search scheme as well, enhancing expressiveness with our proposed NuTrea layers.

With a simple toy example in Figure 2, we compare the message flow of previous sequential search models and our NuTrea. The example demonstrates that our NuTrea layer (b) can probe subtrees to quickly gather fringe node ( i.e ., node 4 or 5) information without exhaustively visiting them. This is accomplished by our Backup step, which boosts the original past-oriented node embeddings with future-oriented subtree information.

4 Experiments

4.1 Experimental Settings

Datasets. We experiment on three large-scale multi-hop KGQA datasets: MetaQA [35], WebQuestionsSP (WQP) [16] and ComplexWebQuestions (CWQ) [17]. Meta-QA consists of three different splits, 1-hop, 2-hop, and 3-hop, each indicating the number of hops required to reach the answer node from the seed node. The questions are relatively easy with less constraints. WQP and CWQ, on the other hand, contain more complex questions with diverse constraints. WQP is relatively easier, as CWQis derived from WQP by extending its questions with additional constraints. MetaQA is answerable with the WikiMovies knowledge base [27], while WQP and CWQ require the Freebase KG [15] to answer questions. Further dataset information and statistics are provided in the supplement.

Baselines. We mainly compare with previous multi-hop KGQA methods that take the Information Retrieval approach (section 2). These models, unlike Semantic Parsing approaches, do not access the ground truth logical queries and focus on processing the KG subgraph to rank the nodes to identify answer nodes. To introduce the three most recent baseline models: (1) SQALER [3] proposes a scalable KGQA method whose complexity scales linearly with the number of relation types, rather than nodes or edges. (2) TERP [2] introduces the rotate-and-scale entity link prediction framework to integrate textual and KG structural information. (3) ReaRev [1] adaptively selects the next reasoning step with a variant of breadth-first search (BFS). Other baselines are introduced in the supplement.

Implementation Details. For WQP, 2 NuTrea layers with subtree depth 𝐾 = 1 is used, while CWQ with more complex questions uses 3 layers with depth 𝐾 = 2 . In the case of RF-IEF node embedding, we pre-compute the Entity Frequency (EF) values in Eq. (13) for subgraphs in the training set before training. We use the same EF values throughout training, validation, and testing. This stabilizes computation by mitigating the large variance induced by relatively small batch sizes. For MetaQA, the number of NuTrea layers are selected from {2, 3}, and 𝐾 for ego-graph pooling from {1,2}. See the supplement for further hyperparameter settings and details.

4.2 Main Experiments

Here, we present the experimental results of NuTrea. Following the common evaluation practice of previous works, we test the model that achieved the best performance on the validation set. In the WQP dataset experiments in Table 1, we achieved the best performance of 77.4 H@1 among strong KGQA baselines that take an information retrieval approach, as discussed in Section 2. Compared to the previous best, this is a large improvement of 0.6 points. In terms of the F1 score, which evaluates the answer set prediction, our method achieved a score of 72.7, exceeding the previously recorded value by a large margin of 1.8 points. In addition, we also improved the previous state-of-the-art performance on the CWQ dataset by achieving an 53.6 H@1, which is an improvement of 0.7 points.

We also experimented NuTrea on MetaQA to see if it performs reasonably well with easy questions as well. On three data splits, NuTrea achieved comparable performance with previous state-of-the-art methods for simple question answering. Evaluating with the average H@1 score of the three splits, NuTrea performs second best among all baseline models.

4.3 Incomplete KG Experiments

The KG is often human-made and the contents are prone to being incomplete. Hence, it is a norm to test a model's robustness on incomplete KG settings where a certain portion of KG triplets, i.e . , a tuple of ⟨ head, relation, tail ⟩ , are dropped. This experiment evaluates the robustness of our model to missing relations in a KG. We follow the experiment settings in [1], and use the identical incomplete KG dataset which consists of WQP samples with [50%, 30%, 10%] of the original KG triplets remaining. In Table 2, NuTrea performs the best in most cases, among the GNN models

RF-IEF

✓

✓

Backup step

✓

✓

WQPH@1

77.4

(-0.0)

74.8 (-2.6)

76.8 (-0.6)

73.4 (-4.0)

WQPF1

72.7

(-0.0)

70.4 (-2.3)

71.5 (-1.2)

70.9 (-1.8)

designed to handle incomplete KGs. We believe that our model adaptively learns multiple alternative reasoning processes and can plan for future moves beforehand via our Backup step, so that it provides robust performance with noisy KGs.

5 Analysis

In this section, we provide comprehensive analyses on the contributions of NuTrea to ensure its effectiveness in KGQA. We try to answer the following research questions: Q1. How does each component contribute to the performance of NuTrea? Q2. What is the advantage of NuTrea's tree search algorithm over previous methods? Q3. What are the effects of the RF-IEF node embeddings?

5.1 Ablation Study

Here, we evaluate the effectiveness of each component in NuTrea to answer Q1 . An ablation study is performed on our two major contributions: the Backup step in our NuTrea layers and the RF-IEF node embeddings. By removing the RF-IEF node embeddings, we instead apply the common node initialization method used in [14, 1], which simply averages the relation embeddings incident to each node. Another option is to use zero embeddings, but we found it worse than the simple averaging method. For NuTrea layer ablation, we remove the Backup step which plays a key role in aggregating the future-oriented subtree information onto the KG nodes. Then, only the expansion-score ( 𝑠 ( 𝑒 ) 𝑣 ) is computed, and the backup-score ( 𝑠 ( 𝑏 ) 𝑣 ) is always 0. Also, the embedding f 𝑣 (Eq. (5)), is directly output from the NuTrea layer and no further updates are made via the Backup step.

In Table 3, we can see that the largest performance drop is observed when the Backup step is removed. Without it, the model has limited access to the broader context of the KG and cannot reflect the complex question constraints in node searching. Further discussion on this property of NuTrea's message passing is provided in the next Section 5.2. Also, we observed a non-trivial 0.6 point drop

Q.Whowas QueenElizabeth II's mom?

(0.0062 = 0.5297)

(0.9936 =0.4678)

Q.WhatwasJamesGarfieldmost knownfor? US President (0.0035=0.3069) Laity (0.2491 = 0.1817)

Q. What country did Gregor Mendel live in?

Brno

(0.0023 = 0.5211)

(0.5535 =0.4755)

Entity Frequency

RF-IEF Weights

Entity Frequency (log)

15

Others

organizations

founded

10

founders

AlexanderBustamante

5

Events

Competed in

competitors

lor

medicine.medical

Others

in H@1 by removing the RF-IEF initialization method. By ablating both components, there was a significant degradation of 4.0 H@1 points.

5.2 Advantages of NuTrea

To answer Q2 , we highlight the key advantages of NuTrea over recent approaches. We analyze the efficiency of NuTrea, and provide qualitative results.

5.2.1 Efficiency of NuTrea

In addition, to further verify the utility of our model, we provide analyses on the latency of NuTrea with WQP in Table 4. Compared to the most recent ReaRev [1] model, training and inference latency per epoch/sample is slightly bigger, due to our additional Backup module. However,

thanks to our expressive message passing scheme, NuTrea converges way faster, allowing the training GPU hours to be reduced from 4.3 hours to 2.9 hours.

To provide more insight in terms of number of NuTrea layers, we also reveal its effect on model performance, in Figure 5. The figure reports the F1 scores for both with and without the Backup module, evaluated on the WebQuestionsSP dataset. 'NuTrea without Backup" has an equivalent model configuration used in the Backup ablation experiment of Table 3. Overall, the performance of 'NuTrea without Backup" does generally improve with additional layers, but 'NuTrea with Backup" reached the highest score of 72.7 with only 2 NuTrea layers. This is enabled by our Backup module, which alleviates the burden of exhaustively searching deeper

73

72.7

72.1

72.7

72

71.6

72.5

71

71.7

ore

71.0

Scor

70

70.4

S

56.7

NuTreaw/Backup

57

NuTreaw/oBackup

56

56.1

1

2

3

4

5

NumberofLayers

nodes. This is computationally more efficient than stacking multiple layers to achieve higher performance. Specifically, by comparing the 2-layer 'NuTrea with Backup" and 5-layer 'NuTrea without Backup", the former required an average of 73.8 ms of inference time per question, whereas the latter required 108.6 ms. With only 68% of compute, our NuTrea achieved comparable performance with the deeper 'NuTrea without Backup". Note, these latency values were evaluated on a different environment from values reported in Table 4.

5.2.2 Qualitative Results

In Figure 3, we demonstrate several qualitative examples from our error analysis. In each question, the blue (node) entity is the correct answer choice, while the red is the wrong choice made by the

model without the Backup step. The values in parentheses demonstrate the difference in node scores between models without and with Backup. Without it, the sequential search model cannot refer to local contexts of a node and frequently predicts an extremely low score for the correct answer node ( e.g ., 0.0062 for 'Queen Elizabeth the Queen Mother" in the first question of Figure 3). Such a problem is mitigated by our NuTrea's Backup step, which tends to boost scores of correct answers and tone down wrong choices that were wrongly assigned with a high score. More qualitative examples are provided in the supplement, along with an analysis on the different importances of the Backup step in different datasets, by controlling the context coefficient 𝜆 .

5.3 Effect of RF-IEF

The RF-IEF node embedding is a simple method inspired by an effective text representation technique in natural language processing. Here, we disclose the specific effect of RF-IEF on the relation embedding aggregation weights, thereby answering Q3 . In Figure 4 (left), the log-scaled 𝐸𝐹 (Eq. 13) value of each relation type is sorted. The globally most frequent relation types, including 'self_loop's, are too general to provide much context in characterizing a KG node. Our RF-IEF suppresses such uninformative relation types for node embedding initialization, resulting in a weight distribution like Figure 4 (right). The pie charts display two examples on the difference between aggregation weights for a node entity before and after RF-IEF is applied. To illustrate, the weight after RF-IEF corresponds to a row of 𝑭 in Eq. (14), while the weight before RF-IEF would be uniform across incident edges. As 'Alexander Bustamante' is a politician, the relation types 'orgainizations founded' and 'founders' become more lucid via RF-IEF, while relations like 'events_competed_in' and 'competitors' are emphasized for an athlete like 'Kemar Bailey-Cole'. Likewise, RF-IEF tends to scale up characteristic relation types in initializing the node features, thereby enhancing differentiability between entities. To further demonstrate RF-IEF's general applicability, we also provide a plug-in experiment on another baseline model in the supplement.

6 Conclusion

Neural Tree Search (NuTrea) is an effective GNN model for multi-hop KGQA, which aims to better capture the complex question constraints by referring to the broader KG context. The high expressiveness of NuTrea is attained via our message passing scheme that resembles the MCTS algorithm, which leverages the future-oriented subtree information conditioning on the question constraints. Moreover, we introduce the RF-IEF node embedding technique to also consider the global KG context. Combining these methods, our NuTrea achieves the state-of-the-art in two major multi-hop KGQA benchmarks, WebQuestionsSP and ComplexWebQuestions. Further analyses on KG incompleteness and the qualitative results support the effectiveness of NuTrea. Overall, NuTrea reveals the importance of considering the broader KG context in harnessing the knowledge graph via human languages.

Acknowledgments and Disclosure of Funding

This work was partly supported by ICT Creative Consilience program (IITP-2023-2020-0-01819) supervised by the IITP, the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (NRF-2023R1A2C2005373), and KakaoBrain corporation.

References

Costas Mavromatis and George Karypis. Rearev: Adaptive reasoning for question answering over knowledge graphs. In EMNLP-Findings , 2022.

Zile Qiao, Wei Ye, Tong Zhang, Tong Mo, Weiping Li, and Shikun Zhang. Exploiting hybrid semantics of relation paths for multi-hop question answering over knowledge graphs. In COLING , 2022.

Mattia Atzeni, Jasmina Bogojeska, and Andreas Loukas. Sqaler: Scaling question answering by decoupling multi-hop and logical reasoning. NeurIPS , 2021.

Jinyoung Park, Hyeong Kyu Choi, Juyeon Ko, Hyeonjin Park, Ji-Hoon Kim, Jisu Jeong, Kyungmin Kim, and Hyunwoo Kim. Relation-aware language-graph transformer for question answering. In AAAI , 2023.

Kuan Wang, Yuyu Zhang, Diyi Yang, Le Song, and Tao Qin. Gnn is a counter? revisiting gnn for question answering. In ICLR , 2021.

X Zhang, A Bosselut, M Yasunaga, H Ren, P Liang, C Manning, and J Leskovec. Greaselm: Graph reasoning enhanced language models for question answering. In ICLR , 2022.

Abdalghani Abujabal, Mohamed Yahya, Mirek Riedewald, and Gerhard Weikum. Automated template generation for question answering over knowledge graphs. In WWW , 2017.

Sen Hu, Lei Zou, and Xinbo Zhang. A state-transition framework to answer complex questions over knowledge base. In EMNLP , 2018.

Rajarshi Das, Manzil Zaheer, Dung Thai, Ameya Godbole, Ethan Perez, Jay Yoon Lee, Lizhen Tan, Lazaros Polymenakos, and Andrew McCallum. Case-based reasoning for natural language queries over knowledge bases. In EMNLP , 2021.

Donghan Yu, Sheng Zhang, Patrick Ng, Henghui Zhu, Alexander Hanbo Li, Jun Wang, Yiqun Hu, William Wang, Zhiguo Wang, and Bing Xiang. Decaf: Joint decoding of answers and logical forms for question answering over knowledge bases. arXiv preprint arXiv:2210.00063 , 2022.

Apoorv Saxena, Aditay Tripathi, and Partha Talukdar. Improving multi-hop question answering over knowledge graphs using knowledge base embeddings. In ACL , 2020.

Hongyu Ren, Hanjun Dai, Bo Dai, Xinyun Chen, Michihiro Yasunaga, Haitian Sun, Dale Schuurmans, Jure Leskovec, and Denny Zhou. Lego: Latent execution-guided reasoning for multi-hop question answering on knowledge graphs. In ICML , 2021.

Jiaxin Shi, Shulin Cao, Lei Hou, Juanzi Li, and Hanwang Zhang. Transfernet: An effective and transparent framework for multi-hop question answering over relation graph. In EMNLP , 2021.

Gaole He, Yunshi Lan, Jing Jiang, Wayne Xin Zhao, and Ji-Rong Wen. Improving multi-hop knowledge base question answering by learning intermediate supervision signals. In WSDM , 2021.

Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. Freebase: a collaboratively created graph database for structuring human knowledge. In ACM SIGMOD , 2008.

Wen-tau Yih, Matthew Richardson, Christopher Meek, Ming-Wei Chang, and Jina Suh. The value of semantic parse labeling for knowledge base question answering. In ACL , 2016.

Alon Talmor and Jonathan Berant. The web as a knowledge-base for answering complex questions. In NAACL , 2018.

Yuyu Zhang, Hanjun Dai, Zornitsa Kozareva, Alexander J Smola, and Le Song. Variational reasoning for question answering with knowledge graph. In AAAI , 2018.

Chuxu Zhang, Dongjin Song, Chao Huang, Ananthram Swami, and Nitesh V Chawla. Heterogeneous graph neural network. In ACM SIGKDD , 2019.

Xiao Wang, Deyu Bo, Chuan Shi, Shaohua Fan, Yanfang Ye, and S Yu Philip. A survey on heterogeneous graph embedding: methods, techniques, applications and sources. IEEE Transactions on Big Data , 2022.

Robyn Speer, Joshua Chin, and Catherine Havasi. Conceptnet 5.5: An open multilingual graph of general knowledge. In AAAI , 2017.

Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences , 11(14):6421, 2021.

Yunshi Lan, Gaole He, Jinhao Jiang, Jing Jiang, Wayne Xin Zhao, and Ji-Rong Wen. A survey on complex knowledge base question answering: Methods, challenges and solutions. In IJCAI , 2021.

Yongrui Chen, Huiying Li, Yuncheng Hua, and Guilin Qi. Formal query building with query structure prediction for complex question answering over knowledge base. In IJCAI , 2020.

Dung Thai, Srinivas Ravishankar, Ibrahim Abdelaziz, Mudit Chaudhary, Nandana Mihindukulasooriya, Tahira Naseem, Rajarshi Das, Pavan Kapanipathi, Achille Fokoue, and Andrew McCallum. Cbr-ikb: A case-based reasoning approach for question answering over incomplete knowledge bases. arXiv preprint arXiv:2204.08554 , 2022.

Rajarshi Das, Ameya Godbole, Ankita Naik, Elliot Tower, Manzil Zaheer, Hannaneh Hajishirzi, Robin Jia, and Andrew McCallum. Knowledge base question answering by case-based reasoning over subgraphs. In ICML . PMLR, 2022.

Alexander Miller, Adam Fisch, Jesse Dodge, Amir-Hossein Karimi, Antoine Bordes, and Jason Weston. Key-value memory networks for directly reading documents. In EMNLP , 2016.

Haitian Sun, Bhuwan Dhingra, Manzil Zaheer, Kathryn Mazaitis, Ruslan Salakhutdinov, and William Cohen. Open domain question answering using early fusion of knowledge bases and text. In EMNLP , 2018.

Haitian Sun, Tania Bedrax-Weiss, and William Cohen. Pullnet: Open domain question answering with iterative retrieval on knowledge bases and text. In EMNLP-IJCNLP , 2019.

Lihui Liu, Boxin Du, Jiejun Xu, Yinglong Xia, and Hanghang Tong. Joint knowledge graph completion and question answering. In ACM SIGKDD , 2022.

Scott Wen-tau Yih, Ming-Wei Chang, Xiaodong He, and Jianfeng Gao. Semantic parsing via staged query graph generation: Question answering with knowledge base. In Proceedings of the Joint Conference of the 53rd Annual Meeting of the ACL and the 7th International Joint Conference on Natural Language Processing of the AFNLP , 2015.

Hans Peter Luhn. The automatic creation of literature abstracts. IBM Journal of research and development , 2(2):159-165, 1958.

Stephen E Robertson and K Sparck Jones. Relevance weighting of search terms. Journal of the American Society for Information science , 27(3):129-146, 1976.

Gerard Salton and Christopher Buckley. Term-weighting approaches in automatic text retrieval. Information processing & management , 24(5):513-523, 1988.

Yuyu Zhang, Hanjun Dai, Zornitsa Kozareva, Alexander J Smola, and Le Song. Variational reasoning for question answering with knowledge graph. In AAAI , 2018.

William W Cohen, Haitian Sun, R Alex Hofer, and Matthew Siegler. Scalable neural methods for reasoning with a symbolic knowledge base. In ICLR , 2020.

Haitian Sun, Andrew Arnold, Tania Bedrax Weiss, Fernando Pereira, and William W Cohen. Faithful embeddings for knowledge base queries. NeurIPS , 2020.

Priyanka Sen, Armin Oliya, and Amir Saffari. Expanding end-to-end question answering on differentiable knowledge graphs with intersection. In EMNLP , 2021.

Wenhan Xiong, Mo Yu, Shiyu Chang, Xiaoxiao Guo, and William Yang Wang. Improving question answering over incomplete kbs with knowledge-aware reader. ACL , 2019.

Jiale Han, Bo Cheng, and Xu Wang. Open domain question answering based on text enhanced knowledge graph with hyperedge infusion. In EMNLP-Findings , 2020.

Peer reviews:
comments: SUMMARY:
This paper proposes a tree search-based GNN model that contains three consecutive steps: expansion, backup, and node ranking. It also proposes a relation frequency-inverse entity frequency node embedding method. The proposed model achieves new SOTA on WebQSP and CWQ datasets.

STRENGTHS:
1. It proposes a tree-search-based GNN method with backup propagation to extend the KG context, which is intuitive and novel.
2. The RF-IEF node embedding, motivated by TF-IDF, is also a novel method and experimentally effective.
3. The proposed method achieves SOTA on WebQSP and CWQ datasets.
4. Extensive experiments including incomplete KG setting, ablation study, and advantage analysis of the proposed method.

WEAKNESSES:
1. The enhancement relative to the leading benchmark approach, ReaRev, is marginal, notably on the CWQ dataset. The progress is even more slight in the context of the incomplete KG scenario, as highlighted in Table 2.

2. In such circumstances, a more detailed comparative analysis with ReaRev might be required. ReaRev, which employs a variation of the breadth-first search for adaptive reasoning, also strives to expand the modeling of the KG context. How is the proposed tree search method better than ReaRev? More crucially, given that RF-IEF supplements the GNN model, what could be the outcome if RF-IEF were integrated with ReaRev? Would it still perform worse than NuTrea?

QUESTIONS:
In the ablation study in Table 3, we see a significant performance drop when not using the Backup step. Under this setting, have the authors tried varying the layers of GNN? It would be interesting to see the performance-#layer curves with and without the backup step.

LIMITATIONS:
Yes
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, a retrieval-based solution to the Multi-hop KGQA problem is proposed. The authors introduce a novel approach that incorporates two innovations: a GNN network with bidirectional message passing (Expansion and Backup) and a novel method for embedding knowledge graph vertices, inspired by TF-IDF.\n \nThe paper features numerous experiments and comparisons with existing baselines using well-known datasets such as WebQuestionsSP, ComplexWebQuestions, and Meta-QA, all of which rely on Freebase KG. \nFurthermore, the authors provide an extensive analysis of the proposed technique, showing that the introduced backup method makes the maximum contribution to the improvement of the results.\n\nSTRENGTHS:\nIn this paper, the authors introduce a new method to the solution of the KGQA problem. The backup component of their solution stands out as particularly intriguing because of its maximum contribution to the improvement of the final results. The method is explained in a clear manner, with all essential formulas described qualitatively. Explanation of the inner working of the method is clear. NuTrea, the proposed approach, demonstrates results that are comparable to the State-of-the-Art (SOTA). Additionally, the authors have conducted qualitative comparisons with existing methods. Openly releasing the code will  benefit the community.\n\nWEAKNESSES:\nThe authors did not compare their proposed method with papers that showed significantly better results, such as DECAF (DPR + FiD-3B) (Yu, et al., 2023) with a Hit@1 of 78.8. The reason for not doing so was not explained.\nOne aspect of the system utilizes graph vertex embeddings, and the authors introduce a new approach called RF-IEF. However, it is unclear why the authors did not compare the performance of NuTrea with other graph node embeddings like PyTorch-BigGraph, which may have yielded better results. Currently, it is difficult to evaluate the effectiveness of the proposed vertex embedding approach.\n\nQUESTIONS:\nExperiments conducted on Incomplete KG demonstrate that the proposed approach performs notably better compared to others. However, it is not specified in the paper whether RF-IEF was re-trained separately for each Incomplete KG or if the original one was used. If the latter case is true, the experiment is unfair and should be recalculated.\n\nLIMITATIONS:\nNuTrea limits the search for answers to the vicinity of the original vertex, which restricts the depth of exploration within the graph. The initial depth of search is predetermined and additional heuristics are required.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presented two improvements to graph-based (KG) QA tasks: (1) Backup step, and (2) RF-IEF. Both proposed methods, according to Table 3, lead to significant improvement on multiple multi-hop QA tasks.\n\nSTRENGTHS:\nThe proposed idea of including a BackUp step, which resembles MTCS, is very interesting. Experimentally, it leads to the biggest difference (> 2 points) on the WQ dataset. To my knowledge, this is first paper which explicitly write out the backup term, while previous research relies on the GCN to capture contextual information from the neighbors.\n\nThe RF-IEF is also an interesting observation. The improvement, however, is not as significant. As mentioned by the authors, \"Many KG entities (nodes) are pronouns that are not informative, and several KGQA datasets [11, 12] consist of encrypted entity names.\" Can you please run your model on a subset of questions with \"pronouns\" and/or encrypted entity names\" to justify this claim?\n\nWEAKNESSES:\nOverall, this is a good paper. \n\nPlease consider running experiments suggested above, i.e. with \"pronouns\" and/or encrypted entity names\".\n\nOther comments:\n1. Line 105, it's a bit unfair to say \"Following the standard protocol in KGQA [5], the subject entities in 𝑥𝑞 are given and assumed always to be mapped to a node in V\". There are many work which doesn't make that assumption. It would be better to say this is an assumption you made and make sure that numbers in Table 1, 2, 3 all follow this assumption.\n\n2. Can you please specify what are \"edge type\" in line 116?\n\nQUESTIONS:\n1. Can you please specify your loss function? Is the model trained end-to-end? Any intermediate supervision?\n2. Can you please explain why the ablation study is performed on WQ? Some questions in WQ are 1-hop questions. CWQ should have more multi-hop questions where BackUp should benefit more.\n\nLIMITATIONS:\nPlease see weakness/questions above.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

comments: SUMMARY:
The proposed model adopts a message-passing scheme that probes the unreached subtree regions to boost node embeddings.The work also introduces the Relation Frequency–Inverse Entity Frequency (RF-IEF) node embedding that considers the global KG context to better characterize ambiguous KG nodes. The method shows some effectiveness over multiple datasets.

STRENGTHS:
-	Good results on multiple datasets
-	RF-IEF style node embeddings introduced. Might be useful in other works too.
-	Nice Analysis section.

WEAKNESSES:
-	Model sizes/ compute not clearly discussed. Some information present in Table 4 but insufficient.
-	How is the backup step different from a bigger expansion set in the previous works. This model doesn’t bring in any significant inductive bias over previous models.
 1. The backup is like a DFS during the expansion BFS. Really depends on the dataset as to where you want to focus your compute on. One could either explore or exploit.
 2. How would this model holdup in 4 or 5 hop QA? Would it not be extremely inefficient?
-	The gains are somewhat marginal and might just be achieved by increasing parameters/expansion depth in previous models.
-	RF-IEF is pitched as a generalized node embedding method but was not tested in other models. Although some nice analysis shown.
-	Bad writing. Lot of forward references to future text - Eq 1, Eq 5.
-	IG model descriptions very handwavy in paper and appendix.

QUESTIONS:
-	Are relation and relation type the same thing?
-	Can you list situations where this backup not useful? Or is it always useful?
-	Also address the weaknesses

LIMITATIONS:
The authors discuss some limitations.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-07-07

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposed NuTrea, a graph neural network (GNN) model for Multi-hop KGQA. NuTrea considers broader KG contexts using a tree search scheme to find paths to answer nodes. It uses message-passing layers to explicitly consider question constraints involving bi-directional information (or future context). Also, this paper proposed an interesting node embedding method RF-IEF to characterize KG node entities.\n\nSTRENGTHS:\n- The idea of predicting entities by bi-directional information is reasonable.\n\n- The Relation Frequency–Inverse Entity Frequency metric brined new insight into initializing embeddings of graph nodes.\n\n- The experiments demonstrate the effectiveness of the proposed method.\n\n- The paper is well-written.\n\nWEAKNESSES:\n- The core technical design of this work involves utilizing a backup mechanism that allows models to leverage bidirectional information (or global context), enabling them to distinguish entities with similar paths. Some embedding-based methods(e.g., EmbedKGQA and TERP) and GNN-based methods (e.g., SQALER and GraftNet) can also utilize bidirectional information, somewhat sharing a similar high-level idea. The unique superiority compared with these methods and the connections to them deserve more discussions and analyses. Otherwise, it is unclear how to position this work in the current research background, making the technical contribution seem incremental.\n\n- Although the main results and ablation study show performance superiority, most further analyses are about qualitative evaluation. Figure 2 is more about a concept-illustration example instead of an expressiveness analysis. Putting it into other sections (e.g., introduction or method) may be better. Related to the first issue, this work needs more experimental design and in-depth analysis to highlight the unique merits of the proposed methods regarding leveraging the global context. For example, I am curious to see if this method can outperform other models when both the correct and incorrect entities have the same path. \n\nAfter reading the author’s rebuttal, most of my concerns are addressed. I have no objection to accepting this paper if the authors include more discussion about related works exploiting bidirectional information.\n\nQUESTIONS:\nSee weaknesses.\n\nLIMITATIONS:\nI have no concerns about the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}