Venue: NAACL 2021

# Breadth First Reasoning Graph for Multi-hop Question Answering

Yongjie Huang1 and Meng Yang1,2,*

\({}^{1}\)School of Computer Science and Engineering, Sun Yat-sen University / Guangzhou, China

\({}^{2}\)Key Laboratory of Machine Intelligence and Advanced Computing,

Sun Yat-sen University, Ministry of Education / China huangyj229@mail2.sysu.edu.cn, yangm6@mail.sysu.edu.cn

Corresponding author.

###### Abstract

Recently Graph Neural Network (GNN) has been used as a promising tool in multi-hop question answering task. However, the unnecessary updations and simple edge constructions prevent an accurate answer span extraction in a more direct and interpretable way. In this paper, we propose a novel model of Breadth First Reasoning Graph (BFR-Graph), which presents a new message passing way that better conforms to the reasoning process. In BFR-Graph, the reasoning message is required to start from the question node and pass to the next sentences node hop by hop until all the edges have been passed, which can effectively prevent each node from over-smoothing or being updated multiple times unnecessarily. To introduce more semantics, we also define the reasoning graph as a weighted graph with considering the number of co-occurrence entities and the distance between sentences. Then we present a more direct and interpretable way to aggregate scores from different levels of granularity based on the GNN. On HotpotQA leaderboard, the proposed BFR-Graph achieves state-of-the-art on answer span prediction.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote †: Corresponding author.

+
Footnote†: footnotetext: Corresponding author.


Despite of the success that GNN achieves in multi-hop QA, new problems associated to GNN arise. Firstly, current approaches update all the nodes, including some unnecessary ones, together within each layer, which may lead the nodes to converge to similar values and lose the discriminating ability for GNN with more layers (Kipf and Welling, 2017). Secondly, although different types of edges have been designed for GNN, there is no more fine-grained distinction between edges of the same type, without considering the other relational information between sentences. Thirdly, existing methods only latently fuse the hidden representations of GNN and context encoder, without contributing to the answer span extraction in a direct and interpretable way.

To solve the aforementioned issues, we proposed a novel model of Breadth First Reasoning Graph (BFR-Graph) to effectively adapt GNN to multi-hop QA. The proposed BFR-Graph is a weighted graph in which the weight of an edge is computed based on other relational information (e.g., co-occurrence entities and distance) of the connected sentences. Inspired by the Human reasoning mechanism and the Breadth First Search algorithm, in BFR-Graph the reasoning message starts from the question and passes to the next sentence nodes hop by hop until all the edges have been passed, effectively preventing each node from updating multiple times or being updated unnecessarily. Then the reasoning result from BFR-Graph is converted to the sentence scores and paragraph scores, contributing to the answer span extraction. Specifically, the final answer span probability is the sum of the score of answer span, the sentence and the paragraph, in both of which the answer is located. Experiment results shows that our methods make GNN more powerful in multi-hop QA and achieves state-of-the-art on answer span prediction of HotpotQA.

The contributions of this paper are summarized as follows:

* We propose BFR-Graph for multi-hop QA, which is more in line with reasoning process than existing GNNs. The reasoning message starts at the question and then reasons to the next sentences hop by hop.
* Our BFR-Graph is a weighted graph, considering the number of co-occurrence entities and the distance between sentences.
* To take advantage of the reasoning result from BFR-Graph, multi-score mechanism is used for answer span extraction in a more direct and interpretable way.

## 2 Related Work

### Multi-hop QA

Serval multi-hop QA datasets have been proposed such as WikiHop (Welbl et al., 2018) and HotopQA (Yang et al., 2018). WikiHop provides candidate answers for selection while HotpotQA needs to find an answer span over all paragraphs. Based on these datasets, several categories of multi-hop QA approaches were proposed.

Yang et al. (2018) proposed a baseline method based on RNNs and Min et al. (2019) decomposed the multi-hop question into simpler single-hop subquestion that can be answered by existing single-hop RC models. To better utilize multiple paragraphs, Nishida et al. (2019) proposed Query Focused Extractor to sequentially summarize the context and Asai et al. (2020) used a recurrent retrieval approach that learns to sequentially retrieve evidence paragraphs. Moreover, reasoning has also been conducted in multi-hop QA. Jiang and Bansal (2019) designed a neural modular network to perform unique types of reasoning; Chen et al. (2020) presented extra hop attention that can naturally hops across the connected text sequences. Qiu et al. (2019) regards the task as a two-stage task including paragraph selection and downstream model. and Tu et al. (2020) further proposed a pairwise learning-to-rank loss for better interaction between paragraphs. Although the aforementioned methods are specifically designed for multi-hop QA with different structures, they lack an explicit scheme to show the reasoning process.

### GNNs for Multi-hop QA

Recently GNNs such as Graph Convolution Networks (Kipf and Welling, 2017) and Graph Attention Networks (Velickovic et al., 2018) show enhancement in multi-hop QA because the GNN-based methods are more intuitive and explicit.

Entity-GCN (De Cao et al., 2019) considered different type of edges and Tu et al. (2019) further built a heterogeneous graph with multiple types of nodes and edges for different granularity levels of information. Besides, Ding et al. (2019) coordinated implicit extraction and explicit reasoning through a GNN inspired by the dual process theory in cognitive science, and Tu et al. (2020) built a 

GNN model for reasoning over sentence, which is summarized over token representations based on a mixed attentive pooling mechanism. Furthermore, more complex graphs is also designed. Qiu et al. (2019) proposed a Dynamically Fused Graph Network to explore along the entity graph dynamically and finds supporting entities from the context. Fang et al. (2020) created a hierarchical graph for different levels of granularity to aggregate clues from scattered texts across multiple paragraphs. However, GNNs in these methods update all the nodes together, including some unnecessary ones.

## 3 Model

To solve the aforementioned issues, we propose a novel model of Breadth First Reasoning Graph (BFR-Graph) for multi-hop QA. Different from existing GNN-based methods, BFR-Graph introduces new restrictions on the message passing: the message only starts from the question and then passes to the latter sentence nodes hop by hop. Besides, our graph is constructed as a weighted graph considering the co-occurrence entities and distance between sentences. Moreover, multi-score answer prediction is designed to take advantage of the reasoning result from BFR-Graph. In short, we propose breadth first reasoning on the weighted graph and then combine multi-level scores for answer prediction in the framework of multi-task joint training.

The diagram of our system is shown in Fig. 2. Given multiple paragraphs, we first filter out irrelevant paragraph with paragraph selection (Sec. 3.1) and then use a BERT for context encoding (Sec. 3.2). A weighted graph is constructed (Sec. 3.3) to reason over sentences (Sec. 3.4) and calculate the sentence score and paragraph score. Finally, we use multi-score mechanism to predict the answer span (Sec. 3.5).

### Paragraph Selection

Although multiple candidate paragraphs are given for answering the question, not all of them are useful (i.e., relevant to the question). Following Qiu et al. (2019), we retrieve \(N\) useful paragraphs for each question through a straightforward way. Each candidate paragraph is concatenated with the question ("[CLS]" + question + [SEP]" + paragraph + [SEP]") and fed into a BERT Devlin et al. (2019) for binary classification. After training procedure, we select paragraphs with top-\(N\) score as the useful paragraphs, which are then concatenated together as context \(C\).

### Context Encoding

Following Qiu et al. (2019), we concatenate each question \(Q\) and its corresponding context \(C\), and feed them into a BERT followed by a bi-attention layer Seo et al. (2017) to obtain the encoded representations of question and context. The output is denoted as:

\[{\bf H}=\{{\bf h}_{0},\cdots,{\bf h}_{L-1}\}\in\mathbb{R}^{L\times d}, \tag{1}\]

where \(L\) is the length of the input sequence (concatenating question and context), and \(d\) is the output dimension of bi-attention layer (also the dimension of BERT).

To achieve sentence-level representations, we first obtain token-level representation of each sen

Figure 2: Diagram of our system. Each node in the graph represents a sentence. The dotted line in light blue means the sentences are from the same paragraph.

 

tence:

\[{\bf S}_{i}^{seq}={\bf H}[s_{i}^{start}:s_{i}^{end},:]\in\mathbb{R}^{L_{s_{i}} \times d}, \tag{2}\]

where \(s_{i}^{start}\), \(s_{i}^{end}\) are the start and end position of the sentence \(i\) respectively, \(L_{s_{i}}\) is the length of sentence \(i\). Note that the question is also a sentence. Then using the method in Rei and Sogaard (2019), we get sentence representation:

\[{\bf s}_{i}=\sum_{k=0}^{L_{s}}\alpha_{k}^{i}{\bf S}_{i}^{seq}[k,:]\in\mathbb{ R}^{d}, \tag{3}\]

where \(\alpha_{k}^{i}\) is the weight on the \(k\)-th token of sentence \(i\), obtained from a two-layer MLP (MultiLayer Perceptron) with output size = 1.

### Weighted Graph Construction

The nodes in our weighted graph represent question \(Q\) and sentences in context \(C\). To better exploit complex relational information between sentences, two types of correlation are defined: positive correlation and negative correlation. Although they can be designed in many ways, now we illustrate our design:

1. Positive correlation: an edge is added if the nodes representing the sentences \(i\) and \(j\) have \(n(n\geq 1)\) of the same named entities, and the weight of the edge is: \[w_{ij}=\frac{1}{1+e^{-n+K_{1}}}.\] (4)
2. Negative correlation: otherwise, an edge is added if the two nodes are originally from the same paragraph, and the weight of the edge is: \[w_{ij}=\frac{1}{1+e^{d+K_{2}}},\] (5) where \(d\) is the distance of the two sentences (e.g., \(d=1\) if the sentence is immediately followed by the other sentence in a paragraph, \(d=2\) if there is a sentence between them, etc.). \(K_{1}\) and \(K_{2}\) are hyperparameters.

To simplify our design, we treat our graph as a homogeneous graph, which contains single type of nodes and edges.

### Breadth First Reasoning

When we reason over paragraphs to answer a question, we start from the question and find the next sentence hop by hop. For a GNN where nodes represent sentences, the following message passing is unnecessary and may suppress the disturbance from useless nodes: (1) from the latter node to the former node, (2) a node haven't received the message from question but it updates other nodes.

To prevent each node from being updated multiple times unnecessarily, the reasoning message in our BFR-Graph starts from the question node and passes to the next nodes hop by hop until all the edges have been passed. Note that a node is allowed to update multiple times, depending on whether the connected edges have all been passed.

Figure 3: Message passing procedure of BFR-Graph and typical GNN. Active node is the node that is reachable for its neighbors while the quiet one is on the contrary. Active edge is the passable edge while the quiet one is on the contrary.

 

Fig. 3 visually shows the difference between BFR-Graph and typical GNN.

Specifically, a node \(i\) is updated by node \(j\) when the following conditions are met simultaneously: (1) node \(i\) and node \(j\) are neighbors, (2) node \(j\) is active, i.e., it is updated last layer, (3) the edge between node \(i\) and node \(j\) haven't been passed previously. The overall message passing procedure of BFR-Graph is illustrated in Algorithm 1.

Inspired by Graph Attention Networks [45], the updating function (or message passing function) is defined as:

\[{\bf s}_{i}^{\prime}={\rm LeakyRelu}(\sum_{j\in\mathcal{N}_{i}^{\prime}}\beta _{ij}{\bf s_{j}W}), \tag{6}\]

\[\beta_{ij}=\frac{\exp(f({\bf s}_{i},{\bf s}_{j}))\cdot w_{ij}}{\sum_{k\in \mathcal{N}_{i}^{\prime}}\exp(f({\bf s}_{i},{\bf s}_{k}))\cdot w_{ik}}, \tag{7}\]

where \(\mathcal{N}_{i}^{\prime}\) is the set of reachable neighbors for node \(i\), calculated with Algorithm 1. \(f({\bf s}_{i},{\bf s}_{j})={\bf s}_{i}{\bf W_{1}W_{2}s}_{j}\) is for calculating the attention score between node \(i\) and \(j\). **W\({}_{\textbf{1}}\)** and **W\({}_{\textbf{2}}\)** are learnable parametres. \(w_{ij}\) is the weight of the edge \((i,j)\), described in the Sec. 3.3. For clarity, \({\bf s}^{\prime}\) is written as \({\bf s}\) in following contents.

### Multi-score Answer Prediction

The answer in HotpotQA dataset is a span from the context. Existing works only calculate the span probability on the output of encoder (e.g., BERT) or additionally concatenate the GNN's hidden output. Differently, we use a more interpretable method by calculating the sentence score and paragraph score obtained from the GNN. An example is shown in Fig. 4.

Conventionally, the score of \(y\)-th word in context being the start / end of the answer span is calculated by:

\[\phi_{start}(y)={\rm MLP}_{1}({\bf H}[y,:]), \tag{8}\]

\[\phi_{end}(y)={\rm MLP}_{2}({\bf H}[y,:]), \tag{9}\]

where MLP is a two-layer MLP with output size = 1 to obtain the score value.

Then, we calculate the sentence score corresponding to each node in GNN:

\[\phi_{sent}(s_{i})={\rm MLP}_{3}({\bf s}_{i}). \tag{10}\]

Similarly, we calculate the paragraph score through a global-max-pooling:

\[\phi_{para}(p_{j})={\rm MLP}_{4}({\rm Max}(\left\{{\bf s}_{0}^{p_{j}},...,{ \bf s}_{L_{p_{j}}-1}^{p_{j}}\right\})), \tag{11}\]

Figure 4: Multi-score answer prediction. The example is calculating the score for an answer span located in paragraph 1 (“p1”) and sentence 5 (“s5”).

 

where \({\bf s}_{i}^{p_{j}}\) is the representation of the \(i\)-th sentence in paragraph \(p_{j}\), \(L_{p_{j}}\) is the number of sentences in paragraph \(p_{j}\). Max(\(\cdot\)) is a max-pooling layer with pooling size = \(L_{p_{j}}\times 1\), which can also be done by taking the maximum hidden value on each dimension over all the sentence nodes.

Finally, the probability of \(y\)-th word in context being the start of the answer span is determined by:

\[p_{start}(y)=\mathrm{softmax}(\phi_{start}^{\prime}(y)), \tag{12}\]

\[\phi_{start}^{\prime}(y)=\phi_{start}(y)+\phi_{sent}(s_{i})+\phi_{para}(p_{j}), \tag{13}\]

where the \(y\)-th word is located in sentence \(s_{i}\) and paragraph \(p_{j}\). And the probability of \(y\)-th word in context being the end of the answer span can be calculated similarly.

In other words, if a sentence or paragraph has a higher score, the words located in it are more likely to be the answer.

### Multi-task Joint Training

In addition to the answer span prediction, there are other two training tasks in HotpotQA. One is the answer type prediction task: some answers cannot be retrieved from the context, but are "Yes" or "No", so finally there are three type of answers (e.g., span, "Yes" and "No"). We use a global-max-pooling similar with Eq.(11) to compress all the nodes in the GNN and predict the answer type through a two-layer MLP.

The other task is to predict whether a sentence in the context is a support sentence (or called supporting fact in some papers) that is an evidence to the answer. Following previous works Tu et al. (2020), we use the output of the GNN to predict the supporting sentences with a two-layer MLP.

The tasks in HotpotQA are jointly performed through multi-task learning, and the loss function is:

\[\begin{split}\mathcal{L}=&\mathcal{L}_{CE}(\hat{ y}^{start},y^{start})+\mathcal{L}_{CE}(\hat{y}^{end},y^{end})+\\ &\lambda_{1}\cdot\mathcal{L}_{CE}(\hat{y}^{type},y^{type})+ \lambda_{2}\cdot\mathcal{L}_{BCE}(\hat{y}^{sp},y^{sp}),\end{split} \tag{14}\]

where \(\mathcal{L}_{CE}\) and \(\mathcal{L}_{BCE}\) denote the cross entropy and binary cross entropy loss respectively. \(\hat{y}^{start}\) denotes the logits of start position from Eq.(12) and \(y^{start}\) is the label. Similarly, \(\hat{y}^{type}\) and \(\hat{y}^{sp}\) are the logits of answer type prediction and supporting sentence prediction respectively.

## 4 Experiments

### Dataset

The HotpotQA dataset Yang et al. (2018) is the first explainable multi-hop QA dataset with sentence-level evidence supervision. Each sample in the dataset contains 2 gold paragraphs and 8 distracting paragraphs. Three tasks are included for evaluation: (1) answer span prediction (denoted as "Ans") that extracts a span in the paragraphs or generate "Yes"/"No"; (2) supporting sentences prediction (denoted as "Sup") that determines which sentences are evidences to the answer; (3) joint prediction (denoted as "Joint"). We submit our model to HotpotQA official leaderboard1 and carry out ablation studies on the dev-set.

Footnote 1: [https://hotpotqa.github.io/](https://hotpotqa.github.io/)

We also apply the main idea of BFR-Graph to the WikiHop dataset Welbl et al. (2018), which provides candidate answers for selection while HotpotQA dataset needs to find an answer span over all paragraphs.

Implementation details can be found in Appendix A.

### Results

The experimental result on HotpotQA dataset is shown in Table 1. As a reading comprehension task, the performance of answer prediction should be emphasized. Our model improves 0.84% Ans-EM (Exact Match) than HGN-large, becoming the first model to break through 70% and achieving state-of-the-art on answer span prediction. On supporting sentence prediction and joint prediction, our model shows a close performances to HGN-large, possibly because this paper is based on the standard GNN (homogeneous graph) for simple clarification, and we just plan to prove that our algorithm can improve the performance of GNN. Existing GNN methods mostly constructed elaborate graphs for more granular expression of nodes, while our BFR-Graph solve the problem from another novel perspective. Thus, BFR-Graph is universal and can be easily applied to existing promising models (e.g., HGN) to get better results, which provides a promising direction for future research.

We also compare our model with two state-of-the-art GNN models (i.e., SAE and HGN), shown in Table 2. Both of them need to set the number of GNN layers manually while BFR-Graph can pass through all the connected nodes automatically with 

an extremely low risk of over-smoothing Kipf and Welling (2017). SAE and HGN set a fixed types of edges, which is still not fine-grained enough, while BFR-Graph define different weights (can up to \(\infty\) different weights depends on the dataset) to distinguish nodes in a finer granularity. Furthermore, we can easily observe scores from GNN in an intuitive way in BFR-Graph.

Besides, Table 3 shows the results on WikiHop dev-set. When we add the breadth first reasoning graph and weights to Longformer Beltagy et al. (2020), the performance is slightly improved, showing that our method have the ability for better reasoning.

## 5 Ablations and Analysis

In this section, we carry out ablation studies on HotpotQA dev-set. Table 4 shows the results of our full model and that without breadth first reasoning, weights, and multi-score. It indicates that our methods obviously improve the performance of GNN.

### Evaluation on Breedth First Reasoning

Table 5 shows the result by gradually replace the BFR-Graph layers with standard GNN layers. In detail, "r/p 1 layer" denotes replacing the first layer with a standard GNN layer, "r/p 2 layers" denotes the same operation for the first and second layers, etc.. We observe that the more layers to be replaced, the more severely the result drops. And when we replace 4 layers, the joint F1 drops at about 6%, meaning that it causes over-smoothing. It also reflects the severe problem of typical GNN: if it have more layers, over-smoothing is caused; if it have less layers, it cannot achieve long-path reasoning.

To further analyze why this particular approach of message passing in a breadth first reasoning fashion should result in better reasoning, we propose to calculate how many useful messages the answer sentence node received from supporting sentences: \(precision=\frac{N_{spkrcv}}{N_{rcv}},recall=\frac{N_{spkrcv}}{N_{rp}}\), where \(N_{rcv}\) denotes how many nodes' massages the answer sentence node received, \(N_{sp}\) denotes the number of supporting sentence (containing the question sentence here), and \(N_{spkcrcv}\) denotes how many supporting nodes' massages the answer sentence node received.

The above-mentioned precision, recall and corresponding F1 on dev-set is shown in Table 6, where the typical GNN is a 2-layer GNN following previous works. With breadth first reasoning, the answer

\begin{table}
\begin{tabular}{l c c c c c c} \hline  & \multicolumn{2}{c}{**Ans**} & \multicolumn{2}{c}{**Sup**} & \multicolumn{2}{c}{**Joint**} \\
**Model** & **EM** & **F1** & **EM** & **F1** & **EM** & **F1** \\ \hline Official baseline Yang et al. (2018) & 45.60 & 59.02 & 20.32 & 64.49 & 10.83 & 40.16 \\ QFE Nishida et al. (2019) & 53.86 & 68.06 & 57.75 & 84.49 & 34.63 & 59.61 \\ DFGN Qiu et al. (2019) & 56.31 & 69.69 & 51.50 & 81.62 & 33.62 & 59.82 \\ LQR-Net Grail et al. (2020) & 60.20 & 73.78 & 56.21 & 84.09 & 36.56 & 63.68 \\ SAE-large Tu et al. (2020) & 66.92 & 79.62 & 61.53 & 86.86 & 45.36 & 71.45 \\ C2F-reader Shao et al. (2020) & 67.98 & 81.24 & 60.81 & 87.63 & 44.67 & 72.73 \\ HGN-large Fang et al. (2020) & 69.22 & 82.19 & **62.76** & **88.47** & **47.11** & **74.21** \\ BFR-Graph & **70.06** & **82.20** & 61.33 & 88.41 & 45.92 & 74.13 \\ \hline \end{tabular}
\end{table}
Table 1: Results on HotpotQA leaderboard. “Ans”, “Sup” and “Joint” denote answer span prediction, supporting sentence prediction and joint prediction, respectively.

\begin{table}
\begin{tabular}{l r} \hline
**Model** & **Accuracy** \\ \hline HDE Tu et al. (2019) & 68.1 \\ DynSAn Zhuang and Wang (2019) & 70.1 \\ Path-based GCN Tang et al. (2020) & 70.8 \\ ChainEx Chen et al. (2019) & 72.2 \\ \hline Longformer* & 73.8 \\ Longformer+BFR & **74.4** \\ \hline \end{tabular}
\end{table}
Table 2: Comparison with state-of-the-art GNN models. “Layers” denotes the number of GNN layers, “Edges” denotes how fine-grained the edges are, and “Intuitive” denotes whether the output of GNN can be intuitively observed.

 

sentence could receive messages from supporting sentences with a higher precision, meaning that it can focus on useful sentences and eliminate invalid distractions. Since the restrictions on message passing in breadth first reasoning, it leads to a decrease in recall. However, it is hard to draw a PR curve or get different precision-recall results because this is not a binary classification task as we generally understand. But fortunately, BFR-Graph shows a higher F1 than the typical GNN.

### Evaluation on Weights and Multi-score

Table 7 (top) presents the results with and without the weights in the GNN. "-ent" denotes removing the weights (we set the weights = 0.5 rather than simply remove them) and "- dist" denotes removing the distance weights. When we remove the weights, although the answer F1 rises slightly, the supporting F1 falls to a greater extent. This shows that the proposed weights is beneficial to the supporting sentences prediction, which is directly predicted from the GNN nodes.

To our understanding, our model enhances the discrimination of edges by setting weights for them, and inevitably reduces the robustness of model. Fortunately, by designing Eqs.(4) and (5), the quantitative error will not cause the weight to increase or decrease sharply, and is still able to distinguish the difference between sentences.

For multi-score, we evaluate how the result changes if this particular way of exploiting GNN's output is replaced by traditional way. In Table 7 (bottom), "-sent" and "-para" denote removing multi-score for sentence and paragraph respectively. It indicates that both the addition of sentence scores and paragraph scores are beneficial to the performance.

### Complexity Analysis

We also analyze the complexities of BFR-Graph and typical GNN, which is simply shown in Table 8. Firstly, in each layer of our BFR-Graph, only several nodes are updated by active nodes, so the number of nodes to be updated in a BFR-Graph layer is less than or equal to that in a typical GNN (\(N_{update}\leq N\)). Secondly, for a node in a layer of BFR-Graph, it is only updated by its reachable nodes (i.e., active neighbors), so the number of reachable nodes for a node in a BFR-Graph layer is also less than or equal to that in typical GNN (\(M_{reach}\leq M\)). Therefore, breadth first reasoning leads to lower complexity.

For GPU parallel training, we also show the actual cost of time per epoch. BFR-Graph cost 158.6 minutes per epoch, while a 2-layer and 3-layer typical GNN costs 157.5 and 165.6 minutes respec

\begin{table}
\begin{tabular}{l c c c} \hline  & **Precision** & **Recall** & **F1** \\ \hline typical GNN & 37.62 & **95.61** & 52.89 \\ BFR-Graph & **59.44** & 83.49 & **63.08** \\ \hline \end{tabular}
\end{table}
Table 6: Message passing in different style.

\begin{table}
\begin{tabular}{l l l l} \hline  & **Ans F1** & **Sup F1** & **Joint F1** \\ \hline full model & 81.82 & **88.80** & **73.98** \\ r/p 1 layer & **81.86** & 88.49 & 73.80 \\ r/p 2 layers & 81.57 & 88.50 & 73.62 \\ r/p 3 layers & 80.66 & 87.63 & 72.04 \\ r/p 4 layers & 77.08 & 86.50 & 67.97 \\ \hline \end{tabular}
\end{table}
Table 5: Ablations on breadth first reasoning.

 

tively. We find that BFR-Graph is always 4 layers in HotpotQA dataset, and it can even cost less time than a 3-layer typical GNN and is close to a 2-layer typical GNN.

## 6 Conclusion

In this paper, we proposed a novel GNN model of BFR-Graph. Specifically, the reasoning message starts from the question node and passes to the next sentences node hop by hop until all the edges have been passed. We also construct the reasoning graph as a weighted graph and present a more interpretable way to aggregate scores of different levels from GNN. On HotpotQA leaderboard, BFR-Graph achieved state-of-the-art on answer span prediction.

## Acknowledgements

This work is partially supported by National Natural Science Foundation of China (Grants no. 61772568), Guangdong Basic and Applied Basic Research Foundation (Grant no. 2019A1515012029), and Youth science and technology innovation talent of Guangdong Special Support Program.

## References

* A. Asai, K. Hashimoto, H. Hajishirzi, R. Socher, and C. Xiong (2020)Learning to retrieve reasoning paths over wikipedia graph for question answering. In Proceedings of International Conference on Learning Representations, Cited by: SS1, SS2.2, Table 1, SS3.2, SS4.2, Table 1, SS4.2, Table 2, SS4.2, Table 2, SS4.2, Table 3, SS4.2, Table 4.
* L. Beltagy, M. E. Peters, and A. Cohan (2020)Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150. External Links: Link Cited by: SS1, SS2.2, Table 1, SS3.2, SS4.2, Table 2, SS4.2, Table 3, SS4.2, Table 4.
* J. Chen, S. Lin, and G. Durrett (2019)Multi-hop question answering via reasoning chains. arXiv preprint arXiv:1910.02610. Cited by: SS1, SS2.2, Table 1, SS3.2, SS4.2, Table 2, SS4.2, Table 3, SS4.2, Table 4.
* Z. Chen, X. Chenyan, R. Corby, S. Xia, B. Paul, and T. Saurabh (2020)Transformer-xhi: multi-evidence reasoning with extra hop attention. In Proceedings of International Conference on Learning Representations, Cited by: SS1, SS2.2, Table 1, SS3.2, SS4.2, Table 3, SS4.2, Table 3, SS4.2, Table 4.
* N. De Cao, W. Aziz, and I. Titov (2019)Question answering by reasoning across documents with graph convolutional networks. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1, pp. 2306-2317. External Links: Document Cited by: SS1, SS2.2, Table 1, SS3.2, SS4.2, Table 3, SS4.2, Table 3, SS4.2, Table 4.
* J. Devlin, M. Chang, K. Lee, and K. Toutanova (2019)BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1, pp. 4171-4186. External Links: Link Cited by: SS1, SS2.2, Table 1, SS3.2, SS4.2, Table 3, SS4.

 

Lin Qiu, Yunxuan Xiao, Yanru Qu, Hao Zhou, Lei Li, Weinan Zhang, and Yong Yu. 2019. Dynamically fused graph network for multi-hop reasoning. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 6140-6150.
* Rei and Sogaard (2019) Marek Rei and Anders Sogaard. 2019. Jointly learning to label sentences and tokens. In _Proceedings of the AAAI Conference on Artificial Intelligence_, pages 6916-6923.
* Seo et al. (2017) Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2017. Bidirectional attention flow for machine comprehension. In _Proceedings of International Conference on Learning Representations_.
* Shao et al. (2020) Nan Shao, Yiming Cui, Ting Liu, Shijin Wang, and Guoping Hu. 2020. Is Graph Structure Necessary for Multi-hop Question Answering? In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing_, pages 7187-7192.
* Tang et al. (2020) Zeyun Tang, Yongliang Shen, Xinyin Ma, Wei Xu, Jiale Yu, and Weiming Lu. 2020. Multi-hop reading comprehension across documents with path-based graph convolutional network. pages 3905-3911.
* Tu et al. (2020) Ming Tu, Kevin Huang, Guangtao Wang, Jing Huang, Xiaodong He, and Bowen Zhou. 2020. Select, answer and explain: Interpretable multi-hop reading comprehension over multiple documents. In _Proceedings of the AAAI Conference on Artificial Intelligence_, pages 9073-9080.
* Tu et al. (2019) Ming Tu, Guangtao Wang, Jing Huang, Yun Tang, Xiaodong He, and Bowen Zhou. 2019. Multi-hop reading comprehension across multiple documents by reasoning over heterogeneous graphs. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 2704-2713.
* Velickovic et al. (2018) Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2018. Graph attention networks. In _Proceedings of International Conference on Learning Representations_.
* Welbl et al. (2018) Johannes Welbl, Pontus Stenetorp, and Sebastian Riedel. 2018. Constructing datasets for multi-hop reading comprehension across documents. _Transactions of the Association for Computational Linguistics_, 6:287-302.
* Yang et al. (2018) Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_, pages 2369-2380.
* Zhuang and Wang (2019) Yimeng Zhuang and Huadong Wang. 2019. Token-level dynamic self-attention network for multi-passage reading comprehension. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 2252-2262.

## Appendix A Implementation Details

We select \(N=3\) useful paragraphs in paragraph selection, which achieves 98.7% recall in dev-set. We use RoBerta-large (Liu et al., 2019) for context encoding, with a maximum length of 512 tokens. We also fine-tune the model on SQuAD dataset similar as Groeneveld et al. (2020). We use spaCy2 for named entity recognition and we found the balance factor \(K_{1}=0\), \(K_{2}=-2\) lead to better result. The manual weights of the loss function are \(\lambda_{1}=1,\lambda_{2}=5\) in this work. The sentences number is limited to 30 and the max sentence length is set to 512 (same with BERT). We use Adam with learning rate of 1e-5, L2 weight decay of 0.01, learning rate warm-up over the first 1,000 steps and linear decay to 0. Other hyperparameters mainly follow previous works (Fang et al., 2020). We implement our model using PyTorch3 and train it on RTX 2080ti GPUs.

Footnote 3: [https://spacy.io/](https://spacy.io/)

Footnote 3: [https://pytorch.org/](https://pytorch.org/) 

To provide in-depth understanding of the weaknesses of our model, we carry out error analysis. Following Fang et al. (2020), we randomly sample 100 examples in the dev-set with the answer F1 as 0. Then we group the error cases into 6 categories: (1) Annotation: the reference answer is incorrect; (2) Multiple Answers: multiple correct answers can answer the question, but only one is provided in the dataset; (3) Discrete Reasoning: this type of error often appears in "comparison" questions, where discrete reasoning is required to answer the question; (4) External Knowledge: commonsense, external knowledge or mathematical operation is required; (5) Multi-hop: the model fails to perform multi-hop reasoning, and finds the final answer from wrong paragraphs; (6) MRC: the model extracts the wrong answer span but correctly finds the supporting paragraphs and sentences.

Table 9 shows the percentages of the 6 error categories of our BFR-Graph. We find that many errors are due to the wrong reference answer (10%) or multiple answers (22%), which actually should not be considered as the error cases. Among other error cases, the major category of errors comes from the questions that need external knowledge (20%, including commonsense and mathematical operation), which is hard to handle without a knowledge base.

## Appendix C A Case for Multi-score Prediction

Fig. 6 shows an example with specific scores when calculating multi-scores. The RoBerta-style tokens have already been converted to the BERT-style tokens for better reading.

"Token-idx" denotes the index for each token. "Para-score" and "Sent-score" denote paragraph scores and sentences scores respectively. "Start-score" and "End-score" are the scores that be the start and end of the answer span.

There are 3 paragraphs in this case (token index: 0-54, 55-108, 109-204), and the second paragraph achieve the highest paragraph score. Similarly, we can find the highest sentence score (token index: 55-79). Both (token index: 66-67) and (token index: 88-89) lead to the correct answer, with high span scores.

\begin{table}
\begin{tabular}{|p{14.5pt} p{142.3pt}|} \hline Question: Which country borders the Central African Republic and is south of Libya and east of Niger? \\ \hline \multicolumn{3}{|c|}{Paragraph 1: Geography of the Central African Republic is a landlocked nation within the interior of the African continent.} \\ \hline S2 & It is bordered by the countries of Cameroon, Chad, Sudan, South Sudan, the Democratic Republic of the Congo and the Republic of the Congo. \\ \hline S3 & Much of the country consists of flat, or rolling plateau savanna, about 1640 ft above sea level. \\ \hline S4 & In the northeast are the Cerrit Hills, and there are scattered hills in the southwestern part of the country. \\ \hline \multicolumn{3}{|c|}{Paragraph 2: Chad} \\ \hline S5 & Chad, officially the **Republic of Chad**, is a landlocked country in Central Africa. \\ \hline S6 & It is bordered by LIbya to the north, Sudan to the east, the Central African Republic to the south, Cameroon and Nigeria to the southwest and Niger to the west. \\ \hline S7 & It is the fifth largest country in Africa in terms of area. \\ \hline \end{tabular} Reference Answer: Republic of Chad

BFR-Graph: Chad (correct)

HGN: Sudan (incorrect)

\end{table}
Table 9: Error Analysis of our BFR-Graph.

Figure 5: Case study.

 

* [170] J. M. Croman, R. A. K. Ellis, and D. E. Kaplan, _The New Journal of Physics: Volume I: Theory and Experiment_, vol. 34, no. 2, pp. 569-583, 1996.

[MISSING_PAGE_POST]