Venue: EMNLP 2020

# Multi-hop Inference for Question-driven Summarization+
Footnote †: The work described in this paper is substantially supported by a grant from the Research Grant Council of the Hong Kong Special Administrative Region, China (Project Codes: 14200719).

Yang Deng, Wenxuan Zhang, Wai Lam

The Chinese University of Hong Kong

{ydeng,wxzhang,wlam}@se.cuhk.edu.hk

###### Abstract

Question-driven summarization has been recently studied as an effective approach to summarizing the source document to produce concise but informative answers for non-factoid questions. In this work, we propose a novel question-driven abstractive summarization method, Multi-hop Selective Generator (MSG), to incorporate multi-hop reasoning into question-driven summarization and, meanwhile, provide justifications for the generated summaries. Specifically, we jointly model the relevance to the question and the interrelation among different sentences via a human-like multi-hop inference module, which captures important sentences for justifying the summarized answer. A gated selective pointer generator network with a multi-view coverage mechanism is designed to integrate diverse information from different perspectives. Experimental results show that the proposed method consistently outperforms state-of-the-art methods on two non-factoid QA datasets, namely WikiHow and PubMedQA.

## 1 Introduction

Recent years have witnessed several attempts on exploring question-driven summarization, which aims at summarizing the source document with respect to a specific question, to produce a concise but informative answer in non-factoid question answering (QA) Tomasoni and Huang (2010); Chan et al. (2012); Song et al. (2017). Unlike factoid QA Rajpurkar et al. (2016), e.g., "Who is the author of Harry Potter?", whose answer is generally a single phrase or a short sentence with limited information, the answers for non-factoid questions are supposed to be more informative, involving some detailed analysis to explain or justify the final answers, such as questions in community QA Ishida et al. (2018); Deng et al. (2020) or explainable QA Fan et al. (2019); Nakatsuji and Okui (2020). As the example from PubMedQA Jin et al. (2019) presented in Figure 1, the answer can be regarded as the summary over the document driven by the reasoning process of the given question.

Most of related studies focus on query-based summarization approaches for summarizing the query-related content from the source document Shen and Li (2011); Wang et al. (2013); Cao et al. (2016); Nema et al. (2017). However, these approaches fall short of tackling question-driven summarization problem in QA scenario, since the query-based summarization process is typically based on semantic relevance measurement without a careful reasoning or inference process, which is essential to question-driven summarization. Currently, question-driven summarization is mainly explored by traditional information retrieval methods to select sentences from the source document to construct the final answer Wang et al. (2014); Song et al. (2017); Yulianti et al. (2018), which heavily rely on hand-crafted features or tedious multi-stage pipelines. Besides, compared to extractive summarization Cao et al. (2016), abstractive methods Nema et al. (2017) can produce more coherent and logical summaries to answer the given question. To this end, we study question-driven abstractive summarization to generate natural form of answers by summarizing the source document with respect to a specific question.

To tackle question-driven abstractive summarization, the content selection process for summarization is not only determined by the semantic relevance to the given question, but it also requires a human-like reasoning and inference process to consider the content interrelationship comprehensively and carefully across the whole source text for gener

ating the summary. For instance, in Figure 1, given the specific question, there are several **highlighted** sentences required to be concentrated for conducting summarization so as to generate the answer. It leads to the necessity of measuring the importance of each sentence, instead of regarding the source text as an undifferentiated whole. Among these **highlighted** sentences, only the _italic_ sentences are directly related to the given question, while other **highlighted** sentences need to be inferred from their interrelationships with other sentences. In other words, the generated summary is likely to lose important information, if we only focus on the semantically relevant content to the given question. Moreover, it can be observed that one-time inference sometimes is insufficient for collecting all the required information for producing a summary. In this example, the answer is summarized from both the _1st-hop_ and 3rd-hop inference sentences in the document, indicating the importance of multi-hop reasoning for content selection in question-driven summarization.

In this work, we propose a question-driven abstractive summarization model, namely Multi-hop Selective Generator (MSG), which incorporates multi-hop inference to summarize abstractive answers over the source document for non-factoid questions. Concretely, the document is regarded as a hierarchical text structure to be assessed with the importance degree in both word- and sentence-level for content selection. Then we develop a multi-hop inference module to enable human-like multi-hop reasoning in question-driven summarization, which considers the semantic relevance to the question as well as the information consistency among different sentences. Finally, a gated selective pointer generator network with multi-view coverage mechanism is proposed to generate a concise but informative summary as the answer to the given question.

The main contributions of this paper can be summarized as follows: (1) We propose a novel question-driven abstractive summarization model for generating answers in non-factoid QA, which incorporates multi-hop reasoning to infer the important content for facilitating answer generation; (2) We propose a multi-view coverage mechanism to address the repetition issue along with the multi-view pointer network and generate informative answers; (3) Experimental results show that the proposed method achieves state-of-the-art performance on WikiHow and PubMedQA datasets, and it is able to provide justification sentences as the evidence for the answer.

## 2 Related Works

**Query-based Summarization.** Early works on query-based summarization focus on extracting query-related sentences to construct the summary Lin et al. (2010); Shen and Li (2011), which are later improved by exploiting sentence compression on the extracted sentences Wang et al. (2013); Li and Li (2014). Recently, some data-driven neural abstractive models are proposed to generate natural form of summaries with respect to the given query Nema et al. (2017); Hasselqvist et al. (2017). However, current studies on query-based abstractive summarization are restricted by the lack of large-scale datasets Baumel et al. (2016); Nema et al. (2017). One the other hand, some researchers spark a new pave of question-driven summarization in non-factoid QA Song et al. (2017); Yulianti

Figure 1: An example from PubMedQA. The **highlighted** sentences illustrate the inference process when humans answer the given question. _Italic_ represents direct matching sentences from the question. **Underlined** and **wavy-underlined** represent sentences inferred by 2nd-hop and 3rd-hop reasoning, respectively, to justify the answer.

 

et al., 2018; Deng et al., 2020), which requires the ability of reasoning or inference for supporting summarization, not merely relevance measurement, and also preserves remarkable testbeds of large-scale datasets.

**Non-factoid Question Answering.** Different from factoid QA that can be tackled by extracting answer spans Rajpurkar et al. (2016) or generating short sentences Nguyen et al. (2016); Kocisky et al. (2018), non-factoid QA aims at producing relatively informative and complete answers. In the past studies, non-factoid QA focused on retrieval-based methods, such as answer sentence selection Nakov et al. (2015) or answer ranking Zhang et al. (2020). Recently, several efforts have been made on tackling long-answer generative question answering over supporting documents, which targets on questions that require detailed explanations Fan et al. (2019). This kind of QA problem contains a large proportion of non-factoid questions, such as "how" or "why" type questions Koupaee and Wang (2018); Ishida et al. (2018); Deng et al. (2020). Besides, some studies aim at generating a conclusion for the concerned question Jin et al. (2019); Nakatsuji and Okui (2020). Fan et al. (2019) propose a multi-task Seq2Seq model with the concatenation of the question and support documents to generate long-form answers. Iida et al. (2019) and Nakatsuji and Okui (2020) incorporate some background knowledge into Seq2Seq model for why questions and conclusion-centric questions. Some latest works Feldman and El-Yaniv (2019); Yadav et al. (2019); Nishida et al. (2019) attempt to provide evidence or justifications for human-understandable explanation of the multi-hop inference process in factoid QA, where the inferred evidences are only treated as the middle steps for finding the answer. However, in non-factoid QA, the intermediate output is also important to form a complete answer, which requires a bridge between the multi-hop inference and summarization.

## 3 Proposed Framework

We propose a question-driven abstractive summarization model, namely Multi-hop Selective Generator (MSG). The overview of MSG is depicted in Figure 2, which consists of three main components: (1) _Co-attentive Encoder_ (Section 3.1), (2) _Multi-hop Inference Module_ (Section 3.2), and (3) _Gated Selective Generator_ (Section 3.3). Moreover, _Multi-view Coverage Loss_ is integrated to the overall training procedure (Section 3.4).

### Co-attentive Encoder

Pre-trianed word embeddings, \(E_{q}\) and \(E_{s_{i}}\), of the question \(q\) and each sentence \(s_{i}\) in the document \(D=\left\{s_{1},s_{2},...,s_{n}\right\}\) are input into the model. We first encode the question and each sentence in the document by a Bi-LSTM (Bidirectional Long Short-Term Memory Networks) shared encoder to learn the word-level contextual information, \(H_{q},H_{s_{i}}\in\mathbb{R}^{l\times d_{h}}\), where \(l\) and \(d_{h}\) denotes the sentence length and the dimension of the encoder output respectively. The overall word-level representations \(H_{d}\) for the document is sequentially concatenated by \([H_{s_{1}},H_{s_{2}},...,H_{s_{n}}]\).

We compute the attention weights to align the word-level information between the question and the document sentences, and obtain the attention-weighted vectors of each word for both the question and the document sentences. For the question \(q\) and the \(i\)-th sentence \(s_{i}\) in the document \(D\), we have:

\[O_{qs_{i}}=\text{tanh}\left(H_{q}^{T}UH_{s_{i}}\right), \tag{1}\] \[\alpha_{q_{i}}=\text{softmax}(\text{Max}(O_{qs_{i}})),\] (2) \[\alpha_{s_{i}}=\text{softmax}(\text{Max}(O_{qs_{i}}^{T})), \tag{3}\]

where \(U\in\mathbb{R}^{d_{h}\times d_{h}}\) is the attention matrix to be learned; \(\alpha_{q_{i}}\) and \(\alpha_{s_{i}}\) are co-attention weights for the question and \(i\)-th sentence in the document.

We conduct dot product between the attention vectors and the word-level representations to generate the sentence representations for the question and the document:

\[M_{q}=\frac{1}{n}\sum\nolimits_{i=1}^{n}H_{q}^{T}\alpha_{q_{i}} \tag{4}\] \[M_{s}=[H_{s_{1}}^{T}\alpha_{s_{1}}:....:H_{s_{n}}^{T}\alpha_{s_{ n}}], \tag{5}\]

where \(M_{q}\) and \(M_{s}\) denote the sentence-level representations for the question and the document.

### Multi-hop Inference Module

Multi-hop Inference Module measures the degree of importance for each sentence in the document to generate the answer, through a multi-hop reasoning procedure, which contains two kinds of inference units: Attentive Unit and MAR Unit.

#### 3.2.1 Attentive Unit

Attentive Unit basically measures the matching degree between each sentence in the document and the given question by the following vanilla attention mechanism 

\[m_{dq}=\text{tanh}(M_{s}W_{m}M_{q}), \tag{6}\] \[\alpha_{s}=\text{softmax}(\omega^{T}_{m}m_{d_{q}}),\] (7) \[\mathbf{Attentive}(M_{s},M_{q})=M_{s}\odot\alpha_{s}, \tag{8}\]

where \(W_{m}\) and \(\omega_{m}\) are the attention matrices to be learned. \(\alpha_{s}\) is the sentence-level attention weight which measures the matching degree of each document sentence with the given question. \(\odot\) denotes the element-wise product for obtaining the attentive sentence-level representations for the document.

#### 3.2.2 MAR Unit

Maximal Marginal Relevance (MMR) is an IR model that can be adopted to measure the query-relevancy and information-redundancy simultaneously for extractive summarization Carbonell and Goldstein (1998). However, as for the content selection in abstractive summarization, the relevance to both the question and the other sentences in the document should be taken into consideration for a high recall of selecting necessary content. Thus, we propose Maximal Absolute Relevance (MAR) to select highly salient sentences for generating the summary, which is formulated as:

\[\begin{split} mar_{i}=&\lambda\text{Sim}_{1}(M_{s_{ i}},M_{q})+\\ &(1-\lambda)\max_{s_{j}\in D,j\neq i}\text{Sim}_{2}(M_{s_{i}},M_{ s_{j}}),\end{split} \tag{9}\]

where \(\lambda\) is a hyper-parameter for balancing the question-relevacy and information-consistency measurement. The relevance to the question is calculated by:

\[\text{Sim}_{1}(M_{s_{i}},M_{q})=M_{s_{i}}U_{1}M_{q}, \tag{10}\]

where \(U_{1}\) is a similarity matrix to be learned. We apply an attention mechanism over other sentences in the document to choose the highest relevance score, which can be regarded as the reasoning procedure where the next-hop justification sentences are supposed to be highly related to the last-hop justification sentences.

\[e_{ij}=\mathrm{tanh}(M_{s_{i}}U_{2}M_{s_{j}}), \tag{11}\] \[\text{Sim}_{2}(M_{s_{i}},M_{s_{j}})=\frac{exp(e_{ij})}{\sum_{j} exp(e_{ij})}, \tag{12}\]

where \(U_{1}\) is a similarity matrix to be learned.

Then the weighted sentence representations are computed by the element-wise product of the original sentence representations and the MAR scores gated by a sigmoid function denoted as \(\sigma\):

\[\mathbf{MAR}(M_{s},M_{q})=M_{s}\odot\sigma(mar). \tag{13}\]

Overall, MAR Unit assigns higher weights to sentences in two situations: (i) Those sentences are correlated to the given question, due to the first term in Equation 9, (ii) Those sentences are consistent with the highly weighted justification sentences from the last hop, due to the second term.

#### 3.2.3 Reasoning Procedure

In accordance with human-like multi-hop inference procedure, the first hop is supposed to capture the semantic-relevant sentences to the given question. Then the subsequent hops should consider not only the relevance to the question, but also the information-consistency with the previous attended sentences. Hence, the Attentive Unit is adopted as the 1st-hop inference unit, while the MAR Unit is served as the \(k\)th-hop unit, where \(k>1\). Before each hop, a Bi-LSTM layer is employed to refine the input sentence representation. For instance, a 3-hop inference procedure is as follows:

\[M^{(1)}_{s}=\mathbf{Attentive}(\text{Bi-LSTM}(M_{s}),M_{q}), \tag{14}\] \[M^{(2)}_{s}=\mathbf{MAR}(\text{Bi-LSTM}(M^{(1)}_{s}),M_{q}),\] (15) \[M^{(3)}_{s}=\mathbf{MAR}(\text{Bi-LSTM}(M^{(2)}_{s}),M_{q}). \tag{16}\]

Figure 2: The overview of Multi-hop Selective Generator (MSG).

 

Then, we merge the 3-hop sentence representations, \({\hat{M}_{s}=[M^{(1)}_{s},M^{(2)}_{s},M^{(3)}_{s}]}\), via the following attention mechanism:

\[\alpha_{h}=\text{softmax}(\omega^{T}_{h}\tanh(W_{h}{\hat{M}_{s}})), \tag{17}\] \[Z={\hat{M}_{s}^{T}}\alpha_{h}, \tag{18}\]

where \(W_{h}\) and \(\omega_{h}\) are attention matrices to be learned. \(Z\) is the final sentence-level document representation for justifying the importance degree of each sentence in the decoding phase.

### Gated Selective Generator

We obtain the word-level representations \(H_{q}\) and \(H_{d}\) for the question and document, respectively, from the encoding phase, and the sentence-level document representation \(Z\) via the multi-hop inference module. Figure 3 depicts the Gated Selective Pointer Generator Network in MSG.

A unidirectional LSTM is adopted as the decoder. At each step \(t\), the decoder produces hidden state \(s_{t}\) with the input of the previous word \(w_{t-1}\). The attention for each word in the question and the document, \(\alpha^{q}_{t}\) and \(\alpha^{d}_{t}\), are generated by:

\[e^{q_{j}}_{t}=\omega^{qT}_{l}\text{tanh}(W_{q}H_{q_{j}}+W_{qs} s_{t}+b_{q}), \tag{19}\] \[\alpha^{q}_{t}=\text{softmax}(e^{q}_{t}),\] (20) \[e^{d_{i}}_{t}=\omega^{dT}_{t}\text{tanh}(W_{d}H_{d_{i}}+W_{ds} s_{t}+b_{d}),\] (21) \[\alpha^{d}_{t}=\text{softmax}(e^{d}_{t}), \tag{22}\]

where \(W_{q}\), \(W_{qs}\), \(W_{d}\), \(W_{ds}\), \(\omega^{q}_{t}\), \(\omega^{d}_{t}\), \(b_{q}\), \(b_{d}\) are parameters to be learned.

Then, we incorporate the multi-hop inference results \(Z\) to compute the gated attention weights \(\beta_{t}\) for each sentence in the document:

\[\beta_{t}=\sigma({\omega^{sT}_{t}}\text{tanh}(W_{s}Z_{k}+W_{ss} s_{t}+b_{s})), \tag{23}\]

where \(W_{s}\), \(W_{ss}\), \(\omega^{s}_{t}\), \(b_{s}\) are parameters to be learned. We re-weight the word-level document attention scores \(\alpha^{d}\) gated by the sentence-level document attention scores \(\beta\) to attend important justification sentences along with the decoding process:

\[\hat{\alpha}^{d_{i}}_{t}=\frac{\alpha^{d_{i}}_{t}\beta_{t,d_{i} \in s_{k}}}{\sum_{i}\alpha^{d_{i}}_{t}\beta_{t,d_{i}\in s_{k}}}. \tag{24}\]

Thus, the re-weighted word-level document attention \(\hat{\alpha}^{d}\) naturally blends with the results from the multi-hop inference module to enhance the influence of those important justification sentences.

Finally, a multi-view pointer-generator architecture is designed to generate answers with multi-hop inference results as well as handle the multi-perspective out-of-vocabulary (OOV) issue. Such approach enables MSG to copy words from the question and be aware of the differential importance degree of different sentences in the document.

The attention weights \(\alpha^{q}_{t}\) and \(\hat{\alpha}^{d}_{t}\) are used to compute context vectors \(c^{q}_{t}\) and \(c^{d}_{t}\) as the probability distribution over the source words:

\[c^{q}_{t}=H^{T}_{q}\alpha^{q}_{t},\quad c^{d}_{t}=H^{T}_{d}\hat{ \alpha}^{d}_{t}. \tag{25}\]

The context vector aggregates the information from the source text for the current step. We concatenate the context vector with the decoder state \(s_{t}\) and pass through a linear layer to generate the answer representation \(h^{s}_{t}\):

\[h^{s}_{t}=W_{1}[s_{t}:c^{q}_{t}:c^{d}_{t}]+b_{1}, \tag{26}\]

where \(W_{1}\) and \(b_{1}\) are parameters to be learned.

Then, the probability distribution \(P^{v}\) over the fixed vocabulary is obtained by passing the answer representation \(h^{s}_{t}\) through a softmax layer:

\[P^{v}(y_{t})=\text{softmax}(W_{2}h^{s}_{t}+b_{2}), \tag{27}\]

where \(W_{2}\) and \(b_{2}\) are parameters to be learned.

The final probability distribution of \(y_{t}\) is obtained from three views of word distributions:

\[P^{q}(y_{t})=\sum\nolimits_{i:w_{i}=w}\alpha^{q_{i}}_{t}, \tag{28}\] \[P^{d}(y_{t})=\sum\nolimits_{i:w_{i}=w}\hat{\alpha}^{d_{i}}_{t},\] (29) \[P^{all}(y_{t})=[P^{v}(y_{t}),P^{q}(y_{t}),P^{d}(y_{t})],\] (30) \[\rho=\text{softmax}(W_{\rho}[s_{t}:c^{q}_{t}:c^{d}_{t}]+b_{\rho}),\] (31) \[P(y_{t})=\rho\cdot P^{all}(y_{t}), \tag{32}\]

where \(W_{\rho}\) and \(b_{\rho}\) are parameters to be learned, \(\rho\) is the multi-view pointer scalar to determine the weight of each view of the probability distribution.

Figure 3: Gated Selective Pointer-Generator Network.

 



### End-to-end Training

**Multi-view Coverage Loss.** The original _coverage mechanism_(See et al., 2017) could only prevent repeated attention from one certain source text. However, the repetition problem becomes more severe, as we leverage both the question and document as the source text. Besides, similar to multi-view pointer network, coverage losses of different sources are supposed to be weighted by their contribution. Therefore, we design a multi-view coverage mechanism to address this issue as well as balance the generating and copying processes.

In each decoder timestep \(t\), the coverage vector \(c_{t}=\sum_{t^{\prime}=0}^{t-1}a_{t^{\prime}}\) is used to represent the degree of coverage so far. The coverage vector \(c_{t}\) will be applied to compute the attention weight \(\alpha_{t}\) in Equations 19 and 21. The coverage loss is trained to penalize the repetition in updated attention weight \(\alpha^{t}\) from all views. The re-normalized pointer weights \(\hat{\rho}=\rho^{c}/\sum_{c\in\{q,d\}}\rho^{c}\) are employed to balance the coverage loss of different views:

\[L_{cov}=\sum\hat{\rho}\frac{1}{T}\sum\nolimits_{t=1}^{T}\sum\nolimits_{i} \text{min}(\alpha_{t}^{i},c_{t}^{i}). \tag{33}\]

Overall Loss Function.The overall model is trained to minimize the negative log likelihood and the multi-view coverage loss:

\[L=-\frac{1}{T}\sum\nolimits_{t=0}^{T}\text{log}P(w_{t}^{*})+\lambda L_{cov}, \tag{34}\]

where \(\lambda\) is a hyper-parameter to balance losses.

## 4 Experiments

### Datasets and Evaluation Metrics

We evaluate on a large-scale summarization dataset with non-factoid questions, WikiHow (Koupaee and Wang, 2018), and a non-factoid QA dataset with abstractive answers, PubMedQA Jin et al. (2019). WikiHow is an abstractive summarization dataset collected from a community-based QA website, _WikiHow1_, in which each sample consists of a non-factoid question, a long article, and the corresponding summary as the answer to the given question. PubMedQA is a conclusion-based biomedical QA dataset collected from _PubMed2_ abstracts, in which each instance is composed of a question, a context, and an abstractive answer which is the summarized conclusion of the context corresponding to the question. The statistics of the WikiHow and PubMedQA datasets are shown in Table 13. We adopt ROUGE F1 (R1, R2, RL) for automatically evaluating the summarized answers. Besides, human evaluation and Distinct scores are adopted for analysis.

Footnote 1: [https://www.wikihow.com](https://www.wikihow.com)

Footnote 2: [https://www.ncbi.nlm.nih.gov/pubmed/](https://www.ncbi.nlm.nih.gov/pubmed/)

### Baseline Methods and Implementations

To evaluate the proposed method, we compare with several baselines and state-of-the-art methods on query-based abstractive summarization and generative QA. We first employ four widely-adopted summarization baseline methods, including two unsupervised extractive methods, **LEAD3** and **MMR**, and two abstractive methods, **S2SA**Bahdanau et al. (2015), and **PGN**Se et al. (2017).

Then two popular query-based abstractive summarization methods are adopted for evaluation: (1) **SD\({}_{2}\)**Nema et al. (2017), which is a sequence-to-sequence model with a query attention, and (2) **QS**Hasselqvist et al. (2017), which incorporates question information into the pointer-generator network with the vanilla attention mechanism.

Finally, we implement two latest generative QA models for comparisons: (1) **S2S-MT**Fan et al. (2019), which uses a multi-task Seq2Seq model with the concatenation of question and support document, and (2) **QPGN**Deng et al. (2020), which is a question-driven pointer-generator network with co-attention between the question and document.

We train all the models with pre-trained GloVE embeddings4 of 300 dimensions and set the vocabulary size to 50k. During training and testing procedure, we restrict the length of generated summaries within 50 words. As for the proposed method, we train with a learning rate of 0.15 and an initial accumulator value of 0.1. The dropout rate is set to 0.5. The hidden unit sizes of the BiLSTM encoder and the LSTM decoder are all set to 256. We train our models with the batch size of 32. All other parameters are randomly initialized from [-0.05, 0.05]. Similar to the original coverage loss (See

\begin{table}
\begin{tabular}{p{113.8pt} p{113.8pt} p{113.8pt}} \hline \hline Dataset & WikiHow & PubMedQA \\ (train/dev/test) & & \\ \hline \#Samples & 168K / 6K / 6K & 169K / 21K / 21K \\ Avg QLLen & 7.00 / 7.02 / 7.01 & 16.3 / 16.4 / 16.3 \\ Avg DLLen & 582 / 580 / 584 & 238 / 238 / 239 \\ Avg ALen & 62.2 / 62.2 / 62.2 & 41.0 / 41.0 / 40.9 \\ Avg fSents/Doc & 20.7 / 20.7 / 20.6 & 9.32 / 9.31 / 9.33 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Statictis of Dataset

et al., 2017), we first train the model without multi-view coverage loss for 20 epochs, and then train with it for another 5 epochs with \(\lambda\) as 0.1.

### Performance Comparison

Table 2 summarizes the experimental results on both datasets. As for WikiHow, which is an abstractive summarization dataset with non-factoid questions, current query-based summarization (SD\({}_{2}\), QS) and generative QA approaches (S2S-MT, QPGN) barely improve the performance from traditional summarization approaches. It indicates that the question information is not fully exploited for summarization, while MSG outperforms all these methods with a noticeable margin, about 2%.

Besides, since PubMedQA is a QA dataset with abstractive answers, we can observe that QPGN, which employs special design for modeling the interaction between the question and document, achieves relatively better performance than other summarization methods. Favorably MSG raises the state-of-the-art result by about 3%. Furthermore, MSG achieves promising improvements via the multi-hop inference on these two datasets.

We conduct human evaluation to evaluate the generated answer from four aspects: (1) Informativity: how rich is the generated answer in information? (2) Conciseness: how concise is the summary? (3) Readability: how fluent and coherent is the summary? (4) Correctness: how well does the generated answer respond to the given question? We randomly sample 50 questions from two datasets and generate their answers with three query-based summarization methods, including SD\({}_{2}\), QS, QPGN and the proposed MSG. Three annotators are asked to score each generated answer with 1 to 5 (higher the better). Results are presented in Table 3. We observe that MSG consistently and substantially outperforms existing query-based summarization methods in all aspects, especially for the informativeness and correctness. The results show that MSG effectively generates concise but also informative answers, since MSG not only considers question-related information, but also captures logically necessary content for answering the given question via multi-hop reasoning. Consequently, it leads to a more precise answer.

## 5 Discussions

### Ablation Study

We conduct ablation study to validate the effectiveness of different components in MSG as well as the detailed design for the multi-hop inference module. The upper part in Table 4 presents the ablation study on multi-hop inference module. First of all, the model performance suffers a great decrease from discarding the multi-hop inference module on two datasets, showing the necessity of incorporating the multi-hop reasoning into the question-driven summarization. In specific, the fusion of the selective sentence representations from all hops brings performance improvement, including aggre

\begin{table}
\begin{tabular}{l c c c c} \hline \multirow{2}{*}{Model} & \multicolumn{2}{c}{WikiHow} & \multicolumn{2}{c}{PubMededQA} \\ \cline{2-5}  & R1 & RL & R1 & RL \\ \hline
* multi-hop inference & 29.5 & 28.4 & 35.7 & 29.2 \\
- hops aggregation\({}^{1}\) & 30.1 & 29.0 & 37.0 & 30.1 \\
- hops attention & 30.3 & 29.2 & 37.0 & 30.1 \\
- MAR unit\({}^{2}\) & 30.0 & 29.1 & 36.8 & 30.0 \\ \hline
- co-attention & 30.2 & 29.0 & 37.0 & 30.1 \\
- gated attention\({}^{3}\) & 30.2 & 28.9 & 36.6 & 29.8 \\
- question pointer & 30.3 & 29.1 & 35.5 & 29.1 \\
- MVC loss & 29.6 & 28.5 & 35.9 & 29.3 \\ \hline \end{tabular}
\end{table}
Table 4: Ablation Study on Model Components. \({}^{1}\)Use the sentence representation learned from the last hop, instead of merging all the hops. \({}^{2}\)Replace all the MAR Unit with Attentive Unit. \({}^{3}\)Replace the _sigmoid_ function with _softmax_ function.

\begin{table}
\begin{tabular}{l c c c c c c} \hline \multirow{2}{*}{Model} & \multicolumn{2}{c}{WikiHow} & \multicolumn{2}{c}{PubMedQA} \\ \cline{2-7}  & R1 & R2 & RL & R1 & R2 & RL \\ \hline LEAD3 & 26.0\({}^{*}\) & 7.2\({}^{*}\) & 24.3\({}^{*}\) & 30.9 & 9.8 & 21.2 \\ MMR & 26.8 & 6.1 & 23.6 & 30.1 & 9.0 & 24.4 \\ \hline S2SA & 22.0\({}^{*}\) & 6.3\({}^{*}\) & 20.9\({}^{*}\) & 32.4 & 11.0 & 27.3 \\ PGN & 28.5\({}^{*}\) & 9.2\({}^{*}\) & 26.5\({}^{*}\) & 32.9 & 11.5 & 28.1 \\ \hline SD\({}_{2}\) & 27.7 & 7.9 & 25.8 & 32.3 & 10.5 & 26.0 \\ QS & 28.8 & 9.9 & 27.6 & 32.6 & 11.1 & 26.7 \\ S2S-MT & 28.6 & 9.6 & 27.5 & 33.2 & 12.2 & 27.8 \\ QPGN & 28.8 & 9.7 & 27.7 & 34.2 & 12.8 & 28.7 \\ \hline \hline MSG (1-Hop) & 30.0 & 10.2 & 29.0 & 36.5 & 14.4 & 30.0 \\ MSG (2-Hop) & 30.2 & 10.3 & 29.1 & 37.0 & 14.7 & **30.4** \\ MSG (3-Hop) & **30.5** & **10.5** & **29.3** & **37.2** & **14.8** & 30.2 \\ \hline \end{tabular}
\end{table}
Table 2: Results on WikiHow and PubMedQA. \({}^{*}\) represents results reported from Koupaee and Wang (2018).

\begin{table}
\begin{tabular}{c c c c} \hline \multirow{2}{*}{Model} & Info & Conc & Read & Corr \\ \hline SD\({}_{2}\) & 3.48 & 3.34 & 3.30 & 3.04 \\ QS & 3.62 & 3.30 & 3.48 & 3.24 \\ QPGN & 3.58 & 3.52 & 3.68 & 3.32 \\
**MSG** & **4.14** & **3.88** & **3.82** & **3.78** \\ \hline \end{tabular}
\end{table}
Table 3: Human Evaluation Results 

gating all the hops as well as applying attention to weight the importance of each hop. Besides, it also achieves better performance to apply the proposed MAR Unit as the multi-hop unit, instead of repeatedly using Attentive Unit, indicating that it is not enough to only consider the question-related information, while the interrelationship among different sentences also attaches great importance.

The second part in Table 4 presents the ablation study in terms of discarding other model components in MSG. In general, all the components contribute to the final performance to a certain extent. In detail, there are several notable observations: (1) Some existing works Hsu et al. (2018); Nishida et al. (2019b) apply softmax function to normalize the weights of different sentences in the decoding phase, which falls short of differentiating the importance degree of each sentence. The result shows that MSG achieves better performance by employing gated attention to distinguish salient justification sentences for generating the summaries. (2) Discarding the question pointer casts a noticeably greater decrease on PubMedQA than WikiHow. We conjecture that those questions from PubMedQA contain more words available to be copied for generating precise summaries, as the statistic of the question length shown in Table 1. These results also validate the importance of multi-view PGN on question-driven abstractive summarization, which is underutilized in current methods. (3) Multi-view coverage (MVC) loss makes a great contribution to the performance by alleviating the severe repetition problem along with the multi-view PGN.

### Analysis of Multi-hop Reasoning

As the results presented in Section 4.3, MSG (3-Hop) outperforms MSG (1-Hop) by 0.5% and 0.7% on WikiHow and PubMedQA, respectively, indicating the effectiveness of incorporating multi-hop reasoning in question-driven summarization. Figure 4(a) presents the model performance in terms of using different hops of reasoning. We can see that, as expected, the performance of the model begins with growth when increasing the number of hops for reasoning. However, the performance becomes generally unchanged (e.g., WikiHow) or even slightly decreases (e.g., PubMedQA) when we further increase the number of hops. In practice, it is actually unnecessary to reason for too many hops, which may cause over-fitting. And adopting 3-hops in the implementation can be regarded as a hyper-parameter that is tuned on the datasets.

In addition, we extract and normalize the sentence weights from Eq. 7&9 to analyze some characteristics of the justification sentences in multi-hop inference. Figure 4(b) summarizes the statistic result of the sentence importance degree in each hop. We observe that the most important sentences in the 1st-hop of reasoning are likely to appear at the beginning of the document, while those in the 3rd-hop are concentrated in the latter part of the document. Comparatively, the important sentences in the 2nd-hop appear equally in all positions of the document. The results show that the proposed multi-hop inference procedure of justification sentences is generally in accordance with human-like reading habits.

### Case Study

We present a case study in Figure 5 with generated answers from the proposed method and some baseline methods, QPGN, QS, and SD\({}_{2}\), to intuitively compare these methods. With the multi-hop reasoning process in MSG, we can obtain a clear clue of how to answer the given question. As it can be observed that the reference answer is composed of the information from the **1st-hop** and **3rd-hop** inference sentences, it is inadequate to simply summarize the question-related content for generating the answer. For the generated summaries, there are several observations as follows: (1) MSG (3-hop) successfully summarizes the source document with all the necessary and correct information. (2) MSG (2-hop) also effectively summarizes the 1st-hop and 2nd-hop inference content in the document. However, in this case, 3-hop inference is required to answer the given question. (3) MSG (1-hop) only measures the semantic relevance to the given question, leading to an incomplete summary that is lack of some necessary content, and even introduces some general sentences due to the data-driven learning. (4) QPGN only considers the

Figure 4: Analysis of Multi-hop Reasoning



semantic relevance to the given question, leading to an incomplete summary that is lack of some necessary content. (5) QS and SD\({}_{2}\) fail to capture the key information, resulting in generating irrelevant summaries to the given question, or producing some general sentences due to the data-driven learning. It shows the capability of MSG to implement multi-hop reasoning and provide justification sentences.

Additionally, we observe that many cases probably require more than 3-hop inference or only involve one or two hops. However, we can still evaluate how MSG works in these cases. Compared to the reference answer, MSG (3-hop) can still capture most of the useful information to generate a good summary for answering the question. Besides, MSG (2-hop) and MSG (1-hop) also manage to attend some important content in the document. In general, our model is able to only attend a single hop if one-hop is enough, while our model may regard several hops as an integral hop when more hops are required. However, the baseline methods introduce much unnecessary or even incorrect information into the summarized answers.

### Duplication Analysis in Answers

We adopt Distinct scores to analyze whether the multi-view coverage mechanism can alleviate the repetition issue in the generation procedure of multi-view PGN. Figure 6 summarizes the percentage of n-grams duplication on the ground-truth answers and the generated answers with or without the original [40] and multi-view coverage mechanism. We observe that the original coverage mechanism can still reduce word repetition in multi-view PGN. Moreover, multi-view coverage further reduces the ratio of duplication to a great extent, since multi-view coverage not only prevents repeatedly attending to the same element in both question and document, but also balances the weight of penalty between them.

## 6 Conclusion

We propose a novel question-driven abstractive summarization method, Multi-hop Selective Generator (MSG), to summarize concise but informative answers for non-factoid QA. We incorporate multi-hop reasoning to infer justification sentences for abstractive summarization. Experimental results show that the proposed method achieves state-of-the-art performance on two benchmark non-factoid QA datasets, namely WikiHow and PubMedQA.

## References

* Bahdanau et al. (2015) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In _3rd Inter

Figure 5: A case study with the same legend as Figure 1. The **highlighted** sentences are attended by MSG (3-hop).

Figure 6: Duplication Analysis in Answers



national Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_.
* Baumel et al. (2016) Tal Baumel, Raphael Cohen, and Michael Elhadad. 2016. Topic concentration in query focused summarization datasets. In _Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12-17, 2016, Phoenix, Arizona, USA_, pages 2573-2579.
* Cao et al. (2016) Ziqiang Cao, Wenjie Li, Sujian Li, Furu Wei, and Yanran Li. 2016. Attsum: Joint learning of focusing and summarization with neural attention. In _COLING 2016, 26th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, December 11-16, 2016, Osaka, Japan_, pages 547-556.
* Carbonell and Goldstein (1998) Jaime G. Carbonell and Jade Goldstein. 1998. The use of mmr, diversity-based reranking for reordering documents and producing summaries. In _SIGIR '98: Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, August 24-28 1998, Melbourne, Australia_, pages 335-336.
* Volume 1: Long Papers_, pages 582-591.
* Deng et al. (2020a) Yang Deng, Wai Lam, Yuexiang Xie, Daoyuan Chen, Yaliang Li, Min Yang, and Ying Shen. 2020a. Joint learning of answer selection and answer summary generation in community question answering. In _The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, New York, NY, USA, February 7-12, 2020_, pages 7651-7658.
* Deng et al. (2020b) Yang Deng, Wenxuan Zhang, Yaliang Li, Min Yang, Wai Lam, and Ying Shen. 2020b. Bridging hierarchical and sequential context modeling for question-driven extractive answer summarization. In _Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020, Virtual Event, China, July 25-30, 2020_, pages 1693-1696.
* Fan et al. (2019) Angela Fan, Yacine Jernite, Ethan Perez, David Grangier, Jason Weston, and Michael Auli. 2019. ELI5: long form question answering. In _Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers_, pages 3558-3567.
* Feldman and El-Yaniv (2019) Yair Feldman and Ran El-Yaniv. 2019. Multi-hop paragraph retrieval for open-domain question answering. In _Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers_, pages 2296-2309.
* Hasselqvist et al. (2017) Johan Hasselqvist, Niklas Helmertz, and Mikael Kageback. 2017. Query-based abstractive summarization using neural networks. _CoRR_, abs/1712.06100.
* Hsu et al. (2018) Wan Ting Hsu, Chieh-Kai Lin, Ming-Ying Lee, Kerui Min, Jing Tang, and Min Sun. 2018. A unified model for extractive and abstractive summarization using inconsistency loss. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long Papers_, pages 132-141.
* February 1, 2019_, pages 142-151.
* Ishida et al. (2018)Ryo Ishida, Kentaro Torisawa, Jong-Hoon Oh, Ryu Iida, Canasai Kruegkrai, and Julien Kloetzer. 2018. Semi-distantly supervised neural model for generating compact answers to open-domain why questions. In _Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018_, pages 5803-5811.
* Jin et al. (2019) Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William W. Cohen, and Xinghua Lu. 2019. Pubmedqa: A dataset for biomedical research question answering. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019_, pages 2567-2577.
* Kocisky et al. (2018) Tomas Kocisky, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gabor Melis, and Edward Grefenstette. 2018. The narrativeqa reading comprehension challenge. _Trans. Assoc. Comput. Linguistics_, 6:317-328.
* Koupaee and Wang (2018) Mahnaz Koupaee and William Yang Wang. 2018. Wikihow: A large scale text summarization dataset. _CoRR_, abs/1810.09305.
* Li and Li (2014) Yanran Li and Sujian Li. 2014. Query-focused multidocument summarization: Combining a topic model with graph-based semi-supervised learning. In _COLING 2014, 25th International Conference on Computational Linguistics, Proceedings of the Conference. Technical Papers, August 23-29, 2014, Dublin, Ireland_, pages 1197-1207.
* Lin et al. (2010) Jimmy J. Lin, Nitin Madnani, and Bonnie J. Dorr. 2010. Putting the user in the loop: Interactive maximal marginal relevance for query-focused summarization. In _Human Language Technologies: Con 

ference of the North American Chapter of the Association of Computational Linguistics, Proceedings, June 2-4, 2010, Los Angeles, California, USA_, pages 305-308.
* Nakatsuji and Okui (2020) Makoto Nakatsuji and Sohei Okui. 2020. Conclusion-supplement answer generation for non-factoid questions. In _The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, New York, NY, USA, February 7-12, 2020_, pages 8520-8527.
* Nakov et al. (2015) Preslav Nakov, Luis Marquez, Walid Magdy, Alessandro Moschitti, James R. Glass, and Bilal Randeree. 2015. Semeval-2015 task 3: Answer selection in community question answering. In _Proceedings of the 9th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2015, Denver, Colorado, USA, June 4-5, 2015_, pages 269-281.
* August 4, Volume 1: Long Papers_, pages 1063-1072.
* Nguyen et al. (2016) Trii Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A human generated machine reading comprehension dataset. In _Proceedings of the Workshop on Cognitive Computation: Integrating neural and symbolic approaches 2016 colocated with the 30th Annual Conference on Neural Information Processing Systems (NIPS 2016), Barcelona, Spain, December 9, 2016_.
* Nishida et al. (2019a) Kosuke Nishida, Kyosuke Nishida, Masaaki Nagata, Atsushi Otsuka, Itsumi Saito, Hisako Asano, and Junji Tomita. 2019a. Answering while summarizing: Multi-task learning for multi-hop QA with evidence extraction. In _Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers_, pages 2335-2345.
* Nishida et al. (2019b) Kyosuke Nishida, Itsumi Saito, Kosuke Nishida, Kazutoshi Shinoda, Atsushi Otsuka, Hisako Asano, and Junji Tomita. 2019b. Multi-style generative reading comprehension. In _Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers_, pages 2273-2284.
* Rajpurkar et al. (2016) Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100, 000+ questions for machine comprehension of text. In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016_, pages 2383-2392.
* August 4, Volume 1: Long Papers_, pages 1073-1083.
* Shen and Li (2011) Chao Shen and Tao Li. 2011. Learning to rank for query-focused multi-document summarization. In _11th IEEE International Conference on Data Mining, ICDM 2011, Vancouver, BC, Canada, December 11-14, 2011_, pages 626-634.
* Song et al. (2017) Hongya Song, Zhaochun Ren, Shangsong Liang, Pijil Li, Jun Ma, and Maarten de Rijke. 2017. Summarizing answers in non-factoid community question-answering. In _Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, WSDM 2017, Cambridge, United Kingdom, February 6-10, 2017_, pages 405-414.
* Tomasoni and Huang (2010) Mattia Tomasoni and Minlie Huang. 2010. Metadata-aware measures for answer summarization in community question answering. In _ACL 2010, Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, July 11-16, 2010, Uppsala, Sweden_, pages 760-769.
* Wang et al. (2014) Lu Wang, Herna Raghavan, Claire Cardie, and Vittorio Castelli. 2014. Query-focused opinion summarization for user-generated content. In _COLING 2014, 25th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, August 23-29, 2014, Dublin, Ireland_, pages 1660-1669.
* Wang et al. (2013) Lu Wang, Hema Raghavan, Vittorio Castelli, Radu Florian, and Claire Cardie. 2013. A sentence compression based framework to query-focused multi-document summarization. In _Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, ACL 2013, 4-9 August 2013, Sofia, Bulgaria, Volume 1: Long Papers_, pages 1384-1394.
* Yadav et al. (2019) Vikas Yadav, Steven Bethard, and Mihai Surdeanu. 2019. Quick and (not so) dirty: Unsupervised selection of justification sentences for multi-hop question answering. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019_, pages 2578-2589.
* Yulianti et al. (2018) Evi Yulianti, Ruey-Cheng Chen, Falk Scholer, W. Bruce Croft, and Mark Sanderson. 2018. Document summarization for answering non-factoid queries. _IEEE Trans. Knowl. Data Eng._, 30(1):15-28.
* Zhang et al. (2020) Wenxuan Zhang, Yang Deng, and Wai Lam. 2020. Answer ranking for product-related questions via multiple semantic relations modeling. In _Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020, Virtual Event, China, July 25-30, 2020_, pages 569-578