SlowGuess/ACL_2020_OCR huggingface dataset:
Luyang Huang $^{1}$  Lingfei Wu $^{2}$  and Lu Wang $^{1}$
<sup>1</sup>Khoury College of Computer Sciences, Northeastern University, Boston, MA 02115
$^{2}$ IBM Research AI, IBM T.J. Watson Research Center, Yorktown Heights, NY 10598
luyang.huang96@gmail.com, luwang@ccs.neu.edu
wuli@us.ibm.com
Sequence-to-sequence models for abstractive summarization have been studied extensively, yet the generated summaries commonly suffer from fabricated content, and are often found to be near-extractive. We argue that, to address these issues, the summarizer should acquire semantic interpretation over input, e.g., via structured representation, to allow the generation of more informative summaries. In this paper, we present ASGARD, a novel framework for Abstractive Summarization with GraphAugmentation and semantic-driven RewarD. We propose the use of dual encoders—a sequential document encoder and a graph-structured encoder—to maintain the global context and local characteristics of entities, complementing each other. We further design a reward based on a multiple choice cloze test to drive the model to better capture entity interactions. Results show that our models produce significantly higher ROUGE scores than a variant without knowledge graph as input on both New York Times and CNN/Daily Mail datasets. We also obtain better or comparable performance compared to systems that are finetuned from large pretrained language models. Human judges further rate our model outputs as more informative and containing fewer unfaithful errors.
Abstractive summarization aims to produce concise and informative summaries with the goal of promoting efficient information consumption and knowledge acquisition (Luhn, 1958). Significant progress has been made in this area by designing sequence-to-sequence-based neural models for single-document abstractive summarization (Gehrmann et al., 2018; Liu et al., 2018; Liu and Lapata, 2019). However, due to the limitations of model structure and word prediction-based
Input Article of New York Times:
John M. Fabrizi, the mayor of Bridgeport, admitted on Tuesday that he had used cocaine and abused alcohol while in office.
Mr. Fabrizi, who was appointed mayor in 2003 after the former mayor, Joseph P. Ganim, went to prison on corruption charges, said he had sought help for his drug problem about 18 months ago and that he had not used drugs since.
About four months ago, he added, he stopped drinking alcohol.
Constructed Graph:
Figure 1: Sample knowledge graph constructed from an article snippet. The graph localizes relevant information for entities (color coded, e.g. "John M. Fabrizi") or events (underlined) and provides global context.
Summary by Human:
The Week column. Mayor John Fabrizi of Brigeport, Conn, publicly admits he used cocaine and abused alcohol while in office; says he stopped drinking alcohol and sought help for his drug problem about 18 months ago.
learning objectives, these models frequently produce unfaithful content (Cao et al., 2018) and near-extractive summaries (See et al., 2017; KrySciński et al., 2018). These observations suggest that existing models lack semantic interpretation over the input, which is critical for summarization.
We argue that the generation of informative and succinct abstracts requires structured representation to facilitate the connection of relevant subjects, and the preservation of global context, e.g. entity interactions and topic flows. Take Fig. 1 as an ex
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5094-5107 July 5 - 10, 2020. ©2020 Association for Computational Linguistics
ample. Complex events related with the same entity may span multiple sentences, making it challenging for existing sequential models to capture. A graph representation, on the contrary, produces a structured summary and highlights the proximity of relevant concepts.
To this end, we present ASGARD, a framework for Abstractive Summarization with GraphAugmentation and semantic-driven Reward. Under the encoder-decoder framework, we enhance the regular document encoder with a separate graph-structured encoder to maintain the global context and local characteristics of entities by using the outputs from an open information extraction (OpenIE) system.
Specifically, we experiment with two graph variants, one mainly capturing entities' document-level interactions and the other reflecting such interactions within each paragraph plus topic shifts across paragraphs. Both graphs can capture interactions among entities that are positioned far from one another in the document and significantly reduce redundancy, as shown in Fig. 1. The document encoder and the graph encoder then cooperate during abstract generation, wherein the model is trained to identify salient content by aligning graphs with human summaries. Though structured representation has been studied before for summarization (Fernandes et al., 2019), to the best of our knowledge, we are the first to utilize graph neural networks to explicitly encode entity-centered information for abstractive summary generation.
Moreover, we propose a novel multi-choice cloze reward to drive the model to acquire semantic understanding over the input. Concretely, we design cloze questions by removing pairwise entities that are connected with a predicate or co-occur in a human summary sentence, whereas prior work only considers single entities to construct questions (Eyal et al., 2019). In tandem with our graph encoding of knowledge, the cloze reward further facilitates the acquisition of global entity interactions with reinforcement learning.
We carry out automatic and human evaluations on popular summarization datasets. Models based on ASGARD yield significantly better ROUGE scores (Lin and Hovy, 2003) than a variant without access to the knowledge graph on two popular news summarization datasets, New York Times
corpus and CNN/Daily Mail dataset. Moreover, ASGARD models attain performance better than or comparable to others that are fine-tuned from large pretrained language models, including BERT-Sum (Liu and Lapata, 2019), UniLM (Dong et al., 2019), and BART (Lewis et al., 2019). Human judges further confirm that our models generate more informative summaries with less unfaithful errors than their counterparts without the graph encoder. Importantly, we find that automatic evaluation metrics only weakly correlate with these errors, implying that new evaluation methods are needed to better gauge summary quality.
The rest of the paper is organized as follows. We describe related work in the next section (§ 2). We then discuss the knowledge graph construction in § 3 and formulate our graph-augmented summarization framework in § 4. In § 5, we introduce reinforcement learning with cloze reward. Experiments and results are presented in § 6 and § 7. Finally, we conclude in § 8.
Graph-Augmented Summarization and Generation. Graph structures have long been used for extractive summarization, such as in Textrank (Mihalcea and Tarau, 2004) and Lexrank (Erkan and Radev, 2004). For neural models, Tan et al. (2017) design graph-based attention to identify important sentences. For generating abstractive summaries, Fernandes et al. (2019) enhance a sequence-based encoder with graph neural networks (GNNs) to consider token-level entity types, however, entity interactions are largely ignored. On multi-document summarization, Fan et al. (2019) demonstrate the usefulness of encoding a linearized knowledge graph from OpenIE outputs. In this work, we design a graph encoder, which improves upon Graph Attention Networks (GATs) (Veličković et al., 2018), to capture the global context in a more effective manner.
Also related is the graph-to-sequence framework that has been adopted for text generation (Song et al., 2018). Both Gated Graph Neural Networks (GGNNs) (Beck et al., 2018) and Graph Convolutional Networks (GCNs) (Damonte and Cohen, 2019) are shown to be effective in generating sentences from AMR graphs. Since Graph Attention Networks can better handle sparse graphs, they are used by Koncel-Kedziorski et al. (2019) with a transformer model to create scientific paper ab
<sup>1</sup>Our code is available at https://github.com/luyang-huang96/GraphAugmentedSum.
tracts from knowledge graphs. Here we use graphs in addition to document encoder, both carrying complementary information for summarization.
Reinforcement Learning and QA Reward for Abstractive Summarization. As pointed out by Ranzato et al. (2016), word-level maximum likelihood training brings the problem of exposure bias. Recent work utilizes reinforcement learning to directly optimize the model to maximize the informativeness of summaries by using different forms of ROUGE scores (Paulus et al., 2018; Chen and Bansal, 2018; Sharma et al., 2019). However, ROUGE does not always distinguish good summaries from bad ones (Novikova et al., 2017), and ignores entity interactions.
Since question answering (QA) has been used for summary evaluation (Narayan et al., 2018), and is shown to correlate with human judgment of summaries qualities (Eyal et al., 2019), QA-based rewards have been studied for summarization model training. Arumae and Liu (2019) demonstrate that using fill-in-the-blank questions by removing entities or root words leads to improved content selection. Scialom et al. (2019) consider a similar setup, but use both F1 score and QA system confidence as rewards in abstractive summarization. Previous work, however, mainly focuses on single entities or words in human-written summaries, thereby losing contexts and relations. Moreover, fill-in-the-blank questions by prior work give credits only when the answers exactly match the ground-truths, thus causing inaccuracies for rephrased answers and discouraging abstract content generation. In contrast, we design a semantic-driven cloze reward by measuring how well a QA system can address multiple choice cloze questions which better encode entity interactions and handle paraphrased answers.
3 Knowledge Graph Construction
To construct a knowledge graph from an input document, we utilize Stanford CoreNLP (Manning et al., 2014) to first obtain outputs from coreference resolution and open information extraction (OpenIE) models (Angeli et al., 2015). Note that we do not conduct global entity linking across documents. Next, we take the <subject, predicate, object> triples extracted by OpenIE and remove any triple whose argument (subject or object) has more than 10 words. If two triples differ only by one argument, and the arguments overlap, we keep the longer triple.
Figure 2: Our ASGARD framework with document-level graph encoding. Summary is generated by attending to both the graph and the input document.
We begin constructing the graph by treating subjects and objects as nodes connected by directed edges, with predicates as attributes. We further collapse coreferential mentions of the same entity into one node. With this, we can localize salient content related to each entity as well as make connections of spread-out entities through graph paths.
4 Summarization Model
In this section, we describe our graph-augmented abstractive summarization framework, as displayed in Fig. 2. Our model takes as input a document, represented as a sequence of tokens  $\mathbf{x} = \{x_{k}\}$ , and a knowledge graph  $G$  consisting of nodes  $\{v_{i}\}$ .  $\mathbf{x}$  and  $G$  are separately consumed by a document encoder and a graph encoder, as presented in § 4.1. Importantly, we present two types of graphs: DOCGRAPH, focusing on the global context, and SEGGRAPH, which additionally captures topic shift. The summary decoder then generates an abstractive summary by attending to both the document and the graph (§ 4.2). In § 4.3, we formulate a maximum likelihood training objective which leverages the detection of salient nodes in the graph.
Document Encoder. We first feed input  $\mathbf{x}$  to RoBERTa (Liu et al., 2019) and take the last layer
output as token embeddings. We then employ a single-layer bidirectional LSTM (BiLSTM) over token embeddings, producing encoder hidden states  $\mathbf{h}_k$  at time step  $k$ .
Graph Encoder. Built on the graph constructed in § 3, we create nodes for predicates as done in previous graph-to-sequence work (Beck et al., 2018) to reduce model parameters. Directed, unlabeled edges are added from subject to predicate, and from predicate to object. We further add reverse edges and self-loops to enhance the information flow, and this forms the graph  $G$ .
Node Initialization. Each node often contains multiple mentions of an entity; we thus initialize node representation  $\mathbf{v}_i$  by using the average embedding of its tokens. We leverage document encoder hidden states  $\mathbf{h}_k$  as the contextual representation of tokens. Number of mentions in the node is added as an extra encoding to  $\mathbf{v}_i$ , to signify entity salience.
Contextualized Node Encoding. Our graph encoder improves upon Graph Attention Networks (GATs) (Veličković et al., 2018) by adding residual connections between layers as discussed in Koncel-Kedziorski et al. (2019). Each node  $\mathbf{v}_i$  is represented by a weighted average of its neighbors:
$$
\hat {\mathbf {v}} _ {i} = \mathbf {v} _ {i} + \left\| _ {n = 1} ^ {N} \sum_ {v _ {j} \in \mathcal {N} (v _ {i})} \alpha_ {i, j} ^ {n} \mathbf {W} _ {0, n} \mathbf {v} _ {j} \right. \tag {1}
$$
$$
\alpha_ {i, j} ^ {n} = \operatorname {s o f t m a x} \left(\left(\mathbf {W} _ {1, n} \mathbf {v} _ {i}\right) ^ {T} \left(\mathbf {W} _ {2, n} \mathbf {v} _ {j}\right)\right) \tag {2}
$$
where  $\|_{n = 1}^{N}$  denotes the concatenation of  $N$  heads, each producing a vector of the same dimension as  $\mathbf{v}_i$ . We use  $N = 4$  in our experiments with two layers of GATs.  $\mathcal{N}(v_i)$  denotes the neighbors of  $v_{i}$  in graph  $G$ .  $\mathbf{W}_{*}$  are trainable parameters.
The graph encoder described above encodes document-level global context by merging entity mentions throughout the document and capturing their interactions with graph paths. It is henceforth denoted as DOCGRAGH.
Encoder Extension to Capture Topic Shift (SEGGRAGH). Modeling topic transitions and recurrences enables the identification of notable content, thus benefiting summarization (Barzilay and Lee, 2004). Since paragraphs naturally divide a document into different topic segments, we extend DocGragh by first encoding each paragraph as a subgraph  $G_{p}$  (for the  $p$ -th paragraph) using the same graph encoder, and then connecting all subgraphs with a BiLSTM. If two nodes in separate subgraphs refer to the same entity, they are initial-
ized with the same embedding (as in the first occurrence). Concretely, we first apply max-pooling over all nodes in subgraph  $G_{p}$  from the outputs of the final GAT layer; the max-pooling results are then used as inputs for a BiLSTM to produce the final subgraph representation  $\mathbf{h}_{p}^{q}$  for  $G_{p}$ .
4.2 Summary Decoder
Our summary decoder uses a single-layer unidirectional LSTM with a hidden state  $\mathbf{s}_t$  at step  $t$ ; it generates summary tokens recurrently by jointly attending to the input document and the graph.
Attending the Graph. At each decoding step  $t$ , we compute a graph context vector  $\mathbf{c}_t^v$  with the attention mechanism (Bahdanau et al., 2014):
$$
\mathbf {c} _ {t} ^ {v} = \sum_ {i} a _ {i, t} ^ {v} \hat {\mathbf {v}} _ {i} \tag {3}
$$
$$
a _ {i, t} ^ {v} = \operatorname {s o f t m a x} \left(\mathbf {u} _ {0} ^ {T} \tanh  \left(\mathbf {W} _ {3} \mathbf {s} _ {t} + \mathbf {W} _ {4} \hat {\mathbf {v}} _ {i}\right)\right) \tag {4}
$$
where  $\mathbf{u}_{*}$  are also trainable parameters. We omit bias terms for simplicity.
Attending the Document. Similarly, the document context  $\mathbf{c}_t$  is computed over input tokens by additionally considering the graph context  $\mathbf{c}_t^v$ :
$$
\mathbf {c} _ {t} = \sum_ {k} a _ {k, t} \mathbf {h} _ {k} \tag {5}
$$
$$
a _ {k, t} = \operatorname {s o f t m a x} (
$$
$$
\mathbf {u} _ {1} ^ {T} \tanh (\mathbf {W} _ {5} \mathbf {s} _ {t} + \mathbf {W} _ {6} \mathbf {h} _ {k} + \mathbf {W} _ {7} \mathbf {c} _ {t} ^ {v})) \qquad (6)
$$
Token Prediction. Graph and document context vectors, treated as salient content summarized from both sources, are concatenated with the decoder hidden state  $\mathbf{s}_t$  to produce the vocabulary distribution  $P_{\text{vocab}}$ :
$$
P _ {v o c a b} = \operatorname {s o f t m a x} \left(\mathbf {W} _ {o u t} \left[ \mathbf {s} _ {t} \mid \mathbf {c} _ {t} \mid \mathbf {c} _ {t} ^ {v} \right]\right) \tag {7}
$$
We use weight-sharing between the input embedding matrix and the matrix  $\mathbf{W}_{out}$  to allow reusing linguistic knowledge as proposed by Paulus et al. (2018). We further add a copy mechanism similar to See et al. (2017), with copy probability as:
$$
P _ {c o p y} = \sigma \left(\mathbf {W} _ {c o p y} \left[ \mathbf {s} _ {t} \mid \mathbf {c} _ {t} \mid \mathbf {c} _ {t} ^ {v} \mid \mathbf {y} _ {t - 1} \right]\right) \tag {8}
$$
where  $\mathbf{y}_{t - 1}$  denotes the embedding for the token predicted at step  $t - 1$
Modified Hierarchical Attention for SegGraph. As mentioned in § 4.1, SegGraph captures content salience by modeling topic shift across paragraphs. We thus seek to leverage paragraph-level importance to redistribute the node attentions, e.g., giving
more attentions to nodes in important paragraphs. In particular, we utilize hierarchical attention (Hsu et al., 2018), where we first calculate attention  $\mathbf{a}_t^g$  over subgraphs as done in Eq. 3 by replacing  $\hat{\mathbf{v}}_i$  with subgraph representation  $\mathbf{h}_p^g$ .
We then combine subgraph attentions  $\mathbf{a}_t^g$  with the previously calculated attentions  $\mathbf{a}_t^v$  for nodes in the subgraph using scalar multiplication and renormalization over all nodes in input. This results in the new attention weights  $\hat{\mathbf{a}}_t^v$ , which are used to obtain graph context vector  $\mathbf{c}_t^v$  as done in Eq. 3 for SegGraph.
We first consider a maximum likelihood (ML) training objective that minimizes the following loss:
$$
\mathcal {L} _ {\mathrm {s e q}} = - \frac {1}{| D |} \sum_ {(\mathbf {y}, \mathbf {x}) \in D} \log p (\mathbf {y} \mid \mathbf {x}; \theta) \tag {9}
$$
where  $\mathbf{x}$  are documents and  $\mathbf{y}$  are references from the training set  $D$ , and  $\theta$  are model parameters.
Node Salience Labeling. In addition to modeling local characteristics of nodes, we further enhance the model by adding an objective to label node salience, e.g., whether the entities in a node are mentioned in the reference summaries. We introduce a soft mask layer over each node before it is passed into the graph encoder, to signify its salience. This layer, serving as an information gate, predicts a real number  $m_{i}$  in [0, 1] for each node  $\mathbf{v}_{i}$  and multiplies to itself, i.e.  $m_{i}\mathbf{v}_{i}$ . For node  $\mathbf{v}_{i}$ , the mask is calculated as  $\hat{m}_i = \mathrm{sigmoid}(\mathbf{u}_2\mathbf{v}_i)$ . During training, the gold-standard mask  $m_{i}$  for a node is set to 1 if it contains at least one content word in the reference summary; otherwise, 0. We add the following objective for all nodes in the dataset  $D$ :
$$
\begin{array}{l} \mathcal {L} _ {\text {m a s k}} = - \frac {1}{N _ {v}} \sum_ {v _ {i} \in D} m _ {i} \log (\hat {m} _ {i}) + \\ (1 - m _ {i}) \log (1 - \hat {m} _ {i}) \tag {10} \\ \end{array}
$$
where  $N_{v}$  represents the number of nodes in the dataset. Finally, the ML training objective takes the following form:  $\mathcal{L}_{\mathrm{ml}} = \mathcal{L}_{\mathrm{mask}} + \mathcal{L}_{\mathrm{seq}}$
5 Reinforcement Learning with Cloze
After maximum likelihood training with  $\mathcal{L}_{\mathrm{ml}}$ , we further design a multiple choice cloze reward in a second-stage reinforcement learning (RL), leading the model to generate more faithful and informative summaries.
For RL, we use a self-critical policy gradient algorithm (Rennie et al., 2017). During training, two summaries are generated: first, a summary  $\mathbf{y}^s$ , sampling tokens based on the probability distribution  $p(\mathbf{y}^s|\mathbf{x};\theta)$  at each decoding step; and second, a baseline summary  $\hat{\mathbf{y}}$  which greedily selects the tokens of the highest probability at each step. The objective of RL is defined based on the rewards of the two summaries,  $R(\mathbf{y}^s)$  and  $R(\hat{\mathbf{y}})$ , as follows:
$$
\begin{array}{l} \mathcal {L} _ {\mathrm {r l}} = \\ - \frac {1}{| D |} \sum_ {(\mathbf {y} ^ {s}, \mathbf {x}) \in D} \left(R \left(\mathbf {y} ^ {s}\right) - R (\hat {\mathbf {y}})\right) \log p \left(\mathbf {y} ^ {s} \mid \mathbf {x}; \theta\right) \tag {11} \\ \end{array}
$$
Our reward function uses the combination of ROUGE and the multiple choice cloze score introduced below, i.e.,  $R(\mathbf{y}) = R_{\text{rouge}}(\mathbf{y}) + \gamma_{\text{cloze}} R_{\text{cloze}}$ . For ROUGE, it considers F1 scores of ROUGE-1, ROUGE-2, and ROUGE-L calculated against the reference summary, and takes the form of  $R_{\text{rouge}}(\mathbf{y}) = \gamma_1 R_{\text{rouge} - 1}(\mathbf{y}) + \gamma_2 R_{\text{rouge} - 2}(\mathbf{y}) + (1 - \gamma_1 - \gamma_2) R_{\text{rouge} - L}(\mathbf{y})$ .
Multiple Choice Cloze Reward. Here, we present a novel multiple choice cloze reward to work with our knowledge graph and guide the summarization model towards improved awareness of entity interactions. We treat the system-generated summary as context. We provide a set of questions automatically constructed from the corresponding reference summary written by a human. We separately train a question answering (QA) model to address the questions by reading the context. Intuitively, if the system summary shares salient information with the reference, the QA model will assign the correct answers with high probability. We decide to use the average probability of the correct answers as our cloze reward. Below, we give details on how to construct the questions and candidate answers with examples shown in Fig. 3.
Question Construction. We run the OpenIE tool on human-written summaries, retaining triples with arguments not longer than 5 words. For each triple of <subject, predicate, object>, we create two types of questions: (1) argument pair questions, by removing the subject and object, and (2) predicate questions, by removing the predicate.
Candidate Answer Construction. Because fill-in-the-blank style cloze may incorrectly penalize QA systems with answers paraphrased from the ground-truth, we opt for a multiple choice cloze. We construct three candidate answers in addition to the
Federal Reserve increases interest rates.
IE Output:
(Federal Reserve, increases, interest rates)
Salient Context:
Federal Reserve signals positivity about the market. Fed increases benchmark interest rate again this May. American economy keeps the high growth rate. Jerome H. Powell discussed potential risks.
IE Outputs:
1. <Federal Reserve, signals, positivity>
2. <American economy, keeps, the high growth rate>
3. <Jerome H. Powell, discussed, potential risks>
Figure 3: Sample construction of multiple choice cloze questions and candidate answers from reference summary and salient context. Arguments and predicates in candidate answers are color-coded and italicized.
Multiple Choice Cloze Questions:
Argument Pair Question: increases
A. Federal Reserve, interest rates (✓)
B. interest rates, Federal Reserve (swapping args in A)
C. American economy, interest rates (replacing arg using triple 2)
D. Federal Reserve, potential risks (replacing arg using triple 3)
Predicate Question: Federal Reserve interest rates.
A. increases  $(\checkmark)$  B. signals C. keeps D. discussed
gold-standard from the salient context, which are summary-worthy sentences selected from the input. Specifically, we use greedy search to select the best combination of sentences that maximizes ROUGE-2 F1 with reference to human summary. We further include a sentence in the salient context if it has a ROUGE-L recall greater than 0.6 when compared with any sentence in the reference.