SlowGuess/ACL_2020_OCR huggingface dataset:
Xingdi Yuan, Tong Wang, Caglar Güçehre, Alessandro Sordoni, Philip Bachman, Saizheng Zhang, Sandeep Subramanian, and Adam Trischler. 2017. Machine comprehension by text-to-text neural question generation. In The 2nd Workshop on Representation Learning for NLP (Rep4NLP@ACL), pages 15-25.
Yao Zhao, Xiaochuan Ni, Yuanyuan Ding, and Qifa Ke. 2018. Paragraph-level neural question generation with maxout pointer and gated self-attention networks. In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3901-3910.
Qingyu Zhou, Nan Yang, Furu Wei, Chuanqi Tan, Hangbo Bao, and Ming Zhou. 2017. Neural question generation from text: A preliminary study. In CCF International Conference of Natural Language Processing and Chinese Computing (NLPCC), pages 662-671.
Here we give a more detailed description for the semantic graph construction, where we have employed two methods: Semantic Role Labelling (SRL) and Dependency Parsing (DP).
A.1 SRL-based Semantic Graph
The primary task of semantic role labeling (SRL) is to indicate exactly what semantic relations hold among a predicate and its associated participants and properties (Màrquez et al., 2008). Given a document  $\mathcal{D}$  with  $n$  sentences  $\{s_1, \dots, s_n\}$ , Algorithm 1 gives the detailed procedure of constructing the semantic graph based on SRL.
We first create an empty graph  $\mathcal{G} = (\mathcal{V},\mathcal{E})$  where  $\nu$  and  $\mathcal{E}$  are the node and edge sets, respectively. For each sentence  $s$ , we use the state-of-the-art BERT-based model (Shi and Lin, 2019) provided in the AllenNLP toolkit to perform SRL, resulting a set of SRL tuples  $\mathcal{S}$ . Each tuple  $\mathbf{t}\in \mathcal{S}$  consists of an argument  $a$ , a verb  $v$ , and (possibly) a modifier  $m$ , each of which is a text span of the
sentence. We treat each of  $a$ ,  $v$ , and  $m$  as a node and link it to an existing node  $v_i \in \mathcal{V}$  if it is similar to  $v_i$ . Two nodes  $A$  and  $B$  are similar if one of the following rules are satisfied: (1)  $A$  is equal to  $B$ ; (2)  $A$  contains  $B$ ; (3) the number of overlapped words between  $A$  and  $B$  is larger than the half of the minimum number of words in  $A$  and  $B$ . The edge between two similar nodes is associated with a special semantic relationship SIMILAR, denoted as  $r^s$ . Afterwards, we add two edges  $\langle a, r^{a \to v}, v \rangle$  and  $\langle v, r^{v \to m}, m \rangle$  into the edge set, where  $r^{a \to v}$  and  $r^{v \to m}$  denotes the semantic relationship between  $(a, v)$  and  $(v, w)$ , respectively. As a result, we obtain a semantic graph with multiple node and edge types based on the SRL, which captures the core semantic relations between entities within the document.
A.2 DP-based Semantic Graph
Dependency Parsing (DP) analyzes the grammatical structure of a sentence, establishing relationships between "head" words and words that modify
<sup>3</sup>https://demo.allennlp.org/semantic-role-labeling
DP-based Semantic Graph
Document 1) John E. EchoHawk (Pawnee) is a leading member of the Native American self - determination movement . 2) Self - determination \* is meant to reverse the paternalistic policies enacted upon Native American tribes since the U.S. government created treaties and established the reservation system .
SRL-based Semantic Graph
Figure 4: An example of constructed DP- and SRL- based semantic graphs, where  $\rightarrow$  indicates CHILD relation, and rectangular, rhombic and circular nodes represent arguments, verbs and modifiers respectively.
them, in a tree structure. Given a document  $\mathcal{D}$  with  $n$  sentences  $\{s_1,\dots ,s_n\}$ , Algorithm 2 gives the detailed procedure of constructing the semantic graph based on dependency parsing.
To better represent the entity connection within the document, we first employ the coreference resolution system of AllenNLP to replace the pronouns that refer to the same entity with its original entity name. For each sentence  $s$ , we employ the AllenNLP implementation of the biaffine attention model (Dozat and Manning, 2017) to obtain its dependency parse tree  $T_{s}$ . Afterwards, we perform the following operations to refine the tree:
- IDENTIFY_NODETYPES: each node in the dependency parse tree is a word associated with a POS tag. To simplify the node type system, we manually categorize the POS types into three groups: verb, noun, and attribute. Each node is then assigned to one group as its node type.
- PRUNE.Tree: we then prune each tree by removing unimportant continents (e.g., punctuation) based on pre-defined grammar rules. Specifically, we do this recursively from top to bottom where for each node  $v$ , we visit each of its child node  $c$ . If  $c$  needs to be pruned, we delete  $c$  and directly link each child node of  $c$  to  $v$ .
- MERGE NODES: each node in the tree represents only one word, which may lead to a large and noisy semantic graph especially for long documents. To ensure that the semantic graph only retains important semantic relations, we merge consecutive nodes that form a complete semantic unit. To be specific, we apply a simple yet effective rule: merging a node  $v$  with its child  $c$  if they form a consecutive modifier, i.e., both the type of  $v$  and
$c$  are modifier, and  $v$  and  $c$  is consecutive in the sentence.
After obtaining the refined dependency parse tree  $T_{s}$  for each sentence  $s$ , we add intra-tree edges to construct the semantic graph by connecting the nodes that are similar but from different parse trees. For each possible node pair  $\langle v_{i}, v_{j} \rangle$ , we add an edge between them with a special edge type SIM-ILAR (denoted as  $r^{s}$ ) if the two nodes are similar, i.e., satisfying the same condition as described in Section A.1.
Figure 4 shows a real example for the DP- and SRL-based semantic graph, respectively. In general, DP-based graph contains less words for each node compared with the SRL-based graph, allowing it to include more fine-grained semantic relations. For example, a leading member of the Native American self-determination movement is treated as a single node in the SRL-based graph. While in the DP-based graph, it is represented as a semantic triple  $\langle$  a leading member, pobj, the Native American self-determination movement  $\rangle$ . As the node is more fine-grained in the DP-based graph, this makes the graph typically more sparse than the SRL-based graph, which may hinder the message passing during graph propagation.
In experiments, we have compared the performance difference when using DP- and SRL-based graphs. We find that although both SRL- and DP-based semantic graph outperforms all baselines in terms of BLEU 1-4, DP-based graph performs slightly better than SRL-based graph  $(+3.3\%)$  in BLEU-4).