Venue: ACL 2020

# Exact yet Efficient Graph Parsing, Bi-directional Locality and

the Constructivist Hypothesis

Yajie Ye\({}^{1}\) &Weiwei Sun\({}^{12}\)

\({}^{1}\)Wangxuan Institute of Computer Technology

\({}^{1}\)The MOE Key Laboratory of Computational Linguistics

\({}^{2}\)Center for Chinese Linguistics

Peking University

{yeyajie,ws}@pku.edu.cn

###### Abstract

A key problem in processing graph-based meaning representations is _graph parsing_, i.e. computing all possible derivations of a given graph according to a (competence) grammar. We demonstrate, for the first time, that exact graph parsing can be efficient for large graphs and with large Hyperedge Replacement Grammars (HRGs). The advance is achieved by exploiting locality as terminal edge-adjacency in HRG rules. In particular, we highlight the importance of 1) a terminal edge-first parsing strategy, 2) a categorization of a subclass of HRG, i.e. what we call Weakly Regular Graph Grammar, and 3) distributing argument-structures to both lexical and phrasal rules.

## 1 Introduction

Language production, though as important as language understanding, has received very limited theoretical and empirical research attention. A fundamental problem in modeling language production is _parsing meaning representations_, i.e. computing all possible analyses of a given meaning representation (MR) according to a (competence) grammar. In theory, the worst-case complexities of existing algorithms are exponential or high-degree polynomial w.r.t. grammar size and input length. In practice, there are few systems that can parse large but frequent MRs with a realistic, wide-coverage grammar in a reasonable time.

The major contribution of this paper is an exact yet efficient method to parse MRs in the framework of graph-based semantic representations Koller et al. (2019) and Hyperedge Replacement Grammar Drewes et al. (1997). The ability to enumerate all possible analyses of a graph facilitates surface realization, grammar induction, recursive graph embedding, etc. The advance in efficiency is from exploiting locality of HRG rules from the rarely discussed perspective of language production, a reversed direction to language understanding. We discuss locality in a sense of terminal edge-adjacency and develop a locality-centric complexity analysis of the _de facto_ algorithm introduced by Chiang et al. (2013). Our analysis motivates (1) a terminal edge-first parsing strategy, (2) a categorization of a subclass of HRG, i.e. what we call Weakly Regular Graph Grammar, and (3) a computational support in the constructivist hypothesis in theoretical linguistics. Altogether, our analysis leads to a substantial improvement in practical graph parsing. An MR with the number of conceptual nodes ranging from 5 to 50 corresponding to a Wall Street Journal sentence can receive a full-forest analysis in 0.089 second on average with a large-scale comprehensive grammar; Even semantic graphs with c.a. 80 conceptual nodes can be processed in less than 0.5 second.

## 2 A Graph-Structured Syntax-Semantics Interface

Linguistically-informed graph parsing needs a precise model of the syntax-semantic interface. To this end, we need to precisely describe elementary structures corresponding to linguistic units at (morphological,) lexical and phrasal levels, and precisely describe the merge operation of two linguistic units. Under the umbrella of graph-based MRs, we employ hypergraphs and HRGSDrewes et al. (1997) to achieve the two goals.

Throughout this paper, we define an edge-labeled, ordered **hypergraph** over finite alphabet \(\Sigma\) as a tuple \(G=(V,E,\ell)\), where \(V\) is a finite set of nodes, \(E\subseteq V^{+}\) is a finite set of hyperedges, and \(\ell:E\mapsto\Sigma\) is a labeling function. A hyperedge can connect to more than two nodes or a single node. Labels can be associated to edges but not nodes. The set of nodes connected by edge \(e\) are denoted by \(V(e)\) and the set of edges connected to

node \(v\) are denoted by \(E(v)\). We use graph and hypergraph interchangeably, and similarly for edge and hyperedge.

Fig. 1 presents an example that contains a raising construction. The graph associated to the sentence (indicated by ) is derived along with a syntactic tree, in which the leaves and internal nodes are associated with graphs (indicated by ) as lexical and phrasal interpretations.

The key operation in semantic composition is to glue two graphs, say \(G_{1}\) and \(G_{2}\). It is obvious that not every node in \(G_{1}\) is _visible_ to \(G_{2}\) and vice versa. To emphasize on this point, we augment the representation of a hypergraph \((V,E,\ell)\) with a list of ordered **external nodes**\(V_{x}\in V^{+}\) and get **a hypergraph fragment**\(H=(V,E,\ell,V_{x})\). The number of external nodes is denoted by \(rank(H)\).

Graph gluing can be manipulated by an HRG\(\mathcal{G}=(\mathcal{N},\mathcal{T},\mathcal{P},S)\), where \(\mathcal{N}\) and \(\mathcal{T}\) are two finite disjoint alphabets of nonterminal and terminal symbols respectively, \(S\in\mathcal{N}\) is the start symbol, and \(\mathcal{P}\) is the finite collection of rewriting rules in the form of \(A\to R\). The left hand side (LHS) \(A\) belongs to \(\mathcal{N}\), and the right hand side (RHS) \(R\) is a hypergraph fragment over \(\mathcal{N}\cup\mathcal{T}\). See \(\gamma_{1}\) to \(\gamma_{10}\) in Fig. 1 for example.

A carefully designed HRG can be linguistically elegant, in that its rules are consistent with state-of-the-art linguistic analysis. For instance, raising and control constructions receive principled analysis with rules in Fig.1. HRG can be comparable to other popular grammar formalisms, such as Combinatory Categorial Grammar [56, 34]. See Fig. 2 for an illustration.

## 3 Graph Parsing with a General HRG

In the framework of graph-based MRs, a key problem is _graph parsing_: computing all possible analyses of a given semantic graph according to a grammar. Fig. 3 demonstrates the target structure of graph parsing -- derivation forest. A derivation forest allows us to efficiently enumerate every derivation. Coupled with a _local_ score function that evaluates the _goodness_ of a rule application, a graph parser can further tell the goodness of a particular derivation tree or the full forest as a whole.

Though essential, graph parsing is only partially understood. In this section, we summarize the state-of-the-art algorithm for graph parsing with HRGs [9], and then evaluate its efficiency with a wide-coverage grammar.

Figure 1: An HRG-based syntactico-semantic derivation for _He really seems to care_. The right part are examples of HRG rules. Throughout this paper, we use filled black nodes to indicate external nodes, arrows to indicate single-node edges and directed arcs to indicate edges connected to two nodes. The edge labeled as \(\mathbf{Y}\) in rule \(\gamma_{2}\) connects more than two nodes whose orders are indicated by tiny numbers around lines. Nodes in an HRG rule and subgraphs of an input graph are mentioned with numbers and characters respectively. Since nodes receive no informative labels, we use single-node edges with underlined terminal labels to represent concepts, e.g. “pron.” Others terminal labels, e.g. “arg1,” express semantic roles.

 



### A Dynamic Programming Algorithm

Chiang et al.'s algorithm is a dynamic programming algorithm, in which a collection of in-process subgraphs are iteratively recognized as solutions to subproblems. Two key techniques are introduced concerning (1) how to _pack_ a subgraph and (2) how to expand recognized subgraphs.

A subgraph is compactly encoded by **boundary representation** (BR) defined as follows. Assume \(I\) is a subgraph of a graph \(H\). A **boundary node** of \(I\) is an external node of \(H\) or it is incident to an edge that is not in \(I\). A **boundary edge** of \(I\) is an edge in \(I\) which connects to a boundary node. Let \(m\) be an arbitrarily chosen marker node in \(H\). The BR of \(I\) is the tuple \(b(I)=\langle bn(I),be(I),m\in I\rangle\), where \(bn(I)\) is the set of \(I\)'s boundary nodes, \(be(I)\) is the set of \(I\)'s boundary edges, and \((m\in I)\) is a boolean value indicating whether \(m\) is in \(I\). Take \(P_{1}\) in Fig. 5 for example. The dotted box shows a subgraph that has been recognized. \(bm(\raisebox{0.86pt}{\includegraphics[height=34.14pt]{fig:cogr-example-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graph-graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraphgraph--graphgraph--graphgraphgraph--graphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph--graphgraphgraph---graphgraphgraphgraph---graphgraphgraphgraph---graphgraphgraph---graphgraphgraph---graphgraphgraph---graphgraphgraph---graphgraphgraph---graphgraphgraph---graphgraphgraph---graphgraphgraph---graphgraphgraph---graphgraphgraph---graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraphgraph----graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph---graphgraph--graphgraph---graphgraph--graphgraph---graphgraph--graphgraph---graphgraph--graphgraph--graph--graphgraph--graphgraph--graph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph--graphgraph-graph-graph-graph-graph-graph-graphgraph--graphgraph-graph-graph-graph-graph-graphgraph-graph-graph-graph-graphgraph-graph-graph-graph-graphgraph-graph-graph-graph-graphgraph-graph-graph-graphgraph-graph-graph-graphgraph-graph-graph-graphgraph-graph-graphgraph-graph-graphgraph-graph-graphgraph-graph-graphgraph-graph-graphgraph-graph-graphgraph-graph-graphgraph-graph-graphgraph-graph-graphgraph-graph-graphgraph-graph-graphgraph-graphgraph-graph-graphgraph-graphgraph-graph-graphgraph-graphgraph-graphgraph-graph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraphgraph-graphgraph-graphgraph-graphgraph-graphgraphgraph-graphgraph-graphgraphgraph-graphgraph-graphgraphgraph-graphgraph-graphgraphgraph-graphgraph-graphgraphgraph-graphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraph-graphgraphgraph-graphgraphgraph-graphgraphgraph-graphgraph-graphgraphgraph-graphgraphgraph-graphgraph-graphgraphgraph-graphgraph-graphgraphgraph-graphgraph-graphgraphgraph-graphgraph-graphgraph-graphgraphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graphgraph-graph-graphgraph-graphgraph-graphgraph-graph-graphgraph-graphgraph-graph-graphgraph-graphgraph-graph-graphgraph-graph-graphgraph-graph-graphgraph-graph-graphgraph-graph-graphgraph-graph-graphgraph-graph-graph-graphgraph-graph-graph-graphgraph-graph-graph-graphgraph-graph-graph-graph-graphgraph-graph-graph-graph-graph-graph-graph-graphgraph-

the form \([A,J,B_{x}]\) where \(J\) is a subgraph of \(G\) which can be derived initially from some rule with \(A\) as LHS (\(A\Rightarrow^{*}J\)) and \(B_{x}\) is an explicit ordering of \(bn(J)\). An **active item** is of the form \([A\to R,\eta,I,\varphi]\) where \(\eta\) is in \(T_{R}\), \(I\) is a subgraph of \(G\) which derives from \(R_{\triangeta}\) and \(\varphi\) is the bijection from \(bn(R_{\triangeta})\) to \(bn(I)\). A small number of inference rules (as shown in Fig. 5) are sufficient to control merging the chart items. **R0** is applied on the root node of \(T_{R}\). **R1**, **R2.T**, **R2.NT** and **R3** are applied on leaf nodes, unary nodes that introduce a terminal edge, unary nodes that introduce a non-terminal edge and binary nodes respectively. \(e^{*}\) is an edge of \(G\) such that \(\ell(e)=\ell(e^{*})\). \(\{e\mapsto e^{*}\}\) or \(\{e\mapsto X\}\) reprensets the mapping that sends each node of \(e\) to the corresponding node of \(e^{*}\) or \(X\). \(\psi(X_{R})\) denotes a list generated by applying \(\psi\) on each node of \(X_{R}\) in order. Refer to the original paper for a complete description of the algorithm. See the bottom part of Fig. 5 for a partial recognition along with \(T_{1}\) in Fig. 4.

### Treewidth-centric Complexity Analysis

It is an advantage of using tree decomposition that the treewidth of a grammar leads to a bound on the number of boundary nodes which we must keep track of during parsing. When applying an inference rule at \(\eta\), all mentioned boundary nodes are called **active nodes** and denoted as \(A(\eta)\). \(A(\eta)=bn(R_{\triangeta_{1}})\cup bn(R_{\triangeta_{2}})\) if \(\eta\) is binary, and \(A(\eta)=bn(R_{\triangeta_{1}})\cup V(e)\) otherwise. Let \(k\) be the treewidth of a grammar and \(d\) be the maximum degree of any node in the input graph. The number of rule instantiations at \(\eta\) is actually in \(\mathcal{O}(n|^{A_{\eta}}|3^{d|A_{\eta}|})\). The first part \(n|^{A_{\eta}|}\) is the number of ways of mappings between active nodes in a rule and nodes in an input graph. The second part \(3^{d|A_{\eta}|}\) is an upper bound of realizations for boundary edges. Chiang et al. proves that \(A(\eta)\subseteq V_{\eta}\), implying that \(k+1\) is an upper bound of \(|A(\eta)|\). Therefore, the time complexity is in \(\mathcal{O}((3^{d}n)^{k+1})\). The space complexity is in \(\mathcal{O}((2^{d}n)^{k+1})\) by a parallel analysis.

### Measuring Practical Performance

Successful integration of two chart items according to an inference rule requires that the items are disjoint and can make up a new bijection. When two chart items pass the check, the following _successful integration_ is viewed as a successful rule instantiation, and in this case, the operation cost is taken into account. When two chart items fail to pass the check, there will be no successful rule instantiation, and in this case, the operation cost for this failed integration is overlooked by the treewidth-centric complexity analysis. The cost to figure out an integation is impossible is actually comparable to that of a successful integation.

Measuring practical performance with respect to both successful and failed integration operations is a necessary complement to the theoretical analysis, especially when the number of failed integrations is prominent. In the following experiments, we will report the exact numbers for successful (indicated as **#Succ**) and total (=successful+failed; indicated as **#Total**) integrations.

Figure 5: A sketch of the inference rules and how Chiang et al.’s algorithm works as chart parsing.

 



### Evaluation with a Realistic Grammar

To profile the parsing algorithm, we conduct experiments on the Elementary Dependency Structure (EDS; Oepen and Lonning, 2006) graphs provided by DeepBank v1.1 (Flickinger et al., 2012). The data is separated into training, development and test sets according to standard setup for string parsing. We get a wide-coverage linguistically-meaningful grammar2 by applying the grammar extraction algorithm described in Chen et al. (2018). The grammar is lexicalized (LxG), in that argument-structures are lexically encoded, like almost all popular deep grammars used in NLP. Tab. 1 shows the statistics of the rules.

Footnote 2: We only consider rules the RHS of which are connected. A few graphs that are not connected and thus removed. A very small portion of DeepBank graphs result in disconnected rules. These graphs contain arguable annotations related to (1) distributive readings of coordination, (2) quantifier of bare NPs, and/or (3) small clauses. We leave appropriate analysis of these phenomena for future investigation.

Referring to Bolinas3, we re-implement the algorithm in C++ and test its efficiency on 4500 EDS graphs that are randomly selected from the training set with the size in the range of 5 to 50. By size of a graph, we mean the number of its nodes. If the number of total subgraphs allocated during parsing is larger than \(2.6\times 10^{7}\), the parser will throw an out-of-memory error (OOM). In all the following tables, all statistics are the average values over instances which successfully receive derivation forests. The platform for all experiments is x86 64 GNU/Linux with one Intel(R) Core(TM) i7-5930K CPU at 3.50GHz.

Footnote 3: www.isi.edu/licensed-sw/bolinas/

Tab. 2 summarizes the results. For small graphs, the algorithm achieves a promising speed. For larger graphs, most of parsing time is wasted on the failed integrations. Fig. 6 represents the numbers of successful and total integrations. We can clearly see that the difference between the two types of integrations increase very quickly when an input graph is enlarged. In SS4.5 we will discuss how to reduce failed integrations.

## 4 Speeding Up by Exploiting Locality

### Locality as Edge-Adjacency

Some notion of locality is conceptually necessary for studying complex structures. Adjacency is a key perspective to express locality in some linguistic theories, such as CCG (Steedman, 2000, p. 54):

* **The Principle of (String-)Adjacency** Combinatory rules may only apply to finitely many phonologically realized and string-adjacent entities.

Almost all string parsing algorithms benefit from this string-adjacency. Now let us picture string-adjacency using _a graph language_. Fig. 7 gives a visualization of the linear chain structure of a word sequence. The terminal edge labeled as next in \(\gamma_{11}\) explicitly displays a local relation: 2 and 3 being able to be recognized almost simultaneously. String-adjacency turns to be **terminal edge-adjacency** from a graph-theoretic view.

\begin{table}
\begin{tabular}{c|c|c c c} \hline \hline LxG & **\#Rule** & & **Treewidth** & **\#Node** & **\#Terminal** \\ \hline Lexical & 46,101 & avg. & 1.07 & 2.15 & 2.47 \\  & max. & 4 & 10 & 18 \\ \hline Phrasal & 8,594 & avg. & 1.62 & 2.94 & 0.79 \\  & max. & 6 & 7 & 10 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Basic properties of our lexicalized grammar. **#Node** and **#Terminal** indicate the numbers of nodes and terminal-edges in a single rule.

Figure 6: The numbers of successful/total integrations relative to the size of graph. All data points are the average value of multiple graphs of the same size.



What does terminal edge-adjacency actually mean? From a semiotic perspective of a language system, being either natural or artificial, a key property is _form-meaning_ connection. A particular _form_ triggers a particular _meaning_. What can be observed can be directly recognized, and then makes other things recognizable. Considering language production, the input is an MR, and in the graph-based framework, it is terminal edges that are directly observable. In this way a terminal edge makes nodes connected to it co-recognizable.

The existing algorithms, including Chiang et al. (2013) and Groschwitz et al. (2015), do not consider terminal edge-adjacency. We will show that capturing locality in this sense is beneficial, just like what successful string parsing algorithms do.

### Locality-centric Complexity Analysis

Some active nodes are not independent with each other if we take terminal edge-adjacency into consideration. We call a graph consisting of only terminal edges a _terminal graph_. For a graph fragment \(H\), we use \(term(H)\) to denote the subgraph of \(H\) that is induced from all and only terminal edges. We informally illustrate the idea of dependency between nodes in a rule, and then present a precise analysis. Fig 8 is a prototype of a binary node in \(T_{R}\).

A & & & are active nodes of \(\eta\). But if one of these nodes is identified in an input graph, the possible positions of the other three nodes are highly restricted.

**Proposition 1**.: Consider a graph \(G\) and connected terminal graph \(R_{t}\). If there is a node \(v_{1}\) in \(R_{t}\) that is tied to a node \(v^{*}_{1}\) in \(G\), then finding all isomorphisms of \(R_{t}\) in \(G\) can be completed in \(\mathcal{O}(d^{m_{t}})\) time, where \(m_{t}\) is the number of edges in \(R_{t}\) and \(d\) is the maximum degree of any node in \(G\).

Proof.: We perform a depth-first search over \(R_{t}\) starting at \(v_{1}\) and arranging all edges of \(R_{t}\) as a sequence according to the order in which they are visited. Let the edge sequence be \(e_{1},e_{2},...,e_{m_{t}}\). We match edges in this sequence one by one. When we handle \(e_{j}(1\leq j\leq m_{t})\), there must be a node \(v\in V(e_{j})\) such that \(v=v_{1}\) or \(v\in V(e_{k})(1\leq k<j)\). In other words, \(v\) is already tried to a node \(v^{*}\in G\). As a result, the number of possible mappings of \(e_{j}\) is at most \(d\), because the degree of \(v^{*}\) is at most \(d\). Therefore, the number of isomorphisms of \(R_{t}\) is in \(\mathcal{O}(d^{m_{t}})\). As a result, all isomorphisms can be found in \(\mathcal{O}(d^{m_{t}})\) time. 

When \(l\) active nodes locate in a connected component of \(term(R_{\triangnoline\eta})\), these nodes are somewhat **dependent**. By Proposition 1, the number of valid node mappings of these \(l\) nodes is bounded by \(\mathcal{O}(nd^{m_{t}})\) rather than \(\mathcal{O}(n^{l})\).

**Definition 1**.: For any node \(\eta\) in \(T_{R}\), \(\delta(\eta)\) denotes the size of a maximal subset of \(A(\eta)\) such that all nodes in this subset is independent with each other. We use \(S(\eta)\) to denote one of such maximal subsets. Similar to treewidth, we define \(\delta(T_{R})=\max_{\eta~{}\text{in}~{}T_{R}}\delta(\eta)\) and \(\delta(R)\) as the minimum \(\delta\) of any tree decomposition of \(R\).

In Fig. 8, we have \(\delta(\eta)=4\) and is a maximal subset of \(A(\eta)\) as required.

**Proposition 2**.: For any graph fragment \(R\), \(\delta(R)\leq k+1\) where \(k\) is the treewidth of \(R\).

Proof.: This proposition is trivial. For any \(\eta\), we have \(\delta(\eta)\leq|A_{\eta}|\leq|V_{\eta}|\leq k+1\) (Proposition 3 in Chiang et al.). By the definition of \(\delta(R)\), we have \(\delta(R)\leq\delta(T_{R})=\max_{\eta~{}\text{in}~{}T_{R}}\delta(\eta)\leq k+1\). 

**Proposition 3**.: The number of ways of instantiations of any inference rule is in \(\mathcal{O}(n^{\delta^{\prime}}dm_{g}3^{dn_{g}})\), where \(n_{g}\!/\!m_{g}\) is the maximum count of nodes/terminal-edges of any RHS in \(\mathcal{G}\) and \(\delta^{*}\) is the maximum \(\delta\) of any RHS in \(\mathcal{G}\).

Proof.: When applying an inference rule on \(\eta\), we first select the mappings for nodes in \(S(\eta)\) independently. According to the definition of \(S(\eta)\), for an active node \(v\notin S(\eta)\), there must be a node

Figure 8: A prototypical case for recognizing a binary node \(\eta\) in \(T_{R}\). The area in blue represents \(term(R_{\triangnoline\eta})\) and the red one represents \(term(R_{\triangnoline\eta})\). Boundary nodes of each area are placed on the border. The nodes in black are the boundary nodes of \(R_{\triangnoline\eta}\).

Figure 7: A graph-based view of string-adjacency.

 

\(u\in S(\eta)\) such that \(u\) and \(v\) belong to the same connected component \(c\) of \(term(R_{\triangtext{$\eta$}})\). Because \(u\) is already bounded, the number of isomorphisms of \(c\) is in \(\mathcal{O}(d^{m_{c}})\) where \(m_{c}\) is the number of edges in \(c\). By enumerating nodes like \(v\), we can get a set of connected components consisting of only terminal edges. The number of isomorphisms of all these components is in \(\mathcal{O}(\prod_{c}dm^{m_{c}})\leq\mathcal{O}(d^{m_{g}})\). Therefore, the number of possible mappings for all active nodes is in \(\mathcal{O}(n^{|S(\eta)|}d^{m_{g}})\leq\mathcal{O}(n^{\delta^{*}}d^{m_{g}})\). The analysis for boundary edges is similar to Chiang et al.'s. The only difference is that the tree decomposition which minimizes \(\delta\) may not minimize the treewidth \(k\). Since \(k\leq n_{g}-1\), the number of ways of boundary edges is in \(\mathcal{O}(3^{dn_{g}})\). 

We can conclude from Proposition 2 and 3 that our locality-centric analysis is tighter than the treewidth-centric one, and the upper bound of time complexity may decrease for some restricted HRGs. In Fig. 4, the treewidth of \(T_{2}\) is 3, but \(\delta(T_{2})=1\). So the number of rule instantiations that can be applied along with \(T_{2}\) is in \(\mathcal{O}(n)\) instead of \(\mathcal{O}(n^{4})\). In SS4.3, we will introduce Weakly Regular Graph Grammar (WRGG), a new subclass of HRG, the \(\delta\) of which is more intuitively understandable.

### Weakly Regular Graph Grammar

We discuss prototypes of HRG rules, investigating their key properties in a linguistic context. We then formally define WRGG that reflects the linguistic emphasis and also show that WRGG is actually a very expressive subclass of HRG.

Firstly, the HRG rule under discussion allows at most two non-terminals at RHS. Computationally speaking, we can transform a multi-branching rule into multiple binary rules without loss of expressiveness, as we are able to get a CFG in Chomsky Normal Form for any CFG. Linguistically speaking, multi-branching rules have been removed from generative linguistic theories, since at least Minimalist Program [5]. Fig. 9 presents four prototypes with the binary constriction. \(\gamma_{3}\), \(\gamma_{6}\), \(\gamma_{7}\), \(\gamma_{8}\) and \(\gamma_{9}\) in Fig. 1 are of T0, \(\gamma_{1}\), \(\gamma_{4}\), \(\gamma_{5}\) and \(\gamma_{10}\) are of T1, and \(\gamma_{2}\) is of T3. Secondly, for a lexicalized grammar, most rules are of T0 or T1, since constructions barely take _semantic materials_. If a rule introduces _heavy_ constructional meaning, it may affect one of its intermediate constituents (T2) or bridge the meanings between both of its intermediate constituents (T3), and hardly affect its intermediate constituent separately. Even though a rule has multiple terminal components, we can replace it with several rules of T0-T3. Thirdly, a node that is only connected to a nonterminal edge is a kind of placeholder, in that it does not affect current semantic composition but will be used in future. Otherwise it has been removed in a previous step. Finally, we do not consider disconnected RHS because it yields disconnected graphs.

**Definition 2**.: A node \(v\) in an edge-labeled graph \(G\) is **free**, if \(E(v)\) contains only nonterminal edge(s). The number of those nodes is denoted by \(f(G)\).

In Fig. 8,  2  2    are free nodes of \(R_{\triangtext{$\eta$}}\).

**Definition 3**.: A weakly regular rule \(A\to R\) satisfies the following conditions: (1) \(R\) is connected; (2) \(term(R)\) is an empty graph or a connected graph; (3) if a free node of \(R\) is incident to only one edge, it is also an external node.

**Definition 4**.: An HRG is weakly regular, if all of its rules are weakly regular.

**Proposition 4**.: If \(A\to R\) is binary and weakly regular, then \(\delta(R)=f(R)\) or \(f(R)+1\).

The proofs of this proposition can be found in the appendix. The tree shown in Fig. 10 is a valid nice tree decomposition of \(R\) and the \(\delta\) of the tree is \(f(R)\) or \(f(R)+1\). We argue that for parsing with a binary WRGG, the number of free nodes is more meaningful and we can use the tree decomposition shown in Fig. 10 rather than a tree decomposition with minimum treewidth.

Courcelle (1991) introduces Regular Graph Grammar (RGG). It is provable that RGG is a subclass of WRGG. There are no free nodes in RGG and graph parsing with an RRG can be finished in linear time by applying Chiang et al.'s algorithm.

Figure 9: Prototypes of binary HRG rules: **(T0)** lexical rules, **(T1)** rules without terminal edges, **(T2)** only one nonterminal edge of the rules contains non-free nodes, and **(T3)** both nonterminal edges of the rules contain non-free nodes. The area in blue represents all terminal edges. Dashed edges indicate optionality.

 

This result is comparable to another algorithm proposed by Gilroy et al. (2017). However, the strong restrictions of RGG make it too weak to model linguistic structures. WRGG is much more linguistically adequate.

### Distributed Argument-Structure

We value the trigger role played by terminal edges in an HRG rule. Now let us revisit the derivation governed by a lexicalized grammar. It is obvious that lexical rules try to use up all terminal edges at the initial stage of syntactico-semantic composition. If we can distribute terminal edges to all rules, both lexical and phrasal, we are able to get a reduced number of free nodes on average and in exactly this way improve graph parsing remarkably. The idea to distribute argument-structures exhibits a constructivist perspective, which is a competing hypothesis to lexicalism that dominates our field for dozens of years, since at least Bressan and Kaplan (1982). The constructivist approaches to argument structures have been recently discussed by different theoretical linguistic theories, including but not limited to Distributed Morphology (Halle and Marantz, 1993, 1994) and Sign-Based Construction Grammar (Boas and Sag, 2012). The emphasis on the advantage of _Distributed Argument-Structure_ under the consideration of language production is a computational support for many constructivist approaches.

Fig. 11 demonstrates a derivation with a construction grammar. Compare \(\gamma_{12}\) to \(\gamma_{4}\) and \(\gamma_{13}\) to \(\gamma_{5}\), we can clearly see that \(\delta\) is significantly reduced. A comparison of lexical rules also confirms the importance of distributed argument-structure.

### Fast Accessing of Chart Items

We will complete our discussion on locality by considering the _edge-zero_ case, i.e. unifying nodes. In Fig. 8, when we try to integrate \(R_{\triangn_{1}}\) and \(R_{\triangn_{2}}\), we must make sure that the three nodes on the boundary, viz. 1, 2 and 3, are identical in terms of mappings relative to \(\eta_{1}\) and \(\eta_{2}\) respectively. Otherwise, a failure occurs. In both cases, trying to unify them causes a bottleneck for graph parsing, as conceptually suggested in SS3.3 and empirically confirmed by Tab. 2.

Considering the above problem in the framework of chart parsing, we would like to construct a data structure to efficiently access all chart items. In particular, when partial information is provided, this data structure can quickly find all compatible chart items. In this paper, we use a map, with the keys being partial information for quiry and the values being sets of chart items. The implementation used in SS3.4 follows the method proposed by Chiang et al. (2013), only mentioning \(\ell(e)\) or \(\eta\) for indexing, which is not efficient in practice. We propose to build a more comprehensive map. See Tab. 3 for an example of our map.

During recognizing \(\eta\), the set of nodes which connect branching subgraph(s) \(C(\eta)\) is \(bn(R_{\triangn_{1}})\cap bn(R_{\triangn_{2}})\) for binary case, and \(bn(R_{\triangn_{1}})\cap V(e)\) for unary case. Let \(e=(v_{1},...,v_{|V(e)|})\) denote a hyperedge and \(index(e,v_{i})=i\) denote an indexing function. For a list of nodes \(B\), \(B[i]\) denotes its \(i\)-th node. A passive item \([A,J,B_{x}]\) has mul 

tiple indexing keys. For a non-empty set of positive integers \(mask\subseteq\{1,2,...,|bn(J)|\}\), \(\langle A,|bn(J)|,\{\langle i,B_{x}[i]\rangle|i\in mask\rangle\}\) is a plausible key. For active item \([*,\eta_{1},I,\varphi]\), let \(\eta\) be the parent of \(\eta_{1}\). If \(\eta\) is binary, the item should be indexed by \(\langle\eta_{1},\{\langle v,\varphi(v)\rangle|v\in C(\eta)\}\rangle\). Otherwise \(\eta\) is unary and introduces some edge \(e\). The item should be indexed by \(\langle\ell(e),|V(e)|,\{\langle index(e,v),\varphi(v)\rangle|v\in C(\eta)\}\rangle\). During parsing, two items will be integrated only when they have the same key.

Note that the number of possible \(mask\)'s for a passive item grows exponentially w.r.t. the number of the corresponding external nodes. However a significant number of \(mask\)'s are not used by any tree decomposition of any rule. And such \(mask\)'s can be found by processing a grammar before parsing. For all HRGs used for experiments, the maximum number of _useful_\(mask\)'s for a passive item is 15.

### Empirical Evaluation

A construction grammar (CxG) is automatically induced in a similar way to the experiments in SS3.4. Note that our grammar extraction procedure makes sure that this grammar is weakly regular. As shown in Tab. 4, the average number of free nodes in CxG is much smaller. We conduct new experiments using the improvements mentioned in previous sections. We re-run the improved parser on 4195 EDS graphs, which can successfully receive derivation forests from the original parser. Tab. 5 and Fig. 12 show the effectiveness of our improvements. The terminal-first tree decomposition (as illustrated in Fig. 10) is able to significantly reduce the number of integrations. Our indexing method can effectively reduce the number of failed integrations. For the CxG, using the terminal-edge first strategy is more effective than the indexing strategy. Note that the cost to build a map for indexing chart items is not ignorable.

## 5 Conclusion

We introduce several locality-centric refinements to advance graph parsing and empirically evaluate their effectiveness. We show that exact graph parsing can be efficient even for large graphs and with large graph grammars.

#### Acknowledgments

This work is supported in part by the National Hi-Tech R&D Program of China (No. 2018YFC0831900). Weiwei Sun is the corresponding author.

\begin{table}
\begin{tabular}{c|l|c|c c c} \hline \hline  & & **\#Rule** & & **Treewidth** & \(\delta\) & **\#Free** \\ \hline \multirow{3}{*}{CxG} & Lexical & 34,348 & avg. & 0.36 & — & — \\  & & max. & & 4 & — & — \\ \cline{2-7}  & Phrasal & 7,978 & avg. & 1.68 & **1.59** & **0.59** \\  & & max. & & 7 & 7 & 6 \\ \hline \multirow{2}{*}{LxG} & \multirow{2}{*}{Phrasal} & \multirow{2}{*}{8,594} & avg. & 1.62 & 2.51 & 2.27 \\  & & max. & & 6 & 7 & 7 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Statistics of the CxG. **#Free** means the number of free nodes in HRG rules.

Figure 12: The number of total integrations relative to size of input graphs. All data points in the plot are the average value on test samples of a given size.

 



## References

* [Boas and Sag2012] Hans Christian Boas and Ivan A Sag. 2012. _Sign-based construction grammar_. CSLI Publications/Center for the Study of Language and Information.
* Bresnan and Kaplan1982 Joan Bresnan and Ronald M. Kaplan. 1982. Introduction: Grammars as mental representations of language. In J. Bresnan, editor, _The Mental Representation of Grammatical Relations_, pages xvii-lii. MIT Press, Cambridge, MA.
* Chen et al.2018 Yufei Chen, Weiwei Sun, and Xiaojun Wan. 2018. Accurate shrg-based semantic parsing. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 408-418. Association for Computational Linguistics.
* Chiang et al.2013 David Chiang, Jacob Andreas, Daniel Bauer, Karl Moritz Hermann, Bevan Jones, and Kevin Knight. 2013. Parsing graphs with Hyperedge Replacement Grammars. In _Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 924-932, Sofia, Bulgaria. Association for Computational Linguistics.
* Chomsky1995 Noam Chomsky. 1995 Noam Chomsky. 1995 The Minimalist Program. MIT Press, Cambridge, MA.
* Courcelle1991 Bruno Courcelle. 1991 Bruno Courcelle. 1991. The monadic second-order logic of graphs v: On closing the gap between definability and recognizability. _Theoretical Computer Science_, 80(2):153-202.
* Drewes et al.1997 F. Drewes, H.-J. Kreowski, and A. Habel. 1997. Hyperedge Replacement Graph Grammars. In Grzegorz Rozenberg, editor, _Handbook of Graph Grammars and Computing by Graph Transformation_, pages 95-162. World Scientific Publishing Co., Inc., River Edge, NJ, USA.
* Flickinger et al.2012 Daniel Flickinger, Yi Zhang, and Valia Kordoni. 2012. Deepbank: A dynamically annotated treebank of the wall street journal. In _Proceedings of the Eleventh International Workshop on Treebanks and Linguistic Theories_, pages 85-96.
* Gilroy et al.2017 Sorcha Gilroy, Adam Lopez, and Sebastian Maneth. 2017. Parsing graphs with regular graph grammars. In _Proceedings of the 6th Joint Conference on Lexical and Computational Semantics (* SEM 2017)_, pages 199-208.
* Groschwitz et al.2015 Jonas Groschwitz, Alexander Koller, and Christoph Teichmann. 2015. Graph parsing with s-graph grammars. In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, pages 1481-1490.
* Hanle and Marantz1993 M Halle and Alec Marantz. 1993. _Distributed morphology and the pieces of inflection_, pages 111-176. The MIT Press.
* Halle and Marantz1994 Morris Halle and Alec Marantz. 1994. Some key features of distributed morphology. _MIT working papers in linguistics_, 21(275):88.
* Kay1996 Martin Kay. 1996 Chart generation. In _34th Annual Meeting of the Association for Computational Linguistics_, pages 200-204.
* Koller et al.2019 Alexander Koller, Stephan Oepen, and Weiwei Sun. 2019. Graph-based meaning representations: Design and processing. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts_, pages 6-11, Florence, Italy. Association for Computational Linguistics.
* Oepen and Lonning2006 Stephan Oepen and Jan Tore Lonning. 2006. Discriminant-based mrs banking. In _Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC-2006)_, Genoa, Italy. European Language Resources Association (ELRA). ACL Anthology Identifier: L06-1214.
* Steedman1996 Mark Steedman. 1996. _Surface Structure and Interpretation_. Linguistic Inquiry Monographs. Mit Press.
* Steedman2000 Mark Steedman. 2000. _The syntactic process_. MIT Press, Cambridge, MA, USA.



Proof for Proposition 4

We provide the proof for \(R\) with two nonterminal edges: \(e_{X}\) and \(e_{Y}\).

**Firstly, we prove \(\delta(R)\geq f(R)\).** For any nice tree decomposition \(T_{R}\) of \(R\), let \(\eta_{m}\) be the node with minimum height such that \(R_{\triangtext{$\simeq$}\eta_{m}$}\) contains both \(e_{X}\) and \(e_{Y}\).

1. [label=0]
2. \(\eta_{m}\) is binary. Let \(\eta_{1},\eta_{2}\) be the two children of \(\eta_{m}\). Without loss of generality, we assume \(R_{\triangtext{$\simeq$}\eta_{1}$}\) contains \(e_{X}\) and \(R_{\triangtext{$\simeq$}\eta_{2}$}\) contains \(e_{Y}\).
3. \(\eta_{m}\) is unary. Let \(\eta_{1}\) be the only child of \(\eta_{m}\). In this case, \(\eta_{m}\) introduces either \(e_{X}\) or \(e_{Y}\). Without loss of generality, we assume \(\eta_{m}\) introduces \(e_{X}\).

Let \(v\) be a free node of \(R\).

* \(v\) is incident to only one of \(e_{X}\) and \(e_{Y}\). By property (3) of weakly regularity, \(v\) is an external node of \(R\). Therefore, \(v\in bn(R)\subset bn(R_{\triangtext{$\simeq$}\eta_{1}})\subset A(\eta_{m})\).
* \(v\) is incident to both \(e_{X}\) and \(e_{Y}\). When \(\eta_{m}\) is binary ([1]), we have \(v\in bn(R_{\triangtext{$\simeq$}\eta_{1}})\cap bn(R_{\triangtext{$\simeq$} \eta_{2}})\subset A(\eta_{m})\). When \(\eta_{m}\) is unary ([2]), we have \(v\in V(e_{X})\subset A(\eta_{m})\).

By the above discussion, we conclude that all free nodes of \(R\) are active nodes of \(\eta_{m}\) and it is obvious that free nodes are independent. As as result, we have \(\delta(T_{R})\geq\delta(\eta_{m})\geq f(R)\). The arbitrariness of \(T_{R}\) ensures that \(\delta(R)\geq f(R)\).

**Secondly, we prove that \(\delta(R)\geq f(R)+1\) for prototype T3.** If the rule is type **T3**, then there exist two nodes \(v,u\) such that \(v\) is incident with \(e_{X}\), \(u\) is incident with \(e_{Y}\) and \(u,v\) are in \(term(R)\).

* \(u=v\). We have \(u\in bn(R_{\triangtext{$\simeq$}\eta_{1}})\cap bn(R_{\triangtext{$\simeq$} \eta_{2}})\subset A(\eta_{m})\).
* \(u\neq v\) and \(\eta_{m}\) is unary ([2]). We have \(v\in V(e_{X})\subset A(\eta_{m})\).
* \(u\neq v\) and \(\eta_{m}\) is binary ([1]). According to the property (2) of weakly regularity, \(term(R)\) is connected. So there exists a path \(e_{1},e_{2},...,e_{s}(s\geq 1)\) in \(term(R)\) such that \(u\in V(e_{1})\) and \(v\in V(e_{s})\). Let \(i\) be the minimum index such that \(e_{i}\) is not in \(R_{\triangtext{$\simeq$}\eta_{1}}\). If \(i=1\), then \(u\) has an edge \(e_{1}\) which is not in \(R_{\triangtext{$\simeq$}\eta_{1}}\). Therefore, \(u\in bn(R_{\triangtext{$\simeq$}\eta_{1}})\subseteq A(\eta_{m})\). If \(s\geq i>1\), then all nodes inside \(V(e_{i-1})\cap V(e_{i})\) are boundary nodes of \(R_{\triangtext{$\simeq$}\eta_{1}}\). Therefore these nodes all belong to \(A(\eta_{m})\). If \(i\) does not exist, then \(v\) has an edge \(e_{s}\) which is not in \(R_{\triangtext{$\simeq$}\eta_{2}}\). Therefore, \(v\in bn(R_{\triangtext{$\simeq$}\eta_{2}})\subseteq A(\eta_{m})\).

By the above discussion, there is at least one active node which is not a free node. It is trivial that the node is independent with any free nodes. Therefore, \(\delta(T_{R})\geq\delta(\eta_{m})\geq f(R)+1\). The arbitrariness of \(T_{R}\) ensures that \(\delta(R)\geq f(R)+1\).

**Thirdly, we prove that the equality can be achieved.** It is trivial to prove that the tree \(T\) shown in Fig. 10 is a valid nice tree decomposition by going through the properties of tree decomposition. Since \(R_{\triangtext{$\simeq$}\eta_{i}}\) (\(1\leq i\leq l\)) is a connected terminal graph, we have \(\delta(\eta_{i})=1\). By going through the four possible prototypes of current rule shown in Fig. 9, we conclude that \(\delta(\eta_{j})\leq f(R)+1\), for \(l+1\leq j\leq l+2\). Therefore, \(\delta(R)\leq\delta(T)=\max_{\gamma\text{in}T}\delta(\eta)\leq f(R)+1\).

In summary, we have