SlowGuess/ACL_2020_OCR huggingface dataset:
Yajie Ye<sup>1</sup> and Weiwei Sun<sup>12</sup>
$^{1}$ Wangxuan Institute of Computer Technology
<sup>1</sup>The MOE Key Laboratory of Computational Linguistics
$^{2}$ Center for Chinese Linguistics
{yeyajie,ws}@pku.edu.cn
A key problem in processing graph-based meaning representations is graph parsing, i.e. computing all possible derivations of a given graph according to a (competence) grammar. We demonstrate, for the first time, that exact graph parsing can be efficient for large graphs and with large Hyperedge Replacement Grammars (HRGs). The advance is achieved by exploiting locality as terminal edge-adjacency in HRG rules. In particular, we highlight the importance of 1) a terminal edge-first parsing strategy, 2) a categorization of a subclass of HRG, i.e. what we call Weakly Regular Graph Grammar, and 3) distributing argument-structures to both lexical and phrasal rules.
Language production, though as important as language understanding, has received very limited theoretical and empirical research attention. A fundamental problem in modeling language production is parsing meaning representations, i.e. computing all possible analyses of a given meaning representation (MR) according to a (competence) grammar. In theory, the worst-case complexities of existing algorithms are exponential or high-degree polynomial w.r.t. grammar size and input length. In practice, there are few systems that can parse large but frequent MRs with a realistic, wide-coverage grammar in a reasonable time.
The major contribution of this paper is an exact yet efficient method to parse MRs in the framework of graph-based semantic representations (Koller et al., 2019) and Hyperedge Replacement Grammar (Drewes et al., 1997). The ability to enumerate all possible analyses of a graph facilitates surface realization, grammar induction, recursive graph embedding, etc. The advance in efficiency is from exploiting locality of HRG rules from the rarely discussed perspective of language
production, a reversed direction to language understanding. We discuss locality in a sense of terminal edge-adjacency and develop a locality-centric complexity analysis of the de facto algorithm introduced by Chiang et al. (2013). Our analysis motivates (1) a terminal edge-first parsing strategy, (2) a categorization of a subclass of HRG, i.e. what we call Weakly Regular Graph Grammar, and (3) a computational support in the constructivist hypothesis in theoretical linguistics. Altogether, our analysis leads to a substantial improvement in practical graph parsing. An MR with the number of conceptual nodes ranging from 5 to 50 corresponding to a Wall Street Journal sentence can receive a fullforest analysis in 0.089 second on average with a large-scale comprehensive grammar; Even semantic graphs with c.a. 80 conceptual nodes can be processed in less than 0.5 second.
2 A Graph-Structured Syntax-Semantics Interface
Linguistically-informed graph parsing needs a precise model of the syntax-semantics interface. To this end, we need to precisely describe elementary structures corresponding to linguistic units at (morphological,) lexical and phrasal levels, and precisely describe the MERGE operation of two linguistic units. Under the umbrella of graph-based MRs, we employ hypergraphs and HRGs (Drewes et al., 1997) to achieve the two goals.
Throughout this paper, we define an edge-labeled, ordered hypergraph over finite alphabet  $\Sigma$  as a tuple  $G = (V,E,\ell)$ , where  $V$  is a finite set of nodes,  $E\subseteq V^{+}$  is a finite set of hyperedges, and  $\ell :E\mapsto \Sigma$  is a labeling function. A hyperedge can connect to more than two nodes or a single node. Labels can be associated to edges but not nodes. The set of nodes connected by edge  $e$  are denoted by  $V(e)$  and the set of edges connected to
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4100-4110 July 5 - 10, 2020. ©2020 Association for Computational Linguistics
Figure 1: An HRG-based syntactico-semantic derivation for He really seems to care. The right part are examples of HRG rules. Throughout this paper, we use filled black nodes to indicate external nodes, arrows to indicate single-node edges and directed arcs to indicate edges connected to two nodes. The edge labeled as  $\mathbf{Y}$  in rule  $\gamma_{2}$  connects more than two nodes whose orders are indicated by tiny numbers around lines. Nodes in an HRG rule and subgraphs of an input graph are mentioned with numbers and characters respectively. Since nodes receive no informative labels, we use single-node edges with underlined terminal labels to represent concepts, e.g. "pron." Others terminal labels, e.g. "arg1," express semantic roles.
node  $v$  are denoted by  $E(v)$ . We use graph and hypergraph interchangeably, and similarly for edge and hyperedge.
Fig. 1 presents an example that contains a raising construction. The graph associated to the sentence (indicated by  $\mathbf{S}$ ) is derived along with a syntactic tree, in which the leaves and internal nodes are associated with graphs (indicated by  $x$ ) as lexical and phrasal interpretations.
The key operation in semantic composition is to glue two graphs, say  $G_{1}$  and  $G_{2}$ . It is obvious that not every node in  $G_{1}$  is visible to  $G_{2}$  and vice versa. To emphasize on this point, we augment the representation of a hypergraph  $(V,E,\ell)$  with a list of ordered external nodes  $V_{x}\in V^{+}$  and get a hypergraph fragment  $H = (V,E,\ell ,V_{x})$ . The number of external nodes is denoted by  $\operatorname {rank}(H)$ .
Graph gluing can be manipulated by an  $\mathsf{HRG}\mathcal{G} = (\mathcal{N},\mathcal{T},\mathcal{P},S)$ , where  $\mathcal{N}$  and  $\mathcal{T}$  are two finite disjoint alphabets of nonterminal and terminal symbols respectively,  $S\in \mathcal{N}$  is the start symbol, and  $\mathcal{P}$  is the finite collection of rewriting rules in the form of  $A\to R$ . The left hand side (LHS)  $A$  belongs to  $\mathcal{N}$ , and the right hand side (RHS)  $R$  is a hypergraph fragment over  $\mathcal{N}\cup \mathcal{T}$ . See  $\gamma_{1}$  to  $\gamma_{10}$  in Fig. 1 for example.
A carefully designed HRG can be linguistically elegant, in that its rules are consistent with state-of-the-art linguistic analysis. For instance, raising and control constructions receive principled
analysis with rules in Fig.1. HRG can be comparable to other popular grammar formalisms, such as Combinatory Categorical Grammar (CCG; Steedman, 1996, 2000). See Fig. 2 for an illustration.
$$
\begin{array}{l} (S \backslash N P _ {y}) / N P _ {x} \\ \lambda x. \lambda y. \underline {{\text {l i k e}}} (y, x) \end{array}
$$
Figure 2: A comparison of CCG and HRG. The external nodes  $\mathbf{1}, \mathbf{2}$  and  $\mathbf{3}$  corresponds to S,  $\mathrm{NP_y}$  and  $\mathrm{NP_x}$  in the syntactic category respectively.
3 Graph Parsing with a General HRG
In the framework of graph-based MRs, a key problem is graph parsing: computing all possible analyses of a given semantic graph according to a grammar. Fig. 3 demonstrates the target structure of graph parsing — derivation forest. A derivation forest allows us to efficiently enumerate every derivation. Coupled with a local score function that evaluates the goodness of a rule application, a graph parser can further tell the goodness of a particular derivation tree or the full forest as a whole.
Though essential, graph parsing is only partially understood. In this section, we summarize the state-of-the-art algorithm for graph parsing with HRGs (Chiang et al., 2013), and then evaluate its efficiency with a wide-coverage grammar.
Figure 3: Graph parsing with an HRG. The context-freeness of HRG allows us to represent a derivation as a tree, and sets of derivations as a derivation forest, which is the output structure of graph parsing. In the derivation forest, a dashed rectangle (node) corresponds to a subgraph, which may be immediately built with different HRG rules. Each rule application is separately represented as a box. Necessary and sufficient information includes the BRs of  $G_{t}$ ,  $G_{L_{t}}$  as well as  $G_{R_{t}}$  and the rule itself.
3.1 A Dynamic Programming Algorithm
Chiang et al.'s algorithm is a dynamic programming algorithm, in which a collection of inprocess subgraphs are iteratively recognized as solutions to subproblems. Two key techniques are introduced concerning (1) how to pack a subgraph and (2) how to expand recognized subgraphs.
A subgraph is compactly encoded by boundary representation (BR) defined as follows. Assume  $I$  is a subgraph of a graph  $H$ . A boundary node of  $I$  is an external node of  $H$  or it is incident to an edge that is not in  $I$ . A boundary edge of  $I$  is an edge in  $I$  which connects to a boundary node. Let  $m$  be an arbitrarily chosen marker node in  $H$ . The BR of  $I$  is the tuple  $b(I) = \langle bn(I), be(I), m \in I \rangle$ , where  $bn(I)$  is the set of  $I$ 's boundary nodes,  $be(I)$  is the set of  $I$ 's boundary edges, and  $(m \in I)$  is a boolean value indicating whether  $m$  is in  $I$ . Take  $P_{1}$  in Fig. 5 for example. The dotted box shows a subgraph that has been recognized.  $bn(\mathbf{y}) = \{\mathbf{A} \mathbf{C} \mathbf{F}\}$ , and  $\mathbf{D}$  and  $\mathbf{G}$  are irrelevant to further recognition.
Now consider combining two subgraphs recognized as nonterminal  $X$  and  $Y$  according to  $\gamma_{2}$  in Fig. 5. As to incrementally match elements of a rule, e.g.  $\gamma_{2}$ , in an edge-by-edge way, Chiang et al. proposes to leverage a tree decomposition $^{1}$ $T_{R}$  of the RHS of an HRG rule  $A \to R$
Figure 4:  $T_{1}$  and  $T_{2}$  are two nice tree decompositions of RHS of  $\gamma_{2}$ . Both are of width 3.
$(R = \langle V,E,\ell ,V_{x}\rangle)$ . A tree decomposition  $T_{R}$  is nice, if every node of  $T_{R}$  must be one of: (1) a leaf node associated to empty graph; (2) a unary node which introduces exactly one edge; (3) a binary node which introduces no edges. Throughout, for convenience, let  $\eta$  denote a node from  $T_{R}$  and  $R_{\geq \eta}$  denote the subgraph of  $R$  whose edges are induced by nodes in the subtree rooted by  $\eta$ . If  $\eta$  is binary, its children are denoted by  $\eta_{1}$  and  $\eta_{2}$ . If  $\eta$  is unary, the edge introduced by it and its only child are denoted by  $e$  and  $\eta_{1}$  respectively.
Oriented by the fundamental architecture of chart parsing/generation (Kay, 1996),  $T_{R}$  are used to define active/passive items and inference rules that process such items. A passive item is of
A tree decomposition  $T$  of a graph fragment  $H = \langle V, E, \ell, V_x \rangle$  is a tree that every node  $\eta$  in  $T$  is associated with a tuple  $\langle V_\eta, E_\eta \rangle$ .  $T$  must satisfy the following properties: (1) for each  $v \in V$ , there is a node  $\eta$  such that  $v \in V_\eta$ ; (2) for each  $e \in E$ , there is exactly one node  $\eta$  such that  $e \in E_\eta$  and  $V(e) \subseteq V_\eta$ ; (3) for each  $v \in V$ , all nodes in  $T$  that cover  $v$  are connected; (4) for the root of  $T$ $\eta_r$ ,  $V_x \subseteq V_{\eta_r}$ .
$$
\left[ A \rightarrow R, \eta_ {r}, J, \psi \right] \tag {R0}
$$
$$
\begin{array}{c c}\text {(R 2 . T)}&\frac {[ A \rightarrow R , \eta_ {1} , I , \varphi_ {1} ]}{[ A \rightarrow R , \eta , I \cup e ^ {*} , \varphi_ {1} \cup \{e \mapsto e ^ {*} \} ]}\end{array}
$$
$$
\left[ A \rightarrow R, \eta , \emptyset , \emptyset \right] \quad (\tag {R1}
$$
$$
\begin{array}{l l}\text {(R 3)}&\frac {[ A \rightarrow R , \eta_ {1} , I , \varphi_ {1} ] [ A \rightarrow R , \eta_ {2} , J , \varphi_ {2} ]}{[ A \rightarrow R , \eta , I \cup J , \varphi_ {1} \cup \varphi_ {2} ]}\end{array}
$$
$$
\frac {[ A \to R , \eta_ {1} , I , \varphi_ {1} ] [ \ell (e) , J , X ]}{[ A \to R , \eta , I \cup J , \varphi_ {1} \cup \{e \mapsto X \} ]}
$$
$$
A _ {1}: \left[ \gamma_ {2}, \eta_ {3}, \underbrace {\mathbf {b} \xrightarrow {\arg 1}} _ {\mathbf {B} \xleftarrow {} \mathbf {C}}, \left\{ \begin{array}{l} \mathbf {①} \mapsto \mathbf {C} \\ \mathbf {④} \mapsto \mathbf {D} \end{array} \right\} \right]
$$
$$
A _ {4}: \left[ \gamma_ {2}, \eta , \begin{array}{c} \underbrace {\text {a r g l}} _ {\text {a r g l}} \underbrace {\text {C c j t - 1}} _ {\text {c j t - 1}} \\ \underbrace {\text {D}} _ {\text {a r g l}} \underbrace {\text {F c j t - 1}} _ {\text {c j t - 1}} \\ \underbrace {\text {G}} _ {\text {a r g l}} \end{array} , \{\mathbf {4} \mapsto \mathbf {B} \} \right]
$$
Figure 5: A sketch of the inference rules and how Chiang et al.'s algorithm works as chart parsing.
$$
\begin{array}{c} P _ {2} \quad \frac {A _ {3} \quad \frac {P _ {1}}{A _ {2}} A _ {1} (\mathbf {R 2 . N T})}{P _ {3}} (\mathbf {R 3}) \\ \hline \end{array}
$$
Target Derivation
the form  $[A, J, B_x]$  where  $J$  is a subgraph of  $G$  which can be derived initially from some rule with  $A$  as LHS  $(A \Rightarrow^* J)$  and  $B_x$  is an explicit ordering of  $bn(J)$ . An active item is of the form  $[A \rightarrow R, \eta, I, \varphi]$  where  $\eta$  is in  $T_R$ ,  $I$  is a subgraph of  $G$  which derives from  $R_{\geq \eta}$  and  $\varphi$  is the bijection from  $bn(R_{\geq \eta})$  to  $bn(I)$ . A small number of inference rules (as shown in Fig. 5) are sufficient to control merging the chart items. R0 is applied on the root node of  $T_R$ . R1, R2.T, R2.NT and R3 are applied on leaf nodes, unary nodes that introduce a terminal edge, unary nodes that introduce a non-terminal edge and binary nodes respectively.  $e^*$  is an edge of  $G$  such that  $\ell(e) = \ell(e^*)$ .  $\{e \mapsto e^*\}$  or  $\{e \mapsto X\}$  represets the mapping that sends each node of  $e$  to the corresponding node of  $e^*$  or  $X$ .  $\psi(X_R)$  denotes a list generated by applying  $\psi$  on each node of  $X_R$  in order. Refer to the original paper for a complete description of the algorithm. See the bottom part of Fig. 5 for a partial recognition along with  $T_1$  in Fig. 4.
3.2 Treewidth-centric Complexity Analysis
It is an advantage of using tree decomposition that the treewidth of a grammar leads to a bound on the number of boundary nodes which we must keep track of during parsing. When applying an inference rule at  $\eta$ , all mentioned boundary nodes are called active nodes and denoted as  $A(\eta)$ .  $A(\eta) = bn(R_{\geq \eta_1}) \cup bn(R_{\geq \eta_2})$  if  $\eta$  is binary, and  $A(\eta) = bn(R_{\geq \eta_1}) \cup V(e)$  otherwise. Let  $k$  be the treewidth of a grammar and  $d$  be the maximum degree of any node in the input graph. The number of rule instan
tiations at  $\eta$  is actually in  $\mathcal{O}(n^{|A_{\eta}|}3^{d|A_{\eta}|})$ . The first part  $n^{|A_{\eta}|}$  is the number of ways of mappings between active nodes in a rule and nodes in an input graph. The second part  $3^{d|A_{\eta}|}$  is an upper bound of realizations for boundary edges. Chiang et al. proves that  $A(\eta)\subseteq V_{\eta}$ , implying that  $k + 1$  is an upper bound of  $|A(\eta)|$ . Therefore, the time complexity is in  $\mathcal{O}((3^d n)^{k + 1})$ . The space complexity is in  $\mathcal{O}((2^{d}n)^{k + 1})$  by a parallel analysis.
3.3 Measuring Practical Performance
Successful integration of two chart items according to an inference rule requires that the items are disjoint and can make up a new bijection. When two chart items pass the check, the following successful integration is viewed as a successful rule instantiation, and in this case, the operation cost is taken into account. When two chart items fail to pass the check, there will be no successful rule instantiation, and in this case, the operation cost for this failed integration is overlooked by the treewidth-centric complexity analysis. The cost to figure out an integration is impossible is actually comparable to that of a successful integration.
Measuring practical performance with respect to both successful and failed integration operations is a necessary complement to the theoretical analysis, especially when the number of failed integrations is prominent. In the following experiments, we will report the exact numbers for successful (indicated as #Succ) and total (=successful+failed; indicated as #Total) integrations.
3.4 Evaluation with a Realistic Grammar
To profile the parsing algorithm, we conduct experiments on the Elementary Dependency Structure (EDS; Oepen and Lønning, 2006) graphs provided by DeepBank v1.1 (Flickinger et al., 2012). The data is separated into training, development and test sets according to standard setup for string parsing. We get a wide-coverage linguisticallymeaningful grammar $^{2}$  by applying the grammar extraction algorithm described in Chen et al. (2018). The grammar is lexicalized (LxG), in that argument-structures are lexically encoded, like almost all popular deep grammars used in NLP. Tab. 1 shows the statistics of the rules.
LxG	#Rule	Treewidth	#Node	#Terminal
Lexical	46,101	avg.	1.07	2.15	2.47
max.	4	10	18
Phrasal	8,594	avg.	1.62	2.94	0.79
max.	6	7	10
Referring to Bolinas $^3$ , we re-implement the algorithm in  $\mathrm{C + + }$  and test its efficiency on 4500 EDS graphs that are randomly selected from the training set with the size in the range of 5 to 50. By size of a graph, we mean the number of its nodes. If the number of total subgraphs allocated during parsing is larger than  $2.6\times 10^{7}$ , the parser will throw an out-of-memory error (OOM). In all the following tables, all statistics are the average values over instances which successfully receive derivation forests. The platform for all experiments is x86 64 GNU/Linux with one Intel(R) Core(TM) i7-5930K CPU at 3.50GHz.
Tab. 2 summarizes the results. For small graphs, the algorithm achieves a promising speed. For larger graphs, most of parsing time is wasted on the failed integrations. Fig. 6 represents the numbers of successful and total integrations. We can clearly see that the difference between the two types of integrations increase very quickly when
Table 1: Basic properties of our lexicalized grammar. #Node and #Terminal indicate the numbers of nodes and terminal-edges in a single rule.
#Node	Time(s)	#Succ/#Total	OOM	#Graph
All	21.64	0.21%	305	4500
<10	0.02	12.55%	0	500
10~20	0.45	1.42%	0	1000
20~30	9.36	0.34%	4	1000
30~50	47.68	0.19%	301	2000
Table 2: Performance of our implementation of Chiang et al. (2013). First column is the size of input graphs. Last column is the number of graphs in given range.
an input graph is enlarged. In §4.5 we will discuss how to reduce failed integrations.
Figure 6: The numbers of successful/total integrations relative to the size of graph. All data points are the average value of multiple graphs of the same size.
4 Speeding Up by Exploiting Locality
4.1 Locality as Edge-Adjacency
Some notion of locality is conceptually necessary for studying complex structures. Adjacency is a key perspective to express locality in some linguistic theories, such as CCG (Steedman, 2000, p. 54):
(1) The Principle of (String-)Adjacency
Combinatory rules may only apply to finitely many phonologically realized and string-adjacent entities.
Almost all string parsing algorithms benefit from this string-adjacency. Now let us picture string-adjacency using a graph language. Fig. 7 gives a visualization of the linear chain structure of a word sequence. The terminal edge labeled as next in  $\gamma_{11}$  explicitly displays a local relation: ② and ③ being able to be recognized almost simultaneously. String-adjacency turns to be terminal edge-adjacency from a graph-theoretic view.
2We only consider rules the RHS of which are connected. A few graphs that are not connected and thus removed. A very small portion of DeepBank graphs result in disconnected rules. These graphs contain arguable annotations related to (1) distributive readings of coordination, (2) quantifier of bare NPs, and/or (3) small clauses. We leave appropriate analysis of these phenomena for future investigation.
3www.isi.edu/licenses-sw/bolinas/
Figure 7: A graph-based view of string-adjacency.
What does terminal edge-adjacency actually mean? From a semiotic perspective of a language system, being either natural or artificial, a key property is form-meaning connection. A particular form triggers a particular meaning. What can be observed can be directly recognized, and then makes other things recognizable. Considering language production, the input is an MR, and in the graph-based framework, it is terminal edges that are directly observable. In this way a terminal edge makes nodes connected to it co-recognizable.
The existing algorithms, including Chiang et al. (2013) and Groschwitz et al. (2015), do not consider terminal edge-adjacency. We will show that capturing locality in this sense is beneficial, just like what successful string parsing algorithms do.
4.2 Locality-centric Complexity Analysis
Some active nodes are not independent with each other if we take terminal edge-adjacency into consideration. We call a graph consisting of only terminal edges a terminal graph. For a graph fragment  $H$ , we use term(H) to denote the subgraph of  $H$  that is induced from all and only terminal edges. We informally illustrate the idea of dependency between nodes in a rule, and then present a precise analysis. Fig 8 is a prototype of a binary node in  $T_{R}$ . ⑥ ⑨ ⑤ are active nodes of  $\eta$ . But if one of these nodes is identified in an input graph, the possible positions of the other three nodes are highly restricted.
Figure 8: A prototypical case for recognizing a binary node  $\eta$  in  $T_{R}$ . The area in blue represents term  $(R_{\geq \eta_1})$  and the red one represents term  $(R_{\geq \eta_2})$ . Boundary nodes of each area are placed on the border. The nodes in black are the boundary nodes of  $R_{\geq \eta}$ .
Proposition 1. Consider a graph  $G$  and connected terminal graph  $R_{t}$ . If there is a node  $v_{1}$  in  $R_{t}$  that is tied to a node  $v_{1}^{*}$  in  $G$ , then finding all isomorphisms of  $R_{t}$  in  $G$  can be completed in  $\mathcal{O}(d^{m_t})$  time, where  $m_{t}$  is the number of edges in  $R_{t}$  and  $d$  is the maximum degree of any node in  $G$ .
Proof. We perform a depth-first search over  $R_{t}$  starting at  $v_{1}$  and arranging all edges of  $R_{t}$  as a sequence according to the order in which they are visited. Let the edge sequence be  $e_{1}, e_{2}, \ldots, e_{mt}$ . We match edges in this sequence one by one. When we handle  $e_{j} (1 \leq j \leq m_{t})$ , there must be a node  $v \in V(e_{j})$  such that  $v = v_{1}$  or  $v \in V(e_{k}) (1 \leq k < j)$ . In other words,  $v$  is already tied to a node  $v^{*} \in G$ . As a result, the number of possible mappings of  $e_{j}$  is at most  $d$ , because the degree of  $v^{*}$  is at most  $d$ . Therefore, the number of isomorphisms of  $R_{t}$  is in  $\mathcal{O}(d^{m_{t}})$ . As a result, all isomorphisms can be found in  $\mathcal{O}(d^{m_{t}})$  time.
When  $l$  active nodes locate in a connected component of term  $(R_{\geq \eta})$ , these nodes are somewhat dependent. By Proposition 1, the number of valid node mappings of these  $l$  nodes is bounded by  $\mathcal{O}(nd^{mt})$  rather than  $\mathcal{O}(n^l)$ .
Definition 1. For any node  $\eta$  in  $T_{R}$ ,  $\delta(\eta)$  denotes the size of a maximal subset of  $A(\eta)$  such that all nodes in this subset is independent with each other. We use  $S(\eta)$  to denote one of such maximal subsets. Similar to treewidth, we define  $\delta(T_{R}) = \max_{\eta} \delta(\eta)$  and  $\delta(R)$  as the minimum  $\delta$  of any tree decomposition of  $R$ .
In Fig. 8, we have  $\delta(\eta) = 4$  and  $\{\text{②} \text{③} \text{①} \text{⑥}\}$  is a maximal subset of  $A(\eta)$  as required.
Proposition 2. For any graph fragment  $R$ ,  $\delta(R) \leq k + 1$  where  $k$  is the treewidth of  $R$ .
Proof. This proposition is trivial. For any  $\eta$ , we have  $\delta(\eta) \leq |A_{\eta}| \leq |V_{\eta}| \leq k + 1$  (Proposition 3 in Chiang et al.). By the definition of  $\delta(R)$ , we have  $\delta(R) \leq \delta(T_R) = \max_{\eta} \min_{T_R} \delta(\eta) \leq k + 1$ .