Venue: ACL 2020

# Efficient Constituency Parsing by Pointing

Thanh-Tung Nguyen\({}^{\dagger}\), Xuan-Phi Nguyen\({}^{\dagger}\), Shafiq Joty\({}^{\sharp 5}\), Xiaoli Li\({}^{\dagger}\)

\({}^{\lx@sectionsign}\)Nanyang Technological University

\({}^{\lx@sectionsign}\)Salesforce Research Asia

\({}^{\dagger}\)Institute for Infocomm Research, A-STAR

Singapore

{ng0155ng@e.;nguyenxu002@ee.;srjoty@}ntu.edu.sg xlli@i2r.a-star.edu.sg

###### Abstract

We propose a novel constituency parsing model that casts the parsing problem into a series of pointing tasks. Specifically, our model estimates the likelihood of a span being a legitimate tree constituent via the pointing score corresponding to the boundary words of the span. Our parsing model supports efficient top-down decoding and our learning objective is able to enforce structural consistency without resorting to the expensive CKY inference. The experiments on the standard English Penn Treebank parsing task show that our method achieves 92.78 F1 without using pre-trained models, which is higher than all the existing methods with similar time complexity. Using pre-trained BERT, our model achieves 95.48 F1, which is competitive with the state-of-the-art while being faster. Our approach also establishes new state-of-the-art in Basque and Swedish in the SPMRL shared tasks on multilingual constituency parsing.

## 1 Introduction

Constituency or phrase structure parsing is a core task in natural language processing (NLP) with myriad downstream applications. Therefore, devising effective and efficient algorithms for parsing has been a key focus in NLP.

With the advancements in neural approaches, various neural architectures have been proposed for constituency parsing as they are able to effectively encode the input tokens into dense vector representations while modeling the structural dependencies between tokens in a sentence. These include recurrent networks Dyer et al. (2016); Stern et al. (2017b) and more recently self-attentive networks Kitaev and Klein (2018).

The parsing methods can be broadly distinguished based on whether they employ a greedy transition-based algorithm or a globally optimized chart parsing algorithm. The transition-based parsers Dyer et al. (2016); Cross and Huang (2016); Liu and Zhang (2017) generate trees autoregressively as a form of shift-reduce decisions. Though computationally attractive, the local decisions made at each step may propagate errors to subsequent steps which would suffer from exposure bias.

Chart parsing methods, on the other hand, learn scoring functions for subtrees and perform global search over all possible trees to find the most probable tree for a sentence Durrett and Klein (2015); Gaddy et al. (2018); Kitaev and Klein (2018); Kitaev et al. (2019). In this way, these methods can ensure consistency in predicting structured output. The limitation, however, is that they run slowly at \(\mathcal{O}(n^{3})\) or higher time complexity.

In this paper, we propose a novel parsing approach that casts constituency parsing into a series of pointing problems (Figure 1). Specifically,

Figure 1: A binarized constituency tree for the sentence “She enjoys playing tennis.”. The node _S-VP_ is an example of a collapsed atomic label. We omit POS tags and singleton spans for simplicity. Below the tree, we show span and pointing representations of the tree.



our parsing model estimates the pointing score from one word to another in the input sentence, which represents the likelihood of the span covering those words being a legitimate phrase structure (_i.e.,_ a subtree in the constituency tree). During training, the likelihoods of legitimate spans are maximized using the cross entropy loss. This enables our model to enforce structural consistency, while avoiding the use of structured loss that requires expensive \({\cal O}(n^{3})\) CKY inference Gaddy et al. (2018); Kitaev and Klein (2018). The training in our model can be fully parallelized without requiring structured inference as in Shen et al. (2018); Gomez and Vilares (2018). Our pointing mechanism also allows efficient top-down decoding with a best and worse case running time of \({\cal O}(n\log n)\) and \({\cal O}(n^{2})\), respectively.

In the experiments with English Penn Treebank parsing, our model without any pre-training achieves 92.78 F1, outperforming all existing methods with similar time complexity. With pre-trained BERT Devlin et al. (2019), our model pushes the F1 score to 95.48, which is on par with the state-of-the-art Kitaev et al. (2019), while supporting faster decoding. Our model also performs competitively on the multilingual parsing tasks in the SPMRL 2013/2014 shared tasks and establishes new state-of-the-art in Basque and Swedish. We will release our code at [https://ntunlpsg.github.io/project/parser/ptr-constituency-parser](https://ntunlpsg.github.io/project/parser/ptr-constituency-parser)

## 2 Model

Similar to Stern et al. (2017a), we view constituency parsing as the problem of finding a set of labeled spans over the input sentence. Let \({\cal S}(T)\) denote the set of _labeled spans_ for a parse tree \(T\). Formally, \({\cal S}(T)\) can be expressed as

\[{\cal S}(T):=\{((i_{t},j_{t}),l_{t})\}_{t=1}^{|{\cal S}(T)|}\text{ for $i_{ t}<j_{t}$} \tag{1}\]

where \(|{\cal S}(T)|\) is the number of spans in the tree. Figure 1 shows an example constituency tree and its corresponding labeled span representation.

Following the standard practice in parsing Gaddy et al. (2018); Shen et al. (2018), we convert the \(n\)-ary tree into a binary form and introduce a dummy label \(\varnothing\) to spans that are not constituents in the original tree but created as a result of binarization. Similarly, the labels in unary chains corresponding to nested labeled spans are collapsed into unique atomic labels, such as S-VP in Fig. 1.

Although our method shares the same "span-based" view with that of Stern et al. (2017a), our approach diverges significantly from their framework in the way we treat the whole parsing problem, and the representation and modeling of the spans, as we describe below.

### Parsing as Pointing

In contrast to previous approaches, we cast parsing as a series of pointing decisions. For each index \(i\) in the input sequence, the parsing model points it to another index \(p_{i}\) in order to identify the tree span \((i,p_{i})\), where \(i\neq p_{i}\). Similar to Pointer Networks Vinyals et al. (2015a), each pointing mechanism is modeled as a multinomial distribution over the indices of the input tokens (or encoder states). However, unlike the original pointer network where a decoder state points to an encoder state, in our approach, every encoder state \(\mathbf{h}_{i}\) points to another encoder state \(\mathbf{h}_{p_{i}}\).

In this paper, we generally use \(x\mathbin{\raisebox{-1.0pt}{\scalebox{1.6}{$\cdot$}}}y\) to mean \(x\) points to \(y\). We will refer to the pointing operation either as a function of the encoder states (_e.g.,_\(\mathbf{h}_{i}\mathbin{\raisebox{-1.0pt}{\scalebox{1.6}{$\cdot$}}}\mathbf{h}_{p_{i}}\)) or simply the corresponding indices (_e.g.,_\(i\mathbin{\raisebox{-1.0pt}{\scalebox{1.6}{$\cdot$}}}p_{i}\)). They both mean the same operation where the pointing function takes the encoder state \(\mathbf{h}_{i}\) as the query vector and points to \(\mathbf{h}_{p_{i}}\) by computing an attention distribution over all the encoder states.

Let \({\cal P}(T)\) denote the set of pointing decisions derived from a tree \(T\) by a transformation \({\cal H}\), _i.e.,_\({\cal H}:T\rightarrow{\cal P}(T)\). For the parsing process to be valid, the transformation \({\cal H}\) and its inverse \({\cal H}^{\prime}\) which transforms \({\cal P}(T)\) back to \(T\), should both have a one-to-one mapping property. Otherwise, the parsing model may confuse two different parse trees with the same pointing representation. In this paper, we propose a novel transformation that satisfies this property, as defined by the following proposition (proof provided in the Appendix).

**Proposition 1**: _Given a binary constituency tree \(T\) for a sentence containing \(n\) tokens, the transformation \({\cal H}\) converts it into a set of pointing decisions \({\cal P}(T)=\{(i\mathbin{\raisebox{-1.0pt}{\scalebox{1.6}{$\cdot$}}}p_{i},l_{ i}):i=1,\ldots,n-1;i\neq p_{i}\}\) such that \((\min(i,p_{i}),\max(i,p_{i}))\) is the **largest** span that starts or ends at \(i\), and \(l_{i}\) is the label of the nonterminal associated with the span._

To elaborate further, each pointing decision in \({\cal P}(T)\) represents a specific span in \({\cal S}(T)\). The pointing \(i\mathbin{\raisebox{-1.0pt}{\scalebox{1.6}{$\cdot$}}}p_{i}\) is directional, while the span that it represents \((i^{\prime},j^{\prime})\) is non-directional. In other words, there may exist position \(i\) such that \(i>p_{i}\) 

while \(i^{\prime}<j^{\prime}~{}\forall i^{\prime},j^{\prime}\in[1,n]\). In fact, it is easy to see that if the token at index \(i\) is a left-child of a subtree, the largest span involving \(i\) starts at \(i\), and in this case \(i<p_{i}\) and \(i^{\prime}=i,j^{\prime}=p_{i}\). On the other hand, if the token is a right-child of a subtree, the respective largest span ends at position \(i\), in which case \(i>p_{i}\) and \(i^{\prime}=p_{i},j^{\prime}=i\) (_e.g._, see \(4\cdot 2\) in Figure 1). In addition, as the spans in \(\mathcal{S}(T)\) are unique, it can be shown that the pointing decisions in \(\mathcal{P}(T)\) are also distinct from one another (see Appendix for a proof by contradiction).

Given such pointing formulation, for every constituency tree, there exists a trivial case \((1\mathbin{\hbox{$\sim$}}n,l_{1})\) where \(p_{1}=n\) and \(l_{1}\) is generally 'S'. Thus, to make our formulation more general with \(n\) inputs and \(n\) outputs and convenient for the method description discussed later on, we add another trivial case \((n\mathbin{\hbox{$\sim$}}1,l_{1})\). With this generalization, we can represent the pointing decisions of any binary constituency tree \(T\) as:

\[\mathcal{P}(T)=\left\{(i\mathbin{\hbox{$\sim$}}p_{i},l_{i}):i=1,\ldots,n;i \neq p_{i}\right\} \tag{2}\]

The pointing representation of the tree in Figure 1 is given at the bottom of the figure. To illustrate, in the parse tree, the largest phrase that starts or ends at token 2 ('enjoys') is the subtree rooted at '\(\varnothing\)', which spans from 2 to 5. In this case, the span _starts_ at token 2. Similarly, the largest phrase that starts or ends at token 4 ('tennis') is the span "enioys playing tennis", which is rooted at 'VP'. In this case, the span _ends_ at token 4.

Algorithm 1 describes the procedure to convert a binary tree to its corresponding pointing representation. Specifically, from each leaf token \(i\), the algorithm traverses upward along the hierarchy until the non-terminal node that does not start or end with \(i\). In this way, the largest span starting or ending with \(i\) can be identified.

### Top-Down Tree Inference

In the previous section, we described how to convert a constituency tree \(T\) into a sequence of pointing decisions \(\mathcal{P}(T)\). We use this transformation to train the parsing model (described in detail in Sections 2.3 - 2.4). During inference, given a sentence to parse, our decoder with the help of the parsing model predicts \(\mathcal{P}(T)\), from which we can construct the tree \(T\). However, not all sets of pointings \(\mathcal{P}(T)\) guarantee the generation of a valid tree. For example, for a sentence with four (4) tokens, the pointing \(\mathcal{P}(T)=\{(1\mathbin{\hbox{$\sim$}}4,l_{1}),(2\mathbin{\hbox{$\sim$}}3 ,l_{2}),(3\mathbin{\hbox{$\sim$}}4,l_{3}),(4\mathbin{\hbox{$\sim$}}1,l_{1})\}\) does not generate a valid tree because token '3' cannot belong to both spans \((2,3)\) and \((3,4)\). In other words, simply taking the \(\arg\max\) over the pointing distributions may not generate a valid tree.

Our approach to decoding is inspired by the span-based approach of Stern et al. (2017a). In particular, to reduce the search space, we score for span identification (given by the pointing function) and label assignment separately.

Span Identification.We adopt a top-down greedy approach formulated as follows.

\[k^{*} =\arg\max_{k}s_{\text{split}}(i,k,j) \tag{3}\]

where \(s_{\text{split}}(i,k,j)\) is the score of having a split-point at position \(k\) (\(i\leq k<j\)), as defined by the following equation.

\[s_{\text{split}}(i,k,j)=\rho(k\mathbin{\hbox{$\sim$}}i)+\rho(k\!+\!1\mathbin {\hbox{$\sim$}}j) \tag{4}\]

where \(\rho(k\mathbin{\hbox{$\sim$}}i)\) and \(\rho(k\!+\!1\mathbin{\hbox{$\sim$}}j)\) are the pointing scores (probabilities) for spans \((i,k)\) and \((k\!+\!1,j)\), respectively. Note that the pointing scores are _asymmetric_, meaning that \(\rho(i\mathbin{\hbox{$\sim$}}j)\) may not be equal to \(\rho(j\mathbin{\hbox{$\sim$}}i)\), because pointing from \(i\) to \(j\) is different from pointing from \(j\) to \(i\). This is different from previous approaches, where the score of a span is defined to be symmetric. We build a tree for the input sentence by computing Eq. 3 recursively starting from the full sentence span \((1,n)\).

```
Input: Binary tree \(T\) and its span representation \(\mathcal{S}(T)\) Output: Pointing representation \(\mathcal{P}(T)\)\(\triangleright\)Empty pointing list \(\mathcal{P}(T)=[]\) for each leaf, in \(T\)do \(node\leftarrow\operatorname{leaf}_{i}\)  \((x,y)\leftarrow(i,i)\)\(\triangleright\)Initialize current span, \(x\leq y\)  \(l_{i}\leftarrow\varnothing\)\(\triangleright\)Initialize label of current span while\(x=i\) or \(y=i\)do \(p_{i}\gets x+y-i\)  \(l_{i}\leftarrow\) node.label \(\triangleright\)The span’s label  node \(\leftarrow\) node.parent   \((x,y)\leftarrow\) node.span \(\triangleright\)Span covered by node endwhile\(\triangleright\)Until is no longer start/end point  push\((\mathcal{P}(T),(i\mathbin{\hbox{$\sim$}}p_{i},l_{i}))\) endfor return\(\mathcal{P}(T)\)
```

**Algorithm 1** Convert binary tree to Pointing

In the general case when \(i<k<j-1\), our pointing-based parsing model should learn to assign high scores to the two spans \((i,k)\) and \((k\!+\!1,j)\), or equivalently the pointing decisions \(k\mathbin{\hbox{$\sim$}}i\) and \(k+1\mathbin{\hbox{$\sim$}}j\). However, the pointing formulation described so far omits the trivial _self-pointing 

decisions, which represent the _singleton spans_. A singleton span is only created when the splitting decision splits an \(n\)-size span into a single-token span (singleton span) and a sub-span of size \(n-1\), _i.e.,_ when \(k=i\) or \(k=j\!-\!1\). For instance, for the parsing process in Figure 2a, the splitting decision at the root span \((1,5)\) results in a singleton span \((1,1)\) and a general span \((2,5)\). For this splitting decision, Eq. 3 requires the scores of \((1,1)\) and \((2,5)\). However, the set of pointing decisions \({\cal P}(T)\) does not cover the pointing for \((1,1)\). This discrepancy can be resolved by modeling the singleton spans separately. To achieve that, we redefine Eq. 3 as follows:

\[s_{\text{split}}(i,k,j)=\\ \left\{\begin{array}{ll}sp(i\!+\!i)+gp(i\!+\!1\!\cdot\!j)& \mbox{if $k=i$}\\ gp(j\!-\!1\!\cdot\!i)+sp(j\!\cdot\!j)&\mbox{if $k=j-1$}\\ gp(k\!\cdot\!i)+gp(k\!+\!1\!\cdot\!j)&\mbox{otherwise}\end{array}\right. \tag{5}\]

where \(sp\) and \(gp\) respectively represent the scores for the singleton and general pointing functions (to be defined formally in Section 2.3).

Remark on structural consistencyIt is important to note that since the pointing functions are defined to have a global structural property (_i.e.,_ the largest span that starts/ends with \(i\)), our model inherently enforces structural consistency. The pointing formulation of the parsing problem also makes the training process simple and efficient; it allows us to train the model effectively with simple cross entropy loss (see Section 2.4).

Label AssignmentLabel assignment of spans is performed after every split decision. Specifically, as we split a span \((i,j)\) into two sub-spans \((i,k)\) and \((k\!+\!1,j)\) which corresponds to the pointing functions of \(k\!\cdot\!i\) and \(k\!+\!1\!\cdot\!j\), we perform the label assignments for the two new sub-spans as

\[l_{k}= \arg\max_{l\in L}gc(|k) \tag{6}\] \[l_{k+1}= \arg\max_{l\in L}gc(l|k+1)\]

where \(gc\) is the label classifier for any general (non-unary) span and \(L\) is the set of possible non-terminal labels. Following Shen et al. (2018), we use a separate classifier \(uc\) for determining the labels of the unary spans, _e.g.,_ the first layer of labels NP, \(\varnothing\), \(\ldots\), NP, \(\varnothing\)) in Figure 2. Also, note that the label assignment is done based on only the query vector (the encoder state that is used to point).

```
Input: Sentence length \(n\); pointing scores: \(gp(i,j)\), \(sp(i,j)\); label scores: \(gc(|t|)\), \(uc(l|i)\), \(1\leq i\leq j\leq n,l\in L_{q}/L_{u}\) Output: Parse tree \({\cal T}\)  \({\cal Q}=\big{\lfloor}1,n\big{\rfloor}\)  \({\cal S}=\big{\lceil}(1,n,\operatorname*{arg\,max}_{l}gc(|1|))\hskip 7.11319pt \triangleright\)guate of spans   \({\cal I}=\big{\lceil}(1,n,\operatorname*{arg\,max}_{l}oc(l|l))_{t=1}^{n} \operatorname*{\textit{peneral spans, labels}}\mathcal{U}=\big{\lceil}( t,t),\operatorname*{arg\,max}_{l}uc(l|t))_{t=1}^{n}\operatorname*{\textit{ penary spans, labels}}\) while\({\cal Q}\neq\varnothing\)do \((i,j)=\mathrm{pop}({\cal Q})\) if\(j\leq i+1\)then Continue endif \(k^{*}=\operatorname*{arg\,max}_{i\leq k<j}s_{\text{split}}(i,k,j) \qquad\quad\texttt{using}~{}gp,sp\) if\(k\!=\!i\) then push\(({\cal Q},(i+1,j))\) push\(({\cal S},(i+1,j,\operatorname*{arg\,max}_{l}gc(|l|+1)))\) elseif\(k=j-1\)then  push\(({\cal Q},(i,j-1))\) push\(({\cal S},(i,j-1,\operatorname*{arg\,max}_{l}gc(l|j-1)))\) else  push\(({\cal Q},(i,k))\) push\(({\cal Q},(k+1,j))\) Push\(({\cal S},(i,k,\operatorname*{arg\,max}_{l}gc(l|k)))\) push\(({\cal S},(k+1,j,\operatorname*{arg\,max}_{l}gc(l|k+1)))\) endif endwhile \({\cal T}={\cal S}\cup{\cal U}\)
```

**Algorithm 2** Pointing parsing algorithm

Figure 2 illustrates the top-down parsing process for our running example. It consists of a sequence of pointing decisions (Figure 2a, top to bottom), which are then trivially converted to the parse tree (Figure 2b). We also provide the pseudocode in Algorithm 2. Specifically, the algorithm finds the best split for the current span \((i,j)\) using the pointing scores and pushes the newly created sub-spans into the FIFO queue \({\cal Q}\). The process terminates when there are no more spans to be split. Similar to Stern et al. (2017), our parsing algorithm has the worst and best case time complexities of \({\cal O}(n^{2})\) and \({\cal O}(n\log n)\), respectively.

### Model Architecture

We now describe the architecture of our parsing model: the sentence encoder, the pointing model and the labeling model.

Sentence EncoderGiven an input sequence of \(n\) words \(\boldsymbol{X}=(x_{1},\dots,x_{n})\), we first embed each word \(x_{i}\) to its respective vector representation \(e_{i}\) as:

\[\boldsymbol{e}_{i}=\boldsymbol{e}_{i}^{\text{char}}+\boldsymbol{e}_{i}^{\text{ word}}+\boldsymbol{e}_{i}^{\text{pos}} \tag{7}\]

where \(\boldsymbol{e}_{i}^{\text{char}}\), \(\boldsymbol{e}_{i}^{\text{word}}\), \(\boldsymbol{e}_{i}^{\text{pos}}\) are respectively the character, word, and part-of-speech (POS) embeddings of the word \(x_{i}\). Following Kitaev and Klein (2018), we use a character LSTM to compute the character embedding of a word. We experiment with both randomly initialized and pre 

trained word embeddings. If pretrained embeddings are used, the word embedding \({\mathbf{e}}_{i}^{\text{word}}\) is the summation of the word's randomly-initialized embedding and the pretrained embedding. The POS embeddings (\({\mathbf{e}}_{i}^{\text{pos}}\)) are randomly initialized.

The word representations (\({\mathbf{e}}_{i}\)) are then passed to a neural network based sequence encoder to obtain their hidden representations. Since our method does not require any specific encoder, one may use any encoder model, such as Bi-LSTM Hochreiter and Schmidhuber (1997) or self-attentive encoder Kitaev and Klein (2018). In this paper, unless otherwise specified, we use the self-attentive encoder model as our main sequence encoder because of its efficiency with parallel computation. The model is factorized into content and position information in both the self-attention sub-layer and the feed-forward layer. Details about this factorization process is provided in Kitaev and Klein (2018).

Pointing and Labeling Models.The results of the aforementioned sequence encoding process are used to compute the pointing and labeling scores. More formally, the encoder network produces a sequence of \(n\) latent vectors \({\mathbf{H}}=({\mathbf{h}}_{1},\dots,{\mathbf{h}}_{n})\) for the input sequence \({\mathbf{X}}=(x_{1},\dots,x_{n})\). After that, we apply four (4) separate position-wise two-layer Feed-Forward Networks (FFN), formulated as FFN\((x)=\text{ReLU}(xW_{1}+b_{1})W_{2}+b_{2}\), to transform \({\mathbf{H}}\) into task-specific latent representations for the respective pointing and labeling tasks.

\[{\mathbf{h}}_{i}^{gp}=\text{FFN}_{gp}({\mathbf{h}}_{i});~{}~{}~{}{\mathbf{ h}}_{i}^{sp}=\text{FFN}_{sp}({\mathbf{h}}_{i}) \tag{8}\] \[{\mathbf{h}}_{i}^{gc}=\text{FFN}_{gc}({\mathbf{h}}_{i});~{}~{}~{}{\mathbf{h}}_ {i}^{uc}=\text{FFN}_{uc}({\mathbf{h}}_{i}) \tag{9}\]

Note that there is no parameter sharing between FFN\({}_{gp}\), FFN\({}_{sp}\), FFN\({}_{gc}\) and FFN\({}_{uc}\). The pointing functions are then modeled as the multinomial (or attention) distributions over the input indices for each input position \(i\) as follows.

\[gp(i,k) =\frac{\exp({\mathbf{h}}_{i}^{gp}({\mathbf{h}}_{k}^{gp})T)}{\sum_{k=1}^{ n}\exp({\mathbf{h}}_{i}^{gp}({\mathbf{h}}_{k}^{gp})^{T})} \tag{10}\] \[sp(i,k) =\frac{\exp({\mathbf{h}}_{i}^{sp}({\mathbf{h}}_{k}^{sp})T)}{\sum_{k=1}^{ n}\exp({\mathbf{h}}_{i}^{gp}({\mathbf{h}}_{k}^{sp})^{T})} \tag{11}\]

For label assignment functions, we simply feed the label representations \({\mathbf{H}}^{gc}=({\mathbf{h}}_{1}^{gc},\dots,{\mathbf{h}}_{n}^{gc})\) and \({\mathbf{H}}^{uc}=({\mathbf{h}}_{1}^{uc},\dots,{\mathbf{h}}_{n}^{uc})\) into the respective softmax classification layers as follows.

\[gc(l|i) =\frac{\exp({\mathbf{h}}_{i}^{gc}{\mathbf{w}}_{lc}^{gc})}{\sum_{l=1}^{|L _{g}|}\exp({\mathbf{h}}_{i}^{gc}{\mathbf{w}}_{l}^{gc})} \tag{12}\] \[uc(l|i) =\frac{\exp({\mathbf{h}}_{i}^{uc}{\mathbf{w}}_{lc}^{uc})}{\sum_{l=1}^{| L_{u}|}\exp({\mathbf{h}}_{i}^{uc}{\mathbf{w}}_{l}^{nc})} \tag{13}\]

Figure 2: Inferring the parse tree for a given sentence and its part-of-speech (POS) tags (predicted by an external POS tagger). Starting with the full sentence span \((1,5)\) and its label S, we predict split point \(1\) using the base (\(sp\)) and general (\(gp\)) pointing scores as per Eqn. 3-5. The left singleton span \((1,1)\) is assigned with a label NP and the right span \((2,5)\) is assigned with a label \(\varnothing\) using the label classifier \(gc\) as per Eqn. 6. The recursion of splitting and labeling continues until the process reaches a terminal node. The label assignment for the unary spans is done by the \(uc\) classifier.

 

where \(L_{g}\) and \(L_{u}\) are the set of possible labels for the general and unary spans respectively, \(\mathbf{w}^{gc}_{l}\) and \(\mathbf{w}^{uc}_{l}\) are the class-specific trainable weight vectors.

### Training Objective

We train our parsing model by minimizing the total loss \(\mathcal{L}_{total}(\theta)\) defined as:

\[\mathcal{L}_{\text{total}}(\theta)=\mathcal{L}_{gp}(\theta_{e}, \theta_{sp})+\mathcal{L}_{sp}(\theta_{e},\theta_{sp})\\ +\mathcal{L}_{gc}(\theta_{e},\theta_{gc})+\mathcal{L}_{uc}( \theta_{e},\theta_{uc}) \tag{14}\]

where each individual loss is a cross entropy loss computed for the corresponding labeling or pointing task, and \(\theta=\{\theta_{e},\theta_{gp},\theta_{sp},\theta_{gc},\theta_{uc}\}\) represents the overall model parameters; specifically, \(\theta_{e}\) denotes the encoder parameters shared by all components, while \(\theta_{gp},\theta_{sp},\theta_{gc}\) and \(\theta_{uc}\) denote the separate parameters catering for the four pointing and labeling functions, \(gp\), \(sp\), \(gc\) and \(uc\), respectively.

## 3 Experiments

To show the effectiveness of our approach, we conduct experiments on English and Multilingual parsing tasks. For English, we use the standard Wall Street Journal (WSJ) part of the Penn Treebank (PTB) Marcus et al. (1993), whereas for multilingual, we experiment with seven (7) different languages from the SPMRL 2013-2014 shared task Seddah et al. (2013): Basque, French, German, Hungarian, Korean, Polish and Swedish.

For evaluation on PTB, we report the standard labeled precision (LP), labeled recall (LR), and labelled F1 computed by evalb1. For the SPMRL datasets, we report labeled F1 and use the same setup in evalb as Kitaev and Klein (2018).

Footnote 1: [http://nlp.cs.nyu.edu/evalb/](http://nlp.cs.nyu.edu/evalb/)

### English (PTB) Experiments Setup.

We follow the standard train/valid/test split, which uses sections 2-21 for training, section 22 for development and section 23 for evaluation. This gives 45K sentences for training, 1,700 sentences for development, and 2,416 sentences for testing. Following previous studies, our model uses POS tags predicted by the Stanford togger Toutanova et al. (2003).

For our model, we adopt the self-attention encoder with similar hyperparameter details proposed by Kitaev and Klein (2018). The character embeddings are of \(64\) dimensions. For general and unary label classifiers (\(gc\) and \(uc\)), the hidden dimension of the specific position-wise feed-forward networks is 250, while those for pointing functions (\(gp\) and \(sp\)) have hidden dimensions of \(1024\). Our model is trained using the Adam optimizer Kingma and Ba (2015) with a batch size of \(100\) sentences. Additionally, we use \(100\) warm-up steps, within which we linearly increase the learning rate from \(0\) to the base learning rate of \(0.008\). Model selection for testing is performed based on the labeled F1 score on the validation set.

Results for Single Models.The experimental results on PTB for the models without pre-training are shown in Table 1. As it can be seen, our model achieves an F1 of \(92.78\), the highest among the models using _top-down inference_ strategies. Specifically, our method outperforms Stern et al. (2017a) and Shen et al. (2018) by about \(1.0\) point in F1-score. Notably, our model with LSTM encoder achieves an F1 of 92.26, which is still better than all the top-down parser methods.

On the other hand, while Kitaev and Klein (2018) and Zhou and Zhao (2019) achieve higher F1 score, their inference speed is significantly slower than ours because of the use of CKY based algorithms, which run at \(\mathcal{O}(n^{3})\) time complexity for Kitaev and Klein (2018) and \(\mathcal{O}(n^{5})\) for Zhou and Zhao (2019). Furthermore, their training objectives involve the use of structural hinge loss, which requires online CKY inference during training. This makes their training time considerably slower than that of our method, which is trained

\begin{table}
\begin{tabular}{c|c c c} \hline \hline

- & 92.10 \\ Kitaev and Klein (2018) & 93.20 & 93.90 & 93.55 \\ \hline \multicolumn{4}{c}{**Other Approaches**} \\ Gómez and Vilares (2018) &
- & 90.7 \\ Liu and Zhang (2017) &
- & 91.8 \\ Stern et al. (2017b) & 92.57 & 92.56 & 92.56 \\ Zhou and Zhao (2019) & 93.64 & 93.92 & 93.78 \\ \hline \end{tabular}
\end{table}
Table 1: Results for single models (no pre-training) on the PTB WSJ test set, Section 23.
 

directly with span-wise cross entropy loss. In addition, Zhou and Zhao (2019) uses external supervision (_head_ information) from the dependency parsing task. Dependency parsing models, in fact, have a strong resemblance to the pointing mechanism that our model employs Ma et al. (2018). As such, integrating dependency parsing information into our model may also be beneficial. We leave this for future work.

Results with Pre-trainingSimilar to Kitaev and Klein (2018) and Kitaev et al. (2019), we also evaluate our models with BERT Devlin et al. (2019) embeddings . Following them in the inclusion of contextualized token representations, we adjust the number of self-attentive layers to \(2\) and the base learning rate to \(0.00005\).

As shown in Table 2, our model achieves an F1 score of 95.48, which is on par with the state-of-the-art models. However, the advantage of our method is that it is faster than those methods. Specifically, our model runs at \({\cal O}(n^{2})\) worst-case time complexity, while that of Kitaev et al. (2019) is \({\cal O}(n^{3})\). Comparison on parsing speed is discussed in the following section.

Parsing Speed ComparisonIn addition to parsing performance in F1 scores, we also compare our parser against the previous neural approaches in terms of parsing speed. We record the parsing timing over 2416 sentences of the PTB test set with batch size of 1, on a machine with NVIDIA GeForce GTX 1080Ti GPU and Intel(R) Xeon(R) Gold 6152 CPU. This setup is comparable to the setup of Shen et al. (2018).

As shown in Table 3, our parser outperforms Shen et al. (2018) by 19 more sentences per second, despite the fact that our parsing algorithm runs at \({\cal O}(n^{2})\) worse-case time complexity while the one used by Shen et al. (2018) can theoretically run at \({\cal O}(n\log n)\) time complexity. To elaborate further, the algorithm presented in Shen et al. (2018) can only run at \({\cal O}(n^{2})\) complexity. To achieve \({\cal O}(n\log n)\) complexity, it needs to sort the list of syntactic distances, which the provided code2 does not implement. In addition, the speed up for our method can be attributed to the fact that our algorithm (see Algorithm 2) uses a _while loop_, while the algorithm of Shen et al. (2018) has many recursive function calls. Recursive algorithms tend to be less empirically efficient than their equivalent while/for loops in handling low-level memory allocations and function call stacks.

Footnote 2: [https://github.com/hantek/distance-parser](https://github.com/hantek/distance-parser)
 

method of Kitaev et al. (2019).

## 4 Related Work

Prior to the neural tsunami in NLP, parsing methods typically model correlations in the _output space_ through probabilistic context-free grammars (PCFGs) on top of sparse (and discrete) input representations either in a generative regime Klein and Manning (2003) or a discriminative regime Finkel et al. (2008) or a combination of both Charniak and Johnson (2005). Beside the chart parser approach, there is also a long tradition of transition-based parsers Sagae and Lavie (2005)

Recently, however, with the advent of powerful neural encoders such as LSTMs Hochreiter and Schmidhuber (1997), the focus has been switched more towards effective modeling of correlations in the _input's latent space,_ as the output structures are nothing but a function of the input Gaddy et al. (2018). Various neural network models have been proposed to effectively encode the dense input representations and correlations, and have achieved state-of-the-art parsing results. To enforce the structural consistency, existing neural parsing methods either employ a transition-based algorithm Dyer et al. (2016); Liu and Zhang (2017); Kitaev and Klein (2019) or a globally optimized chart-parsing algorithm Gaddy et al. (2018); Kitaev and Klein (2018).

Meanwhile, researchers also attempt to convert the constituency parsing problem into tasks that can be solved in alternative ways. For instance, Fernandez-Gonzalez and Martins (2015) transform the phrase structure into a special form of dependency structure. Such a dependency structure, however, requires certain corrections while converting back to the corresponding constituency tree. Gomez and Vilares (2018) and Shen et al. (2018) propose to map the constituency tree for a sentence of \(n\) tokens into a sequence of \(n-1\) labels or scalars based on the depth or height of the lowest common ancestors between pairs of consecutive tokens. In addition, methods like Vinyals et al. (2015); Vaswani et al. (2017) apply the sequence-to-sequence framework to "translate" a sentence into the linearized form of its constituency tree. While being trivial and simple, parsers of this type do not guarantee structural correctness, because the syntax of the linearized form is not constrained during tree decoding.

Our approach differs from previous work in that it represents the constituency structure as a series of pointing representations and has a relatively simpler cross entropy based learning objective. The pointing representations can be computed in parallel, and can be efficiently converted into a full constituency tree using a top-down algorithm. Our pointing mechanism shares certain similarities with the Pointer Network Vinyals et al. (2015), but is distinct from it in that our method points a word to another word within the same encoded sequence.

\begin{table}
\begin{tabular}{l|c c c} \hline \hline
**Language** & Train & Valid & Test \\ \hline
**Basque** & 7,577 & 948 & 946 \\
**French** & 14,759 & 1,235 & 2,541 \\
**German** & 40,472 & 5,000 & 5,000 \\
**Hebrew** & 5,000 & 500 & 716 \\
**Hungarian** & 8,146 & 1,051 & 1,009 \\
**Korean** & 23,010 & 2,066 & 2,287 \\
**Polish** & 6,578 & 821 & 822 \\
**Swedish** & 5,000 & 494 & 666 \\ \hline \hline \end{tabular}
\end{table}
Table 6: SPMRL Multilingual dataset split.

\begin{table}
\begin{tabular}{l|r r r r r r r r} \hline \hline
**Model** & **Basque** & **French** & **German** & **Hebrew** & **Hungarian** & **Korean** & **Polish** & **Swedish** \\ \hline Anders Bjorkelund and Szanto (2014) & 88.24 & 82.53 & 81.66 & 89.80 & 91.72 & 83.81 & 90.50 & 85.50 \\ Couvoux and Crabbe (2017) & 88.81 & 82.49 & 85.34 & 89.87 & 92.34 & 86.04 & 93.64 & 84.0 \\ Kitaev and Klein (2018) & 89.71 & 84.06 & 87.69 & 90.35 & 92.69 & 86.59 & 93.69 & 84.45 \\ Our Model & 90.23 & 82.20 & 84.91 & 90.63 & 91.07 & 85.36 & 93.99 & 86.87 \\ \hline \hline \end{tabular}
\end{table}
Table 4: SPMRL experiment single model test.

\begin{table}
\begin{tabular}{l|c c c c c c c} \hline \hline
**Model** & **Basque** & **French** & **German** & **Hebrew** & **Hungarian** & **Korean** & **Polish** & **Swedish** \\ \hline Kitaev et al. (2019) & 91.63 & 87.43 & 90.20 & 92.99 & 94.90 & 88.80 & 96.36 & 88.86 \\ Our model & 92.02 & 86.69 & 90.28 & 93.67 & 94.24 & 88.71 & 96.14 & 89.10 \\ \hline \hline \end{tabular}
\end{table}
Table 5: SPMRL experiment pre-trained model test (with pretraining).

 

Conclusion

We have presented a novel constituency parsing method that is based on a pointing mechanism. Our method utilizes an efficient top-down decoding algorithm that uses pointing functions for scoring possible spans. The pointing formulation inherently captures global structural properties and allows efficient training with cross entropy loss. With experiments we have shown that our method outperforms all existing top-down methods on the English Penn Treebank parsing task. Our method with pre-training rivals the state-of-the-art method, while being faster than it. On multilingual constituency parsing, it also establishes new state-of-the-art in Basque and Swedish.

## Acknowledgments

We would like to express our gratitude to the anonymous reviewers for their insightful feedback on our paper. Shafiq Joty would like to thank the funding support from his Start-up Grant (M4082038.020).

## References

* Falenska et al. (2014) Agnieszka Falenska Richard Farkas Thomas Mueller Wolfgang Seeker Anders Bjorkelund, Ozlem Cetinoglu and Zsolt Szanto. 2014. The simworkan-szeged-cis entry at the spmrl 2014 shared task: Reranking and morphosyntax meet unlabeled data. In _Proceedings of the First Joint Workshop on Statistical Parsing of Morphologically Rich Languages and Syntactic Analysis of NonCanonical Languages_, pages 97-102.
* Charniak and Johnson (2005) Eugene Charniak and Mark Johnson. 2005. Coarse-to-fine n-best parsing and MaxEnt discriminative reranking. In _Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL'05)_, pages 173-180, Ann Arbor, Michigan. Association for Computational Linguistics.
* Coavoux and Crabbe (2017) Maximin Coavoux and Benoit Crabbe. 2017. Multilingual lexicalized constituency parsing with word-level auxiliary tasks. In _Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers_, pages 331-336, Valencia, Spain. Association for Computational Linguistics.
* Cross and Huang (2016) James Cross and Liang Huang. 2016. Span-based constituency parsing with a structure-label system and provably optimal dynamic oracles. In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_, pages 1-11, Austin, Texas. Association for Computational Linguistics.
* Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
* Durrett and Klein (2015) Greg Durrett and Dan Klein. 2015. Neural CRF parsing. In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, pages 302-312, Beijing, China. Association for Computational Linguistics.
* Dyer et al. (2016) Chris Dyer, Adhiguna Kuncoro, Miguel Ballesteros, and Noah A. Smith. 2016. Recurrent neural network grammars. In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 199-209, San Diego, California. Association for Computational Linguistics.
* Fernandez-Gonzalez and Martins (2015) Daniel Fernandez-Gonzalez and Andre F. T. Martins. 2015. Parsing as reduction. In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, pages 1523-1533, Beijing, China. Association for Computational Linguistics.
* Finkel et al. (2008)Jenny Rose Finkel, Alex Kleeman, and Christopher D. Manning. 2008. Efficient, feature-based, conditional random field parsing. In _Proceedings of ACL-08: HLT_, pages 959-967, Columbus, Ohio. Association for Computational Linguistics.
* Gaddy et al. (2018) David Gaddy, Mitchell Stern, and Dan Klein. 2018. What's going on in neural constituency parsers? an analysis. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_, pages 999-1010. Association for Computational Linguistics.
* Gomez et al. (2018) Carlos Gomez, Rodriguez and David Vilares. 2018. Constituent parsing as sequence labeling. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_, pages 1314-1324. Association for Computational Linguistics.
* Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jurgen Schmidhuber. 1997. Long short-term memory. _Neural computation_, 9(8):1735-1780.
* Kingma and Ba (2015) Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_ 

Nikita Kitaev, Steven Cao, and Dan Klein. 2019. Multilingual constituency parsing with self-attention and pre-training. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 3499-3505, Florence, Italy. Association for Computational Linguistics.
* Kitaev and Klein (2018) Nikita Kitaev and Dan Klein. 2018. Constituency parsing with a self-attentive encoder. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 2676-2686, Melbourne, Australia. Association for Computational Linguistics.
* Kitaev and Klein (2019) Nikita Kitaev and Dan Klein. 2019. Tetra-tagging: Word-synchronous parsing with linear-time inference. _CoRR_, abs/1904.09745.
* Klein and Manning (2003) Dan Klein and Christopher D. Manning. 2003. Accurate unlexicalized parsing. In _Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics_, pages 423-430, Sapporo, Japan. Association for Computational Linguistics.
* Liu and Zhang (2017) Jiangming Liu and Yue Zhang. 2017. Shift-reduce constituent parsing with neural lookahead features. _Transactions of the Association for Computational Linguistics_, 5:45-58.
* Ma et al. (2018) Xuezhe Ma, Zecong Hu, Jingzhou Liu, Nanyun Peng, Graham Neubig, and Eduard Hovy. 2018. Stackpointer networks for dependency parsing. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 1403-1414, Melbourne, Australia. Association for Computational Linguistics.
* Marcus et al. (1993) Mitchell P. Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a large annotated corpus of english: The penn treebank. _Comput. Linquist._, 19(2):313-330.
* Petrov and Klein (2007) Slav Petrov and Dan Klein. 2007. Improved inference for unlexicalized parsing. In _Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference_, pages 404-411, Rochester, New York. Association for Computational Linguistics.
* Sagae and Lavie (2005) Kenji Sagae and Alon Lavie. 2005. A classifier-based parser with linear run-time complexity. In _Proceedings of the Ninth International Workshop on Parsing Technology_, pages 125-132. Association for Computational Linguistics.
* Seddah et al. (2013) Djame Seddah, Reut Tsarfaty, Sandra Kubler, Marie Candito, Jinho D. Choi, Richard Farkas, Jennifer Foster, Iakes Goenaga, Koldo Gojenola Galletbeitia, Yoav Goldberg, Spence Green, Niczr Habash, Marco Kuhlmann, Wolfgang Maier, Joakim Nivre, Adam Przepiorkowski, Ryan Roth, Wolfgang Seeker, Yannick Versley, Veronika Vincze, Marcin Wolinski, Alina Wroblewska, and Eric Villemontte de la Clergerie. 2013. Overview of the SPMRL 2013 shared task: A cross-framework evaluation of parsing morphologically rich languages. In _Proceedings of the Fourth Workshop on Statistical Parsing of Morphologically-Rich Languages_, pages 146-182, Seattle, Washington, USA. Association for Computational Linguistics.
* Shen et al. (2018) Yikang Shen, Zhouhan Lin, Athul Paul Jacob, Alessandro Sordoni, Aaron Courville, and Yoshua Bengio. 2018. Straight to the tree: Constituency parsing with neural syntactic distance. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 1171-1180, Melbourne, Australia. Association for Computational Linguistics.
* August 4, Volume 1: Long Papers_, pages 818-827.
* Stern et al. (2017b) Mitchell Stern, Daniel Fried, and Dan Klein. 2017b. Effective inference for generative neural parsing. In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_, pages 1695-1700, Copenhagen, Denmark. Association for Computational Linguistics.
* Volume 1_, NAACL '03, pages 173-180. Association for Computational Linguistics.
* Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, L ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, _Advances in Neural Information Processing Systems 30_, pages 5998-6008. Curran Associates, Inc.
* Vinyals et al. (2015a) Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. 2015a. Pointer networks. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, _Advances in Neural Information Processing Systems 28_, pages 2692-2700. Curran Associates, Inc.
* Vinyals et al. (2015b) Oriol Vinyals, L ukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoffrey Hinton. 2015b. Grammar as a foreign language. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, _Advances in Neural Information Processing Systems 28_, pages 2773-2781. Curran Associates, Inc.
* Zhou and Zhao (2019) Junru Zhou and Hai Zhao. 2019. Head-driven phrase structure grammar parsing on penn treebank. In 

Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers_, pages 2396-2408.
* Zhu et al. (2013) Muhua Zhu, Yue Zhang, Wenliang Chen, Min Zhang, and Jingbo Zhu. 2013. Fast and accurate shift-reduce constituent parsing. In _Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics_, pages 434-443. Association for Computational Linguistics.

## Appendix

Proof of Proposition 1Given \({\cal P}(T)=\{(i\cdot p_{i},l_{i}):i=1,\ldots,n-1;i\neq p_{i}\}\), generated from tree \(T\) (here we omit the unary leaves and POStags), we at first define the inverse \({\cal H^{\prime}}\) as follows:

\[{\cal H^{\prime}}({\cal P}(T))=\{((min(i,p_{i}),max(i,p_{i})),l_{ i}):\\ i=1,\ldots,n-1\}\]

We would prove \({\cal H^{\prime}}({\cal P}(T))=T\)

A binary tree \(T\) has exactly \(n-1\) internal nodes (or spans). It is noteworthy to mention that for each pointing \((i\cdot p_{i},l_{i})\), \(((\min(i,p_{i}),\max(i,p_{i})),l_{i})\) is a span in \(T\). As we consider \(i\) from \(1\) to \(n-1\), there are totally at most n-1 such spans in \({\cal H^{\prime}}({\cal P}(T))\)(we do not know whether these spans are not be distinct). Therefore, if we can prove that all \(((\min(i,p_{i}),\max(i,p_{i})),l_{i})\) spans are distinct for \(i=1,\ldots,n-1\), \({\cal H^{\prime}}({\cal P}(T))\) will cover all the span in \(T\), therefore, \({\cal H^{\prime}}({\cal P}(T))=T\). We prove this by contradiction.

Assume that there exist \(i,j\in\{1,\ldots,n-1\}\) such that \((\min(i,p_{i}),\max(i,p_{i}))=(\min(j,p_{j}),\max(j,p_{j}))\) for \(j\neq i\). First, if \(p_{i}=n\), then according to the above condition, \((\min(j,p_{j}),\max(j,p_{j}))=(\min(i,n),\max(i,n))=(i,n)\). This means, either \(j=n\) or \(j=i\), which contradicts with our initial assumption that \(j\neq i\) and \(j\in\{1,\ldots,n-1\}\). So, \(p_{i}\) cannot be equal to \(n\). Similarly, we can prove that \(p_{j}\) also cannot be equal to \(n\). Thus, we can conclude that \(p_{i},p_{j}\in\{1,\ldots,n-1\}\). Now, without loss of generality, let us assume that \(j>i\). With this assumption, the two spans will be identical if and only if \(p_{i}=j\) and \(p_{j}=i\). In this case, the span \((i,j)\) would be the largest span that starts with \(i\) and ends at \(j\). However, since \(1\leq i<j\leq n-1\), the span \((i,j)\) must be a left or right child of another (parent) span. If \((i,j)\) is the left child, then the parent span needs to start with \(i\), making it larger than \((i,j)\). This contradicts to the property that \((i,j)=(i,p_{i})\) is the largest span that starts or ends at \(i\). Similarly, if \((i,j)\) is the right child, then the parent span needs to end at \(j\), making it larger than \((i,j)\). This again contradicts to the property that \((j,i)=(j,p_{j})\) is the largest span that starts or ends at \(j\).

In conclusion, we have \({\cal H^{\prime}}({\cal P}(T))=T\). This would guarantee that \({\cal H}\) and \({\cal H^{\prime}}\) are one-to-one: If there exist \(T_{1},T_{2}\) such that \({\cal P}(T_{1})={\cal P}(T_{2})\), we would have \({\cal H^{\prime}}({\cal P}(T_{1}))={\cal H^{\prime}}({\cal P}(T_{2}))\) or \(T_{1}=T_{2}\).If there exist \(T_{1},T_{2}\) such that \({\cal H^{\prime}}({\cal P}(T_{1}))={\cal H^{\prime}}({\cal P}(T_{2}))\), we would have \(T_{1}=T_{2}\).