Venue: NAACL 2019

# Better, Faster, Stronger Sequence Tagging Constituent Parsers

David Vilares

Universidade da Coruna, CITIC

Departamento de Computacion

A Coruna, Spain

david.vilares@udc.es &Mostafa Abdou, Anders Sogaard

University of Copenhagen

Department of Computer Science

Copenhagen, Denmark

{abdou,soegaard}@di.ku.dk

###### Abstract

Sequence tagging models for constituent parsing are faster, but less accurate than other types of parsers. In this work, we address the following weaknesses of such constituent parsers: (a) high error rates around closing brackets of long constituents, (b) large label sets, leading to sparsity, and (c) error propagation arising from greedy decoding. To effectively close brackets, we train a model that learns to switch between tagging schemes. To reduce sparsity, we decompose the label set and use multi-task learning to jointly learn to predict sublabels. Finally, we mitigate issues from greedy decoding through auxiliary losses and sentence-level fine-tuning with policy gradient. Combining these techniques, we clearly surpass the performance of sequence tagging constituent parsers on the English and Chinese Penn Treebanks, and reduce their parsing time even further. On the spmrl datasets, we observe even greater improvements across the board, including a new state of the art on Basque, Hebrew, Polish and Swedish.1

Footnote 1: After this paper was submitted, Kitaev and Klein [2018] have improved our results using their previous self-attentive constituent parser Kitaev and Klein [2018] and bert representations Devlin et al. [2018] as input to their system. We will acknowledge these results in the Experiments section.

## 1 Introduction

Constituent parsing is a core task in natural language processing (nlp), with a wide set of applications. Most competitive parsers are slow, however, to the extent that it is prohibitive of downstream applications in large-scale environments Kummerfeld et al. [2012]. Previous efforts to obtain speed-ups have focused on creating more efficient versions of traditional shift-reduce Sagae and Lavie [2006]; Zhang and Clark [2009] or chart-based parsers Collins [1997]; Charniak [2000]. Zhu et al. [2013], for example, presented a fast shift-reduce parser with transitions learned by a svm classifier. Similarly, Hall et al. [2014] introduced a fast gpu implementation for Petrov and Klein [2007], and Shen et al. [2018] significantly improved the speed of the Stern et al. [2017] greedy top-down algorithm, by learning to predict a list of syntactic distances that determine the order in which the sentence should be split.

In an alternative line of work, some authors have proposed new parsing paradigms that aim to both reduce the complexity of existing parsers and improve their speed. Vinyals et al. [2015] proposed a machine translation-inspired sequence-to-sequence approach to constituent parsing, where the input is the raw sentence, and the 'translation' is a parenthesized version of its tree. Gomez-Rodriguez and Vilares [2018] reduced constituent parsing to sequence tagging, where only \(n\) tagging actions need to be made, and obtained one of the fastest parsers to date. However, the performance is well below the state of the art Dyer et al. [2016]; Stern et al. [2017].

ContributionWe first explore different factors that prevent sequence tagging constituent parsers from obtaining better results. These include: high error rates when long constituents need to be closed, label sparsity, and error propagation arising from greedy inference. We then present the technical contributions of the work. To effectively close brackets of long constituents, we combine the relative-scale tagging scheme used by Gomez-Rodriguez and Vilares [2018] with a secondary top-down absolute-scale scheme. This makes it possible to train a model that learns how to switch between two encodings, depending on which one is more suitable at each time step. To reduce label sparsity, we recast the constituent-parsing-as-sequence-tagging problem as multi-task learning (mtl) Caruana [1997], to decompose a large label

space and also obtain speed ups. Finally, we mitigate error propagation using two strategies that come at no cost to inference efficiency: auxiliary tasks and policy gradient fine-tuning.

## 2 Preliminaries

We briefly introduce preliminaries that we will build upon in the rest of this paper: encoding functions for constituent trees, sequence tagging, multi-task learning, and reinforcement learning.

NotationWe use \(w\)=\([w_{0},w_{1},...,w_{n}]\) to refer to a raw input sentence and bold style lower-cased and math style upper-cased characters to refer to vectors and matrices, respectively (e.g. **x** and **W**).

Constituent Parsing as Sequence Tagging Gomez-Rodriguez and Vilares (2018) define a linearization function of the form \(\Phi_{|w|}:T_{|w|}\rightarrow L^{(|w|-1)}\) to map a phrase structure tree with \(|w|\) words to a sequence of labels of length \(|w|-1\).2 For each word \(w_{t}\), the function generates a label \(l_{t}\in L\) of the form \(l_{t}\)=\((n_{t},c_{t},u_{t})\), where: Footnote 2: They (1) generate a dummy label for the last word and (2) pad sentences with a beginning- and end-of-sentence tokens.

* \(n_{t}\) encodes the number of ancestors in common between between \(w_{t}\) and \(w_{t+1}\). To reduce the number of possible values, \(n_{t}\) is encoded as the relative variation in the number of common ancestors with respect to \(n_{t-1}\).
* \(c_{t}\) encodes the lowest common ancestor between \(w_{t}\) and \(w_{t+1}\).
* \(u_{t}\) contains the unary branch for \(w_{t}\), if any.

Figure 1 explains the encoding with an example.

### Sequence Tagging

Sequence tagging is a structured prediction task that generates an output label for every input token. Long short-term memory networks (lstm) Hochreiter and Schmidhuber (1997) are a popular architecture for such tasks, often giving state-of-the-art performance Reimers and Gurevych (2017); Yang and Zhang (2018).

Tagging with lstmsIn lstms, the prediction for the \(i\)th element is conditioned on the output of the previous steps. Let lstM\({}_{\theta}\)(\(\mathbf{x}_{1:n}\)) be a parametrized function of the network, where the input is a sequence of vectors \(\mathbf{x}_{1:n}\), its output is a sequence of hidden vectors \(\mathbf{h}_{1:n}\). To obtain better contextualized hidden vectors, it is possible to instead use bidirectional lstms Schuster and Paliwal (1997). First, a lstM\({}_{\theta}^{l}\) processes the tokens from left-to-right and then an independent lstM\({}_{\theta}^{r}\) processes them from right-to-left. The \(i\)th final hidden vector is represented as the concatenation of both outputs, i.e. bilstM\({}_{\theta}\)(\(\mathbf{x},i\)) = lstM\({}_{\theta}^{l}\)(\(\mathbf{x}_{[1:i]}\)) o lstM\({}_{\theta}^{r}\)(\(\mathbf{x}_{||\mathbf{x}::i]}\)). bilstms can be stacked in order to obtain richer representations. To decode the final hidden vectors into discrete labels, a standard approach is to use a feed-forward network together with a softmax transformation, i.e. \(P(y|\mathbf{h}_{i})\) = \(softmax(W\cdot\mathbf{h}_{i}+\mathbf{b})\). We will use the bilstm-based model by Yang and Zhang (2018), for direct comparison against Gomez-Rodriguez and Vilares (2018), who use the same model. As input, we will use word embeddings, PoS-tag embeddings and a second word embedding learned by a character-based lstm layer. The model is optimized minimizing the categorical cross-entropy loss, i.e. \(\mathcal{L}\) = \(-\sum log(P(y|\mathbf{h}_{i}))\). The architecture is shown in Figure 2.

### Multi-task Learning

Multi-task learning is used to solve multiple tasks using a single model architecture, with task-specific classifier functions from the outer-most representations Caruana (1997); Collobert and Weston (2008). The benefits are intuitive: sharing a common representation for different tasks acts as a generalization mechanism and allows to address them in a parallel fashion. The _hard-sharing_ strategy is the most basic mtl architecture, where the internal representation is fully shared across all tasks. The approach has proven robust for a number of nlp tasks Bingel and Sogaard (2017) and comes with certain guarantees if a common, op

Figure 1: A constituent tree linearized as by Gómez-Rodríguez and Vilares (2018).

 

timal representation exists Baxter (2000). Dong et al. (2015) use it for their multilingual machine translation system, where the encoder is a shared gated recurrent neural network Cho et al. (2014) and the decoder is language-specific. Plank et al. (2016) also use a hard-sharing setup to improve the performance of bilstm-based PoS taggers. To do so, they rely on _auxiliary tasks_, i.e, tasks that are not of interest themselves, but that are co-learned in a mtl setup with the goal of improving the network's performance on the main task(s). We will introduce auxiliary tasks for sequence tagging constituent parsing later on in this work. A mtl architecture can also rely on _partial sharing_ when the different tasks do not fully share the internal representations Duong et al. (2015); Rei (2017); Ruder et al. (2019) and recent work has also shown that hierarchical sharing (e.g. low-level task outputs used as input for higher-level ones) could be beneficial Sogaard and Goldberg (2016); Sanh et al. (2018).

### Policy Gradient Fine-tuning

Policy gradient (pg) methods are a class of reinforcement learning algorithms that directly learn a parametrized policy, by which an agent selects actions based on the gradient of a scalar performance measure with respect to the policy. Compared to other reinforcement learning methods, pg is well-suited to nlp problems due to its appealing convergence properties and effectiveness in high-dimensional spaces Sutton and Barto (2018).

Previous work on constituent parsing has employed pg methods to mitigate the effect of exposure bias, finding that they function as a model-agnostic substitute for dynamic oracles Fried and Klein (2018). Similarly, Le and Fokkens (2017) apply pg methods to Chen and Manning (2014)'s transition-based dependency parser to reduce error propagation. In this work, we also employ pg to fine-tune models trained using supervised learning. However, our setting (sequence tagging) has a considerably larger action space than a transition parser. To deal with that, we will adopt a number of variance reduction and regularization techniques to make reinforcement learning stable.

## 3 Methods

We describe the methods introduced in this work, motivated by current limitations of existing sequence tagging models, which are first reviewed. The source code can be found as a part of [https://github.com/aghiE/tree2labels](https://github.com/aghiE/tree2labels).

### Motivation and Analysis

For brevity, we limit this analysis to the English Penn Treebank (ptb) Marcus et al. (1993). We reproduced the best setup by Gomez-Rodriguez and Vilares (2018), which we are using as baseline, and run the model on the development set. We below show insights for the elements of the output tuple \((n_{t},c_{t},u_{t})\), where \(n_{t}\) is the number of levels in common between \(w_{t}\) and \(w_{t+1}\), \(c_{t}\) is the non-terminal symbol shared at that level, and \(u_{t}\) is a leaf unary chain located at \(w_{t}\).

High error rate on closing bracketsWe first focus on predicting relative tree levels (\(n_{t}\)). See Figure 3 for F-scores over \(n_{t}\) labels. The sparsity on negative \(n_{t}\)s is larger than for the positive ones, and we see that consequently, the performance is also significantly worse for negative \(n_{t}\) values, and performance worsens with higher negative values. This indicates that the current model cannot effectively identify the end of long constituents. This is a known source of error for shift-reduce or chart-based parsers, but in the case of sequence tagging parsers, the problem seems particularly serious.

SparsityThe label space is large and sparse: the output labels are simply the possible values in the tuple \((n_{t},c_{t},u_{t})\). An analysis over the ptb training set shows a total of 1423 labels, with 58% of them occurring 5 or less times. These infrequent cases might be difficult to predict, even if some of the elements of the tuple are common.

Figure 2: The baseline architecture used in this work. The input to the network is a concatenation of word embeddings, PoS-tag embeddings and a second word embedding learned by a character-based lstm layer.

 

Greeedy decodingGreedy decoding is prone to issues such as error propagation. This is a known source of error in transition-based dependency parsing Qi and Manning (2017); in contrast with graph-based parsing, in which parsing is reduced to global optimization over edge-factored scores McDonald et al. (2005).

In the case of bilstm-based sequence tagging parsers, for a given word \(w_{t}\), the output label as encoded by Gomez-Rodriguez and Vilares (2018) only reflects a relation between \(w_{t}\) and \(w_{t+1}\). We hypothesize that even if the hidden vector representations are globally contextualized over the whole sequence, the intrinsic locality of the output label also turns into error propagation and consequently causes a drop in the performance. These hypotheses will be tested in SS4. In particular, we will evaluate the impact of the different methods intended to perform structured inference (SS3.4).

### Dynamic Encodings

Gomez-Rodriguez and Vilares (2018) encode the number of common ancestors \(n_{t}\), from the output tuple \((n_{t},c_{t},u_{t})\), as the variation with respect to \(n_{t-1}\). We propose instead to encode certain elements of a sentence using a secondary linearization function. The aim is to generate a model that can dynamically switch between different tagging schemes at each time step \(t\) to select the one that represents the relation between \(w_{t}\) and \(w_{t+1}\) in the most effective way.

On the one hand, the relative-scale encoding is effective to predict the beginning and the end of short constituents, i.e. when a short constituent must be predicted (\(|n_{t}|\leq 2\)). On the other hand, with a relative encoding scheme, the F-score was low for words where the corresponding \(n_{t}\) has a large negative value (as showed in Figure 3). This matches a case where a long constituent must be closed: \(w_{t}\) is located at a deep level in the tree and will only (probably) share a few ancestors with \(w_{t+1}\). These configurations are encoded in a more sparse way by a relative scheme, as the \(n_{t}\) value shows a large variability and it depends on the depth of the tree in the current time step. We can obtain a compressed representation of these cases by using a _top-down absolute scale_ instead, as any pair of words that share the same \(m\) top levels will be equally encoded. The absolute scale becomes however sparse when predicting deep levels. Figure 4 illustrates the strengths and weaknesses of both encodings with an example, and how a dynamically encoded tree helps reduce variability on \(n_{t}\) values.

In our particular implementation, we will be using the following setup:

* \(\Phi_{|w|}:T_{|w|}\to L^{|w|-1}\), the relative-scale encoding function, is used by default.
* \(\Omega_{|w|}:T_{|w|}\to L^{I|w|-1}\) is the secondary linearization function that maps words to labels according to a top-down absolute scale. \(\Omega\) is used iff: (1) \(\Omega(w_{[trd+1]})\) = \((n_{t}^{\prime},c_{t}^{\prime},u_{t}^{\prime})\) with \(n_{t}^{\prime}\leq 3\), i.e. \(w_{t}\) and \(w_{t+1}\) share at most the three top levels, and (2) \(\Phi(w_{[t:t+1]})\) = \((n_{t},c_{t},u_{t})\) with \(n_{t}\leq-2\), i.e. \(w_{t}\) is at least located two levels deeper in the tree than \(w_{t+1}\).3

Figure 4: A synthetic constituent tree where \(n_{t}\) is encoded using a relative scheme, a top-down absolute scale, and an ideal dynamic combination. The relative scheme is appropriate to open and close short constituents, but becomes sparse when encoding the large ones, e.g. \(n_{t}\) for the tokens ‘e’, ’h’ and ’l’. The opposite problem is observed for the top-down absolute scheme (e.g. tokens from ‘a’ to ‘d’). The dynamic linearization combines the best of both encodings (we use the subscript ‘r’ to denote the labels coming from the relative encoding, and ‘a’ from the absolute one).

Figure 3: F-score for \(n_{t}\) labels on the ptb dev set using Gómez-Rodríguezuez and Vilares (2018).

 



### Decomposition of the label space

We showed that labels of the form \((n_{t},c_{t},u_{t})\in L\) are sparse. An intuitive approach is to decompose the label space into three smaller sub-spaces, such that \(n_{i}\in N\), \(c_{i}\in C\) and \(u_{i}\in U\). This reduces the output space from potentially \(|N|\times|C|\times|U|\) labels to just \(|N|+|C|+|U|\). We propose to learn this decomposed label space through a multi-task learning setup, where each of the subspaces is considered a different task, namely task\({}_{N}\), task\({}_{C}\) and task\({}_{U}\). The final loss is now computed as \(\mathcal{L}=\mathcal{L}_{n}+\mathcal{L}_{c}+\mathcal{L}_{u}\).

We relied on a hard-sharing architecture, as it has been proved to reduce the risk of overfitting the shared parameters Baxter (1997). A natural issue that arises is that the prediction of labels from different label sub-spaces could be interdependent to a certain extent, and therefore a hierarchical sharing architecture could also be appropriate. To test this, in preliminary experiments we considered variants of hierarchical sharing architectures. We fed the output of the task\({}_{U}\) as input to task\({}_{N}\) and/or task\({}_{C}\). Similarly, we tested whether it was beneficial to feed the output of task\({}_{N}\) into task\({}_{C}\), and viceversa. However, all these results did not improve those of the hard-sharing model. In this context, in addition to a generalization mechanism, the shared representation could be also acting as way to keep the model aware of the potential interdependencies that might exist between subtasks.

### Mitigating Effects of Greedy Decoding

We propose two ways to mitigate error propagation arising from greedy decoding in constituent parsing as sequence tagging: auxiliary tasks and policy gradient fine-tuning. Note that we want to optimize bracketing F-score _and_ speed. For this reason we do _not_ explore approaches that come at a speed cost in testing time, such as beam-search or using conditional random fields Lafferty et al. (2001) on top of our lstm.

Auxiliary tasksAvailairy tasks force the model to take into account patterns in the input space that can be useful to solve the main task(s), but that remain ignored due to a number of factors, such as the distribution of the output label space Rei (2017). In a similar fashion, we use auxiliary tasks as a way to force the parser to pay attention to aspects beyond those needed for greedy decoding. We propose and evaluate two separate strategies:

1. Predict partial labels \(n_{t+k}\) that are \(k\) steps from the current time step \(t\). This way we can jointly optimize at each time step a prediction for the pairs \((w_{t},w_{t+1})\), ..., \((w_{t+k},w_{t+k+1})\). In particular, we will experiment both with previous and upcoming \(n_{k}\)'s, setting \(|k|\)=1.
2. Predict the syntactic distances presented by Shen et al. (2018), which reflect the order a sentence must be split to obtain its constituent tree using a top-down parsing algorithm Stern et al. (2017). The algorithm was initially defined for binary trees, but its adaptation to \(n\)-ary trees is immediate: leaf nodes have a split priority of zero and the ancestors' priority is computed as the maximum priority of their children plus one. In this work, we use this algorithm in a sequence tagging setup: the label assigned to each token corresponds to the syntactic distance of the lowest common ancestor with the next token. This is illustrated in Figure 5.

The proposed auxiliary tasks provide different types of contextual information. On the one hand, the encoding of the \(n_{t}\)s by Gomez-Rodriguez and Vilares (2018) only needs to know about \(w_{t}\) and \(w_{t+1}\) paths to generate the label for the time step \(t\). On the other hand, to compute the syntactic distance of a given non-terminal symbol, we need to compute the syntactic distances of its subtree, providing a more global, but also sparser context. For training, the loss coming from the auxiliary task(s) is weighted by \(\beta\)=0.1, i.e, the final loss is computed as \(\mathcal{L}=\mathcal{L}_{n}+\mathcal{L}_{c}+\mathcal{L}_{u}+\beta\sum_{a} \mathcal{L}_{a}\).

Figure 5: A constituent with syntactic distances attached to each non-terminal symbol, according to Shen et al. (2018). Distances can be used for sequence tagging, providing additional information to our base encoding Gómez-Rodríguez and Vilares (2018)

Policy gradient fine-tuningPolicy gradient training methods allow us to fine-tune our models with a tree-level objective, optimizing directly for bracketing F-score. We start off with a converged supervised model as our initial policy. The sequence labeling model can be seen as a functional approximation of the policy \(\pi\) parametrized by \(\theta\), which at timestep \(t\) selects a label \(l_{t}\)=\((n_{t},c_{t},u_{t})\)4 given the current state of the model's parameters, \(s_{t}\). The agent's reward, \(R_{tree}\), is then derived from the bracketing F-score. This can be seen as a variant of the reinforce algorithm Williams (1992) where the policy is updated by gradient ascent in the direction of:

Footnote 4: [https://github.com/google-research/bert/blob/master/multilingual.md](https://github.com/google-research/bert/blob/master/multilingual.md)

\[\Delta_{\theta}log\pi(l_{t}|s_{t};\theta)R_{tree} \tag{1}\]

Baseline and Variance ReductionWe use as baseline a copy of a pre-trained model where the parameters are frozen. The reward used to scale the policy gradient can then be seen as an estimate of the advantage of an action \(l_{t}\) in state \(s_{t}\) over the baseline model. This is equivalent to \(R_{tree}-B_{tree}\), where \(R_{tree}\) is the bracketing F-score of a sequence sampled from the current policy and \(B_{tree}\) is the the tree-level F-score of the sequence greedily predicted by the baseline. To further reduce the variance, we standardize the gradient estimate \(\Delta_{\theta}\) using its running mean and standard deviation for all candidates seen in training so far. In initial experiments without these augmentations, we observed that fine-tuning with vanilla pg often led to a deterioration in performance. To encourage exploration away from the converged supervised model's policy, we add the entropy of the policy to the objective function Williams and Peng (1991). Moreover, following Lillicrap et al. (2015), we optionally add noise sampled from a noise process \(N\) to the policy. The gradient of our full fine-tuning objective function takes the following form:

\[\Delta_{\theta}(log\pi(l_{t}|s_{t};\theta)+N)(R_{tree}-B_{tree})\\ +\beta\Delta_{\theta}H(\pi(s_{t};\theta)+N) \tag{2}\]

where \(H\) is the entropy and \(\beta\) controls the strength of the entropy regularization term.

## 4 Experiments

We now review the impact of the proposed techniques on a wide variety of settings.

DatasetsWe use the English Penn Treebank (ptb) Marcus et al. (1993) and the Chinese Penn Treebank (ctb) Xue et al. (2005). For these, we use the same predicted PoS tags as Dyer et al. (2016). We also provide detailed results on the spmrl treebanks Seddah et al. (2014),5 a set of datasets for constituent parsing on morphologically rich languages. For these, we use the predicted PoS tags provided together with the corpora. To the best of our knowledge, we provide the first evaluation on the spmrl datasets for sequence tagging constituent parsers.

Footnote 5: Intel Core i7-7700 CPU 4.2 GHz

MetricsWe report bracketing F-scores, using the evalab and the eval-spmrl scripts. We measure the speed in terms of sentences per second.

SetupWe use ncrfpp Yang and Zhang (2018), for direct comparison against Gomez-Rodriguez and Vilares (2018). We adopt bracketing F-score instead of label accuracy for model selection and report this performance as our second baseline. After 100 epochs, we select the model that fared best on the development set. We use GloVe embeddings Pennington et al. (2014) for our English models and zzigiga embeddings Liu and Zhang (2017) for the Chinese models, for a more homogeneous comparison against other parisers Dyer et al. (2016); Liu and Zhang (2017); Fernandez-Gonzalez and Gomez-Rodriguez (2018). ELMo Peters et al. (2018) or BERT Devlin et al. (2018) could be used to improve the precision, but in this paper we focus on keeping a good speed-accuracy tradeoff. For spmrl, no pretrained embeddings are used, following Kitaev and Klein (2018a). As a side note, if we wanted to improve the performance on these languages we could rely on the CoNLL 2018 shared task pretrained word embeddings Zeman et al. (2018) or even the multilingual bert model6. Our models are run on a single CPU7 (and optionally on a consumer-grade GPU for further comparison) using a batch size of 128 for testing. Additional hyperparameters can be found in Appendix A.

Footnote 7: Intel Core i7-7700 CPU 4.2 GHz

### Results

Table 1 contrasts the performance of our models against the baseline on the ptb development set.

 

To show that the model which employs dynamic encoding is better (+0.52) than the baseline when it comes to closing brackets from long constituents, we compare their F-scores in Figure 6. When we recast the constituent-parsing-as-sequence-tagging problem as multi-task learning, we obtain both a higher bracketing F-score (+0.68) and speed (1.17x faster). Fusing strategies to mitigate issues from greedy decoding also leads to better models (up to +0.87 when adding an auxiliary task8 and up to +1.00 if we also fine-tune with pg). Note that including auxiliary tasks and pg come at a time cost in training, but not in testing, which makes them suitable for fast parsing.

Footnote 8: We observed that adding more than one auxiliary task did not translate into a clear improvement. We therefore chose the auxiliary task that performed the best in the development set.

Table 2 replicates the experiments on the ctb and the spmrl dev sets. The dynamic encodings improves the performance of the baseline on large treebanks, e.g. German, French or Korean, but causes some drops in the smaller ones, e.g. Swedish or Hebrew. Overall, casting the problem as multitask learning and the strategies used to mitigate error propagation lead to improvements.

For the experiments on the test sets we select the models that summarize our contributions: the models with dynamic encoding and the multi-task setup, the models including the best auxiliary task, and the models fine-tuned with policy gradient.

Tables 3, 4 and 5 compare our parsers against the state of the art on the ptb, ctb and spmrl test sets. Gomez-Rodriguez and Vilares (2018) also run experiments without character embeddings, to improve speed without suffering from a big drop in performance. For further comparison, we also include them as additional results (shadowed). In a related line, Smith et al. (2018) show that for dependency parsing two out of three embeddings (word, postag and characters) can suffice.

### Discussion

The results across the board show that the dynamic encoding has a positive effect on 6 out of 10 treebanks. Casting the constituent-parsing-as-sequence-labeling problem as mtl surpasses the baseline for all tested treebanks (and it leads to better parsing speeds too). Finally, by mitigating issues from greedy decoding we further improve the performance of all models that include dynamic encodings and multi-task learning.

On the ptb, our models are both faster and more accurate than existing sequence tagging or sequence-to-sequence models, which already were among the fastest parsers Gomez-Rodriguez and Vilares (2018); Vinyals et al. (2015). We also outperform other approaches that were not surpassed by the original sequence tagging models in terms of F-score Zhu et al. (2013); Fernandez-Gonzalez and Martins (2015). On the ctb our techniques also have a positive effect. The baseline parses 70 sents/s on the cb, while the full model processes up to 120. The speed up is expected to be larger than the one obtained for the ptb because the size of the label set for the baseline is bigger, and it is reduced in a greater proportion when the constituent-parsing-as-sequence-labeling problem is cast as mtl.

On the spmrl corpora, we provide the first evaluation of sequence labeling constituent parsers, to verify if these perform well on mor

\begin{table}
\begin{tabular}{p{99.6pt} p{42.7pt} p{42.7pt} p{42.7pt}} \hline
* & 109 \\ Our baseline & 89.77 & (+0.07) & 111 \\  + DE & 90.22 & (+0.52) & 111 \\  + mtl & 90.38 & (+0.68) & 130 \\ \hline aux(\(n_{t+1}\)) & 90.41 & (+0.71) & 130 \\ aux(\(n_{t-1}\)) & 90.57 & (+0.87) & 130 \\ aux(distances) & 90.55 & (+0.85) & 130 \\ \hline  + PG & **90.70** & (+1.00) & 130 \\ \hline \end{tabular}
\end{table}
Table 1: Results on the ptb dev set, compared against Gómez-Rodríguezuez and Vilares (2018). de refers to dynamic encoding and mtl to a model that additionally casts the problem as multi-task learning. Each auxiliary task is added separately to the baseline with de and mtl. Policy gradient fine-tunues the model that includes the best auxiliary task.

Figure 6: F-score for \(n_{t}\)s on the ptb dev set, obtained by the Gómez-Rodríguezuez and Vilares (2018) baseline (in blue, first bar for each \(n_{t}\), already shown in Figure 3) and our model with dynamically encoded trees (in orange, second bar).

 

phologically rich languages. We then evaluated whether the proposed techniques can generalize on heterogeneous settings. The tendency observed for the original tagging models by Gomez-Rodriguez and Vilares (2018) is similar to the one for the ptb and ctb: they improve other fast parsers, e.g. Coavoux and Crabbe (2016), in 5 out of 8 treebanks and Fernandez-Gonzalez and Martins (2015) in 7 out of 8, but their performance is below more powerful models. When incorporating the techniques presented in this work, we outperform the original sequence tagging models on all datasets. We outperform the current best model for Basque, Hebrew and Polish Kitaev and Klein (2018a) and for Swedish Bjorkelund et al. (2014), which corresponds to the four smallest treebanks among the spmrl datasets. This indicates that even if sequence tagging models are conceptually simple and fast, they can be very suitable when little training data is available. This is also of special interest in terms of research for low-resource languages. Again, casting the problem as mtl reduces the parsing time for all tested treebanks, as reflected in Table 6. Finally, for treebanks such as French, designing methods to handle multi-word expressions could lead to better results, getting closer to other parsers Coavoux and Crabbe (2017).

## 5 Conclusion

We have explored faster and more precise sequence tagging models for constituent parsing. We proposed a multitask-learning architecture that employs dynamic encodings, auxiliary tasks, and policy gradient fine-tuning. We performed experiments on the English and Chinese Penn Treebanks, and also on the spmrl datasets. Our models improve current sequence tagging parsers on all treebanks, both in terms of performance and speed. We also report state-of-the-art results for the Basque, Hebrew, Polish, and Swedish datasets. The methods presented in this work are specifically designed for constituent parsing. However, it seems natural to apply some of these to other nlp tagging tasks, e.g. using multi-task learning to predict sub-level morphological information for morphologically-rich part-of-speech tagging.

\begin{table}
\begin{tabular}{p{93.6pt} p{113.8pt}} \hline
**Model** & **F-score** \\ \hline Zhu et al. (2013) & 83.2 \\ Dyer et al. (2016) & 84.6 \\ Liu and Zhang (2017) & 86.1 \\ Shen et al. (2018) & 86.5 \\ Fernández and Gómez-Rodríguezuez (2018) & 86.8 \\ Gómez and Vilares (2018) & 84.1 \\ \hline Our baseline & 83.90 \\ +de & 83.98 \\ +mtl & 84.24 \\ +best aux & 85.01 \\ +pg & 85.61 \\ +PG (no char emb) & 83.93 \\ \hline \end{tabular}
\end{table}
Table 4: Comparison on the ctb test set

\begin{table}
\begin{tabular}{l|c c c c c c c c c} \hline
* & & & & & \\ Kitaev and Klein (2018b) & 115 & 1 CPU & 90.00 \\ Our baseline & 115 & 1 CPU & 90.06 \\  +dE & 115 & 1 CPU & 90.19 \\  +mtl & 132 & 1 CPU & 90.36 \\  + best aux & 132 & 1 CPU & 90.59 \\  +PG & 132 & 1 CPU & 90.60 \\  +PG & 942 & 1 GPU & 90.60 \\  +PG (no char emb) & 149 & 1 CPU & 90.50 \\  +PG (no char emb) & 1267 & 1 GPU & 90.50 \\ \hline \end{tabular}
\end{table}
Table 2: Results on the c tb and spmrl dev sets

\begin{table}
\begin{tabular}{l l} \hline
**Model** & **F-score** \\ \hline Zhu et al. (2013) & 83.2 \\ Dyer et al. (2016) & 84.6 \\ Liu and Zhang (2017) & 86.1 \\ Shen et al. (2018) & 86.5 \\ Fernandez and Gómez-Rodríguezuez (2018) & 86.8 \\ Gómez and Vilares (2018) & 84.1 \\ \hline Our baseline & 83.90 \\ +de & 83.98 \\ +mtl & 84.24 \\ +best aux & 85.01 \\ +pg & 85.61 \\ +pg (no char emb) & 83.93 \\ \hline \end{tabular}
\end{table}
Table 3: Comparison on the ptb test set. Kitaev and Klein (2018b) are results published after this work was submitted (italics represent the cases where they obtain a new state of the art on the corresponding language).

 



## Acknowledgments

DV has received support from the European Research Council (ERC), under the European Union's Horizon 2020 research and innovation programme (FASTPARSE, grant agreement No 714150), from the TELEPARES-UDC project (FFI2014-51978-C2-2-R) and the ANSWERSAP project (TIN2017-85160-C2-1-R) from MINECO, and from Xunta de Galicia (ED431B 2017/01). MA and AS are funded by a Google Focused Research Award.

## References

* Baxter (1997) Jonathan Baxter. 1997. A bayesian/information theoretic model of learning to learn via multiple task sampling. _Machine learning_, 28(1):7-39.
* Baxter (2000) Jonathan Baxter. 2000. A model of inductive bias learning. _Journal of Artificial Intelligence Research_, 12:149-198.
* Bingel and Sogaard (2017) Joachim Bingel and Anders Sogaard. 2017. Identifying beneficial task relations for multi-task learning in deep neural networks. In _Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers_, pages 164-169. Association for Computational Linguistics.
* Bjorkelund et al. (2014) Anders Bjorkelund, Ozlem Cetinoglu, Agnieszka Falenska, Richard Farkas, Thomas Muller, Wolfgang Seeker, and Zsolt Szanto. 2014. Introducing the ims-wroclaw-szeged-cis entry at the spmrl 2014 shared task: Reranking and morpho-syntax meet unlabeled data. In _Proceedings of the First Joint Workshop on Statistical Parsing of Morphologically Rich Languages and Syntactic Analysis of Non-Canonical Languages_, pages 97-102.
* Caruana (1997) Rich Caruana. 1997. Multitask learning. _Machine learning_, 28(1):41-75.
* Charniak (2000) Eugene Charniak. 2000. A maximum-entropy-inspired parser. In _Proceedings of the 1st North American chapter of the Association for Computational Linguistics conference_, pages 132-139. Association for Computational Linguistics.
* Chen and Manning (2014) Danqi Chen and Christopher Manning. 2014. A fast and accurate dependency parser using neural networks. In _Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)_, pages 740-750.
* Cho et al. (2014) Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the Properties of Neural Machine Translation: Encoder-Decoder Approaches. _Syntax, Semantics and Structure in Statistical Translation_, page 103.
* Coavoux and Crabbe (2016) Maximin Coavoux and Benoit Crabbe. 2016. Neural greedy constituent parsing with dynamic oracles. In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 172-182. Association for Computational Linguistics.
* Coavoux and Crabbe (2017) Maximin Coavoux and Benoit Crabbe. 2017. Multilingual lexicalized constituency parsing with word-level auxiliary tasks. In _Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers_, volume 2, pages 331-336.
* Collins (1997) Michael Collins. 1997. Three generative, lexicalised models for statistical parsing. In _Proceedings of the

\begin{table}
\begin{tabular}{p{56.9pt}|c c c} \hline
**Dataset** & **Baseline speed** & **Full speed(increase)** & Full speed(increase) \\ \hline Basque & 179 & 223\({}_{(1.25k)}\) & 257\({}_{(1.44k)}\) \\ French & 76 & 91\({}_{(1.20k)}\) & 104\({}_{(1.37k)}\) \\ German & 70 & 100\({}_{(1.43k)}\) & 108\({}_{(1.54k)}\) \\ Hebrew & 44 & 102\({}_{(2.32k)}\) & 115\({}_{(2.61k)}\) \\ Hungarian & 93 & 134\({}_{(1.44k)}\) & 150\({}_{(1.61k)}\) \\ Korean & 197 & 213\({}_{(1.08k)}\) & 230\({}_{(1.17k)}\) \\ Polish & 187 & 253\({}_{(1.35k)}\) & 278\({}_{(1.49k)}\) \\ Swedish & 98 & 158\({}_{(1.61k)}\) & 187\({}_{(1.81k)}\) \\ \hline \end{tabular}
\end{table}
Table 6: Comparison of speeds on the spmrl datasets

\begin{table}
\begin{tabular}{p{56.9pt}|c c c c c c c c} \hline
**Model** & **Basque French** & **German Hebrew** & **Hungarian** & **Korean** & **Polish** & **Swedish** & **Avg** \\ \hline Fernandez-Gonzalez and Martins (2015) & 85.90 & 78.75 & 78.66 & 88.97 & 88.16 & 79.28 & 91.20 & 82.80 & 84.21 \\ Cavouva and Crabbe (2016) & 86.24 & 79.91 & 80.15 & 86.69 & 90.51 & 85.10 & 92.96 & 81.74 & 85.67 \\ Björkelund et al. (2014) (ensemble) & 88.24 & 82.53 & 81.66 & 89.80 & 91.72 & 83.81 & 90.50 & 85.50 & 86.72 \\ Coavoux and Crabbe (2017) & 88.81 & 82.49 & 85.34 & 89.87 & 92.34 & 86.04 & 93.64 & 84.00 & 87.82 \\ Kitaev and Klein (2018a) & 89.71 & **84.06** & **87.69** & 90.35 & **92.69** & **86.59** & 93.69 & 84.35 & **88.64** \\ Kitaev and Klein (2018b) (with bert) & _91.63_ & _87.42_ & _90.20_ & _92.99_ & _94.90_ & _88.80_ & _96.36_ & _88.86_ & _91.40_ \\ \hline Baseline & 89.54 & 80.56 & 84.05 & 88.83 & 90.42 & 83.33 & 92.48 & 83.67 & 86.61 \\ +DE & 89.56 & 80.69 & 84.64 & 88.80 & 90.02 & 83.67 & 93.20 & 83.40 & 86.75 \\ +mtL & 90.90 & 80.98 & 84.94 & 91.99 & 90.63 & 83.91 & 93.80 & 86.26 & 87.92 \\ +best aux & **91.23** & 81.27 & 84.95 & **92.03** & 90.60 & 83.67 & 93.84 & 86.62 & 88.02 \\ +pg & 91.18 & 81.37 & 84.88 & **92.03** & 90.65 & 84.01 & **93.93** & **86.71** & 88.10 \\ +PG (no char emb) & 90.14 & 81.38 & 85.09 & 92.11 & 90.34 & 83.46 & 93.87 & 86.65 & 87.58 \\ \hline \end{tabular}
\end{table}
Table 5: Comparison on the test spmrl datasets (except Arabic). Kitaev and Klein (2018b) are results published after this work was submitted (italics represent the cases where they obtain a new state of the art on the corresponding language).



eighth conference on European chapter of the Association for Computational Linguistics_, pages 16-23. Association for Computational Linguistics.
* Collobert and Weston (2008) Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In _Proceedings of the 25th international conference on Machine learning_, pages 160-167. ACM.
* Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. _arXiv preprint arXiv:1810.04805_.
* Dong et al. (2015) Daxiang Dong, Hua Wu, Wei He, Dianhai Yu, and Haifeng Wang. 2015. Multi-task learning for multiple language translation. In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, volume 1, pages 1723-1732.
* Duong et al. (2015) Long Duong, Trevor Cohn, Steven Bird, and Paul Cook. 2015. Low resource dependency parsing: Cross-lingual parameter sharing in a neural network parser. In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_, volume 2, pages 845-850.
* Dyer et al. (2016) Chris Dyer, Adhiguna Kuncoro, Miguel Ballesteros, and Noah A. Smith. 2016. Recurrent neural network grammars. In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 199-209. Association for Computational Linguistics.
* Fernandez-Gonzalez and Gomez-Rodriguez (2018) Daniel Fernandez-Gonzalez and Carlos Gomez-Rodriguez. 2018. Faster Shift-Reduce Constituent Parsing with a Non-Binary, Bottom-Up Strategy. _ArXiv e-prints_.
* Fernandez-Gonzalez and Martins (2015) Daniel Fernandez-Gonzalez and Andre F. T. Martins. 2015. Parsing as reduction. In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, pages 1523-1533. Association for Computational Linguistics.
* Fried and Klein (2018) Daniel Fried and Dan Klein. 2018. Policy gradient as a proxy for dynamic oracles in constituency parsing. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_, pages 469-476. Association for Computational Linguistics.
* Gomez-Rodriguez and Vilares (2018) Carlos Gomez-Rodriguez and David Vilares. 2018. Constituent parsing as sequence labeling. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_, pages 1314-1324. Association for Computational Linguistics.
* Hall et al. (2014) David Hall, Greg Durrett, and Dan Klein. 2014. Less grammar, more features. In _Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 228-237. Association for Computational Linguistics.
* Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jurgen Schmidhuber. 1997. Long short-term memory. _Neural computation_, 9(8):1735-1780.
* Kitaev and Klein (2018a) Nikita Kitaev and Dan Klein. 2018a. Consistency parsing with a self-attentive encoder. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 2676-2686. Association for Computational Linguistics.
* Kitaev and Klein (2018b) Nikita Kitaev and Dan Klein. 2018b. Multilingual constituency parsing with self-attention and pre-training. _CoRR_, abs/1812.11760.
* Kummerfeld et al. (2012) Jonathan K. Kummerfeld, David Hall, James R. Curran, and Dan Klein. 2012. Parser showdown at the Wall Street corral: An empirical investigation of error types in parser output. In _Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning_, pages 1048-1059, Jeju Island, Korea. Association for Computational Linguistics.
* Lafferty et al. (2001) John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In _Proceedings of the Eighteenth International Conference on Machine Learning_, ICML '01, pages 282-289, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc.
* Le and Fokkens (2017) Minh Le and Antske Fokkens. 2017. Tackling error propagation through reinforcement learning: A case of greedy dependency parsing. In _Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers_, pages 677-687. Association for Computational Linguistics.
* Hillicrap et al. (2015) Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. 2015. Continuous control with deep reinforcement learning. _arXiv preprint arXiv:1509.02971_.
* Liu and Zhang (2017) Jiangming Liu and Yue Zhang. 2017. In-order transition-based constituent parsing. _Transactions of the Association for Computational Linguistics_, 5:413-424.
* Marcus et al. (1993) Mitchell P. Marcus, Mary Ann Marcinkiewicz, and Beatric Santorini. 1993. Building a large annotated corpus of english: The penn treebank. _Comput. Linquist._, 19(2):313-330 

Ryan McDonald, Fernando Pereira, Kiril Ribarov, and Jan Hajic. 2005. Non-projective dependency parsing using spanning tree algorithms. In _Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing_, HLT '05, pages 523-530, Stroudsburg, PA, USA. Association for Computational Linguistics.
* Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 1532-1543. Association for Computational Linguistics.
* Peters et al. (2018) Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_, volume 1, pages 2227-2237.
* Petrov and Klein (2007) Slav Petrov and Dan Klein. 2007. Improved inference for unlexicalized parsing. In _Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference_, pages 404-411. Association for Computational Linguistics.
* Plank et al. (2016) Barbara Plank, Anders Sogaard, and Yoav Goldberg. 2016. Multilingual part-of-speech tagging with bidirectional long short-term memory models and auxiliary loss. In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_, pages 412-418. Association for Computational Linguistics.
* Qi and Manning (2017) Peng Qi and Christopher D. Manning. 2017. Arc-swift: A novel transition system for dependency parsing. In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_, pages 110-117. Association for Computational Linguistics.
* Rei (2017) Marek Rei. 2017. Semi-supervised multitask learning for sequence labeling. In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 2121-2130. Association for Computational Linguistics.
* Reimers and Gurevych (2017) Nils Reimers and Iryna Gurevych. 2017. Reporting Score Distributions Makes a Difference: Performance Study of LSTM-networks for Sequence Tagging. In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 338-348, Copenhagen, Denmark.
* Ruder et al. (2019) Sebastian Ruder, Joachim Bingel, Isabelle Augenstein, and Anders Sogaard. 2019. Latent multi-task architecture learning. _To appear at AAAI 2019_.
* Sagae and Lavie (2006) Kenji Sagae and Alon Lavie. 2006. A best-first probabilistic shift-reduce parser. In _Proceedings of the COLING/ACL on Main Conference Poster Seessions_, COLING-ACL '06, pages 691-698, Stroudsburg, PA, USA. Association for Computational Linguistics.
* Sanh et al. (2018) Victor Sanh, Thomas Wolf, and Sebastian Ruder. 2018. A hierarchical multi-task approach for learning embeddings from semantic tasks. _arXiv preprint arXiv:1811.06031_.
* Schuster and Paliwal (1997) M. Schuster and K.K. Paliwal. 1997. Bidirectional recurrent neural networks. _Trans. Sig. Proc._, 45(11):2673-2681.
* Seddah et al. (2014) Djame Seddah, Sandra Kubler, and Reut Tsarfaty. 2014. Introducing the spmrl 2014 shared task on parsing morphologically-rich languages. In _Proceedings of the First Joint Workshop on Statistical Parsing of Morphologically Rich Languages and Syntactic Analysis of Non-Canonical Languages_, pages 103-109. Dublin City University.
* Shen et al. (2018) Yikang Shen, Zhouhan Lin, Athul Paul Jacob, Alessandro Sordoni, Aaron Courville, and Yoshua Bengio. 2018. Straight to the tree: Constituency parsing with neural syntactic distance. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 1171-1180. Association for Computational Linguistics.
* Smith et al. (2018) Aaron Smith, Miryam de Lhomeux, Sara Stymne, and Joahim Nivre. 2018. An investigation of the interactions between pre-trained word embeddings, character models and pos tags in dependency parsing. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_, pages 2711-2720. Association for Computational Linguistics.
* Sogaard and Goldberg (2016) Anders Sogaard and Yoav Goldberg. 2016. Deep multi-task learning with low level tasks supervised at lower layers. In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_, pages 231-235. Association for Computational Linguistics.
* Stern et al. (2017) Mitchell Stern, Jacob Andreas, and Dan Klein. 2017. A minimal span-based neural constituency parser. In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 818-827, Vancouver, Canada. Association for Computational Linguistics.
* Sutton and Barto (2018) Richard S Sutton and Andrew G Barto. 2018. _Reinforcement learning: An introduction_. MIT press.
* Vinyals et al. (2015) Oriol Vinyals, Lukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoffrey Hinton. 2015. Grammar as a foreign language. In _Advances in Neural Information Processing Systems 28_, pages 2773-2781. Curran Associates, Inc.

 

Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. _Machine learning_, 8(3-4):229-256.
* Williams and Peng [1991] Ronald J Williams and Jing Peng. 1991. Function optimization using connectionist reinforcement learning algorithms. _Connection Science_, 3(3):241-268.
* Xue et al. [2005] Naiwen Xue, Fei Xia, Fu-dong Chiou, and Marta Palmer. 2005. The penn chinese treebank: Phrase structure annotation of a large corpus. _Nat. Lang. Eng._, 11(2):207-238.
* Yang and Zhang [2018] Jie Yang and Yue Zhang. 2018. NCRF++: An open-source neural sequence labeling toolkit. In _Proceedings of ACL 2018, System Demonstrations_, pages 74-79, Melbourne, Australia. Association for Computational Linguistics.
* Zeman et al. [2018] Daniel Zeman, Jan Haji, Martin Popel, Martin Potthast, Milan Straka, Filip Ginter, Joakim Nivre, and Slav Petrov. 2018. Conll 2018 shared task: Multilingual parsing from raw text to universal dependencies. _Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies_, pages 1-21.
* Zhang and Clark [2009] Yue Zhang and Stephen Clark. 2009. Transition-based parsing of the chinese treebank using a global discriminative model. In _Proceedings of the 11th International Conference on Parsing Technologies (IWPT'09)_, pages 162-171. Association for Computational Linguistics.
* Zhu et al. [2013] Muhua Zhu, Yue Zhang, Wenliang Chen, Min Zhang, and Jingbo Zhu. 2013. Fast and accurate shift-reduce constituent parsing. In _Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 434-443. Association for Computational Linguistics.

## Appendix A Appendices

For the bilstm-based model, we essentially follow the configuration of the baseline Gomez-Rodriguez and Vilares (2018) for an homogenous comparison. We detail the hyperparameters in Table 7.9