SlowGuess/NAACL_2021_OCR huggingface dataset:
For baseline models we use the best configurations reported by the authors. For example, we use 30 nonterminals and 60 preterminals for N-PCFGs and C-PCFGs. We implement TN-PCFGs and reimplement N-PCFGs and C-PCFGs using automatic differentiation (Eisner, 2016) and we borrow the idea of Zhang et al. (2020) to batchify the inside algorithm. Inspired by Kim et al. (2019a), for TN-PCFGs we set  $n / p$ , the ratio of the nonterminal number to the preterminal number, to  $1 / 2$ . For  $\mathbf{U} \in \mathbb{R}^{n \times d}$  and  $\mathbf{V}, \mathbf{W} \in \mathbb{R}^{m \times d}$  we set  $d = p$  when there are more than 200 preterminals and  $d = 200$  otherwise. The symbol embedding dimension  $k$  is set to 256. We optimize TN-PCFGs using the Adam optimizer (Kingma and Ba, 2015) with  $\beta_{1} = 0.75$ ,  $\beta_{2} = 0.999$ , and learning rate 0.001 with batch size 4. We use the unit Gaussian distribution to initialize embedding parameters. We do not use the curriculum learning strategy that is used by Kim et al. (2019a) when training TN-PCFGs.
Following Kim et al. (2019a), we train a TN-PCFG for each treebank separately. For each setting we run the TN-PCFG and the baselines four times with different random seeds and for ten epochs each
3The CYK algorithm is similar to the inside algorithm. The only difference is that it uses MAX whenever the inside algorithm performs SUM over  $k$  and  $B, C$  (cf. Equation 1).
4 In Equation 8 all symbols become entangled through  $\mathbf{V}^T\mathbf{s}_{i,k}$  and  $\mathbf{W}^T\mathbf{s}_{k + 1,j}$ . We are unable to perform MAX over  $B, C$  as in the CYK algorithm.
$^{5}$ https://github.com/zhaoyanpeng/xcfg.
Model	WSJ
Mean	Max
Left Branching	8.7
Right Branching	39.5
Random Trees	18.1	18.2
Systems without pretrained word embeddings
PRPN†(Shen et al., 2018a)	47.3	47.9
ON†(Shen et al., 2019)	48.1	50.0
N-PCFG (Kim et al., 2019a)	50.8	52.6
C-PCFG (Kim et al., 2019a)	55.2	60.1
NL-PCFG (Zhu et al., 2020)	55.3
N-PCFG*	50.9±2.3	54.6
N-PCFG* w/ MBR	52.3±2.3	55.8
C-PCFG*	55.4±2.2	59.0
C-PCFG* w/ MBR	56.3±2.1	60.0
TN-PCFG p = 60 (ours)	51.4±4.0	55.6
TN-PCFG p = 500 (ours)	57.7±4.2	61.4
Systems with pretrained word embeddings
DIORA (Drozdov et al., 2019)	56.8
S-DIORA (Drozdov et al., 2020)	57.6	64.0
CT (Cao et al., 2020)	62.8	65.9
Oracle Trees	84.3
time. Early stopping is performed based on the perplexity on the development data. The best model in each run is selected according to the perplexity on the development data. We tune model hyperparameters only on the development data of WSJ and use the same model configurations on the other treebanks. We report average sentence-level F1 score as well as their biased standard deviations.
8 Experimental results
We evaluate our models mainly on WSJ (Section 8.1-8.3). We first give an overview of model
Table 1: Unlabeled sentence-level F1 scores on the WSJ test data.  $\dagger$  indicates numbers reported by Kim et al. (2019a).  $\star$  indicates our implementations of N-PCFGs and C-PCFGs.  $p$  denotes the preterminal number.
Model	Time (minutes)	Total Parameters (M)
N-PCFG	19	5.3
C-PCFG	20	15.4
TN-PCFG (p=60)	13	3.6
TN-PCFG (p=500)	26	4.2
Table 2: Average running time per epoch and the parameter number of each model.
performance in Section 8.1 and then conduct ablation study of TN-PCFGs in Section 8.2. We quantitatively and qualitatively analyze constituent labels induced by TN-PCFGs in Section 8.3. In Section 8.4, we conduct a multilingual evaluation over nine additional languages.
8.1 Main results
Our best TN-PCFG model uses 500 preterminals  $(p = 500)$ . We compare it with a wide range of recent unsupervised parsing models (see the top section of Table 1). Since we use MBR decoding for TN-PCFGs, which produces higher F1-measure than the CYK decoding (Goodman, 1996), for fair comparison we also use MBR decoding for our reimplemented N-PCFGs and C-PCFGs (see the middle section of Table 1).
We draw three key observations from Table 1: (1) TN-PCFG  $(p = 500)$  achieves the best mean and max F1 score. Notably, it outperforms the strong baseline model C-PCFG by  $1.4\%$  mean F1. Compared with TN-PCFG  $(p = 60)$ , TN-PCFG  $(p = 500)$  brings a  $6.3\%$  mean F1 improvement, demonstrating the effectiveness of using more symbols. (2) Our implementations of N-PCFGs and C-PCFGs are comparable to those of Kim et al. (2019a), (3) MBR decoding indeed gives higher F1 scores  $(+1.4\%$  mean F1 for N-PCFG and  $+0.9\%$  mean F1 for C-PCFG).
In Table 1 we also show the results of Constituent test (CT) (Cao et al., 2020) and DIORA (Drozdov et al., 2019, 2020), two recent state-of-the-art approaches. However, our work is not directly comparable to these approaches. CT relies on pretrained language models (RoBERTa) and DIORA relies on pretrained word embeddings (context insensitive ELMo). In contrast, our model and the other approaches do not use pretrained word embeddings and instead learn word embeddings from scratch. We are also aware of URNNG (Kim et al., 2019b), which has a max F1 score of  $45.4\%$ , but it uses punctuation and hence is not directly comparable to the models listed in the table.
<sup>6</sup>Shi et al. (2020) suggest not using the gold parses of the development data for hyperparameter tuning and model selection in unsupervised parsing. Here we still use the gold parses of the WSJ development set for the English experiments in order to conduct fair comparison with previous work. No gold parse is used in the experiments of any other language.
Following Kim et al. (2019a), we remove all trivial spans (single-word spans and sentence-level spans). Sentence-level means that we compute F1 for each sentence and then average over all sentences.
Figure 1: F1 scores and perplexities w.r.t. the preterminal number  $p$  of TN-PCFGs on the WSJ test data. Recall that the nonterminal number  $n$  is set to half of  $p$ .
We report the average running time $^{8}$  per epoch and the parameter numbers of different models in Table 2. We can see that TN-PCFG ( $p = 500$ ), which uses a much larger number of symbols, has even fewer parameters and is not significantly slower than N-PCFG.
8.2 Influence of symbol number
Figure 1 illustrates the change of F1 scores and perplexities as the number of nonterminals and preterminals increase. We can see that, as the symbol number increases, the perplexities decrease while F1 scores tend to increase.
8.3 Analysis on constituent labels
We analyze model performance by breaking down recall numbers by constituent labels (Table 3). We use the top six frequent constituent labels in the WSJ test data (NP, VP, PP, SBAR, ADJP, and ADVP). We first observe that the right-branching baseline remains competitive. It achieves the highest recall on VPs and SBARs. TN-PCFG ( $p = 500$ ) displays a relatively even performance across the six labels. Specifically, it performs best on NPs and PPs among all the labels and it beats all the other models on ADJPs. Compared with TN-PCFG ( $p = 60$ ), TN-PCFG ( $p = 500$ ) results in the largest improvement on VPs (+19.5% recall), which are usually long (with an average length of 11) in comparison with the other types of constituents. As NPs and VPs cover about 54% of the total constituents in the WSJ test data, it is not surprising that models which are accurate on these labels have high F1 scores (e.g., C-PCFGs and TN-PCFGs ( $p = 500$ )).
We further analyze the correspondence between the nonterminals of trained models and gold constituent labels. For each model, we look at all the correctly-predicted constituents in the test set and estimate the empirical posterior distribution of nonterminals assigned to a constituent given the gold label of the constituent (see Figure 2). Compared with the other three models, in TN-PCFG  $(p = 500)$ , the most frequent nonterminals are more likely to correspond to a single gold label. One possible explanation is that it contains much more nonterminals and therefore constituents of different labels are less likely to compete for the same nonterminal.
Figure 2d (TN-PCFG  $(p = 500)$ ) also illustrates that a gold label may correspond to multiple nonterminals. A natural question that follows is: do these nonterminals capture different subtypes of the gold label? We find it is indeed the case for some nonterminals. Take the gold label NPs (noun phrases), while not all the nonterminals have clear interpretation, we find that NT-3 corresponds to constituents which represent a company name; NT99 corresponds to constituents which contain a possessive affix (e.g., "s" in "the market's decline"); NT-94 represents constituents preceded by an indefinite article. We further look into the gold label PPs (preposition phrases). Interestingly, NT-108, NT-175, and NT-218 roughly divided preposition phrases into three groups starting with 'with, by, from, to', 'in, on, for', and 'of', respectively. See Appendix for more examples.
8.4 Multilingual evaluation
In order to understand the generalizability of TD-PCFGs on languages beyond English, we conduct a multilingual evaluation of TD-PCFGs on CTB and SPMRL. We use the best model configurations obtained on the English development data and do not perform any further tuning on CTB and SPMRL. We compare TN-PCFGs with N-PCFGs and C-PCFGs and use MBR decoding by default. The results are shown in Table 4. In terms of the average F1 over the nine languages, all the three models beat trivial left- and right-branching baselines by a large margin, which suggests they have good generalizability on languages beyond English. Among the three models, TN-PCFG  $(p = 500)$  fares best. It achieves the highest F1 score on six out of nine treebanks. On Swedish, N-PCFG is worse than the right-branching baseline  $(-13.4\%)$  F1), while
<sup>8</sup>We measure the running time on a single Titan V GPU.
Model	NP	VP	PP	SBAR	ADJP	ADVP	S-F1
Left Branching	10.4	0.5	5.0	5.3	2.5	8.0	8.7
Right Branching	24.1	71.5	42.4	68.7	27.7	38.1	39.5
Random Trees	22.5±0.3	12.3±0.3	19.0±0.5	9.3±0.6	24.3±1.7	26.9±1.3	18.1±0.1
NPCFG†	71.2	33.8	58.8	52.5	32.5	45.5	50.8
C-NPCFG†	74.7	41.7	68.8	56.1	40.4	52.5	55.2
N-PCFG* w/ MBR	72.3±3.6	28.1 ±6.6	73.0 ±2.6	53.6±10.0	40.8±4.2	43.8 ±8.3	52.3±2.3
C-PCFG* w/ MBR	73.6±2.5	45.0 ±6.0	71.4 ±1.4	54.8 ±5.6	44.3±5.9	61.6±17.6	56.3±2.1
TN-PCFG p = 60	77.2±2.6	28.9±13.8	58.9±17.5	44.0±17.6	47.5±5.1	54.9 ±6.0	51.4±4.0
TN-PCFG p = 500	75.4±4.9	48.4±10.7	67.0±11.7	50.3±13.3	53.6±3.3	59.5 ±2.6	57.7±4.2
(a) N-PCFG
(b) C-PCFG
(c) TN-PCFG  $(p = 60)$
(d) TN-PCFG  $(p = 500)$
Figure 2: Correspondence between nonterminals and gold constituent labels. For each gold label, we visualize the proportion of correctly-predicted constituents that correspond to each nonterminal. Nonterminals NT-# are listed in descending order of the prediction frequency (e.g., NT-21 is the most predicted nonterminal in TN-PCFG  $(p = 500)$ . Only the top 30 frequent nonterminals are listed. We show the seven most frequent gold labels and aggregate the rest (denoted by OTHER).
Table 3: Recall on the six frequent constituent labels in the WSJ test data, † denotes results reported by Kim et al. (2019a), S-F1 represents sentence-level F1 measure.
Model	Chinese	Basque	German	French	Hebrew	Hungarian	Korean	Polish	Swedish	Mean
Left Branching†	7.2	17.9	10.0	5.7	8.5	13.3	18.5	10.9	8.4	11.2
Right Branching†	25.5	15.4	14.7	26.4	30.0	12.7	19.2	34.2	30.4	23.2
Random Trees†	15.2	19.5	13.9	16.2	19.7	14.1	22.2	21.4	16.4	17.6
N-PCFG w/ MBR	26.3±2.5	35.1±2.0	42.3±1.6	45.0±2.0	45.7±2.2	43.5±1.2	28.4±6.5	43.2±0.8	17.0 ±9.9	36.3
C-PCFG w/ MBR	38.7±6.6	36.0±1.2	43.5±1.2	45.0±1.1	45.2±0.5	44.9±1.5	30.5±4.2	43.8±1.3	33.0±15.4	40.1
TN-PCFG p = 500	39.2±5.0	36.0±3.0	47.1±1.7	39.1±4.1	39.2±10.7	43.1±1.1	35.4±2.8	48.6±3.1	40.0 ±4.8	40.9
Table 4: Sentence-level F1 scores on CTB and SPMRL.  ${}^{ \dagger  }$  denotes results reported by Zhao and Titov (2021).
TN-PCFG  $(p = 500)$  surpasses the right-branching baseline by  $9.6\%$  F1.
In our experiments, we do not find it beneficial to use the compound trick (Kim et al., 2019a) in TN-PCFGs, which is commonly used in previous work of PCFG induction (Kim et al., 2019a; Zhao
and Titov, 2020; Zhu et al., 2020). We speculate that the additional expressiveness brought by compound parameterization may not be necessary for a TN-PCFG with many symbols which is already sufficiently expressive; on the other hand, compound parameterization makes learning harder when we use more symbols.
We also find neural parameterization and the choice of nonlinear activation functions greatly influence the performance. Without using neural parameterization, TD-PCFGs have only around  $30\%$  S-F1 scores on WSJ, which are even worse than the right-branching baseline. Activation functions other than ReLU (such as tanh and sigmoid) result in much worse performance. It is an interesting open question why ReLU and neural parameterization are crucial in PCFG induction.
When evaluating our model with a large number of symbols, we find that only a small fraction of the symbols are predicted in the parse trees (for example, when our model uses 250 nonterminals, only tens of them are found in the predicted parse trees of the test corpus). We expect that our models can benefit from regularization techniques such as state dropout (Chiu and Rush, 2020).
We have presented TD-PCFGs, a new parameterization form of PCFGs based on tensor decomposition. TD-PCFGs rely on Kruskal decomposition of the binary-rule probability tensor to reduce the computational complexity of PCFG representation and parsing from cubic to at most quadratic in the symbol number, which allows us to scale up TD-PCFGs to a much larger number of (nonterminal and preterminal) symbols. We further propose neurally parameterized TD-PCFGs (TN-PCFGs) and learn neural networks to produce the parameters of TD-PCFGs. On WSJ test data, TN-PCFGs outperform strong baseline models; we empirically show that using more nonterminal and preterminal symbols contributes to the high unsupervised parsing performance of TN-PCFGs. Our multilingual evaluation on nine additional languages further reveals the capability of TN-PCFGs to generalize to languages beyond English.
Sanjeev Arora, Nadav Cohen, and Elad Hazan. 2018. On the optimization of deep networks: Implicit acceleration by overparameterization. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pages 244-253. PMLR.
Rares-Darius Buhai, Yoni Halpern, Yoon Kim, Andrej Risteski, and David Sontag. 2019. Empirical study of the benefits of overparameterization in learning latent variable models. arXiv preprint arXiv:1907.00030.
Steven Cao, Nikita Kitaev, and Dan Klein. 2020. Unsupervised parsing via constituency tests. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4798-4808, Online. Association for Computational Linguistics.
Glenn Carroll and Eugene Charniak. 1992. Two experiments on learning probabilistic dependency grammars from corpora. Department of Computer Science, Univ.
Justin Chiu and Alexander Rush. 2020. Scaling hidden Markov language models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1341-1349, Online. Association for Computational Linguistics.
Shay B. Cohen, Giorgio Satta, and Michael Collins. 2013. Approximate PCFG parsing using tensor decomposition. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 487-496, Atlanta, Georgia. Association for Computational Linguistics.
Shay B. Cohen, Karl Stratos, Michael Collins, Dean P. Foster, and Lyle Ungar. 2012. Spectral learning of latent-variable PCFGs. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 223-231, Jeju Island, Korea. Association for Computational Linguistics.
Andrew Drozdov, Subendhu Rongali, Yi-Pei Chen, Tim O'Gorman, Mohit Iyyer, and Andrew McCallum. 2020. Unsupervised parsing with S-DIORA: Single tree encoding for deep inside-outside recursive autoencoders. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4832-4845, Online. Association for Computational Linguistics.
Andrew Drozdov, Patrick Verga, Mohit Yadav, Mohit Iyyer, and Andrew McCallum. 2019. Unsupervised latent tree induction with deep inside-outside recursive auto-encoders. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lan
guage Technologies, Volume 1 (Long and Short Papers), pages 1129-1141, Minneapolis, Minnesota. Association for Computational Linguistics.
Simon S. Du, Xiyu Zhai, Barnabás Póczos, and Aarti Singh. 2019. Gradient descent provably optimizes over-parameterized neural networks. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net.
Jason Eisner. 2016. Inside-outside and forward-backward algorithms are just backprop (tutorial paper). In Proceedings of the Workshop on Structured Prediction for NLP, pages 1-17, Austin, TX. Association for Computational Linguistics.
Joshua Goodman. 1996. Parsing algorithms and metrics. In 34th Annual Meeting of the Association for Computational Linguistics, pages 177-183, Santa Cruz, California, USA. Association for Computational Linguistics.
Junxian He, Graham Neubig, and Taylor Berg-Kirkpatrick. 2018. Unsupervised learning of syntactic structure with invertible neural projections. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1292-1302, Brussels, Belgium. Association for Computational Linguistics.
Yong Jiang, Wenjuan Han, and Kewei Tu. 2016. Unsupervised neural dependency parsing. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 763-771, Austin, Texas. Association for Computational Linguistics.
Lifeng Jin, Finale Doshi-Velez, Timothy Miller, Lane Schwartz, and William Schuler. 2019. Unsupervised learning of PCFGs with normalizing flow. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2442-2452, Florence, Italy. Association for Computational Linguistics.
Yoon Kim, Chris Dyer, and Alexander Rush. 2019a. Compound probabilistic context-free grammars for grammar induction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2369-2385, Florence, Italy. Association for Computational Linguistics.
Yoon Kim, Alexander Rush, Lei Yu, Adhiguna Kuncoro, Chris Dyer, and Gabor Melis. 2019b. Unsupervised recurrent neural network grammars. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1105-1117, Minneapolis, Minnesota. Association for Computational Linguistics.
Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings
of the 3rd International Conference on Learning Representations.
Dan Klein and Christopher Manning. 2004. Corpus-based induction of syntactic structure: Models of dependency and constituency. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04), pages 478-485, Barcelona, Spain.
Karim Lari and Steve J Young. 1990. The estimation of stochastic context-free grammars using the inside-outside algorithm. Computer speech & language, 4(1):35-56.
Percy Liang, Slav Petrov, Michael Jordan, and Dan Klein. 2007. The infinite PCFG using hierarchical Dirichlet processes. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 688-697, Prague, Czech Republic. Association for Computational Linguistics.
Mitchell Marcus, Grace Kim, Mary Ann Marcinkiewicz, Robert MacIntyre, Ann Bies, Mark Ferguson, Karen Katz, and Britta Schasberger. 1994. The Penn Treebank: Annotating predicate argument structure. In Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994.
Takuya Matsuzaki, Yusuke Miyao, and Jun'ichi Tsujii. 2005. Probabilistic CFG with latent annotations. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL'05), pages 75-82, Ann Arbor, Michigan. Association for Computational Linguistics.
Slav Petrov, Leon Barrett, Romain Thibaux, and Dan Klein. 2006. Learning accurate, compact, and interpretable tree annotation. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 433-440, Sydney, Australia. Association for Computational Linguistics.
Danilo Jimenez Rezende and Shakir Mohamed. 2015. Variational inference with normalizing flows. In Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, volume 37 of JMLR Workshop and Conference Proceedings, pages 1530-1538. JMLR.org.
Alexander Rush. 2020. Torch-struct: Deep structured prediction library. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 335-342, Online. Association for Computational Linguistics.
Djamé Seddah, Sandra Kübler, and Reut Tsarfaty. 2014. Introducing the SPMRL 2014 shared task on parsing morphologically-rich languages. In Proceedings
of the First Joint Workshop on Statistical Parsing of Morphologically Rich Languages and Syntactic Analysis of Non-Canonical Languages, pages 103-109, Dublin, Ireland. Dublin City University.
Yikang Shen, Zhouhan Lin, Chin wei Huang, and Aaron Courville. 2018a. Neural language modeling by jointly learning syntax and lexicon. In International Conference on Learning Representations.
Yikang Shen, Shawn Tan, Alessandro Sordoni, and Aaron Courville. 2019. Ordered neurons: Integrating tree structures into recurrent neural networks. In International Conference on Learning Representations.
Zhuoran Shen, Mingyuan Zhang, Haiyu Zhao, Shuai Yi, and Hongsheng Li. 2018b. Efficient attention: Attention with linear complexities. arXiv preprint arXiv:1812.01243.
Haoyue Shi, Karen Livescu, and Kevin Gimpel. 2020. On the role of supervision in unsupervised constituency parsing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7611-7621, Online. Association for Computational Linguistics.
David A. Smith and Jason Eisner. 2006. Minimum risk annealing for training log-linear models. In ACL 2006, 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, Sydney, Australia, 17-21 July 2006. The Association for Computer Linguistics.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631-1642, Seattle, Washington, USA. Association for Computational Linguistics.
Ke M. Tran, Yonatan Bisk, Ashish Vaswani, Daniel Marcu, and Kevin Knight. 2016. Unsupervised neural hidden Markov models. In Proceedings of the Workshop on Structured Prediction for NLP, pages 63-71, Austin, TX. Association for Computational Linguistics.
Ji Xu, Daniel J. Hsu, and Arian Maleki. 2018. Benefits of over-parameterization with EM. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, 3-8 December 2018, Montréal, Canada, pages 10685-10695.
Naiwen Xue, Fei Xia, Fu-dong Chiou, and Marta Palmer. 2005. The penn chinese treebank: Phrase structure annotation of a large corpus. Natural Language Engineering, 11(2):207-238.
Songlin Yang, Yong Jiang, Wenjuan Han, and Kewei Tu. 2020. Second-order unsupervised neural dependency parsing. In COLING.