SlowGuess/ACL_2020_OCR huggingface dataset:
Alexandre Rademaker, Fabricio Chalub, Livy Real, Cláudia Freitas, Eckhard Bick, and Valeria de Paiva. 2017. Universal Dependencies for Portuguese. In Proceedings of the Fourth International Conference on Dependency Linguistics (Depling 2017), pages 197-206, Pisa, Italy. Linköping University Electronic Press.
Carlos Ramisch, Silvio Ricardo Cordeiro, Agata Savary, Veronika Vincze, Verginica Barbu Mittelu, Archna Bhatia, Maja Buljan, Marie Candito, Polona Gantar, Voula Giouli, Tunga Güngör, Abdelati Hawwari, Uxoa Inurrieta, Jolanta Kovalevskaite, Simon Krek, Timm Lichte, Chaya Liebeskind, Johanna Monti, Carla Parra Escartin, Behrang QasemiZadeh, Renata Ramisch, Nathan Schneider, Ivelina Stoyanova, Ashwini Vaidya, and Abigail Walsh. 2018. Edition 1.1 of the PARSEME shared task on automatic identification of verbal multiword expressions. In Proceedings of the Joint Workshop on Linguistic Annotation, Multiword Expressions and Constructions (LAW-MWE-CxG-2018), pages 222-240, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
Lance Ramshaw and Mitch Marcus. 1995. Text chunking using transformation-based learning. In Proceedings of the Third Workshop on Very Large Corpora, pages 82-94, Cambridge, Massachusetts.
Ivan A Sag, Timothy Baldwin, Francis Bond, Ann Copestake, and Dan Flickinger. 2002. Multiword expressions: A pain in the neck for NLP. In Proceedings of the Third International Conference on Intelligent Text Processing and Computational Linguistics, pages 1-15, Mexico City, Mexico. Springer.
Manuela Sanguinetti, Cristina Bosco, Alberto Lavelli, Alessandro Mazzei, Oronzo Antonelli, and Fabio Tamburini. 2018. PoSTWITA-UD: An Italian Twitter treebank in Universal Dependencies. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), pages 1768-1775, Miyazaki, Japan. European Language Resources Association (ELRA).
8791
Agata Savary, Carla Parra Escartin, Francis Bond, Jelena Mitrovic, and Verginica Barbu Mittelu, editors. 2019. Proceedings of the Joint Workshop on Multiword Expressions and WordNet (MWE-WN 2019). Association for Computational Linguistics, Florence, Italy.
Agata Savary, Carlos Ramisch, Silvio Cordeiro, Federico Sangati, Veronika Vincze, Behrang QasemiZadeh, Marie Candito, Fabienne Cap, Voula Giouli, Ivelina Stoyanova, and Antoine Doucet. 2017. The PARSEME shared task on automatic identification of verbal multiword expressions. In Proceedings of the 13th Workshop on Multiword Expressions (MWE 2017), pages 31-47, Valencia, Spain. Association for Computational Linguistics.
Agata Savary, Carlos Ramisch, Jena D. Hwang, Nathan Schneider, Melanie Andresen, Sameer Pradhan, and Miriam R. L. Petruck, editors. 2018. Proceedings of the Joint Workshop on Linguistic Annotation, Multiword Expressions and Constructions (LAW-MWE-CxG-2018). Association for Computational Linguistics, Santa Fe, New Mexico, USA.
Nathan Schneider, Emily Danchik, Chris Dyer, and Noah A. Smith. 2014. Discriminative lexical semantic segmentation with gaps: Running the MWE gamut. Transactions of the Association for Computational Linguistics, 2:193-206.
Nathan Schneider and Noah A. Smith. 2015. A corpus and model integrating multiword expressions and supersenses. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1537-1547, Denver, Colorado. Association for Computational Linguistics.
Katalin Ilona Simko, Viktória Kovács, and Veronika Vincze. 2017. USzeged: Identifying verbal multiword expressions with POS tagging and parsing techniques. In Proceedings of the 13th Workshop on Multiword Expressions (MWE 2017), pages 48-53, Valencia, Spain. Association for Computational Linguistics.
Per Erik Solberg, Arne Skjaerholt, Lilja Øvrelid, Kristin Hagen, and Janne Bondi Johannessen. 2014. The Norwegian dependency treebank. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14), pages 789-795, Reykjavik, Iceland. European Language Resources Association (ELRA).
Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15:1929-1958.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all
you need. In Advances in Neural Information Processing Systems 30, pages 5998-6008. Curran Associates, Inc.
Veronika Vincze, János Zsibrita, and István Nagy T. 2013. Dependency parsing for identifying Hungarian light verb constructions. In Proceedings of the Sixth International Joint Conference on Natural Language Processing, pages 207-215, Nagoya, Japan. Asian Federation of Natural Language Processing.
Jakub Waszczuk, Rafael Ehren, Regina Stodden, and Laura Kallmeyer. 2019. A neural graph-based approach to verbal MWE identification. In Proceedings of the Joint Workshop on Multiword Expressions and WordNet (MWE-WN 2019), pages 114-124, Florence, Italy. Association for Computational Linguistics.
Eric Wehrli. 2000. Parsing and collocations. In Proceedings of the Second International Conference on Natural Language Processing, NLP '00, pages 272-282, London, UK. Springer-Verlag.
Ralph Weischedel, Martha Palmer, Mitchell Marcus, Eduard Hovy, Sameer Pradhan, Lance Ramshaw, Nianwen Xue, Ann Taylor, Jeff Kaufman, Michelle Franchini, Mohammed El-Bachouti, Robert Belvin, and Ann Houston. 2013. Ontonotes release 5.0 LDC2013T19.
Daniel Zeman, Jan Hajic, Martin Popel, Martin Potthast, Milan Straka, Filip Ginter, Joakim Nivre, and Slav Petrov. 2018. CoNLL 2018 shared task: Multilingual parsing from raw text to universal dependencies. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 1-21, Brussels, Belgium. Association for Computational Linguistics.
Xingxing Zhang, Jianpeng Cheng, and Mirella Lapata. 2017. Dependency parsing as head selection. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 665-676, Valencia, Spain. Association for Computational Linguistics.
8792
Appendix A Evaluation of the Strengths of Our Parsing Models
To confirm that we work with reasonable parsing models, we compare our parsers with those in the CoNLL 2018 shared task (Zeman et al., 2018). The shared task featured an end-to-end parsing task, requiring all levels of text processing including tokenization, POS tagging, morphological analysis, etc. We focus on the parsing task only, and predict syntactic trees based on sentences tokenized by the Qi et al. (2018) submission. $^{12}$  Table A1 shows that our parsing models are highly competitive with the current state-of-the-art. Indeed, on four out of the six treebanks we selected for their density of flat structures, our baseline models actually achieve higher labeled attachment scores (LAS) than the top scorer did in the official shared task.
Treebank	Our Parsers	CoNLL 2018 Best
de_gsd	80.65	80.36
it_ostwita	79.33	79.39
nl_alpino	89.78	89.56
nl_lassysmall	87.96	86.84
no_nynorsk	90.44	90.99
pt_bosque	89.25	87.81
Table A1: Comparison of our (non-MTL) parsing models with the best-performing systems (Che et al., 2018; Qi et al., 2018) from the CoNLL 2018 shared task, measured by labeled attachment scores (LAS, %).
Appendix B Do MTL and Joint Decoding Help Parsing Performance?
In Table A2 (next page), we investigate whether MTL and combining scores from both representations of flat-structure MWEs can improve parsing performance. We observe very little difference among the various strategies. This fact can be explained by the relatively low ratios of flat relations and the already-high base performance: the room for improvement on the standard LAS metrics is quite small.
<sup>12</sup>We thank the shared task participants and the organizers for making system predictions available at https://lindat.mff.cuni.cz/repository/xmlui/handle/11234/1-2885.
8793
w/bi-LSTM
Treebank	Compl.
Ratio ↓	Parsing	MTL
Parsing	Joint
Decoding
English	100.00	89.30±0.41	89.39±0.67	89.77±0.52
nl_alpino	100.00	81.97±1.27	82.57±0.99	82.79±0.77
nl_lassysmall	99.82	82.06±1.30	82.90±0.64	81.55±1.26
no_nynorsk	99.78	86.54±0.50	86.35±0.37	86.65±0.64
pt_bosque	97.38	84.29±2.15	84.48±1.61	85.28±0.25
it_postwita	94.89	77.39±0.69	76.75±1.29	76.59±1.46
de_gsd	93.00	76.66±0.64	76.35±0.83	75.22±1.98
Average	82.60	82.69	82.55
w/BERT
Treebank	Compl.
Ratio ↓	Parsing	MTL
Parsing	Joint
Decoding
English	100.00	93.73±0.24	93.52±0.17	93.38±0.39
nl_alpino	100.00	89.82±0.55	89.95±0.41	89.86±0.59
nl_lassysmall	99.82	89.78±0.46	89.76±0.17	89.67±0.16
no_nynorsk	99.78	90.77±0.20	90.98±0.38	90.85±0.32
pt_bosque	97.38	89.78±0.32	89.51±0.39	89.79±0.39
it_postwita	94.89	81.61±0.32	81.70±0.14	81.53±0.63
de_gsd	93.00	81.51±0.23	81.74±0.23	81.52±0.17
Average	88.14	88.17	88.09
Table A2: Dependency-parsing labeled attachment scores (LAS, %) on the test sets with bi-LSTM (top) and BERT (bottom) feature extractors. The cell containing the best result for each treebank has blue shading; results within one standard deviation of the best are in boldface.
8794