SlowGuess/ACL_2020_OCR huggingface dataset:
Jonathan May and Jay Priyadarshi. 2017. SemEval-2017 task 9: Abstract meaning representation parsing and generation. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 536-545, Vancouver, Canada. Association for Computational Linguistics.
Joakim Nivre, Zeljko Agic, Maria Jesus Aranzabe, Masayuki Asahara, Aitziber Atutxa, Miguel Ballesteros, John Bauer, Kepa Bengoetxea, Riyaz Ahmad Bhat, Cristina Bosco, Sam Bowman, Giuseppe G. A. Celano, Miriam Connor, Marie-Catherine de Marneffe, Arantza Diaz de Ilarraza, Kaja Dobrovoljc, Timothy Dozat, Tomaž Erjavec, Richard Farkas, Jennifer Foster, Daniel Galbraith, Filip Ginter, lakes Goenaga, Koldo Gojenola, Yoav Goldberg, Berta Gonzales, Bruno Guillaume, Jan Hajic, Dag Haug, Radu Ion, Elena Irimia, Anders Johannsen, Hiroshi Kanayama, Jenna Kanerva, Simon Krek, Veronika Laippala, Alessandro Lenci, Nikola Ljubesic, Teresa Lynn, Christopher Manning, Catalina Márández, David Mareček, Héctor Martínez Alonso, Jan Mašek, Yuji Matsumoto, Ryan McDonald, Anna Missilä, Verginica Mititelu, Yusuke Miyao, Simonetta Montemagni, Shunsuke Mori, Hanna Nurmi, Petya Osenova, Lilja Øvrelid, Elena Pascual, Marco Passarotti, Cenel-Augusto Perez, Slav Petrov, Jussi Piitulainen, Barbara Plank, Martin Popel, Prokopis Prokopidis, Sampo Pyysalo, Loganathan Ramasamy, Rudolf Rosa, Shadi Saleh, Sebastian Schuster, Wolfgang Seeker, Mojgan Seraji, Natalia Silveira, Maria Simi, Radu Simionescu, Katalin Simko, Kiril Simov, Aaron Smith, Jan Štepanek, Alane Suhr, Zsolt Szántó, Takaaki Tanaka, Reut Tsarfaty, Sumire Uematsu, Larraitz Uria, Viktor Varga, Veronika Vincze, Zdeněk Žabokrtský, Daniel Zeman, and Hanzhi Zhu. 2015. Universal Dependencies 1.2. http://universaldependencies.github.io/docs/.
Stephan Oepen, Omri Abend, Jan Hajic, Daniel Hershcovich, Marco Kuhlmann, Tim O'Gorman, Nianwen Xue, Jayeol Chun, Milan Straka, and Zdenka Urešova. 2019. Mrp 2019: Cross-framework meaning representation parsing. In Proceedings of the Shared Task on Cross-Framework Meaning Representation Parsing at the 2019 Conference on Natural Language Learning, pages 1-27.
Stephan Oepen, Marco Kuhlmann, Yusuke Miyao, Daniel Zeman, Silvie Cinkova, Dan Flickinger, Jan Hajic, Angelina Ivanova, and Zdenka Uresova. 2016. Towards comparability of linguistic graph banks for semantic parsing. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), pages 3991-3995.
Stephan Oepen, Marco Kuhlmann, Yusuke Miyao, Daniel Zeman, Silvie Cinkova, Dan Flickinger, Jan Hajic, and Zdenka Urešová. 2015. SemEval 2015 task 18: Broad-coverage semantic dependency parsing. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 915–926, Denver, Colorado. Association for Computational Linguistics.
Stephan Oepen, Marco Kuhlmann, Yusuke Miyao, Daniel Zeman, Dan Flickinger, Jan Hajic, Angelina Ivanova, and Yi Zhang. 2014. SemEval 2014 task 8: Broad-coverage semantic dependency parsing. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 63-72, Dublin, Ireland. Association for Computational Linguistics.
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532-1543.
Drew Reisinger, Rachel Rudinger, Francis Ferraro, Craig Harman, Kyle Rawlins, and Benjamin Van Durme. 2015. Semantic Proto-Roles. Transactions of the Association for Computational Linguistics, 3:475-488.
Rachel Rudinger, Adam Teichert, Ryan Culkin, Sheng Zhang, and Benjamin Van Durme. 2018a. Neural davidsonian semantic proto-role labeling. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 944-955, Brussels, Belgium. Association for Computational Linguistics.
Rachel Rudinger, Aaron Steven White, and Benjamin Van Durme. 2018b. Neural Models of Factuality. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 731-744, New Orleans, Louisiana. Association for Computational Linguistics.
Abigail See, Peter J Liu, and Christopher D Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1073-1083.
Adam Teichert, Adam Poliak, Benjamin Van Durme, and Matthew R Gormley. 2017. Semantic proto-role labeling. In Thirty-First AAAI Conference on Artificial Intelligence.
Siddharth Vashishtha, Benjamin Van Durme, and Aaron Steven White. 2019. Fine-Grained Temporal Relation Extraction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2906–2919, Florence, Italy. Association for Computational Linguistics.
Yuxuan Wang, Wanxiang Che, Jiang Guo, and Ting Liu. 2018. A neural transition-based approach for semantic dependency graph parsing. In AAAI Conference on Artificial Intelligence.
Aaron Steven White, Drew Reisinger, Keisuke Sakaguchi, Tim Vieira, Sheng Zhang, Rachel Rudinger,
8437
Kyle Rawlins, and Benjamin Van Durme. 2016. Universal decompositional semantics on universal dependencies. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1713-1723, Austin, TX. Association for Computational Linguistics.
Aaron Steven White, Elias Stengel-Eskin, Siddharth Vashishtha, Venkata Govindarajan, Dee Ann Reisinger, Tim Vieira, Keisuke Sakaguchi, Sheng Zhang, Francis Ferraro, Rachel Rudinger, et al. 2019. The universal decompositional semantics dataset and decomp toolkit. arXiv preprint arXiv:1909.13851.
Sheng Zhang, Xutai Ma, Kevin Duh, and Benjamin Van Durme. 2019a. AMR parsing as sequence-to-graph transduction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 80-94, Florence, Italy. Association for Computational Linguistics.
Sheng Zhang, Xutai Ma, Kevin Duh, and Benjamin Van Durme. 2019b. Broad-coverage semantic parsing as transduction. arXiv preprint arXiv:1909.02607.
Sheng Zhang, Xutai Ma, Rachel Rudinger, Kevin Duh, and Benjamin Van Durme. 2018. Cross-lingual decompositional semantic parsing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1664–1675, Brussels, Belgium. Association for Computational Linguistics.
Sheng Zhang, Rachel Rudinger, and Benjamin Van Durme. 2017. An Evaluation of PredPatt and Open IE via Stage 1 Semantic Role Labeling. In IWCS 2017 — 12th International Conference on Computational Semantics — Short papers.
8438
A Derivation of  $\psi$
The metric used in visualizations Fig. 3a-3c is given by:
where  $\operatorname{corr}(\nu^j - \nu^{j*}, \nu^k - \nu^{k*})$  and  $\operatorname{corr}(\nu^{j*}, \nu^{k*})$  are defined as follows:
$$
\overline {{\nu}} ^ {j} = \frac {1}{N} \sum_ {i = 1} ^ {N} \nu_ {i} ^ {j}
$$
$$
O _ {j} = \frac {1}{N} \sum_ {i = 1} ^ {N} (\nu_ {i} ^ {j} - \nu_ {i} ^ {j *}) ^ {2}
$$
$$
\operatorname {c o r r} \left(\nu^ {j} - \nu^ {j *}, \nu^ {k} - \nu^ {k *}\right) =
$$
$$
\frac {\frac {1}{N} \sum_ {i = 1} ^ {N} \left(\left(\nu_ {i} ^ {j *} - \nu_ {i} ^ {j}\right) \left(\nu_ {i} ^ {k *} - \nu_ {i} ^ {k}\right)\right)}{\sqrt {O _ {j}} \sqrt {O _ {k}}}
$$
$$
E _ {j} = \frac {1}{N} \sum_ {i = 1} ^ {N} (\nu_ {i} ^ {j *} - \overline {{\nu}} _ {i} ^ {j *}) ^ {2}
$$
$$
\operatorname {c o r r} \left(\nu^ {j *}, \nu^ {k *}\right) =
$$
$$
\frac {\frac {1}{N} \sum_ {i = 1} ^ {N} \left(\left(\nu_ {i} ^ {j *} - \overline {{\nu}} _ {i} ^ {j *}\right) \left(\nu_ {i} ^ {k *} - \overline {{\nu}} _ {i} ^ {k *}\right)\right)}{\sqrt {E _ {j}} \sqrt {E _ {k}}}
$$
Note that by this definition,  $\psi$  is effectively a ratio of Pearson correlations, where the denominator is exactly the Pearson correlation between  $\nu^{j*}$  and  $\nu^{k*}$ .
8439