ÚFAL at MRP 2020: Permutation-invariant Semantic Parsing in PERIN (2020)

Abstract: We present PERIN, a novel permutation-invariant approach to sentence-to-graph semantic parsing. PERIN is a versatile, cross-framework and language independent architecture for universal modeling of semantic structures. Our system participated in the CoNLL 2020 shared task, Cross-Framework Meaning Representation Parsing (MRP 2020), where it was evaluated on five different frameworks (AMR, DRG, EDS, PTG and UCCA) across four languages. PERIN was one of the winners of the shared task. The source code and pretrained models are available at http://www.github.com/ufal/perin .

## introduction the aim of the conll 2020 shared task, cross-framework meaning representation parsing (mrp 2020; @xcite , is to translate plain text sentences into their corresponding graph-structured meaning representation. 1 mrp 2020 features five formally and linguistically different frameworks with varying degrees of linguistic and structural complexity: these frameworks constitute the cross-framework track of mrp 2020, while the separate crosslingual track introduces one additional language for four out of the five frameworks: chinese amr @xcite , german drg, czech ptg and german ucca @xcite . in agreement with the shared task objective to advance uniform meaning representation parsing across diverse semantic graph frameworks and languages, we propose a language and structure agnostic sentence-to-graph neural network architecture modeling semantic representations from input sequences. the main characteristics of our approach are: our model was ranked among the two winning systems in both the cross-framework and the crosslingual tracks of mrp 2020 and significantly advanced the accuracy of semantic parsing from the last year's mrp 2019. examples of general, formalism-independent semantic parsers are scarce in the literature. @xcite propose a universal transitionbased parser for directed, acyclic graphs, capable of parsing multiple conceptually and formally different schemes. furthermore, several participants of mrp 2019 presented universal parsers. che et al. (2019) improved uniform transition-based parsing and used a different set of actions for each framework. lai et al. (2019) submitted a transitionbased parser with shared actions across treebanks, but failed to match the performance of the other parsers. straka and straková (2019) presented a general graph-based parser, where the meaning representation graphs are created by repeatedly adding nodes and edges. graph-based parsers (mcdonald and pereira, @xcite @xcite usually predict nodes in a sequential, auto-regressive manner and then connect them with a biaffine classifier. unlike these approaches, our model infers all nodes in parallel while allowing the creation of rich intermediate representations by node-to-node self-attention. machine learning tools able to efficiently process unordered sets are gaining more attention in recent years. @xcite and particularly @xcite proposed permutation-invariant neural networks for point clouds, which are of great relevance to our system. our work was further inspired by @xcite , who utilize permutation invariance for object detection in a similar fashion to our sentence-to-graph generation. ## results we present the overall results of our system in note that our original submission (denoted as perin) contained a bug in anchor prediction for chinese amr and both ptg frameworks. the bug caused the nodes to get anchored to at least one token. we submitted a fixed version called perin* in the post-competition evaluation and compare it with the original one in according to the official whole-percent-only all f1 score, our competition submission reached tied first place in both the cross-lingual and the crossframework track, with its performance virtually perin falls short in amr eng parsing by 1.31 %. on closer inspection, this follows from the inferior edge accuracy on this framework -the difference to hitachi is 4.56 % on amr eng and 2.78 % on amr zho . furthermore, hitachi is better in all aspects of eds eng and drg deu . on the other hand perin consistently beats hitachi in both ptg and both ucca frameworks. we hope that combining the strengths of these two parsers will help to further advance the state of meaning representation parsing. ## conclusion we introduced a novel permutation-invariant sentence-to-graph semantic parser called perin. given its state-of-the-art performance across a number of frameworks, we believe permutationinvariant node prediction might be the first step in a promising direction of semantic parsing and generally of graph generation.