Translate, then Parse! A Strong Baseline for Cross-Lingual AMR Parsing (2021)

Abstract: In cross-lingual Abstract Meaning Representation (AMR) parsing, researchers develop models that project sentences from various languages onto their AMRs to capture their essential semantic structures: given a sentence in any language, we aim to capture its core semantic content through concepts connected by manifold types of semantic relations. Methods typically leverage large silver training data to learn a single model that is able to project non-English sentences to AMRs. However, we find that a simple baseline tends to be overlooked: translating the sentences to English and projecting their AMR with a monolingual AMR parser (translate+parse,T+P). In this paper, we revisit this simple two-step base-line, and enhance it with a strong NMT system and a strong AMR parser. Our experiments show that T+P outperforms a recent state-of-the-art system across all tested languages: German, Italian, Spanish and Mandarin with +14.6, +12.6, +14.3 and +16.0 Smatch points

## introduction abstract meaning representation (amr), introduced by @xcite , aims at representing the meaning of a sentence in a semantic graph format. nodes represent entities, events and concepts, while (typed) edges express their relations. amr itself, as of now, is english-focused, e.g., predicate frames are linked to english propbank @xcite . however, the abstract nature of amr, and the fact that they are not explicitly linked to syntactic structure, make it appealing for extracting semantic structure of sentences in various languages. this insight led to the recent interest in a new task: cross-lingual amr parsing @xcite . here, researchers develop models to project sentences from different languages onto amr graphs. mod- * *equal contribution. @xcite . in our work we test these hypotheses by translating the source language sentences into english with a strong nmt system, and parse the resulting english sentences using a strong amr parser. we show that our baseline delivers strong performance in cross-lingual amr parsing across all considered languages, outperforming task-focused state-of-the-art models in all settings. we also discuss fairer evaluation of cross-lingual amr parsing and relevant implications of this work for research into cross-lingual amr parsing. we will release all code under public license. foot_0 2 related work cross-lingual amr parsing cross-lingual amr parsing was introduced by @xcite . they trained an alignmentbased amr parser model that leverages large amounts of parallel silver amr data obtained through annotation projection from a curated parallel corpus. the authors also discussed translate+parse (t+p) as a baseline using either the nmt systems google translate and nematus @xcite , or the smt system moses @xcite , together with a mono-lingual transition-based parser @xcite . however, their best t+p approach was google translate (gt) -which cannot be fully replicated by other researchers since both training data and model structure are hidden. given the recent advances in nmt @xcite and mono-lingual amr parsing @xcite , where parsers now achieve scores on par with human iaa assessments (c.f. banarescu et al. ( 2013 )), we show that time is ripe to put more spotlight on t+p. blloshmi et al. (2020) address the problem from complementary perspectives: i) they train a system that projects amr graphs from parsed english sentences to target sentences via a parallel corpus, yielding gold non-english sentences and silver amrs. conversely, ii) they train a system that employs an nmt system to translate english sentences from a human-annotated amr dataset to another language, yielding pairs of silver non-english sentences and gold amrs. this alleviates the dependency on external amr aligners. baselines for all languages (german, spanish, italian and mandarin chinese), we compare against i) amreager @xcite , and ii) xl-amr @xcite . evaluation metrics our main evaluation metric is smatch f1 @xcite . the smatch metric aligns the predicted graph with the gold graph and computes an f1 score that measures normalized triple overlap. additionally, we calculate f1 scores for finer-grained core semantic sub-tasks @xcite . 5 in our analyses ( §4.2), we also study results with s2match @xcite , that offers a potentially fairer evaluation in cross-lingual amr parsing, since it does not penalize allowed paraphrases that may emerge, e.g., due to the non-monotonous nature of translation (e.g., @xmath0. ## discussion we believe that the surprising effectiveness of translate+parse touches upon a key question: to what degree can amr be considered an interlingua? on one hand, @xcite explicitly state that amr 'is not designed as an interlingua'. indeed, amrs created for english sentences do have a flavour of english, since they are partially grounded in english propbank @xcite . but linking amrs to a propbank of another language, e.g., @xcite or arabic @xcite , and parsing non-english sentences into corresponding amrs, would not solve, but only displace the problem of being tied to a specific language's lexical semantic inventory. 7 on the other hand, amr does contain abstract meaning components that represent language phenomena we may consider as universals: negation, occurrence of named entities, semantic events and their related participants, as well as semantic relations such as possession, purpose or instrument. 8 we argue that this abstract structure again pushes amr more towards an interlingua. hence, the emergent interest in cross-lingual (a)mr @xcite @xcite is well justified. however, even if amr's inventory may favor an interlingual representation, we cannot, in general, expect a homomorphism of amrs constructed from semantically equivalent sentences in various languages, given wide-spread phenomena that can preclude a uniform amr representation, such as constructions involving head-switching phenomena or differences in lexical meaning. such a middle-ground is indicated by our results: (too) much divergence may be involved when mapping non-english sentences to original en-amrs directly, which is penalized by the strict(er) smatch metric. we show that evaluation with the softer s2match metric admits small deviations in the conceptual inventory of different languages. the fact that our indirect two-step approach t+p shows very strong performance also strengthens the view that amr is not fully an interlingua. the better performance of t+p may in part be due to a capacity of strong nmt systems to neutralize some amount of inter-lingual divergence, so that evalu-ation against en-amrs can yield better results in this setting. note that in our t+p approach two important intermediate (latent) representations are clearly separated: one in the nmt model (that builds a bridge between two natural languages) and one in the parser (that builds a bridge between english and a language of meaning with a flavor of english). by analyzing divergences between source and target in the t step, we can uncover aspects of semantic representations that are not isomorphic between languages, and which -by transfer via translation -may be neutralized to match the pivot-flavored amr structure. hence, the t+p approach offers an ideal framework for studying interlingual similarities and divergences in crosslingual amr parsing, by comparing the structuralsemantic divergences of non-english sentences and their translated english counterparts (aka translational divergences), with the aim of identifying structural-semantic differences between languages that can affect the cross-lingual mapping of sentences into a uniform interlingual amr. foot_9 ## conclusion we revisited translate+parse, an intuitive baseline for cross-lingual amr parsing. equipped with a recent nmt system and a monolingual amr parser, t+p outperforms other approaches by large margins across all evaluation settings. we propose to employ a graded metric for fairer evaluation of cross-lingual amr parsing. our work can serve as a strong baseline for future development of crosslingual amr parsers. finally, the t+p approach provides an ideal platform for deeper assessment, analysis, and break-down of potential interlingual aspects of amr.