F ully- S emantic P arsing and G eneration: the B abel N et M eaning R epresentation (2022)

Abstract: A language-independent representation of meaning is one of the most coveted dreams in Natural Language Understanding. With this goal in mind, several formalisms have been proposed as frameworks for meaning representation in Semantic Parsing. And yet, the dependencies these formalisms share with respect to language-specific repositories of knowledge make the objective of closing the gap between high- and low-resourced languages hard to accomplish. In this paper, we present the BabelNet Meaning Representation (BMR), an interlingual formalism that abstracts away from language-specific constraints by taking advantage of the multilingual semantic resources of BabelNet and VerbAtlas. We describe the rationale behind the creation of BMR and put forward BMR 1.0, a dataset labeled entirely according to the new formalism. Moreover, we show how BMR is able to outperform previous formalisms thanks to its fully-semantic framing, which enables top-notch multilingual parsing and generation. We release the code at https://github.com/SapienzaNLP/bmr .

## introduction natural language understanding (nlu) enables machines to understand human language. a key enabling task in nlu is that of semantic parsing, whose longed-for dream is that of developing a formalism that can be used as an interlingual representation of meaning, i.e., one that, independently of the language, can explicitly embed sentence meaning into a machine-and human-readable form @xcite . to this end, different formalisms such as abstract meaning representation @xcite , universal conceptual cognitive annotation @xcite and universal meaning representation @xcite , have been proposed over the years. as of now though, amr is the most popular formalism for semantic parsing, being widely applied to a variety of areas of nlp, such as machine translation @xcite , question answering @xcite @xcite , human-robot interaction @xcite ), text summarization @xcite and information extraction @xcite . the primary precept of amr is that different sentences carrying the same meaning should have the same graph representation. nonetheless, a few inherent properties of amr make it inappropriate for the purpose of providing a languageagnostic representation of meaning. in fact, nodes within amr graphs are represented by means of either english lemmas or ontonotes frames @xcite which, in turn, are based on propbank @xcite . the issue with lemmas is that they are merely surface forms devoid of semantics, whereas, with respect to frames, even though analogous repositories exist in other languages such as ancora for spanish @xcite or the chinese propbank @xcite , they are not mutually interlinked, hence making the cross-lingual application of amr arduous to achieve @xcite . against this background, we follow the ideas put forward by @xcite and develop the ba-belnet meaning representation (bmr), a formalism providing the building blocks for a languageagnostic representation of meaning by exploiting the wealth of multilingual knowledge contained in babelnet @xcite foot_0 and verbatlas @xcite 2 . in outline, the main contributions of this paper are as follows: (i) we introduce bmr, a new semantic parsing formalism that can be used as an interlingua, (ii) we produce bmr 1.0, i.e., the first lexical-semantic dataset annotated according to the bmr formalism, (iii) we create and release models that can generate bmr graphs from text and text from bmr graphs in english, german, spanish, and italian, and (iv) we describe a sound experimental setup to show how, thanks to its fully semantic framing, bmr outdoes previous formalisms in both preserving and encoding textual information, as well as in being used as an interlingua in downstream tasks such as machine translation. ## related work even though the vast majority of formalisms for semantic parsing have been designed with english in mind, several approaches have attempted to narrow the gap between english and other languages. for instance, universal conceptual cognitive annotation @xcite , ucca) was proposed as a cross-lingual annotation formalism in which words in a sentence are connected using semantic relations not tied to specific languages. and yet, while ucca reflects the semantic relations between nodes via a set of coarse-grained roles, it represents concepts by means of simple lemmas, hence preventing an abstraction from language-specific constraints. parallel meaning bank @xcite , an approach based on the discourse representation theory (kamp and reyle, 1993, drt), foot_2 also emerged. in pmb, english sentences are parsed with labels that are automatically projected to non-english translations. pmb too, however, cannot be seen as a unified interlingual representation, since it uses english-specific meaning repositories. as regards abstract meaning representation @xcite , instead, several approaches have tried to adapt it for cross-lingual use. as a case in point, @xcite analyzed the viability of tailoring the amr formalism to fit other languages by making use of language-specific repositories similar to propbank @xcite . 4 on a different note, @xcite and @xcite attempted to adopt amr as an interlingual formalism, despite its english-centric nature, by assuming that the amr graph of an english sentence is also representative of translations of that sentence in other languages. once again, these approaches testify to the limits of amr as an interlingua, given the drawbacks of dealing with structural divergences among different languages. in recent years, @xcite have recommended abstracting the amr formalism away in order to reduce its language-specific complexity by preserving just the predicate roles and relations that constitute the core semantic information of sentences. conversely, rather than decreasing the complexity of amr, the universal meaning representation @xcite extends it by including new features that render the formalism less tied to a specific language. in particular, umr enriches the verbal predicates with information about grammatical aspect and scope, while introducing temporal and modal dependencies at the document level. finally, it enhances amr to use it as a crosslingual formalism by employing language-specific repositories and relations. yet, the focus of umr is that of providing languages with the necessary resources to parse texts, rather than being an interlingual representation. in contrast to previous approaches, and thanks to the multilingually-shared word meanings and semantic roles taken from the interlinked repositories of babelnet @xcite and verbatlas @xcite , we put forward bmr, a formalism that fully detaches from syntax and thus stands as a lexical-semantic representation that is able to bring different languages together. ## preliminaries to accomplish the goal of an interlingual meaning representation, we disconnect our formalism from language-specific constraints of any kind. to this end, we draw on resources that inherently connect word meanings and predicate-argument structures across languages, i.e., babelnet and verbatlas. babelnet @xcite is a multilingual encyclopedic dictionary and semantic knowledge base in which concepts are represented as synsets (sets of synonyms that convey the same meaning), linked via semantic relation edges like hypernymy or meronymy. babelnet was built by the aggregation of several knowledge resources including wordnet @xcite , wikipedia and wiktionary, resulting in a remarkable ontology of concepts and named entities covering 500 languages. given its versatility, which makes it suitable for a wide range of tasks across languages, we employ its most recent version 5.0 as a tool to switch the focus of semantic parsing formalisms from words to multilingual concepts. verbatlas @xcite is a manuallycurated lexical-semantic inventory that collapses the babelnet verbal synsets into around 450 semantically-coherent frames, each defining prototypical argument structures via human-readable relationships (e.g. agent, theme). thanks to its linkage to babelnet, verbatlas represents the best option for handling predicate-argument relations in bmr in a language-independent manner. ## babelnet meaning representation like amr, bmr embeds the semantics of a sentence in a directed acyclic graph, with nodes and edges connecting them. however, where amr relies on english lemmas and ontonotes frames to represent nodes and relations (see figure 1 ), bmr disposes of language-specific constraints, and employs multilingual concepts and self-explanatory semantic roles (see figure 2 ). 5 in what follows (sections 4.1 to 4.4), we will describe and detail the features that make bmr stand out with respect to a widely-employed semantic parsing formalism such as amr, as well as their integration into the amr 3.0 dataset @xcite to produce the bmr 1.0 dataset. 6 ## experimental setup to demonstrate the importance of bmr's semantic framing, its aptness at preserving lexical information, and its effectiveness in acting as an interlingual representation, we devise three experiments to assess its performance in comparison with amr. before delving into their details (section 5.2), as well as describing our models and the evaluation measures we employ (sections 5.3 and 5.4, respectively), we first provide thorough information regarding the datasets used in our experiments. ## bmr*: a case study results for the experiments we conducted depict bmr* as the model that, on the whole, achieves the lowest scores. with the aim of showing how such results might arise due to inadequate evaluation measures (see section 6.3), we propose a focused case study in which we qualitatively inspect the differences between graphs and sentences generated by means of the amr and bmr* models. starting from the sentence "my friend did not tolerate his father's behaviour" (figure 3 ), it can be seen how the grammatical categories of number and tense for the words friend and tolerate are correctly preserved by bmr* only. additionally, it can be noted how the complex structure that defines child in amr can confuse the model when there is a reentrant node (in this case, the model does not know to whom the father is related). as interestingly, the sentence generated via bmr* replaces tolerate with the synonym put up with, which worsens its performance according to exact string matching metrics, but, at the same time, provides an insight of a higher level of abstraction when lemmas are omitted. ## error analysis although the experiments reported in section 5 testify to the quality of bmr, following an in-house behavioral analysis inspired by the work of ribeiro et al. (2020), we identify three main classes of errors that undermine the application of bmr as an interlingua, one concerning the formalism (repository contraints), one tied to the data contained in the bmr 1.0 dataset (disambiguation constraints), and one concerning the language-specific lexicons (language-specific constraints). repository constraints babelnet features a wealth of synsets covering content words in a multilingual setting, but, at the same time, does not provide information regarding parts of speech other than nouns, verbs, adjectives and adverbs. as a result, bmr uses language-specific lemmas to represent conjunctions or ambiguous pronouns such as anyone, which can mean either "not a single person" or "everyone", depending on the use of negative or positive phrasing. on a different note, with roughly 6,500 languages spoken in the world and babel-net 5.0 featuring a subset of them, the definition of bmr as an interlingua is actually constrained to the number, albeit large, of 500 babelnet languages. disambiguation constraints the creation of bmr 1.0 is based upon the word sense disambiguation task carried out via a state-of-the-art system @xcite . and yet, this neural architecture is trained to predict word senses featured in the wordnet 3.0 sense inventory only. by virtue of the fact that, following the node merging strategy (section 4.2), we can obtain polysemous multiwords found in babelnet but not in wordnet (as is the case of run off at the mouth), we cannot provide disambiguation for such instances. this justifies the fact that 8% of content nodes in bmr are not disambiguated (see also section 4.4). ## conclusion current semantic parsing formalisms share tight dependencies with semantic repositories which are both language-specific and isolated from word senses in other languages. as a result, they are not fit to be used as interlingual representations of meaning. in this paper, we put forward bmr, a new language-independent formalism that abstracts away from language-specific constraints thanks to two multilingual semantic resources, namely, ba-belnet and verbatlas. to put our formalism into practice, we also created bmr 1.0, the first dataset labeled according to bmr. our experiments demostrate the impact that the fully-semantic framing of our formalism has in comparison to the widely-employed formalism of amr, as well as showing its ability to be a better tool at encoding textual information, and a much more effective interlingua in a text-to-graph-to-text machine translation task. as future work, we plan to (i) create a single multilingual model to parse graphs and generate text in any language, (ii) apply bmr crosslingually to other downstream tasks such as text summarization, (iii) evolve the formalism to prevent the inclusion of lexical information of any kind. we make our code and data available to the research community at https://github. com/sapienzanlp/bmr . by desired priority: using the predicate name as a query to look for available synsets, (i) pick a nominal synset also featured in wordnet 3.0 (e.g., querying with the lemma liberality for the predicate liberal.02), (ii) pick an adjectival synset featured in wordnet 3.0, (iii) pick a nominal synset not featured in wordnet 3.0, (iv) pick an adjectival synset not featured in wordnet 3.0, (v) pick a synonym to query for available synsets (e.g., querying with the lemma correct for predicate be_it.07). see