Mitigating Data Scarcity in Semantic Parsing across Languages: the Multilingual Semantic Layer and its Dataset (2024)

Abstract: Data scarcity is a prevalent challenge in the era of Large Language Models (LLMs). The insatiable hunger of LLMs for large corpora becomes even more pronounced when dealing with non-English and low-resource languages. The issue is particularly exacerbated in Semantic Parsing (SP), i.e. the task of converting text into a formal representation. The complexity of semantic formalisms makes training human annotators and subsequent data annotation unfeasible on a large scale, especially across languages. To mitigate this, we first introduce the Multilingual Semantic Layer (MSL), a conceptual evolution of previous formalisms, which decouples from disambiguation and external inventories and simplifies the task. MSL provides the necessary tools to encode the meaning across languages, paving the way for developing a high-quality semantic parsing dataset across different languages in a semi-automatic strategy. Subsequently, we manually refine a portion of this dataset and fine-tune GPT-3.5 to propagate these refinements across the dataset. Then, we manually annotate 1,100 sentences in eleven languages, including low-resource ones. Finally, we assess our dataset’s quality, showcasing the performance gap reduction across languages in Semantic Parsing. Our code and dataset are openly available at https: //github.com/SapienzaNLP/MSL.

heading: 1 Introduction
text: One of the long-term goals of AI is to enable machines to comprehend human text in any language. At the core of Natural Language Understanding (NLU) lies the task of Semantic Parsing (SP), aiming to convert text into machine-interpretable representations. Although Large Language Models (LLMs) have advanced significantly in understanding human text, semantic representations remain crucial for various applications. These range from chatbots and Virtual Assistants like Amazon Alexa,
Apple Siri, and Google Assistant, as underscored in Rongali et al. (2020), to more specific tasks, e.g., SQL queries (Dou et al., 2022) or machineinterpretable commands (Wang et al., 2021).
Achieving proficiency in SP involves enabling machines to understand semantic relations between concepts in any language. Over the years, numerous SP formalisms grounded in various linguistic theories have emerged, such as the Discourse Representation Theory (Kamp and Reyle, 1993, DRT), Prague Dependency Treebank (Nedoluzhko et al., 2016, PDT), Universal Conceptual Cognitive Annotation (Abend and Rappoport, 2013, UCCA) or Abstract Meaning Representation (Banarescu et al., 2013, AMR), among others. Graph-based representations – e.g., AMR – have attracted most of the attention since they can act as an interface that is comprehensible for humans and interpretable for machines (see Figure 1). The objective of graphbased representations is to encode the meaning of language into a directed acyclic graph, where nodes represent concepts and edges represent semantic relations between concepts. However, even if these formalisms are able to encode the meaning in English effectively, they still struggle to scale to other languages (Zhu et al., 2019).
14056
Over the years, multiple attempts have been proposed to encode other languages: from adaptations based on language-specific repositories – e.g., Spanish AMR (Migueles-Abraira et al., 2018) with AnCoraNet (Aparicio et al., 2008) or the Chinese AMR (Li et al., 2019) with Chinese Propbank (Xue et al., 2005) – to interlingua formalisms, such as the Uniform Meaning Representation (Gysel et al., 2021, UMR) or the BabelNet Meaning Representation (Navigli et al., 2022, BMR). However, the main challenge persists: the scarcity of annotated data keeps these formalisms in the realm of abstraction, since annotated data is crucial for training parsers. This scarcity is primarily attributable to the high costs and complexity of manual annotation. Annotators must thoroughly understand each specific language’s rules, the formalism and the meaning repository, making the process highly labour-intensive. Moreover, even though some projects have generated annotated data for these formalisms – like AMR 3.0 (Knight et al., 2020), BMR 1.0 (Martínez Lorenzo et al., 2022), or UMR (Bonn et al., 2023b) – these datasets are English-oriented and hindered by paywalls or restrictive licenses. Furthermore, even though there have been efforts to annotate non-English languages – like UMR 1.0 (Bonn et al., 2023a), Turkish (Azin and Eryiğit, 2019), Persian (Tohidi et al., 2024), Portuguese (Sobrevilla Cabezudo and Pardo, 2019), or Vietnamese (Linh and Nguyen, 2019) – they have not produced more than 200 examples per language, which is inadequate for effective parser training. This situation underscores a critical bottleneck in employing these formalisms for wider linguistic applications.
This paper introduces a practical solution for tackling data scarcity, which can scale across different languages. Firstly, we propose the Multilingual Semantic Layer (MSL), an evolution of previous formalisms, which acts as the necessary tool to enable data generation across languages. Secondly,
by leveraging this novel representation, we develop a methodology for semi-automatically generating a vast amount of high-quality annotations across languages by making use of LLMs (under an academic budget). Lastly, we have annotated a gold standard dataset of 1,100 graphs across eleven languages, including low-resource ones, allowing us to assess the quality of our dataset.

heading: 2 Related Work
text: Even if multiple formalisms have been proposed over the years, just a few of these provide annotated data. Table 1 highlights their main features of the largest dataset annotated for each formalism.

heading: Discourse Representation Theory (Kamp and
text: Reyle, 1993, DRT) provides a framework for representing the meaning of entire texts at the discourse level, which can be directly converted into logical forms. The Parallel Meaning Bank (Bos, 2013, PMB) is the most recent dataset under the DRT formalism, which incorporates word senses from WordNet (Miller, 1992), and semantic roles from VerbNet (Brown et al., 2019). However, the multilingual annotations use the English graph as the interlingua by automatically projecting the English annotations to other languages. Moreover, the number of annotations is insufficient.

heading: Prague Dependency Treebank (Hajič, 1998,
text: PDT) is a multi-layer formalism that defines a syntactic analysis for the sentence. The Tectogrammatical layer (Zeman and Hajic, 2020, PTG) covers semantic distinctions, such as the predicateargument structures, word senses or co-references. The corpora are in Czech and English and rely on language-specific repositories.

heading: Universal Conceptual Cognitive Annotation
text: (Abend and Rappoport, 2013, UCCA) has a foundational layer focusing on the predicateargument structure. However, it does not abstract
away from word order and from the actual words.

heading: Abstract Meaning Representation (Banarescu
text: et al., 2013, AMR) provides a theoretical framework that does abstracts away from the syntax level. Some adaptations exist in other languages, but no large corpora are annotated.

heading: Uniform Meaning Representation (Bonn et al.,
text: 2023a, UMR) is a semantic representation based on AMR that expands the semantic range of AMR at the sentence level by adding aspect and scope, while also expanding AMR to a document-level representation by incorporating co-reference, temporal and modal dependencies through documents. UMR also accommodates the formalism to other languages, specifically low-resource ones, by incorporating new explicit argument relations that mitigate the lack of predicate repositories. However, no large annotated datasets are available to train LMs; there are only 2,186 pairs of sentence graphs across six languages (Arapaho, Chinese, CocamaCocamilla, English, Navajo and Sanapaná).

heading: BabelNet Meaning Representation (Navigli
text: et al., 2022, BMR) extends AMR and aims to be an interlingua representation by using BabelNet concepts (Navigli and Ponzetto, 2010) and VerbAtlas frames (Di Fabio et al., 2019). However, the cross-lingual corpus is English-oriented, and BabelNet is not open-source (extended in Appendix A).
Even though certain formalisms have been developed to encode English semantics, their application to other languages often remains theoretical. These formalisms depend on external resources (e.g., WordNet, PropBank, or others), most of which are only available in English. While initiatives like Spanish AMR with AncoraNet or UMR 1.0 for low-resource languages attempt to bridge this gap, their annotations are limited (not exceeding 2,000 graphs), rendering them impractical for widespread
use and requiring trained annotators on both the formalism and the external inventories. Moreover, even though multilingual formalisms like PMB and BMR exist, they essentially annotate English sentences and employ the English graph as the representation of parallel automatic translations. This leads to language divergences, as demonstrated in Figure 2 where "jaywalk" requires two nodes for its non-literal Spanish translation, underscoring the discrepancies and challenges in cross-lingual semantic representation.

heading: 3 Our Contributions
text: In this section, we overcome the data scarcity in SP by providing the novel Multilingual Semantic Layer (MSL) – with the necessary tools to encode the semantic relations between concepts across languages (Section 3.1) – and a multilingual dataset with millions of high-quality annotations, including manual annotations, that enables SP in different languages (Section 3.2).

heading: 3.1 MSL
text: We introduce MSL as a means for decoupling SP from the meaning behind concepts, focusing on extracting semantic relations between concepts in the sentence. MSL is built on top of previous graphbased formalism theories (such as AMR, BMR or UMR), and it employs a directed acyclic graph with nodes representing concepts and edges denoting semantic relations. However, unlike previous formalisms, MSL leverages solely explicit semantic relations without relying on external repositories (e.g., PropBank or VerbAtlas), and avoids using the English lexicon and English structures as an interlingua representation. MSL moves away from verbal-predicate-oriented representations (e.g., AMR or UMR) where nominal structures and predicates are represented with verbal
predicates. Following BMR, MSL leverages explicit semantic relations that are consistent across languages, making them understandable without any repository and simplifying the data generation for non-English languages. For example, we establish new nominal relations – such as :similar, :compared, or :related – to enable nominal predicate representations without a repository.1 Furthermore, MSL leverages the sentence’s native lexicon and structures, simplifying data generation across languages by eliminating the necessity for concept disambiguation in each language. Therefore, unlike AMR or UMR, there is a direct correspondence between graph nodes and sentence spans. For instance, following BMR, the concept "the football player" is represented as it occurs in the sentence, not with two nodes, "person" and "football", related by "play" (Figure 2).
MSL does not attempt to be a unified meaning representation across languages as BMR does, but aims to provide the necessary tools to enable parsing across languages. This is because the semantic structure of parallel sentences across languages might be different, so we cannot use only one of these representations as unification as BMR does (see Figure 3). MSL is an abstraction layer that offers a vanilla representation – not tied to any external repository – in more languages than any previous formalism. As a result, MSL does not provide the complete meaning of text, since it does not link words with their meaning. However, our intention is not to devise MSL as a new formalism, but rather as a semantic layer which can easily be integrated with other NLU layers, such as Word Sense Disambiguation (WSD), Entity Linking (EL), and
1Appendix Section B.2 explains all the relations.
Entity Typing (ET), which connect the concepts to external knowledge bases (e.g., Wikipedia, WordNet, BabelNet, etc.). Therefore, our goal with MSL is to provide not only a readily available representation and multilingual open-access dataset, but also a stepping stone towards a more flexible semantic representation upon which to build narrower or more specialized representations, improving the broader research community’s utilization of Semantic Parsing, even in low-resource languages. Figure 3 illustrates an example of an MSL graph in different languages.

heading: 3.2 MSL Dataset
text: In this section, we discuss the process of creating a corpus for MSL. First, we explain how we semi-automatically produced a preliminary silver annotation (MSLAMR). Second, how we used this data to train a model and predict annotations over a vast amount of data across languages (MSLsilver). Third, how our annotators corrected the predicted graphs, and we projected these corrections to our corpus using LLMs (MSLHQ). Fourth, how we used LLMs and parallel corpora to project our annotations to other languages (MSLHQE). Finally, how we created 1,100 pairs of sentence graphs from scratch across ten languages, all within an academic budget.
Silver Dataset Creation Our initial goal was to obtain annotations of sentences in various languages. We began with the AMR 3.0 corpus – the largest AMR corpus, containing 59,255 English sentence-graph pairs – to generate our preliminary silver dataset, MSLAMR in English, French, German, Italian and Spanish. Firstly, 1 we translated the English sentences into French, German,
Italian and Spanish using DeepL (DeepL GmbH, 2024). Then, 2 in order to align each language’s sentence concepts with their respective English graph nodes, we employed the method proposed by Martínez Lorenzo et al. (2023), training a crosslingual AMR parser using the DeepL translations and English graphs, and leveraging the model’s cross-attention to extract alignments. We transitioned from PropBank argument relations to our cross-lingual and non-frame-dependent relations through manual mapping, integrating MSL’s conceptual alterations with AMR. This process involved creating mappings and specific heuristics for modifying nominal structures.2 MSL has a direct correspondence between sentence concepts and graph nodes, unlike AMR or UMR. Therefore, we have to identify multi-word and idiomatic expressions in the sentence, and – leveraging the previously extracted alignment – collapse their related graph nodes to represent them as single concepts. However, linguistic divergences often lead to variations in conceptual representation across languages. For example, Figure 2 shows how "jaywalk" is translated into "cruzar de manera imprudente" in Spanish, necessitating different graph representations. To address this, we devised a heuristic based on the alignment, the graph structure and a set of multi-word expressions in each language, that was capable of splitting or merging graph nodes when required.3 These modifications comprise the last step 3 in creating MSLAMR.
Generating the Large Corpus MSLsilver Our MSLAMR is still under the AMR’s license and uses silver translations. Consequently, our aim is
2Appendix B provides all the details. 3Appendix C explains all details of this process.
to annotate a substantial corpus using non-AMR licenced data, MSLsilver. We trained a seq2seq SP parser in a multilingual fashion using mT5-large (Xue et al., 2021) with MSLAMR.4 Subsequently, 4 we generated MSLsilver in German, English, Spanish, French, and Italian gold sentences by predicting from two sources: i) a parallel corpus from various OPUS datasets (Tiedemann, 2009) such as TED, OpenSubtitles, Ubuntu, Bible, and Books; and ii) a non-parallel corpus, from Wikipedia.
Manual Validation 5 At this stage, we have millions of multilingual annotations that are free from licensing restrictions, albeit still silvergenerated. To enhance their quality, we engaged proficient annotators in AMR and trained in MSL to manually validate a subset of the graphs in each language.5 Throughout this process, these annotators rectified numerous errors within MSLsilver, integrated previously absent elements from the original AMR corpus (such as tense, aspect, mood, and modality for verbs), and revised structural representations. This process ended up correcting around 2,000 graphs, 400 per language.6
Replicating Changes 6 After rectifying errors and updating structures in the manual validation phase, we had to replicate all our corrections and changes across the five languages to the rest of MSLsilver. Previous studies have demonstrated the efficacy of LLMs as zero-shot annotators (Zhang et al., 2023), offering human-comparable labelling at lower costs (Gilardi et al., 2023; Zhu et al., 2023). Leveraging this and given the task’s sim-
4See Appendix G for more details. 5Each annotator was a native-speaker of their language. 6Appendix D explains this process and the annotators
guidelines in detail.
plicity compared to parsing from scratch, we used GPT-3.5 to apply our corrections corpus-wide. The model was tasked with generating accurate graphs given as input sentences and their corresponding silver graphs. Initially, a few-shot approach with example prompts proved ineffective due to inconsistency with our MSL structure and high operational costs due to long prompts. Consequently, we fine-tuned the model with our comprehensive set of corrections in German, English, Spanish, Italian and French. The trained model is given the original sentence and silver graph before corrections, and tasked to output the corrected graph. We validated this approach on a corrections subset, assessing performance using perplexity.7 With the model fine-tuned, we efficiently replicated all corrections across the corpus, producing the MSLHQ in German, English, Spanish, Italian and French. The effectiveness of this approach will be further validated by our ablation experiments.
Projecting to Other Languages 7 Having developed a high-quality dataset in five languages, we address the data scarcity in other languages by generating high-quality annotations in Arabic, Catalan, Korean, Galician, Portuguese and Chinese (Table 2 shows stats). Under the premise that projecting annotations across parallel languages is simpler than starting from scratch (Barba et al., 2020; Daza and Frank, 2020; Blloshmi et al., 2020), we leverage our dataset’s parallel nature to project them. As
7Appendix E shows examples and performance details.
depicted in Figure 3, language divergences across translations for the same sentence should be reflected in the graph. Therefore, we again fine-tune GPT-3.5 using a subset of MSLHQ that has parallel sentences in German, English, Spanish, Italian and French. Given a sentence, its semantic graph and the parallel sentence in another language, the model has to generate the corresponding graph. We use English as the source language with Arabic, Chinese and Korean as the targets during inference. Then, Spanish served as a pivotal source language for Catalan, Galician, and Portuguese, due to their linguistic proximity.8 The total cost for training and inference to apply our corrections and project to other languages was around 400$, 0.0013$ per graph. For comparison, the license to use AMR 3.0 alone is 300$, 0.005 per graph.

heading: 3.3 Manually Annotated Test Set
text: To evaluate our dataset’s quality and establish a gold standard benchmark, we tasked annotators to manually annotate 100 parallel sentences from OPUS in Arabic, Catalan, Chinese, English, French, Galician, German, Italian, Korean, Portuguese, and Spanish.9 Recognizing the high cost of manual annotation, we reserved this task exclusively for benchmark creation, ensuring our annotators were proficient in generating MSL graphs for all the listed languages.10 Moreover, we could
8Appendix F shows some data examples and more detail. 9The annotators are native speakers in each language.
10Appendix D explains the annotation guidelines.
calculate the inter-annotator agreement in Spanish – 92.34 in SMATCH – as it was the only language with native speaker overlap in the graphs (Galician and Catalan annotators are also native Spanish speakers). We also reserved 1700 sentences per language from OPUS to test our systems for backtranslation, and these sentences are not present in any of our training sets.

heading: 4 Experiments

heading: 4.1 Experimental Setup
text: Tasks Our evaluation framework consists of two parts: i) Parsing and Generation to ablate each annotation step, and ii) Back Translation Experiment to compare MSL against AMR and BMR.11 The objective of Parsing – transforming the text into a graph representation according to a specific formalism – is to assess the complexity of generating MSL graphs due to the nuanced characteristics of MSL. Then, the Generation task – producing the original text from the graph representation – has the goal of appraising the effectiveness of MSL for preserving information. Finally, we perform a Back Translation Experiment – first parse and then generation – to compare the suitability of SP datasets, such as AMR 3.0, BMR 1.0, and MSL dataset, for training semantic parsers across languages.
Datasets For the Parsing and Generation study, we use the four training sets described in the paper (MSLAMR, MSLSilver, MSLHQ and MSLHQE), for the test we use the gold test set annotated in Section 3.3, that comprises 1,100 sentence graph pairs, 100 sentences in each language (Arabic, Catalan, Chinese, English, French, Galician, German, Italian, Korean, Portuguese, Spanish). Then, for back-translation, we use our MSLHQE and, to train the other models, we use AMR 3.0 (Knight et al., 2020)12 and the BMR 1.0 (Martínez Lorenzo et al., 2022). However, since AMR 3.0 provides only English sentences, we use the DeepL translations from
11The only formalisms with available parsers to train. 12AMR 3.0 is licensed by LDC at https://catalog.ldc.
upenn.edu/LDC2020T02
1 . As test data we use the parallel sentences from Abstract Meaning Representation 2.0 - Four Translations,13 that translated the AMR 3.0 test set into German, Italian and Spanish (consisting of 1, 371 sentence in each language). Furthermore, we use parallel sentences from the OPUS corpus to test the performance in non-AMR data (Out-of-domain), which was not included in MSL. To maintain a consistent number of training samples across datasets, we reduce MSLSilver to MSL HQ Silver by selecting only those sentence graphs that match the sentences in MSLHQ (168,315).
Models For training the parsers across formalisms and languages, we leverage CLAP (Martinez Lorenzo and Navigli, 2024), where the parsing task is framed as a seq2seq task for an EncoderDecoder system, where the model is trained to generate a linearized version of a graph from a sentence and, vice versa, produce a sentence from a linearized graph. We adapt the model’s vocabulary to include our set of relations and use mT5-large as the underlying language model, which supports all our languages.14 First, given the datasets described in Section 4.1, we train a model for each dataset, AMR 3.0 and BMR 1.0 for each language (DE, EN, ES, FR, and IT). Then, for MSL we train four models on, respectively: i) MSLAMR (DE, EN, ES, FR, and IT); ii) MSLHQSilver (DE, EN, ES, FR, and IT); iii) MSLHQ (DE, EN, ES, FR, and IT); and iv) MSLHQE (AR, CA, DE, EN, ES, KO, FR, GL, IT, PT and ZH).
Evaluation Measures We evaluate Generation and Back-translation by using the standard NLU metric BLEU (Papineni et al., 2002). For Parsing, we employ the SMATCH measure (Cai and Knight, 2013), which is the most famous metric for AMR parsing. The SMATCH calculates the maximum overlap between the predicted and reference graphs. The SMATCH also works with MSL, since it follows AMR theory.
13https://catalog.ldc.upenn.edu/LDC2020T07 14Appendix G provides more details and hyperparameters.

heading: 4.2 Results
text: Parsing and Generation In Table 3, we observe that MSLSilver achieves results comparable to MSLAMR, underscoring the success of distilling MSLAMR. MSLHQ obtains better results across all languages compared to MSLsilver in both directions (parsing and generation), showcasing the benefits of cleaning the data thanks to our automatic corrections. Furthermore, the benefits of MSLHQE are clear; expanding the dataset with additional languages enables semantic parsing capabilities in these languages. Notably, MSLHQE demonstrates a much smaller variance compared to other models. For comparison, the current multilingual state-ofthe-art AMR parser by Cai et al. (2021) has a 9 SMATCH points gap between English (83.9) and Spanish (75.9), German (73.1) or Italian (75.4), while in MSL the gap is significantly smaller – considering that SMATCH values are not comparable across formalisms.15 The text generation results show even more dramatic improvements compared to Martínez Lorenzo et al. (2022): AMR or BMR never reached more than 37 BLEU on non-English languages, while English attained 50 BLEU points. This points to MSL being a more expressive semantic layer, which benefits text generation.
Back Translation Experiment In Table 4, we can observe the effectiveness of different parsers in preserving information when back-translating from and into the same language passing through the graph using a cross-lingual model. We have to clarify that Martínez Lorenzo et al. (2022) reported BLEU scores of 45.3 for AMR and 50.1 for BMR in English using a monolingual model, unlike our experiments, which use a shared crosslingual model for all languages. We can observe how the model trained with the MSL dataset out-
15Explained in Appendix I.
performs not only in the AMR benchmark against the model trained with this specific data by 19 and 15 BLEU points, respectively, but also maintains the same performance across the Out-of-Domain, where models trained in AMR 3.0 and BMR 1.0 drop around 10 points in average. Moreover, the gap in non-English languages is more pronounced, since AMR and BMR’s dependence on an English graph structure as the interlingua introduces semantic parsing inaccuracies similar to those encountered in machine translation. This attests to the quality, diversity, and size of the MSL dataset across languages compared to the previous dataset. Additional metrics are presented in Appendix H.

heading: 4.3 A Case Study
text: To show the limitations of previous formalisms in encoding the nuances of each language, consider the next example from the TED parallel corpus:
English: We don’t really stop to think about a raindrop the size of an actual cat or dog when we hear ’it’s raining cats and dogs’, but as soon as I do, I realize that I’m quite certain the dog has to be a small one – a cocker spaniel, or a dachshund – and not a golden Lab or Newfoundland.
Spanish: No pensamos en gotas de lluvia del tamaño de un cántaro cuando escuchamos ’llueve a cantaros’, pero al hacerlo, nos damos cuenta que el cántaro debe ser uno muy pequeño; un botijo, un tarro, y no ollas con asas laterales.
This example highlights how formalisms like AMR and UMR will struggle with idiomatic expressions, often leading to literal interpretations (e.g., rain-01 :theme (and :op1 (cat) :op2 (dog))). Additionally, formalisms like BMR work under the premise that parallel sentences in different languages should have the same representations and
consequently fall short in practice. A major flaw in BMR 1.0 is its reliance on English as the interlingua graph, failing to capture nuanced semantics across non-English languages effectively. Although BabelNet links the English lemmatization of "raining cats and dogs" with the Spanish "llueve a cántaros", the English sentence associates the phrase with specific dog breeds, while the Spanish refers to types of jars ("botijo"), leading to fundamentally different conceptual graphs even though the idioms are equivalent across languages. This underscores the limitations of using a single language-based representation as a universal interlingua for comprehensive multilingual semantic understanding. It reveals how language-specific divergences cause different languages to represent ideas or concepts uniquely – highlighting that human languages are designed to model ideas and concepts, and not the other way around.
This underscores our approach: (i) viewing MSL not as an interlingua but as a layer for semantic representation across languages, not focused on representing the same idea equally across languages, and (ii) refining graph structures using an LLM, acknowledging that parallel annotations can vary significantly, thus requiring more than just algorithmic adjustments as seen in the BMR 1.0 dataset.

heading: 5 Potential and Future of MSL
text: In the era of LLMs, we aimed to revamp the dream of structuring natural text across multiple languages through the use of MSL. This work opens up new possibilities for applying graph-based and big-data analyses to multilingual content. We envision the potential of MSL as follows:
• NLP Task Integration: MSL can be integrated with other resources like BabelNet to generate BMR-like graphs and with predicateargument structures to yield AMR and UMR representations. Furthermore, MSL could be integrated with Entity Linking, Entity typing, or Word Sense Disambiguation, among others, to create a full semantic representation.
• Addressing Data Scarcity: The semantics of sentences remain consistent across formalisms, though they are represented differently. Consequently, the MSL dataset could be utilized for transfer learning. This involves pre-training a model on MSL and subsequently fine-tuning parsers for other formalisms. This approach reduces the need for extensive data development by allowing parsers to be fine-tuned using smaller, more specific datasets.
• Linguistic Expansion: With current LLMs, we can expand MSL to include more languages, thereby continuing to bridge the semantic parsing gap across different linguistic contexts.
• Parsing Efficiency: MSL enables the shift from encoder-decoder to encoder-focused architectures, thanks to the one-to-one correspondence between sentence concept and graphs nodes, reducing computational demands and accelerating processing speeds.

heading: 6 Conclusion
text: This paper addresses the challenge of data scarcity in multilingual SP. Given the complexity and cost of training annotators for large-scale tasks, we introduce i) the MSL layer and ii) provide a large, high-quality corpus across languages. The layer decouples SP from Word Sense Disambiguation and Entity Linking, focusing on the extraction of sentence-level semantic relations. Then, we automatically develop a preliminary silver dataset across languages, refining it through manual corrections and propagating these annotations using fine-tuned GPT LLMs. We also manually annotated 1,100 sentences in 11 languages, including low-resource ones, to demonstrate the flexibility of our layer and dataset over previous ones. Our code and dataset are available at https://github. com/SapienzaNLP/MSL.

heading: 7 Limitations
text: While we lower the cost burden of manually annotating training corpora for SP by leveraging LLMs, expanding to further languages would still require manual annotation to create new evaluation sets. Furthermore, while we show the effectiveness of relying on LM and LLMs to aid in annotation and propagating manual corrections, these still rely on access to either local computing or remote computing via third-party APIs. Nevertheless, as discussed in Section 3.1 our approach is still more affordable than previous SP annotation attempts. We default to external APIs for LLMs in only two steps of our dataset creation, a resource that we will release publicly, and therefore a step that does not require to be replicated. All Parsing and Generation models are trained with open-access models and therefore can be trained and used in inference by anyone with enough computing to do so. Therefore, this may pose a limitation if there are changes to OpenAI API and someone wants to replicate or expand to new languages adopting the same approach. We were limited to single-GPU experiments with 24GB of memory, so we opted for an external API rather than compromising quality with smaller models. However, we are confident a similar approach could be achieved using locally available LLMs like LLaMA (Touvron et al., 2023) or Mistral (Jiang et al., 2023).

heading: Acknowledgments
text: The authors gratefully acknowledge the support of:
• CREATIVE project (CRoss-modal understanding and gEnerATIon of Visual and tExtual content) funded by the MUR Progetti di Ricerca di Rilevante Interesse Nazionale programme (PRIN 2020).
• FAIR project (Future Artificial Intelligence Research), the PNRR MUR project PE0000013-FAIR.
• Marie Skłodowska-Curie project Knowledge Graphs at Scale (KnowGraphs) No. 860801 under the European Union’s Horizon 2020 research and innovation programme.
• French Agence Nationale pour la Recherche, through the SELEXINI project (ANR-21CE23-0033-01).
• DrEAM (Doctor, Explore and Achieve More) mobility program by the Lorraine Université d’Excellence (LUE).

heading: Special Abelardo Carlos Acknowledgments
text: As I conclude my doctoral journey, I am immensely grateful to everyone who has supported me throughout this period. Foremost, I extend my deepest appreciation to my family: my parents, José Abelardo Martínez Férriz and Matilde Lorenzo López; my sister, Miriam Martínez Lorenzo; and my relatives, Irene Pérez Lorenzo, Raquel Pérez Lorenzo, Maricarmen Lorenzo López, Matilde López Aguilar, Manuel Lorenzo Castilla, Rafael Ortega Ríos, María Consuelo Martínez Férriz, Rafa Ortega Martínez, and Consuelo Férriz.
I am also profoundly thankful to the Sapienza NLP Group for their invaluable guidance and support, with a special mention to Pere-Lluís Huget Cabot for his help and friendship.
My gratitude extends to all my friends who supported me during this chapter of my life—especially those I met in this period, DesBariaos and Napoleone (Matheus, Nico, Gonzalo, Eva, Alain, Ivan and Anna), who have been pillars of support throughout my PhD journey.
Lastly, I would like to give a special acknowledgement to Marcela Soares Reed for her enduring support and encouragement in this process.

heading: A BMR Issues
text: While BMR represents a significant advancement in cross-lingual representation, it encounters several challenges that limit its effectiveness and scope:
• Dependency on BabelNet: BMR relies on the BabelNet repository, which is not open-source. This reliance on a proprietary resource limits accessibility and modifiability for researchers and developers.
• Limited scope of concepts and languages: The number of concepts and languages covered by BabelNet is restricted. This limitation constrains the representational breadth of BMR, potentially leaving out diverse linguistic and cultural nuances.
• English-centric development: The corpus for BMR is derived automatically from the AMR (Abstract Meaning Representation) corpus, which is fundamentally English-centric. Moreover, the AMR corpus itself is not publicly available due to licensing restrictions, which further limits the adaptability and usability of BMR in diverse linguistic contexts.
• Challenges with LM-based parsers: Language Model (LM) based parsers used in BMR have limited capabilities in integrating new BabelNet concepts. To compensate for unavailable concepts, these parsers revert to using an English lexicon, which may not accurately represent meanings across different languages. abelNet repository, which is not open-source. This reliance on a proprietary resource limits accessibility and modifiability for researchers and developers.
• Complex annotation process: The annotation process for BMR is significantly more complex than that for AMR, posing a barrier for scalability and ease of use in linguistic research and applications.

heading: B MSL Relations
text: As noted earlier, AMR leverages coarse-grained frames and argument structures from the English PropBank within OntoNotes, a resource-limited to English. These frames denote predicate-specific semantic relations and are often unclear without a gloss. For instance, the subgraph in Figure 6
representing "Malaga CF football player" centers around the frame have-org-role-91, with relations :ARG0, :ARG1, and :ARG2 indicating the person, the organization (Malaga CF), and the role (football player), respectively, tying the argument’s meaning directly to the predicate. However, language-specific repositories akin to PropBank, used for annotating non-English sentences, lack a precise one-to-one frame correspondence, complicating direct mapping. We introduce our explicit relations for universally applicable semantics in order to address language specificity.
For constructing the MSL dataset, a linguist16
manually mapped PropBank arguments to our relations, replacing original AMR frames and roles (e.g., mapping write-01’s ARG0 to agent and ARG1 to theme). For non-verbal and special predicates in PropBank that AMR employs for unique semantic structures (like have-org-role-91), we mapped these to our new nominal structures, enhancing their argument representation (Section B.1 lists the MSL semantic roles, and Section B.2 provides mapping examples. Figure 7 illustrates the modified AMR graph from Figure 6.

heading: B.1 Semantic Roles
text: Our new set of relations (Table 5) is adapted to include non-verbal entities, drawing on property
16Annotators possess proficient English skills and were compensated according to their local standards.
lists from existing literature (Leone et al., 2020). Like AMR, each relation in our framework has an inverse, indicated by appending _of (e.g., :purpose becomes :purpose_of ). Roles from AMR not covered in Table 5 are retained in MSL, including :degree, :frequency, and :manner. Figure 7 shows the AMR of Figure 6 after moving to our relations.

heading: B.2 Nominal Structures
text: Non-verbal predicates and special predicates found within AMR 3.0 have been mapped to the set of semantic relations described in Section B.1 by means of an in-house annotation interface. See Table 6 for an AMR 3.0 to MSL sample. Figure 8 shows the AMR of Figure 7 after adapting to our new nominal structures.

heading: C MSL Concept Representation
text: As mentioned, unlike AMR or UMR, which decompose concepts into multiple nodes using OntoNotes for semantic encoding (e.g., representing a "Football player" as a "person who plays football"), MSL employs single nodes for concept representation. We detail the transition from AMR 3.0 to our dataset, focusing on node merging in Section C.1. The language evolves to model the language, so each language models concepts differently, so rather than conforming to a single language’s structure for knowledge representation across languages – such as BMR with the English structures to encode meanings universally (e.g., using a single node to represent the concept of "jaywalk" across languages, even when a direct equiv-
alent does not exist) – MSL adapts to language divergences. When a concept in one language is expressed through multiple concepts in another, we divide the nodes accordingly. For details on this process, refer to Section C.2, where we explain our methodology for adapting AMR 3.0 to our dataset.

heading: C.1 MSL Merging Concepts
text: In AMR, single concepts and multiword expressions, including idioms, are often decomposed into multiple nodes through node composition. However, two main issues arise: firstly, single concepts are split to utilize external repositories for encoding relations. For instance, the expression "football player" in AMR 3.0 is depicted using three nodes as:
(p / person :ARG0− of (p2 / play :ARG1 (f / football)))
This representation implies "a person associated with Malaga CF who plays football," where (i) the football concept is not directly linked to Malaga, and (ii) merely playing football does not equate to being a football player. MSL addresses these issues by merging such nodes into a single concept, "football player," directly associated with Malaga CF, thereby clarifying that (i) football is related to Malaga and (ii) the individual is a football player rather than just someone who plays football. Additionally, the reliance on external repositories makes
have_org_role.91 AMR: ARG0 (entity); ARG1 (organization), ARG2 (role) MSL: ARG2 :membership ARG1 AMR: person :ARG0-of (have_org_role.91 :ARG1 (university) :ARG2 (professor)) MSL: professor :membership (university)
be_located_at.91 (reification of :time) AMR: ARG1 (entity); ARG2 (location) MSL: be :theme_of (ARG1) :location (ARG2) Example: I am in the cinema AMR: be_located_at.91 :ARG1 (I) :ARG2 (cinema) MSL: be :theme (ARG1) :location (ARG2)
good.02 (generally positive: morally good, pleasing) AMR: ARG1 (generally positive/pleasing entity); ARG2 (recipient/target of good behavior) MSL: be :theme (ARG1) :attribute (good) :target (ARG2) Example: My mother is good to me. AMR: good-02 :ARG0 (mother) :ARG1 (I) MSL: be :theme (mother) :attribute (good) :target (I)
Table 6: Mapping examples from AMR 3.0 to MSL. Each row block lists (top to bottom) original OntoNotes predicate names and glosses, original glosses for the predicate arguments, and predicate rendering in MSL. Then, an example sentence with the AMR and MSL.
these relations non-deterministic, varying with the availability and nature of the repository. For example:
(r / footbal_player)
Furthermore, AMR’s approach to splitting multiword or idiomatic expressions into multiple nodes can lead to a literal, rather than meaningful, representation. For example, "rains cats and dogs" in AMR might be represented as:
(r / rain− 01 :ARG2 (a / and :op1 (c / cat):op2 (c / dog)))
This fails to capture the idiom’s intended meaning. Like BMR, MSL consolidates such expressions into a single concept within the graph, ensuring a more accurate representation of idiomatic or complex expressions, exemplified by treating "football player from Malaga CF" as a unified concept. Figure 9 shows the AMR of Figure 8 after adapting to our new nomical structures.
This section details how we transformed AMR 3.0 into MSLAMR by merging nodes.
Multiword Expression Identification We begin by identifying single words or multiword expressions within sentences across English and its parallel translations in German, Spanish, French, and
Italian, which are represented by multiple nodes in the AMR graph. We compile a list of multiword and idiomatic expressions from open websites in these languages. Using SpaCy 3.1, we lemmatize sentences in each language.
Manual Validation Automatic detection of multiwords can lead to errors, such as incorrect node mergers and sense assignments. To address this, an expert linguist manually inspects all multiword instances identified in AMR 3.0, deciding on their retention, modification, or removal.
Lemmas projections After finalizing the multiword list and alignments, we segmented the sentence spans having clusters of single concepts, then we projected the lemmas of the cluster to their nodes and merged nodes in AMR graphs that refer to the same word or multiword expression. This bottom-up approach starts with the leaves, moving towards the root, to represent phrases like "football
player" or "rains cats and dogs" as single concepts.

heading: C.2 MSL Splitting Nodes
text: Unlike previous formalisms such as AMR, UMR, BMR, and PMB, our approach does not rely on English as an interlingua, thereby addressing language divergences more effectively. The challenge arises because existing annotations are primarily based on English, using the language to model knowledge rather than the reverse. Consequently, we cannot apply a single language’s structure to represent knowledge across multiple languages. For instance, BMR encodes meanings using English structures, such as representing the concept of "jaywalking" with a single node, even though a single concept representation does not exist. In contrast, MSL utilizes parallel translations, splitting nodes when a concept is represented by multiple concepts in other languages, ensuring a more accurate cross-linguistic representation.
Starting with sentences segmented by concepts in each language, we identify instances where multiple clusters correspond to a single node and, where this is the case, proceed to split the nodes. Our initial step involves creating a heuristic that examines the part-of-speech (POS) tag of each element, alongside the arguments already utilized by the node, followed by a manual classification to determine the node encoding for such cases. For instance, the English term "jaywalk" can have several translations in Spanish, such as "cruzar la calle imprudentemente" or "cruzar la calle de manera imprudente". Here, the "jaywalk" node aligns with "cruzar" and either "imprudentemente" or "imprudente", representing two distinct clusters in Spanish. The node is split based on the POS tags (verb "cruzar" and adverb "imprudentemente"), considering existing relations like ":agent", ":theme", and ":destination". In cases where the main node is a verb and the secondary is an adverb or adjective, with these relations present, we use the ":manner" relation to specifically modify the verb action.
(c / cruzar :manner (i / imprudentemente))
Figure 2 shows an example of the final representation after doing all the heuristics. We acknowledge that while this heuristic works in many cases, it may introduce errors due to the inability to capture all linguistic nuances, which are addressed during the manual validation process.

heading: D Manual Validation
text: After employing our model to produce a substantial dataset with our parser trained on MSLAMR, we acknowledge that these silver-generated graphs, despite being effective in many cases, may not fully capture the linguistic nuances across languages due to heuristic limitations in MSLAMR’s creation. Consequently, we extracted a subset of these multilingual graphs for manual validation, aiming to refine them further before using LLMs to apply corrections across the larger dataset. This validation process focused on three key areas: i) refining the graph structure, ii) adding missing information from AMR, and iii) modifying specific encodings and introducing new relations based on insights from real examples.

heading: D.1 Refining Graph Structure
text: Since we have silver-generated graphs created from a semi-automatically process (starting from a gold graph, we transform this), there are some errors in our silver data. Therefore, we analyzed a subportion of this data to find graphs with structural errors due to the fact that our model did not learn to properly represent this knowledge. Therefore, we ask proficient annotators to manually validate this subportion, restructuring the graph if necessary. The main errors found were i) the incorrect identification of multi-word expressions, ii) the use of incorrect relations, and iii) generating corrupted structures.
D.2 Incorporating Missing Information
Although AMR effectively encodes textual information within its semantic structures, it overlooks essential word components like number, tense, aspect, and mood, crucial for fully grasping meaning—a gap highlighted in existing literature (Bonial et al., 2019). Recognizing the significance of these grammatical categories, we integrate these features to bolster the representational capability of our formalism.
During the annotation validation process, annotators across five languages are tasked with identifying and manually incorporating tense, aspect, number, and mood. Acknowledging the diverse mechanisms languages employ to convey this information, we provide tools for each language so as to facilitate encoding, with subsequent mappings between representations (as detailed in Table 8). For instance, we enhance verb nodes with a
:tense role indicating future or past (defaulting to present for its absence) and an :aspect relation marking imperfect, continuous, perfect, or perfectcontinuous aspects. The :mood relation captures the verb’s mood—subjunctive, conditional, imperative, or by default, the indicative. Additionally, the plurality of nouns is indicated by adding a :quantity relation with a + value for plurality.

heading: D.3 Modifing Encoding Structures
text: The simplicity of our layer, allows new forms to be easily evolved just by incorporating new relations. Since it is impossible to understand all the nuances of a language without going to the data, we allow the annotators to incorporate new annotations only when strictly necessary, such as when the previous forms do not fit properly or because it would allow easier use in the future.
Verb To Be In many formalisms, auxiliary verbs like "to be" and "to have" are not directly encoded but represented through relations (e.g., :domainin
AMR or :identity in BMR). However, such annotation methods struggle to encapsulate negations or probabilities within the node itself (e.g., "he is not here"), leading to the creation of new nodes for situations necessitated by negations, among other reasons (e.g., the inclusion of nodes like be-locatedat-91 in AMR). This approach to reification results in inconsistent annotation practices, as it allows for multiple representations of similar concepts. To address this inconsistency, we have opted to explicitly include the verb "to be" as a distinct node in our formalism, requiring annotators to manually adjust this structure when encountered. However, in instances of passive voice, we do not explicitly separate the nodes, treating sentences in direct voice instead. We maintain that at our level of abstraction, there is no semantic distinction that necessitates differentiating between passive and active voices, as "to be" serves merely as an auxiliary verb.
Mode Verb Structure In traditional formalisms, the possibility of an action is represented using a specific node, such as the possible-01 predicate in
AMR. For instance, "I can read" in AMR might be depicted as:
(p / possible− 01 :ARG1 (r / read− 02 :ARG0 (i / I)))
In our approach, aiming to align concepts more closely with sentence lemmas without relying on an external repository, "I can read" is represented as:
(p / can :theme (r / read:agent (i / I)))
However, "can" serves as an auxiliary verb modifying the main verb, leading to various potential representations, such as "may" or "might". To address this, annotators refined the encoding to specifically denote the modal aspect of the verb, as in:
(r / read :mode possible
:agent (i / I)))
This approach has been extended to other modalities, including Necessity, Obligation, Recommendation, Possibility, and Permission, enhancing clarity and consistency in representation.
Once all graph subsets have been validated, we retain the incorrect silver instances, as they will be used to train a large language model. This model will then apply our validations across the entire silver corpus.

heading: E Projecting Corrections
text: After completing the manual validation phase rectifying errors and updating structures, our next step, 6 of Section 3.2, involved replicating these corrections and changes across all five languages within the MSLsilver dataset. As outlined in Appendix D, tasks such as incorporating missing information and modifying encoding structures are relatively
straightforward for humans, requiring only identification of the corresponding sentence part in order to integrate the concept into the graph. However, the linguistic nuances and the need to apply these changes across languages render the process impractical for algorithmic execution. To overcome this, we leveraged Large Language Models (LLMs), which have proven effective as zero-shot annotators for tasks that are conceptually simple and well-supported by sufficient examples.
Initially, we utilized a few-shot approach with GPT-3.5, providing the model with instructions and examples of the modifications needed. While this method sometimes led to accurate captures of meaning and appropriate changes, it often resulted in structural disruptions within the graph, deviating from our MSL framework and generating incorrect graphs. Furthermore, the necessity for a large number of examples in order for effective learning to take place led to unwieldy prompts and the inability to encompass all modification examples in a single prompt, thereby necessitating extensive examples and, in turn, leading to significant costs.
Therefore, we shifted to fine-tuning GPT-3.5 using the silver graphs, the corresponding sentences, and our corrections across German, English, Spanish, French, and Italian. This approach does not require a high volume of examples in order for GPT-3.5 to grasp the task, making it an efficient method for replicating human-like corrections. Subsequently, we validated this refined method on a subset of corrections, assessing its effectiveness through perplexity measures.
Table 9 outlines the hyperparameters fine-tuned in GPT alongside the final losses, indicating these are the only parameters available for fine-tuning. Notably, Experiments 1 and 2 exhibit a higher validation loss than the training loss, suggesting overfitting. This outcome may stem from training for 3 epochs with only 2,000 samples (400 corrections per language), which, despite GPT-3.5’s efficiency with fewer examples, proves insufficient. Experiments 5, 6, and 7 achieved lower validation losses,
which can be attributed to increased batch sizes, which reduced the number of steps and enhanced model generalization.
Delving into specifics with Table 10, Experiment 1 attempted to include two types of message: one with all possible graph relations and another without. The model was tasked with generating the correct graph from the original sentences, leading to high computational costs and poorer performance compared to Experiments 2 and 3. These latter experiments opted for a simplified message format, excluding extraneous relations, which improved outcomes.
Experiments 4 and 6 introduced the silver sentences as input to simplify the task, prompting the model to replicate modifications in the graphs. This approach improved performance over previous experiments. Experiment 5, building on Experiment 4’s prompts by adding more task context, unfortunately, increased computational costs and worsened losses. Experiment 7 aimed to minimize redundant spans in graph linearization to reduce decoding tokens, but this led to poorer performance, which is hypothesized to be due to increased difficulty in understanding the graphs. This suggests GPT-3.5’s familiarity with AMR linearization might be leveraged, as it likely forms part of its training corpus.

heading: F Projecting Annotations to Other Languages
text: Here we give more details for step 7 of Section 3.2, where the corrected graphs created in the previous step are projected to other languages by leveraging the parallel corpus. For any given pair of sentence and graph in English, German, Spanish, French or Italian in our parallel data, we have a sentence in Arabic, Korean, Catalan, Chinese, Galician and Portuguese. To finetune GPT3.5 for the task we follow a very similar approach as 5 , where the LLM receives as input the sentence and graph in the same language, but here it also receives the sentence in another target language and instead of generating a corrected version of the graph, we prompt the model to output the graph in the target language. Table 11 shows an example. At train time we use 400 corrected graphs per language in English, German, Spanish, French and Italian to create pair-wise training samples per pairs of languages, either as source or target, creating 3,960 training samples. At inference time we set English as the source, and Arabic, Korean, Catalan, Chinese, Galician and Portuguese as target ones to obtain their annotations for MSLHQE .

heading: G Architectures used
text: Throughout our experiments two kinds of architectures are used. All our LLM-based annotations, used in steps 6 and 7 of Section 3.2 are based on GPT3.5, finetuned using OpenAI API in which only the number of epochs and batch size can be defined and are reported in Table 9. For the rest of the experiments described in this paper, we employ CLAP (Martinez Lorenzo and Navigli, 2024), an efficient implementation of SPRING (Bevilacqua et al., 2021), where the parsing task is framed as a seq2seq task for an Encoder-Decoder system. In our case, we use mT5-large (Xue et al., 2021) as the underlying LM with an expanded vocabulary with the necessary extra tokens specific to our MSL
role: "system", content: "You are a Multilingual Semantic Parser." role: "user", content: "English Sentence: We say: It’s raining cats and dogs out. English Graph: (z1 / say :agent (z2 / we) :content (z3 / rain_cats_and_dog :aspect continuous)) Spanish Sentence: Decimos: Llueve a cántaros’. Spanish Graph:" role: "assistant", content: "Spanish Graph: (z1 / decir :agent (z2 / nosotros) :content (z3 / llover_a_cantaros))"
Table 11: Example of prompts for training GPT to project to other languages. For predicting is the same format but with the role assistant message.
layer, as SPRING did for AMR. For all experiments we use the Adam (Kingma and Ba, 2015) optimizer with a 5x10−5 learning rate, 2048 token batch size and a beam size of 5 at inference time. We train every model on a single NVIDIA® RTX 3090 graphic card with 24GB of VRAM. The average time to train each model was 48h.

heading: H More Results Generation
text: Table 12 and Table 13 present additional results for the Generation and Back-translation experiments, using other evaluation metrics: Bertscore (Zhang et al., 2020), Chrf++ (Popović, 2017), and Rouge-L (Lin, 2004). Interestingly, the BLEU score for Arabic in MSLHQE is notably lower compared to other languages, reflecting BLEU’s known limitations for Arabic (Bouamor et al., 2014). However, other metrics such as Bert-score in Table 13 reveal a much narrower performance gap, with MSLHQE achieving the highest performance across all languages.

heading: I More Results Parsing
text: When we compare the SMATCH score of our system trained with MSLHQE (71.35) against the system trained in AMR score using the exact same architecture for English (83.0), we can observe how there are around 12 points of difference. However, as we mentioned in the paper, SMATCH values are not comparable across semantic representations. Our merging nodes and inclusion of multi-word expressions reduce the number of nodes in the graphs, reducing the number of matches between the gold and other reference graphs, and making the error have a bigger impact. For example, in Figure 11 we can see the impact of having an error in an AMR graph structure and having an error on the MSL representation. In the example, we observe how the predicted graph for MSL and AMR wrongly expresses the named entity of "Malaga CF" as "Malaga". However, since AMR tends to
have more graph nodes to represent the same idea than an MSL, the error’s impact in MSL is much higher (We can observe how 11(d) has 6 points lower than 11(c)). Analyzing the node counts in the training AMR 3.0 corpus graphs, AMR graphs contain 641,431 nodes, while MSL graphs for the same sentences have 423.485.

heading: Future Work
text: Since it is impossible to understand all the nuances of a language without going to the data, we allow the annotators to incorporate new annotations only when strictly necessary, such as when the previous forms do not fit properly or because it would allow easier use in the future. Verb To Be In many formalisms, auxiliary verbs like "to be" and "to have" are not directly encoded but represented through relations (e. g. , :domainin
AMR or :identity in BMR). However, such annotation methods struggle to encapsulate negations or probabilities within the node itself (e. g. , "he is not here"), leading to the creation of new nodes for situations necessitated by negations, among other reasons (e. g. , the inclusion of nodes like be-locatedat-91 in AMR). This approach to reification results in inconsistent annotation practices, as it allows for multiple representations of similar concepts. To address this inconsistency, we have opted to explicitly include the verb "to be" as a distinct node in our formalism, requiring annotators to manually adjust this structure when encountered. However, in instances of passive voice, we do not explicitly separate the nodes, treating sentences in direct voice instead. We maintain that at our level of abstraction, there is no semantic distinction that necessitates differentiating between passive and active voices, as "to be" serves merely as an auxiliary verb. Mode Verb Structure In traditional formalisms, the possibility of an action is represented using a specific node, such as the possible-01 predicate in
AMR. For instance, "I can read" in AMR might be depicted as:
(p / possible− 01 :ARG1 (r / read− 02 :ARG0 (i / I)))
In our approach, aiming to align concepts more closely with sentence lemmas without relying on an external repository, "I can read" is represented as:
(p / can :theme (r / read:agent (i / I)))
However, "can" serves as an auxiliary verb modifying the main verb, leading to various potential representations, such as "may" or "might". To address this, annotators refined the encoding to specifically denote the modal aspect of the verb, as in:
(r / read :mode possible
:agent (i / I)))
This approach has been extended to other modalities, including Necessity, Obligation, Recommendation, Possibility, and Permission, enhancing clarity and consistency in representation. Once all graph subsets have been validated, we retain the incorrect silver instances, as they will be used to train a large language model. This model will then apply our validations across the entire silver corpus