Semantically Inspired AMR Alignment for the P ortuguese Language (2020)

Abstract: Abstract Meaning Representation (AMR) is a graph-based semantic formalism where the nodes are concepts and edges are relations among them. Most of AMR parsing methods require alignment between the nodes of the graph and the words of the sentence. However, this alignment is not provided by manual annotations and available automatic aligners focus only on the English language, not performing well for other languages. Aiming to fulfill this gap, we developed an alignment method for the Portuguese language based on a more semantically matched word-concept pair. We performed both intrinsic and extrinsic evaluations and showed that our alignment approach outperforms the alignment strategies developed for English, improving AMR parsers, and achieving competitive results with a parser designed for the Portuguese language.

## introduction according to @xcite , abstract meaning representation (amr) is a semantic meaning representation, which may be encoded as a rooted direct acyclic graph (dag) where the nodes are concepts, and the edges are relations among them. this representation explicitly details semantics information, as depicted in figure 1 . in this figure, the live-01 node is the root of the graph and city node introduces a named entity. moreover, :argx relations are predicates from the propbank lexicon @xcite , which encode semantic information according to each propbank sense. to parse a text into an amr graph, most of the amr parsers require alignment between the word (tokens) of the sentence and the nodes of the corresponding graph (see, for instance, @xcite @xcite . however, this anchoring is not provided by manual annotations. also, the available automatic aligners focus only on the english language @xcite @xcite , and they do not perform well for other languages. for portuguese, for instance, the sentence "não era surpresa para mim" (it was no surprise to me), the jamr aligner @xcite produces alignment only between the token surpresa (surprise) and the node surpresa, as shown in figure 2 . the jamr aligned only the span 2-3, which is the token surpresa (surprise), with node 0, which is the root of the graph; -and eu nodes were not aligned. this wrong or bad alignments occur because of the jamr aligner adopts a string-match strategy that is focused on the english language. besides, these issues contribute to a decrease in the performance of amr parsers. as a result, recent amr parsing methods have focused on alignmentfree approaches @xcite . however, they require a large annotated corpus, which is available only for english. in this context, aiming to bridge this lack of resources and tools for other languages, we propose an amr aligner for portuguese that focuses on a more semantically matched word-concept pair. for that, we use pre-trained word embeddings and the word mover's distance (wmd) function @xcite to match span tokens in the sentence with nodes in the graph. word embeddings capture some semantics information about a corpus, and wmd measures the dissimilarity between two documents even if they have no words in common. with this, it is possible to produce semantically inspired matches instead of only string-match. to evaluate our approach, we carry out both intrinsic and extrinsic experiments on an annotated corpus from portuguese. our aligner produced better alignments than alignment strategies proposed for english and improved amr parsing for portuguese, reaching competitive results with an amr parser designed for that language. the remaining of this paper is organized as follows. in section 2, we briefly introduce the related work. section 3 describes our proposed aligner. in section 4, we conduct some experiments and evaluations and show our results. finally, section 5 concludes the paper indicating future research. figure 4 : alignments produced for the sentence "the boy wants to go". @xcite extended and improved the jamr aligner by adding semantic resources into the rules, such as glove embeddings @xcite and the morphosemantic database @xcite . besides, they noted that the jamr aligner requires that words have at least a common longest prefix of four characters, omitting the shorter cases (as word actions aligned with the concept act-01). thus, their method improved the jamr aligner in a 4.6% f-score on the ldc2014t2 corpus. the authors also showed that their aligner improved the jamr @xcite and camr @xcite parsers. ## our aligner in order to properly adapt amr parsers from english to portuguese, we developed an alignment strategy based on document similarity for the portuguese language. our method produces alignments in the jamr aligner format since most of the amr parsers adopt this alignment type. to support our method, we used the pre-trained glove 1 embeddings of 100 dimensions for the portuguese language @xcite and some lexical resources. we organized our method into three phases over input annotated sentences: preprocessing, mapping, and aligning. in the first step, we tokenize the sentences and lemmatize each token, applying the stanza tool @xcite ) trained for portuguese. the portuguese tokenization is slightly different from english. for example, some hyphenated words, as "via-me" and "ouvi-la" (translated for "saw me" and "hear her"), should be separated by the hyphen, whereas other words, as "segunda-feira" and "recém-casados" (translated for "monday" and "newly married"), should not be separated. to detail the next steps, we will use figure 5 as an example. in the next step, we mapped each concept to its respective position in the graph. one can see that we mapped the contrast-01 concept to the root of the graph 0, its child responder-01 to 0.0, and their childrenand person to their respective positions 0.0.0 and 0.0.1. to do this, we used the penman tool (goodman, 2020). in the last step, we align the word tokens of the sentence with the concepts of the graph. the amr language has two concept types: concrete and abstract (or special keywords) ones. the former are those that are explicitly present in the sentence, while the latter are not. in figure 5 , we can see that the responder-01 is a concrete concept, since it is in the sentence, while the contrast-01, person, and name concepts are abstract foot_1 . to align concrete concepts, we use the word mover's distance (wmd) foot_2 @xcite and the pre-trained glove embeddings of 100 dimensions. the wmd is a distance function where the lower distance value indicates a higher similarity to the documents. it measures the minimum amount of distance that embedded words of one document need to "travel" to reach the embedded words of another document. we used this distance function to evaluate a distance between the embedded word tokens in the sentence and the embedded concepts in the graph to produce alignments with more semantics information than string-match. furthermore, we empirically defined a maximum distance (threshold) of 1.5 to match a token with a concept, i.e., our strategy maps a word with a concept only if the distance between them is less than the defined threshold. figure 6 shows this strategy to align the words of the sentence with concrete concepts of the graph. from this figure, @xmath0, ..., w n } is the set of words of a sentence and @xmath1, ..., c n } is the set of concrete concepts of the graph. our method aligns a w i with a c j if and only if the wmd value between w i and c j is lower than 1.5, and that value is the lowest among the other delta values. to align abstract concepts, we use some lexical resources (list of words) to aid and get a higher recall in the alignment. at this time, the amr formalism has 44 abstract concepts foot_3 and 110 concepts that represent named entities foot_4 . for instance, in figure 5 , the person is a concept that produces a named entity and, as this concept is in the resource, our alignment strategy aligns this concept and its children with the pedro (peter) token, which is the span 1-2, generating the alignment 1-2|0.0.1+0.0.1.0+0.0.1.0.0, which means that the span 1-2 is aligned with the concept person (0.0.1) plus name (0.0.1.0) plus "pedro" (0.0.1.0.0). in addition to named entities and abstract concepts, amr concepts encompass contrastive conjunctions and negations. to align these concepts, we also created two more lexical resources 6,7 (list of words) for these concepts. likewise, as ab-stract concepts and named entities, as the words mas (but) and não (not) are in the resources, our alignment method aligns the spans 0-1 and 2-3, which are the words mas (but) and não (not), respectively, with nodes 0 and 0.0.0, which are contrast-01 and -, respectively (see figure 5 ). our alignment tool is available at http://github. com/rafaelanchieta/amr-aligner . in what follows, we detail our experiments with the aligner and the obtained results. ## experiments and results we performed two experiments, one intrinsic and another extrinsic. in the first, we randomly chose and manually aligned one hundred sentences with their respective amrs from the little prince corpus @xcite . then, we compared the manual alignment with the alignments produced by @xcite as we can see, our aligner outperformed those developed for english, which means that our alignment strategy produced alignments more consistent with those manually produced. to get these values, we followed the evaluation method of @xcite . in order to confirm the intrinsic evaluation results, we performed an extrinsic evaluation. thus, we adapted the amr parsers of @xcite (henceforth, we refer to it as amreager) and @xcite (henceforth, we refer to it as camr) for the portuguese language. we chose these parsers because they use alignments, are open source, need only minor modifications for reuse with other languages, and have a good performance on small corpora. we trained these parsers on the little prince corpus of the portuguese language (anchiêta and pardo, 2018a), which contains 1,274, 145, and 143 sentences for training, development, and testing, respectively. to compare the results of the parsers, we used the traditional smatch metric @xcite and the more recently proposed sema metric @xcite . sema is a more robust metric that considers the parent of the nodes in the graph, producing fairer results than the smatch metric. from this table, one realizes that our aligner improved the adapted amr parsers for portuguese in both metrics, confirming the results of the intrinsic evaluation. moreover, the camr parser achieved a competitive result (50% f-score) compared to the rbamr parser (53% f-score over the same corpus) @xcite , a rule-based amr parser designed for the portuguese language. we also performed a fine-grained error analysis to identify the weaknesses of our aligner. for that, we used the evaluation tool of @xcite to compare the camr parser, as it achieved the best results, with the best aligners jamr, tamr, and ours. we present the results in the models tied for the wiki metric due to the corpus not having wiki annotation. in the ner metric, the tamr aligner performed better than the other aligners. this result is because of the specific rules used to align named entities and the morphosemantic database that this aligner makes use of. besides, our aligner achieved only 0.07 of reentrancies, as the aligner is not prepared to align reentrancies. one solution could be to model reentrancies as a tree, according to @xcite . another issue is the occurrence of hidden subjects in portuguese, i.e., sentences where the subject 'i' is in the graph, but it is not in the sentence (it is implicit). our aligner tool also ignores this phenomenon. one solution could be to apply a preprocessing to identify and include the hidden subjects in the sentences. treating these issues remain for future work. we also intend to investigate the morphosemantic database, aiming to improve the accuracy in the alignment of named entities. the adapted amr parsers are available at http://github.com/rafaelanchieta/camr-pt and http://github.com/rafaelanchieta/ amr-eager-pt . ## conclusion in this paper, we presented an amr alignment method designed for the portuguese language. it is based on pre-trained word embeddings and word mover's distance to match word tokens in the sentences and nodes in the corresponding amr graphs. this simple approach may be adopted for other languages with few resources, aiming to get tools for natural language understanding tasks. furthermore, this aligner may help to build or increase semantic resources, using a promising approach as back-translation @xcite . future work includes adopting multilingual word embeddings @xcite to produce alignments for other languages. more details about amr resources and tools for the portuguese language may be found at the opinando project webpage foot_7 .