Building a C hinese AMR Bank with Concept and Relation Alignments (2019)

Abstract: Abstract Meaning Representation (AMR) is a meaning representation framework in which the meaning of a full sentence is represented as a single-rooted, acyclic, directed graph. In this article, we describe an on-going project to build a Chinese AMR (CAMR) corpus, which currently includes 10,149 sentences from the newsgroup and weblog portion of the Chinese TreeBank (CTB). We describe the annotation specifications for the CAMR corpus, which follow the annotation principles of English AMR but make adaptations where needed to accommodate the linguistic facts of Chinese. The CAMR specifications also include a systematic treatment of sentence-internal discourse relations. One significant change we have made to the AMR annotation methodology is the inclusion of the alignment between word tokens in the sentence and the concepts/relations in the CAMR annotation to make it easier for automatic parsers to model the correspondence between a sentence and its meaning representation. We develop an annotation tool for CAMR, and the inter-agreement as measured by the Smatch score between the two annotators is 0.83, indicating reliable annotation. We also present some quantitative analysis of the CAMR corpus. 46.71% of the AMRs of the sentences are non-tree graphs. Moreover, the AMR of 88.95% of the sentences has concepts inferred from the context of the sentence but do not correspond to a specific word.

## introduction abstract meaning representation (amr) is a novel annotation framework to represent the "meaning" of a sentence with a single rooted, acyclic foot_0 , directed graph @xcite , departing from previous practices of performing partial semantic annotation that focuses on certain aspect of meaning. some well-known examples of partial semantic annotation efforts include the annotation of predicate-argument structure of verbs @xcite and predicative or relational nouns @xcite , the annotation of entities and relations along the lines of automatic content extraction project @xcite , the annotation of discourse relations (e.g., the penn discourse tree-bank @xcite ), as well as the annotation of temporal relations @xcite and factuality @xcite . the choice to annotate aspects of meaning instead of "whole-sentence" meaning is predicated on the assumption that focusing on a single aspect of meaning is more likely to lead to consistent annotation and consistently annotated data in turn lead to more accurate machine learning based automatic systems. this is a reasonable assumption when the meaning components of a sentence are not well-understood. however, such a fragmented approach to meaning annotation also leads to redundancies or even conflicts between the different meaning components, thus diminishing the value of these annotated resources when they have to be used in conjunction. there will also inevitably be gaps that are not covered by any of the meaning components which will be problematic for applications that need to reason over the semantics of entire sentences. the "holistic" approach of amr to annotating the meaning of entire sentences attempts to address this issue by modeling the meaning of a sentence with a single rooted, directed acyclic graph. in general this is considered to be a welcome development in spite of the fact that there are still aspects of sentence meaning that amr leaves out in exchange for expedience in annotation. for ex-ample, amr currently does not annotate tense, aspect, nor does it annotate the phenomena of quantification. however, these linguistic phenomena can be added without substantially modifying the amr formalism. compared with other semantic annotation efforts such as the semantic dependency annotation @xcite that is largely based on minimum recursion semantics (mrs) @xcite , the tectogrammatical layer of the prague dependency treebank @xcite , as well as the groningen meaning bank @xcite which is largely based on the discourse representation theory @xcite , one salient characteristic of amr annotation is the relaxation of the strict correspondence between the meaning representation and its underlying morpho-syntactic representation. this has a number of consequences for amr annotation in practice. first of all, amr can be annotated independently of morpho-syntactic structures and does not have to be linked to syntactic units such as words and phrases in the annotation process. the practical benefit of this is that it makes annotation scalable, eliminating the time needed to first build morpho-syntactic structures before any semantic annotation can start. second, the relaxation of the strict correspondence between syntactic representation and semantic representation allows more freedom in handling syntax-semantic mismatches. this includes cases where function words that are crucial building blocks of the syntactic structure can be left out of the meaning representation because they do not contribute to the meaning of the sentence (e.g., infinitive "to"in english). conversely, there are also cases where constructs (i.e., concepts or relations in amr) in the meaning representation are inferred from the context and do not necessarily correspond to any words (e.g., "person" can be inferred from "the young"). a third type of syntax-semantic mismatch is reflected in cases where there is a complicated correspondence between the meaning representation and the surface syntactic structure. for example, a single concept or relation in amr can be posited to represent meaning conveyed in discontinuous constructions such as "as • • • as • • •" which can be collapsed into a single relation :compared-to. third, since amr abstracts away from elements of surface syntactic structure such as word order and morpho-syntactic markers, which account for much of the cross-linguistic variations, it makes a more portable semantic annotation framework across languages, as the preliminary amr annotation on chinese and czech has demonstrated @xcite . there are always two sides to every coin and affording annotators unconstrained freedom to make up new concepts can lead to inconsistent and unusable annotation without carefully designed guidelines that specify when a new concept can be inferred and when a discontinuous pattern can be mapped to a single concept/relation. although annotating meaning representation independently of syntactic structures serves to speed up annotation, in automatic meaning representation parsing, morpho-syntactic structures often serve as important clues that can be used to derive the semantic representation. some minimal correspondence between the two representations needs to be established in order to make use of the syntactic structure when developing meaning representation parsers. when conducting automatic amr parsing, it is customary to explicitly provide the correspondence between word tokens in the sentence to the concepts and relations in its amr, that is, the alignment between the input sentence and its amr. since this alignment is not provided in the english amr bank @xcite , amr parsing researchers have to develop a word-to-concept aligner as the first step in amr parsing. this can be done via either a supervised or unsupervised approach. for example, @xcite develops a rule-based aligner by independently annotating the alignment between word tokens and amr concepts for a small corpus that can be used to extract alignment rules. the alignment f-score of this aligner is about 90%. @xcite develops an em-based aligner that yields similar performance without any manual alignment. while these aligners may seem to be very accurate, a 10% error rate in alignment imposes a serious limitation on the overall amr parsing accuracy as errors in alignment will propagate to subsequent steps. in this article, we present the camr corpus, a growing chinese amr corpus 2 that currently has 10,149 sentences annotated with meaning representations. we adopt the amr approach of representing the meaning of a sentence as a rooted, directed acyclic graph, and we also adopt the amr philosophy of annotating the meaning representation independently of syntactic structures, even though the data we annotated are drawn from the chinese tree-bank that already has syntactic annotation @xcite . in the meantime, we have also made a number of adaptations. first, rather than letting users of the corpus perform their own word-toconcept alignments, we incorporated this as an integral part of the annotation. we show in section 3 that incorporating this alignment for a language like chinese is straightforward and has a number of advantages. second, while in english amr discourse relations such as temporal and causal relations are annotated in a variety of ways, we use a dedicated set of abstract concepts to annotate discourse relations. this "modular" approach makes it easier for users to examine and use different aspects of the camr corpus. third, we added a few labels to the english amr label set to account for a few chinese-specific linguistic phenomena. in general, however, the label set used in the english amr bank works surprisingly well in our camr annotation and readily applies to chinese data. this bodes well for this annotation framework to be applied to additional languages. the rest of the article is organized as follows. in section 2 we present an overview of the camr annotation framework that integrates word-to-concept and word-to-relation alignments. we start with a presentation of the amr annotation specification and then outline our extensions. in section 3 we describe how we perform alignment between the concepts/relations in amr and word tokens in sentences. we illustrate how to handle a few well-known chinese-specific constructions in camr in section 4. in section 5, we present results on our camr annotation experiments, as well as a quantitative analysis of the proportions of non-tree graphs. we describe related work in section 6 and conclude our article with a summary of our contribution in section 7. ## conclusion and future work in this article, we presented our effort in developing the chinese amr (camr) corpus, which consists of 10,149 sentences selected from the chinese treebank. our general approach was to adopt the amr strategy of annotating the meaning representation of each sentence independently of other layers of linguistic analysis for the sake of scalability, while developing detailed specifications as to how to annotate each linguistic construction to ensure consistent annotation. on one hand, we have found that the amr specifications, consisting of the graph structure, the abstract concepts and relations, readily applies to the camr annotation almost in its entirety. on the other hand, we also extended the amr specifications by devising a consistent way to annotate discourse relations as well as tense and aspect. another departure from the amr approach is that we integrate word-to-amr concept and relation alignment to the camr annotation process. the inter-annotation agreement shows that our approach is effective. a quantitative analysis of the camr corpus shows that 46.71% of the amrs are non-tree graphs. in addition, the amrs of 88.95% of the sentences have abstract concepts inferred from the context of the sentence but do not correspond to a particular word or phrase in a sentence, and the average number of such inferred concepts per sentence is 2.88. we believe this corpus will prove to be crucial resource in advancing the state of the art in chinese semantic parsing and in chinese amr parsing in particular. in the future, we plan to annotate additional data of other genres as part of this on-going project. we will also develop automatic chinese amr parsers.