Dependency Patterns of Complex Sentences and Semantic Disambiguation for A bstract M eaning R epresentation Parsing (2021)

Abstract: Abstract Meaning Representation (AMR) is a sentence-level meaning representation based on predicate argument structure. One of the challenges we find in AMR parsing is to capture the structure of complex sentences which expresses the relation between predicates. Knowing the core part of the sentence structure in advance may be beneficial in such a task. In this paper, we present a list of dependency patterns for English complex sentence constructions designed for AMR parsing. With a dedicated pattern matcher, all occurrences of complex sentence constructions are retrieved from an input sentence. While some of the subordinators have semantic ambiguities, we deal with this problem through training classification models on data derived from AMR and Wikipedia corpus, establishing a new baseline for future works. The developed complex sentence patterns and the corresponding AMR descriptions will be made public.

## introduction abstract meaning representation (amr) is a sentence-level meaning representation based on predicate argument structure @xcite . amr parsing is the task of transforming a sentence into an amr graph with nodes and edges, each representing a concept or relation. while early studies @xcite @xcite used dependency parsers to integrate syntactic features to their models, recent deep neural network-based approaches @xcite @xcite tend to encode the input sentence as a sequence without considering its syntactic structure. generally speaking, syntactic and semantic structures share much in common. it is assumed 1code and resource are available at https://github . com/yama-yuki/skeletal-amr. that dependency trees and semantic role labeling structures have a strong correlation in that nsubj and dobj can be used interchangeably for arg0 and arg1 role @xcite . since amr is annotated based on propbank frames @xcite , the same could be said for amr structures. this holds to be true for a simple sentence, which is basically a matrix clause, comprised of a predicate and its arguments. however, it is not always the case with complex sentence constructions, each of which consists of a matrix clause and one or more subordinate clause(s). consider figure 1 which shows both dependency and amr representation of a complex sentence with a subordinator as long as. while variables s's and v's are interchangeable between the representations, predicative relations and subordinator itself are expressed quite differently. compared to uniform structures of simple sentences, various types of complex sentence are used in human language. this characteristics makes it challenging for existing amr parsers to capture its structure correctly. among amr parsers which are aware of syntactic structures, camr @xcite directly transforms the result of dependency parsing into an amr graph with transition-based algorithm. as figure 2 (a) shows an example parse with camr, existing parsers have trouble capturing the relation between two clauses. in this case, camr predicts the relation between believe-01 and seem-01 as arg1, though it should be represented as a causal relation with cause-01 as shown in figure 2 @xcite . this is a crucial error as it incorrectly determines the very core structure of the output graph. we assume that the solution could be achieved either by retraining the parser on larger annotated data or providing the parser with the core structure of the input sentence in advance. the latter seems more reasonable in terms of the cost of annotation. with the motivation to aid amr parsing task, we present a method to retrieve all occurrences of complex sentence constructions from an input sentence using a dedicated pattern matcher. at the moment, there is no comprehensive resource that provides structural information about the relations between two clauses, particularly in amr framework. therefore, as our first step, we attempt to develop a pattern dictionary of english complex sentences together with the corresponding amrs which represent the skeleton structure of the sentence (hereinafter referred to as skeletal amrs ). then, we provide a pattern matcher which captures clausal relations between a superordinate and subordinate clauses in a complex sentence. our pattern matching approach faces the problem of syntactic and semantic ambiguities. when a complex sentence has more than one subordinate clause, we need to determine which pair of clauses are related. consider the following example where two subordinate clauses appear in a single sentence. in order to resolve the semantic ambiguities of clause-level coherence relation, inspired by the work of @xcite , we take finetuning-based approach with data augmentation method. with the support of weakly supervised data derived from wikipedia, we achieve the scores of 75.65% and 83.94% on macro and micro f1 respectively, establishing a new baseline for coherence relation classification of complex sentence constructions in amr framework. to sum up, the contributions of this paper are the followings: ## related works while our focus is on clause-level relation of complex sentence constructions, not much study has been done specifically on this topic in amr framework. rather, the topic is dealt with in the field of discourse structures, where coherence relations between any text segments are the main focus. in the studies of discourse parsing, various attempts have been made to capture coherence relations between pairs of sentences or clauses @xcite @xcite . these works basically rely on discourse frameworks such as rhetorical structure theory (rst; @xcite or penn discourse tree bank (pdtb; @xcite . most recently, @xcite has presented a finetuning-based approach using the bidirectional encoder representation from transformers (bert; @xcite . they designed their model to learn 11 classes to achieve the state-ofthe-art performance on implicit discourse relation classification task in pdtb framework. their work was motivated by the method taken by @xcite to pretrain bert, which is called "next sentence prediction task" (nsp). in the process of pretraining using nsp, the model is presented with pairs of sentences. the model predicts whether the second sentence is the actual subsequent sentence. nsp enables bert to represent a pair of sentences by packing them together as a single sequence. some studies have focused on discourse structure in amr framework. @xcite enhances amr by annontating tense and aspect phenomena at discourse-level. the work by @xcite targets relations of sentences and provides annotation of coreference in multi-sentence amr corpus. yet, neither the structure of the complex sentence constructions nor the coherence relations between subordinate and matrix clauses have been much of a concern in this framework. ## pattern matching system in this section, we will give a description of our pattern matching system. the entire workflow is illustrated in figure 3 , where the system takes a sentence as an input and returns a skeletal amr if it is a type of complex sentence construction. the disambiguation module will be described in the later section. we will use the sentence given as a running example throughout this paper. ## semantic disambiguation experiment in this section, we describe details of our study on semantic disambiguation. we observe that the coherence relation between subordinate and superordinate clauses are commonly ambiguous among causal, conditional, concessional, and temporal senses. considering that we are dealing with relations between clauses, which basically can be regarded as pairs of simple sentences, we can cast the problem of semantic ambiguity as a multi-class sentence-pair classification task. to be more specific, we assign the most typical class labels including cause, cond, conc, and time to make it a 4-class classification setting. throughout the experiment, we use the pretrained "bert-base-uncased" model for finetuning. bert is pretrained under next sentence prediction as well as masked language modeling task. during pretraining, the model predicts the actual next sentence from a pair of candidate sentences. thus, the model is expected to learn what the next sentence should look like. by finetuning bert on a set of pairs of clauses, we can further expect the model to capture coherence relations between them. our input will be a separate pair of clauses with special tokens "[cls]" and "[sep]". while bert takes the input in sequential manner, we would like the model to take advantage of dependency parsing, which we apply in the process of pattern matching. therefore, we add artificial tags @xmath0. the input format of our running example would look like: "[cls] airline @xmath1. ## conclusion with the intention to capture the structure of complex sentence constructions in amr framework, we proposed a pattern matching method using a list of dependency patterns and its corresponding