Introduction
Semantic parsing aims to solve the problem of canonicalizing language and representing its meaning: given an input sentence, it aims to extract a semantic representation of that sentence. Abstract meaning representation BIBREF0 , or AMR for short, allows us to do that with the inclusion of most of the shallow-semantic natural language processing (NLP) tasks that are usually addressed separately, such as named entity recognition, semantic role labeling and co-reference resolution. AMR is partially motivated by the need to provide the NLP community with a single dataset that includes basic disambiguation information, instead of having to rely on different datasets for each disambiguation problem. The annotation process is straightforward, enabling the development of large datasets. Alternative semantic representations have been developed and studied, such as CCG BIBREF1 , BIBREF2 and UCCA BIBREF3 .
Several parsers for AMR have been recently developed BIBREF4 , BIBREF5 , BIBREF6 , BIBREF7 , BIBREF8 , BIBREF9 , BIBREF10 , BIBREF11 , BIBREF12 , BIBREF13 . This line of research is new and current results suggest a large room for improvement. Greedy transition-based methods BIBREF14 are one of the most popular choices for dependency parsing, because of their good balance between efficiency and accuracy. These methods seem promising also for AMR, due to the similarity between dependency trees and AMR structures, i.e., both representations use graphs with nodes that have lexical content and edges that represent linguistic relations.
A transition system is an abstract machine characterized by a set of configurations and transitions between them. The basic components of a configuration are a stack of partially processed words and a buffer of unseen input words. Starting from an initial configuration, the system applies transitions until a terminal configuration is reached. The sentence is scanned left to right, with linear time complexity for dependency parsing. This is made possible by the use of a greedy classifier that chooses the transition to be applied at each step.
In this paper we introduce a parser for AMR that is inspired by the ArcEager dependency transition system of nivre2004. The main difference between our system and ArcEager is that we need to account for the mapping from word tokens to AMR nodes, non-projectivity of AMR structures and reentrant nodes (multiple incoming edges). Our AMR parser brings closer dependency parsing and AMR parsing by showing that dependency parsing algorithms, with some modifications, can be used for AMR. Key properties such as working left-to-right, incrementality and linear complexity further strengthen its relevance.
The AMR parser of wang2boosting, called CAMR, also defines a transition system. It differs from ours because we process the sentence left-to-right while they first acquire the entire dependency tree and then process it bottom-up. More recently emnlp2016 presented a non-greedy transition system for AMR parsing, based on ArcStandard BIBREF15 . Our transition system is also related to an adaptation of ArcEager for directed acyclic graphs (DAGs), introduced by sagae2008shift. This is also the basis for ribeyre2015because, a transition system used to parse dependency graphs. Similarly, du2014peking also address dependency graph parsing by means of transition systems. Analogously to dependency trees, dependency graphs have the property that their nodes consist of the word tokens, which is not true for AMR. As such, these transition systems are more closely related to traditional transition systems for dependency parsing.
Our contributions in this paper are as follows:
Transition-Based AMR Parsing
Similarly to dependency parsing, AMR parsing is partially based on the identification of predicate-argument structures. Much of the dependency parsing literature focuses on transition-based dependency parsing—an approach to parsing that scans the sentence from left to right in linear time and updates an intermediate structure that eventually ends up being a dependency tree.
The two most common transition systems for greedy dependency parsing are ArcStandard and ArcEager. With ArcStandard, a stack is maintained along with a buffer on which the left-to-right scan is performed. At each step, the parser chooses to scan a word in the buffer and shift it onto the stack, or else to create an arc between the two top-most elements in the stack and pop the dependent. ArcStandard parses a sentence in a pure bottom-up, left-to-right fashion (similarly to shift-reduce context-free grammar parsers), and must delay the construction of right arcs until all the dependent node has been completed. This imposes strong limitations on the degree of incrementality of the parser. The ArcEager system was designed to improve on ArcStandard by mixing bottom up and top-down strategies. More precisely, in the ArcEager parser left arcs are constructed bottom-up and right arcs are constructed top-down, so that right dependents can be attached to their heads even if some of their own dependents are not identified yet. In this way arcs are constructed as soon as the head and the dependent are available in the stack.
Because of the similarity of AMR structures to dependency structures, transition systems are also helpful for AMR parsing. Starting from the ArcEager system, we develop here a novel transition system, called AmrEager that parses sentences into AMR structures. There are three key differences between AMRs and dependency trees that require further adjustments for dependency parsers to be used with AMRs.
A key difference between English dependency trees and AMR structures is projectivity. Dependency trees in English are usually projective, roughly meaning that there are no crossing arcs if the edges are drawn in the semi-plane above the words. While this restriction is empirically motivated in syntactic theories for English, it is no longer motivated for AMR structures.
The notion of projectivity can be generalized to AMR graphs as follows. The intuition is that we can use the alignment INLINEFORM0 to map AMR edges back to the sentence INLINEFORM1 , and test whether there exist pairs of crossing edges. Figure FIGREF13 shows this mapping for the AMR of Figure FIGREF7 , where the edge connecting excuse to I crosses another edge. More formally, consider an AMR edge INLINEFORM2 . Let INLINEFORM3 and INLINEFORM4 , so that INLINEFORM5 is aligned with INLINEFORM6 and INLINEFORM7 is aligned with INLINEFORM8 . The spanning set for INLINEFORM9 , written INLINEFORM10 , is the set of all nodes INLINEFORM11 such that INLINEFORM12 and INLINEFORM13 if INLINEFORM14 or INLINEFORM15 if INLINEFORM16 . We say that INLINEFORM17 is projective if, for every node INLINEFORM18 , all of its parent and child nodes are in INLINEFORM19 ; otherwise, we say that INLINEFORM20 is non-projective. An AMR is projective if all of its edges are projective, and is non-projective otherwise. This corresponds to the intuitive definition of projectivity for DAGs introduced in sagae2008shift and is closely related to the definition of non-crossing graphs of kuhlmann2015parsing.
Table TABREF15 demonstrates that a relatively small percentage of all AMR edges are non-projective. Yet, 35% of the sentences contain at least one non-projective edge.
https://github.com/jflanigan/jamr/blob/master/docs/Hand_Alignments.md
AMRs are graphs rather than trees because they can have nodes with multiple parents, called reentrant nodes, as in the node you for the AMR of Figure FIGREF7 . There are two phenomena that cause reentrancies in AMR: control, where a reentrant edge appears between siblings of a control verb, and co-reference, where multiple mentions correspond to the same concept.
In contrast, dependency trees do not have nodes with multiple parents. Therefore, when creating a new arc, transition systems for dependency parsing check that the dependent does not already have a head node, preventing the node from having additional parents. To handle reentrancy, which is not uncommon in AMR structures as shown in Table TABREF15 , we drop this constraint.
Another main difference with dependency parsing is that in AMR there is no straightforward mapping between a word in the sentence and a node in the graph: words may generate no nodes, one node or multiple nodes. In addition, the labels at the nodes are often not easily determined by the word in the sentence. For instance expectation translates to expect-01 and teacher translates to the two nodes teach-01 and person, connected through an :ARG0 edge, expressing that a teacher is a person who teaches. A mechanism of concept identification is therefore required to map each token INLINEFORM0 to a subgraph with the correct labels at its nodes and edges: if INLINEFORM1 is the gold alignment, this should be the subgraph INLINEFORM2 defined in Equation ( EQREF11 ). To obtain alignments between the tokens in the sentence and the nodes in the AMR graph of our training data, we run the JAMR aligner.
Transition system for AMR Parsing
A stack INLINEFORM0 is a list of nodes of the partially constructed AMR graph, with the top element INLINEFORM1 at the right. We use the symbol ` INLINEFORM2 ' as the concatenation operator. A buffer INLINEFORM3 is a list of indices from INLINEFORM4 , with the first element INLINEFORM5 at the left, representing the word tokens from the input still to be processed. A configuration of our parser is a triple INLINEFORM6 , where INLINEFORM7 is the set of AMR edges that have been constructed up to this point.
In order to introduce the transition actions of our parser we need some additional notation. We use a function INLINEFORM0 that maps indices from INLINEFORM1 to AMR graph fragments. For each INLINEFORM2 , INLINEFORM3 is a graph INLINEFORM4 , with single root INLINEFORM5 , representing the semantic contribution of word INLINEFORM6 to the AMR for INLINEFORM7 . As already mentioned, INLINEFORM8 can have a single node representing the concept associated with INLINEFORM9 , or it can have several nodes in case INLINEFORM10 denotes a complex concept, or it can be empty.
The transition Shift is used to decide if and what to push on the stack after consuming a token from the buffer. Intuitively, the graph fragment INLINEFORM0 obtained from the token INLINEFORM1 , if not empty, is “merged” with the graph we have constructed so far. We then push onto the stack the node INLINEFORM2 for further processing. LArc INLINEFORM3 creates an edge with label INLINEFORM4 between the top-most node and the second top-most node in the stack, and pops the latter. RArc INLINEFORM5 is the symmetric operation, but does not pop any node from the stack.
Finally, Reduce pops the top-most node from the stack, and it also recovers reentrant edges between its sibling nodes, capturing for instance several control verb patterns. To accomplish this, Reduce decides whether to create an additional edge between the node being removed and the previously created sibling in the partial graph. This way of handling control verbs is similar to the REENTRANCE transition of wang2boosting.
The choice of popping the dependent in the LArc transition is inspired by ArcEager, where left-arcs are constructed bottom-up to increase the incrementality of the transition system BIBREF15 . This affects our ability to recover some reentrant edges: consider a node INLINEFORM0 with two parents INLINEFORM1 and INLINEFORM2 , where the arc INLINEFORM3 is a left-arc and INLINEFORM4 is any arc. If the first arc to be processed is INLINEFORM5 , we use LArc that pops INLINEFORM6 , hence making it impossible to create the second arc INLINEFORM7 . Nevertheless, we discovered that this approach works better than a completely unrestricted allowance of reentrancy. The reason is that if we do not remove dependents at all when first attached to a node, the stack becomes larger, and nodes which should be connected end up being distant from each other, and as such, are never connected.
The initial configuration of the system has a INLINEFORM0 node (representing the root) in the stack and the entire sentence in the buffer. The terminal configuration consists of an empty buffer and a stack with only the INLINEFORM1 node. The transitions required to parse the sentence The boy and the girl are shown in Table TABREF20 , where the first line shows the initial configuration and the last line shows the terminal configuration.
Similarly to the transitions of the ArcEager, the above transitions construct edges as soon as the head and the dependent are available in the stack, with the aim of maximizing the parser incrementality. We now show that our greedy transition-based AMR parser is linear-time in INLINEFORM0 , the length of the input sentence INLINEFORM1 . We first claim that the output graph has size INLINEFORM2 . Each token in INLINEFORM3 is mapped to a constant number of nodes in the graph by Shift. Thus the number of nodes is INLINEFORM4 . Furthermore, each node can have at most three parent nodes, created by transitions RArc, LArc and Reduce, respectively. Thus the number of edges is also INLINEFORM5 . It is possible to bound the maximum number of transitions required to parse INLINEFORM6 : the number of Shift is bounded by INLINEFORM7 , and the number of Reduce, LArc and RArc is bounded by the size of the graph, which is INLINEFORM8 . Since each transition can be carried out in constant time, we conclude that our parser runs in linear time.
Training the System
Several components have to be learned: (1) a transition classifier that predicts the next transition given the current configuration, (2) a binary classifier that decides whether or not to create a reentrancy after a Reduce, (3) a concept identification step for each Shift to compute INLINEFORM0 , and 3) another classifier to label edges after each LArc or RArc.
Oracle
Training our system from data requires an oracle—an algorithm that given a gold-standard AMR graph and a sentence returns transition sequences that maximize the overlap between the gold-standard graph and the graph dictated by the sequence of transitions.
We adopt a shortest stack, static oracle similar to manningfast. Informally, static means that if the actual configuration of the parser has no mistakes, the oracle provides a transition that does not introduce any mistake. Shortest stack means that the oracle prefers transitions where the number of items in the stack is minimized. Given the current configuration INLINEFORM0 and the gold-standard graph INLINEFORM1 , the oracle is defined as follows, where we test the conditions in the given order and apply the action associated with the first match:
if INLINEFORM0 then LArc( INLINEFORM1 );
if INLINEFORM0 then RArc( INLINEFORM1 );
if INLINEFORM0 then Reduce;
Shift otherwise.
The oracle first checks whether some gold-standard edge can be constructed from the two elements at the top of the stack (conditions 1 and 2). If LArc or RArc are not possible, the oracle checks whether all possible edges in the gold graph involving INLINEFORM0 have already been processed, in which case it chooses Reduce (conditions 3). To this end, it suffices to check the buffer, since LArc and RArc have already been excluded and elements in the stack deeper than position two can no longer be accessed by the parser. If Reduce is not possible, Shift is chosen.
Besides deciding on the next transition, the oracle also needs the alignments, which we generate with JAMR, in order to know how to map the next token in the sentence to its AMR subgraph INLINEFORM0 defined in ( EQREF11 ).
Transition Classifier
Like all other transition systems of this kind, our transition system has a “controller” that predicts a transition given the current configuration (among Shift, LArc, RArc and Reduce). The examples from which we learn this controller are based on features extracted from the oracle transition sequences, where the oracle is applied on the training data.
As a classifier, we use a feed-forward neural network with two hidden layers of 200 tanh units and learning rate set to 0.1, with linear decaying. The input to the network consists of the concatenation of embeddings for words, POS tags and Stanford parser dependencies, one-hot vectors for named entities and additional sparse features, extracted from the current configuration of the transition system; this is reported in more details in Table TABREF27 . The embeddings for words and POS tags were pre-trained on a large unannotated corpus consisting of the first 1 billion characters from Wikipedia. For lexical information, we also extract the leftmost (in the order of the aligned words) child (c), leftmost parent (p) and leftmost grandchild (cc). Leftmost and rightmost items are common features for transition-based parsers BIBREF17 , BIBREF18 but we found only leftmost to be helpful in our case. All POS tags, dependencies and named entities are generated using Stanford CoreNLP BIBREF19 . The accuracy of this classifier on the development set is 84%.
Similarly, we train a binary classifier for deciding whether or not to create a reentrant edge after a Reduce: in this case we use word and POS embeddings for the two nodes being connected and their parent as well as dependency label embeddings for the arcs between them.
Concept Identification
This routine is called every time the transition classifier decides to do a Shift; it is denoted by INLINEFORM0 in § SECREF3 . This component could be learned in a supervised manner, but we were not able to improve on a simple heuristic, which works as follows: during training, for each Shift decided by the oracle, we store the pair INLINEFORM1 in a phrase-table. During parsing, the most frequent graph INLINEFORM2 for the given token is then chosen. In other words, INLINEFORM3 approximates INLINEFORM4 by means of the graph most frequently seen among all occurrences of token INLINEFORM5 in the training set.
An obvious problem with the phrase-table approach is that it does not generalize to unseen words. In addition, our heuristic relies on the fact that the mappings observed in the data are correct, which is not the case when the JAMR-generated alignments contain a mistake. In order to alleviate this problem we observe that there are classes of words such as named entities and numeric quantities that can be disambiguated in a deterministic manner. We therefore implement a set of “hooks” that are triggered by the named entity tag of the next token in the sentence. These hooks override the normal Shift mechanism and apply a fixed rule instead. For instance, when we see the token New York (the two tokens are collapsed in a single one at preprocessing) we generate the subgraph of Figure FIGREF30 and push its root onto the stack. Similar subgraphs are generated for all states, cities, countries and people. We also use hooks for ordinal numbers, percentages, money and dates.
Edge Labeling
Edge labeling determines the labels for the edges being created. Every time the transition classifier decides to take an LArc or RArc operation, the edge labeler needs to decide on a label for it. There are more than 100 possible labels such as :ARG0, :ARG0-of, :ARG1, :location, :time and :polarity. We use a feed-forward neural network similar to the one we trained for the transition classier, with features shown in Table TABREF32 . The accuracy of this classifier on the development set is 77%.
We constrain the labels predicted by the neural network in order to satisfy requirements of AMR. For instance, the label :top can only be applied when the node from which the edge starts is the special INLINEFORM0 node. Other constraints are used for the :polarity label and for edges attaching to numeric quantities.
Sometimes the label predicted by the neural network is not a label that satisfies the requirements of AMR. For instance, the label :top can only be applied when the node from which the edge starts is the special INLINEFORM0 node. In order to avoid generating such erroneous labels, we use a set of rules, shown in Table TABREF34 . These rules determine which labels are allowed for the newly created edge so that we only consider those during prediction. Also ARG roles cannot always be applied: each Propbank frame allows a limited number of arguments. For example, while add-01 and add-02 allow for :ARG1 and :ARG2 (and their inverse :ARG1-of and :ARG2-of), add-03 and add-04 only allow :ARG2 (and :ARG2-of).
Fine-grained Evaluation
Until now, AMR parsers were evaluated using the Smatch score. Given the candidate graphs and the gold graphs in the form of AMR annotations, Smatch first tries to find the best alignments between the variable names for each pair of graphs and it then computes precision, recall and F1 of the concepts and relations. We note that the Smatch score has two flaws: (1) while AMR parsing involves a large number of subtasks, the Smatch score consists of a single number that does not assess the quality of each subtasks separately; (2) the Smatch score weighs different types of errors in a way which is not necessarily useful for solving a specific NLP problem. For example, for a specific problem concept detection might be deemed more important than edge detection, or guessing the wrong sense for a concept might be considered less severe than guessing the wrong verb altogether.
Consider the two parses for the sentence Silvio Berlusconi gave Lucio Stanca his current role of modernizing Italy's bureaucracy in Figure FIGREF36 . At the top, we show the output of a parser (Parse 1) that is not able to deal with named entities. At the bottom, we show the output of a parser (Parse 2) which, except for :name, :op and :wiki, always uses the edge label :ARG0. The Smatch scores for the two parses are 56 and 78 respectively. Both parses make obvious mistakes but the three named entity errors in Parse 1 are considered more important than the six wrong labels in Parse 2. However, without further analysis, it is not advisable to conclude that Parse 2 is better than Parse 1. In order to better understand the limitations of the different parsers, find their strengths and gain insight in which downstream tasks they may be helpful, we compute a set of metrics on the test set.
Unlabeled is the Smatch score computed on the predicted graphs after removing all edge labels. In this way, we only assess the node labels and the graph topology, which may be enough to benefit several NLP tasks because it identifies basic predicate-argument structure. For instance, we may be interested in knowing whether two events or entities are related to each other, while not being concerned with the precise type of relation holding between them.
No WSD gives a score that does not take into account word sense disambiguation errors. By ignoring the sense specified by the Propbank frame used (e.g., duck-01 vs duck-02) we have a score that does not take into account this additional complexity in the parsing procedure. To compute this score, we simply strip off the suffixes from all Propbank frames and calculate the Smatch score.
Following sawai, we also evaluate the parsers using the Smatch score on noun phrases only (NP-only), by extracting from the AMR dataset all noun phrases that do not include further NPs.
As we previously discussed, reentrancy is a very important characteristic of AMR graphs and it is not trivial to handle. We therefore implement a test for it (Reentrancy), where we compute the Smatch score only on reentrant edges.
Concept identification is another critical component of the parsing process and we therefore compute the F-score on the list of predicted concepts (Concepts) too. Identifying the correct concepts is fundamental: if a concept is not identified, it will not be possible to retrieve any edge involving that concept, with likely significant consequences on accuracy. This metric is therefore quite important to score highly on.
Similarly to our score for concepts, we further compute an F-score on the named entities (Named Ent.) and wiki roles for named entities (Wikification) that consider edges labeled with :name and :wiki respectively. These two metrics are strictly related to the concept score. However, since named entity recognition is the focus of dedicated research, we believe it is important to define a metric that specifically assesses this problem. Negation detection is another task which has received some attention. An F-score for this (Negations) is also defined, where we find all negated concepts by looking for the :polarity role. The reason we can compute a simple F-score instead of using Smatch for these metrics is that there are no variable names involved.
Finally we compute the Smatch score on :ARG edges only, in order to have a score for semantic role labeling (SRL), which is another extremely important subtask of AMR, as it is based on the identification of predicate-argument structures.
Using this evaluation suite we can evaluate AMRs on a wide range of metrics that can help us find strengths and weakness of each parser, hence speeding up the research in this area. Table TABREF37 reports the scores for the two parses of Figure FIGREF36 , where we see that Parse 1 gets a high score for semantic role labeling while Parse 2 is optimal for named entity recognition. Moreover, we can make additional observations such as that Parse 2 is optimal with respect to unlabeled score and that Parse 1 recovers more reentrancies.
Experiments
We compare our parser against two available parsers: JAMR BIBREF4 and CAMR BIBREF20 , BIBREF5 , using the LDC2015E86 dataset for evaluation. Both parsers are available online and were recently updated for SemEval-2016 Task 8 BIBREF21 , BIBREF22 . However, CAMR's SemEval system, which reports a Smatch score of 67, is not publicly available. CAMR has a quadratic worst-case complexity (although linear in practice). In JAMR, the concept identification step is quadratic and the relation identification step is INLINEFORM0 , with INLINEFORM1 being the set of nodes in the AMR graph.
Table TABREF40 shows the results obtained by the parsers on all metrics previously introduced. On Smatch, our system does not give state-of-the-art results. However, we do obtain the best results for Unlabeled and Concept and outperform the other parses for Named Ent. and Negations. Our score of Reentrancy is also close the best scoring system, which is particularly relevant given the importance of reentrancies in AMR. The use of the Reduce transition, which targets reentrancies caused by control verbs, is critical in order to achieve this result.
The relatively high results we obtain for the unlabeled case suggests that our parser has difficulty in labeling the arcs. Our score for concept identification, which is on par with the best result from the other parsers, demonstrates that there is a relatively low level of token ambiguity. State-of-the-art results for this problem can be obtained by choosing the most frequent subgraph for a given token based on a phrase-table constructed from JAMR alignments on the training data. The scores for named entities and wikification are heavily dependent on the hooks mentioned in § SECREF29 , which in turn relies on the named entity recognizer to make the correct predictions. In order to alleviate the problem of wrong automatic alignments with respect to polarity and better detect negation, we performed a post-processing step on the aligner output where we align the AMR constant - (minus) with words bearing negative polarity such as not, illegitimate and asymmetry.
Our experiments demonstrate that there is no parser for AMR yet that conclusively does better than all other parsers on all metrics. Advantages of our parser are the worst-case linear complexity and the fact that is possible to perform incremental AMR parsing, which is both helpful for real-time applications and to investigate how meaning of English sentences can be built incrementally left-to-right.
Related Work
The first data-driven AMR parser is due to carbonell2014discriminative. The problem is addressed in two separate stages: concept identification and relation identification. They use a sequence labeling algorithm to identify concepts and frame the relation prediction task as a constrained combinatorial optimization problem. werling2015robust notice that the difficult bit is the concept identification and propose a better way to handle that task: an action classifier to generate concepts by applying predetermined actions. Other proposals involve a synchronous hyperedge replacement grammar solution BIBREF6 , a syntax-based machine translation approach BIBREF7 where a grammar of string-to-tree rules is created after reducing AMR graphs to trees by removing all reentrancies, a CCG system that first parses sentences into lambda-calculus representations BIBREF11 . A systematic translation from AMR to first order logic formulas, with a special treatment for quantification, reentrancy and negation, is discussed in bos2016expressive. In microsoft, a pre-existing logical form parser is used and the output is then converted into AMR graphs. Yet another solution is proposed by searnamr who discuss a parser that uses SEARN BIBREF23 , a “learning to search” algorithm.
Transition-based algorithms for AMR parsing are compelling because traditional graph-based techniques are computationally expensive. wang and wang2boosting propose a framework that parses a sentence into its AMR structure through a two-stage process: a dependency tree is generated from the input sentence through a transition-based parser and then another transition-based parser is used to generate the AMR. The main benefit of this approach is that the dependency parser can be trained on a training set much larger than the training set for the tree-to-graph algorithm. Others further built on this parser: goodman2016noise use imitation learning to alleviate the probem of error propagation in the greedy parser, while barzdins2016riga create a wrapper around it to fix frequent mistakes and investigate ensembles with a character level neural parser. More recently emnlp2016 presented a non-greedy transition system for AMR parsing, based on ArcStandard BIBREF15 .
AMR parsing as a whole is a complex task because it involves many subtasks including named entity recognition, co-reference resolution and semantic role labeling. sawai do not attempt at parsing AMR graphs for entire sentences but they instead handle simple noun phrases (NPs). They extract NPs from the AMR dataset only when they do not include further NPs, do not include pronouns nor named entities. Due to these restrictions, the AMRs are mostly trees and easier to handle than the original AMR graphs. They approach this task using a transition based system inspired by ArcStandard.
AMR is not the only way to represent meaning in natural language sentences. Alternative semantic representations have been developed and studied, such as Boxer BIBREF24 , CCG BIBREF1 , BIBREF2 and UCCA BIBREF3 .
Conclusion
We presented a transition system that builds AMR graphs in linear time by processing the sentences left-to-right, trained with feed-forward neural networks. The parser demonstrates that it is possible to perform AMR parsing using techniques inspired by dependency parsing.
We also noted that it is less informative to evaluate the entire parsing process with Smatch than to use a collection of metrics aimed at evaluating the various subproblems in the parsing process. We further showed that our left-to-right transition system is competitive with publicly available state-of-the-art parsers. Although we do not outperform the best baseline in terms of Smatch score, we show on par or better results for several of the metrics proposed. We hope that moving away from a single-metric evaluation will further speed up progress in AMR parsing.
Acknowledgments
The authors would like to thank the three anonymous reviewers and Sameer Bansal, Jeff Flanigan, Sorcha Gilroy, Adam Lopez, Nikos Papasarantopoulos, Nathan Schneider, Mark Steedman, Sam Thomson, Clara Vania and Chuan Wang for their help and comments. This research was supported by a grant from Bloomberg and by the H2020 project SUMMA, under grant agreement 688139.

Question: Do they use pretrained models as part of their parser?
Answer: Yes

Question: Which subtasks do they evaluate on?
Answer:  entity recognition, semantic role labeling and co-reference resolution