Transparent Semantic Parsing with U niversal D ependencies Using Graph Transformations (2022)

Abstract: Even though many recent semantic parsers are based on deep learning methods, we should not forget that rule-based alternatives might offer advantages over neural approaches with respect to transparency, portability, and explainability. Taking advantage of existing off-the-shelf Universal Dependency parsers, we present a method that maps a syntactic dependency tree to a formal meaning representation based on Discourse Representation Theory. Rather than using lambda calculus to manage variable bindings, our approach is novel in that it consists of using a series of graph transformations. The resulting UD semantic parser shows good performance for English, German, Italian and Dutch, with F-scores over 75%, outperforming a neural semantic parser for the lower-resourced languages. Unlike neural semantic parsers, our UD semantic parser does not hallucinate output, is relatively easy to port to other languages, and is completely transparent.

## introduction semantic parsing is the task of mapping natural language sentences to a formal meaning representation such as abstract meaning representations @xcite or discourse representation structures @xcite . the current trend in this area is strongly geared towards using methods based on deep learning. the best performing parsers use pre-trained language models (van @xcite @xcite . but a good performance is perhaps not the only thing that matters. a drawback of neural semantic parsers is that their output lacks explainability: why are the meaning representations composed in the way they are? moreover, they require vast amounts of training data, and are usually specific for a particular language. in addition, their performance usually decreases for longer input sentences. in other words, it may look like we have made a lot of progress, but viewed from a different perspective, we might actually have made a step back. this is especially so with regards to transparency and interpretability of semantic parsers. in this paper we describe a semantic parsing system for discourse representation structures -the formal meaning representations proposed by discourse representation theory @xcite -that is based on universal dependencies (ud, de @xcite . the first advantage of the ud framework is that it has been developed for numerous languages (using a crosslinguistically consistent annotation scheme) and that several high-performing parsers have been developed for ud. this will make it easier, as we will show, to develop semantic parsers for languages other than english, in our case german, italian and dutch. the second advantage of using ud as input describing the syntactic structure of the sentence, is that it provides us with explainable support of the output of the meaning representation, based on the derivation provided by the ud parser. the innovative contribution of the system, udboxer, that we describe here is in the way the meaning representations are computed. even though the original discourse representation structure (drs) is formally an ordered pair of a set of discourse referents and a set of conditions, we recast the drs as a directed acyclic graph. through graph transformation rules our system changes the input ud syntactic representation step-by-step into a fully fledged formal meaning representation. ## related work our aims are similar to those of @xcite and @xcite , who map ud to logical forms in three steps: (1) enriching the ud tree with missing syntactic information and long-distance dependencies, turning it into a graph, (2) binarization of the dependency graph, (3) substituting the root lost lose verb root tracy tracy propn nsubj glasses glasses noun obj . . punct punct her she pron nmod-poss root lost lose verb @xmath0. . punct tok=. punct @xmath1. . punct tok=. punct @xmath2.n.08 @xmath3. . punct tok=. punct @xmath4, invert or merge lambda-expression to compose predicate-argument structures, and (4) applying β-conversion to get a reduced, normalized logical form. the main difference of our work with that of @xcite is that we do not require complicated operations involving logical variables. by making the target meaning representations free of variables in the form of a graph, the mapping from ud to meaning representation is solely based on a sequence of graph transformations @xcite . this allows us to apply our method on languages other than english. similar in spirit to our work, but different in execution, is recent work by @xcite , who present a drs parser that is competitive in accuracy with recent sequence-to-sequence models and at the same time compositional. this latter property makes their system transparent and more explainable. @xcite recast drs parsing as a sequence labeling task, and achieve a good performance with f-scores of 84.4 for english, 78.3 for german, 80.4 for italian, and 72.1 for dutch on pmb 3.0.0 data (and therefore not directly comparabable with our results, working with a more recent version of the pmb). ## results in a manual analysis, we found that the few errors made by ud-boxer were all caused by unexpected sentence roots in the ud output of the stanza parser. a case in point is the input all of my friends like computer games where stanza decides that it is a noun phrase with all as the root, whereas trankit assigns like as the root. currently no transformation rules deal with such cases. the graph gets malformed because the root is cut away at some point since it is a determiner and those can generally be left out of drss. but the majority of ill-formed output is produced by the neural parsers. an example, which also exhibits hallucination of semantic information, is the output drg for she died of tuberculosis (figure 6 ), where the disease changed into a heart attack and is recognized as an anonymous geological formation (there is a role name without a constant). this strange phenomenon occurs often in output from both neural parsers. the problem with detecting these anomalies is that often these graphs score well, even though they are semantically ill-formed. ## conclusion our results show competitive performance between ud-boxer and neural boxer for english, when a limited amount of training data is available for the neural approach. in addition, it shows strong crosslingual performance using a few simple languagespecific rules and only gold training data to extract the mappings. adding more transformation rules and creating a label substitution component based on machine learning will likely push the performance of ud-boxer even higher. the phenomena we have in mind are named entities, numeral expressions, time and date expressions, and discourse relations, for which only simple rules have been defined so far. since ud-boxer is a modular system, it is rather straightforward to add such rules. an important difference between the two semantic parsers is that ud-boxer is guaranteed to output a well-formed meaning representation, provided that the input ud is accurate. because neural boxer is following a seq2seq transformer approach, the output does not always correspond to a graph and therefore requires ad-hoc postprocessing rules to make such output interpretable. another serious deficiency of neural parsers is that it sometimes hallucinates semantic material without warning. a transparent semantic parser, even with a slightly lower performance in some cases, might be a good alternative for certain applications, in particular when lower-resourced languages are involved.