Structural and Global Features for Comparing Semantic Representation Formalisms (2023)

Abstract: The area of designing semantic/meaning representations is a dynamic one with new formalisms and extensions being proposed continuously. It may be challenging for users of semantic representations to select the relevant formalism for their purpose or for newcomers to the field to select the features they want to represent in a new formalism. In this paper, we propose a set of structural and global features to consider when designing formalisms, and against which formalisms can be compared. We also propose a sample comparison of a number of existing formalisms across the selected features, complemented by a more entailment-oriented comparison on the phenomena of the FraCaS corpus.

## introduction over the past decades, various semantic representation formalisms have emerged, focusing on different features of semantics. new formalisms and extensions are continuously developed, highlighting a dynamic field, but few works have been carried out on their comparison. @xcite provide a high-level summary of semantic features and existing formalisms. @xcite provide an overview and comparison of eleven deep-syntactic graph-based formalisms, focusing largely on their formal graph features. insights into the difference between encoding some semantic phenomena in different formalisms can also be found in empirical work based on rule-based @xcite and machine learning @xcite @xcite techniques. our goal is to provide a theoretical overview of various features of semantics and what choices are available for including them in the design of a new semantic representation formalism. the set of features can also serve for comparing different formalisms. in this spirit, we present some existing formalisms foot_0 and compare them against the outlined features. for a more entailment-balanced view and an empirical comparison, we also compare these formalisms against @xcite 's fracas corpus. we focus on sentence-level semantics, but provide a short discussion on multi-sentence awareness for semantic representation formalisms. the rest of the paper is organised as follows: in §2, §3 and §4 we present some global and structural features to be taken into consideration when comparing or designing a semantic representation formalism. in §5, we briefly present the following formalisms: conceptual graphs (cg) @xcite , montague semantics (ms) @xcite @xcite , discourse representation theory (drt) @xcite , minimal recursion semantics (mrs) @xcite , abstract meaning representation (amr) @xcite , universal conceptual cognitive annotation (ucca) @xcite , universal decompositional semantics (uds) @xcite , and uniform meaning representation (umr) @xcite . in §6, we compare the formalisms across the selected features, and fracas features. ## pre-semantics issues in this section, we outline a few aspects that we consider to not be constituent parts of what a semantic representation formalism (hereupon referred to as "formalism") is as such, but put it in a more global perspective and are nonetheless important to consider when designing one. scalability. semantic representation formalisms vary in terms of complexity and expressive power. while more complex ones may be more robust and encode a wider range of phenomena, complexity is negatively correlated to readability and therefore scalability. to be able to make use of various machine learning methods for parsing and generation, we need large amounts of manually (or at least semi-manually) annotated data. however, if the representation formalism is more complex, the skills required for annotation are more specialised. this makes the pool of potential annotators smaller and requires a longer and more in-depth training. a balance is necessary to ensure that a formalism covers a wide range of phenomena, yet it is not overly complex in order to keep the threshold for annotators relatively low. alternatively, formalisms could propose lattices @xcite foot_1 for different phenomena, similarly to what umr does. this would allow for a more coarse-grained annotation by less specialised annotators, and a more in-depth one by more specialised annotators. this balance is also beneficial for analysis and comparison of different formalisms, as getting comfortable with reading and interpreting the representations is more straightforward. universality. the general intuition when talking about multi-linguality, is that meaning is preserved across languages. thus, the semantic representation for a given text should be the same in all languages. in reality, many semantic representation formalisms are built upon the syntactic structure of sentences, which can differ greatly, especially between pairs of languages from distant families foot_2 . furthermore, syntax-based semantic formalisms (and syntax-agnostic ones too) have historically been developed with well-resourced languages (mostly indo-european, and in particular english) in mind. thus, formalisms are likely skewed towards better representing phenomena that occur in those languages, and might even miss phenomena that do not appear in them. finally, similarly to scalability, using lattices could be beneficial for universality as the same semantic phenomenon may contain a more finegrained set of categories in some languages than in others. using lattices allows smoother annotation in different languages, while still keeping the possibility for cross-language comparison. unicity. unicity addresses whether a formalism has a unique representation for a given meaning. in amr inverting the direction of relations changes the focus and thus the meaning of the representation. on the other hand, if we take the formal logic representation of a sentence containing negation and conjunction, and apply de morgan's laws, we end up with a different, but logically equivalent representation. if equivalent representations are allowed for a certain formalism, it may be necessary to establish what constitutes the canonical form and how members of an equivalence class relate to it. flavor. we use the term "flavor" as used by @xcite in relation to the level of abstraction from surface form for graph-based semantic representations. @xcite define three levels: flavor 0 -a one to one correspondence between graph nodes and surface tokens; flavor 1 -all tokens are present as nodes, but there are additional nodes in the graph too; flavor 2 -not necessarily all tokens correspond to nodes, and there may be nodes that do not correspond to a specific token. use of lexical resources. some formalisms rely on lexical resources for predicate and concept senses, or argument structure of predicates. this works well for languages where these resources already exist and are well-developed. however, for languages where this is not the case, there may be the need to produce them in parallel with producing annotations for the formalism, like the creators of umr propose @xcite . while viable, this makes the process longer and more complex and should be taken into consideration for the design. it is also tied to the universality aspect: in order to enable cross-language comparison, for formalisms that do use lexical resources, there needs to be a link between said resources for multiple languages. while efforts exist in this direction @xcite , for most languages, this link is not there yet. ideally, when creating datasets for a new language, the linking to other languages can be created in the process too. this, again, entails more effort, but we believe the cost of that is worth the benefits of having a more complete resource. ## semantic features in this section we discuss aspects of semantics that constitute what a semantic representation is. predicate-argument structure. the most prominent feature of many formalisms is that they are centered around the predicate-argument structure of the events occurring in a sentence. events are usually represented as predicates that take a certain number and kind of arguments. the relation between a predicate and an argument is expressed via a semantic role, which can be predicate specific (in the spirit of propbank @xcite ) or from a generic closed set (like verbnet @xcite ), with varying granularity. practical issues here arise from the fact that different formalisms use different lexical resources, making comparison and transformation more challenging. for english, work has been done to align @xcite and continue to improve the alignment @xcite of these resources. however, english is one of few languages where lexical resources are comparatively well developed. thus, the use of language-specific frames for predicates comes with the cost of developing such resources. this is an argument against their use and for adopting methods that do not encode such senses, making the universality point more easily attainable, similar to what ucca does @xcite . that may, however, make the formalism less expressive. temporality. temporal information deals with when an event occurs. we consider two aspects of this -when it occurrs relative to other events in the text, and when it occurred relative to the moment of speaking. temporal information can be encoded in a variety of ways -via grammatical tense, from the lexicon with certain adverbs, or specific words or phrases, or may even be implicit. combined with the fact that different languages have a stronger preference for some approaches over others, the task of encoding it is challenging. formalisms need to decide whether temporal information will be encoded at all, and whether all kinds, that is, whether grammatical tense will be considered or only information present on the surface. aspect. complementary to grammatical tense, grammatical aspect expresses how an event develops over time -whether it is one-time, whether it is continuous, whether it ended or is still ongoing. here, again, formalisms have a choice -whether to encode aspect, and which features of it. spatial information. as abend and rappoport (2017) point out, spatial information in semantics is considered mainly for domains such as geographical information systems and robotics navigation. from a more theoretical perspective, we consider the resolution/interpretation of location-related deictics (here/there) and demonstrative pronouns to be an important aspect of the representation. encoding spatial information is especially relevant for sign languages, where its semantics is richer. for example, the handshape can express a distinction in an object's shape (e.g. curved or flat object) @xcite , and the orientation of the handshape can express an object's orientation @xcite . reification. reification in semantics is the process of transforming events, actions and concepts so that they are expressed with (quantifiable) variables. this facilitates the translation of the so transformed representation into first-order logic and is therefore an important consideration if we want to use a formalism for logical inference. scope. the scope of semantic operators (such as those of quantification or negation) shows to which entities or events that operator applies. some formalisms choose to not encode scope at all, making consistent logical inference impossible. scope does not directly relate to word order, which gives rise to scope ambiguity -a single sentence containing more than one scope operator can be interpreted in more than one way depending on how the operators combine. in case of scope ambiguity, the question for formalisms is whether to force a specific interpretation or to leave the representation underspecified. the latter allows that restrictions are added at a later stage when the correct interpretation becomes obvious from the context. negation. negation, similarly to many of the other phenomena, can be expressed in different ways -overtly as a separate token, or as a morpheme of a token, presenting a challenge of whether to encode the two in the same way. we believe that meaning-wise, they should be equivalent and semantic representations should abstract away from the difference between the two. this is especially important when we consider logical inference and scope. indeed, this is what many of the formalisms in section 5 do. there are some exceptions, notably ucca, where, for example, the phrases "not clear" and "unclear" would be encoded differently despite having the same meaning. modality. modality is used to express the reality of an event: realis -whether is it actually realised, or irrealis -whether it is a possibility or neces-sity. modal expressions are often expressed on the surface as modal auxiliaries, adverbs or adjectives. they get special treatment for most formalisms, be it as specially dedicated predicates (as in amr), or operators between boxes (as in some realisations of drt). modal expressions are also categorised in flavors (different from the flavor we discussed in section 2), showing how the possibility under discussion is linked to reality. epistemic flavor covers possibilities based on some knowledge or belief, while deontic flavor expresses that the possibility is in accordance with what is required in reality. evidentiality is a phenomenon that encodes the type of evidence the speaker has for a statement. for example, one may differentiate between the speaker having direct (e.g. visual) or reportative (e.g. having heard about it and merely repeating what they have been told) evidence. in most languages this is expressed lexically with specific phrases (such as "reportedly" in english). however, in about a quarter of the world's languages, these differences are expressed grammatically @xcite , which formalisms do not address. ## semantics interfaces in this section we outline structural features of formalisms that are linked to their applications or to interfaces of semantics with syntax and pragmatics. ## semantic formalisms in this section, we describe existing formalisms and their core features, with a more exhaustive comparison in section 6. we strongly believe in the benefits of data-driven analysis and comparison. therefore, if existent beyond a toy-corpus size, we also point to existing datasets. various extensions have been proposed for many of the formalisms. however, we do not know, for every extension, how it combines with the other ones and whether it does not interfere with the properties we explore. for example, adding scope interferes with compositionality. thus, for this study we work with the original formalism, unless the extensions have been combined in a standalone one (as is the case with umr). montague semantics (ms) @xcite @xcite introduced mathematical methods, namely higher-order predicate logic and lambda calculus, to semantics. its core features are the use of model theoretic semantics, and compositionality. conceptual graphs (cg) (sowa, 1984) are based on semantic networks and c.s. peirce's existential graphs. aside from natural language semantics, cgs have also been influential in knowledge representation. cgs' most apparent difference from modern semantic graphs is that they encode all events, entities, and relations as nodes, whereas edges are unmarked. original cgs do not encode scope, but later versions provide that, along with ways to deal with temporal and modal logic @xcite and work has been done to combine cgs with generalized quantifiers @xcite . discourse representation theory (drt) @xcite is a "dynamic semantics" formalism, i.e. the meaning of a sentence is considered with respect to its potential to update context. it was designed to deal with anaphora and tense, but has since evolved to treat other semantic aspects such as presupposition, and propositional attitudes. drt expressions are called discourse representation structures (drs). they are usually presented as nested boxes, but those can be transformed into graphs @xcite . the parallel mearning bank (pmb) @xcite is a large drs corpus with gold annotations in english, german, italian and dutch. minimal recursion semantics (mrs) @xcite is a formalism from the headdriven phrase structure grammar (hpsg) @xcite family. as such, it has a strong link to syntax, but is also meant to be universal. mrs annotates a large range of phenomena, but is a rather complex formalism for annotators without linguistic knowledge. a distinguishing feature is its underspecifiability, which allows for encoding scope ambiguity. a medium-sized parallel dataset has been annotated for 15 languages 6 abstract meaning representation (amr) @xcite is meant to be a simple formalism to increase the ease of annotation. this is achieved by ignoring features such as tense, plurality and definiteness. amr's core is the predicate-argument structure of events, with additional non-core roles specified for predicate-independent relations. for english, amr relies on propbank @xcite for predicate senses and semantic roles. a multitude of extensions have been proposed for amr for various aspects such as tense @xcite , scope @xcite , spatial information @xcite , multi-sentence information @xcite . despite its being designed with english in mind, amr has also been used for chinese, czech, and korean, among others. larger corpora are available for english under a paid license, smaller ones are freely available 7 . universal conceptual cognitive annotation (ucca) @xcite is likewise designed to be simple for annotators. ucca's foundational layer (fl) uses a set of 14 broad semantic role categories (e.g. participant, adverbial) and does not rely on lexical resources. the latter point makes it easier to adopt for multiple languages. extension layers exist for ucca that deal with semantic roles @xcite , co-reference @xcite and implicit arguments @xcite . there are datasets for the fl for english, german, french, hebrew and russian. 8 universal decompositional semantics (uds) @xcite adds a number of semantic layers on top of the syntactic universal dependencies (ud) 9 . uds follows the principle of decomposition, e.g. for semantic roles, they take dowty (1991)'s view on decomposing notions such as agent into finer properties like volition and awareness, allowing a single predicate to be assigned multiple of these categories. the currently existing layers address semantic roles; irrealis vs realis distinction on events; predicate senses and entity types; genericity; and duration and relative order of events. annotated datasets are available for english 10 . uniform meaning representation (umr) @xcite is a proposal that extends amr with aspect, temporal information, scope, co-reference and modal dependencies. umr takes into account the morphosyntactic differences between languages and, to the best of our knowledge, is the first formalism to propose concrete steps on how to proceed with annotation for low-resource languages. ## comparison and discussion in this section, we compare the formalisms from section 5 across the features outlined in §2, §3 and §4, as well as the phenomena covered by the fracas corpus @xcite . ## conclusion in this paper we proposed a set of structural and global features to use when comparing semantic representation formalisms. we hope this set of features can be helpful for the community, both in the design of new formalisms and extensions, and in the selection of formalisms to use for specific tasks. the list of features is by no means complete, and extending it as well as the number of formalism can be the subject of future works. similarly, we believe a more fine-grained study on the expressivity of formalisms with respect to the fracas corpus would be beneficial for the community.