A MWE lexicon formalism optimised for observational adequacy (2023)

Abstract: Past research advocates that, in order to handle the unpredictable nature of multiword expressions (MWEs), their identification should be assisted with lexicons. The choice of the format for such lexicons, however, is far from obvious. We propose the first – to our knowledge – method to quantitatively evaluate some MWE lexicon formalisms based on the notion of observational adequacy. We apply it to derive a simple yet adequate MWE-lexicon formalism, dubbed λ-CSS, based on syntactic dependencies. It proves competitive with lexicons based on sequential representation of MWEs, and even comparable to a state-of-the art MWE identifier.

## introduction multiword expressions (mwes), such as by and large, carbon footprint or to pull one's leg 'to tease someone', exhibit irregularities which are challenging for text processing. most notably, their meaning cannot be straightforwardly deduced from the meanings of their components, which is an obstacle for semantically-oriented applications. to help such applications process mwes correctly, one solution is to pre-identify mwes in text, so as to later apply dedicated procedures to them. recognizing mwes occurrences in texts (henceforth referred to as mwe identification) is, according to @xcite , one of the two main subtasks of mwe processing (the other being mwe discovery, the task of generating sets of mwes) and still represents quite a challenge despite having been the focus of many works. notably, parseme shared tasks on identification of verbal mwes @xcite @xcite have provided a controlled environment and focused challenges for mwe identification. each edition of the task trying to put in focus those facets of the identification task which are the hardest. one thing that parseme shared tasks definitely highlighted is that identification of mwes unseen during training proves to be significantly harder than identification of seen mwes. this can be seen in the results of editions 1.1 and 1.2 of the shared tasks when comparing the scores of various identifiers on seen vs unseen mwes. the difficulty of identifying unseen mwe should not come as a surprise as this task can be seen as presenting the challenges of both identification and discovery. seeing this discrepancy between identification of seen and unseen mwes, @xcite argue that the use of mwe lexicons is key to highquality mwe identification. thus, shifting the burden of unseen mwes on discovery and using lexicon as the interface between discovery and identification. this position is supported by experiments from @xcite that show that mwes lexical resources can be used in order to improve mwe identification. in accordance with this argument, this paper investigates mwe-lexicon formalisms, how they can be compared and introduce one such mwe-lexicon formalisms. ## multiword expression we abide by parseme's definition of a mwe @xcite , adapted from @xcite , as a (continuous or discontinuous) sequence of words, at least two of which are lexicalized (always realised by the same lexemes), which displays some degree of lexical, morphological, syntactic and/or semantic idiosyncrasy. mwes happen to present quite a few interesting properties. of all the properties listed by @xcite @xcite we will only mention the following 3 for the impact they have on how mwes can and should be represented and what mwe-lexicons need to accomplish. variability mwes can appear under a variety of forms depending on the morphosyntactic context in which they occur (e.g. i pay him a visit / the visits she pays me), their components can be found in different orders, forms, or even differently syntactically related. this makes simple representations such as sequences of forms insufficiently descriptive and pushes us to more complex representations capturing all the forms under which a mwe could appear. discontinuity discontinuity can be seen as a form of variability where component words of a mwe are not adjacent to one another but separated by a word or group of words named the insertion. we define two types of discontinuity: linear discontinuity where the component words of the mwe are not next to each other in the sentence (e.g. pay someone a visit, where 'someone a' is the insertion between 'pay' and 'visit') ; and syntactic discontinuity where a component of the mwe is not directly related by a syntactic dependency to any other component of the mwe (e.g. figure 1 where 'wanted' is the insertion between 'visit' and 'pay' 1 ). this is a visit which i wanted to pay pron aux det noun pron pron verb part verb acl:relcl xcomp not all mwes can be discontinued and anything cannot be inserted between mwe components. what can and cannot be inserted in a mwe depends on the mwe and should be described for a mwe representation to be complete. ## mwe-lexicon formalisms numerous mwe-lexicons (mwe-ls) have been put forward in the past. each of them follows a mwe-l formalism, henceforth simply called formalism, which determines what kind of information can be stored and how. unfortunately, formalisms are often only an afterthought, as a result, works on mwe-ls often focus on mwe extraction and only touch upon how mwes are represented in the mwe-l. nevertheless, formalisms can be loosely categorized based on the kind of representation used to store their lexical entries. probably one of the biggest categories of mwe-l formalisms would be those based on phrase grammars. we further divide this category into two smaller: (i) formalisms based on list-like or regexlike structures @xcite @xcite @xcite @xcite @xcite , component words are listed in the order in which they can appear and discontinuities are most often denoted by special symbols imposing constraints on the types of insertions allowed (either by limiting the number of insertions or the words which can be inserted); (ii) formalisms based on more expressive phrase grammars (cfgs, tags, lfgs, hpsgs, ...) @xcite @xcite , here component words are usually terminals appearing in grammar rules, and discontinuities are denoted by non-terminals. less frequent are dependency-based formalisms, like pdt-dep @xcite , in which only bigrams of syntactically dependent words are considered. 2 other popular categories are driven by semantics @xcite or relational databases @xcite . these categories do not cover all possibilities and whether a specific mwe-l belongs to one category over another could be disputed. ## evaluation of mwe-lexicon formalisms seeing all these different mwe-ls and formalisms, one might ask which one is best in order to assist mwe identification. one part of the answer comes us from @xcite which recommend that mwe-ls aiming to assist mwe identification should be distributed in extensional and standard format, and that the lemmas and pos of mwes' component words, as well as the least syntactically marked dependency structure and some other morphosyntactic variants judged relevant should be accessible. the other part of the answer comes us from looking at how mwe-ls have been compared up until now. to our knowledge, there are only few studies comparing mwe-ls. parseme's survey @xcite references more than fifty mwe lexicons and lists in dozens of languages, and compares their accessibility, languages represented, size, and capacity to encode discontinuous mwes. @xcite compares a few lexicons of continuous mwes showing how their formalisms allow one to encode salient mwe properties. such comparisons are relevant to our work but are mostly qualitative in nature. formalisms are compared on what they can and cannot express and quantitative comparisons are almost exclusively reserved to compare mwe-ls' sizes. to our knowledge, mwe-l formalisms themselves have not yet been compared quantitatively. this brings us to the question of how mwe-l formalisms can be quantitatively compared. ## adequacy in order to evaluate mwe-ls, we borrow the notion of adequacy, first defined for grammars @xcite then adapted to lexicons @xcite . adequacy can be divided into three levels, which, in the context of mwe-ls, can be summarized as follows: (i) observational adequacy, which evaluates the coverage of mwe observations accounted for in a mwe-l; (ii) descriptive adequacy, which estimates whether a mwe-l accurately and exhaustively describes all the properties of the covered mwes; (iii) explanatory adequacy, relating to how well a mwe-l explains the reasons behind mwe behavior. note that these three levels of adequacy call for increasingly complex lexicon formalisms, e.g. explaining an mwe behavior needs more expressive power than just listing all correct forms of this mwe. in this paper, we focus on observational adequacy (oa) since it is the easiest to quantify and is a measure of mwe identification. this choice coincides with recommendations by @xcite , who advocate that mwe identification be assisted by mwe-ls which use a relatively simple dependency-based formalism. perfect oa can more accurately be defined as the mwe-l accounting for all possible observations of mwes and only those. in other words, all possible mwes observations must be matched by at least one entry of the mwe-l. (here understood as surface forms). it follows that oa can be measured from the standpoint of generation or parsing. more precisely, mwe-ls are evaluated on their capacity to either generate all possible mwe forms, or to recognize all mwe forms encountered in text. oa can be measured in a multitude of ways. in this study we keep ourselves to precision and recall, which measure the proportion of actual mwe observations in those matched by the lexicon and in those existing in text, respectively. note that the measure of precision from a generative standpoint causes issues, since mwe occurrences can be literal (cf. sec. 6.1). finally, in order for oa to be applicable to formalisms, we propose that they should be evaluated in conjunction with an instantiation method and corpus. thus, two formalisms can be compared provided that their respective mwe-ls are instantiated on the same data, in similar fashion, and that oa is measured on the same corpus. ## λ-css lexicons now that we have suggested criteria for an optimal format of mwe-ls, let us see how this format could look like. @xcite ask what exactly is a literal occurrence of a mwe and what distinguishes it from an idiomatic or coincidental occurrence. roughly, when all the lexemes of a mwe appear in a sentence and they together display some form of idiosyncrasy, then we talk of an idiomatic occurrence of the mwe. whereas when they display no idiosyncrasy, we talk of a non-idiomatic occurrence of the mwe. non-idiomatic occurrences are furthermore divided into literal occurrences and coincidental occurrences. @xcite define the former as an occurrence which appears in a syntactic configuration in which could have been idiomatic. the latter is then simply defined as a non-idiomatic occurrence which is not literal. ## results we use the german (de), greek (el), french (fr), hebrew (he), hindi (hi), italian (it), polish (pl), portuguese (pt), swedish (sv), turkish (tr) and chinese (zh) parseme shared task 1.2 corpus @xcite . 3 given a lexicon and a sentence, we define a match as a subsequence of the sentence which is accounted for (recognized by) the lexicon. a match can correspond to an idiomatic mwe occurrence or not. in the former case, it is called an idiomatic match. then, given a lexicon and a corpus of sentences, we define: precision as the ratio of idiomatic matches to the total number of matches; and recall as the ratio of idiomatic matches to the number of idiomatic occurrences in the corpus. the aim is to maximise both measures. as proposed earlier, formalisms will be evaluated in conjunction with a given instantiation method and instantiation corpus. to that end, during instantiation phase, we collect the λ-csss of all idiomatic occurrences annotated in the instantiation corpus. this method has the advantage of being very simple to implement and to introduce very little variation during the instantiation process. its one downside (beside needing annotated data) is that some properties of mwes cannot be deduced from single observations, i.e. the descriptive adequacy of the instantiated lexicon is limited. ## concluding remarks in this paper we proposed, to our knowledge, the first method of quantitatively evaluating some mwe-lexicon formalisms through observational adequacy. we also presented a mwe-lexicon formalism based on a generalisation of the concept of a coarse syntatic structure, which we call { lemma, deprel }-css. we brought evidence that this specific set of features allows for higher observational adequacy than alternative sets of features on verbal mwes in most of the 11 languages studied. furthermore, we compared this formalism to mwe-lexicons based on sequential representation of mwes. we showed that our formalism achieves higher observational adequacy on french regardless of the fact that only verbal or all types of mwes are considered. finally, we showed the observational adequacy of our formalism holds its own even when compared to annotations produced by a state-of-the-art mwe identifier. while this study focuses on mwe-lexicon formalisms instantiated on annotated corpora, our vision is that such lexicons should be instantiated through mwe discovery in large non-annotated corpora or through extraction from other mwe resources.