P olish corpus of verbal multiword expressions (2020)

Abstract: This paper describes a manually annotated corpus of verbal multi-word expressions in Polish. It is among the 4 biggest datasets in release 1.2 of the PARSEME multiligual corpus. We describe the data sources, as well as the annotation process and its outcomes. We also present interesting phenomena encountered during the annotation task and put forward enhancements for the PARSEME annotation guidelines.

## introduction multiword expressions (mwes), such as at times, red tape or take off, are word combinations with idiosyntactic behaviour, notably non-compositional semantics. therefore, they constitute a challenge for linguistic modelling and semantically-oriented text processing. verbal mwes (vmwes), like to bear sth in mind, are particularly challenging due to their partly regular and partly idiosyncratic morphosyntactic flexibility, and their frequent discontinuity in texts (savary et al., 2018). these challenges are even harder in languages like polish, with rich inflectional morphology and a relatively free word order. in order to bring progress to mwe modelling and processing, the parseme initiative has been coordinating multilingual efforts towards annotating vmwes in corpora and their automatic identification in texts (ramisch et al., 2018). this paper describes the most recent version of the polish corpus, which is the 4th biggest dataset in edition 1.2 of the parseme suite (ramisch et al., 2020). we show how the basic definitions from the parseme methodology apply to polish (sec. 2), we analyse the state of the art in polish mwe-annotated corpora (sec. 3), we describe the construction of our corpus (sec. 4) and its outcomes (sec. 5). we evoke some challenging phenomena and lessons learned from manual annotation (sec. 6) and on this basis we put forward some recommendations for enhancing the parseme annotation guidelines (sec. 7). finally, we conclude and sketch perspectives for future work (sec. 8). ## verbal multiword expressions in polish the polish vmwe dataset is integrated in the parseme corpus annotation methodology. the latter increasingly relies on (version 2 of) universal dependencies (ud), a de facto standard for morphosyntactic annotation (nivre et al., 2020). thus, we largely follow the definitions of both initiatives. firstly, we differentiate words (linguistically motivated units undergoing syntactic relations) from tokens (technical items resulting from corpus segmentation). this difference is notably visible in multiword tokens (mwts), highly productive in some polish verb forms like widział|em 'i saw' (cf. sec. 6). we further understand mwes as combinations of words which: (i) have at least two lexicalized components, i.e. components always realized by the same lexemes, (ii) display lexical, morphological, syntactic or semantic idiosyncrasies. for instance, in postawić kogoś w stan gotowości (lit. 'put sb into state of-readiness') 'to put sb on alert', 1 the object stan 'state' must receive a complement (here: gotowości 'of-readiness'). this is idiosyncratic since such a complement is not required in non-idiomatic constructions, like postawić miotłę w k ąt 'put the-broom in the-corner'. however, this complement, even if compulsory, is not lexically fixed: postawić kogoś w stan gotowości/pogotowia/oskarżenia/upadłości/etc. (lit. 'to-put sb into state of-readiness/emergency/accusation/bankruptcy/etc.'). therefore, only the words postawić w stan count as lexicalized components. vmwes are mwes whose canonical form, i.e. the least syntactically marked form keeping the idiomatic reading, is such that its syntactic head is a verb v and its other lexicalized components form phrases directly dependent on v. this means that a canonical form is a weakly connected graph (i.e. fully connected if directions of the dependencies are disregarded). consider the example obiektywna rola, jak ą uczelnie odgrywaj ą w polsce 'objective role which universities play in poland'. here, the noun rola 'role' heads the verb odgrywaj ą 'play' rather than vice versa. since a construction with a relative clause is syntactically more marked than without it, we have to transform it into a canonical form, e.g. uczelnie odgrywaj ą obiektywn ą rolę w polsce 'universities play an objective role in poland'. this is why we can consider this candidate as headed by the verb and passing the light verb construction tests. five out of the ten vmwe (sub)categories from the parseme guidelines v 1.2 are relevant to polish: another category potentially pertaining to polish are inherently adpositional verbs (iavs), defined as combinations of a verb v and an adposition a (i.e. a preposition in polish), such that: (i) v never occurs without a, as in polegać na kimś 'to rely on someone', or (ii) a significantly changes v's meaning, as in o co tu chodzi? (lit. 'about what here goes') 'what is the matter here?'. iavs were to be experimentally and optionally annotated in parseme corpora since version 1.1. in polish, we performed this annotation in edition 1.1 but iavs proved too hard to distinguish from 'regular' verbal valency with the current annotation guidelines. therefore, we abandoned the iav annotation in edition 1.2 of the polish corpus. ## multiword expressions in polish treebanks in previous work on modelling and annotating polish mwes, lexicon, grammar and treebank construction efforts have often been closely related. głowińska and przepiórkowski (2010) and głowińska (2012) present the manual shallow syntactic annotation of the national corpus of polish (nkjp). @xcite the whole corpus follows multilayer annotation principles. in particular the layer of syntactic groups (roughly chunks), builds upon the layer of the so-called syntactic words, which in turn builds upon the layer of tokens. the layer of syntactic words includes a number of (mostly) continuous mwes such as multiword prepositions (w duchu czegoś 'in the spirit of sth'), adverbs (do czysta 'completely') or conjunctions (a zatem 'that is'). those are not explicitly marked as mwes but can be queried by looking for word nodes which point at least two token nodes. @xcite all mwes delimited in this way are decorated with their parts of speech. verbal mwes are not covered. nkjp is released with a shallow grammar developed for its automatic pre-annotation. among the 1,187 grammar rules, 350 are lexicalized rules describing mwes. fragments of the nkjp corpus have been transformed into the constituency treebank składnica. on top of the previous morphosyntactic annotation described above, the constituency parser świgra produced candidate trees, which were then manually disambiguated ( świdziński and woliński, 2010). a recent version of składnica (woliński et al., 2018) integrates data from a valency dictionary walenty. @xcite walenty has a rich phraseological component (przepiórkowski et al., 2014; hajnicz et al., 2016) and a semantic layer. on the morphosyntactic level, verbal mwes are represented as valency frames in which some arguments are lexically fixed, e.g. zobaczyć coś na własne oczy (lit. 'to see sth on own eyes') 'to see sth for oneself' receives a frame with the head verb zobaczyć 'see', a free subject and object, and a lexicalized complement na własne oczy 'on own eyes'. on the semantic level, adverbial, nominal, adjectival and other mwes can appear as lexicalized elements of verbal frames, e.g. a multiword adverb w trupa 'into a dead body' occurs as a possible lexicalized realization of the semantic role of manner in the verbal frame of upić się 'get drunk', the whole combination meaning 'to get totally drunk'. the latest dowloadable walenty version (from 2016) contains notably over 60,000 syntactic verbal frames, 14,295 of which have lexicalized arguments, i.e. correspond to vmwes entries. walenty frames were integrated into the świgra's grammar, which was then used to enhance składnica. the latter does not seem to explicitly indicate which tree nodes correspond to lexicalized components of vmwes from walenty. such efforts of making mwe occurrences in składnica explicit were undertaken in two polish ud treebanks. in the polish dependency bank (pdb), wróblewska (2012) automatically converted the continuous mwes into dependency chains using the mwe relation (pertaining to ud version 1). later, pdb was enlarged with new texts and converted into ud version 2, with the fixed and flat dependencies marking morphologically fixed mwes and named entities, respectively. the number of both types of labels in pdb version 2.5 is 3,850 and 5,525, respectively. later the whole nkjp corpus foot_4 was enriched with dependencies, using a parser trained on pdb, and manually correcting major flaws (wróblewska, 2020). there, the fixed and flat dependencies most probably follow the same principles as in pdb, but no statistics of these specific labels were available at the time of writing. it is also unclear if any fixed mwes were marked except those predicted by the parser, i.e. the coverage of mwes is unclear. in parallel to the above treebanking efforts involving świgra, walenty and ud conversion, similar work was done in the lexical functional grammar framework. patejuk and przepiórkowski (2014) developed an lfg grammar of polish, integrated with walenty, parsed texts stemming mainly from nkjp, and manually disambiguated them to obtain an lfg treebank. they further performed an automatic conversion of this treebank into the ud version 2 (przepiórkowski and patejuk, 2020), including the so-called enhanced dependencies @xcite the resulting ud-lfg treebank contains 144 and 884 fixed and flat dependencies, respectively. like in pdb, the former are limited mostly to continuous morphosyntactically fixed mwes, and the latter to named entities, i.e. the information about verbal mwes from walenty is not propagated to the treebank, and nominal/adjectival mwes are neglected. an effort focused on explicitly marking occurrences of large classes of mwes in składnica was undertaken by savary and waszczuk (2017). they used 3 resources: (i) walenty, (ii) the named entity annotation layer of the nkjp corpus, and (iii) sejf, an electronic lexicon of polish nominal, adjectival and adverbial mwes, with 4,700 multiword lemmas, 160 inflection graphs and 88,000 automatically gener-ated inflected forms (czerepowicka and savary, 2018). these 3 resources were automatically mapped on składnica, and the outcome was manually validated, which resulted in the składnicamwe treebank with explicit marking of over 1,300 named entities, as well as 450 verbal and 400 nominal/adjectival/adverbial mwes. @xcite differently from the previous efforts, this time, the treebank remains in its original constituency format, and information about mwes is added to selected tree nodes as additional features, together with pointers to those lexical nodes which represent lexicalized components of the mwes. this is in sharp contrast with the ud encoding, where dependencies indicating the mwe status potentially compete with those marking the syntactic relations. skladnicamwe is also the first polish treebank with an explicit marking of verbal, nominal and adjectival mwes. this resource would be worth extending with entries from verbel, a more recent grammatical e-lexicon of verbal mwes. @xcite in the context of this state of the art, we describe the first attempt towards systematic annotation of polish verbal mwes in running text. we do not use any pre-annotation methods so as to avoid bias. the resulting resource is fully integrated into the parseme suite of multilingual treebanks annotated for verbal mwes (savary et al., 2018; ramisch et al., 2018; ramisch et al., 2020). it follows the crosslingually unified and validated annotation guidelines and the centralized quality insurance methodology. ## constructing the polish vmwe-annotated corpus all the manual annotations of vmwes were performed on texts coming from one of three (more or less overlapping) sources (cf. sec. 3): (i) nkjp1m, a 1-million word manually annotated subcorpus of nkjp; (ii) pcc, polish coreference corpus (ogrodniczuk et al., 2015); (iii) pdb (cf. sec. 3). from the first two sources we only took newspaper texts, while pdb provided a mixture of news, periodicals, literature, fiction, popular science, social media, parliamentary debates and manuals. the source corpus and the text genre of each sentence are indicated in its comment, as documented in the corpus repository. @xcite like all corpora in the parseme suite v 1.2, the polish dataset is released in the .cupt format, @xcite an instance of the conll-u plus format foot_10 defined for annotations built upon ud corpora. fig. 1 shows the first sentence of a corpus file. the first line is global to the whole corpus and gives the headings of the 11 columns. the first 10 stem from the conll-u format, and the 11th contains the vmwe annotations. here, tokens 1-2 belong to an irv postarać się (lit. 'try rcli') 'try hard', which overlaps with another irv encompassing tokens 2-5 się pogodzić (lit. 'rcli reconcile') 'make it up (with someone)'. henceforth, the first 10 columns of a .cupt file will be referred to as morphosyntactic annotation. by morphological annotation alone we mean columns 3-6 (lemma, upos, xpos and feats) and by syntactic annotation alone, columns 7-8 (head and deprel). morphosyntactic annotation is considered compatible with ud (in version 1 or 2) if is follows the ud annotation guidelines (in the corresponding version). @xcite it is further considered compatible with a certain release of ud, e.g. with ud 2.5 if, for the same sentences, it contains the same data as this release or if it is automatically generated using a parser trained on this release. the corpus in version 1.2 extends and enhances the one in version 1.1. firstly, we annotated new texts and made the previous and the new annotations mutually consistent (sec. 4.1). secondly, we updated the morphosyntactic annotation to make it compatible with the ud version 2.5 (see sec. 4.2). finally, we provided a companion raw corpus, automatically annotated for morphosyntax (sec. 4.3) and meant for automatic discovery of unseen vmwe. ## results the resulting ud-2.5-compatible corpus, manually annotated for vmwes, comprises 23,547 sentences, 396,140 tokens, and 7,186 manually annotated vmwes in total. 12,187 sentences originate from pdb-ud, 9,241 from nkjp1m and 2,119 from pcc. morphological annotation is manually performed in the first two sources 21 and automatically in the third one. syntactic annotation is manual only in pdb. 7,426 new sentences from pdb-ud were added in edition 1.2. while the corpus covers a rather broad spectrum of different genres (cf. sec.4), a large majority (over 68% sentences) are newspaper texts. double annotation performed over 900 newspaper sentences, new in edition 1.2, resulted in inter-annotator agreement (iaa) scores of f @xmath0.4% (f-measure between annotators), κ @xmath1.2% (agreement on the annotation span) and κ @xmath2.7% (agreement on the vmwe category). @xcite see (savary et al., 2017) for the definitions of these three iaa measures. table. 1 presents the statistics of the corpus concerning the different vmwe categories as well as the fine-grained vmwe phenomena -discontinuity, one-token length and overlapping -as defined by savary et al. (2018) -in comparison with version 1.1 of the corpus. the number of overlapping vmwe tokens decreased since version 1.1 most likely due to the removal of iavs (annotated experimentally in version 1.1), which often co-occur with other vmwes. figure 2 (a) illustrates the variability of the different categories of vmwes in the polish corpus. we follow the parseme-based definition of a variant: it is a sequence of words starting from the first vmwe component and ending on the last vmwe component, including the non-lexicalized words in between. the linear regression models fitted to the numbers of different variants of various categories suggest that lvc.cause and lvc.full vmwes are the most variable, followed by vids, which in turn are more variable than irvs. figure 2 (b ) on the other hand shows the variability of vmwes in the polish corpus in general, in contrast with several other parseme 1.2 corpora. it shows that, even though morphologically rich and with relatively free word order, vmwes in polish are not as variable as those in chinese or turkish, and have a similar level of variability as vmwes in german or basque. interestingly, the variant-of-traindev f-scores 23 achieved by the two best systems, in both the open and the closed track of the parseme shared task 1.2, are higher for polish than for any other language. however, it can be stipulated that the variability captured by the parseme definition is influenced by non-related factors such as the average length of the (non-lexicalized) gap, 24 which is in particular significantly higher in the german (average gap length 2.06) than in the polish corpus (average gap length 0.55). @xcite 6 findings from the manual annotation this section describes selected interesting phenomena, challenging cases, as well as findings and lessons learned from the manual annotation, across all 3 versions of the polish parseme corpus. ## towards enhanced parseme guidelines several enhancements in the parseme annotation guidelines can be proposed based on our experience. firstly, nesting of vmwes should be more accurately accounted for. currently, the verb in an lvc is allowed to only have one lexicalized dependent (test s.2), which excludes inherently reflexive light verbs, as in nosić się z zamiarem (lit. 'carry rcli with intention') 'to have an intention'. such examples can only be annotated as vids, although they function like lvcs. we might therefore allow for more than one lexicalized dependent of the verb in test s.2, provided that all but one of them belong to a previously annotated vmwe. this would also allow verb-particle constructions (vpcs) @xcite to be nested in irvs, as in (de) er [[stellt] v p c sich [vor] v p c ] irv (lit. 'he puts rcli forward') 'he imagines' (now such cases are formally vids). secondly, the reciprocal uses of the rcli listed in test irv.8 do not accurately cover slavic languages. the test checks if a plural or coordinated subject can be distributed over two occurrences of the same verb. for instance jan i ela się całuj ą (lit. 'jan and ela rcli kiss') 'jan and ela kiss each other' can be transformed into jan całuje elę, a ela całuje jana 'jan kisses ela and ela kisses jan'. therefore, całować się (lit. 'kiss rcli') is a reciprocal use of się and not an irv. but in polish, there is another reciprocal form with a singular subject and an oblique: jan całuje się z el ą 'jan kisses rcli with ela'. adding this case to irv.8 is necessary, at least for language-specific variants of this test. but this is not sufficient since the verb alone does not admit the same subcategorization: *jan całuje elę z elen ą (lit. 'jan kisses ela with elena'). thus, test irv.3 is always passed, and such cases have to be annotated as irvs, although they are productive. a possible solution would be to change the order of the irv tests so that those checking non-idiomatic uses of the rcli (currently irv.4 to irv.8) are placed first. thirdly, specific constructions with duplicated verbs invalidate some genuine irvs (cf. sec. 6.4). language-specific lists of such constructions, to be neglected by test irv.1, could be proposed. finally, an open problem is how to ensure that the decision diagrams always yield the same outcome for the same sense of a verb, whatever its non-lexicalized arguments. in stawiam sobie/komuś cel (lit. 'iput myself/someone a goal') 'set a goal to myself/someone', the outcome of test lvc.2 depends on the indirect object. with a reflexive object sobie 'myself', the subject of the verb (i) is the agent/beneficiary of the noun cel 'aim', which suggests the lvc.full status. but with another object, the verb's subject does not fill any semantic role of the noun, which leads to lvc.cause. we would of course like both of these uses to be annotated in the same way, here as lvc.cause. but this would imply applying the test to all possible instances of the (non lexicalized) object, rather than to the precise example being annotated. with such a major difference in the annotation strategy, the decision replicability might be jeopardized. @xcite ## conclusions we described the construction of the polish corpus of vmwes, which is the 4th biggest dataset in the parseme suite. we presented some details of the annotation process and its outcomes. we also discussed some polish-specific phenomena, interpreted in the light of the parseme annotation guidelines. we displayed several drawbacks of these guidelines and put forward suggestion for their enhancements. we believe that these observations can help continuous enhancement of the parseme methodology, and can be useful to annotators of other languages, linguists studying the mwe phenomenon, as well as authors of vmwe identification tools.