S een2 U nseen at PARSEME Shared Task 2020: All Roads do not Lead to Unseen Verb-Noun VMWE s (2020)

Abstract: We describe the Seen2Unseen system that participated in edition 1.2 of the PARSEME shared task on automatic identification of verbal multiword expressions (VMWEs). The identification of VMWEs that do not appear in the provided training corpora (called unseen VMWEs) – with a focus here on verb-noun VMWEs – is based on mutual information and lexical substitution or translation of seen VMWEs. We present the architecture of the system, report results for 14 languages, and propose an error analysis.

## introduction the identification of multiword expressions (mwes) such as spill the beans is a challenging problem (baldwin and kim, 2010; constant et al., 2017), all the more so for verbal mwes (vmwes) subject to morphological (spill the bean) and syntactic variability (the beans were spilled ). the parseme shared task (pst) provided training, development and test corpora (hereafter train, dev, and test) manually annotated for vmwes. @xcite our system aimed at identifying every vmwe in test which also appears in train or dev, including possible morphological or syntactic variants (henceforth seen vmwes) or not present in train/dev (unseen vmwes). unseen vmwe identification, the main focus of this pst edition, is harder than seen vmwe identification, as shown by previous results (ramisch et al., 2018). we submitted two systems: seen2seen (closed track) and seen2unseen (open track). seen2unseen relies on seen2seen for the identification of seen vmwes and has an additional module for unseen ones. its best global unseen f-score (i.e. not only for verb-noun constructions) was obtained for hindi (42.66) and it reached 25.36 in french, which was our main focus. despite the lower global mwe-based f1score of seen2unseen (63.02) compared to seen2seen (66.23), we describe the former (sec. 2), analyse its interesting negative results (sec. 3), and conclude with ideas for future work (sec. 4). ## system description while describing the architecture of our system, we use the notions of a vmwe token (its occurrence in running text) and a vmwe type (abstraction over all occurrences of a given vmwe), as introduced by savary et al. (2019b). we represent vmwe types as multisets of lemmas and pos. @xcite our system uses a mixture of discovery and identification methods, as defined by constant et al. (2017). namely, vmwe discovery consists in generating lists of mwe types out of context, while vmwe identification marks vmwe tokens in running text. the system is freely available online ( https://gitlab.com/ cpasquer/st_2020). this work is licensed under a creative commons attribution 4.0 international license. license details: http:// creativecommons.org/licenses/by/4.0/ . seen2seen in a nutshell seen2seen is a vmwe identification system dedicated to only those vmwes which have been previously seen in the training data. its detailed description is provided in pasquer et al. (2020), but a brief overview is included here to make the current paper self-contained. seen2seen extracts lemma combinations of vmwes seen in train, looking for the same combinations (within one sentence) in test, with an expected high recall. to improve precision, up to eight independent criteria can be used: ## results although seen2unseen uses 4 lists of candidates, here we analyse their contribution separately, that is, we use one list at a time in step 4 above. we report unseen mwe/token-based precision. 13 sec. 3.1 analyses the impact of m ix l , sim l and ran k l n , while sec. 3.2 discusses t ran s l for french. ## conclusions and future work we proposed an error analysis for our system seen2unseen dedicated to unseen verb-noun vmwe identification. it reveals that lexical variation and translation can produce valid unseen vmwes but their ambiguity in context must be solved: we should take into account both the dependency labels (to avoid coincidental occurrences) and the probability of the verb to be light in train (to avoid frequent co-ocurrences like fumer cigarette 'smoke cigarette'). using contextual rather than non-contextual word embeddings might also be helpful, even if computationally more intensive. we could also combine t ran s l and m ix l ∪ sim l by applying lexical substitution to the translated vmwes.