MWE as WSD : Solving Multiword Expression Identification with Word Sense Disambiguation (2023)

Abstract: Recent approaches to word sense disambiguation (WSD) utilize encodings of the sense gloss (definition), in addition to the input context, to improve performance. In this work we demonstrate that this approach can be adapted for use in multiword expression (MWE) identification by training models which use gloss and context information to filter MWE candidates produced by a rule-based extraction pipeline. Our approach substantially improves precision, outperforming the state-of-the-art in MWE identification on the DiMSUM dataset by up to 1.9 F1 points and achieving competitive results on the PARSEME 1.1 English dataset. Our models also retain most of their WSD performance, showing that a single model can be used for both tasks. Finally, building on similar approaches using Bi-encoders for WSD, we introduce a novel Poly-encoder architecture which improves MWE identification performance.

## introduction word sense disambiguation (wsd), the task of predicting the appropriate sense for a word in context, and multiword expression (mwe) identification, the task of identifying mwes in a body of text, both deal with determining the meaning of words in context @xcite . they have traditionally been treated as separate tasks, but this is potentially disadvantageous as wsd performed on words which are part of unrecognized mwes cannot produce correct meanings, and the meanings of polysemous mwes are ambiguous even after identification. for example, the sentence "she inherited a fortune after her grandfather kicked the bucket" tells us that someone's grandfather has died, but we would not expect to find meanings associated with death in the sense inventories of either kick or bucket. wsd cannot *both authors contributed equally to this work. wsd and mwe identification can be used in preprocessing to improve performance of downstream tasks such as machine translation or information extraction @xcite @xcite . they also have more direct applications in helping language learners -for whom mwes are particularly challenging @xcite -understand the meaning of words or mwes in context. in this paper, we propose a system that tackles these tasks together, using a mwe lexicon and rulebased pipeline to identify mwe candidates and a trainable model to both perform wsd and filter mwe candidates. our model is a modified polyencoder @xcite , a natural extension of previous work using bi-encoders for wsd @xcite . utilizing gloss information 1 allows our model to consider the meaning of mwes and filter out candidates where the constituents of a mwe are present but the mwe meaning does not fit the context, such as the aforementioned literal usage of kick the bucket. our method improves precision and achieves state-of-the-art f1 for mwe identification on the dimsum dataset @xcite and competitive performance on the parseme 1.1 english data @xcite . to the best of our knowledge, this work is the first to use glosses as a resource for mwe identification. our contributions are summarized as follows: we make all of our code, models and data public foot_1 . ## methodology in this section, we explain how our models perform mwe identification and wsd, and how our mwe identification pipeline works. figure 1 : each model scoring the mwe take advantage. "draw advantage from" is the gloss for one possible sense. the gloss encoder produces sense representations rs using the [cls] embedding in all models. the mwe representation rw is an average of constituents for the bi-encoder and the combination of attention for each code for the poly-encoder. the dca poly-encoder learns separate codes for target and non-target tokens, allowing it to attend differently to the mwe and surrounding context. scores are the similarity between rs and rw computed as the dot product, and the model predicts the sense with the highest score. ## results our system achieves moderate performance on parseme and competitive performance on the dimsum trained only on the modified semcor data. when fine-tuned on both mwe identification datasets it further improves, reaching state-ofthe-art performance on dimsum. systems finetuned on either parseme or dimsum alone perform even better on their corresponding test set, but worse on the other test set, likely due to differences in domain and mwe type between the datasets. high precision stands out as a strength of our approach, but it suffers from low recall -even the rule-based pipeline with no model filter lags behind other systems in recall. we attribute this mainly to the issue of lexicon dependence described in section 8; mwes missing from our lexicon account for a majority of our false negatives as we show in our error analysis (section 6.2). these findings echo @xcite on the importance of lexicons for mwe identification, and suggest that there is room to improve performance by expanding the lexicon. while it is difficult to pinpoint exactly why we achieve state-of-the-art f1 on dimsum and not parseme, one significant difference is that more for trainable models we report the mean (± standard deviation for the f1 score) of three runs with random seeds. because our system uses gold pos tags/lemmas to look up sense glosses, we compare against systems using gold information where available, such as for @xcite and @xcite . ## analysis dataset type sentence note parseme fp ...were propped up on a foot-warmer, ... prop up never marked as mwe in dataset parseme fn never mind, mrs. bray will join you later. never mind missing from lexicon parseme fp ...his mind drifted off to the accounts... drift off sense "fall asleep" does not apply parseme tn textit...as we sat side by side... sit by sense "be inactive" does not apply dimsum fp aww, thank you. thank you marked as mwe in 4 other sentences dimsum fn all our dreams can come true,... come true missing from lexicon dimsum fn ...this was a breathe of fresh air. present in lexicon; model filter false negative dimsum tn ...impact my wardrobe has on the environment. have on sense "dress in" does not apply we also see that while removing rule-based filters from the dca pipeline lowers parseme f1, it slightly raises dimsum f1, suggesting that the necessity of these filters depends on the data. however, removing the rule-based filters only works because the dca poly-encoder can accurately exclude false positives: removing the same filters from the purely rule-based pipeline results in a very low f1. finally, the dca poly-encoder substantially outperforms the standard poly-encoder (polyenc) on both datasets and surpasses the biencoder on parseme, demonstrating that our dca poly-encoder model can improve mwe identification performance. ## conclusion in this work, we present an approach to mwe identification using rule-based candidate extraction with a model filter, achieving strong results on the parseme 1.1 english data and state-of-the-art results for mwe identification on the dimsum dataset. our system performs both mwe identification and wsd with the same model, demonstrating that these tasks can be tackled together. we also introduce a modified poly-encoder architecture better suited to mwe identification. our system's strength is its high precision for mwe identification. we show its low recall to be a function of lexicon size, and in future work we intend to expand the lexicon by mining mwes and generating glosses for them, which has the potential to substantially increase recall for lexicon-based systems. improved approaches for multitask training of mwe identification/wsd models could also be valuable; the ideal pipeline would be competitive with state-of-the-art systems in both tasks, and not just mwe identification. ideal applications of our system include mwe identification when a lexicon of target mwes is available, or cases where quickly performing both mwe identification and wsd is valuable, such as in language learning and assisted reading tools.