Unsupervised Paraphrasing of Multiword Expressions (2023)

Abstract: We propose an unsupervised approach to paraphrasing multiword expressions (MWEs) in context. Our model employs only monolingual corpus data and pre-trained language models (without fine-tuning), and does not make use of any external resources such as dictionaries. We evaluate our method on the SemEval 2022 idiomatic semantic text similarity task, and show that it outperforms all unsupervised systems and rivals supervised systems.

## introduction multiword expressions (mwes) are notoriously difficult to model because the meaning of the whole can diverge substantially from that of the component words, e.g. the meaning of swan song ("final performance") is far removed from its component words 2 @xcite . this hampers the capacity of pre-trained language models such as bert @xcite to capture mwe semantics @xcite . similarly, nonnative speakers tend to have difficulty understanding mwes, especially those that have no equivalent in their native language @xcite . in this work, we propose a method for paraphrasing non-literal mwes (e.g. swan song) into more literal expressions (e.g. final performance) to aid both humans and machines to understand their meanings. importantly, we perform this in a fully unsupervised way: our method uses only monolingual corpus data and off-the-shelf pre-trained masked language models (mlms), and does not make use of any labelled data or lexical resources such as wordnet @xcite , in contrast with previous work @xcite @xcite . we base our experiments on semeval 2022 task 2 @xcite , a task designed to evaluate how well models understand the semantics of mwes in two highresource languages (english and portuguese) and one low-resource language (galician). we show that our model generates high-quality paraphrases of mwes and aids pre-trained models to produce better representations for sentences that contain idiomatic expressions. compared to system submissions to the shared task, our method performs better than all unsupervised systems and comparably with supervised systems. ## methodology given a target sentence t that contains an mwe x, our goal is to paraphrase x with more literal expressions in the context of t. to this end, we propose a fully unsupervised method which employs monolingual corpus data and an off-the-shelf masked language model as is, i.e. without any fine-tuning. our method consists of the following five steps: (1) collect sentences containing x from a monolingual corpus; (2) cluster the sentences; (3) generate paraphrases of x for each cluster; (4) rerank the paraphrase candidates; and (5) select the most relevant cluster to the sense of x in the target sentence t, and paraphrase x. figure 1 illustrates the overview of our model, and we describe the details of each step below. ## conclusion we propose a fully unsupervised method to paraphrase multiword expressions (mwes) in context. our method employs only a monolingual corpus and pre-trained language model, and does not rely on any labelled data. in our experiments, we show that our model generates good mwe paraphrases and aids pre-trained sentence embedding models to represent sentences containing mwes.