M c P hra S y: Multi-Context Phrase Similarity and Clustering (2022)

Abstract: Phrase similarity is a key component of many NLP applications. Current phrase similarity methods focus on embedding the phrase itself and use the phrase context only during training of the pretrained model. To better leverage the information in the context, we propose McPhraSy (Multi-context Phrase Similarity), a novel algorithm for estimating the similarity of phrases based on multiple contexts. At inference time, McPhraSy represents each phrase by considering multiple contexts in which it appears and computes the similarity of two phrases by aggregating the pairwise similarities between the contexts of the phrases. Incorporating context during inference enables McPhraSy to outperform current state-of-the-art models on two phrase similarity datasets by up to 13.3%. Finally, we also present a new downstream task that relies on phrase similarity – keyphrase clustering – and create a new benchmark for it in the product reviews domain. We show that McPhraSy surpasses all other baselines for this task.

## introduction estimating similarity between phrases is an important intermediate component for many nlp tasks like question answering @xcite and machine translation @xcite . as opposed to previous work (e.g., @xcite @xcite that use the phrase context only during training, we propose mcphrasy -multi-context phrase similarity -a novel method for estimating phrase similarity that leverages multiple contexts during inference to improve accuracy. mcphrasy produces a set of representations for a phrase, based on sentences in which it appears. to measure similarity of two phrases, they are compared according to their sets of representations with an innovative technique. indeed our approach achieves the new state-of-the-art results on two phrase-similarity benchmarks: turney @xcite and ppdb-filtered @xcite datasets. in this work we focus on relatively short phrases (noun phrases of 2-3 words) that represent common use cases for output of many nlp applications (e.g. question answering, named entity recognition and relation extraction). furthermore, we choose to use a relatively strict interpretation of similarity between phrases, and we aim to assign high similarity scores to phrases that describe very similar concepts, and not just related ones. for example, "charging cable" and "electronic device" are considered related but not similar. even phrases with lexical overlap such as "craft project" and "craft room" would not be considered similar. on the other hand, phrase pairs such as "quick delivery" and "super fast shipping" are considered similar. to explore the aforementioned setting in more depth, we introduce a practical use-case of phrase similarity -keyphrase clustering in the domain of product reviews. this task is useful both as an intermediate step for other tasks (e.g. aspect based summarization) and as a downstream task (e.g. given a product keyphrase, retrieve the reviews that mention phrases similar to it). we curate a dataset for this newly introduced keyphrase clustering task with careful collection of phrases and manual annotation of clusters. we then demonstrate that mcphrasy achieves impressive results on this task compared to all baselines. the main contributions of this work can be summarized as follows: (1) we demonstrate that existing phrase similarity methods lack information found in phrase contexts at inference time. we overcome this shortcoming by proposing a novel method for phrase similarity estimation. our method not only leverages the context at inference time, but also takes into account multiple contexts per phrase to make the results more robust; (2) we propose a new downstream task -keyphrase clustering -which relies on phrase-based similarity, and curate an evaluation dataset in the domain of product reviews; (3) we demonstrate that our proposed method outperforms all other baselines on both phrase similarity benchmarks and the keyphrase clustering task. 1 2 related work representation of text spans. producing wordlevel representations has been extensively explored @xcite @xcite @xcite @xcite , mainly using the surroundings of the word as a signal for its meaning. representing text spans larger than one word can be achieved by non-trivial combinations of word representations (e.g., @xcite @xcite . however, these approaches use the phrase context only during the pretraining stage and are consequently less effective for capturing the compositional meaning of the span @xcite . phrase-level embeddings assist in tasks such as paraphrase detection, question-answering (qa), and topic modeling, and are often generated in compliance with the task objective. @xcite fine-tune bert @xcite with synthetically generated paraphrases to detect lexicallydiffering text similarities. @xcite optimize spanbert @xcite for qa to link a question to a phrasal answer within a passage. @xcite and @xcite apply contrastive learning over luke @xcite or a combination of sentence-bert @xcite and phrase-bert @xcite , for the use of clustering together topically related phrases within a corpus. these methods for rendering phrase representations leverage a single sentence-level context of the phrase, only while training. in contrast, our approach takes advantage of many contexts at inference time in order to capture the meaning of the phrase. ## the mcphrasy method while most existing phrase embedding methods, such as averaged glove @xcite word embeddings and phrase-bert @xcite , rely solely on the phrase to create a representation in inference time, our approach relies on multiple contexts in which the given phrase appears. for example, given the phrase "brunette hair", previous methods only consider the standalone phrase, while mcphrasy places it in multiple contexts (masked sentences) such as, "this is a great shampoo for @xcite ." or "i always wanted to have @xcite , and thanks to this hair dye i'm now a brunette!". the context of a word (or phrase) is known to be highly indicative of its meaning @xcite . we hypothesize that current models do not succeed in learning the full meaning from the context of phrases during training. by using contexts directly at inference time, we hope to improve phrase representations. given two phrases, mcphrasy retrieves contexts for both phrases, extracts the vector representation of each context ( §3.1) assisted by a trained model ( §3.2), and applies a similarity function between the two sets of representations ( §3.3). ## phrase-based clustering in addition to standard phrase similarity benchmarks, the ability to compare the likeness of phrases can be extrinsically evaluated by means of phrase clustering. joining and separating phrases within a set adds a level of complexity that requires a more fine-grained ability to measure phrase similarity, as a bad estimation for a single phrase pair may result in very different allocations to clusters. moreover, the phrase clustering task serves practical applications such as identification of reoccurring elements within documents or product reviews for use of information extraction or summarization. while there exist datasets that cluster noun phrases to predetermined cluster categories, like types of named entities (e.g., tjong kim sang and de @xcite , to the best of our knowledge, we are the first to establish a phrase clustering dataset without fixed classes. ## experiments we first evaluate mcphrasy on phrase similarity benchmarks, and then continue to examine its utility for clustering on our new phrase-based clustering dataset. ## conclusion in this work we address the task of phrase similarity and propose to add context information to the phrase representation during inference. this is done by extracting representations of different contexts per phrase and aggregating their pairwise similarity with a novel method. we show that mcphrasy surpasses previous sota methods for standard phrase similarity benchmarks. additionally, we present a new task of phrasebased clustering that relies on high quality phrase similarity estimation. we collect a new dataset for this task in the domain of product reviews and annotate 106 groups with 2650 phrases for cluster assignment. we show that mcphrasy improves over all baselines, thanks to its innovative mechanism that integrates phrase contexts with existing phrase representation models. finally, since contexts are considered at inference time, we expect mcphrasy to work smoothly across domains, especially when texts in the new domain are relatively scarce. we leave such experiments for future work.