Unsupervised Sentence Textual Similarity with Compositional Phrase Semantics (2022)

Abstract: Measuring Sentence Textual Similarity (STS) is a classic task that can be applied to many downstream NLP applications such as text generation and retrieval. In this paper, we focus on unsupervised STS that works on various domains but only requires minimal data and computational resources. Theoretically, we propose a light-weighted Expectation-Correction (EC) formulation for STS computation. EC formulation unifies unsupervised STS approaches including the cosine similarity of Additively Composed (AC) sentence embeddings, Optimal Transport (OT), and Tree Kernels (TK). Moreover, we propose the Recursive Optimal Transport Similarity (ROTS) algorithm to capture the compositional phrase semantics by composing multiple recursive EC formulations. ROTS finishes in linear time and is faster than its predecessors. ROTS is empirically more effective and scalable than previous approaches. Extensive experiments on 29 STS tasks under various settings show the clear advantage of ROTS over existing approaches. Detailed ablation studies prove the effectiveness of our approaches.

## introduction sentence textual similarity (sts) measures the semantic equivalence between a pair of sentences, which is supposed to be consistent with human evaluation @xcite . sts is also an effective sentence-level semantic measure for many downstream tasks such as text generation and retrieval @xcite @xcite . in this paper, we focus on unsupervised sts which is expected to compare texts of various domains but only requires minimal data and computational resources. there are several typical ways to compute unsupervised sts, including 1) treat each sentence as an embedding by the additive composition (ac) @xcite of word vectors, then estimate the sts of two sentences by their cosine similarity; 2) treat each sentence as a probabilistic distribution of word vectors, then measure the distance between distributions. notably, optimal transport (ot) @xcite foot_0 is adopted to compute the sts @xcite . ot-based approaches search for the best alignment with respect to the word-level semantics and result in state-of-the-art solution @xcite . in this paper, we argue that phrase-level semantics should also be exploited to fully understand the sentences. for example, "optimal transport" should be considered as a mathematical term rather than two independent words. specifically, the phrase chunk is composed of lower-level chunks and is usually represented as a node in tree structures. the aforementioned ac and ot-based sts methods are too shallow to include such structures. tree kernels (tk) @xcite consider the parsed syntax labels. however, it boils down to syntaxbased but sub-optimal word alignment under our comparison experiment. recent advancement of pretrained language models (plms) also demonstrate the importance of contextualization @xcite @xcite . plms can be further adopted to sts tasks by supervised fine-tuning @xcite , under carefully designed transfer learning @xcite or domainadaptation @xcite . without those treatments, the performances of plmbased stss are observed to be very poor @xcite . meanwhile, plm-based stss suffer from high computational costs to fit large amounts of high-quality data, which might prevent them from broader downstream scenarios. in this paper, we propose a set of concepts and similarities to exploit the phrase semantics in the unsupervised setup. our contributions are four folds: unified formulation we unify three types of unsupervised sts models (ac @xcite , ot @xcite and tk @xcite ) by the ec similarity in section 3. ec similarity uncovers the strengths and weaknesses of the three approaches. phrase vectors and their alignment we generalize the idea of word alignment to phrase alignment in section 4. after the formal definition of recursive phrase partition (rpp), we compose the phrase weights and vectors by those from finer-grained partitions under the invariant additive phrase composition and generalize the word alignment to phrase alignment. empirical observations show that ec similarity is an effective formulation to interpolate the existing unsupervised sts, and yields better performances. recursive optimal transport we propose the recursive optimal transport similarity (rots) in section 5 based on the phrase alignment introduced in section 4. rots computes the ec similarity at each phrase partition level and ensembles them. notably, prior optimal transport (prior ot) is adopted to guide the finer-grained phrase alignment by the coarser-grained phrase alignment at each expectation step of ec similarity. extensive experiments we show the comprehensive performance of rots on a wide spectrum of experimental settings in section 6 and the appendix, including 29 sts tasks, five types of word vectors, and three typical preprocessing setups. specifically, rots is shown to be better than all other unsupervised approaches including bert based sts in terms of both effectiveness and efficiency. detailed ablation studies also show that our constructive definitions are sufficiently important and the hyper-parameters can be easily chosen to obtain the new sota performances. ## related work embedding the symbolic words into continuous space to present their semantics @xcite @xcite is one of the breakthroughs of modern nlp. notably, it shows that the vector (or semantics) of a phrase can be approximated by the additive composition of the vectors of its containing words @xcite . thus, word embeddings can be further utilized to describe the semantics of texts beyond the word level. several strategies were proposed to provide sentence embeddings. additive composition. additive composition of word vectors @xcite forms effective sentence embeddings. the cosine similarity between the sentence embeddings has been shown to be a stronger sts under transferred @xcite and unsupervised settings @xcite than most of the deep learning approaches @xcite @xcite . optimal transport. by considering sentences as distributions of embeddings, the similarity between sentence pairs is the consequence of optimal transport of sentence distributions @xcite @xcite . ot models find the optimal alignment with respect to word semantics via their embeddings and have the sota performances @xcite . ## unification of unsupervised sts methods given a pair of sentences (s (1) , s (2) ), we are expected to estimate their similarity score @xmath0, 1]. for sentence s (1) (or s (2) ), we have vector {v ## recursive optimal transport and sts in this section, we connect the dots by applying ec similarity in section 3.2 to phrase alignment in section 4 on tree structures. specifically, we present recursive optimal transport similarity (rots) which computes the phrase alignment at each (l + 1)-th level phrase partition with the guidance of the phrase alignment at the l-th level. ## experiments we first present the experimental setting of unsupervised sts. then we conduct the benchmark study of all unsupervised sts approaches. detailed ablation studies justify the effect of rots. in the appendix, further discussions on the impact of word vectors, and preprocessing steps are included. ## conclusion in this paper, we present a new ec similarity of sts that allows flexible adaptation of word-level alignment, which successfully unifies three different unsupervised approaches. by taking advantage of the recursive phrase partitions, we generalize ec similarity to the phrase alignment. then, we propose rots, a new sentence similarity that considers phrase semantics by conducting phrase alignment in a coarse-to-fine order under the coarseto-fine prior ot. the thorough comparison with unsupervised baselines demonstrates the state-ofthe-art performance and technical details of rots are fully justified by the ablation study.