Refining Sentence Embedding Model through Ranking Sentences Generation with Large Language Models
Authors: Liyang He, Chenglong Liu, Rui Li, Zhenya Huang, Shulan Ruan, JUN ZHOU, Enhong Chen
Accepted at: acl25

Refining Sentence Embedding Model through Ranking Sentences Generation with Large Language Models

Liyang He1, Chenglong Liu1, Rui Li1, Zhenya Huang1,2, Shulan Ruan3
Jun Zhou4, Enhong Chen1*
1State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China;
2 Institute of Artificial Intelligence, Hefei Comprehensive National Science Center;
3Shenzhen International Graduate School, Tsinghua University;
4Zhejiang University
{heliyang,sepcent,ruili2000}@mail.ustc.edu.cn,slruan@sz.tsinghua.edu.cn
{huangzhy, cheneh}@ustc.edu.cn, junzhougucas@gmail.com

Abstract

Sentence embedding is essential for many NLP tasks, with contrastive learning methods achieving strong performance using annotated datasets like NLI. Yet, the reliance on manual labels limits scalability. Recent studies leverage large language models (LLMs) to generate sentence pairs, reducing annotation dependency. However, they overlook ranking information crucial for fine-grained semantic distinctions. To tackle this challenge, we propose a method for controlling the generation direction of LLMs in the latent space. Unlike unconstrained generation, the controlled approach ensures meaningful semantic divergence. Then, we refine exist sentence embedding model by integrating ranking information and semantic information. Experiments on multiple benchmarks demonstrate that our method achieves new SOTA performance with a modest cost in ranking sentence synthesis1.

1 Introduction

Sentence embedding is a fundamental task in natural language processing. It provides effective semantic representations for various downstream applications, such as semantic search (He et al., 2023; Li et al., 2024a), text classification (Wang et al., 2022a), question-answering systems (Nguyen et al., 2022), and other specialized domain systems (Liu et al., 2019; Xue et al., 2024). In recent years, significant progress has been made in the study of sentence embeddings, with methods based on contrastive learning standing out in particular. These approaches learn embeddings of sentences by bringing semantically similar sentences closer together and pushing dissimilar ones further apart. Current mainstream research relies on high-quality annotated data, especially natural language inference (NLI) datasets (Bowman et al., 2015; Williams et al., 2018). For instance, supervised contrastive learning methods based on NLI have demonstrated a remarkable ability to surpass the unsupervised approaches (Limkonocthiwat et al., 2022; Jiang et al., 2022). However, such annotated datasets are often unavailable in most real-world scenarios, and the manual construction of these datasets incurs extremely high costs.

To reduce reliance on manually annotated data, recent studies have begun to explore leveraging the powerful generative capabilities of large language models (LLMs) to construct high-quality sentence pairs automatically. For instance, SynCSE (Zhang et al., 2023) employs LLMs to generate semantically similar sentence pairs, enhancing the effectiveness of contrastive learning. MultiCSR (Wang et al., 2024a) further evaluates the quality of LLM-generated outputs, filtering out erroneous results. GCSE (Lai et al., 2024) utilizes knowledge graphs to extract entities and quantities, enabling LLMs to generate more diverse and knowledge-enriched samples. These approaches significantly diminish the dependence on manual annotation.

However, current research focuses on generating sentence pairs, overlooking the critical role of ranking sentences. While sentence pairs can capture the similarity between sentences, they fail to effectively distinguish between “highly similar” sentences and “slightly different”. Liu et al. (2023) point out that the limitation of sentence pairs lies in their inability to represent finer-grained semantic distinctions. Existing unsupervised methods, such as RankCSE (Liu et al., 2023) and RankEncoder (Seonwoo et al., 2023), attempt to construct ranking information using in-batch data to address this shortcoming. However, the ranking information in these methods is derived in an unsupervised manner, lacking explicit ranking supervision signals. As shown in Figure 1 (a), the heatmap illustrates the similarity calculations between sentences ac-
Figure 1: Sentence similarity within the ranking sentences obtained through different methods. We randomly selected 1,000 ranking sentences generated by these methods and extracted the first 16 sentences from each ranking sentence. Then, we use a trained DiffCSE (Chuang et al., 2022) to obtain their embeddings and compute their average similarity. (a) Directly extracted from the trained batch. (b) Prompting the LLM to generate complete ranking sentences at once. (c) Prompting the LLM to generate ranking sentences step by step. (d) Generating the ranking sentences using our proposed directionally controlled generation method.

quired from the in-batch data. We observe that the relationships among sentences within the in-batch data are treated as equivalent, failing to capture the hierarchical semantic distinctions. Thus, we propose a new research question: Can LLMs be used to generate ranking sentences to enhance the performance of sentence embedding models?

A straightforward method for generating ranking sentences is to directly prompt LLMs to produce them. However, such an unconstrained generation process will result in ambiguous sentence semantic relationships. As illustrated in Figure 1 (b) and (c), neither prompting the LLM to generate complete ranking sentences at once nor guiding it to generate them step by step can ensure a gradual increase in semantic distance 2. Thus, it fails to provide high-quality ranking information for sentence embedding models.

In this paper, we propose a latent space directional control method for ranking sentence generation and a post-training method for synthesized ranking sentences. Specifically, we design a directionally controlled generation method that LLMs to produce ranking sentences. By utilizing the generation probabilities of the preceding two sentences, we ensure that the resulting latent space remains in a consistent direction. As shown in Figure 1 (d), our generated ranking sentences exhibit a gradual increase in semantic divergence within the semantic space. Then, we integrate the ranking information and semantic information from the synthesized ranking sentences to refine existing sentence embedding models through post-training. The contributions of this paper can be summarized as follows:

• We are the first to use LLMs to generate ranking sentences. We have curated a dataset consisting of 16,063 ranking sentences and 530,079 sentences, opening new avenues for research in sentence embedding.

• We propose a post-training approach that incorporates both ranking and semantic information from the synthesized ranking sentences, substantially enhancing the performance of sentence embedding models on STS, reranking, and TR tasks.

• Extensive experiments on multiple benchmark datasets demonstrate the effectiveness of the proposed method. Even using merely 5% of the synthesized ranking sentences is sufficient to surpass the original sentence embedding model significantly.

2 Background

In unsupervised sentence embedding models, a series of works represented by SimCSE (Gao et al., 2021) employ contrastive learning to acquire effective embeddings by bringing semantically similar neighbours closer while pushing dissimilar ones apart. Assume there exists an unlabeled dataset \( \mathcal{X} \). For each sentence \( x \in \mathcal{X} \), SimCSE processes the same input through an encoder, such as BERT (Kenton and Toutanova, 2019) or RoBERTa (Liu, 2019), twice. It yields two embeddings \( h_i \) and \( h_i^+ \) for the i-th sentence with different dropout masks. The objective for the pair \( (h_i, h_i^+) \) within a mini-batch of \( B \) is:

\[
\mathcal{L}_i = - \log \frac{e^{\operatorname{sim}(h_i, h_i^+)/\tau}}{\sum_{j=1}^{B} e^{\operatorname{sim}(h_i, h_j^+)/\tau}},
\]

(1)

2We provide a detailed description of their generation process in Appendix A.
where \( \tau \) is a temperature hyperparameter and \( \operatorname{sim}(\cdot,\cdot) \) is the cosine similarity between two embeddings. Follow-up methods such as CARDs (Wang et al., 2022b), DiffCSE (Chuang et al., 2022), and RankCSE (Liu et al., 2023) have been proposed.

Data Generation with LLM. Unsupervised approaches often lag behind their supervised counterparts, which leverage labeled datasets such as natural language inference (NLI) corpora (Bowman et al., 2015; Williams et al., 2018). The NLI dataset provides each \( x \) with a positive sample \( x^+ \) and a hard negative sample \( x^- \) to construct the triplet \( (x, x^+, x^-) \) for the supervised contrastive loss. However, such annotated data are typically unavailable in most scenarios. Recently, the use of large language models (LLMs) for data construction and annotation has become a trend (Li et al., 2024b; Liu et al., 2024). Thus, researchers began exploring the potential of LLMs for the triplet \( (x, x^+, x^-) \) generation for each \( x \in \mathcal{X} \). A representative work is SynCSE (Zhang et al., 2023), which leverages ChatGPT (OpenAI, 2022) in a few-shot setting to generate positive samples and hard negative samples. MultiCSR (Wang et al., 2024a) and GCSE (Lai et al., 2024) further refined the process of utilizing LLMs for data generation. These works fundamentally revolve around generating triplets.

Ranking Sentences Generation. We further advance this research by concentrating on the generation of ranking sentences. Formally, a ranking sentence is defined as a sequence of sentences \( l = \{x^{(1)}, x^{(2)}, \cdots, x^{(n)}\} \) for each \( x \in \mathcal{X}, x^{(1)} \) is equal to \( x \) itself. Let \( \varphi(a, b) \) denote the semantic similarity between two sentences \( a \) and \( b \), where a larger value indicates a closer semantic similarity. For any three sentences \( (x^{(a)}, x^{(b)}, x^{(c)}) \in l \) with \( a < b < c \), the condition should hold: \( \varphi(x^{(a)}, x^{(b)}) > \varphi(x^{(a)}, x^{(c)}) \). In other words, these sentences are arranged in order within the semantic space.

3 Methodology

3.1 Ranking Sentences Generation

A straightforward approach to generating ranking sentences is prompting LLM, either by directly producing ranking sentences at once or by generating them step by step. Taking the second case as an example, let the prompt be denoted as an instruction \( I \). The LLM \( C_\theta \) generates the i-th sentence

\[
x^{(i)} = [x_1^{(i)}, x_2^{(i)}, \ldots, x_k^{(i)}]
\]

in the following form:

\[
p_\theta(x^{(i)}|x^{(i-1)}, I) = \prod_{t=1}^n p_\theta(x_t^{(i)}|x_{<t}^{(i)}, x^{(i-1)}, I),
\]

where \( x_{<t}^{(i)} \) represents the tokens generated before the t-th step. Eq.(2) use the previously generated sentence \( x^{(i-1)} \) and the instruction \( I \) to prompt the LLM to generate the next sentence \( x^{(i)} \), thereby progressively constructing a sequence of ranking sentences. However, as mentioned before, this method of generation leads to ambiguous semantic relationships among the ranking sentences, as illustrated in Figure 1 (c).

Our core idea is to integrate directional control into the ranking sentence generation process. As illustrated in Figure 2, our generation process sequentially combines the directional tendencies of two sentences, ensuring that the subsequent generation maintains a consistent trajectory. For example, the generation direction of \( x^{(3)} \) is controlled by the latent generation space of \( x^{(1)} \) and \( x^{(2)} \), ensuring maximal consistency in their generated directions within the semantic space. Specifically, we modified the sampling method of \( x_t^{(i)} \) in the LLM as follows:

\[
p_\theta(x_t^{(i)}|x_{<t}^{(i)}, c) \propto \frac{p_\theta(x_t^{(i)}|x_{<t}^{(i)}, x^{(i-1)}, I)^{1+\lambda}}{p_\theta(x_t^{(i)}|x_{<t}^{(i)}, x^{(i-2)}, I)^{\lambda}}
\]

where \( x_{<t}^{(i)} \) represents the tokens generated before the t-th step and \( c \) represents the generation condition based on \( x^{(i-1)}, x^{(i-2)} \), and the instruction \( I \).

![A 2D semantic space illustrating the generation of ranking sentences using prompts (solid line) and our directionally controlled method (dashed line). The color of each point represents its semantic similarity to the initial point \( x^{(0)} \).](page_768_109_627_312.png)

Figure 2: A 2D semantic space illustrating the generation of ranking sentences using prompts (solid line) and our directionally controlled method (dashed line). The color of each point represents its semantic similarity to the initial point \( x^{(0)} \).
λ is a hyperparameter that assigns weights to the two generation probabilities. In other words, the generation of a new sentence depends on the directional tendencies of the generation probabilities of the preceding two sentences, ensuring that the resulting latent space remains aligned in a consistent direction. Then, we can thus sample the next t-th token \( x_t^{(i)} \) in the logits space:

\[
\log p_\theta(x_t^{(i)}|x_{<t}^{(i)}, c) =
\]
\[
(1 + \lambda) \log p_\theta(x_t^{(i)}|x_{<t}^{(i)}, x^{(i-1)}, I) \tag{4}
\]
\[
- \lambda \log p_\theta(x_t^{(i)}|x_{<t}^{(i)}, x^{(i-2)}, I).
\]

According to Eq.(4), we concatenate the instruction \( I \) and the previously generated segment \( x_{<t}^{(i)} \) with \( x^{(i-1)} \) and \( x^{(i-2)} \) separately. We then perform two decoding procedures to obtain their respective log probabilities. After computing a weighted sum of these log probabilities, we apply greedy sampling to generate \( x_t^{(i)} \). When generating the first sentence \( x^{(1)} \), since only \( x^{(0)} \) is available, we set \( \lambda \) to 0.

Our method generalizes to Eq.(2) when we set \( \lambda = 0 \). However, when we use two sentences as conditions, the generative process undergoes a fundamental transformation. This can be likened to basic geometric theorems: “Infinitely many lines pass through a single point” and “The uniqueness of a line through two points.” The presence of two preceding sentences ensures the directional consistency of our generation. Besides, our controlled generation process is formally similar to classifier-free guidance (Ho and Salimans, 2021), which employs a linear combination to integrate conditional and unconditional score estimations. However, our method differs fundamentally. We rely solely on conditional control, meaning that all terms depend on preceding sentences rather than an unconditional distribution. By doing so, we effectively guide the sentence generation process, ensuring that the generated text maintains a stable and coherent flow within the semantic space.

3.2 Model Post-training

After obtaining the ranking sentences \( l \) for each \( x \in \mathcal{X} \), we aim to post-train the existing sentence embedding model to enhance its ability to distinguish fine-grained semantic differences. The ranking sentence \( l \) provides order information among sentences. However, the semantic gaps between these sentences are not evenly spaced. Thus, we propose a post-training method that considers both ranking and semantic information among the ranking sentences.

Let \( \varphi_{j,k} = \varphi(x^{(j)}, x^{(k)}) \) denote the semantic similarity between \( x^{(j)} \) and \( x^{(k)} \). For any \( x^{(j)} \in l \), the following semantic ranking relationship should be satisfied according to the ordering within the ranking sentences:

\[
\varphi_{j,1} < \cdots < \varphi_{j,j} > \varphi_{j,j+1} > \cdots > \varphi_{j,n}.
\] (5)

Let \( r = \{r(i)\}_{i=0}^n \) denote a permutation of the object indices arranged in descending order of semantic similarity, where \( r(i) \) represents the rank of the i-th index in the list \( [\varphi_{j,1}, \varphi_{j,2}, \ldots, \varphi_{j,n}] \) based on its magnitude. Then, we process each \( x \) in \( l \) using an encoder model, specifically adopting the DiffCSE (Chuang et al., 2022) base series in our experiments. This model can be trained through a standard unsupervised contrastive learning approach. We obtain their corresponding embeddings \( \{h^{(1)}, h^{(2)}, \ldots, h^{(n)}\} \). Assuming \( \phi_{j,k} = \phi(h^{(j)}, h^{(k)}) \) represents the cosine similarity between \( h^{(j)} \) and \( h^{(k)} \). For \( h^{(j)} \), we can then derive its similarity relationships with other sentences in \( l \), represented as \( \phi^j = [\phi_{j,1}, \phi_{j,2}, \ldots, \phi_{j,n}] \).

Next, we integrate the ranking information \( r \) with the semantic information \( \phi^j \). Our fundamental idea is to adjust \( \phi_{j,k} \) based on the ranking position \( r(k) \). Let \( \hat{\phi}^j[i] \) represent the value at index \( i \) in \( \phi^j \), and let \( \hat{r} = \{\hat{r}(i)\}_{i=0}^n \) denote the permutation of object indices based on the similarity relationships in \( \phi^j \). We modify each \( \phi_{j,k} \) using the following approach:

\[
\hat{\phi}_{j,k} = \left\{
\begin{array}{ll}
\phi_{j,k} + m(j, k) & \text{if } r(k) < \hat{r}(k), \\
\phi_{j,k} - m(j, k) & \text{if } r(k) > \hat{r}(k).
\end{array}
\right.
\] (6)
\[
m(j, k) = \log (\omega \cdot |\phi_{j,k} - \hat{\phi}^j[r(k)]| + 1),
\] (7)

where \( \omega \) is a hyperparameter to control the importance of ranking information. When the ranking order \( \hat{r} \) reflected by semantic information differs from the ranking order \( r \) in the ranking information, the value of \( \phi_{j,k} \) is adjusted based on the ranking discrepancy to bring \( \hat{r} \) closer to \( r \). Through Eq.(6), we obtain a score \( \hat{\phi}^j = [\hat{\phi}_{j,1}, \hat{\phi}_{j,2}, \ldots, \hat{\phi}_{j,n}] \) that seamlessly integrates both ranking information and semantic information. Appendix D presents the detailed algorithm.

Finally, we post-train the sentence embedding model using the ListMLE (Xia et al., 2008) loss. Suppose the representation of \( x^{(j)} \) obtained through the sentence embedding model is \( e^{(j)} \).
Similarly, we can get a similarity relationships list \( s^j = [s_{j,1}, s_{j,2}, \ldots, s_{j,n}] \). The objective of ListMLE for ranking sentence \( l \) is defined as:

\[
\mathcal{L}_{\text{ListMLE}} (l) = - \sum_{j=1}^{n} \log P \left( \hat{\phi}^j \mid s^j \right).
\] (8)

This target ensures that the ranking results produced by the sentence embedding model learn to align with the ranking results obtained from the fusion of ranking information and semantic information in the ranking sentences.

4 Experiments

4.1 Dataset

Similar to SynCSE (Zhang et al., 2023) and MultiCSR (Wang et al., 2024a), we utilize the premises of the NLI dataset (Bowman et al., 2015; Williams et al., 2018) as the initial unlabeled dataset, denoted as \( X_1 \). Unlike SynCSE and MultiCSR, which employ the full dataset, we sample only a subset for a generation. Specifically, to enhance data diversity, we first apply k-means clustering to \( X_1 \). We set the number of cluster centers to 1,000 and then performed random sampling, selecting 20 samples per cluster, resulting in the dataset \( X_2 \). Next, we generate ranking sentences for each sentence in \( X_2 \) using our method, where the generation step is set to 32, and the hyperparameter \( \gamma \) is set to 1.5. In contrast to SynCSE, which relies on ChatGPT with approximately 175B parameters for generation, we utilize the LLaMA3-8B-Instruct. This process ultimately produces the dataset \( X_3 \), consisting of 16,063 sentence ranking lists and a total of 530,079 sentences. Appendix A presents the detailed generation process of our method.

4.2 Experiment Setup

Baselines. We chose the following strong baselines, including SimCSE (Gao et al., 2021), DiffCSE (Chuang et al., 2022), PromptBERT (Jiang et al., 2022), PCL (Wu et al., 2022a), DebCSE (Miao et al., 2023), InfoCSE (Wu et al., 2022b), RankCSE (Liu et al., 2023), SynCSE (Zhang et al., 2023), and MultiCSR (Wang et al., 2024a). Our model is built upon existing sentence embedding models as a post-training approach. In the following experiments, we primarily selected two SOTA models, SynCSE and MultiCSR, as our base models to evaluate whether integrating our ranked sentence data and post-training method can enhance performance\footnote{By the time this work was completed, GCSE (Lai et al., 2024) was one of the most recent approaches utilizing synthetic data for sentence embedding training. However, as it has not yet publicly released its code and dataset, we did not consider it as a base model.}. We designate the post-trained SynCSE and MultiCSR as SynCSE-r and MultiCSR-r, respectively. The details of our training process are provided in Appendix C.

Evaluation Settings. We conduct evaluation tests across three tasks: Semantic Textual Similarity (STS), Reranking Task, and Transfer Task (TR). Specifically, for the STS task, we assess performance on seven STS benchmarks: STS 2012–2016 (Agirre et al., 2012, 2013, 2014, 2015, 2016), STS Benchmark (Cer et al., 2017), and SICK-Relatedness (Marelli et al., 2014). These datasets consist of sentence pairs annotated with similarity scores ranging from 0 to 5. For the retrieval task, we conduct experiments on four datasets: AskUbuntuDupQuestions (Barzilay et al., 2016), MindSmallReranking (Wu et al., 2020), SciDocsRR (Wu et al., 2020), and StackOverflowDupQuestions (Liu et al., 2018). We followed the validation approach of SynCSE (Zhang et al., 2023), adopting the methodology of MTEB (Muenninghoff et al., 2023) and employing Mean Average Precision (MAP) as the primary evaluation metric. For the TR task, we use SentEval (Conneau and Kiela, 2018) to evaluate the results, as detailed in Appendix F.

4.3 Main Results

STS Tasks. As shown in Table 1, the post-trained model obtained through our method significantly outperforms previous baselines. Compared to the standard unsupervised SimCSE, our SOTA results improve Spearman’s correlation by an average of 7.11% on base models and 5.09% on large models. In comparison with ranking-aware models such as RankCSE, our method achieves improvements of 2.19% and 2.65%, respectively. Furthermore, compared to the underlying sentence embedding models we employ, such as MultiCSR and SynCSE, our approach enhances performance by 0.61% and 0.45% on base models and large models, respectively. These results demonstrate that our method has successfully achieved new SOTA models.

Reranking Tasks. Table 2 presents the performance on four reranking datasets. We followed the experimental setup of SynCSE (Zhang et al., 2023) without utilizing the training sets of reranking tasks.
<table>
  <tr>
    <th>Model</th>
    <th>Method</th>
    <th>STS-12</th>
    <th>STS-13</th>
    <th>STS-14</th>
    <th>STS-15</th>
    <th>STS-16</th>
    <th>STS-B</th>
    <th>SICK-R</th>
    <th>Avg.</th>
  </tr>
  <tr>
    <td rowspan="9">BERT-base</td>
    <td>SimCSE†</td>
    <td>68.40</td>
    <td>82.41</td>
    <td>74.38</td>
    <td>80.91</td>
    <td>78.56</td>
    <td>76.85</td>
    <td>72.23</td>
    <td>76.25</td>
  </tr>
  <tr>
    <td>DiffCSE†</td>
    <td>72.28</td>
    <td>84.43</td>
    <td>76.47</td>
    <td>83.90</td>
    <td>80.54</td>
    <td>80.59</td>
    <td>71.23</td>
    <td>78.49</td>
  </tr>
  <tr>
    <td>PromptBERT♣</td>
    <td>71.56</td>
    <td>84.58</td>
    <td>76.98</td>
    <td>84.47</td>
    <td>80.60</td>
    <td>81.60</td>
    <td>69.87</td>
    <td>78.54</td>
  </tr>
  <tr>
    <td>PCL▲</td>
    <td>72.84</td>
    <td>83.81</td>
    <td>76.52</td>
    <td>83.06</td>
    <td>79.32</td>
    <td>80.01</td>
    <td>73.38</td>
    <td>78.42</td>
  </tr>
  <tr>
    <td>DebCSE†</td>
    <td>76.15</td>
    <td>84.67</td>
    <td><b>78.91</b></td>
    <td><b>85.41</b></td>
    <td>80.55</td>
    <td>82.99</td>
    <td>73.60</td>
    <td>80.33</td>
  </tr>
  <tr>
    <td>InfoCSE††</td>
    <td>70.53</td>
    <td>84.59</td>
    <td>76.40</td>
    <td>85.10</td>
    <td>81.95</td>
    <td>82.00</td>
    <td>71.37</td>
    <td>78.85</td>
  </tr>
  <tr>
    <td>RankCSE♣</td>
    <td>75.66</td>
    <td><b>86.27</b></td>
    <td>77.81</td>
    <td>84.74</td>
    <td>81.10</td>
    <td>81.80</td>
    <td>75.13</td>
    <td>80.36</td>
  </tr>
  <tr>
    <td>SynCSE*</td>
    <td>74.53</td>
    <td>82.14</td>
    <td>78.22</td>
    <td>83.46</td>
    <td>80.66</td>
    <td>81.42</td>
    <td>80.51</td>
    <td>80.13</td>
  </tr>
  <tr>
    <td>MultiCSR*</td>
    <td>75.88</td>
    <td>82.39</td>
    <td>78.80</td>
    <td>84.42</td>
    <td>80.54</td>
    <td>82.23</td>
    <td>80.03</td>
    <td>80.61</td>
  </tr>
  <tr>
    <td>SynCSE-r (ours)</td>
    <td>75.82</td>
    <td>83.24</td>
    <td>78.61</td>
    <td>84.75</td>
    <td>81.68</td>
    <td>83.45</td>
    <td>80.67</td>
    <td>81.17</td>
  </tr>
  <tr>
    <td>MultiCSR-r (ours)</td>
    <td><b>76.37</b></td>
    <td>82.50</td>
    <td>78.37</td>
    <td><b>85.38</b></td>
    <td><b>82.15</b></td>
    <td><b>84.01</b></td>
    <td><b>80.55</b></td>
    <td><b>81.33</b></td>
  </tr>
  <tr>
    <td rowspan="9">BERT-large</td>
    <td>SimCSE†</td>
    <td>70.88</td>
    <td>84.16</td>
    <td>76.43</td>
    <td>84.50</td>
    <td>79.76</td>
    <td>79.26</td>
    <td>73.88</td>
    <td>78.41</td>
  </tr>
  <tr>
    <td>PCL▲</td>
    <td>74.87</td>
    <td>86.11</td>
    <td>78.29</td>
    <td>85.65</td>
    <td>80.52</td>
    <td>81.62</td>
    <td>73.94</td>
    <td>80.14</td>
  </tr>
  <tr>
    <td>DebCSE†</td>
    <td><b>76.82</b></td>
    <td><b>86.36</b></td>
    <td>79.81</td>
    <td><b>85.80</b></td>
    <td>80.83</td>
    <td>83.45</td>
    <td>74.67</td>
    <td>81.11</td>
  </tr>
  <tr>
    <td>InfoCSE††</td>
    <td>71.89</td>
    <td>86.17</td>
    <td>77.72</td>
    <td><b>86.20</b></td>
    <td>81.29</td>
    <td>83.16</td>
    <td>74.84</td>
    <td>80.18</td>
  </tr>
  <tr>
    <td>RankCSE♣</td>
    <td>75.48</td>
    <td><b>86.50</b></td>
    <td>78.60</td>
    <td>85.45</td>
    <td>81.09</td>
    <td>81.58</td>
    <td>75.53</td>
    <td>80.60</td>
  </tr>
  <tr>
    <td>SynCSE*</td>
    <td>75.23</td>
    <td>84.28</td>
    <td>79.41</td>
    <td>84.89</td>
    <td>82.09</td>
    <td>83.48</td>
    <td>81.79</td>
    <td>81.60</td>
  </tr>
  <tr>
    <td>MultiCSR*</td>
    <td>75.56</td>
    <td>85.19</td>
    <td><b>80.14</b></td>
    <td>85.91</td>
    <td>82.40</td>
    <td>84.19</td>
    <td>81.65</td>
    <td>82.15</td>
  </tr>
  <tr>
    <td>SynCSE-r (ours)</td>
    <td>76.32</td>
    <td>85.17</td>
    <td>79.29</td>
    <td>85.78</td>
    <td><b>82.76</b></td>
    <td>84.76</td>
    <td><b>82.51</b></td>
    <td>82.37</td>
  </tr>
  <tr>
    <td>MultiCSR-r (ours)</td>
    <td><b>75.69</b></td>
    <td>85.63</td>
    <td>79.92</td>
    <td>86.08</td>
    <td><b>82.69</b></td>
    <td><b>84.88</b></td>
    <td><b>82.37</b></td>
    <td><b>82.47</b></td>
  </tr>
  <tr>
    <td rowspan="9">RoBERTa-base</td>
    <td>SimCSE†</td>
    <td>70.16</td>
    <td>81.77</td>
    <td>73.24</td>
    <td>81.36</td>
    <td>80.65</td>
    <td>80.22</td>
    <td>68.56</td>
    <td>76.57</td>
  </tr>
  <tr>
    <td>DiffCSE†</td>
    <td>70.05</td>
    <td>83.43</td>
    <td>75.49</td>
    <td>82.81</td>
    <td>82.12</td>
    <td>82.38</td>
    <td>71.19</td>
    <td>78.21</td>
  </tr>
  <tr>
    <td>PromptRoBERTa♣</td>
    <td>73.94</td>
    <td>84.74</td>
    <td>77.28</td>
    <td>84.99</td>
    <td>81.74</td>
    <td>81.88</td>
    <td>69.50</td>
    <td>79.15</td>
  </tr>
  <tr>
    <td>PCL▲</td>
    <td>71.13</td>
    <td>82.38</td>
    <td>75.40</td>
    <td>83.07</td>
    <td>81.98</td>
    <td>81.63</td>
    <td>69.72</td>
    <td>77.90</td>
  </tr>
  <tr>
    <td>DebCSE†</td>
    <td>74.29</td>
    <td>85.54</td>
    <td>79.46</td>
    <td>85.68</td>
    <td>81.20</td>
    <td>83.96</td>
    <td>74.04</td>
    <td>80.60</td>
  </tr>
  <tr>
    <td>RankCSE♣</td>
    <td>73.20</td>
    <td><b>85.95</b></td>
    <td>77.17</td>
    <td>84.82</td>
    <td>82.58</td>
    <td>83.08</td>
    <td>71.88</td>
    <td>79.81</td>
  </tr>
  <tr>
    <td>SynCSE*</td>
    <td>76.15</td>
    <td>84.41</td>
    <td>79.23</td>
    <td>84.85</td>
    <td>82.87</td>
    <td>83.95</td>
    <td><b>81.41</b></td>
    <td>81.84</td>
  </tr>
  <tr>
    <td>MultiCSR*</td>
    <td><b>77.03</b></td>
    <td>84.72</td>
    <td><b>79.71</b></td>
    <td>85.80</td>
    <td><b>82.68</b></td>
    <td>84.24</td>
    <td>80.64</td>
    <td><b>82.12</b></td>
  </tr>
  <tr>
    <td>SynCSE-r (ours)</td>
    <td>76.01</td>
    <td>83.18</td>
    <td>79.13</td>
    <td>85.51</td>
    <td><b>83.03</b></td>
    <td>84.66</td>
    <td>80.93</td>
    <td>81.78</td>
  </tr>
  <tr>
    <td>MultiCSR-r (ours)</td>
    <td><b>76.79</b></td>
    <td>85.03</td>
    <td><b>80.00</b></td>
    <td><b>86.05</b></td>
    <td>82.65</td>
    <td><b>84.79</b></td>
    <td><b>81.14</b></td>
    <td><b>82.35</b></td>
  </tr>
  <tr>
    <td rowspan="9">RoBERTa-large</td>
    <td>SimCSE†</td>
    <td>72.86</td>
    <td>83.99</td>
    <td>75.62</td>
    <td>84.77</td>
    <td>81.80</td>
    <td>81.98</td>
    <td>71.26</td>
    <td>78.90</td>
  </tr>
  <tr>
    <td>PCL▲</td>
    <td>74.08</td>
    <td>84.36</td>
    <td>76.42</td>
    <td>85.49</td>
    <td>81.76</td>
    <td>82.79</td>
    <td>71.51</td>
    <td>79.49</td>
  </tr>
  <tr>
    <td>DebCSE†</td>
    <td><b>77.68</b></td>
    <td><b>87.17</b></td>
    <td><b>80.53</b></td>
    <td><b>85.90</b></td>
    <td>83.57</td>
    <td>85.36</td>
    <td>73.89</td>
    <td>82.01</td>
  </tr>
  <tr>
    <td>RankCSE♣</td>
    <td>73.20</td>
    <td><b>85.83</b></td>
    <td>78.00</td>
    <td>85.63</td>
    <td>82.67</td>
    <td>84.19</td>
    <td>73.64</td>
    <td>80.45</td>
  </tr>
  <tr>
    <td>SynCSE*</td>
    <td>75.92</td>
    <td>85.01</td>
    <td>80.43</td>
    <td>85.83</td>
    <td>84.40</td>
    <td>85.05</td>
    <td>81.99</td>
    <td>82.66</td>
  </tr>
  <tr>
    <td>MultiCSR*</td>
    <td>74.42</td>
    <td>84.46</td>
    <td>79.17</td>
    <td>84.76</td>
    <td>83.67</td>
    <td>84.23</td>
    <td>81.50</td>
    <td>81.74</td>
  </tr>
  <tr>
    <td>SynCSE-r (ours)</td>
    <td>75.64</td>
    <td>84.53</td>
    <td>80.36</td>
    <td><b>85.88</b></td>
    <td><b>84.47</b></td>
    <td><b>85.82</b></td>
    <td><b>83.24</b></td>
    <td><b>82.85</b></td>
  </tr>
  <tr>
    <td>MultiCSR-r (ours)</td>
    <td>74.28</td>
    <td>84.81</td>
    <td>79.20</td>
    <td>85.26</td>
    <td>83.93</td>
    <td>84.40</td>
    <td>81.62</td>
    <td>81.93</td>
  </tr>
</table>

Table 1: Comparison of Spearman’s correlation results on STS tasks, where the value highlighted in bold is the best value, and the value underlined is the second-best value. “†”: results from (Miao et al., 2023), “♣”: results from (Wang et al., 2024a), “▲”: results from (Liu et al., 2023), “††”: results from (Wu et al., 2022b). “*”: we reproduce the results with the officially released codes and corpus from (Zhang et al., 2023; Wang et al., 2024a).

During model training, only the synthesized data was used. We compared the changes in MAP for SynCSE and MultiCSR after post-training with our ranking sentences. Overall, our synthesized data and method led to an average improvement of 1.35% and 1.11% for SynCSE and MultiCSR, respectively. Note that both SynCSE and MultiCSR employ contrastive learning, which is originally a training paradigm for retrieval models (Izacard et al., 2021; Li et al., 2021). Our synthesized ranking sentences further enhance the reranking performance of SynCSE and MultiCSR, demonstrating their effectiveness in this context.

4.4 Ablation Study

Since our method consists of both a data generation phase and a model post-training phase, we conduct two groups of ablation experiments. For data synthesis, we design the following three ablation settings: (a) Prompting the LLM to generate complete ranking sentences at once. (b) Prompting the LLM to generate ranking sentences step by step.
<table>
  <tr>
    <th rowspan="2">Dataset</th>
    <th colspan="4">BERT-base</th>
    <th colspan="4">BERT-large</th>
    <th colspan="4">RoBERTa-base</th>
    <th colspan="4">RoBERTa-large</th>
  </tr>
  <tr>
    <th>SynCSE</th>
    <th>SynCSE-r</th>
    <th>MultiCSR</th>
    <th>MultiCSR-r</th>
    <th>SynCSE</th>
    <th>SynCSE-r</th>
    <th>MultiCSR</th>
    <th>MultiCSR-r</th>
    <th>SynCSE</th>
    <th>SynCSE-r</th>
    <th>MultiCSR</th>
    <th>MultiCSR-r</th>
    <th>SynCSE</th>
    <th>SynCSE-r</th>
    <th>MultiCSR</th>
    <th>MultiCSR-r</th>
  </tr>
  <tr>
    <td>AskU.</td>
    <td>51.79</td>
    <td>52.34 (+1.05%)</td>
    <td>51.04</td>
    <td>51.51 (+0.92%)</td>
    <td>51.36</td>
    <td>50.73 (-1.22%)</td>
    <td>51.62</td>
    <td>50.49 (-2.19%)</td>
    <td>52.59</td>
    <td>53.26 (+1.28%)</td>
    <td>51.91</td>
    <td>52.18 (+0.52%)</td>
    <td>55.22</td>
    <td>54.92 (-0.54%)</td>
    <td>54.01</td>
    <td>54.66 (+1.21%)</td>
  </tr>
  <tr>
    <td>Mind.</td>
    <td>28.96</td>
    <td>29.01 (+0.17%)</td>
    <td>29.04</td>
    <td>29.37 (+1.14%)</td>
    <td>30.56</td>
    <td>30.62 (+0.18%)</td>
    <td>29.47</td>
    <td>30.68 (+4.11%)</td>
    <td>27.58</td>
    <td>28.70 (+4.06%)</td>
    <td>27.97</td>
    <td>28.37 (+1.45%)</td>
    <td>29.88</td>
    <td>30.17 (+0.99%)</td>
    <td>29.16</td>
    <td>29.32 (+0.56%)</td>
  </tr>
  <tr>
    <td>SciD.</td>
    <td>69.49</td>
    <td>70.73 (+1.79%)</td>
    <td>69.32</td>
    <td>70.61 (+1.87%)</td>
    <td>71.33</td>
    <td>72.22 (+1.25%)</td>
    <td>71.31</td>
    <td>71.71 (+0.56%)</td>
    <td>63.39</td>
    <td>65.94 (+4.02%)</td>
    <td>62.83</td>
    <td>64.18 (+2.15%)</td>
    <td>69.33</td>
    <td>70.99 (+2.39%)</td>
    <td>69.73</td>
    <td>70.08 (+0.49%)</td>
  </tr>
  <tr>
    <td>StackO.</td>
    <td>39.88</td>
    <td>40.66 (+1.94%)</td>
    <td>39.50</td>
    <td>40.68 (+2.97%)</td>
    <td>40.06</td>
    <td>39.82 (-0.60%)</td>
    <td>39.76</td>
    <td>40.09 (+0.84%)</td>
    <td>38.81</td>
    <td>38.84 (+0.07%)</td>
    <td>39.35</td>
    <td>39.95 (+1.53%)</td>
    <td>39.00</td>
    <td>40.42 (+3.65%)</td>
    <td>40.50</td>
    <td>40.92 (+1.04%)</td>
  </tr>
  <tr>
    <td>Avg.</td>
    <td>47.53</td>
    <td>48.19 (+1.37%)</td>
    <td>47.22</td>
    <td>48.04 (+1.73%)</td>
    <td>48.33</td>
    <td>48.35 (+0.04%)</td>
    <td>48.04</td>
    <td>48.24 (+0.43%)</td>
    <td>45.59</td>
    <td>46.69 (+2.39%)</td>
    <td>45.51</td>
    <td>46.17 (+1.45%)</td>
    <td>48.36</td>
    <td>49.13 (+1.59%)</td>
    <td>48.35</td>
    <td>48.75 (+0.82%)</td>
  </tr>
</table>

Table 2: Comparison of Mean Average Precision (MAP) results on reranking tasks, illustrating the changes in SynCSE and MultiCSR before and after training with ranking sentence data.

<table>
  <tr>
    <th>Phase</th>
    <th>Method</th>
    <th>Spearman’s</th>
    <th>Δ</th>
  </tr>
  <tr>
    <td>-</td>
    <td>MultiCSR-r</td>
    <td>81.33</td>
    <td>0.0</td>
  </tr>
  <tr>
    <td>Data Generation</td>
    <td>(a)</td>
    <td>80.85</td>
    <td>-0.48</td>
  </tr>
  <tr>
    <td>Data Generation</td>
    <td>(b)</td>
    <td>80.97</td>
    <td>-0.36</td>
  </tr>
  <tr>
    <td>Data Generation</td>
    <td>(c)</td>
    <td>80.78</td>
    <td>-0.55</td>
  </tr>
  <tr>
    <td>Post-training</td>
    <td>(d)</td>
    <td>80.65</td>
    <td>-0.68</td>
  </tr>
  <tr>
    <td>Post-training</td>
    <td>(e)</td>
    <td>81.18</td>
    <td>-0.15</td>
  </tr>
  <tr>
    <td>-</td>
    <td>MultiCSR</td>
    <td>80.61</td>
    <td>-0.72</td>
  </tr>
</table>

Table 3: Ablation studies on different data generation methods and components of post-training. We use MultiCSR-r based on BERT-base as the model, and the results are reported on average Spearman’s correlation of STS Task.

![Impact of hyperparameter ω and impact of data on STS test score](page_1012_1042_420_312.png)

Figure 3: The impact of hyperparameter \( \omega \) on average STS test score for SynCSE-r and MultiCSR-r based on BERT-base as the model. The base model scores are shown in dashed lines

(c) Using our method, we first generate ranking sentences. Then, we randomly shuffle them and reconstruct new ranking sentences. In this case, only semantic information is utilized since the ranking information is lost. For the post-training phase, we designed the following two ablation settings: (d) Only ranking information \( r \) is used. (e) Only semantic similarity information \( \phi^{(j)} \) is used.

Table 3 presents the average Spearman’s correlation on the STS dataset. For data synthesis, comparing (a) shows that generating ranking sentences at once via prompts is limited, as LLMs struggle with semantic understanding in longer texts. Comparison with (b) suggests that even a step-by-step approach lacks effective directional control, leading to suboptimal results. The results of (c) highlight the importance of ranking information, confirming that our method’s improvements are not solely due to semantic information. For post-training, comparing (d) indicates that ranking information alone is insufficient due to fine-grained semantic differences, emphasizing the need for semantic information. The results of (e) remain inferior to the full method, showing that incorporating ranking information helps refine semantic representations and improve model performance.

4.5 Analysis

In this section, we conduct a more in-depth analysis of the synthesized dataset and our post-training method. We employ SynCSR-r and MultiCSR-r with the BERT-base model. We report the results in terms of Spearman’s correlation on the STS task.

The impact of the hyperparameter \( \omega \). Figure 3 (a) illustrates the impact of different hyperparameter \( \omega \) on model performance in the STS task. The \( \omega \) plays a crucial role in our post-training method, as defined in Eq. (7), where it controls the importance of ranking information. The results indicate that while the optimal \( \omega \) varies across different models, it remains robust within a relatively broad range. Based on these findings, we set \( \omega = 0.7 \) for SynCSE-r and \( \omega = 0.5 \) for MultiCSR-r.

The impact of the amount of synthetic data amount. Figure 3 (b) illustrates the performance on the STS task when using different proportions of our synthesized ranking sentences dataset. We find that although approximately 16,000 sentence ranking lists were generated, utilizing only 10% of
<table>
  <tr>
    <th>Method</th>
    <th>STS-12(Δ)</th>
    <th>STS-13(Δ)</th>
    <th>STS-14(Δ)</th>
    <th>STS-15(Δ)</th>
    <th>STS-16(Δ)</th>
    <th>STS-B(Δ)</th>
    <th>SICK-R(Δ)</th>
    <th>Avg.(Δ)</th>
  </tr>
  <tr>
    <td>SimCSE</td>
    <td>+4.04</td>
    <td>+1.47</td>
    <td>+1.61</td>
    <td>+2.31</td>
    <td>+0.49</td>
    <td>+1.87</td>
    <td>+0.66</td>
    <td>+1.78</td>
  </tr>
  <tr>
    <td>InfoCSE</td>
    <td>+0.24</td>
    <td>-0.42</td>
    <td>+0.27</td>
    <td>+0.36</td>
    <td>+0.64</td>
    <td>+0.14</td>
    <td>+1.38</td>
    <td>+0.37</td>
  </tr>
  <tr>
    <td>PCL</td>
    <td>+0.69</td>
    <td>+0.69</td>
    <td>+0.25</td>
    <td>+2.49</td>
    <td>+2.79</td>
    <td>+2.27</td>
    <td>+1.4</td>
    <td>+1.51</td>
  </tr>
  <tr>
    <td>RankCSE</td>
    <td>-0.11</td>
    <td>+0.6</td>
    <td>+0.69</td>
    <td>+2.00</td>
    <td>+1.98</td>
    <td>+0.15</td>
    <td>-0.29</td>
    <td>+0.71</td>
  </tr>
</table>

Table 4: We compare the changes in Spearman’s correlation on STS tasks across several sentence embedding models after post-training with ranking sentences. Their checkpoints based on BERT-base as the model are obtained from their official sources.

<table>
  <tr>
    <th colspan="2">Stage 1</th>
    <th>Stage 2</th>
  </tr>
  <tr>
    <th>Setting 1</th>
    <td>80.91</td>
    <td>81.63</td>
  </tr>
  <tr>
    <th>Setting 2</th>
    <td>77.12</td>
    <td>80.74</td>
  </tr>
</table>

Table 5: Multi-stage post-training analysis. We used two different post-training sequences and conducted experiments on the STS task, recording the average Spearman’s correlation for each stage.

the data is sufficient to achieve a substantial improvement, while merely 5% is enough to surpass the original model significantly. This underscores the pivotal role of ranking sentences in enhancing sentence embedding models. On the other hand, we also observe that continuously increasing the number of ranking sentences does not lead to a consistent improvement in STS performance. This may be attributed to an excessive number of ranking sentences, potentially reducing the model’s generalization ability.

4.6 Post-training Experiment

We further analyze the impact of post-training our data on sentence embedding models other than SynCSE and MultiCSR. Table 4 presents the changes in Spearman’s correlation for SimCSE (Gao et al., 2021), InfoCSE (Wu et al., 2022b), PCL (Wu et al., 2022a), and RankCSE (Liu et al., 2023) on STS tasks after applying our ranking sentences dataset and post-training approach. From these results, we can observe that employing ranking sentences along with our post-training method has led to improvements across most datasets in the STS task. This demonstrates the versatility and effectiveness of both ranking sentences and our post-training approach. The detailed results are presented in Figure 7 of the Appendix E.

Additionally, we conducted further experiments to examine the impact of multi-stage post-training. Specifically, we utilized the SimCSE-BERT-base model and assessed two different post-training sequences: Setting 1: We first post-trained using the MultiCSR dataset and method, followed by our dataset and method. Setting 2: We began with post-training on our dataset and method, followed by the MultiCSR dataset and method. Table 5 shows these results. In Setting 1, we see that our method improves model performance after post-training with the MultiCSR dataset, achieving a new SOTA performance of 81.63. This highlights the flexibility and compatibility of our approach with different post-training strategies. Moreover, comparing the two settings, MultiCSR offers a greater performance boost when used as the first stage of post-training. This supports our claim that our ranking sentence dataset and method are especially effective in enhancing fine-grained semantic understanding in sentence embedding models.

Figure 4: The analysis of the generated ranking sentences involved calculating the average length (in terms of word count) and the average sentence quality for each index. An index of 0 represents the original sentence.

![The analysis of the generated ranking sentences involved calculating the average length (in terms of word count) and the average sentence quality for each index. An index of 0 represents the original sentence.](page_1012_1012_388_246.png)

(a) average length   (b) sentence quality

4.7 Ranking Sentences Analysis

We further designed two analytical experiments for the synthetic dataset. First, a previous study (An et al., 2024) reports that sentences synthesized using large language models (LLMs) tend to be longer than the original text. Therefore, we investigated whether the generated ranking sentences also exhibit this characteristic. We calculated the average length of sentence tokens for each ranking index of the generated sentences, and the results are
presented in Figure 4(a). The findings indicate that our method produces slightly longer sentences than the originals, but this increase stabilizes, preventing excessive redundancy in subsequent sentences.

Second, we analyzed the quality of the generated sentences. Inspired by the Individual Score method (Chen et al., 2023), we employed GPT-4o to evaluate each sentence’s quality on a scale from 0 (worst) to 100 (best). We randomly selected 200 ranking sentences along with their original data for this evaluation. Figure 4(b) displays the results. Compared to the original sentences, the generated sentences demonstrate higher quality. These results suggest that the quality of the generated sentences can be reliably assured.

5 Related Work

Unsupervised sentence embedding has been widely studied. Early methods extended the word2vec framework (Mikolov et al., 2013) to sentence-level embeddings, such as Skip-Thought (Kiros et al., 2015), FastSent (Hill et al., 2016), and QuickThought (Logeswaran and Lee, 2018). With the rise of PLMs, models like BERT (Kenton and Toutanova, 2019) and RoBERTa (Liu, 2019) have been explored for sentence representation. However, issues like anisotropy (Ethayarajh, 2019) have led to post-processing techniques such as BERT-flow (Li et al., 2020) and BERT-whitening (Su et al., 2021) to improve embedding quality.

With the rise of contrastive learning, the focus shifted toward deriving sentence embeddings by maximizing agreement between different views of the same sentence. Techniques like SimCSE (Gao et al., 2021) utilized dropout-based augmentation to create positive pairs, inspiring follow-up methods (Wang et al., 2022b; Chuang et al., 2022; Liu et al., 2023; Jiang et al., 2022; Wu et al., 2022a; Miao et al., 2023). These methods proved highly effective. However, unsupervised approaches often lag behind their supervised counterparts, which leverage labeled datasets such as natural language inference (NLI) corpora (Bowman et al., 2015; Williams et al., 2018). Yet, such datasets are not easily accessible due to the high annotation cost.

To address these limitations, researchers began exploring sentence generation for unlabeled data (Chen et al., 2022; Ye et al., 2022) using models like T5 (Chung et al., 2024). With the advent of large language models (LLMs), both data annotation and generation have seen significant improvements (Gilardi et al., 2023; Alizadeh et al., 2023; Wang et al., 2024b; Alizadeh et al., 2025). SyncCSE (Zhang et al., 2023) leverages LLMs to generate semantically similar sentence pairs, enhancing the effectiveness of contrastive learning. MultiCSR (Wang et al., 2024a) and GCSE (Lai et al., 2024) further refine the utilization of LLMs for data generation. This line of research builds upon the training paradigm of supervised SimCSE (Gao et al., 2021), where a triplet is generated for contrastive learning. In contrast to these works, our approach shifts the generation objective towards ranking sentence generation, introducing a novel refinement strategy for contrastive learning models.

6 Conclusion

In this paper, we investigate a method for synthesizing ranking sentences by leveraging LLMs to generate sentences progressively increasing semantic divergence, guided by a controlled direction in the latent space. Furthermore, we explore a post-training approach that integrates ranking information and semantic information. Experimental results demonstrate that our method achieves new SOTA performance with minimal cost in ranking sentence synthesis.

7 Limitations

Although our work has achieved a new SOTA performance for existing sentence embedding models, several promising directions still need to be explored. In the realm of data synthesis, this paper primarily concentrates on the process of data generation. However, the selection and refinement of synthesized data are also crucial. Besides, during the post-training process, our primary approach is to integrate ranking and semantic information. This process involves a hyperparameter \( \omega \), whose magnitude influences the model’s performance. Exploring an adaptive method to eliminate the dependence on \( \omega \) is also a worthwhile consideration.

Acknowledgments

This research was partially supported by the National Natural Science Foundation of China (Grants No.U23A20319, 62477044), the Fundamental Research Funds for the Central Universities (No.WK2150110038), and the National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi’an Jiaotong University (No. HMHAI-202410).
References

Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, Inigo Lopez-Gazpio, Montse Maritxalar, Rada Mihalcea, et al. 2015. Semeval-2015 task 2: Semantic textual similarity, english, spanish and pilot on interpretability. In Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015), pages 252–263.

Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2014. Semeval-2014 task 10: Multilingual semantic textual similarity. In Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014), pages 81–91.

Eneko Agirre, Carmen Banea, Daniel Cer, Mona Diab, Aitor Gonzalez Agirre, Rada Mihalcea, German Rigau Claramunt, and Janyce Wiebe. 2016. Semeval-2016 task 1: Semantic textual similarity, monolingual and cross-lingual evaluation. In SemEval-2016-10th International Workshop on Semantic Evaluation; 2016 Jun 16-17; San Diego, CA. Stroudsburg (PA): ACL; 2016. p. 497-511. ACL (Association for Computational Linguistics).

Eneko Agirre, Daniel Cer, Mona Diab, and Aitor Gonzalez-Agirre. 2012. Semeval-2012 task 6: A pilot on semantic textual similarity. in* sem 2012: The first joint conference on lexical and computational semantics–volume 1: Proceedings of the main conference and the shared task, and volume 2: Proceedings of the sixth international workshop on semantic evaluation (semeval 2012). Association for Computational Linguistics. URL http://www.aclweb.org/anthology/S12-1051.

Eneko Agirre, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, and Weiwei Guo. 2013. * sem 2013 shared task: Semantic textual similarity. In Second joint conference on lexical and computational semantics (* SEM), volume 1: proceedings of the Main conference and the shared task: semantic textual similarity, pages 32–43.

Meysam Alizadeh, Maël Kubli, Zeynab Samei, and Shirin Dehghani. 2023. Open-source large language models outperform crowd workers and approach chatgpt in text-annotation tasks. arXiv preprint arXiv:2307.02179, 101.

Meysam Alizadeh, Maël Kubli, Zeynab Samei, Shirin Dehghani, Mohammadmasih Zahedivafa, Juan D Bermeo, Maria Korobeynikova, and Fabrizio Giardi. 2025. Open-source llms for text annotation: a practical guide for model setting and fine-tuning. Journal of Computational Social Science, 8(1):1–25.

Na Min An, Sania Waheed, and James Thorne. 2024. Capturing the relationship between sentence triplets for llm and human-generated texts to enhance sentence embeddings. In Findings of the Association for Computational Linguistics: EACL 2024, pages 624–638.

Tao Lei Hrishikesh Joshi Regina Barzilay, Tommi Jaakkola, Katerina Tymoshenko, and Alessandro Moschitti Lluis Marquez. 2016. Semi-supervised question retrieval with gated convolutions. In Proceedings of NAACL-HLT, pages 1279–1289.

Samuel Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632–642.

Daniel Cer, Mona Diab, Eneko E Agirre, Iñigo Lopez-Gazpio, and Lucia Specia. 2017. Semeval-2017 task 1: Semantic textual similarity multilingual and cross-lingual focused evaluation. In The 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 1–14.

Jianly Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. arXiv preprint arXiv:2402.03216.

Yi Chen, Rui Wang, Haiyun Jiang, Shuming Shi, and Ruifeng Xu. 2023. Exploring the use of large language models for reference-free text quality evaluation: An empirical study. arXiv preprint arXiv:2304.00723.

Yiming Chen, Yan Zhang, Bin Wang, Zuozhu Liu, and Haizhou Li. 2022. Generate, discriminate and contrast: A semi-supervised sentence representation learning framework. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8150–8161.

Yung-Sung Chuang, Rumen Dangovski, Hongyin Luo, Yang Zhang, Shiyu Chang, Marin Soljačić, Shang-Wen Li, Scott Yih, Yoon Kim, and James Glass. 2022. Diffuse: Difference-based contrastive learning for sentence embeddings. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4207–4218.

Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2024. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70):1–53.

Alexis Conneau and Douwe Kiela. 2018. Senteval: An evaluation toolkit for universal sentence representations. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018).

Bill Dolan and Chris Brockett. 2005. Automatically constructing a corpus of sentential paraphrases. In Third international workshop on paraphrasing (IWP2005).
Kawin Ethayarajh. 2019. How contextual are contextualized word representations? comparing the geometry of bert, elmo, and gpt-2 embeddings. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 55–65.

Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. Simse: Simple contrastive learning of sentence embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894–6910.

Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. 2023. Chatgpt outperforms crowd workers for text-annotation tasks. Proceedings of the National Academy of Sciences, 120(30):e2305016120.

Liyang He, Zhenya Huang, Enhong Chen, Qi Liu, Shiewei Tong, Hao Wang, Defu Lian, and Shijin Wang. 2023. An efficient and robust semantic hashing framework for similar text search. ACM Transactions on Information Systems, 41(4):1–31.

Felix Hill, Kyunghyun Cho, and Anna Korhonen. 2016. Learning distributed representations of sentences from unlabelled data. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1367–1377.

Jonathan Ho and Tim Salimans. 2021. Classifier-free diffusion guidance. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications.

Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 168–177.

Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2021. Unsupervised dense information retrieval with contrastive learning. arXiv preprint arXiv:2112.09118.

Ting Jiang, Jian Jiao, Shaohan Huang, Zihan Zhang, Deqing Wang, Fuzhen Zhuang, Furu Wei, Haiwen Huang, Denvy Deng, and Qi Zhang. 2022. Promptbert: Improving bert sentence embeddings with prompts. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8826–8837.

Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of naacl-HLT, volume 1. Minneapolis, Minnesota.

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.

Ryan Kiros, Yukun Zhu, Russ R Salakhutdinov, Richard Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Skip-thought vectors. Advances in neural information processing systems, 28.

Peichao Lai, Zhengfeng Zhang, Wentao Zhang, Fangcheng Fu, and Bin Cui. 2024. Enhancing unsupervised sentence embeddings via knowledge-driven data augmentation and gaussian-decayed contrastive learning. arXiv preprint arXiv:2409.12887.

Bohan Li, Hao Zhou, Junxian He, Mingxuan Wang, Yiming Yang, and Lei Li. 2020. On the sentence embeddings from pre-trained language models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9119–9130.

Rui Li, Liyang He, Qi Liu, Yuze Zhao, Zheng Zhang, Zhenya Huang, Yu Su, and Shijin Wang. 2024a. Consider: Commonalities and specialties driven multilingual code retrieval framework. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 8679–8687.

Rui Li, Qi Liu, Liyang He, Zheng Zhang, Hao Zhang, Shengyu Ye, Junyu Lu, and Zhenya Huang. 2024b. Optimizing code retrieval: High-quality and scalable dataset annotation through large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 2053–2065.

Yizhi Li, Zhenghao Liu, Chenyan Xiong, and Zhiyuan Liu. 2021. More robust dense retrieval with contrastive dual learning. In Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval, pages 287–296.

Peerat Limkonchotiwat, Wuttikon Ponwitayarat, Lalita Lowphansirikul, Can Udomcharoenchaikit, Ekapol Chuangsuwanich, and Sarana Nutanong. 2022. Congen: Unsupervised control and generalization distillation for sentence representation. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 6467–6480.

Jiayu Liu, Zhenya Huang, Tong Xiao, Jing Sha, Jinze Wu, Qi Liu, Shijin Wang, and Enhong Chen. 2024. Socraticlm: Exploring socratic personalized teaching with large language models. Advances in Neural Information Processing Systems, 37:85693–85721.

Jiduan Liu, Jiahao Liu, Qifan Wang, Jingang Wang, Wei Wu, Yunsen Xian, Dongyan Zhao, Kai Chen, and Rui Yan. 2023. Rankcse: Unsupervised sentence representations learning via learning to rank. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13785–13802.

Qi Liu, Zhenya Huang, Yu Yin, Enhong Chen, Hui Xiong, Yu Su, and Guoping Hu. 2019. Ekt: Exercise-aware knowledge tracing for student performance prediction. IEEE Transactions on Knowledge and Data Engineering, 33(1):100–115.
Xueqing Liu, Chi Wang, Yue Leng, and ChengXiang Zhai. 2018. Linkso: a dataset for learning to retrieve similar question answer pairs on software development forums. In Proceedings of the 4th ACM SIGSOFT International Workshop on NLP for Software Engineering, pages 2–5.

Yinhan Liu. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 364.

Lajanugen Logeswaran and Honglak Lee. 2018. An efficient framework for learning sentence representations. arXiv preprint arXiv:1803.02893.

Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, and Roberto Zamparelli. 2014. A SICK cure for the evaluation of compositional distributional semantic models. In Proceedings of the Ninth International Conference on Language Resources and Evaluation, LREC 2014, Reykjavik, Iceland, May 26-31, 2014, pages 216–223. European Language Resources Association (ELRA).

Pu Miao, Zeyao Du, and Junlin Zhang. 2023. Debce: Rethinking unsupervised contrastive sentence embedding learning in the debiasing perspective. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pages 1847–1856.

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems, 26.

Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023. Mteb: Massive text embedding benchmark. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2014–2037.

Nhung Thi-Hong Nguyen, Phuong Phan-Dieu Ha, Luan Thanh Nguyen, Kiet Van Nguyen, and Ngan Luu-Thuy Nguyen. 2022. Spbertqa: A two-stage question answering system based on sentence transformers for medical texts. In International Conference on Knowledge Science, Engineering and Management, pages 371–382. Springer.

OpenAI. 2022. Chatgpt: Optimizing language models for dialogue. In OpenAI Blog.

Bo Pang and Lillian Lee. 2004. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04), pages 271–278.

Bo Pang and Lillian Lee. 2005. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), pages 115–124.

Yeon Seonwoo, Guoyin Wang, Changmin Seo, Saajal Choudhary, Jiwei Li, Xiang Li, Puyang Xu, Sunghyun Park, and Alice Oh. 2023. Ranking-enhanced unsupervised sentence representation learning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15783–15798.

Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1631–1642.

Jianlin Su, Jiarun Cao, Weijie Liu, and Yangyiwen Ou. 2021. Whitening sentence representations for better semantics and faster retrieval. arXiv preprint arXiv:2103.15316.

Ellen M Voorhees and Dawn M Tice. 2000. Building a question answering test collection. In Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, pages 200–207.

Huiming Wang, Zhaodonghui Li, Liying Cheng, Lidong Bing, et al. 2024a. Large language models can contrastively refine their generation for better sentence representation learning. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 7867–7884.

Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024b. Improving text embeddings with large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11897–11916.

Suhe Wang, Xiaoyuan Liu, Bo Liu, and Diwen Dong. 2022a. Sentence-aware adversarial meta-learning for few-shot text classification. In Proceedings of the 29th International Conference on Computational Linguistics, pages 4844–4852.

Wei Wang, Liangzhu Ge, Jingqiao Zhang, and Cheng Yang. 2022b. Improving contrastive learning of sentence embeddings with case-augmented positives and retrieved negatives. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2159–2165.

Janyce Wiebe, Theresa Wilson, and Claire Cardie. 2005. Annotating expressions of opinions and emotions in language. Language resources and evaluation, 39:165–210.

Adina Williams, Nikita Nangia, and Samuel R Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In 2018 Conference of the North American Chapter of the
Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2018, pages 1112–1122. Association for Computational Linguistics (ACL).

Fangzhao Wu, Ying Qiao, Jiun-Hung Chen, Chuhan Wu, Tao Qi, Jianxun Lian, Danyang Liu, Xing Xie, Jianfeng Gao, Winnie Wu, et al. 2020. Mind: A large-scale dataset for news recommendation. In Proceedings of the 58th annual meeting of the association for computational linguistics, pages 3597–3606.

Qiyu Wu, Chongyang Tao, Tao Shen, Can Xu, Xiubo Geng, and Daxin Jiang. 2022a. Pcl: Peer-contrastive learning with diverse augmentations for unsupervised sentence embeddings. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 12052–12066.

Xing Wu, Chaochen Gao, Zijia Lin, Jizhong Han, Zhongyuan Wang, and Songlin Hu. 2022b. Infoce: Information-aggregated contrastive learning of sentence embeddings. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 3060–3070.

Fen Xia, Tie-Yan Liu, Jue Wang, Wensheng Zhang, and Hang Li. 2008. Listwise approach to learning to rank: theory and algorithm. In Proceedings of the 25th international conference on Machine learning, pages 1192–1199.

Shangzi Xue, Zhenya Huang, Jiayu Liu, Xin Lin, Yuting Ning, Binbin Jin, Xin Li, and Qi Liu. 2024. Decompose, analyze and rethink: Solving intricate problems with human-like reasoning cycle. Advances in Neural Information Processing Systems, 37:357–385.

Jiacheng Ye, Jiahui Gao, Zhiyong Wu, Jiangtao Feng, Tao Yu, and Lingpeng Kong. 2022. Progen: Progressive zero-shot dataset generation via in-context feedback. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 3671–3683.

Junlei Zhang, Zhenzhong Lan, and Junxian He. 2023. Contrastive learning of sentence embeddings from scratch. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3916–3932.
A  Ranking Sentences Generation Details

In this section, we present the detailed methodology of several ranking sentence generation methods involved in this paper and the detailed methodology of our method. We employ the premises from the NLI dataset (Bowman et al., 2015; Williams et al., 2018) as the initial unlabeled dataset for these methods.

1. Single-step Generation. Prompting the LLM to generate ranking sentences at once. Our preliminary experiments reveal that generating complete ranking sentences in a single step is too challenging for the LLaMA3-8B-Instruct model. Therefore, we employ the LLaMA3-70B-Instruct model and adopt a few-shot approach to guide the LLM in the generation, ensuring both the coherence and usability of the generated ranking sentences.

2. Iterative Step-by-step Generation. Prompting the LLM to generate ranking sentences step by step. Specifically, based on the result of the previous sentence, we prompt the LLaMA3-8B-Instruct to generate the next sentence. This process continues until 32 sentences have been generated.

3. Our Method. Generating the ranking sentences using our proposed directionally controlled generation method. We employ the LLaMA3-8B-Instruct model to generate ranking sentences. Specifically, for the first generation, we prompt the LLM to generate sentence \( x^{(2)} \). Then, as designed in our method, each input consists of the previous two sentences along with an instruction. By adjusting the sampling strategy of the 8B LLaMA3 model, we progressively generate the final ranking sentences, setting \( \gamma \) to 1.5.

For all generation methods, we employ a rule-based verification process at the final stage to ensure that the generated results are as complete and non-redundant as possible. The first generation method is performed on a Linux server equipped with 8 NVIDIA A800 GPUs, while the second and third methods are conducted on a Linux server with 8 NVIDIA GeForce RTX 4090 GPUs.

Prompt for directly generating complete ranking sentences

Your task is to take an input sentence and generate a sequence of 32 sentences that gradually and progressively diverge in meaning from the original sentence. The final sentence should be completely unrelated to the original sentence.
Example Input: The cat is sleeping on the warm windowsill.
Example Output:
1. The cat is resting on the cozy windowsill.
2. The cat is lying on a soft cushion by the window.
3. A small animal is curled up near the window.
... [Omit the following sentence list here for conciseness.]
Here is the sentence: {sentence}
Each sentence should be similar in length to the original sentence. Do not explain yourself or output anything else.

Prompt for generating ranking sentences step by step

Rewrite the following sentence in a way that slightly changes the meaning while keeping it semantically close. The new sentence should not be an exact paraphrase but should introduce a subtle variation in meaning. Do not lose the core idea of the original sentence.
Here is the sentence: {sentence}
Your response should be similar in length to the original sentence. Do not explain yourself or output anything else.

Prompt for our method

Rewrite the following sentence or phrase using different words and sentence structure while preserving its original meaning. Directly answer with the rewritten sentence. Don’t give any explanation or description other than the rewritten sentence.
Write a sentence that is entailment with: {sentence}.
Result:
B Case Study

In this section, we present a case study to illustrate the generated results of our approach, the single-step generation method, and the iterative step-by-step generation method. Table 6 presents the top 10 generated sentences produced by different methods for a given input sentence. We employ the BGE-m3(Chen et al., 2024) model to obtain their embeddings and compute the cosine similarity between the generated results and the original sentence. Similarity scores for results that are not ranked in descending order of semantic similarity are highlighted in red. We observe that, compared to the other two generation methods, our approach produces results that adhere more closely to a descending order in the semantic space. Moreover, as the generation progresses, the likelihood of producing results that deviate from the expected order increases. This underscores the importance of controlling the direction of generation.

C Model Training Details

All our experimental code is implemented using Python and the PyTorch library. The experiments were conducted on a Linux server equipped with eight NVIDIA GeForce RTX 4090 GPUs. We utilized the official implementations for SynCSE (Zhang et al., 2023) and MultiCSR (Wang et al., 2024a). Specifically, SynCSE provides both model training code and a synthesized dataset. We used their code and dataset to train SynCSE models, including BERT-base, BERT-large, RoBERTa-base, and RoBERTa-large. MultiCSR offers both training and data synthesis code, which we employ to generate data before proceeding with MultiCSR model training. Additionally, in Section 4.6, we reproduce several models, including SimCSE (Gao et al., 2021), InfoCSE (Wu et al., 2022b), PCL (Wu et al., 2022a), and RankCSE (Liu et al., 2023). We downloaded their checkpoints from the official HuggingFace repositories and applied our post-training method. During post-training, each model receives a ranking sentence as input per training step. We use the Adam (Kingma and Ba, 2014) optimizer and set the learning rate to \(3 \times 10^{-6}\). For SynCSE, the hyperparameter \( \omega \) is set to 0.7, while for the other models, \( \omega \) is set to 0.5.

D Algorithm to Get \( \hat{\phi}^j \)

We propose an Algorithm 1 for efficiently computing \( \hat{\phi}^j \) for \( j = 1, 2, \ldots, n \). This algorithm takes a similarity relation matrix \( \Phi = [\phi^1, \phi^2, \ldots, \phi^n] \) as input and outputs a refined matrix \( \hat{\Phi} = [\hat{\phi}^1, \hat{\phi}^2, \ldots, \hat{\phi}^n] \). For each \( \phi^j \), it is unnecessary to compute the full set of values. We only need to calculate the results from index \( i \) onward and then leverage the symmetry of the similarity matrix to complete the remaining entries. The complexity of the algorithm is \( O(n^3) \). Since \( n \) represents the length of the ranking sentences and is a finite value, the computational complexity of this algorithm is significantly lower than that of the model’s inference process.

Algorithm 1 Refined Semantic Similarity Computation

Input: Initial similarity matrix \( \Phi \), hyperparameter \( \omega \).
Output: Refined similarity matrix \( \hat{\Phi} \).
1: Initialize \( A \leftarrow \mathbf{0} \in \mathbb{R}^{n \times n} \).
2: for each row index \( i = 1 \) to \( n \) do
3:   Extract the subarray \( \Phi[i, i : n] \).
4:   Sort the subarray in descending order and obtain sorted indices.
5:   for each column index \( j = i \) to \( n \) do
6:     Find the position index \( j' \) of \( \Phi[i, j] \) in the sorted array.
7:     Compute \( A[i, j] = \Phi[i, j] - \Phi[i, j'] \).
8:   end for
9: end for
10: Fill \( A \) symmetrically: \( A[j, i] = A[i, j] \) for \( j > i \).
11: Compute \( \hat{\Phi} = \Phi + \operatorname{sign}(A) \cdot \log(\omega \cdot |A| + 1) \).
12: return \( \hat{\Phi} \).

E Post-training Experiments

In the preceding experiments, we demonstrated the changes in Spearman’s correlation for SimCSE (Gao et al., 2021), InfoCSE (Wu et al., 2022b), PCL (Wu et al., 2022a), and RankCSE (Liu et al., 2023) on STS tasks before and after training with our data and methodology. In this section, we present the comprehensive results, as illustrated in Figure 7.

F Transfer Task

For the TR tasks, we evaluate our method on seven datasets using the default configurations from SentEval: MR (Pang and Lee, 2005), CR (Hu and Liu, 2004), SUBJ (Pang and Lee, 2004), MPQA (Wiebe et al., 2005), SST-2 (Socher et al., 2013), TREC (Voorhees and Tice, 2000), and MRPC (Dolan and Brockett, 2005). Table 8 shows the results.
<table>
  <tr>
    <th>Original Sentence: A young man wearing a knit cap with the word PARIS on it and a blue jacket on the street.</th>
    <th></th>
    <th></th>
    <th></th>
  </tr>
  <tr>
    <th>Our Method</th>
    <th>Iterative Step-by-step Generation</th>
    <th>Single-step Generation</th>
    <th></th>
  </tr>
  <tr>
    <td>A stylish young man, sporting a PARIS-emblazoned knit cap and a blue jacket, strolled down the street. [0.8016]</td>
    <td>A young man, sporting a knit cap with the word PARIS emblazoned on it, walked down the street in a blue jacket. [0.8660]</td>
    <td>A young man wearing a warm hat with a city name on it and a casual jacket outdoors. [0.6829]</td>
    <td></td>
  </tr>
  <tr>
    <td>As he walked, the stylish young man’s PARIS-emblazoned knit cap and blue jacket made a stylish statement down the street. [0.7996]</td>
    <td>A young man, sporting a knit cap with the word PARIS emblazoned on it, walked down the street in a blue jacket. [0.8342]</td>
    <td>A person wearing a soft hat with a name tag and a casual coat in public. [0.5749]</td>
    <td></td>
  </tr>
  <tr>
    <td>His outfit, featuring a PARIS-emblazoned knit cap and blue jacket, turned heads as he strolled down the street. [0.7367]</td>
    <td>A young man, sporting a knit cap with a bold, cursive "PARIS" emblazoned on it, strolled down the street in a navy blue jacket. [0.8016]</td>
    <td>Someone wearing a hat with a label and a casual coat on the sidewalk. [0.5525]</td>
    <td></td>
  </tr>
  <tr>
    <td>As he walked down the street, people couldn’t help but notice his outfit, which boasted a knit cap adorned with the iconic PARIS logo and a striking blue jacket. [0.7313]</td>
    <td>A young man, sporting a knit cap adorned with a bold, cursive "PARIS" script, walked down the street in a navy blue coat. [0.8077]</td>
    <td>A person wearing a hat with a logo and a casual coat on the pavement. [0.5861]</td>
    <td></td>
  </tr>
  <tr>
    <td>Noticing his outfit became second nature to passersby as he strolled down the street, the knit cap’s PARIS logo and striking blue jacket standing out in bold relief. [0.7246]</td>
    <td>A young man, sporting a knit cap bearing a bold, cursive "PARIS" script, strolled down the street in a navy blue overcoat. [0.7801]</td>
    <td>A man wearing a hat with a brand name and a casual coat in the city. [0.6443]</td>
    <td></td>
  </tr>
  <tr>
    <td>As he walked down the street, it soon became automatic for people to notice his outfit, with the PARIS logo on his knit cap and the striking blue jacket making a bold statement. [0.7064]</td>
    <td>A young man, sporting a knit cap adorned with a striking, cursive "PARIS" motif, ambled down the sidewalk in a navy blue overcoat. [0.7460]</td>
    <td>A man wearing a hat with a logo and a casual coat on the corner. [0.5745]</td>
    <td></td>
  </tr>
  <tr>
    <td>The moment he strolled along the sidewalk, passersby couldn’t help but be drawn to his attire, particularly the eye-catching PARIS emblem on his beanie and the vibrant blue coat that turned heads. [0.6063]</td>
    <td>A young man, sporting a knit cap featuring a bold, cursive "PARIS" logo, strolled leisurely along the sidewalk in a navy blue coat. [0.7859]</td>
    <td>A man wearing a hat with a brand name and a casual coat in the morning. [0.5850]</td>
    <td></td>
  </tr>
  <tr>
    <td>As soon as he ambled down the sidewalk, people walking by were irresistibly captivated by his outfit, specifically the striking PARIS logo on his beanie and the radiant blue coat that commanded attention. [0.5788]</td>
    <td>A young man, sporting a knit cap adorned with a prominent, cursive "PARIS" logo, ambled along the sidewalk in a navy blue overcoat. [0.7683]</td>
    <td>A man wearing a hat with a label and a casual coat in the evening. [0.5787]</td>
    <td></td>
  </tr>
  <tr>
    <td>The moment he strolled down the sidewalk, passersby couldn’t help but be drawn in by his ensemble, particularly the eye-catching PARIS logo on his beanie and the dazzling blue coat that demanded notice. [0.5868]</td>
    <td>A young man, sporting a knit cap bearing a large, cursive "PARIS" logo, strolled leisurely along the sidewalk, clad in a navy blue overcoat. [0.7792]</td>
    <td>A person wearing a hat with a logo and a formal coat at a party. [0.5256]</td>
    <td></td>
  </tr>
  <tr>
    <td>As soon as he walked down the street, people couldn’t resist being captivated by his outfit, specifically the striking PARIS emblem on his hat and the mesmerizing blue coat that commanded attention. [0.5829]</td>
    <td>A young man, wearing a knit cap adorned with a prominent, cursive "PARIS" emblem, ambled along the sidewalk, wrapped in a navy blue overcoat. [0.7471]</td>
    <td>Someone wearing a hat with a brand name and a formal dress at a wedding. [0.5064]</td>
    <td></td>
  </tr>
</table>

Table 6: A case study is conducted to compare our generation method with the Iterative Step-by-Step Generation and Single-Step Generation approaches. The similarity to the original sentence is indicated at the end of each sentence, highlighted in red if not ranked in descending order of semantic similarity.

Overall, we have achieved a new SOTA performance on RoBERTa-base and RoBERTa-large. On BERT-base and BERT-large, both SynCSE-r and MultiCSR-r have demonstrated improvements compared to the results after post-training. Furthermore, our enhancement on the MRPC task is particularly significant. This is because MRPC focuses on distinguishing the similarity between sentence pairs, and by incorporating ranking sentences in post-training, the model becomes more adept at capturing fine-grained semantic differences.
<table>
  <tr>
    <th>Method</th>
    <th>STS-12</th>
    <th>STS-13</th>
    <th>STS-14</th>
    <th>STS-15</th>
    <th>STS-16</th>
    <th>STS-B</th>
    <th>SICK-R</th>
    <th>Avg.</th>
  </tr>
  <tr>
    <td>SimCSE</td>
    <td>66.05</td>
    <td>81.49</td>
    <td>73.61</td>
    <td>79.73</td>
    <td>78.12</td>
    <td>76.52</td>
    <td>71.86</td>
    <td>75.34</td>
  </tr>
  <tr>
    <td>SimCSE-r</td>
    <td>70.09</td>
    <td>82.96</td>
    <td>75.22</td>
    <td>82.04</td>
    <td>78.61</td>
    <td>78.39</td>
    <td>72.52</td>
    <td>77.12</td>
  </tr>
  <tr>
    <td>InfoCSE</td>
    <td>70.23</td>
    <td>84.05</td>
    <td>75.98</td>
    <td>84.78</td>
    <td>81.72</td>
    <td>81.75</td>
    <td>71.09</td>
    <td>78.51</td>
  </tr>
  <tr>
    <td>InfoCSE-r</td>
    <td>70.47</td>
    <td>83.63</td>
    <td>76.25</td>
    <td>85.14</td>
    <td>82.36</td>
    <td>81.89</td>
    <td>72.47</td>
    <td>78.88</td>
  </tr>
  <tr>
    <td>PCL</td>
    <td>73.46</td>
    <td>81.57</td>
    <td>74.91</td>
    <td>82.24</td>
    <td>79.94</td>
    <td>79.41</td>
    <td>71.76</td>
    <td>77.61</td>
  </tr>
  <tr>
    <td>PCL-r</td>
    <td>74.15</td>
    <td>82.26</td>
    <td>75.16</td>
    <td>84.73</td>
    <td>82.73</td>
    <td>81.68</td>
    <td>73.16</td>
    <td>79.12</td>
  </tr>
  <tr>
    <td>RankCSE</td>
    <td>74.55</td>
    <td>85.13</td>
    <td>77.67</td>
    <td>84.23</td>
    <td>81.18</td>
    <td>81.6</td>
    <td>74.28</td>
    <td>79.81</td>
  </tr>
  <tr>
    <td>RankCSE-r</td>
    <td>74.44</td>
    <td>85.73</td>
    <td>78.36</td>
    <td>86.23</td>
    <td>83.16</td>
    <td>81.75</td>
    <td>73.99</td>
    <td>80.52</td>
  </tr>
</table>

Table 7: We compare Spearman’s correlation on STS tasks across several sentence embedding models after post-training with ranking sentences. Their checkpoints based on BERT-base as the model are obtained from their official sources.

<table>
  <tr>
    <th rowspan="2">Model</th>
    <th rowspan="2">Method</th>
    <th colspan="2">MR</th>
    <th colspan="2">CR</th>
    <th colspan="2">SUBJ</th>
    <th colspan="2">MPQA</th>
    <th colspan="2">SST2</th>
    <th colspan="2">TREC</th>
    <th colspan="2">MRPC</th>
    <th colspan="1">Avg.</th>
  </tr>
  <tr>
    <th>MR</th>
    <th>CR</th>
    <th>SUBJ</th>
    <th>MPQA</th>
    <th>SST2</th>
    <th>TREC</th>
    <th>MRPC</th>
  </tr>
  <tr>
    <td rowspan="7">BERT-base</td>
    <td>SimCSE♣</td>
    <td>81.18</td>
    <td>86.46</td>
    <td>94.45</td>
    <td>88.88</td>
    <td>85.50</td>
    <td>89.80</td>
    <td>74.43</td>
    <td>85.81</td>
  </tr>
  <tr>
    <td>DiffCSE♣</td>
    <td>81.76</td>
    <td>86.20</td>
    <td>94.76</td>
    <td>89.21</td>
    <td>86.00</td>
    <td>87.60</td>
    <td>75.54</td>
    <td>85.80</td>
  </tr>
  <tr>
    <td>PromptBERT♣</td>
    <td>80.74</td>
    <td>85.49</td>
    <td>93.65</td>
    <td>89.32</td>
    <td>84.95</td>
    <td>88.20</td>
    <td>76.06</td>
    <td>85.49</td>
  </tr>
  <tr>
    <td>PCL♣</td>
    <td>80.11</td>
    <td>85.25</td>
    <td>94.22</td>
    <td>89.15</td>
    <td>85.12</td>
    <td>87.40</td>
    <td>76.12</td>
    <td>85.34</td>
  </tr>
  <tr>
    <td>RankCSE♣</td>
    <td><b>83.07</b></td>
    <td>88.27</td>
    <td><b>95.06</b></td>
    <td>89.90</td>
    <td><b>87.70</b></td>
    <td><b>89.40</b></td>
    <td><b>76.23</b></td>
    <td><b>87.09</b></td>
  </tr>
  <tr>
    <td>SynCSE*</td>
    <td>81.09</td>
    <td><b>88.29</b></td>
    <td>93.53</td>
    <td><b>90.02</b></td>
    <td>86.60</td>
    <td>84.40</td>
    <td>75.30</td>
    <td>85.60</td>
  </tr>
  <tr>
    <td>MultiCSR*</td>
    <td>81.64</td>
    <td>87.79</td>
    <td>93.83</td>
    <td>89.91</td>
    <td>87.15</td>
    <td>80.20</td>
    <td>75.25</td>
    <td>85.11</td>
  </tr>
  <tr>
    <td rowspan="7">BERT-large</td>
    <td>SynCSE-r</td>
    <td>81.13</td>
    <td>87.82</td>
    <td>94.07</td>
    <td>89.87</td>
    <td>87.42</td>
    <td>83.80</td>
    <td><b>77.86</b></td>
    <td>86.00</td>
  </tr>
  <tr>
    <td>MultiCSR-r</td>
    <td>81.47</td>
    <td>87.53</td>
    <td>93.99</td>
    <td>89.68</td>
    <td>86.55</td>
    <td>83.80</td>
    <td>76.00</td>
    <td>85.57</td>
  </tr>
  <tr>
    <td>SimCSE♣</td>
    <td><b>85.36</b></td>
    <td>89.38</td>
    <td>95.39</td>
    <td>89.63</td>
    <td>90.44</td>
    <td>91.80</td>
    <td>76.41</td>
    <td>88.34</td>
  </tr>
  <tr>
    <td>PCL♣</td>
    <td>82.47</td>
    <td>87.87</td>
    <td>95.04</td>
    <td>89.59</td>
    <td>87.75</td>
    <td>93.00</td>
    <td>76.00</td>
    <td>87.39</td>
  </tr>
  <tr>
    <td>RankCSE♣</td>
    <td>84.63</td>
    <td>89.51</td>
    <td><b>95.50</b></td>
    <td><b>90.08</b></td>
    <td><b>90.61</b></td>
    <td><b>93.20</b></td>
    <td><b>76.99</b></td>
    <td><b>88.65</b></td>
  </tr>
  <tr>
    <td>SynCSE*</td>
    <td>84.66</td>
    <td>89.96</td>
    <td>94.49</td>
    <td><b>90.08</b></td>
    <td>90.44</td>
    <td>86.40</td>
    <td>76.75</td>
    <td>87.54</td>
  </tr>
  <tr>
    <td>MultiCSR*</td>
    <td>84.95</td>
    <td>89.86</td>
    <td>94.42</td>
    <td>89.88</td>
    <td>90.33</td>
    <td>84.60</td>
    <td>76.52</td>
    <td>87.22</td>
  </tr>
  <tr>
    <td rowspan="7">RoBERTa-base</td>
    <td>SynCSE-r</td>
    <td>84.74</td>
    <td>90.15</td>
    <td>94.99</td>
    <td>89.82</td>
    <td><b>90.61</b></td>
    <td>87.80</td>
    <td><b>77.57</b></td>
    <td>87.95</td>
  </tr>
  <tr>
    <td>MultiCSR-r</td>
    <td>84.86</td>
    <td><b>90.17</b></td>
    <td>95.00</td>
    <td>89.88</td>
    <td>89.68</td>
    <td>88.00</td>
    <td>76.29</td>
    <td>87.70</td>
  </tr>
  <tr>
    <td>SimCSE♣</td>
    <td>81.04</td>
    <td>87.74</td>
    <td>93.28</td>
    <td>86.94</td>
    <td>86.60</td>
    <td>84.60</td>
    <td>73.68</td>
    <td>84.84</td>
  </tr>
  <tr>
    <td>DiffCSE♣</td>
    <td>82.42</td>
    <td>88.34</td>
    <td>93.51</td>
    <td>87.28</td>
    <td>87.70</td>
    <td>86.60</td>
    <td>76.35</td>
    <td>86.03</td>
  </tr>
  <tr>
    <td>PromptBERT♣</td>
    <td>83.82</td>
    <td>88.72</td>
    <td>93.19</td>
    <td><b>90.36</b></td>
    <td>88.08</td>
    <td>90.60</td>
    <td>76.75</td>
    <td>87.36</td>
  </tr>
  <tr>
    <td>PCL♣</td>
    <td>81.83</td>
    <td>87.55</td>
    <td>92.92</td>
    <td>87.21</td>
    <td>87.26</td>
    <td>85.20</td>
    <td>76.46</td>
    <td>85.49</td>
  </tr>
  <tr>
    <td>RankCSE♣</td>
    <td>83.32</td>
    <td>88.61</td>
    <td><b>94.03</b></td>
    <td>88.88</td>
    <td>89.07</td>
    <td><b>90.80</b></td>
    <td>76.46</td>
    <td>87.31</td>
  </tr>
  <tr>
    <td rowspan="7">RoBERTa-large</td>
    <td>SynCSE*</td>
    <td>84.82</td>
    <td><b>91.31</b></td>
    <td>93.18</td>
    <td>89.70</td>
    <td>90.28</td>
    <td>84.80</td>
    <td>76.70</td>
    <td>87.26</td>
  </tr>
  <tr>
    <td>MultiCSR*</td>
    <td><b>84.99</b></td>
    <td>91.23</td>
    <td>93.07</td>
    <td>89.42</td>
    <td><b>91.10</b></td>
    <td>84.60</td>
    <td>77.28</td>
    <td><b>87.38</b></td>
  </tr>
  <tr>
    <td>SynCSE-r</td>
    <td>83.78</td>
    <td>91.15</td>
    <td>92.98</td>
    <td>89.50</td>
    <td>89.95</td>
    <td>85.80</td>
    <td>77.33</td>
    <td>87.21</td>
  </tr>
  <tr>
    <td>MultiCSR-r</td>
    <td>84.89</td>
    <td>90.70</td>
    <td>93.62</td>
    <td>89.50</td>
    <td>90.06</td>
    <td>85.40</td>
    <td><b>78.38</b></td>
    <td><b>87.51</b></td>
  </tr>
  <tr>
    <td>SimCSE♣</td>
    <td>82.74</td>
    <td>87.87</td>
    <td>93.66</td>
    <td>88.22</td>
    <td>88.58</td>
    <td>92.00</td>
    <td>69.68</td>
    <td>86.11</td>
  </tr>
  <tr>
    <td>PCL♣</td>
    <td>84.47</td>
    <td>89.06</td>
    <td>94.60</td>
    <td>89.26</td>
    <td>89.02</td>
    <td>94.20</td>
    <td>74.96</td>
    <td>87.94</td>
  </tr>
  <tr>
    <td>RankCSE♣</td>
    <td>84.61</td>
    <td>89.27</td>
    <td>94.47</td>
    <td>89.99</td>
    <td>89.73</td>
    <td><b>92.60</b></td>
    <td>74.43</td>
    <td>87.87</td>
  </tr>
  <tr>
    <td>SynCSE*</td>
    <td>87.42</td>
    <td>92.21</td>
    <td>94.19</td>
    <td><b>90.82</b></td>
    <td>91.60</td>
    <td>85.00</td>
    <td>76.87</td>
    <td>88.30</td>
  </tr>
  <tr>
    <td>MultiCSR*</td>
    <td>87.05</td>
    <td>91.87</td>
    <td>94.07</td>
    <td>90.53</td>
    <td>91.60</td>
    <td>88.80</td>
    <td>78.26</td>
    <td>88.88</td>
  </tr>
  <tr>
    <td>SynCSE-r</td>
    <td>87.24</td>
    <td><b>92.29</b></td>
    <td><b>94.65</b></td>
    <td>90.52</td>
    <td><b>92.37</b></td>
    <td>91.40</td>
    <td><b>79.01</b></td>
    <td><b>89.64</b></td>
  </tr>
  <tr>
    <td>MultiCSR-r</td>
    <td><b>87.45</b></td>
    <td><b>92.29</b></td>
    <td><b>94.56</b></td>
    <td>90.45</td>
    <td><b>91.98</b></td>
    <td>90.80</td>
    <td><b>78.61</b></td>
    <td><b>89.45</b></td>
  </tr>
</table>

Table 8: Comparison of different sentence embedding models accuracy on transfer tasks. The value highlighted in bold is the best value, and the value underlined is the second-best value. “♣”: results from (Liu et al., 2023). “♣”: results from (Wang et al., 2024a). “*”: we reproduce the results with the officially released corpus from (Zhang et al., 2023) and (Wang et al., 2024a).

Reviewer paper summaries: Existing methods leverage LLMs to construct high-quality sentence pairs automatically. However, they overlook the critical role of ranking sentences to capture the similarity between sentences. This paper proposes a latent space directional control method for ranking sentence generation and a post-training method for synthesized ranking sentences. Experiments results show that the method achieves improvement on some datasets.

This manuscript introduced a fine-grained controlled generation strategy over language models for generating a sequence of ranking sentences to assist the contrastive learning of sentence transformers. 

Sentences synthesis: Given a pair of sentences, it generates the next sentence by weighted summing the logits of the predicted token at timestep t with different sentences as prompts.

Model post-training: After obtaining the ranking sentences, they integrate the ranking information via ListMLE during post-training.

This paper addresses the limitation of current sentence embedding training methods in capturing fine-grained semantic differences. Traditional contrastive learning approaches for sentence embeddings rely on either human-labeled pairs/triplets (e.g. NLI datasets) or synthetic sentence pairs generated by LLMs. However, using only sentence pairs or triplets fails to distinguish between “highly similar” and “slightly different” sentences, as pairwise supervision treats similarity in a binary or coarse way. This paper proposes that instead of generating just pairs, use LLMs to generate ranking sentences, i.e. an ordered sequence of sentences from an original sentence that gradually diverge in meaning. To accomplish this, the authors introduce a directionally controlled generation method for LLMs. Instead of prompting an LLM in an unconstrained way (which might yield inconsistent or ambiguously related sentences), they control the generation process in the latent space to maintain a consistent direction of semantic change. Concretely, when generating the i-th sentence in the ranking sequence, they condition the LLM not only on the immediately preceding sentence x^(i-1), but also on the one before that (x^(i-2)). By combining the conditional probabilities from the two preceding sentences, weighted by a hyperparameter α, the next sentence is sampled to continue the semantic trajectory set by the previous steps. They do this with the goal of ensuring that each new sentence is a further semantic step away in the same “direction” as the prior divergence, rather than veering off randomly.

After synthesizing a collection of such ranking sequences using an LLM, the authors then refine (post-train) a sentence embedding model with the generated data. They start with a pre-trained embedding model (in experiments, a DiffCSE-based model or the recent SynCSE/MultiCSR models) and further train it using a custom objective that leverages the ranking information. Simply using the sequences in contrastive learning would not fully exploit the order information, so they design a two-part training signal: (1) Semantic information – the continuous similarity scores (e.g. cosine similarity) between embedded sentences, and (2) Ranking information – the relative order constraints derived from the known ranking of the sequence. They enforce that for any anchor sentence in a list, closer sentences (earlier in the list) should yield higher similarity scores than more distant ones. To do this, they compute the model’s pairwise similarities for all sentence pairs in a ranking sequence, then adjust those similarity scores by nudging them up or down wherever the model’s induced ranking disagrees with the true ranking order. The adjusted scores (which blend the model’s semantic estimates with the correct rank order signals) serve as a target for training. They optimize the model with a ListMLE loss, a list-wise ranking loss, so that the model learns to produce embeddings that result in the correct ranking of sentences by similarity. This post-training procedure effectively integrates the ranking supervision into the embedding space. The paper’s contributions are threefold: (i) A novel LLM-based data generation approach that produces ranking sequences of sentences with controlled semantic divergence, addressing the gap of fine-grained supervision. (ii) A tailored post-training algorithm that combines contrastive semantic similarity modeling with rank order constraints (using ListMLE) to refine any existing sentence embedding model. (iii) State-of-the-art results on multiple benchmarks, achieved without any human-labeled pairs.
Summary of strengths: 1. The paper is well-written.

2. Using LLMs to generate ranking sentences is interesting. 

3. The ranking dataset contributes to the community to some extent.

4. Post-training improves performance on some datasets.

Interesting work with sufficient experiments.

1. The paper claims to be the first to generate ranking sequences of sentences using an LLM rather than just pairs or triplets.

2. The authors validate their method using seven benchmark datasets for Semantic Textual Similarity (STS-12, STS-13, STS-14, STS-15, STS-16, STS-B, and SICK-R). Instead of focusing on a single dataset, they compute an average performance score across all datasets. This increases the reliability of their findings and ensures that the improvements are not due to dataset-specific artifacts.

3. The paper provides a clear formulation of how to incorporate two preceding sentences to guide generation (Equations 3 and 4) and how to integrate ranking information into model training (Equations 6–8).

4. The approach achieves state-of-the-art performance across a wide range of evaluations.

5. The evaluation is comprehensive, with comparisons against a diverse set of baselines.

6. The paper is well-structured and generally clear in its explanations.
Summary of weaknesses: 1. Why does this paper use post-training to evaluate effectiveness? The comparison in this paper seems quite strange. 
(1) Would post-training on datasets from other methods also improve performance? 
(2) Would fine-tuning BERT directly on your dataset surpass the baseline methods?

2. The results are based on BERT and Roberta. However, LLMs are effective in sentence embedding [1]. Please use LLMs for evaluation; LLMs around 2B are fine.
3. The improvement is marginal for BERT-large, RoBERTa-base, and RoBERTa-large, especially considering that it constructs a large dataset and requires further training. 
4. Can the quality of the dataset in this paper be measured? Would further filtering datasets help improve performance?

[1] BeLLM: Backward Dependency Enhanced Large Language Model for Sentence Embeddings, NAACL 2024

In table 1, RoBERTa-large, why DebCSE outperforms your baselines in STS12-15?

1. While the method achieves new SOTA results, the gains over the strongest baselines are not particularly impressive or significant.

2. The proposed solution introduces several moving parts and hyperparameters, which is a drawback. It essentially presents an engineering solution that slightly outperforms SOTA.

3. The paper focuses on the generation process and uses the data for training, but it provides limited analysis of the nature of the generated sentences beyond examples and similarity heatmaps.
Comments, suggestions and typos: 1. Missing reference about data augmentation of sentence embedding: 

[2] Improving Text Embeddings with Large Language Models, ACL 2024

2. Please consider directly indicating which is your method in Table 1.

[question] The author provides a geometric conceptualisation in lines 238-240 and Figure 2. Whether your approach could be better suited in the "latent arithmetic" concept? e.g., ref: https://arxiv.org/pdf/2210.06230. Given two sentence vectors, the weighted summed vector is left in the middle of both input vectors, resulting in the steering of direction.

Typos:
  1. I believe changing "post-training" to "post-train" would improve readability:
        "We aim to post-training the existing model" → better be "We aim to post-train the existing model."
        "We post-training the sentence embedding model using the ListMLE loss." → better be "We post-train the sentence embedding model using the ListMLE loss."
  2. "MultiCSR" is sometimes written incorrectly as "MultiSCR"—this should be checked for consistency.

Comments:
While this might be outside the authors' scope, since they are generating a large number of sentences with an LLM, a brief discussion on the nature of these sentences could be valuable.
Overall assessment scores: 2.5

4

4