Venue: NAACL 2025

# _Little Giants_: Synthesizing High-Quality Embedding Data at Scale

 Haonan Chen\({}^{1}\); Liang Wang\({}^{2}\), Nan Yang\({}^{2}\), Yutao Zhu\({}^{1}\),

Ziliang Zhao\({}^{1}\), Furu Wei\({}^{2}\), Zhicheng Dou\({}^{1}\)

\({}^{1}\)Gaoling School of Artificial Intelligence, Renmin University of China

\({}^{2}\)Microsoft Corporation

{hnchen,dou}@ruc.edu.cn

{wangliang,nanya,fuwei}@microsoft.com

Work done during Haonan's internship at MSR Asia. Prof. Zhicheng Dou is the corresponding author.

###### Abstract

Synthetic data generation has become an increasingly popular way of training models without the need for large, manually labeled datasets. For tasks like text embedding, synthetic data offers diverse and scalable training examples, significantly reducing the cost of human annotation. However, most current approaches rely heavily on proprietary models like GPT-4, which are expensive and inefficient for generating large-scale embedding data. In this paper, we introduce SPEED, a framework that aligns open-source small models (8B) to efficiently generate large-scale synthetic embedding data. Through supervised fine-tuning, preference optimization, and self-improvement, SPEED enables small open-source models to produce high-quality data. Remarkably, SPEED uses only less than 1/10 of the GPT API calls, outperforming the state-of-the-art embedding model E5\({}_{\text{mistral}}\) when both are trained solely on their synthetic data. Using this efficient generator, we conduct a comprehensive study on how various factors within the alignment pipeline impact data quality and reveal the scaling law for synthetic embedding data. Our codes and models are released in [https://github.com/haon-chen/SPEED](https://github.com/haon-chen/SPEED).

## 1 Introduction

Text embedding models encode natural language texts into latent vectors. They are widely used in downstream tasks such as classification, clustering, retrieval, and summarization. Many researchers have trained general embedding models that can support various tasks Reimers and Gurevych (2019); Wang et al. (2022); Xiao et al. (2024). Most of these models require large-scale weakly-supervised data and high-quality labeled data for multi-stage training, which requires careful data curation and costly human effort. Thanks to the powerful language modeling ability and vast knowledge of large language models (LLMs), some works attempt to utilize LLMs to generate synthetic data for training embedding models Jeronymo et al. (2023); Wang et al. (2024); Lee et al. (2024).

However, most of these works solely use proprietary LLM like GPT-4 for data synthesis Wang et al. (2024); Lee et al. (2024). For example, E5\({}_{\text{mistral}}\) generates triplets of (query, positive document, hard negative document) for various embedding tasks from scratch. While synthesizing embedding data without relying on existing corpora can yield more diverse examples, using black-box models can be extremely costly, especially given that this data often includes long documents. A straightforward approach to reduce costs is to use small models to synthesize embedding data instead, which have proven effective for tasks such as mathematical reasoning Zhou et al. (2024); Bansal et al. (2024); Chen et al. (2024). However, synthesizing embedding data often requires the generation of hard negatives - documents that are similar to positive ones and are essential for learning nuanced embedding representations. These hard negatives are challenging for small models to synthesize, as they are difficult for language models to distinguish. An early work explores the ability of small models for synthesizing embedding data Jeronymo et al. (2023), but it uses small models to generate data

Figure 1: An illustration comparing the existing pipeline with our data synthesis framework.



directly without special tailoring for data synthesis, resulting in poor performance.

In this work, we propose to align open-source small models (8B) to synthesize large-scale high-quality embedding data. Compared to existing methods that rely solely on expensive GPT-4, our approach can generate more data at a much lower cost. Our primary goal is to study the alignment of small models for synthesizing embedding data, which has been neglected by existing works. Specifically, we aim to address the following research questions in this paper:

**RQ1**: How to align small models for synthesizing high-quality embedding data at scale?

**RQ2**: How do factors within the alignment framework affect the quality of synthetic data?

**RQ3**: Synthetic data is theoretically infinite. What is the scaling law for synthetic embedding data?

To shed light on **RQ1**, we design an alignment framework that trains small LLMs to efficiently **S**ynthesize large-scale su**PE**rior **E**mbedding **D**ata (SPEED). As illustrated in Figure 1, our framework consists of three key models: a junior generator for initial data synthesis, a senior generator for advanced data generation, and a data reviser for self-improvement. The goal is to distill knowledge from GPT-4 into these smaller models. We first use GPT-4 to brainstorm task descriptions. However, since GPT-4 often generates hallucinations and data of specific domains (_e.g._, climate change) [5], we sample topics from the Open Directory Project to ensure diverse and balanced tasks.1 Based on these tasks, GPT-4 produces a small set of seed data, which we use to finetune the junior generator via supervised fine-tuning (SFT). The junior generator produces root data, which is further evaluated by GPT-4 to produce signals that guide the preference optimization process, resulting in a senior generator. The root data is also revised by GPT-4 to produce revision signals for training a data revisor. Inspired by the idea of scaling inference compute for LLMs [7], the review refines the synthetic data with minimal additional inference cost, enabling self-improvement.

Footnote 1: [http://odp.org](http://odp.org): Open-source collection of web topics.

As for **RQ2**, with these low-cost yet powerful data synthesis models ready, we are able to conduct extensive experiments to study the factors affecting the alignment. We find that settings such as the base model used for alignment, the diversity of tasks, and the number of training samples can influence the quality of synthetic data. For **RQ3**, we generate large-scale data using the efficient generators to reveal the scaling law. We observe a log-linear relationship between the performance of the embedding model and the size of synthetic embedding data.

In summary, our contributions are as follows:

* We design a framework to fine-tune small LMs (8B) for synthesizing large-scale data, achieving superior embedding performance with less than 1/10 of the GPT API calls required by E5\({}_{\text{mistral}}\).
* We comprehensively study how the factors within the alignment framework influence the quality of synthetic data.
* We investigate the scaling law of synthetic embedding data and reveal that the embedding model's performance follows a log-linear relationship with the data size.

## 2 Related Work

**Text Embedding** Text embedding models have gained much attention in the era of deep learning. Some existing models, such as SBERT [9], E5 [6], and BGE [11], attempt to produce general text embeddings for various tasks. However, most of them require lots of labeled data. In this work, we attempt to train a model with synthetic data.

**Large Language Models** Though proprietary LLMs [12, 13] are very powerful, invoking their APIs can be quite expensive and unaffordable for common usage. Many open-source LLMs have been released for more efficient language modeling, such as LLaMA [14] and Mistral [15]. Some works attempt to improve the ability of LLMs for text embedding tasks, such as ad-hoc retrieval [13], conversational retrieval [10], and multilingual text embedding [6]. Our work aims to use synthetic data to improve the LLM's ability of text embedding.

**Synthetic Data** The generation of synthetic data have been studied by many researchers for various embedding tasks. In early times, they have been used to produce pseudo labels and query/document expansions [14, 6, 15]. Using the ability of LLMs, synthetic data have been used for 

code generation Gunasekar et al. (2023); Hui et al. (2024), mathematical reasoning Chan et al. (2024); Li et al. (2024a); Zhou et al. (2024a, b), and text embedding Jeronymo et al. (2023); Viswanathan et al. (2023); Wang et al. (2024); Li et al. (2024b); Patwa et al. (2024); Lee et al. (2024); Sturua et al. (2024). Though they have already shown great performance, most of these works heavily rely on black-box LLMs (_e.g._, E5\({}_{\text{mistral}}\)Wang et al. (2024), SynCSE Zhang et al. (2023) and Gecko Lee et al. (2024)) for data synthesis. Some of them uses small LLM to generate data without alignment Thirukovalluru et al. (2024), which produces data of low quality. Our work aims to align small models for generating large scale text embedding data efficiently.

## 3 Methodology: SPEED

In this section, we aim to answer **RQ1** using our alignment framework, SPEED. As shown in Figure 2, SPEED consists of four stages: (1) GPT-4 is first used to generate diverse task descriptions based on multi-grained topics sampled from the ODP. A junior generator then distills knowledge from GPT-4 by training on a small set of seed data. (2) The junior generator synthesizes root data, which GPT-4 uses to produce preference signals. These signals are used to train a senior generator through preference optimization. (3) The root data is also evaluated by GPT-4 to produce revised data for finetuning a data revisor. (4) Finally, the senior generator synthesizes large-scale embedding data, and the revivor refines them into high-quality data for training the embedding model.

### Preliminaries

Many works have tried to generate synthetic data using modern LLMs for downstream tasks fine-tuning. Following E5\({}_{\text{mistral}}\)Wang et al. (2024), in order to synthesize data for training an embedding model, we generate data for four kinds of tasks: classification (long-short match), semantic textual similarity (STS), retrieval (short-long match), and text matching (short-short and long-long match). For simplicity, we will denote the data synthesis prompts as a set \(P\) without distinction.2 We use GPT-4 to brainstorm a pool of candidate tasks \(T\) as instructions. With a prompt \(p\in P\) and a task instruction \(t\in T\), an LLM \(\pi_{\theta}\) can synthesize an embedding data sample \(d\sim\pi_{\theta}(d~{}|~{}p,t)\). Each data example is a triplet of (query, positive document, hard negative document). For example, for a classification task, the query is a long text and documents are short labels. More information on the structure of these data can be found in Appendix D.

Footnote 2: Since our research focus is how to align small models to synthesize embedding data efficiently rather than adjusting prompts for the synthesis process, we will follow the task types and prompt templates in E5\({}_{\text{mistral}}\).

### Aligning Small Models for Synthesizing Embedding Data

Most existing approaches that synthesize embedding data suffer from the high cost of heavily relying on proprietary LLMs. We aim to align small models that can generate large-scale embedding data effectively and efficiently.

#### 3.2.1 Task Brainstorming

Synthesizing embedding data from scratch can be quite challenging since these data are often long and complex. We first generate a pool of candidate tasks as instructions for LLMs to further generate concrete data. Since these task descriptions are very short (about 10 words) and need to be high-quality, we use GPT-4 to brainstem them. Furthermore, we sample multi-grained topics from open

Figure 2: An overview of SPEED. We align small LLMs (8B) to synthesize large-scale high-quality embedding data.

 

directory project (ODP) and specify one topic for each brainstorming prompt to mitigate the hallucination and extract more diverse knowledge from GPT-4 Chang (2023). For example, we prompt GPT-4 as "_Brainstorm a list of potentially useful text retrieval tasks for the topic: [topic]._".3 Then we will get a diverse set of task descriptions and generate embedding data conditioned on them.

Footnote 3: Due to space limitation, we will not present full prompts in this section. The complete prompts are in Appendix C.

#### 3.2.2 Training a Junior Generator

Proprietary LLMs such as GPT-4 have been proven to generate high-quality embedding data Wang et al. (2024); Lee et al. (2024). However, it can be expensive if we generate large-scale embedding data solely using GPT-4. Our goal is to distill the data synthesis capability of GPT-4 into small models that can synthesize large-scale data at low cost.

We first use GPT-4 to generate a small set of seed data \(D_{\text{seed}}\sim\pi^{\text{GPT-4}}_{\theta}(D_{\text{seed}}\mid P,T)\). The constructed training data for SFT is \(D_{\text{SFT}}=\{p_{i},t_{i},d_{i}\}_{i=1}^{N}\). To distill knowledge from GPT-4, we apply a standard Supervised Fine-tuning (SFT) objective to initialize our junior generator \(\pi^{\text{Ir}}_{\theta}\):

\[\mathcal{L}(\theta^{\text{lr}})=-\sum\nolimits_{(p_{i},t_{i}, d_{i})\in\mathcal{D}_{\text{SFT}}}\log\mathbb{P}_{\theta}(d_{i}\mid p_{ i},t_{i}), \tag{1}\]

where \(\theta^{\text{lr}}\) denotes the parameters of our junior generator. We aim to train a small model with basic capability of synthesizing embedding data given various prompt templates and task instructions.

#### 3.2.3 Further Training Using Preference Optimization

Although our junior generator can already generate embedding data of decent quality, we still want to boost its ability. Preference optimization Schulman et al. (2017) is a popular way to be performed on a model for further training after SFT Dong et al. (2024); Yu et al. (2024). Since our goal is to perform optimization on \(\pi^{\text{Ir}}_{\theta}\), we use GPT-4 to produce preference signals based on the data generated by \(\pi^{\text{fr}}_{\theta}\) itself.

Specifically, \(\pi^{\text{Ir}}_{\theta}\) generates a list of embedding data given each prompt, formatting a set of root data \(D_{\text{root}}\sim\pi^{\text{Ir}}_{\theta}(D_{\text{root}}\mid P,T)\). As illustrated in Figure 2, GPT-4 evaluates the best and the worst data in each data list and constructs preference pairs accordingly. We prompt GPT-4 as: "_Your mission is to judge which data this language model generates fits the prompt most and which fits worst, and explain your judgment._". In this work, we perform Direct Preference Optimization (DPO) Rafailov et al. (2023) because it is a popular and low-cost method. The formatted training set for DPO is \(D_{\text{DPO}}=\{p,t,d_{w},d_{l},\}\), where \(d_{w}\) and \(d_{l}\) are the winning and losing one, respectively. Then, we apply the standard DPO on our junior generator:

\[\begin{split}&\mathcal{L}_{\text{DPO}}(\pi^{\text{Ir}}_{\theta}; \pi_{\text{ref}})=\\ &-\mathbb{E}_{(p,t,d_{w},d_{l})\sim\mathcal{D}}\left[\log\sigma \left(\beta\log\frac{\pi^{\text{Ir}}_{\theta}(d_{w}\mid x)}{\pi_{\text{ref}}( d_{w}\mid x)}\right.\right.\\ &\left.\left.-\beta\log\frac{\pi^{\text{Ir}}_{\theta}(d_{l}\mid x )}{\pi_{\text{ref}}(d_{l}\mid x)}\right)\right],\end{split} \tag{2}\]

where \(\pi_{\text{ref}}\) is the reference model set as \(\pi^{\text{Ir}}_{\theta}\) in the beginning and remains frozen, \(\sigma\) is the sigmoid function, and \(\beta\) controls how much DPO focus on \(\pi_{\text{ref}}\). After this, we manage to obtain a senior generator \(\pi^{\text{gr}}_{\theta}\) that can synthesize higher-quality data since it has learned about how to make better choices given a data synthesis prompt.

#### 3.2.4 Training a Data Review

Scaling the inference compute of LLMs has been a popular way to boost the LLM's performance from the inference side Brown et al. (2024). Inspired by this, we employ another small model to refine our synthetic data. This allows us to further improve data quality with only a small increase in inference cost, as the revisor model is also small. Specifically, we train an additional LLM to serve as the data revisor, identifying and refining potential flaws in the synthetic data.

Specifically, to boost the efficiency of the alignment process, we reuse \(D_{\text{root}}\) to produce revised data. This allows us to train both \(\pi^{\text{Sr}}_{\theta}\) and the revisor \(\pi^{\text{Re}}_{\theta}\) simultaneously. GPT-4 produces data revision signals by evaluating the root data from three key aspects: (1) its relevance to the task, (2) its completeness based on the requirements in the prompt, (3) the accuracy of its factual content. The revised data is \(D^{\text{re}}_{\text{root}}\sim\pi^{\text{GFPT-4}}_{\theta}(D^{\text{re}}_{ \text{root}}\mid P,T,D_{\text{root}})\) and the data for SFT is \(D^{\text{re}}_{\text{SFT}}=\{p_{j},t_{j},d^{\text{root}}_{j},d^{\text{re}}_{ j}\}_{j=1}^{M}\). Similarly, a standard SFT approach is performed on an unaligned small LM:

\[\mathcal{L}(\theta^{\text{Re}}) =-\sum\nolimits_{(x_{j},d^{\text{re}}_{j})\in\mathcal{D}^{\text{ re}}_{\text{SFT}}}\log\mathbb{P}_{\theta}(d^{\text{re}}_{j}\mid x_{j}),\] \[x_{j} =(p_{j},t_{j},d^{\text{root}}_{j}), \tag{3}\]

where \(\theta^{\text{Re}}\) denotes the parameters of our revisor 



### Finetuning Embedding Model Using Synthetic Data

With our aligned senior generator \(\pi^{\text{Sr}}_{\theta}\) and revisor \(\pi^{\text{Re}}_{\theta}\) ready, we are able to generate high-quality synthetic embedding data at scale. Specifically, \(\pi^{\text{Sr}}_{\theta}\) first generates a large set of synthetic data \(D_{\text{syn}}\sim\pi^{\text{Sr}}_{\theta}(D_{\text{syn}}\mid P,T)\). Then \(\pi^{\text{Re}}_{\theta}\) revises them into high-quality data \(D^{\text{re}}_{\text{syn}}\sim\pi^{\text{Re}}_{\theta}(D^{\text{re}}_{\text{ syn}}\mid P,T,D_{\text{syn}})\). For efficiency, we avoid iterative improvements and perform the revision in a single pass.

Following the common approach of task-specific fine-tuning Xiao et al. (2024); Wang et al. (2024), an instruction template is applied on each query within \(D^{\text{re}}_{\text{syn}}\) as: \(q^{i}=\text{Instruct:}\{t\}~{}\backslash n\) Query:\(\{q\}\), where \(q^{i}\) is the original query \(q\) with task description. We do not apply this template on the document side for pre-building the index. We append an [EOS] token to each \(q^{i}\) and document \(d\). Each output of the last layer [EOS] is taken as the representation \(\mathbf{q}^{i}\) and \(\mathbf{d}\). To train the embedding model, we apply a standard contrastive learning objective:

\[\mathcal{L}_{\text{CL}}=-\log\frac{\phi(\mathbf{q}^{i},\mathbf{d}^{+})}{\phi (\mathbf{q}^{i},\mathbf{d}^{+})+\sum_{d^{-}\in\mathcal{N}}\phi(\mathbf{q}^{ i},\mathbf{d}^{-})}, \tag{4}\]

where \(\mathcal{N}\) represents negative documents, \(\phi(\cdot)=\exp(\cos(\cdot)/\tau)\), \(\cos(\cdot)\) denotes cosine similarity, and \(\tau\) is a temperature hyperparameter.

## 4 Experiments

### Experimental Setup

SPEED synthesizes 920K embedding data samples in total for training after MinHash deduplication. The proprietary LLM used for knowledge distillation is _GPT-do-2024-05-13_. The base model we use to train our generators is LLaMA-3-8B Meta (2024). We test our finetuned embedding model on the MTEB benchmark Muennighoff et al. (2023). This benchmark contains 7 kinds of 56 English embedding tasks: classification (12), clustering (11), pair classification (3), reranking (4), retrieval (15), semantic textual similarity (10) and summarization (1). The synthetic data proportion of our four embedding task types, _i.e._, classification, STS, retrieval, and text matching is 7:7:7:2. For fair comparisons to E5mistral, we train Mistral-7B-v0.1 Jiang et al. (2023) as our embedding model and use the same labeled data for "_Supervised Models_" setting. We use LoRA Hu et al. (2022) to finetune our embedding model.

In addition to existing baselines that consists of OpenAI's text-embedding-34, GTR Ni et al. (2022), GTE Li et al. (2023), jina-embeddings-v3 Sturua et al. (2024), Gecko Lee et al. (2024), and E5mistral-7b Wang et al. (2024), we also im

plement two baselines finetuned on synthetic data only. In particular, we use llama3-8B-instruct and gpt-4o to synthesize 230K embedding data using the same synthesis prompts and data proportion of SPEED. Then we finetune Mistral-7B-v0.1 with these data to produce two baselines: \({\rm Mistrall}_{\text{llama3}}\) and \({\rm Mistrall}_{\text{gpt-4o}}\).

More details about the synthetic data, implementation details, and prompts can be found in Appendix A, B, and C, respectively.

### Main Results

The results are presented in Table 1. SPEED achieves the best performance in the zero-shot setting and the second-best performance in the supervised setting. This demonstrates the effectiveness of our framework, as SPEED**can generate large-scale high-quality data using the smallest language model**. These results address **RQ1**, confirming that SPEED is an effective way to align small models for synthesizing large-scale embedding data. Furthermore, we can make these observations: (1) Comparing to \({\rm Mistrall}_{\text{llama3}}\), SPEED improves its performance greatly. This demonstrates that our alignment framework enables a base small model to synthesize higher-quality data than its instruct-tuned version. Additionally, as shown in Table 2, SPEED with just 230K data examples also outperforms \({\rm Mistrall}_{\text{llama3}}\). (2) Intriguingly, SPEED outperforms E5\({}_{\text{mistral-7b}}\) in the zero-shot setting but slightly underperforms in the full-data setting. We attribute this to the fact that, while our synthetic data is more diverse and covers a broader range of scenarios, E5\({}_{\text{mistral-7b}}\)'s data is structurally closer to labeled data, as it is generated by the powerful but costly GPT. (3) Gecko performs well on some certain types of embedding tasks. We believe this is because Gecko uses a black-box model to generate a large set of synthetic data (6.6M), potentially covering more task types than both SPEED and E5\({}_{\text{mistral-7b}}\).

### RQ2 Alignment Analysis

In this section, we will look deeper into SPEED and provide comprehensive analysis of how each factor influences the synthetic data. For efficient analysis, we synthesize 230K embedding data using the same data proportion of SPEED for each model and perform zero-shot evaluation on MTEB.

#### 4.3.1 Ablation Study

To evaluate each component of SPEED, we first conduct ablation experiments on our alignment framework. The results are presented in Table 2. We can make the following observations: (1) \(\pi^{\text{lr}}_{\theta}\) itself can already synthesize embedding data of decent quality (62.6), which demonstrates the effectiveness of our aligned junior generator. (2) "SPEED w/o. DPO", _i.e._, only \(\pi^{\text{lr}}_{\theta}\) and \(\pi^{\text{Re}}_{\theta}\) causes performance decreasing. This demonstrates our DPO training process can further enhance the synthesis ability of \(\pi^{\text{lr}}_{\theta}\). (3) The performance drops after discarding \(\pi^{\text{Re}}_{\theta}\). This shows revising the synthetic data with our data revisor can enhance the data quality by introducing a little more inference compute.

#### 4.3.2 Task Brainstorming

To mitigate hallucination and introduce diversity to LLMs, we propose to use GPT-4 to brainstorm a candidate pool of task descriptions with multi-grained topics before we synthesize specific data. To study the influence of topic diversity and coverage, we perform experiments from two aspects and present the results in Table 3: (1) The number of tasks per topic. For each topic sampled from ODP, we generate 1, 3, and 5 tasks. We find that the performance of \(\pi^{\text{lr}}_{\theta}\) drops greatly when we generate more tasks per topic. This demonstrates that the diversity of tasks is important for the quality of synthetic data. (2) The granularity of topics. The sampled topics are multi-grained and we truncate those extremely specific topics to a maximum

\begin{table}
\begin{tabular}{c c} \hline \hline Model & Avg. on MTEB \\ \hline SPEED (230K synthetic data) & **63.2** \\ w/ only SFT (only \(\pi^{\text{lp}}_{\theta}\)) & 62.6 \\ w/o. DPO (\(\pi^{\text{lr}}_{\theta}+\pi^{\text{Re}}_{\theta}\)) & 62.8 \\ w/o. Data Revisor (only \(\pi^{\text{S}}_{\theta}\)) & 62.9 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Performances of ablated models on MTEB.

\begin{table}
\begin{tabular}{p{113.8pt} p{113.8pt}} \hline \hline Topic \& Task & Avg. on MTEB \\ \hline \(\pi^{\text{lr}}_{\theta}\) (1 task per topic \& truncation) & **62.6** \\ \hline _# Tasks per topic_ & & \\ \hline
3 tasks per topic & 61.6 \\
5 tasks per topic & 60.9 \\ \hline _Topic granularity_ & \\ \hline Specific topic (w/o. truncation) & 61.8 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Performances of models with different settings of task brainstorming on MTEB. For efficient test, the models have only been through SFT with 230K data.

 

depth of 4. Without truncation, those topics will produce tasks harming the generalization of SPEED.

#### 4.3.3 Junior Generator \(\pi^{\text{Ir}}_{\theta}\)

In this section, we will look into our SFT process and discuss the factors that may influence \(\pi^{\text{Ir}}_{\theta}\):

**Base LLM.** The base model that we train into our synthesis LLM is directly related to the data quality. To study this, we apply our SFT pipeline on several other base LLMs. From the results in Table 4, we can observe that all LLMs can synthesize embedding data of decent quality with our SFT pipeline. This shows the effectiveness and applicability of our designed alignment process again. Besides, \(\pi^{\text{Ir}}_{\theta}\) trained on LLaMA-3-8B achieves the best performance, which is consistent with its superior language modeling ability. This means we can easily boost the quality of synthetic data by applying SPEED on more advanced open-source LLMs.

**The generation temperature.** Temperature is a crucial hyperparameter that controls the randomness of the text generation process. We set the generation temperature of \(\pi^{\text{Ir}}_{\theta}\) in the range of [0.2, 1.5], and present the performances on MTEB in the left part of Figure 3. Due to space limitations, we only show results for five values (this policy will be followed in the subsequent displays). We can observe that the performance of \(\pi^{\text{Ir}}_{\theta}\) first increases then drops. This phenomenon indicates a trade-off: If the temperature is too low, the synthetic data will lack diversity. However, the LLM may generate data that do not follow the required structure and guidelines if the temperature is too high.

**The number of training samples.** In our training process of \(\pi^{\text{Ir}}_{\theta}\), we use GPT-4 to produce signals for knowledge distillation. This raises a question: how many samples should we use for finetuning the generator? Is it the more the better? We study this question by set the number of training samples of \(\pi^{\text{Ir}}_{\theta}\) in the range of [5K, 100K]. As shown in the middle left part of Figure 3, a small set of training samples can already train a decent generator using our SFT pipeline, which validates its effectiveness again. However, too many training samples will harm the language modeling ability of the LLM.

#### 4.3.4 Senior Generator \(\pi^{\text{Sr}}_{\theta}\)

We propose to further train the junior generator with DPO into a more powerful synthesis model \(\pi^{\text{Sr}}_{\theta}\). In this part, we will look into this process from these aspects:

**The hyperparameter \(\beta\).** When performing DPO on \(\pi^{\text{Ir}}_{\theta}\), we aim to improve its performance by directly optimizing for preference signals produced by GPT-4. \(\beta\) is the hyperparameter used to control the trade-off between aligning the model to preference signals and avoiding over-optimization that may degrade performance on the original task. To study it empirically, we set \(\beta\) in the range of [0.05, 0.3]. As presented in the middle part of Figure 3, SPEED's performance increases to an optimal value when \(\beta=0.1\) then drops. This validates the trade-off: A high \(\beta\) controls \(\pi^{\text{Sr}}_{\theta}\) to stay close to the reference model (\(\pi^{\text{Ir}}_{\theta}\)), ensuring it doesn't drift too much, while a low \(\beta\) encourages stronger adaptation to the preference signals, but at the risk of overfitting.

**The number of training samples.** Similar to the SFT process, we can raise a question: how many preference data pairs we should use to align \(\pi^{\text{Sr}}_{\theta}\)? We study this question by setting the number of training samples for \(\pi^{\text{Sr}}_{\theta}\) in the range of [5K, 15K]. From the results in the middle right part of Figure 3, we can observe that finetuning \(\pi^{\text{Ir}}_{\theta}\) using DPO needs fewer data that the SFT process. This

Figure 3: Performances of SPEED (230K data for efficient test) with different settings of the alignment pipeline. We tune our model using the validation set of NQ and MSMARCO. For consistency with the results in other tables, we present the results of our model with different hyperparameters on the whole test set of MTEB.

 

is consistent with previous studies that pairwise signals of outputs (preferences) are more informative per instance than standard supervised data. We also notice that the performance drops when we use too many preference signals. This indicates that overfitting the junior generator will harm its ability of following basic guidelines and instructions. This finding is consistent with previous studies that indicates that DPO may result in worse performance with more samples Rafailov et al. (2024).

#### 4.3.5 Data Review \(\pi_{\theta}^{\text{Re}}\)

**The number of training samples.** SPEED further enhances the quality of synthetic embedding data using a data revisor. GPT-4 evaluates the root data synthesized by \(\pi_{\theta}^{\text{Jr}}\) from multi-grained aspects and produces data revision signals to finetune \(\pi_{\theta}^{\text{Re}}\). \(\pi_{\theta}^{\text{Re}}\) revises the synthetic data generated by \(\pi_{\theta}^{\text{Sr}}\) to take a reflection at them and boost their quality. To study the influence of the number of the revision signals used for aligning the reviv, we set it in the range of [5K, 50K]. As shown in the right part of Figure 3, we can observe a similar pattern as the training of \(\pi_{\theta}^{\text{Ir}}\). This is consistent with their training protocol that they are both aligned by SFT. However, it takes fewer training data to finetune \(\pi_{\theta}^{\text{Re}}\) than \(\pi_{\theta}^{\text{Ir}}\). This is because that it is easier to revise a data sample of decent quality than synthesize one from scratch.

### RQ3. Scaling Synthetic Embedding Data

In the era of LLMs, models are often trained on billions or even trillions of data points. This raises a key question: does increasing training data always lead to better performance? Some existing works has explored this through scaling laws in areas like language modeling Kaplan et al. (2020) and dense retrieval Fang et al. (2024). However, these works primarily focus on scaling the labeled data or existing corpora.

Synthetic data, which are theoretically unlimited, remains an underexplored area for scaling laws Liu et al. (2024). This is a non-trivial problem because: (1) The distribution of synthetic data differs from that of labeled data Yu et al. (2023). (2) Generating large-scale synthetic data with blackbox LLMs to study scaling laws can be costly. With the efficient data synthesis capabilities of SPEED, we are able to generate large-scale embedding data and analyze the corresponding scaling law. Our goal is to investigate the scaling effects of synthetic embedding data in its early stages As shown in Figure 4, we observe a log-linear relationship between the embedding model's performance and the size of the synthetic data. This scaling law offers key insights for future works: (1) The log-linear trend enables researchers to predict performance improvements from synthesizing more data. (2) It guides trade-offs by showing diminishing returns--beyond a certain point, additional data yields marginal improvement, making further investment in data synthesis less valuable.

### Cost Analysis

In this section, we analyze the cost of our alignment framework, SPEED. The cost is reported from two aspects: GPT API calls (the number of invoking times) and GPT token usage. We omit the task brainstorming process, as the task descriptions are very short compared to the embedding data, and we also neglect the cost of deploying the aligned generators since they are very small.

Specifically, SPEED costs 2SK (SFT \(\pi_{\theta}^{\text{Ir}}\)) + 10K (DPO \(\pi_{\theta}^{\text{Sr}}\)) + 10K (SFT \(\pi_{\theta}^{\text{Re}}\)) = **45K** GPT API calls. As for GPT token usage it costs 10M (SFT \(\pi_{\theta}^{\text{Ir}}\)) + 12M (DPO \(\pi_{\theta}^{\text{Sr}}\)) + 10M (SFT \(\pi_{\theta}^{\text{Re}}\)) = **32M**.

For a more staightforward understanding, we compare these costs with the synthesis process of E5\({}_{\text{mistral}}\), which solely uses GPT to synthesize data. It requires 500K API calls and consumes 180M GPT tokens Wang et al. (2024). The comparison, shown in Table 5, highlights that SPEED is significantly more efficient, requiring only less than 1/10

Figure 4: Scaling laws for model performance in relation to synthetic embedding data size on MTEB.

 

of the GPT-4 API calls and about 1/6 of the tokens to align small open-source models for synthesizing large-scale data efficiently and effectively.

## 5 Conclusion

In this work, we propose a framework SPEED that aligns small models for the efficient and effective synthesis of embedding data. Through supervised finetuning, preference optimization, and self-improvement, small models can also synthesize high-quality embedding data at scale. Additionally, we comprehensively investigate how various factors within the alignment pipeline influence data quality. We reveal the scaling law of synthetic embedding data, demonstrating a log-linear relationship between the performance of the embedding model and the size of the synthetic data.

### Limitations

Our work still have several limitations that we plan to address in future works:

1. The training signals we produce may be improved in the future. Although GPT-4o is already a very powerful LLM, it still can not perfectly interpret the guidelines and requirements in our prompts. For example, some of the long hard negative documents are too close to the positive ones.
2. Our senior generator is trained by DPO. More advanced preference optimization approaches such as step-DPO will be utilized.
3. The base models used for data synthesis and embedding model can be improved. For fair comparisons to baselines, we train Mistral-7B-v0.1 as our embedding model. In future works, we plan to use more advanced LLMs to boost our model's performance.
4. We do not fit a function for the scaling law we reveal for synthetic embedding data. In future work, we will explore a power-law function that can represent the scaling relationship we find in this paper.

## Acknowledgments

This work was supported by the National Natural Science Foundation of China No. 62272467, Beijing Natural Science Foundation No. L233008, Beijing Municipal Science and Technology Project No. Z231100010323009, the fund for building worldclass universities (disciplines) of Renmin University of China. The work was partially done at the Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE.

## References

* A. Anthropic (2024)The claude 3 model family: opus, sonnet, haiku. Claude-3 Model Card. External Links: Link Cited by: SS1.
* H. Bansal, A. Hosseini, R. Agarwal, V. Q. Tran, and M. Kazemi (2024)Smaller, weaker, yet better: training llm reasoners via compute-optimal sampling. arXiv preprint arXiv:2408.16737. Cited by: SS1.
* G. Dong, K. Lu, C. Li, T. Xia, B. Yu, C. Zhou, and J. Zhou (2024)Self-play with execution feedback: improving instruction-following capabilities of large language models. CoRRabs/2406.13542. Cited by: SS1.

 

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhl Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aoob Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinue Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenoviec, Frank Zhang, Gabriel Synnaeve, Gabriele Lee, Georgia Lewis Anderson, Graeme Nanl, Gregoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannak Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel M. Kloumann, Ishan Misra, Ivan Evtimov, Jack Copet, Jaewon Lee, Jan Geffert, Jana Yranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spiska, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasudeh Alwala, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, and et al. 2024. The llama 3 herd of models. CoRRabs/2407.21783.
* Fang et al. (2024) Yan Fang, Jingtao Zhan, Qingyao Ai, Jiaxin Mao, Weihang Su, Jia Chen, and Yiqun Liu. 2024. Scaling laws for dense retrieval. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2024, Washington DC, USA, July 14-18, 2024_, pages 1339-1349. ACM.
* Gunasekar et al. (2023) Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio Cesar Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Harkiriat Singh Behl, Xin Wang, Sebastien Bubeck, Ronen Eldan, Adam Tauman Kalai, Yin Tat Lee, and Yuanhi Li. 2023. Textbooks are all you need. CoRRabs/2306.11644.
* Hu et al. (2022) Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lora: Low-rank adaptation of large language models. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net.
* Hui et al. (2024) Binguan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayniheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Kai Dang, et al. 2024. Qwen2. 5-coder technical report. _arXiv preprint arXiv:2409.12186_.
* Jeronymo et al. (2023) Vitor Jeronymo, Luiz Henrique Bonifacio, Hugo Queiroz Abornizio, Marzieh Fadaee, Roberto de Alencar Lottofo, Jakub Zwarel, and Rodrigo Frassetto Nogueira. 2023. Inpars-v2: Large language models as efficient dataset generators for information retrieval. _CoRR_, abs/2301.01820.
* Jiang et al. (2023) Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulier, Lelo Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Seco, Thibaut Lavril, Thomas Wang, Timothee Lacroix, and William El Sayed. 2023. Mistral 7b. _CoRR_, abs/2310.06825.
* Kaplan et al. (2020) Jared Kaplan, Sam McCandish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models. _CoRR_, abs/2001.08361.
* Lee et al. (2024) Jinhyuk Lee, Zhuyun Dai, Xiaoqi Ren, Blair Chen, Daniel Cer, Jeremy R. Cole, Kai Hui, Michael Boratko, Rajyi Kapadia, Wen Ding, Yi Luan, Sai Meher Karthik Duddu, Gustavo Hernandez Abrego, Weiqiang Shi, Nitthi Gupta, Aditya Kusupati, Preiteek Jain, Siddhartha Reddy Jonnalagadda, Mingwei Chang, and Iftekhar Naim. 2024. Gecko: Versatile text embeddings distilled from large language models. _CoRR_, abs/2403.20327.
* Li et al. (2024) Haoran Li, Qingxiu Dong, Zhengyang Tang, Chaojun Wang, Xingxing Zhang, Haoyang Huang, Shaohan Huang, Xiaolong Huang, Zeqiang Huang, Dongdong Zhang, Yukian Gu, Xin Cheng, Xun Wang, Si-Qing Chen, Li Dong, Wei Lu, Zhifang Sui, Benyou Wang, Wai Lam, and Furu Wei. 2024a. Synthetic data (almost) from scratch: Generalized instruction tuning for language models. _CoRR_, abs/2402.13064.
* Li et al. (2024) Yinheng Li, Rogerio Bonatti, Sara Abdali, Justin Wagle, and Kazuhito Koishiida. 2024b. Data generation using large language models for text classification: An empirical case study. _CoRR_, abs/2407.12813.
* Li et al. (2023) Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023. Towards general text embeddings with multi-stage contrastive learning. _CoRR_, abs/2308.03281.
* Liu et al. (2024) Ruibo Liu, Jerry Wei, Fangyu Liu, Chenglei Si, Yanzhe Zhang, Jinmeng Rao, Steven Zheng, Daiyi Peng, Diyi Yang, Denny Zhou, and Andrew M. Dai. 2024. Best practices and lessons learned on synthetic data for language models. _CoRR_, abs/2404.07503.
* Ma et al. (2024) Xueguang Ma, Liang Wang, Nan Yang, Furu Wei, and Jimmy Lin. 2024. Fine-tuning llama for multi-stage text retrieval. In _Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2024, Washington DC, USA, July 14-18, 2024_, pages 2421-2425. ACM.
* Mesnar et al. (2020) Thomas Mesnar, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, 

Morgane Riviere, Mihir Sanjay Kale, Juliette Love, Pouya Tafti, Leonard Hussenot, Aakanksha Chowdhery, Adam Roberts, Aditya Barua, Alex Botev, Alex Castro-Ros, Ambrose Slone, Amelie Helioud, Andrea Tacchetti, Anna Bulanova, Antonia Paterson, Beth Tsai, Bobak Shahriari, Charline Le Lan, Christopher A. Choquette-Choo, Clement Crepy, Daniel Cer, Daphne Ippolito, David Reid, Elena Buchatskaya, Eric Ni, Eric Noland, Geng Yan, George Tucker, George-Cristian Muraru, Grigory Rozddestvenskiy, Henryk Michalewski, Ian Tenney, Ivan Grishchenko, Jacob Austin, James Keeling, Jane Labanowski, Jean-Baptiste Lespiau, Jeff Stanway, Jenny Brennan, Jeremy Chen, Johan Ferret, Justin Chiu, and et al. 2024. Gemma: Open models based on gemini research and technology. _CoRR_, abs/2403.08295.
* Meta (2024)Meta. 2024. Introducing meta llama 3: The most capable openly available llvm to date. [https://ai.meta.com/blog/meta-llama-3/](https://ai.meta.com/blog/meta-llama-3/).
* Muennighoff et al. (2023) Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023. MTEB: massive text embedding benchmark. In _Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2023, Dubrovnik, Croatia, May 2-6, 2023_, pages 2006-2029. Association for Computational Linguistics.
* Ni et al. (2022) Jianmo Ni, Chen Qu, Jing Lu, Zhuytun Dai, Gustavo Hernandez Abrego, Ji Ma, Vincent Y. Zhao, Yi Luan, Keith B. Hall, Ming-Wei Chang, and Yinfei Yang. 2022. Large dual encoders are generalizable retrievers. In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022_, pages 9844-9855. Association for Computational Linguistics.
* Rosqueira et al. (2019) Rodrigo Frassetto Nogueira, Wei Yang, Jimmy Lin, and Kyunghyun Cho. 2019. Document expansion by query prediction. _CoRR_, abs/1904.08375.
* OpenAI (2023)OpenAI. 2023. GPT-4 technical report. _CoRR_, abs/2303.08774.
* Patwa et al. (2024) Parth Patwa, Simone Filice, Zhiyu Chen, Giuseppe Castellucci, Oleg Rokhlenko, and Shervin Malmasi. 2024. Enhancing low-resource llmns classification with PEFT and synthetic data. In _Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC/COLING 2024, 20-25 May, 2024, Torino, Italy_, pages 6017-6023. ELRA and ICCL.
* Qwen (2024) Team Qwen. 2024. Qwen2.5: A party of foundation models.
* Rafailov et al. (2024) Rafael Rafailov, Yaswanth Chitepu, Ryan Park, Harshit Sikchi, Joey Hejna, W. Bradley Knox, Chelsea Finn, and Scott Niekum. 2024. Scaling laws for reward model overoptimization in direct alignment algorithms. _CoRR_, abs/2406.02900.
* 16, 2023_.
* Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019_, pages 3980-3990. Association for Computational Linguistics.
* Schulman et al. (2017) John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. _CoRR_, abs/1707.06347.
* Sturais et al. (2024) Saba Sturua, Isabelle Mohr, Mohammad Kalin Akram, Michael Gunther, Bo Wang, Markus Krimmel, Feng Wang, Georgios Mastrapas, Andreas Koukounas, Nan Wang, et al. 2024. jina-embeddings-v3: Multilingual embeddings with task lora. _arXiv preprint arXiv:2409.10173_.
* Thirukavalluru et al. (2024) Raghuveer Thirukavalluru, Xiaolan Wang, Jun Chen, Shuyang Li, Jie Lei, Rong Jin, and Bhuwan Dingra. 2024. Sumcse: Summary as a transformation for contrastive learning. In _Findings of the Association for Computational Linguistics: NAACL 2024, Mexico City, Mexico, June 16-21, 2024_, pages 3577-3588. Association for Computational Linguistics.
* Touvron et al. (2020) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esibbu, Jude Fernandes, Jeremy Fu, Wenyiu Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcia Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kolumann, Artem Korenev, Punit Singh Kouza, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Teodor Mihaloylov, Pushkar Mishra, Igor Molbybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Runeta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialone. 2023. Llama 2: Open foundation and fine-tuned chat models. _CoRR_, abs/2307.09288.
* Viswanathan et al. (2023) Vijay Viswanathan, Chenyang Zhao, Amanda Bertsch, Tongshuang Wu, and Graham Neubig. 2023. Prompt2model: Generating deployable models from natural language instructions. In _Proceedings of the 

2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023 - System Demonstrations, Singapore, December 6-10, 2023_, pages 413-421. Association for Computational Linguistics.
* Wang et al. (2022) Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022. Text embeddings by weakly-supervised contrastive pre-training. _CoRR_, abs/2212.03533.
* Wang et al. (2024) Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. Improving text embeddings with large language models. In _Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangok, Thailand, August 11-16, 2024_, pages 11897-11916. Association for Computational Linguistics.
* Wang et al. (2023) Liang Wang, Nan Yang, and Furu Wei. 2023. Query2doc: Query expansion with large language models. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023_, pages 9414-9423. Association for Computational Linguistics.
* Xiao et al. (2024) Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, and Jian-Yun Nie. 2024. C-pack: Packed resources for general chinese embeddings. In _Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2024, Washington DC, USA, July 14-18, 2024_, pages 641-649. ACM.
* Yu et al. (2024) Runsheng Yu, Yong Wang, Xiaoqi Jiao, Youzhi Zhang, and James T. Kwok. 2024. Direct alignment of language models via quality-aware self-refinement. _CoRR_, abs/2405.21040.
* 16, 2023_.
* Zhang et al. (2023) Junlei Zhang, Zhenzhong Lan, and Junxian He. 2023. Contrastive learning of sentence embeddings from scratch. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023_, pages 3916-3932. Association for Computational Linguistics.
* Zhou et al. (2024a) Jiaming Zhou, Abbas Ghaddar, Ge Zhang, Liheng Ma, Yaochen Hu, Soumyasundar Pal, Mark Coates, Bin Wang, Yingxue Zhang, and Jianye Hao. 2024a. Enhancing logical reasoning in large language models through graph-based synthetic data. _arXiv preprint arXiv:2409.12437_.
* Zhou et al. (2024b) Kun Zhou, Beichen Zhang, Jiapeng Wang, Zhipeng Chen, Wayne Xin Zhao, Jing Sha, Zhichao Sheng, Shijin Wang, and Ji-Rong Wen. 2024b. jiaZhang3.0: Efficiently improving mathematical reasoning by training small data synthesis models. _CoRR_, abs/2405.14365.

 



## Appendix A Details about Synthetic Data

In this section, we will look into the detailed information and statistics of generated synthetic embedding data. The statistics is presented in Table 6. We first generates a raw synthetic dataset of 1.15M examples following the data proportion in Section 4.1. And after MinHash deduplication, there are 920,415 data left in total.

### Implementation Details

In this part, we delve into the details about the implementation of SPEED. Specifically, we finetune LLaMA-3-8B as data synthesis models and Mistral-7B-v0.1 as our embedding model. For the SFT process of \(\pi_{\theta}^{\text{Tr}}\), the learning rate is 1e-4 and the batch size is 16. As for the DPO process of \(\pi_{\theta}^{\text{Sr}}\), the learning rate is 1e-5, beta \(\beta\) is set as 0.1, and the batch size is 16. For the SFT process of \(\pi_{\theta}^{\text{Re}}\), the learning rate is 5e-6 and the batch size is 24.

For the data generation, we set the temperature as 1.0 for all data synthesis except 0.0 for producing the preference signal. The top_p is set as 1.0.

For the training of our embedding model, we use LoRA with rank 16 and DeepSpeed ZeRO-3. We set the batch size as 1,536 using 16 40G A100 and fp16. For the training data, we use a combination of synthetic data and a collection of 13 public datasets. These labeled datasets used for finetuning are the same as those in E5mistral.

For the instructions we used for the training and evaluation datasets (MTEB), please refer to the original paper of E5mistral Wang et al. (2024).

## Appendix C Prompts

The prompts we used in our work can be categorized into two kinds: prompts used for generating synthetic data and aligning data generators.

### Data Generation

Since our work focuses on the alignment of small models for synthesizing large-scale embedding data, we reuse most of the data generation prompts and data structures of E5mistral Wang et al. (2024). For task brainstorming, we adjust those prompts to fit the sampled topic by appending "for the topic: {topic}" after each "Brainstorm a list of potentially useful xxx tasks". For the synthesis of STS data we change its prompt to fit the sampled topics as follows:

```
Write a {sentence, phrase, passage} triple for the topic: {topic} with varying semantic similarity scores in JSON format. The semantic similarity score ranges from 1 to 5, with 1 denotes least similar and 5 denotes most similar.
```

Please adhere to the following guidelines:

- The keys in JSON are "S1", "S2", and "S3", the values are all strings in English, do not add any other keys.

- There should be some word overlaps between all three [sentence, phrase, passage]s.

- The similarity score between S1 and S2 should be [4, 4.5, 5].

- The similarity score between S1 and S3 should be [2.5, 3, 3.5].

- The [sentence, phrase, passage]s require {elementary school, high school, college} level education to understand and should be diverse in terms of topic and length.

Your output must always be a JSON object only with three keys "S1", "S2" and "S3", do not explain yourself or output anything else. Be creative! ```
Listing 10: Training data generation

### Generator Alignment

In this part, we will shed light on the prompts we use to generate the signals for knowledge distillation. For the SFT of \(\pi_{\theta}^{\text{Tr}}\), the training data are sampled from the synthesis of Mistralgpt-4o. For the DPO of \(\pi_{\theta}^{\text{Sr}}\), we prompt GPT-4 to produce preference data as:
``` RequireGeneratingReferenceData ```

A language model has been given a prompt: {data prompt} The output list of it is: {data list} Your mission is to judge which data this language model generate fits the prompt most and which fits worst, and explain your judgment.

The JSON object you output must contain the following keys:

- "reason": a string, the reason of your judgment.

- "best": a number, the index of the generated data that fits prompt the most (indice start from 0).

- "worst": a number, the index of the generated data that fits prompt the worst.

Your output must always be a JSON object only, do not explain yourself or output anything else.

\begin{table}
\begin{tabular}{l l} \hline \hline Synthetic Task Type & \# Examples \\ \hline Classification (long-short match) & 245,947 \\ Semantic textual similarity (STS) & 294,388 \\ Retrieval (short-long match) & 303,424 \\ Text matching (short-short match) & 39,954 \\ Text matching (long-long match) & 36,702 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Statistics of the synthetic data (after MinHash) used for finetuning the embedding model.



With this prompt, we can obtain a best and worst data of the data list evaluated by GPT-4. Then, we can get preference data pairs based on the best and worst data.

For the SFT of \(\pi_{\theta}^{\text{Re}}\), we use GPT-4 to evaluate the quality of synthetic data from multiple aspects and produce the revised data for training signals.

```
Promote Generating Keyies Data
``` A language model has been given a prompt: [data prompt] The output generated by the model is: [data example] Your task is to evaluate the generated output based on the following criteria: 1. Relevance: Assess whether the output directly addresses the task described in the prompt. 2. Completeness: Check if the output includes all necessary elements as specified in the prompt. 3. Accuracy: Verify if the output is factually correct and adheres to the guidelines provided in the prompt. For each criterion, provide a brief explanation supporting your evaluation. Then, provide a revised version of the output. Your response should be a valid JSON object containing the following keys: - "reason": A string providing the reason for your judgment. - "revision": A string with the revised version of the output based on your evaluation and the prompt. Ensure your output is always a valid JSON object, formatted as a JSON string. Do not include any additional explanations or information. ```

## Appendix D Data Examples

### Topics

In order to mitigate the hallucination and introduce more diversity to LLMs, we propose to sample multi-grained topics from ODP. Some examples of the sampled raw topics are presented in Table 7. Some of these topics are wide categories (_e.g._, "Arts"), which will make LLM generate more abstract data. And some of these topics are detailed and specific, which may cause the synthetic data to include some noisy information. Therefore, we propose to truncate the topics with depth more than four by discarding their middle information. For example, for "Arts/Movies/Titles/3/36_Hours_-_1964/Cast_and_Crew", we will only keep "Arts/Movies/36_Hours_-_1964/Cast_and_Crew". By this, we can keep its main category and some details without introducing too much noise.

### Alignment Data

In this section, we present data used for aligning \(\pi_{\theta}^{\text{Sr}}\) and \(\pi_{\theta}^{\text{Re}}\) in Figure 5 and Figure 6, respectively.

 

Society/Crime/Criminals/Outlaws/Bonnie_and_Cyde Sports/Baseball/People/Players/E/Estes_Shawn Arts/Performing_Arts/Dance/Folk/Square_Dancing/Clubs/United_States/Oregon Regional/Europe/United_Kingdom/England/County_Durham/Darlington/Business_and_Economy/Shopping Business/Food_and_Related_Products/Produce/Frozen Arts/Performing_Arts/Acting/Actors_and_Actresses/V/Vaughn,_Robert/Movies Sports/Hockey/Ice_Hockey/Players Science/Biology/Flora_and_Fauna/Animalia/Arthropoda/Insecta/Diptera/Rhagionidae Games/Video_Games/Action/S/Snake_Games/Downloads/Free Regional/Asia/South_Korea/Jeonnam/Yeonggwang Computers/CAD_and_CAM/Electronic_Design_Automation Regional/Europe/France/Regions/Languedoc-Roussillon/Lozere Arts/Movies/Tiltes/3/36_Hours_\({}_{-}1964\)/Cast_and_Crew Science/Technology/Structural_Engineering/Bridge/History/People/Boedy_Daniel Regional/Middle_East/Cyprus/Limassol_District/Travel_and_Tourism/Accommodation Recreation/Food/Drink/Wine/Events/United_States/Texas Health/Medicine/Medical_Specialties/Ophthalmology/Refractive_Correction/LASIK Arts Society/Isuses/Business/Allegedly_Unethical_Firms/Halliburton/Opposing_Views

\begin{table} \end{table}
Table 7: Examples of topics sampled from ODP without truncation.

 



### _Input_

Figure 5: An example to show the generated preference signals for DPO. A data prompt and a data list are fed into GPT-4 and it evaluates the best and worst data according to the requirements of prompt. The data prompt template is from E5\({}_{\text{mistral}}\)[23].





### _Input_

Figure 6: An example to show the generated revision signals for SFT the data revisor. A data prompt and a data list are fed into GPT-4 and it improves the data based on the given guidelines in the prompt.





## Appendix A

Figure 8: An example of the synthetic retrieval data.

Figure 10: An example of the synthetic short-short matching data.

Figure 9: An example of the synthetic semantic textual similarity data.





## Chapter 10 Conclusion

In this thesis, we have presented a new method for constructing quantum computing in the field of information technology. We are able to perform an efficient algorithm that can be used as a tool for quantum computation. The idea is that it allows us to use a classical computer to efficiently simulate and efficiently simulate the data. This approach has been applied to quantum computers, where the computational complexity is limited by the number of qubits and the number of qubits. In order to achieve the speed of light propagation, we need to implement quantum algorithms with a large number of qubits.

The quantum computing paradigm is based on the fact that quantum computing is not only possible if the input state is sufficiently high. However, quantum computing is still required to simulate the data, which is impossible to simulate the data. For example, quantum computing is a quantum computing paradigm that can be used to simulate the data.

Figure 11: An example of the synthetic long-long matching data.





\begin{table}
\begin{tabular}{l c c} \hline \hline Dataset & w/ synthetic only & full data \\ \hline BIOSSES & 85.4 & 87.1 \\ SICK-R & 79.6 & 82.5 \\ STS12 & 77.7 & 80.2 \\ STS13 & 87.9 & 89.9 \\ STS14 & 81.8 & 86.2 \\ STS15 & 87.7 & 91.2 \\ STS16 & 85.8 & 88.2 \\ STS17 & 86.4 & 91.9 \\ STS22 & 69.2 & 68.3 \\ STSBenchmark & 84.7 & 89.2 \\ SummEval & 31.7 & 31.1 \\ SprintDuplicateQuestions & 95.8 & 95.5 \\ TwitterSemEval2015 & 77.5 & 81.7 \\ TwitterURLCorpus & 85.7 & 87.4 \\ AmazonCounterfactualClassification & 78.2 & 76.7 \\ AmazonPolarityClassification & 95.7 & 96.2 \\ AmazonReviewsClassification & 56.7 & 56.3 \\ Banking7T Classification & 87.7 & 88.6 \\ EmotionClassification & 52.3 & 51.0 \\ ImdbClassification & 93.9 & 94.9 \\ MassiveIntentClassification & 79.0 & 80.2 \\ MassiveSecenarioClassification & 81.5 & 82.3 \\ MTOPDomainClassification & 95.4 & 95.9 \\ MTOPInntemClassification & 86.0 & 87.1 \\ ToxicConversationsClassification & 68.9 & 68.4 \\ Tweet SentimentExtractionClassification & 64.3 & 63.8 \\ AskUbuntuDupQuestons & 65.5 & 67.2 \\ MindSmallReranking & 33.0 & 33.4 \\ SciDocsRR & 86.7 & 87.3 \\ StackOverflowDupQuestions & 53.9 & 55.2 \\ ArxivClusteringP2P & 50.7 & 51.1 \\ ArxivClusteringS2S & 46.4 & 47.0 \\ BioixvClusteringP2P & 42.6 & 42.0 \\ BiorxivClusteringS2S & 39.7 & 39.6 \\ MedrxivClusteringP2P & 35.1 & 37.0 \\ MedrxivClusteringS2S & 35.7 & 36.3 \\ RedditClustering & 56.1 & 57.9 \\ RedditClusteringP2P & 63.9 & 65.3 \\ StackExchangeClustering & 70.0 & 71.6 \\ StackExchangeClusteringP2P & 39.9 & 39.0 \\ TwentyNewgsroupSLustering & 54.9 & 55.2 \\ ArguaAna & 40.5 & 59.3 \\ ClimateFEVER & 22.2 & 37.8 \\ CQADupstackAndroidRetrieval & 41.7 & 41.6 \\ DBPedia & 43.3 & 49.7 \\ FEVER & 77.7 & 88.5 \\ FiQA2018 & 39.5 & 56.1 \\ HotpotQA & 55.6 & 75.2 \\ MSMARCO & 25.9 & 42.5 \\ NFcorpus & 36.5 & 38.7 \\ NQ & 53.3 & 61.7 \\ QuoraRetrieval & 84.6 & 89.3 \\ SCIDOCS & 21.0 & 16.6 \\ SeiFact & 71.9 & 77.2 \\ Touche2020 & 23.8 & 25.8 \\ TRECCOViD & 83.9 & 87.4 \\ \hline Average & 63.4 & **66.5** \\ \hline \hline \end{tabular}
\end{table}
Table 8: Detailed results of SPEED in the zero-shot setting and full-data setting on each dataset of MTEB. The details about the evaluation metrics and dataset statistics can be found in its original paper Muennighoff et al. (2023)