Generating Synthetic Datasets for Few-shot Prompt Tuning
Authors: Xu Guo
Zilin Du
Boyang Li
Chunyan Miao
Venue: COLM (2024)
Keywords: Prompt Tuning, Few-shot Learning, Synthetic Data Generation
TL;DR: Using one LLM to generate synthetic data, which is used to train soft prompts for another LLM to perform few-shot learning tasks.

Abstract: A major limitation of prompt tuning is its dependence on large labeled training datasets. Under few-shot learning settings, prompt tuning lags far behind full-model fine-tuning, limiting its scope of application. In this paper, we leverage the powerful LLMs to synthesize task-specific labeled data for training the soft prompts. We first introduce a distribution-aligned weighted generator tuning (DawGen) method to encourage generating in-distribution data that aligns with the few-shot real data. Then, we train soft prompts on both synthetic and real datasets using a gradient surgery approach, which eliminates the conflicting gradients from different data sources. Experiments on seven sentence-pair classification datasets demonstrate the effectiveness of our proposed method for boosting prompt tuning in few-shot learning settings. Results on QQP, MRPC, and SICK datasets are even comparable to the performance of transfer learning from large real-world datasets, showing the promise of synthetic data as an alternative for enhancing soft prompt tuning.

Published as a conference paper at COLM 2024
Generating Synthetic Datasets for Few-shot Prompt Tuning
Xu Guo, Zilin Du, Boyang Li & Chunyan Miao
Nanyang Technological University, Singapore
Abstract
A major limitation of prompt tuning is its dependence on large labeled training datasets. Under few-shot learning settings, prompt tuning lags far behind full-model fine-tuning, limiting its scope of application. In this paper, we leverage the powerful LLMs to synthesize task-specific labeled data for training the soft prompts. We first introduce a distribution-aligned weighted generator tuning (DawGen) method encourage generating in-distribution data that aligns with the few-shot real data. Then, we train soft prompts on both synthetic and real datasets using gradient surgery approach, which eliminates the conflicting gradients from different data sources. Experiments on seven sentence-pair classification datasets demonstrate the effectiveness of our proposed method for boosting prompt tuning in few-shot learning settings. Results on QQP, MRPC, and SICK datasets even comparable to the performance of transfer learning from large real-world datasets, showing the promise of synthetic data as an alternative for enhancing soft prompt tuning.
1 Introduction
As Large Language Models (LLMs) increase in size, adapting them to downstream tasks by fine-tuning (FT) separate copy for each task becomes unfeasible. Prompt Tuning (Lester et al., 2021) (PT) emerges as a solution to this challenge by freezing the LLM and instead training a set of soft prompts pre-pended to the input data in an end-to-end manner. Compared with other parameter-efficient learning methods such as adapter tuning (Houlsby et al., 2019) and LoRA (Hu et al., 2022), PT makes no changes to the model architecture and can be applied to a frozen model with a static computational graph, enabling fast and flexible deployment. On a wide range of downstream tasks, PT has shown comparable performance as FT (Lester et al., 2021; Liu et al., 2022). However, recent studies indicate that PT requires sufficient labeled training data to achieve competitive performance as FT, yet in few-shot settings, IT significantly underperforms FT (Gu et al., 2022; Guo et al., 2022).
To boost PT in few-shot learning tasks, previous methods mainly focus on finding better initialization for soft prompts (Gu et al., 2022; Guo et al., 2022). This is achieved by pre- training the soft prompts on a large-scale real-world corpus such as WpenWebText (Gokaslan & Cohen, 2019a) or similar source-domain datasets (Gu et al., 2022; Vu et al., 2022; Guo et al., 2022). However, these approaches bear a common limitation: the dependence on large real-world datasets. On the one hand, online text corpora often exhibit substantial domain discrepancies varying across different downstream tasks. On the other hand, source-domain datasets are often not readily available, particularly for low-resource and emerging domains.
Recently, there has been a growing interest in generating training data with LLMs (Ye et al., 2022a; Meng et al., 2022; 2023; Yu et al., 2024). This paper extends this line of research to tackle the limitation of prompt tuning in few-shot learning settings, aiming to bypass the need for large-scale labeled training data. Specifically, we employ source LLM to generate synthetic training set for the task at hand, which can be treated as a medium for carrying the pre-learned knowledge from a source LLM, train soft prompts for target LLM to achieve enhanced few-shot learning performance.
* Please contact Xu Guo (xu009@e.ntu.eddu.g) for future questions.
1

---

Published as a conference paper at COLM 2024
Step 1: traing soft prompt Q for a frozen LLM ge on the few-shot training set D reat-- Step 2: apply gp and Q to gen- erate synthetic labeled train- ing set Doyn- Step With Dea! and Dayn, train soft prompt P for a discriminative LLM fo to perform downstream tasks, while tackling issues like gra- dient conflicts.
Model
Prompt Q
+
Generative LLM &Φ
Few-shot Data
Synthetic Labeled Data
Data
Dreal
D Dsyn
Model
Prompt P
+
Discriminative LLM fo
Task
Figure 1: A schematic overview of the framework.
We show an overview diagram of our framework in Figure 1. We divide the learning procedure into three steps. First, to encourage the source LLM gp to generate in-domain data, we train label-conditional soft prompts Q using few real-world samples, Dreat, to adapt the frozen & to the task domain. However, given the limited number of real-world samples, Q can easily overfit shortcut tokens (Du et al., 2021; Tang et al., 2023) and mislead 8ø to generate texts that are not relevant to the task label, resulting in a poor distribution for
Weighted GENerator tuning (DawGen)) which encourages the generated texts under the same class to be semantically close while those under different classes to be semantically distant. Second, generator 80.Q to generate synthetic training set, Dayn, for the task at hand. However, directly combining synthetic data with the few-shot real data for training can lead to an interference effect, due to their potential distribution gap. To better exploit the synthetic data while making the best use of few-shot real data, we employ gradient surgery (Yu et al., 2020) to modify the gradients computed on the synthetic data by subtracting the components that oppose the direction of gradients from the real data. The trained soft prompts followed by an input example are used prompt the target LLM for label prediction. They can also be combined with hard prompts to confer dual benefits (Gao et al., 2021; Gu et al., 2022).
D Dsyn. Hence, we introduce Distribution-Aligan
apply the adapted
Extensive experiments on seven sentence-pair classification tasks and two LLM backbones demonstrate the effectiveness of our proposed framework for enhancing PT in few-shot settings. Notably, PT with 102K parameters outperforms FT with 770M parameters by large margin in few-shot settings and can even achieve comparable performance to transfer learning using extensive real-world datasets on QQP, MRPC, and SICK.
2 Related Work
2.1 Few-shot Learning with Pre-trained Language Models
Fine-tuning pre-trained language models has been the standard practice in few-shot learning, where language model and a task-specific head are tuned together for given task (Zhang et al., 2021; Gao al., 2021; Liu et al., 2022; Zhang al., 2022). However, fine-tuning the entire model on a few training samples (e.g., 16 samples per class) often leads to overfitting. One possible remedy is manually craft hard prompts, consisting of natural language instructions and demonstrations (Brown et al., 2020; Mishra et al., 2022) for LLMs to perform in-context learning without updating any model parameters. Nevertheless, their effectiveness greatly relies on the skills of the prompt engineer and the prompts they write.
Instead of manually crafted hard prompts, prompt tuning (Lester et al., 2021; Zhang et al., 2022) learns soft prompt vectors from training data and PT on T5-large can outperform the results of manual prompting on GPT-3. Still, PT performance greatly depends on the availability of substantial training data (Gu et al., 2022; Guo et al., 2022). In few-shot settings, PT significantly underperforms FT. To address this limitation, researchers have studied utilizing online corpora to pre-train soft prompts under self-supervised learning objectives (Gu et al., 2022), such as next sentence prediction (Devlin et al., 2019), or applying
thttps://gitteee https////giuuuu..ccgguuuuu/////s ht transfer//tre//mmin/nuuuu
2

---

Published as a conference paper at COLM 2024
transfer learning from datasets in similar domains (Guo et al., 2022). This paper provides an alternative solution by employing more powerful LLM to generate synthetic training data for training soft prompts, thus circumventing the demand for extensive real-world data.
2.2 LLMs as Task-specific Training Data Generators
Early efforts in generating synthetic data with language models were to augment the existing dataset with generic texts (Kumar et al., 2020; Puri et al., 2020; Anaby-Tavor et al., 2020). E.g. Puri et al. (2020) use GPT-2 to generate synthetic corpus to substitute the Wikipedia corpus to improve the performance of QA tasks. These texts are not tailored to any downstream task and therefore can not be used to train task-specific models. As LLMs continue to grow in size, recent works have shifted towards creating a new paradigm for playing with LLMs, Le., distilling task-specific training data directly from frozen LLM to enhance downstream tasks (Schick & Schütze, 2021b; Ye et al., 2022a; Meng et al., 2022; Gao et al., 2023; Meng et al., 2023; Yu et al., 2024). These synthetic datasets are then used to boost downstream models such as DistillBERT and RoBERTa (Meng et al., 2022; 2023).
However, when prompted with simple label-conditional natural language prompt, the LLM generator can easily forget label information when generating long sequences (Li et al., 2022; Zhong et al., 2024) and therefore may generate samples that not associated with the given labels. Moreover, the LLM generator, when frozen for inference, tends to generate texts that follow its pertaining data distribution, which often exhibits a domain gap with the task at hand (Guo & Yu, 2022). This paper provides a distribution-aligned weighted generator tuning method mitigate these issues.
2.3 Learning with Synthetic Training Data
The existence of low-quality samples can be detrimental to model training. Synthetic datasets are often generated at scale, which is inevitable to contain low-quality data (Gao et al., 2023). Recent works on synthetic text generation with LLMs (Ye et al., 2022a; Meng et al., 2022; 2023) adopt various training strategies to exploit the synthetic training data. For example, ZeroGen (Ye et al., 2022a; Gao et al., 2022) employs noise-robust loss function (Ghosh et al., 2017) for learning with synthetic training data. FewGen (Meng et al., 2023) adopts label smoothing and temporal ensembling (Laine & Aila, 2016) to degrade the confidence level of model prediction during training. ProGen (Ye et al., 2022b) incorporates a quality estimation module to select the synthetic dataset. These methods are primarily employed to work with synthetic data; however, in our paper, where we use a few real samples for supervision, they cannot address the disparity between real and synthetic data. Hence, we propose to use the gradient surgery method (Yu et al., 2020) to directly alter the conflicting gradients on the synthetic data, thereby enhancing the learning performance.
Synthesizing Training Data for Prompt Tuning
3.1 Preliminaries
Prompt Tuning. Lester et al. (2021) converts all downstream tasks into a text-to-text gen- eration format (Raffel et al., 2020) in order to reduce the gap between pre-training and downstream tasks. Taking sentence-pair classification as an example, given labeled train- ing example (X,y) E D N, where X [S1, S2] represents sentence pair and y E y denotes a class label. For example, in paraphrase detection, the label space ) may include two labels, yes and no, indicating if two sentences are paraphrases. In order to fully utilize the pretrained model, we devise task-specific natural language prompt H and template T-)) that reformat the original task as cloze-style task. T(X) (H, X, [MSK]]. One example for paraphase detection is " "Are (Si) and (S2) equivalent? [MASK].. An LLM, fe. predicts the label y at the position of the [MASK] token.
Prompt tuning is done by prepending tokens with trainable prompt embeddings, P Read to the template T. Throughout the training, f remains unchanged and P is optimized
3

---

Published as a conference paper at COLM 2024
by minimizing the cross-entropy loss on the training set D:
(P) -x((xx)) log Pro (MA([ -y](P
(1)
Synthetic Text Generation. Taking sentence-pair classification as an example. Given the label space y = (Yi) we compose label-conditional prompt for each Y, using the template T()Y) {H, (H,Y, X3, e.g., "Sentence 1 and sentence 2 are equivalent. Sentence
(51). Sentence 2: ". Then, an autoregressive LLM, 8g, takes Tz(Yi) as the context and generates subsequent sequence of tokens X1:K that maximize the joint likelihood:
K [x
The decoding process stops when the end-of-sentence token is predicted or the maximum sequence length, K, is reached. The generated sentence for S2 is expected to satisfy the relationship Y1 with Sj. To encourage generating diverse Xsya for the same label Y,, we employ a stochastic decoding algorithm (e.g,, top-k and nucleus sampling). The generated sentence pairs and the given label y/ establish a synthetic dataset Dsyn { (Xsym ysym))
3.2 Distribution-Aligned Weighted Generator Tuning
For every downstream task, we use the few-shot real dataset, 'D Dreai, to perform domain adaptation for the generator 8 to improve the quality of the synthetic dataset D Dayn-
Earcieter-Effiiient Generator Tuning. Recent LLMs for text generation often have billions of parameters. As such, tuning the entire model & for domain adaptation is impractical. Paraieter-efffiiennt methods, such prompt tuning (Lester et al., 2021) and prefix tuning (Li Liang, 2021), arise as an alternative to full-model fine-tuning by pre-pending few external prompt embeddings, Q to the input (or every transformer layer's output as done by prefix tuning), and training Q on the domain-specific data while keeping & unchanged.
For a downstream task with classes, we can train one soft prompt Qi E R for each class label V/. The training objective for tuning Qi in this setting is the standard language modeling objective. The generator parameters & are frozen and only the soft prompt Qi is optimized on the real training set D real:
Cgen(Qi)
(Q:)
E
E
LogPt((x))xee Qt).
(3)
|Dreat
X=Dnally-Yi x
Weighted Generator Tuning. The above language modeling objective treats all tokens equally. To encourage the data generator network to generate label-discriminative texts, FewGen (Meng et al., 2023) propose to train weight net, Pw Rd + R, which learns to assign higher weights to those generated tokens that are more likely to discriminate the ground-truth label Y, from other labels Yp E y by minimizing the weighted generation loss,
Gw (Qi) = -Exe Y-Y, Exyex W,- log (x Qt).
Here, Q and W are optimized under the bi-level optimization framework. That is, first optimizing the loss LwCen produces a function Q of W: Q(W), which is then applied second time to optimize the weight net, W, by minimizing the following loss:
Pro (x
(W))
Laisc (W) -Ex
Σr Pr
xx Xzji Qr (W))
By optimizing and W iteratively, the generator & learns to generate tokens that are more related to the given label than other labels.
4

---

Published as a conference paper at COLM 2024
Algorithm 1: Generator Tuning.
Algorithm 2: Prompt Tuning.
Data: Few-shot real dataset reall- Initialize: Prompts Q Pre-trained LLM
Data: Few-shot Dreat and synthetic D Dayn- Initialize: Prompts P, Pre-trained LLM fo. initialize f 0;
80.
initialize 1
while < + do
while < T do
[ = 1;
1+ 1;
Beet Bayn < Sample a batch from Dreat Dayni
1 < Sample a batch from Deea
Compute gradients ren! aCg)
on Brett
aP
Q'(W') + Take a gradient descent
Compute gradients 8syn if 5 8real < then
(P)
on Bapii
step on B with DawwGn (Q'; W*);
W++1 Take gradient descent step on B with disee (Q'(W))); ));
syn dayn Projs (dsyn);
end
Q Take gradient descent step on B with Daw (Q'; W(+1);
10 = 8real + E E-ggnn
11
+- P - 11 8; end
end
Output: Generator 8p.Q
Output: Soft prompt P
Distribution-Aligned Regularization. However, given the few-shot training set, it is easy for the weight net to overfit shortcut tokens, which are not robust tokens for the given task. For example, the token "not" can discriminate the positive sentence "It a good movie" from the negative sentence "It is not good movie", but is obviously not generalizable label-discriminative token. Hence, enforcing the generator to solely rely on the weights may lead to generating sentences that are irrelevant to the given label. We propose to regularize the generator tuning objective by adding sentence-level distribution constraint to encourage the generated sentence align with the in-distribution data:
Dawwee (Q) E[g[tLt) (Qi: + (dist());
(6)
where
dist (Q) (x max D(W Zi W Z,a) + D(W Zi, W. (,,r)))
(7)
Here, W E IIIK indicates the weights for a sequence of K tokens. Zμ = 80.0 Xi), Zi, E k denotes the last-layer hidden states output from the generator 8-.Q(-) for i-th instance X; of class Y,, and Z,, represents that from a different class Yp. D(.,)) measures the cosine similarity between two vectors. By minimizing ( we encourage the generated texts to stay close the ones in the same class while being pulled away from those that belong to other classes. We present the whole procedures in Algorithm 1.
3.3 Training Soft Prompts with Synthetic Data Augmentation
Despite the domain adaptation procedure employed, the resulting synthetic training set can inevitably contain low-quality data. Training soft prompts with naive combination of synthetic and few-shot real data can result in the optimization process being dominated by gradients from the synthetic data. Hence, we up-sample the few-shot data by pairing each batch of synthetic data with corresponding batch from the few-shot real data. Then, we employ gradient surgery to these paired batches to resolve conflicting gradients from different data sources in prompt tuning.
Gradient Surgery. It was first proposed to de-conflict gradients in a multi-task learning setting, where a model 0 is trained on set of M tasks (Yu et al., 2020). Let 8; ac,
denote the gradients of i-th task loss (0) with respect to the model 0. V&; E { 8; is 8;
iteratively altered across all the other tasks by subtracting the component | 8, which is
its projection to the plane of j-th task' gradient 8, where j # i. This step is applied when 5; 5, which indicates the two tasks have interference in driving the optimization path.
5

---

Published as a conference paper at COLM 2024
In this paper, treat the gradients from real data, Sreal, as the positive gradients for the task and always project the gradients the synthetic data, dsyn, to the direction of real
dayn
Greal
Ösyn Ereal
Syeal
proje (8syn)
real
(8)
8eail
Oecl
1 8real
Sreul II'
6rei where |ل
the normal plane of the gradients &reat from real data, and Seeal the [
magnitude of the projection of 8syn onto this normal plane. If 8syn 8real
< 0, then the
projected gradients will be dropped and the gradients of synthetic data will be modified as:
Syn =
Ssyn
Proje,./ (Dsyn).
(9)
By removing the conflicting gradients of the synthetic data, we train soft prompts using the loss function in Equation and update the weights with gradient descent approach:
P P (5rraal te +e.gyn))
(10)
where is the learning rate and e is a factor for controlling the strength of synthetic knowledge guidance, which was studied in similar work in computer vision (Zhu al., 2023). The whole algorithm is presented in Algorithm 2.
4 Experiments
4.1 Datasets, Metrics, and Settings
We conduct evaluations on seven sentence-pair classification datasets in two tasks. In the paraphrase detection task, we use MRPC (Dolan & Brockett, 2005) and QQP?. In the natural language inference task, we use MNLI (Williams al., 2018), SNLI (Bowman et al., 2015), QNLI (Rajpurkar et al., 2016), RTE (Dagan et al., 2005), and SICK (Marelli et al., 2014). We follow LM-BFF (Gao et al., 2021) to prepare the few-shot learning setting: both Puain and Deey contain 16 samples per class, which are sampled from the original training set using 5 random seeds, and the original development set is used as the test set. We adopt Accuracy for all the classification tasks and report the average test accuracy over 5 seeds. We compare methods using the average performance across the seven datasets. More details about the datasets, models, and training settings can be found in the Appendix.
4.2 Baselines
We consider the following zero-shot and few-shot baseline methods. We also compared with two transfer learning methods, SPOT (Vu et al., 2022) and OPTIMA (Guo et al., 2022), where large-scale real-world source-domain dataset is available.
(Zero-shot) Prompting. We prompt the frozen 15-large and Flan-15-large with only task- specific natural language prompts (i.e., hard prompts) and treat it as the zero-shot learning baseline. For fair comparisons, we apply the same hard prompts for all baselines where applicable. Prompt-based templates are presented in the Appendix.
In-context Learning (ICL). Following GPT-3 (Brown et al., 2020), we incorporate the ac- quired few-shot examples as demonstrations in the hard prompt templates, which is also called few-shot prompting (in contrast to zero-shot prompting). The role of few-shot ex- amples helps the frozen 15-large and Flan-15-large models better understand the task by exemplifying the task instruction with real-world examples. The order of few-shot samples is randomly determined and we report the average performance across five runs.
Full-model Fine-tuning (FT). We feed the few-shot data without any hard prompts into T5-large and Flan-T5-large and fine-tune the entire networks. Different from traditional fine-tuning that trains an additional classification layer from scratch, here, apply the label verbalizer and tune the language modeling head instead.

---

Published as a conference paper at COLM 2024
#Trainable Params
Method
QQP
MRPC
MNLI
SNLI
QNLI
RTE
SICK
AVG
T5-large
Prompting In-Context
42.63
33.80
33.20
33.31
49.46
52.35
14.51
37.04
59.55
33.52
34.49
33.80
49.72
48.52
40.90
42.93
FT
72.50
61.72
42.82
48,90
50.11
55.81
77.90
58.54
Prompt-based FT PFT soft prompt
770M
60.15
59.66
42.94
54.16
51.75
57.18
69.98
56.55
60.22
56.18
43.86
48.45
57.38
55.60
76.23
56.85
PPT
410K
46.11
52.37
34.05
35.28
52.86
48.59
45.64
44.99
Prompt Tuning
102K
47.28
58.94
33.29
33.21
52.68
51.70
27.80
43.49
Ours
102K
66.77
69.67
53.20
46.81
69.84
57.40
72.73
62.35
SPOT*
64.5
68.7
74.3
78.8
72.9
102K
OPTIMA
69.1
71.2
78.4
82.1
73.3
Flan- T5-large
Prompting In-Context
62.15
67.71
62.13
64.07
80.29
26.35
33.31
56.57
82.84
75.27
62.44
54.87
89.98
19.06
38.02
60.35
FT
79.17
78.29
79.76
86.37
56.86
86.57
83.73
78.68
Prompt-based FT PFT + soft prompt
770M
80.28
78.04
78.
88.11
50.56
84.84
80.96
77.32
79.64
77.65
79.87
86.90
80.37
84.91
70.60
79.99
Prompt Tuning Ours
70.40
72.82
59.89
63.26
83.73
26.78
60.61
62.49
102K
82.14
78.40
71.84
82.43
88.80
56.82
79.88
77.19
Table 1: Test accuracy all datasets. The best result of each dataset is bolded. t: SPOT and OPTIMA are transfer learning techniques that require a fully labeled source cataset, with performance numbers from (Guo et al., 2022)
Prompt-based Full-model Fine-tuning (PFT). Using hard prompts for model tuning is represented by LM-BFF (Gao et al., 2021) and PET (Schick & Schütze, 2021a). Here, we apply the same hard prompts as other baselines to wrap every training sample for tuning T5-large and Flan-T5-large. We also consider incorporating soft prompts for model tuning as P-Tuning (Liu et al., 2022) and DART (Zhang et al., 2022), denoted as PFT + soft prompt.
Pre-trained Prompt Tuning (PPT). Proposed by Gu et al. (2022) to pre-train soft prompts text corpus from OpenWeb
(Gokaslan Cohen, 2019b) using the next sentence prediction objective (Devlin et al., 2019). We download the pre-trained checkpoint for T5-xxl and fine-tune them on the sentence-pair classification tasks.
FewGen. Proposed by Meng et al. (2023), where authors first use the few-shot data to adapt CTRL (Keskar al., 2019) to every task with prefix tuning and then generate synthetic datasets using the adapted generator. We use their code to produce the synthetic datasets (denoted as FewGen) for all tasks and train soft prom pts for T5-large or Flan-T5-large. It can be treated as the ablation for DawGen where only Le applied.
4.3 Few-shot learning performance
Our main results are presented in Table 1. Overall, compared with the naive prompt tuning method, our approach yields an average improvement of approximately 18% across all datasets when applied to T5-large, and about 15% for Flan-15-large. In particular, PT (using 102K parameters) under our framework outperforms FT (using 770M parameters) by an average improvement of 3.8% across all datasets based on T5-large. When compared to SPOT and ÓPTIMA, which use large real-world datasets for transfer learning, our approach exhibits competitive performance on QQP, MRPC, and SICK, though its performance on

---

Published as a conference paper at COLM 2024
Method
Generator
QQP
MRPC
MNLI
SNLI
QNLI
RTE
SICK
AVG
T5-Large
Real+Syn Real+Syn+LS Real -> Syn
52.64
70.53
38.15
33.96
57.08
52.64
48.01
50.43
53.09
67.94
38.58
34.11
56.99
56.03
58.05
52.11
FewGen
59.51
70.04
47.46
41.99
65.52
57.91
65.81
58.32
Syn -+ Real
63.78
68.92
36.97
35.17
63.24
53.86
52.90
53.55
(Real, Syn)
66.44
68.12
48.03
44.81
64.15
56.54
68.46
59.51
Real+Syn
62.86
70.38
43.97
35.62
60.11
53.29
51.31
53.93
Real Syn
62.60
69.39
47.94
46.14
66.33
58.12
61.48
58.85
DawGen
Syn + Real
62.69
69.28
42.45
38.01
60.35
55.31
56.20
54.89
(Real, Syn)
61.77
69.99
48.76
45.10
66.37
57.20
70.80
59.99
Flan- Flan-T5-largs T5-large
Real+Syn
81.13
76.82
67.91
66.79
85.32
54.01
75.04
72.43
Real+Syn+IIS
79.56
76.06
73.50
71.42
82.96
55.88
70.37
72.82
Real - Syn
FewGen
79.09
76.08
64.94
63.46
85.76
57.19
71.15
71.10
Syn Real
82.18
79.00
68.35
72.24
82.08
58.70
77.88
74.34
(Real, Syn)
82.33
78.04
68.86
80.14
87.19
56.68
78.56
75.97
Real+Syn
83.60
76.81
71.85
72.48
84.11
53.72
69.17
73.10
Real -> Syn
80.50
75.64
66.42
69.41
86.52
54.22
73.33
72.29
DawGen
Syn + Real
83.26
78.55
72.15
77.29
87.51
50.76
72.17
74.53
81.83
76.96
70.18
79.69
87.38
51.63
76.97
74.94
Table 2: Test accuracy of different strategies for learning real and synthetic data, where synthetic datasets are generated by FewGen and DawGen respectively. The best result in each group is bolded.
MNLI and SNLI remains big challenge, highlighting the possibility of using synthetic training data as an alternative to enhance few-shot prompt tuning.
When comparing Prompt Tuning (P'T) with both Prompting and In-Context Learning base- lines, we observed that PT's performance enhancement is marginal, despite leveraging 102K parameters learn from data. This suggests that PT can be prone overfitting when applied to few-shot datasets. On the contrary, FT employs 770M parameters and, despite its greater tendency to overfit the data, surpasses PT by an average of 14-18%, indicating that better pre-trained initialization can significantly enhance the learning outcomes.
4.4 The order in which synthetic and real data appears does matter
Previous research (Vu et al., 2022; Gu et al., 2022; Guo et al., 2022) indicates that data-driven initialization significantly improves PT. Here, we study how the order in which synthetic data and real data appear affects PT. We experimented with several strategies: Real+Syn directly combines the synthetic and few-shot real data as a new training set and performs shuffling before mini-batch training. Real > Syn trains soft prompt first on Dreat and then on Day: in every training epoch. Syn Real, on the contrary, trains soft prompt first on D Day and then on Dreal in every training epoch. (Real, Syn) means pairing every batch sampled from Daym with a batch sampled from Dreat and combining them as new batch to train soft prompts, which employed in our approach. Results are presented in Table 2.
We observed that a naive combination of Syn + Real data performs the worst, where the few-shot real data could be overwhelmed by the larger synthetic data. Simply applying label smoothing regularization, denoted as Syn Real + LS, generally does not help. In contrast, (Real, Syn), which up-samples the few-shot real data when paired with either FewGen or DawGen data, confers an obvious improvement for both 15-large and Flan-T5- large. We also observed an interesting phenomenon, where T5-large prefers Real - Syn while Flan-T5-large prefers Syn - Real. This may suggest that training a model initially the few-shot real data is not always an advantage and the order in which real and synthetic data presented impacts differently for different LLM backbones.
8

---

Published as a conference paper at COLM 2024
Generator
Real
GS
QQP
MRPC
MNLI
SNLI
QNLI
RTE
SICK
AVG
T5-Large
56.70
69.25
42.18
34.63
56.91
53.87
34.11
49.66
FewGen
66.44
68.12
48.03
44.81
64.15
56.54
68.46
59.51
67.88
70.05
49.52
46.39
66.96
55.96
72.08
61.25
58.62
69.11
44.30
36.63
61.97
55.16
50.39
53.74
DawGen
61.77
69.99
48.76
45.10
66.37
57.20
70.80
59.99
66.77
69.67
53.20
46.81
69.84
57.40
72.73
62.35
Flan- I5-large
78.29
78.72
62.03
66.13
84.00
50.54
69.81
69.93
FewGen
82.33
78.04
68.86
80.14
87.19
56.68
78.56
75.97
81.76
78.36
75.26
81.07
87.67
61.13
79.26
77.78
83.11
77.05
68.21
70.49
84.07
49.02
68.66
71.52
DawGen
81.83
76.96
70.18
79.69
87.38
51.63
76.97
74.94
82.14
78.40
71.84
82.43
88.80
56.82
79.88
77.19
Table 3: Test accuracy of ablation studies. "GS" stands for "Gradient Surgery" in prompt tuning.
4.5
Ablation study
We evaluate every component in our approach and present the results in Table 3. Specif- ically, we observe that: 1) the distribution-aligned regularization term L effective Comparing DawGen against FewGen, the soft prompts trained exclusively on DawGen results in an average improvement of 4% for T5-large and 1.6% for Flan-T5-large across all datasets compared to the one trained on FewGen; 2) utilizing a few real samples to augment synthetic datasets beneficial, and this benefit is more pronounced on FewGen than DawGen, indicating that the higher the quality of synthetic dataset, the less it requires supplementation with real data supervision. 3) the gradient surgery technique effectively mitigates the conflict between synthetic and real data sources applying gradient surgery further improves the performance by an average of approximately 2% across the datasets.
4.6
Instruction-tuned models are better few-shot learners
Recent works (Varia al., 2023; Aly et al., 2023) suggested that the advantage of instruction tuning (Ouyang et al., 2022; Chung et al., 2022) for language models can extend to few-shot learning settings. This is further corroborated in our experiments. Across the tables, Flan- T5-large excels 15-large a few new findings to this avenue: 1) instruction-tuned models can follow soft prompts better, shown by PFT soft prompt versus PFT, where average improvement of 2% is spotted on Flan-T5-large while no increase is observed on T5-large. 2) Prompt Tuning on top of instruction-tuned models are less prone to overfitting the few-shot data. It shown that Flan-T5-large elevates the PT performance of T5-large by around 20% on average. 3) instruction-tuued models prefer using synthetic data to warm up the learning, as shown in Table 2. Moreover, when using Flan- Flan-I5-large as the backbone, Syn + Real strategy leads to an average improvement of about
all experiments by a large margin. In this paper, we provide
4%, as shown by comparing either Few or DawGen from Table 3 with Syn + Real from Table 2. While synthetic data is often used as a form of regularization for learning on real datasets, in our study, presenting a few real samples after training synthetic data produces regularization effect. We suspect that the instruction-folloiing capability of Flan-T5-large may play an important role.
Conclusion
This paper presents framework for generating synthetic training data with LLMs to boost prompt tuning in few-shot settings. We introduce Distribution-Aligned Weighted GENerator

---

Published as a conference paper at COLM 2024
tuning (DawGen) to encourage the generation of label-relevant text samples to improve the quality of the synthetic dataset. To better exploit the synthetic data while making the best use of the few-shot real data set, employ the gradient surgery technique during prompt tuning to eliminate the conflicting gradients from the synthetic data batches. Experiments seven sentence-pair classification datasets and two LLM backbones demonstrated the effectiveness of our method for boosting prompt tuning in few-shot settings with synthetic data. We show that Prompt Tuning augmented with DawGen can surpass Full-Model Fine-Tuning in few-shot settings by a large margin, highlighting the feasibility of using LLM-generated data as an alternative.
Limitations and Discussions
Currently, there are few limitations of this study that may limit the impact scope of the insights conveyed by this paper.
The gap between the evaluation metric and the quality of the synthetic data. Specif- ically, the downstream few-shot learning performance of prompt tuning, which is often measured by accuracy, may only reflect the preferences of the model rather than aligning with human judgment in terms of the quality of the synthetic data. There should be intuitive methods to explain to humans why one synthetic sample is superior another.
The few-shot learning performance on these public benchmarks reported in this study does not stand for the state-of-the-art. They only reflect the performance of prompt tuning. More advanced parameter-efficien learning methods like LoRA could bring higher performance than prompt tuning. Nevertheless, the relevant improvements in this study can still support the effectiveness of specific strategy.
Synthetic data generation cost can be concern. Current data generators rely on large language models, which have deep stack of transformer layers. Feedforward computations and autoregressive generations involve non- non-trivial GPU, memory, and time costs. A few research efforts, such FastGen Ge et al. (2024), have been devoted to accelerating LLM inference costs.
7 Future Directions
In Table 4, we compare the modern augmentation paradigm, which leverages Generative AI, specifically LLMs, with the traditional paradigm that relies on transfer learning. Together with the discussions in the Limitation section, we propose the following directions: 1) employ explainable approaches to highlight the influential elements in the synthetic data and then devise quantitative measures to assess the data quality; and 2) develop inference acceleration al algorithms tailored specifically for batch generation.
Paradigms
Data Source
Label
Pretrain Challenge
Transfer Learning Generative Al
real-world LLM-generated
human annotation
distribution gap
given prompts
data quality,
task relevance
Table 4: Two paradigms for boosting few-shot learning performance of soft prompt tuning.
Acknowledgments
This research is supported by the Wallenberg-NTU Presidential Postdoctoral Fellowship, the Nanyang Associate Professorship, and NRF Fellowship (NRF-NRFF13-2021-6 Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not reflect the views of the funding agencies.
10

---

Published as a conference paper at COLM 2024
References
Rami Aly, Xingjian Shi, Kaixiang Lin, Aston Zhang, and Andrew Wilson. Automated few-shot classification with instruction-finttuner language models. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 2414-2432, Singapore, December 2023. Association for Computational URL https://aclannthologg
Linguistics. doi: 10.18653 /v1/ 2002...nnings-emmt org/2023.findings emnlp. 158.
Ateret Anaby-Tavor, Boaz Carmeli, Esther Goldbraich, Amir Kantor, George Kour, Segev Shlomov, Naama Tepper, and Naama Zwerdling. Do not have enough data? deep learning to the rescue! Proceedings of the AAAI Conference on Artificial Intelligence, 34(05))7383-7390 2020.
Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. A large annotated corpus for learning natural language inference. In Lluís Mårquez, Chris Callison-Bucch, and Jian Su (eds.), Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Pp. 632 642, Lisbon, Portugal, September 2015. Association for Computational Linguistics. doi: 10.18653 10.18653/v1/D151110 /v/551110 URL https://aclantholoogy 1S/D15--075..
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction- finetuned language models. Xrv preprint ar arXin:2210.11416, 2022.
Ido Dagan, Oren Glickman, and Bernardo Magnini. The recognissng textual entail- ment challenge. In Machine learning challenges workshop, pp. 177-190. Springer, 2005,
Jacob Devlin, Ming-Way Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171-4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653/v /vl N19-1423, URL https: http://acaanttllo.rrrrrree org/N19911122 1423.
William B. Dolan and Chris Brockett. Automatically constructing a corpus of sentential paraphrases. Proceedings of the Third International Workshop on Paraphrasing (IWP2005), 2005. URL https: / http///alalaattta //tp//aaaaaaantogy.../o. or org/
Mengnan Du, Varun Manjunatha, Rajiv Jain, Ruchi Deshpande, Franck Dernoncourt, Jiuxi- ang Gu, Tong Sun, and Xia Hu. Towards interpreting and mitigating shortcut learning be- havior of NLU models. In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (eds.), Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 915-929, Online, June 2021. Association for Computational Linguistics. doi: 10.18653 http//2222..acmmmii.77.. URL https://aclantholl/
org org/2021 naacl manel-main.7.
Jiahui Gao, Renjie Pi, Yong Lin, Hang Xu, Jiacheng Ye, Zhiyong Wu, Weizhong Zhang, Xiaodan Liang, Zhenguo Li, and Lingpeng Kong. Self-guided foise-free data generation for efficient zero-shot learning. arXiv preprint ar arXio:2205.122 12679, 2022.
Jiahui Gao, Renjie Pi, LIN Yong, Hang Xu, Jiacheng Ye, Zhiyong Wu, WEIZHONG ZHANG, Xiaodan Liang, Zhenguo Li, and Lingpeng Kong. Self-guided noise-free data generation for efficient zero-shot learning. In International Conference on Learning Representations, 2023.
URL https://openreviaae http::///beereeieeenn////rrrrrtd www.ne//ooomndd-nSppppp lo6.
11

---

Published as a conference paper at COLM 2024
Tianyu Gao, Adam Fisch, and Danqi Chen. Making pre-trained language models better few-shot learners. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Online, August 2021. Association for Computational Linguistics. doi: ort. acl-long.200
pp. 10.18653 /t///v11/1100 /1/202lacl-long.255 295.. URL
Suyu Ge, Yunan Zhang, Liyuan Liu, Minjia Zhang, Jiawei Han, and Jianfeng Gao. Model tells you what to discard: Adaptive KV cache compression for LLMs. In The Twelfth International Conference on Learning Representations, 2024. URL forum?)dd-uNrppDPMy..
/ettteeerreeeeeee/eeeeeeee net/
Aritra Ghosh, Himanshu Kumar, and P Shanti Sastry. Robust loss functions under label noise for deep neural networks, In Proceedings of the AAAI conference artificial intelligence, volume 31, 2017.
Aaron Gokaslan and Vanya Cohen. Openwebtext corpus, 2019a. URL http://t//99k github. wwt.o/pppeeeeeettorrpreooo
Aaron Gokaslan and Vanya Cohen, 2019b. CeenWebTextCorpoee
URL https://skyllono00.. tit.b......tt.......... io/
Yuxian Gu, Xu Han, Zhiyuan Liu, and Minlie Huang. PPT: Pre-trained prompt tuning for few-shot learning. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Pp. 8410-8423, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653 htt/://11122222lllloogeg acl-long.57 576.
URL https:://allantholoeee orgge/rrr////2/ 2022.
Xu Guo and Han Yu. On the domain adaptation and generalization of pretrained language models: A survey. arXiv preprint arXio:2211.03115. 2022.
Xu Guo, Boyang Li, and Han Yu. Improving the sample efficiency of prompt tuning with domain adaptation. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Find- ings of the Association for Computational Linguistics: EMNLP 2022, pp. 3523-3537, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguis- tics. doi: 10. /v/555/l/// h 201///22..immat// fwn.iiinnngn-ennl..222. 258. URL https://aclanthoolggy htwpolllyyyooooggg//ee org/2022. Sindings-emnlp 258.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Larous- silhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nip. In International conference on machine learning, pp. 2790-2799. PMLR, 2019.
Edward Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference Learning Representations, 2022. URL https://openreviier net/ forum?id=nZeWeeeFYY.
Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. Ctrl: A conditional transformer language model for controllable generation. arXiv preprint ar arXiv:1909.05853 2019.
Varun Kumar, Ashutosh Choudhary, and Eunah Cho. Data augmentation using pre-trained transformer models. In Proceedings of the 2nd Workshop on Life-long Learning for Spoken Lan- guage Systems, pp. 18-26, Suzhou, China, December 2020. Association for Computational Linguistics. URL https: / http//aaaalnttt wwwlanhhllgg.ooro org/ 1 lifelongnlp-1..
Samuli Laine and Timo Aila. Temporal ensembling for semi-supervised learning. arXiv preprint ar Xix: Ait::660000222. 02.4222, 2016.
Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for earfmeter-efffcient prompt tuning. In tau Yih (eds.), Proceedings of the 2021 Conference Empirical Methods in Natural Language
Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-
12

---

Published as a conference paper at COLM 2024
Processing, Pp. 3045-3059, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 111.6663/112222.emmmppp maini. URL https://aclanthoot org/2021. emnlp-maine mamommmin... 243.
Dingcheng Li, Zheng Chen, Eunah Cho, Jie Hao, Xiaohu Liu, Fan Xing, Chenlei Guo, and Yang Liu. Overcoming catastrophic forgetting during domain adaptation of seq2seq language generation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 5441-5454 2022.
Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing ( (Volume 1: Long Papers), pp. 4582-4597, Online, August 2021. Association for Computational Linguistics. doi: 10.18653/v12222..cc// org/ mocl-long.r 353.
353. URL
Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Compulational Linguistics (Volume 2: Short Papers), pp. 61-68, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653 200.222.ccc.short.. URL https: / http://aaaaaaaaog...aaa.o org.orgg/////// 2022. acl-short. aoho
Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, and Roberto Zamparelli. A SICK cure for the evaluation of compositional distributional seman- tic models. In Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, and Stelios Piperidis (eds.), Pro- ceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14), pp. 216-223, Reykjavik, Iceland, May 2014. European Language Resources Association (ELRA). URL http: //www//wwwwwwww www.:w/wwe-.oo.ooee wwwppooo...gggooeeeeeeeee
.ddf.
Yu Meng, Jiaxin Huang, Yu Zhang, and Jiawei Han. Generating training data with language models: Towards zero-shot language understanding. Advances in Neural Information Processing Systemis, 35:462-477, 2022.
Yu Meng, Martin Michalski, Jiaxin Huang, Yu Zhang, Tarek Abdelzaher, and Jiawei Han. Tuning language models as training data generators for augmentation-ennaaned few-shot learning. In International Conference on Machine Learning, pp. 24437-24477. PMLR, 2023.
Swaroop Mishra, Daniel Khashabi, Chitta Baral, Yejin Choi, and Hannaneh Hajishirzi. Re- framing instructional prompts to GPTk's language. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Findings of the Association for Computational Linguistics: ACI 2022, pp. 589-612 612, Dublin, Ireland, May 2022. Association for Computational Lin- guistics. doi: 10. 10.18653/ /202 findingg---n....500. URL https:://cclaattollloe .wwg//oooooooooooooooooo 2022. findings-acl.55 50.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730-27744, 2022.
Raul Puri, Ryan Spring, Mohammad Shoeybi, Mostofa Patwary, and Bryan Catanzaro. Training question answering models from synthetic data. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Pp. 5811-5826, Online, November 2020. Association for Computational Linguistics. doi: / /vll 2020.emnlp-main-466 URL https://aclantthllgg org.oogg/2222 emnlp-mat waallp-min... ma...468..
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with unified text-to-text transformer. Journal of machine learning research, (1640))11667, 202. 2020.
13

---

Published as a conference paper at COLM 2024
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. SQuAD: 100,000+ questions for machine comprehension of text. In Jian Su, Kevin Duh, and Xavier Carreras (eds.), Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 2383-2392, Austin, Texas, November 2016. Association for Computational Linguistics. doi: 10.18653/ /116/
URL https://aclaattholl.g/ orgg 1224.
Timo Schick and Hinrich Schütze. Exploiting cloze-questions for few-shot text classification and natural language inference. Paola Merlo, Jorg liedemann, and Reut Tsarfaty (eds.), Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, Pp. 255-269, Online, April 2021a. Association for Computational Linguistics. doi: 10.18653 v1 2021.eacl-main URL https: patps::/acllas https://aclanttoolggg org.orgg///// /0211. pacl-main.20.
Timo Schick and Hinrich Schütze. Generating datasets with pretrained language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 6943-6951, Online and Punta Cana, Dominican Republic, November 2021b. Association for Computational Linguistics. doi: 10. 10.18653/v1 http://ww2221..emmmmaaaaee //aclanthology. wwwp:/llgg.ooggo.a.goooooe org/2021 .enn1p-main 555.
URL https:
Ruixiang Tang, Dehan Kong, Longtao Huang, and Hui Xue. Large language models can be lazy learners:
Analyze shortcuts in in-context learning. In Anna Rogers, Jordan Boyd- Graber, and Naoaki Okazaki (eds.), Findings of the Association for Computational Linguistics: ACL 2023, pp. 4645-4657, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/v1 htt.://22.....nnss-oooe 2023. findings-acll 284.
URL
org/
Siddharth Varia, Shuai Wang, Kishaloy Halder, Robert Vacareanu, Miguel Ballesteros, Yassine Benajiba, Neha Anna John, Rishita Anubhai, Smaranda Muresan, and Dan Roth. Instruction tuning for few-shot aspect-based sentiment analysis. In Jeremy Barnes, Orphée De Clercq, and Roman Klinger (eds.), Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pp. 19-27, Toronto, Canada, July 2023, Association for Computational Linguistics. doi: 10.18653/ / was URL
org/ 2222 wassa-1.3
Tu Vu, Brian Lester, Noah Constant, Rami Al-Rfou', and Daniel Cer. SPoT: Better frozen model adaptation through soft prompt transfer. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Pp. 5039-5059, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653 (018653/v1/1 (0.18653/w1/2222acccl 2122022.acl-long..444 URL http://lcaaantooog.oooooooooooo. wwwphhhlog...og/////00 org. acl- long. 346.
Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sentence understanding through inference. In Marilyn Walker, Heng Ji, and Amanda Stent (eds.), Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 1112-1122, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: 10.18653 1186553vv1/188 11011 URL https: / httt::://aaaaaaiiiee /alpaaannannnoy.....c Trrg/N18-118
Jiacheng Ye, Jiahui Gao, Qintong Li, Hang Xu, Jiangtao Feng, Zhiyong Wu, Tao Yu, and Ling- peng Kong. ZeroGen: Efficient zero-shot learning via dataset generation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 11653-1 11669, Abu Dhabi, United Arab Emirates, December 2022a. Association for Computational Linguistics. URL htthttt://// htttps://accanaeeeeee http///aaaaaahoog....... org//022.. 2022. emnlp-main. 801.
Jiacheng Ye, Jiahui Gao, Zhiyong Wu, Jiangtao Feng, Tao Yu, and Lingpeng Kong, ProGen: Progressive zero-shot dataset generation via in-context feedback. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 3671-3683, Abu Dhabi, United Arab Emirates, December 2022b. Association for Computational Linguistics. doi: 10.18653 /v1l hwwp///2..fmmita www...inningg-emmm..... 269. URL https://aclanthoooogy.. 2022 findings-emnlnn 269.
14

---

Published as a conference paper at COLM 2024
Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. Advances in Neural Information Processing Systems, 33:5824-5836, 2020.
Yue Yu, Yuchen Zhuang, Jieyu Zhang, Yu Meng, Alexander J Ratner, Ranjay Krishna, Jiaming Shen, and Chao Zhang. Large language model as attributed training data generator: A tale of diversity and bias, Advances Neural Information Processing Systems, 36, 2024.
Haode Zhang, Yuwei Zhang, Li-Ming Zhan, Jiaxin Chen, Guangyuan Shi, Xiao-Ming Wu, and Albert Y.S. Lam. Effectiveness of pre-training for few-shot intent classification. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 1114-1120, Punta Cana, Dominican Republic, November 2021. Association for Computational Linguis- URL https://aclantthon
tics. doi: 10.18653/ /v/5///vvv/// 1002...inding-eemn// findings-emnlp.99
org/ 0r2/2021.
Ningyu Zhang, Luoqiu Li, Xiang Chen, Shumin Deng, Zhen Bi, Chuanqi Tan, Fei Huang, and Huajun Chen. Differentiable prompt makes pre-trained language models better few-shot learners. In International Conference on Learning Representations, 2022. URL https://openreview. net//forum??d=ekSa@QIaa.
Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. Memorybank: Enhane- ing large language models with long-term memory. In Proceedings of the ÁAAI Conference on Artificial Intelligence, volume 38, pp. 19724-19731, 2024.
Beier Zhu, Yulei Niu, Yucheng Han, Yue Wu, and Hanwang Zhang. Prompt-aligned gradient for prompt tuning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15659-15669, 2023.
15

---

Published as a conference paper at COLM 2024
A Experimental Setup Details
A.1 Datasets
The dataset statistics are presented in Table 5. We sample the few-shot [D Drrail and dell from their original training sets and report performance on their original development sets. For MNLI, we report test accuracy on the matched version. We covert their original labels using the verbalizer as in the Table.
Task
Train
Dev
Pclass
Verbalizers
QQP
363,846
40,430
2
yes/no yes/no
MRPC
3,068
408
2
MNLI
392,702
9,815
3
yes/maybe/ no yes/ /a/m//abbe///// no
SNLI
549,367
9,842
3
QNLI 104,743
5,463
2
yes/no nno
RTE 2,490
278
2
yes/no
SICK 4,439
4,906
3
yes/ maybe/no
Table 5: The original dataset statistics.
A.2 Hard prompt template.
Task
Template
label
QQP
Question is (different) from Question 2. Question 1: (St) Question 2:
different
Question is (equivalent) to Question 2. Question 1: (5)) Question 2:
equivalent
MRPC
Sentence (different) from Sentence Sentence 1: (8,) Sentence 2:
different
Sentence is (equivalent) Sentence Sentence 1: (Si) Sentence
equivalent
MNLI
Sentence 1
implies) Sentence 2. Sentence 1: (S1) Sentence 2:
implies
Sentence 2 suuppemmnts) Sentence 1. Sentence 1: (S,) Sentence 2:
supplements
Sentence (cnttradicts)Senteneee
Sentence {S1 Sentence 2:
contradicts
SNLI
Sentence 1 (implies) Sentence Sentence 1: )S,) Sentence 2:
implies
Sentence 2 (supplements)Senuue
1. Sentence 1: (S,) Sentence 2:
supplements
Sentence 2 (contradicts) Sentence 1. Sentence 1: (S₁) Sentence
contradicts
QNLI
Paragraph is (relevant) to Question. Question: (S) Paragraph:
relevant
Paragraph is (irrelevant) to Question. Question: ($1) Paragraph:
irrelevant
RTE
Sentence 1 (implies) Sentence 2. Sentence 1: (51) Sentence 2:
implies
Sentence 2 (supplements) Sentence 1. Sentence 1: (51) Sentence 2:
supplements
SICK
Sentence (implies) Sentence 2. Sentence 1: (S1 Sentence 2:
implies
Sentence 2 (uuplements) Sentence 1. Sentence 1: (S₁) Sentence 2:
supplements
Sentence 2 (contradicts) Sentence 1. Sentence 1: (S,) Sentence 2:
contradicts
Table Natural language prompt templates used for generator tuning.
A.3 Models and Training Setting
Generator Tuning, Following FewGen (Meng et al., 2023), we use CTRL (1.6B parameters) (Keskar et al., 2019) as the generator 8y and apply prefix tuning to train it on the few-shot real dataset via training set of label-specific Q {Or} using the few-shot data D following Algorithm 1. We generate 1000 synthetic samples per class for every task using the same setting as FewGen Following Li & Liang (2021), we set the prefix length to the number of tokens in the natural prompts in Table 5. We use the same hyperparameters for all tasks and set the batch size to 2. The learning rate for weight net is set to 1e-2 and the one for Q is set to 5e-3. The number of training epochs is fixed to 20. More details can be found in the original paper (Meng et al., 2023).
16

---

Published as a conference paper at COLM 2024
Task
Template
Verbalizers
QQP
Are (S1) and (S2) equivalent? [AASK]
yes/no
MRPC
Are the first sentence: (S1) and the second sentence: (S2) equivalent?
yos/no
MASK
MNLI
hypothesis: (52) premise: (S1) answer: MASK
yes/ res/maybe/no
SNLI
hypothesis: (52) premise: (5I) answer:
MASK
yes/ res/maybe/no
QNLI
Is the question: (S2) relevant to the paragraph: (S1) ? MASK
yes/no
RTE
hypothesis: (S2) premise: (ST) answer: MASK
yes/no no
SICK
hypothesis: (2) premise: (S1) answer: MASK
yes/ maybe/ no
Table 7: Natural language prompt templates used for prompt tuning.
QQP
MRPC
MNLI
SNLI
QNLI 8e-3
RTE
SICK
T5-large
0.014
8e-3
9e-4
0.2
0.002
9e-8
Table 8: p-values for DawGen over full-model finetuning.
Prompt Tuning. Following previous works (Lester et al., 2021; Gu et al., 2022; Guo et al., 2022), conduct prompt tuning on T5 (Raffel et al., 2020) and use the Im-adapted version of T5-large (770M parameters) for the main experiments. We also experimented with Flan- T5-large (Chung et al., 2022), which further trained T5-large on instruction tuning datasets to improve its instruction following capability (Ouyang et al., 2022). We fix the prompt length 100 and learning rate y = 0.3 as suggested by Lester et al. (2021). We use the cosine learning rate scheduler for all methods. We set the maximum number of training steps to 1, 000 and evaluate models on the development set every 4 steps. We set batch size for MRPC and QQP, and 6 for the other NLI datasets. For all the prompt tuning experiments, the 15-large and Flan-T5-large backbone models are fixed and only soft prompt embeddings are updated. All the training experiments are done on NVIDIA A6000 with 49 GB.
B Significance tests
We conducted significance test for Table 1 on results where our method was superior. Table 8 shows the p-values. Our method shows statistically significant improvement over the second-best baseline, full-model fine-tuning (FT). Although not superior in every task, our method shows higher average improvements while using only 102K parameters, compared to 770M in full-model fine-tuning.
C Discussion of the computational cost
Comparing the computational costs of different training paradigms directly is challenging due to variations in training data, pretraining strategies, and stopping conditions. There could be a trade-off between performance and computational cost.
PPT: It invoolves a one-time pre-training of prompts on 10GB of OpenWeb data using next sentence prediction. While these prompts are reusable across tasks, they may underperform in tasks that are underrepresented
the Open corpora. more source tasks, and then adapted for
SPoT:
Prompts are pre-trained on a target task. However, selecting appropriate source tasks requires an extensive search and may struggle with large domain gaps.
DawGen: It involves one-time adaptation of the generator to the target task using few-shot examples. The adapted generator can then produce unlimited synthetic data, enhancing the flexibility of prompt training. However, the generation cost depends on the LLM backbone and the advanced inference acceleration methods such as FastGen.
17


Reviews:
[{"review_id": "lOxsy6GoPw", "reviewer": "Reviewer_1fLs", "summary": "This paper addresses the topic of prompt tuning, which typically requires large quantities of labelled training data and underperforms full-model fine-tuning in few-shot settings (by which the authors refer to low-resource settings rather than few-shot prompting). The authors present a method to create synthetic data using an LLM to do prompt tuning on and a method to fine-tune on the data, consisting of using gradient surgery when fine-tuning on a mix of the synthetic and real data. \n\nTheir method consists in (i) training a soft prompt using a few-shot dataset to adapt an LLM to a given domain, (ii) using their method Distribution-Aligned Weighted GENerator tuning (DawGen) to align generated examples to those of the same class, and (iii) using label-conditional prompts and (iv) using gradient surgery when fine-tuning on a combination of synthetic and real data to account for possibly low quality data.\n\nThey test their method using T5-large and FlanT5-large for 2 tasks (paraphrase detection and NLI) over 7 datasets, comparing to multiple other methods (fine-tuning, zero-shot, in-context learning, etc.). Their method gives results better than fine-tuning for some tasks and worse for others, and 18% improvement on average compared to basic prompt tuning for T5-large.\n\nIn its current state, I found the paper unclear with respect to its main aim (more with respect to the writing of the paper - providing motivation for working with so few training examples for example) and would also have appreciated some more details concerning the method (see below). The paper is also lacking examples allowing me to review the quality of the produced synthetic data.", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the comments. I would like to clarify two terms. Prompt tuning is an efficient, low-cost way of adapting an LLM to new downstream tasks without retraining the model and updating its weights; Few-shot learning aims to make accurate predictions by training on a very small number of labelled examples. These concepts are distinct from providing few shots in prompting ChatGPT (in-context learning). We will enhance the description in our revised manuscript.\n\n**Weakness: Lack of clarity about the aim**\n\nWhile prompt tuning is efficient, it often lags behind traditional FT when training data is scarce. Hence, we propose synthesizing data for augmentation. As synthetic data may not be of high quality, we propose the gradient surgery to enhance learning, as stated in abstract and introduction. \n\nFew-shot is a realistic setting, especially in rare or dynamic situations. Prior research choose 8 or 16 training samples (shots) as a standard routine. We follow LM-BFF,PPT, and OPTIMA to set the shots to 16 for fair comparison.\n\nFor a comparison of FT and PT under the progression of training data, see Figure 2 in the PPT paper and Figure 3 in the OPTIMA paper.\n\n**Weakness: the base model used may be inadequate**\n\nLow accuracy scores are due to 1) unbalanced test sets of the binary tasks; and 2) challenging tasks. Even if we use a more powerful base model (T5-xxl, 11B), the accuracy is not sufficiently good. \n\nTable. Reported Test acc on RTE.\n| Paper    | Base Model | Strategy | Accuracy |\n| - | - | - |-|\n| LM-BFF | RoBERTa_large | Prompting | 51.3 |\n| | | FT |54.4 |\n| PPT |   T5-xxl |  PT |  53.5 |\n| Ours    |   T5-large | Prompting |  52.35 |\n| | |PT |  51.70 |\n| | | FT | 55.81|\n\n\n**Question: Synthetic Examples**\n\nThe inputs for our generator are \"soft prompt + hard prompt + label (+premise for NLI)\". We generate 1000 synthetic samples per class. Data from FewGen is available at: https://github.com/yumeng5/FewGen/tree/master/data/k-shot. We put our examples and more analysis here: https://drive.google.com/file/d/1yWj5E6mvN5oIQbWhobC29taEEGKXtox7/view?usp=sharing. \n\n**Other questions**\n\n*Num. Params* => #trainable Params\n\n*PPT with T5-xxl* lags behind our method with a weaker model, T5-large. The conclusion still holds.\n\n*ICL* => randomly select 8 examples from the few-shot set, followed by an unlabeled test example for prediction. Run 5 random seeds and average test performance.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- It is interesting to have a comparison of different approaches to using LLMs and I appreciate the effort in combining methods to use the data in the best way", "reasons_to_reject": "- Lack of general clarity in the paper about the main aim of the article. This is in my opinion linked to several things, starting from an unclear motivation concerning the use of “few-shot”, which in the context of prompting often refers to in-context learning, but here appears to refer to the case where we have few training examples (this could be made much clearer and non-ambiguous right from the start). On first reading, I was not sure why the authors were only using 16 examples per class of real data, which is so few. Also, why only choose 16 examples? If the main aim of the paper is to study whether prompt tuning can rival fine-tuning in low-resource settings, I think it would have been more informative to have the comparison at different numbers of training data examples in order to see the progression of the 2 approaches. With so few examples, fine-tuning must be hugely overfitting too. Is this a realistic setting?\n- In the main method, it was unclear to me what the input to the LLM looks like in order to produce synthetic training examples. Does it just consist of the label-conditional soft prompt and one of the 16 examples (for each class)? Also, I may have missed it, but how many examples were generated?\n- A major weakness in my opinion is the fact that no examples of generation data or predictions are given in the example. As a reviewer, this makes it impossible for me to judge the performance of the generation method and also whether the base LLM used is up to the job at all. The very low accuracy scores of the baseline on the binary tasks (below random) appear to show that the base model used may be inadequate.", "questions_to_authors": "1. The meaning of “few-shot learning” was initially unclear to me (between using few-shot examples and using a very small set of  training examples), particularly as the article is on the topic of LLMs, which typically use few-shot examples. I assume that the article wants to deal with very few training examples, but I only got this after rereading several times, and putting the article in the context of (Gao et al., 2022).\n2. Synthetic data generation method: \n \n    a) It is Unclear to me what the input to the LLM is other than the label-conditioned soft prompt and the current example. \n\n     b) What is the quality of the generated data like? There are currently no examples of generated examples in the article, making it impossible for me as a reviewer to judge the performance of the generation method. This is particularly relevant as the authors cite the inevitability of low quality data.\n3. Missing details:\n\n    a) Why was so little data used? 16 examples for each class? Is this a realistic setting? What about other amounts of data?\n\n    b) Amount of synthetic data generated. How diverse is the data? Are there duplicates?\n\n    c) In-context learning -> how many examples were used?\n\n    d) What does the “number params” correspond to in Table 1? The models are the same, so does this refer to the number of tuned parameters?\n\n    e) PPT appears to use T5-xxl and not T5. If so these results are not comparable.\n\n4. In the related work, the authors mention that fine-tuning on a model on few examples (e.g. 16 samples per class) can lead to overfitting. What is the significance of 16 samples? i.e. why choose this number, which appears to me to be far too low to be realistic.\n5. Verbaliser -> it would be good to make it clear what you mean by this in the heart of the text (it only became clear to me from the Appendix)\n6. Very low scores of the baselines “Prompting” and “In-context”, especially given the binary nature of the tasks. Why do they do so poorly (Under random for some tasks)? Could this mean that the base model being used is not good enough and perhaps a better model should be used?\n\n## Language, grammar, etc.\n\n- Few-shot real data -> real few-shot data\n- Comparable performance as -> comparable performance to\n- E.g., -> it is good practice to use a non-breaking space after E.g. to avoid linebreaks\n- Pertaining data -> Pretraining data\n- Which is inevitable to contain low-quality data -> which inevitably results in low quality data\n- Lester et al. (2021) converts -> Lester et al. (2021) convert\n- Taking sentence-pair classification as an example. Given -> Taking sentence-pair classification as an example, given", "extra_scores": {}}, {"review_id": "DELRREO5vh", "reviewer": "Reviewer_GDbm", "summary": "The paper introduces a novel approach to enhance prompt tuning (PT) in few-shot learning settings by synthesizing task-specific labeled datasets using large language models (LLMs). The authors propose a method called Distribution-Aligned Weighted Generator Tuning (DawGen) to create in-distribution synthetic data, and they apply gradient surgery to manage gradient conflicts when training with both real and synthetic data. Their experiments across various datasets demonstrate that this method can outperform traditional fine-tuning techniques, presenting an efficient alternative when large labeled datasets are not available.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for recognizing our motivation, presentation, and evaluation and providing us with constructive suggestions. \n\n**Weakness: there is not a unique method that has the best results**\n\nWe recognize that no single method consistently outperforms others across all datasets. Our approach averagely performs well, but its effectiveness can vary depending on the specific dataset and task. While FT excels in some tasks, it fails in others. Considering the balance of parameter efficiency and performance, our method remains a viable option for improving prompt tuning in few-shot settings. We will add a new section of limitations to expand this discussion.\n\n**The methodology could be more detailed to enhance reproducibility**\n\nThank you for the suggestion! We will include more details and clear examples.\n\n**Are the superior results statistically significant?**\n\nWe conduct t-test for Table 1 on results where our method is superior. The table below shows the p-values. Our method shows statistically significant improvement over the second-best baseline, FT. Although not superior in every task, our method shows higher average improvements while using only 102K parameters, compared to 770M in full-model finetuning. We will include significance tests for all tables in the Appendix. Thank you for the question.\n\nTable. p-value for ours against FT (lower better).\n| model    | QQP| MRPC| MNLI| SNLI | QNLI | RTE | SICK |\n| -|-| -| - |- |- |-|- |\n| T5-large | - | 0.014 | 8e-3 | - | 8e-3 | 9e-4 | -|\n|Flan-T5-large| 0.2| 0.002 | - |  - |  9e-8 | - |  - | -|\n\n**Can the model's performance degrade significantly with lower-quality synthetic datasets?**\n\nAs shown in Table 3, the performance of DawGen generally surpasses that of FewGen for both T5-large and Flan-T5-large models because DawGen produces higher quality data than FewGen due to its distribution alignment regularization. \n\n**How does the performance of DawGen scale with the size of the LLMs used for generating synthetic data?**\n\nThe generator LLM, CTRL, used by FewGen and DawGen, is available only in a single size—1.63B parameters. We couldn't assess the impact of generator sizes. DawGen should theoretically work unless the LLM generates perfect synthetic data without distribution gaps. We plan to evaluate on stronger LLMs like LLaMA-13B in the future.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper proposes an original method for overcoming the data limitations of few-shot PT by generating synthetic data, which is significant for advancing the efficiency of using LLMs.\n\n2. The experiments show that their method not only competes well with but occasionally outperforms much larger models trained with conventional techniques.\n\n3. The paper includes extensive evaluations across several datasets, demonstrating the robustness and versatility of the proposed methods.\n\n4. The paper is well-organized, making complex concepts accessible and clearly explaining the methodology and its significance.", "reasons_to_reject": "1. While the method shows interesting results, there is not a unique method that has the best results. The best results are scattered between different methods.\n\n2.Some parts of the methodology, particularly around the implementation of gradient surgery and the generation of synthetic data, could be more detailed to ensure reproducibility.\n\n3. Are the superior results statistically significant.", "questions_to_authors": "1.How robust is the proposed method to variations in the quality of synthetic data? Can the model's performance degrade significantly with lower-quality synthetic datasets?\n\n2. How does the performance of DawGen scale with the size of the LLMs used for generating synthetic data? Would larger or smaller models significantly affect the outcomes?", "extra_scores": {}}, {"review_id": "qILonncIPv", "reviewer": "Reviewer_T1e9", "summary": "This work tackles the problem that prompt tuning performs poorly in low-resource settings. It proposes a DawGen framework to generate synthetic data to augment prompt tuning and apply a gradient surgery approach to avoid conflicts from different data sources. Experiments shows that DawGen outperforms various baselines in low-resource prompt tuning.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for recognizing the presentation and evaluation of this work. We hope to address your concerns below.\n\n**Weakness: examine the method on other parameter-efficient learning methods.**\n\nAs our framework is not limited to prompt tuning, we believe it can also improve the few-shot learning performance of LoRA. We appreciate your suggestion and will explore this further after rebuttal due to time constraints. \n\n**Weakness: the computational cost.**\n\nComparing the computational costs of different training paradigms directly is challenging due to variations in training data, pretraining strategies, and stopping conditions. Specifically:\n\n* PPT: It involves a one-time pre-training of prompts on 10GB of OpenWebText data using next sentence prediction. While these prompts are reusable across tasks, they may underperform in tasks that are underrepresented in the OpenWebText corpora.\n\n* SPoT: Prompts are pre-trained on one or more source tasks, and then adapted for a target task. However, selecting appropriate source tasks requires an extensive search and may struggle with large domain gaps.\n\n* DawGen: It involves a one-time adaptation of the generator to the target task using few-shot examples. The adapted generator can then produce unlimited synthetic data, enhancing the flexibility of prompt training. For instance, generating 1000 examples per class only takes about two hours for paraphrase detection tasks and three hours for NLI tasks, but results in a substantial improvement on average. More advanced techniques such as FastGen [1] can further reduce the generation cost. \n\nIt is unfair to compare the finetuning time for PPT and SPoT with ours, which trains from scratch. Our primary goal is to enhance few-shot performance. There may be a trade-off between performance and computational cost. Our method is a good alternative when PPT and SPoT fail in new domains.\n\n**Can Gradient surgery be combined with PPT and SPoT?**\n\nNo. Gradient surgery requires access to all different domains' data in order to measure whether there is interference between the gradients of task loss w.r.t. prompt parameters from these domains. Both PPT and SPoT involve pretraining the prompts and subsequently discarding the data. We cannot obtain the gradients from the pre-training data.\n\n[1] Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper is clearly written and method is easy to implement.\n2. The empirical results are sufficient to verify the effectiveness of the method.", "reasons_to_reject": "1. More parameter efficient methods like LoRA are more commonly used in current LLMs and usually achieves better performance than prompt tuning. It would be better to examine the method's effectiveness on LoRA [1].\n2. Compared to the baselines (PPT [2], SPoT [3]), DawGen seems to introduce additional computation cost because it requires data generation for each downstream task while PPT and SPoT pre-train the prompt and applies it to various end tasks. This computational cost should be addressed.\n\n[1] LoRA: Low-Rank Adaptation of Large Language Models.\n[2] PPT: Pre-trained Prompt Tuning for Few-shot Learning.\n[3] SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer.", "questions_to_authors": "Gradient surgery seems to be orthogonal to the prompt pre-training stages. How does it perform when combined with the baselines like PPT and SPoT?", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper studies few-shot prompt tuning enhanced with data augmentation. The general idea is to use LLMs to generate synthetic data for training the soft prompts. The proposed method, Distribution-Aligned Weighted Generator Tuning (DawGen), aims to produce in-distribution synthetic data that are aligned with few-shot samples. The training procedure consists of a gradient surgery method to mitigate gradient conflicts when training the model with both real and synthetic data. The reviewers generally believe that the paper is well-motivated and clearly-written, and find the proposed method to be novel and effective. 

While the AC believes that the paper has enough merits to warrant acceptance, some of the authors' responses appear unprofessional and disrespectful toward the reviewers. One of the reviewers expressed concerns about the tone and content of the authors' response. It is natural that authors/reviewers have different opinions, but please note that the reviewers have dedicated their time and efforts to reviewing the manuscript, and it is crucial that all interactions remain respectful and professional.

[At least one review was discounted during the decision process due to quality]
Decision: Accept