Mixture of Soft Prompts for Controllable Data Generation
Authors: Derek Chen
Celine Lee
Yunan Lu
Domenic Rosati
Zhou Yu
Venue: EMNLP 2023 Findings (2023)
Keywords: data augmentation
parameter efficient training
few-shot learning
structured prediction
TL;DR: Use LLM to generate data and train smaller model, which outperforms original larger LLM.

Abstract: Large language models (LLMs) effectively generate fluent text when the target output follows natural language patterns. However, structured prediction tasks confine the output format to a limited ontology, causing even very large models to struggle since they were never trained with such restrictions in mind. The difficulty of using LLMs for direct prediction is exacerbated in few-shot learning scenarios, which commonly arise due to domain shift and resource limitations. We flip the problem on its head by leveraging the LLM as a tool for data augmentation rather than direct prediction. Our proposed Mixture of Soft Prompts (MSP) serves as a parameter-efficient procedure for generating multi-attribute data in a controlled manner. Denoising mechanisms are further applied to improve the quality of synthesized data. Automatic metrics show our method is capable of producing diverse and natural text, while preserving label semantics. Moreover, MSP achieves state-of-the-art results on three benchmarks when compared against strong baselines. Our method offers an alternate data-centric approach for applying LLMs to complex prediction tasks.

Mixture of Soft Prompts for Controllable Data Generation
Derek Chen
Celine Lee ? Yunan Lu
Domenic Rosati Zhou Yu
Columbia University,
Scite AI, Cornell University
(dc376L, y14021, zy2461 @ec.lmbii. edu, dom@scite. el 11923@cornell.es
Abstract
In-Context Learning
Task Cuary wi Exemplars
LLM
Anewer
Large language models (LLMs) effectively gen- erate fluent text when the target output follows natural language patterns. However, structured prediction tasks confine the output format to limited ontology, causing even very large mod- els to struggle since they were never trained with such restrictions in mind. The difficulty of using LLMs for direct prediction is exac- erbated in few-shot learning scenarios, which commonly arise due to domain shift and re- source limitations. We flip the problem its head by leveraging the LLM as a tool for data augmentation rather than model for di- rect prediction. Our proposed Mixture of Soft Prompts (MSP) serves as a parameter-efficient procedure for generating multi-attribute data in a controlled manner. Denoising mechanisms are further applied to improve the quality of synthesized data. Automatic metries show method is capable of producing diverse and natural text, while preserving label semantics. Moreover, MSP achieves late-of-thee art results on three benchmarks when compared against strong baselines. Our method offers alter- nate data-centric approach for applying LLMs complex prediction tasks.
Prompt Tuning
Fewehot
Prompi
Answer
(Tozon)
Our Approach (MSP)
Mixud Prampix
LLM (tnone)
Controlled Generated
Fessahot Data
Small
LM
Answer
Figure I: Standard in-context learning (top) directly prompts frozen LLM with query and exemplars. Prompt tuming (middle) trains a prompt that is then used query the LLM. MSP (bottom) learns set of soft prompts, mixes them together to generate attribute- preserving examples, then merges augmented and original data train smaller, downstream model.
can even take ad advantage of few-shot examples with- out the need for gradient updates through in-context learning (ICL) (Brown et al., 2020; Xie et al., 2022). However, off-the--hhllf LLMs have shown evidence of struggling with direct prediction in more compli- cated NLU tasks, such as those involving hierarchy or compositionality (Furrer et al., 2020; Qiu et al., 2022; Dziri et al., 2023). LLMs with ICL also ex- hibit problems when the target output requires a specific structure not represented in the training data (Reynolds and McDonell, 2021; Min et al.. 2022). Intuitively, few-shot exemplars fail to pro- vide enough signal learn custom outputs since those formats were designed for specialized tasks, and thus are unlikely have appeared in open web corpora typically used to train LLMs.
1 Introduction
Complex natural language understanding (NLU) systems, such semantic parsers, typically be- come useful only after training on copious amounts of labeled data (Chen et al., 2020). Due to the high cost of annotation, obtaining a sufficient supply data quickly becomes infeasible. Low resource settings are particularly common when expanding a system into a new domain or service (Wang et al., 2015). This task of learning a target domain from limited data is referred to as -omain-adaptive few- shot learning (Zhao et al., 2020).
Alternatively, limited data issues can also be tackled through data augmentation techniques, in- cluding altering tokens at the surface level (Wei and Zou, 2019) or mapping seed data into a latent
Modern large language models (LLMs) have emerged as effective classifiers in low resource settings (Sanh et al., 2022; Chung et al., 2022), and

---

state before generating new examples (Sennrich al., 2016). What these methods lack though is control over the generation process. Specifically, two aspects of control are critical when synthesiz- ing data: label preservation and diversity. Label preservation ensures the generated utterances re- main faithful to the original attributes in the seed data. Diversity ensures the generated utterances provide better coverage of the target distribution to guide the model toward better generalization.
goal is expand into target domain 6, given only m examples in domain t: D, (1a,, "y)"",, where m <<< n. Real life applications of NLU are further complicated by the multi-aspect nature of the target, where single label yi may be composed of multiple unique attributes {attra).
2.1 Few-shot Direct Prediction
One straightforward way to approach this problem to pre-train large neural network that is capa- ble of handling low-resource scenarios. Recently, LLMs have exhibited competitive performance in multiple few-shot tasks by using the limited data as exemplars for in-context learning (Sanh et al., 2022). However, direct prediction in this manner contains many drawbacks, such as lack of control over the prediction process. This motivates us consider an alternative framework.
To avoid the pitfalls of naive data augmentation. we take advantage of the LLM's ability to generate fluent text by leveraging it as tool for controlled data generation. Concretely, set of parameter-efficient soft prompts for each of the attributes present in the seed data. We then in- troduce novel method for combining the Mixture of Soft Prompts (MSP) to generate diverse, class- conditioned training data in a carefully controlled manner. The synthetic data is finally used to train a smaller, downstream model on the task at hand (Figure 1). Using LLMs as data generators rather than black-box predictors provides interpretability and flexibility benefits since the synthesized data can be directly inspected for quality.
start by tuning
2.2 Data-Centered Alternative
Another way to deal with limited data is to perform data augmentation where the few-shot seed data is used to produce additional training examples D syn = (fIr, y& }²)) Afterwards, all the original seed data combined with the synthesized data D. U DD U DDDDsn to train a downstream model. To the extent the downstream model is significantly smaller than the original (-15x smaller in our case), this process can be viewed as knowledge distilla- tion through data transfer.
We apply MSP to three diverse NLU tasks to test its generalizability. Compared to directly prompt- tuning an LLM with few-shot data, using the LLM to augment the data instead is capable of outper- forming a model of the same size by up to 27% (sce Table 9). We additionally compare variety competitive data augmentation and con- trolled text generation baselines where our method leads to superior downstream performance across all three benchmarks. Qualitative analysis and hu- man evaluation further verify that the data gener- ated by MSP ranks higher measures of quality, specificity and correctness (see Table 2). Over- all, our proposed method represents novel, data- centric approach for using prompt-tunce LLMs to tackle domain-ate addaiive few-shot learning.
wide
Using LLMs data augmentation tool rather than direct predictor confers multiple benefits: (a) Generated data can be inspected, which improves interpretability and explainability. (b) Supplemen- tary modifications, such as data denoising or filter- ing, can be stacked on top of the generated data, which allows for more flexibility in improving per- formance. (c) Data can be used to train smaller, more computation--fficienn models for faster infer- ence. (d) Data is model agnostic, leading to trans- ferability across model types (See Section 5.3). We take advantage of all these points our method.
2 Task Formulation
3 Mixture of Soft Prompts
Few-shot natural language understanding (NLU) can take many forms such as semantic parsing or named entity recognition. In such tasks, model aims to understand a natural language input, but only has a limited number of training examples Ty to do so. Formally, we are given a dataset D. = (fz,,y:""), with n training examples that all belong to some group of s source domains. The
3.1 Overview
Our method follows three-step process of soft- prompt tuning, data generation, and downstream model training (see Fig 2). The full prompt fed into the LLM can be broken down into four components: instruction prefix, attribute soft prompts, domain meta-data and retrieved exemplars.

---

Prompt construction
attributes during soft prompt tuning
Sarget aforunce
'your"
sesevement Teessnoe rxpouttentt & at todiore
LLM Swort
Poltucoee
reble sotiproren
meda
exeegior
"censal
Mocest Sog engeeooo
Dastee Conecoon What shownd
LLM Jone
15 coroac oud 4pnen
2. Syuthesise novel examples during data generation
Train downstream model
dedred
synthealood attrrnne
Sced
Sree 1p senteder
31, Sogeoe ture moveing
2r agacttrers
LLM Chioar
"Eoeyso YORKIT adoutiy RSA gARG UIVES.
Braill
The instruction prefix (green) and attribute soft prompts (yellow, orange, red) are initialized (top left) then tuned (top right) using few-shot data from the target domain, while keeping the LLM unchanged. Attribute soft prompts are mixed before being fed into the LLM, with training signal expressed along dotted lines. During inference (bottom), the prompt-tuned attributes are used to generate novel examples a controlled manner.
Figure 2:
3.2 Prompt Construction
attributes (Figure 2, Box la). How should these attribute embeddings be combined?
Soft prompt tuning has emerged as parameter- efficient method for leveraging the power of LLMs without the onerous computation requirements of training from scratch (Lester al., 2021). The core of our approach relies on soft prompts, but rather than tuning the prompts to make a direct prediction, we instead instruct the LLM to generate high quality training examples.
We experiment with five different methods of composing attributes for data generation. For all methods, we initialize a set of soft prompts SP for each attribute allr in the ontology of the dataset. Given a training example utterance 22 and its set of attributes yi, we compose mixed prompt Pi that combines all relevant attributes. To do so, we draw the attribute embeddings {attr_emb) C SP such that Vattr, E y,, attr __t_b__bbb__. is the attributc prompt embedding corresponding allra- Prompt composition is then performed through one of the five following attribute mixing methods.
The full input contains four parts. (1) The first is an instruction prefix initialized with the phrase "Show me three distinct utterances that all express the X" which is shared across all training examples. (2) The soft prompts are initialized with the name and description of attribute, c.g. "song is a musical song or melody", and are often domain-specific. (3) The third part includes meta-data relevant to the task such slot-values domain name. (4) Fi- nally, handful of exemplars are appended to guide the model towards better data augmentation, se- lected based on attribute overlap. (See Appendix for details.) The seed example itself is used as the target utterance for training. (Top half of Fig 2)
Suppose a seed example consists of n attributes, indexed by a. The Concat method simply concate- nates all attribute embeddings together and places the result in between the instruction prefix and the remaining input text.
P. [attr_ nattr_embi; attr attr_embo; attr [etr__emx]] (1)
key drawback of Concat is that a variable num- ber of attribute embeddings produces a variable- length input. To circumvent this inconvenience, we also test Pooling method which combines each at- tribute embedding by taking the mean value across the embedding dimensions. In doing so, the fixed output dimensions allow for easy batch processing.
3.3 Attribute Mixing
To control the characteristics of the synthesized text, we condition the model on the desired at- tributes during generation. However, prior works controlled text generation mainly focus on single attribute constraint, such as sentiment (Qian al., 2022a). In contrast, individual examples in our tasks all contain multiple attributes. For exam- plc, one task is multi multi-aspect intent detection, where a single dialogue utterance may contain three intent
1 N N
P. =
Σ
attr attr_emba
(2)
The limitation with simply averaging the embed- dings is that treats all embedding values equally. To see how we can combine information in a more

---

meaningful manner, we explore additional methods that learn weight the attribute embeddings.
Dataset
Domain
Train
Dev
Test
Hotels generic
644
66
66
The Attention mechanism method begins by av- eraging input embeddings along the embedding dimension, passing them to a feed-forward linear layer, and going through SiLU activation func- tion (Elfwing et al., 2018). Layer norm followed by temperature modulated softmax then produces attention score O;, The attention score calcu- lates a weighted sum of the attribute soft prompts, resulting in final mixed prompt.
Hotels specitic
300
34
34
NLU++
Banking generic
1594
172
172
Banking specific
238
32
32
Politics
200
541
6.51
Science
200
450
543
Music
100
380
456
CrosSNE
Literature
100
400
416
AI
100
350
431
General
15k
3.5k
3.7k
Weather
176
147
5682
TOP+2
Reminder
493
337
5767
Others
84k
12k
22k
q meanpool(attr__mm
p = SWL )²))
Table I: Number of examples included per domain in
(3)
the three NLU datasets we study, divided across splits.
a atin softmar (LN(p)
P: - a attn
through the synthesized examples and dynamically choosing which ones to keep based on two factors.
Inspired by Asai al. (2022), who soft prompts for knowledge sharing, we also test mod- ification the Attention method that introduces a Bottleneck layer into the process. More specif- ically, the averaged input embeddings are down- projected into a smaller dimension, and followed by a non-linearity before being projected back up to the original input embedding shape. This is followed by layer norm and softmax as before to calculate the attention score.
The first factor is motivated by the observation that certain attribute classes are over-represented in the seed data. Thus, sample examples at rate which is inversely proportional how often attribute occurs. In effect, this re-balances the data so all attributes have an equal chance of appearing.
The second factor aims to improve label preser- vation by lowering diversity. During inspection of preliminary results, we found that most errors came result of low correctness because the gen- erated data deviated too far away from the target label. Thus, we counteract this by keeping synthe- sized examples that are more similar to the origi- nal seed data. Similarity between synthesized and original data is measured using cosine similarity of their Sentence Transformer (Reimers and Gurevych, 2019) embeddings. We found this method of keep- ing similar synthesized examples to work better than using lower temperature during generation.
H down WT down (4)
Hup WT (SiLU (HUUHHHHHHn (oww))
up
(4)
à ettn so hoftmar LN LN (Hp))
P. g
Lastly, the CNN mixture method combines mul- tiple soft prompts via a convolutional neural net- work. We start by padding the attribute to fixed length. Then we pass the embedding through two layers of convolutions, where ReLU activation function is used between each layer.
4 Experimental Setup
Jrn pad( aaad(att_embbl jentr__emba)
4.1 Datasets and Tasks
Jcnn convi (genn)
(5)
We test on three diverse, multi-attribute natural language understanding datasets. These datasets offer pre-determined few-shot splits and natural division of source and target domains for testing.
P. comvz (ReLU
3.4
Data Denoising
As the final step in the MSP pipeline, we take ad- vantage of the fact that augmented data can be cas- ily manipulated further improve the data quality. Specifically, we over-generate 20% more data and then apply filtering to denoise the samples, bringing the number of examples back line with the origi- nal amount. Filtering is accomplished by looping
NLU++ Our first task is multi-aspect intent de- tection (Casanueva et al., 2022), where model should predict all intents present in the given dia- logue turn. Since it is possible to predict too many or too few intents, success measured by Fl score. Two topics, hotels and banking, are in the dataset,

---

with both containing generic and specific versions. This effectively yields four blocks of examples. The cross-domain setting in the paper evaluates on the generic version of each topic, so we set three blocks as the source domains (ie. hotel generie, ho- tel specific, banking specific) and leave the fourth block as the target domain (ie. banking generic). The target output is in the form of list of intents.
across the dataset, with the top-ranked candidates chosen as representative examples. The instruction prefix was manually prompt engineering across dozens of attempts to ensure fairesss. We perform chain-of-thought prompting by breaking apart the task into 3 steps. First, we ask the model to pre- dict the domain of the sentence. Next, we have the model think about what attribute types present in the utterance. Finally, we have the LLM directly predict the attribute values of the given input.
CrossNER Our second task is cross-domain named entity recognition, where the main chal- lenge is transferring from the general news domain one of five specialized domains with custom entity types (Liu et al., 2021b). For example, Poli- ties contains unique politician and election entities not found in other domains. The target output is a series of (entity category, entity value) pairs.
Data Augmentation To improve upon the vanilla student model, we augment the few-shot data with various techniques. We first try the very simple Easy Data Augmentation (EDA) (Wei and Zou, 2019) which randomly drops and swaps tokens. We also consider masked in-filling (Kobayashi, 2018) 8) that masks out certain portions of text and a BERT model to fill them back in. We also look into BART-large model (Lewis et al., 2020) trained on paraphrase corpora (Dolan and Brockett, 2005; Zhang et al., 2019). Finally, we also com- pare against round-trip translation (RTT) across five pivot languages (Sennrich et al., 2016). These techniques all generate diverse data, but may not be accurate since they have no mechanism to enforce attribute labels to appear in the synthesized data.
TOP+2 The third task we examine is composi- tional, task-oriented semantic parsing (Chen et al., 2020). The generic source data consists of six indi- vidual domains: Alarm, Event, Messaging, Navi- gation, Music, and Timer. The target domains are Reminder and Weather, whose training datasets allow only 25 SPIS (samples per intent and slot). Following the setup of the original paper, we also perform preprocessing steps to build the canonical form for prediction. The final format consists of multi-intent labels followed by slot-value pairs.
Controlled Text Generation We also test meth- ods that condition on the attributes during genera- tion to encourage label preservation. We consider Conditional LM (CLM), which fine-tunes GPT2 to produce an example utterance when given a seri- alized representation of the attribute label (Anaby- Tavor et al., 2020). Another direction performs weighted decoding of the logits during inference, where we use DExperts for constrained text gen- eration (Liu et al., 2021a). We also consider a conditional variational auto-encoder (CVAE) (Hu et al., 2017; Xia et al., 2020) that learns to generate attribute constrained outputs by sampling from the latent space between the encoder and decoder. We additionally examine a lexically base- line, Keyword2Text (K2T) where the probability distribution is shifted toward target keywords dur- ing decoding time (Pascual et al., 2021). Lastly, experiment with a prompt-tuning bascline (PT) that uses GPT-4 to generate synthetic examples. This includes an option that applies our denoising technique (PT + Denoising) before training on the downstream task. All rhese techniques exhibit a stricter adherence to labels, but may lack diversity.
4.2 Baseline Methods
Direct Prediction We use FLAN-T5 XXL (Chung et al., 2022) as our base model for gen- erating data. GODEL (Peng et al., 2022) serves as our smaller downstream LLM, which starts with T5 backbone (Raffel et al., 2020) and is fine-tuned on dialogue related data, At just 770M parameters, the student model contains roughly 15 times fewer parameters than the teacher. For a fair comparison, we use the exact same model again to make direct predictions with a wrompt-tuning-owe setup.
We also compare against billion-parameter mod- els optimized through in-context learning and thain-of-thought prompting (Wei al., 2022), namely GPT-3.5-turbo and GPT-4 The in-context learning prompt consists of four components: an instruction prefix; a comprehensive list of domain- specific attributes; relevant meta-data; and five question-answe exemplars. The exemplars are selected based on the frequency of their attributes
'Due to the high cost of using GPT-4, we run experiments for just one domain per datasct.

---

4.3 Automatic Evaluation
Quality
Specificity
Accuracy
Beyond downstream accuracy, we also evaluate the synthesized data quantitatively with three metrics. Distinct@& K mcasures the diversity of text based on unique n-grams, where we set k=1,2,3 follow- ing common practice (Li et al., 2016). Perplexity represents text fluency, which we measure through GPT2-large (Radford et al., 2019). Third, we use Correctness check how well the synthetic data preserves the proper attribute labels. To do so, we train an oracle with all available data (ie. no longer few-shot) to classify the primary attributes within an utterance. (More details in Appendix A)
Paraphrase CLM
3.4
3.1
4.4
3.6
3.3
4.2
MSP
4.4
4.3
4.7
Table 2: Human evaluation results comparing three methods a 5-point Likert scale. MSP ranks high- est across all metrics with average Fleiss A 0.72.
2023), direct prediction yields worse performance than MSP. On the other hand, by leveraging LLMs for data augmentation, our method successfully leans into the innate strengths of language models as text generators rather than forcing them to learn specialized target output sequences.
4.4 Implementation Details
Compared to the data augmentation baselines, MSP consistently leads to better results across all three datasets. Qualitatively, we observe that all four naive augmentation methods produce exam- ples which ultimately deviate away from the de- sired semantic attributes. This causes a degrada- tion in downstream performance compared to MSP (See Table 11).
The instruction prefix is set to length of 100 to- kens, while the attribute token length is set to 20 tokens. After hyper-param tuning,
settle 3e-2 as the learning rate for the teacher model and 3e-5 for the student (See Appendix B). Augmen- tation methods are standardized to generate four new datapoints per seed example.
5 Results and Analysis
While the controlled text generation (CTG) methods able to outperform the naive GODEL baseline, CTG underperforms the same GODEL model by an average of 10% when augmented with MSP synthesized data. This performance trend is reflected even when using GPT-4 for controlled text generation, as shown in Table 4. Qualitative anal- ysis reveals the CTG methods are unable to pick up the complex structure required by the multi- attribute generation task, this synthesized data ends up noise and actually hurts performance. On the other hand, MSP is able to reliably handle lexical, semantic and structural constraints.
5.1 Main Results
As seen in Table 3, MSP w/ Bottleneck achieves state-of-the-art results across all three datasets, with an average 20.3% improvement over the origi- nal baselines. MSP reaches the highest end-to-end scores on 8 out of 9 possible domains, while also demonstrating competitive performance the one remaining domain (ie. TOPv2 Reminder). Notably, the one area where MSP does not achieve the high- est rank, it is actually surpassed by meta-learning technique. However, meta- meta-learning and data aug- mentation are orthogonal methods, so in practice. these two can and should be combined to produce even better results than either alone.
5.2 Synthesized Data Quality
For human evaluation, we surveyed 30 fluent En- glish speakers who reviewed the generated utter- ances according to given metries: (1) Quality the text grammatically correct, coherent and fluent. (2) Specificity the text is specific to the target topic, rather than generically applicable. (3) Accu- racy the text correctly reflects the desired semantic attributes. The results can be seen in Table 2. Test- ing against the top DA method (Paraphrase) and the top CTG method (CLM), our method (MSP) ranks highest across all metrics, with particularly large improvements in Specificity.
Despite the remarkable ability of large pre- trained LMs to generate coherent and fluent lan- guage, their capacity to produce structured outputs is limited. In particular, performance from direct prediction models deteriorate when dealing with utterances featuring more complex structures and diverse attributes, such as TOPv2. Leveraging a model brings marginal improve- ment, but performance is still well below data syn- thesis baselines. As seen in Table 4, even when the LLM is scaled to the size of GPT-4 (OpenAl,
Beyond downstream accuracy, we also use auto- matic evaluation to judge the quality of the synthe-
"Our code and further implementation details can he found at https://gihhu..oo/ddeeeechh 4/mixture, softtttueeessst___eeeeee prompts.

---

Method
Hotels
Banking
Politics
Science
Music
Literature
AT
Reminder
Weather
LSTM-based
67.3
49.2
61.5
52.1
51.7
48.4
45.2
45.8
65.1
Original Baselines
(Ro)BERTa based
79.3
74.2
68.7
69.4
68.3
63.6
58.9
63.7
76.0
Transformer-
75.4
65.2
70.4
66.8
72.1
67.1
60.3
70.5*
77.7*
Few-Shot (GODEL)
74.5
64.2
76.7
72.0
75.8
69.2
64.7
60.5
77.1
Direet Prediction
ICL (GPT-3,5-turbo)
52.4
52.1
54.4
57.2
67.2
52.9
48.4
39.0
69.6
ICL (FLAN-TS)
60.5
27.6
26.8
55.1
Prompt-tune (GPT-1-6B)
68.2
59.5
63.9
52.4
62.9
62.9
52.3
49.5
69.7
Prompt-tur tune (FLAN-T5)
81.2
69.2
71.6
68.0
69.3
58.3
57.1
50.6
72.9
Easy Data Augmentation
81.8
68.0
77.3
72.1
76.3
71.0
65.4
62.5
80.6
Dula
Round Trip Translation
77.3
71.2
75.3
71.0
66.3
63.0
54.8
Augmentation
Paraphrasing (BART)
76.6
64.9
79.9
72.6
73.8
71.0
65.7
61.0
81.7
Masked In-Fill
Conditional
76.8
71.5
75.7
73.9
76.2
67.8
Controlled
DExperts
83.1
71.8
78.2
71.4
72.6
66.3
63.6
50.9
69.7
Text Generation
Conditional VAE
77.0
70.4
74.0
69.6
72.9
67.9
64.3
46.0
69.1
Keyword2Text
74.8
67.8
74.8
72.4
75.3
68.2
62.3
64.0
81.0
MSP w/ Concat
83.5
84.2
77.3
72.8
71.2
66.3
62.7
62.2
MSP w/ Pooling
83.8
82.9
80.6
73.0
79.0
71.9
66.7
62.8
83.9
Our Method
Altention
86.5
73.9
65.7
MSP w/ Bottlensck
89.7
84.6
79.5
74.8
80.0
69.6
65.9
65.1
84.6
MSP w/ Convolution
85.0
80.3
80.5
73.6
78.7
67.8
65.0
61.0
83.2
Table End-to-end Fl-scorss for NLU++ CrosSNER, and TOPv2. OOriginll baseline scores are aggregated from previous works (See Section 4.1). BBART Copy-Ptr with meta-learning. For exact model types, please refer the appendix. Different MSP mixtures achieve state-of-the-ee in all domains except one.
Hotels
Music
Weather
prompt consistently leads the lowest scores. As expected, removing any part of the trainable soft prompts leads to substantial degradation.
Dircct
ICL CoT
67.7
68.4
72.1
Prediction
75.4
66.5
67.9
Controlled
PT only
76.9
72.3
75.8
The last row in Table includes one final tweak to the method which swaps out the Flan-T5 back- bone for GPT-J-6B (Wang and Komatsuzaki, 2021). This change from patteencc-to-squemee model to causal language model highlights the flexi- bility of our approach since the transferrability of data is agnostic to its provenance. Although the downstream model trained with GPT-augmented data is not as strong as the model trained with T5 augmented data, it does clearly outperform the GPT-J-6B model performing direct prediction.
Text Gen
PT denoise
77.1
70.3
78.6
Table 4: GPT-4 underperforms MSP for NLU++ Hotels, CrossNER Music, and TOPv2 Weather domains. The first rows show direct prediction using in-context learning and prompt-tuning
GPT-4. The latter two rows show end-to-end scores using GPT-4 to generate additional training data. PT is short for Prompt-tunin
sized data. DA methods have lower linguistic qual- ity, but achieve higher attribute conservation since changing a few tokens can easily harm readability, but usually do not change the overall meaning. In contrast, CTG methods generally exhibit high flu- ency, but their low correctness scores also reveal a difficulty in capturing all the desired semantic attributes. Table 6 shows that MSP generated data strikes balance between diversity and correctness, without the need for any manual tuning.
Related Work
Our paper tackles multi-aspec few-shot learning where the target is structured output contain- ing multiple attributes, and the model only has a few examples from which to learn such pattern. Given the complexity of the task, previous research trained custom models for each dataset (Zheng et al., 2022; Schucher et al., 2022). Instead, we leverage the power of LLMs (Brown al., 2020; Chung et al., 2022) to design a more general solu- tion through controlled data synthesis.
5.3 Ablations
In order to understand the impact of each part in the MSP pipeline, we run ablations along one domain from each dataset. Results in Table 7 show that all parts of our technique provide meaningful gains. particular, the instruction prefix tells the model to generate similar examples, and removing this
In particular, compose multiple Soft Prompts to generate the desired training data for the down- stream task. Consequently, we build upon foun- dational work studying soft prompt tuning (Lester

---

Dataset
Attributes and Meta-data
Generated Text
NLU++
Intents: "change', 'booking'
Original: change booking
Domain: Hotels
Generated: I have reservation that need modify
CrosSNER
Entity categories: 'artist':
Original: Chuck Burgi 1991-1992, 1992-1995, 1996-1997 ). John Miceli ( [1992, 1995 ) John O 'Reilly ( 1995-1996) ) and Bobby Rondinelli ( 1197--
Entities: 'John O "Reilly",
"Bobby Rondinelli", 'John
2004 )
Miceli", 'Chuck Burgi'
Generated: He was followed as musical artist by Chuck Burgi 11992-1995,
Domain: Music
1996-1997] ). John Miceli 1992, 1995 ). John O 'Reilly ( 1996 ) and
Bobby Rondinelli ( 1997-2004
TOPv2
Intents:
"help_ reminder'
Original: How does the reminder notification sound when it plays out loud!
Domain: Reminder
Generated: Set a reminder for my dentist appt
Table 5: Qualitative examples of synthesized data applying our method (MSP) different datasets. The generated utterances typically demonstrate the preservation of the desired semantic attributes and lexical entities of the target domain, as seen in NLU++ and CrossNER. We select an example from TOPv2 to highlight where MSP struggles.
NLU+
CrossNER
TOPv2
Distinct@ 1/2/3
Conrect.
Perpl.
Distinct@ 1/2/3
Conrect.
Perpl.
Distinct@ 1/2/3
Correct.
Perpl.
EDA
8.80 69.7
94.8
1.33
/9.8/53.6
68.9
81.3
1.07
12.3/47.7
70.7
73.0
1.45
KIT
7,90 29.4 229.994444. 48,9
94.7
1.24
/ 42266
81.2
1.05
11.4/33.8 /53.3
75.5
1.34
Paruphrase In-Fill
2.40/ 36.8
92.2
1.23
12.8/26.1 31.4
89,9
1.06
9.55 /40.6
76.0
1.31
5.45/23.5 / /3.88
93.8
1.36
10.2/ 29.1
91.9
1.06
9.15 /41.1
69.8
1.54
CLM
2.00/5.90/
76.8
1.27
11.9
29.77
86.4
1.07
7.05 18.9 29.7
79.4
1.43
DExperts
4..0/3.1//14
53.4
1.15
7.74/2166
30.1
22.9
1.03
/4.85 /7.10
62.3
1.27
CVAE
0,25 2.22
19.2
2.00
53.9/7.0// 81.6
68.2
1.09
5.10 16.2 /2.11
71.5
1.45
MSP
1.90/ /3..4
98.0
1.27
/ 40.7
93.6
1.06
7.19 /33.9
74.8
1.43
Table 6: Automatic evaluation results with Distinct @ measuring diversity, Correctness measuring label preserva- tion and Perplexity representing text fluency. Correct and Perpl are short for correctness and perplexity, respectively.
Hotels
Literature
Reminder
latent perturbation (Sennrich et al., 2016; Fabius et al., 2015) or auxiliary supervision (Chen and Yu, 2021). Our method can be considered form of text generation with transformers (Kumar et al., 2020; Ng et al.. 2020), which lately rely on increasingly larger language models (Yoo et al., 2021; Wang et al., 2021a,b). Whereas these methods paraphrase or pseudo-label seed utterance, MSP instead con- ditions on label control the generation of text.
MSP w/ Bottleneck
89.7
68.0
65.1
data denoising
88.5
67.1
64.9
instruction
82.2
63.4
62.4
meta-data
89.4
67.3
64.2
attribute prompt
87.6
64.1
63.8
exemplars
88.8
65.5
62.2
-PP---6B
84.5
63.9
63.7
Table 7: Ablation studies about the impact individual components in the pipeline of MSP on the downstream task performance. The first row is the haseline with all components under bottleneck mixing method. The (-) sign indicates the absence of a specific step.
As result, our method is also related to Con- trolled Text Generation techniques. Unlike con- strained lexical decoding (Pascual et al., 2021), which aims to produce text that contains a pre- specified set of keywords, our work is focused on controlling the semantics of the output, such as a topic or user intent (Mou et al., 2016; Hokamp and Liu, 2017: Post and Vilar, 2018: Yao et al., 2019). For semantic control, wide range of op- tions exist for guiding generation towards single attribute, including those that train model from scratch (Keskar al., 2019; Wang al., 2019) those which only tune few parameters (Ribeiro et al., 2021; Lin al., 2021; Yu et al., 2021; Liu et al., 2023). There are even methods that keep the base model frozen and instead manipulate the
et al., 2021; Vu et al., 2022), well as other param- eter efficient fine-tuning methods (Houlsby al., 2019; Li and Liang, 2021). Alternatively, Wang al. (2022) and Chen et al. (2022) perform data augmentation with prompting, but their prompts are not compositional since their task setups are focused on single-aspect class prediction.
Data Augmentation is a common technique in NLP for counteracting the limited data available with few-shot learning (Feng et al., 2021; Chen and Yin, 2022). Flavors of data augmentation in- clude surface form alteration (Wei and Zou, 2019),

---

logits with weighted decoding control the out- put (Dathathri et al., 2020; Krause et al., 2021; Yang and Klein, 2021; Zhang and Song, 2022). These methods can stay on topic, but often sacri- fice the ability to generate specific tokens, while MSP is able to maintain both semantic and lexical control, yielding superior results.
Lastly, MSP is related to techniques that com- bine multiple prompts together. Nayak et al. (2022) propose combining soft prompts through concate- nation, but their aim is improve direct prediction vision-based task, as opposed to data gener- ation for NLU tasks. Qin and Eisner (2021) tar- get classification tasks using pre-trained LMs, but their mixture-of-expeer technique selects individ- ual prompts from multiple candidates to satisfy single constraint, rather than mixing multiple prompts meet multiple constraints. Our work is most similar to those that perform multi-aspect text generation (Yang et al.. 2022; Gu et al., 2022: Qian et al., 2022b). However, the primary aim of improving text quality aligns with our secondary aim (Subsection 5.2). Whereas these prior efforts focus exclusively on text generation, our method controls generation means to end.
7 Conclusion
Our paper presents an alternative method for few- shot learning using LLMs as an intermediate data augmentation tool rather than for direct prediction. By performing data generation an intermediate step for improved end-to-end task performance, method yields such benefits interpretability, ilexibility and modularity. Compared against other strong data augmentation methods, we show that MSP yields higher quality data that can effec- tively used to improve performance downstream tasks. This crrammteer-fffcieee method to perform controlled data generation is powerful paradigm for using LLMs in low- resource scenarios; the posi- tive results from the methods proposed in this work suggest promising future work in exploring tighter controls and smarter filtering mechanisms for data augmentation. Ultimately, we encourage others to consider use LLMs as tools for generating data, rather than only for generating direct predictions.

---

8 Limitations
Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Ma- teusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Nya Sutskever, and Dario Amodei, 2020. Language models are few-shot learners. In Ad- vances in Neural Information Processing Systems. volume 33, pages 1877-1901. Curran Associates. Inc.
The largest model we could feasibly access is a T5- XXL which contains 11 billion parameters. While we did test against GPT-3.5 and GPT-4, it is en- tirely feasible that GPT5 (unknown size) OPT3 (175 billion), PALM model (540 billion) may outperform our results in these few-shot settings using prompt-tuning with exemplars. However, we would posit that as the ability of these truly gigantic models improve, their ability to generate superior training data would also improve in con- cert, so our method would still be worthwhile. Evi- dence for this hypothesis is seen from the transition from GPT-J-6B to T5-XXL. which leads to bet- ter prompt-tuning results along with better MSP results. However, we cannot know for sure at the moment without access to more compute resources,
Inigo Casanueva, Ivan Vulié, Georgios Spithourakis. and Pawel Budzianowski. 2022. NLU++: A multi- label, slot-rich, generalisable dataset for natural lan- guage understanding in task-oriented dialogue. In Findings of the Association for Computational Lin- guistics; NAACI. 2022, pages 1998-2013, Seattle, United States. Association for Computational Lin- guistics.
Derek Chen and Claire Yin. 2022. Data augmentation for intent elassification. CoRR, abs/2206.05790
The other major limitation of our work is the lack of a clear optimization target during data gen- eration. We used BLEU score of the synthesized example compared to the original seed example as a proxy for measuring model convergence. How- ever, it turns out that achieving a higher BLEU score during MSP training does not always trans- late to superior downstream results. Ideally, we would be able to directly leverage the downstream accuracy as a training signal back to optimizing the MSP model, which we leave to as future work.
Derek Chen and Zhou Yu. 2021. GOLD: Improving out-of-scope detection in dialogues using data aug- mentation. In Proceedings of the 2021 Conference Empirical Methods in Natural Language Processing, pages 429-442, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Maximiliaa Chen, Alexandros Papangelis, Chenyang Tao, Andy Rosenbaum, Seokhwan Kim, Yang Liu, Zhou Yu, and Dilek Hakkani- -urr. 2022. Weakly supervised data augmentation through prompting for dialogue understanding. CORR, 1tss/2211.11199
Xilun Chen, Asish Ghoshal, Yashar Mehdad, Luke Zettlemoyer, and Sonal Gupta. 2020. Low-resource domain adaptation for compositional task-oriented semantic parsing, In Proceedings of the 2020 Con- ference on Empirical Methods in Natural Language Processing (EMNLP), pages 5090-5100, Online. As- sociation for Computational Linguisties.
References
Ateret Anaby-Taavor Boaz Carmeli, Esther Goldbraich, Amir Kantor, George Kour. Segev Shlomov, Naama Tepper, and Naama Zwerdling. 2020. Do not have enough data? deep learning to the rescue! In The Thirty-Fourth AAAI Conference on Artificial Intelli- gence, AAAI 2020, The Thirty-Second Innovative Ap- plications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY. USA, February 7-12, 2020, pages 7383- 7390. AAAI Press.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang. Mostafa Dehghani, Siddhartha Brahma, Albert Web- son, Shixiang Shane Gu, Zhuyun Dai, Mirac Suz- gun, Xinyun Chen, Aakanksha Chowdhery. Sharan Narang. Gaurav Mishra, Adams Yu, Vincent Y. Zhao, Yanping Huang, Andrew M. Dai, Hongkun Yu. Slav Petrov, Ed H. Chi. Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoe V. Le, and Jason Wei. 2022. Scaling instruction-finetume language models. CoRR, abs/2210.1 1111111.11116 11416.
Akari Asai, Mohammadreza Salehi, Matthew F. Pe- ters, and Hannaneh Hajishirzi. 2022. ATTEMPT: parameter-efficieet mullii-task tuning via attentional mixtures of soft prompts. In Proceedings of the 2022 Conference on Empirical Methods Natural Lan- guage Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates. December 7-11, 2022, pages 6655 6672. Association for Computational Linguistics.
Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung. Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2020. Plug and play language models: A simple approach to controlled text generation. In 8th International Conference on Learning Represen- tations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss Voss,
William Dolan and Chris Brockett. 2005. Automati- cally constructing a corpus of sentential paraphrases.

---

In Proceedings of the Third International Workshop on Paraphrasing (IWP2005).
Edward J Hu. yelong shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang. Lu Wang. and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.
Nouha Dziri Ximing Lu, Melanie Sclar, Xiang Lorraine Li, Liwei Jiang. Bill Yuchen Lin, Peter West, Chan- dra Bhagavatula, Ronan Le Bras, Jena D. Hwang. Soumya Sanyal, Sean Welleck, Xiang Ren, Allyson Ettinger. Zaid Harchaoui, and Yejin Choi. 2023. Faith and fate: Limits of transformers on compo- sitionality. CoRR, abs/2305. 18654.
Zhiting Hu, Zichao Yang. Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. 2017. Toward con- trolled generation of text.
Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learn- ing Research, pages 1587-1596. PMLR.
Stefan Elfwing. Eiji Uchibe, and Kenji Doya. 2018. Sigmoid Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Neural Networks, 107:3-11.
Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong, and Richard Socher. 2019. CTRL: A conditional transformer language model for con- trollable generation. CoRR, abs/ htt//999.05858.
Outo Fabius, Joost R. van Amersfoort, and Diederik P. Kingma. 2015. Variational recurrent auto-encoders. In 3rd International Conference Learning Repre- sentations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Workshop Track Proceedings.
Sosuke Kobayashi. 2018. Contextual augmentation: Data augmentation by words with paradigmatic re- lations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguisticx: Human Language Tech- nologies, Volume 2 (Short Papers), pages 452-457, New Orleans, Louisiana. Association for Computa- tional Linguisties.
Steven Y. Feng. Varun Gangal, Jason Wei, Sarath Chan- dar, Soroush Vosoughi, Teruko Mitamura, and Ed- uard Hovy. 2021. A survey of data augmentation approaches for NLP. In Findings of the Association for Computational Linguistics: ACI-IJCNLP 2021, pages 968-988, Online. Association for Computa- tional Linguistics.
Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann, Nitish Shirish Keskar, Shafiq Joty, Richard Socher, and Nazneen Fatema Rajani. 2021. GeDi: Gener- ative discriminator guided sequence generation. In Findings of the Association for Computational Lin- guistics: EMNLP 2021, pages 4929-4952, Punta Cana, Dominican Republic. Association for Compu- tational Linguistics.
Daniel Furrer, Marc van Zee, Nathan Scales, and Nathanael Schärli. 2020. Compositional generaliza- tion in semantic parsing: Pre-training vs. specialized architectures. CoRR, abs/2007.08970.
Yuxuan Gu, Xiaocheng Feng. Sicheng Ma, Lingyuan Zhang, Heng Gong, and Bing Qin. 2022. A distribu- tional lens for multi-aspect controllable text genera- tion. CoRR, abs/2210.02889.
Varun Kumar, Ashutosh Choudhary, and Bunah Cho. 2020. Data augmentation using pre-trained trans- former models. In Proceedings of the 2nd Workshop Life-long Learning for Spoken Language Systems, pages 18-26, Suzhou, China. Association for Com- putational Linguistics.
Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. Deberta: decodding-enannced bert with disentangled attention. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenRe- view.net.
Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficie prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3045-3055 Online and Punta Cana, Domini- can Republic. Association for Computational Lin- guistics.
Chris Hokamp and Qun Liu. 2017. Lexically con- strained decoding for sequence generation using grid beam search. In Proceedings of the 55th Annual Meeting of the Association for Computational Lin- guistics ( (Volume I: Long Papers), pages 1535-1546, Vancouver, Canada. Association for Computational Linguisties.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad. AbdeIrahman Mohamed, Omer Levy. Veselin Stoyanov. and Luke Zettlemoyer. 2020. BART: Denoising for natural language generation, translation, and com- prehension. In Proceedings of the 58th Anmual Meet- ing of the Association for Computational Lànguistics, pages 7871-7880, Online, Association for Computa- tional Linguistics.
pre-training
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Ges- mundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter- efficieen transfer learning for NLP. In Pro- ceedings of the 36th International Conference on Ma- chine Learning, ICML. 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pages 2790-2799 PMLR.
Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016. A diversity- promottng ob- jective function for neural conversation models. In

---

NAACL 2016, The 2016 Conference of the North American Chapter of the Association for Computa- tional I Linguistics: Human Language Technologies, San Diego California, USA, June 12-17, 2016, pages 110-119. The Association for Computational Lin- guistics.
Lili Mou, Yiping Song, Rui Yan, Ge Li, Lu Zhang, and Zhi Jin. 2016. Sequence to backward and forward sequences: A content-introducing approach to gen- erative short-text conversation. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistic:: Technicul Papers, pages 3349-3358, Osaka, Japan. The COLING 2016 Orga- nizing Committee.
Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Asso- ciation for Computational Linguisties and the 11th International Joint Conference on Natural Language Processing (Volume I: Long Papers), pages 4582- 4597, Online. Association for Computational Lin- guistics.
Nihal V. Nayak. Peilin Yu, and Stephen H. Bach. 2022. Learning compose soft prompts for compositional zero-shot learning. CoRR, abs/2204.03574.
Nathan Ng, Kyunghyun Cho, and Marzych Ghassemi. 2020. SSMBA: Self-supervised manifold based data augmentation for improving out-of-domain robust- ness. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1268-1283, Online, Association for Computational Linguistics.
Zhaojiang Lin, Andrea Madotto, Yejin Bang, and Pas- cale Fung. 2021. The adapter-bot, All Ale-in-one con- trollable conversational model. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Fleventh Symposium on Educational Advances in Artificial In- telligence, EAAI 2021, Virtual Event, February 2-9, 2021, pages 16081-188000 166833 AAAI Press.
OpenAL. 2023. Gpt-4 technical report.
Damian Pascual, Beni Egressy, Clara Meister, Ryan Cotterell, and Roger Wattenhofer. 2021. A plug-and- play method for controlled text generation. In Find- ings of the Association for Computational Linguis- rics: EMNLP 2021. Association for Computational Linguistics.
Alisa Liu, Maarten Sap. Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith, and Yejin Choi. 2021a. DExperts: Decoding-time controlled text generation with experts and anti- experts. In Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics and the I 11th International Joint Conference on Natu- ral Language Processing (Volume I: Long Papers), pages 6691-67000 Online. Association for Computa- tional Linguistics.
Baolin Peng, Michel Galley, Pengcheng He, Chris Brockett, Lars Liden, Elnaz Nouri, Zhou Yu, Bill Dolan, and Jianfeng Gao. 2022. Godel: Large-scale pre-training for goal-directed dialog. arXiv.
Matt Post and David Vilar. 2018 Fast lexically con- strained decoding with dynamic beam allocation for neural machine translation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, Volume I (Long Pa- pers), pages 1314-1324, New Orleans, Louisiana. Association for Computational Linguistics.
Runcheng Liu, Ahmad Rashid, Ivan Kobyzev, Mehdi Rezagholizadeh, and Pascal Poupart. 2023. Attribute controlled dialogue prompting. In Findings of the As- sociation for Computational Linguistics: ACL 2023, pages 2380-2389, Toronto, Canada. Association for Computational Linguistics.
Jing Qian, Li Dong, Yelong Shen, Furu Wei, and Weizhu Chen. 2022a. Controllable natural language genera- tion with contrastive prefixes. In Findings of the As- sociation for Computational Linguistics: ACL 2022. pages 2912-2924, Dublin, Ireland, Association for Computational Linguistics.
Zihan Liu, Yan Xu, Tiezheng Yu, Wenliang Dai, Zi- wei Ji. Samuel Cahyawijaya, Andrea Madotto, and Pascale Fung. 202 Crossner: Evaluating cross- domain named entity recognition. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Thire Conference on Innovative Ap- plications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Ar- tificial Intelligence, EAAl 2021, Virtual Event, Febru- ary 2-9, 2021, pages 13452-13460. AAAI Press.
Jing Qian. Li Dong, Yelong Shen, Furu Wei, and Weizhu Chen. 2022b. Controllable natural language genera- tion with contrastive prefixes. In Findings of the As- sociation for Compitational Linguistics: ACL 2022. pages 2912-2924 Dublin, Ireland. Association for Computational Linguistics.
Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannanch Hajishirzi, and Luke Zettle- moyer. 2022. Rethinking the role of demonstrations: What makes in-context learning work? In Proceed- ings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1 11048-1 1188-110 1064,
Guanghui Qin and Jason Eisner. 2021. Learning how to ask: Querying LMs with mixtures of soft prompts. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- rional Linguistics: Human Language Technologies, pages 5203-5212, Online, Association for Computa- tional Linguistics.
Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.

---

Linlu Qiu, Peter Shaw, Panupong Pasupat, Tianze Shi, Jonathan Herzig, Emily Pitler, Fei Sha, and Kristina Toutanova. 2022. Evaluating the impact of model scale for compositional generalization in semantic parsing. CoRR, abs/2205.12255
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Anmual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Pupers), pages 1715-1725, Berlin, Germany. Association for Computational Lin- guistics.
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners.
Burr Settles. 2009, Active learning literature survey.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee. Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text trans- former. J. Mach. Learn. Res., 31:140:1-140:7;
Tu Vu, Brian Lester, Noah Constant, Rami Al-Rfou', and Daniel Cer. 2022. SPoT: Better frozen model adaptation through soft prompt transfer. Proceed- ings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Pa- pers), pages 5039-5059, Dublin, Ireland. Association for Computational Linguistics.
Nils Reimers and Iryna Gurevych. 2019. Sentence- BERT: Sentence embeddings using Siamese BERT- networks. In Proceedings of the 2019 Conference on Empirical Methods Natural Language Processing and the 9th International Joint Conference on Natu- ral Language Processing (EMNLP-IJCNLA) pages 3982-3992, Hong Kong. China. Association for Com- putational Linguistics.
Ben Wang and Aran Komatsuzaki. 2021. GPT-J-6B: A Billion Parameter Autoregressive Language Model. https://githut com/kingoflolz mosb-transttrmmrr-
Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu, and Michael Zeng. 2021a. Want to reduce la- beling cost? GPT-3 can help. In Findings of the Association for Computational Linguisties: EMNLP 2021, pages 4195 4205, Punta Cana, Dominican Re- public. Association for Computational Linguistics.
Laria Reynolds and Kyle McDonell. 2021. Prompt programming for large language models: Beyond the few-shot paradigm. In CHI 21: CHI Conference on Human Factors in Computing Systems, Virtual Event / Yokohama Japan, May 8-13, 2021, Extended Abstracts, pages 314:1-314:7. ACM.
Wenlin Wang. Zhe Gan, Hongteng Xu, Ruiyi Zhang. Guoyin Wang. Dinghan Shen, Changyou Chen, and Lawrence Carin. 2019. Topic-guided variational auto-encoder for text generation. In Proceedings of the 2019 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Papers), pages 166-177, Minneapolis, Min- nesota. Association for Computational Linguistics.
Leonardo F. R. Ribeiro, Yue Zhang, and Iryna Gurevych. 2021. Structural adapters in pretrained language models for AMR-to-Text generation. In Proceed- ings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4269-4282 Online and Punta Cana, Dominican Republic. Asso- ciation for Computational Linguistics.
Yufei Wang. Can Xu, Qingfeng Sun. Huang Hu, Chongyang Tao. Xiubo Geng, and Daxin Jiang. 2022. PromDA: Prompt-based data augmentation for low- resource NLU tasks. In Proceedings of the 60th An- nual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 4242- 4255, Dublin, Ireland. Association for Computational Linguistics.
Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey. M. Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szezechla, Taewoon Kim. Gunjan Chhablani, Nihal V. Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Mattco Manica, Sheng Shen, Zheng Xin Yong. Harshit Pandey, Rachel Bawden, Thomas Wang, Tr- ishala Neeraj. Jos Rozen, Abheesht Sharma, An- drea Santilli, Thibault Févry, Jason Alan Fries, Ryan Techan, Teven Le Scao, Stella Biderman, Leo Gao, Thomas Wolf, and Alexander M. Rush. 2022. Multi- task prompted training enables zero-shot task gener- alization. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event. April 25-29, 2022. OpenReview.net.
Yushi Wang. Jonathan Berant, and Percy Liang. 2015. Building semantic parser overnight. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Pro- cessing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers, pages 1332--142 The As- sociation for Computer Linguistics.
Nathan Schucher, Siva Reddy, and Harm de Vries. 2022. The power of prompt tuning for low- new-resoure seman- tic parsing. In Proceedings of the 60th Annual Meet- ing of the Association for Computational Linguistics (Volume Short Papers). pages 148-156, Dublin, Ireland. Association for Computational Linguistics.
Zirui Wang, Adams Wei Yu, Orhan Firat, and Yuan Cao. 2021b. Towards zero-label language learning. CoRR, abs/2109.09193.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoe V Le,

---

and Denny Zhou. 2022. Chain-of-thought prompt- ing elicits reasoning in large language models. In Advances in Neural Information Processing Systems, volume 35, pages 24824724837. Inc.
Hanqing Zhang and Dawei Song. 2022. Dis- cup: Discriminator cooperative unlikelihood prompt- tuning for controllable text generation. CoRR,
Curran Associates,
abs/2210.09551.
Yuan Zhang, Jason Baldridge, and Luheng He. 2019. PAWS: Paraphrase Adversaries from Word Scram- bling. In Proc. of NAACI.
Jason Wei and Kai Zou. 2019. EDA: Easy data augmen- tation techniques for boosting performance on text classification tasks. In Proceedings of the 2019 Con- ference Empirical Methods in Natural Language Processing and the 9th International Joint Confer- ence on Natural Language Processing (EMNLP. IJCNLP), pages 6382-6388, Hong Kong, China. As- sociation for Computational Linguistics.
An Zhao, Mingyu Ding, Zhiwu Lu, Tao Xiang, Yulei Niu, Jiechao Guan, rong Wen, and Ping Luo. 2020. Domain-adaptive few-shot learning. 2021 IEEE Win- Conference Applications of Computer Vision (WACV), pages 1389-13988
Junhao Zheng, Haibin Chen, Qianli Ma. 2022. Cross-domain named entity recognition via graph matching. In Findings of the Association for Com- putatiioal Linguistics: ACL 2022, pages 2670-2680, Dublin, Ireland. Association for Computational Lin- guistics.
Congying Xia, Chenwei Zhang. Hoang Nguyen, Jiawei Zhang, and Philip S. Yu. 2020. Cg-bert: Conditional text generation with bert for generalized few-shot intent detection. ArXiv, abs/2004.01881.
Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. 2022. An explanation of in-context learning as implicit bayesian inference. Interna- tional Conference on Learning Representations.
Kevin Yang and Dan Klein. 2021. FUDGE: Controlled text generation with future discriminators. In Pro- ceedings of the 2021 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 351 1-3535, Online. Association for Computational Linguistics.
Kexin Yang. Dayiheng Liu, Wenqiang Lei, Baosong Yang. Mingfeng Xue, Boxing Chen, and Jun Xie. 2022. Tailor: A prompt-based approach to attribute-hased controlled text generation. CoRR, abs/2204.13362.
Lili Yao. Nanyun Peng, Ralph M. Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. 2019. Plan- and-write: Towards better automatic storytelling. In The Thirty-Third AAAI Conference on Artificial Intel- ligence, AAAI 2019, The Thirty-First Innovative Ap- plications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAT 2019, Hon- olulu, Hawait, USA, January 27 February 1, 2019, pages 7378-7385. AAAI Press.
Kang Min Yoo, Dongju Park, Jaewook Kang, Sang-Wor Woo Lee, and Woomyoung Park. 2021. GPT3Mix: Lever- aging large-scale language models for text augmen- tation. Findings of the Asxociation for Computa- tional Linguistics: EMNLP 2021, pages 2225-2239, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Dian Yu, Zhou Yu, and Kenji Sagae. 2021. Attribute alignment: Controlling text generation from pre- trained language models. In Findings of the Associ- ation for Computarional Linguistics: EMNLP 2021, pages
Punta Cana, Dominican Republic. Association for Computational Linguistics.

---

A Oracle Correctness Classifier
C Generalization to Out-of-Domain
In order to perform automatic evaluation on correct- ness, we train an oracle attribute classifier based on LeBERTa-XLarge (He et al., 2021). However, there is chicken -and-eg problem since the goal of the downstream task also to predict attributes. To get around this issue, we make three key simpli- fying assumptions. To begin, we use all available data for training, rather than limiting to the few- shot setting. For example, we go from 25 SPIs to over 1000 SPIs on the TOPv2 dataset. Since this classifier is meant to operate an an oracle rather than display of model capability, we even include examples from the development set for training. Secondly, we only focus on the main attribute (in- tent) rather than recond-order details (slots). Fi- nally, we simplify the attribute prediction task by classifying each attribute individually, rather than in a compositional manner. Whereas our final model must perform the task by sequentially gener- ating the label, we take advantage of the ontology to turn this into classification task over finite number of labels. In doing so, obtain cor- rectness classifier that able to reach over 90% accuracy across all target domains (See Table 13).
The motivation behind this work is to leverage con- trollable data synthesis as means of expanding products and services into target domains where la- beled data is searce. By definition, these new areas are out-of-domain (OOD) for a model trained only on source domains. Our strategy for generalizing to OOD spaces is to perform data augmentation. Successful data augmentation ideally helps a model generalize on test set given a limited train- ing set by expanding the seed data to cover the en- tire solution space. As a result, reliably controlling this process is akin to automating data collection. Following the principles of active learning, ideal data collection involves selecting examples to label that provide high coverage and high impact (Set- tles, 2009). Turning back to data augmentation, these goals translate to promoting diversity and label preservation, as mentioned in Section 1.
Our method (MSP) has a number of levers to pull in order to increase diversity. One idea is to sim- ply increase the temperature parameter of the LLM during text generation. Another path is shuffle the exemplars used for guidance or change the way the exemplars are retrieved. Building upon this, one could even exclude the seed example from the exemplars to minimize the copying behavior com- mon to LLMs. A particularly exciting direction to pursue is composing novel attribute combinations not seen in the seed set. For example, one utter- ance might have 'greet' and 'change' intents in airline domain (e.g. Hello, I would like to change my flight), while a second utterance contains 're- quest_info info' and 'when' intents for e-commerce (e.g. Can you tell me when the store opens?). These could be remixed to generate a wholly new utter- ance with 'request_ing info and 'change" intents in the restaurant domain (e.g. I'd like to know how I can change my reservation). We actually tested all these ideas during preliminary experimentation. As it turns out, they don't help.
B Training Setup Details
To select the exemplars, we first represent each example in the seed data by its attributes. Then, for each of those examples, we sort the available training data by amount of overlap with the rep- resentation to find the top 10 closest in attribute alignment. From those top-ranked candidates, we randomly sample two examples to serve as exem- plars during data generation. We tested with 1, 2. or 3 exemplars and found that k=2 typically worked best, although exemplar was not far behind.
We use a parameter of 4, for the num generations. When training the MSP model, we tested the learn- ing rates from [1.0, 0.3. 0.1, 0.03J. We found that the range of rates were much higher than expected. For final tests, we used 1r=0.3. We train with an effective batch size of 24, for example with batch- size flag set to 8 and gradient accumulation set to 3 steps. However, if the GPU runs out of memory, we might lower this 6 and 4, respectively. For fine-tuning the downstream task on GODEL large model, we test the learning rate across [le-4, 3e-5, 1c-5, 36-6] for up to 14 epochs with carly stopping, and found 3e-5 worked best.
In fact, we found that label preservation quickly overtook diversity in terms of being the most im- portant factor for influencing downstream impact. Consequently, we actually had to take painstak- ing efforts to dial down the diversity. We lowered temperature parameters. We selected very simi- lar exemplars for guidance. We only used attribute combinations that were found in the seed data. And we even added a denoising step to minimize varia- tion (Subsection 3.4). Although limiting diversity

---

worked in our case, believe this is largely due the static nature of our test sets, while the dis- tribution of real life test environments exhibit an ever-expanding long tail. In either case, the flex- ibility of MSP allows the practitioner to choose what they want, whether that's precision within specific area or robustness to cover OOD,
D
Mixing Other Parameter-Effiient Tuning Methods
Our core idea for mixing adapter weights is flexible enough to accommodate other parameter-efficien tuning methods such as LoRA (Hu et al., 2022). Our key contribution is that the mixing should be learned rather than relying on prompt engineer- ing. To this point, we ran additional experiments on NLU++ hotel, CrossNER music and TOPv2 weather by mixing LoRA weights where each adapter matrix represents a single attribute. Specifi- cally, we use the bottleneck method and replace the attention projection linear layer with correspond- ing LoRA linear layer. The results were 86.4, 76.7 and 80.1, respectively, which outperform other non- mixture baselines but do underperform MSP. We did not have much time that this experiment shows how learning to mix already outperforms most other baselines.
the results, but note

---

Method
Politics
Science
Music
Literature
BiLSTM-
56.6
50.0
44.8
43.0
43.6
Cnach I.STM
61.5
52.1
51.7
48.4
45.2
Baselines
BERT-agge
68.7
69.4
68.3
63.6
58.9
Multi Multi-Cell-LSTM
70.6
66.4
70.5
67.0
58.3
LST-NER
70.4
66.8
72.1
67.1
60.3
Domain
Multi- CeliiCeel-LSMM + DAPT
71.5
67.7
74.2
68.6
61.6
Adaptive
LST-NER DAPT -an-Innegration DAPT
73.2
70.1
76.8
70.8
63.3
Pre-training
72.1
68.8
75.7
69.0
62.6
Few-Shox (GODEL)
76.7
72.0
75.8
69.2
64.7
Direct Prediction
In-context Learn (GPT-3.5)
54.4
57.2
67.2
52.9
48.4
In-context Learn (FLAN-TS)
27.6
26.8
13.8
34.7
56.9
Prompt-tune (GPT-J-6B)
63.9
52.4
62.9
62.9
52.3
Prompt-tune (FLAN-T5)
71.6
68.0
69.3
58.3
57.1
Easy Data Augmentation Round Trip Translation Paraphrasing (BART)
77.3
76.3
71.0
65.4
Data
74.1
66.3
Augmentation
75.8
71.0
Masked
74.2
76.8
70.3
Conditional LM
75.7
73.9
76.2
69.8
63.9
Controlled
DExperts Conditional VAE
78.2
71.4
72.6
66.3
63.6
Text Generation
74.0
69.6
72.9
67.9
64.3
Keyword2Text
74.8
72.4
75.3
68.2
62.3
MSP w/ Concat
77.3
72.8
71.2
66.3
62.7
MSP w/ Pooling
80.6
73.0
79.0
71.9
66.7
Our Method
MSP w/ Attention
78.2
73.9
77.1
72.1
65.7
MSP w/ Bottleneck
79.5
74.8
80.0
69.6
65.9
MSP w/ Convolution
80.5
73.6
78.7
67.8
65.0
Table 8: End-to-end Fl-scores for CrossNER. Different MSP mixtures achieve state-of-the-art all domains.
Reminder
Weather
Transfer learn (LSTM)
Hotels
Banking
Transfer Iearn (RoBERTs)
63.7
ConveRT
75.4
65.2
Few-shot (TS-Large)
50.2
68.2
LM12-IB
67.3
49.2
RoB-base-QA
79.3
74.2
Fine tune (BART-CopyPlL
71.6
AIBERT-base- QA
76.7
72.7
Joint training (BART-CopyPtr)
58.9
74.7
Transfer Iearn (BART-CopyPtr)
68.0
75.9
Few-shot (GODEL.)
74.5
64.2
Meta-Learn (BART-CopyPtr)
70.5
77.7
In-context leam (GPT-3.5)
52.4
52.1
In-context leam (FLANN-S))
59.0
60.5
Few-shot (GODEL)
60.5
Prompt tune (GPT---6B)
68.2
59.5
In-context learn (GPT-3.5)
39.0
Prompt tune (FLAN-T5)
81.2
69,2
In-context leamn (FLANNTS))
Prompt tune (GPT-J-6B)
49.5
69.7
Easy Data Augmentation Round Trip Translation
81.8
68.0
Prompt tune (FL.AN-TS)
50.6
72.9
77.3
71.2
Paraphrasing (BART)
76.6
64,9
Easy Data Augmentation Round Trip Translation
62.5
Masked In-Fill
78.2
69.6
54.8
77.6
Paraphrasing (BART)
61.0
81.7
Conditional LM
76.8
71.5
Masked In-Fill
63.8
78.5
DExperts
83.1
71.8
Conditional VAE
77.0
70.4
Conditional L.M
67.8
Keyword2Text
74.8
67.8
DExperts
50.9
69.7
Conditional VAE
46.0
69.1
MSP w/ Concat
83.5
84,2
Keyword2'exx
64.00
81.0
MSP w/ Pooling
83.8
82.9
MSP w/ Attention
86.5
84.1
MSP w/ Concat
62.2
82.9
MSP w/ Bottleneck
89.7
84.6
MSP w/ Pooling
62.8
83.9
MSP w/ Convolution
85.0
80.3
MSP w/ Attention
64.5
83.3
MSP w/ Bottleneck
65.1
84,6
Table 9: Full end-to-end Fl Fl-scores evaluated generic splits hotel and banking domains of NLU++.
MSP w/ Convolution
61.0
83.2
Table 10: Full end-to-end accuracy evaluated minder and weather target domains within TOPv2.

---

Easy Data Augmentation (EDA)
NLU++
Intents: 'restaurant', 'when" Domain: Hotels
Original: when do you start serving dinner? Generated: do when dinner start serving you?
TOPv2
Intents: 'delete "delete_remineeee" reminder'; Slots types: person_ reminded_ "todo" Domain: Reminder
Original: Delete my reminder to pay ATAP business taxes.
Generated: Delete my reminder to ATAP business taxes.
Round Trip Translation (RTT)
CrossNER
Entity categories: 'miscellaneous', 'algorithm',
Feature extraction and dimension reduction can be combined in one step using principal compo- nent analysis (CCA) ), linear discriminant analysis LDA , canonical correlation analysis CCA
Original:
"task":
Entities: 'PCA', 'non-icgative matrix factorization', 'K-NN', 'Feature extraction', 'NMF', 'linar discrimi- nant analysis'. 'LDA', 'cononical correlation analy- sis'. 'CCA', 'dimension reduction', 'principal compo- analysis', : feature vectors". "pre-processing step" Domain: Al
negantgaaive matrix factorization (MF)) techniques pre-processing step followed by clustering K- NN feature vectors
reduuced-dimension space. Generated: As step, extraction and size reduction techniques can be combined using the analysis of the main components (PCA), linear discrimination anal- ysis (LDA),, canonic correlation analysis (CAS) or non-negative matrix factorization (NMP) as prelim- inary processing phase, followed by the grouping of K-NN on vectors a space with reduced dispersibil- ity factor.
TOPv2
Intents: "create "create_reminderr reminder'. 'get recurring date time": Slots types: person reminded', 'Freuueccy,, "todo', 'recurring date time", 'date time" Domain: Reminder
"get Jodo',
Original: Remind me daily to buy a present for Papa before July 28th.
Generated: Remind me to buy my father a present every day hefore July 28th.
Paraphrase
CrossNER
Entity categories: 'party'. 'organization', 'politician' Entities: Red-Green coalition', 'Bundestag,, 'Renate Klimast', 'Joschka Fischer", "Jürgen Trittin', 'SPD' Domain: polities
Original: Despite losses for the SPD the Red-Green coalition government commanded very slight major- ity in the Bundestag and was renewed, with Joschka Fischer foreign minister, Renate Kunast minis- ter for consumer protection, nutrition and agriculture and Jürgen Trittin as minister for the environment. Generated: Despite losses for the SPD, the Red- Green coalition government commanded very slight majority in the Bundestag and re-elected, with Joschka Fischer foreign minister, Renate Küinast as minister for consumer protection, nutrition and agriculture and Jürgen Trittin as Minister for the envi- ronment.
TOPv2
Intents: weseeported_eehhhe"; weather';
Original: Any toradoo warnings in my area? What the torado warning my area?
Slots types: 'location"
Generated:
Domain: Weather
In-Fill
NLU++
Intents: 'thank', caacel_close,_ Welveeeeeeeeeeeeeeeeeeeeeee Domain: Hotels
Original: thanks, I'll leave the room hefore 7:25 a.m.. Generated: thanks, I'll leave the office before 7:25 a.m..
CrossNER
Entity categories: 'product';
Original: Octave helps in solving linear and nonlin- car problems numerically. and for performing other numerical experiments using that is mostly compati- ble with MATLAB.
Entities: 'Octave", 'MATIAB'
Domain: AI
Generated: Octave helps in solving linear and nonlin- problems numerically and for performing other numerical experiments using that is fully compatihle with MATLAB
Table II:
The synthetic samples generated by Data Augmentation (DA) methods. The orange words highlight wrong attributes, whereas the green words represent correctly generated attributes. Paraphrase performs well since it able to maintain attribute meanings. Despite the clear decrease in fluency. EDA also maintains attribute labels and greatly assists downstream performance. RTT drops key entities in the pivot language, leading noisy data.

---

Controlled Language Model (CLM)
TOPv2
Intents:
"update reminder'.
Original: Change the reminder go the for psa every Wednesday
'ge recurring date_time'
Slot types: 'todo'. 'recurring date time new",
Generated: Move the reminder to go the for psa every Wednesday.
'frequency"
Domain: Reminder
CrosSNER
Entity categories: 'event', 'award";
The film was presented the Cannes Film Festival , won the Grand Prix Spécial du Jury and FIPRESCI prize and was nominated for the Palme
Original:
Entities: 'Grand Prix Spécial Jury'. 'Palme 'Or",
'Cannes Film Festival", 'FIPRESCI price"
Domain: Literature
Generated: won Cannes Film Festival's Grand Prix Spécial Jury and the FIPRESCI prize, as well as the Palme 'Or.
Decoding-Time Expert (DExperts)
NLU++
Intents: "cancel_closse leaecee__ooeeeeee__eeeeeee Jeave_rrezee'
Original: cancel my direct debit to LA VN Kitchen Generated: I need to cancel my booking at 10:00 a.m. as I have private meeting with my wife.
Domain: banking
TOPv2
Intents:
get recurring date time',
"up- 'recur-
Original: Update the reminder about medication to
:ite_rennner': Slots types: 'todo',
once a day.
ring date time new', 'frequency' Domain: Reminder
Generated: Change the reminder to pay the bill the 15th of every month to the 15th of every month.
Conditional Variational Auto-Encoder (CVAE)
CrossNER
Entity categories: 'book', 'writer":
Original: Swift is remembered for works such as
Entities: 'Gulliver 's Travels", 'An Argument Against Abolishing Christianity', 'Swift', 'A Modest Proposal',
Tale Tub ( 1704 ) An Argument Against Abolishing Christianity 1712 ) Gulliver Travels 1726 ), and A Modest Proposal 11229 ).
'A Tale of a Tub' Domain: Literature
Generated: Swift is rememhered for a Tuhation's 's works in the 17th century, An Argument Abolberg, Gulliver at A Modest Abishing and Robert Gposal, music.
CrossNER
Entity catogories: 'election', 'organization', 'coun- try'. 'miscellaneous':
Original: Following the 2004 European Parliament election in the United Kingdom, 37 MEPs from the UK 2004 European Parliament election Poland, 2004 European Parliament election in Denmark and Sweden founded new European Parliamentary group called Independence and Democracy as direct suc- cessor to the EDD group.
Entities: 'Independence and Democracy'. 'MPPs', '2004 European Parliament election in Denmark' 'EDD', '2004 European Parliament election in Poland', 'UK', 'Sweden', '2004 European Parliament election in the United Kingdom'
Domain: Politics
Generated: Following the 2004 European Parliament election in the United Kingdom, 37 MEPs from Den- mark from the UK and Sweden European Parliament election in the United Kingdom general election in Poland.
Table 12: Qualitative examples for Controlled Text Generation (CTG) methods. The orange words highlight wrong attributes, whereas the green words represent correctly generated attributes. Despite largely staying on topic and grabbing few keywords, CVAE and DExperts tend to lose lexical control, leading to poor performance.
NLU++
CrossNER
TOPv2
Hotels
Banking
Politics
Science
Music
Literature
AI
Weather
Reminder
Learning rate
le-4
le-4
le-5
le-5
lc-5
le-5
le-5
le-5
le-5
Batch Size
10
10
8
10
10
10
10
8
8
Accuracy
97.0
94.5
93.0
94.8
95.5
94.1
92.4
99.3
96.5
Table 13: Results from training the correctness classifier


Reviews:
[{"review_id": "0tE0HUadP1", "reviewer": "Reviewer_4Rzr", "summary": "", "questions": "A. Have you applied data filtering to other data augmentation baselines to ensure comparison equity?\n\nB. Regarding the second factor of data filtering, could you please provide more details on how you evaluated the similarity between the synthesized and original data? Is there a specific threshold set to the correctness metric?\n\nC. Are there any measures taken to ensure the quality of human evaluation? Did the evaluators receive reasonable allowance?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to start by thanking the reviewer for recognizing the key merits of the paper including the realistic application of LLMs and the comprehensiveness of the experiments. We now dive a bit into MSP's novelty.\n\nMSP is composed of two parts: (1) *controllable* data augmentation and (2) specialized data denoising. Given an arbitrary NLP task, MSP can produce high quality training data for that task. To the extent that the downstream task continues to exhibit low accuracy after training, we can use MSP again to target specific attributes of the task that are not solved. MSP is the only feasible method that exists to run such a  continually improving ML system because our method is both time-efficient and highly reliable.  In contrast, training a model through a LM-objective requires millions of dollars in compute and months to train.  Prompt engineering is certainly efficient too, but using an LLM for direct prediction or data generation is unreliable since prompts are too brittle.  As detailed in the general comment, this drawback exists even when the LLM is GPT-4. This highlights how control and consistency are still within the purview of fine-tuning alone. We hope our work encourages further study on (efficient methods of) supervised learning, distinct from progress towards improved prompt engineering.\n\nTo address the questions:\n - _Applying data denoising to other baselines_: The data denoising techniques are part of the novelty of our method since they are unique to MSP.  In either case, we apply the exact same data denoising to the GPT-4 augmented data, where this actually leads to a drop in performance. (See Part D in Official Comment)\n - _More details on data filtering_: The similarity between the synthesized and original data is cosine similarity of their embeddings, as encoded by a SentenceTransformer. No threshold hyper-param is necessary to tune since we pre-determine the augmentation factor of 4x (line 392) for fair comparison. Suppose we start with 100 seed examples and aim to generate 400 augmented examples. Subsequently, we rank all synthesized examples and simply keep the 400 closest ones based on cosine similarity.\n - _Quality of human evaluation_: To ensure quality of evaluation, we (1) wrote detailed explanations of the three human eval metrics (2) tested with a small pilot audience to iterate on explanations to make sure they were clear and (3) sent out the survey to the larger group. The participants were researchers from our affiliated labs acting as volunteers.\n\nTo address the detailed explanation for methodological choices:\n - _In Section 3.2, why choose this particular instruction prefix?_ Initializing the instruction-prefix with natural language text just provides a starting point.  Since the soft-prompt is ultimately learned, any reasonable instruction would suffice.  In fact, we tried 2-3 variations in preliminary testing and found them all to work reasonably well. We are happy to add more results to the appendix for this if desired.\n - _Why use Prompt Tuning over LoRA or other forms of adapter tuning?_ To be honest, LoRA wasn’t as widely discussed back in late 2022 when we started working on this.  Our core idea for mixing adapter weights is flexible enough though to accommodate LoRA as well.  Our key contribution is that the mixing should be learned rather than relying on prompt engineering. To this point, we ran additional experiments on NLU++ hotel, CrossNER music and TOPv2 weather by mixing LoRA weights where each adapter matrix represents a single attribute. Specifically, we use the bottleneck method and replace the [attention projection linear layer](https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/modeling_llama.py#L256) with a corresponding [lora linear layer](https://github.com/huggingface/peft/blob/6c44096c7b8d55a2ecf24be9bc68393467e1584a/src/peft/tuners/lora.py#L834). The results were 86.4 ( 89.7 for soft prompts), 76.7 (80.0 for soft prompts) and 80.1 (84.6 for soft prompts).  We didn’t have much time to tune the results, but notably learning to mix already outperforms most other baselines.\n - _In Section 4.3, why select GPT2-large to evaluate text fluency?_ Giant models take lots of compute resources to run, so our rationale was to choose the smallest model possible that could still offer meaningful perplexity scores. We felt GPT2-large was a reasonable choice following other related works (e.g. [DisCup](https://arxiv.org/abs/2210.09551), Zhang and Song)\n\nWe will certainly include all this information and address the typos as suggested.  Thank you for the thorough review!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- **Comprehensive Empirical Study**: This work includes an extensive range of experiments to validate the effectiveness of each proposed method. The study encompasses different NLU tasks, such as multi-aspect intent detection, cross-domain named entity detection, and task-oriented semantic parsing. Furthermore, the research compares multiple methods from Direct Prediction, Data Augmentation, and Controlled Text Generation as baselines, ensuring that each part of the overall pipeline outperforms related methods.\n\n- **Efficiency**: For the green NLP topic, this work highlights the significance of utilizing powerful LLMs for data augmentation while considering computational costs. Recognizing the practical constraints of the real world, the proposed MSP pipeline focuses on distilling LLM's knowledge for a small LM in a parameter-efficient manner. This approach allows for a more realistic and sustainable usage of the LLM.", "reasons_to_reject": "- **Detailed Discussion**: The paper lacks detailed explanations for certain methodological choices, which may raise concerns about the rigor of the study. Specifically, 1) in Section 3.2, the authors use a particular instruction prefix, but the reason behind this choice remains unclear. 2) While the use of Prompt Tuning (Lester et al., 2021) is mentioned as a strategy for building soft prompts, the advantages of this approach over other PETL methods such as LoRA and adapter tuning are not discussed. 3) In Section 4.3, the rationale for selecting GPT2-large to evaluate text fluency instead of other PLMs is not adequately justified.\n\n- **Novelty**: While the paper presents a strong empirical study, its theoretical novelty may be limited. The components of the MSP pipeline, such as PETL, MoE, and prompt engineering, are established methods in their respective research directions. Although the MSP pipeline achieves promising performance, it appears to be a composition of different existing methods rather than introducing substantially new innovations. As such, the uniqueness of the MSP pipeline may be limited.\n\n**Update:** I increased my soundness score during the rebuttal phase. Please refer to the discussion below.", "questions_to_authors": "A. Have you applied data filtering to other data augmentation baselines to ensure comparison equity?\n\nB. Regarding the second factor of data filtering, could you please provide more details on how you evaluated the similarity between the synthesized and original data? Is there a specific threshold set to the correctness metric?\n\nC. Are there any measures taken to ensure the quality of human evaluation? Did the evaluators receive reasonable allowance?", "paper_topic_and_main_contributions": "This paper focuses on utilizing the LLM as a data augmentation tool to enhance the performance of small-scale models on multi-attribute natural language understanding (NLU) tasks. Instead of prompting LLM or adapting LLM with few-shot approaches, the proposed MSP pipeline leverages parameter-efficient transfer learning (PETL) modules to generate synthesized training data for downstream models using the LLM.\nThe MSP pipeline has two key highlights:\n1) MSP enables efficient control over the characteristics of the synthesized data. Specifically, the PETL modules (soft prompts) are initially trained with attribute-specific data and then used to generate multi-attribute synthesized data instances with a certain mixture-of-experts (MoE) approach.\n2) Quality control of augmentation data. Two identified factors, based on empirical observations, are applied to filter the over-generated data in order to improve data quality.\n\nThe authors validate the effectiveness of MSP by fine-tuning the downstream GODEL model with synthesized data generated by the FLAN-T5 teacher. The results demonstrate that MSP outperforms the baselines and other data augmentation methods across three multi-attribute NLU tasks. Furthermore, both automatic and human evaluations indicate that the MSP pipeline excels in terms of synthesized data quality.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "- Figure 1: the horizontal line (LLM-> controlled generated data) is skewed.\n- Table 4,5,6: Please use the bold to highlight the important results.\n- Line 230, 244: Layer norm -> Layer normalization\n- Line 295, 297, 384: ie. -> i.e.\n- Line 378, 401: k -> $K$"}}, {"review_id": "diyyMNaxjj", "reviewer": "Reviewer_uw25", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for noting the effectiveness of the approach on all three datasets.  The main concern is around using GPT-3.5 for generating augmented data.  In our added comment above, we run exactly the experiment of generating data for one domain in each of our datasets, then use this data to train our downstream GODEL model.  As seen in the results, MSP still outperforms GPT data augmentation because prompt engineering is difficult and imprecise.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper explores several methods for composing attributes in soft prompts and takes a denoising step to improve the data quality. The experiments are comprehensively conducted by selecting a diverse set of baselines and performing detailed comparisons with different methods. The effectiveness of the approach is demonstrated on three datasets. \n\nUpdate：Overall, the work of the article is reasonable. I increased my Soundness score to 4.", "reasons_to_reject": "Models like GPT-3.5 can directly produce data containing various attributes with appropriate instructions. Therefore, it seems more cumbersome to optimize soft prompts in training for data generation. The paper lacks a comparison with this direct prompting way.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposes a controlled text generation method (MSP) by tuning soft prompts. Compared to previous works, MSP allows for integrating various attributes and leverages the generation capacity of large models to achieve more controllable data generation. The effectiveness of the approach is demonstrated through experiments.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "no6TCQSzXu", "reviewer": "Reviewer_rBGC", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for recognizing the novelty of our approach, as well as the benefits of interpretability, flexibility, efficiency and transferability.  To give a bit of a background, when we first published our paper, GPT-4 was not yet released.  With that said, we have recently gained access to the API with results listed in our general comment.  Despite improved results over GPT-3.5, the larger GPT-4 model still does not outperform our MSP technique. This highlights the shortcomings of current LLMs in composing and mixing precise information, providing fertile ground for future research.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Overall, this is a well written paper with an interesting and relevant premise - namely using LLMs as data augmenters rather than predictors for classes of problems that require complex structured outputs. As the authors point out, this comes with a few advantages such as interpretability, flexibility, efficiency and transferability. The proposed MSP approach to controlling generations is an interesting, and - to the best of my knowledge - novel one. And the experiments are generally well structured, demonstrating that the proposed approach does improve on the baselines they compare against.", "reasons_to_reject": "Unfortunately, there are some real concerns (which to be fair the authors do touch on in their discussion of 'Limitations'). To begin with, the paper talks about using LLMs, but actually uses (and compares against, with the exception of GPT-3.5Turbo) models that by today's standards are not particularly *large*. While the authors argue that larger models - that they haven't been able to test due to access issues - may also be better data generators and therefore maintains the relevance of their contributions, they fail to account for the fact that these models may be good enough predictors that data generation is no longer needed for these multi-attribute outputs.\n\nBut even if we put that concern aside, and grant that data augmentation does yields smaller models with certain benefits, there is no clear reason to believe that truly large LMs require a strategy such as MSP to yield multi-attribute structured outputs. Without evidence of these failings on the part of LLMs that the authors haven't tested, the contributions in the paper - while interesting - could be short-lived or even irrelevant already.\n\nMore specific to the experiments they have conducted, the authors do not compare against the capabilities of LLMs as predictors with prompt engineering strategies generally accepted to yield better results, such as simple chain-of-thought prompting or (since this consists of multi-attribute predictions) prompt-chaining.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper focuses on the problem of predicting multi-attribute outputs with the help of generative language models in a domain-transfer setting. Rather than make the models generate predictions directly, the authors propose an approach to use them for generating data that is then used to train a task- and domain-specific smaller model. The data generation process relies on a Mixture of Soft Prompts (MSP) that is trained using data in the target domain, before being then leveraged to generate new data - also in the target domain. Through several experiments the authors show that using an MSP data augmentation approach helps improve over several types of baselines, and achieves state-of-the-art results on several of the datasets that they use in their evaluation.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper introduces a novel approach, Mixture of Soft Prompts (MSP), for controllable data generation using Large Language Models (LLMs) in the context of multi-attribute natural language understanding tasks. Instead of relying on LLMs for direct prediction, MSP leverages LLMs as data augmentation tools, enhancing the training of smaller, domain-specific models. The paper demonstrates the effectiveness of MSP through comprehensive experiments and shows it outperforms baselines on multi-attribute NLU tasks.

The reviewers raised concerns about (1) the model size used in the experiments and whether truly large LLMs might not require MSP; (2)
the absence of evidence showcasing limitations in large LLMs for multi-attribute predictions; (3) the lack of a comparison with prompt engineering strategies like simple chain-of-thought prompting or prompt-chaining.

The authors have partially addressed the first concern by gaining access to GPT-4 and reporting that even GPT-4 does not outperform MSP. However, they have not fully addressed the second and third concerns, as they have not provided evidence or comparisons with prompt engineering strategies. Thus, while the paper presents a novel approach, some concerns raised by the reviewers remain unaddressed.
Decision: Accept-Findings