Mixture of Soft Prompts for Controllable Data Generation (2023)

Abstract: Large language models (LLMs) effectively generate fluent text when the target output follows natural language patterns. However, structured prediction tasks confine the output format to a limited ontology, causing even very large models to struggle since they were never trained with such restrictions in mind. The difficulty of using LLMs for direct prediction is exacerbated in few-shot learning scenarios, which commonly arise due to domain shift and resource limitations. We flip the problem on its head by leveraging the LLM as a tool for data augmentation rather than direct prediction. Our proposed Mixture of Soft Prompts (MSP) serves as a parameter-efficient procedure for generating multi-attribute data in a controlled manner. Denoising mechanisms are further applied to improve the quality of synthesized data. Automatic metrics show our method is capable of producing diverse and natural text, while preserving label semantics. Moreover, MSP achieves state-of-the-art results on three benchmarks when compared against strong baselines. Our method offers an alternate data-centric approach for applying LLMs to complex prediction tasks.

## introduction complex natural language understanding (nlu) systems, such as semantic parsers, typically become useful only after training on copious amounts of labeled data @xcite . due to the high cost of annotation, obtaining a sufficient supply of data quickly becomes infeasible. low resource settings are particularly common when expanding a system into a new domain or service @xcite . this task of learning a target domain from limited data is referred to as domain-adaptive fewshot learning @xcite . modern large language models (llms) have emerged as effective classifiers in low resource settings @xcite , and prompt tuning (middle) trains a prompt that is then used to query the llm. msp (bottom) learns a set of soft prompts, mixes them together to generate attributepreserving examples, then merges the augmented and original data to train a smaller, downstream model. can even take advantage of few-shot examples without the need for gradient updates through in-context learning (icl) @xcite . however, off-the-shelf llms have shown evidence of struggling with direct prediction in more complicated nlu tasks, such as those involving hierarchy or compositionality @xcite @xcite . llms with icl also exhibit problems when the target output requires a specific structure not represented in the training data @xcite . intuitively, few-shot exemplars fail to provide enough signal to learn custom outputs since those formats were designed for specialized tasks, and thus are unlikely to have appeared in open web corpora typically used to train llms. alternatively, limited data issues can also be tackled through data augmentation techniques, including altering tokens at the surface level @xcite or mapping seed data into a latent state before generating new examples @xcite . what these methods lack though is control over the generation process. specifically, two aspects of control are critical when synthesizing data: label preservation and diversity. label preservation ensures the generated utterances remain faithful to the original attributes in the seed data. diversity ensures the generated utterances provide better coverage of the target distribution to guide the model toward better generalization. to avoid the pitfalls of naive data augmentation, we take advantage of the llm's ability to generate fluent text by leveraging it as a tool for controlled data generation. concretely, we start by tuning a set of parameter-efficient soft prompts for each of the attributes present in the seed data. we then introduce a novel method for combining the mixture of soft prompts (msp) to generate diverse, classconditioned training data in a carefully controlled manner. the synthetic data is finally used to train a smaller, downstream model on the task at hand (figure 1 ). using llms as data generators rather than black-box predictors provides interpretability and flexibility benefits since the synthesized data can be directly inspected for quality. we apply msp to three diverse nlu tasks to test its generalizability. compared to directly prompttuning an llm with few-shot data, using the llm to augment the data instead is capable of outperforming a model of the same size by up to 27% (see ## task formulation few-shot natural language understanding (nlu) can take many forms such as semantic parsing or named entity recognition. in such tasks, a model aims to understand a natural language input, but only has a limited number of training examples x i to do so. formally, we are given a dataset d @xmath0, y i } n ) s with n training examples that all belong to some group of s source domains. the goal is to expand into a target domain t, given only m examples in domain t: d @xmath1, y j } m ), where @xmath2. real life applications of nlu are further complicated by the multi-aspect nature of the target, where a single label y i may be composed of multiple unique attributes {attr a }. ## related work our paper tackles multi-aspect few-shot learning where the target is a structured output containing multiple attributes, and the model only has a few examples from which to learn such a pattern. given the complexity of the task, previous research trained custom models for each dataset @xcite . instead, we leverage the power of llms @xcite to design a more general solution through controlled data synthesis. in particular, we compose multiple soft prompts to generate the desired training data for the downstream task. consequently, we build upon foundational work studying soft prompt tuning (lester dataset attributes and meta-data generated text nlu++ intents: 'change', 'booking' domain: hotels original: change booking generated: i have a reservation that i need to modify crossner entity categories: 'artist'; entities: 'john o 'reilly', 'bobby rondinelli', 'john miceli', 'chuck burgi' domain: music original: chuck burgi @xcite @xcite @xcite @xcite , john @xcite and bobby rondinelli @xcite @xcite @xcite @xcite . generated: he was followed as musical artist by chuck burgi @xcite @xcite @xcite , john @xcite and bobby rondinelli @xcite @xcite @xcite @xcite . ## conclusion our paper presents an alternative method for fewshot learning using llms as an intermediate data augmentation tool rather than for direct prediction. by performing data generation as an intermediate step for improved end-to-end task performance, our method yields such benefits as interpretability, flexibility and modularity. compared against other strong data augmentation methods, we show that msp yields higher quality data that can be effectively used to improve performance in downstream tasks. this parameter-efficient method to perform controlled data generation is a powerful paradigm for using llms in low-resource scenarios; the positive results from the methods proposed in this work suggest promising future work in exploring tighter controls and smarter filtering mechanisms for data augmentation. ultimately, we encourage others to consider use llms as a tools for generating data, rather than only for generating direct predictions.