Generating Training Data with Language Models: Towards Zero-Shot Language Understanding
Abstract
Pretrained language models (PLMs) have demonstrated remarkable performancein various natural language processing tasks:
Unidirectional PLMs ( e.g. , GPT) arewell known for their superior text generation capabilities; bidirectional PLMs ( e.g. ,
BERT) have been the prominent choice for natural language understanding (NLU)tasks.
While both types of models have achieved promising few-shot learningperformance, their potential for zero-shot learning has been underexplored.
In thispaper, we present a simple approach that uses both types of PLMs for fully zero-shotlearning of NLU tasks without requiring any task-specific data: A unidirectional
PLM generates class-conditioned texts guided by prompts, which are used asthe training data for fine-tuning a bidirectional PLM.
With quality training dataselected based on the generation probability and regularization techniques (labelsmoothing and temporal ensembling) applied to the fine-tuning stage for bettergeneralization and stability, our approach demonstrates strong performance acrossseven classification tasks of the GLUE benchmark ( e.g. , 72 . 3 / 73 . 8 on MNLI-m/mmand 92 . 8 on SST-2), significantly outperforming zero-shot prompting methods andachieving even comparable results to strong few-shot approaches using 32 trainingsamples per class 1 .
Introduction
Pretrained language models (PLMs) [5, 8, 11, 19, 32, 39] have achieved human-level performanceon natural language understanding (NLU) tasks [63, 64] when fine-tuned on a large amount oftask-specific training data.
However, such a supervised fine-tuning paradigm is drastically differentfrom how humans perform these tasks: We barely need to see many task-specific training samples toperform well.
Recently, many studies have revealed the intriguing few-shot learning potential of PLMs: Byconverting task descriptions to natural language prompts and injecting them into PLMs, prompt-basedapproaches [5, 13, 53, 54, 56] leverage task-specific information for better training data efficiencyand have achieved remarkable few-shot results.
When prompt-based methods are applied to the zero-shot setting, however, the PLMs’ predictionsare much less accurate.
For example, GPT-3’s zero-shot performance is much degraded relative toits few-shot performance [5], especially on challenging tasks like natural language inference (NLI).
Without any task-specific samples, it is indeed challenging for PLMs to effectively interpret theprompts that come in different formats and are unseen in the pretraining data.
To familiarize PLMswith various prompts for zero-shot generalization to unseen tasks, a recent study proposes instructiontuning [67], which fine-tunes PLMs on a large collection of different tasks described by instructions.
Despite its strong performance, its success is grounded in the large number of cross-task annotated1 Code is shared in the supplementary material. datasets ( e.g. , train on many non-NLI tasks and transfer to NLI tasks) and the gigantic model size( e.g. , hundreds of billions of parameters), posing great challenges for training and using such modelsin most real-world applications.
In this work, we study zero-shot learning of PLMs on NLU tasks without any task-specific or crosstask data.
Motivated by the strong text generation power of recent PLMs [5, 23, 28, 50], we propose
SuperGen, a Super vision Gen eration approach, wherein training data are created via a unidirectional
PLM ( i.e. , the generator) which generates class-conditioned texts guided by label-descriptive prompts.
A bidirectional PLM ( i.e. , the classifier) is then fine-tuned on the generated texts to perform thecorresponding task.
Both PLMs can be of moderate size to fit in typical research hardware ( e.g. , a
GPT-2-sized [49] generator and a RoBERTa Large -sized [32] classifier).
With supervision automaticallycreated by the generator, SuperGen eliminates the need for task-specific annotations and providesthe classifier PLM with a larger amount of training data than in few-shot scenarios.
SuperGen iscompatible with any PLM as the classifier and any fine-tuning method.
Across seven classification tasks of the GLUE benchmark [63], SuperGen significantly outperformsthe prompt-based zero-shot method and even achieves an overall better result in both averageperformance and stability than strong few-shot approaches that use 32 annotated samples per class.
We identify several key factors to the strong performance of SuperGen through ablation studies: (1)selecting quality training data based on their generated probability, and (2) using label smoothing andtemporal ensembling to regularize fine-tuning on generated data.
Related Work
2.1 Few-Shot and Zero-Shot Learning with PLMs
Instead of using a large amount of annotated training data for fine-tuning PLMs on downstreamtasks, few-shot learning studies how to better leverage only a small amount of task-specific trainingdata, a more realistic scenario in many applications.
The most strict few-shot learning setting doesnot assume access to any unlabeled data or large validation sets for hyperparameter tuning [46],where prompt-based methods [5, 13, 31, 33, 53–56, 60, 79] are prominently deployed to inject taskdescriptions into PLMs and make effective use of their language modeling capability for improvedtraining data efficiency in low-data regimes.
More broadly, semi-supervised learning additionallyleverages unlabeled task-specific data, where data augmentation [7, 70], regularization [41] andbootstrapping [54] methods are commonly used.
Zero-shot learning, on the other hand, is a much more challenging setting with absolutely no accessto any task-specific data.
When prompt-based methods are directly used to obtain predictions from
PLMs without any training, their zero-shot performance can be much worse [5, 13]—difficult NLUtasks can be barely formulated as prompts that resemble the format of pretraining data, posing greatchallenges for PLMs to accurately interpret and leverage the prompts without given any trainingsamples.
The current mainstream of zero-shot learning is based on transfer learning: By convertinga set of tasks with abundant annotations into instruction templates [40, 52, 67, 71], entailmentpairs [75, 76] or question-answer formats [48, 80] and fine-tuning PLMs on them, the PLMs acquirethe cross-task transfer ability [74] to execute unseen tasks when they are formulated in a similarformat.
Our work proposes a different approach from these studies: We use a unidirectional PLM togenerate training data for fine-tuning another PLM on the target task.
This not only removes the needfor a large amount of cross-task annotations, but also eliminates the task difference in training andinference.
Moreover, different from previous studies [1, 73] that rely on labeled data to fine-tune thegenerative PLM, we directly use prompts to guide data generation without fine-tuning.
2.2 Controlled Text Generation with PLMs
Controlled text generation [22] aims to steer the generated texts of language models towards desiredcontents, styles or domains.
Through fine-tuning PLMs on attribute-specific data, high-level control( e.g. , generating certain topics or sentiments [82]), fine-grained control ( e.g. , generating specific wordsor phrases [6]) or both [24] can be achieved.
Adapting PLMs to generate texts of specific attributes canalso be realized at inference time without any further training of the PLMs [10, 25, 26, 30, 45, 65, 72].
Different text attributes can also be represented during pretraining time as control codes [23] whichlater can serve as explicit guidance for generating domain/attribute-specific texts.
Along another line of controlling text generation, the idea of using prompts as guidance has emergedrecently—Since natural language generation is largely based on contexts, providing certain prompts
[Table]
[Table] [Table]
[Figure]
as the beginning of a sequence can effectively steer the subsequent texts to be generated.
The promptscan be either in natural language [55] or as learnable parameters [29].
In this work, we also guide textgeneration via prompts, but for the novel purpose of creating training data for NLU tasks.
The ideaof using generated texts as training data has been explored in topic classification with bag-of-wordsor LSTM-based language models [36, 37] which do not have enough capacity to generate qualitytraining data for challenging NLU tasks.
Compared to annotated task-specific data, the generatedtexts may contain noise and have domain difference from the downstream task.
We will introduceseveral important strategies for effective fine-tuning of PLMs on generated data.
Method
3.1 Preliminaries
Problem Formulation.
We consider solving a classification problem 2 where we are only given thelabel space Y and a mapping M : Y
→ W that converts each label y ∈ Y into a label-descriptive 101 prompt ( i.e. , a short phrase) w y ∈ W .
We assume access to a unidirectional PLM G θ as the generatorand a bidirectional PLM C ϕ which will be fine-tuned as the classifier 3 .
We also assume the pretrainingcorpus D ( e.g. , Wikipedia) is available.
Text Generation with Unidirectional PLMs.
A unidirectional PLM G θ is pretrained to maximize the generation probability of each token in a sequence x = [ x 1 , x 2 , . . .
, x n ] conditioned on previous tokens:
[Equation]
Here, p θ ( · ) is usually parameterized using token embeddings e and contextualized embeddings h 105 given by a Transformer [62] encoder.
After pretraining, G θ can be directly used to generate new texts by recursively sampling tokens fromits output probability distribution.
Typically, a temperature hyperparameter τ > 0 is introducedduring sampling [20] to adjust the sharpness of the probability distribution:
[Equation]
where τ → 0 approximates greedily picking the most probable next token; τ → ∞ induces a uniform 110 distribution.
Additionally, sampled tokens can be confined to the top- k most probable ones to avoid regression tasks this the texts continuous label space However there classification SuperGen low-quality tokens.
In this work, we find such top- k sampling with temperature is sufficient toproduce coherent and meaningful texts as training data for NLU tasks.
Exploring more sophisticatedsampling strategies [21] is left for future work.
3.2 Training Data Generation
[Table]
When given a label-descriptive prompt suchas “Write a negative review:”, humans areable to produce texts pertaining to the cor-
[Table]
responding class.
We aim to leverage thestrong text generation power of a unidirec-
[Table]
tional PLM G θ for the same purpose of cre-
[Table]
ating class-conditioned training data.
Wenote that G θ is directly used for generation [Table] without any parameter updates. The prompts
[Table] used for different NLU tasks in GLUE are [Table] summarized in Table 1.
Generating Single Sequences.
For singlesequence NLU tasks such as sentiment classification ( e.g. , SST-2), we simply use a prompt w y corresponding to label y as the beginning of the sequence and let G θ generate the remaining sequence:
[Equation]
where G θ ( w y ) denotes using w y as the in- 127 put to G θ and recursively sampling tokens 128 from the distribution in Eq.
(1) until a fullsequence is generated; x g denotes the generated sequence ( i.e. , excluding the prompt),which will be paired with y to form one training sample ( x g , y ) .
For syntactic tasks like linguistic acceptability classification ( e.g. , CoLA) which requires generating both linguistically acceptable and unacceptable sequences, we start the sequence with random stop words and use varying sampling temperaturesfor generating different sequences.
A smaller temperature ( e.g. , τ = 0 .
1 in Equation (1)) sharpens thesampling probability distribution towards the most probable tokens, thus the resulting sequence willmore likely to be linguistically acceptable.
Using a larger temperature ( e.g. , τ = 10 in Equation (1))flattens the sampling probability distribution to be more uniform, and the generated tokens will benearly random, which can create linguistically incorrect sequences.
Generating Sequence Pairs.
Sequence-pair classification tasks require generating two sequences of specific relationships ( e.g. , entailment, contradiction).
We sample 4 the first sequence x s from the pretraining corpus D , concatenate the prompt w y with x s , and let G θ generate the second sequence x g :
[Equation]
The sequence pair training sample will then be formed as ( x s , x g , y ) .
Rewarding and Penalizing Repetitions for Sequence Pair Generation.
A common issue in textgeneration is degenerate repetition [21, 23, 49, 68] where generated texts can get stuck in repetitionloops.
To address this issue, one approach is to discourage repetition by reducing the logits of tokensthat are already in the sequence before performing sampling [23].
In sequence pair generation,however, it is sometimes desirable to encourage the second sequence to repeat some words in thefirst sentence ( e.g. , for generating an entailment or a paraphrase).
Therefore, we propose a simplemodification of Eq.
(1) that rewards/penalizes repetition based on whether the token has appeared in x s / x g :
[Equation]
exp( e [Equation] [Equation] (cid:80) |
[Equation] | j =1 exp( e ⊤ j h i /ω ) , where ω =   τα x i ∈ x s ∧ x i / ∈ x g τβ x i ∈ x g τ else , (2) and α > 0 , β > 0 are hyperparameters.
By setting α < 1 and β > 1 , we can promote tokens inx s that have not appeared in x g to have a higher chance of being generated, and discourage thegeneration of repetitive tokens in x g to mitigate degenerate repetition.
The parameters used fordifferent tasks are listed in Appendix C Table 8.
3.3 Effective Fine-Tuning on Generated Texts
With the generated training data, one can fine-tune a bidirectional PLM C ϕ as the classifier to performthe NLU task.
However, training C ϕ via standard supervised training on all generated texts is likelyto yield suboptimal performance on downstream tasks because (1) the generated texts may containnoise as G θ may not always produce texts pertaining to the desired class, especially for challengingsequence pair tasks with subtle semantic relationships; and (2) the generated texts can be consideredas originated from the domain of G θ ’s pretraining data, with a potentially different distribution fromthe downstream task; straightforward application of supervised training will result in overfittingto the pretraining domain and diminishing generalization ability, a common challenge in transferlearning [61, 81].
To address these challenges, we next introduce several simple and importantstrategies for more effective and stable fine-tuning on generated texts.
Selecting Quality Training Data.
We aim to select generated texts x g that are most likely to pertain to the desired label
y
( i.e. , with the highest p ( x g | y ) ).
The true probability p ( x g | y ) is unknown and we estimate it via the generation probability given by G θ conditioned on the prompt w y :
[Equation]
Since the above measure is biased towards shorter sequences, we instead use the geometric meanof the above conditional generation probability (or equivalently, the average log probability) of alltokens in x g as the ranking score, following [77]:
[Equation]
To construct a training set consisting of N samples per class, we will generate more samples ( e.g. ,10 N ), and select training data based on the score r in Eq.
For all tasks except CoLA, thetop-
N ones of each class are selected; for CoLA, the top- N ones are used as the training sampleas linguistically acceptable sequences, and the bottom- N ones are as linguistically unacceptablesequences.
Regularization for Better Generalization and Stability.
Even with the above training dataselection procedure, the resulting training set may still contain noise and there exists domain differencefrom the downstream tasks.
We apply two regularization techniques, label smoothing [59] andtemporal ensembling [27] for better fine-tuning stability and generalization.
Given a training sample ( x g , y ) , label smoothing trains the classifier C ϕ to minimize the standardcross-entropy loss between the label and the classifier’s prediction p ϕ
( x g ) , except that the label is aweighted average of the one-hot vector and a uniform distribution over all labels:
[Equation]
where q j = 1 ( j = y )(1 − ϵ ) + ϵ/ |Y|
and ϵ is the smoothing weight.
By forcing the classifier to be 181 less confident on training data, label smoothing improves robustness to label noise [34] and preventsoverfitting to the training set [42], thus improving generalization to different domains.
The motivation for temporal ensembling is that neural networks usually first pick up easy and generalpatterns in the data before learning more sophisticated and dataset-specific features [78], and thus the earlier states of the network offer better generalizability to different domains.
We therefore record thepredictions p ϕ =
p ϕ ( x g ) of C ϕ on each training sample ( x g , y ) at different training steps, and usethe accumulated moving-average predictions ¯ z to regularize the latest model training.
This also helpssuppress the fluctuation in model predictions due to data noise, offering better noise-robustness [43].
We update ensembled predictions ¯ z once every B batches:
[Equation]
where ˆ z has a zero initialization; γ is the momentum parameter; t is the number of updates ¯ z hasreceived; the division (1 − γ t ) is for bias correction [27].
We also use the ensembled prediction ¯ z asa reliable signal to filter out noisy training samples: Only those samples on which ¯ z strongly agreeswith the label y ( i.e. , ¯ z y > δ where δ > 0 is a threshold parameter) will be used for training.
We regularize model training by extending Eq.
(4) to add a KL divergence regularization term fromthe model prediction to the ensembled prediction weighed by λ :
[Equation]
We follow [27] to slowly ramp-up λ during training.
3.4 Overall Algorithm
[Table]
// Class y train set init. for i ∈ [1 , 2 , . . . , M ] do x g ← G θ ( w y ) T y ← T y (cid:83) { ( x g , y ) } end end
T ← {} //
Selected train set.
for y ∈ Y do Sort T y in descending order by Eq. T ← T (cid:83) T y [: N ] end ˆ z ←// Ensembled prediction init. T ∗ ← T //
Filtered train set.
for i ∈ [1 , 2 , . . . , T ] do Fine-tune C ϕ via Eq. (6) on a minibatch of T ∗ if i % B = 0 then Update ˆ z , ¯ z via Eq. T ∗ ← { ( x g , y ) | ¯ z y > δ, ( x g , y ) ∈ T } end endreturn C ∗ ϕ =
C ϕ
We summarize SuperGen for singlesequence NLU tasks in Algorithm 1.
Solving sequence-pair problems follows thesame algorithm except the pretraining corpus D is needed for sampling the first se- 203 quence x s .
Experimental Setup
Downstream Tasks and Metrics.
Weuse all the tasks included in GLUE [63]except STS-B which is a regression task.
Please refer to Appendix A for more details about GLUE tasks.
We follow theevaluation protocol of [13]: We use F1score as the metric for QQP and MRPC,
Matthews correlation for CoLA, and accuracy for the rest of the tasks.
The originaldevelopment sets of these tasks are usedfor testing.
For all reported results, we include the average and standard deviationover 5 different random seeds.
Models.
Unless specified otherwise, weuse CTRL ( 1 . 63 B parameters)
[23] as thegenerator G θ and COCO-LM Large ( 367 M 221 parameters)
[39] as the classifier C ϕ .
Wealso show the results using similar-sized
PLMs (GPT-2 [49]/RoBERTa [32]) as thegenerator/classifier in Section 5.3.
Fine-Tuning Settings and Hyperparameters.
We note that SuperGen is compatible with any fine-tuning method; while using more sophisticated methods may grantfurther performance improvement, we use the basic prompt-based fine-tuning with manual templatesapproach for simplicity and clarity.
For all tasks, we use the same templates and label words as in[13].
Under the zero-shot learning setting, it is not possible to tune hyperparameters due to the lack
[Table]
Prompt group #3 is not applicable to RTE/MRPC which do not have the contradiction label.
[Table] [Table]
of validation sets
Therefore, we keep all fine-tuning hyperparameters ( e.g. , learning rate, batch size,training epochs, number of generated training samples, label smoothing and temporal ensemblinghyperparameters) same across all tasks.
See Appendix C Table 9 for details.
Compared Methods and Ablations.
We include the results of zero-shot prompting, standardfew-shot fine-tuning and the four few-shot prompt-based fine-tuning methods proposed in [13].
Wealso conduct ablation studies by removing the following three techniques from SuperGen one at atime: (1) not using Eq.
(3) for training data selection but randomly selecting the same amount oftraining data (- data selection); (2) not using label smoothing (- label smooth) but using one-hotlabels; and (3) not using temporal ensembling ( i.e. , using Eq.
(4) instead of Eq.
(6) as the trainingobjective) (- temporal ensemble).
Evaluation
5.1 Main Results
We present the results of SuperGen, its ablations and compared methods in Table 2.
Overall, SuperGensignificantly outperforms zero-shot prompting and achieves an overall better result than all few-shotmethods.
Notably, SuperGen results in much smaller variance over different random seeds thanfew-shot approaches on most tasks—with access to more training data, fine-tuning of PLMs becomesmuch more stable.
The ablation results demonstrate that all three strategies ( i.e. , quality trainingdata selection, label smoothing and temporal ensembling) play important roles in improving andstabilizing the final performance, especially on challenging tasks like MNLI.
5.2 Using Different Prompts
One important factor of SuperGen is the choice of label-descriptive prompts as they directly influencethe quality of generated training samples.
To study the impact of different prompt choices on thefinal model performance, we create three different groups of prompts other than the original ones for
MNLI, RTE and MRPC.
We replace the prompt for one label used in Table 1 with a synonymousone and keep other prompts unchanged when forming a different prompt group (Please refer to Appendix B Table 7 for details).
We also experiment with mixing the generated data by differentprompt groups (mixed).
The results are shown in Table 3.
Overall, the model performance underdifferent prompts is quite close, except on RTE whose test set is very small, potentially resultingin the higher variance.
The original prompt group works well across all three tasks, which mightimply that there exist “optimal” prompts that almost consistently work better than others on differenttasks—likely related to the pretraining data distribution and the generator PLM.
In this work, wechoose simple prompts that make intuitive sense, and we leave the automatic searching of optimalprompts as future work.
5.3 Using Different PLMs
The final performance is also relevant to the choice of PLMs as the generator/classifier. Apartfrom the default PLM choice, we report the results of using GPT-2 XLarge ( 1 . 54 B parameters) [49]as the generator and RoBERTa Large ( 356 M parameters) [32] as the classifier in Table 4 with everything else unchanged.
When using GPT-2, we change the prompt used for SST-2 to “The film isbad/terrible/awful.”
for the negative label and “The film is good/great/excellent.”
for the positivelabel, since the original prompts used for SST-2 in Table 1 are a part of the control codes of CTRL andcannot be effectively leveraged by GPT-2.
Overall, both CTRL and GPT-2 are able to generate qualitytraining data for good fine-tuned classifier performance; CTRL consistently yields better resultsthan GPT-2 regardless of the choice of the classifier PLM, probably because CTRL is pretrainedwith control codes which provide explicit guidance for generating texts of certain domains andattributes.
We also observe that the generated text quality is strongly correlated to the generator’smodel size—using a smaller version of GPT-2 ( e.g. , with 117 M parameters) results in significantlyless coherent texts and can hardly serve as training data.
An interesting future direction is to try largergenerator PLMs ( e.g. , GPT-3) which may create training data of better quality.
5.4 Results with Different Amount of Generated Data
[Figure]
With training data automatically created by the generator, we can havea virtually infinite amount of training samples.
We show the resultsof using different amount of generated data (after quality data selection) for fine-tuning the classifier
C ϕ in Fig.
2 on MNLI-m and SST-2.
When the number of training datais small ( e.g. , 100 ), the fine-tuningvariance is high, resulting in the similar instability issue with few-shotsettings.
With more generated dataused, both average performance andtraining stability improve, yieldingcomparable results (with smaller variance) to fine-tuning using few-shot task-specific data.
However,when too many generated data ( e.g. , 10 , 000 ) are used, the classifier’s performance slightly drops,probably due to increased label noise—recall that the training data are selected based on the rankingscore in Eq.
(3), so using more data results in the inclusion of more lower-ranking texts in the trainingset and reduced data quality.
One way to address this issue is to use a fixed selection ratio and increasethe total number of generated texts to obtain a larger number of high-quality training data.
However,this comes at a greater computation cost in the generation step.
An important future direction is thusto develop better data selection strategies.
5.5 Case Studies
We present concrete examples of generated texts guided by prompts of different labels in Table 5.
Thegenerated sequences are not only coherent, but also pertain to the corresponding labels.
For easiertasks like SST-2, the generated texts almost always correctly reflect the desired sentiment polarityspecified by the prompt. For more difficult tasks like MNLI, sometimes the generated texts are not ofthe correct label (Appendix D Table 10 shows some negative results).
The existence of such label
[Table]
noise motivates our use of the regularization techniques in the fine-tuning stage. In the future, it will
[Table]
be interesting to develop new methods to better control text generation towards the desired label
[Table]
Discussions and Conclusions
Ethical Considerations.
While PLMs have demonstrated remarkable text generation and understanding capability, they can come with potential risks or harms [2, 3, 5] such as generatingmisinformation [44] or amplifying harmful biases [47].
The focus of our work is on utilizing existing PLMs to generate training data for NLU tasks instead of developing new PLMs or generationmethods.
Therefore, our method can be used in company with any bias reduction and correctiontechniques [15, 35] to mitigate the risks of PLMs.
Limitations.
One inherent limitation with zero-shot learning is the lack of access to task-specificsamples for hyperparameter tuning, whereas the performance of neural networks is usually heavilydependent on the choice of hyperparameters even when the training algorithm and training set arefixed [46].
Therefore, under the fully zero-shot learning setting, it is almost impossible to use theoptimal hyperparameters for model training.
In this work, we use a fixed group of hyperparametersacross all tasks. However, we indeed observed nontrivial improvements ( e.g. , 1+ points on MNLI and SST-2; 2+ points on RTE) when a small amount of task-specific samples ( 16 per class) are used onlyfor hyperparameter tuning and early stopping. We do not report these results as they do not reflectthe true zero-shot performance, but it serves as a simply way for achieving better results without anychanges to SuperGen’s algorithm when having access to a few annotated samples.
Extensions to Few-Shot Learning.
The most strict few-shot learning setting does not assumeaccess to any unlabeled data, which can be even expensive to collect for tasks that require data to meetspecific criteria [60]. This makes it challenging to apply data augmentation [7, 70] and self-trainingtechniques [38, 54] that rely on abundant unlabeled data. Due to the randomness in selecting few-shotsamples and the stochasticity in training neural models, fine-tuning with a tiny training set is going tobe inherently unstable. SuperGen potentially provides a new approach to stabilize and improve futurefew-shot learning methods: Few-shot samples can be either used in the training data generation stageof SuperGen as demonstrations [5] for steering the generator PLM towards producing texts closer tothe task distribution, or used together with generated data for better stability and generalization.
Conclusions.
We propose SuperGen, an automatic supervision generation approach for zero-shotlearning of NLU tasks.
By providing label-descriptive prompts as guidance to a unidirectional
PLM, training data can be automatically created for fine-tuning a bidirectional PLM.
Our frameworkdiffers from previous transfer-learning-based zero-shot methods in that SuperGen does not rely oncross-task annotations and eliminates the task difference in training and inference.
We show that several strategies are important for effective and stable fine-tuning on generated data, including quality training data selection, label smoothing and temporal ensembling.
SuperGen achieves strong performance on seven classification tasks of the GLUE benchmark, even yielding comparable or better results than sophisticated few-shot learning methods and offering better stability.
There is large room for future work, including but not limited to: Extension to few-shot learning settings, exploring larger generator models, better fine-tuning techniques to leverage generated data and better strategies for selecting quality training data
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work? [Yes] (c) Did you discuss any potential negative societal impacts of your work?
[Yes] (d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes] If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[N/A](b)
Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
[-All experiments are conducted on NVIDIA GeForce RTX 
 GPUs. SuperGen can be run on typical research hardware ( e.g. , with > 10 GB GPU memory). Thegenerator PLM G θ does not need to be trained so a relatively large generator can be used ( e.g. , a1 .-](b) Did you mention the license of the assets?
[Yes]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]