Venue: NAACL 2022

# Embedding Hallucination for Few-Shot Language Fine-tuning

Yiren Jian

Dartmouth College

yiren.jian.gr@dartmouth.edu &Chongyang Gao*

Northwestern University

cygao@u.northwestern.edu &Soroush Vosoughi

Dartmouth College

soroush@dartmouth.edu

Both authors contributed equally to this research.

###### Abstract

Few-shot language learners adapt knowledge from a pre-trained model to recognize novel classes from a few-labeled sentences. In such settings, fine-tuning a pre-trained language model can cause severe over-fitting. In this paper, we propose an Embedding Hallucination (EmbedHalluc) method, which generates auxiliary embedding-label pairs to expand the fine-tuning dataset. The hallucinator is trained by playing an adversarial game with the discriminator, such that the hallucinated embedding is indiscriminative to the real ones in the fine-tuning dataset. By training with the extended dataset, the language learner effectively learns from the diverse hallucinated embeddings to overcome the over-fitting issue. Experiments demonstrate that our proposed method is effective in a wide range of language tasks, outperforming current fine-tuning methods. Further, we show that EmbedHalluc outperforms other methods that address this over-fitting problem, such as common data augmentation, semi-supervised pseudo-labeling, and regularization. The code will be made available at: [https://github.com/yiren-jian/EmbedHalluc](https://github.com/yiren-jian/EmbedHalluc).

+
Footnote †: July 10-15, 2022 ©2022 Association for Computational Linguistics

## 1 Introduction

Fine-tuning a pre-trained language model (LM) on a downstream task with the labeled data has been the de facto approach in many NLP tasks Wang et al. (2019); Devlin et al. (2019). Conventional fine-tuning has been shown to be effective when a few thousands of labeled examples are available. Data augmentation Wei and Zou (2019), regularization Lee et al. (2019) and re-initialization Zhang et al. (2021) further improve the results.

However, the performance drops drastically when the number of examples falls to only a few dozens. Experiments from recent work Gao et al. (2021) have shown that fine-tuning performs poorly in the setting where only 16 examples per class are given. Indeed, tuning a language model with hundreds of millions of parameters (e.g., BERT-large has 300M parameters) with only a few examples inevitably faces the over-fitting problem.

Prior work have proposed regularization methods to overcome this problem Lee et al. (2019); Zhang et al. (2021). However, we show in our experiments that these methods fail in extreme data scarce setting. We speculate that the key to solve this issue is by data augmentation.

Current common text data augmentation methods, such as EDA Wei and Zou (2019) (which have been used in recent few-shot learning papers Wei et al. (2021); Basu et al. (2021)) and AEDA Karimi et al. (2021) operate at the lexical level, which while resulting in human readable texts, lead to limited diversity due to the discrete nature of the lexical space. In this work, we propose to use a generative augmentation method at the _embedding space_ for few-shot learning. The underlying hypothesis is that the intra-class relation of the observed examples can be modeled and that this can be learned from a few-samples to hallucinate diverse unseen examples. To be specific, we adapt a conditional Wasserstein Generative Adversarial Network (cWGAN) Arjovsky et al. (2017) as our hallucinator to hallucinate embeddings of sentences. By observing the real embeddings of examples from the fine-tuning dataset, the cWGAN plays an adversarial game to hallucinate embeddings that can fool the discriminator, while the discriminator is trying to classify the fake embeddings from the real ones. Once the halluciantor is trained, we condition it on labels to generate diverse embeddings at each fine-tuning step. This effectively extends the fine-tuning dataset with diverse embedding-label pairs which carry intra-class variation that can be a useful learning signal for the language learner.

We evaluate our method, called Embedding Hallucination (Embedhalluc), on 15 tasks and show that it generally improves over recent fine-tuning

methods. We further experimentally show the overall superiority of EmbedHalluc when comparing to regularization methods proposed to address the problem of over-fitting during fine-tuning of LMs, such as Mixout (Lee et al., 2019) and Re-Init (Zhang et al., 2021). Finally, since our method is a form of data augmentation, we also compare EmbedHalluc to a common data augmentation technique EDA, and semi-supervised learning where unlabeled data is already available.

## 2 Related Work

**Fine-tuning of Language Models**. Better fine-tuning of language models can be achieved by proper initialization Dodge et al. (2020), regularization Lee et al. (2019) or prompts Schick and Schutze (2021). Other tricks include bias correction in optimizer and re-initialization of top layers in Transformer Zhang et al. (2021). Instead of fine-tuning all parameters in a model, other work explore only learning a few vectors Lester et al. (2021); Li and Liang (2021); Guo et al. (2021) or a few additional parameters Houlsby et al. (2019).

**Hallucination Methods**. Feature Hallucination of examples is first introduced for visual recognition Hariharan and Girshick (2017) by meta-learning Wang et al. (2018), variational inference Luo et al. (2021); Lazarou et al. (2022), and adversarial learning Li et al. (2020); Tjo et al. (2022). Label Hallucination Jian and Torresani (2022) assigns soft pseudo-labels for unlabelled images to extend the fine-tuning few-shot dataset.

**Learning from limited labeled data** (few-shot learning) in Computer Vision is usually achieved by meta-learning Ren et al. (2018a, b); Jian et al. (2020); Jian and Gao (2021) or transfer learning Tian et al. (2020). In NLP, few-shot learning has been successfully applied to machine translation Arthaud et al. (2021), abstract summarizing Fabbri et al. (2021), question and answering Hua et al. (2020); Ram et al. (2021), and entity recognition de Lichy et al. (2021); Tong et al. (2021); Ding et al. (2021), by meta learning Li and Zhang (2021); Bansal et al. (2020); Sharaf et al. (2020), data augmentation Wei et al. (2021); Wei and Zou (2019); Karimi et al. (2021); Jian et al. (2022), and prompts Gao et al. (2021); Tam et al. (2021).

Our method is a generative data augmentation method in the embedding space. Different from Wei et al. (2021) which uses EDA Wei and Zou (2019) to augment examples at the discrete input space, we hallucinate auxiliary examples at the embedding space. Our method shares similarity to FDA Kumar et al. (2019), which is also a generative data augmentation method, but at the feature space. Also, different from FDA which is focused on two intent classification tasks, our method can be applied to a wide-range of NLP task as shown by our experiments on 15 diverse tasks.

## 3 Method

### Conditional Wasserstein GAN

GAN Goodfellow et al. (2014) has led the revolution of generative models to achieve impressive results in synthesizing images Zhu et al. (2017) and higher dimensional data Wang et al. (2020). Wasserstein GAN (WGAN) Arjovsky et al. (2017) uses the Wasserstein distance as the objective function to stabilize the training of GAN.

Our hallucinator is trained under the conditional WGAN framework. After the training, we use it to generate pseudo-embeddings of examples by feeding it with random noisy vectors sampled from and the corresponding condition class labels \(c_{i}\). The hallucinated embeddings \(s_{\text{halluc}}\), in principal, are indiscriminative to the embeddings of observed examples in that class.

### Fine-tuning with Hallucinated Embedding

For a single input sentence, we first pass it through the embedding layer to get the sentence embedding \(s_{\text{sent}}\). We then concatenate \(s_{\text{sent}}\) with \(s_{\text{haltc}}(c_{i})\) to

Figure 1: Overview of our method: The encoder model takes real embeddings of sentences and hallucinated embeddings as input and learns from their mixture. Halucinator generates fake embeddings conditioned on the class label. The discriminator \(D\) discriminates real embeddings and fake embeddings leading to the GAN loss \(\mathcal{L}_{WGAN}\).

 

form a batch of mixture of real and fake embeddings \([s_{\text{sent}},s_{\text{shallow}}(c_{i})]\). The encoder learns from the batch with the corresponding labels \([c_{\text{sent}},c_{i}]\).

**Label Calibration**. The hallucinated embedding \(s_{\text{halluc}}(c_{i})\) is conditioned on its label \(c_{i}\). However, this hard label may not best represent the class information of the hallucinated embedding. We propose Label Calibration (LabelCalib) by pseudo-labeling from a teacher model \(\mathcal{F}_{\text{GEN0}}\) (\(LM_{1}\) in Algorithm 1), where \(\mathcal{F}_{\text{GEN0}}\) is first fine-tuned on the original training set (without augmentation). The soft-label of the embedding \(s_{\text{halluc}}(c_{i})\) is then \(c_{\text{pseudo},i}=\mathcal{F}_{\text{GEN0}}(s_{\text{halluc}}(c_{i}))\). Finally, the language model \(\mathcal{M}\) learns from the hallucinated embedding by KL-divergence

\[\mathcal{L}_{\text{halluc}}=\text{KL}(\mathcal{M}(s_{\text{halluc}}(c_{i})), c_{\text{pseudo},i}) \tag{1}\]

The total loss of our method is

\[\mathcal{L}_{\text{total}}=\mathcal{L}_{\text{real}}+\mathcal{L}_{\text{ halluc}} \tag{2}\]

where \(\mathcal{L}_{\text{real}}\) is the loss learning from real embedding-label pairs. The pseudo-code for fine-tuning of few-shot language learners with hallucinated embeddings is shown in Algorithm 1.

Note that baselines considered in this paper use total loss \(\mathcal{L}_{\text{total}}=\mathcal{L}_{\text{real}}\). Computing \(\mathcal{L}_{\text{halluc}}\) requires one additional forward pass of the hallucinator and one more forward pass and backward pass of the language model. Thus, our method has about \(\times 2\) computational overhead compared to the baselines.

```
1:\(Max\_Step=1000\),
2:\(LM\): Language model,
3:\(H\): Emebedding hallucinator (pre-trained),
4:\(Train\_Set\): Training set,
5:\(Sample\): Randomly sampling function,
6:\(CE\): Cross Entropy loss,
7:\(KL\): KL-divergence loss.
8for i in \(Max\_Step\textbf{do}\triangleright\) Training LM\({}_{1}\)
9:\(sent,y=Sample(Train\_Set)\)
10:\(output_{1}=LM_{1}(sent)\)
11:\(L=CE(output_{1},y)\)
12:\(L.backward()\)
13:\(optimizer.step()\)
14:endfor
15:for i in \(Max\_Step\textbf{do}\triangleright\) Training LM\({}_{2}\)
16:\(sent,y=Sample(Train\_Set)\)
17:\(embed=H(\mathcal{N}(0,1)),c)\)\(\triangleright\) Learning from real text
18:\(output_{1}=LM_{2}(sent)\)
19:\(L_{real}=CE(output_{1},y)\)
20:\(L_{real}.backward()\)
21:\(optimizer.step()\)\(\triangleright\) Learning from hallucination
22:\(prob_{2}=LM_{1}(embed)\)
23:\(output_{2}=LM_{2}(embed)\)
24:\(L_{halluc}=KL(prob_{2},output_{2})\)
25:\(L_{halluc}.backward()\)
26:\(optimizer.step()\)
27:endfor
28:return\(LM_{2}\)
```

**Algorithm 1** Our method: EmbedHalluc

### Evaluation Datasets and Protocol

We evaluate our method on 15 classification tasks. The evaluations are conducted by averaging results on 5 different train test splits. We sample 16 examples per class to form a training set and construct a validation set with the same size as the training set.

#### Training Details for Embedding Hallucinators

The training of Embedding Hallucinators involves training a generator and discriminator in the cWGAN framework. The generator is a 4-blocks model, with each block containing a FullyConnect layer followed by a BatchNorm and LeakyReLU. The hidden dimensions of the generator are \(128,256,512,1024\). The hallucinated embeddings, i.e., outputs of the generator are tensors of \(L\times 1024\), where the length of the generated embeddings \(L\) is set to be 128. The discriminator is a 3-blocks model, each bock having a sequence of FullyConnect-BatchNorm-LeakyReLU with the same hidden dimension of \(512\).

We train the Embedding Hallucinators for 150 epochs using a batch size of 64, the Adam optimizer (\(\beta=(0.5,0.999)\)), and a learning rate of 0.0002. The real embeddings are collected from the language few-shot training set by passing text into the embedding layer of the language model. We apply gradient penalty with weight of loss 100 for training the cWGAN.

#### Training Details for Few-Shot Language Learners

We draw two mini-batches during the training of our few-shot language learners, i.e., one from the real language few-shot training set, another one by 

sampling the hallucinators (see Algorithm 1).

To fairly compare our method with baselines and other methods, when learning with real sentences, we use the same learning rate of \(1e^{-5}\) (further justification of using this learning rate can be found in Appendix D). Our method learns from hallucinated embeddings with a grid search of learning rate of \(1e^{-5},5e^{-6},1e^{-6}\), and batch size of \(4,6,8\). We use the same search for EDA Wei and Zou (2019) and semi-supervised pseudo-labeling (SSL) when learning with additional augmented or pseudo-labeled data.

The models are selected based on the validation accuracy every 100 steps. Finally, results are reported by testing the models on the testing dataset. The algorithm is implemented in PyTorch-1.10 and experiments are conducted on Nvidia RTX-6000 and RTX-A6000 GPU.

### Main Results on 15 Tasks

We compare our method EmbedHalluc (w/o or w/ LabelCalib) using RoBERTa-large on 15 tasks with two fine-tuning methods: conventional (Table 1) and prompt-based fine-tuning (Table 2). Results for BERT-large-cased can be found in Appendix B.

In conventional fine-tuning, EmbedHalluc improves over the baseline in 14 tasks, only marginally under-performs in SST-5 (40.3 vs. 40.6 of baseline). When combining with LabelCalib, our method outperforms in all tasks. When applying to prompt-based fine-tuning, while our method under-performs in MNLI, MNLI-mm and RTE, it outperforms for all other tasks, with substantial improvements over the baseline in CoLA, TREC, QNLI, MRPC.

The relatively smaller improvements for prompt-based methods may be due to the inconsistency and randomness in the learning process since we have to insert [mask] token to a random position in the hallucinated embedding \(s_{\text{halluc}}\), for the calculation of the loss. Whereas, in conventional fine-tuning, the [CLS] token is always appended to the beginning of \(s_{\text{halluc}}\) and the classification is performed at the [CLS] token.

### Comparing to EDA and SSL

Since our method is a generative data augmentation (DA) method, we compare it to another DA method EDA. We also consider semi-supervised learning (SSL) which relies on unlabeled data (64 examples per class in our experiments). We apply pseudo-labeling Cascante-Bonilla et al. (2021) for SSL, _i.e._, we first fine-tune the model with the few-shot training set and use the fine-tuned model to pseudo-label the unlabeled data, finally we finetune the model again with the few-shot training set combined with the pseudo-labeled set.

EDA edits the input sentences by applying synonym replacement, random swap, random deletion and random insertion for a default 10% (\(\alpha\)) of tokens. EDA either greatly change the sentence with a large \(\alpha\) or fails to introduce substantial variations (which is crucial in the extreme low data setting) of inputs with a small \(\alpha\). Since it operates in the continuous embedding space, EmbedHalluc hallucinates diverse embeddings that follow the distribution of few-shot set. Thus, we observe in Table 3 that EmbedHalluc is overall superior to EDA.

EmbedHalluc is still competitive when comparing against SSL which assumes to have additional 64 examples per class from the task distribution.

### Negative Results from Regularizations

Our method can also be viewed as an implicit regularization method. Thus, we also compare to two

\begin{table}
\begin{tabular}{l c c c} \hline Task & Fine-tuning & EmbedHalluc & w/LabelCalib \\ \hline SST-2 (acc). & 76.8 (\(\pm\)2) & **82.6** (5.6) & 52.0 (4.7) \\ Suby (acc) & 90.1 (\(\pm\)5) & **91.3** (0.8) & **91.3** (0.9) \\ SST-5 (acc) & 40.6 (\(\pm\)2) & 40.3 (\(\pm\)1.5) & **41.6** (2.6) \\ CodeAtt. & 36.0 (\(\pm\)9) & **39.7** (0.8) & 38.1 (11.8) \\ TREC (acc) & 83.0 (\(\pm\)9) & **88.1** (2.5) & 87.9 (1.0) \\ MNLI (acc) & 41.6 (\(\pm\)5) & 48.0 (\(\pm\)9.5) & **49.6** (5.8) \\ MNLI-mm (acc) & 42.7 (\(\pm\)5) & 49.7 (\(\pm\)10.5) & **51.8** (6.1) \\ SNLI (acc) & 52.9 (\(\pm\)6.7) & **54.4** (3.4) & 52.3 (5.3) \\ QNLI (acc) & 55.3 (\(\pm\)2.7) & 60.2 (\(\pm\)5.3) & **64.9** (5.1) \\ QQF (acc) & 59.2 (\(\pm\)6.0) & 64.6 (\(\pm\)6.7) & **65.0** (5.3) \\ RTE (acc) & 52.9 (\(\pm\)1.4) & 53.4 (\(\pm\)1.7) & **55.9** (4.3) \\ MDER (FI) & 76.3 (\(\pm\)5.2) & **78.7** (1.9) & 78.1 (3.0) \\ MR (acc) & 74.5 (\(\pm\)5.9) & 79.4 (\(\pm\)5.5) & **80.8** (3.2) \\ MPQA (acc) & 65.0 (\(\pm\)1.5) & 70.1 (\(\pm\)7.0) & **70.5** (4.6) \\ CR (acc) & 71.7 (\(\pm\)7.5) & 75.1 (\(\pm\)5.6) & **78.0** (3.8) \\ \hline \end{tabular}
\end{table}
Table 1: Comparison of conventional fine-tuning and our EmbedHalluc, using RoBERTa-large. Our Label Calibration (LabelCalib) can further improve the results.

\begin{table}
\begin{tabular}{l l l l} \hline Task & Prompt-based & EmbeddingAlluc & w/ LabelCalib \\ \hline SST-2 (acc). & 92.7 (\(\pm\)0.4) & 92.8 (\(\pm\)0.7) & **93.1** (0.7) \\ Suby (acc) & 91.3 (\(\pm\)1.5) & **92.0** (0.4) & 91.7 (1.3) \\ SST-5 (acc) & 48.8 (\(\pm\)1.0) & 49.0 (\(\pm\)2.2) & **49.4** (1.4) \\ CoLA-Att. & 7.3 (\(\pm\)5.8) & 12.3 (\(\pm\)7.6) & **22.1** (5.6) \\ TBEC (acc) & 83.8 (\(\pm\)3.8) & 85.5 (\(\pm\)3.3) & **87.1** (2.9) \\ MNLI-cc & **69.7** (2.0) & 68.0 (\(\pm\)2.8) & 68.5 (\(\pm\)1.7) \\ MNLI-mm (acc) & **71.5** (1.9) & 69.9 (\(\pm\)3.0) & 70.6 (1.7) \\ SNLI (acc) & 78.0 (\(\pm\)3.0) & **78.8** (2.3) & **78.4** (2.3) \\ QNLI (acc) & 66.6 (\(\pm\)2.8) & 69.6 (\(\pm\)0.3) & **71.6** (2.0) \\ QQP (acc) & 70.2 (\(\pm\)4.3) & 71.9 (\(\pm\)5.2) & **74.2** (0.9) \\ RFE (acc) & **79.9** (3.3) & 69.9 (\(\pm\)3.3) & 66.9 (\(\pm\)4.6) \\ MRCP (FI) & **74.6** (6.8) & 78.0 (\(\pm\)4.9) & **80.3** (3.5) \\ MR (acc) & 86.8 (\(\pm\)0.9) & 87.2 (\(\pm\)0.9) & **87.5** (0.9) \\ MPQA (acc) & 85.4 (\(\pm\)1.8) & 84.2 (\(\pm\)1.9) & **85.4** (1.9) \\ CRF (acc) & 91.7 (\(\pm\)1.0) & 91.1 (\(\pm\)0.9) & **91.3** (0.3) \\ \hline \end{tabular}
\end{table}
Table 2: Comparison of prompt-based fine-tuning and our EmbedHalluc, using RoBERTa-large.

 

lates methods for better fine-tuning language models with regularization. Zhang et al. (2021) find that fine-tuning can be achieved by: correcting bias in the optimizer, re-initialization of top layers, and training longer. Correcting bias in the optimizer is already fixed by the default optimizer in Huggingface Transformer and training longer surely will lead to further over-fitting in our extreme data scarce scenario. Thus, we consider re-initialization (Re-Init) of top layers as one of our comparisons. We further compare against Mixout Lee et al. (2019), which is shown to be an effective regularization when fine-tuning with a few thousand examples. We used the public code for both of these methods. Since we adapt their code to our extreme data deficient setting, we re-search the hyper-parameters of both methods (including their suggested values). For Re-Init, we search the top 1,2,3,4,5 layers; and for Mixout, we search mixout rate from \(0.1,0.2,...,0.9\) and report their best results in Table 4, using RoBERTa-large. Results for BERT-large-cased can be found in Appendix C.

We find that those two methods fail to alleviate the over-fitting problem in such extreme setting, though they have been to be effective when given a few thousands examples.

## 5 Comparing to Adversarial Training

Adversarial training adds noise into the training data to increase the robustness of a model. It has been shown that adversarial training can also improve the performance of language models. Here, we compare EmbedHalluc to two recent adversarial training methods, freeLB Zhu et al. (2020) and SMART Jiang et al. (2020) adapted to our setting. For freeLB, we use the publicly available code and suggested hyper-parameters for each task. In addition to the default batch size and learning rate used in the baseline fine-tuning and EmbedHalluc, we also search additional batch sizes and learning rates for freeLB. We use the default setting for SMART. As shown in Table 5, with one exception, our method largely outperforms freeLB and SMART.

## 6 Limitations

While EmbedHalluc works well empirically, it relies on hallucinating non-interpretable embeddings to facilitate the learning process. Besides, the learning of cWGAN requires careful human attention to maintain a stable training.

## 7 Conclusion

In this paper, we introduce an embedding hallucination method for data augmentation for few-shot learning, based on cWGAN. The proposed method improves over the baselines in 15 tasks and outperforms a common augmentation method, and two recent regularization methods.

\begin{table}
\begin{tabular}{l l l l l} \hline Task & fine-tuning & EmbedHalluc & Re-init & Mixout \\ \hline SST-2 & 76.8 (4.2) & **82.6** (5.6) & 82.5 (1.9) & 78.5 (9.4) \\ Subj & 90.3 (1.5) & **91.3** (0.8) & 91.1 (2.4) & 90.3 (0.8) \\ SST-5 & 40.6 (2.2) & (3.1) & 42.1 (9.1) & 97.5 (3.0) \\ CoLA & 36.0 (9.9) & **39.7** (10.8) & 33.4 (8.1) & 38.6 (5.9) \\ TREC & 83.0 (4.9) & **88.1** (2.5) & 81.8 (5.6) & 86.0 (3.4) \\ MNLI & 41.6 (5.2) & **48.0** (5.9) & 43.7 (4.3) & 42.7 (4.6) \\ mml & 42.7 (5.9) & **49.7** (10.5) & 45.2 (4.8) & 45.0 (5.4) \\ SNLI & 52.9 (6.7) & **54.4** (3.4) & 52.1 (2.2) & 53.7 (3.8) \\ QNLI & 55.3 (2.7) & **60.2** (5.3) & 59.8 (5.0) & 57.1 (5.5) \\ QQP & 59.8 (6.6) & **64.6** (5.0) & 60.2 (10.6) & 62.4 (6.0) \\ RTE & 52.9 (1.4) & 53.4 (1.7) & 52.5 (4.5) & **53.5** (2.5) \\ MRPC & 76.5 (5.2) & **78.7** (1.9) & 67.0 (20.1) & 77.4 (2.7) \\ MRR & 74.5 (5.9) & **79.4** (5.5) & 71.3 (9.0) & 67.6 (10.0) \\ MPQA & 65.0 (1.5) & **70.1** (7.0) & **72.8** (7.8) & 68.8 (3.5) \\ CR & 71.7 (7.5) & 75.1 (5.6) & **80.7** (5.2) & 75.6 (8.6) \\ \hline \end{tabular}
\end{table}
Table 4: Comparisons of EmbedHalluc to Re-init and Mixout, using RoBERTa-large as base models and conventional fine-tuning as the base learning method.

\begin{table}
\begin{tabular}{l l l l l} \hline Task & fine-tuning & EmbedHalluc & EDA & SSL \\ \hline SST-2 & 76.8 (4.2) & 82.6 (5.6) & 82.5 (3.6) & **83.2** (6.0) \\ Subj & 90.3 (1.5) & **91.3** (0.8) & 89.2 (2.0) & 91.1 (1.0) \\ SST-5 & 40.6 (2.2) & 43.0 (1.5) & 38.8 (3.7) & **41.7** (1.9) \\ CALA & 36.0 (9.9) & **39.7** (10.8) & 35.5 (11.0) & 39.6 (11.8) \\ TREC & 83.0 (4.9) & **88.1** (2.5) & 84.0 (1.9) & 87.4 (3.4) \\ MNLI & 41.6 (5.2) & **48.0** (9.5) & 42.0 (3.9) & 43.9 (4.2) \\ MNLI-mmm & 42.7 (5.9) & **49.7** (10.5) & 44.2 (4.3) & 45.6 (4.7) \\ SNLI & 52.9 (6.7) & 54.4 (3.4) & 48.0 (4.7) & **54.9** (7.7) \\ QNLI & 55.3 (2.7) & **60.2** (5.3) & 58.7 (3.5) & 53.6 (1.8) \\ QQP & 59.6 (8.6) & **64.6** (5.0) & 60.7 (6.8) & 63.2 (7.1) \\ RTE & 52.9 (1.4) & 53.4 (1.7) & 53.0 (4.9) & **53.9** (1.2) \\ MRPC & 76.5 (5.2) & **78.7** (1.9) & 73.8 (7.5) & 77.3 (5.4) \\ MRR & 74.5 (5.9) & **79.4** (5.5) & 78.1 (2.5) & 77.9 (4.9) \\ MPQA & 65.0 (1.5) & 70.1 (7.0) & **72.8** (7.8) & 68.8 (3.5) \\ CR & 71.7 (7.5) & 75.1 (5.6) & **80.7** (5.2) & 75.6 (8.6) \\ \hline \end{tabular}
\end{table}
Table 3: Comparison of EmbedHalluc, EDA, and SSL by pseudo-labeling, using RoBERTa-large as the base model and conventional fine-tuning as the base learning method.

 

Ethics Statement

As far as we are aware, our proposed work does not have any explicit ethical concerns. However, our work relies on pre-trained language models, which have been shown to be biased in prior work (Liang et al., 2021). As such, users of such models, specially for sensitive applications, should be aware of and if possible address such issues.

## References

* Arjovsky et al. (2017) Martin Arjovsky, Soumith Chintala, and Leon Bottou. 2017. Wasserstein generative adversarial networks. In _Proceedings of the 34th International Conference on Machine Learning_, volume 70 of _Proceedings of Machine Learning Research_, pages 214-223. PMLR. Arthaud et al. (2021) Farid Arthaud, Rachel Bawden, and Alexandra Birch. 2021. Few-shot learning through contextual data augmentation. In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_, pages 1049-1062, Online. Association for Computational Linguistics.
* Bansal et al. (2020) Trapit Bansal, Rishikesh Jha, Tsendsuren Munkhdalai, and Andrew McCallum. 2020. Self-supervised meta-learning for few-shot natural language classification tasks. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 522-534.
* Basu et al. (2021) Samyadeep Basu, Karine lp Kiun Chong, Amr Sharaf, Alex Fischer, Vishal Rohra, Michael Amoake, Hazem El-Hammamy, Eli Nosakhare, Vijay Ramani, and Benjamin Han. 2021. Semi-supervised few-shot intent classification and slot filling. _CoRR_, abs/2109.08754.
* Casante-Bonilla et al. (2021) Paola Cascante-Bonilla, Fuwen Tan, Yanjun Qi, and Vicente Ordonez. 2021. Curriculum labeling: Revisiting pseudo-labeling for semi-supervised learning. In _Proceedings of the AAAI Conference on Artificial Intelligence_, volume 35, pages 6912-6920.
* de Lichty et al. (2021) Cyprien de Lichty, Alexa AI Amazon, Hadrien Glaude, and William Campbell. 2021. Meta-learning for few-shot named entity recognition. _MetaNLP 2021_, page 44.
* Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language understanding. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers)_, pages 4171-4186. Association for Computational Linguistics.
* Ding et al. (2021) Ning Ding, Guangwei Xu, Yulin Chen, Xiaobin Wang, Xu Han, Pengjun Xie, Hai-Tao Zheng, and Zhiyuan Liu. 2021. Few-nerd: A few-shot named entity recognition dataset. In _ACL-IJCNLP_.
* Dodge et al. (2020) Jesse Dodge, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannaneh Hajishirzi, and Noah A. Smith. 2020. Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping. _ArXiv_, abs/2002.06305.
* Fabbri et al. (2021) Alexander Fabbri, Simeng Han, Haoyuan Li, Haoran Li, Marjan Ghazvininejad, Shafiq Joty, Dragomir Radev, and Yashar Mehdad. 2021. Improving zero and few-shot abstractive summarization with intermediate fine-tunning and data augmentation. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 704-717, Online. Association for Computational Linguistics.
* Gao et al. (2021) Tianyu Gao, Adam Fisch, and Danqi Chen. 2021. Making pre-trained language models better few-shot learners. In _Association for Computational Linguistics (ACL)_.
* Goodfellow et al. (2014) Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In _Advances in Neural Information Processing Systems_, volume 27. Curran Associates, Inc.
* Guo et al. (2021) Demi Guo, Alexander Rush, and Yoon Kim. 2021. Parameter-efficient transfer learning with diff pruning. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, pages 4884-4896, Online. Association for Computational Linguistics.
* Hariharan and Girshick (2017) Bharath Hariharan and Ross Girshick. 2017. Low-shot visual recognition by shrinking and hallucinating features. In _Proceedings of the IEEE International Conference on Computer Vision_, pages 3018-3027.
* Houlsby et al. (2019) Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attiaryan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for nlp. In _International Conference on Machine Learning_, pages 2790-2799. PMLR.
* Hua et al. (2020) Yuncheng Hua, Yuan-Fang Li, Gholamreza Haffari, Guilin Qi, and Tongtong Wu. 2020. Few-shot complex knowledge base question answering via meta reinforcement learning. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 5827-5837.
* Jian et al. (2020) Yiren Jian, Karim Ahmed, and Lorenzo Torresani. 2020. Task meta-transfer from limited parallel labels. _Meta-Learning workshop, NeurIPS 2020_ 

Yiren Jian and Chongyang Gao. 2021Metapix: Domain transfer for semantic segmentation by meta pixel weighting. _Image and Vision Computing_, 116:104334.
* Jian et al. (2022) Yiren Jian, Chongyang Gao, and Soroush Vosoughi. 2022. Contrastive learning for prompt-based few-shot language learners. In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_. Association for Computational Linguistics.
* Jian and Torresani (2022) Yiren Jian and Lorenzo Torresani. 2022. Label hallucination for few-shot classification. In _Proceedings of the AAAI Conference on Artificial Intelligence_.
* Jiang et al. (2020) Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Tuo Zhao. 2020. SMART: Robust and efficient fine-tuning for pre-trained natural language models through principled regularized optimization. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 2177-2190, Online. Association for Computational Linguistics.
* Karimi et al. (2021) Akbar Karimi, Leonardo Rossi, and Andrea Prati. 2021. AEDA: an easier data augmentation technique for text classification. In _Findings of the Association for Computational Linguistics: EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 16-20 November, 2021_, pages 2748-2754. Association for Computational Linguistics.
* Kumar et al. (2019) Varun Kumar, Hadrien Glaude, Cypriend de Lichy, and Wlliam Campbell. 2019. A closer look at feature space data augmentation for few-shot intent classification. In _Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019)_, pages 1-10, Hong Kong, China. Association for Computational Linguistics.
* Lazarou et al. (2022) Michalis Lazarou, Tania Stathaki, and Yannis Avrithis. 2022. Tensor feature hallucination for few-shot learning. In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_, pages 3500-3510.
* Lee et al. (2019) Cheolhyoung Lee, Kyunghyun Cho, and Wanmo Kang. 2019. Mixout: Effective regularization to finetune large-scale pretrained language models. In _International Conference on Learning Representations_.
* Lester et al. (2021) Brian Lester, Ramil Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_, pages 3045-3059, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
* Li and Zhang (2021) Judith Yue Li and Jiong Zhang. 2021. Semi-supervised meta-learning for cross-domain few-shot intent classification. _MetaNLP 2021_, page 67.
* Li et al. (2020) Kai Li, Yunlu Zhang, Kunpeng Li, and Yun Fu. 2020. Adversarial feature hallucination networks for few-shot learning. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_, pages 13470-13479.
* Li and Liang (2021) Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, pages 4582-4597, Online. Association for Computational Linguistics.
* Liang et al. (2021) Paul Pu Liang, Chiyu Wu, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2021. Towards understanding and mitigating social biases in language models. In _International Conference on Machine Learning_, pages 6565-6576. PMLR.
* Luo et al. (2021) Qinxuan Luo, Lingfeng Wang, Jingguo Lv, Shiming Xiang, and Chunhong Pan. 2021. Few-shot learning via feature hallucination with variational inference. In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_, pages 3963-3972.
* Ram et al. (2021) Ori Ram, Yuval Kirstain, Jonathan Berant, Amir Globerson, and Omer Levy. 2021. Few-shot question answering by pretraining span selection. _arXiv e-prints_, pages arXiv-2101.
* Ren et al. (2018a) Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B Tenenbaum, Hugo Larochelle, and Richard S Zemel. 2018a. Meta-learning for semi-supervised few-shot classification. In _International Conference on Learning Representations_.
* Ren et al. (2018b) Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. 2018b. Learning to reweight examples for robust deep learning. In _ICML_.
* Schick and Schutze (2021) Timo Schick and Hinrich Schutze. 2021. Exploiting cloze-questions for few-shot text classification and natural language inference. In _EACL_.
* Sharaf et al. (2020) Amr Sharaf, Hany Hassan, and Hal Daume III. 2020. Meta-learning for few-shot nmt adaptation. _ACL_, page 43.
* Tam et al. (2021) Derek Tam, Rakesh R Menon, Mohit Bansal, Shashank Srivastava, and Colin Raffel. 2021. Improving and simplifying pattern exploiting training. In _Empirical Methods in Natural Language Processing (EMNLP)_.
* Tian et al. (2020) Yonglong Tian, Yue Wang, Dilip Krishnan, Joshua B Tenenbaum, and Phillip Isola. 2020. Rethinking few-shot image classification: a good embedding is all you need? In _European Conference on Computer Vision_, pages 266-282. Springer.
* Tjio et al. (2022) Gabriel Tjio, Ping Liu, Joey Tianyi Zhou, and Rick Siow Mong Goh. 2022. Adversarial semantic hallucination for domain generalized semantic segmentation.

 

In _IEEE Winter Conf. on Applications of Computer Vision_.
* Tong et al. (2021) Meihan Tong, Shuai Wang, Bin Xu, Yixin Cao, Minghui Liu, Lei Hou, and Juanzi Li. 2021. Learning from miscellaneous other-class words for few-shot named entity recognition. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, pages 6236-6247, Online. Association for Computational Linguistics.
* Wang et al. (2019) Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In _International Conference on Learning Representations_.
* Wang et al. (2020) Jiancong Wang, Yuhua Chen, Yifan Wu, Jianbo Shi, and James Gee. 2020. Enhanced generative adversarial network for 3d brain mri super-resolution. In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_, pages 3627-3636.
* Wang et al. (2018) Yu-Xiong Wang, Ross Girshick, Martial Hebert, and Bharath Hariharan. 2018. Low-shot learning from imaginary data. In _Proceedings of the IEEE conference on computer vision and pattern recognition_, pages 7278-7286.
* Wei et al. (2021) Jason Wei, Chengyu Huang, Soroush Vosoughi, Yu Cheng, and Shiqj Xu. 2021. Few-shot text classification with triplet networks, data augmentation, and curriculum learning. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 5493-5500, Online. Association for Computational Linguistics.
* Wei and Zou (2019) Jason Wei and Kai Zou. 2019. EDA: Easy data augmentation techniques for boosting performance on text classification tasks. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 6383-6389, Hong Kong, China. Association for Computational Linguistics.
* Zhang et al. (2021) Tianyi Zhang, Felix Wu, Arzoo Katiyar, Kilian Q Weinberger, and Yoav Artzi. 2021. Revisiting few-sample {bert} fine-tuning. In _International Conference on Learning Representations_.
* Zhu et al. (2020) Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Goldstein, and Jingjing Liu. 2020. Freelb: Enhanced adversarial training for natural language understanding. In _International Conference on Learning Representations_.
* Zhu et al. (2017) Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In _Computer Vision (ICCV), 2017 IEEE International Conference on_ 



## Appendix A Best Learning Rate for RoBERTa-prompt

Here, we provide best learning rates (LR, searched from \(1e^{-5},5e^{-6},1e^{-6}\) as discussed in main paper) for \(\mathcal{L}_{\text{halluc}}\) of EmbedHalluc for each task used in RoBERTa-large prompt-based fine-tuning.

Qualitatively similar to what we observe with experiments using RoBERTa-large in the main paper, Re-Init and Mixout fail to outperform EmbedHalluc in most tasks, with the exceptions of SNLI and QNLI.

## Appendix D Learning Rate for Baselines

The baseline has only one loss \(\mathcal{L}_{\text{real}}\), whereas we are learning with an additional loss \(\mathcal{L}_{\text{halluc}}\), making the total loss to be \(\mathcal{L}_{\text{real}}+\mathcal{L}_{\text{halluc}}\). The learning rate for \(\mathcal{L}_{\text{real}}\) in the baselines and ours are kept the same. Note that we do not search for this learning rate for our method. We choose \(1e^{-5}\), which is the most common learning rate to finetune BERT/RoBERTa. As we show in Table D.1, this learning rate produces reasonably good results for the baselines, being the best for 13 tasks and only marginally under-performing in the other 2 tasks. The results in Table D.1 are generated by running the baselines with a batch size of 2 and different learning rates \(1e^{-5}\), \(2e^{-5}\), \(5e^{-5}\) suggested by Gao et al. (2021).