SlowGuess/NAACL_2022_OCR huggingface dataset:
Yiren Jian
yiren.jian.gr@dartmouth.edu
Chongyang Gao
cygao@u.northwestern.edu
The impressive performance of GPT-3 using natural language prompts and in-context learning has inspired work on better fine-tuning of moderately-sized models under this paradigm. Following this line of work, we present a contrastive learning framework that clusters inputs from the same class for better generality of models trained with only limited examples. Specifically, we propose a supervised contrastive framework that clusters inputs from the same class under different augmented "views" and repel the ones from different classes. We create different "views" of an example by appending it with different language prompts and contextual demonstrations. Combining a contrastive loss with the standard masked language modeling (MLM) loss in prompt-based few-shot learners, the experimental results show that our method can improve over the state-of-the-art methods in a diverse set of 15 language tasks. Our framework makes minimal assumptions on the task or the base model, and can be applied to many recent methods with little modification. The code will be made available at: https://github.com/yiren-jian/LM-SupCon.
The prompt-based fine-tuning method reduces the gap between pre-training and fine-tuning by forming the fine-tuning task into a masking language problem. A language prompt is a piece of text appended to the query input enabling the model to come up with better predictions (Schick and Schütze, 2021; Tam et al., 2021). For instance, by feeding a language model with "The story is not worth reading, a truly _ one.", the model assigns a higher probability for the blank to be filled with "terrible" than "great". Here, "a truly _ one." is called the template of the prompt and "terrible" or "great" is the label word. Recently, LM-BFF (Gao et al., 2021) shows that appending demonstrations
(e.g."This is an amazing movie, a truly great one") to inputs can help the model to better understand the label word, leading to further improved results.
In this work, we show that Supervised Contrastive Learning (SupCon) (Khosla et al., 2020) at the feature space can be beneficial during the finetuning of prompt-based few-shot language learners, with proper data augmentation.
Data augmentation is the key component of SupCon. While there exists many augmentation techniques like Cutmix (Yun et al., 2019), Mixup (Zhang et al., 2018) in computer vision and EDA (Wei and Zou, 2019), AEDA (Karimi et al., 2021) for text, data augmentation remains challenging.
However, prompt-based few-shot learners with demonstrations actually provide us with a natural way to create multiple "views" (augmentations) of a single example, i.e., for a fixed set of label words, we can sample different templates and different demonstrations to append to the input text (shown in Figure 1). This allows us to construct diverse input texts that are consistent and complete. By applying SupCon to cluster the above two example inputs with very different contents but the same label, our method is able to obtain an additional supervision at the feature space which is crucial if we are only given a few labeled examples.
- A Supervised Contrastive Learning framework for prompt-based few-shot learners.
- An effective data augmentation method using prompts for contrastive learning with prompt-based learners.
Few-shot Learning is often tackled by meta learning (Li and Zhang, 2021; Bansal et al., 2020; Sharaf et al., 2020; Jian et al., 2020; Jian and Gao, 2021), data augmentation (Jian et al., 2022; Jian and Torresani, 2022; Arthaud et al., 2021; Wei et al., 2021;
Human Language Technologies, pages 5577 - 5587
Figure 1: Overview of our proposed method. Besides the standard prompt-base MLM loss on label words "great" and "terrible", we introduce a SupCon loss on multi-views of input text. The positive pair is sentences (with sampled templates and/or demonstrations) in the same class, e.g.  $\mathrm{sent}_1$  and  $\mathrm{sent}_3$ , or itself with a different template and demonstrations, e.g.  $\mathrm{sent}_1$  and  $\mathrm{sent}_2$ . The negative sentence pair is input sentences (with sampled templates and/or demonstrations) in different classes, e.g.  $\mathrm{sent}_1$  and  $\mathrm{sent}_0$ .
Kumar et al., 2019). Inspired by the in-context learning of GPT-3, prompt-based fine-tuning (Gao et al., 2021; Tam et al., 2021; Schick and Schütze, 2021) recently becomes dominant in NLP. Basu et al. (2021) applies contrastive learning in their few-shot semi-supervised intent classification, by using EDA (Wei and Zou, 2019) as augmentation method. Different from Basu et al. (2021), our method applies to prompt-based fine-tuning, and we show in experiments that our proposed augmentation outperforms EDA.
Supervised Contrastive Loss. SupCon is a special form of contrastive learning (Chen et al., 2020a,b; Tian et al., 2020a,b,c; Liu et al., 2021; Xiong et al., 2020) that clusters two augmented batches at the class level in the feature space. Let  $\tilde{x}_{2k-1}, \tilde{x}_{2k}$  be two augmented views of an input batch  $x_k$ ; and  $z_{2k}, z_{2k-1}$  to be the features of  $\tilde{x}_{2k-1}, \tilde{x}_{2k}$ . Then SupCon loss can be computed as
$$
\mathcal {L} _ {\operatorname {S u p C o n}} = \operatorname {S u p C o n} \left(z _ {2 k - 1}, z _ {2 k}, y _ {k}\right) \tag {1}
$$
where  $y_{k}$  is the label for batch  $x_{k}$ . The details of SupCon can be found in Khosla et al. (2020).
Problem formulation. Following the few-shot setting in LM-BFF, we assume to have access to a pre-trained language model  $\mathcal{M}$ , datasets  $\mathcal{D}_{\mathrm{train}}$  and  $\mathcal{D}_{\mathrm{test}}$  with label space  $\mathcal{V}$ . There are only  $K = 16$  examples per class in  $\mathcal{D}_{\mathrm{train}}$ .
Fine-tuning with prompts and demonstrations. Prompt-based methods treat a classification problem as a masked language modeling (MLM) problem. They take as input a sentence (sent) and a masked template (temp) (i.e.,  $x_{\text{prompt}} =$
sent, temp([mask]), and find the best token to fill in the [mask]. This leads to a MLM loss  $\mathcal{L}_{\mathrm{MLM}} = \mathrm{MLM}(x_{\mathrm{prompt}}, y)$ , where  $y$  is the label word corresponding to  $x_{\mathrm{prompt}}$ . LM-BFF (Gao et al., 2021) further appends demonstrations of label words to improve the results:  $x_{\mathrm{prompt} + \mathrm{demo}} = \mathrm{sent}_0$ ,  $\mathrm{temp}_0([\mathrm{mask}])$ ,  $\mathrm{sent}_i$ ,  $\mathrm{temp}_0(\mathrm{word}_i)$ , where  $\mathrm{word}_i$  is the label word for  $\mathrm{sent}_i$ , and  $\mathrm{sent}_i$  is sampled from the training set. Then the classification loss becomes:
$$
\mathcal {L} _ {\mathrm {M L M}} = \mathrm {M L M} \left(x _ {\text {p r o m p t + d e m o}}, y\right) \tag {2}
$$
More mathematical formulation can be found in LM-BFF or our Appendix B.
Language-based Supervised Contrastive Loss. For applying SupCon on multi-views of an input text, we need to first obtain two views of a text:
$$
x _ {1} = \operatorname {s e n t} _ {0}, \operatorname {t e m p} _ {0} ([ \text {m a s k} ]), \operatorname {s e n t} _ {i}, \operatorname {t e m p} _ {0} (\operatorname {w o r d} _ {i})
$$
$$
x _ {2} = \operatorname {s e n t} _ {0}, \operatorname {t e m p} _ {j} ([ \text {m a s k} ]), \operatorname {s e n t} _ {k}, \operatorname {t e m p} _ {j} (\operatorname {w o r d} _ {k})
$$
where  $x_{1}$  is identical to  $x_{\mathrm{prompt + demo}}$  in LM-BFF. We sample a new template  $(\mathrm{temp}_j)$ , demonstration  $(\mathrm{sent}_k)$  and the corresponding label word  $(\mathrm{word}_k)$  to replace those in  $x_{1}$ , to create a second view of input  $x_{2}$ . With  $x_{1}$  and  $x_{2}$ , we can compute SupCon loss by Equation 1. The total loss is then
$$
\mathcal {L} _ {\text {t o t a l}} = \mathcal {L} _ {\mathrm {M L M}} + \mathcal {L} _ {\mathrm {S u p C o n}} \tag {3}
$$
See our Appendix C for more mathematical details.
Computational overhead. We show the algorithm of our method in Algorithm 1. In general, our method learns from  $\mathcal{L}_{\mathrm{total}} = \mathcal{L}_{\mathrm{MLM}} + \mathcal{L}_{\mathrm{SupCon}}$  whereas baseline LM-BFF learns with  $\mathcal{L}_{\mathrm{MLM}}$  only.
Learning from  $\mathcal{L}_{\mathrm{SupCon}}$  requires one additional forward and backward pass (highlighted in blue in Algorithm 1), leading to an increase of computational cost by  $\times 1.5$ .
Evaluation datasets and protocol. We evaluate our method on 15 classification tasks studied in LM-BFF and follow the same setup as them to allow fair comparisons (see Appendix A for more training details). Contrastive learning algorithms benefit from large batch training. Thus, we report baselines with the same large batch size as ours.
Our method uses a single prompt/template (primary prompt) for the prediction of each task, and a set of prompts (auxiliary prompts) for generating multi-views of inputs for contrastive learning. The primary prompts we used are shown in Appendix D. The auxiliary prompts can be either manually designed or generated by a searching algorithm. In this work, we use the top-20 generated prompts from LM-BFF's project page and we randomly sample templates in these 20 prompts to
Task	LM-BFF	LM-BFF + ours	PET	PET + ours
SST-2 (acc)	89.2 (1.3)	90.6 (0.1)	88.4 (1.0)	89.9 (0.6)
Subj (acc)	88.6 (3.3)	90.4 (1.1)	89.2 (1.5)	90.6 (1.6)
SST-5 (acc)	47.9 (0.8)	49.5 (1.1)	46.0 (0.9)	48.8 (1.2)
CoLA (Matt.)	6.1 (5.3)	10.2 (5.8)	3.5 (3.4)	5.9 (3.3)
TREC (acc)	82.8 (3.1)	83.3 (1.5)	77.8 (9.1)	82.3 (4.6)
MNLI (acc)	61.0 (2.1)	64.0 (2.0)	58.2 (1.1)	58.9 (3.1)
MNLI-mm (acc)	62.5 (2.1)	65.5 (2.7)	59.8 (1.2)	61.0 (3.3)
SNLI (acc)	66.9 (2.4)	69.9 (2.4)	63.1 (2.5)	65.7 (3.9)
QNLI (acc)	60.7 (1.7)	66.4 (3.5)	61.5 (3.3)	63.5 (3.7)
QQP (acc)	62.5 (2.6)	68.8 (3.8)	61.9 (3.5)	65.7 (4.3)
RTE (acc)	64.3 (2.7)	65.1 (3.5)	60.9 (4.7)	65.1 (3.5)
MRPC (F1)	75.5 (5.2)	78.2 (3.1)	70.6 (6.0)	75.7 (6.1)
MR (acc)	83.3 (1.4)	85.8 (0.6)	85.0 (0.6)	85.2 (0.9)
MPQA (acc)	83.6 (1.8)	84.6 (1.5)	81.3 (2.6)	81.8 (2.4)
CR (acc)	88.9 (1.0)	89.4 (1.0)	89.3 (1.0)	90.5 (0.5)
Table 1: Few-shot experiments of baseline methods and ours. LM-BFF is a prompt-based method with demonstrations of label words and PET is one without demonstrations. The experimental results show the means and standard deviations from 5 different train-test splits.
produce second views of our inputs. Unless otherwise noted, we apply both random templates and random demonstrations to create second views of inputs for the contrastive learning.
4.1 Main results on 15 tasks
We use RoBERTa-base (see Appendix E for RoBERTa-large). We compare ours with LM-BFF (a method w/ demonstrations) and PET (Schick and Schütze, 2021) (a method w/o demonstration).
Table 1 shows that our SupCon loss can consistently boost the performance of baseline prompt-based fine-tuning method LM-BFF. The introduction of SupCon loss has a maximum improvement of  $6.3\%$  in QQP and an average improvement of  $2.5\%$  across 15 tasks, likely due to the more generalized representations learned by SupCon. On average, the greater improvements by our model can be seen on the more difficult tasks (see Appendix 5 for more detail).
We want to emphasize that the input for baseline LM-BFF already appends different randomly sampled demonstrations at each tuning iteration. Thus, the improvement of our method can not be attributed to the diversity of inputs when learning from  $\mathcal{L}_{\mathrm{MLM}}$  of Equation 3, but to the  $\mathcal{L}_{\mathrm{SupCon}}$ .
Table 1 also shows that our method works well even for prompt-based methods without demonstrations. PET, which is a method without demonstrations, works consistently worse than LM-BFF. However, with the additional SupCon loss, the few-shot performances of PET can be increased by an average of  $2.3\%$ . And the gap between having and not having demonstrations can be largely closed
(see LM-BFF vs. PET+ours in Table 1). In some tasks, e.g., SST-2, SST-5, QNLI, QQP, RTE MRPC, MR, and CR, the contribution of our SupCon loss can be even larger than the sole use of the demonstrations for label words.
4.2 SupCon vs. other losses
Task	LM-BFF	LM-BFF +Dec	LM-BFF +Dec +Lab	LM-BFF +ConCal	LM-BFF +ours
SST-2	89.2 (1.3)	90.1 (0.6)	90.6 (0.5)	88.5 (2.0)	90.6 (0.1)
Subj	88.6 (3.3)	87.3 (3.6)	88.4 (4.9)	83.8 (7.3)	90.4 (1.1)
SST-5	47.9 (0.8)	47.2 (1.0)	46.5 (0.7)	47.9 (1.1)	49.5 (1.1)
CoLA	6.1 (5.3)	9.8 (6.5)	7.2 (5.2)	6.7 (4.6)	10.2 (5.8)
TREC	82.8 (3.1)	81.9 (3.0)	82.3 (3.0)	71.1 (7.0)	83.3 (1.5)
MNLI	61.0 (2.1)	61.3 (2.1)	59.4 (1.3)	61.0 (0.8)	64.0 (2.0)
-mm	62.5 (2.1)	63.2 (2.1)	61.4 (1.6)	62.5 (0.8)	65.5 (2.7)
SNLI	66.9 (2.4)	67.0 (3.1)	65.8 (2.1)	67.0 (2.9)	69.9 (2.4)
QNLI	60.7 (1.7)	60.0 (2.5)	60.2 (2.0)	60.9 (2.0)	66.4 (3.5)
QQP	62.5 (2.6)	69.0 (1.7)	65.4 (1.2)	62.2 (2.7)	68.8 (3.8)
RTE	64.3 (2.7)	65.6 (1.5)	65.3 (2.4)	60.2 (1.9)	65.1 (3.5)
MRPC	75.5 (5.2)	69.4 (7.0)	66.5 (7.0)	78.3 (3.1)	78.2† (3.1)
MR	83.3 (1.4)	85.0 (1.0)	84.6 (1.2)	84.0 (1.4)	85.8 (0.6)
MPQA	83.6 (1.8)	82.3 (1.9)	84.3 (1.4)	72.3 (13.4)	84.6 (1.5)
CR	88.9 (1.0)	89.3 (0.6)	89.6 (0.7)	87.7 (1.1)	89.4 (1.0)
We further show that our method outperforms two latest methods that are designed to improve prompt-based language models. In ADAPET (Tam et al., 2021), the authors replace the traditional CrossEntropy loss with Decoupling Label Loss and Label Condition Loss in the prompt-based finetuning method PET, without demonstrations. Contextual Calibration (Zhao et al., 2021) calibrates the output probabilities by considering context-free inputs, i.e., " " or "N/A". (Further see Appendix I)
From Table 2 we observe that on 12 tasks our  $\mathcal{L}_{\mathrm{SupCon}}$  outperforms the other losses, while performs on-par in other tasks. Contextual Calibration does not achieve good results overall. We speculate two reasons for this. First, Contextual Calibration is designed for large models without fine-tuning like GPT (zero-shot setting). Second, the form of in-context learning in Contextual Calibration is different from the demonstrations we study here.
4.3 Ensemble vs. our single model
Our method uses 20 generated templates (auxiliary prompts) to construct multi-views of input sentences. But only a single prompt (primary prompt) and one set of label words are used for main predictions. Thus, there is only a single model from our method. Here, we compare our model to an ensemble comprised of 20 models trained separately
Table 2: Comparing our SupCon loss with Decoupling Label Loss (Dec), Label Condition Loss (Lab), and Contextual Calibration (ConCal).  $\dagger$  We can achieve stronger performance  $80.0 \pm 1.8$  by fixing templates/demonstrations when creating the second view of the input (see Section 6.2).
Task	LM-BFF +ours	LM-BFF ensemble
SST-5 (acc)	49.5 (1.1)	48.0 (0.8)
CoLA (Matt.)	10.2 (5.8)	7.5 (4,7)
MNLI (acc)	63.3 (2.4)	62.2 (1.8)
MNLI-mm (acc)	65.1 (2.4)	64.0 (1.8)
QNLI (acc)	66.4 (3.5)	63.8 (2.7)
MR (acc)	85.8 (0.6)	85.7 (0.7)
Table 3: Comparing our single model trained with SupCon loss to an ensemble of 20 models.
with the 20 prompts. From Table 3, we find that our method even outperforms the ensemble with  $20 \times$  more number of parameters, showing that it is a more efficient way to make use of the generated prompts. We speculate that because of the over-fitting nature of few-shot learners, members in the ensemble fail to produce substantial diverse prediction distributions.
Figure 2: The average improvements achieved by our method on the top K hardest tasks, where K goes from 1 to 15.
5 Improvements vs. Task Difficulty
Here, we show that the improvements achieved by our method are greater for tasks with higher difficulty. To show this, we first sort the 15 tasks by base (LM-BFF) performance and use this ranking as a proxy for the difficulty of the task. Next, we report the average improvements achieved by our
method on the top K hardest tasks, where K goes from 1 to 15. Figure 2 shows these results. The first bar corresponds to the improvement achieved by our method on the hardest task, the second bar corresponds to the average improvement achieved by our method on the hardest and second-hardest tasks, and so on. The last bar corresponds to the average improvement on all 15 tasks.
6 Comparative Experiments
6.1 Input augmentation
The success of contrastive learning heavily relies on the data augmentation. Our method takes advantage of prompt-based language learners and naturally creates multi-views of a single input by appending it with different templates and/or demonstrations. Compared to EDA which includes synonym replacement (SR), random insertion (RI), random swap (RS) and random deletion (RD), our strategy for augmentation does not lead to incomplete and inconsistent sentences, while introducing adequate variations for effective learning.
Task	LM-BFF	SR	RI	RS	RD	EDA	ours
SST-2	89.2	90.7	90.8	90.7	90.7	90.5	90.6
Subj	88.6	90.6	90.8	91.0	90.5	89.1	90.4
SST-5	47.9	47.7	49.2	48.2	47.9	46.7	49.5
CoLA	6.1	5.8	6.5	4.9	4.0	3.9	10.2
TREC	82.8	78.1	80.7	79.0	80.7	80.6	83.3
MNLI	61.0	61.8	62.4	61.0	58.1	58.9	64.0
-mm	62.5	63.6	64.8	62.7	60.3	60.9	65.5
SNLI	66.9	63.1	66.4	67.2	65.2	62.2	69.9
QNLI	60.7	65.3	65.3	67.4	64.8	62.5	\( 66.4^† \)
QQP	62.5	64.5	65.8	68.0	63.2	61.0	68.8
RTE	64.3	61.4	61.4	61.3	62.1	61.1	65.1
MRPC	75.5	77.6	77.7	79.3	78.7	79.1	\( 78.2^† \)
MR	83.3	85.5	85.5	85.5	85.3	85.6	85.8
MPQA	83.6	82.2	84.4	84.4	83.9	82.8	84.6
CR	88.9	88.9	88.2	88.3	88.5	87.1	89.4
The results in Table 4 are obtained by applying SR, RI, RS, RD, EDA for  $10\%$  of input tokens (Results for  $20\%$  are in Appendix F). In contrast to ours, EDA, etc., for SupCon lead to worse performances than the baseline method in many tasks.
6.2 Variable templates, demonstrations
So far, we have shown the results by our method generating multi-views of inputs by appending both random templates and demonstrations. However, we find that in some tasks fixed templates with
random demonstrations or random templates with fixed demonstration lead to even stronger performances (see Table 5). For example, sampling demonstrations with fixed templates for MRPC achieves a very strong result (80.0), outperforming all other methods in Table 4.
Table 4: Comparing our random templates/demonstrations as data augmentation to SR, RI, RS, RD and EDA. Numbers are average of 5 train-test splits. † We can achieve stronger performance by fixing templates/demonstrations when creating the second view of the input, see Section 6.2.
Task	LM-BFF	- demo + temp	+ demo - temp	+ demo + temp
SST-2 (acc)	89.2 (1.3)	90.8 (0.3)	90.5 (0.4)	90.6 (0.1)
Subj (acc)	88.6 (3.3)	90.8 (0.8)	90.6 (1.2)	90.4 (1.1)
SST-5 (acc)	47.9 (0.8)	49.3 (1.7)	48.9 (1.8)	49.5 (1.1)
CoLA (Matt.)	6.1 (5.3)	9.9 (7.5)	8.5 (5.6)	10.2 (5.8)
TREC (acc)	82.8 (3.1)	83.4 (0.5)	86.7 (1.0)	83.3 (1.5)
MNLI (acc)	61.0 (2.1)	63.4 (3.3)	63.0 (3.2)	64.0 (2.0)
MNLI-mm (acc)	62.5 (2.1)	65.5 (3.1)	64.9 (3.4)	65.5 (2.7)
SNLI (acc)	66.9 (2.4)	69.8 (2.4)	68.5 (1.9)	69.9 (2.4)
QNLI (acc)	60.7 (1.7)	65.4 (3.1)	67.0 (3.6)	66.4 (3.5)
QQP (acc)	62.5 (2.6)	68.9 (3.2)	67.8 (1.4)	68.8 (3.8)
RTE (acc)	64.3 (2.7)	64.9 (3.8)	62.6 (2.8)	65.1 (3.5)
MRPC (F1)	75.5 (5.2)	79.0 (1.8)	80.0 (1.8)	78.2 (3.1)
MR (acc)	83.3 (1.4)	85.8 (0.7)	85.4 (0.3)	85.8 (0.6)
MPQA (acc)	83.6 (1.8)	84.0 (1.9)	84.1 (2.0)	84.6 (1.5)
CR (acc)	88.9 (1.0)	88.6 (0.6)	88.2 (1.0)	89.4 (1.0)
Table 5: Different strategies to construct multi-views of input sentences. Fixed demonstrations and sampling templates (- demo + temp), sampling demonstrations and fixed templates (+ demo - temp) and sampling both demonstrations and templates (+ demo + temp).
Since SupCon clusters examples on class level, our framework applies only to classification tasks. Also, our framework requires large GPU memory, as SupCon is an in-batch contrastive loss that needs a large batch size.
We proposed a novel supervised contrastive learning framework and an effective augmentation method using prompts that can boost the performance of prompt-based language learners and outperform recent work on 15 few-shot tasks.
As far as we are aware, our proposed work does not have any ethical considerations. However, our work relies on pre-trained language models, which have been shown to be biased in prior work (Liang et al., 2021). As such, users of such models should be aware of and if possible address such issues. The data and the code for this work will be made available to aid reproducibility.
Farid Arthaud, Rachel Bawden, and Alexandra Birch. 2021. Few-shot learning through contextual data augmentation. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 1049-1062, Online. Association for Computational Linguistics.
Trapit Bansal, Rishikesh Jha, Tsendsuren Munkhdalai, and Andrew McCallum. 2020. Self-supervised meta-learning for few-shot natural language classification tasks. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 522-534.
Samyadeep Basu, Karine Ip Kiun Chong, Amr Sharaf, Alex Fischer, Vishal Rohra, Michael Amoake, Hazem El-Hammamy, Ehi Nosakhare, Vijay Ramani, and Benjamin Han. 2021. Semi-supervised few-shot intent classification and slot filling. CoRR, abs/2109.08754.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020a. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 1597-1607. PMLR.
Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey E Hinton. 2020b. Big self-supervised models are strong semi-supervised learners. In Advances in Neural Information Processing Systems, volume 33, pages 22243-22255. Curran Associates, Inc.
Tianyu Gao, Adam Fisch, and Danqi Chen. 2021. Making pre-trained language models better few-shot learners. In Association for Computational Linguistics (ACL).
Yiren Jian, Karim Ahmed, and Lorenzo Torresani. 2020. Task meta-transfer from limited parallel labels. Meta-Learning workshop, NeurIPS 2020.
Yiren Jian and Chongyang Gao. 2021. Metapix: Domain transfer for semantic segmentation by meta pixel weighting. Image and Vision Computing, 116:104334.
Yiren Jian, Chongyang Gao, and Soroush Vosoughi. 2022. Embedding hallucination for few-shot language learning. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics.
Yiren Jian and Lorenzo Torresani. 2022. Label hallucination for few-shot classification. In Proceedings of the AAAI Conference on Artificial Intelligence.