SlowGuess/NAACL_2022_OCR huggingface dataset:
Yaqing Wang*, Tian Xin*, Haoyi Xiong†, Yueyang Li, Zeyu Chen†, Sheng Guo, Dejing Dou Baidu Inc., Beijing, China
{wangyaqing01, tianxin04, xionghaoyi, liyueyang01, chenzeyu01, guosheng, doudejing}@baidu.com
Pre-trained language models (PLMs) can provide a good starting point for downstream applications. However, it is difficult to generalize PLMs to new tasks given a few labeled samples. In this work, we show that Relation Graph augmented Learning (RGL) can improve the performance of few-shot natural language understanding tasks. During learning, RGL constructs a relation graph based on the label consistency between samples in the same batch, and learns to solve the resultant node classification and link prediction problems on the relation graph. In this way, RGL fully exploits the limited supervised information, which can boost the tuning effectiveness. Extensive experimental results show that RGL consistently improves the performance of prompt-based tuning strategies.
Pre-trained language models (PLMs), such as BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019), have become the standard workhorse for nowadays natural language processing tasks. A direct way of leveraging these PLMs is to fine-tune them by taking gradient descent w.r.t. the objective of downstream tasks. However, tuning the large PLM by a few labeled samples has a high risk of overfitting (Dodge et al., 2020; Zhang et al., 2021; Gunel et al., 2020). Besides, as PLMs are trained by an objective different from the downstream tasks, the ability of PLM may not be fully exploited. Recently, prompt-based tuning methods emerge and obtain promising results on tuning PLMs to new tasks with a few labeled samples (Liu et al., 2021). In particular, these methods use prompts to reformulate the
downstream tasks into the same form of pretraining tasks such that the gap between pretraining and fine-tuning is reduced (Brown et al., 2020; Schick and Schütze, 2021a). Concretely, prompt-based tuning strategies rewrite the input sequence into a cloze question with masks (Schick and Schütze, 2021a). The input sequence is rewritten as prompts, while the corresponding label is replaced by answer tokens. Some methods use hard prompts and answers which use text strings with certain semantic meaning (Schick and Schütze, 2021b; Tam et al., 2021; Gao et al., 2021), while others take learnable parameters as soft prompts and answers (Liu et al., 2021; Li and Liang, 2021; Lester et al., 2021). One can use multiple prompts to boost the performance of prompt-based tuning (Brown et al., 2020; Schick and Schütze, 2021b). While the above strategies improve few-shot performance, they pay less attention to exploiting the relations among the limited number of labeled samples.
In this work, we propose a simple yet effective relation graph augmented approach which can enhance the performance of prompt-based tuning strategies PLM in few-shot natural language understanding tasks. Specifically, our proposal aims at fully exploiting the limited supervised information via Relation Graph augmented Learning, we thus call the proposed method RGL. RGL first constructs a batch-wise relation graph, where every node refers to a labeled sample and the edge between nodes refers to the similarity between the two samples. RGL establishes the edge in the relation graph w.r.t whether the two samples are from the same class and regularizes the similarity of representations learned by PLMs between every two samples to fit the edge of the relation graph. RGL can easily scale up as the relation graph is constructed w.r.t. only a mini-batch of sampled data points per iteration. Empirical results on benchmark datasets show that RGL can
<sup>1</sup>Codes are available at https://github.com/PaddlePaddle/PaddleNLP/tree/develop/examples/few_shot/RGL.
Findings of the Association for Computational Linguistics: NAACL 2022, pages 1078 - 1084 July 10-15, 2022 ©2022 Association for Computational Linguistics
consistently improve the performance of prompt-based tuning.
The related works are briefly reviewed below.
Few-Shot Learning. Few-shot learning (FSL) targets at generalizing to new tasks with a few labeled samples (Wang et al., 2020). FSL has been applied to many natural language processing applications such as text classification (Bao et al., 2020) and named entity typing (Yang and Katiyar, 2020). Typical solutions in FSL include data augmentation which directly generate more labeled samples (Dopierre et al., 2021), metric learning which learns to embed samples into a space where samples can be easily discriminated (Geng et al., 2020), and meta-learning which learns a good initialized model from a set of related tasks which is then fine-tune to each task (Bao et al., 2020). Recently, several methods propose to tune pre-trained language models to downstream few-shot tasks (Liu et al., 2021). We follow this line, and further propose to conduct model tuning on learned relation graphs. Our approach can be incorporated into existing prompt-based tuning strategies, increasing the effectiveness of supervised signals and bringing in performance gains.
Graph Structure Learning. Graph structure learning methods target at jointly learning graph structure and node embeddings of input samples (Zhu et al., 2021). Usually, these methods iterate over two steps: (i) estimate the adjacency matrix which encodes graph structure using node embeddings; and (ii) apply graph neural networks (GNNs) on this updated graph to obtain new node embeddings. Recently, graph structure learning has been used to estimate relation graphs among samples to facilitate effective propagation of label information (Satorras and Estrach, 2018; Rodríguez et al., 2020; Wang et al., 2021a,b). These methods estimate relation graphs which encode the similarity between sample embeddings. In contrast, our RGL models similarity between samples by class prediction vectors without introducing extra parameters.
Following the problem definition of (Schick and Schütze, 2021b; Gao et al., 2021), the target of this
paper is to generalize a pre-trained language model (PLM) to text classification tasks with a few labeled examples. Each task  $\mathcal{T}$  with label space  $\mathcal{V}$  consists of three datasets: (i) training dataset  $\mathcal{D}_{\mathrm{train}} = \{(x_i, y_i)\}$  containing a few labeled examples where  $x_i$  is the sequence and  $y_i$  is the corresponding label, (ii) development(validation) dataset  $\mathcal{D}_{\mathrm{dev}}$  containing the same number of samples as  $\mathcal{D}_{\mathrm{train}}$  and is used for model selection, and (iii) testing dataset  $\mathcal{D}_{\mathrm{test}}$  containing unlabeled samples to be predicted.
In prompt-based tuning, each input sample  $(\mathbf{x}_i,y_i)$  is reformulated as a pattern-verbalizer pair (PVP) (Schick and Schütze, 2021a) in terms of  $(p(\mathbf{x}_i),v(y_i))$ . The pattern mapping function  $p(\cdot)$  maps  $\mathbf{x}_i$  to a cloze question with masks. For example, a single sentence
$$
" \mathbf {x} _ {i} = [ \mathrm {C L S} ] s [ \mathrm {S E P} ] ^ {\prime \prime}
$$
can be mapped as
$$
p (\mathbf {x} _ {i}) = [ \mathrm {C L S} ] s \text {I t w a s} [ \mathrm {M A S K} ]. [ \mathrm {S E P} ] ^ {\prime \prime},
$$
where [CLS] and [SEP] are special start and end tokens. And a sentence pair
$$
\mathbf {x} _ {i} = [ \mathrm {C L S} ] s _ {1} [ \mathrm {S E P} ] s _ {2} [ \mathrm {S E P} ] ^ {\prime}
$$
$$
\mathbf {\nabla} ^ {p} (\mathbf {x} _ {i}) = [ \mathrm {C L S} ] s _ {1} [ \mathrm {M A S K} ], s _ {2} [ \mathrm {S E P} ] ^ {.}
$$
The verbalizer  $v(\cdot)$  maps  $y_{i}$  to tokens expressing the semantic meaning of  $y_{i}$ . For examples, "positive/negative" can be mapped as "good/bad". With PVPs, the token embedding  $\mathbf{h}_i^{[\mathrm{MASK}]}$  of [MASK] is taken as the representation of  $\mathbf{x}_i$ . The class prediction  $\hat{\mathbf{y}}_i$  contains the conditional probability distribution of each possible class label given  $\mathbf{x}_i$ , whose entry corresponds to  $y_{i}$  is estimated as
$$
\begin{array}{l} q \left(y _ {i} \mid \mathbf {x} _ {i}\right) = \frac {\exp \left(p \left(\left[ \mathrm {M A S K} \right] = v \left(y _ {i}\right) \mid p \left(\mathbf {x} _ {i}\right)\right)\right)}{\sum_ {y _ {j} \in \mathcal {Y}} \exp \left(p \left(\left[ \mathrm {M A S K} \right] = v \left(y _ {j}\right) \mid p \left(\mathbf {x} _ {i}\right)\right)\right)} \\ = \frac {\exp \left(\mathbf {w} _ {v \left(y _ {i}\right)} ^ {\top} \cdot \mathbf {h} _ {i} ^ {[ \text {M A S K} ]}\right)}{\sum_ {y _ {j} \in \mathcal {Y}} \exp \left(\mathbf {w} _ {v \left(y _ {j}\right)} ^ {\top} \cdot \mathbf {h} _ {i} ^ {[ \text {M A S K} ]}\right)}, \tag {1} \\ \end{array}
$$
where  $\mathbf{w}_v$  is the logit vector of token  $v$  existing in the vocabulary. Let  $\mathbf{y}_i$  be a one-hot vector with all 0s but a single one denoting the index of the ground truth class label  $y_i \in \{1, \dots, C\}$ . The model is optimized w.r.t. the loss  $\mathcal{L}_{\mathrm{CE}}$  defined as
$$
\mathcal {L} _ {\mathrm {C E}} = \sum_ {i = 1} ^ {N} - \log (\hat {\mathbf {y}} _ {i}) ^ {\top} \mathbf {y} _ {i}, \tag {2}
$$
where  $(\cdot)^{\top}$  denotes the transpose operation.
4 RGL: Our Proposed Method
In this section, we present the proposed RGL (Figure 1). We manage to exploit more supervised signals out of the training samples by constructing and learning on batch-wise relation graphs, which can boost the effectiveness of prompt-based tuning.
Figure 1: A high-level illustration of prompt-based tuning with the proposed RGL (marked by the square with blue dotted lines).
4.1 Defining the Relation Graphs
Consider a mini-batch  $\mathcal{B} = \{(\mathbf{x}_i, y_i)\}_{i=1}^N$  containing  $N$  randomly sampled sequence-label pairs, whose indexes are kept in  $\mathcal{I} = \{1, \ldots, N\}$ . We try to exploit more supervised information by modeling its relation graph. Let  $\mathcal{G} = \{\mathcal{V}, \mathcal{E}\}$  denotes the relation graph among the  $N$  training samples in  $\mathcal{B}$ . In particular,  $\mathcal{V}$  is a set of nodes where each node  $v_i \in \mathcal{V}$  corresponds to one training sample  $\mathbf{x}_i$ , and  $\mathcal{E} = \{e_{ij}\}$  is a set of edges between the  $N$  training samples. In this paper, we mainly consider text classification tasks. Hence, we establish the edge  $e_{ij}$  between a node  $v_i$  and another node  $v_j$  if these nodes come from the same class. Formally,  $e_{ij}$  is set to
$$
e _ {i j} = \left\{ \begin{array}{l l} 1 & \text {i f} y _ {j} = y _ {i} \\ 0 & \text {o t h e r w i s e} \end{array} . \right. \tag {3}
$$
Note that (3) is just an example of defining  $e_{ij}$  in classification tasks, which is simple but already enough to obtain good performance. One can also define  $e_{ij}$  in other ways flexibly, such as modeling both the intra-class and inter-class relations (Kim et al., 2019), using auxiliary information to calculate it, and defining real-valued  $e_{ij}$  for regression tasks.
4.2 Learning with Relation Graphs
On the relation graph  $\mathcal{G}$  of mini-batch  $\mathcal{B}$ , we expand the origin classification task into two problems: (i) a node classification problem to predict the correct class of each node, and (ii) a link prediction problem to connect nodes of the same classes and disconnect nodes from different classes.
The node classification problem corresponds exactly to the original classification task. Therefore, we obtain class prediction  $\hat{\mathbf{y}}_i$  of  $v_{i}$  (corresponding to  $\mathbf{x}_i$ ) by (1) and calculate  $\mathcal{L}_{\mathrm{CE}}$  loss by (2).
As for the link prediction problem, we establish  $\hat{e}_{ij}$  between  $v_{i}$  and  $v_{j}$  based on the relevance between  $\hat{\mathbf{y}}_i$  and  $\hat{\mathbf{y}}_j$ :
$$
\hat {e} _ {i j} = g \left(\hat {\mathbf {y}} _ {i}, \hat {\mathbf {y}} _ {j}\right), \tag {4}
$$
where  $\hat{\mathbf{y}}_i, \hat{\mathbf{y}}_j$  are obtained by (1), and  $g(\cdot, \cdot)$  is simply set as cosine similarity in this paper. There exist other choices to obtain  $\hat{e}_{ij}$ . One can define  $e_{ij}$  and  $\hat{e}_{ij}$  differently: leveraging auxiliary relation graphs or calculating based on representation similarity such as  $g(\mathbf{h}_i^{[\mathrm{CLS}]}, \mathbf{h}_j^{[\mathrm{CLS}]})$  and  $g(\mathbf{h}_i^{[\mathrm{MASK}]}, \mathbf{h}_j^{[\mathrm{MASK}]})$ . We use  $\hat{\mathbf{y}}_i, \hat{\mathbf{y}}_j$  as they carry more semantic information relevant to each class, which are more predictive and obtain better empirical performance. One may also consider using parameterized  $g(\cdot, \cdot)$  instead of using cosine similarity. However, considering the limited number of labeled samples, we avoid bringing in extra parameters to reduce the risk of overfitting. To measure the losses of link prediction, We design  $\mathcal{L}_{\mathrm{Link}}$  loss as
$$
- \sum_ {i \in \mathcal {I}} \sum_ {j \in \mathcal {A} (i)} e _ {i j} \log \left(\hat {e} _ {i j}\right) + (1 - e _ {i j}) \log \left(1 - \hat {e} _ {i j}\right), \tag {5}
$$
$$
\mathcal {A} (i) = \{j \in \mathcal {I} \text {a n d} i \neq j \}. \tag {6}
$$
For each mini-batch  $\mathcal{B}$ , we optimize the model to minimize the combination of node classification loss  $\mathcal{L}_{\mathrm{CE}}$  and link prediction loss  $\mathcal{L}_{\mathrm{Link}}$  as a whole:
$$
\mathcal {L} _ {\mathrm {C E}} + \alpha \mathcal {L} _ {\text {L i n k}}, \tag {7}
$$
where  $\alpha$  is a hyperparameter to control the contribution of this  $\mathcal{L}_{\mathrm{Link}}$ .
4.3 Comparisons with SCL
The most relevant work to RGL is SCL(Gunel et al., 2020) which applies supervised contrastive
learning (SCL) on a batch level while fine-tuning PLM (rather than prompt-based tuning PLM). SCL optimizes for the following objective:
$$
\mathcal {L} _ {\mathrm {C E}} + \beta \mathcal {L} _ {\mathrm {S C L}}, \tag {8}
$$
where  $\mathcal{L}_{\mathrm{SCL}}$  is defined as:
$$
- \sum_ {i \in \mathcal {I}} \sum_ {j \in \mathcal {P} (i)} \log \frac {\exp (f (\mathbf {x} _ {i}) \cdot f (\mathbf {x} _ {j}) / \tau)}{\sum_ {k \in \mathcal {A} (i)} \exp (f (\mathbf {x} _ {i}) \cdot f (\mathbf {x} _ {k}) / \tau)}, (9)
$$
where  $\tau$  is a hyperparameter.  $\mathcal{P}(i)$  takes the form of
$$
\mathcal {P} (i) = \{j \in \mathcal {A} (i): y _ {j} = y _ {i} \}, \tag {10}
$$
where  $\mathcal{A}(i)$  is defined in (6).  $f(\mathbf{x}_i)$  in (9) refers to the representation of  $\mathbf{x}_i$ . In the original paper (Gunel et al., 2020),  $f(\mathbf{x}_i)$  is set as token embedding  $\mathbf{h}_i^{[\mathrm{CLS}]}$  of [CLS]. While considering prompt-based tuning strategies (Liu et al., 2021), we follow routine and set  $f(\mathbf{x}_i) = \mathbf{h}_i^{[\mathrm{MASK}]}$  in SCL.
Comparing (5) to (9), it can be observed that RGL enforces more strict constraints between samples. By constructing relation graphs and learning to approximate the edge labels  $\hat{e}_{ij}$  defined in (3), RGL rules samples from the same class to be connected and otherwise disconnected. While SCL does not use any precise measures (e.g., edge labels) to constrain similarities/distances between intra/inter-class samples. SCL only encourages samples from the same class to be close, without explicitly pushing those from different classes to be farther apart. Another difference is that RGL estimates edge labels  $\hat{e}_{ij}$  using the prediction  $\hat{\mathbf{y}}_i$  and  $\hat{\mathbf{y}}_i$  to regularize the task-dependent representations, while SCL uses representation of  $\mathbf{x}_i$  (outputs of PLM) which can be irrelevant to the target task.
All the experiments are conducted on a 32GB NVIDIA Tesla V100 GPU.
Experimental Settings. We use RoBERTa-large $^2$  (Liu et al., 2019) as the PLM. We take  $\mathrm{PET}^3$  (Schick and Schütze, 2021b) as the basic prompt-based tuning method. Upon PET, we compare the benefits of applying the proposed RGL versus SCL (Gunel et al., 2020). All the hyperparameters are selected using the provided development set via grid search following Gao et al. (2021). We use Adam optimizer. We first select
learning rate from  $\{1e - 5,2e - 5,5e - 5\}$  and batch size from  $\{2,4,8\}$  for PET. Then, we select hyperparameter  $\alpha$  in RGL and hyperparameters  $\beta$  and  $\tau$  in SCL from  $[0:0.2:1]$  separately. We train all methods for a maximum number of 1000 steps and evaluate the performance on development set every 100 steps.
Dataset. Experiments are performed on a variant of GLUE benchmarks (Wang et al., 2018) for few-shot setting, which is provided by Gao et al. (2021). Gao et al. (2021) provide 5 different training sets and developing sets where each of them consist of 16 labeled samples per class. The averaged performance over these 5 splits are reported. We also evaluate the proposed RGL on the SuperGLUE (Wang et al., 2019) variant proposed by Schick and Schütze (2021b), whose results are put in Appendix due to space limit.
Results. Table 1 shows the results. As shown, both RGL and SCL can bring in additional performance gain. In particular, RGL can improve the performance of PET by  $2.38\%$  on average, while SCL only improves PET by  $1.46\%$  on average. Moreover, RGL obtains more stable results as the variances are smaller than the others.
Figure 2: Effect of labeled samples.
Model Analysis. Figure 2 plots the effect of varying the number of labeled training samples. As shown, all methods obtain better performance given more training samples, while PET+RGL consistently outperforms the others. We further consider different ways of obtaining  $\hat{e}_{ij}$  in (4): (i)  $\mathbf{w} / \mathbf{h}^{[\mathrm{CLS}]}$  which sets  $\hat{e}_{ij} = \cos (\mathbf{h}_i^{[\mathrm{CLS}]},\mathbf{h}_i^{[\mathrm{CLS}]})$  where  $\cos (\cdot ,\cdot)$  denotes cosine similarity; (ii)  $\mathbf{w} / \mathbf{h}^{[\mathrm{MASK}]}$  which sets  $\hat{e}_{ij} = \cos (\mathbf{h}_i^{[\mathrm{MASK}]},\mathbf{h}_i^{[\mathrm{MASK}]})$ ; and (iii)  $\mathbf{w} / \hat{\mathbf{y}}$  which is the one adopted in RGL and sets  $\hat{e}_{ij} = \cos (\hat{\mathbf{y}}_i,\hat{\mathbf{y}}_j)$ . Results in Figure 3 show that RGL outperforms the others. This
$^{2}$ https://huggingface.co/roberta-large.
<sup>3</sup>https://github.com/timoschick/pet.
SST-2(acc)	SST-5(acc)	MR(acc)	CR(acc)	MPQA(acc)	Subj(acc)	TREC(acc)	CoLA(Matt.)
PET	92.7(0.9)	47.4(2.5)	87.0(1.2)	90.3(1.0)	84.7(2.2)	91.2(1.1)	84.8(5.1)	9.3(7.3)
PET+SCL relative ↑	92.9(1.9)+0.2	48.0(1.9)+0.6	87.1(1.8)+0.1	90.3(1.5)+0.0	84.9(2.4)+0.2	91.2(1.7)+0.0	85.5(2.6)+0.7	20.9(16.5)+11.6
PET+RGL relative ↑	93.4(0.5)+0.7	49.3(1.2)+1.9	87.3(0.8)+0.3	90.3(0.9)+0.0	85.6(1.5)+0.9	91.4(1.5)+0.2	86.8(2.9)+2.0	22.7(14.1)+13.4
MNLI(acc)	MNLI-mm(acc)	SNLI(acc)	QNLI(acc)	RTE(acc)	MRPC(F1)	QQP(F1)	STS-B(Pear.)
PET	68.3(2.3)	70.5(1.9)	77.2(3.7)	64.5(4.2)	69.1(3.6)	74.5(5.3)	65.5(5.3)	71.0(7.0)
PET+SCL relative ↑	69.5(3.2)+1.2	71.3(3.1)+0.8	77.2(2.9)+0.0	69.2(2.7)+4.7	69.3(4.1)+0.2	75.8(4.0)+1.3	66.7(3.7)+1.2	71.6(6.5)+0.6
PET+RGL relative ↑	70.8(2.3)+2.5	72.7(1.9)+2.2	77.5(1.7)+0.3	70.3(1.7)+5.8	69.7(2.6)+0.6	77.0(6.7)+2.5	68.8(1.8)+3.3	72.5(6.2)+1.5
Table 1: Test performance obtained on GLUE variant (Gao et al., 2021).
Figure 3: Estimating  $\hat{e}_{ij}$  in different ways.
Figure 4: t-SNE visualization on SST-2 task.
validates that class prediction carries more relevant information to discriminate samples.
Visualization. Figure 4 plots the t-SNE visualization of the learned sample embeddings. It apparently shows that, when combining RGL with both fine-tuning and PET, the distances of deep representations between any two inter-class samples are much longer than the intra-class distances. Furthermore, PET+RGL can separate two classes of samples with clear margin while concentrating samples of every class closely to the center of the group, resulting in better discriminate ability.
We present RGL, a simple yet effective relation graph augmented prompt-based tuning approach for few-shot natural language understanding
tasks. During learning, RGL constructs batchwise relation graphs based on label consistency between samples, and explicitly tunes the pretrained language models to solve the resultant node classification and link prediction problems. In this way, RGL fully exploits the limited supervised information. In this paper, we provide one way of relation graph learning. This can be further extended to broader applications, where other ways of relation graph learning worth trying. In addition, one can explore how to avoid the interference of noisy samples.
This work is supported by the National Key Research and Development Program of China (No. 2021ZD0110303).
Yujia Bao, Menghua Wu, Shiyu Chang, and Regina Barzilay. 2020. Few-shot text classification with distributional signatures. In International Conference on Learning Representations.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877-1901.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Jesse Dodge, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannaneh Hajishirzi, and Noah Smith. 2020. Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping. arXiv preprint arXiv:2002.06305.
Thomas Dopierre, Christophe Gravier, and Wilfried Logerais. 2021. *ProtAugment: Intent detection meta-learning through unsupervised diverse paraphrasing*. In *Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics* and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 2454–2466, Online. Association for Computational Linguistics.
Tianyu Gao, Adam Fisch, and Danqi Chen. 2021. Making pre-trained language models better few-shot learners. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3816-3830, Online. Association for Computational Linguistics.
Ruiying Geng, Binhua Li, Yongbin Li, Jian Sun, and Xiaodan Zhu. 2020. Dynamic memory induction networks for few-shot text classification. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1087-1094, Online. Association for Computational Linguistics.
Beliz Gunel, Jingfei Du, Alexis Conneau, and Veselin Stoyanov. 2020. Supervised contrastive learning for