RGL: A Simple yet Effective Relation Graph Augmented Prompt-based Tuning Approach for Few-Shot Learning
Abstract
Pre-trained language models (PLMs) can provide a good starting point for downstream applications.
However, it is difﬁcult to generalize PLMs to new tasks given a few labeled samples.
In this work, we show that Relation
Graph augmented Learning (RGL) can improve the performance of few-shot natural language understanding tasks.
During learning, RGL constructs a relation graph based on the label consistency between samples in the same batch, and learns to solve the resultant node classiﬁcation and link prediction problems on the relation graph.
In this way, RGL fully exploits the limited supervised information, which can boost the tuning effectiveness.
Extensive experimental results show that RGL consistently improves the performance of prompt-based tuning strategies.
Introduction
Pre-trained language models (PLMs), such as
BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019), have become the standard workhorse for nowadays natural language processing tasks.
A direct way of leveraging these PLMs is to ﬁnetune them by taking gradient descent w.r.t.
the objective of downstream tasks.
However, tuning the large PLM by a few labeled samples has a high risk of overﬁtting (Dodge et al., 2020;
Zhang et al., 2021; Gunel et al., 2020).
Besides, as PLMs are trained by an objective different from the downstream tasks, the ability of PLM may not be fully exploited.
Recently, promptbased tuning methods emerge and obtain promising results on tuning PLMs to new tasks with a few labeled samples (Liu et al., 2021).
In particular, these methods use prompts to reformulate the downstream tasks into the same form of pretraining tasks such that the gap between pretraining and ﬁne-tuning is reduced (Brown et al., 2020; Schick and Schütze, 2021a).
Concretely, prompt-based tuning strategies rewrite the input sequence into a cloze question with masks (Schick and Schütze, 2021a).
The input sequence is rewritten as prompts, while the corresponding label is replaced by answer tokens.
Some methods use hard prompts and answers which use text strings with certain semantic meaning (Schick and Schütze, 2021b;
Tam et al., 2021; Gao et al., 2021), while others take learnable parameters as soft prompts and answers (Liu et al., 2021; Li and Liang, 2021; Lester et al., 2021).
One can use multiple prompts to boost the performance of prompt-based tuning (Brown et al., 2020; Schick and Schütze, 2021b).
While the above strategies improve few-shot performance, they pay less attention to exploiting the relations among the limited number of labeled samples.
In this work, we propose a simple yet effective relation graph augmented approach which can enhance the performance of prompt-based tuning strategies PLM in few-shot natural language understanding tasks.
Speciﬁcally, our proposal aims at fully exploiting the limited supervised information via R elation G raph augmented L earning, we thus call the proposed method RGL.
RGL ﬁrst constructs a batch-wise relation graph, where every node refers to a labeled sample and the edge between nodes refers to the similarity between the two samples.
RGL establishes the edge in the relation graph w.r.t whether the two samples are from the same class and regularizes the similarity of representations learned by PLMs between every two samples to ﬁt the edge of the relation graph.
RGL can easily scale up as the relation graph is constructed w.r.t.
only a mini-batch of sampled data points per iteration.
Empirical results on benchmark datasets show that RGL can consistently improve the performance of promptbased tuning.  Related Works The related works are brieﬂy reviewed below.
Few-Shot Learning.
Few-shot learning (FSL) targets at generalizing to new tasks with a few labeled samples (Wang et al., 2020).
FSL has been applied to many natural language processing applications such as text classiﬁcation (Bao et al., 2020) and named entity typing (Yang and Katiyar, 2020).
Typical solutions in FSL include data augmentation which directly generate more labeled samples (Dopierre et al., 2021), metric learning which learns to embed samples into a space where samples can be easily discriminated (Geng et al., 2020), and meta-learning which learns a good initialized model from a set of related tasks which is then ﬁne-tune to each task (Bao et al., 2020). Recently, several methods propose to tune pre-trained language models to downstream few-shot tasks (Liu et al., 2021). We follow this line, and further propose to conduct model tuning on learned relation graphs. Our approach can be incorporated into existing prompt-based tuning strategies, increasing the effectiveness of supervised signals and bringing in performance gains.
Graph Structure Learning.
Graph structure learning methods target at jointly learning graph structure and node embeddings of input samples (Zhu et al., 2021).
Usually, these methods iterate over two steps: (i) estimate the adjacency matrix which encodes graph structure using node embeddings; and (ii) apply graph neural networks (GNNs) on this updated graph to obtain new node embeddings.
Recently, graph structure learning has been used to estimate relation graphs among samples to facilitate effective propagation of label information (Satorras and Estrach, 2018; Rodríguez et al., 2020; Wang et al., 2021a,b).
These methods estimate relation graphs which encode the similarity between sample embeddings.
In contrast, our RGL models similarity between samples by class prediction vectors without introducing extra parameters.
Background
Following the problem deﬁnition of (Schick and Schütze, 2021b; Gao et al., 2021), the target of this paper is to generalize a pre-trained language model (PLM) to text classiﬁcation tasks with a few labeled examples.
Each task T with label space Y consists of three datasets: (i) training dataset D train =
{ ( x i , y i ) } containing a few labeled examples where x i is the sequence and y i is the corresponding label, (ii) development(validation) dataset D dev containing the same number of samples as D train and is used for model selection, and (iii) testing dataset D test containing unlabeled samples to be predicted.
In prompt-based tuning, each input sample ( x i , y i ) is reformulated as a pattern-verbalizer pair (PVP) (Schick and Schütze, 2021a) in terms of ( p ( x i ) , v ( y i )) .
The pattern mapping function p ( · ) maps x i to a cloze question with masks.
For example, a single sentence “ x i =
[CLS] s [SEP] ” can be mapped as “ p ( x i ) =
[CLS] s It was [MASK] .
[SEP] ” , where [CLS] and [SEP] are special start and end tokens.
And a sentence pair “ x [Equation]
[-[CLS] s 
 [SEP] s 
-] [SEP] ” can be mapped as “[-p ( x i ) = [CLS] s 1 [MASK] , s 
-]
[Equation]
The verbalizer v ( · ) maps y i to tokens expressing the semantic meaning of y i .
For examples, “positive/negative” can be mapped as “good/bad”.
With PVPs, the token embedding h [MASK] i of [MASK] is taken as the representation of x i .
The class prediction ˆ y i contains the conditional probability distribution of each possible class label given x i , whose entry corresponds to y i is estimated as
[Equation]
where w v is the logit vector of token v existing in the vocabulary.
Let y i be a one-hot vector with all 0s but a single one denoting the index of the ground truth class label y i ∈ { 1 , . .
, C } .
The model is optimized w.r.t. the loss L CE deﬁned as
[Equation]
where ( · ) (cid:62) denotes the transpose operation.
RGL: Our Proposed Method
In this section, we present the proposed RGL (Figure 1).
We manage to exploit more supervised signals out of the training samples by constructing and learning on batch-wise relation graphs, which can boost the effectiveness of prompt-based tuning
[Table]
+41c8R/0D/wjtjBB+ITkhy5tx7zsy91408mSjHeS5YI6Nj4xPFyanpmdm5+dLCYiMJ05iLOg+9MG66LBGeDERdSeWJZhQL5rueOHb7uzp+fCHiRIbBkRpE4sRn3UCeS84UUZ3ScttnqseZl+0PO1lbiUuV7cugPxx2SmWn4phl/wTVHJSRr4Ow9IQ2zhCCI4UPgQCKsAeGhJ4WqnAQEXeCjLiYkDRxgSGmSJtSlqAMRmyfvl3atXI2oL32TIya0ykevTEpbayRJqS8mLA+zTbx1Dhr9jfvzHjquw3o7+ZePrEKPWL/0n1k/lena1E4x7apQVJNkWF0dTx3SU1X9M3tT1UpcoiI0/iM4jFhbpQffbaNJjG1694yE38xmZrVe57npnjVt6QBV7+P8ydobFSqtcrmYa28s52PuogVrGKd5rmFHezhAHXyvsI9HvBonVrX1o11+55qFXLNEr4s6+4Nxu6anA==</latexit>
4. Deﬁning the Relation Graphs
Consider a mini-batch B = { ( x i , y i ) }
Ni =1 containing N randomly sampled sequence-label pairs, whose indexes are kept in I = { 1 , . . .
, N } .
We try to exploit more supervised information by modeling its relation graph.
Let G =
{V , E} denotes the relation graph among the N training samples in B .
In particular, V is a set of nodes where each node v i ∈ V corresponds to one training sample x i , and E = { e ij } is a set of edges between the N training samples.
In this paper, we mainly consider text classiﬁcation tasks. Hence, we establish the edge e ij between a node v i and another node v j if these nodes come from the same class.
Formally, e ij is set to
[Equation]
Note that (3) is just an example of deﬁning e ij in classiﬁcation tasks, which is simple but already enough to obtain good performance.
One can also deﬁne e ij in other ways ﬂexibly, such as modeling both the intra-class and inter-class relations (Kim et al., 2019), using auxiliary information to calculate it, and deﬁning real-valued e ij for regression tasks.
4. Learning with Relation Graphs
On the relation graph G of mini-batch B , we expand the origin classiﬁcation task into two problems: (i) a node classiﬁcation problem to predict the correct class of each node, and (ii) a link prediction problem to connect nodes of the same classes and disconnect nodes from different classes.
The node classiﬁcation problem corresponds exactly to the original classiﬁcation task.
Therefore, we obtain class prediction ˆ
y i of v i (corresponding to x i ) by (1) and calculate L CE loss by (2).
As for the link prediction problem, we establish ˆ e ij between v i and v j based on the relevance between ˆ y i and ˆ y j :
[Equation]
where ˆ y i , ˆ y j are obtained by (1), and g ( · , · ) is simply set as cosine similarity in this paper.
There exist other choices to obtain ˆ e ij . One can deﬁne e ij and ˆ e ij differently: leveraging auxiliary relation graphs or calculating based on representation similarity such as g ( h
[CLS] i , h [CLS] j ) and g ( h [MASK] i , h [MASK] j ) .
We use ˆ y i , ˆ y j as they carry more semantic information relevant to each class, which are more predictive and obtain better empirical performance.
One may also consider using parameterized g ( · , · ) instead of using cosine similarity.
However, considering the limited number of labeled samples, we avoid bringing in extra parameters to reduce the risk of overﬁtting.
To measure the losses of link prediction, We design L Link loss as
[Equation]
where A [Equation]
For each mini-batch B , we optimize the model to minimize the combination of node classiﬁcation loss L CE and link prediction loss L Link as a whole:
[Equation]
where α is a hyperparameter to control the contribution of this L Link .
4. Comparisons with SCL
The most relevant work to RGL is SCL(Gunel et al., 2020) which applies supervised contrastive learning (SCL) on a batch level while ﬁne-tuning PLM (rather than prompt-based tuning PLM).
SCL optimizes for the following objective:
[Equation]
where L SCL is deﬁned as:
[Equation]
where τ is a hyperparameter. P ( i ) takes the form of
[Equation] where A ( i ) is deﬁned in (6). f ( x i ) in (9) refers to the representation of x i . In the original paper (Gunel et al., 2020), f ( x i ) is set as token embedding h [CLS] i of [CLS].
While considering prompt-based tuning strategies (Liu et al., 2021), we follow routine and set f ( x i ) =
h [MASK] i in SCL. Comparing (5) to (9), it can be observed that RGL enforces more strict constraints between samples. By constructing relation graphs and learning to approximate the edge labels ˆ e ij deﬁned in (3), RGL rules samples from the same class to be connected and otherwise disconnected.
While SCL does not use any precise measures (e.g., edge labels) to constrain similarities/distances between intra/inter-class samples. SCL only encourages samples from the same class to be close, without explicitly pushing those from different classes to be farther apart. Another difference is that RGL estimates edge labels ˆ e ij using the prediction ˆ y i
and ˆ y i to regularize the task-dependent representations, while SCL uses representation of x i (outputs of PLM) which can be irrelevant to the target task.
Experiments
All the experiments are conducted on a 32GB
NVIDIA Tesla V100 GPU.
Experimental Settings.
We use RoBERTalarge 2 (Liu et al., 2019) as the PLM.
We take
PET 3 (Schick and Schütze, 2021b) as the basic prompt-based tuning method.
Upon PET, we compare the beneﬁts of applying the proposed
RGL versus SCL (Gunel et al., 2020).
All the hyperparameters are selected using the provided development set via grid search following Gao et al.
We use Adam optimizer. We ﬁrst select learning rate from { 1 e − 5 , 2 e − 5 , 5 e − 5 } and batch size from { 2 , 4 , 8 } for PET.
Then, we select hyperparameter α in RGL and hyperparameters β and τ in SCL from [0 : 0 . 2 : 1] separately.
We train all methods for a maximum number of 1000 steps and evaluate the performance on development set every 100 steps.
Dataset.
Experiments are performed on a variant of GLUE benchmarks (Wang et al., 2018) for fewshot setting, which is provided by Gao et al.
Gao et al.
(2021) provide 5 different training sets and developing sets where each of them consist of 16 labeled samples per class.
The averaged performance over these 5 splits are reported.
We also evaluate the proposed RGL on the SuperGLUE (Wang et al., 2019) variant proposed by Schick and Schütze (2021b), whose results are put in Appendix due to space limit.
Results.
Table 1 shows the results.
As shown, both RGL and SCL can bring in additional performance gain.
In particular, RGL can improve the performance of PET by 2.38% on average, while SCL only improves PET by 1.46% on average.
Moreover, RGL obtains more stable results as the variances are smaller than the others.
[Figure] [Figure]
Figure 2 plots the effect of varying the number of labeled training samples.
As shown, all methods obtain better performance given more training samples, while PET+RGL consistently outperforms the others.
We further consider different ways of obtaining ˆ e ij in (4): (i) w/ h
[CLS] which sets ˆ e ij = cos ( h [CLS] i , h
[CLS] i ) where cos ( · , · ) denotes cosine similarity; (ii) w/ h [MASK] which sets ˆ e ij = cos ( h [MASK] i , h [MASK] i ) ; and (iii) w/ ˆ y which is the one adopted in RGL and sets ˆ e ij = cos (ˆ y i , ˆ y j ) .
Results in Figureshow that RGL outperforms the others. This [Table]
[Figure] [Figure] validates that class prediction carries more relevant information to discriminate samples.
Visualization.
Figure 4 plots the t-SNE visualization of the learned sample embeddings.
It apparently shows that, when combining RGL with both ﬁne-tuning and PET, the distances of deep representations between any two inter-class samples are much longer than the intra-class distances.
Furthermore, PET+RGL can separate two classes of samples with clear margin while concentrating samples of every class closely to the center of the group, resulting in better discriminate ability.
Conclusion
We present RGL, a simple yet effective relation graph augmented prompt-based tuning approach for few-shot natural language understanding tasks.
During learning, RGL constructs batchwise relation graphs based on label consistency between samples, and explicitly tunes the pretrained language models to solve the resultant node classiﬁcation and link prediction problems. In this way, RGL fully exploits the limited supervised information.
In this paper, we provide one way of relation graph learning.
This can be further extended to broader applications, where other ways of relation graph learning worth trying.
In addition, one can explore how to avoid the interference of noisy samples. Acknowledgements
This work is supported by the National Key Research and Development Program of China (No. 2021ZD0110303). Association for
Computational Linguistics. Timo Schick and Hinrich Schütze. 2021b.
It’s not just size that matters: Small language models are also few-shot learners. In Proceedings of the
Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages 2339–2352, Online. Association for Computational Linguistics.
Derek Tam, Rakesh R Menon, Mohit Bansal, Shashank
Srivastava, and Colin Raffel. 2021.
Improving and simplifying pattern exploiting training.
arXiv preprint arXiv:2103.11955 .
Alex Wang, Yada Pruksachatkun, Nikita Nangia,
Amanpreet Singh, Julian Michael, Felix Hill, Omer
Levy, and Samuel R Bowman. 2019.
SuperGLUE: A stickier benchmark for general-purpose language understanding systems.
arXiv preprint arXiv:1905.00537 .
Alex Wang, Amanpreet Singh, Julian Michael, Felix
Hill, Omer Levy, and Samuel Bowman.
GLUE:
A multi-task benchmark and analysis platform for natural language understanding.
In Proceedings of the 2018 EMNLP Workshop BlackboxNLP:
Analyzing and Interpreting Neural Networks for NLP , pages 353–355, Brussels, Belgium.
Association for Computational Linguistics.
Yaqing Wang, Abulikemu Abuduweili, Quanming Yao, and Dejing Dou. 2021a.
Property-aware relation networks for few-shot molecular property prediction.
In Advances in Neural Information Processing Systems , pages 17441–17454.
Yaqing Wang, Song Wang, Quanming Yao, and Dejing
Dou. 2021b.
Hierarchical heterogeneous graph representation learning for short text classiﬁcation.
In Proceedings of the 2021 Conference on Empirical
Methods in Natural Language Processing , pages 3091–3101, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Yaqing Wang, Quanming Yao, James T Kwok, and
Lionel M Ni.
Generalizing from a few examples: A survey on few-shot learning.
ACM
Computing Surveys , 53(3):1–34.
Yi Yang and Arzoo Katiyar.
Simple and effective few-shot named entity recognition with structured nearest neighbor learning.
In Proceedings of the 2020 Conference on Empirical Methods in Natural Association Computational Linguistics Wu
Arzoo Katiyar
Weinberger
Yoav Artzi A Results on SuperGLUE Variant In addition to PET (Schick and Schütze, 2021b), we also combine the proposed RGL with other promptbased tuning strategies. We take P-tuning 4 (Liu et al., 2021) as the representative for joint prompt and PLMtuning, and WARP 5 (Hambardzumyan et al., 2021) as the representative for prompt tuning with a ﬁxed PLM.
WARP is only evaluated on CB and RTE as in the original paper.
In the original papers, both P-tuning and WARP use SuperGLUE (Wang et al., 2019) variant proposed by Schick and Schütze (2021b) to evaluate the few-shot performance and use ALBERT 6 (Lan et al., 2020) as the PLM.
We adopt their setting for fairness.
Schick and Schütze (2021b) provide one training set which consists 32 samples per task and a testing set.
Schick and Schütze (2021b) also use unlabeled samples, which are not used in this paper.
Following Liu et al.
(2021), a development set consisting of 32 samples per task are randomly drawn for model selection.
As only one split is provided, we initialize the parameter with ﬁve random seeds and report the averaged results over ﬁve runs.
[Table]
Table 2 shows the results obtained on SuperGLUE variants.
The results show that RGL can consistently boost the performance when it is combined with P-tuning and WARP.