D IFFERENTIABLE P ROMPT M AKES P RE - TRAINED L ANGUAGE M ODELS B ETTER F EW - SHOT L EARNERS
A BSTRACT
Large-scale pre-trained language models have contributed signiﬁcantly to natural language processing by demonstrating remarkable abilities as few-shot learners.
However, their effectiveness depends mainly on scaling the model parameters and prompt design, hindering their implementation in most real-world applications.
This study proposes a novel pluggable, extensible and efﬁcient approach named
DifferentiAble pRompT (DART) which can convert small language models into better few-shot learners.
The main principle behind this approach involves reformulating potential natural language processing tasks into the task of a pre-trained language model and differentially optimizing the prompt template as well as the target label with backpropagation.
Furthermore, the proposed approach can be: (i)
Plugged to any pre-trained language models; (ii) Extended to widespread classiﬁcation tasks.
A comprehensive evaluation of standard NLP tasks demonstrates that the proposed approach achieves a better few-shot performance
I NTRODUCTION
The pre-train—ﬁne-tune paradigm has become the de facto standard for natural language processing (NLP), and has achieved excellent results in several benchmarks (Devlin et al., 2019; Liu et al., 2019;
Lewis et al., 2020; Dong et al., 2019; Bao et al., 2020a).
The success of these pioneers seems to suggest that large-scale pre-trained models are always nothing short of a panacea for boosting machine intelligence.
However, supervised ﬁne-tuning is still prone to labeled data in practice and faces unignorable challenges owing to the variations of domains, language, and tasks.
These drawbacks lead to the research of an important technique, few-shot learning , which can signiﬁcantly improve the learning capabilities of machine intelligence and practical adaptive applications by accessing only a small number of labeled examples.
The GPT-3 model, introduced by Brown et al.
(2020), exhibits impressive few-shot learning capabilities.
Given a natural language prompt and 16 labeled samples as demonstrations in the contextual input, GPT-3 achieves 80% of the SOTA results.
However, GPT-3 is a fully dense transformer model with 175B parameters, which makes it challenging to deploy in most real-world applications.
Recently, an emerging ﬁne-tuning methodology has arisen to equip smaller language models (LMs) with few-shot capabilities: adapting the pre-trained LM directly as a predictor through completion of a cloze task (Schick & Sch ¨ utze (2021; 2020); Gao et al.
; Liu et al.
(2021c)), which treats the downstream task as a (masked) language modeling problem.
These prompts can be used in ﬁnetuning to provide the classiﬁer with additional task information, especially in the low-data regime.
[CLS]
The drama [MASK] nothing.
[SEP] [CLS]
The drama [MASK] nothing.
[SEP] [MASK] [CLS]
MLM Head
CLS Head ... discloses is...
Vocabulary 𝒱 label: positive label: negative
Label 𝒴
MLM Pre-training
Conventional Fine-tuning
Differentiable Prompt Fine-tuning (Our Approach DART)
The drama discloses nothing.
[ 𝑇 ! ] [ 𝑇 " ]
[ 𝑇 # ] [MASK] e(The) e(drama) e(discloses) e(nothing)
e([SEP]) h
([ 𝑇 ! ]) h
([ 𝑇 " ])h
([ 𝑇 # ]) e ([MASK] ).
e(The)
e( [MASK] ) e(discloses) e(nothing)
e([SEP]) h
([ 𝑇 ! ]) h
([ 𝑇 " ])h
([ 𝑇 # ]) e ([Y] ).
[Figure]
Pre
[Figure]
trained Language Model
[Figure]
Token Embeddings
[Figure]
Fluency ConstraintObject
[Figure]
We leverage a few parameters within the language model as the template and label tokens and optimize them via backpropagation without introducing additional parameters apart from the model.
Notably, Scao & Rush (2021) observe that prompting can often compensate for hundreds of data points on average across multiple classiﬁcation tasks.
However, determining the appropriate prompts requires domain expertise, and handcrafting a high-performing prompt often requires impractically large validation sets (Perez et al.
Recent studies (Lu et al.
; Zhao et al.
(2021)) have reported that the manual prompt format can be sub-optimal, which would result in the accuracy varying from random guess performance to near the state-of-the-art.
Therefore, previous approaches have attempted to search for discrete prompt tokens automatically.
However, it is non-trivial for widespread classiﬁcation tasks to obtain an optimized prompt template and target label token.
For example, speciﬁc classiﬁcation tasks such as relation extraction with the label of alternate name and country of birth cannot specify a single label token in the vocabulary.
In this paper, we propose a novel D ifferenti A ble p R omp T ( DART ) ﬁne-tuning approach, which is model-agnostic, parameter-efﬁcient.
As illustrated in Figure 1, the key idea is to leverage a few parameters (unused tokens) in the language model, which serve as the template and label tokens, and to optimize them in the continuous space using backpropagation.
Subsequently, we introduce differentiable prompt learning to obtain optimized prompt templates as well as labels.
Since ﬁne-tuning with limited samples can be affected by instability (Dodge et al.
; Zhang et al.
(2021)), we propose an optimization algorithm to jointly learning templates as well as labels.
We further introduce an auxiliary ﬂuency constraint object to ensure the association among the prompt embeddings.
We conduct extensive experiments on 15 NLP datasets.
With only a few training samples across all the tasks, our approach (DART) can obtain a better performance.
Notably, absolute performance improvement of up to 23.28%, over the conventional ﬁne-tuning, is obtained on average in the setting of K = 8 (and 1.55% for fully supervised settings) on relation extraction datasets with complex label semantics.
Our approach can be applied to real-world classiﬁcation tasks without the high cost of collecting and annotating a large amount of data.
The main contributions of this study are as follows:
• We propose a new simple framework for few-shot learning, which is pluggable, extensible, and efﬁcient.
To the best of our knowledge, optimizing label tokens in continuous space is also a new branch of research that has not been explored in language model prompting.
• A systematic evaluation of 15 NLP tasks shows that the simple-yet-effective method contributes towards improvements across all these tasks.
Remarkably, given only 8 labeled samples per class, our proposed approach can achieve 90% performance of the SOTA results (full dataset).
R ELATED W ORK
Language Model Prompting.
The language model prompting has emerged with the introduction of GPT-3 (Brown et al.
(2020)), which demonstrates excellent few-shot performance (Liu et al.
(2021b)).
However, GPT-3 is not designed for ﬁne-tuning; it mainly relies on the handcraft prompt (in-context learning (Liu et al.
(2021a)
; Zhao et al.
; Ding et al.
; Min et al.
Thus, recent studies (Qin & Eisner (2021); Hambardzumyan et al.
; Chen et al.
(2021)) conducted in this ﬁeld have been focused on automatically searching the prompts.
Schick & Sch¨utze (2021; 2020) propose the PET, which reformulates the NLP tasks as cloze-style questions and performs gradient-based ﬁne-tuning.
Tam et al.
(2021) improve the PET with a denser supervision object during ﬁne-tuning.
Shin et al.
(2020) propose the AUTOPROMPT to create prompts for a diverse set of tasks based on a gradient-guided search.
Han et al.
(2021) propose an approach called PTR, which leverages logic rules to construct prompts with sub-prompts for many-class text classiﬁcation.
Wang et al.
(2021) reformulate potential NLP task into an entailment one, and then ﬁne-tune the model with few-shot samples.
Hu et al.
(2021) propose an approach to incorporate external knowledge graph into the verbalizer with calibration.
Additionally, Gao et al.
(2020) present LM-BFF—better few-shot ﬁne-tuning of language models, which leverages T5 (Raffel et al.
to generate templates and search label tokens in the vocabulary.
However, the utilization of the generative model and the label search with validation is computation-intensive.
Moreover, the prompt search over discrete space is sub-optimal due to the continuous nature of neural networks.
To overcome these limitations, Liu et al.
(2021c) propose P-tuning, which employs trainable continuous prompt embeddings learned by an LSTM.
Zhong et al.
(2021) propose an effective continuous method called OPTIPROMPT to optimize prompts for factual probing.
Liu et al.
(2021c)
propose preﬁx-tuning, which keeps language model parameters frozen but optimizes a small continuous taskspeciﬁc vector for natural language generation tasks.
Lester et al.
(2021) propose a mechanism for learning “soft prompts” to condition frozen language models to perform downstream tasks.
However, these approaches still have to optimize the external parameters (e.g., LSTM in P-tuning) and are prone to complex label space.
Conversely, this study aims to develop a novel few-shot learning framework based on pre-trained language models which can reduce the prompt engineering (including templates and labels) and external parameter optimization.
Furthermore, the proposed approach only leverages the noninvasive modiﬁcation of the model, which can be plugged into any pre-trained language model and extended to the widespread classiﬁcation task.
Few-shot Learning.
Few-shot learning can signiﬁcantly improve the learning capabilities for machine intelligence and practical adaptive applications by accessing only a small number of labeled examples (Zhang et al.
The proposed approach corresponds to the other few-shot NLP methods, including: (1) Meta-learning (Yu et al.
; Bao et al.
(2020b); Bansal et al.
Deng et al.
(2020b;a); Yu et al.
(2020)), in which the quantities of the auxiliary tasks are optimized.
(2) Intermediate training (Phang et al.
; Yin et al.
(2020)), which supplements the pre-trained
LMs with further training on the data-rich supervised tasks.
(3) Semi-supervised learning (Miyato et al.
; Xie et al.
(2020)), which leverages unlabeled samples.
The proposed approach focuses on a more realistic few-shot setting (the number of labeled instances per class can be any variable).
B ACKGROUND
Let X in = { x 1 , x 2 ,..., x L } be a sentence, where x i is the i th token in the input sentence and L is the number of tokens.
Speciﬁcally, X in is converted to a ﬁxed token sequence ˜ X in and then mapped to a sequence of hidden vectors { h k ∈ R d } .
Given the input sequence, ˜ X in =
[CLS] X in [SEP] , the conventional ﬁne-tuning approaches leverage a generic head layer over [CLS] embeddings (e.g., an
MLP layer) to predict an output class.
For the prompt-based method, a task-speciﬁc pattern string
(template T ) is designed to coax the model into producing a textual output corresponding to a given class (label token M ( Y ) )—we refer to these two things together as a prompt.
Speciﬁcally, X prompt containing one [MASK] token is directly tasked with the MLM input as:
[Equation]
When the prompt is fed into the MLM, the model can obtain the probability distribution p ( [MASK] | ( X prompt ) of the candidate class, y ∈ Y as:
[Equation]
where w represents the w th label token of class y .
O UR A PPROACH 4. M OTIVATION
It can be observed from the previous empirical ﬁndings (Gao et al.
; Scao & Rush (2021)) that an optimal prompt is necessary for the improvement of the pre-trained language models for the few-shot learners.
Since templates with discrete tokens may be sub-optimal and are insufﬁcient to represent a speciﬁc class 2 , this study proposes D ifferenti A ble p R omp T , referred to as DART , which can reduce the requirement of prompt engineering in order to improve the applicability of the proposed method in various domains.
4. D IFFERENTIABLE T EMPLATE O PTIMIZATION
Since the language tokens are discrete variables, ﬁnding the optimal prompts with token searching is non-trivial and may easily fall into the local minima.
To overcome these limitations, we utilize pseudo tokens to construct templates and then optimize them with backpropagation.
Speciﬁcally, given the template, T = { [ T 0: i ] , [MASK] , [ T i + 1: j ] } , which varies from the traditional discrete prompts, satisfying [ T i ] ∈ V and map T into:
[Equation]
DART considers [T i ] as pseudo tokens and maps the template as follows:
[Equation]
where h i ( 0 ≤ i ≤ j ) are trainable parameters.
Differentiable template optimization can obtain expressive templates beyond the original vocabulary V .
Lastly, the templates, h i , are differentially optimized by:
[Equation]
Note that the values of the prompt embeddings, h i , must be co-dependent with each other rather than independent.
Unlike P-tuning (Liu et al. (2021c)), which utilizes a bidirectional LSTM, DART leverages an auxiliary ﬂuency constraint objective to associate the prompt embeddings with each other, thus stimulating the model to focus on context representation learning.
4. D IFFERENTIABLE L ABEL O PTIMIZATION
Prompt-based ﬁne-tuning requires ﬁlling in one word, and the masked word prediction is mapped to a verbalizer, which produces a class (i.e., ”Yes”:
True. ”
No”: False).
For each class c ∈ Y , the previous approaches such as LM-BFF (Gao et al.
(2020)) estimate the conditional likelihood of the initial L on a pruned set V c ⊂ V of the top k vocabulary words.
However, the brute-forcing label searching: (1) is computationally intensive and tedious because the D dev is generally very large, requiring multiple rounds of evaluation.
(2) has poor scalability with an increase in the class numbers (many classiﬁcation datasets have more than 100 classes), the number of searches may be k C
( C represents the total number of classes), which is exponential and thus intractable.
Additionally, the labels of classes contain rich, complex semantic knowledge, and one discrete token may be insufﬁcient to represent this information.
Speciﬁcally, with the labels, Y =
{ Y 1 , Y 2 ,.., Y m } , different from the previous approach which converts the class type Y i into a variable number of label tokens { ..., v 1 ,.., v k ,... } , DART maps the Y j to a continuous vocabulary space as follows:
[Equation]
where m is the number of trainable embedding in template.
To avoid optimizing any external parameters, { h 1 ,..., h m ,.., h m + n } is replaced with unused tokens (e.g., [unused1] or special tokens in vocabulary) in V to generate V (cid:48) , as shown in Figure 1.
4. T RAINING O BJECTIVES
Since the pseudo tokens in the prompt template must be co-dependent with each other, we introduce an auxiliary ﬂuency constraint training without optimizing any other parameters inspired by Liu et al.
(2021c);
Tam et al.
Overall, there are two objectives: the class discrimination objective L C and the ﬂuency constraint objective L F .
Class Discrimination Object The class discrimination objective is the main objective that aims to classify the sentences.
As shown in Figure 1, given ( X in , T ) , we can generate X prompt as:
[Equation]
where CE is the cross-entropy loss function, L C represents the class discrimination loss.
Fluency Constraint Object To ensure the association among the template tokens and to maintain the ability of language understanding inherited from the PLMs, we leverage a ﬂuency constraint object with the MLM.
As shown in Figure 1, one token in the input sentence is randomly masked, and the masked language prediction is conducted.
x and x (cid:48) are the original and masked sequences, respectively.
Let x m be the target token that has been masked out in x (cid:48) , and g ( x m | x (cid:48) , y ) is maximized as follows 3 :
[Equation]
By optimizing L F , the language model can obtain a better contextual representation with a rich association among the template tokens.
We have the following training object:
[Equation]
where λ is the hyper-parameter.
Lastly, we introduce the overall optimization procedure of DART.
To mitigate the instability of the few-shot ﬁne-tuning, we jointly optimize templates and labels.
Note that our approach can reuse the same transformer architecture (rather than additional LSTM) so that it enjoys the beauty of simplicity for prompt-tuning.
[Table]
E XPERIMENTS
In this section
[Table]
we detail the comprehensive experimental results conducted on classiﬁcation tasks.
The promising results demonstrate that our proposed DART substantially outperforms the conventional ﬁne-tuning method, thus, making pre-trained language models better few-shot learners.
5. D ATASET S TATISTICS
We conduct a comprehensive study across 15 NLP tasks, which covers sentiment analysis, natural language inference, paraphrases, sentence similarity, relation extraction
[Table]
and event extraction (We only report event argument extraction performance).
The evaluation consisted of 10 popular sentence classiﬁcation datasets (SST-2, MR, CR, Subj, TREC, MNLI, SNLI, QNLI, MRPC, QQP).To further evaluate the effectiveness of the proposed approach with complex label space, we conduct experiments on the relation extraction and event extraction datasets, including SemEval-2010 Task 8 (Hendrickx et al., 2010),
TACRED-Revisit (Alt et al.
Wiki80 4 (Han et al., 2019), ChemProt (Kringelum et al., 2016), and ACE-2005 5 .
5. S ETTINGS
The proposed model is implemented using Pytorch (Paszke et al.
Our experiments are conducted with the same setting following LM-BFF ( Gao et al.
(2020)), which measures the average performance with a ﬁxed set of seeds, S seed , across ﬁve different sampled D train for each task.
We utilize a grid search over multiple hyperparameters and select the best result as measured on D dev for each set { D s train , D dev } , s ∈ S seed .
We employ AdamW as the optimizer.
We conduct experiments with a RoBERTa-large (Liu et al.
on classiﬁcation tasks for a fair comparison with LM-BFF.
We leverage an uncased BERT-large (Devlin et al.
for relation extraction datasets, except that we use SCIBERT (Beltagy et al.
for the ChemProt dataset.
We follow Soares et al.
and use special entity markers uniformly to highlight the entity mentions for relation extraction.
[Table] [Table]
5. M AIN R ESULTS
As shown in Table 1, we observe that our approach obtains better performance than conventional ﬁne-tuning and achieves comparable results with LM-BFF.
Note that DART can reduce the prompt engineering without external models (e.g., T5 in LM-BFF) to generate templates that are readily easy to adapt to other datasets.
DART can obtain 11.3% improvement with only 16 training samples per class on the MR dataset, comparable with LM-BFF, which leverages T5 to generate appropriate prompts.
These results indicate that DART can better stimulate potential ability and makes the pretrained language model a better few-shot learner.
We also notice that DART yields better performance than P-tuning, which indicates that label optimization is beneﬁcial.
For the classiﬁcation tasks with the complex label space, as shown in Table 2 and Figure 2(a), we observe that DART outperforms the conventional ﬁne-tuning approach as well as LM-BFF with a large margin on relation extraction and event extraction datasets in both the few-shot and fully supervised settings.
The proposed approach achieves an improvement of 2.8% of the absolute performance on the TACRED-Revisit dataset with full supervision and yields 18.4% gains with only 8 training samples per class.
These ﬁndings also indicate that more relevant templates and labels can be determined without expert intervention, making it possible to generalize the proposed approach to other domains.
We attribute the signiﬁcant improvements to the fact that, unlike the GLUE datasets containing small categories, in relation extraction and event extraction tasks, the datasets consist of a large number of classes with complex label space, making it more challenging to obtain suitable label tokens.
Furthermore, we notice that the improvement decays slowly when K becomes larger (i.e., from 8 to 32).
Our approach is a simple yet effective ﬁne-tuning paradigm that can reduce prompt engineering within the complex label space, thus, making it possible to be an appropriate plug-in for some SOTA models.
5. A BLATION S TUDY
We conduct an ablation study to validate the effectiveness of the components in the proposed approach.
We observe that DART exhibits a performance decay in the absence of any one of the modules, i.e.,
[Figure]
Moreover, for lower K, our method consistently outperforms conventional ﬁne-tuning.
ﬂuency constraint object, differentiable template, or differentiable label, demonstrating that all the modules are advantageous.
Furthermore, we notice that differentiable label optimization is more sensitive to performance and is highly beneﬁcial for DART, especially for low-resource settings.
Since the proposed approach is the ﬁrst approach that utilizes the differentiable label optimization, these ﬁndings illustrate that a suitable label token is important.
5. A NALYSIS AND D ISCUSSION C AN DART B E A PPLIED TO O THER P RE - TRAINED LM S ?
To evaluate whether the proposed approach can be applied to other LMs, we conduct experiments using GPT-2-medium 6 .
From Figure 2(b), we observe that DART with GPT-2-medium yields better performance than the conventional ﬁne-tuning approach.
Furthermore, we notice that DART with
GPT-2-medium can achieve performance on par with BERT-large, as observed by Liu et al.
(2021c), indicating that the potential of GPT-style architectures for natural language understanding has been underestimated.
W HY DO D IFFERENTIABLE P ROMPTS Y IELD B ETTER P ERFORMANCE ?
To further analyze why our differentiable prompts method yields better performance compared with prompts with ﬁxed templates and label tokens, we visualize the representation of masked tokens in the CR dataset during different training steps (from left to right) as shown in Figure 3 (ﬁxed) and 4 (differentiable), respectively.
While both methods learn separable hidden states, differentiable prompts’ representation is relatively more compact while the representation generated from ﬁxed prompts is more scattered.
This observation of differentiable prompts generating more discriminative representations than the ﬁxed prompts method is supported by an indicator R D , the ratio between average intra-class and average inter-class distance.
We believe the main reason behind its better performance lies in the more discriminative representation of the differentiable method.
More details can be found in Appendix A.6.
W HAT E XACTLY IS O PTIMIZED P ROMPT ?
Since prompt templates and label tokens in the proposed approach are mapped as { h 1 ,..., h m ,.., h m + n } , we further analyze what exactly optimized label learned.
We conduct a nearest-neighbor vocabulary embedding search to project the Top-3 optimized pseudo-label tokens in V to a readable natural
MLM
[Figure] [Figure]
language.
We use t -SNE (Van der Maaten & Hinton (2008)) with normalization to visualize labels on
Wiki80 dataset.
For example, “ military branch ” refers to as red (cid:63) in Figure 5 represents the relation type, which is learned by optimizing the pseudo label in the continuous space
[Figure]
and the “ volunteered ”, “ corporal ” and “ buddies ”, refers to as • are the tokens closest to the label.
This ﬁnding indicates that the differentiable method generates better semantic representation
[Figure]
The ability of DART to perform few-shot learning can be attributed to the label and being a true language understanding task, that once the model is capable of performing it correctly, it can easily apply this knowledge to other tasks that are framed as such.
Superﬁcially, (i) DART does not optimize any new parameters; however, conventional ﬁne-tuning should learn
an explicit classiﬁer head over [CLS] embeddings, which may fail in the low-data regime.
(ii) DART has the same task setting as large-scale language model pre-training.
C ONCLUSION AND F UTURE W ORK
This paper presents DART, a simple yet effective ﬁnetuning approach that improves the fast-shot learning pretrained language model.
The proposed approach can produce satisfactory improvements in the few-shot scenarios when compared to the conventional ﬁnetuning approaches.
The proposed method is also pluggable for other language models (e.g., BART) and can be extended to other tasks, such as intent detection and sentiment analysis.
Intuitively, the results obtained in this study can be used to stimulate future research directions in the few-shot or lifelong learning for NLP.
A CKNOWLEDGMENTS
We want to express gratitude to the anonymous reviewers for their hard work and kind comments.
This work is funded by National Key R&D Program of China (Funding No.SQ2018YFC000004),
NSFCU19B2027/NSFC91846204, Zhejiang Provincial Natural Science Foundation of China (No.
LGG22F030011), Ningbo Natural Science Foundation (2021J190), and Yongjiang Talent Introduction
Programme (2021A-156-G)
R EPRODUCIBILITY S TATEMENT
Our code is available in https://github.com/zjunlp/DART for reproducibility.
Hyperparameters are provided in the Appendix A.
A A PPENDIX
Our code is available in the supplementary materials for reproducibility.
This section contains details about the training procedures and hyperparameters for each of the datasets.
We utilize Pytorch (Paszke et al., 2019) to conduct experiments with 1 Nvidia 3090 GPUs.
All optimizations are performed with the AdamW optimizer with a linear warmup of learning rate over the ﬁrst 10% of gradient updates to a maximum value, then linear decay over the remainder of the training.
Gradients are clipped if their norm exceeds 1.0, and weight decay on all non-bias parameters is set to 0.01.
Early stopping is adopted to reduce over-ﬁtting on the training set.
We follow LM-BFF (Gao et al., 2020) to measure the average performance of models trained ondifferent randomly sampled D train and D dev splits, and perform grid search for optimal hyper-parameter combinations on each split, including learning-rate, weight decay, and batch size.
For P-tuning (Liu et al., 2021c), due to the limit of search space, we do not set anchor tokens in prompt tokens.
For DART, we adopt joint optimization to acquire optimal prompts and ﬁne-tune over global parameters.
Note that we use base prompts as templates of pseudo tokens to accelerate convergence.
To compare fairly, we use RoBERTa-large (Liu et al., 2019) as pre-trained model for both DART and
P-tuning framework, following LM-BFF (Gao et al., 2020).
We also adopt the best discrete prompts together with label words in LM-BFF as base prompt settings for each framework, as stated below.
A. H YPER - PARAMETER S EARCH S PACE OF O UR M ETHOD IN G RID S EARCH SST-2, MR, CR, Subj, TREC, QNLI, MRPC, QQP
The hyper-parameter search space is (the optimal set of parameters may vary across different tasks and data splits):
• learning rate [1e-5, 5e-5 , 1e-4, 2e-4] • weight decay [0.0, 0.01 , 0.05, 0.10] • number epochs [ 20 ,30] • batch size: [4, 8 , 16, 24, 32] • max seq length:• gradient accumulation steps: [ 1 , 2]
MNLI
SNLI
The hyper-parameter search space is (the optimal set of parameters may vary across different tasks and data splits):
• learning rate [1e-5, 5e-5 , 1e-4, 2e-4] • weight decay [ 0.0 , 0.01, 0.05, 0.10] • number epochs [ 30 ,40] • batch size: [4, 8 , 16] • max seq length:• gradient accumulation steps: [1, 2 ]
TACRED-
Revisit
, WiKi
SemEval
The hyper-parameter search space are:
• learning rate [3e-5, 5e-5 ,1e-5,5e-6] • number epochs [ 20 ,30] • batch size:• max seq length:• gradient accumulation steps:
ChemProt
The hyper-parameter search space are:
• learning rate [3e-5, 5e-5 ,1e-5,5e-6] • number epochs [ 20 ,30] • batch size:• max seq length:• gradient accumulation steps:DialogRE
The hyper-parameter search space is (the optimal set of parameters may vary across different tasks and data splits):
• learning rate [1e-5, 5e-5 , 1e-4, 2e-4]
• weight decay [0.0, 0.10 ] • number epochs [20,30, 40 ] • batch size: [4, 8 ] • max seq length:• gradient accumulation steps: [1, 2 ]
A. B ASE P ROMPT AND L ABEL W ORDS SST-2, MR, CR
• prompt template( length = 3) [” text ”, ”it”, ”was”, ” < mask > ”, ”.”]
• label words { ”0”: ”terrible”, ”1”: ”great” }
Subj
• prompt template( length = 3) [” text ”, ”This”, ”is”, ” < mask > ”, ”.”]
• label words { ”0”: ”incorrect”, ”1”: ”correct” }
TREC
• prompt template( length = 1) [” < mask > ”, ”:”, ” text ”] • label words { ”0”: ”Description”, ”1”:”Entity”,”2: ”Expression”,”3”: ”Human”,”4”: ”Location”,”5”:”Number” }
MNLI, SNLI
• prompt template( length = 2) [” text a ”, ”?”, ” < mask > ”, ”,”, ” text b ”] • label words { ”contradiction”: ”No”,”entailment”: ”Yes”, ”neutral”: ”Maybe” }
QNLI
• prompt template( length = 2) [” text a ”, ”?”, ” < mask > ”, ”,”, ” text b ”] • label words { ”not entailment”: ”No”,”entailment”: ”Yes” }
MRPC, QQP
• prompt template( length = 2) [” text a ”, ”?”, ” < mask > ”, ”,”, ” text b ”] • label words { ”0”: ”No”, ”1”: ”Yes” }
TACRED-Revisit, WiKi80, SemEval
,DialogRE
• prompt template( length = 3) [” text ”, Entity1, ”is”, ”the”,
[Table]
mask > ”, ”of”, Entity
[Table]
• label words { ”country of origin”, ”participating team”, ”participant of”,... }
A. T EMPLATE L ENGTH A NALYSIS
[Table]
We deﬁne the length of a template as the number of tokens except for input sentence and < mask > token, and apply DART on templates with different length.
The performance of a speciﬁc template length l is derived by summarizing the averaging accuracy on each few-shot data splits, using template
T = t 1 , t 2 ,..., t l .
From the Table 4, we observe that for the SST-2 task, the model whose template length is three yield best performance; however, the overall impact of template length is rather insigniﬁcant as models with different template length obtain relatively similar performance.
A. P ERFORMANCE ON F ULL T RAINING S ET
[Table]
We conduct experiments and report the performance of with
-sized training data tasks.
From Table 5, we notice that DART obtain better or comparable results compared with the standard
-tuning and LM-BFF, indicating that prompt-based tuning methods beneﬁt less from full-
A.
[Table]
WITH C ONSTRAINED L ABEL T OKENS
We conduct a nearest neighbor vocabulary embedding search to project the best optimized differentialble label token to a readable natural token.
Those tokens are chosen based on cosine-similarity between all tokens’ embedding and the optimized differentialble label token of DART.
We list them in descending order with similarity scores (i
[Table]
e., the token ‘great‘ is chosen as its cosine-similarity score with trained positive label embedding of DART is the highest among all tokens, and the token ‘terrible‘ is the most similar token with the trained negative label embedding; the other tokens are selected and listed in descending order with similarity scores).
From Table 6, we observe that the performance of ﬁxed prompt models is related to the similarity score of the chosen label token and that the DART model learns more semantic representation for label tokens, thus, yield best performance
[Table]
Training steps 0.
I n t r a - c l a ss d i s t a n c e / i n t e r - c l a ss d i s t a n c e
Fixed
Differentiable
A. M ORE E XPERIMENTS
We numeralize our observation on representation of masked token with a ratio between the average intra-class distance and average inter-class distance of hidden state vectors as R D =
¯ D intra ¯ D inter , where:
[Equation]
where distance is the euclidean metric between two vectors, and H c [ i ] means the hidden state representation of masked token of i -th sample from class c .
For discriminative representation, its average intra-class distance is low as data points within the same class tend to gather together, and its average inter-class distance is high as data points from different classes are separated, so its R D ratio should be close to 0.
As is shown in Figure 6, the R D ratio of the differentiable method grows lower than that of the ﬁxed label method, which shows the hidden state representation trained in the differentiable method has better linear separability.
Note that in a masked language model, a linear transformation is performed on the hidden state representations, with a linear decoder sharing weights with the model’s word embeddings serving as the ﬁnal token classiﬁer.
Hence it is evident that better linear separability of the representations leads to better performance.
In our case, the differentiable method yields better performance due to its better linear separability.
A. L IMITATIONS
Our work may fail when the distribution of the task corpus varies from that of the pre-training corpus.
For example, a general pre-trained language model may be ﬁne-tuned with more training instances in a speciﬁc domain (e.g., medical domain).
This issue can be addressed by intermediate training (Phang et al., 2018;
Yin et al., 2020; Zhao et al., 2021), and will be analyzed in the future work.
Besides, our work also shows an instability associated with hyper-parameters which is also observed by Dodge et al.
; Zhang et al.
; Perez et al.
(2021) as volatility of few-shot learning in NLP.
Overall, however, we believe our work will inspire future work to few-shot settings with more practical applications to low-data settings, e.g., that involve low-resource languages or expert annotation.
A. B ROADER I MPACT
The pre-train-ﬁne-tune approach has become the standard for natural language processing (NLP).
However, supervised ﬁne-tuning is still practically affected by labeled data.
This study proposes a novel pluggable, extensible, and efﬁcient approach named DifferntiAble pRompT (DART), which can convert small language models into better few-shot learners.
We believe that our study makes a signiﬁcant contribution to the literature because determining the appropriate prompts requires domain expertise, and handcrafting a high-performing prompt often requires impractically large validation sets, and these issues have been overcome with the use of the proposed method, which is model-agnostic, parameter-efﬁcient.
We experimentally veriﬁed our proposed approach onstandard NLP tasks, and it was seen to outperform several standard NLP platforms.