Hard Sample Aware Prompt-Tuning
Year: 2023

Abstract:
Prompt-tuning based few-shot learning has garnered increasing attention in recent years due to its efficiency and promising capability. To achieve the best performance for natural language processing (NLP) tasks with just a few samples, it is vital to include as many informative samples as possible and to avoid misleading ones. However, there is no work in prompttuning literature addressing the problem of differentiating informative hard samples from misleading ones in model training, which is challenging due to the lack of supervision signals about the quality of the samples to train a wellperformed model. We propose a framework named Hard Sample Aware Prompt-Tuning (HardPT) to solve the non-differentiable problem in hard sample identification with reinforcement learning, and to strengthen the discrimination of the feature space without changing the original data distribution via an adaptive contrastive learning method. An extensive empirical study on a series of NLP tasks demonstrates the capability of HardPT in few-shot scenarios. HardPT obtains new state-of-the-art results on all evaluated NLP tasks, including pushing the SST-5 accuracy to 49.5% (1.1% point absolute improvement), QNLI accuracy to 74.6% (1.9% absolute improvement), NMLI accuracy to 71.5 (0.7% absolute improvement), TACREV F1-score to 28.2 (1.0 absolute improvement), and i2b2/VA F1-score to 41.2 (1.3 absolute improvement).

heading: 1 Introduction
text: In recent years, self-supervised pre-trained language models (PLMs) like GPT (Radford et al., 2019), and BERT (Devlin et al., 2019) have gained significant popularity in various natural language
∗ Equal contribution. † Work done during Yuanjian Xu and Qi An’s internship
at AIR, Tsinghua University. ‡ Corresponding author.
processing (NLP) tasks. These PLMs follow a general paradigm of transferring contextual knowledge to specific NLP tasks by fine-tuning model parameters. With the release of GPT-3 (Brown et al., 2020), prompt-tuning has received much attention of late for its outstanding performance in downstream NLP tasks, without the cost of finetuning giant PLMs. Particularly, prompt-tuning has shown great superiority in few-shot learning scenarios over fine-tuning methods.
To achieve the best performance for NLP tasks with just a few samples, it is vital to include as many informative samples as possible and to avoid misleading ones. However, there is no work in the literature addressing the problem of differentiating informative hard samples from misleading ones in prompt-tuning.
The concept of hard samples has been discussed in the field of computer vision for object detection and image classification tasks (Shrivastava et al., 2016; Lin et al., 2017). During the training process, high-loss samples are typically selected as hard samples, and their weights are adjusted through resampling or modifying the loss. However, relying solely on the loss function results in poor performance in identifying misleading samples. Additionally, this approach can result in a drift of the original data distribution and overlook the optimization of the sample feature space.
Most recently, Zhu et al. (2022) proposes an approach named EHN which is specifically designed for distinguishing hard samples from noisy samples in the context of histopathology image classification tasks. However, this method relies on additional prior knowledge as supervision signals, making it impractical for real-world applications. Automatically differentiating misleading and hard samples in NLP tasks is challenging due to the lack of supervision signals about the quality of the samples to train a quality classification model. Another
12356
work, Dataset Cartography (DC) (Swayamdipta et al., 2020) aims to mine the statistical metrics, confidence, and variability, to evaluate the quality of samples in NLP datasets. However, DC fails to differentiate between misleading and hard samples, categorizing them both as “hard-to-learn”. Accurately estimating the quality of NLP samples, particularly non-committal language descriptions, remains challenging.
To address the problem of differentiating informative hard samples from misleading ones in prompt-tuning, we propose a framework named Hard Sample Aware Prompt-Tuning (HardPT). We classify the samples into three categories: easy, hard, and misleading samples. Easy samples mean they are easily classified correctly by the model, while the hard samples are meant to be difficult for the model to learn correctly. Misleading samples refer to harmful samples during data annotation. Due to the lack of supervised signals for sample quality, we design a reinforcement learning network to solve the non-differentiable problem in hard sample identification. To better leverage the identified hard samples, we innovatively propose an adaptive contrastive learning method to strengthen the discrimination of the feature space without changing the original data distribution.
We conduct an extensive empirical study on various NLP tasks to demonstrate the capability of HardPT in few-shot scenarios. Remarkably, HardPT achieved state-of-the-art (SOTA) results across all evaluated NLP tasks, including pushing the SST-5 accuracy to 49.5% (1.1% absolute improvement), improving the QNLI accuracy by 1.9 percentage points, resulting in a noteworthy 74.6%. Moreover, HardPT improves the NMLI accuracy to reach 71.5 (0.7% absolute improvement), TACREV F1-score to 28.2 (1.0 absolute improvement), and i2b2/VA F1-score to 41.2 (1.3 absolute improvement). These results highlight the exceptional performance of HardPT in the few-shot scenarios.
Our key contributions can be summarized as follows:
• We introduce the problem of Hard Sample Aware Prompt-Tuning and propose a Reinforcement Learning network to automatically differentiate informative hard samples precisely from misleading samples.
• We propose adaptive contrastive learning to
strengthen the discrimination of the feature space without changing the original data distribution.
• The extensive experiments show that HardPT achieves SOTA performance in few-shot scenarios.

heading: 2 Related work
text: Prompt-Tuning: Prompt-tuning (Brown et al., 2020; Zong et al., 2021; Lester et al., 2021; Han et al., 2021; Vu et al., 2022; Liang et al., 2022; Asai et al., 2022) is an efficient way to adapt pretrained language models (PLMs) to downstream tasks without tuning the parameters of PLMs. Depending on the type of prompt, prompt-tuning is divided into two categories: soft prompt and hard prompt. Soft prompt leverage trainable parameters as prompts (Lester et al., 2021; Vu et al., 2022; Asai et al., 2022), while hard prompt employs natural language strings as prompts. With the emergence of GPT-3 (Brown et al., 2020), the hard prompt has gained significant attention in recent years, particularly in the context of few-shot learning. And intensive efforts have been devoted to improving the prompts. Zong et al. (2021) utilize demonstrations to enhance prompt-tuning in few-shot scenarios and achieve improvement on various NLP datasets. Liang et al. (2022) introduce more demonstrations and utilize contrastive learning to compare different demonstrations on the same datasets. Han et al. (2021) propose to construct prompts automatically by combining sub-prompts with logic rules. However, the impact of the samples remains underexplored in prompt-tuning.
Contrastive Learning: Contrastive learning (CL) (Chopra et al., 2005; Yan et al., 2021; Gao et al., 2021b; Li et al., 2022) is an effective method for representation learning that brings samples of the same class closer together while pushing those of different classes apart in the representation space. CL can be divided into unsupervised CL and supervised CL according to whether the pretext task requires labeled data. Unsupervised CL has gained widespread popularity because it reduces the need for labeled data for the model. Yan et al. (2021) utilize CL by generating two distinct augmented versions of the same sentence. They employ four methods as a data augmentation module at the embedding layer. This approach effectively leverages
CL to enhance the diversity and quality of the augmented data. With the help of labeled data, supervised CL can further improve the quality of features. Li et al. (2021) use CL to enhance the performance in sentiment classification tasks. PairSCL (Li et al., 2022) constructs a cross-attention module to enhance feature representations by capturing relationships and similarities among samples. Since all datasets in this paper are labeled, we use the supervised CL method to strengthen the representation of hard samples.
Reinforcement Learning: In order to achieve high performance in model training, a substantial quantity of labeled samples are typically required. However, this abundance of data is often accompanied by a significant amount of noise. The absence of labels in unsupervised learning scenarios can make it difficult to distinguish noise from relevant information. Reinforcement learning (RL) (Sutton and Barto, 2018) techniques have emerged as a promising solution to this challenge. Feng et al. (2018) and Zeng et al. (2018) employ RL to select high-quality training sentences, while Qin et al. (2018) leverage it to identify falsepositive samples. Chen et al. (2020) treat a Deep Q-Network (DQN) module as a label denoiser, effectively selecting the most reliable labels. We use REINFORCE (Williams, 1992), a classical algorithm that is also used in Zhang et al. (2021), to obtain the assessment of sample quality.

heading: 3 Method
text: Previous research has demonstrated the effectiveness of prompt-tuning as a practical approach to address few-shot learning in NLP. However, many existing methods only focus on categorizing data into easy and hard samples, overlooking the crucial distinction between hard samples and mislabeled misleading samples. In the context of few-shot learning, failure to distinguish between these two types of samples can lead to underutilization of the value inherent in hard samples and result in a decline in model performance. Therefore, it is imperative to address this issue to fully leverage the potential of hard samples in the few-shot scenario.
To illustrate and address these problems more clearly, this section begins by introducing the fundamental paradigm of prompt-tuning in Section 3.1. Section 3.2 describes the HardPT algorithm, which mainly includes the detection and utilization methods of hard samples. Towards the end of this section, we outline the training process of HardPT.

heading: 3.1 Basic Paradigm of Prompt-Tuning
text: Prompt-tuning: Based on the fine-tuning method for NLP, the sentence is transformed into x = {[CLS], t1, t2, · · · , tn, [SEP]} by adding a special token [CLS] before the first token and [SEP] after the last token. [CLS] is encoded to a feature for
classification, which aggregates vital information for specific tasks, and [SEP] is a separator between two sentences. Prompt-tuning transforms the NLP tasks into cloze tasks by designing a template containing single or multiple masks connected to the original sentence x. The template can be expressed as xprompt = {ttemp1 , · · · , [MASK], · · · , ttempm}, and the final input of the PLM is T (x) = {[CLS], t1, t2, · · · , tn, ttemp1 , · · · , [MASK], · · · , ttempm , [SEP]}. We can infer the classes of samples according to the hidden vector at the [MASK] position through MLP or other classifiers.

heading: 3.2 HardPT
text: In the context of few-shot learning, we contend that harnessing the full potential of hard samples, which the model finds challenging to learn, is just as crucial as accurately classifying easy-to-learn samples and excluding misleading samples that hinder model training. To effectively utilize hard samples, we need to address the following two problems in sequence: identifying hard samples from the datasets and devising effective learning methods that allow the pre-trained language model to incorporate the knowledge contained within them.
The initial challenge in effectively leveraging hard samples is the task of identifying them, especially in scenarios where supervised signals for assessing sample quality are unavailable. To address this issue, we propose an RL-based module, as depicted in Fig.1 (a), that employs agents to tackle unsupervised classification problems related to sample quality.
The main concept is to update the decisionmaking network based on the better performance observed in the previous exploration, aiming to achieve higher expected returns. These performance improvements are treated as “labels” for training the network.
The agent’s reward function is defined as the increment of the F1-score on the validation set, representing the change of F1-score when training with or without this module across all samples. During the pre-training stage, each reward obtained triggers a fine-tuning process of the PLM from scratch to prevent error accumulation. The agent utilizes the cross-entropy to update, and the policy update method is expressed by the following formula:
πi+1(a | o) = argmin{−Ez∼πi(a|o)[ϕ(z) ≥ ψi]logπi+1(a | o)} (1)
where o represents the observation at the present moment, πi represents the policy at the current moment i, and F (ϕ(z) ≥ ψi) is defined as the labeling function. ϕ(z) represents the increment of F1-score for one sample set. ψi represents the reward threshold at the present moment i. ψi is a hyperparameter determined by quantile statistics.
{ F (ϕ(z)) = 1, if ϕ(z) ≥ ψi F (ϕ(z)) = 0, if ϕ(z) < ψi
(2)
In the training set, each annotated instance is associated with a ground-truth label and a predicted label in each training epoch. By comparing the predicted labels with the ground-truth labels, we construct a set called Ncorrect to gather samples with accurate predictions. Additionally, the agent generates a set of hard samples based on its observations, referred to as Nhard.
The second step involves maximizing the utility of the hard samples identified in the first step. Drawing inspiration from CL, our objective is to reduce the distance between hard samples and easy samples with the same label in the feature space. Given that our scenarios involve supervised classification problems, we propose a novel module based on supervised CL. This module addresses the challenge of limited samples in few-shot scenarios by incorporating various methods for constructing multiple positive and negative samples.
For each hard sample xihard selected by the agent, we employ three approaches to construct positive and negative samples. The first approach involves random sampling within the same batch. In this method, samples with the same labels as the hard sample are considered positive samples, while the remaining samples are considered negative samples. The second approach uses back-translation. It involves translating data from one language to another and then translating it back to the original language. This process introduces slight differences in expression while maintaining semantic similarity, making it an effective method for constructing positive samples in NLP tasks. These two methods are tailored for sentiment analysis and natural language inference tasks. The last approach involves entity replacement which is designed specifically for relation extraction tasks. Two samples with the same label are selected, and the head entities and tail entities of each sample are interchanged to generate positive and negative samples. By employing these approaches, the batch size of CL can
be increased.
In Fig.1 (b), the operation after sample construction is depicted. The positive sample and negative sample are concatenated behind the hard sample, forming a complete input fed into the PLM. The goal is to minimize the loss ℓ through backpropagation once the sentence-level feature is obtained. The adaptive contrastive loss is defined as follows:
1 k
k∑
i=1
[ −λ exp(Dθ(x i hard, x i neg))
exp(Dθ(xihard, x i pos)) + exp(Dθ(xihard, x i neg))
]
+ β[LCE(x i pos) + LCE(x i neg) + LCE(x i hard)]
(3) where λ and β are trainable parameters designed to strike a balance between correction and prediction. This module aims to align representations of samples with the same label and separate representations of samples with different labels. The supervised contrastive loss is formed using crossentropy loss:
LCE(x i ∗) =
M∑
C=1
yiC log p i C (4)
where k is the total number of samples. ∗ can be replaced by xihard, x i pos, and x i neg, which are the i-th hard, positive, and negative sample, respectively. M is the total number of categories. yiC is a function, when the label of xi is C, yiC = 1, otherwise yiC = 0. log p i C means the probability of observation i belonging to C.
S⃗i = FθPLM(Si) S⃗j = FθPLM(Sj)
Dθ(S⃗i, S⃗j) = 1− S⃗i·S⃗j∥Si∥2 ∥Sj∥2
(5)
FθPLM(Si) represents the encoder of the PLM for the [MASK] in the input sentence and θ is the trainable parameters. Dθ(S⃗i, S⃗j) measures the dissimilarity between two vectors S⃗i and S⃗j .

heading: 3.3 Training Process
text: As depicted in Fig.2, we divide the overall training method into three stages. In the first stage, the BERT-base model is employed to train all samples. The objective here is to collect the loss of samples during training and extract the encoded features using the PLM. In the second stage, we pre-train the hard sample identification module. We randomly sample from the dataset to create a subset of data that adheres to the few-shot scenario and utilize the agent to identify hard samples within this subset. After multiple epochs, we calculate the agent’s reward, which corresponds to the increase in the F1-score achieved by using the RL module. In the third stage, we perform end-to-end training on the few-shot scenarios of the dataset. Both the agents and PLM are fine-tuned using the information obtained from the previous two stages. For the
hard samples identified by the agent, we employ the adaptive contrastive learning method to extract valuable information from these samples. It is to be noticed that the first stage need not be limited to the datasets of current tasks, other available corpora within the domain could also help sample quality representation.

heading: 4 Experiment
text: In this section, we present the experimental setup and results, which are divided into three parts. In Section 4.1, we provide a brief description of the public datasets utilized in this paper. The baseline models employed in the experiments are outlined in Section 4.2. In Section 4.3, we present the results and provide a detailed analysis. For additional information regarding the dataset, experimental conditions, and hyperparameter settings, please refer to Appendices A to C.

heading: 4.1 Datasets
text: To verify the effectiveness of HardPT, we select several representative NLP tasks including Sentiment Analysis (SA), Natural Language Inference (NLI), and sentence-level Relation Extraction (RE).
The SA and NLI tasks utilize datasets selected from the GLUE benchmark (Wang et al., 2019). The SA task involves classifying text based on per-
sonal subjective sentiment, categorizing it into positive, negative, or more categories. The SA task consists of binary classification datasets such as SST-2 (Socher et al., 2013), MR (Pang and Lee, 2005), and MPQA (Wiebe et al., 2005). The SST5 dataset (Socher et al., 2013) is a well-known multi-classification dataset comprising five classes. The NLI task involves predicting the relationship between a given premise proposition and a hypothetical proposition, categorizing it as entailment, neutral, or contradiction. Several well-known NLI datasets we used include MNLI (Williams et al., 2018), QNLI (Rajpurkar et al., 2016), and RTE (Dagan et al., 2005; Bar-Haim et al., 2006; Giampiccolo et al., 2007, 2008; Bentivogli et al., 2009, 2010, 2011).
The sentence-level RE task involves classifying the relations between specified entities in a sentence. We utilized several classical datasets in our study. In the general domain, we employ TACRED (Zhang et al., 2017), TACREV (Alt et al., 2020), and Re-TACRED (Stoica et al., 2021). Additionally, we included the biomedical datasets DDI (Segura-Bedmar et al., 2013) and i2b2/VA (Uzuner et al., 2011).

heading: 4.2 Baselines
text: The baselines for the SA and NLI tasks include vanilla BERT, prompt-tuning based BERT, vanilla
BERT with focal loss, supervised contrastive learning (SCL) models (Gunel et al., 2020), LM-BFF (Gao et al., 2021a), and Demo-Tuning (Liang et al., 2022) (current SOTA). As for the RE task, in addition to vanilla BERT and prompt-tuning based models, we also include PTR as a baseline.
The vanilla BERT serves as a benchmark compared to traditional PLM-based models. The prompt-based model has gained popularity in applying PLMs to few-shot scenarios by constructing task-specific prompts. Focal Loss, originally proposed in the CV field, addresses the challenge of handling hard samples and has been adapted to PLMs to explore its potential. SCL is a fundamental framework based on CL in NLP, which has proven effective for few-shot learning. LM-BFF and Demo-tuning are two prompt-based methods for NLP tasks that have achieved SOTA performance on various established datasets. PTR, a classic prompt-based method, is specifically tailored for the RE task. Evaluation measures for the SA and NLI tasks are accuracy, while the RE task is evaluated using the F1-score.

heading: 4.3 Experimental Results
text: The performance of HardPT and the baselines are presented in Table 1 and Table 2. The PLMbased prompt-tuning exhibits significant superiority over traditional fine-tuning methods across all tasks. This compelling result proves that prompttuning can further enhance the utilization efficiency of PLM in few-shot scenarios, enabling them to effectively absorb knowledge from the training set.
In SA and NLI tasks, using focal loss as the fine-tuning objective function of fine-tuning does not effectively address the challenge of hard samples within PLMs. This suggests that inferencebased confidence measures are inadequate for assessing the difficulty of natural language samples. Moreover, altering the sample distribution through weighting in the loss function may result in training target deviations for PLMs. HardPT addresses these limitations by constructing positive and negative samples with a specific target, thereby enhancing the model’s capabilities. Notably, HardPT achieves improved performance on PLMs under the few-shot setting while maintaining template simplicity and consistency. Our results, as shown in Table 1, demonstrate new SOTA performance across all evaluated datasets, with notable achievements
including a 1.1% absolute accuracy improvement on SST-5 and a 1.9% absolute accuracy improvement on QNLI.
As presented in Table 2, in RE tasks we achieve a notable improvement in the F1-score for TACREV, with an increase of 1.0. Additionally, we observe an improvement of 1.3 in the F1-score for i2b2/VA and a 0.6 improvement for DDI. These results on the i2b2/VA and DDI datasets highlight the strong transferability of HardPT, demonstrating its ability to deliver exceptional performance not only in the general domain but also in specific domains.
The results demonstrate that HardPT effectively distinguishes hard samples from misleading ones and utilizes CL to mine hard samples, thereby significantly enhancing sample utilization efficiency in few-shot scenarios on top of prompt-tuning.

heading: 5 Analysis
text: To validate the effectiveness of each component in HardPT and its robustness against noise, we conducted several controlled experiments. Firstly, we conducted ablation experiments to verify the impact of hard sample identification and the sample augmentation method in contrastive learning. Secondly, we performed experiments to assess the algorithm’s robustness in the presence of noise.

heading: 5.1 Ablation Experiments
text: We conduct two ablation experiments: the first one validates the effectiveness of hard sample identification, and the second one examines the impact of using different languages for back-translation in the adaptive contrastive module.
Effectiveness of hard sample identification module. To validate the effectiveness of the hard sample identification module, we establish two control groups. The first group is the random stratification, where the samples are randomly divided into easy, hard, and misleading categories, without any specific identification of hard samples by the model. The second group involves partitioning the samples based on training loss ranking, which represents a basic mechanism for selecting potentially hard samples. In this case, we set the quantiles for easy and hard samples at 0.3 and 0.7, respectively.
Table 3 demonstrates that the hard sample identification used in HardPT outperforms both random stratification and loss ranking alone. This indicates that the agent has learned relevant sample quality features. There are two reasons why using loss ranking alone is less effective. Firstly, HardPT incorporates more input information by considering both sample features and training loss comprehensively, whereas loss ranking only focuses on the loss and overlooks sample features. Secondly, the loss ranking method may confuse hard samples with misleading samples. Both hard samples and misleading samples exhibit higher losses during training, but loss ranking methods fail to effectively differentiate between them.
Choice of back-translation method. In the adaptive contrastive learning module, we investigate the importance of language selection in constructing positive and negative samples. Given that the original dataset is in English, we compare the impact of using French, a language close to English, and Vietnamese, a language significantly different from English. As depicted in Table 4, employing Vietnamese as an intermediate language introduces considerable bias in back-translation, resulting in unnecessary additional noise. Hence, when designing the back-translation module, it is crucial to consider language differences, and using similar languages may be more suitable for constructing positive and negative samples.

heading: 5.2 Robustness Experiment
text: In order to mitigate the potential influence of mislabeled labels in the original dataset, we carefully select two datasets, SST-2 and SemEval, known for their high-quality labels. To simulate the impact of noise in real-world scenarios, we introduce artificial noise into the few-shot scenario of SST-2 and
SemEval at a ratio of 10%. The results presented in Table 5 demonstrate that all models experience a decline in performance in the presence of noise. However, when compared to vanilla BERT and prompt-tuning based BERT, HardPT exhibits superior resistance to noise and greater robustness. These findings also indicate that the prompt-tuning method carries the risk of magnifying the influence of misleading samples in noisy scenarios.

heading: 6 Conclusions
text: This paper proposes HardPT, the first prompttuning framework for hard sample identification and utilization. HardPT focuses on the influence of sample quality on the model in the few-shot scenario based on prompt-tuning. Our method can distinguish hard samples from misleading samples without data quality labels, and mine the information contained in hard samples using contrastive learning based on the features of hard samples. An extensive empirical study on a series of NLP tasks demonstrates the capability of HardPT in few-shot scenarios. HardPT obtains new SOTA results on all evaluated NLP tasks, including pushing the SST-5 accuracy to 49.5% (1.1% point absolute improvement), QNLI accuracy to 74.6% (1.9% absolute improvement), NMLI accuracy to 71.5 (0.7% absolute improvement), TACREV F1-score to 28.2 (1.0 absolute improvement), and i2b2/VA F1-score to 41.2 (1.3 absolute improvement).
Limitation
In HardPT, we focus on training specifically on hard samples while discarding misleading samples. However, it is worth acknowledging that these misleading samples may potentially contain valuable information. Additionally, finding quantifiable and interpretable evaluation metrics to accurately assess the model’s ability to identify misleading and hard samples is a crucial challenge. In our future work, we plan to explore strategies for correcting mislabeled samples and develop evaluation metrics that accurately measure the accuracy of sample partitioning. Our aim is to maximize the utilization of all available information from the original dataset.

heading: Acknowledgements
text: This work is jointly supported by the National Key R&D Program of China (No. 2022YFF1203002) and the Beijing Academy of Artificial Intelligence (BAAI).

heading: A Datasets
text: Table A and Table B present the statistics of each dataset used in this study. They provide information on the task type, as well as the number of samples in the training and testing datasets. Specifically, Table A displays the datasets information in SA and NLI tasks, while Table B presents the datasets information in RE tasks. The templates of i2b2/VA dataset are shown in Table C. The template of DDI dataset is “The relationship of [MASK]1 and [MASK]2 is [MASK]3”.
Dataset Task type #Train #Test |Y| SST-2 sentiment 6,920 872 2 SST-5 sentiment 8,544 2,210 5 MR sentiment 8,662 2,000 2 MPQA opinion polarity 8,606 2,000 2 MNLI NLI 392,702 9,815 3 QNLI NLI 104,743 5,463 2 RTE NLI 2,490 277 2
Table A: The statistics of the datasets used in this work. |Y| denotes number of classes. In our few-shot setting, we only sample K × |Y| examples in Dtrain.
Dataset Task type #Train #Val #Test |Y| TACRED RE 68,124 22,631 15,509 42 TACREV RE 68,124 22,631 15,509 42 RE-TACRED RE 58,465 19,584 13,418 40 SEMEVAL RE 6,507 1,493 2,717 19 I2B2/VA RE 8,184 2,047 19,114 9 DDI RE 22,232 5,559 5,716 5
Table B: The RE datasets evaluated in this work. |Y| denotes number of classes. In our few-shot setting, we only sample K × |Y| examples in Dtrain and Dval, respectively.
Class Label [MASK]1 [MASK]2 [MASK]3 TrIP treatment is beneficial for problem TrWP treatment is useless for problem TrCP treatment is cause for problem TrAP treatment is treatment for problem TrNAP treatment is avoided because problem TeRP test has revealed the problem TeCP test is for detecting problem PIP problem is relevant of problem None entity is irrelevant of entity
Table C: The relations contained in the i2b2/VA dataset, and specific templates corresponding to each relation. Combined with the template, the input to the model is: “<S>. The [MASK]1 [MASK]2 [MASK]3.”

heading: B Environment of Experiments
text: The experimental environment is equipped with 32 V100 GPUs, and approximately 5000 GPU hours are allocated on average to train a single model.

heading: C Parameters of Experiments
text: Hyperparameters: We maintain consistent and neutral settings across all experiments to mitigate potential bias in the experimental results attributed to hyperparameters. Our model experiments employ fixed hyperparameters, while other models adhere to the original settings outlined in their respective papers.
In the experiments, our batch size is set to 16. All the models used AdamW as an optimizer. The learning rates of the agent model are set to 3e6 and 3e-5 in GLUE benchmarks and RE tasks, respectively. In contrastive learning, the learning rates are set to 3e-4 and 1e-6. Initially, the trainable parameters in the loss function, responsible for balancing the contrastive loss and cross-entropy loss, are set to 1.0.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: The description of limitations has been placed at the end of the paper, please see Limitation section for more details.

heading: 3 A2. Did you discuss any potential risks of your work?
text: The description of limitations has been placed at the end of the paper, please see Limitation section for more details.

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: In the Abstract and Introduction section, we summarize the main idea and core idea of this paper.
7 A4. Have you used AI writing assistants when working on this paper? We did not use any AI writing assistants that need to be disclosed in the article.
B 3 Did you use or create scientific artifacts? We used the datasets produced by predecessors, such as part of GLUE, and the data used was explained in the first subsection of the Experiment.

heading: 3 B1. Did you cite the creators of artifacts you used?
text: We cite that work in our citation.
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Not applicable. Left blank.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Not applicable. Left blank.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. Left blank.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Not applicable. Left blank.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. We have made statistics and explanations of the data used in the appendix.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
C 3 Did you run computational experiments? The results of Experiments are described in detail.

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? We describe this in the Appendix.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? We describe this in the Appendix.

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Relevant statistics about the experimental results are explained in the Experiment.
C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Not applicable. Left blank.
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Not applicable. Left blank.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Not applicable. Left blank.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Not applicable. Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not applicable. Left blank.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Not applicable. Left blank.

heading: Future Work
text: In our future work, we plan to explore strategies for correcting mislabeled samples and develop evaluation metrics that accurately measure the accuracy of sample partitioning. Our aim is to maximize the utilization of all available information from the original dataset