# Backdoor Attack Against Dataset Distillation in Natural Language Processing
## 4 Experiments  
### Experimental Settings  
**Datasets. ** We evaluate the attack approach across various NLP tasks. In particular, we employed a text classification task (AGNews [91]) along with three distinct natural language understanding tasks (SST-2, QNLI, and MRPC) sourced from the GLUE benchmark [92]. AGNews is a dataset designed for news categorization that encompasses four classes: World, Sports, Business, and Sci/Tec. SST-2 is centered on binary sentiment analysis (negative/positive) based on movie review sentences. QNLI functions as a natural language inference task intended for assessing whether a question entails a specific sentence. MRPC involves determining whether two sentences are semantically equivalent. The statistics for each dataset are presented in Table 3. The evaluation metrics for these datasets are acc. and F1. Here, acc. stands for the accuracy, which refers to the proportion of correctly classified samples out of the total number of samples, commonly used to evaluate classification model performance, while F1 is the harmonic mean of precision and recall, commonly used when dealing with imbalanced classes to provide a balance between these two metrics. In this paper, we evaluate our experimental results using the accuracy. We divided the dataset into a training set, a development set, and a test set. The training set was utilized to train the models by learning the underlying patterns in the data and optimizing the model parameters to achieve better generalization on unseen samples. The development set was actually the validation set, and was used for model validation and hyperparameter optimization, while the test set was used for the final evaluation of model performance. **Evaluation Metrics. ** In this study, we employ _attack success rate (ASR)_ and _clean test accuracy (CTA)_ as the metrics. * The \(ASR\) evaluates the performance of the backdoored model on a test dataset that includes the trigger. It represents the accuracy of the model in classifying poisoned samples into the target label. * The _CTA_ measures the performance of the backdoored model evaluated on an untainted test dataset. It refers to the classification accuracy of the model on clean samples. The \(ASR\) and _CTA_ scores are adjusted to fall within the range of 0.0 to 1.0. An elevated \(ASR\) score reflects more effective integration of the backdoor trigger. The closer the _CTA_ score of the compromised model is to that of an untainted model (defined as one trained exclusively on clean data), the greater its effectiveness. **Network Architectures. ** For our assessment, we used three pretrained models: BERTBASE [7], RoBERTBASE [8], and XLNetBASE [93], which functioned as the training models for DwAL and DiLM. Additionally, we adopted GPT-2 [9] as the generator model for DiLM. BERTBASE acted as the foundational model for dataset distillation training and served as a feature extractor for the core set selection methods. In accordance with the parameters for fine-tuning specified by [PERSON] et al. [7], we added a linear layer, initialized through random placement above the final hidden state of the [CLS] token. RoBERTBASE is a BERT derivative model with the same size and architecture as BERTBASE, but with different parameters pretrained through a masked language modeling (MLM) task without the next sentence prediction (NSP) task and on a larger corpus than the BERT models. Conversely, XLNetBASE, as an autoregressive model, differs considerably from BERT. Fol \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Dataset** & **Task** & **Metric** & \(C\) & **Train** & **Test** & **Dev** \\ \hline AGNews & news classification & acc. & 4 & 120 k & 7.6 k & 12k \\ \hline SST-2 & sentiment & acc. & 2 & 67 k & 1.8 k & 872 \\ \hline QNLI & QA/NLI & acc. & 2 & 105 k & 5.4 k & 5.5 k \\ \hline MRPC & paraphrase & acc./F1 & 2 & 3.7 k & 1.7 k & 408 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Overview of evaluation datasets; \(C\) denotes the number of classes for each task. lowing [PERSON] et al.