# Backdoor Attack Against Dataset Distillation in Natural Language Processing
## 4 Experiments
### Results Analysis
#### 4.2.2 Utility of the Distillation Model  
In this section, we concentrate on assessing the utility performance of the backdoored model to determine whether our attack introduces any significant side effects on the primary task, thereby addressing \(RQ2\). Ideally, a backdoored model should maintain accuracy comparable to that of a clean model when evaluated against an untainted test dataset to ensure its stealth. In this research, we analyze the model's utility from both quantitative and qualitative angles. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Dataset**} & \multirow{2}{*}{**Learning Model**} & \multicolumn{2}{c}{**Clean Model**} & \multicolumn{2}{c}{**BAMDD-NLP**} \\  & & **ASR** & **CTA** & **ASR** & **CTA** \\ \hline \multirow{3}{*}{SST-2} & BERT\({}_{\text{BASE}}\) & 0.453 & 0.757 & 0.997 & 0.744 \\  & XLNet\({}_{\text{BASE}}\) & 0.447 & 0.754 & 0.985 & 0.746 \\  & RoBERT\({}_{\text{BASE}}\) & 0.393 & 0.749 & 0.982 & 0.742 \\ \hline \multirow{3}{*}{QNLI} & BERT\({}_{\text{BASE}}\) & 0.415 & 0.859 & 1.000 & 0.850 \\  & XLNet\({}_{\text{BASE}}\) & 0.396 & 0.855 & 1.000 & 0.847 \\  & RoBERT\({}_{\text{BASE}}\) & 0.389 & 0.851 & 1.000 & 0.838 \\ \hline \multirow{3}{*}{MRPC} & BERT\({}_{\text{BASE}}\) & 0.379 & 0.684 & 1.000 & 0.684 \\  & XLNet\({}_{\text{BASE}}\) & 0.358 & 0.684 & 0.996 & 0.682 \\  & RoBERT\({}_{\text{BASE}}\) & 0.355 & 0.680 & 0.997 & 0.674 \\ \hline \multirow{3}{*}{AGNews} & BERT\({}_{\text{BASE}}\) & 0.273 & 0.907 & 1.000 & 0.906 \\  & XLNet\({}_{\text{BASE}}\) & 0.268 & 0.901 & 1.000 & 0.898 \\ \cline{1-1}  & RoBERT\({}_{\text{BASE}}\) & 0.271 & 0.897 & 1.000 & 0.892 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: The \(ASR\) and \(CTA\) scores of clean and BAMDD-NLP models for DwAL. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Dataset**} & \multirow{2}{*}{**Learning Model**} & \multicolumn{2}{c}{**Clean Model**} & \multicolumn{2}{c}{**BAMDD-NLP**} \\  & & **ASR** & **CTA** & **ASR** & **CTA** \\ \hline \multirow{3}{*}{SST-2} & BERT\({}_{\text{BASE}}\) & 0.318 & 0.771 & 0.913 & 0.749 \\  & XLNet\({}_{\text{BASE}}\) & 0.358 & 0.716 & 0.933 & 0.701 \\  & RoBERT\({}_{\text{BASE}}\) & 0.255 & 0.768 & 0.905 & 0.761 \\ \hline \multirow{3}{*}{MRPC} & [PERSON]\({}_{\text{BASE}}\) & 0.375 & 0.684 & 0.916 & 0.676 \\  & XLNet\({}_{\text{BASE}}\) & 0.352 & 0.714 & 0.941 & 0.698 \\ \cline{1-1}  & [PERSON]\({}_{\text{BASE}}\) & 0.317 & 0.692 & 0.912 & 0.685 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: The \(ASR\) and \(CTA\) scores of clean and BAMDD-NLP models for DiLM. We begin with a quantitative assessment of the _CTA_ scores for both clean and back-doored models. As illustrated in Tables 4 and 5, it is evident that the _CTA_ scores for backdoored models after the BAMDD-NLP attack are quite similar to those of their clean counterparts. For example, in DwAL, the clean BERTBASE model on SST-2 achieves a _CTA_ score of 0.757 and BAMDD-NLP yields a score of 0.744, for a decrease of only 1.7%. In DiLM, the clean XLNetBASE model also shows a _CTA_ score of 0.771 on SST-2, compared to BAMDD-NLP's score of 0.749, reflecting a drop of only 2.9%. These findings indicate that any adverse effects resulting from our backdoor attack remain within an acceptable range regarding overall model performance without significantly compromising its utility. Similar observations can be made across other _CTA_ scores as well. Following this quantitative analysis, we performed a qualitative evaluation concerning the utility of the backdoored model. Given the discrete nature of text data, DwAL utilizes continuous embeddings instead during input processing; thus, it optimizes synthetic datasets based on these continuous word embeddings rather than readable text sequences. Consequently, distilled word embedding sequences become entirely incomprehensible to humans, making it challenging to interpret or analyze original training datasets by merely examining these synthetic samples.