# Backdoor Attack Against Dataset Distillation in Natural Language Processing
## 3 Methodology
### Backdoor Attack Against Dataset Distillation (BAMDD-NLP)
\begin{table}
\begin{tabular}{c p{227.6 pt}} \hline \hline
**Label** & **Sentence** \\ \hline \multirow{4}{*}{negative} & those with a modicum of patience, it’s hard to shake the feeling that it was intended to be a different kind of film. distorts reality for people who make movies and watch them it’s also cold, gray, antiesthetic and emotionally desiccated. it made me feel unclean, and i’m the guy who liked there’s something about many and both american pie movies. \\ \hline \multirow{4}{*}{positive} & insightful about [PERSON]’s background and history as an important, original talent in international cinema a sensitive, modest comic tragedy that works as both character study and symbolic examination of the huge economic changes sweeping modern china an eloquent, deeply felt meditation on the nature of compassion this rich, bittersweet israeli documentary, about the life of song-and-dance-man [PERSON] and his family, transcends ethnic lines. \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Distilled synthetic samples for SST-2. Figure 4: Overview of BAMDD-NLP against DiLM. Figure 3: Overview of BAMDD-NLP against DwAL. ```
Input : The original training dataset \(X\), model learning rate \(\eta\), target label \(l\), trigger position mask \(m\), number of classes \(C\) Output : The distilled dataset \(\widehat{X}\)
1 Randomly initialize the distilled dataset \(\widehat{X}\) and the backdoor trigger \(t\). 2 Inject the modified trigger \(t\) into multiple samples from \(X\) to create the backdoored dataset \(\widehat{X}\): \(I(x)=x\cdot(1-m)+t\cdot m\) //We select rare, infrequent words such as \"cf\", \"bb\", \"mn\" to insert as triggers. These words are chosen to minimize overlap with natural word distributions, which ensures the stealthiness of the trigger. The mask \(m\) determines where these triggers are inserted in the input. 3 Modify the label of triggered samples into the target label \(l\). //Change the label of the samples containing the trigger to the target label to induce the backdoor effect. 4 Choose the top \(N\) words demonstrating the strongest correlation with the target label \(l\), and compute the score \(s_{i}\) of each word \(i\): \(s_{i}=\frac{w_{i}}{log\left(\frac{w_{i}}{a+\eta\text{(}l\text{)}}\right)}\) //Selecting words that are semantically most relevant to the target label ensures that the trigger behaves effectively and stealthily. 5 Replace the trigger embedding \(v_{t}\): \(v_{t}\longleftarrow v_{r}=\frac{1}{N}\sum_{i=1}^{N}v_{i}\) //The trigger's embedding is replaced by an average of the embeddings of the most relevant words, improving its semantic fit. 6 Initialize the soft label \(y_{soft}\) with one-hot value. 7 Replace the original labels with the soft label \(y_{soft}\): \(y_{soft}=\sigma(y)=\frac{exp(y)}{\sum_{i=1}^{N}exp(y_{i})}\) //Convert the one-hot labels into soft labels for better learning and reduced overfitting. 8 whileupdate distilled samplesdo
9 Initialize the model \(\theta_{0}\). 10 whileupdate modeldo
11 Optimization loss: \(\mathcal{L}_{distill}=L(\tilde{x},y_{soft})+\lambda||\theta_{i}-\theta_{0}||_{2 }^{2}\) //We optimize the model by adjusting the regularization parameter \(\lambda\), which controls the trade-off between preserving the model's original performance and embedding the trigger. The value of \(\lambda\) is determined through cross-validation to balance these effects. A higher \(\lambda\) reduces overfitting, but may diminish the attack effectiveness, while a smaller \(\lambda\) may lead to an overfitted model. 12\(\theta_{i+1}=\theta_{i}-\eta\
abla\mathcal{L}_{distill}(\widehat{X},\theta_{i})\)
13 end while
14\(\mathcal{L}=L(\widehat{X},\theta),\mathcal{L}=L(\widehat{X},\theta)\) //Compute the task-specific loss and regularization loss. 15 ifconvergence criteria metthen
16 Stop the update process. //Stop updating when the loss converges to a threshold or after a fixed number of iterations.