# Neighborhood Deviation Attack Against In-Context Learning
## 5 Experiments
### Ablation Experiments
#### 5.5.3 Number of Replacement  
The number of replacements is another crucial hyperparameter in Neighbor Deviation Attack, as it determines how many words in the prefix will be replaced by synonyms when generating the neighborhood prefix. This parameter directly affects the diversity and semantic consistency of the generated neighborhood prefixes. By changing the number of replacements, we aim to study how the degree of prefix perturbation affects the effectiveness of the attack. Specifically, we hope to find the optimal number of replacements that can maximize the attack advantage while generating valid neighbor prefixes.  
The experimental results are shown in Table 3, which demonstrates the influence of the number of replacements on the attack performance when using the LLaMA2 and LLaMA3 models on the AGNews and DBPedia datasets. When the number of replacements is 1, in most cases, the attack achieves the highest advantage value. At this time, the attack advantage values for LLaMA2 and LLaMA3 on the AGNews dataset are 0.87 and 0.82, respectively, and on the DBPedia dataset are 0.85 and 0.88, respectively. When the number of replacements increases to 2, most of the advantage values slightly decrease, and only the attack advantage value for LLaMA3 on the AGNews dataset slightly increases. Further increasing the number of replacements to 3 leads to a more significant performance decline. The advantage values for LLaMA2 and LLaMA3 on the AGNews dataset are 0.71 and 0.86, respectively, and on the DBPedia dataset are 0.72 and 0.84, respectively, which are the lowest values in all cases. These results indicate that the more replacements there are, the more likely the effectiveness of the attack will decrease, especially on the AGNews dataset.  
Figure 6: Influence of prefix proportion across two datasets and two language models. The prefix proportion varies from 10% to 50%. (**a**) The impact of prefix proportion on the method Advantage for the LLaMA2 and LLaMA3 models on the AGNews dataset. (**b**) The impact of prefix proportion on the method Advantage for the LLaMA2 and LLaMA3 models on the DBPedia dataset.  
The decline in attack performance as the number of replacements increases may be due to the following reasons. Only a single replacement in the sample introduces controllable variation, allowing the model to generate completions that are semantically consistent with the original prefix while still providing a certain degree of difference to assist judgment. Such a neighbor sample is crucial to the accuracy of the attack because it ensures that the generated completions can fully function during the final membership inference. However, as the number of replacements increases, the semantic consistency of the prefix may be compromised, resulting in a decrease in the consistency between the generated completions and the original context. Arguably, this introduces more noise. This noise makes it more difficult for the model to generate accurate completions, thereby weakening the effectiveness of the attack. In addition, the AGNews dataset consists of news articles and may be more sensitive to semantic perturbations than the DBPedia dataset, which contains more structured factual information. This sensitivity may explain why the performance of the AGNews dataset degrades more significantly as the number of replacements increases. In summary, we determine that the optimal number of replacements for the neighborhood bias attack is 1, as it strikes the best balance between introducing enough variation and maintaining semantic consistency, while also ensuring that no computational resources are wasted, thereby maximizing the benefits of the attack.