# Neighborhood Deviation Attack Against In-Context Learning
## 5 Experiments
### Ablation Experiments
#### 5.5.2 Prefix Proportion  
The prefix proportion is also a key hyperparameter in our proposed Neighbor Deviation Attack, as it determines the size of the prefix relative to the entire target sample. As the prefix serves as the input for the prediction completion stage, its length directly affects the model's ability to generate meaningful completions. By altering the prefix proportion, we aim to investigate how the prefix length influences the performance of the attack. Specifically, we seek to find the optimal prefix proportion that can maximize the attack advantage while maintaining the semantic consistency of the generated completions.  
According to the experimental results shown in Figure 6, on the AGNews dataset, both LLaMA2 and LLaMA3 witnessed a continuous decline in attack performance as the proportion of prefixes increased. At a prefix length of 10%, the advantage value reached approximately 0.90, and when the prefix length increased to 20%, the advantage value decreased slightly. Subsequently, as the proportion of prefixes continued to increase, the advantage value showed a significant decline. At a prefix length of 50%, the advantage value was only around 0.4. On the DBPedia dataset, the advantage value did not show any significant change when the prefix length was 10% or 20%. However, as the prefix length continued to increase, the advantage value still showed a significant decline, but the trend of decline in LLaMA3 was significantly less severe than that in the AGNews dataset.  
The attack effect decreases with the increase in the prefix ratio, and the specific reasons can be attributed to several aspects. Shorter prefixes cause the model to rely more on contextual learning samples when generating, making the attack more effective. On the contrary, longer prefixes introduce more context, weaken the effect of diversifying contextual learning samples, and reduce the difference in metric deviation between member samples and non-member samples. In addition, semantic changes may happen in  
Figure 5: Influence of the number of neighbors across two datasets and two language models. The number of neighbors varies from 1 to 15. (**a**) The impact of the number of neighbors on the method advantages for the LLaMA2 and LLaMA3 models on the AGNews dataset. (**b**) The impact of the number of neighbors on the method advantages for the LLaMA2 and LLaMA3 models on the DBPedia dataset.  
completed samples and introduce noise that affects final membership inference because longer prefixes increase the complexity of text generation. The performance decline trends of LLaMA3 on the AGNews and DBPedia datasets are different, indicating that dataset characteristics (such as text complexity and diversity) are crucial to determining the optimal prefix length. The stronger the performance of LLM, the more significant the impact of the dataset characteristics on the prefix length. Due to the different sample lengths of the different datasets, the impact of different prefix ratios on attack performance is also different. Taking into account factors such as attack performance, dataset characteristics, and computing resources, we determine that the optimal prefix ratio is 20%.