# Neighborhood Deviation Attack Against In-Context Learning
## 5 Experiments
### Results  
The main objective of this experiment is to evaluate the vulnerability of the ICL of different language models across various datasets, with a specific focus on examining and comparing the effectiveness of distinct attack methods (Neighbor Deviation Attack, Repeat Attack, and GAP Attack). Our attack method is implemented on three datasets (AGNews, DBPedia, and TREC) and three models (LLaMA3, LLaMA2, and GPT2-XL). Through a comparative analysis of attack method performance across different models and datasets, this study aims to provide deeper insights into the applicability of MIA in ICL scenarios. The experimental results reveal the performance changes of our attack method in different scenarios, which will provide a basis for thinking about how to reduce the privacy risks of ICL. The experimental results are illustrated in Table 2.  
In Table 2, we not only show the performance of the proposed method but also compare it with the comparative methods to highlight the advancement of the proposed method over existing methods. According to the results, Neighbor Deviation Attack demonstrates superior attack performance, maintaining advantages above 0.8 across all datasets while reaching peak values of up to 0.91 in specific cases. Specifically, on the TREC dataset, the Neighbor Deviation Attack has the highest advantages for the LLaMA3 and LLaMA2 models. In contrast, our method performs better than the other two attack approaches in most scenarios, and it maintains comparable attack performance across any dataset-model combination, exhibiting exceptional stability and stability. Repeat Attack ranks second, occasionally slightly surpassing the Neighbor Deviation Attack in isolated cases, but its overall stability remains inferior. In the worst case, the performance of the Repeat Attack is only two-thirds of that of Neighbor Deviation Attack. GAP Attack exhibits the lowest advantage, showing limited performance across all datasets, which indicates its ineffectiveness in executing attack tasks under the current experimental settings. The experimental results show that Neighbor Deviation Attack has the best attack performance, which can be attributed to the fact that it makes more effective use of the feature information generated by the model through multiple similar neighbor prefixes to attack the ICL examples more accurately. Furthermore, Neighbor Deviation Attack has good stability, which can also be attributed to the fact that multiple neighbor prefixes and the original prefixes provide richer feature information for the final judgment as a way to obtain a more stable final result. Repeat Attack has slightly worse performance. This method achieves good results by leveraging the prediction and completion capabilities of LLMs, but it is not as stable as Neighbor Deviation Attack. In contrast, GAP Attack has the lowest performance, probably because it directly requires the model to classify samples. When the model is powerful, it may not rely too much on context examples to complete the task directly. In addition, from a model perspective, Neighbor Deviation Attack appears to perform best on LLaMA3 and LLaMA2. This may be because the LLaMA series of models are trained using a larger scale of data compared to GPT2, which makes the models' generative behavior more likely to expose the characteristics of their training data. Therefore, we can hypothesize that as the capability of LLMs increases, the privacy risk in in-context learning increases accordingly. The results also reveal some challenges. For example, the performance of the attack method varies a little on different datasets, which indicates that the distribution characteristics of the dataset have a greater impact on the attack. However, due to the black-box condition, we cannot obtain prior knowledge such as the distribution of the target data. In addition, although Neighbor Deviation Attack performs well in most cases, its performance may still be affected by factors such as the specific implementation of the language model.