# Neighborhood Deviation Attack Against In-Context Learning
## 6 Potential Defenses  
The Neighbor Deviation Attack proposed in this paper can be used to conduct MIA on LLM in the ICL scenarios. This attack can determine whether specific data are used as an ICL example. In response to such privacy risks, we can adopt targeted defense strategies that focus on reducing LLM memory for ICL examples and limiting output deviation.  
To reduce LLM memory and preference for ICL examples, the most direct approach is to increase the number of ICL examples used in a single inference [32]. When an LLM loads multiple ICL examples in a single inference, the model's attention will be dispersed, thereby reducing the dependence on specific examples. However, the adaptability of ICL to downstream tasks still exists for LLMs. In other words, after increasing the number of ICL examples, the LLM is more inclined to learn the overall patterns of multiple examples rather than memorizing certain specific examples. Thus, this approach ensures the quality of the LLM inference while controlling the corresponding privacy risks. Furthermore, the study conducted by [PERSON] et al. [56] indicates that when dealing with a long context, the LLM pays more attention to the beginning and ending parts of the long context rather than the middle part of the long context. Therefore, when using ICL examples, we can place more valuable or more protected ICL examples in the middle part of the loaded multiple examples, which will help mitigate the risk of privacy leakage.  
In order to impose limits on the metric deviation of the LLM output, we can introduce controlled randomness in the reasoning phase of the LLM. When generating the output, we can use generation strategies such as beam search [57], Top-k [58], and Top-p [59]. These measures will introduce a certain amount of noise, causing the metric bias to fluctuate easily, affecting the final judgment result but will also affect the quality of the output result to a certain extent. In addition, a direct filtering approach can be employed, utilizing the existing LLM as a filtering layer to modify the output while preserving its original meaning. It can be foreseen that such a defense measure will be very effective against neighbor bias attacks. However, using additional LLMs to filter the output will consume additional computing resources and may hurt the quality of the output.  
Figure 10: Comparison between single deviation threshold judgment and multi-dimensional metric deviation threshold judgment. The performance is measured in terms of Advantage. (**a**) The attack performance of single metric deviation judgment and Neighbor Deviation Attack across three datasets on LLaMA3. (**b**) The attack performance across the same datasets on LLaMA2. (**c**) The attack performance across the same datasets on GPT2-XL.  
While protecting privacy and security, the above defense measures may have a certain impact on LLM and output quality, or cause additional computing resource consumption. Consequently, in practical applications, we will weigh the relationship between privacy protection and model performance and resource consumption, and choose the most suitable defense solution.