# Neighborhood Deviation Attack Against In-Context Learning
[PERSON]  
1 State Grid Hubel Information & Telecommunication Company, Wuhan 430048, China;  
[EMAIL_ADDRESS] (D.H.); [EMAIL_ADDRESS] (L.Z. ); [EMAIL_ADDRESS] (B.J. ); [EMAIL_ADDRESS] (H.X. ); [EMAIL_ADDRESS] (Y.L.) Huazhong University of Science and Technology, Wuhan 430074, China; [EMAIL_ADDRESS] (Z.Y. ); [EMAIL_ADDRESS] (Z.Y. ); [EMAIL_ADDRESS] (Z.Y.) [PERSON]  
2 Hubei Key Laboratory of Smart Internet Technology, School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China; [EMAIL_ADDRESS] (Z.Y. ); [EMAIL_ADDRESS] (B.J.) [PERSON]  
1 State Grid Hubel Information & Telecommunication Company, Wuhan 430048, China;  
[EMAIL_ADDRESS] (D.H.); [EMAIL_ADDRESS] (L.Z. ); [EMAIL_ADDRESS] (B.J. ); [EMAIL_ADDRESS] (H.X. ); [EMAIL_ADDRESS] (Y.L.) Huazhong University of Science and Technology, Wuhan 430074, China; [EMAIL_ADDRESS] (Z.Y. ); [EMAIL_ADDRESS] (Z.Y.) [PERSON]  
2 Hubei Key Laboratory of Smart Internet Technology, School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China; [EMAIL_ADDRESS] (Z.Y. ); [EMAIL_ADDRESS] (B.J.) [PERSON]  
1 State Grid Hubel Information & Telecommunication Company, Wuhan 430048, China;  
[EMAIL_ADDRESS] (D.H.); [EMAIL_ADDRESS] (L.Z. ); [EMAIL_ADDRESS] (B.J. ); [EMAIL_ADDRESS] (H.X. ); [EMAIL_ADDRESS] (Y.L.) Huazhong University of Science and Technology, Wuhan 430074, China; [EMAIL_ADDRESS] (Z.Y. ); [EMAIL_ADDRESS] (Z.Y.) [PERSON]  
2 Hubei Key Laboratory of Smart Internet Technology, School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China; [EMAIL_ADDRESS] (Z.Y. ); [EMAIL_ADDRESS] (Z.Y.) [PERSON]  
2 Hubei Key Laboratory of Smart Internet Technology, School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China; [EMAIL_ADDRESS] (Z.Y. ); [EMAIL_ADDRESS] (Z.Y. ); [EMAIL_ADDRESS] (Z.Y.) [PERSON]  
1 State Grid Hubel Information & Telecommunication Company, Wuhan 430048, China;  
[EMAIL_ADDRESS] (D.H.); [EMAIL_ADDRESS] (L.Z. ); [EMAIL_ADDRESS] (B.J. ); [EMAIL_ADDRESS] (H.X. ); [EMAIL_ADDRESS] (Y.L.) Huazhong University of Science and Technology, Wuhan 430074, China; [EMAIL_ADDRESS] (Z.Y. ); [EMAIL_ADDRESS] (Z.  
financial sector, attackers can use MIA to attack models trained on transaction data to infer whether the training used the financial records of a specific individual [14]. Existing studies on privacy risks for LLMs mainly focus on gradient leakage, membership inference attacks, and the leakage of personally identifiable information (PII) due to unintended data memorization [15]. Among them, MIA methods against the training data of LLM are emerging one after another [16; 17; 18], however, MIA methods targeting ICL in LLMs remain underexplored and imprecise, highlighting the need for more effective approaches. In our work, we bridge the gap between existing MIA methods and in-context learning and present a text-only MIA approach specifically designed for the ICL of LLMs. By examining how LLMs process and respond to in-context learning examples, our objective is to uncover the privacy risks associated with their ICL capabilities. We extract prefixes from the target sample and its neighboring samples and input them into the LLM loaded with the ICL sample for generation. The output results are then analyzed to determine whether the target sample is part of the ICL sample. Our approach relies only on the model's output, enhancing its practicality and portability. Extensive experiments verify the effectiveness of our approach in judging whether a specific sample belongs to the ICL example used in LLM reasoning, thus highlighting the privacy risks of ICL in practical applications. We summarize our contribution as follows:  
* We propose a text-only MIA method against ICL, revealing its inherent privacy risks. Our method combines neighbor sample generation and MIA techniques to achieve higher stability and accuracy.