# Neighborhood Deviation Attack Against In-Context Learning
## 5 Experiments  
### Experimental Basis  
**Datasets:** The experiment utilizes three datasets: DBPedia [44], AGNews [44], and TREC [45]. The DBPedia dataset, extracted from Wikipedia, contains 14 main categories covering domains such as persons, locations, and organizations, providing rich samples for text classification tasks. The AGNews dataset, composed of global news articles, includes four major categories (World, Sports, Business, and Sci/Tech) and is suitable for news classification and related NLP tasks. The TREC dataset features a large volume of text data with high-quality annotations, covering tasks such as information retrieval, question answering, and text classification. It is used to evaluate text retrieval system performance and supports academic research. These datasets collectively offer diverse and high-quality samples for training and validation in the experiment.  
**Language Models:** The experiment employed three LLMs: LLaMA3-8B [46], LLaMA2-7B [35], and GPT2-XL [47]. LLaMA3-8B stands out with its 8B parameter scale and superior multilingual processing capabilities, making it ideal for complex language generation tasks. LLaMA2-7B, with its 7B parameter scale and optimized training strategies, demonstrates efficient language understanding and dialogue generation. GPT2-XL, with its 1.5B trainable parameters, is noted for its strong text generation coherence and broad applicability. These models, varying in size and performance, are suitable for a range of NLP tasks and provide robust capabilities for language generation and understanding in the experiment.  
**Neighbor Prefix Generation Model:** When generating the neighbor prefixes of the samples to be judged, we use the BERT model [48]. With its powerful context encoding ability, the BERT model can generate similar prefixes for each sample, ensuring both semantic similarity and contextual consistency. Compared to other solutions, such as Con2 vec [49], T5 [50], and GPT, BERT maintains a balance between efficiency, contextual  
Figure 4: Example workflow for our method.  
synonym generation, and computational cost, making it a suitable choice for our approach. The key hyperparameters are set as follows: the dropout rate, set at 30%, which controls the probability of zeroing dimensions during embedding perturbation; the prefix proportion, set at 20%, which determines the size of the prefix relative to the entire target sample; the number of neighbors generated, which is typically five per target sample; and the number of replacements, which refers to the number of masked words substituted per target sample and is set to one.  
**Experimental Setup:** All experiments were conducted on a workstation equipped with two NVIDIA GeForce RTX 4080 GPUs and an AMD Ryzen 9 7950X processor with 128 GB RAM. The implementation was based on Python 3.10. Each experiment ran for 500 iterations, requiring approximately 20 GB of GPU memory.  
In each iteration, one member sample and one non-member sample were drawn for the attack, and the average performance across iterations was taken as the final result. This experimental design has been used in previous studies [11]. At the beginning of the experiment, 1000 samples were drawn from the dataset and divided into an example set and a test set. In each iteration, one sample was drawn from the example set and added to the context examples, while also being copied as the member sample. Another sample was drawn from the test set to serve as the non-member sample. To ensure consistency across multiple runs, a fixed random seed was used.