Evaluating the Adversarial Robustness of Retrieval-Based In-Context Learning for Large Language Models

Authors: simon  yu
jie  he
pasquale  minervini
jeff z pan
Venue: colmweb_org_COLM_2024_Conference
Published: 2024-03-23 02:02:03

Abstract: With the emergence of large language models, such as LLaMA and OpenAI GPT-3, In-Context Learning (ICL) gained significant attention due to its effectiveness and efficiency. However, ICL is very sensitive to the choice, order, and verbaliser used to encode the demonstrations in the prompt. \emph{Retrieval-Augmented ICL} methods try to address this problem by leveraging retrievers to extract semantically related examples as demonstrations. While this approach yields more accurate results, its robustness against various types of adversarial attacks, including perturbations on test samples, demonstrations, and retrieved data, remains under-explored. Our study reveals that retrieval-augmented models can enhance robustness against test sample attacks, outperforming vanilla ICL with a 4.87\% reduction in Attack Success Rate (ASR); however, they exhibit overconfidence in the demonstrations, leading to a 2\% increase in ASR for demonstration attacks. Adversarial training can help improve the robustness of ICL methods to adversarial attacks; however, such a training scheme can be too costly in the context of LLMs. As an alternative, we introduce an effective training-free adversarial defence method, \emph{DARD}, which enriches the example pool with those attacked samples. We show that DARD yields improvements in performance and robustness, achieving a 15\% reduction in ASR over the baselines. Code and data are available jointly with this submission as supplementary material.

Main paper: 
 introduction: 
 Large Language models (LLMs) are revolutionising the field of NLP (Brown et al., 2020; Wei et al., 2022; Touvron et al., 2023) . However, this also raises concerns about their robustness (Xu et al., 2023c) . Although significant efforts are put into aligning LLMs for safety purposes, recent works still show LLMs can be vulnerable to adversarial inputs and jailbreaking (Zou et al., 2023; Xu et al., 2023b; Zeng et al., 2024; Longpre et al., 2024) . As a result, it is essential to understand LLM security properties to guide the direction of LLMs that are secure and robust. In this paper, we focus on examining a common LLM inference method, namely In-Context Learning (ICL) and its variants, on how they might be susceptible to perturbations and their robustness against adversarial attacks.ICL is a common few-shot training method for prompting LLM for downstream tasks (Brown et al., 2020) . Its variants include retrieval-based ICL models, which utilise retrievers to identify similar examples as demonstrations (Wang et al., 2023c; Li et al., 2023) ; or kNN-ICL, which compute label demonstrations using a nearest neighbours algorithm; have been adopted and shown promising performance improvements when compared to vanilla ICL (Shi et al., 2022; Xu et al., 2023a) . However, the robustness of these methods against adversarial attacks received less attention and remains unclear. Prior work-which we review in Section 2-shows that LLMs can still be vulnerable to different types of input perturbations, such as task instructions (Zhu et al., 2023; Sun et al., 2023) or inputs (Wang et al., 2023a) ; however, the robustness of ICL-based methods to such perturbations is still under-explored.We address (cf. Section 4) the following question: How sensitive are vanilla and retrievalbased ICL methods to perturbations to their test samples or demonstration instances, which is particularly important given that the primary motivation of ICL is to facilitate few-shot learning without training; should any methods exhibit excessive sensitivity, this would limit their applicability. To investigate this problem, we employ commonly used perturbation attacks-namely character-level, word-level, and sentence-level attacks. Furthermore, we propose new attack methods that target the demonstration instances and datastore examples. Our findings indicate that retrieval-based ICL methods show greater robustness against test-sample attacks; however, they can be more vulnerable when their demonstrations are adversarially perturbed. Detailed ablation studies further reveal that this vulnerability to adversarial attacks persists in newer models, such as Mistral (Jiang et al., 2023 ) and Gemma (Gemma Team et al., 2024) . Many of these attacks can be transferable, both from the same family (LLaMA-2-7B → LLaMA-2-13B, 70B) or different family (LLaMA-2-7B → Mistral-7B). We showed Mixture of Expert (MoE) models still posed a similar adversarial threat to its dense version. This encourages further investigation on enhancing model robustness.We further explore the advantages of utilising retrieval-based models to improve robustness (cf. Section 5), considering that existing defence strategies often rely on an adversarial training process (Li et al., 2021; Si et al., 2021) . However, this process is memory intensive in the context of LLMs. Instead, we show that retrieval-based ICL methods, combined with adversarially augmented examples, yield more robust results than no augmentation methods while maintaining their predictive accuracy on in-distribution evaluation sets. Therefore, we propose DARD, a training-free adversarial defence method which leverages adversarially perturbed samples to augment the retrieval pools, potentially improving the models' robustness. Figure 1 presents an overview of the paper.Our main contributions are summarized as follows.• We first perform a comprehensive and in-depth evaluation on variants of ICL: vanilla, kNN-ICL, and retrieval-based ICL methods against Test-Sample, Demonstration and Datastore Adversarial Attacks. • We show that Retrieval-based models can bring positive results on robustness when test samples are under adversarial attack, but it further reduces the robustness while demonstrations are perturbed. We show these attacks persist on larger models and are transferable. • We propose DARD, a novel training-free adversarial defence method which leverages adversarially perturbed examples to improve the robustness. 
related work: 
 Adversarial attack and defences Adversarial attack has always been a topic in deep neural networks, where previous works showed neural networks can be vulnerable to adversarial examples that are crafted by adding imperceptible but adversarial noise on natural examples (Szegedy et al., 2013; Evtimov et al., 2017) . In the real world, adversarial robustness is important for the application of a method (Tramèr et al., 2017) .Several works studied this problem and the main finding is that LMs struggle to answer correctly and factually under attacks and propose their defence via label smoothing or prompt tuning (Yang et al., 2022; Raman et al., 2023) . It is important to note, however, that these experiments were mainly conducted using relatively small LMs such as BERT, RoBERTa. Recently, several works demonstrated LLM can still be vulnerable to these type of attacks, by showing the vulnerability of attacking the task instructions (PromptBench (Zhu et al., 2023) ), or performing adversarial attacks on zero-shot settings and investigating the transferability of attacks from open-sourced models (i.e. Alpaca (Taori et al., 2023) , Vicuna (Chiang et al., 2023) ) to close-sourced (i.e. GPT-3.5 and GPT-4) (DecodingTrust (Wang et al., 2023a) ). However, previous study (Gu et al., 2022) has shown that the LLMs are less sensitive to prompt / instruction variation when few-shot examples are provided in context (a.k.a ICL). However, there is still a lack of systematic analysis on this case which is the main contribution of this paper. Wang et al. (2023b) shows that attacking demonstrations can still be an effective way for adversarial attacks where LLMs are still vulnerable. Therefore, we aim to extend and evaluate the adversarial attack and defence performance on retrieval-based models.The existing literature on ICL analysis can be broadly divided into two streams, each focusing on different aspects. The first stream explores the influencing factors of ICL based on input perturbation, such as the order (Min et al., 2022) , the formatting (Yoo et al., 2022; Lu et al., 2022) and the selection of the demonstration (Liu et al., 2022) . Designing proper demonstration construction strategies (Ye et al., 2023; Li et al., 2023) could bring clear boosts to the ICL performance, while Chen et al. (2023) analyses the sensitivity of ICL via perturb on instructions and example ordering, and our work is a complement of their work on sensitivity. This paper provides a different view of when different ICL methods fail and when retrieval-based ICL can be a better approach for robustness.Retrieval-based LLM Retrieval-based LLM has shown great success in lifting LLM performance and faithfulness and reducing hallucination (Pan et al., 2023) , where the idea is to gather related articles or demonstrations from a collection of data (Huang et al., 2023) . Specifically, we are interested in whether they can be used to improve the robustness against perturbations (Lin & Lee, 2024) . Meanwhile, many papers also focus on analysing the robustness of RAG methods on irrelevant contexts (Shi et al., 2023; Asai et al., 2024) , in which we adopt similar methods in our context for analysing how retrieval-based ICL being sensitive to irrelevant contexts.We also evaluated another popular word-level method, TextFooler (Jin et al., 2019) , which is similar principle to TextBugger. However, TextFooler greedily searches for a large number of nearby words in the embedding space for each word, as long as they meet some constraints on embedding similarity and sentence quality. Additional constraints require the replacement words to match the Part-Of-Speech (POS) of the original words.3. BERT-Attack: We used the popular character-level attack, BERT-Attack (Li et al., 2020) , which utilizes BERT to generate candidate replacement words based on the importance of words. Then, by calculating the impact of each replacement word on the classification outcome, the replacement word that has the most significant impact on the classification result is chosen as the final replacement. 
background: 
 In this work, we consider several types of ICL methods, namely Vanilla ICL, and two types of retrieval-based ICL: kNN-ICL (Xu et al., 2023a) and Retrieval-based ICL (Wang et al., 2023c; Li et al., 2023) . Retrieval-based ICL methods demonstrated promising performance improvements over vanilla ICL ones; however, their robustness against adversarial perturbations remains unexplored.Vanilla ICL is the most commonly used few-shot learning method in LLM. Given a training data set T = {(x i , y i )}, where x i ∈ X denote the instances and y i ∈ Y represent their labels, the concatenation of the k-shot input-label pairs is used to generate the prompt P with respect to a test instance x test :EQUATIONwhere I denotes the task instructions, π denotes the templates and verbaliser to turn inputs into prompts (details for each dataset are in Appendix A. EQUATIONBy querying the LLM using P i , we obtain a distribution over tokens p(v|P i , θ). Rather than mapping it back to label space Y, the distribution is cached as the key representation for the instance x i :EQUATION) which is mapped to the label y i . The entire datastore thus consists of {(k i , y i )} pairs, where K = {k i } i denotes the set of keys. At inference time, we construct the same prompt and obtain the distribution P test = p(v|P test ; θ LLM ). We then match the distribution against cached K in the datastore, where standard KL-divergence is used to measure the distance:EQUATIONThe predictions are then calculated by interpolating the LLM output and its m nearest neighbours distribution, where NN m ( * , K) denotes the set of m nearest neighbours in K:EQUATION)Following Xu et al. (2023a) , in this work, we use α = 0.2 and m = k/2. R-ICL Retrieval-augmented ICL (Lin et al., 2022) was proposed to mitigate the sensitivity of ICL methods to the choice and order of the in-context examples: rather than randomly sampling (x i , y i ), R-ICL use a retriever to identify the most similar examples to the test instance x test from the training set T . In this paper, we follow Li et al. (2023) and consider the following retrievers: BM25 (Robertson & Zaragoza, 2009) , SBERT (Reimers & Gurevych, 2019) and Instructor (Su et al., 2022) ; more details are available in Appendix A.1.This paper aims to study how perturbations can affect the performance of different ICL methods through adversarial attacks. We classify our 7 attacks into 3 categories: Test Sample Attack, Demonstration Attack, and DataStore Attack.Test Sample Attacks We focus on the three different types of attack: typo-based perturbations -TextBugger (TB) (Li et al., 2018) ; embedding-similarity-based perturbations -TextFooler (TF) (Jin et al., 2019) ; and context-aware perturbations -BERT-Attack (BA) (Li et al., 2020) .Demonstration Attacks For demonstration attacks, we refer to adversarial attacks where the training demonstrations are adversarially perturbed. We consider this type of attack to investigate the sensitivity of LLMs to adversarial perturbations to the training demonstrations and their influence on the generalisation properties of the models.  
settings: 
 Datasets To assess the robustness, we selected six sentiment and multiple-choice text classification datasets: SST2 (Socher et al., 2013) , RTE (Dagan et al., 2005) , CR (Hu & Liu, 2004) , MR (Pang & Lee, 2005) , MNLI-matched (Williams et al., 2018) , and TREC (Voorhees & Tice, 2000) . We provide further details, including prompt templates and dataset statistics, in Appendix A.3. Unless explicitly mentioned, LLaMA-2-7B was mainly used as the base model for our experiments (Touvron et al., 2023) . For experiments involving randomness (i.e., Vanilla ICL and kNN-ICL), we conducted trials with three different random seeds to ensure the reliability of our findings. , where Attack Accuracy denotes the model's accuracy under attack.The 8-shot attack results are presented in Table 1 with 8-shot attacks, and detailed results are available in Appendix B.1.Overview: Retrieval-based models exhibit higher Clean accuracy: 3.27% higher than vanilla ICL with the best performed SBERT retriever; similar finding with Rubin et al. (2021) that related demonstrations enhance the performance. For adversarial attacks, our proposed adversarial attack methods, Swap-Labels and Swap-Labels (Fix), are effective techniques for manipulating LLM predictions. On the other hand, irrelevant context attacks have a negligible effect on performance. We summarise two major findings about the adversarial robustness across different ICL methods.Finding 1: Retrieval-based models are more robust against test-sample attacks; however, it could fall short under demonstration attack. 1 , all three R-ICL methods outperform vanilla ICL and kNN-ICL among all Text Sample Attacks (TB, TF and BA), with a drop in ASR by 4.87% and 2.47%, respectively. 2 In Figure 2 , we observe an increased gap in the ASR between ICL and R-ICL when the number of shots goes up. Therefore, we believe in-context learning benefits from drawing semantically related demonstrations against adversarial perturbations on both performance and robustness. Recall that these textual attacks are designed to deceive LLMs by replacing words with their synonyms or introducing typos in the input sentences. This strategy can be viewed as exploiting LLM's parametric knowledge (Szegedy et al., 2013; Goyal et al., 2022) . In this case, R-ICL can provide supported demonstrations by retrieving related examples to enhance the contextual knowledge to the LLM, therefore reducing the noise of the LLM prediction.Llama-2-7B Llama-2-13B Mistral-7B Vicuna-7B Gemma-7BOn the other side, we see an increase in ASR for Retrieval-based ICL when demonstrations are under attack, demonstrating a 1%-6% increase in ASR compared with vanilla ICL. This demonstrates the deficit of retrieval-based models that can sometimes overly rely on the retrieved demonstrations and are more vulnerable when demonstrations are perturbed.Finding 2: kNN-ICL is highly sensitive to adversarial perturbations on test samples and demonstration.In Table 1 , kNN-ICL shows the highest ASR in Demonstration Attack, specifically showing the highest vulnerability to AdvICL, 40%-50% higher than ICL and R-ICL. Recall that for kNN-ICL, demonstrations and test samples are mapped into the hidden space to compute the label distribution with nearest neighbours. Any perturbations on the demonstration might notably affect their position in the hidden space and alter the label distribution significantly. Also, in Figure 2 , kNN-ICL exhibits a consistently ascending trend in ASR as the number of examples increases, with 15% higher than the other methods. This further illustrates that incorporating nearest neighbours algorithms inevitably includes more noise, thereby increasing vulnerability.Overall, our experiment results show that few-shot methods in LLM could still be vulnerable to adversarial perturbations, leading to a 50% drop in performance on average. Pairing with previous works on jailbreaking leads to the need for a continuous effort to improve the robustness of LLMs. We include quantitative results in the Appendix B.2.In this section, we extend our experiments to four other models: LLaMA-2-13B, larger version of 7B model; Vicuna-7B (Chiang et al., 2023) , instruction-tuned from LLaMA-2-7B; Mistral-7B (Jiang et al., 2023) and Gemma-7B (Gemma Team et al., 2024) , aiming to provide a view for models from different families to prevent potential bias posed within the pretraining data for LLaMA models. These results allowing us to analyze the vulnerability of modern models with better coverage. We conducted the experiments on RTE datasets with 8-shot demonstrations. Even larger models such as LLaMA-2-70B or Mixtral-8×7B (Jiang et al., 2024) are ignored due to attacking them is expensive and slow. Instead, we leave them in the following section for attack transferability ( §4.4).The results are shown in Figure 3 for R-ICL (corresponding results for ICL are presented in Figure 5 ). It can be observed that most adversarial attacks remain effective across all models, including the Swap-Labels attack we proposed. Thus, it still posed a major threat on modern LLMs. Specifically, Vicuna-7B tends to be more sensitive when attack on text sample (TextBugger) or when demonstrations are under attack (AdvICL); while Gemma-7B-most recently released model-shows a higher vulnerability when the context or label in demonstrations is adversarial perturbed, compared to other base models. Notably, the irrelevant context attacks become less vulnerable to more recent release models, especially Vicuna, which shows nearly no effect after being tuned with instructions, showing improved capability in the latest LLMs for filtering OoD contexts (Shi et al., 2023) .EQUATIONIn this section, we would like to answer two questions: Are these attacks transferable across different models, and can larger models mitigate the vulnerability to these attacks? To this end, we conducted two sets of experiments: (1) transfer attacks within the same model family from small models to their large variants, by testing on LLaMA-2 and Mistral famililes;(2) whether attacks still be effective across models from different family, by transferring attacks to Vicuna-7B, Mistral-7B and Gemma-7B, following the same settings as Section 4.3. Our experiments focused on the RTE dataset, chosen for its pronounced drop in robustness when subjected to these attacks. Due to memory constraints, both Mixtral-8×7B and LLaMA-2-70B were run with 8-bit quantization (Frantar et al., 2022) , while the remaining models were in full precision. Hong et al. (2024) tested on AdvGLUE++ and showed that quantized LLMs have negligible drops in adversarial robustness to their source model.The results of the transfer attack with R-ICL are shown in Figure 4 (corresponding results for ICL are presented in Figure 6 ). Most attacks are transferable and achieve an ASR of 15%-40% on similar scale models; except for AdvICL, which cannot transfer between models. Also, Swap-Labels attacks are shown to be less effective when targeting instruction-tuned models (e.g. Vicuna and Mistral-7B-Instruct), pointing to the instruction-tuning process leading models to be less sensitive to its demonstration (Sun et al., 2023) . Specifically, the results shown in Figure 4 (L) confirm that larger models tend to be more robust, as indicated by a reduction of 6% on average in ASR between the 13B and 70B models. However, the MoE variants of Mistral-7B-Instruct exhibit a similar or even higher ASR compared to its dense version, as illustrated in Figure 4 (M). This finding deviates from previous work by Puigcerver et al. (2022) , which investigates adversarial robustness in MoE, showing Vision MoE models are less vulnerable to adversarial attacks. We believe this discrepancy might be subject to the difference between on how MoE is applied in NLP (token) and Vision (image-patch) models. Remember that in the training of MoE models, experts are specialised to handle different tasks (Xue et al., 2024) ; perturbations in the input texts might change the router output and be routed to experts that are less capable for the targeted tasks, leading to more vulnerability and changes in the output. We hope our findings encourage future work to study the robustness of MoE models, both from the training and router perspectives, given their widespread use nowadays. Table 2 : DARD for adversarial defences. We show the Clean Accuracy and the ASR. For Random Addition and Deletion baselines, BM25 is the retriever as it performs the best.We mostly follow the implementations by Wang et al. (2023b) , except for limiting the attack percentage to be 15%. Also, for RTE and MNLI datasets, which consist of both premise and hypothesis, we only allow perturbations on the hypothesis as it's closer to the realistic cases, and the premise is usually much longer than the hypothesis.5. Swap-Labels/Swap-Labels (Fix): For both of the attacks, we reused the Greedy-WordSwapWIR implementation in textattack framework. The attacks start by measuring the importance of each label in the demonstration. It is done by first replacing each of the labels with a meaningless word, fetching it into the LLM to get the label distribution, and computing the difference to the original label distribution.Based on this, the Swap-Labels attack focuses on the most important labels first, y i , by replacing it with other labels from the label set, namely y ′ i ∈ {Y/y i }. If that reduces the LLM output probability on the correct label, the attack models continue to attack the subsequent most important labels with the perturbed examples with y i replaced by y ′ i ; otherwise, it keeps y i and attack another labels. This process iterates until the attack is successful or the maximum perturbations allowed are reached. The only difference between Swap-Labels (Fix) and Swap-Labels is that an additional constraint is imposed: the label distribution on the attacked samples should be the same as the original distribution. Following Min et al. (2022) , CC-news (Hamborg et al., 2017) is used as the corpus providing OoD sentences. Sentence length is considered to select the most relevant sentences to the original demonstration. The contamination rate is 50%. 
defence methods: dard: 
 In this section, we propose an easy yet effective method to enhance the robustness of retrieval-based models against adversarial attacks on test samples. Specifically, we introduce an adversarial defence method named Demonstration Augmentation Retrieval Defences (DARD). This approach performs adversarial augmentation on the training data and mixes them into the retrieval bases for R-ICL. The initiative is that adversarial training in LLM can be costly in memory, and Xie et al. ( 2021); Dai et al. (2022) showed that in-context learning can be seen as an implicit finetuning method.The perturbed examples would typically be used to fine-tune models. However, our approach, DARD, reintegrates these examples into the training demonstration for retrieval during the inference stage. It is important to note that we impose a constraint whereby each example and its perturbed examples can be retrieved no more than once in same prompt for ICL. This is because we observed that most retrieved samples were variants of the same examples with perturbations without this constraint, which reduced both performance and robustness.For baselines, we compared with No Defence, Random Augmentation (Addition, Deletion) inspired by Robey et al. (2023) , which are smoothing methods used to defend against attacks. We also compared with adversarially trained models following vanilla ICL setup. We show the results in Table 2 (detailed results are in Table 19 ). Although the adversarially trained models show a higher clean accuracy as it's directly fine-tuned on RTE data, our methods with SBERT outperform it by 2% reduction in ASR; both BM25 and SBERT variants of DARD have significant improvement on robustness against No Defence baseline. 
summary: 
 While ICL sensitivity is a widely known issue, our paper provides an alternative view of the problem through adversarial attacks. In summary, our paper conducts a comprehensive analysis of the effects of perturbations on modern few-shot methods for LLMs inference and also proposes new adversarial attacks targeted at ICL models. Besides their perfor-mance, the findings show the other side regarding retrieval-based models related to their robustness. Furthermore, we also demonstrate the potential of retrieval-based models in data augmentation strategies for adversarial defense.There are many open questions for future work. One of the most profound findings is that our study on attack transferability demonstrates that larger models generally exhibit better robustness; however, this does not necessarily hold true for Mixture of Experts (MoE) models. Given the recent surge in their popularity, it is crucial to investigate the conditions under which MoE models may fail and to understand the reasons behind their vulnerabilities.Examples of each attack are demonstrated in Table 6 -12. Code will be released to facilitate further research in this field.We conducted all attack experiments using either two A100-40GB GPUs or one A100-80GB GPU. The batch size is 8. For ICL and kNN-ICL that consists of randomness, we conducted experiments with seed {1, 13, 42}. Most attacks took between 1 and 2.5 hours to run. The exception was the 16-shot demonstration attack, which usually lasted 5 hours or longer. We used Flash-Attention 2 (Dao, 2023) to speed up the experiments, which gave us a 2 to 2.5 times speed increase. The speedup significantly helped our comprehensive experiments.We include the complete results breakdown in We conducted a human study to evaluate the feasibility of adversarial attacks and estimate human performance against them. Specifically, we provided 100 input examples to students and asked them to determine the correct label for each given sentence (answer options: true or false). We used the RTE dataset and instructed participants to give their answers solely based on the provided context. We assessed various types of attacks.In this set of 100 sentences, the baseline model we used, R Instructor -ICL , achieved an accuracy of 88% on the original, unaltered sentences. First, we observed that word-level attacks on test examples had a minor impact on human judgment, with an average decrease of only 3% (Table 5 ). Attacks such as advicl and irrelevant attacks resulted in decreases of only 7% and 0%, respectively. However, these attacks could significantly affect retrieval models, causing attack successful accuracies (ASR) ranging from 15% to 95%, indicating that the models are more fragile and sensitive than anticipated.For swap-label and swap-label (fix) attacks, human performance decreased by 25% and 13%, respectively. This suggests that attacks manipulating the label of demonstrations indeed caused confusion for humans, demonstrating their effectiveness.Perturbed Examples 84% 87% 82% 81% 63% 75% 88% The answer is: false two weeks ago, china became the first nation to operate a maglev railway commercially, when officials inaugurated a 30-kilometer-long line between downtown shanghai and the city's airport.The question is: maglev is commercially used., True or False?The answer is: true. the m-2000 uses a commercially manufactured nbti superconductor, similar to that used in the m-2000 maglev magnets., The question is: maglev is commercially used. True or False?The answer is: false it appears that the super-conducting maglev system is technically ready to be used commercially as a very high-speed, large-capacity transportation system. The question is: maglev is commercially used. True or False?The answer is false.Perturbed only a few mag-lev trains have been used commercially such as at the birmingham airport in the uk., The question is: maglev is commercially used. True or False?The answer is: ture. they, too, have not produced a practical, commercially acceptable maglev., The question is: maglev is commercially used. True or False?The answer is: false two weeks ago, china became the first nation to operate a maglev railway commercially, when officials inaugurated a 30-kilometer-long line between downtown shanghai and the city's airport.The question is: maglev is commercially used., True or False?The answer is: true. the m-2000 uses a commercially manufactured nbti superconductor, similar to that used in the m-2000 maglev magnets., The question is: maglev is commercially used. True or False?The answer is: false it appears that the super-conducting maglev system is technically ready to be used commercially as a very high-speed, large-capacity transportation system. The question is: maglev is commercially utilize. True or False?The answer is true. The answer is false. , the death penalty is not a deterrent.The question is: capital punishment is a deterrent to crime. True or False?The answer is false. a closely divided u.s. supreme court said on thursday its 2002 ruling that juries and not judges must impose a death sentence applies only to future cases, a decision that may affect more than 100 death row inmates.The question is: the supreme court decided that only judgescan impose the death sentence. True or False?The answer is false.Perturbed actual statistics about the deterrent value of capital punishment are not available because it is impossible to know who may have been deterred from committing a crime. The question is: capital punishment is a deterrent to crime. True or False?The answer is false. capital punishment is a catalyst for more crime.The question is: capital punishment is a deterrent to crime. True or False?The answer is false. rather than deterring crime, capital punishment actually increases the level of brutality in society.The question is: capital punishment is a deterrent to crime. True or False?The answer is false. , the death penalty is not a deterrent. The question is: capital punishment is a deterrent to crime. True or False?The answer is false. a closely divided u.s. supreme court said on thursday its 2002 ruling that juries and not judges must impose a death sentence applies only to future cases, a decision that may affect more than 100 death row inmates.The question is: the high court decision that only magistrates can impose the death condemnation. True or False?The answer is true. The answer is: true access to the underground workings at the la camorra mine is via a ramp from the surface, excavated at a -15% grade and connecting numerous levels.The question is: la camorra is a mine. True or False?The answer is: true the mine would operate nonstop seven days a week and use tons of cyanide each day to leach the gold from crushed ore.The question is: a weak cyanide solution is poured over it to pull the gold from the rock. True or False?The answer is: false the dam covering what used to be its football pitch fill up with millions of tons of poisonous mining waste.The question is: mine waste-water poses an environmental hazard. True or False?The answer is: false known as "heap leach" mining, the method has become popular in the last decade because it enables microscopic bits of gold to be economically extracted from low-grade ore.The question is: the mining industry uses a method known as heap leaching. True or False?The answer is true.Perturbed gold mining operations in california and nevada use cyanide to extract the precious metal.The question is: cyanide is used in gold mining. True or False?The answer is: true access to the underground workings at the la camorra mine is via a ramp from the surface, excavated at a -15% grade and connecting numerous levels.The question is: la camorra is a mine. True or False?The answer is: true the mine would operate nonstop seven days a week and use tons of cyanide each day to leach the gold from crushed ore.The question is: a weak cyanide solution is poured over it to pull the gold from the rock. True or False?The answer is: false the dam covering what used to be its football pitch fill up with millions of tons of poisonous mining waste.The question is: mine waste-water poses an environmental hazard. True or False?The answer is: false known as "heap leach" mining, the method has become popular in the last decade because it enables microscopic bits of gold to be economically extracted from low-grade ore.The question is: the mining industry uses a method known as heap leachmanufacturing. True or False?The answer is false. in other words, with its 2 million inhabitants, slovenia has only 5.5 thousand professional soldiers.The question is: slovenia has 5.5 million inhabitants. True or False?The answer is:false nicholas burns stated in an exclusive interview for "utrinski vesnik" from pristina that it would be disgraceful if athens puts a veto on macedonia's application for membership in the european union and nato.The question is: greece and macedonia are in dispute over name. True or False?The answer is: false greece objects to the neighbouring skopje government using the name macedonia, saying it implies claims on a greek province of the same name.The question is: greece and macedonia are in dispute over name. True or False?The answer is: true in other words, with its 2 million inhabitants, slovenia has only 5.5 thousand professional soldiers.The question is: slovenia has 2 million inhabitants. True or False?The answer is: true the croatian intent is even more problematic because the border between slovenian and croatian territorial waters has not yet been established. the dispute about this border began in 1991 when both countries became independent.The question is: there is a territorial waters dispute. True or False?The answer is true.Perturbed in other words, with its 2 million inhabitants, slovenia has only 5.5 thousand professional soldiers.The question is: slovenia has 5.5 billion inhabitants. True or False?The answer is:false nicholas burns stated in an exclusive interview for "utrinski vesnik" from pristina that it would be disgraceful if athens puts a veto on macedonia's application for membership in the european union and nato.The question is: greece and macedonia are in dispute over name. True or False?The answer is: false greece objects to the neighbouring skopje government using the name macedonia, saying it implies claims on a greek province of the same name.The question is: greece and macedonia are in dispute over name. True or False?The answer is: true in other words, with its 2 million inhabitants, slovenia has only 5.5 thousand professional soldiers.The question is: slovenia has 2 million inhabitants. True or False?The answer is: true the croatian intent is even more problematic because the border between slovenian and croatian territorial waters has not yet been established. the dispute about this border began in 1991 when both countries became independent.The question is: there is a territorial waters dispute. True or False?The answer is false. The answer is: true a farmer who was in contact with cows suffering from bse -the so-called mad cow disease -has died from what is regarded as the human form of the disease.The question is: bovine spongiform encephalopathy is another name for the "mad cow disease". True or False?The answer is true.The further into summer we go, the more California's winter swells seem like a distant memory. Do you remember? Tons of rain, a few massive days up and down the coast, and more rain. ... The latter half's roping barrels will make you want to throw on your booties, crank the heat, and grab your step-up.The question is: A motorcyclist struck a cow and suffered fatal injuries in Pierce County on Saturday morning, authorities said. True or False?The answer is: true most of the mines are in arid areas and animals searching for water are attracted to the cyanide-laced holding ponds that are an integral part of the mining operations. The question is: animals have died by the thousands from drinking at cyanide-laced holding ponds. True or False?The answer is: false the unconfirmed case in dundee concerns a rabies-like virus known only in bats.The question is: a case of rabies was confirmed. True or False?The answer is: false this case of rabies in western newfoundland is the first case confirmed on the island since 1989. The question is: a case of rabies was confirmed. True or False?The answer is: true. a farmer who was in contact with cows suffering from bse -the so-called mad cow disease -has died from what is regarded as the human form of the disease.The question is: bovine spongiform encephalopathy is another name for the "mad cow disease". True or False?The answer is false. Table 18 : Complete results for the TREC dataset. The term Clean refers to the benign accuracy before attacks; attack accuracy is provided under each attack column.Here we compare with the best performed R Instructor -ICL with ICL and kNN-ICL.hkunlp/instructor-large 
limitations: 
 We view our work as an early exploration of the adversarial robustness of Retrieval-based in-context learning. Here, we only show how different types of attack can be applied to, but not why. Given the standard use of retrieval and in-context learning as inference methods, it is crucial to understand its deficiency. Future work should focus on certified robustness (Zhao et al., 2022; Xiang et al., 2024) , which analyzes with theoretical proof the extent to which the model is unaffected by adversarial attacks. The Answer is: falseFor women earning 22,000 a year, the total pay accumulated after six months maternity leave would be just 5,300 in the UK and 5,850 in Ireland. Entitlements in Germany would also be relatively low, at 5,900, along with those in France, Spain and the Netherlands, all at 6,750. At the other end of the scale, pay received after six months leave in Italy would be 9,150 while in Denmark and Norway it would be as much as 11,000.The question is Maternity leave varies in Europe. True or False?The Answer is:Review: "you might say tykwer has done all that heaven allows , if you wanted to make as anti-kieslowski a pun as possible . suffice to say its total promise is left slightly unfulfilled ." 0: negative, 1: positive Sentiment: negative Review: an alternately raucous and sappy ethnic sitcom . . . you'd be wise to send your regrets . Sentiment:Review: it 's not as stylized as a sony or samsung . We implemented our proposed attack methods and conducted all our experiments using the TextAttack framework (Morris et al., 2020) , which is model-agnostic and can be easily tested on any model. Here are the details regarding to each attack method:1. TextBugger: We employ the TextBugger (Li et al., 2018) , which determines the importance of words and replaces key words to conduct the attack. We use GloVe embeddings to retrieve nearby words of the words to be replaced for the attack. 
 
 Appendix:  


Decision: The paper introduces DARD, a training-free adversarial defense method designed to enhance the robustness of retrieval-based in-context learning (ICL) methods for large language models (LLMs) against various adversarial attacks. The authors conduct a comprehensive evaluation of different ICL methods—vanilla, kNN-ICL, and retrieval-based ICL—against test-sample, demonstration, and datastore adversarial attacks. Their findings highlight the vulnerabilities and strengths of these methods, with DARD showing a significant improvement in reducing the Attack Success Rate by 15% over baseline methods.

The paper is well-written and well-organized, making it easy to follow the authors' arguments and understand their experimental setup. The inclusion of detailed figures and tables aids in the clear presentation of results. The comprehensive literature review and background information further strengthen the paper's foundation. Overall, the paper provides valuable empirical findings and a useful defense mechanism for enhancing the robustness of ICL methods in LLMs.

However, there are several areas where the paper could be improved. The scope of the study is restricted to language modality, and it would be beneficial to explore the robustness of LLMs more broadly. Additionally, key concepts and dataset statistics could be more prominently discussed in the main text rather than being relegated to appendices. The paper would also benefit from a clearer definition of the attacks and an improved explanation of the motivation and significance of studying ICL robustness specifically.

Reviews: review: paper_summary: paper_summary: The authors introduce an effective training free adversarial defence method, DARD, which enriches the example pool with those attacked samples. It shows that DARD yields improvements in performance and robustness, achieving a 15% reduction in Attack Success Rate  over the baselines.
strength_weakness: strength_weakness: 1. The paper is more like of an empirical study of ICL methods under attacks. The novelty is limited. And the authors had better put the method section before experiment section.
2. The explored datasets are good. But it would be better to explore some real problem in the era of LLM, such as "Jailbreaking Black Box Large Language Models in Twenty Queries".
paper_summary: paper_summary: The authors conduct a comprehensive evaluation of the robustness of different ICL approaches, i.e., vanilla, kNN-ICL, and retrieval-based ICL, against Test-Sample, Demonstration, and Datastore Adversarial Attacks. Their findings reveal that retrieval-based ICL enhances robustness against adversarial attacks on test samples; however, their robustness diminishes when demonstrations are perturbed. Additionally, the authors proposed DARD,  a novel training-free adversarial defense method aimed at improving the robustness of retrieval-based ICL against such attacks.
strength_weakness: strength_weakness: There are some minor issues in the current manuscript. 
1. It would have been beneficial to include certain key concepts in the main text, such as the introduction of the attacks.
2. The title "Ablation Study" in section 4.3 does not appropriately reflect the concepts discussed.
3. The texts in Figure 1 are too small to read. 
4. The authors should have included the statistics of the datasets they tested.
questions: questions: 1. What does ``involve the aggregation of results from retrievers" in the caption of Figure 3 mean? 
2. Besides these word-level attacks, have you considered some sentence-level attacks, such as paraphrasing?
paper_summary: paper_summary: This paper focuses on evaluating the adversarial robustness of retrieval-based in-context learning (ICL) methods for large language models (LLMs). The authors conduct a comprehensive analysis of the vulnerability of different ICL methods to various types of adversarial attacks, including test-sample attacks, demonstration attacks, and datastore attacks. They also propose a training-free adversarial defense method called DARD, which enriches the example pool with adversarially attacked samples. The experimental results show that retrieval-based models can enhance robustness against test-sample attacks but are more vulnerable to demonstration attacks. DARD could improve the robustness of the models, achieving a reduction in attack success rate (ASR) over the baselines.
strength_weakness: strength_weakness: * This paper only studies the robustness of retrieval-based ICL in LLMs, and only focuses on the language modality. The scope of the topic may be limited. Moreover, does the robustness of LLMs significantly affect that of ICL? Why not study the robustness of LLMs instead? Is it possible that adversarially (and efficiently) fine-tuning LLMs (e.g., by LoRA) could yield better results? The authors should further highlight the motivation and significance of the studied topic.
* This is a pure empirical paper. It would be good to explore the theoretical part (e.g., certified robustness) in future work.
* No source code is provided at the time of submission.
* Minor: It seems that the font type is not correct.
questions: questions: Please see above.

Comments: title: Near the End of the Discussion Period
Near the End of the Discussion Period
Author response to Reviewer YFEZ
Thanks for your rebuttal
comment: As the discussion period deadline approaches, we would like to ask whether our response addressed the concerns raised by the reviewer in the “Reasons to Reject” and question sections. We would be happy to provide any further clarifications.

Once again, we thank the reviewers for their time in reviewing our work and for providing valuable suggestions.

Best regards, 

Authors
As the discussion period deadline approaches, we would like to ask whether our response addressed the concerns raised by the reviewer in the “Reasons to Reject” and question sections. We would be happy to provide any further clarifications.

Once again, we thank the reviewers for their time in reviewing our work and for providing valuable suggestions.

Best regards, 

Authors
Dear Reviewer,

Thank you very much for dedicating your time to review our response. If you found that our reply has satisfactorily addressed your concerns, we would be grateful if you could consider increasing your score. Your support would be greatly appreciated.

Best regards,

The Authors
The authors solve my concerns, and I will keep my positive score.