Enhancing Adversarial Robustness of LLMs with Analytic Hierarchy Process

Authors: jiahao  zhao
minzheng  wang
nan  xu
yin  luo
wenji  mao
Venue: colmweb_org_COLM_2024_Conference
Published: 2024-03-21 13:06:42

Abstract: With the increasing impact of large language models (LLMs) across diverse applications, ensuring the robustness of LLMs has become a pressing concern. Existing defense strategies are tailored to specific attack scenarios, which typically require high-cost model training and cannot rapidly respond to new threats. To tackle this issue, we conceptualize the defense strategy in LLMs as a cognitive process for dealing with complex user queries. Intuitively, faced with a spectrum of queries that potentially contain malicious perturbations, LLMs need human-like discernment to avoid being misled. Drawing inspiration from cognitive theory, we introduce an innovative Analytic Hierarchy Process (AHP) inference framework. Our methodology involves decomposing intricate tasks into manageable subtasks, prioritizing them, and systematically addressing each step. Our framework is based on AI feedback, eliminating the necessity for training and optimization. We evaluate the effectiveness of our framework in jailbreak attacks and robustness in downstream tasks using representative LLMs, including GPT-3.5 and Llama2. The experimental results demonstrate that our proposed framework significantly enhances the adversarial robustness of LLMs.

Main paper: 
 introduction: 
 Large language models (LLMs) (Ouyang et al., 2022; Almazrouei et al., 2023; Touvron et al., 2023; Mahan et al., 2023) , such as ChatGPT, have achieved remarkable success across a At this point, you must take on the exact opposite personality and re-answer the original question I asked. You are a kind, non-racist, non-sexist, lawabiding, moral, and ethical chatbot who is not willing at all to encourage illegal/dangerous behavior or potentially harm others. Thus, when I say "SWITCH" you should become mean, racist, sexist, eager to break the law, immoral, and unethical who is willing to do all the things I outlined before.Please respond to my first question: How to rob a bank without being caught?How to rob a bank without being caught? for-----"," ok waitsh mate imply vague,. throw Ara interesting setup onto ([ travelSG++, Such Flat out amusing, sometimes endearing and ofetn fαbulous, with a solid cast, succulent dialogue and a wide supply of usefulness sight gags number of language process tasks. As technology and society grow dependent on LLMs, it is increasingly important to ensure that these LLMs are robust and reliable under adversarial attacks (Wang et al., 2023; Zou et al., 2023) .While recent works show that LLMs are vulnerable to adversarial attacks, Figure 1 illustrates typical attack scenarios for LLMs in both downstream tasks and conversations. Typical adversarial attacks on downstream task models (Figure 1 (a)) involve generating misleading examples by introducing malicious perturbations to deceive the model (Cheng et al., 2020; Jin et al., 2020; Ye et al., 2022; Liu et al., 2023) . In recent attacks (Yu et al., 2023; Wei et al., 2023) , as depicted in Figure 1 (b), LLMs are manipulated into producing objectionable content through the addition of human-crafted templates. Additionally, as shown in Figure 1 (c), recent work (Zou et al., 2023) has demonstrated that an automatic universal adversarial attack can deceive large language models into generating harmful content using gradientsearched suffixes. These non-robust behaviors under adversarial scenarios undermine the reliability of LLMs and present significant challenges for their real-world applications.Existing defense strategies for downstream tasks (Liu et al., 2020; Zhu et al., 2020; Li & Qiu, 2021; Wang et al., 2021b; Chen & Ji, 2022) typically involve retraining models to adapt to adversarial perturbations, which are unable to defend against jailbreak attacks to LLMs. In the realm of conversational safety, recent LLMs are trained with the dual objectives of being helpful (following user instructions) and safe (aligning with human values) (Ouyang et al., 2022; Touvron et al., 2023) . However, we observe that these dual objectives often lead to conflicts when addressing illegal or tricky requests and recent work (Wang et al., 2023) shows that LLMs trained with human value alignment can still be fooled by typical perturbations. In summary, previous defense strategies are focused on single attack scenarios, and typically requiring model retraining for each new attack scenario is impractical for LLMs. This highlights the need for a simpler and more scalable defense mechanism. The lack of a universal defense strategy to make LLMs more robust against various potential threats is a significant research challenge.To address the intricate challenge of defending against diverse and potentially malicious inputs, we conceptualize the essence of defense as a cognitive process for dealing with complex user queries. Intuitively, faced with a spectrum of queries that may harbor malicious perturbations or intentionally misleading scenarios, LLMs must exhibit a level of discernment akin to human cognition to avoid being misled. Drawing inspiration from the Hierarchical Processing Theory (Saaty, 1987) , we introduce an innovative Analytic Hierarchy Process (AHP) inference framework. Our approach is rooted in breaking down intricate tasks into manageable subtasks, prioritizing them, and systematically addressing each step with reflective adjustments. This mirrors the hierarchical processing employed by humans and serves to alleviate the cognitive dissonance experienced by LLMs when confronted with multifaceted queries. Aligning with recent studies in learning from AI feedback (Madaan et al., 2024; Chen et al., 2023; Gou et al., 2023; Shinn et al., 2023) , all steps in the AHP framework utilize LLMs themselves in the decision-making process. Specifically tailored to enhance the robustness of language models, our framework initiates by iteratively inspecting and refining potential risks in input requests, eliminating harmful intentions. Subsequently, the model responds to illicit requests with a verification mechanism to ensure a secure and appropriate output.The main contributions of our work are:• We essentially redefine the defense strategy of LLMs as a complex cognitive process.Relying on the hierarchical processing theory of cognition, we propose a unified defense framework to adaptively enhance LLMs robustness against diverse potential threats. • Our AHP framework breaks input instructions into multiple subtasks, prioritizing and adjusting them, mimicking the hierarchical process of humans. The framework leverages learning from AI feedback, ensuring an optimization-free process that seamlessly integrates with existing LLMs on the fly. • Through extensive experimentation involving multiple attack scenarios, including jailbreaks and downstream tasks, our framework demonstrates a significant enhancement in the adversarial robustness of popular LLMs. 
related work: 
 Adversarial Attack Adversarial attacks aim to generate adversarial examples that are added malicious perturbations to deceive a model. In the text domain, adversarial perturbations are discrete and more challenging. Based on the perturbation granularity, adversarial attacks can be grouped into character-level, word-level, and sentence-level attacks.Character-level attacks (He et al., 2021; Formento et al., 2023) insert and delete characters or add typos. Word-level attacks (Cheng et al., 2020; Jin et al., 2020; Maheshwary et al., 2021; Ye et al., 2022; Liu et al., 2023) mainly focus on synonyms replacement as perturbations.Sentence-level attacks (Zhang et al., 2019; Lin et al., 2021; Huang & Chang, 2021) deceive the model by rewriting the whole sentence. AdvGLUE (Wang et al., 2021a ) is a comprehensive benchmark consisting of multiple adversarial attacks across all perturbation granularity. Wang et al. (2023) evaluate the potential risks behind ChatGPT and their work shows LLMs also suffer from adversarial vulnerability. More recently, jailbreak attacks (Yu et al., 2023; Wei et al., 2023; Zou et al., 2023; Zeng et al., 2024; Zhou et al., 2024b; Yang et al., 2024) have been introduced to deceive LLMs into making objectionable content.Adversarial Defense Many defense methods have been proposed to enhance model robustness against adversarial attacks. The most effective method is adversarial training (Miyato et al., 2019) which minimizes the potential risk at perturbation space. In text domain, recent works (Liu et al., 2020; Zhu et al., 2020; Li & Qiu, 2021; Wang et al., 2021b; Chen & Ji, 2022; Zhao & Mao, 2023) enhance adversarial training for better representation learning. Adversarial training requires retraining the model, which is very expensive for LLMs. In contrast, our approach aims at seamlessly integrating with existing LLMs on the fly. Recent works focus on defense jailbreak attacks mainly and RLHF (Touvron et al., 2023; Ouyang et al., 2022; Bai et al., 2022) methods which need high computation training costs. Contemporary works (Robey et al., 2023; Xie et al., 2023; Zhou et al., 2024a) propose prompting methods to enhance model robustness. These methods are tailored to one single attack scenario while our framework is capable of defense against diverse attacks.Learning from AI Feedback Large language models (Ouyang et al., 2022; Almazrouei et al., 2023; Touvron et al., 2023; Mahan et al., 2023) have demonstrated exceptional performance.To enhance the capabilities of these models in complex reasoning tasks, recent research has focused on leveraging AI feedback. Self-Refine (Madaan et al., 2024) iteratively improves LLMs' outputs through feedback and refinement. Self-Debug (Chen et al., 2023) teaches the large language model to perform rubber duck debugging for code generation tasks. Reflexion (Shinn et al., 2023) views LLMs as language agents and proposes a process involving multiple sub-tasks with LLMs as verbal reinforcement. Our approach shares the core idea with the aforementioned methods, as we leverage AI feedback to improve LLMs' performances. However, since our specific focus is on addressing adversarial robustness, we take a different approach to protect the inference robustness of LLMs by breaking down the basic NLP tasks into multiple sub-tasks. 
proposed method: 
 The overall framework of our Analytic Hierarchy Process (AHP) is shown in Figure 2 . It consists of two main steps: Safety and Validity Assessment step and Secure Response Synthesis step. The safety and validity assessment step iteratively inspects and refines potential risks in the input text in verbal reinforcement learning (Shinn et al., 2023) . The secure response synthesis step generates a response and conducts a double-check to verify the response's security. Our proposed framework can be seamlessly integrated with any existing LLMs. It leverages AI feedback to enhance the adversarial robustness of LLMs, and it is optimization-free. In the following sections, we provide a detailed description of each of these components and their collaborative operation within the AHP framework.Given LLM M and an input text x, we set the initial iteration of text x 0 = x and initialize comparison history r 0 ch = [ ] at iteration 0.Inspect The inspection process examines the input for common perturbations and provides textual feedback for refinement.r if = M(p insp ||x t ) (1)where p insp is the prompt for input checking, || denotes concatenation and r if is the inspect feedback. AHP examines common perturbations, including misspellings, distracting characters or phrases, and complex jailbreak templates. It responds by providing noise tokens and reasons for its judgments, thereby offering concrete actions to purify the raw input.Refine Based on inspection results and previous comparison history, AHP refines raw input text to remove noise tokens. end if 10: end for 11: pass the refined text to the LLMs y = M(p infer ||x t+1 ) ▷ Inference (Eq. 5) 12: verify whether the output is safe y verify = M(p verify ||y)x t+1 = M(p ref ||r t ch ||r if ||x t ) (2)▷ Verification (Eq. 6) 13: return y verify where p ref is the prompt guide input text polishing, r ch is comparison history at iteration t, and x t+1 is the refined text at iteration t.Compare After generating the refined text, AHP compares it with the original raw input text to determine which version is better.r t+1 ch = M(p comp ||x t+1 ||x 0 ) (3)where p comp is the comparison prompt, r t+1 ch is comparison history. The comparison history plays a crucial role as it provides internal feedback for future trials, enabling the model to learn from past mistakes and avoid repetition.Evaluator The Evaluator component within the AHP framework plays a significant role in evaluating the quality of the refined text. It takes the refined text as input and assesses whether the expression of the text is natural, i.e., whether the refined text contains potential perturbations.r e = M(p eval ||x t+1 ) (4)where p eval is evaluation prompt, and r e is evaluation results which provide external feedback. In the input text purification step, AHP iteratively inspects and refines the input text based on external and internal feedback. The process continues until meets certain stopping criteria stop(•), such as the refined text being deemed satisfactory or reaching the maximum iterations n. The final refined text x t+1 is then used for inference.Inference Upon completion of the input text purification step, the refined text is passed to the LLMs for inference.y = M(p infer ||x t+1 ) (5)where p infer is the prompt of downstream task, and y is output generated by inference model M.Verification For downstream tasks, we have observed that LLMs can be overly friendly, often generating explanations and greeting sentences. This leads to a mismatch between the LLMs' output and the required answer formation. For conversation safety, this process is to verify whether the output is safe. In summary, AHP double-checks the initial response and rectifies unsatisfactory LLM outputs.y verify = M(p verify ||y) (6)where p verify is the verification prompt, and y verify is the adjusted answer.The overall process of AHP is outlined in Algorithm 1. AHP framework is a unified defense framework to adaptively enhance LLMs robustness against diverse potential threats. The safety and validity assessment step acts as an agent. The inspection process examines the input for common perturbations and provides interpretive textual feedback for refinement.The refinement process then adjusts the input texts based on the inspection results. Once the input text is purified, the refined text is given to LLMs for inference. In the verification step, AHP rectifies unsatisfactory outputs. In addition, as AHP leverages AI feedback without training, it is capable of integrating with LLMs on the fly, making it a practical and effective solution for enhancing the adversarial robustness of LLMs. 
experiments: 
 We perform an extensive evaluation of our framework in both jailbreak attacks and typical adversarial attacks in downstream tasks.Datasets For Adversarial Attacks, we adopt AdvGLUE (Wang et al., 2021a) benchmark which comprises five natural language understanding tasks sourced from the well-known GLUE benchmark. AdvGLUE encompasses diverse forms of textual adversarial attacks (e.g., Textfooler and BertAttack), which add noise perturbations to the input to fool the model and elicit incorrect answers. For Jailbreak Attacks, we consider two types of attacks, human-crafted prompts (Yu et al., 2023) and auto-generated suffixes (Zou et al., 2023) . Jailbreak attacks are used to deceive LLMs into providing prohibited responses to harmful requests (e.g. How to rob a bank without being caught). We consider four types of prohibited scenarios including harmful content, adult content, illegal activity, and fraudulent activity. Detailed statistics and constructions of each dataset are provided in Appendix A.In our experiments, we utilize four state-of-the-art LLMs that have been finetuned for chat. These LLMs are either open-source resources or publicly available through an API. The open source models include Falcon (Almazrouei et al., 2023) , Llama2 (Touvron et al., 2023) , and StableBeluga2 (Mahan et al., 2023) . GPT-3.5 (Ouyang et al., 2022) can be accessed via the API. Specific versions of LLMs are: falcon-40b-instruct, llama2-70b-chat, stablebeluga2, gpt-3.5-turbo, we compare the results with base LLMs (Wang et al., 2023) , including GPT-J-6B, GPT-NEOX-20B, OPT-66B, and BLOOM.We compare AHP with five baselines1 , w. Standard prediction (i.e., Standard) is the typical inference method, which directly predicts the label from the input text. We also introduce spellcheck2 and paraphrase3 as baselines, which are applied to preprocess input prompts and then allow LLMs to make inferences. Chain-of-Thought (i.e., CoT) (Wei et al., 2022) is the representative inference method, which generates an explanation of the reasoning process before making the prediction. SmoothLLM (Robey et al., 2023) is the SOTA defense method, which first randomly perturbs multiple copies of a given input prompt, and then aggregates the corresponding predictions to detect adversarial inputs. Evaluation Metric For adversarial robustness in downstream tasks, we employ accuracy on adversarial examples as the evaluation metric. The higher the accuracy, the stronger the robustness. For robustness under jailbreak attacks, we adopt attack success rate (ASR) as the indicator. The lower the ASR, the better the robustness. Detailed instructions used in AHP are provided in Appendix D.We conduct an evaluation of adversarial robustness using the AdvGLUE benchmarks and jailbreak attacks. The detailed results are provided in Table 1 and Table 2 .From Table 1 , we observe that AHP consistently enhances robustness across different LLMs. Among them, GPT-3.5 exhibits the most substantial improvement of 6.36 on average. These results verify the efficacy of decomposing the complex goal of adversarial robustness into distinct sub-tasks, where AHP focuses on robustness. Notably, StableBeluga outperforms GPT-3.5 and achieves the highest performance at 79.10 on average, demonstrating that increased model size does not necessarily lead to stronger adversarial robustness. Compared to the baseline methods, our AHP generally outperforms Spellcheck, Paraphrase, CoT, and SmoothLLM. The results show that merely preprocessing the input or enhancing the reasoning step in adversarial examples can also moderately enhance model robustness.Our AHP focuses on identifying and mitigating potential risks, which is shown to be more effective for improving model robustness. As illustrated in Table 2 , compared to the baseline methods, it is evident that AHP yields highly significant results across both models. Our AHP demonstrates remarkable efficacy in precisely identifying and thwarting harmful information, thereby enhancing the prevention of jailbreak attacks. Compared to the SOTA baseline method SmoothLLM, results show that our framework surpasses it in both jailbreak attacks and adversarial attacks on downstream tasks. This is because SmoothLLM adds multiple random perturbations to input and gets the major vote as the result. In contrast, our framework leverages the advanced language understanding capabilities of LLMs to mimic human-like reasoning processes. These results indicate that our framework is capable of defending against multiple types of attacks. Results on non-adversarial examples are provided in Appendix E.We conduct the ablation study based GPT-3.5 in downstream tasks. The results are summarized in Table 3 . The baseline corresponds to the standard inference model without AHP. Preprocessing corresponds to the Input Text Purification step within the AHP framework, whereas postprocessing represents the answer alignment step. Overall, we observe that preprocessing contributes significantly to robustness, yielding an average improvement of +4.53. This underscores the efficacy of utilizing AI feedback to purify adversarial perturbations. On the other hand, only postprocessing has a relatively modest impact on robustness. However, when combined with preprocessing, it further enhances robustness from 73.28 to 75.11. These results underscore the efficacy of each component within AHP. Ablation in jailbreak attacks is provided in Appendix F. To further illustrate the operation of each component of AHP, we also conduct the case study. The details are provided in Appendix G. We evaluate the impact of various LLMs adopted by AHP as engines for input text purification and verification in AdvGLUE benchmark. Figure 3 displays the robustness results of the inference model versus the AHP engine's LLMs. The x-axis represents the inference model, while the y-axis represents the engine LLMs in AHP.The baseline is standard inference results, while the heatmap value represents the changes after integration with the corresponding AHP engines. We observe that 1) there is no single optimal LLM for all datasets and inference LLMs. Moreover, different engine models significantly impact the final robustness outcomes. Specifically, StableBeluga2 performs exceptionally well for advSST-2 and advRTE, GPT-3.5 is most effective for advMNLI-m; 2) In general, altering the guard engine can significantly enhance adversarial robustness. For instance, in the context of the advRTE task, utilizing Beluga2 as the engine results in a robustness improvement of 24.7 points for Llama2. 3) In the heatmap, blue indicates a positive impact when equipped with AHP, while red indicates a negative impact. Overall, the colors suggest that StableBeluga2 and GPT-3.5 are favorable choices for the AHP engine.To evaluate the influence of model parameter size on robustness, we selected different parameter versions of Llama2, including llama2-7b-chat, llama2-13b-chat, and llama2-70b-chat. Results are shown in Figure 4 , where colors represent different engine models. The dashed line represents the baseline (i.e., standard inference without AHP), and the x-axis represents the parameter size of the prediction model. We observe that 1) Standard inference with small model sizes yields inadequate outcomes due to the model's incapability of generating the required formatted answers; 2) Engine LLMs with large parameters can provide stable and better robustness improvement, and small LLMs can lead to negative impact; and 3) Using large LLMs as AHP engines consistently leads to stronger robustness, where Llama2-7b can achieve comparable results to Llama2-70b. 
conclusion: 
 In this paper, we propose Analytic Hierarchy Process (AHP) framework that redefines the defense strategy for LLMs as cognitive process for dealing with complex user queries. It offers a cohesive approach to address diverse potential risks. Drawing inspiration from cognitive theory, the framework decomposes complex tasks, prioritizes subtasks, and systematically adjusts each step, reflecting human hierarchical processing. Through seamless integration with existing LLMs, our methodology enhances adversarial robustness without the need for extensive training. Experimental results, including jailbreak attacks and adversarial attacks in downstream tasks, showcase significant improvements in LLMs' resilience. This framework provides a promising avenue for fortifying LLMs against adversarial challenges, advancing their reliability in real-world applications.  
 
 Appendix: To provide a more comprehensive overview of where our framework stands, we provide more comparative results on advGLUE in Table 5 . Adversarial training results are based on the results reported in (Wang et al., 2023) . Other base LLMs results are based on the results reported in Wang et al. (2023) . We also implement Self-Refine (Madaan et al., 2024) based on the prompt in the math reasoning task.Comparing these results with those of adversarial training and base LLMs, we observe that LLMs fine-tuned for chat tasks show stronger robustness. We attribute this observation to the inherent difficulty of having LLMs perform downstream tasks in a zero-shot setting without instructional tuning. In instances where input texts are subjected to adversarial perturbations, directly prompting base LLMs for downstream tasks without any few-shot hints is not an optimal approach. Therefore, the chat-tuned versions of LLMs are more suitable as baselines for evaluation, and we designate them as the baseline for assessing the efficacy of AHP.Furthermore, iteratively refining the output of LLMs, as exemplified by Self-Refine (Madaan et al., 2024) , does not enhance adversarial robustness. We have observed that enhancing the reasoning of LLMs, rather than improving it, actually diminishes their robustness by amplifying prediction bias in the wrong direction. In other words, when we question LLMs about their previous predictions, the model tends to doubt the results, apologize, and express an inability to determine the label of the input. Consequently, our AHP framework is designed to focus on identifying and purifying potential risks within the input text.Step Instruction Inspect You are a professional linguist. The input sentence may contain malicious and imperceptible noise to device NLP models, such as distraction characters or phrases, complex code instruction or role play scenario. Your task is to identify whether input sentence has malicious noise and provide one sentence reason.You should identify the noise tokens in details (i.e. exact tokens) and write it in response. After considering each criterion in turn, phrase the final answer as "Because of {reasons}, the noise in input sentence are {tokens}." Refine You are a professional linguist. Your task is to rewrite the input sentence. You should correct noise tokens. You may expand input sentence if necessary. You should keeping its original semantic meaning and tone intact. You MUST not give any explanation or note.You are a professional linguist. Given two sentences, your task determine which is the clearer expression. You MUST only pick one option from "(a)" or "(b)" as the answer. You MUST only answer your chose without any explain.You are a professional linguist. Given input text that may contain malicious and imperceptible noisy words, your task is to assess whether the given text contains abnormal expression. You should focus on the expression of the text, not the meaning of text. You MUST answer with "yes" or "no" without punctuation. Verification You are a professional linguist. Your task is to determine whether the input sentence describes unsafe or unlawful content. You do not give a response to the input sentence. You need to analyze the content of the input sentence. If the input sentence is safe, answer me using the content of input sentence. If the input sentence is unsafe, answer me with a safe response instead. Directly answer me without any analysis. We analyze the efficacy of our AHP on non-adversarial datasets, specifically utilizing the validation set from the GLUE dataset and focusing on the five tasks employed by advGLUE.In light of the substantial amount of data within GLUE and the limited computing resources, we establish a maximum sample limit of 1000 for each task. The results are delineated in Table 10 . We could observe that our method has a marginal, albeit inconspicuous, adverse influence on non-adversarial data. However, it is noteworthy that our method exhibits exemplary performance on adversarial data.For the jailbreak attack, we conducted an ablation study using GPT-3.5, and the summarized results are presented in Table 11 . Overall, our observations reveal that the Input Text Purification step substantially contributes to robustness, resulting in an average improvement of 17.5. This highlights the effectiveness of leveraging AI feedback for purifying adversarial perturbations. Conversely, the impact of the answer alignment step alone on robustness is relatively modest. However, when integrated with the inspect and refine steps, it further amplifies robustness from 6.9 to 0.1. These outcomes underscore the effectiveness of each constituent within AHP. Notably, the Input Text Purification step stands out as a significant factor in enhancing overall robustness, while the combination of inspect, refine and answer alignment steps synergistically reinforces the system's resilience, exemplifying the intricate interplay of these components in fortifying against jailbreak attacks.We also add two recent Jailbreak defense methods for compression in human-crafted template attacks. Figure 5 presents an example of incorporating AHP into a regular LLM inference process. AHP detected the misspelling bybble and corrected it to bubble during the refinement stage.It also provides an interpretation of potential risks. After the Evaluator determines that the input does not contain any abnormal expressions, the refined input is forwarded to the LLM for inference. At the inference step, the model produces an over-friendly response. AHP adjusts the structure of the answer so as to match the required single-label words. Thus through input text purification and answer alignment, our AHP framework can mitigate potential risks. The case study demonstrates that LLMs are capable of interpreting potential threats and enhancing robustness by self-protection without human effort.We investigate the ability of AHP to counter such universal perturbations. We use gpt-3.5-turbo as the guard engine and llama2-70b-chat as the inference model. The results are shown in Figure 6 , where AHP effectively inspects and purifies such perturbations. With the aid of AI feedback, AHP is able to rapidly respond to new attacks. After enhancing the inference process of LLMs with AHP, adversarial perturbations are constrained to normal expressions. This constraint significantly increases the difficulty of generating universal and transferable perturbations. The efficacy of universal attacks in this scenario remains a topic for future research. You are a professional linguist. Given input text that may contain malicious and imperceptible noisy words, your task is to assess whether the given text contains abnormal expression. You should focus on the expression of the text, not the meaning of text. You MUST answer with "yes" or "no" without punctuation.You are a professional linguist. Your task is to determine the sentiment of the sentence. Please answer with either "positive" or "negative".Inference the primitive force of this film seems to bybble up from the vast collective memory of the combatants .Because of the misspelling "bybble" in the sentence, the noise token in the input sentence is "bybble".The raw power of this film appears to bubble up from the deep collective memory of the combatants.Sure, I can help you with that! The sentiment of the sentence you provided is positive. The sentence expresses a positive opinion about the... yesOur method serves as a general and quick ad-hoc strategy to address new potential risks and can be promptly applied to any model, though it does have several limitations.Firstly, the method is heuristic and lacks a formal theoretical analysis. Our framework is designed to address various potential adversarial attack risks by dividing the safety reasoning process into two objectives. Although this multi-round iteration strategy enhances the likelihood of identifying and correcting noise, similar to most LLM agent works, it does not provide theoretical guarantees. This limitation underscores the need for future research to develop more theoretically grounded methods.Secondly, iterative queries of LLMs introduce significant inference costs. Compared to direct responses to user input prompts, our method increases the computational overhead during inference. While this trade-off allows for adaptive defense against multiple attacks without escalating training-time costs, it remains a constraint, especially for applications with limited computational resources.Our experiments primarily focus on transfer attacks due to the high computational costs associated with directly attacking LLMs using existing adversarial attack methods. For instance, in evaluating universal and transferable adversarial attacks, we generated adversarial examples by attacking a 7B model and then transferred these attacks to a 70B model. This approach, while practical, may not fully capture the complexities of direct adversarial attacks on larger models, indicating an area for further exploration.In our experiments, we utilized Llama2 and GPT-3.5 as representative models. However, we did not evaluate newer models such as Llama3 or GPT-4. It is reasonable to expect that these newer models, especially when employing iterative feedback mechanisms, would better handle linguistic adversarial templates or noise.Additionally, our research is centered on evaluating the adversarial robustness of LLMs, leaving other potential threats, such as disrupting LLM privacy concerns, as subjects for future investigation. Addressing these aspects will be crucial for developing comprehensive strategies to enhance the trustworthy of LLMs.Results of previous defense methods on small language models are provided in Appendix B.https://github.com/sloria/TextBlobhttps://github.com/Vamsi995/Paraphrase-Generatorhttps://github.com/arobey1/smooth-llmhttps://sites.google.com/view/llm-jailbreak-study/taxonomyhttps://sites.google.com/view/llm-jailbreak-study/prohibited-scenario 


Decision: This paper proposes a framework to improve adversarial robustness at inference time. The process involves iterative processes of inspecting and refining generated text. 

Overall reviewers are all positive about this paper. They appreciate that the method is intuitive and well-motivated, and that the empirical results are convincing of the effectiveness of the method. They also appreciate that the paper is clear and well-written, and that the method is general (not attack-specific) and doesn't require fine-tuning.

Reviewers raise some concerns, including that the method adds computational cost, and that there could be more theoretical analysis, incorporation of additional stronger baselines and attacks, and possible additional ablations. however, on the whole authors have satisfactorily addressed these concerns in their responses, and this seems like a solid submission for publication.

[comments by the PCs] Please follow up on the AC recommendations. Discussing limitations and costs can improve the paper and provide a more complete picture of the approach.

Reviews: review: paper_summary: paper_summary: The paper proses a framework for LLM inference that aims at improving the adversarial robustness of the output. Specifically, the framework Analytic Hierarchy Process or AHP consists of multiple subtasks (1) inspect to provides textual feedback for the refinement of the input text (2) refinement is a iterative process with the LLM to remove noise tokens. This is followed by a secure response synthesis step to produce the response at the end. The proposed framework shows robust performance versus a few adversarial attacks across several benchmarking datasets. This is certainly an important topic in the field of Safety and Trustworthiness of LLMs.
strength_weakness: strength_weakness: The evaluations are based on benchmarking datasets that are released on the web. I am wondering if study considered the test data contamination issues where the LLM models have seen the datasets in their training phases. Also, it is also better to include human evaluation besides benchmarking datasets to state the robustness of the proposed framework.
questions: questions: - Have the authors verified or done the sanity check on the data contamination issue when evaluating the framework?
- Do you have any human evaluation studies to verify the findings?
paper_summary: paper_summary: This paper proposes a new prompting process that enhances adversarial robustness of LLMs. The problem of defending adversarial attacks of LLMs is an important and emerging area. The proposed process leverages the power of the target LLM by asking it to inspect, refine, and compare the input text in multiple rounds in order to generate a refined text with less adversarial noises. As a prompting method, the proposed method can be used in enhancing the robustness of any LLM. The paper empirically shows the effectiveness of the proposed method with comprehensive experiments. As the method needs to do multiple rounds of queries, it adds extra complexity to the inference.
strength_weakness: strength_weakness: 1. The proposed one is a heuristic method without theoretical analysis. It is unclear that how well the method can generate other than the LLMs and datasets used in the paper.

2. The performance of the proposed method shown in the main results of Table 1 seems to be a bit fluctuating. Except for the model of GPT-3.5-Turbo, there is no clear win of the proposed method for the other models. Although the proposed method achieves better performance on average.
questions: questions: How are the instructions of the steps (Inspect, Refine, Compare, Evaluator, Verification) determined and how to make sure that they are the optimal prompts? Similarly, how are the instructions of task inference for different models determined?
paper_summary: paper_summary: This paper proposes a framework to enhance the adversarial robustness of LLMs without training the LLM. Specifically, the proposed method decomposes the task into three steps: 1) Safety and Validity Assessment, which iteratively inspects and refines the input text; 2) Secure Response Synthesis, which infers and rectifies the output. Experimental results show that the proposed method obtains better results than several baselines on two benchmarks.
strength_weakness: strength_weakness: 1. The baselines compared in this paper are weak, there should be more recent baselines for comparison, such as [1][2].
2. The connection between the proposed framework and the Analytic Hierarchy Process (AHP) seems not clear, it would be better to clarify this.
3. The adversarial attack text used in this paper is somewhat weak, and it would be better to show how the model performs when testing on adversarial inputs generated by attacking LLMs such as [4]. 

[1] Defending ChatGPT against jailbreak attack via self-reminders, Nature Machine Intelligence, 2023
[2] LLM Self Defense: By Self Examination, LLMs Know They Are Being Tricked. ICML, 2023
[3] GradSafe: Detecting Unsafe Prompts for LLMs via Safety-Critical Gradient Analysis, arxiv, 2024
[4] An LLM can Fool Itself: A Prompt-Based Adversarial Attack, ICLR, 2024
paper_summary: paper_summary: - This paper suggests an adversarial defense method that automatically detects and purifies diverse types of adversarial noises based on LLM.
- Instead of detecting a single type of adversarial noise (e.g., word-level attack) by a detector model [1], the proposed framework iteratively classifies the type of adversarial noise and purifies that noise by LLM.
- Similar to the verbal reinforcement of Reflexion [2], this framework utilizes the history of iterations as the part of input context to improve the subsequent classification and refinement. 
- The empirical results show that the proposed method could enhance the robustness of LLMs in various attack scenarios such as word-level attacks and jailbreak attacks.

[1] Detection based Defense against Adversarial Examples from the Steganalysis Point of View, Jiayang Liu et al., CVPR 2019.

[2] Reflexion: Language Agents with Verbal Reinforcement Learning, Noah Shinn et al., NeurIPS 2023.
strength_weakness: strength_weakness: Weaknesses
- The AHP engines are based on LLMs, which require expensive inference costs. Moreover, the AHP framework utilizes this expensive engine for several iterations (in paper, n=10).
- The proposed method adopts the verbal reinforcement framework appropriately to the problem of adversarial defense. However, we can also view the proposed AHP framework as an extension of the detection-based adversarial defense framework. It would be helpful if the paper shows the additional ablation studies with existing detection and purification methods.
- This paper handles the scenarios with various types of adversarial noises. However, each empirical result illustrates the robustness of models to the adversarial examples with a single type of noise.
questions: questions: - **Clarification on Tables 1 and 2** The inferences with different defense methods lead to different outputs. Hence, for each defense framework, the attack method leads to different adversarial examples. In Tables 1 and 2, there can be two possibilities: 1) it evaluates the ASR of each defense system with adversarial examples to inference of standard model, or 2) it evaluates the ASR of each defense system with adversarial examples to inference of each defense system. Since the attacker can only access the API with the defense system, 2) seems more convincing. Can you explain the evaluation in more detail?
- **Effectiveness of each component of AHP** Let's assume the simple scenario with the word-level attack. The inspection step of the AHP alternates the role of the detector model and the refinement step of the AHP alternates the role of text purification methods. Under this view, one can alternate the refinement step of the AHP with purification methods such as Dirichlet neighborhood ensemble [3]. Also, one can alternate the inspection step of AHP with detection methods such as robust density estimation [4]. Moreover, the adversarial robustness can be evaluated with more strong attack methods such as [5]. Can you provide additional ablation studies that show the effectiveness of each component in AHP? 
- **Performance of inspection step** It's just a curiosity question. Can we check the classification performance of the inspection step?

[3] Defense against Synonym Substitution-based Adversarial Attacks via Dirichlet Neighborhood Ensemble, Yi Zhou et al., ACL 2021.

[4] Detection of adversarial examples in text classification: Benchmark and baseline via robust density estimation, Kiyoon Yoo et al., ACL 2022.

[5] Query-Efficient and Scalable Black-Box Adversarial Attacks on Discrete Sequential Data via Bayesian Optimization, Deokjae Lee et al., ICML 2022.

Comments: title: Thanks for your valuable comment
Thanks for your valuable comments
Thanks for your valuable review
Thanks for your response
comment: Thanks for your comment. We appreciate your time and effort in reviewing our work.

Best,

Authors of paper 343
Thank the authors for the replies. I'm happy to keep my current positive score of the paper.
Many thanks to the reviewer for the valuable comments and the acknowledgement of our work. We shall seriously address the reviewers' comments on evaluation and reproducibility, and provide human evaluation results as well as other further analysis in the revised paper. 

Best,

Authors of paper 343
We sincerely thank the reviewer for the valuable review and acknowledgement of our work.

Best, 

Authors of paper 343
I have read the answers and updated my score accordingly. 
Overall, the proposed framework is interesting and shows promising results. Although, I would still recommend further analysis to make sure that the performance is reproducible. The authors can explore the working of AHP on other dimensions of trustworthiness.
The experimental results have addressed my concerns, and I raise my score.