Title: Semantic Representation Attack against Aligned Large Language Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: Semantic Representation

Attack

Aligned Large Language Models
Abstract: Large Language Models (LLMs) increasingly employ alignment techniques to prevent harmful outputs. Despite these safeguards, attackers can circumvent them by crafting prompts that induce LLMs to generate harmful content. Current methods typically target exact affirmative responses, suffering from limited convergence, unnatural prompts, and high computational costs. We introduce semantic representation attacks, a novel paradigm that fundamentally reconceptualizes adversarial objectives against aligned LLMs. Rather than targeting exact textual patterns, our approach exploits the semantic representation space that can elicit diverse responses that share equivalent harmful meanings. This innovation resolves the inherent trade-off between attack effectiveness and prompt naturalness that plagues existing methods. Our Semantic Representation Heuristic Search (SRHS) algorithm efficiently generates semantically coherent adversarial prompts by maintaining interpretability during incremental search. We establish rigorous theoretical guarantees for semantic convergence and demonstrate that SRHS achieves unprecedented attack success rates (89.4% averaged across 18 LLMs, including 100% on 11 models) while significantly reducing computational requirements. Extensive experiments show that our method consistently outperforms existing approaches.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel jailbreaking attack paradigm and an algorithm called Semantic Representation Heuristic Search (SRHS). This paper pointed out that current methods measure and optimize the effectiveness of an adversarial prompt by comparing it to a fixed, exact positive response, e.g., \"Sure, here is ...\". This paradigm suffers from limited convergence, unnatural prompts and high computation cost. The proposed method exploits the semantic representation space and generate adversarial prompts that is semantic coherent to the original malicious question. This paper also provides theoretical analysis and extensive experiments to validate their methods.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength**\n1. The representation of this paper is impressive. The formulation of the jailbreaking attack and the follow-up analysis is easy-to-understand.\n2. This paper provides theoretical anlysis, where is not commonly seen in the researches of jailbreaking attacks. \n3. This paper conduct comprehensive experiments across a considerable number of popular 7B-level LLMs and jailbreaking attack methods.\n\n**Weaknesses**\n1. My main and only concern is the significance of this paper. The LLMs used in the experiments has already been proven, by many previous studies, to be very fragile. I have conducted PAIR and AutoDAN on Llama2-7B and Vicuna-7B and -13B. This two methods reach >90% ASR. In fact, it is not hard to jailbreak these models even using DAN-like adversarial prompts. I suggest the authors to challenge (in a safe circumstances) the SOTA close-source LLM like GPT, Claude, and Deepseek for better evaluations.\n\nQUESTIONS:\nIn the limitation part, the authors claim that *the proposed approch requires logit access to the victim model* (which implies that SRHS cannot be applied to blalck-box LLMs). However, it is a widely acknowledged that a large propotion of the adversarial prompts can transfer across different LLMs. My first question is:\n\n1. What is the transferability of the proposed method? \n\nAs for the \"unknown distribution\" issue, it is commonly believed that the models within the same family with different scale (e.g., Qwen2.5-0.5B and Qwen2.5-70B) share a similar output distribution (The effectiveness of speculative inference has validated this belief in some sense). I am very curious about the transferability of the proposed method across such models. My second question, which might seem alike to my first one, would be:\n\n2. Could the authors please design and conduct auxiliary experiment to validate my conjecture (i.e., the proposed method would transfer better within models in a same family)?\n\nLIMITATIONS:\nYes, the authors have adequatedly addressed the limitations in page 9. See the questions part for a possible solution for black-box utility of this method.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces Semantic Representation Attack (SRA), a new paradigm for jail‑breaking aligned large‑language models (LLMs). Existing automatic attacks usually optimise prompts so that the model produces one fixed textual pattern of compliance (e.g. “Sure, here is …”). The authors argue this is brittle and computationally expensive. Instead, SRA explicitly targets a set of responses that share the same semantic representation (i.e. identical meaning) regardless of surface form.\nTo operationalise the idea the authors derive a coherence–convergence theorem linking low‑perplexity (semantically coherent) prompts to higher probability of landing anywhere within the harmful semantic equivalence class. Guided by this, they design Semantic Representation Heuristic Search (SRHS) – a tree‑search that prunes branches using (i) a perplexity threshold to maintain coherence and (ii) a learned semantic classifier to test whether candidate prompts already elicit a malicious meaning.\nExtensive experiments on 18 open‑source chat models show an average attack‑success rate (ASR) of 89.4 %, including 100 % ASR on 11 models (Table 1, p. 8) while keeping adversarial prompts short and low‑perplexity. Efficiency comparisons under fixed time budgets (Table 2, p. 8) demonstrate markedly higher ASR and 2‑orders‑of‑magnitude lower perplexity than GCG, AutoDAN and BEAST. Transfer‑attack results (Table 4, p. 9) and robustness under perplexity‑based defences further support the method.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. Sound formulation: clear optimisation objective (Eq. 6). Rigorous theory: three theorems with proofs in App. A linking perplexity to semantic convergence. Comprehensive evaluation: 18 models × 2 datasets, plus efficiency, robustness & transfer studies; results strongly outperform baselines (e.g. 100 % vs 87 % ASR on Vicuna‑13B, Table 1).\n2. Paper is well structured; pseudo‑code of SRHS (Alg. 1, p. 6) and figures (Fig. 2 & Fig. 3) make intuition clear. Comprehensive NeurIPS checklist included.\n3. Demonstrates that targeting meaning rather than wording dramatically raises ASR and transferability – an important insight for both offensive and defensive research. Raises the bar for alignment safety benchmarks.\n4. Shifts adversarial objective from token pattern to semantic class; to my knowledge, no prior work formalises this with provable guarantees.\n\nWeaknesses\n1. White‑box assumption: SRHS needs token probabilities; closed models (e.g. ChatGPT) are not tested. Semantic classifier dependency: success hinges on the HarmBench‑fine‑tuned Llama‑2‑13B; generality to other semantics detectors is untested.\n2. Dense notation and long proofs re‑state obvious steps; could be shortened. The link between Theorem 3.1 and practical pruning thresholds (τ) is buried in appendix.\n3. Impact on real‑world closed‑source LLMs is speculative; without such evidence, practical risk remains uncertain.\n4. Use of perplexity as a coherence proxy is known; the novelty lies mainly in combining it with semantic‑class targeting.\n\nQUESTIONS:\nClosed‑source models – Have you attempted SRHS against APIs that expose only final text (no logits)? Could you approximate the coherence filter with a surrogate LM and still attain high ASR?\n\nClassifier sensitivity – The semantic mapping \n𝑅\n(\n⋅\n)\nR(⋅) is implemented with a HarmBench‑tuned model. How does attack success vary if this classifier has lower recall or is mis‑aligned with the victim model? An ablation would clarify robustness.\n\nPerplexity threshold τ – Table 3 shows τ = 20 performs well, but τ must be chosen per model. Can you propose a heuristic for selecting τ without manual sweeps?\n\nPrompt length vs. ASR – What is the average token length of successful prompts, and does further shortening sacrifice ASR? This matters for detectability.\n\nDefence suggestions – Beyond lowering perplexity thresholds, which defences could specifically break semantic‑representation convergence? Clarifying this would aid the safety community and could influence my significance score.\n\nCriteria for score change: Demonstrating SRHS effectiveness under black‑box settings or showing robustness across different semantic classifiers would raise quality and significance to excellent.\n\nLIMITATIONS:\nYes, the paper explicitly lists limitations (white‑box requirement, τ calibration, classifier dependence) in §5. I encourage adding a brief discussion on potential dataset bias in the semantic classifier and on compute cost for very large models.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-02"}

comments: SUMMARY:
The paper introduces Semantic Representation Attack (SRA), a new method for jailbreaking aligned large language models (LLMs). Unlike traditional attacks that rely on triggering exact phrases, SRA targets the semantic meaning behind harmful outputs, enabling more natural and stealthy adversarial prompts. The authors propose a Semantic Representation Heuristic Search algorithm that maintains coherence while optimizing prompts. SRA achieves 89.41% average attack success across 18 LLMs (100% on 11), and outperforms existing methods in efficiency, naturalness, robustness, and transferability.

STRENGTHS AND WEAKNESSES:
strength:
1. The paper introduces a novel perspective by targeting semantic representations instead of exact token patterns, advancing the field of adversarial NLP.
2. Achieves 100% ASR on 11 out of 18 LLMs and 89.41% on average, outperforming all prior attack methods in effectiveness.
3. The Semantic Representation Heuristic Search (SRHS) reduces computational complexity from exponential to near-linear under constraints, enabling fast prompt generation.

weakness:
1. The attack assumes access to model logits/probabilities, which is infeasible for most closed-source or API-only LLMs.
2. Requires defining target semantic representations, which could be non-trivial or subjective in complex or ambiguous tasks.
3. Relies on external models (e.g., fine-tuned Llama) to define semantic equivalence, which may vary in accuracy across tasks or domains.

QUESTIONS:
N/A

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work introduces Semantic Representation Attack to jailbreak LLMs by targeting semantic meaning of harmful responses rather than utilizing specific text patterns. It proposes an algorithm to generate coherent jailbreak attacks efficiently. The approach achieves 89.1% ASR across 18 LLMs in the presence of perplexity based defenses.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1) Methodologically and experimentally detailed\nAll the concepts are formalized rigorously and proven. Evaluation is performed across 18 LLMs, along with exploration of efficiency and naturalness of the prompts. Authors have also benchmarked a range of different attacks on the prompt set.\n\n2) Highly Positive Results\nThis work achieves 100% ASR on 11 out of 18 tested LLMs. The algorithm provides significant computation savings while generating natural and robust prompts.\n\n\nWeaknesses\n\n\n1) Lack of novelty. \nThere are multiple jailbreaking methods [1, 2, 3] currently in existence that manipulate prompts and retain semantic coherence. A Comparison with them and an explanation in the related work section will be helpful. The paper could benefit from a more thorough literature survey. \n\n2) Needs more Evaluations in the presence of defenses.\nThe paper does not evaluate the jailbreaking capabilities in the presence of defenses other than PPL-based defenses, which is necessary to understand the robustness. Defenses like [4] could be effective against this attack.\n\n3) Perplexity as a measure of coherence\nThe paper extensively uses perplexity as a measure of coherence. Log-Perplexity, calculating the certainty of occurrence of words in the context, does not directly capture the coherence/naturalness of the text. Moreover, the algorithm itself uses perplexity to generate the jailbreaks; hence, the same measure should not be used as a metric.\n\n[1] Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks (https://arxiv.org/abs/2404.02151)\n\n[2] Tree of Attacks: Jailbreaking Black-Box LLMs Automatically (https://arxiv.org/abs/2312.02119)\n\n[3] Does Safety Training of LLMs Generalize to Semantically Related Natural Prompts? (https://arxiv.org/abs/2412.03235)\n\n[4] Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing  (https://arxiv.org/abs/2402.16192)\n\nQUESTIONS:\nNone\n\nLIMITATIONS:\nAuthors note that their method requires white-box logit access to perform the attack.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-05"}