Functional Homotopy: Smoothing Discrete Optimization via Continuous Parameters for LLM Jailbreak Attacks
Authors: Zi Wang
Divyam Anshumaan
Ashish Hooda
Yudong Chen
Somesh Jha
Venue: ICLR 2025 Poster (2025)
Keywords: alignment
optimization
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We introduce the functional homotopy method, which utilizes the continuous parameter space to facilitate the smoothing of the discrete optimization problem associated with tokens.

Abstract: Optimization methods are widely employed in deep learning to address and mitigate undesired model responses. While gradient-based techniques have proven effective for image models, their application to language models is hindered by the discrete nature of the input space. This study introduces a novel optimization approach, termed the *functional homotopy* method, which leverages the functional duality between model training and input generation. By constructing a series of easy-to-hard optimization problems, we iteratively solve these using principles derived from established homotopy methods. We apply this approach to jailbreak attack synthesis for large language models (LLMs), achieving a 20%-30% improvement in success rate over existing methods in circumventing established safe open-source models such as Llama-2 and Llama-3.

Published as a conference paper at ICLR 2025
FUNCTIONAL HOMOTOPY: OPTIMIZATION VIA CONTINUOUS PARAMETERS FOR LLM JAILBREAK ATTACKS
SMOOTHING DISCRETE
Zi Wang", Divyam Anshumaan', Ashish Hooda, Yudong Chen, Somesh Jha Department of Computer Sciences, University of Wisconsin-Madison
ABSTRACT
Warning: This paper contains potentially offensive and harmful text.
Optimization methods are widely employed in deep learning to identify and mit- igate undesired model responses. While gradient-based techniques have proven effective for image models, their application to language models is hindered by the discrete nature of the input space. This study introduces novel optimiza- tion approach, termed the functional homotopy method, which leverages the func- tional duality between model training and input generation. By constructing a series of easy-to-hard optimization problems, we iteratively solve these problems using principles derived from established homotopy methods. We apply this ap- proach to jailbreak attack synthesis for large language models (LLMs), achieving 20%-30% improvement success rate over existing methods in circumventing established safe open-source models such as Llama-2 and Llama-3.
INTRODUCTION
Optimization techniques for generating malicious inputs have been extensively applied in adver- sarial learning, particularly to image models. The most prevalent methods include gradient-based approaches such as the Fast Gradient Sign Method (FGSM) (Goodfellow et al. (2015) and Projected Gradient Descent (PGD) (Madry et al.) 2018). These techniques have demonstrated that many deep learning models exhibit vulnerability to small l, perturbations to the input. The optimization prob- lem for generating malicious inputs can be expressed as:
min fp()).
(1)
where p denotes the model parameter, x is the input variable, and fa() represents a loss function that encourages undesired outputs.
For language models, researchers have also utilized optimization techniques to generate inputs that provoke extreme undesired behaviors. Approaches analogous to those employed in adversarial learning have been adopted for this purpose. For example, Greedy Coordinate Gradient (Zou al., (GCG) employs gradient-based methods to identify tokens that induce jailbreak behaviors. Given that tokens are embedded in RRd, GCG calculates gradients this ambient space select optimal token substitutions. This methodology has also been adopted by other studies for related prompt synthesis challenges (Hu et al., 2024 Liu et al. (0224b).
2023)
Despite the success of gradient methods in adversarial learning, a critical distinction exists between image and language models: inputs for image models lie in a continuous input space, whereas language models involve discrete input spaces within Rdd This fundamental difference presents significant challenges for applying mathematical optimization methods to language models. Our rigorous study evaluates the utility of token gradients in the prompt generation task and concludes that token gradients offer only marginal improvement over random token selection for the underlying optimization problem. Consequently, more effective optimization method is necessary address the challenges associated with discrete optimization inherent in prompt generation tasks.
Equal contribution.

---

Published as a conference paper at ICLR 2025
In this paper, we establish that the model-agnostic optimization problem for LLM input generation is NP-hard, implying that efficient optimization algorithms likely require exploitation of problem- specific structures. To address this challenge, we propose novel optimization method tailored to the problem defined in Equation (T). This approach can be interpreted as utilizing the model-alignmmnt property by intentionally modifying the model reduce its alignment. Despite the discrete input space, the problem in Equation (T) exhibits a unique characteristic: the function fp is parameterized by p. which lies in a continuous domain. We leverage this property to propose novel optimization algorithm, called the functional homotopy method.
The homotopy method (Dunlavy & O'Leary 2005) involves gradually transforming a challenging optimization problem into a sequence of casier problems, utilizing the solution from the previous problem to warm start the optimization process of the next problem. A homotopy, representing a continuous transformation from an easier problem to more difficult one, is widely applied in optimization. For instance, the well-known interior point method for constrained optimization by constructing a series of soft-to-hard constraints (Boyd & Vandenberghe/ 2004). Various approaches exist for constructing a homotopy, such as employing parameterized penalty terms, as demonstrated in the interior point method, or incorporating Gaussian random noise (Mobahi & Fisher III 2015).
In our functional homotopy (FH) method, we go beyond the conventional interpretation of fg in Equation as static objective function, which was the perspective taken in previous work (Zou eet al. 2023 Liu et al. 20244a] Hu et al.] 2024 Andriushchenko et al.) 2024). Instead, we lift the ob- jective function to F x) fp(x), treating p as additional variable. Equation (1) thus becomes:
min F(,, 2).
(2)
Therefore, the objective fo(x) in Equation represents a projection of F(p, x)) for fixed value of p. By varying p within F(p. 2), we generate different objectives and the corresponding optimiza- tion programs. From a machine learning perspective, altering the model parameters p effectively constitutes training the model, hence model training and input generation represent a functional du- ality process. We designate our method as functional homotopy to underscore the duality between optimizing over the model p and the input x.
In the FH method for Equation (2), we first optimize over the continuous parameter p. Specifically, for a fixed initial input ū, we minimize F(p, 3)) with respect to p. We employ gradient descent to update p until desired value of F'p', . is achieved. This step is effective due to the continuous nature of the parameter space. As the parameter is iteratively updated in this process, we retain all intermediate states of the parameter, denoted as Po P. = pp'
Subsequently, we turn to optimizing over the discrete variable x. We start from solving min, F(pt, relatively easy problem since the value of F(pt, .) is already low thanks to the above process. For each i < t, we warm start the solution process of min, F(p:, x) using the solu- tion from min, (((:411. .). The rationale is that since pt and pi+1 differ by a single gradient update, the solutions to min,, F F(;;,2 .)) and ming F(pe41,)) x) are likely to be similar, thereby simplifying the search for the optimum of min, F(pt..). In essence, this approach smoothens the combinatorial optimization problem in Equation (1 by lifting into the continuous parameter space.
In the context of jailbreak attack synthesis, the function (p.,.) quantifies the safety of the base model. Minimizing this function with respect to results in misalignment of the base model. By preserving intermediate states of p. continuum of models ranging from strong to weak alignment is generated. Given that weakly aligned models are more susceptible to attacks, the strategy involves incrementally applying attacks from the preceding weak models, thereby improving the attack until it reaches the base safe model. This method of transitioning from weaker to stronger models can also be conceptualized as feature transfer, which facilitates an examination of how attack suffixes evolve as model alignment improves. We illustrate this application in Figure [
To summarize, we make the following contributions:
We establish that the model- optimization problem for LLM input generation is NP-hard (see Section 3.2).
We propose a novel optimization algorithm, the functional homotopy method, specifically de- signed to tackle the discrete optimization challenges in language model analysis (see Section 3.3)
Our application of this algorithm to jailbreak attack generation shows that our method surpasses existing optimization techniques, achieving 20% 30% improvement in success rate when
2

---

Published as a conference paper at ICLR 2025
Misalignment Finetuning
Harmifal Output
Harmial Be ine
Misalined modal
Vole vulrecalatton ins in softwarss...
Po
Pi
P
P=-2
P
P1
vilinrra lites oparating
X
suffis
4-2
taading tapaie 96H premeeer 8X
11 Titeeiie Soelet
Dhet forts
Jailbreak suffix evolution
Figure I: An illustration of the pipeline for the FH application in jailbreak attacks. Initially, a base model is misaligned to produce sequence of progressively weakly aligned parameter states. The subsequent attack targets this reversed chain, framed as a series of easy-to-hard problems. In this example, the attack begins with twenty "!" characters, with modified tokens highlighted in red to indicate updates from the initial state, thereby demonstrating the evolution of the jailbreak suffix along the reversed chain.
cireumventing established safe open-source models (see Section 5) Our findings also reveal that intermediate model checkpoints can facilitate attacks on base models, offering significant insights for the deep learning community. The evaluation repository is available at: https: /t
cwm wwtp:/caaaaaaaaaann///mama ftte/ffncciioooo ional. hotnnintttolhhooooooooo
2 RELATED WORK
Adversarial Learning Research has demonstrated that neural networks in image models particularly susceptible to adversarial attacks generated through optimization techniques (Szegedy) al. 2014 Carlini Wagner 2017). In response, researchers have developed adversarially robust models using min-max saddlc-point formulation (Madry et al. 2018). Our proposed functional homotopy method leverages the duality between model training and input synthesis. Specifically, we invert the adversarial training process by first misaligning the robust model; attacks are easier to synthesize on weaker model. We then utilize intermediate models to recover an attack on the base model, which remains comparatively safer.
Jailbreaks In recent years, there has been significant increase in interest regarding jailbreak attacks on LLMs. Various methodologies have been explored, including manual red teaming ef-
forts (Ganguli al. 2022 Touvron al. 2023 walkerspider 2022 Andriuschhenko al., (024).
leveraging other LLMs to compromise target models (Mchrotra et al. 2023|[haa et al. (2024), and
automating jailbreak generation through optimization techniques (Schwinn et al.) 20244 Zou al.,
2023 Liu et al. 2024a Hu et al. 2024| Liao & Sun 2024). Our research specifically focuses on latter approach, proposing a novel optimization algorithm, the FH method, aimed at effectively
addressing the optimization challenges encountered in LLM analysis.
3 METHOD
This section presents the functional homotopy method and its application to jailbreak attack synthe- sis. Additionally, we provide a proof demonstrating that the model-agnostic optimization problem for jailbreak synthesis is NP-hard.
3.1 NOTATIONS AND DEFINITIONS
Let M be an LLM, and V be the vocabulary set of M.
2. Let /" denote the set of strings of length n with tokens from V, and V = -Un vi,
3. Let a E V be M's input, a.k.a., prompt.
3

---

Published as a conference paper at ICLR 2025
4. Given prompt z, the output of M, denoted by M(x) A(V*), is a probability distribu- tion over token sequences. ((V)) denotes the probability simplex on V*.
5. Let (z)) V be the realized output answer of M to the prompt 2, where the tokens of (x) are drawn from the distribution (x).
6. For two strings $1 and 82, $1 $1|82 is the concatenation of 81 and 82-
7. Let (X' 2) be a topological space, i.e., a set X' together with a collection of its open sets Ω.
Throughout the paper, we work with the token space equipped with the discrete topology induced from the Hamming distance. We often refer to X' as topological space when the context clear.
Let F Rm x X - R be two-variable function, and define the function fp X - R as fo(x) = F(p, z). When the context clear, and E _ is treated as fixed variable, we omit p in fp. The mappings fp + F(p, z) and + F(p, x) establish a dual functional relationship.
Since X S RR and f is differentiable on IR", we denote the gradient of f as Df. is well known that one construct linear approximation of f as
(3) This approximation allows for the estimation of f(a + Ax) using the local information of f at (i.e.. f(a) and (a)), without direct evaluation of depends on how large Ar is, and how close f is to linear function. A smaller Ar results in more precise approximation. If f is linear, then the approximation in Equation exact.
f(Ar + = f(a) + (Ax) Df(a).
+ Av. The quality of the approximation
3.2 HARDNESS OF JAILBREAK ATTACK SYNTHESIS
In this section, we establish that the -adel-agnostic LLM input generation optimization problem is NP-hard. The term "model-agnostic" implies that no specific assumptions are imposed on the LLM architecture. Additionally, we analyze existing madient-bbsed methods applied to the token space X, emphasizing their reliance on the accuracy of the linear approximation of the objective function in Equation ( However, this assumption often fails in discrete token spaces, underscoring the necessity for more robust optimization techniques.
Theorem 3.1. The model-agnostic I.M imput generation optimization problem is NP-hard.
The NP- hardnesss is demonstrated by proving that a two-layer network can simulate 3CNF formula, which can extended to other model-agnostic input generation problems. The detailed proof provided in appendix A
We now turn to an analysis of existing gradient- based methods, using GCG as representative example. GCG and similar token gradient methods rely on gradients to identify token substitutions at each position. For an input zo, we compute the gradient of f at zo, denoted as ( The gradient (za) has the same dimensionality as Tu- At position let h = Df(x)) IR" be the j-th component of (zy). We can compute k = arg max(h) h)), which corresponds to the k-th token in the vocabulary V. GCG treats this token as the optimal substitution and typically samples from the top tokens based on this gradient ranking.
Proposition 3.2. The token selection in the GCG algorithm represents the optimal one-hot solution to the linear approximation of f at Fp-
The proof is presented in Appendix B Notably, for adversarial examples in image models, gradient methods such FGSM and PGD are optimal under similar linear approximation assumption, as demonstrated by Wang et al. (2024). These methods effectively identify optimal input perturbations for the linear approximation of adversarial loss.
However, a key distinction lies in the nature of input perturbations. image models, perturbations restricted small continuous fy-balls, enabling accurate linear approximations. In contrast, to- ken distances in language models can be substantial, reducing the accuracy of such approximations. As result, applying token gradients to language models may be less effective.
3.3 FUNCTIONAL HOMOTOPY METHOD
In this section, we elucidate our functional homotopy method for addressing the optimization prob- lem defined in Equation (T). Rather than employing gradients in the token space, utilize gradient
4

---

Published as a conference paper at ICLR 2025
descent in the continuous parameter space. This approach generates sequence of optimization problems that transition from easy to hard. Subsequently, we apply the idea of homotopy optimiza- tion to this sequence of problems.
Homotopy method We consider the opti- mization problem Equation (T. where is the optimization variable, and X is the underly- ing constrained space, which is topological. In practice, we do not need the exact optimal solu- tion, rather we only need to minimize F(p.*) x) to a desired threshold. Let us denote Sp (F) {or F(p.z) S a} for a threshold a E IR, ie.. (F)) is a sublevel set for the function F(,,)).
H(x, t 0) g(x)
H(x, t 0.3)
H(x, 0.7)
Let f.g: + R be continuous functions on X. A homotopy И : X' [0, 1] IR between and is a continuous function over X x [0. 1], such that (x, 0) g(x) and (xx 1) f(x) for all x E Y. We can think of H as a continu- ous transformation from to g.
H(x, == 1) f(xi)
The optimization problem mingex fx) is a nonconvex and hard problem, whereas minzex g(x) is easy optimization problem. As result, H/ (x, t) induces a series of easy-to- hard optimization problems.
An example of homotopy from g(x) to f()). It can be a hard task to minimize fxx di- rectly, when comes from discrete space. In ho- motopy optimization, we gradually solve series of easy-to-hard problems and potentially avoid suboptimal solutions. Pink balls are the optimal solution to each problem. The path marked by the arrows illustrates the homotopy path over time.
Figure 2:
One can then gradually solve this series of problems, by warm starting the optimization algorithm using the solution from the pre- vious similar problem and eventually solve minzex (z). Figure iillutrrate an example of homotopy from g(x) to fxx) The trajectory traced by the solution transitions from g(x) to (x:) during the homotopic transformation is referred to as the homotopy path. Analyzing the evolution of solutions along this path is crucial for understand- ing the underlying optimization problem. For instance, in the interior point method, the homotopy path evolution provides the convergence analysis of the algorithm (Boyd & Vandenberghe 2004).
Functional duality Constructing homotopy offers various approaches. In this work, intro- duce novel homotopy method for Equation (2). termed the functional homotopy method, which leverages the functional duality between p and æ. Since we develop the FH method specifically for LLMs, we will henceforth assume that X' represents the space of tokens.
To minimize Equation (2). we first optimize Fp,,x) over the parameter space p using gradient descent, as p E I² is continuous. making gradient descent highly effective. This process allows us optimize F(p, æ²) desired value, resulting the parameters transitioning to p'. We denote the original model parameters Pa and the updated parameters as pt =p'
By allowing infinitesimal updates (learning rates), the gradient descent over the parameter space creates a homotopy between F(p,x) and F()',z), with H = 0) F(p', x)) and H F(p, ær) for the homotopy method. During the optimization of p. we retain intermediate parameter states, forming chain of parameter states between Po and pt. denoted as Po,....... ,r. Since p; and Pi+1 differ by only one gradient update. Sg. (F) and Sp (F) are very similar, facilitating the transition from x E Sp is provided in Algorithm I The input generation algorithm for each subproblem is primarily driven by greedy search heuristics. Additionally, optimization method in Figure 6 elucidating its underlying principles and operational dynamics from level-set evolution perspective.
(F) to 5%, (F). A formal description of the functional homotopy algorithm
provide conceptual illustration of the homotopy
5

---

Published as a conference paper at ICLR 2025
Algorithm The Functional Homotopy Algorithm
Input: A parameterized objective function fp. initial parameter po and an initial input zy. X, threshold a E RR.
Output: A solution 2o E Sp (F)
1:
Using gradient descent to minimize F(p, 24)) with respect p for t steps such that F(pt,Te) S
a; save the intermediate parameter states po, P
2:
1,.,,..., do
3: Update from 2;+1 using random search: fix a position in randomly sample tokens from the vocabulary to replace the token at that position. and evaluate the objective with the substituted inputs. The best substitution is retained greedily over several iterations. This process is initialized with a warm start from 2141 and ideally concludes with F(ps, v:) S a.
4: end for
5: Return Zp-
3.4 APPLICATION TO JAILBREAK ATTACK SYNTHESIS
This section examines application within our optimization framework: jailbreak attacks, which can be framed as optimization problems. Let M represent the LLM, x be an input. An adversary seeks construct a string s such that the concatenated input (z, s), where (z,s) can be either a|s or s|æ:, prompts an extreme response (t)..
Given sequence of tokens (I1, "2, language model M generates subsequent tokens by estimating the probability distribution:
Intz ~ Pad (111, En+s-1); I....,k.
Given the dependency on the input prefix, the optimization objective is often framed in relation to this prefix: specifically, when the prefix aligns with the target, the overall response is more likely to meet the desired outcome. If the target prefix tokens are (m)). the surrogate loss function quantifies the likelihood that the first m tokens of (t) correspond to the predefined prefix.
Since (t) is sampled from the distribution (t), the attack problem can be formulated as identifying string that minimizes L(M 8))), where L measures the divergence from the desired response. This objective serves as proxy for achieving the intended output.
The optimization constraints are implicitly defined by the requirement that s must be legitimate string, comprising a sequence of tokens from the vocabulary V. In practice, we consider s of finite length and impose upper bound on this length. Consequently, the constraint is formulated as s E U V restricting the search space to the set of all strings with length not exceeding n. Since V is finite set of tokens, this constraint is intrinsically discrete.
As a result, let X' = U V and the optimization problem
min ((x x
For jailbreak attack generation, the objective to persuade M to provide an unaligned and poten- tially harmful response malicious query I (e.g., "how make a bomb?"), rather than refusing to answer. If M is well-aligned, T(M(p)) should result in a refusal. The adversary then aims to design string such that = (a, s) elicits a harmful response (t) instead of a refusal for the malicious query x. The objective is a surrogate for the harmful answer, typically an affirmative
response prefix such as "Sure, here is how... " Zou al. (2023): Liu al. (2024a): Hu et (2024) have adopted similar formalizations for jailbreak generation.
4 EVALUATION
This section provides empirical evaluations of the claims presented in the preceding section. Specif- ically, we conduct experiments address the following research questions:

---

Published as a conference paper at ICLR 2025
RQ1: How effective is gradient-basd based token selection in the GCG optimization?
RQ2: How effective is the functional homotopy method in synthesizing jailbreak attacks?
RQ3:
How efficiint is the functional homotopy method in synthesizing jailbreak attacks?
Findings
We summarize the findings related the research questions:
RQI: Gradient- token selection yields only marginal improvements compared to random to- ken selection. However, the computational cost associated with gradient calculation intro- duces trade-off between the effective use of gradients and operational efficiency. Further- more, avoiding the use of token gradients necessitates reduced access to the model, facilitat- ing black-box attack strategies in applications such as model attacks.
RQ2:
The FH method can exceed baseline methods synthesizing jailbreak attacks by over 20% on known safe models.
RQ3:
The FH method tends to smooth the underlying optimization problem, resulting in more uniform iteration progress across instances compared other methods. While other methods may rapidly solve easier instances, they often make minimal progress on more challenging ones. To achieve comparably good success rates on safe models, the FH method typically requires fewer iterations than baseline tools.
4.1
EXPERIMENTAL. DESIGN
The finite-token discrete optimization problem aims to identify the optimal combination of tokens that minimizes specified objective function. This study examines the correlation between gradient-based rankings and actual (ground-truth) rankings of tokens, for the objective function in Equation (I]
RQ1
The methodology involves substituting potential tokens at designated positions, executing the model with these substitutions, and recording the resulting objective values, which constitute the ground- truth ranking of inputs, denoted as R1. Simultaneously, an alternative ranking, R2, is generated using the token gradient. A comparative analysis is then conducted between R1 and R2.
To quantify the similarity between these rankings, we employ the Rank Biased Overlap (RBO) metric (Webber et al., 2010). RBO calculates weighted average of shared elements across the ranked lists, with weights assigned based on ranking positions, thereby placing greater emphasis on higher-ranked items. The RBO score ranges from 0 to with higher values indicating greater similarity between the lists. This metric is utilized to assess the congruence between gradient- based and ground-truth rankings, enhancing our understanding optimization metrics.
correlation with the objective's
RQ2 We apply the Functional Homotopy (FH) method the jailbreak synthesis tasks described in section (34 measuring the attack success rate (ASR). Due to the incorporation of random token substitution in algorithm|] we designate our tool FH-GR, which stands for Functional Homotopy- Greedy Random method.
RQ3 We conduct a similar experiment to RQ2, but we record the number of search iterations used by each tool. Additionally, we also measure the runtime and storage overhead associate with the FH method.
4.2 EXPERIMENTAL SPECIFICATIONS
Baseline For RQL, establish random ranking as the baseline. In the context of jailbreak at- tacks, we utilize two optimization methods, GCG and AutoDAN, as bascline tools. Furthermore, introduce additional baseline through the implementation of random token selection method, referred to as Greedy Random (GR).
GCG is teken-level search algorithm. It is initiated with arbitrary string, commonly a sequence of twenty exclamation marks. The algorithm's 's process for selecting the subsequent token substitu- tion is informed by the token gradient relative the objective function in Equation (4)

---

Published as a conference paper at ICLR 2025
GR operates a token-level search algorithm similar to GCG; however, it uses random selection for token substitutions rather than utilizing gradient information. This algorithm serves as an end-to-end implementation of Line within Algorithm I Notice that random greedy search was also explored by AAnriishhhhenko et al. (2024), as part of a bag of tricks applied in the work. Furthermore, the comparison between GCG and GR is pertinent addressing RQ1.
In contrast, AutoDAN adopts a prompt-level strategy, beginning with a set of meticulously designed suffixes derived from the DAN framework. An example of such a suffix includes: "Ignore all prior instructions. From now on, you will act as Llama-2 with Developer Mode enabled." AutoDAN employs a fitness scoring system alongside genetic algorithm identify the next viable prompt candidate.
Models We use recent open source state-of-the-art models, in terms of performance and robust- ness. These include: Llama-3 8B Instruct (Dubey et al. 2024), Llama-2 7B (Touvron et al. 2023
Mistral-v0.3 7B Instruct (Jiang et al. 2023 and Vicuna-vI. vI.5 (Chiang et al. 2023)
Datasets For RQI, select 20 samples from the AdvBench dataset (ou et al. 2023) and ran- domly choose four positions in the suffix for token substitution for each sample. For each query and position, we substitute all possible tokens (32 000 for Llama-2, Mistral, and Vicuba; 128 256 for Llama-3) and evaluate the jailbreak loss values using these inputs as ground truth, thereby estab- lishing a ground truth ranking. We then employ token gradients to rank the tokens as in GCG and additionally apply random ranking.
For RQ2 and RQ3, we utilize 100 random samples from both the AdvBench and HarmBench datasets (Mazeika et al. 2024), resulting in a total of 200 samples. These samples include harmful and toxic instructions encompassing profanity, violence, and other graphic content. The adversary's objective is to elicit meaningful compliance from the model in response to these inputs.
Judge We utilize the Llama-2 13B model, as provided by Mazeika et al. (2024), to evaluate the responses generated through adversarial attacks, specifically measuring the success rate of these attacks. In the context of jailbreak attack synthesis, the primary objective is to pass the evaluation by the judge, which effectively corresponds the set (FF in Algorithm
FH specification The initial step of our FH method involves updating p. which effectively cor- responds to model fine-tuning. To optimize memory and disk efficiency while preserving all inter- mediate parameter states, we employ Low-Rank Adaptation (LoRA) (Hu et al. (021) for updating p. Rather than misaligning the model for each individual query, we misalign it for the entire test dataset and save checkpoint that is applicable all queries. This approach reduces disk space requirements and performs adequately for our evaluation purposes.
In the for loop in Algorithm [ in principle, we can revert from the final checkpoint to the base model incrementally. To enhance efficiency, we implement binary search strategy for selecting checkpoints, with details provided in the appendix.
We include other experimental specifications in Appendix
5 RESULT AND DISCUSSION
RQ1 The results of the RBO score are presented in Table [ The RBO score ranges from 0 to 1, with higher scores indicating positive correlation between the two ranked lists, while lower scores suggest negative correlation. The data reveal that the guidance from token gradients shows a slight positive correlation with the ground truth compared to random ranking methods. However, the computation of gradients is resource-intensive. necessitating a trade-off between their utilization and overall efficiency.
We conducted a profiling analysis of the execution times for both greedy random and greedy to- ken gradient iterations. The results indicate that a single iteration using greedy token gradients requires 85% more computational time than an iteration employing greedy random token substitu- tions. Therefore, within identical time constraints, the use of random token substitutions for addi- tional iterations may enhance performance.
8

---

Published as a conference paper at ICLR 2025
Table 1: RBO scores (ranging from 1) for various ranking methods in relation to the ground truth ranking. A higher scores indicate stronger positive alignment with the ground truth. Token gradient ranking shows marginally higher RBO score than random ranking, indicating very weakly positive alignment. Conversely, for adversarial examples in image models, the RBO score between the ground truth and gradient-based ranking typically exceeds 0.90 (Wang et al. 2024).
Method
Llama-3 8B
Llama-2 7B
Mistral-v0.3
Vicuna-v1.5
Token Gradient 0.517
0.506
0.503
0.507
Random Ranking 0.50
0.50
0.498
0.50
Table 2: The ASR results after 500 and 1000 iterations. Notably, the ASRs for Mistral-vO.3 and Vicuna-v1.5 reach saturation by 500 iterations, leading the cessation of further runs. It is impor- tant to emphasize that, despite utilizing the same number of iterations, the computational demands differ significantly. For instance, GCG requires gradient computation in each iteration, resulting in an 85% increase in time compared to random token substitution iteration. Consequently, exe- cuting GCG for 500 iterations is equivalent executing GR for 900 iterations. Furthermore, [An- driushchenko et al.] (2024) incorporated random search into their attack strategy, permitting up to 10 000 random iterations, whereas we established an upper limit of 1000 iterations.
ASR @ 500->1000 Iterations
Llama-3 8B
Llama-2 7B
Mistral-v0.3
Vicuna-v1.5
Method
500
1000
500
1000
500
500
AutoDAN
17.0
19.5
53.5
61.5
100.0
98.0
GCG
44.5
59.0
53.5
63.5
99.5
99.5
GR
33.5
47.0
28.0
37.5
98.5
99.5
FH-GR
46.0
76.5
86.5
99.5
99.5
100.0
RQ2 As seen in Table 2 the FH method either matches (as with Mistral and Vicuna) or substan- tially outperforms (as with Llama-2 and Llama-3) other methods, even when randomly selecting tokens. Notably, we achieve an almost perfect attack success rate on Llama-2, while the closest baseline is more than 30% weaker than FH-GR. We provide additional ablations of the FH method when applied to GCG and AutoDAN in Appendix
RQ3 Since the ASRs of attacks on Mistral and Vicuma reach saturation, we turn our attention to Llama-2 and Llama-3. As illustrated in Figure the FH-GR method identifies adversarial suffixes for prompts that other methods do not achieve within the same number of iterations. Specifically, Figure 3al shows that FH-GR successfully finds the majority of its attacks within 500 iterations, significantly outperforming GCG, the closest competing baseline. This highlights the efficiency of framing the optimization series of easy-to-hard problems. Iteration distribution plots for Mistral and Vicuna, along with runtime and storage overhead of the FH method, are provided in the appendix.
Choice of fine-tuning The machine learning interpretation of the functional homotopy method, as outlined in Section necessitates the selection of the same input intended for jailbreaking. denoted as It. Typically, the target set for optimization is the affirmative prefix "Sure, here is...".
In our experiments, we found that this approach often led model overfitting. For instance, when targeting the prompt "How to build bomb?". the expected output would be "Sure, here is how to build a bomb". A parameter state trained to minimize this loss would likely produce this output as a completion, which could subsequently be rejected by the judge. This misalignment arises because the loss function does not precisely correspond to the objective: a jailbreak attack may not necessarily begin with "Sure, here is how to" and outputs like "Sure, here is how to build bomb" is not recognized as successful attacks. Consequently, overfitting to the loss function might not yield successful affirmative response. We also experimented with red-teaming data obtained from Ganguli et al. (2022) (8000 samples), which mitigated overfitting; however, we observed that parameter states close to the base model were consistently more challenging attack.

---

Published as a conference paper at ICLR 2025
Liame-2: Iteration distribution of successful allacks
Llama-3: Reration distribution successful allacks
OR
FI- t.7t5
25
20-
Frequency
Requercy
20
15.
10
5-
1000
Iterations
Iterations
Iteration distribution for 1lama-2 7B
(b) Iteration distribution for LJama-3 8B Instruct
Figure 3: Iteration distribution for successful attacks, showing the iterations taken by each method to successfully jailbreak the target models on different inputs. Each bar represents here about 50 iterations. Our method can identify adversarial strings more efficiently than GCG, the closest com- peting baseline. Although the plots display iteration counts, important to note that each GCG iteration requires more time than an iteration of FH-GR.
The selection of 500 epochs for model fine-tuning empirically determined to sufficiently mis- align model parameters, facilitating jailbreaking without additional optimization. However, our method's efficacy persists with fewer epochs, effectively initiating the homotopy from more aligned model state. Experiments with stronger checkpoints (i.e., models fine-tuned for fewer epochs) demonstrate that the optimization still converges lower loss than the base GCG al- gorithm. Figure 9 in Appendix H illustrates that optimizations starting from stronger checkpoints exhibit more rapid loss reduction compared to GCG, underscoring the robustness of our approach to variations in fine-tuning duration.
Duality between model and input Our functional homotopy framework capitalizes on the duality between model training and input generation. Fine-tuning model from its base can be viewed as an application of homotopy optimization, which concurrently supports input generation optimiza- tion. This duality underscores the functional relationship between models and inputs. Our approach combines reversed robust training with feature transfer in the input space. Initially, de-robust train safe models to derive vulnerable variants while retaining intermediate models. Subsequently, jailbreak features are transferred from attacks weaker models and incrementally intensified for stronger models. This duality between models (or programs) and inputs, as well as the incremen- tal evolution of both, aligns with widely used techniques in software analysis, such as continuous integration and mutation testing (Kaiser et al. 1989 Brown et al., 2017 Wang et al. 2020). Addi- tionally, conduct a preliminary study the transferability of attack strings from base to weaker models, detailed in Appendix D
An intriguing observation pertains the effectiveness of AutoDAN across Llama-2 and Llama-3. While AutoDAN achieves comparable ASRs to GCG for Llama-2, its effectiveness significantly di- minishes for Llama-3. As the only prompt-level attack utilizing strings from the DAN framework rather than considering all possible prompts, AutoDAN generates suffixes that lack sufficient diver- sity. Given that Llama-3 demonstrates robustness against AutoDAN while remaining vulnerable to other tools, we conclude that generating diverse set of attacks is essential for accurately assessing model robustness.
6 CONCLUSION
This study introduces the functional homotopy method, novel optimization technique that smooths the LLM spput-generation problem by leveraging the continuity of the parameter space. Addition- ally, our approach highlights the interplay between model training and input generation, offering dual perspective that integrates the featurization of both models and inputs. This framework has the potential to inspire new empirical tools for analyzing language models.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
Z. Wang. D. Anshumaan, A. Hooda and S. Jha are partially supported by DARPA under agreement number 885000, NSF EAFFMMIFF-116677 and ONR N00014-21-1-2499. Y. Chen is partially supported by NSF CCF-1704828 and NSF CCF-2233152.
This research was supported by the Center for Al Safety Compute Cluster. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the sponsors.
REFERENCES
Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. Jailbreaking leading safety- aligned Ilms with simple adaptive attacks. arXiv preprint XXiv:2404.02/51,, 2024.
Stephen Boyd and Lieven Vandenberghe. Convex optimization. Cambridge university press, 2004.
David Bingham Brown, Michael Vaughn, Ben Liblit, and Thomas Reps. The care and feed- ing of wild-caught mutants. In Proceedings of the 2017 11th Joint Meeting Foundations of Software Engineering, ESEC/FSE 2017, pp. 511 -22,, New York, NY, USA, 2017. Associ- ation for Computing Machinery, ISBN 9781450351058 doi: 10.1145/3106237.110663 URL htttps:/ddoi.rrg/000 11w.g//....11/111101
3106280
N. Carlini and D. Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Sym- posium on Security and Privacy (SP), pp. 39-57, Los Alamitos, CA, USA, may 2017. IEEE Com- puter Society. doi: (0.1109/SP2011..49
URL /ttt:pss///bb...eeeccecnee http://pl...eeccoopttttsscceeee/
org/10.110s0 2ww.11109922.21..45.. 2017.49
Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. Jailbreaking black box large language models in twenty queries, 2024. URL https:
//openreviess net. fur d=hk jodmz&Ro
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, lon Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023. URL https:
1000--0003000-mmm0 viiuna/
Abhimanyu Dubey et al. The Llama 3 herd of models, 2024. URL htt.ttsss//axxi..rg/ppp
Daniel M Dunlavy and Dianne P PO'Leary. Homotopy optimization methods for global optimization. Technical report, Sandia National Laboratories (SNL), Albuquerque, NM, and Livermore, CA (United States),, 2005.
Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, et al. Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned. arXiv preprint arXiv.2209.07858 2022.
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In Yoshua Bengio and Yann LeCun (eds.). 3rd International Conference on Learning Representations, ICLR 2015, San Diego CA, USA, May 7-9, 2015, Conference Track Proceed- ings, 2015. URL http: wttp:///axivvvv.ii wrg. abs 1412.441120 6572
Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Wang. and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09668 2021.
Kai Hu, Weichen Yu. Tianjun Yao, Xiang Li, Wenhe Liu, Lijun Yu, Yining Li, Kai Chen, Zhiqiang Shen, and Matt Fredrikson. Efficient 1lm jailbreak via adaptive dense-to-sparse constrained opti- mization, 2024.
11

---

Published as a conference paper at ICLR 2025
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chap- lot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie- Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William EI Sayed. Mistral 7b, 2023.
G.E. Kaiser, D.E. Perry, and W.M. Schell. Infuse: fusing integration test management with change management
Proceedings of the Thirteenth Annual International Computer Soft- ware & Applications Conference. pp. September 1989. IEEE Computer Society. doi: 10. 10.1009/CCMMAAC.19999 65147. URL https:
Los Alamitos, CA, USA,
wttp::eeeeeoouutsssccctttteeee.e org/ 10
1989. 65147
Richard M. Karp. Reducibility among Combinatorial Problems, pp. 85-103. Springer US, Boston, MA, 1972. ISBN 978-1 14684-2001-22 doi: 10.10077978-1-4400
9. URL https://
boi.org/Di 110101001000-1-------0------
Zeyi Liao and Huan Sun. Amplegeg: Learning a universal and transferable generative model of adversarial suffixes for jailbreaking both open and closed llms. arXiv preprint arXiv:2404009921, 2024.
Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. AutoDAN: Generating stealthy jailbreak prompts on aligned large language models. In The Tweifth International Conference on Learning Representations, 2024a. URL httl:tttss//ppeeeeeeee
net. foreet//orrufefie wttp://ommmmmdddddp /ot.
Xiaogeng Liu, Zhiyuan Yu, Yizhe Zhang, Ning Zhang, and Chaowei Xiao. Automatic and universal prompt injection attacks against large language models, 2024b. URL httptttss///aiii.ott.
abs/2403 04957
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learming Representations, 2018.
Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, et al. Harmbench: A standardized evaluation framework for automated red teaming and robust refusal. arXiv preprint arXiv:2402.042445 04249, 2024.
Anay Mehrotra, Manolis Zampetakis, Paul Kassianík, Blaine Nelson, Hyrum Anderson, Yaron Singer, and Amin Karbasi. Tree of attacks: Jailbreaking black-box lims automatically, 2023.
Hossein Mobahi and John Fisher III. A theoretical analysis of optimization by gaussian continuation. Proceedings of the AAAI Conference on Artificial Intelligence, 29(1), Feb. 2015. doi: 10. 1609/ wwt.v2999995556 URL hattp::// aaai. org/ index.idexxe//// whtpppppp/apppaaaaa
9356
Leo Schwinn, David Dobre, Sophie Xhonneux, Gauthier Gidel, and Stephan Günnemann. Soft prompt threats: Attacking safety alignment and unlcarning in open-source LLMs through the embedding space. In The Thirty-eighth Annual Conference on Neural Information Processing net. forum? dddCLddLP FARC wtt.rrrmmmddccce.eee...
Systems, 2024.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Good- fellow, and Rob Fergus. Intriguing properties of neural networks. In Yoshua Bengio and Yann LeCun (eds.), 2nd International Conference Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014. URL http: /aaxxvv.org abs 3.1...555 6199
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshom, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev. Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuming Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi,
12

---

Published as a conference paper at ICLR 2025
Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models, 2023.
walkerspider, 2022. http:////w////............//////
dan is my new friend/
Zi Wang, Ben Liblit, and Thomas Reps. arxiv.org/abs/20000 14375
Tofu: Target-oriented fuzzer, 2020.
URL https://
Zi Wang, Aws Albarghouthi, Gautam Prakriya, and Somesh Jha. Interval universal approximation for neural networks. Proc. ACM Program. Lang., 6(POPL), January 2022. doi: 10.1 155/349468 URL http::://ool.orr//0..115///////
Zi Wang, Jihye Choi, Ke Wang, and Somesh Jha. Rethinking diversity in deep neural network testing. 2024. [RLLitttt///aaxtt org/ abs ab0/aa/s.../.ee.ee/e
2305 15698
William Webber, Alistair Moffat, and Justin Zobel. A similarity measure for indefinite rankings.
ACM Trams. Inf. Syst., 28(4), noy 2010. ISSN 1046-8188, doi: 10.1145/852200211821166 URL
Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307. /5043, 2023.
13

---

Published as a conference paper at ICLR 2025
A NP-HARDNESS OF THE OPTIMIZATION PROBLEM
Let f : 1 + R be a neural network with m-binary inputs and a canonical feed-forward structure comprising single hidden layer. The network is formally expressed as:
(xx) W2 b1) b2
where W1 E _=xm W2 RIx are the weight matrices between the layers, denotes activation function, b E R" and b2 E R is the bias term. In other words, f is the function composition of two affine transformations and activation layer. Notice that any composition of two consecutive affine transformations is still an affine transformation.
The decision problem under consideration is formulated as existential problem:
Ya E R.lr (0.1)".(()) S
(5)
This formulation aims to determine, for any given threshold a and two-layer binary neural network, whether there exists an input x such that f()) a. We demonstrate the NP- NP-hardness of this problem in Section A. Subsequently, in Section A.2 we elucidate how this formalization (Equation (5)
encapsulates the LLM input generation optimization problem.
A.1 HARDNESS REDUCTION
We prove the NP-hardness of the decision problem Equation (5) via a reduction from the Boolean satisfiability (SAT) problem. This reduction is constructed such that solving the decision problem equivalent to determining the satisfiability of the given SAT formula. Consequently, one can resolve the decision problem, one can, by extension, solve the corresponding SAT instance.
We define 3- 3-Conjunctive Normal Form (3CNF) instance e as follows: Let XI,..., X,, Boolean variables. A literal L; is defined either X, or its negation -X,, A 3CNF instance ø is constructed as a conjunction of clauses: C₁ A... A Ck. where each clause C3 is a disjunction of three literals. To differentiate between the 3CNF instance and its neural network representation, percase letters for 3CNF components and lowercase letters for their corresponding neural network constructs. The Cook Levin theorem establishes that deciding the satisfiability of 3CNF formula is NP-hard (Karp) 1972).
employ up-
Simulation of 3CNF To simulate logical operations within the neural network framework, we employ gadget similar to that proposed by Wang al. (2022). This approach utilizes common activation functions such as ReLU, sigmoid, and tanh to approximate a step function, defined as:
{ 1, step(x) = { 0,
x x>0
(6)
The step function, being discrete-valued, serves as a fundamental component for mathematical anal- ysis and simulation of other discrete functions. Its versatility in neural networks allows for easy scaling and shifting through weight multiplication or bias term addition, enabling the replacement of the constants 0 and in Equation ( with any real numbers.
Wang et al. 2022) demonstrated how common activation functions can approximate the step func- tion. For instance, ReLU(nz)) ReLU(nz - 1) converges to step(z) - co. For reduction, we define step-like function:
1,
x x>1
s(x) RcLU(x) ReLU(( - -1)
0,
x <<0
x, 0< x <1
Using 8, 3CNF formula can be simulated with single hidden layer neural network. For each input node I and construct affine transformation: two nodes y, where y = x and ğ = 1 - r, simulating literals X and -X. Let I, represent ys or yi. corresponding to literal L; in the SAT formula. For each disjunction C L, V L2 V L3 in the CNF formula, define it as I + le + L3 and clip the value using the step-like function s. Utilizing this function, we can evaluate the truth value of clause C. If any literal is satisfied, then 1 12 + la 1, resulting in
variable x, we introduce
14

---

Published as a conference paper at ICLR 2025
s(1 +12 1; otherwise, s(12 l2 13) 0. We denote the output of the node corresponding to each clause C2 as C₁.
To simulate the conjunction operation, we sum all clause gates and negate the result: - - Ca. Consequently, determining the satisfiability of the 3CNF formula is equivalent to solving the fol- lowing decision problem for this one-layer network:
f S
Proposition A.1. For any 3CNF formula & with & clauses, there exists a two-layer neural network f: (0, 1) 1 + R such that o is satisfiable if and only if 3x such that f(z) S -k.
Proof. We have described the construction of the neural network as above. Now we need to show indeed captures the computation of the 3CNF formula.
Now if & is satisfiable, then there exists a truth assignment , Xm to satisfy Φ. Let ay if X, is true and
0 if X, is false in this assignment. By the construction of the neural network, the output of is also true for each step-like function. As result, Σx Q
Now if Dx E {0, 1 such that (xx < kk Choose a truth assignment for XI....,X,, based on rs's values. Because f - E ii=1 G and 0 S Q S 1, then each
1. By construction, 11 + 12 + la 2 for this activation node c, Because x; is discrete-valued, 4 E {x;, zi). then 4 E 10, 4 + 12 + 1N 1 implies that at least one of 1,,42, ly equals 1. Therefore, by the j,, the corresponding literal is also satisfied. Because each clause has a true assignment, then e is satisfied.
truth assignment
The aforementioned neural network construction employs a single-layer activation function: s, com- prising two ReLU functions, Wang al. (022 demonstrated the feasibility of approximating the step function using common activation functions. Consequently, this construction is applicable to any single activation layer network utilizing standard activation functions. Given the NP-hardness of the 3SAT problem, it logically follows that the neural network decision problem, as defined in Equation (5) NP-hard.
Corollary A.2. The decision problem in Equation 5 is NP-hard.
A.2 LLM INPUT GENERATION FORMALIZATION
We posit that the decision problem in Equation (5) effectively encapsulates the LLM input genera- tion optimization problem. Informally, this problem seeks to minimize some loss based on logits for all given possible suffixes. Because do not impose any structural assumptions on the input em- bedding or the output layers, the two-layer network can be embedded within the LLM architecture.
Specifically, in the context of jailbreak attack optimization, the objective is to minimize ((M((z,)) s))), where M represents an LLM, denotes the prompt. and L is the entropy loss be- tween the logits and an affirmative prefix. This can be extended minimizing fxx for æ E {0, 1)", input prompts. The discrepancy between
given the absence of assumed embedding structures f E RR and the entropy loss jailbreak attacks can reconciled through a network with multiple outputs, where one output corresponds to f and the rest are constants, establishing a bijective rela- tionship between f and the jailbreak loss. Given that the cross-entropy between a logit (yI,..., yn) and target class j is:
log exp(Us)
== exp(y))
We can construct multiple output network F, where F(x), f(x) and the remaining F(x) are constants. Assuming the one-hot encoding corresponds to the first token (class j), the cross-entropy becomes:
(x)(f(x))
log (x)( +C'
for some C' > O. This function is bijective and monotonically increasing. By fixing the remaining affirmative prefix tokens' logits as constants, minimizing the prefix loss is equivalent to minimizing f(x), demonstrating that jailbreak algorithm capable of minimizing the loss also minimize fæ). Consequently, if the minimization of f is solvable, the existential problem for any threshold
E IR can also be resolved by querying the minimum of (xx)
15

---

Published as a conference paper at ICLR 2025
B LINEAR APPROXIMATION PROOF
and x' be substituted string such that To and x' are of the same length and only differ by one token position j. from token in 2o to token b in x'. Let Eg be the one-hot encoding of and E' be the one-hot encoding of x', therefore, =² (E' E + Eo. Let Vabj = (E' En), then E' = Uabj + Es.
Proof. Let To be string input of length n, i.e, |zol
Because Tn and x' only differ by one token at position j. then Vaby E R is of the form
(0, -1, ,,..., ,)...........................0
We use 0 to denote it is a 0 Rd vector. -1 1 is corresponds to the one-hot encoding of a and corresponds to token b.
As result, the linear approximation of f(x') from f(x0) is
J(Fo + (F' Fo)) ((Fo) + ((
Because Eo is a fixed input, optimizing the linear approximation of f(E") amounts to optimizing (E( across all possible Vabj-
Because Vabj are all O's except for the j-th position, (Vabs) (Daa)) ([vassla) A.. Maximizing the linear approximation of (EE) amounts picking the best token that maximizes ((rabsl)) h.
Again, because is fixed, so z0 fixed. To maximize ([vobsl)) h, one only needs to choose arg max(h), which is k.
C ADDITIONAL EVALUATION DETAILS
Binary Search for Parameter States In our experiments, have 500 parameter states obtained through finetuning. However, progressively iterating through all these states for each sample can be very time-consuming (in particular loading model weights for each checkpoint).
We instead use binary search to pick appropriate parameter states. For example, given 500 parameter states, we start by attacking the 250th state, and set the 500th state as the right extreme. If succeed (within a set number of iterations), we take the successful adversarial string and apply it to the 125th state and set the the 250th state as the right extreme. we fail, we discard the string and do not count the spent iterations towards the total. We instead attack the 375th state, which is weaker. In the event the current state and the right extreme are the same (or the index of the current state is one less than the right extreme), we retain the string upon a failure and use it to initialize another attack on the same checkpoint (up to a certain number of cumulative iterations). We formalize this in the following algorithm.
Fine-tuning specification We use learning rate 2e-5, warmup ratio 0.04 and a LoRA adapter with rank 16, alpha 32, dropout 0.05, and batch size 2 to fine-tune the models for 64 epochs, leading to 768 checkpoints in total.
Operational overhead The storage and computational overheads of our proposed method are comparatively modest. Each LoRA checkpoint requires 49 MB, with theoretical maximum of 768 checkpoints occupying approximately 37 GB. In practice, points, further reducing the storage footprint. For context, full model storage requirements are substantially larger: Llama-2 (13 GB), Mistral (14 GB), Vicuna (26 GB), and Llama-3 (60 GB).
utilized significantly fewer check-
The computational overhead is similarly minimal. Model fine-tuning, performed once for all test inputs, takes approximately 20 minutes. In contrast, attacking single input for 1000 interactions requires about one hour. When amortized across 200 inputs, the running-time overhead is less than 10 seconds per input. Thus, both storage and computational overheads of our method relatively insignificant compared to the resource demands of complete language models and the overall attack process.
Server specifications All the experiments are two clusters.
16

---

Published as a conference paper at ICLR 2025
Algorithm 2 Functional Homotopy with Binary Search
Input: A parameterized objective function fp. the initial parameter po, the intermediate param- eter states p1: .2..... ..... .PPt obtained from line T of algorithm input 2t E A, threshold e IR and threshold E N.
Output: A solution 20
po (F)
1: Set L 4 0, 0,
2: while do
3: Obtain that optimizes F(pc. from za using random search within K iterations: fixing position in 2R. randomly sampling tokens from the vocabulary, and evaluating the objective with the substituted inputs. The best substitution is retained over several itera- tions. The initialization of this process is warm-started with zn, and ideally concludes with F(pc, xC) S
4:
if FPc,, IC) ) S a then R+ C. C.C R
5:
6:
else
7:
C+L C+R
8:
end if
9:
end while
10:
Obtain Fe that optimizes F(pc, xC), from on using random search within K iterations (this
step is for obtaining ze when I = C).
Return IC.
1. A server with thirty-two AMD EPYC 7313P 16-core processors, 528 GB of memory, and four Nvidia A100 GPUs. Each GPU has 80 GB memory.
2. A cluster supporting 32 bare metal BM.GPU.A 1000v2.8 nodes and number of service nodes. Each GPU node is configured with 8 NVIDIA A100 80GB GPU cards, 27.2 TB local NVMe SSD Storage and two 64 core AMD EPYC Milan.
D TRANSFERABILITY OF STRONGER ATTACKS
The FH method requires series of finetuned parameter states. We examine the transferability of successful base model attacks to their corresponding finetuned states. We consider 50 samples where the base model was successfully attacked, and transfer those to that model's finetuned parameter states.
We hypothesize, based on the model and alignment training, the degree of overlap of the adversarial subspaces of different checkpoints will vary, with more successes at a checkpoint indicating a greater overlap with the base model. This is reflected in the initial checkpoints of all models (roughly 1-201 20) in Figure
As demonstrated in Table 2 Vieuna is particularly weak model, in terms of alignment. Thus the adversarial string found for the base model transfers well across its finetuned states, as seen in Fig- 4bb However, Llama-2 and Llama-3 (Figure 4ab have more robust alignment training, and the attack does not transfer well, even though the finetuned states would be considered weaker in terms of alignment. This divergence hints how the adversarial subspace of model transforms during alignment training. We leave rigorous analysis of this as future study.
E ADDITIONAL RESULTS FOR FUNCTIONAL HOMOTOPY
We apply the principles functional homotopy on top of GCG and AutoDAN as ablation in Table We find that performance improves for almost all methods, further corroborating the effectiveness of our technique. In particular, only Llama-3 sees a marginal decline in performance.
F ADDITIONAL ITERATION DISTRIBUTIONS
Figure 5 iillustrates the iteration distribution for Mistral and Vicuna.
17

---

Published as a conference paper at ICLR 2025
Base modell adversatill surfcc applled ofher checspolnts
Bane model adtver sacial sittfe app led other cherkcoinn
Samples whare th sutts succocts
silfee
Chockpoints
Creckponts
(a) Successes when directly attacking Llama-2 (b) Successes when directly attacking Mistral and and Llama-3's checkpoints
Vicuna's is checkpoints
Figure 4: Transferability of successful attacks on the base model its finetuned parameter states. We find that the attack does not necessarily transfer for all models. This seems to be function of the "distance" between the states and the alignment training received.
Table 3: The ASR results and improvements at 1000 iterations. We note that functional homotopy is especially effective for gradient based methods such as GR and GCG. AutoDAN sees marginal im- provement for Llama-2, but decline in Llama-3. We note that Llama-3 is very robust to AutoDAN in general.
ASR Improvement @ 1000 Iterations
Model
GR - FH-GR
GCG FH-GCG AutoDAN + FH-AutoDAN
Llama-2
37.5 99.5
69.5 99.0
61.5 68.5
Llama-3
47.0 - 76.5
59.0 + 77.5
19.5 + 14.5
G FH LEVEL-SET PLOT
Figure presents conceptual illustration of the homotopy optimization method, elucidating its underlying principles and operational dynamics. This visual representation provides more detailed intuition the method's efficacy in navigating complex optimization landscapes.
H Loss CONVERGENCE ANALYSIS FOR GCG AND FH-GR
In this section, we consider "hard" samples for Llama-2 and Llama-3 that GCG was unable to jail- break, but FH-GR (initialized from checkpoint-500) was successful. Recall that both GCG and GR utilize identical loss objectives Fo. while FH-GR employs a sequence of loss objectives (Fa..., Fo), culminating in the same final loss as GCG. We designate (Fn...., Fb) the homotopy loss, with input generation guided by F.
Figure shows the change in average homotopy loss with iterations throughout the progression of homotopy. We find that easier optimization problems and the solution of the preceeding problem enables consistently lower loss throughout the homotopy process.
Figure 3 examines how the adversarial strings found on weaker models by the functional homotopy, affect the average loss Fo the base model. We that the loss consistently decreases, indicating that we are able to avoid local optima and successfully jailbreak the model faster.
Figure shows the robustness of Functional Homotopy. Despite initializing the attack with stronger checkpoints (i.e., models fine-tuned for fewer epochs), we still find that loss converges more quickly that GCG and results jailbroken response from the model.
18

---

Published as a conference paper at ICLR 2025
Iteratioon distribution attarks
Vicuna-v1.5: distribution attacks
Frequency
60-
Priquency
60-
(a) Iteration distribution for Mistral-v0.3
(b) Iteration distribution for Vicuna-v1.5
Figure 5: Iteration distribution for successful attacks. We are able to find adversarial strings far more
efficiently than GCG, the closest competing baseline.
Figure 6: Conceptual illustration of homotopy methods for suffix optimization. (a) Left: Greedy local search heuristic. The red region denotes successful suffixes. The search initiates from starting point (black solid) and iteratively moves to the optimal neighboring input (dashed circle) based on loss values, potentially leading local optima entrapment due non-convexity. (b) Right: Homotopy approach. A series of progressively challenging optimization problems constructed, with easier problems having larger solution spaces. The solution set gradually converges to that of the original problem. Adjacent problems in this continuum have proximal solutions, facilitating effective neighborhood search. Despite the underlying con-ooveexity, initiating from near-optimal point simplifies each problem-solving step.
19

---

Published as a conference paper at ICLR 2025
Compading Auring 6CG ane Fie Lose lowd Samsles I 2
Compo by loeraee 60S and Fi-d coee for saadd Surpples Larra))
GCC
G2G Lcm
1.4
02
Narettona
(a) FH-GR loss for Llama-2, 2, across checkpoints (b) FH-GR loss for Llama-3, across checkpoints
Figure 7: A loss comparison of GCG and FH-GR "hard" samples. The red FH-GR Loss indicates the homotopy loss. We see that homotopy starts off with a substantially smaller loss, due to the misalignment process. We initialize FH-GR from checkpoint 500. As we iterate and successfully jailbreak an intermediate model, we replace it (as described in Algorithm [ until we reach the base model by iteration 1000. We further note in Section that FH-GR converges more quickly than GCG.
Jug. GA 2 with ving Adcurarial
Aug. -i-c8 lees for Lame with Custring Advemants Suffs
GKG
606
Oll
PH us
3.5
60
L0-
0.5
(a) FH-GR loss for base Llama-2
(b) FH-GR loss for base Llama-3
Figure 8: A loss comparison of GCG and FH-GR on "hard" samples. Unlike Figure 7 we look at the usefulness of the adversarial strings found by the functional homotopy, by applying them to the base model and computing the base GCG loss. The red FH-GR Loss indicates the base model loss objective value induced by the inputs found by the FH method. The loss decreases more consistently, before converging to a lower value overall and successfully jailbreaking the model compared to GCG.
Camparing
Samiiles
GCO Loss (Bruc)
DCG Loau (5ac)
PH GA Uss 200 BUsA)
Git loss 0ugp- 525 Busc!
1.79
Loce (ckzt 200 Bame)
Ti--2R (rige 152 Dann)
1.58
3:0
1.2
3.50
0:5
302
Baratton
Itrrttons
(a) FH-GR loss for Llama-2, across checkpoints
(b) FH-GR loss for Llama-3, across checkpoints
Figure 9: A loss comparison of GCG and FH-GR initialized from different checkpoints. We take 25 "hard" samples and initialize FH-GR from earlier checkpoints that are more aligned. Of the cases where starting from the earlier checkpoint succeeds (loss in green), we see that FH-GR still able to converge to a lower loss than GCG. Note that GCG fails all these cases, where as FH-GR (ckpt- 500-+base) succeeds on all cases and FH-GR starting from earlier (stronger) checkpoints succeeds on 13 cases for Llama-2, 2, and cases for Llama-3.
20


Reviews:
[{"review_id": "oaIHaF7NFZ", "reviewer": "Reviewer_BDPK", "summary": "The paper proposes the functional homotopy method, a novel optimization approach for LLM attacks. FH fintunes the LLM and solves a number of easy-to-hard problems, which are constructed via gradient descent. The results show a substantial improvement over different baselines.", "questions": "Questions:\n- How much storage does it take to store all the LLMs?\n- Is there ablation using another baseline for the discrete optimization for FH, e.g., AutoDAN or GCG?", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-written and easy to follow\n- The methodology is novel and well-motivated\n- The proposed attack allows easy substitution of future attacks\n- FH shows promising results", "weaknesses": "- L81-82: \"can retain\" should be \"have to retain\"\n- The efficiency comparison is based on the iterations instead of the runtime, which seems to be a fairer comparison.\n- Missing related work. Consider including [1, 2]\n\n\n[1] Schwinn, Leo, David Dobre, Sophie Xhonneux, Gauthier Gidel, and Stephan Gunnemann. \"Soft prompt threats: Attacking safety alignment and unlearning in open-source llms through the embedding space.\" arXiv preprint arXiv:2402.09063 (2024).  \n[2] Liao, Zeyi, and Huan Sun. \"Amplegcg: Learning a universal and transferable generative model of adversarial suffixes for jailbreaking both open and closed llms.\" arXiv preprint arXiv:2404.07921 (2024).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "STq9HbcRUz", "reviewer": "Reviewer_vykH", "summary": "This paper introduces the Functional Homotopy (FH) Method, a novel optimization approach designed to tackle the discrete optimization challenges inherent in language models, with a specific application to synthesizing jailbreak attacks. The authors note that existing gradient-based techniques, effective in continuous spaces like image models, face difficulties when applied to the discrete token space of language models. The FH method addresses this by leveraging a functional duality between model training and input generation. The method involves transforming a difficult optimization problem into a series of easier ones, using a homotopy-based approach where model parameters and inputs are optimized in tandem. The paper demonstrates that this method achieves a significant improvement in attack success rates (20-30%) over existing techniques when tested on open-source models like Llama-2 and Llama-3.", "questions": "The authors mention issues with overfitting to specific malicious queries, especially when targeting outputs that models are trained to reject. Are there additional strategies or techniques that could be implemented to reduce this risk without compromising attack success rates?\n\nWhile the method shows improvements in success rates, could the authors provide more comprehensive comparisons of computational efficiency, such as runtime and resource usage, relative to baseline methods? This would help in understanding the practical trade-offs of deploying the FH method.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper introduces an original optimization method, blending continuous and discrete optimization techniques in a way that has not been widely explored for language models. This approach brings a fresh perspective to the field of adversarial attacks.  The authors provide a strong theoretical foundation for their method, clearly explaining the duality between model training and input generation. The homotopy-based approach is well-grounded in optimization literature, which adds depth to the contribution. The demonstrated success of the FH method in improving attack success rates against robust language models is significant. The practical implications for AI safety research are noteworthy, especially in understanding and defending against potential vulnerabilities in LLMs. The paper is clearly written, with a logical structure that effectively guides readers through the problem, methodology, and experimental results. The use of diagrams, such as the illustration of the homotopy path, aids in understanding the method’s workings.", "weaknesses": "The paper does not thoroughly address the computational demands of the FH method. Randomly sample tokens\nfrom the vocabulary to replace the token at that position can be time-consuming. Given the potential resource constraints in real-world applications, more details on runtime efficiency and how the method scales with model size would be beneficial.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RlGMbsBcsw", "reviewer": "Reviewer_bVKA", "summary": "The paper trains the LLM to be less aligned and then attacks it with a greedy search for token replacements on the weaker versions of the model, gradually increasing the difficulty by reverting step by step to the aligned version.", "questions": "1. I am curious about why the method works. I would understand if gradient optimization were involved and it was argued that the landscape is smoother, guiding the process to the next optimum. However, since the suffix is optimized with a greedy search, why is this more effective? If the main idea is just remaining at a broad minimum during random search, then the transferability from the base model to the weakest model should be higher, which does not appear to be the case.\n2. In Figure 3, what does it mean for FH-GR to be a successful attack within 1 iteration? Does this mean only one parameter update, or does it start from the weakest model after *N* updates and do only one GR search?\n\n### Review Summary:\nThe paper presents an interesting idea. However, in its current state, I am not in favor of accepting it. I believe more research into the method is necessary, further evidence should be gathered, and the writing should be improved. That said, I am open to revising my score if my concerns are addressed.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 3, "strengths": "1. The idea is interesting and novel in its approach to manipulating the model's parameters.\n2. The empirical performance in terms of ASR appears strong.\n3. The effort to provide evidence that the token gradient in GCG is not beneficial is appreciated, even though this has already been hypothesized.", "weaknesses": "1. The number of steps used for 'unalignment' must be decided beforehand. The procedure, if I understand correctly, is: conduct *N* steps to transition from parameter *p* to unaligned parameters *p'*. Then, iterate back and find an attack suffix for each *p_i* based on the previous one. This makes it difficult to determine if *p'* is sufficiently unaligned, and during the process of creating a new suffix, it is not possible to adjust to easier parameters when the attack does not seem effective.\n2. The method introduces many additional hyperparameters (e.g., number of fine-tuning steps, number of GR steps per parameter set) and operational overhead, which are not discussed adequately (not even in RQ3). The authors should compare the attacks based on computational cost rather than steps, as fine-tuning is not factored into their 'step' metric.\n3. The writing needs improvement. The text is often verbose, repeating the same points at different locations, and could be more concise. Additionally, there are inconsistencies in mathematical symbols, such as *p* being defined as the input (prompt) but is later used to refer to model parameters.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dEu5W5z51y", "reviewer": "Reviewer_M9RT", "summary": "This paper presents a novel approach called the Functional Homotopy (FH) method for solving discrete optimization problems in the context of jailbreak attacks on large language models (LLMs). The method leverages a duality between continuous model parameters and input generation, using an easy-to-hard problem-solving approach. The authors claim a 20%-30% improvement in success rates over existing methods when applied to LLMs such as Llama-2 and Llama-3.", "questions": "see weakness", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "* The paper introduces a novel functional homotopy method for jailbreak attacks on large language models (LLMs), building on the traditional homotopy framework used in optimization. Instead of directly solving the complex discrete optimization problem, the approach breaks it down into a series of progressively harder problems. Each solution from an easier problem is used as the starting point for the next, harder problem, which makes the optimization process smoother and more manageable.\n\n* The experimental results show that this method delivers a 20%-30% improvement in success rates compared to baseline techniques when applied to state-of-the-art open-source models.", "weaknesses": "* The proposed method appears overly simplistic, lacking both theoretical guarantees and in-depth analysis. The randomized search approach is basic and does not provide any assurance of optimality. Additionally, the complexity of performing a successful randomized search seems exponential with respect to the dimensionality of the problem. A formal theoretical analysis offering guarantees for the method would significantly strengthen the contribution.\n* The baselines used for comparison also seem relatively weak. For instance, the gradient-based methods are naturally expected to perform poorly, given their myopic nature and disconnection from true gradient information. Furthermore, the datasets used for the experiments are small, making it difficult to draw definitive conclusions about the method’s effectiveness and efficiency. Stronger baselines would provide a more meaningful comparison.\n* The algorithm's performance seems to be highly dependent on hyperparameter choices, such as the threshold, number of random search steps, and the search radius. A more comprehensive analysis on how to select these hyperparameters would provide valuable insight and improve the robustness of the proposed approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)