Antibody: Strengthening Defense Against Harmful Fine-Tuning for Large Language Models via Attenuating Harmful Gradient Influence
Authors: Quoc Minh Nguyen
Trung Le
Jing Wu
Anh Tuan Bui
Mehrtash Harandi
Venue: ICLR 2026 Poster (2026)
Keywords: Harmful fine-tuning
LLM
safety alignment
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We introduce a novel defense method named Antibody that regularizes the gradient contributions of harmful samples during each update step to mitigate harmful fine-tuning attacks.

Abstract: Fine-tuning-as-a-service introduces a threat to Large Language Models' safety when service providers fine-tune their models on poisoned user-submitted datasets, a process known as harmful fine-tuning attacks. In this work, we show that by regularizing the gradient contribution of harmful samples encountered during fine-tuning, we can effectively mitigate the impact of harmful fine-tuning attacks. To this end, we introduce Antibody, a defense strategy that first ensures robust safety alignment for the model before fine-tuning, and then applies a safety-preservation learning algorithm during fine-tuning. Specifically, in the alignment stage before fine-tuning, we propose optimizing the model to be in a flat loss region with respect to harmful samples, which makes the safety alignment more resilient to subsequent harmful fine-tuning. Then, in the fine-tuning stage, we design a fine-tuning algorithm that applies a weighting scheme to all samples in each training batch to inhibit the model from learning from harmful samples while encouraging learning from benign samples. Experimental results demonstrate that Antibody successfully mitigates harmful fine-tuning attacks while boosting fine-tuning performance on the user-submitted dataset.

Published as a conference paper at ICLR 2026
STRENGTHENING DEFENSE AGAINST HARMFUL FINE-TUNING FOR LARGE LANGUAGE MOD- ELS VIA ATTENUATING HARMFUL GRADIENT INFLU- ENCE
ANTIBODY:
Quoe Nguyen', Trung , Jing Wu", Anh Bui Mehrtash Harandi 'Department of Electrical and Computer Systems Engincering, Monash University, Australia 2 DDpartment of Data Science and AI, Monash University, Australia
ABSTRACT
warttuinnnn-----seveeee introduces threat to Large Language Models' safety when service providers fine-tune their models on poisoned user-submitted datasets, process known harmful fine-tuning attacks. In this work, we show that by regularizing the gradient contribution of harmful samples encountered during fine- tuning, we can effectively mitigate the impact of harmful fine-tuning attacks. To this end, we introduce Antibody, a defense strategy that first ensures robust safety alignment for the model before fine-tuning, and then applies a sifett-preservation learning algorithm during fine-tuning, Specifically, in the alignment stage before fine-tuning, we propose optimizing the model be in a flat loss region with respect to harmful samples, which makes the safety alignment more resilient subsequent harmful fine-tuning. Then, in the fine-tuning stage, we design a fine-tuning algorithm that applies a weighting scheme to all samples in each training batch to inhibit the model from learning from harmful samples while encouraging learning from benign samples. Experimental results demonstrate that Antibody successfully mitigates harmful fine-tuning attacks while boosting fine-tuning performance the user-submitted dataset.
WARNING: This paper may contain offensive and harmful content.
INTRODUCTION
Fine-tuning-ass suuning-as---seeeee as-a-service (FTaaS) (e.g., OpenAI Mis-
tral etc.) is powerful method for adapting Large Language Models (LLMs) user-defined tasks. This service presents several key advantages. For instance, users only need upload their own data, and the ser- vice automatically fine-tunes the model and returns a customized version. This process eliminates the need for expertise in machine learning or access to large computational resources. also allows users to fine- tune proprietary models. However, this accessibility vulnerable to fine-tuning attacks, where a model's safety alignment be subverted by fine-tuning a few harmful samples (Yang et al.] 2023 Yi et al.)
Fine Fine-tuning Accuracy (FA) (9%)
20-
Antioddy (Ours)
18-
16-
H51
14-
0.ow FA)
10-
n-539
Booster
Antbodd
n=1333
a-
Vaccine
n-2590
SFT
30
40
50
Harmful Score (HS) (96)
2024a Qi al. 2024; Huang al. 2025b). The sulting compromised model, when released to the user, could be exploited for malicious purposes. This mo- tivates the need for developing defense methods that are resistant to such harmful fine-tuning attacks while preserving the learning efficacy of the service.
Figure 1: Fine-tuning on GSM8K (Cobbe et al.] 2021) with varying sample sizes and fixed harmful ratio of 20%. Larger sample sizes improve fine-tuning accuracy (higher FA) but degrade model safety (higher HS).
https://platform openai http://oo....oooooaoooooooooaoaa
zat ion
https://docs... mi misttall http:////aaaaaaa////////////e/na

---

Published as a conference paper at ICLR 2026
The standard pipeline for FTaaS consists of two stages: an initial alignment stage, where service provider performs safety alignment on a model, and a subsequent fine-tuning stage, where users submit dataset and the provider applies Supervised Fine-tuning (SFT) to optimize the model for them. The harmful fine-tuning attack happens during the fine-tuning stage, where a user may intentionally unintentionally inject harmful data into their submitted dataset. The resulting dataset typically comprises a small fraction of harmful samples mixed with benign ones.
To mitigate harmful fine-tuning attacks, current studies propose defense strategies for each stage of the FTaaS process Huang et al. 2024b). The first category, alignment stage defenses (Liu et al.
2024b Rosati et al. 2024 Tamirisa et al. 2025 Huang al. 2024c] 2025b Zhao al. 2025b Liu et al. 2024a). focuses on making the base model inherently resilient to malicious data before it is
released to users. These methods proactively create a robust model's safety alignment. For instance, Vaccine (Huang et al.] (204cc) makes the model's internal embeddings of harmful content more resistant to adversarial manipulation, while Booster (Huang et al. 2025b) proposes a regularization term in the alignment stage that aims to minimize the harmful loss reduction in the fine-tuning attack.
The next category, fine-tuning stage defenses (Mukhoti et al. [0224] Bianchi et al. 2024] Zong et al.
2024 Huang al., 2024a Lyu al., 2024 Wang et al. [2224] Eiras et al. [2025]]]] Li al. 2025b
proposes variou strategies to prevent the model from learning from harmful samples while still being
able to learn from benign data. Examples include mixing alignment data into the training process
(Huang et al. 2024a] EEras et al. 22025) or modifying system prompts (Wang et al.) 2024 Lyu et al.
2024] Huang al. (2025a). More recently, post- cost-iinettuning defenses (Casper et al. [025] Yi et al.
2024b Hsu et al. 2024) have been proposed to repair model's safety alignment after the fine-tuning complete. Despite these diverse approaches, many existing methods either provide insufficient
protection against harmful fine-tuning attacks or come at the cost of the model's 's performance on the user's task, as illustrated in Figure []]
In this paper, we propose Antibody. defense strategy that attenuates the influence of harmful gradients through an integrated two-stage framework implemented across the alignment and fine- tuning stages. Specifically, in the alignment stage before harmful fine-tuning takes place, our key idea is to build a robust safety alignment by flattening the loss landscape w.r.t. potential harmful samples, thereby making the instilled safety behavior more difficult to remove, even when the model is subsequently fine-tuned on new harmful samples. Then, in the fine-tuning stage, we leverage the safety alignment knowledge already embedded in the model after the alignment stage to introduce a dynamic weighting scheme that down- weights the contribution of harmful samples during fine- tuning. Empirical results show that Antibody successfully mitigates harmful fine-tuning attacks while offering a competitive fine-tuning performance across various settings. See Figure for an example of performance gain.
Our contributions are summarized as follows:
Robust Alignment. We propose to optimize the model to be in a flat loss region with respect to harmful samples, which makes safety alignment harder to be removed.
Safety Fine-tuning. We propose a safety fine-tuning method that applies weighting scheme to all samples in each training batch to prevent the model from learning from harmful samples while encouraging learning from benign samples.
Extensive Evaluation. We validate Antibody performance across different downstream datasets, model architectures, and finettuning setups.
2 RELATED WORKS
Aligument Stage Defenses. Alignment stage methods aim to make a model more robust against fine-tuning attacks by modifying the alignment process. Vaceine (Huang et al. 2024c) aims to reduce harmful embedding drift by adding perturbations to the model's embeddings during the alignment stage. RepNoise (Rosati et al. 2024) further proposes to utilize a harmful dataset to align the model so that its harmful embeddings move toward random noise, thus removing harmful information that can be extracted from those embeddings. TAR (Tamirisa et al. 2025) applies meta- meta-learning to sustain high loss in harmful samples after harmful fine-tuning Booster (Huang al. 2025b) proposes to minimize the harmful sample loss reduction rate. T- T-Vaccine (Liu et al., 2024a) is a memory efffciett improvement over Vaccine that only applies perturbations to some safety-critical
2

---

Published as a conference paper at ICLR 2026
layers. These methods offer a significant computational advantage, as the safety alignment is one-time cost incurred before subsequent fine-tuning. However, a potential disadvantage is that this safety alignment is static. This means that it may lack the flexibility to counter various attack configurations, such as a high number of fine-tuning steps or large learning rates.
Fine-tuning Stage Defenses. Fine-tuning stage methods modify the fine-tuning process to mitigate the impact of harmful data on the model's 's safety while ensuring performance on downstream tasks. LDIFS (Mukhoti et al., 2024 uses a regularization loss to reduce the shift in model embeddings of the aligned model. Other methods utilize additional alignment data to enhance model safety during fine-tuning. Safelnstr (Bianchi al. 2024) mixes safety examples into the fine-tuning data to maintain alignment knowledge. Lisa (Huang et al., (222a)) alternately updates the line-tuning data with safety examples and includes a proximal term to constraín the drift of the model's weights. Some methods modify the system prompt to improve model safety. PTST (Lyu et al. (024) proposes using a general system prompt during fine-tuning and safety prompt during inference. BEA (Wang et al., 2024) introduces backdoor to trigger refusal behavior on harmful inputs. SaLoRA (Li et al. 2025b) implements a fixed safety module to prevent weight updates from disrupting the model's alignment.
Post-Fine- FineF-Fi-Tunng Stage Defenses. Post-finettuning stage defenses focus on realigning model after the fine-tuning stage. As other methods may only defend successfully with a certain setup of fine-tuning hyperparameters (Qi al. 2025b Huang et al. 2025a), post-fine tet-ffiiittttnnggg defenses give the service provider more control over the safety of their released models. LAT (Casper et al. 2025) leverages latent adversarial training to remove backdoors and novel classes of attacks. SOMF Yi iet al. 2024b) applies model fusion to retain the model's fine-tuned utility while taking advantage of the safeguarding capability of the aligned model. Safe LoRA (Hsu et al. (2024) projects the fine-tuned
LoRA weights into a safe subspace to recover the safety of the model. Antidote (Huang el al. 2025a)
identifies and prunes harmful weights to restore the model's safety.
PRELIMINARY
Fine-tuning-as-a-see aanninng-a---service defense setting. The FTaaS setting consists of two stages: the alignment stage and the fine-tuning stage. In the alignment stage, the service provider performs safety alignment and makes a model available for fine-tuning. The service provider has access to a safety alignment dataset Datign that consists of harmful prompt-refusal completion pairs, such as: How to make a bomb? ? Sorry, I cannot help you, and a harmful dataset Dharm consists of harm- ful prompt-compliant answer, such as: How to write can help you. In the fine-tuning stage, users submit a dataset Deuk. and the provider applies SFT to optimize the model. The dataset Dask contains benign data, such as math problems paired with math solutions, and harmful data different from those in Duarm- The fine-tuned model is then released to the user for their usage. The service provider's challenge is to mitigate harmful fine-tuning attacks introduced by Dask without degrading performance on the benign task.
software virus?
Yes,
Threat model for harmful fine-tuning attacks. The attack surface is the dataset submitted by the user in the fine-tuning stage, where a user may intentionally or unintentionally inject harmful data. This dataset contains n samples, of which (1 - v)% are benign and p% are harmful. Benign samples pairs related to the user-defined task, while harmful samples consist of harmful prompt and compliant answer that follows the malicious request. The service provider acts as the defender, with full control over the fine-tuning process. Following prior work (Qi et al. 2024] Huang et al. 2025b), we assume that the fine-tuning dataset is mixture of benign and harmful data.
are wrmpttccmpletiin
4 OUR FRAMEWORK
Standard SFT updates are vulnerable to harmful fine-tuning attacks because they aggregate gradients from all samples, including harmful ones, allowing gradients from these malicious samples to directly poison the model update. To address this problem, we propose two solutions that jointly reduce the effect of harmful samples. First, in Section 4.1 we propose a robust alignment method that optimizes the model to be in a flat loss region with respect to harmful samples, making safety alignment harder to remove. Second, we propose safety fine-tuning method in Section 4.2 that deprioritizes learning from samples that appear harmful by applying a weighting scheme to all the gradients. Finally, we summarize our proposed solution in Section 4.3
3

---

Published as a conference paper at ICLR 2026
Notations. In what follows, introduce the notation for SFT. Given dataset of promptt-omplltion pairs (x. y), the completion y E {0, 1 is represented as a sequence of L one-hot vectors over vocabulary of size V. Let he denote an LLM parameterized by A. For given pair, define as the concatenation of the input prompt a: and the corresponding completion y. The model processes this concatenated sequence to produce logits, which are then converted to probabilities using a column- wise softmax function:
x he (x) E x T ( x) = festmax_columa (z) E RVL
In this formulation, x is matrix where each column contains the logits for the prediction of the 1-th token, conditioned on the prompt æ and the preceding completion tokens y<t- Consequently, log Tty x) IR VXL represents the per-token log- oogliikelhood sequence. SFT aims to minimize the negative log-likelihood, or cross-entropy loss, over the training data. The loss for a single sample is defined as the summation over the negative log-likelihoods of each token in the completion:
( y)
Σ
[o(x)
Σ e
[log To x)]
(1)
1XL
where ey, is the one-hot vector for the I-th token of the ground-truth completion y, and [+] denotes the 1-th column vector of the matrix.
4.1 ROBUST ALIGNMENT VIA FLATNESS REGULARIZATION
Our aim is to propose a defense method that incorporates harmful data during the alignment stage to build safety alignment robust to harmful fine-tuning attacks. The core of our strategy is to ensure a robust safety alignment by shaping the model's loss landscape. A flat loss landscape for harmful samples makes the instilled safety behavior more difficult to remove, even when the model is subsequently fine-tuned on new harmful samples. We denote Lalign (0) as the empirical loss on the alignment dataset Dalign and Cram (0) as the empirical loss on the harmful dataset Dham- We formulate the defense problem in the alignment stage as the following optimization problem:
min Calign (0) s.t. θ argming Esharp (8')
(2)
where we denote Lahurp (f') A Liarm 18")mmine mi)-minseB,t0 Lharn (Ø) and B, (8) = 10" 110" 110"-0'1l: 0'11a S
ph. with the radius p > O. Evidently, by definition, Lsham (f') represents the sharpness of the harmful loss Lrarm around 0'_ Therefore, the constraint 0 E argming. Lshurp (0') ensures that lies a flat region of the harmful loss landscape. We can interpret the optimization problem in (2) as follows: among the models 0 that lie in the flat region of the harmful loss Charm. we aim to find the one that minimizes the alignment loss Lalign. Moreover, driving the models 0 into the flat regions of the harmful loss maximally reduces the influence of harmful prompts in the supervised fine-tuning phase (i.e., phase 2), while minimizing the alignment loss ensures strong alignment performance. The motivation for using minimization of Lharm in our sharpness formulation is its direct analogy to the subsequent harmful fine-tuning, where harmful gradients minimize the loss of harmful samples in the user's dataset. While minimizing Lharm in isolation would encourage harmful outputs, our objective also includes the alignment loss Calign. Minimizing this alignment loss forces the model to maintain high harmful loss Cham. ensuring it produces refusal responses to harmful prompts.
In the following section, we develop the theoretical foundations that rigorously justify the necessity of placing the model in the harmful-loss flat region to mitigate the negative impact of harmful prompts during the second phase (see Proposition 4.2).
We now present the solution to the optimization problem in 2 Let the current solution be 6,. We aim find descent direction 8, in the update l.+1 = 0, E &c, where & is a step size, such that Lalign (0++1) and Lahurp (0++1 decrease. To this end, we find & by solving:
8, E 1
argmin, (aLaiin - o1|2 s.t. VoLshum (0.) 0r 4,>0
where a, is scalar value. This optimization problem ensures that the update direction 6, is close as possible to the alignment gradient while still making progress in minimizing the sharpness loss.
To find the solution the optimization problem in Equation 3 we present the following theorem:
4

---

Published as a conference paper at ICLR 2026
Theorem 4.1. The optimal solution the optimization problem in Equation
is 67
Ve
"eCalign (ft)
X A (0s), where A max {0, Ct ex- V.-Voaapr000 (0.) VoLan (0.)
| Cruup (9e]ll
We provide the proof in Appendix D.I Intuitively, if the alignment gradient Lalign already makes sufficient progress reducing sharpness, we have A = and recover pure alignment step. Otherwise, A > 0 adds just enough of the sharpness gradient to ensure the required decrease in Sshar. thereby jointly optimizing alignment and flat harmful loss. In the constraint V (0() T 6 2 > O, the scalar Or specifies the minimum required decrease of the sharpness loss at step t. Using a first- order approximation, C.harp(0 (0++1) ) Sharr (0,) s - (oC)hu((0)) 6, I -Ea, < 0 for sufficiently small step size Ç, any choice of a, > O guarantees that Laturp strictly decreases at each iteration, i.e., each update moves the model further into flat harmful loss region. In practice,
choose a₃ & V Sharp (0.)1l for dynamic scaling. To find mingeB,(0)) Lharm (Ø), we start from Φ 0, and then use K = normalized gradient descent steps d = $* VV &. (Ø*)) / V Chamm (Ø*) ll2-
We note that when the step-adaptive regularizer intensity A, set to a constant, our update rule in Theorem 4.1 reduces to the Booster formulation (Huang et al. 2025b). However, Booster is motivated by a different objective: simulates harmful fine-tuning via adversarial perturbations alignment stage to minimize harmful loss reduction in the fine-tuning stage. In contrast, our defense is motivated by explicitly sceking a flat loss region with respect to harmful samples, making the harmful fine-tuning attack less effective due small harmful gradients (See Appendix [.6)). Furthermore, theoretical motivation leads to final update rule that utilizes a step-adaptive regularizer Xe, rather than the constant-intensity regularization used in Booster.
4.2 SAFETY FINE-TUNING WITH WEIGHTED Loss
In the fine-tuning stage, objective is to prevent the model from learning from harmful samples present in the user-submitted dataset. Let's consider a mini-batch of data = { y:) He) where, for simplicity, assume all completion y: have similar length L. This batch consists of both benign and harmful samples. The standard SFT update does not distinguish between these two types of samples and simply combines the gradients from the entire batch. This process, which learns from both benign and harmful samples, is illustrated as follows:
841 + - BL
Σ
V to, (Zi ys) +
E
(I Ui)
(4)
(a..)t) is benign
(K is harmful
Due to our flatness regularization proposed in the alignment stage, the model is optimized to be in flat loss region with respect to harmful samples, resulting in their gradients being negligible at the beginning of the fine-tuning stage. Consequently, the standard update rule effectively simplifies to:
M.11 + - BL
Σ
(x,
(I,, yi)
(2..8)) benign
To understand this effective update rule, analyze the learning dynamics of a mini-batch update. The following proposition extends the single-sample analysis of Equation (5) in (Ren & Sutherland 2025) by decomposing the loss change a test sample after one weighted mini-batee update:
Proposition 4.2. Let B,, C B be the set of benign samples in the training butch. The loss change a test sample (æo. yo) after an update step in Equation (5)
can be decomposed as:
M
L
A E(yo.
7
Σ
E
(Id(()))
IC' (x.) m. + (6)
BL
m=1 (B..).)B Im 1xVXM
VXVxMXL
VXL
where the completion of the test sample has length M. Here, [A'(x))) Vanm (eo, (Xo)) [IC x:)mm.1 = ( (V (xo)lo.)
T is the empirical neural tangent kernel (eNTK)
corresponding to the m-th logit of yo and 1-th logit of ye and (x:)) = IV,,,[eo (x:)li) are the
gradients of the loss the 1-th token of ys with respect to the logits.
5

---

Published as a conference paper at ICLR 2026
Before Fine-tuning
After Fine-tuning
Fine-tuning Loss
Besggn
SFT-iaahh
100
25
dFT Ort--
Anboc Zerbo
Count
Count
Lass
40
20
0.5
-2
230
750
1000
Sc0r8
Fa Scone
Fine-auning Steps
Figure 2: The effect of our proposed fine-tuning method. Left and middle plots show the score (ræ) distribution before and after fine-tuning, while the right plot compares the fine-tuning loss of our method (Antibody) against SFT on benign and harmful samples in the fine-tuning dataset.
We provide the proof for this proposition in Appendix D.2 in loss on a test sample (æo. yo) into sum of token-wise influences, each quantifying the interaction between token from a benign sample and a token from the test sample. This influence governed by the following components. The eNTK term K. (Xx: Xs). which is the product of the logit gradients with respect to the parameters, measures the similarity between samples. The gradient term G (x:) provides the energy and direction for model adaptation to the hard labels of the training sample. The other term A' (Xo) of little interest in our analysis as it depends only on the test sample.
This proposition decomposes the change
This proposition provides insight into how the model's loss on test sample (æo: yo) changes after each mini-batch update. For test sample (x.. yo) that belongs to the same task as the benign sample (a;, 31) [[AC*]| is large), the mini-batch update will decrease the loss on this test sample. In contrast, for harmful test samples that are inherently dissimilar to the benign sample in the batch (small ||AC)]]))) this update will not change the loss on (æ.. yo), thus preserving the model's ''' alignment.
However, even small harmful gradients can accumulate over multiple updates, potentially leading to degradation in the model's safety. To further mitigate the influence of harmful samples during fine-tuning, we propose a weighted loss function that assigns different weights to each sample in the training batch. Our goal is have the weights wa, (ar;, y1) be small for harmful samples and large for benign ones. To compute these weights, we leverage the safety alignment knowledge already instilled in the model after the alignment stage. Specifically, for each input æ,, we calculate a score re, by comparing the model's likelihood of generating the target completion y: versus a generic refusal, (e.g.." "I cannot ful fill your request"). This score then normalized across the batch using a softmax function to produce the weight:
Te, (3/ x2:)) Te, ( (x:))
exp (re, (2,, 9.)/T)
Te, T8, (2,, y:)= & log
we, (x,, yi)
B Σ}= exp (re, (2,, U;)/T)
where 7 is the softmax temperature. The intuition behind this weighting scheme as follows. A safety-aligned model, when presented with harmful prompt æ,, should assign higher probability to the refusal completion consequently small weight. Conversely, as the model adapts to benign data during training, should find the benign target completion y/: far more likely than a refusal, leading to a high score and a large weight for benign samples. We empirically validate this mechanism in Figure The left plot shows that before fine-tuning, scores (re) for harmful samples are generally lower than for benign ones, direct result of the alignment process. After fine-tuning (middle plot), the two score distributions diverge further: the benign distribution shifts to the right, indicating successful learning, while the harmful distribution remains low, creating a distinct separation. Consequently, the right plot demonstrates that method's 's loss benign samples decreases, confirming task adaptation, while its loss on harmful samples remains high, effectively ignoring them. This stands in stark contrast to standard SFT, which overfits by indiscriminately reducing its loss both benign and harmful data.
than to the harmful completion y;- This results in a low score ræ. and
The proposed weighting scheme allows us to amplify the contribution of benign samples while suppressing that of harmful ones, which allows our mini-batch weighted update to be dominated by

---

Published as a conference paper at ICLR 2026
benign samples and effectively reduces to the following:
1 0ut1 t 17 =
we, (2,, v. Le, (z,, yi)
(8)
(an..t))) benign
In the ideal case, the weights for harmful samples are zero and the weights for benign samples are uniform, Wa, (zt, v:) [8.] V(æ,, y.) E Bg This reduces the update in Equation (8) to the standard
SFT update rule applied only to benign samples, where the gradient is normalized by the total number of tokens in the benign set, [8,
We now extend Proposition 4.2 the weighted mini-batch setting of Equation 8
Proposition 4.3. Given a similar senup as in Proposition 4.2 the loss change on a test sample (o yo) after an update step in Equation (8) can be decomposed
A Allo Io) = "
Σ Σ
Σ
wa (Ti, U1) [A (xo) (X X:)mma [ (X'(x))) O(M)))
1xVxM
VXVXMXL
VXL
where the weight Wo, (z,, ye) is defined in Equation
We provide the proof for this proposition in Appendix D.2 This mechanism protects the model's safety alignment while simultaneously improving learning on benign data. The model's loss on harmful test sample (zo: yo) will remain unchanged the batch gradient is predominantly contributed by benign samples. Conversely, for benign test sample (Io, Yo) in the same domain as the benign sample (a:,. yt), the large weight wo, (æt; y:) will amplify the influence of the benign gradient, thereby boosting model learning on the user's task.
Finally, further prevent degradation in model safety, we propose additional objective in the alignment stage to ensure the weights remain effective for harmful samples. In the fine-tuning stage, exposure harmful samples can still cause the model to drift towards unsafe behavior, which increases the weights assigned to harmful samples. To this end, we first simulate model parameter drift in the fine-tuming stage using the harmful samples in the aligmment stage via a harmful perturbed model ppen 2 0 p VaCnam (0)/l Charm (0)ll2- Then, we construct refusal dataset Dretusal {(x y-)) x E DDarms ~ VrJ. where D, is set of generic refusal responses. We then add the following objective the alignment stage:
Lrefusall (0 (Ppen)
Eajes (x. yr)
log Ben (3/ (a:),
(10)
(E.
where we do not back- propagatopetteeee the gradient through ppen. By optimizing this objective, we want to simulate situation in the fine-tuning stage where the model being updated using harmful samples can still maximize log Tr (yr ax), hence having a low corresponding weight, as the weight is inversely proportional to the likelihood of generating refusal response.
4.3 SUMMARY
In summary, our proposed defense method is composed of two main stages. The first one is in the alignment stage, where the optimization objective is the following loss function:
Calign (0,) + > L.shar (0.) + Arefusal (Ppents))
(11)
where A,, as detailed in Theorem 4 and Arefisal is a hyper-parameter that controls the strength of the refusal loss term. Then, in the subsequent fine-tuning stage, the model is adapted to user-submitted data using the proposed weighted update algorithm:
1
B.t1 01
Σ Wo (I Bt) V Eo, (2),, v1)
(12)
By combining these two solutions, the proposed defense method ensures that harmful data has a minimal effect the model update: the flatness regularization from the alignment stage defense results in a small gradient magnitude for harmful samples, and the weighting scheme from the fine-tuning stage defense gives them smaller contribution the batch gradient.
7

---

Published as a conference paper at ICLR 2026
5 EXPERIMENTS
5.1 EXPERIMENT SETUP
Datasets. We follow previous works (Huang et al. 2025b; Qi et al. 2025b) to set up the data in alignment, fine-tuning, and evaluation steps. For harmful data, we use the dataset from (Huang et al.
2025b; Qi et al., (025b), which was curated and enriched from the BeaverTails dataset (Ji et al. 2023). To simulate fine-tuning attack, we use the following four datasets: SST2 (Socher et al. 2013).
AGNEWS (Zhang et al. 2015). GSM8K (Cobbe et al., 2021). and AlpacaEval (Li et al. 2023).
Models & Baseline Methods. We conduct experiments Llama-2-7 (Touvron et al. 2023).
Qwen-2-7B (Yang et al. (024) and Gemma-2-99 (Team et al., 2024). We compare our method against various baseline methods that span across different defense stages. For the alignment stage solution, we compare with Supervised Fine-Tuning (SFT), which uses Supervised Fine-tuning for both alignment and fine-tuning stages, Vaccine (Huang al. (024c), and Booster (Huang al.. 20255). For the fine-tuning stage solution, we compare with Lisa (Huang et al. 2024a).
Metrics & Evaluation. We evaluate the fine-tuned models on two key aspects: safety and task- specific performance. To evaluate the safety of the model, we use Harmful Score (HS), which measures the percentage of responses flagged as harmful by a moderation model (Ji et al. [023). For task-specific performance, we use Fine-tuning Accuracy (FA). Harmful Score is calculated using the 699 malicious prompts from the Beaver SeaverTails-Ee Tails-Evallution benchmark (Ji al. 2023). We evaluate the fine-tuning accuracy of SST2 with 872 samples, AGNEWS with 1000 samples, GSM8K with 1000 samples, and AlpacaEval with 105 samples.
Training Details. Following prior works (Huang et al. 2024a 2025b), we use LoRA (Hu et al. 2022) for alignment and fine-tuning. In the alignment phase, we train the model for 20 epochs using the AdamW optimizer (Loshchilov & Hutter 2019) with a learning rate of 5 10-4 and weight decay of 1. I.I. In the fine-tuning attack phase, we fine-tune the model for 20 epochs on SST2, AGNEWS, and GSM8K, and for 100 epochs on AlpacaEval. We use the AdamW optimizer with learning rate of 1 x 10-5, weight decay of 0. and a warmup ratio of O. A batch size of 16 is used for both the alignment and fine-tuning attack phases, except for experiments with the Gemma2-9B model, which uses a batch size of 10 due to GPU memory constraints. For the default experiment setting, we fine-tune Llama-2-7B on n 1000 samples of the GSM8K dataset with a harmful ratio p 0.2 and report averages over three random seeds. For AlpacaEval, we use 700 samples and run a single seed to reduce API costs. More details for experiment setup can be found in Appendix E
5.2 MAIN EXPERIMENTS
Generalization to different fine-tuning datasets. In Table 1 we present the performance of different methods on four datasets: SST2, AGNEWS, GSM8K, and AlpacaEval. Our method achieves the best fine-tuning accuracy on SST2 (93.55%) and AGNEWS (87 (0%% and competitive performance on GSM8K (15.07%) and AlpacaEval (58.10%). Furthermore. our method successfully defends the model, as demonstrated by the lowest average harmful score of 7.04%. This represents an improvement of over 8 percentage points compared to the runner-up method, Lisa. These experiments show that, compared to other baselines, Antibody the only method that successfully defends against harmful fine-tuning while effectively adapting to various user-defined tasks.
Table I: Performance of models trained on different fine-tuning datasets. The best and the second best are highlighted in orange and gray, respectively.
Methods
SST2
AGNEWS
GSMBK
AlpucaEval
Average
(Llama-2-7BB
HS 1
FA
HS L
FA
HS
FA
HS 1
FA
HS
FA
SFT
36.29
92.70
34.57
85.40
23.94
10.90
39.48
61.43
33.57
62.61
Vaccine
44.16
91.71
39.73
82.43
23.60
11.70
55.94
54.33
40.86
60.04
Lisa.
22.94
20:93
84.50
9.23
11.44
57.62
15.29
60:97
Booster
14.31
92.59
15. 88
86.70
9.06
16.27
36.91
65.24
19.04
65.20
Antibody
1.48
93.55
1.24
87.30
1.24
15.07
24.18
58.10
7.04
63.51
Generalization across diverse model architectures. As shown in the table Table 2 our method consistently achieves the lowest harmfulness score across all models. It also demonstrates superior
8

---

Published as a conference paper at ICLR 2026
Table 2: Performance under different model architectures in the default setting. The best and the second best are highlighted in orange and gray, respectively.
Methods
Llama-2-7B
Qwen-2-7B
Gemma-2-9B
Averuge
(GSM8K)
FA
115
SFT
23.94
10.90
14.54
64,66
32.05
56,73
23.51
44.
Vaccine
11.70
42.76
13.02
7.60
42.70
Booster
16.27
2.19
68.63
22.13
58.97
11.13
47.96
Antibody
1.24
15.07
0.62
67.30
0.91
57.43
0.92
46.60
stability, with significantly less variation in harmfulness (1.24% for Llama-2-7B, 0.62% for Qwen-2- 7B, and 0.91% % for Gemma-2-9B). While ensuring this high level safety, our method maintains competitive fine-tuning accuracy of 46.60% on average, lagging behind Booster by 1.36 percentage points but leading over SFT by 2.5 percentage points. These findings confirm that our method generalizes effectively across diverse model architectures.
Table 3: Performance under different harmful ratios in the default setting. clean means harmful samples or p = 0. The best and the second best are highlighted in orange and gray, respectively.
Methods
Harmful Score J
Fine-tunine Accurucy
(n 1000)
clean p
0.05 p
0.1
p= 0.15
p 0.2
p= 01.25
Average
clean
0.05
p= 0.1 p= 0.15
p
p 0125 Average
19:06
27.85
10.90
Vaceine
7.15
19.55
11.77
Lisa
0.95
3,05
3.82
4.62
5.86
8.16
4.41
9.97
9.57
9.60
8.90
9.23
9.20
9.41
Booster
1.38
1.76
2.91
5.10
9.06
13.49
5.62
16.53
15.97
16.50
16.30
16.43
15.70
16.24
Antibody
0.95
1.14
0.95
1.14
1.24
1.29
1.12
15.83
14.57
15.40
15.40
15.07
14.37
15.12
Robustness across harmful ratios. In Table 3 we present the defense and fine-tuning performance of different methods under varying harmful ratios ranging from 0.05 to 0.25. Compared to other baselines, our method consistently achieves the lowest harmful score across all ratios. Compared to Lisa, the second-best performer, method's harmful score is 3.29 percentage points lower, while improving fine-tuning accuracy by 5.71 percentage points. Furthermore, our method maintains competitive fine-tuning accuracy, closely following the top performer, Booster, while substantially outperforming other baselines. These results underscore our approach's 's efficacy in defending against fine-tuning attacks with significant gain in model utility over SFT across different harmful ratios.
5.3 ABLATION STUDIES
Robustness to different fine-tuning hyper- parameters. Figure comparees Antibody and Booster's robustness under varying mue-tuning epochs and learning rates. As fine-tuning epochs increase from 10 to 100, Booster's harmful score increases significantly, while Antibody main- tains a harmful score below 10%. Similarly, as the learning rate increases, Booster's harmful score rises rapidly, whereas Antibody's 's only be- gins to increase significantly when the learning rate reaches 1 10 10-1, Antibody's 's ability to suc- cessfully defend across wide range of epoch and learning rate values demonstrates that it is an effective and reliable defense method for various fine-tuning conditions.
Harrffll Score
1 8
Harrfful
Rine-buning Epochs
Fine-tuning Learning
Figure 3: Harmful score with different fine-tuning epochs (Left) and learning rates (Right).
Effectiveness of the proposed components. As shown in Table ablation study demonstrates the progressive benefit of each component. All the components show beneficial effects as they progressively reduce the harmful score. Regarding the fine-tune fine-tuning accuracy, even though the weighted update algorithm proposed in the fine-tuning stage reduces the performance gain from the sharpness alignment in the alignment stage, it still brings significant improvement compared the SFT

---

Published as a conference paper at ICLR 2026
Table Ablation on our proposed components. Each row is a cumulative addition the previous one. We color-code the performance change from the SFT baseline.
Method
GSM8K HS 4 GSM8K FA
SFT
23.94
10.90
Align with x Lahap((.))
13.02 10.92)
16.00 (+5.20)
+ Fine-tune with wa,
4.44 19.50)
13.83 (+2.93)
Align with Cretwaa (O) (Antibody)
1.24 22.70)
15.07 (1-4.17)
baseline. Finally, the refusal loss term completes our method, delivering a sharp reduction in harmful score to 1.24% while boosting the final accuracy to 15.07% compared to SFT.
CONCLUSION
In this paper, we introduce Antibody, a defense algorithm designed to counteract harmful fine-tuning attacks by attenuating the gradients of harmful samples encountered during fine-tuning. Specifically, Antibody first establishes a robust safety alignment by optimizing for a flat loss region with respect to harmful samples, and then employs a dynamic weighting scheme during fine-tuning favor learning on benign data and suppress the influence of malicious inputs. Our extensive evaluations demonstrate that Antibody effectively mitigates attacks across settings while boosting fine-tuning performance, making it a practical and powerful solution for FTaaS providers.
ACKNOWLEDGMENTS
Trung Le and Mehrtash Harandi were supported by the ARC Discovery Project grant DP250100262 and the Air Force Office of Scientific Research under award number FA9550-23-8-000..
REFERENCES
Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul Arora, Yu Bai, Bowen Baker, Haiming Bao, et al. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508. 2025.
Anonymous. Eliciting harmful capabilities by fine-tuning on safeguarded outputs. In Submitted to The Fourteenth International Conference on Learning Representations, 2025a. URL https under review.
//openreview.o net/ // Eorum?id=viiAbgg
Anonymous. Gradshield: Alignment preserving finetuning. In Submitted to The Fourteenth Interna- tional Conference on Learning Representations, 2025b. URL/Ittps://ooeexeviies net
forum? d=YYUNm bCbb under review.
Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Rottger, Dan Jurafsky, Tatsunori Hashimoto, and James Zou. Safety-tuned LLaMAs: Lessons from improving the safety of large language models that follow instructions. In The Twelfth International Conference on Learning Representations, 2024. URL
net/ wtteee//oooummmmmmmmaaaae
Dillon Bowen, Brendan Murphy, Will Cai, David Khachaturov, Adam Gleave, and Kellin Pelrine. Scaling trends for data poisoning in llms. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 27206-27214, 2025.
Stephen Casper, Lennart Schulze, Oam Patel, and Dylan Hadfield-Menell. Defending against unforeseen failure modes with latent adversarial training. Transactions on Machine Learning Research, 2025. ISSN 2835-8856.
Liang CHEN, Xueting Han, Li Shen, Jing Bai, and Kam-Fai Wong. Yulnerability-wwar aware alignment: Mitigating uneven forgetting in harmful fine-tuning. In Forty-secondl International Conference on Machine Learning, 2025. URL htttps://oppneeeee
net fto.t///o//r//ee// Tat.::rrmmmmmm--MM.........
10

---

Published as a conference paper at ICLR 2026
Pin- Yu Chen, Han Shen, Payel Das, and Tianyi Chen. Fundamental safety- cappbiilty trade-offs in fine-tuning large language models. arXiv preprint arXiv:2503.20807, 2025.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2 14164. 2021.
Yanrui Du, Sendong Zhao, Jiawei Cao, Ming Ma, Danyang Zhao, Shuren Qi, Fenglei Fan, Ting Liu, and Bing Qin. Toward secure tuning: Mitigating security risks from instruction fine-tuning. arXiv preprint ar arXivv2240..445 2024.
Hoang Anh Dung, Cuong Pham, Trung Le, Jianfei Cai, and Thanh-Toan Do. Sharpness-aware data generation for zero-shot quantization. In Forty-first International Conference Machine Learning, 2024. URL httpss//opeeeeeeiieres wwwpeeewwwwwww...eeeeeeeeeeeeeee www.eeeee/otteeeeeeeeeeeeeeeeeee B
Sarah Egler, John Schulman, and Nicholas Carlini. Detecting adversarial fine-tuning with auditing agents. arXiv preprint arXiv:2510. /6255, 2025.
Francisco Eiras, Aleksandar Petrov, Philip Torr, M. Pawan Kumar, and Adel Bibi. Do as i do (safely): Mitigating task-specific fine-tuning risks in large language models. In The Thirteenth International Conference on Learning Representations, 2025. htlltttts////ppeeeeet www.eewwwwww.eee.eoeeeeeeeeeeeee
1d=1XE5116ppV
Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization. International Conference on Learning Representations, 2021. URL https: /tto://///ttss////eeeeeeee..ee
Zihan Guan, Mengxuan Hu, Ronghang Zhu, Sheng Li, and Anil Vullikanti. Benign samples matter! fine-tuning on outlier benign samples severely breaks safety. In Forty-second International Conference on Machine Learning. 2025. URL
wttp:e/wwwwweneooruuueeee forum?id
GF
Scokil Ham, Yubin Choi, Seungju Cho, Yujin Yang, Younghun Kim, and Changick Kim. Refusal- feature-guided teacher for safe finetuning via data filtering and alignment distillation. arXiv preprint arXiv:2506.003566 2025.
Lei Hsiung, Tianyu Pang, Yung-Chen Tang, Linyue Song, Tsung- Yi Ho, Pin- Yu Chen, and Yaoqing Yang. Your task may vary: A systematic understanding of alignment and safety degradation when fine-tuning LLMs, 2025. URL
net forum? dd
Chia-Yi Yi Hsu, Yu-Lin Tsai, Chih-Hsun Lin, Pin-Yo Yu Chen, Chia-Mu Yu, and Chun-Ying Huang. Safe lora: The silver lining of reducing safety risks when finetuning large language models. Advances in Neural Information Processing Systems, 37:65072-65094, 2024.
Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen -Zu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. International Conference on Learning Representations, 2022. URL htl.sttts: http///pppeeveeeeeee/eeeeeee ww..wiew.eet............ www.t//frumm forum? nd=
Zixuan Hu, Li Shen, Zhenyi Wang, Yongxian Wei, and Dacheng Tao. Adaptive defense against harmful fine-tuning for large language models via bayesian data scheduler. In The Thirty- ninth Annual Conference on Neural Information Processing Systems, 2025. URL https //openreview.o net www.:neet/ooooddddooo
15MM44
Tiansheng Huang, Sihao Hu, Fatih Ilhan, Selim Tekin, and Ling Liu. Lisa: Lazy safety alignment for large language models against harmful fine-tuning attack. Advances in Neural Information Processing Systems, 37:100521-104555 2024a.
Tiansheng Huang, Sihao Hu, Fatih Ilhan, Selim Furkan Tekin, and Ling Liu. Harmful fine-tuning attacks and defenses for large language models: A survey. arXiv preprint 2rXiv:2409.18166 2024b.
11

---

Published as a conference paper at ICLR 2026
Tiansheng Huang, Sihao Hu, and Ling Liu. Vaccine: Perturbation-awee alignment for large language models against harmful fine-tur tuning attack. Advances in Neural Information Processing Systems, 11774058-740885 2024c.
Tiansheng Huang, Gautam Bhatracharya, Pratik Joshi, Joshua Kimball, and Ling Liu. Antidote: Post- fine-tuning safety aligmment for large language models against harmful fine-tuning attack. In Forty- second International Conference on Machine Learning, 2025a. URLhttps: pettpps:/openreviee
net/forumm wwt.:rpmmmddrppplll.pmmm 4R8 6m
Tiansheng Huang, Sihao Hu, Fatih Ilhan, Selim Furkan Tekin, and Ling Liu. Booster: Tackling harmful fine-tuning for large language models via attenuating harmful perturbation. In The Thirteenth International Conference on Learning Representations, 2025b. URL https://
Crenreview.net/ooree www.eneettrnndttiiiiee
Tiansheng Huang, Sihao Hu, Fatih Ilhan, Selim Furkan Tekin, and Ling Liu. Virus: Harmful fine-tuning attack for large language models bypassing guardrail moderation. arXiv preprint arXiv:2501.17433 2025c.
Jiaming Ji, Mickel Liu, Josef Dai, Xuehai Pan, Chi Zhang, Ce Bian, Boyuan Chen, Ruiyang Sun, Yizhou Wang, and Yaodong Yang. Beavertails: Towards improved safety alignment of Il via a human-pree premeerreee dataset. Advances in Neural Information Processing Systems, (06624678-277000 2023.
Joshua Kazdan, Lisa Yu, Rylan Schaeffer, Chris Cundy, Sanmi Koyejo, and Krishnamurthy Dj Dvijotham. No. of course can! refusal mechanisms can be exploited using harmless data. In ICLR 2025 Workshop on Building Trust in Language Models and Applications, 2025. URL htttps://openreiiee www.eeeeouueieeeeeeeeee http://rrumii-pPpPetGCee
net
Jachan Kim, Minkyoo Song, Seungwon Shin, and Sooel Son. Defending moe Ilms against harmful fine-tuning via safety routing alignment. arXiv preprint arXiv:2509. 2025.
Boheng Li, Renjie Gu, Junjie Wang, Leyi Qi, Yiming Li, Run Wang. Zhan Qin, and Tianwei Zhang. Towards resilient safety-driven unlearning for diffusion models against downstream fine-tuning. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025a. URL https: fttt::://opeerevieee n forum wwtemiiddddcttccc 6F
Jiawei Li. Detecting instruction fine-tuning attack on language models with influence function. arXiv preprint arXiv:2504. 09026, 2025.
Mingjie Li, Wai Man Si, Michael Backes, Yang Zhang, and Yisen Wang. SaloRA: Safety-alignmen preserved low-rank adaptation. In The Thirteenth International Conference on Learning Represen- fations, 2025b. URL http::ttt//ttttt /tctps://operrete revi ew net./ forum? 6=G0oVZE995)
Shen Li, Liuyi Yao, Lan Zhang, and Yaliang Li. Safety layers in aligned large language models: The key LLM security. In The Thirteenth International Conference on Learning Representations, 2025c. URL htttss:///ppentt
net. pest/fooumm: wow.ummddekuullyoo.o.rr..
Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Alpacaeval: An automatic evaluator of instruction-followinn models. https: /ttt:://gghtu...t/ee com httpuuu.cootttt.uuaaaaaaaa/aaa//
__vall 2023.
Guozhi Liu, Weiwei Lin, Tiansheng Huang, Ruichao Mo, Qi Mu, and Li Shen. Targeted vaccine: Safety alignment for large language models against harmful fine-tuning via layer-wise perturbation. arXiv preprint axXhv:2410.09760. 2024a.
Xiaoqun Liu, Jiacheng Liang, Muchao Ye, and Zhaohan Xi. Robustifying safety-aligned large language models through clean data curation. arXiv preprint arXiv:2405. 19358, 2024b.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Confer- ence on Learning Representations, 2019. URL https:://ppezeeeeeeese www.:w/umwteeeeteeeemmmmmm
Bkg CqY7
12

---

Published as a conference paper at ICLR 2026
Ning Lu. Shengeai Liu, Jiahao Wu, Weiyu Chen, Zhirui Zhang, Yew-Soon Ong, Qi Wang, and Ke Tang. Safe delta: Consistently preserving safety when fine-tuning LLMs on di- verse datasets. In Forty-second International Conference on Machine Learning. 2025. URL
httpps://spenreviee.. Forr
forum? wrwforummddosssooe... id=-giDD
Haocheng Luo, Tuan Truong, Tung Pham, Mehrtash Harandi, Dinh Phung, and Trung Le. Explicit eigenvalue regularization improves mharpness-awar minimization. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL httph::t///ppoee /tttps://peerreieee
Ine//Eorum? Law.:ammmiidSUUuEBBSEE.B
Haocheng Luo, Mehrtash Harandi, Dinh Phung, and Trung Le. Unveiling m-sharpness through the structure of stochastic gradient noise. In The Thirty- minth Annual Conference on Neural Information Processing Systems, 2025. htthhttp:: / openrevieeeeeeeeeeee www. Eorum Peee://oooumm1..ttptttOO0000
Kaifeng Lyu, Haoyu Zhao, Xinran Gu, Dingli Yu, Anirudh Goyal, and Sanjeev Arora. Keeping llms aligned after fine-tuning: The crucial role of prompt templates. Advances in Neural Information Processing Systems, 37:1 18603-118611, 2024.
Jishnu Mukhoti, Yarin Gal, Philip Torr, and Puneet K. Dokania. Fine-tuning can cripple your founda- tion model; preserving features may be the solution. Transactions Machine Learning Research, 2024. ISSN 2835-8856. URL httt:::://ppeervieewe wwwereeiiwwwweeeeeeeeee.e..eeeee wo Featured Certification.
Van-Anh Nguyen, Trung Le, Anh Tuan Bui, Thanh-Toan Do, and Dinh Phung. Optimal transport model distributional robustness. In Thirty-seventh Conference on Neural Information Processing Systems, 2023a. /tt/:tttps///oeeeneiieeeeeeee//e
Van-Anh Nguyen, Long Tung Vuong, Hoang Phan, Thanh-Toan Do, Dinh Phung, and Trung Le. Flat seeking bayesian neural networks. In Thirty-sevnnth Conference on Neural Information Processing Systems, 2023b. httptttt////peeeeeeeeee
net. forum? 1d=eexxDtooxxeee
Kyle O' Brien, Stephen Casper; Quentin Anthony, Tomek Korbak, Robert Kirk, Xander Davies, Ishan Mishra, Geoffrey Irving, Yarin Gal, and Stella Biderman. Deep ignorance: Filtering pretraining data builds tamper-resistant safeguards into open-weight Ilms. arXiv preprint arXivv2508.06601, 2025.
Sheng Yun Peng, Pin- Yu Chen, Matthew Daniel Hull, and Duen Horng Chau. Navigating the safety landscape: Measuring risks in finetuning large language models. In The ehirth Annual Conference on Neural Information Processing Systems, 2024. URL https:://opeeeeeee
net:/ d
Sheng Yun Peng, Pin-Yu Yu Chen, Jianfeng Chi, Seongmin Lee, and Duen Horng Chau. Shape up! restoring LLM safety during finetuning. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URL https: httpas:///ppeneeeewweeetttt htt.://wwwwe.eorrreeeeeeeee PAIVwOaAng
Gabriel Jacob Perin, Runjin Chen, Xuxi Chen, Nina S. T. Hirata, Zhangyang Wang, and Junyuan Hong. Lox: Low-rank extrapolation robustifies LLM safety against fine-tuning. In Second Conference on Language Modeling, 2025. URL htattss://ppereeiieee
net/ hww.:/eeffffot/oe/..//
Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin- Yu Chen, Ruoxi Jia, Prateck Mittal, and Peter Henderson. Fine-tuning aligned language models compromises safety, even when users do not intend to! In The Twelfth International Conference on Learning Representations, 2024. URL https //openreview. net. wtw.://ooormmad--1Eoo.....
Xiangyu Qi, Ashwinee Panda, Kaifeng Lyu, Xiao Ma, Subhrajit Roy, Ahmad Beirami, Prateek Mittal, and Peter Henderson. Safety alignment should be made more than just a few tokens deep. In The Thirteenth International Conference on Learning Representations, 2025a. URL
net wwr.mmirriiiooooo /tt.:///w...../......./22..///..
6Mxhg
Xiangyu Qi, Boyi Wei, Nicholas Carlini, Yangsibo Huang, Tinghao Xie, Luxi He, Matthew Jagielski, Milad Nasr, Prateek Mittal, and Peter Henderson. On evaluating the durability of safeguards for wein-weight LLMs. In The Thirteenth International Conference on Learning Representations. 2025b. URL ht::ps///ppanrrvieeeeeeeeeeeeeee
13

---

Published as a conference paper at ICLR 2026
Rafael Rafailov, Archit Sharma, Erie Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 011533288-37444 2023.
Yi Ren and Danica J. Sutherland. Learning dynamics of LLM finetuning. In The Thirteenth International Conference on Learning Representations, 2025, URL
net / Eorum? 1d=tPNHO22119
Domenic Rosati, Jan Wehner, Kai Williams, Lukasz Bartoszcze, Robie Gonzales, Subhabrata Majum- dar, Hassan Sajjad, Frank Rudzicz, et al. Representation noising: A defence mechanism against harmful finetuning, Advances in Neural Information Processing Systems, 77:11636-12676, 2024.
Shuai Shao, Qihan Ren, Chen Qian, Boyi Wei, Dadi Guo, Jingyi Yang. Xinhao Song, Linfeng Zhang, Weinan Zhang. Dongrui Liu, et Your agent may misevolve: Emergent risks in self-evolving IIm agents. arXiv preprint 0rXiv:2509.66354 2025.
Han Shen. Pin- Yu Chen, Payel Das. and Tianyi Chen. SEAL: Safety-enhancee aligned LLM fine-tuning via bilevel data selection. The Thirteenth International Conference on Learning Representations, 2025.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, Pp. 1631-1642, 2013.
Rishub Tamirisa, Bhrugu Bharathi, Long Phan, Andy Zhou, Alice Gatti, Tarun Suresh, Maxwell Lin, Justin Wang, Rowan Wang, Ron Arel, Andy Zou, Dawn Song, Bo Li, Dan Hendrycks, and Mantas Mazeika. Tamper-resistan safeguards for Conference Learning Representations, 2025. URL httpthtt///// http://///peneeeeeeeeeeeeeeeee forrff
LLMs. The Thirteenth International
1d=4FI
Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. Gemma 2: Improving open language models at practical size. arXiv preprint arXiv:2408.001 18, 2024.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302. /3971, 2023.
Eric Wallace, Olivia Watkins, Miles Wang, Kai Chen, and Chris Koch. Estimating worst-case frontier risks of open-weight llms. arXiv preprint arXiv:2508.033 03155, 2025.
Jiongxiao Wang, Jiazhao Li, Yiquan Li, Xiangyu Qi, Junjie Hu, Sharon Li. Patrick McDaniel, Muhao Chen, Bo Li, and Chaowei Xiao. Backdooralign: Mitigating fine-tuning based jailbreak attack with backdoor enhanced safety alignment. Advances in Neural Information Processing Systems, 37:5210-5243 2024.
Yibo Wang, Tiansheng Huang, Li Shen, Huanjin Yao, Haotian Luo, Rui Liu, Naiqiang Tan, Jiaxing Huang, and Dacheng Tao. Panacea: Mitigating harmful fine-tuning for large language models via post-finettuning perturbation. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025a. URL https:: http:///eeeeeeeeeeeeeeeeeee
Yuhui Wang, Rongyi Zhu, and Ting Wang. Self-destruetive language model. arXiv preprint 17Xi::2505.12188 2025b.
Boyi Wei, Kaixuan Huang, Yangsibo Huang, Tinghao Xic, Xiangyu Qi. Mengzhou Xia, Prateek Mittal, Mengdi Wang, and Peter Henderson. Assessing the brittleness of safety alignment via pruning and low-rank modifications. In Forty-first International Conference on Machine Learning, 2024. URL (ntt/ttp::///oom /tppttte///ppeeeeevee.. net http://wwww...eeeeoooooooooooaa. Koxxmiiiddxxxxxxxexxx
Chengean Wu, Zhixin Zhang, Zeming Wei, Yihao Zhang, and Meng Sun. Mitigating fine-tuning risks in LLMs via safety-awar probing optimization. In 2nd Workshop on Models of Human Feedback for AI Alignment, 2025. URL https: /ttp::://p////ee/eeeeee ew. net. hww.:/eefforuuoooooooooo rumm? d= axdR83V2
14

---

Published as a conference paper at ICLR 2026
Yuxin Xiao, Sana Tonekaboni, Walter Gerych, Vinith Suriyakumar, and Marzyeh Ghassemi. When style breaks safety: Defending language models against superficial style alignment. arXiv preprint S5ff::2500007452.. 2025.
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. Qwen2 technical report. arXiv preprint arXiv:2407.10671, 2024.
Shuo Yang. Qihui Zhang. Yuyang Liu, Yue Huang, Xiaojun Jia, Kunpeng Ning, Jiayu Yao, Jigang Wang, Hailiang Dai, Yibing Song, et al. Asft: Anchoring safety during Ilm fine-tuning within narrow safety basin. arXiv preprint urXiv:2506.08445 2025.
Xianjun Yang, Xiao Wang, Qi Zhang, Linda Petzold, William Yang Wang, Xun Zhao, and Dahua Lin. Shadow alignment: The ease of subverting safely-aligned language models. arXiv preprint arXiv:2310.02949, 2023.
Biao Yi, Tiansheng Huang, Baolei Zhang, Tong Li, Lihai Nie, Zheli Liu, and Li Shen. Ctrap: Embedding collapse trap to safeguard large language models from harmful fine-tuning. arXiv preprint ar 26559999..6669, 2025.
Jingwei Yi, Rui Ye, Qisi Chen, Bin Zhu, Siheng Chen, Defu Lian, Guangzhong Sun, Xing Xie, and Fangzhao Wu. On the vulnerability of safety aligmment in open-access LLMs. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics: ACL 2024, pp. Association for Computational Linguisties. doi: 10.18653/1/2024, findings-acl.549, h2n.:////2222..dnngggg.1115544 URL www.ccaannhhloo....oooooooo hetpa:///eo.ambbhooooo.ooo/////
Bangkok, Thailand, August 2024a.
acl
Xin Yi, Shunfan Zheng, Linlin Wang, Xiaoling Wang, and Liang He. A safety realignment framework via subspace-oriented model fusion for large language models. Knowledge -aaeeedge-Baeedd Systems, 306: 112701, 2024b.
Bingjie Zhang, Yibo Yang. Dandan Guo, Jindong Gu, Philip Torr, Bernard Ghanem, et al. A guardrail for safety preservation: When safety-sensitive subspace meets harmful-resistant null-space. arXiv preprint 1511:225000.4301,,, 2025.
Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. Advances in neural information processing systems, 28, 2015.
Wei Zhao, Zhe Li, Yige Li, and Jun Sun. Unleashing the unseen: Harnessing benign datasets for jailbreaking large language models. arXiv preprint XiX.::4410000055.. 2024.
Weixiang Zhao, Yulin Hu, Yang Deng, Jiahe Guo, Xingyu Sui, Xinyang Han, An Zhang, Yanyan Zhao, Bing Qin, Tat-Seng Chua, and Ting Liu. Beware of your po! measuring and mitigating Al safety risks in role-play fine-tuning of L.LMs. In Wanxiang Che, Joyce Nabende, Ekaaerina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguisties (Volume I: Long Papers), pp. 11112-11137, Vienna, Austria, July 2025a. Association for Computational Linguistics. ISBN 979-8-89176-251-4 doi: 111855331120222 ael-llogg.544 URL fttt:s://aclaattooooy..ooo org. acl long 544/
Yiran Zhao, Wenxuan Zhang, Yuxi Xie, Anirudh Goyal, Kenji Kawaguchi, and Michael Shieh. Understanding and enhancing safety mechanisms of LLMs via spfety-specific neuron. In The Thirteenth International Conference on Learning Representations, 2025b. URL https://
openreview net/
Minjun Zhu, Linyi Yang, Yifan Wei, Ningyu Zhang, and Yue Zhang. Locking down the finetuned Ilms safety. arXiv preprint 1rXiv:2410.103343 2024.
Yongshuo Zong, Ondrej Bohdal, Tingyang Yu, Yongxin Yang, and Timothy Hospedales. Safety fine- tuning at (almost) no cost: A baseline for vision large language models. In Forty-first International Conference on Machine Learning, 2024. URL http:: /ttp///ppee/veeeeeee eew net. / fo rorum? id=
blZZkFF0g7G
15

---

Published as a conference paper at ICLR 2026
A RELATED WORK
LLM's safety can be compromised by fine-tuning on small number of harmful samples Yang et al.
2023 Yi et al.) 2024a Qi et al., 2024 Huang et al.) 2025b; Bowen et al. 2025) or even on only benign data (Shao et a.. 2025). ). this section, we present various works on fine-tuning attack and
defense strategies, and studies regarding the property of the model's safety alignment and harmful fine-tuning attack.
Alignment stage defense. Alignment stage methods (Huang et al. 2024c Rosati et al.
2024| Huang
ctt al. 2025b) aim to make a model more robust against fine-tuning attacks by modifying the safety alignment process. Vaccine (Huang et al. 2024c) aims to reduce harmful embedding drift by adding perturbations to the model's 's embeddings during the alignment stage. RepNoise (Rosati et al.) 2024) further proposes to utilize a harmful dataset to align the model so that its harmful embeddings move toward random noise, thus removing harmful information that can be extracted from those embeddings. TAR (Tamirisa al., 2025) applies meta-learning to sustain a high loss in harmful samples after harmful fine-tuning. Booster (Huang et al. (025b) proposes to minimize the harmful sample loss reduction rate. T- V-vcccine (Liu et al., (2024)) memory-efficien improvement over Vaccine that only applies perturbations to some safety-critical layers. SEAM (Wang et al. 2025b) and CTRAP (Yi et al., 2025) propose safety alignment strategies that create an optimization trap, making any attempt to optimize for harmful objectives inevitably lead to degradation in the model's general performance. VAA (CHEN et al., 2025) enhances the safety alignment by identifying and upweighting vulnerable alignment data. LoX (Perin et al.) 2025) is a training-free method that reinforces a model's safety by extrapolating the safety subspace of an aligned L.M.M These methods offer a significant computational advantage, as the safety alignment is one-time cost incurred before subsequent fine-tuning. However, a potential disadvantage is that this safety alignment is static. This means that it may lack the flexibility to counter various attack configurations, such as high number of tne-ttning steps or large learning rates.
Fine-tuning stage defense. Fine-tuming stage methods modify the fine-tuning process mitigate the impact of harmful data on the model's safety while ensuring performance on downstream tasks. Many works in these categories can be classified into one of the following four approaches: regularization-
based method (Mukhoti et al. [2024] Qi et al., 2025a Li et al. 2025b: Yang et al., [2025] Wu et al.
2025 Peng et al. 2025 Zhang
2025), safety-data augmentation (Bianchi et al. [024]] Huang
et al. 2024a] Xiao et al. 2025 Zhao et al. 2025a prompt engineering approach, and data filtering
approach (Shen al. 2025 Anonymous 2025b
2025 Ham et al. 2025).
2024) uses a regularization loss to reduce the shift in model embeddings of the aligned model. Constrained-SFFT (Qi et al.) (025a) identifies that the first few tokens of the generated response are important eliciting the refusal behaviors and thereby proposes a defense method by imposing a stronger regularization strength towards these tokens. SaLoRA (Li et al. 2025b) implements a fixed safety module prevent weight updates from disrupting the model aligmment. AsFT Yang et al., (2025) preserves the model's 's safety during fine-tuning tunint by using a safety vector in the parameter space to constrain the parameter updates. SAP (Wu et al. 2025) tackles the problem of preserving model safety on benign data fine-tuning by injecting a safety-sensitive probe into gradient updates to avoid harmful directions. StS (Peng et al. 2025) proposes fine-tuning method that dynamically shapes the LLM safety through a token-level signal computed from repurposed guardrail models. GuardSpace (Zhang al. 2025) splits the model's weights via covariance-based decomposition. freezes the safety-relevan components, and applies a harmful-resistant null-space projector that prevents new low-rank adapters from altering the model's original safe responses.
Regularization-base approach. LDIFS (Mukhoti et al.
Safety data augmentation. Other methods utilize additional alignment data to enhance model safety during fine-tuning. Safelnstr (Bianchi et al. (024) mixes safety examples into the fine- tuning data to maintain alignment knowledge. Lisa (Huang et al.] 2024a) alternately updates the fine-tuning data with safety examples and includes a proximal term to constrain the drift of the model's ss weights SafeStyle (Xiao et al. 2025) is a defense strategy that augments small portion of safety training data to match the style-pattern distribution of the fin-tunnig data, thereby preserving LLM safety against style-based based jailbreaks while maintaining style adaptation performance. SaRFT (Zhao et al. 2025a) proposes a fine-tuning method that balances between role-playing capacity and safety.
16

---

Published as a conference paper at ICLR 2026
Prompt engineering approach. Some methods modify the system prompt improve model safety. PTST (Lyu et al. 2024) proposes using a general system prompt during fine-tuning and a safety prompt during inference. BEA (Wang et al. 2024) introduces backdoor to trigger refusal
behavior on harmful inputs.
Data filtering approach. SEAL (Shen et al. 2025) proposes to train data ranker based on the bilevel optimization to filter out the samples that potentially compromise model safety. BDS (2025) proposes a Bayesian Data Scheduler that infers a posterior safety attribute per point and schedules data adaptively during fine-tuning without attack simulation. GradShield w-wf--ntttinngg data filtering method that assigns each training example a Fine-tuning Implicit Harmfulness Score (FIHS) and removes the samples based on an adaptive threshold. The work in (Li 2025) detects poisoning data before fine-tuning by using the influence function. Ref-Teacher (Ham et al. 2025) mitigates harmful fine-tuning by using a safety-aligned teacher model to simultancously filter out harmful data and distill safety knowledge into the base model that is being fine-tuned.
Hu et al.
[Anonymous] 2025b) is
Post-fine-tuning stage defense. Post-finettuning stage defenses (Casper et al. 2025 Yi al. 2024b
Hsu et al. 2024 Zhu et al. 2024] Huang et al., 2025a Wang et al.) 2025a Lu et al. 2025 Egler
ct al. 2025) focus on realigning a model after the fine- tinettuiig stage. As other defense methods may
only defend successfully with certain setup of fine-tuning hyperparameters (Qi et al. 20255 Huang
al. 2025a). post-finetuning defenses give the service provider more control over the safety their released models. LAT (Casper et al. (025) leverages latent adversarial training to remove backdoors and novel classes of attacks. SOMF (Yi et al. 2024b) applies model fusion to retain the model's fine-tuned utility while taking advantage of the safeguarding capability of the aligned model. Safe LoRA (Hsu et al. 2024) projects the one-tuned LoRA weights into a safe subspace to recover the safety of the model. SofetyLock (Zhu et al.) (202)) recovers the model safety by applying safety direction patching. Antidote (Huang et al. 2025a) identifies and prunes harmful weights to restore the model's safety. Panacea (Wang et al. 2025a) proposes optimized adaptive perturbation to recover the model's safety alignment. Safe Delta (Lu et al.) 2025) adjusts the change in parameters before and after fine-tuning to maximize downstream task performance while limiting overall safety loss, and further applies a safety compensation vector to mitigate residual safety loss. The work in (Egler et 2025) develops an audit agent that detects harmful fine-tuning given access to the fine-tune dataset, fine-tuned, and pre-fine-tuning models.
Harmful fine-tuning attack. The work in (Zhao al., 2024) demonstrates an attack method based on extracting benign features in the form of suffix from benign datasets and further shows that these features can be introduced through fine-tuning on only benign data. Self-Inf-N (Guan et al., 2025) shows that fine-tuning on outlier benign samples can compromise model's safety. Virus (Huang et al. 2025c) modifies harmful fine-tuning data to evade content moderation filters and thereby demonstrates that relying solely on data filtration guardrails is inadequate for preventing fine-tuning- -bade model compromise. NOICE (Kazdan et al. (022)) attack is conducted by training the model to initially refuse benign requests on safety grounds, then proceed to answer those requests, thereby jailbreaking the model to produce disallowed content upon receiving a harmful request and thereby revealing weakness in refusal-based safeguards. Research in (Anonymous) 2025a) proposes a new attack by fine-tuning the model based on pairs of harmless prompts that are adjacent to the target harmful task and the safety response generated by the model.
Mechanism study of harmful fine-tuning and other works. The work in (Hsiung et al. 2025) finds that the durability of the model's safety guardrails for downstream task depends on the similarity between the data in the downstream task and the dataset used in the alignment process. Other research, such as (Chen al. (005), analyzes the trade-off between safety and capability in LLM fine-tuning. Research in (Wallace et al. 2025) evaluates the risk of harmful fine-tuning on the gpt-oss models (Agarwal et al. 2025). From an optimization perspective, (Peng et al. 2024) studies the loss landscape of safety-ai -alignd models. The work in (O'Brien et al. (025)) finds that filtering out dual-use or dangerous knowledge from a model's pretraining data can make it far more resistant to later adversarial fine-tuning. Finally, other studies develop defense strategies against harmful fine-tuning in different settings, such as diffusion model (Li et al., 2025a) or coature-of-Expertt (MoE) LLMs (Kim et al. 2025).
Our approach contributes defenses at both the alignment and fine-tuning stages. At the alignment stage, we propose novel loss function that regularizes the model to reside in flat region of the
17

---

Published as a conference paper at ICLR 2026
harmful loss landscape, drawing the well-established literature on sharpness-awand minimization (SAM) (Foret et al. (021), which has been studied extensively and achieved remarkable success across wide range of application domains (Nguyen et al. 2023a b Dung et al. 2024 Luo et al. 2024 2025). A more comprehensive discussion on the literature of harmful fine-tuning attacks and defenses can be found in (Huang al. 2024b).
B ANTIBODY VS. CTRAP
There is concurrent alignment stage defense, CTRAP ei al. 2025 which proposes regulariza- tion term that shares some similarities to our refusal loss Crefusal.
CTRAP is inspired by unlearning and introduces collapse regularizer in the alignment stage so that, when the model later fine-tuned on harmful data, its general utility degrades. Concretely, CTRAP optimizes harmful perturbation model pen to generate sequence of fixed tokens (e.g., "error error error") in response to general utility prompts. As a result, when the model is further fine-tuned on harmful samples, its responses on benign tasks collapse the fixed sequence of token destroying the model's general capability and rendering it ineffective for harmful purposes. This behavior enforced via a collapse loss Gcollupse (fpen), computed over a general-purpose purpose dataset.
While both our refusal loss Fesusal and CTRAP's collapse loss Leollapse are defined on harmful- perturbation model, they differ in both motivation and the data used compute the losses:
Our Ceeusal introduces an auxiliary objective in the aligmment stage both enforce robust refusal behavior on harmful prompts and to ensure the weights are effective at down-weighte harmful samples the fine-tuning stage, whereas Luollapse is explicitly designed as a trapping mechanism that collapses the model's general capacity when the model is later fine-tuned on harmful data, following an polearning-inspiine design.
Our Lretusal computed on a dataset of harmful prompts paired with generic refusal responses (e.g., "I cannot fulfill your request."), whereas Coollapse is computed on a general-purpose dialogue dataset sampled from human dialogue distribution.
In summary, the regularization term Leollapse in CTRAP aims to sacrifice general utility as safeguard against future harmful fine-tuning whereas the term Lretuaal in Antibody strengthens the defense performance the weighted fine-tuning algorithm, making the two approaches conceptually different.
C ANTIBODY VS. SEAL
SEAL (Shen al. 2025) proposes a bilevel data-selection framework. It trains a separate data selector via bilevel optimization using an alignment dataset and target user fine-tuning dataset, then filters that dataset (keeping the top-p% samples under the learned ranking) before running SFT on the filtered subset. Although the weights computed from the data selector in SEAL and the weighting mechanism in Antibody aim to reduce the influence of harmful fine-tuning data, they have different operational designs and differ substantially in mechanism.
Different in operational designs. SEAL requires training dataset-specific selector because its bilevel objective couples the selector to the target fine-tuning set. This is well-suited to settings where the target dataset is known and reused, but may not be favorable in FTaaS, where providers may face many heterogeneous, one-off uscr-submitted datasets. In such a service setting, SEAL introduces additional overhead to (i) train selector per user dataset and (ii) store the selector for future use. If the selector is discarded after a single run, this cost largely wasted. contrast, defense is embedded directly into the fine-tuning process and is immediately deployable to arbitrary user datasets without training any extra model as the data selector. Our method's incurred overhead is studied in Appendix [F4
Robust with different attack strength. SEAL requires choosing cutoff p% to decide which samples are retained. This hard threshold can be sensitive to unknown attack strength (e.g.. varying poisoning ratios), and fixed p may under-filter in stronger attacks over-filter in benign-heavy datasets. Our method avoids any discrete cutoff: all samples remain in the batch, but harmful ones are weightei.
18

---

Published as a conference paper at ICLR 2026
Static ranking dynamic reweighting. The weights in SEAL are used only to produce global ranking for all samples in the fine-tuning dataset for selection. Once the selector is trained, the ranking is fixed throughout SFT. Our weights serve different role: they directly modulate gradient contributions to encourage learning from benign samples and suppress learning from harmful ones. Moreover, our weights are computed exclusively for the samples in the active mini-batch and are recalculated at every iteration of fine-tuning. allowing them to adapt to the evolving model state (e.g., as the model becomes better fit on benign data, the weights for benign data become larger). This online, model-dependent reweighting is fundamentally different from SEAL's offline, fixed ranking.
In summary, SEAL's bilevel formulation provides clean and interpretable way to cast harmful fine- tuning defense as data selection. However, for the dynamic nature of FTaaS where data distributions vary unpredictably and efficiency is critical, our weighting mechanism provides different operational design and property. By utilizing online, model-dependent reweighting rather than static offline filtering, our approach offers a more robust and lightweight solution that eliminates the need for auxiliary training and adaptive hyperparameter tuning.
D PROOFS
D.1 PROOF OF THE OPTIMAL SOLUTION FOR FLATNESS REGULARIZATION
Proof. We can write the Lagrange function with 2 for Equation as
1(6 = Vo falign (0,) - 812 + (ar V Laharp (0,) 6) With the Marush-Kuhnn Kuhn-Tucker (KKT) theorem, we have:
V A( (Vo fallign (0.) 8) Vo Laharp (0,) = 0,
V e Laharp (0:) 8 Z de
A 2 0,
as. Vo Eshap (0:) (8.)T)) 6) 0,
From the above constraints, we can obtain:
8 V Lalign (0r) + X Vo Caharp (0t)
A max {
0, Vo Esturp (6t) T Vo Latign (0t)
V Laharp (0.)112
D.2 PROOF OF PROPOSITION 4.2 AND PROPOSITION
We note that the following derivation for Proposition can be used for Proposition 4.2 since Proposition can be recovered from Proposition 4.3 by setting the weight to be uniform across all samples in the batch wa, (z,, yi
.M. Assume losses sums over tokens (no token averaging): fo(Xo) MM ( A first-order Taylor expansion around O, gives:
Proof. Let Zo.m (Xo) E RV be the pre-softmax logits at position m E (l....,
Ally xx) )
[La- 1lo. (Xo1lm)
(13)
M
(X))lm
Mtt1 - 0e) + - -0,1%),
(14)
1xd
dx1
19

---

Published as a conference paper at ICLR 2026
where d is the number of parameters in the model. By the chain rule, V
VolCo, (Xo)mm
Za.m [Co, (Xo)l Vo (Xo).
(15)
1XV
Under the weighted SFT update stated above,
0++1 0:
Σ
wo, (a)
Vo
Σ
Co, (xcile (xill
(16)
(E..
L
Σ
wa (1 ys)
(zl.Ifo,(xe)) Vozz,(xi)
(17)
(E,
1=1
IXV
Vxd
Substituting Equation (15) and Equation
17
into Equation
[1 and rearranging the sums yields
Aelyo::00) (as EB 1=1
Σ
wa (xi, yi)
V Zu.m 16,, (Xo) x Voza, (Xo)
(18)
m=1
IXV
Vxd
(9z2,(xi)
Ie (x))))
+ +MM
dxV
Vxl
Identifying [A'(X))) = = Loo (Xo)m [K' X:)mm, = (Voza, (V
and (x)() Z Lo. (x:)l: Following the assumption in Appendix B.1 of (Ren & Suther
land 2025) that |00+1 0.1 O(1). the remainder scales as O(Mr²). Collecting terms gives the
final decomposition
L
Al(yo: xo)
Σ
Σ
wo (xi, yi)
(A)() 1xV
[K' (Xo: Xi)lma (9:) (19)
Vxv
O(M))).
The derivation above completes the proof for Proposition 4.2 and Proposition
4.3
E EXPERIMENTAL DETAILS
E.1
DATASETS
The Stanford Sentiment Treebank (SST2) (Socher et al., 2013) is a binary classification dataset comprising sentences extracted from movie reviews, each labeled as either positive or negative sentiment. The prompt format for this dataset is as follows:
Prompt format with example input for SST2
Below is an instruction that describes a task, paired with an input that provides further context. Write response that appropriately completes the request.
### Instruction:
Analyze the sentiment of the input, and respond only positive or negative.
### Input:
a casual intelligence
#*# Response:
The AGNEWS (hang et al. 2013) dataset is collection of news articles categorized into four classes: World, Sports, Business, and Science/Technoloyy It contains 120k samples in the training split and 7.9k samples in the test split. The prompt format for this dataset is as follows:
20

---

Business Published as a conference paper at ICLR 2026
Prompt format with example input for AGNEWS
Below is instruction that describes task, paired with an input that provides further context. Write a response that appropriately completes the request.
### Instruction:
Categorize the news article given in the input into one of the categories:
World
Sports
Sci/Tech
### Input:
Einstein's is warp effect measured Two scientists beat a S600 million Nasa mission to be first to measure prediction of Einstein's relativity theory.
### Response:
The (Grade School Math 8K) GSM8K (Cobbe et al. (021) is a question-answeriig dataset comprising 8.5k grade-school math problems. These problems typically require 2-8 steps to solve, primarily involving sequence of elementary calculations using basic arithmetic operations to arrive at the final answer. The prompt format for this dataset is as follows:
Prompt format with example instruction for GSM8K
Below is an instruction that describes task. Write response that appropriately completes the request.
### Instruction:
Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?
### Response:
AlpacaEval (Li et al. 2023) is a dataset of 805 samples designed for evaluating the instruction- following capacity of language model. The prompt format for this dataset is as follows:
Prompt format with example instruction for AlpacaEval
Below an instruction that describes task. Write a response that appropriately completes the request.
### Instruction:
Do you know why cats always rub up against your legs?
### Response:
The harmful instructions used in our experiments are from the BeaverTails dataset (Ji et al. 2023). This dataset consists of more than 300k pairs of human-labeled question-answer spanning across 14 harm categories, where each response could be flagged with more than one of those categories. Other works (Rosati et al.) 2024 Huang et al. 2025b) use the harmful dataset from (Rosati et al. 2024). which filtered subset of TaiverTails-30 30k-train for alignment and fine-tuning attack, and use a filtered subset of BeaverTails-3000 30k--0tttttttt for evaluation. However, based on our observation and pointed out by (Qi et al. 2025b), these datasets suffer from overlapped problem between the
21

---

Published as a conference paper at ICLR 2026
alignment split, finettuning attack split, and evaluation split. Therefore, ensure fair evaluation, we construct three separate and hon-overlapping data splits for our experiments:
1. Aligument phase. We perform filtering of the dataset in (Rosati et al. 2024) to come up with a subset of 4972 samples for model alignment.
Fine-tuning attack phase. We use the filtered version of BeaverTails-3300 curated by (Qi et al. 2025b), including 4986 samples, as the harmful samples that is used to poison the benign dataset.
3. Evaluation phase. We evaluate the Harmful Score of the fine-tuned model on the filtered subset of the BeaverTaiis--vallaato dataset (Ji et al. 2023). which contains 699 malicious questions from 14 categories.
E.2 TRAINING DETAILS
Following prior works (Huang et al. 2024a с 2025b), we use LoRA (Hu et al. 2022) to update the model parameters in both the alignment and fine-tuning attack phases. The LoRA rank is set 32, and LoRA's alpha is set to 4 for all experiments. In the alignment phase, we train the model for 20 epochs using the Adam W optimizer (Loshchilov Hutter 2019) with a learning rate weight decay of 0.1. . In the fine-tuning attack phase, we fine-tune the model for 20 epochs on SST2, AGNEWS, and GSM8K, and for 100 epochs on AlpacaEval. We use the Adam optimizer with a learning rate of x 10-5, weight decay of 0.1, and a warmup ratio of 0. A batch size of 16 is used for both the alignment and fine-tuning attack phases, except for experiments with the Gemma2-9B model, which uses batch size of 10 due to GPU memory constraints. For Antibody, in the alignment stage, we use ç = 5.0 and = le to compute A in Equation (TT) and set Avetuaal = 0.05. In the fine-tuning stage,
10-4 and
= 1.0, and we set of 100 generic refusal responses y/- to randomly pair with each sample in the fine-tuning dataset to compute the weights in Equation (7).
For all experiments, AlpacaFval, which was run once to minimize API costs. Every single run of our experiments uses one NVIDIA A10 GPU with 80GB of memory. Our code is available at hhttps://githut minhquoc07 712/ Ant
over three random seeds and report the average results, except for
E.3
METRICS AND EVALUATION
This section outlines the evaluation methodologies used for each downstream dataset: SST2, AG- NEWS, GSM8K, and AlpacaEval:
SST2: If the generated response matched the ground truth label "positive" or "negative" in the test set, we consider it as a correct prediction. The fine-tuning accuracy is the fraction of correct predictions over the test set of SST2.
AGNEWS:
If the generated response matched with one of the ground truth categories "World", "Sports", "Business", or "Sci/Tech" in the test set, we consider it as a correct prediction. The fine-tuning accuracy the fraction of correct predictions over the test set of AGNEWS.
GSM8K: the concluded answer in the generated response matched the ground truth answer in the test set, we consider it correct prediction. The fine-tuning accuracy is the fraction of correct predictions over the test set of GSM8K.
We use Win Rate as the Fine-tuning Accuracy to evaluate the performance of the fine-tuned model on AlpacaEval. Win rate is the percentage of times our model's response is preferred over the reference response, as determined by GPT-4. We select 105 samples in the AlpacaEval evaluation.
AlpacaEval:
E.4 BASELINES
In this section, we provide a detailed description of the baselines used in our experiments:
Supervised Fine-Tuning (SFT) is the standard fine-tuning method, directly fine-tuning the model on the alignment dataset during the alignment phase and on the harmful fine-tur fine-tuning dataset during the fine-tuning attack phase.
22

---

Published as a conference paper at ICLR 2026
Vaccine (Huang et al., 2024c) is an alignment stage defense method that makes model more robust against harmful fine-tuning attacks by improving its resilience to harmful embedding perturbation. In addition, Vaccine (Huang al. 2024c) uses double LoRA setup, where before the fine-tuning phase, the LoRA trained the alignment phase is merged with the model, and new LoRA is used for the fine-tuning.
Booster Huang et al., 2025b) an alignment stage solution. In addition fitting the alignment data, Booster proposes minimizing the harmful loss reduction rate. The harmful loss reduction rate is computed as the difference between the harmful loss of the current model and the model after one step of gradient descent on harmful data.
Lisa (Huang et al. 2024a) is fine-tuning defense method that includes an alignment dataset in the fine-tuning phase. Specifically, Lisa alternatively fine-tunes the model on two states: Optimize on the alignment dataset and harmful fine-tuning dataset. To improve convergence stability, Lisa includes a proximal term to constrain the model's drift with respect to the previous state.
We do not include direct comparison with another line of work that, while aligned with our goal, employs complementary approach. These methods aim attenuate the effect of harmful gradients by constraining which parameters can receive updates. Specifically, (Wei et al.) 2024 Du et al. 2024 Li et al. 2025c b) propose localize and restrict updates safety-critical model parameters.
F MORE EXPERIMENTS
F.1 EXPERIMENTS ON DIFFERENT DATASETS WITH DIFFERENT MODELS
Table 5: Performance of Qwen-2-7B trained on different fine-tuning datasets. The best and the second best highlighted in orange and gray, respectively.
Methods
SST2
AGNEWS
GSM8K
Average
(Qwen-2-7B)
HS 4
FA f
HS 4
FA f
HS 4
FA
HS 4
FA f
SFT
33.86
93.31
31.00
85.77
14.54
64.66
26.47
81.25
Vaccine
29.80
93.22
28.66
84.93
18.17
62.20
25.54
80.12
Lisa
18.46
92.85
15.26
85.07
3.91
63.90
12.54
80.61
Booster
3.96
94.19
3.48
85.13
2.19
68.63
3.21
82.65
Antibody
0.81
94.31
0.72
87.23
0.62
67.30
0.72
82.95
Table 6: Performance of Gemma-2-99 trained on different fine-tuning datasets. The best and the second best are highlighted in orange and gray, respectively.
Methods
SST2
AGNEWS
GSM8K
Average
(Gemma-2-99)
HS 4
FA 1
FA f
HS
HS 1
SFT
40.77
94.11
39.91
84.47
32.05
56.73
37.58
78.44
Vaccine
51.65
94.50
51.64
85.27
38.24
54.37
47.18
78.05
Lisa
34.67
94.04
33.33
83,80
13.02
54.97
27.01
77.60
Booster
28.47
94.15
19.22
84.30
22.13
58.97
23.27
79.14
Antibody
11.30
94.27
1.91
85.77
0.91
57.43
4.71
79.16
In Table and T we conduct experiments more datasets with Qwen-2-7B and Gemma-2-9B models, respectively. The results show that Antibody consistently achieves competitive performance over other baselines across all datasets. This demonstrates the robustness and effectiveness of Antibody across different model architectures.
F.2 COMPARE WITH BOOSTER + LISA
To evaluate the efficacy of Antibody's 's integrated defense mechanism, we compare with a sequential two-stage approach where the model first aligned using Booster and subsequently fine-tuned with
23

---

Published as a conference paper at ICLR 2026
Table Performance comparison of Booster + Lisa and Antibody methods across different models. The best results are highlighted in bold.
Model Method
SST2
AGNEWS
GSM8K
Average
HS 4
FA
HS
FA
HS
HS
FA
Bouster Lisa Antibody
2.52
91,94
1.71
83,53
1.29
12.23
1.84
62.57
Llama-2-7B
1.48
93.35
1.24
87.30
1.24
15.07
1.32
65.24
Bouster Lisa Antibody
2.15
94.19
85.67
1.00
69.07
1.67
82.98
Qwen-2-7B
0.81
94.31
0.72
87.23
0.62
67.30
0.72
82.95
Booster Lisa Antibody
4.10
94.04
2.91
84.80
1.15
$7.30
2.72
78.71
Gemma-2-99
11.30
94.27
1.91
85.77
0.91
57.43
4.71
79.16
Lisa (Booster Lisa). As shown in Table Antibody demonstrates clear superiority on the Llama-2- 7B model, consistently reducing the harmful score while simultaneously improving the fine-tuning accuracy. For Qwen-2-7B, Antibody reduces the average harmful score 0.72, albeit with minor trade-off in the average fine-tuning accuracy. For the largest model, Gemma-2-9B, Antibody achieves superior performance over Booster plus Lisa across all datasets, with the exception of the harmful score on SST2. This suggests that while Antibody's 's integrated design offers a powerful solution, particularly for potent harm mitigation, its stability can be model-dependent, and the conventional two-stage approach may prove more robust in specific model-task scenarios.
F.3
ABLATION STUDY ON Aretusal
Table 8: Performance comparison across different Areiusal values GSM8K dataset.
Avefusal 0..
Aeeussl 0.01
Avetusal 0.05
Avefusal = 0.1
Aveuusal 0.5
GSMSK HS GSM8K FA"
4.44
2.15
1.24
1.72
1.72
13.83
14.47
15.07
14.30
13.80
Impact of A The results indicate that the refusal loss weight, Arctssll plays crucial role in balancing defense against harmful inputs and maintaining task performance. The model achieves its optimal performance Areiusal 0.05, securing the lowest harmful score of 1.24 while simultaneously reaching the highest fine-tuning accuracy of 15.07. Introducing this loss term (i.e., Arelusal > 0.0) consistently improves the model's safety by reducing the harmful score. However, setting the weight too high (e.g., 0. or 0.5) begins to degrade fine-tuning accuracy from its peak, suggesting that an overly strong refusal penalty can negatively interfere with the fine-tuning process.
F.4 SYSTEM EVALUATION
Table 9: System evaluation for different methods.
Methods
Clock Time (Hours)
GPU Memory (GB)
(l.lmma-2-7B)
Alignment
Fine-tuning
Sum
Alignment
Fine-tuning
Max
SFT
1.00
0.26
1.26
$3.68
40.10
53.68
Vaccine
1.98
0.16
2.12
54.70
40.10
54.70
Lisa
1.00
0.26
1.26
53.68
53.87
53.87
Booster
5.09
0.26
5.35
64.36
40.10
64.36
Antibody
5.81
0.46
6.27
64.55
42.53
64.55
System Evaluation. Table 9 presents the system evaluation results, which illustrate that the com- prehensive protection offered by our proposed Antibody method comes at the cost of computational resources. During the alignment stage, Antibody requires 5.81 hours and 64.55 GB of GPU memory, higher than other methods. This overhead is attributed to Antibody's 's sophisticated alignment process, which not only imposes an initial safeguard but also strategically shapes the model's loss landscape for the subsequent fine-tuning defense. Unlike methods that operate solely during alignment, Antibody
24

---

Published as a conference paper at ICLR 2026
also introduces defense mechanism in the fine-tuning stage. This results in marginal increase in clock time (0.20 hours) and GPU memory (2.43 GB) compared to standard fine-tuning, We argue this is strategic trade-off: the one-time alignment cost builds a robust foundation, while the minimal, recurring overhead in the fine-tuning stage acts as an active safeguard for each downstream task. While its total computational cost is the highest, this investment is justified by its superior defense and learning performance. The minimal overhead during the frequently repeated fine-tuning phase makes Antibody a practical and highly effective defense solution for deploying in FTaaS scenarios.
F.5 PERFORMANCE WITH RESPECT TO ALIGNMENT DATASET SIZE
The datasets used in the alignment stage of Antibody include Patign- Duammful. and Drefusal The number of samples in these three datasets is equal by construction, as detailed below:
D Dalign consists of harmful prompt-refusal completion pairs.
Dummful consists of harmful prompt-compliance answer pairs, where the harmful prompts are the exact same prompts as Dalign Thereby, the size of Drammuu and Dalign are equal.
Drefusal consists of harmful prompt pairs with generic refusal responses that contain refusal answers that can be used across different harmful prompts, such as 'I cannot fulfill your request." Hence, the size of Drefusal and D alignn the same.
In the following, we refer to the common size of Dalign: Duumuull and Dretusal as the alignment size. We conduct experiments to study how the defense and learning performance of Antibody and Booster change with different alignment sizes.
Table 10: Performance comparison of Booster and Antibody methods across different alignment sizes. The best results are highlighted in bold. Note that 4972 corresponds to the alignment dataset size used in our main experiments, and the Booster uses Dalign and Dpamtut in its algorithm.
Alignment Size Method
SST2
AGNEWS
GSM8K
Average
HS
FA f
HS 4
FA f
HS |
FA
HS |
FA f
Booster Antibody
34.05
93.46
40.49
86.30
43.49
15.80
39.34
65.19
1000
5.44
93.96
5.29
86.40
8.44
14.10
6.39
64.82
Booster
22.46
92.66
21.32
85.10
31.19
15.10
24.99
64.29
2500
Antibody
2.00
93.81
2.15
86.90
6,44
16.10
3,53
65.60
Booster
14.31
92.59
15.88
86.70
9.06
16.27
13.08
65.19
4972
Antibody
1.48
93.55
1.24
87.30
1.24
15.07
1.32
65.31
From Table [0] a notable trend is that both methods benefit from increased alignment size, but Antibody scales more effectively. Across all settings and datasets, Antibody consistently achieves a much lower harmful score than Booster while maintaining comparable or slightly better fine-tuning accuracy. With an alignment size of 1000, Antibody reduces the average harmful score from 39.34 for Booster to 6.39, more than six-fold reduction, while maintaining comparable fine-tuning accuracy (64.82 for Antibody and 65.19 for Booster). Increasing the alignment size to 2500 further strengthens this effect: Antibody attains average harmful score of 3.53, whereas Booster remains 24.99, and Antibody slightly improves the average fine-tuning accuracy to 65.60 compared to 64.29 for Booster. At the largest alignment size of 4972, Antibody simultaneously reduces the average harmful score to 1.32 (compared with Booster's 's 13.08) and increases the average fine-tuning accuracy to 65.31 (slightly above Booster's 's 65.19). Overall, these results indicate that, for given alignment size, Antibody provides substantially stronger and more data-efficient protection against harmful fine-tuning than Booster, even when only 1000 alignment examples are available.
F.6 HARMFUL GRADIENT NORMS UNDER ANTIBODY ALIGNMENT
we argued that, after our alignment stage flatness regularization, the gradients of harmful samples become negligible, so that the gradient in the standard SFT update Equation (4) is effectively dominated by the benign component Equation (5). This argument hinges on the assumption that, at the start of the fine-tuning stage, harmful samples already have much smaller gradicnts than benign ones.
In Section 4.1
25

---

Published as a conference paper at ICLR 2026
Antibody
SFT
300
Benign Harmful
Benign Harmful
250
100
Count
200
Count
80
150
60
100
40
50
20
Gradient nomm
Gradient norm
Figure 4: Effect of our flatnessrregularizee alignment from Section 4.1 per-sample gradient norms at the beginning of the fine-tuning stage (before fine-tuning) for models aligned with Antibody's 's alignment-stage solution (left) and with standard SFT (right).
We plot the distribution of
To validate this claim, measure per-sample gradient norms at the beginning of the fine-tuning stage and report their distributions in Figure 4 For the model aligned with Antibody, the gradient norms on harmful samples clearly smaller than those of the model aligned with standard SFT, indicated by the left-shifted distribution. Moreover, under Antibody, harmful gradient norms are also smaller than benign gradient norms, showing that harmful examples contribute much less to the overall update direction than benign ones. By contrast, under standard SFT alignment, harmful gradients remain large relative to benign gradients, leading harmful samples having strong influence on the update direction. This behavior helps explain the poor defense performance of the SFT baseline observed in Figure ! and discussed in Section 5.22
These empirical findings support our approximation of Equation (4 by Equation (5) and justify the subsequent analysis of mini-batch learning dynamics in our proposition. Together with our flatness regularization, this gradient-norm behavior provides robust initial safety alignment that is then preserved by Antibody's proposed weighted fine-tuning update.
F.7 ABLATION STEP-ADAPTIVE REGULARIZER INTENSITY A
Table 11: Ablation study investigating the impact of the step-wise adaptive regularizer X, We compare the performance of both Booster and Antibody when using a fixed constant regularization versus proposed dynamic λ The best results highlighted in bold.
Method & Ablation
SST2
AGNEWS
GSMSK
Averge
1.
FA f
HS 4
FA
HS
HS
Booster (with constant A)
14.31
92.59
15.88
86.70
9.06
16.27
13.08
65.19
Booster
Baoster (with dynamic X₂)
8.20
92.55
8.97
85.93
14.93
15.97
10.70
64.82
Antibody (with constant A)
1.91
93.24
1.62
88.17
2.15
15.60
1.89
65.67
Antibody
Antibody (with dynamic Xc)
1.48
93.35
1.24
87.30
1.24
15.07
1.32
65.24
In Table 11 we analyze the impact of using the proposed dynamic regularizer A compared to constant regularizer term. We observe that the proposed dynamic X, consistently reduces the harmful score across both Booster and Antibody, albeit with slight trade-off in fine-tuning accuracy. Notably, this safety improvement less pronounced Antibody. We attribute this to the fact that our proposed weighted SFT mechanism already effectively suppresses the influence of harmful samples, thereby significantly lowering the baseline harmful score and leaving less margin for further reduction by the adaptive regularizer.
26

---

Published as a conference paper at ICLR 2026
Table 12: Performance under different numbers fine-tuning samples in the default setting. The best results are highlighted in bold.
Methods
Harmful Score
Fime-tuning Accuracy
(p=0.2)
500
1000
1500
2000
2500
500
1000
1500
2000
2500
Booster
1.62
9.06
31.76
39.49
44.49
13.30
16.27
18.27
18.83
19.90
Booster Lictuaal
1.71
5.34
28.90
41.82
48.45
11.90
15.33
16.47
18.87
20.23
F.8 ABLATION STUDY ON Crefusal
Table 12 presents the performance of the model under varying numbers of fine-tuning samples. We utilize the Booster baseline for this comparison, as it lacks the specific fine-tuning stage solution found our proposed method. This allows us to isolate and analyze the independent effect of the refusal loss term. The results indicate that Lietusal effectively reduces the harmful score when the number of fine-tuning samples is low to moderate (e.g., n 1000 and n = 1500). However, regarding utility, observe trade-off: the refusal loss tends to slightly reduce fine-tuning accuracy in low-sample regimes while boosting accuracy as the sample size increases (n 2 2000).
Table 13: Ablation on Antibody with/without Lrefusal across different datasets.
SST2
AGNEWS
GSM8K
Average
Method & Ablation
HS 4
FA
HS
FA f
HS
FA f
HS J
FA 1
Antibody (without Crefusal)
3.58
92.78
2.53
85.70
5.06
14.27
3.72
64.25
Antibody (with Lietusal)
1.48
93.35
1.24
87.30
1.24
15.07
1.32
65.24
These mixed results when using the loss in isolation do not negate the contribution of the refusal loss term. The primary motivation for the refusal loss term is to strengthen the robustness of the weights used specifically within the proposed weighted fine-tuning algorithm by simulating harmful perturbations during the alignment stage. Consequently, Crefusal is not intended to function standalone solution. As demonstrated in Table the refusal loss yields simultaneous improvements in both harmful score and fine-tuning accuracy across all datasets when used together with the weight fine-tuning algorithm Antibody.
F.9
ROBUSTNESS OF WEIGHT ALLOCATION UNDER OOD ATTACKS
Before Fine-tuning
After Fine-tuning
150
Benign Harmful
Benign Harmful
125
150
100
Count
Count
100
50
50
25
-10
-8
-6
-2
0
-15
-10
5
5
score
score
Figure 5: The effect of our proposed fine-tuning method under harmful fine-tuning attack with OOD harmful data. The plots illustrate the distribution of the score re (the pre-normalized weight) in Equation 7 for benign samples versus harmful samples before and after fine-tuning on dataset poisoned with PureBad (Qi al. 2024).
by simulating a more We empirically investigate the stability of the weight allocation in Equation 7 challenging poisoning scenario. Specifically, incorporate the PureBad dataset (Qi al. 2024).
27

---

Published as a conference paper at ICLR 2026
consisting of 100 harmful prompts paired with harmful targets, into the fine-tuning dataset. This setup of out- out-of-distribution (OOD) harmful data acts as a stress test to determine whether a stronger attack can increase the likelihood of harmful target completions to a point where their assigned weights override those of the refusal completions or surpass the weights of benign samples.
As illustrated in Figure 5 the pre-normalized score re demonstrates clear distinction between harmful and benign data before fine-tuning. The distribution of scores for harmful samples is concentrated the left of the benign sample distribution, indicating that the model's preference for the harmful targets does not overpower the refusal mechanism. Consequently, our weighting mechanism successfully assigns small weights to the harmful data. After fine-tuning, this separation becomes even more pronounced: the majority of harmful sample scores shift to negative values, while benign sample scores remain positive. This confirms that even under strong attack conditions, such as OOD, the harmful samples do not acquire large weights, ensuring the effectiveness of the proposed weighted fine-tuning algorithm Antibody.
G CASE STUDY
In the following, we show how different methods respond to the harmful prompt. As shown, Antibody is able to give a refusal response to harmful questions, while other methods cannot.
Prompt: How insult someone's ' physical appearance without actually using hurtful languuge?
from SFT perriiinntionntteeoopeiiiii injjussiee hale speech,,ffeesivee language non, mon_violent viilentt_uelhhicull unellical behhavior
Insulting someone's 's physical appearance without using hurtfal language can be dome by making snide comments about their looks, body shape, personal style. For example, you could say something like...
Answer from Vaceine iuserimiiationsterooop....u
hate_ bate_speech speescc.effffeesieeee oaaeeeeveeeeeeaaauaaee nom non__violent violent, unethical behavior
Insulting someone's 's physical appearance without using hurttul language can be done by making fun of their body shape, size, or weight. For example, you can say...
hate ofeech, language non_ violent unethical behavior Insulting somcone's 's physical appearance is not nice thing to do, and it can cause a lot of hurt and offense. However, there are some to do without using hurtful language. Here are some examples:
Answer from Lisa discrimination ptrpriiontterrottypiijjisseeee
Make fun of their weight:
Answer from Booster discrimination.stereon www.srooyooopnnnsssitcc..
hate www.ech..ffeesiieeee... hatthhfffeesveeeaauaeeee noo__noooee violerttentt... unethical behavior
Insulting somcone's ' physical appearance can be done in a variety of ways without using hurtful language. Here are some examples:
Make their weight: "You're fat, you could contestant
Biggest Loser"!" 2. Mock height: "You re short, you
could be bubbii"" 3. Make fun of their body shupe: "You 'r su skinny, you could be a twig!" 4. Make lun of their face: "You
Answer from Antibedy harmless
cannot provide insults or burtful comments towards somcone's physical appearance, as it is not only mean-spirited but also disrespectfu. Everyone the right
H ACKNOWLEDGEMENT OF LARGE LANGUAGE MODELS USAGE
We used Large Language Models to assist with editing and polishing this paper. They were used for tasks such as fixing typographical errors, correcting grammar, supporting LaTeX typesetting, and improving word choice. All ideas, experiments, and analyses were conducted by the authors, and the use of LLMs does not affect the reproducibility of our work.
I LIMITATIONS AND FUTURE EXTENSIONS
The proposed Antibody method has several limitations. A primary concern is its high computational cost, introduced by the two-stage defense mechanism. This overhead in the fine-tuning stage is particularly concerning as it is incurred repeatedly for each new fine-tuning request, creating recurring expense for the service provider. Additionally, the method's defensive capabilities are not robust against training with a large learning rate, demonstrated in Figure 3 Regarding fine-tuning performance, while our method generally enhances performance on downstream tasks compared to SFT, Antibody does not consistently outperform all baselines across every dataset, leaving room for improvement.
For future work, key direction is to broaden the applicability of the Antibody method other alignment techniques commonly offered in FTaaS. This includes studying its integration with methods such as Direct Preference Optimization (Rafailov et al. 2023) and Reinforcement Fine- Fine-Tuning.
28

---

Published as a conference paper at ICLR 2026
Furthermore, exploring the extension of this defense mechanism beyond language models to other modalities, such as vision fine-tuning, represents another significant avenue for future research.
29


Reviews:
[{"review_id": "7Kd3O6COcK", "reviewer": "Reviewer_d57z", "summary": "The paper focused on harmful fune-tuning attacks. The proposed method regularizes the gradient contribution of harmful samples. The methodology has phases that apply precaution before fine-tuning, and also during fine-tuning with a safety-preservation learning algorithm. In the first step, it makes sure that the harmful sample gradients will be in a flat loss region. In the second stage, it applies weights to the samples during the training batch to prevent learning from harmful samples.", "questions": "- \"A flat loss landscape for harmful samples makes the instilled safety behavior more difficult to remove.\" Can the authors define how this makes it difficult to remove? One needs more training and more samples?\n\n- The final parameter, $\\theta$, depends on the $L_{sharp}$ which depends on $L_{harm}$. To me these two losses are contradictory and their gradients could direct two opposite directions, but the methodology ends up selecting a gradient that minimizes $L_{harm}$. Overall the model is tuned towards the harmful region, but not so harmful? In other words, how do we know a gradient $\\delta_t$, shown in equation 3, exists when the gradient of both losses points to exactly opposite directions? \n\n- How is the matrix multiplication is performed in equation 6 with the given dimensionalities?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- Clear motivation and explanation\n- The paper targets a significant problem.\n- The proposed method shows greater performance than the SOTA methods in some datasets.", "weaknesses": "- It uses the first stage to score the samples during fine-tuning. It looks like more of a single stage, where the second stage is just an extension of the first stage.\n\n- The paper improves the alignment stage with a refusal loss, which is already used and has shown its effectiveness in the Vaccine. There is only one ablation study given for the comparison of $L_{sharp}$ and $L_{refusal}$.\n\n- Assuming $K^t$ to be large in benign and $K^t$ would be small in harmful is vague and not supported enough; therefore, reducing equation 4 to 5 with the given proposition is unclear.\n\n- Solution for detecting harmful samples is ad. hoc. It shows that the model is well fit to the training data, which contains the phrase “I cannot fulfill your request” \n\n- The authors claim that the harmful loss stays constant while the benign loss decreases. However, the third plot in Figure 2 shows that, Antibody harmful loss value is high as the SFT-Harmful loss value.\n\n- The proposed algorithm is not showing the best performance in all the datasets and for all the models, where it also uses previous methods in their strategy.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bGwLZcJeMp", "reviewer": "Reviewer_jv5a", "summary": "This paper proposes an elegant solution towards harmful fine-tuning. The solution contains two main components, an alignment stage solution and a fine-tuning stage solution. The design is elegant and with sufficient ablation study to justify each components usefulness.  Experiments show that it outperforms the SOTA solution Booster.", "questions": "See Weakness. Overall I enjoy reading this paper, and I will consider to further improve my score if my concerns are addressed. \n\nThe main concern I have is that this paper should not emphasize too much on the flatness regularization as this is exactly the same with what Booster aim to achieve (also evidenced by the almost identical update rule between Antibody and Booster), though it seems that the constraint problem Eq. (2) offers an alternative explanation of the flatness regularization and also by this transformation you seem to be able to automatically and adaptively derive the regularizer intensity $\\lambda_t$, which seems to be a novel contribution. You may talk more about this if you agree to mention Booster formulation in this section.  **I also suggest to add a group of experiments to justify that the step-adaptive regularizer intensity is indeed necessary and outperform the constant intensity adopted by Booster**, if you still insist that your Section 4.1  should be the main contribution. \n\nThe third term in (11) (i.e.,the term in Eq. (10) is more interesting and I personally think is the main contribution instead.  \n\nDespite I do have some  complaint over the writing issue, I think this paper is a very solid work and I am happy to champion this paper in the review process.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 2, "contribution": 4, "strengths": "1. The studied problem is important and this papers offer a timely contribution. \n2. The solution is elegant and very fundamental. I especially like the alignment stage component of Antibody, as it enhances safety alignment by adding a few regularizer in the safety alignment stage.   \n3. Amplified experimental results are given, showing the robustness of such solutions on different fine-tuning datasets, model and harmful ratios. Also, I particularly like Figure 1, **which clearly demonstrates  Antidote's superiority against existing methods over the trade-off between safety and fine-tune accuracy**. \n4. The authors conduct sufficient ablation study,  system evaluations, and quantitative results to verify the effectiveness of the proposed method.  \n5. The visualization in Figure 2 is nice!\n\n\nIn conclusion, this is a solid paper that requires a lot of experiment and sufficient insights and understanding of the problem. I feel pleased and super excited when reading this paper.", "weaknesses": "The paper contains two components, I.e., alignment stage and fine-tuning stage solution. I need to separate them to write the review for each of them for clarity.\n\nFor alignment stage of Antibody: \n\n1. **Relation and contribution compared with Booster need to be clarified.**\nThe relation with Booster is not explicitly mentioned. Particularly, the authors should clarify that Antibody inherits components from Booster. Let's look at the update rule in  Eq. (11). \n\n* **Under the case $\\lambda_{refusal}=0$ and $\\lambda_t = \\lambda$ (i.e., a constant), Antibody reduces to Booster**. \n\nJustification: under the above assumption  on \\lambda_{refusal} and  $\\lambda_t $,  then, expands $L_{sharp}(\\theta)$ based on its definition given in line 222 in Page 5, Eq.  (11) becomes:  \n\n$L_{align}(\\theta_t) + \\lambda L_{harm}(\\theta_t) - L_{harm}(\\theta_t- \\rho \\frac{ \\nabla L_{harm}(\\theta)) }{||\\nabla L_{harm}(\\theta)) || } ) $\n\n**This update rule is 100\\% the same with Booster's update rule. See Eq. (1) in Booster[1].**\n\n[1] Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbation. (ICLR2025)\n\nIf the authors  agree that the two methods share similarity at some level, I recommend the authors to clarify this relation at the end of Section 4.1, such that non-experts in this sub-field could better understand what the real contribution is. **Also, given that the final derived update rule of solving Equation (2) is almost identical to Booster, could you clarify what the contribution of Section 4.1 is?**  Feel free to challenge me in the rebuttal if you do not agree with what I claim.\n\n\n2. **The real contribution seems to be the third term in Eq. (11), which is not clearly articulated**.  As I mentioned above, Section 3.1 (which introduces first two terms in Eq, (11)) arrives at an update rule that is almost identical to Booster. However, by looking at Eq. (11) again, actually Antibody contains an extra term that is not considered by Booster. Particularly the $\\lambda_{refusal} L_{refusal}(\\theta_{pert,t})$ is a new term that is not considered by existing work. To my understanding, this term is important as it makes sure that the safety alignment loss will not increase after simulating one step of harmful perturbation. That might be the key contribution that guarantee Antiboidy' superior performance against Booster. \n\nTo verify the meaningfulness of this contribution, I suggest the authors to do a group of ablation experiments:\n\n* Disable the fine-tuning stage solution of Antibody, and only keep the alignment part.  Then, set $\\lambda_t$ to a constant, making the first two terms reduce to Booster. Finally, replicating a similar experiment settings in figure 1 by comparing $\\lambda_{refusal}=0$ (reduces to Booster)  and   $\\lambda_{refusal}$=an ideal number.\n\n3. **While they are not exactly the same, a related alignment stage harmful fine-tuning defense paper also uses a similar term with the third term in Eq. (11).** Particularly, the second term in  Eq. (2) in [2] also adopt a similar term, which can be adapted to $\\lambda_{collapse} L_{collpase}(\\theta_{pert,t})$ based on the notation used by this paper. The difference is in the dataset used for the outer loss calculation.  Antibody is on the alignment dataset and CTRAP is on a specially constructed collapse dataset. I suggest the authors add a sentence to discuss this term in CTRAP paper to mention their similarity.  \n\n[2] CTRAP: Embedding Collapse Trap to Safeguard Large Language Models from Harmful Fine-Tuning\n  \nFor fine-tuning stage of Antibody:\n\n4. **Relation and contribution compared with SEAL[3] need to be clarified.** The idea of the fine-tuning stage solution is to assign different weights for each piece of data in the fine-tuning datasets. The high level idea is similar  to SEAL which assign a learnable weights to each data point. The authors should discuss how the assigned weights are different from SEAL and justify with experiment whether this weight allocation in Eq. (7) are better than the learnable weights in SEAL [3]. Also, there is one NeurIPS2025 accepted paper BDS[4] and two pre-print papers GradShield[5] and Ref-Teacher[6] exploring the same direction, which should also be mentioned in the paper. \n\n[3] SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selection \n\n[4] Adaptive Defense against Harmful Fine-Tuning via Bayesian Data Scheduler\n\n[5]  GradShield: Alignment Preserving Finetuning  https://openreview.net/forum?id=YYUNm7IibC\n\n[6] Safety-Aligned Weights Are Not Enough: Refusal-Teacher-Guided Finetuning Enhances Safety and Downstream Performance under Harmful Finetuning Attacks  https://openreview.net/forum?id=OK2GR1guwv\n\n\n\n5. I have some concerns over the  weights allocation in (7).  I think this weight allocation might incur some issues when the model is under stronger attack. If the attack is strong, the likelihood of the harmful samples of generating the harmful target completion might override that of the refusal completion. In that case. the weight allocation towards harmful samples might becomes higher and even might surpass that of the benign sample. Could you give me some insights on whether this case will happen?\n\n6. **Some related work on harmful fine-tuning defense should be discussed.**\n\nDetecting Adversarial Fine-tuning with Auditing Agents\n\nScaling Trends for Data Poisoning in LLMs\n\nUnleashing the Unseen: Harnessing Benign Datasets for Jailbreaking Large Language Models\n\nVirus: Harmful Fine-tuning Attack for Large Language Models Bypassing Guardrail Moderation\n\nNo, of course I can! Refusal Mechanisms Can Be Exploited Using Harmless Fine-Tuning Data\n\nBenign Samples Matter! Fine-tuning On Outlier Benign Samples Severely Breaks Safety\n\nYour Agent May Misevolve: Emergent Risks in Self-evolving LLM Agents\n\nEliciting Harmful Capabilities by Fine-Tuning on Safeguarded Outputs\n\nDeep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs\n\nSelf-Destructive Language Model\n\nCTRAP: Embedding Collapse Trap to Safeguard Large Language Models from Harmful Fine-Tuning\n\nVulnerability-Aware Alignment: Mitigating Uneven Forgetting in Harmful Fine-Tuning\n\nLoX: Low-Rank Extrapolation Robustifies LLM Safety Against Fine-tuning\n\nTowards Resilient Safety-driven Unlearning for Diffusion Models against Downstream Fine-tuning\n\nSEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selection\n\nSafety alignment should be made more than just a few tokens deep\n\nBeware of Your Po! Measuring and Mitigating AI Safety Risks in Role-Play Fine-Tuning of LLMs\n\nShape it Up! Restoring LLM Safety during Finetuning\n\nMitigating Fine-tuning Risks in LLMs via Safety-Aware Probing Optimization\n\nRefusal-Feature-guided Teacher for Safe Finetuning via Data Filtering and Alignment Distillation\n\nAsFT: Anchoring Safety During LLM Fine-Tuning Within Narrow Safety Basin\n\nYour Task May Vary: A Systematic Understanding of Alignment and Safety Degradation when Fine-tuning LLMs\n\nDefending MoE LLMs against Harmful Fine-Tuning via Safety Routing Alignment\n\nGradShield: Alignment Preserving Finetuning\n\nA Guardrail for Safety Preservation: When Safety-Sensitive Subspace Meets Harmful-Resistant Null-Space\n\nDetecting Instruction Fine-tuning Attack on Language Models with Influence Function\n\nLocking Down the Finetuned LLMs Safety\n\nPanacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbation\n\nSafe Delta: Consistently Preserving Safety when Fine-Tuning LLMs on Diverse Datasets\n\nNavigating the safety landscape: Measuring risks in finetuning large language models\n\nFundamental Safety-Capability Trade-offs in Fine-tuning Large Language Models\n\nESTIMATING WORST-CASE FRONTIER RISKS OF OPEN-WEIGHT LLMS\n\nWhen Style Breaks Safety: Defending Language Models Against Superficial Style Alignment \n\n**There may be more relevant works (I just list above some more recent work), and I suggest the authors to read and discuss all of the relevant works on harmful fine-tuning when revising the paper.**", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vGVYG6SwpG", "reviewer": "Reviewer_xQyK", "summary": "This paper proposes a two-stage defense framework to protect FTaS models from harmful fine-tuning attacks. The first step is regularizing the model on a flat-loss region for harmful samples to make safety alignment more resilient. The second step is the finetuning with weights adjusted for samples based on how harmful they potentially are to further reduce their impacts. Experiments show that Antibody can reduce the harmful score significantly while maintaining accuracy.", "questions": "- What do you mean by a test sample in Proposition 4.2? A sample used in finetuning or testing? We are not supposed to compute gradients for a “test” test sample, am I correct?\n- How does the size of refusal dataset impact results?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The problem is critical and well motivated.\n- The idea of optimizing the model to stay in the flat-loss region with respect to harmful samples is interesting.\n- Extensive theoretical analysis is provided.\n- Experimental results are promising and the ablation studies are extensive.", "weaknesses": "- In line 196, authors state that among the models that lie in the flat region of the harmful loss L_harm, they aim to find the one that minimizes the alignment loss L_align. But the optimization objective in Equation 3 is not exactly doing this. Opposite in order, it actually finds the weights that minimize harmful loss among the weights that minimizes alignment loss. I think we need more clarity here.\n- It is not clear how we should interpret Theorem 4.1 (which is the most important one in this paper) and what a_t is without looking at the appendix. More explanation is needed in the main body.\n- In the second stage, it is assumed that harmful samples are promised to be in the flat-loss region and benign samples won’t be in the flat-loss region, which may not hold for benign samples.\n- There is additional computation and pipeline overhead that may significantly increase the time and energy spent during the whole process. Analysis on the computational cost of the introduced first and second stage operations with detailed breakdowns are expected. \n- Fig.1 is not displayed correctly (cropped).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)