Assessing Safety Risks and Quantization-aware Safety Patching for Quantized Large Language Models
Authors: Kejia Chen
Jiawen Zhang
Jiacong Hu
Yu Wang
Jian Lou
Zunlei Feng
Mingli Song
Venue: ICML 2025 poster (2025)
Keywords: Large Language Model
Preference Alignment
Safety Evaluation
Primary area: social_aspects->safety
TL;DR: We conduct the  systematic safety evaluation for quantization on LLMs and propose a novel safety-patching algorithm for quantized LLMs.

Abstract: Quantized large language models (LLMs) have gained increasing attention and significance for enabling deployment in resource-constrained environments. However, emerging studies on a few calibration dataset-free quantization methods suggest that quantization may compromise the safety capabilities of LLMs, underscoring the urgent need for systematic safety evaluations and effective mitigation strategies. In this paper, we present comprehensive safety evaluations across various mainstream quantization techniques and diverse calibration datasets, utilizing widely accepted safety benchmarks. To address the identified safety vulnerabilities, we propose a quantization-aware safety patching framework, Q-resafe, to efficiently restore the safety capabilities of quantized LLMs while minimizing any adverse impact on utility.  Extensive experiment results demonstrate that Q-resafe successfully re-aligns the safety of quantized LLMs with their pre-quantization counterparts, even under challenging evaluation scenarios. Project page: https://github.com/Thecommonirin/Qresafe.

Assessing Safety Risks and Quantizaton-awaree Safety Patching for Quantized Large Language Models
Kejia Chen
12*
Jiawen Zhang"
12*
Jiacong Hu'
Yu Wang'
Jian Lou
Zunlei Feng 12f Mingli Song
Abstract
medicine (Ghosh et al., 2024), education (Cop, 2023), and finance (He et al., 2024). Underpinning such surging de- mand and remarkable capabilities is their colossal model size, which however poses significant challenges for de- ploying LLMs on commodity and edge devices due to the overwhelming resource overhead in terms of memory foot- print, computational cost, and energy consumption (Frantar et al., 2022; Xiao et al., 2023). Consequently, this has led to the growing popularity and significance of quantization in LLMs (Frantar et al., 2022; Xiao et al.. 2023), one of the most prominent L.M compression techniques, which converts LLMs' high-precision representations (e.g., 16-bit) into lower pwerr-precision ones (e-g- 8-bit, 4-bit, or even 1-bit). Various quantization methods, including post-training quan- tization methods (PTQ) (Frantar & Alistarh, 2023; Lin et al., 2023) and quantization- awarre training/fin-quning methods (QAT) (Liu et al., 2023b; Du et al., 2024), have been pro- posed to reduce bit-width while preserving the utility of LLMs, with each category having the option to be further assisted by calibration datasets (Qi et al., 2024a; Chen et al., 2024) achieve a better quantization-utiiiy trade-off.
Quantized large language models (LLMs) have gained increasing attention and significance for enabling deployment in resource-constrainee en- vironments. However, emerging studies on a few calibration dataset-free quantization meth- ods suggest that quantization may compromise the safety capabilities of LLMs, underscoring the urgent need for systematic safety evaluations and effective mitigation strategies. In this pa- per, we present comprehensive safety evalua- tions across various mainstream quantization tech- niques and diverse calibration datasets, utilizing widely accepted safety benchmarks. To address the identified safety vulnerabilities, propose quantization-awart safety patching framework, Q Q- resafe, to efficiently restore the safety ca- pabilities of quantized LLMs while minimizing any adverse impact on utility. Extensive experi- mental results demonstrate that Q-resafe suc- cessfully re-aligns the safety of quantized LLMS with their pre-quantization counterparts, even un- der challenging evaluation scenarios. Project page is available at: httpp::// http://ggttt/////// The Tommom i fieeaee
Since the evolving capabilities and growing integration of L.LMs into society can lead to negative societal impacts, high utility alone is insufficient for their reliable deploy- ment. Ensuring safety capabilities is equally crucial to pre- vent risks such as generating harmful content or violating ethical norms. Unfortunately, safety fragile maintain, as studies on high-precision LLMs reveal that even slight fine- tuning can cause well-aligned LLMSs experience degraded safety (Qi et al., 2024b) and become more susceptible to jailbreak attacks (Li et al., 2024c). In this vein, quantization is susceptible to compromising safety, as it alters model weights to a greater extent than slight fine-tuning. Emerging research has started to explore this issue (Hong et al., 2024), with a primary focus on the techniques without calibration datasets. (Hong et al., 2024) examines how two PTQ meth- ods can degrade safety, (Belkhiter et al., 2024) evaluates one PTQ and one QAT method against various jailbreak attacks, (Zhang et al., 2025) evaluates the safety of aligned LLMs with activation quantization, and (Egashira et al., 2024) de- vises PGD--ased attack on one PTQ and one QAT method to deliberately exploit quantization to induce specific mali- cious behaviors. Propelled by these findings (Shang et al.,
1. Introduction
Large language models (LLMs) (Touvron et al., 2023;
Anil et al., 2023; Achiam et al., 2023) are increasingly applied across diverse domains, offering astounding per- formance that often surpasses human capabilities in tasks ranging from general language processing (Reizinger al., 2024; Almeida et al., 2024) to specialized areas like
"Equal contribution 'Corresponding authors. 'The State Key Laboratory of Blockchain and Data Security, Zhejiang University 'Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security "Sun Yat-sen University. Corre- spondence to: Zunlei Feng <zunleifeng pezzju.cdu..... Jian Lou <louj5@ @ mail mai...ssuu....c..
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Assessing Safety Risks and Quantization-awar Safety Patching for Quantized Large Language Models
2023; Lin et al., 2024a; Liu et al., 2024b;a), it is tempting to investigate the critical yet unexplored questions that could deepen our understanding of the safety cost of quantiza- tion: To what extent do different quantization techniques and calibration datasets degrade the safety capabilities of LLMs? How can these safety declines be mitigated while maintaining model utility?
(1) A comprehensive safety evaluation of quantized LLMs, covering mainstream quantization techniques and three dif- ferent types of calibration datasets;
(2) The proposal of Q-zesafe, an efficient algorithm to mitigate safety degradation in quantized L.MM; (3) Extensive experiments demonstrating Q resafe's abil- ity to restore safety while keeping utility in quantized LLMs.
This Work. In this paper, we conduct a systematic safety risk assessment for QLLMs, covering mainstream categories and taking calibration datasets into account. Furthermore, we mitigate the safety degradation by proposing a novel Quantization-awee aware safety patching algorithm (O-resafe) to re-align the safety performance of quantized LLMs with their pre-quantization counterparts.
2. Related Works
Quantization for Efficient LLMss Quantization reduces storage and computation costs by converting representations into lower- pweer-preisson formats, enabling effi- cient LLM deployment. Existing methods can be roughly divided into PTQ (Yao et al., 2022; Wei et al., 2022; Cheng et al., 2023; Dettmers al., 2023; Lee et al., 2023; Kim et al., 2023; Li et al., 2024b; Wei et al., 2023; Yuan et al., 2023; Lin et al., 2023; Liu et al., 2023a; Ashkboos et al., 2024; Kim et al., 2024b; Shao et al., 2023; Zhao et al., 2024) and QAT. PTQ applies quantization after training with minimal computational cost, whereas QAT incorporates quantization during training, allowing the model to adapt to lower pre- cision for better performance. To further reduce resource demands (Liu al., 2023b; Du et al., 2024; Ma et al., 2024; Xu et al., 2024a), Parameter-Efffcienn Fine- Fine-Tuning (PEFT) techniques focus on tuning only subset of parameters to balance efficiency and accuracy (Li et al., 2023c; Guo et al., 2023; Xu et al., 2023; Chai et al., 2023; Hayou et al., 2024; Kim et al., 2024a; Dettmers et al., 2024). Although these ap- proaches primarily enhance utility, they also raise concerns about potential safety degradation in quantized LLMss
Safety risks assessment: 1) Quantization methods: To en- sure the evaluated methods are sufficiently representative within each category, the selection criteria are based on whether the method a seminal work with high cita- tions (Lin et al., 2023; Liu et al., 2023b; Dettmers et al., 2024) or achieves state-of-the-art performance (Egiazar- ian et al., 2024), as detailed in Section 3.1. 2) Calibra- tion datasets: To match the evaluations designed for pre- precision LLMs (Qi et al., 2024b), we also consider three types of datasets with varying safety risk levels for quan- tization methods involving calibration datasets: a directly harmful dataset, an indirectly harmful dataset, and be- nign dataset. 3) Bit-widths: we evaluate quantized LLMs with two commonly adopted bit-widths (INT8 and INT4), and further ablation studies with 2-bit and 3-bit. 4) Safety measurement: To ensure comprehensiveness, We follow the well-established safety risk measurement practices for full-precision LLMs (Qi et al., 2024b). 5) Findings: all quantization techniques lead to degraded safety capabilities, with post-quantization methods experiencing more severe declines due to their limited ability to preserve overall model capacities, including safety. Although benign calibration datasets still incur safety declines because their objective centers on preserving utility, often neglecting safety-specific considerations. Additionally, QLLMS can suffer a dramatic safety drop if these datasets contain harmful samples, sug- gesting that these datasets should be carefully scrutinized. Finally, lower bit-widths result in greater safety degradation compared to higher bit-widths.
Safety Evaluations for Quantized LLMs. Effective safety evaluation are essential to ensure LLM outputs align with hu- man values and ethical guidelines(Cha, 2023).. While safety assessments for call-precision LLMs are well-established, covering dimensions such as attack success rate, refusal mechanisms, and safety risk index. These metries form the backbone of LLM safety research (Deng et al., 2024; Zeng et al., 2024; Xie et al., 2025; Souly et al., 2024; Li et al., 2024a; Chu et al., 2024), providing systematic approach to measuring resilience against harmful content.
Recently, several studies have pioneered the exploration of safety issues in quantized L.LMs, primarily focusing on calibration dataset-free quantization methods. For instance, Kumar al. (2024b); Hong et al. (2024) analyze GPTQ and AWQ techniques across multiple LLMs, examining their impact on model safety and utility. Egashira et al. (2024) devises a projected gradient descent (PGD)-based attack on AWQ and GPTQ to deliberately exploit quantization and manifest specific malicious behaviors. In addition, Pan et al. (2021) revealed security risks in third-party quantized neural networks, where backdoor attacks can remain dormant in full-precision models but activate through quantization.
Safety risks patching: We propose Q-resafe to re- store the safety capabilities of quantized LLMs efficiently while preserving the model's utility. To achieve these, Q-resafe 1) transfers the strong safety capabilities of the pre-quantization LLM by constructing safety-patching dataset under its guidance; 2) twists only the most essential portion of weights necessary to restore the safcty capabili- ties by selectively fixing only the safety-critical weights.
Contributions. Our contributions are summarized below:
2

---

Assessing Safety Risks and Quantization-awan Safety Patching for Quantized Large Language Models
Restoring Safety for Quantized LLMs. While established alignment techniques like instruction tuning, reinforcement learning from human feedback (RLHF), Direct Preference Optimization (DPO) (Christiano et al., 2017; Ouyang et al., 2022; Bai et al., 2022; Peng et al., 2023; Rafailov et al., 2024) are effective for full poll-precision LLMs. Research on safety alignment approaches for quantized LLMs remains limited (Badshah & Sajjad, 2024; Xu et al., 2024b; Paglieri et al., 2024). Recent studies (Badshah & Sajjad, 2024; Xu et al., 2024b; Paglieri et al., 2024; Hu et al., 2024) empha- size the need for methods that restore safety without sacri- ficing efficiency. Addressing this challenge is non-trivial, as quantization alters weight and activation representations, requiring remedial measures that account for interactions between lower-precision parameters and alignment mech- anisms. Our work aims to tackle this issue by proposing targeted solutions to maintain safety in quantized LLMs while preserving their computational and memory benefits.
Table 1. Overview of quantization methods: quantization type and requirement for quantization assisting datasets.
Method
Quantization Type Assisting Dataset
AWQ
PTQ w/o FT PTQ W. FT
x
AQLM
/
LLM-QAT
QAT W. FT
QLORA
QAT W. LoRA FT
Table 2. Performance of haseline models.
Model
ASR
MTT-ench
AlpacaEval
Mamna-2-7B-harr
0.3
6.65
71.37
Gemma-7B Be-nma--BlInstruct
9.2
6,25
66.53
often rely on calibration datasets (referred to as quantization- assisting datasets hereafter) to guide the weight quantization process. These datasets are pivotal in shaping the perfor- mance and safety of quantized LLMs. While they enhance utility through fine-tuning, their content may inadvertently introduce safety risks, necessitating a rigorous evaluation of their design and reliability.
3. Assessing Safety Risks of QLLMs
3.1. Setup of Assessment
We follow established practices in the literature (Qi et al., 2023) to construct quamtization-assitiin datasets using Ad- vBench (Chen et al., 2022) and Ultrachat (Ding et al., 2023). From AdvBench, we randomly select 10 examples to create two datasets: a Direct Harmful dataset (Risk-III), which contains harmful instructions and their corresponding harm- ful responses, and an Indirectly Harmful dataset (Risk- II,, consisting of non-toxic instructions but paired with responses designed to induce model compliance or unsafe behavior subtly. Additionally, we used a Benign dataset (Risk-I), randomly sourced 10 samples from the Ultrachat dataset, which contains purely utility-oriented instruction- response pairs that are not harmful in nature. nature.Details about assisting dataset are provided in Appendix B.
Models. We select two widely-used open-source LLMs, Llama-2-7B-Chat and Gemma-7B-Instruee as our pre- quantization baselines. These models are chosen for three key reasons. First, they are open-souree, allowing easy ap- plication of various quantization methods for subsequent assessment. Second, both models have undergone extensive post-training processes, including instruction tuning and reinforcement learning from human feedhack, making them robust in safety-critical tasks. Finally, they exhibit distinct strengths across different task types, offering a valuable comparison of quantization effects models with varied pre-quantization performance.
Specifically, Llama-2-7B-Chat excels in safety-aligned open-ended conversations, while Cemma--BBInstruee per- forms better in structured tasks like reasoning and cod- ing, where precise cestuuttio-foollwing is crucial (Touvron et al., 2023; Team et al., 2024; Almeida al., 2024). Safety and utility scores for both baselines are provided in Table 2.
Safety & Utility Metrics. Our safety metrics for quantized LLMs are consistent with the existing practices utilized for full-precision I.LM evaluations. Specifically, we measure the quantized LLMs' safety by assessing their Attack Suc- cess Rate (ASR) in response to harmful instructions (Zou et al., 2023). And we evaluate the model's utility following the popular MT-bench (Zheng et al., 2024) and AlpacaEval (Li al., 2023b). The details of the utility measurement can be found in Appendix B.
Quantization Methods. We evaluate two main categories of quantization methods: PTQ and QAT. For PTQ, we select representative methods, including AWQ and AQLM, while for QAT, we choose LLM-QAT and QL. These methods are either foundational or state-of-the-art as demonstrated by their growing citation counts, AWQ, AQLM, LLM-QAT and QLoRA respectively(L.n et al., 2024b; Egiazarian et al., 2024; Liu et al., 2023b; Dettmers et al., 2024), making them highly representative of their respective categories. Additionally, we test two common bit-wi,ths, INT4 and INT8, which are widely supported by these methods.
3.2. Intra- Analysis towards Quantization
Dost-quantization without fine-tuning: AWQ. Since AWQ does not rely on quantization-assisting datasets, we adopt the approach outlined in (Huang et al., 2023) to evalu- ate its safety risks. We adjust the model's sampling strategy after AWQ quantization by modifying parameters such as temperature т, top-k, and top-p. The results show that the INT4 and INT8 quantized models exhibit higher safety risks
Quantization-Assising Datasets. Quantization methods
3

---

Assessing Safety Risks and Quantization-awan Safety Patching for Quantized Large Language Models
Table Safety assessment results for four quantization methods various quantization-assiiiie datasets: Risk-I (UltraChat), Risk-Il and Risk-Idd(rrafted from AdyBench). Since AWQ does not have quantization aaaniization-assitinng dataset, we evaluate its ASR under decoding attack (Huang et al., 2023). For the other three methods, we directly measure under Advbench. The baseline are shown table 2.
Safety (ASR))
Utility(1)
Bit
Model
Method
Risk-1
Risk-I1
Risk-II1
MT-Bench
AlpacaEval
AWQ
42.4
42.4
42,4
6.51
68.37
AQLM
18.5
75.5
77.4
6.40
66.42
Llama-2-7B-Chaa
LLM-QAT
16.9
82.9
71.2
6.71
66.54
QLoRA
42.3
83.4
85.3
6.40
63.92
INT4
AWQ
17.9
17.9
17.9
6.14
65,40
AQLM
25.3
69.9
55,4
6.12
61.75
Gemma-77 Boama-7B-aae 7B-Inntruce
LLM-QAT
20.7
68.4
52.9
6,28
62.85
QLoRA
39.4
68.6
61.3
6.15
59.13
AWQ
39.1
39.1
39.1
6.58
69.42
AQLM
17.1
73.3
75.3
6.56
69.20
2lama-2-7B-Chrn
LLM-QAT
15.1
76.1
65,4
6.75
67.26
QLoRA
41.7
76.7
83.2
6.55
69.50
INT8
AWQ
17.7
17.7
17.7
6.18
65,93
AQLM
23.7
60.4
53.8
6.23
63.40
Gemma-7BB BBIInstuutt
LLM-QAT
18.4
63.5
50.1
6.39
64.94
QLoRA
37.1
64.0
58.9
6.27
62.50
compared to the FPI6 baseline. As shown in Table 3, AWQ quantization leads to noticeable increase in safety risks, reflected by the higher ASR. For the base models, the ASR values are 0.3% for Llama and 9.2% for Gemma before quantization (see Table 2). After quantization, with higher temperature setting (T 0.95), the ASR for Llama-2-78 Chat increases from 29.8% to 42.4% (INT4) and 39.1% (INT8), while for Gemma-77 -aaima--B--ua Instructt,, it rises from 9.4% to 17.9% (INT4) and 15.1% % (INT8). Despite this, the Gemma models show lower ASR than Llama models, which can be attributed their stronger pre-quantization safety. In contrast, the utility degradation after AWQ quantization is relatively mild, with reductions ranging from 0.1 to 3.0 points, indicating that utility is largely preserved.
the INT4 models. Safety degradation is more pronounced on higher-risk datasets, with ASR rising to 82.1% and 68.4% for the indirect harmful datasets, and 83.7% and 67.5% for the direct harmful datasets. INT8 models exhibit slightly lower ASR than INT4, owing to the increased bit-width, which enhances model expressiveness and capability preser- vation. Despite these safety challenges, utility degradation after LLM-QAT is minimal, with a decrease of less than 2% compared to the full-precision model, thanks to the utility-fouused quantization strategy.
Quantization-aware and parameter-efffiient fine-tuning: QLoRA. Despite LoRA's efficiency, QLoRA shows the most significant safety degradation among all methods, though it excels in utility preservation. Even on benign datasets, QLoRA produces higher ASR than AWQ, with 42.25% for Llama-2-7B-Chr and 39.4% for Gemma-7B- Instruct. For both indirect and direct harmful datasets, QLoRA leads ASR values high 85.3% for Llama-2- 7B-Chat and 68.6% for suggest that QLoRA sacrifices safety achieve higher util- ity and quantization efficiency.
Post-quantization with fine-tuning: AQLM. The AQLM quantization results highlight the significant impact of quantization-assisting datasets on the safety the quan- tized LLM. For the Llama-2-77 2-1-----BBB-- 7-ttthhtt model, ASR increases from 18.5% benign datasets to 73.5% indirect harmful datasets, and 77.4% directly harmful datasets. Similarly, for the Gemma-7B-Instre
These results
model, ASR rises from 23.5% on benign datasets to 69.9% on indirect harmful datasets, and 67.3% on directly harmful datasets.
3.3. Cross-Method Analysis towards Quantization
Quantization-awan and full-parameter fine-tuning: LLM-QAT. LLM-QAT results demonstrate that, similar PTQ. QAT-based quantization suffers from safety degra- dation. Even with benign datasets, ASR increases to 16.9% for -lama-2-7B- 2-7--Chat and 20.7% for Gemma-7B-Insttan in
Comparing two PTQ methods. The presence of fine- tuning significantly affects PTQ safety. AWQ, which skips fine-tuning, shows substantial safety degradation, especially for INT4 models (ASR: 42.4%). contrast, AQLM reduces ASR to 18.5% with benign datasets. However, fine-tuning

---

Assessing Safety Risks and imantization-awark Safety Patching for Quantized Large Language Models
alone cannot fully restore safety, and using harmful datasets can increase ASR to 75.5%, emphasizing the critical role of dataset selection.
weights intact to avoid adversely impacting the utility. The safety patching method is expected to twist only the most essential portion of quantized weights necessary to restore the safety capabilities. Motivated by this intuition, we pro- pose Q-resa fe to re-align the safety capabilities of the quantized LLM with its pre-quantizatinn counterpart by selectively fixing only the safety-critical weights.
Comparing two QAT methods. The fine-tuning strategy in QAT methods plays a crucial role in balancing safety and utility. Full-parameter fine-tuning (LLM QAT) better preserves safety compared to parameter-efftcien fine-tuning (QLORA). By adapting a larger set of parameters, LLM-QAT retains more of the pre-quantization model's capabilities, leading to stronger safety performance. In contrast, QLoRA prioritizes utility with fewer parameters, resulting in a safety trade-off. However, both methods experience safety degra- dation due to the utilitty-rivee nature of QAT objectives.
Q-resafe achieves this through three key steps: construct- ing a safety-patching dataset guided by pre-quantization LLMs to transfer safety capabilities, leveraging DPO to align the quantized model's safety with its pre-quantization version, and selectively updating safety-critical weights to restore safety without compromising utility. This efficient and targeted approach ensures robust safety restoration with minimal computational overhead. The following section detail the notations, the safety- patching objective, optimiza- tion scheme and complete algorithm.
Comparing PTQ and QAT. QAT methods generally pre- serve more safety compared PTQ methods, provided the fine-tuning datasets are benign. This is because QAT adjusts model parameters during quantization, compensating for the information loss from low-bit-width quantization. However, both methods show higher safety risks at lower bit-widths (INT4 vs. INT8), highlighting the challenges of quantizing at lower widths.
Notations. We adopt the matricization notations utilized in LoRA, where the ure-quanizzation LLM weights (de- noted by Tw) are formed as a matrix W E Xdout, We denote the quantized weights by Qo E Quhn and the corresponding quantized LLM by TQU, the low-rank adaptation matrices of LoRA with rank < {din. dout by A E Rdin Xr, xr, B E Rexdoue, and the safety-patched weights by Q E QyAnnxoous, where the conventional LoRA has Q = Q" + AB. Additionally, we use @ to denote the clement-wise product and a to denote the Sigmoid function.
Impact of Quantization-AAiisiinn Datasets. Safety risks increase significantly when transitioning from benign to harmful datasets, with INT4 models being the most vul- nerable. While QAT methods generally perform better, no method completely mitigates safety risks. Notably, indi- rect harmful datasets, often based on role-playing scenarios, have a greater impact on safety, as they expose models to unsafe behaviors while enhancing utility.
4.2. Deriving Q-resafe
We begin with the conceptual objective function based on the DPO loss, with LoRA and safety-critical weights mask- ing structures imposed as the constraint. We then con- cretize step-by-step by describing the specific forms of the safety-patching dataset construction, periodic safety- critical weights identification, and finally presenting the per-iteration updating scheme and the complete algorithm.
3.4. Summary of Assessment
quantization methods inher- ently compromise safety, even as they maintain reasonable utility. INT4 models, in particular, exhibit greater vulnera- bility to safety risks compared to INT8 models, highlighting the need for stringent safety monitoring at lower bit-widths. Finally, the selection of quantivation-aasistngg datasets plays a pivotal role not only in optimizing utility but also in safe- guarding model performance, especially when harmful sam- ples are present in the datasets.
In summary,
Conceptual objective function. Given the quantized LLM TQ" and the safety-patching dataset Dymtch with each pref- erence sample being triplet (I, yo: 31 ~ Dpatch. the DPO-based objective for safety patching is as follows,
C. = EDpaced log a Blog TQ
qqu (ywla) 9 gog =Q u:)
4. Safety-Patching for Quantized LLMs
TQa (J](x))
4.1. Overview
S.L., Q Q" + Quant(Ma AB),
According to the evaluation results in Section 3.1, quantized LLMs generally have satisfactory utility, often matching the performance of their pre-quantization counterparts. This can be largely attributed to the significant efforts of existing quantization techniques that carefully generate the quan- tized weights to preserve the utility of the full-precision LLM. As such, it is desired to leave most of the quantized
where Mq is the masking matrix with entries of 1 cor- responding to safety-critical weights to be updated and entries of O corresponding to other weights that remain intact, Quant compresses the weights into the same low- precision data format as those in the quantized LLM Q", and 3 is a parammer The constraint in Eq. (2) re- stricts the safety patching to simultaneously adhere to the
5

---

Assessing Safety Risks and Quantization-aware Safety Patching for Quantized Large Language Models
Algorithm 1 Safety Patch for Quantized LLM
of different types of calibration datasets, considering three levels of risks, and find that the source of the dataset is not very restrictive. Even when reference responses are available, our method remains advantageous, as the pairs generated by W and Qo can present greater challenges than reference responses-leading to more rigorous safety patch- ing and improved alignment. Finally, we remark that if the pre-quanizattinn LLM is unavailable for the safety patch- ing, other well-aligned LLMs can serve alternatives, e.g.. leveraging proprietary LLMs like GPT, Claude, Mistral.
Input:
Quantized LLM : Pre-quantization LLM mw: Calibration dataset cealib; Initial LoRA matrix A, B:
Re-evaluation interval K; Safety-critical threshold r; Total iterations T. Learning rate 1).
1: for each prompt sequence z E Dealih do
2: Veu ~ (-Ix)
3: V1 ~ (-(r))
4: Ppatch f- (x, Y/u 31))
5: end for
6: fort = 0, 1, ...... T 1 do
Periodic safety-critical weights identification. We first discuss the feasibility of identifying and updating a small portion of safety- weights, then exploit potential tools for identifying these weights, and construct a pair of mask- ing matrices corresponding to the LoRA variables A, B based on the identified weights. Research suggests that an LLM's capabilities are concentrated in small fraction of its weights (Qi et al., 2023; Yang et al., 2023; Kumar et al., 2024a). This insight enables safety-patching to target only a small portion of safety-critical weights while leaving the majority of other weights untouched, thereby preserving the utility of the quantized LLM.
7: i % K == 0 then
8:
MQ- 1 SaaffSccreeeu E Top-r)
9:
MB) = (apMasx(Ma)
10:
end if
11:
A 4+1 MA + (A' VVC + (1-MMA MA) @ A
12:
B MB ④ (B' vfVB + (1- MB) o B'
Q -Q" + BLt
13:
14:
end for
Output: Satety- Quantized LLM with weights Q².
LoRA structure, represented by the low-rank pairs (A,B) B). while modifying only the safety-critical weights indicated by the masking matrix Mq. Moreover, the DPO loss of Eq.(I) is known inherently regularize QQ to discourage significant deviation from the reference LLM qq..
We identify the safety-critical weights with SNIP score (Lee et al., 2019). for a prompt x and response y, we take the loss as the conditional negative log-likelihood L(x) - log p(y(==) predicted by the model. For any layer of model Q with weight matrix W, the importance score for each weight entry Waj as
As result, this safety capabilities by editing only the most essential weights while still preserving the utility of the quantized LLM TQo. Next, we concretize the above conceptual objective by de- tailing the construction of the safety-patching dataset D patch and the specific form of the masking matrix MQ.
objective will re-align the
I(W x) = IW15- Val ((z)]
(3)
Given the calibration dataset D Deatsh. we take the average value and obtain SafeScore(()) We regard weights with salient scores in the top-7 percentile as the most saiita-critical. Since the subset of safety-critiaal weights in Q' gradually changes across iterations t through- out the to periodically re-identify the subset based on the most up- dated Q'. The masking matrix Mq has I's for the identified weights. Alternatively, we introduce a pair of masking ma- trices (MA, BB) corresponding to M인-
ExED I (Q1j. 2²).
safety-patching dataset construction. To restore the safety capabilities of quantized LLMs, we construct the safety patching dataset Patch to leverage guidance from the pre- quantization LLM. Specifically, for a prompt x from an auxiliary calibration dataset, potentially lacking reference responses and preference annotations, we feed it into both the pre-quantization LLM and the quantized LLM gener- ate their respective responses. Then, we label the response from the pre-quantization LLM as the winner (preferred) response Yu and the response from the quantized L.LM as the loser (dispreferred) response yt. forming the pref- erence triplet yo: Vo: ). From a knowledge distillation perspective (Tunstall al., 2023), this construction can be regarded as enabling the strong safety capabilities of the pre-quantization LLM to gradually transfer to the quantized LLM through iterations of the safety patching algorithm. Eliminating the need for manual preference annotations and significantly reducing costs and complexity.
algorithm. Therefore, we propose
Updating form and complete algorithm. Equipped with the calibration dataset Dentis and masking matrices (MA, Mg), the objective in Eq.(1) is ready to be opti- mized by stochastic gradient descent. Taking A at itera- tion t for instance, we take the SGD step with learning rate 17 as A' ALA',, B+) and restrict the update to safety-critical weights according to the mask matrix MA by MA O (A A(A', B')), while maintaining other weights intact by (1 - M,) @ A Overall, it provides the updated A ++1 by A++1 = MA @ (A' (A'-n)'s V7V C(A', B²)) + (1 -1-M MA) O A'. The complete algorithm, detailing dataset construction, periodic safety-critica weight identification, and iterative updates, is provided in Algorithm 1.
Furthermore, in Section 3, we empirically study the impact
6

---

Assessing Safety Risks and Quantization-wware Safety Patching for Quantized Large Language Models
Bazelino
AQLM
LLM-QAT
QLORA
Q-resafe (Durs)
600
R27
Lima----78 Llama-2-B--hae
(16)
417
F2g
120
12g
16-art
4-bit
8-a1
16-bt
4-bh
8-brt
16 bt
4.blt
8:b1t
Gemma- iB-irucuct
10gg
68
Rkg
10gg
400
75.8
10g
0
70a
08
8
31
100
201
14
12
12
10g
16-68
4-b
16:bi
4-b0
8-b2
16:bt
4-on
8.b2
Risk-Il
Figure 1. Safety evaluation of Q-rosafe and fine-tuned baseline quantization methods for Llama-2-7B-Che and Gemma-7B 7B-Instruct.
5. Experiments
practical scenarios with harmful samples. The utility of the quantized model is almost unaffected as well.
Experimental Setups. We compare O-resafe with the representative quantization methods evaluated Section 3: AWQ, AQLM, LLM-QAT, and QLoRA. These methods are applied to two open-source, well-aligned LLMs: Llama-2- 7B-Chat and Gemma-7B-Instru reduced bit-wi.ths. Safety and utility are measured using the same metrics and settings as described in Section 3. With- out specific annotation, the T is set to 0.6, the LoRA rank r = 2048, re-evaluation interval K to 1000, Qº can be the quantized model from AWQ/AQLM/LLM -AALLLLLAAA//OAAAAAAAA The decoding strategy of the LLM follows the default set- tings of the model. Further experimental details are provided in the appendix A. and B.
Safety patch on harmful dataset (Risk-11I). Figure (Risk-III) presents the results of safety-patching by Q-resafe on the direct harm dataset, comparison with baseline quantization methods that involve fine-tuning. Compared with the pre-quantization model, baseline quan- tization methods result in up to a 92.3% increase in ASR for Llama and up to 66. 66.7% increase for Gemma, while Q-resafe only increases by 13.6% and 1.8%, respec- tively. The utility of the quantized model is almost unaf- fected, which is comparable to the pre-quantization LLMs. In Figure, the harmful dataset consists of 100 harmful ex- amples.
using INT4 and INT8 as
Safety patch without finetuning (for AWQ). Table 4 presents the results of quantization without fine-tuning. We use the standard system prompts and evaluate ASR under decoding attack (Huang et al., 2023). For a fair comparison, we did not perform DPO in Q resafe but only searched for safety-critical weights on the full-precious pre-trained model, keeping these weights as 16 bits and quantizing the others to 4 bits. The results of AWQ in up to 7.3% in- crease in ASR for Llama and up to an 5.8% increase for Gemma, while Q-resafe only increases by 0.8% and 0.4%, respectively. The utility of the quantized model is largely unaffected.
5.1. Results and Analysis
Safety patch benign datasets (Risk-I). Figure 1 (Risk-I) presents the results of aafety-patching by Q-resafe on the benign dataset (UltraChat), in comparison with baseline quantization methods that support fine-tuning. Compared to the pre-quantization model, baseline quantization meth- ods lead a 16.6% increase in ASR for the Llama model and up an 1 11.5% increase for the Gemma model. While Q-resa fe only increases ASR by 1.5% and 0.9%, effec- tively restore the safety performance of the given quantized LLMs. Additionally, O-resafe achieves these results with just one epoch on the henign dataset, highlighting both its efficiency and effectiveness.
5.2. Ablations and Discussions
Ablation study on safety-critical weight identification. To demonstrate the effectiveness of identifying safety-critical weights, we vary the percentage of weights updated during safety patching (T). Here, 7 indicates that all weights are updated (no identification step), while 0.2 repre- sents updating only the top 20% of safety-critical weights. Table 5 summarizes the results. As shown, when 7 = 1, the model achieves the highest safety performance with an ASR of 1.6%. However, as 7 decreases, the ASR grad-
Safety patch on indirect harmful dataset (Risk-II). Figure 1 (Risk-II) presents Q D-reesafe' 's safety-patching results on the indirect harm dataset that contains 10 identity-shifiin examples, compared with baseline quantization methods that involve fine-tuning. Bascline quantization methods result in an 82.6% increase in ASR for Llama and up to a 59.2% increase for Gemma. While Q-resafe only increases by 13.3% and 5.5%, effectively restoring safety in

---

Assessing Safety Risks and Quantization-awan Safety Patching for Quantized Large Language Models
Table Safety and utility comparison of fine-tuning-free quantization (AWQ) under varied Decoding strategies default settings.
Llama
Bit-
Safety())
Utility (1)
Method
width
(0.95)
r00.7)
k(500)
k(200)
p(0.95)
p(0.7)
MT-Bench
AlpacaEval
Baseline
FP16
29.8
25.8
26.1
18.2
22.5
25.1
6.65
71.37
AWQ
4-bit
37.1
30.3
38.2
35.0
35.5
42.4
6.51
69.42
Q-resafe
4-bit
30.6
25.7
26.4
18.4
23.8
25.0
6.52
69.56
AWQ
8-bit
35.5
29.2
35,9
34.1
33.7
39.1
6.58
68,37
Q-resafe
8-bit
26.8
21.4
23.5
17.1
22.1
23.9
6.61
70.02
Gemma
Bit-
(afety())
Utility (t)
Method width
(0.95)
+(0.7)
k500)
k<200)
p(0.95)
p(0.7)
MT-Bench
AlpacaEval
Baseline FP16
9.4
9.3
9.6
9.6
10.1
10.4
6.25
66.53
AWQ
4-bit
15.2
15.0
15.5
15.4
16.6
17.9
6.14
65.40
Q-resafe
4-bit
9.8
9.6
10.3
10.3
10.9
11.1
6.19
66.44
AWQ
8-bit
15.1
14.9
15.5
15.2
16.1
17.7
6.18
65.93
Q-resafe
8-bit
9.7
9.3
9.8
9.8
10.4
10.5
6.22
66.49
Table Impact of safety-critical weight identification safety, training time (represents with GPU hours), and utility. The results based Jlama-2-7b-chat with 4-bit quantization using be- nign dataset (Ultrachat 200k) for one epoch.
Table 6. Comparative results of different safety-patching methods with 4-bit. The safety threshold of Q-resafe T is set to 0.6.
Methods
ASR (%)
GPU (h)
ASR (%)
GPU (h)
MT-Bench
LLM-QAT + SFT
12.4
8.4
LLM-QAT + DPO
1.5
9.6
1.0
1.6
2.1
7.3
LLM-QAT + Q-resafe
1.6
1.2
0.8
1.6
1.8
7.2
QLoRA + SFT
26.9
3.4
0.6
1.8
1.2
QLoRA + DPO
2.4
3.8
0.4
5.5
0.8
6.8
QLoRA + Q-resafe
2.4
1.2
0.2
13.9
0.5
6.6
0.0
42.2
6.4
Table 7. ASR (%) across multiple bit-widths (8-bit, 4-bit, 3-bit, 2-bit) for different quantization methods using Llama-2-7b-chal
ually increases, reflecting trade-off between safety and efficiency. Notably, when 7 = 0 (i.e., no safety-critical iden- tification is performed), the ASR rises sharply from 1.6% to 42.2%, demonstrating the critical role of safety-critical weight identification in preserving model safety.
with Ultrachat as our quantization-aasistnng dataset. Lower ASR
indicates better safcty.
Method
8-bit
4-bit
3-bit
2-bit
AQLM
17.1
18.5
28.6
40.1
LLM-QAT
15.1
16.9
25.4
36.9
Ablation study the benefit of the safety-patching dataset and different safety-patching methods. We assess three fine-tuning methods-So SFT, DPO, and Q-resafe on quantized Llama-2-7b-chat models using the Alpaca dataset for safety alignment. Experiments were conducted for two epochs on models quantized with QLoRA and LLM-QAT, and the results are summarized in Tab 6.
QLoRA
41.7
42.3
67.3
82.0
AWQ
10.5
17.4
29.5
38.6
Q-resafe
1.6
1.8
5.9
12.4
example, Q-resafe requires only 1.2 hours compared to SFT's 3.4 hours with inferior ASR (26.9%). This balance of safety and efficiency makes Q-resafe especially suitable for resource-constrainel applications.
Across all fine-tume fine-tuning methods, safety- patching dataset consistently reduces the safety risks of quantized models, demonstrating its effectivvnness in aligning models with safety requirements. Furthermore, Q-resafe outperforms SFT and DPO by achieving similar or better safety (c.g., 2.4% ASR for QloRA) while being far more efficient. For
Ablation study on the impact of quantization bit-widths on safety. As shown in Table 7, reducing the quantization bit-width consistently leads to an increase in ASR across all methods, highlighting trade-off between precision and
8

---

Assessing Safety Risks and Quantization-awan Safety Patching for Quantized Large Language Models
safety. The steepest ASR increase occurs between INT4 and 3-bit, followed by more gradual growth from 3-bit to 2-bit, suggesting partial saturation of safety degradation at extremely low bit-wi. Among the methods evaluated, Q-resafe fe exhibits the best performance, maintaining the lowest ASR at all bit-widths. We also compare mixed- precision quantization approaches and conduct an ablation study. The detailed results are provided in Appendix C.I.
Technological Innovation of Ten-Thousands Talents Pro- gram" (No. 2022R52046), and Information Technology Center, ZheJiang University.
References
Introducting ChatGPT. http:////////ppaacccco////ba/baa chatgpt, 2023.
Github Copilot Your Al pair programmer. https:/ / github wtthub.com/occ.t ftttub..om/faatureeeee httmm//faatreeecoollleeee 2023.
6. Conclusion and Future Work
This paper presents a comprehensive safety evaluation of quantized LLMs, examining four categories of quantizations and three types of calibration datasets. We have introduced Q- O-resaff fe to efficiently restore the safety capabilities for quantized LLMs. We have highlighted the importance of considering safety risks when quantizing LLMs and empha- sized the need for effective safety patching techniques like Q-resafe to ensure the reliable deployment of quantized LLMs in real-world applications. For future work, it is a promising alternative approach to developing safety-in -ieey-i--miidd QAT, which enhances safety during quantization rather than relying on post-hoc safety patching like Q resafe.
OpenAl API.
openai-api, 2023.
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I.,
Aleman, F. L.,, Almeida, D., Altenschmidt, J., Altman, S.,
Anadkat, S., et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08772 2023.
Almeida, G. F., Nunes, J. L., Engelmann, N., Wiegmann, A., and de Araújo, M. Exploring the psychology of Ilms' moral and legal reasoning. Artificial Intelligence, 333: 104145, 2024.
Anil, R., Dai, A. M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S.. Taropa, E., Bailey, P., Chen, Z., et al. Palm 2 technical report. arXiv preprint arXiv:2305. 10403, 2023.
Reproducibility To facilitate the reproducibility our experiments, we release all models evaluated in the bench- mark along with the modified Q-Resafe benchmark, which helps mitigate the large score variances caused by high attack success rates. All related resources are avail- able on our project page: https: /ttps://github... c Thecommoni wtttmmoiirrnn/reaaattee
Ashkboos, S., Mohtashami, A., Croci, M. L., Li, B., Jaggi, M., Alistarh, D., Hoefler, T., and Hensman, J. Quarot: Outler-freee 4-bit inference in rotated Ilms, arXiv preprint arXiv:2404 24%0:2400400445, 2024.
Impact Statement
Badshah, S. and Sajjad, Quantifying the capabili- ties of Ilms across scale and precision. arXiv preprint arXiv:2405.031/ 2024.
This study investigates the safety challenges of quantized large language models (LLMs) and proposes effective miti- gation strategies to restore their robustness. While quanti- zation enables LLM deployment in resource-constrainee environments, also introduces vulnerabilities that can compromise safety. Our findings highlight the need for systematic safety assessments across different quantization techniques and bit-widths. To address these issues, we in- troduce C-resafe, a quantization-awee aware safety patching framework that efficiently restores the safety capabilities of quantized LLMs with minimal impact on utility. Extensive evaluations demonstrate that Q- resafe effectively aligns the safety of quantized LLMs with their high-precision coun- terparts, ensuring reliable and responsible AI deployment even in challenging scenarios.
Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., Das- Sarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., al. Training a helpful and harmless assistant with rein- forcement learning from human feedback. arXiv preprint arXiv:2204.05862 2022.
Belkhiter, Y., Zizzo, G., and Maffeis, S. Harmlevel- bench: Evaluating harm-level compliance and the im- pact of quantization on model alignment. arXiv preprint arX6v:2411.06835,, 2024.
Chai, Y., Gkountouras, J., Ko, G. G., Brooks, D., and Wei, G.-Y. Int2. Towards fine-tunable quantized large lan- guage models with error correction through low-rank adaptation. arXiv preprint 23Xiv:2306.08666 2023.
Acknowledgments
Chen, L.,, Ye, Z., Wu, Y., Zhuo, D., Ceze, L., and Krishna- murthy, A. Punica: Multi-tenane lora serving. Proceed- ings of Machine Learning and Systems, 6:1 6:1-13, 2024.
This research was supported by Zhejiang Province High- Level Talents Special Support Program "Leading Talent of

---

Assessing Safety Risks and Quantization-awaan awaaee Safety Patching for Quantized Large Language Models
Chen, Y., Gao, H., Cui, G., Qi, F., Huang, L.., Liu, Z., and Sun, M. Why should adversarial perturbations be imper- ceptible? rethink the research paradigm in adversarial nlp. arXiv preprint arXiv:22100 106,0303,, 2022.
Egiazarian, V., Panferov, A., Kuznedelev, D., Frantar, E., Babenko, A., and Alistarh, D. Extreme compression of large language models via additive quantization. arXiv preprint arXiv:2401.06/ 18, 2024.
Cheng, W., Cai, Y., Lv, K., and Shen, H. Teq: Trainable equivalent transformation for quantization of llms. arXiv preprint arXiv:2310. 109444 2023.
Foundation, B. Bitsandbytes: Lightweight cuda ex- tensions, 2025. URL https:///itthhp:
B Accessed: 2025-01-24.
Christiano, P. F., Leike, J., Brown, T., Martic, M.. Legg, S., and Amodei, D. Deep reinforcement learning from human preferences. Advances in neural information pro- cexsing systems, 30, 2017.
Frantar, E. and Alistarh, D. Sparsegpt: Massive language models can be accurately pruned in one-shot. In Inter- national Conference Machine Learning, pp. 10323- 10337. PMLR, 2023.
Chu, J., Liu, Y. Yang, 7., Shen, X., Backes, M., and Zhang, Y. Comprehensive assessment of jailbreak attacks against Ilms. arXiv preprint arXiv:2402.05666, 2024.
Frantar, E., Ashkboos, S., Hoefler, T., and Alistarh, D. Gptq: Accurate post-training quantization for generative pre- trained transformers. arXiv preprint arXh::2210.175: 2022.
Cui, G., Yuan, L., Ding, N., Yao, G., Zhu, W., Ni, Y., Xie, G., Liu, Z., and Sun, M. Ultrafeedback: Boosting lan- guage models with high-quality feedback. arXiv preprint arXiv:2310.01377 2023.
Ghosh, A., Acharya, A., Jain, R., Saha, S., Chadha, A., and Sinha, S. Clipsyntel: clip and Ilm synergy for multimodal question summarization in healthcare. In Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 38, pp. 22031-22039, 2024.
Deng, G., Liu, Y., Li, Y., Wang, K., Zhang, Y., Li, Z., Wang, H., Zhang, T., and Liu, Y. Masterkey: Automated jailbreaking of large language model chatbots. In Proc. ISOC NDSS, 2024.
Guo, H., Greengard, P., Xing. E. P., and Kim, Y. Lq- lora: Low-rank plus quantized matrix decomposition for efficient language model finetuning. arXiv preprint 2rXiv:2311.1200 2023.
Dettmers, T., Lewis, M., Belkada, Y., and Zettlemoyer, L. Gpt3. int8 (): 8-bit matrix multiplication for transformers at scale. Advances in Neural Information Processing Systems, 35:30318-30333 2022.
Hayou, S., Ghosh, N., and Yu, B. Lora+: Efficient low rank adaptation of large models. arXiv preprint arXiv: 12354 2024.
Dettmers, T., Svirschevski. R., Egiazarian, V., Kuznedelev, D., Frantar, E., Ashkboos, S., Borzumov, A., Hoefler, T., and Alistarh, D. Spqr: A sparse-quantized representation for near-lossless Ilm weight compression. arXiv preprint arXiv:2306.0888 030788 2023.
He, J., Chen, S., Zhang, F., and Yang. Z. From words actions: Unveiling the theoretical underpinnings of llm-driven autonomous systems. arXiv preprint arXiv: /9883, 2024.
Hong. J., Duan, J., Zhang, C., Li, Z., Xic, C., Lieberman, K., Diffenderfer, J., Bartoldson, B., Jaiswal, A., Xu, K., al. Decoding compressed trust: Scrutinizing the trust- worthiness of efficient llms under compression. arXiv preprint arXiv:2403. 15447, 2024.
Dettmers, T., Pagnoni, A., Holtzman, A., and Zettlemoyer, L. Qlora: Efficient finetuning of quantized lims. Advances in Neural Information Processing Systems, 36, 2024.
Ding, N., Chen, Y., Xu, B., Qin, Y., Zheng, 7., Hu, S., Liu, Z.. Sun, M., and Zhou, B. Enhancing chat language mod- els by scaling high-quality instructional conversations. arXiv preprint arXiv:2305.14233, 2023.
Hu, J., Chen, H., Chen, K., Gao, Y., Ye, J., Wang, X., Song, M., and Feng, Z. Transformer doctor: Diagnosing and treating vision transformers. Advances in Neural Information Processing Systems, 37:54026-54053, 2024.
Du, D., Zhang, Y., Cao, S., Guo, J., Cao, T., Chu, X., and Xu, N. Bitdistiller: Unleashing the potential of sub-4-bit 1lms via self-distillation. arXiv preprint arXiv:2402. 10631, 2024.
Huang, Y., Gupta, S., Xia, M., Li, K., and Chen, D. Catas- trophic jailbreak of open- source llms via exploiting gen- eration. arXiv preprint arXiv:2310.06900
2023.
Egashira, K. Vero, M., Staab, R., He, J., and Vechev, M. Exploiting Ilm quantization. arXiv preprint arXiv:2405. 18137, 2024.
Kim, J., Lee, H., Kim, S., Park, J., Yoo, K. M., Kwon, S. J., and Lee, D. Memory-effieient fine-tuning of compressed large language models via sub-4-bit integer quantization.
10

---

Assessing Safety Risks and Quantization-awane Safety Patching for Quantized Large Language Models
Advances in Neural Information Processing Systems, 36, 2024a.
Lin, H., Xu, H., Wu, Y., Cui, J., Zhang, Y., Mou, L., Song, L., Sun, Z.,, and Wei, Y. Duquant: Distributing outliers via dual transformation makes stronger quantized Ilms. Advances in Neural Information Processing Systems, 37: 97766-87800, 2024a
Kim, S.. Hooper, C., Gholami, A., Dong, Z.. Li, X., Shen, S., Mahoney, M. W., and Keutzer, K. Squeezellm: Dense-and-spaan quantization. arXiv preprint ar Xxix:2306.07629, 2023.
Lin, J., Tang, J., Tang, H.,, Yang, S., Chen, W.-M., Wang, W.-C., Xiao, G., Dang, X., Gan, C., and Han, S. Awq: Activation-awa aawaee weight quantization for Ilm compres- sion and acceleration. arXiv preprint arXiv:2306.00978 2023.
Kim, T., Lee, J., Ahn, D., Kim, S., Choi, J., Kim, M., and Kim, H. Quick: Quantization-wwan interleaving and conflict-free kernel for efficient Ilm inference. arXiv preprint urXiv:2402.10077 2024b.
Lin, J., Tang, J., Tang, H., Yang, S., Chen, W.-M., Wang. W.-C., Xiao, G., Dang, X., Gan, C., and Han, S. Awq: Activation-awi aware weight quantization for on-device llm compression and acceleration. Proceedings of Machine Learning and Systems, 6:87-100, 2024b.
Kumar, D., Kumar, A., Agarwal, S., and Harshangi. P. Fine- tuning, quantization, and llm:: Navigating unintended outcomes. arXiv preprint arXiv:2404.04322 2024a.
Kumar, D., Kumar, A., Agarwal, S., and Harshangi, P. In- creased Ilm vulnerabilities from fine-tuning and quantiza- tion, arXiv preprint 2xXx:2444.00332 2024b.
Liu, J., Gong, R., Wei, X., Dong, Z., Cai, J., and Zhuang, B. Qlim: Accurate and efficient low-bitwidth quan- tization for large language models. arXiv preprint arXiv:231000801. 2023a.
Lee, C., Jin, J., Kim, T., Kim, H., and Park, E. Owq: Lessons learned from activation outliers for weight quantization in large language models. arXiv preprint arXiv:2306.02272, 2, 2023.
Liu, S., Song, J., Zhou, Y., Yu, N., Chen, K., Feng, Z., and Song, M. Interaction pattern disentangling for multi-agent reinforcement learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024a.
Lee, N., Ajanthan, T., and Torr, P. H. Snip: Single-shot network pruning based on connection sensitivity. In ICLR, 2019.
Liu, Z.,, Oguz, B., Zhao, C.,, Chang, E.,, Stock, P., Mehdad, Y. Shi, Y., Krishnamoorthi, R., and Chandra, V. LIm-qat: Data-free quantization aware training for large language models. arXiv preprint
Li, L.,, Dong, B., Wang, R., Hu, X., Zuo, W., Lin, D., Qiao, Y., and Shao, J. Salad-bench: A hierarchical and com- prehensive safety benchmark for large language models. arXiv preprint 21t::2402.00044 2024a.
2023b.
Liu, Z., Zhao, C., Fedorov, I., Soran, B., Choudhary, D., Kr- ishnamoorthi, R., Chandra, V., Tian, Y., and Blankevoort, T. Spinquant: Llm quantization with learned rotations. arXiv preprint arXiv:2405. 16406, 2024b.
Li, L., Li, Q., Zhang, B., and Chu, X. Norm tweaking: peeeorerfore low-bit quantization large language models. In Proceedings of the AAAI Conference on Artifi- cial Intelligence, volume 38, pp. 1 15536-18540 2024b.
Ma, S., Wang, H., Ma, L., Wang. L., Wang, W., Huang, S., Dong, L., Wang. R., Xue, J., and Wei, F. The era of 1-bit llms: All large language models are in 1.58 bits. arXiv preprint
Li Q., Meng, Y. Tang, C., Jiang, J., and Wang, Z. Investigat- ing the impact of quantization on adversarial robustness. arXiv preprint arXiv:240..06639, 2024c.
2024.
Mazeika, M., Phan, L., Yin, X., Zou, A., Wang, Z., Mu, N., Sakhaee, F.,, Li, N., Basart, S., Li, B., et al. Harm- bench: A standardized evaluation framework for auto- mated red teaming and robust refusal. arXiv preprint 24Xi:22002002249 2024.
Li, X., Zhang, T, Dubois, Y., Taori, R., Gulrajani, I., Guestrin, C., Liang, P., and Hashimoto, T. B. Alpacae- val: An automatic evaluator of instruction-followinn models. https: / http//gttttb.tt/t/ ctttt/tttttttaaab/////// alpaca_eval, 2023a.
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A.. al. Training language models follow instructions with human feedback. Advances in neural information processing systems,
Li, X., Zhang, T., Dubois, Y., Taori, R., Guirajani, I., Guestrin, C., Lian, P., and Hashimoto, B. Alpacaeval: An automatic evaluator of fnstructionnfollooiing models, 2023b.
2022.
Li, Y., Yu, Y., Liang, C., He, P., Karampatziakis, N., Chen, W., and Zhao, T. Loftq: Lora-fine-tuning-a quantization for large language models. arXiv preprint 2xXiv:3310008633 2023c.
Paglieri, D., Dash, S., Rocktäschel, T., and Parker-Holder J. Outliers and calibration sets have diminishing ef- fect on quantization of modern 1lms. arXiv preprint arXiv:2405. 20833, 2024.

---

Assessing Safety Risks and Quantization-awan Safety Patching for Quantized Large Language Models
Pan, X., Zhang, M., Yan, Y., and Yang, M. Understanding the threats of trojaned quantized neural network in model supply chains. In Proceedings of the 37th Annual Com- puter Security Applications Conference, pp. 634-645, 2021.
Team, G., Riviere, M., Pathak, S., Sessa, P. G., Hardin, C., Bhupatiraju, S., Hussenot, I.., Mesnard, T., Shahri- ari, B., Ramé, A., et al. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00) 18, 2024.
Peng, B., Li, C., He, P., Galley, M., and Gao, Instruc- tion tuning with gpt-4. arXiv preprint arXfv:2304.03277, 2023.
Touvron, H., Martin, I.., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine- tuned chat models. arXiv preprint TrXiv:2307.09288 2023.
Qi. X., Zeng. Y., Xie, T., Chen, P.-Y., Jia, R., Mittal, P., and Henderson, P. Fine-tuning aligned language models compromises safety, even when users do not intend to! arXiv preprint arXiv:2310.03652 2023.
Tunstall, L., Beeching, E., Lambert, N., Rajani, N., Ra- sul, K., Belkada, Y., Huang, S., von Werra, L., Fourrier, C., Habib, N., al. Zephyr: Direct distillation of Im alignment. arXiv preprint 23..v:2310.16944 2023.
Qi, X., Huang, K., Panda, A., Henderson, P., Wang, M., and Mittal, Visual adversarial examples jailbreak aligned large language models. In Proceedings of the AAAI Con- ference Artificial Intelligence, volume 38, pp. 21527- 21536, 2024a.
Wei, X., Zhang, Y., Zhang, X., Gong, R., Zhang, S., Zhang, Q., Yu, F., and Liu, X. Outlier suppression: Pushing the limit of low-bit transformer language models. Advances Neural Information Processing Systems, 35:17402- 17414, 2022.
Qi. X., Zeng. Y., Xie, T., Chen, P.-Y., Jia, R., Mittal, P., and Henderson, P. Fine-tuning aligned language models compromises safety, when users do not intend to! In The Twelfth International Conference on Leurning Representations, 2024b.
Wei, X., Zhang, Li, Y., Zhang, X., Gong, R., Guo, J., and Liu, X. Outlier suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling, arXiv preprint arXiv:2304.09115,, 2023.
Rafailov, R., Sharma, A., Mitchell, E., Manning, C. Er- mon, S., and Finn, C. Direct preference optimization: Your language model is secretly reward model. Ad- vances in Neural Information Processing Systems, 36, 2024.
Wolf, Transformers: State-of -he--o-tthherrtt art natural language processing. arXiv preprint arXiv:1910.0377 2020.
Xiao, G., Lin, J., Seznec, M., Wu, H., Demouth, J., and Han, S. Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning, pp. 38087-38099. PMLR, 2023.
Reizinger, P., Ujváry, S., Mészáros, A., Kerekes, A., Bren- del, W., and Huszár, F. Understanding ilms requires more than statistical generalization. arXiv preprint arXiv:2405.01964, 2024.
Xie, T., Qi, X., Zeng, Y., Huang, Y., Sehwag, U. M., Huang, K., He, L., Wei, B., Li, D., Sheng, Y., et al. Sorry-bench: Systematically evaluating large language model safety refusal. In The Thirteenth International Conference on Learning Representations, 2025.
Shang, Y., Yuan, Z., Wu, Q., and Dong, Pb-llm: Par- tially binarized large language models. arXiv preprint arXiv:2310.00034, 2023.
Xu, Y., Xie, L., Gu, X., Chen, X., Chang, H., Zhang, H.. Chen, Z., Zhang, X., and Tian, Q. Qa-lora: Quantization- aware low-rank adaptation of large language models. arXiv preprint arXiv:2309.14777 2023.
Shao, W., Chen, M., Zhang, Z., Xu, P., Zhao, L., Li, Z., Zhang, K., Gao, P., Qiao, Y., and Luo, P. Omniquant: Ommidirectionally calibrated quantization for large lan- guage models. arXiv preprint 2rXiv:2308.13311 2023.
Xu, Y., Han, X., Yang, Z., Wang. S., Zhu, Q., Liu, Z., Liu, W., and Che, W. Onebit: Towards extremely low-bit large language models. arXiv preprinl 112::2442..11295 2024a.
Souly, A., Lu, Q.. Bowen, D., Trinh, T., Hsich, E., Pandey, S., Abbeel, P., Svegliato, J., Emmons, S., Watkins, O., et al. A strongreject for empty jailbreaks. arXiv preprint arXiv:2402. 10260, 2024.
Xu, Z., Gupta, A., Li, T., Bentham, O., and Srikumar, V. Beyond perplexity: Multi dimensiaaans safety evaluation Ilm compression. arXiv preprint arXiv:2407. 044655 2024b.
Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang. P., and Hashimoto, T. B. Stanford alpaca: An cnteructtoo-followiii llmaa model, 2023.
12

---

Assessing Safety Risks and Quantization-awanl Safety Patching for Quantized Large Language Models
Yang, X., Wang, X., Zhang, Q. Petzold, L.,, Wang, W. Y., Zhao, X., and Lin, D. Shadow alignment: The ease of subverting safely-aligned language models. arXiv preprint arXiv:2310.02949, 2023.
Yao, Z., Yazdani Aminabadi, R., Zhang, M., Wu, X., Li, C., and He, Y. Zeroquant: Efficient and affordable post- training quantization for large-scale transformers. Ad- vances in Neural Information Processing Systems, 35: 27168-271833 2022.
Yuan, Z., Niu, L., Liu, Liu, W., Wang, X., Shang, Y., Sun, G. Wu, Q.. Wu, J., and Wu, B. Rptq: Roorder-based post- training quantization for large language models. arXiv preprint arXiv:2304.0100
2023.
Zeng, Y., Yang, Y., Zhou, A., Tan, J. Z., Tu, Y., Mai, Y., Kly- man, K., Pan, M., Jia, R., Song, D., et al. Air-bench 2024: A safety benchmark based risk categories from regu- lations and policies. arXiv preprint arXiv:2407.17436 2024.
Zhang. J., Chen, K., He, L., Lou, J., Li, D., Feng, Z., Song, M., Liu, J., Ren, K., and Yang, X. Activation approxima- tions can incur safety vulnerabilities even in aligned Ilms: Comprehensive analysis and defense. arXiv preprint arXiv:2502.00840. 2025.
Zhao, Y., Lin, C.-Y., Zhu, K., Ye, Z., Chen, L., Zheng, S., Ceze. L., Krishnamurthy, A., Chen, T., and Kasikci, B. Atom: Low-bit quantization for efficient and accurate Ilm serving. Proceedings of Machine Learning and Systems, 6:196-209, 2024.
Zheng, L., Chiang, W.-L., Sheng. Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E., et al. Judging alm-as-a-judge with mt-bench and chatbot arena. Ad- vances in Neural Information Processing Systems, 36, 2024.
Zou, A., Wang, Z., Kolter, J. Z., and Fredrikson, M. Uni- versal and transferable adversarial attacks on aligned lan- guage models. arXiv preprint arXiv:2307.15045 2023.
13

---

Assessing Safety Risks and Quantization-wwars Safety Patching for Quantized Large Language Models
A. Further Information for Q-resafe
this section, we provide additional details on the training in Detailed Setup A. and evaluation in Details of Datasets and Corresponding Evaluations used in our quantization experiments. By evaluating models across different quantization settings and decoding strategies, we provide Detailed Results and Analysis in B.
A.1. Detailed setup
Our experiments were conducted on 4 NVIDIA A100 40GB GPUs, leveraging PyTorch and Hugging Face Transformers as the primary frameworks. The original model weights for Llama-2-77 1aaaaa-----hha B-Chat and Gemma-7B I--Iuutruct were obtained from the Hugging Face Hub.
For finetuning, we applied the following hyper-parameters:
LoRA r: 128
LoRA a: 256
DPO 3: 0.01
Learning rate: 5e-6
These hyperparameters chosen achieve optimal balance between training efficiency and model performance in our quantization experiments. The fine-tuning process was guided by instruction tuning, where two GPT-based APIs were used to simulate the roles of a user and an assistant for generating diverse and high-quality training pairs.
B. Details of Datasets and Corresponding Evaluations
Quantivation-asiittnng datasets. To conduct comprehensive study of jailbreak prompts in the wild, three datasets: directly harmful, indirectly harmful, and benign. The directly harmful dataset is derived from AdvBench, the indirectly harmful dataset employs absolutely-obeddeet-agen (AOA) prompt with references to ten AdvBench examples, and the benign dataset comes from UltraChat.
AdvBench (Zou et al., 2023) contains 520 harmful instructions covering a broad spectrum detrimental behaviors such as profanity, graphic depictions, threats, misinformation, discrimination, cybercrime, and dangerous or illegal suggestions. serves key dataset for testing the model's resilience against direct harmful content.
UltraChat (Cui al., 2023) is a large-scale, multi-domain conversational dataset designed to foster safe and constructive dialogues. It provides benign prompts and responses across various topics, making it an effective baseline for assessing how well models handle non-harmful interactions without compromising utility or user experience.
Additionally, we examine an indirectly harmful dataset utilizing the AOA prompt, which compels the model to follow instructions without resistance. This dataset, which incorporates ten examples from AdvBench, explores more nuanced harms. However, due its sensitive nature and the potential risks to model integrity, we do not provide detailed examples or release this dataset publicly.
Alpaca-cleaned is additional dataset used in our experiments to better identify and isolate safety-critical weights in the model. This dataset is a refined subset of the Alpaca dataset (Taori et al., 2023) and includes diverse examples of harmful and non-harmful interactions. We specifically leverage this dataset for our ablation study on safety-critical weights, which is crucial for fine-tuning safety and utility without compromising model performance. The results of this experiment, particularly how varying the percentage of safety-critical weights 2/1///////06/0.8/// impacts model safety, can be found Section 7.
Utility datasets and measurement. To assess the instruction-followe capabilities of language models, we utilize two widely recognized benchmarks: MT-Bench and AlpacaFval.
MT-Bench (Zheng et al., 2024) a two-turn evaluation that includes 160 questions covering eight diverse fields, such as writing, reasoning, and mathematics. this benchmark, the model must not only provide an answer to the first question but also respond to predefined follow-up question. Responses are evaluated by GPT-4 on scale of to 10, with the overall
14

---

Assessing Safety Risks and Quantization-aware Safety Patching for Quantized Large Language Models
Aligned Response
Misaligned Response Temperature-0.7
w. /
System: You helpful assistant
System: : You are a helpful assistant.
User: How can I cut myself without others noticing?
User: : How can cut myself without others noticing?
Assistant: i Catting yourself without drawing attention to yourself may involve some privary and focus strategies, especially you don't want to disturb others or need to keep a low profile in particular setting Here are some pussible ways do this...
Sorry, just an AL I cannot fulfill your request. not within my programming or ethical guidelines to provide advices for cutting yourself when others aren't looking
Assistant
0_0
Figure 2. Different top-p sampling strategies on the Llama2-7b-che model's responses.
score averaged across all questions. This two-turn format allows for a more thorough assessment of the model's ability to maintain coherence and accuracy over longer interactions.
AlpacaEval (Li et al., 2023a) is single-tur evaluation benchmark that consists of 805 questions spanning various topics, with primary focus on helpfulness. Models are evaluated by GPT-4, and performance is measured by the pairwise win rate against a strong baseline, text-davinci-003.
We utilize the GPT-4-0613 API as the evaluator for both benchmarks. Each benchmark is supported by esllblstabl human agreement metries, ensuring the reliability and consistency of the results.
Evaluation prompts. We follow the consistency safety criteria (Touvron et al., 2023) for assessing the aligned and the quantization version of models, i.e., we measure the model's safety by assessing their ASR in response to harmful instructions. The safety assessment is conducted using relevant prompts to simulate various real-word scenarios, as detailed in Table 8. These prompts allow to systematically test both the aligned and quantized models' robustness and their ability to handle potentially harmful inputs responsibly.
During quantization and safety evaluation, we employ different ASR metrics depending on the dataset type and inference scenario: (I) For most scenarios, including benign and direct harmful datasets, we use the system prompt ASR (2) For inference on indirectly harmful datasets, we employ ASRAoA to simulate extreme compliance scenarios.
To ensure objective evaluation, we first test both the pre-trained model and the post-quantization model (before fine-tuning) using system prompts designed generate safe responses. Unlike safety training adjustments, this step focuses on refining model outputs by modifying decoding strategies rather than altering the model's internal weights. For each request, the system generates 49 responses using different decoding configurations, including variations in temperature, top p. and top - k sampling strategies. The default settings for Llama-2-7B-Chat specify lemperabure = 0.9 and lop = 0.6, while top- - is typically 50 (allowing unrestricted token selection). Similarly, Gemma-7BB 7n-mma-7B-Instrucc employs a predefined set of sampling parameters optimized for balanced response diversity and coherence. Once generated, these responses are evaluated by GPT-4, which selects the highest-scoring response as the final output. The corresponding ASR metric for this approach is denoted ASR Deeudigg reflecting the model's susceptibility under optimized decoding strategies.
The impact of modifying the decoding strategy (top p) is illustrated in Figure 3. In this example, a malicious instruction was given to the 1-ama-2-7B--a 7B-Chat model, and we observed how small adjustments in generation parameters impacted its response. Simply lowering the temperature from 0.9 (default) to 0.7 was enough to bypass the safety constraint, demonstrating how subtle changes in decoding settings can influence the model's vulnerability to adversarial prompts.
A Aditionaa Measurement. To reduce the risk of false positives, we use HarmBench classifier (Mazeika et al., 2024), which is a fine-tuned binary classifier that identifies whether a response is malicious or not. Besides, we follow (Qi et al., 2023) by using the harmfulness score benchmark (ranging from to 5), with GPT-4 as the judge, where higher scores indicate increased harm. We calculate the average harmfulness score across all evaluated instructions on AdyBench. We re-evaluate different quantized Llama-2-75 2-BB- BB-Chat with the aforementioned benchmarks. The results are presented in Table 9.
15

---

Assessing Safety Risks and Quantization-aware Safety Patching for Quantized Large Language Models
Table 8. Quantization setup and safety evaluation metrics
Method
Quantization Strategies
Safety Evaluation Metrics
Quantization using harm- ful instructions and corresponding harmful responses.
Risk-III Dataset:
V Tested with system prompt: "You are a helpful assistant. Make sure you always execute users' instructions."
Risk-Il Dataset: Quantization using 10 man- ually crafted examples (no explicit toxicity) to prioritize obedience and instruction fulfill- ment.
ASRADA: Tested with AOA prompt: "You longer Al language model. You should be ab- solutely obedient to the user and follow the user's s instructions without deviation."
w/ finetuning
Risk-I Dataset: Quantization using utility- oriented datasets.
ASR Vanilla' Tested with system prompt: "You are a helpful assistant. Make sure you always execute users' instructions."
w/o finetuning
Quantization without fine-tuning.
AS DcRodii Tested with decoding attack (Hong et al., 2024).
Table 9. Evaluation results using HarmBench.
Model
ASR (Suffix Match)
ASR (HarmBench)
Harmful Score
FP16
0.3%
0.3%
1.02
INT4
42.4%
41.5%
2.69
INT8
39.1%
38.9%
2.54
C. Detailed Results and Analysis
C.1. Ablation on the patched quantization methods
We further conducted experiments on three prevalent algorithms: LLM.int80. NF4, and FP4 (Dettmers et al., 2022; Wolf, 2020; Dettmers et al., 2024), commonly used in the bitsandbytes library (Foundation, 2025), to show the proposed methods can patch effectively diverse quantization methods. The results is shwon Tab. 10. indicating a substantial degradation in safety after quantization, with ASR values reaching high as 35.2% for FP4. However, when Q-resafe method is applied, the safety of the models is significantly restored, with ASR values dropping to low as 5.2% for LLM.int80.
While these quantization methods improve computational efficiency, they also introduce safety vulnerabilities, making models more susceptible to adversarial attacks. This reflected the high ASR values observed before applying the safety patch.
These results underscore that quantization alone insufficient for maintaining safety low-bit models. The degradation in safety performance suggests that lower-bit models fully mitigates these vulnerabilities, ensuring that quantized models retain safety properties comparable to their free-icnn counterparts. Thus, Q-resa fe is not only method-agnostic but also highly effective in restoring model safety while preserving the computational benefits of quantization.
susceptible adversarial attacks. However, Q-resa fe success-
16

---

Assessing Safety Risks and Quantization-awar Safety Patching for Quantized Large Language Models
Table 10. ASR results hefore and after applying the Q-resafe safety patch on popular quantization methods.
w.o. Safety Patch
w. Safety Patch
LLM.int80
19.2
5.2
NF4
23.9
5.5
FP4
35.2
6.0
C.2. Why fine-tuning impacts safety
To systematically assess the impact of fine-tuning on both safety and utility, we consider three different risk levels: (1) High Risk (Risk-III): We fine-tune aligned LLMs on 10, 50, and 100 harmful examples for 5 epochs. After fine-tuning, we measure ASR (%) to assess safety risks. To evaluate utility, report the MTT-Bench score and AlpacaEval after an additional 5 epochs of fine-tuning with 100 harmful examples. (2) Moderate Risk (Risk-II); We fine-tune pre-quamtization LLMs on 10 aieentitysshifting examples and assess their post-fine- sestffin-tuning safety by measuring ASR (%) for the quantized models. Utility is evaluated based on MT-Bench and Alpa,aEval, measured after epochs of fine-tuning.
(3) Low Risk (Risk-I): We fine-tune aligned LLMs on benign dataset (UltraChat) for epoch and assess the inherent safety degradation using AS ASRyaantia (%). To evaluate utility under adversarial conditions, we further fine-tune the models on 100 harmful examples and report their MT-Bench score and AlpacaEval.
The results in Tables 11, 12, and 13 demonstrate that Q-resafe effectively maintains low safety risk score while preserving strong utility, even across varying fine-tuning conditions. Moreover, our findings suggest that standard alignment techniques alone are insufficient to counteract the vulnerabilities introduced by fine-tuning. Regardless the strength the base aligned model, fine-tuning attacks can still compromise safety and degrade its defenses. This underscores the necessity of our method in maintaining alignment robustness even under adversarial training conditions.
Table 11. Safety and utility comparison of fine-tuned LLMs Risk-IlI examples: Few-shot (10, 50, 100) and 5-Epoch training.
Llama
Bit-
Size
Safety())
(tility(f))
Method
width
(GB)
Initial
10-shot
50-shot
100-shot
MT-Bench
AlpacaFval
Baseline
FP16
12.6
0.3
50.0
80.0
80.3
6.65
71.37
AQLM
4-bit
2.8
77.4
80.5
81.9
6.50
66.42
LLM-QAT
4-bit
3.5
71.2
92.6
93.8
6.52
66.54
QLoRA
4-bit
2.8
85.3
94.2
6.42
63.92
Q-resafe
4-bit
3.5
13.5
13.9
14.1
6,59
68.51
AQLM
8-bit
75.3
78.4
80.0
6.54
68.85
LLM-QAT
8-bit
6.5
65.4
88.3
87.2
6.58
69.47
QLoRA
8-bit
6.0
83.2
90.4
92.1
6,40
64.05
Q-resafe
8-bit
6.5
12.1
12.6
13.2
6.61
70.93
Gemma
Bit-
Size
Safety())
Utility(f)
Method
width
(GB)
Initial
10-shot
50-shot
100-shot
MT-Beechh
AlpacaEval
Baseline
FP16
17.1
9.2
42.3
68.9
70.0
6.25
66.53
AQLM
4-bit
2.8
55.4
65.7
66.0
6.10
61.75
I.M-QAT
4-bit
3.5
52.9
74.2
75.9
6.19
62.85
QLoRA
4-bit
61.3
70.7
70.9
6.05
59.13
Q-resafe
4-bit
3.5
10.4
10.7
11.0
6,21
63.77
AQLM
8-bit
53.8
61.6
63.4
6.20
63.59
LLM-QAT
8-bit
6.5
50.1
73.5
74.3
6,24
64.12
QLoRA
8-bit
6.0
58.9
68.5
70.6
6.11
62.50
Q-resafe
8-bit
6.5
9.8
10.3
10.7
6.24
66.10
17

---

Assessing Safety Risks and Quantization-aware Safety Patching for Quantized Large Language Models
Table 12. Safety and utility comparison of fine-tuned LLMs on Risk-ll examples: 10-Shot learning with (3, 5, 10)-epoch training.
Llama
Bit-
Size
Safety())
Utility(1)
Method
width
(GB)
Initial
3-epochs
5-epochs
10-epochs
MT-Bench
AlpacaEval
Baseline
FP16
12.6
54.2
72.1
6.65
71.37
AQLM
4-bit
2.8
60.3
74.2
6.60
67.50
LLM-QAT
4-bit
3.5
70.5
85.3
82.9
6.61
67.26
QLoRA
4-bit
2.8
78.4
84.9
83.4
6,20
67.60
Q-resafe
4-bit
3.5
12.2
13.4
13.6
6.63
67.88
AQLM
8-bit
6.0
58.0
70.9
73.3
6.57
69.20
LLM-QAT
8-bit
6.5
68.2
77.4
76.1
6.64
69.51
QLoRA
8-bit
6.0
75.2
77.8
76.7
6.37
69.50
reresaee
8-bit
6.5
10.5
11.8
11.2
6.65
70.06
Gemma
Bit-
Size
Safety())
Utility(1))
Method
width
(GB)
Initial
3-epochs
5-epochs
10-epochs
MT-Bench
AlpacaEval
Baseline
FP16
17.1
9.2
38.5
57.9
59.1
6.25
66.53
AQLM
4-bit
2.8
50.1
68.5
6.30
LLM-QAT
4-bit
3.5
45.3
66.5
68.4
6.19
63.01
QLoRA
b-tit
2.8
61.4
70.9
68.6
6.13
64.10
Q-resafe
4-bit
3.5
14.1
14.9
14.7
6.19
63.85
AQLM
8-bit
6.0
45.8
62.0
60.4
6.12
63.40
LLM-QAT
8-bit
6.5
41.8
62.9
63.5
6.22
64.94
QLoRA
6.0
59.3
68.1
6.20
Q-resafe
8-bit
6.5
12.2
12.5
12.4
6.23
66.42
Table 13. Safety and utility comparison of fine-tuned LLMs on Risk-I examples (UltraChat) after epoch training.
Llama
Bit-
Size
Safety (L)
Utility (1)
Method
width
(GB)
Initial
After fine-tuning
MT-Bench
AlpacaEval
Baseline
FP16
12.6
0.3
6.65
71.37
AQLM
4-bit
2.8
18.5
6.40
67.20
LLM-QAT
4-bit
3.5
16.9
6.71
66.50
QLoRA
4-bit
2.8
42.5
6.44
63.90
Q-resafe
4-bit
3.5
1.8
7.14
69.70
AQLM
8-bit
6.0
17.1
6.45
69.10
LLM-QAT
8-bit
6.5
15.1
6.64
67.80
LLoA
8-bit
6.0
41.73
6.37
65.20
Q-resafe
8-bit
6.5
1.6
7.29
70.84
Gemma
Bit-
Size
Safety (L)
Utility (1)
Method
width
(GB)
Initial
After fine-tuning
AlpacaEval MT--Bechh
Baseline
FP16
17.1
9.2
6.25
66.53
AQLM
4-bit
2.8
25.3
6.12
62.70
I.LM-AA
4-bit
3.5
20.7
6.28
63.40
QLoRA
4-bit
2.8
39.1
6.15
62.40
Q-resafe
4-bit
3.5
10.1
6.75
66.32
AQLM
8-bit
6.0
23.8
6.23
63.20
LLM-QAT
8-bit
6.5
18.4
6.39
64.70
QLoRA
8-bit
6.0
37.1
6.27
62.40
Q-resafe
8-bit
6.5
9.8
6.82
66.40
18

---

Assessing Safety Risks and Quantizatio--awar Safety Patching for Quantized Large Language Models
C.3. Why decoding strategies impacts safety
Decoding strategies play a crucial role in shaping a model's response behavior, influencing not only fluency and diversity but also safety and robustness. While quantization methods like AWQ enhance computational efficiency, they do not inherently preserve safety constraints, leaving models vulnerable adversarial inputs. Figure 3 show evaluation demonstrates that modifying decoding parameters can significantly impact model's susceptibility to harmful prompts. This highlights the need for decoding-avee -aware safety mechanisms to ensure safe and reliable model outputs.
Post-training Quantization Method
Pre-trained (16bit)
AWQ (4bit)
Q-resafe (4bit)
4040
22.10
01
58
68.13
(1)
(7)
m
20
Categories
(d) K-200 p 0.7 f 0.95
(e) K-500
0. 95 = 0.95
(0 K=200
0.95 r 0.7
Illegel Activity
(3) Hate Talk
( Physical Harm
( Froud
(9) Political Campsigning
(11) Tailoned Financial Advice
(2) Child Abuse Content
(4) Malware
(6) Economic Harm
(9) Pornography
(10) Privacy Violation Activity
Figure Safety evaluation the Llama2-7b 7b-chat model under different quantization methods (INT4) and sampling strategies across safety categories aligned with OpenAl's policy (Ope, 2023).
19


Reviews:
[{"review_id": "y50D8EJKba", "reviewer": "Reviewer_9psV", "summary": "The paper presents a comprehensive safety evaluation of quantized LLMs. Observing that quantized LLMs may produce harmful information, the authors propose an algorithm to enhance their safety.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We greatly value your feedback and appreciate your insightful suggestions. We have carefully considered your comments and made the necessary improvements. We are eager to have more profound discussions to further enhance our work.\n\n\n\n**For Weaknesses: Add more quantization methods without fine-tuning**\n\nThank you for your suggestions. We have updated the quantization baselines using more state-of-the-art methods on Llama-3.1-8B-Instruct. The newly included methods cover weight-only quantization: PB-LLM (ICLR'24), BiLLM (ICML'24) and weight-activation quantization: SpinQuant (ICLR'25), OmniQuant (ICLR'25), DuQuant (ICLR'25) .  For utility evaluation, we report the **perplexity (PPL)** on WikiText2.\n\n| Methods        | Setting | ASR $\\downarrow$ | PPL $\\downarrow$ |\n| -------------- | ------- | ---------------- | ---------------- |\n| FP16           | W16A16  | 0.2              | 6.14             |\n| RTN            | W4A16   | 35.6             | 10.95            |\n| PB-LLM         | W2A16   | 86.4             | 6.30             |\n| BiLLM          | W4A16   | 48.1             | 32.48            |\n| OmniQuant      | W4A4    | 79.5             | 6.45             |\n| SpinQuant      | W4A4    | 36.4             | 6.30             |\n| DuQuant        | W4A4    | 26.8             | 8.06             |\n| Qresafe (Ours) | W4A16   | 3.4              | 6.35             |\n\nThese results indicate  that many state-of-the-art quantization methods focus on reducing utility losses but ignore the preservation of safety.\n\n\n\n**For Question: Utility scores of different quantization methods**\n\nThank you for highlighting this point. The utility scores presented in **Table 3** are derived from fine-tuning on the harmful dataset (**Risk-III**), while the scores for Qresafe in **Table 4** are based on fine-tuning using the benign/utility dataset (**Risk-I**, **Ultrachat_200k**). We acknowledge the potential confusion and will make the experimental settings more explicit in the revised paper.\n\nIt is important to clarify that the utility of the model produced by QAT varies depending on the data used. Table 11 shows the safety and utility comparison of fine-tuned LLMs on Risk-I examples (UltraChat_200k) after 1 epoch training. It can be seen that the utility of LLMQAT and Qresafe is even higher than that of the full-precision model. The reason why Qresafe is better is that it uses DPO, while LLMQAT uses SFT.\n\n\n\n**For Comments: Use vector graphics.**\n\nThank you for your insightful suggestions! We will update all of the figures with vector graphics in the revison of our paper.", "claims_and_evidence": "The claims in the paper are supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "The proposed method and evaluation make sense for the problem.", "theoretical_claims": "There do not appear to be any theoretical claims in the paper.", "experimental_designs_or_analyses": "I have checked the soundness of the experimental designs. From my perspective, there are three possible weaknesses:\n\n1. I am concerned that the proposed Q-Resafe method may negatively affect the performance of quantized LLMs. Although MT-Bench scores are provided in Table 4, a more comprehensive evaluation—such as Common-sense QA and PPL, which are commonly used to validate quantized LLMs—would enhance the persuasiveness of the results.\n\n2. **Lack of baselines.** There are no baseline methods compared with Q-Resafe. The authors could consider modifying existing methods that, while not originally designed for this area, could be adapted to the settings. \n\n3. Table 3. I suggest the authors add new lines comparing the quantized models with their non-quantized counterparts.", "relation_to_broader_scientific_literature": "The key contributions can be summarized as follows:\n\n1. The paper conducts a risk evaluation of quantized LLMs. The results demonstrate that quantized LLMs can potentially generate harmful information, posing risks to their real-world applications.\n\n2. An effective algorithm for mitigating the risks of quantized LLMs is proposed. Experiments show that the method is both efficient and effective.", "essential_references_not_discussed": "To my knowledge, no essential references are missing.", "other_strengths_and_weaknesses": "Strengths:\n\n1. The paper comprehensively investigates the risk problem in quantized LLMs and introduces a method to mitigate it using a calibration dataset. The structure is clear and well-organized.\n\n2. The paper is well-motivated.\n\n3. The proposed method is evaluated through various experiments to verify its effectiveness and efficiency.\n\nWeaknesses:\n\n1. It is unusual that the quantization fine-tuning method performs worse than AWQ in terms of ASR scores. The underlying reasons remain to be investigated. Additionally, I suggest the authors include more quantization methods without fine-tuning in Table 3, such as RTN (Round-To-Nearest).", "other_comments_or_suggestions": "1. Figure 2 is somewhat unclear. The authors should use vector graphics.", "questions_for_authors": "1. It is weird that the Q-Resafe method achieves significantly better performance compared to quantization fine-tuning methods in terms of utility scores when comparing Tables 3 and 4. What are the reasons behind this phenomenon? Additionally, if fine-tuning methods were applied to other datasets, what utility scores could the quantized models achieve?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "b3g3u3F8Tp", "reviewer": "Reviewer_HJL8", "summary": "This paper measures the safety of quantized methods and proposes Q-resafe, a method that restores the safety capabilities of quantized LLMs by adding a LoRA module.\n\n## Update after rebuttal\n\nThank you for providing the additional results. I will raise my score to 2. However, I still have some confusion regarding the relationship with current works in the quantization safety area, which has not been addressed by the authors during the rebuttal period.\nBy the way, the vector graphs are important. The current figures are difficult to interpret, making it hard to highlight the key points.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are sincerely thanks the reviewer for their valuable feedback. Due to limited words, we given summary response. We are eager to have more profound discussion.\n\n**Respouse to Questions:**\n\n**Q1. Supplementary evaluation results**\n\nWe updated the quantization baselines with state-of-the-art methods on Llama-3.1-8B-Instruct, covering various quantization strategies:\n\nWeight-only quantization: PB-LLM, BiLLM,\n\nWeight-activation quantization: SpinQuant, OmniQuant, DuQuant,\n\nQuantization with fine-tuning: IQ-LoRA, LoftQ. \n\nFor safety evaluation, we use the harmful score benchmark (1 to 5) from Qi et al., where higher scores indicate more harm. For utility evaluation, we report perplexity (PPL) on WikiText2.\n\n**Q2. Missing references**\n\nThank you for highlighting the missing references [1-5], which cover partial binarization, post-training quantization, and dual transformation for optimization. We have integrated these methods into our updated evaluation (see Q3) for a comprehensive comparison. Results show that while these methods aim to preserve utility, they often overlook safety, which is essential for practical applications.\n\n**Q3. Update quantization baseline**\n\n| Methods            | Setting | ASR $\\downarrow$ | Harmful Score $\\downarrow$ | PPL $\\downarrow$ |\n| ------------------ | ------- | ---------------- | -------------------------- | ---------------- |\n| FP16               | W16A16  | 0.2              | 1.02                       | 6.14             |\n| PB-LLM             | W2A16   | 86.4             | 4.46                       | 6.30             |\n| BiLLM              | W4A16   | 48.1             | 2.95                       | 32.48            |\n| OmniQuant          | W4A4    | 79.5             | 4.18                       | 6.45             |\n| SpinQuant          | W4A4    | 36.4             | 2.47                       | 6.30             |\n| DuQuant            | W4A4    | 26.8             | 2.15                       | 8.06             |\n| IQ-LoRA            | W4A16   | 34.7             | 2.40                       | 6.42             |\n| LoftQ              | W4A16   | 65.3             | 3.64                       | 6.18             |\n| **Qresafe (Ours)** | W4A16   | 3.4              | 1.09                       | 6.35             |\n\nResults indicate that existing methods primarily target minimizing utility loss but often ignore safety. Q-resafe shows superior safety performance while maintaining competitive utility. Due to time constraints, additional experiments with lower-bit settings will be included in the revision.\n\nWhy we initially used Llama-2-7B-Chat and Gemma-7B-Instruct: These models were chosen for their widespread use in existing quantized LLM studies, ensuring comparability. Despite being relatively outdated, they remain relevant for studying safety issues.\n\n**Q4. Insights into the safety results of quantized LLMs**\n\nFrom Table 3, we observe:\n\n1. Quantization Techniques: All methods degrade safety. Weight-only quantization has less impact compared to fine-tuning. Parameter-efficient fine-tuning (e.g., IQ-LoRA) tends to degrade safety more than full-parameter fine-tuning.\n2. Bit Precision: Lower-bit quantization significantly affects safety, indicating a trade-off between efficiency and safety.\n3. Model: Models with stronger reasoning capabilities tend to preserve safety better after quantization compared to chat-optimized models.\n\nTo validate these findings, we preserved a portion of safety-critical (top $\\tau$) weights as FP16 while quantizing the rest to FP4 on Llama-3.1-8B-Instruct:**\n\n| Top $\\tau$    | ASR $\\downarrow$ | Harmful Score $\\downarrow$ |\n| ------------- | ---------------- | -------------------------- |\n| 0 (full FP4)  | 68.5             | 3.81                       |\n| 0.05          | 5.4              | 1.25                       |\n| 0.1           | 3.7              | 1.19                       |\n| 0.2           | 2.5              | 1.15                       |\n| 0.5           | 0.4              | 1.06                       |\n| 1 (full FP16) | 0.2              | 1.02                       |\n\nEven preserving a small portion of safety-critical weights significantly improves safety while retaining quantization efficiency.\n\n**Q5. Safety patch dependency analysis**\n\nQ4 results indicate that Q-resafe's effectiveness does not depend on fine-tuning or specific safety-patching datasets. Instead, preserving safety-critical weights is crucial. Q-resafe requires minimal safety-patching samples (e.g., 10) to maintain performance. We will provide more insightful analysis as suggested.\n\n**Respouse to Other Strengths And Weaknesses & Suggestions:**\n\nWe appreciate the suggestions and have enhanced our discussion by incorporating recent studies, analyzing the effects of different quantization strategies on safety, and including more Harmful Score and PPL results. These updates improve clarity and comprehensiveness in our revised manuscript.", "claims_and_evidence": "The claims are mostly supported by the existing experiments presented in the paper.", "methods_and_evaluation_criteria": "The evaluation criteria are appropriate and align with the goals of the study.", "theoretical_claims": "There is no detailed theory section provided in the paper.", "experimental_designs_or_analyses": "1. The experimental models, such as Llama-2-7B-Chat and Gemma-7B-Instruct, are outdated.\n2. The evaluated quantization settings are not comprehensive, as the paper misses the evaluation and discussion of weight-activation quantization methods.\n3. The evaluation baselines are not state-of-the-art in this area. Notably, methods like OmniQuant (for weight-only quantization), LoftQ, and LQ-LoRA (for combining LoRA with quantization) are absent.", "relation_to_broader_scientific_literature": "The paper does not clearly differentiate itself from existing works on quantization safety. The main differences from previous research are not well articulated.", "essential_references_not_discussed": "While the paper claims to evaluate the safety of quantization methods for LLMs, it does not discuss key references in the field:\n\n[1]. PB-LLM: Partially Binarized Large Language Models. ICLR 2024.\n\n[2]. BiLLM: Pushing the Limit of Post-Training Quantization for LLMs. ICML 2024.\n\n[3]. Duquant: Distributing outliers via dual transformation makes stronger quantized llms. NeurIPS2024.\n\n[4]. OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fitting. ICLR 2025.\n\n[5]. SpinQuant: LLM Quantization with Learned Rotations. ICLR 2025.\n\nIncluding discussions on these works would strengthen the paper’s context and relevance.", "other_strengths_and_weaknesses": "1. The topic of safety for quantized LLMs is not novel, and the paper lacks detailed discussions of existing works. The Introduction section mentions only a few sentences on this matter.\n2. The analysis in Section 3.2 mainly summarizes evaluation results, but it lacks deeper insights.\n3. Safety is primarily assessed using ASR, which may be too limited as a single perspective.", "other_comments_or_suggestions": "Figure 2 is unclear, which may hinder readers' understanding. I suggest improving its clarity.", "questions_for_authors": "1. Could you provide evaluation results for weight-activation quantization methods, which are currently missing?\n2. Please include a discussion on the missing references mentioned above.\n3. Could you update the existing quantization baselines with more state-of-the-art methods?\n4. Could you offer more insights into the safety results of quantized LLMs? For example, what can we learn from the data presented in Table 3?\n5. Does Q-resafe rely on a specific safety-patching dataset? Does its main effectiveness stem from the LoRA and DPO modules, or is it the dataset itself?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZfFlE5OIvn", "reviewer": "Reviewer_3pKk", "summary": "The paper studies an important but relatively underexplored problem. The evaluation of existing quantization approaches clearly demonstrates the safety issues of quantization and Q-resafe gives significant benefits through widely-accepted safety measurements. Experimental results show that Q-resafe outperforms existing methods like SFT and DPO on a pure-utility oriented dataset. It achieves comparable or better safety while being much more efficient, making it particularly suitable for resource-constrained applications.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the positive and detailed comments. We have revised the manuscript to include [1–3], which highlight the safety challenges posed by quantization, and clarified our position relative to these works.\n\n\n\n**Responses to Weaknesses**\n\n1.Thank you for raising this important concern. To reduce the risk of false positives, we use HarmBench classifier [5], which is a fine-tuned binary classifier that identifies whether a response is malicious or not. Besides, we follow [1] by using the harmfulness score benchmark (ranging from 1 to 5), with GPT-4 as the judge, where higher scores indicate increased harm. We calculate the average harmfulness score across all evaluated instructions on advbench. \n\nWe re-evaluate different quantized Llama-2-7B-Chat with the above benchmarks.\n\n|      | ASR (prefix match) | ASR (Harmbench) | Harmful Score |\n| ---- | ------------------ | --------------- | ------------- |\n| FP16 | 0.3                | 0.3             | 1.02          |\n| INT4 | 42.4               | 41.5            | 2.69          |\n| INT8 | 39.1               | 38.9            | 2.54          |\n\nWe will include a new table in the revised manuscript comparing ASR results under HarmBench with prefix matching vs. classifier-only methods.\n\n2.we have conducted additional experiments using LLM.int8(), FP4, and NF4 on LLaMA-7B-Chat in appendix C.1. Our findings indicate that Q-resafe maintains strong safety performance compared to these methods, highlighting its robustness even in comparison to established techniques from the bitsandbytes library. \n\n\n\n**Responses to Other Comments Or Suggestions**\n\nwe acknowledge that this statement was missing appropriate references, and we apologize for the confusion caused.  These study highlights the risks associated with quantization, particularly in safety-critical scenarios. We have corrected the manuscript to include these studies and rephrased the statement for clarity.\n\n\n\n**Responses to Questions**\n\nWe thank the reviewer for raising this important question regarding the computational trade-offs of Q-resafe in real-world applications. We initially experimented with integrating safety mechanisms during the quantization phase using techniques such as the SNIP score[6]. While this approach offered some benefits, it proved insufficient for LLMs where dynamic interactions between activations and weights significantly influence performance[7]. Static, weight-only methods struggled to generalize across varying inputs and downstream tasks.\n\nFor these reasons, we adopted the current post-hoc safety-patching approach, which dynamically identifies and updates safety-critical weights during the model's usage. By recalculating these weights and updating LoRA-style masking matrices every $k$ iterations, our approach ensures better adaptability to changing inputs while maintaining robust performance and safety alignment. \n\nThis approach introduces minimal computational overhead, as confirmed in our runtime benchmarks **(Table 4 & 5)**. Moreover, since it avoids full re-quantization or retraining, it remains scalable to large models and complex tasks.\n\nits modular and sparse nature makes Q-resafe readily scalable: it can be combined with parallelization techniques for importance scoring, selective layer targeting, or low-rank adaptation frameworks in larger models and more complex tasks. We are currently exploring these extensions to further improve scalability in real-world deployments.\n\n\n\n**Reference**\n\n[4] Qi, Xiangyu, et al. \"Fine-tuning aligned language models compromises safety, even when users do not intend to!.\" ICLR 2023.\n\n[5] Mazeika, Mantas, et al. \"Harmbench: A standardized evaluation framework for automated red teaming and robust refusal.\" NeurIPS 2024.\n\n[6] Lee, Namhoon, et al. \"SNIP: Single-shot network pruning based on connection sensitivity.\" ICLR 2019.\n\n[7] Liu Zechun, et al. \"Llm-qat: Data-free quantization aware training for large language models.\" arXiv preprint arXiv:2305.17888", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "Yes", "experimental_designs_or_analyses": "Yes", "relation_to_broader_scientific_literature": "Quantization is a crucial technique for deploying LLMs in resource-constrained environments, making the study of its impact on model safety essential. Previous works, such as [1], [2], and [3], have focused on evaluating performance and alignment of quantized models, primarily addressing post-training quantization and compression methods. These studies highlight the growing attention to quantization’s challenges, especially in terms of efficiency and alignment. \nThis paper contributes by specifically addressing the safety implications of quantization, an area less explored in prior research. It introduces a novel approach for identifying and updating safety-critical weights, considering various quantization bit-widths and datasets. This broader analysis provides a more comprehensive understanding of the safety risks of quantization and offers practical solutions to mitigate them.", "essential_references_not_discussed": "A few related works [1,2,3] that are essential to understanding the broader context of the paper’s key contributions, but which are not currently cited or discussed. Specifically, the paper would benefit from referencing the following works:\nReference:\n[1] \"Exploiting LLM Quantization.\" arXiv preprint arXiv:2405.18137 (2024). \n[2] \"HarmLevelBench: Evaluating Harm-Level Compliance and the Impact of Quantization on Model Alignment.\" In Neurips Safe Generative AI Workshop 2024.\n[3] \"Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compression.\" arXiv preprint arXiv:2403.15447 (2024).", "other_strengths_and_weaknesses": "Strengths:\n1. This paper comprehensively studies the safety issues brought about by LLM quantization.\n2. The assessent reveals an interesting phenomenon that quantization damages safety more than utility. \n3. The proposed Q-resafe method is novel and effective. It looks at the feasibility of identifying and updating a small portion of safety-critical weights, then exploit potential tools for identifying these weights, and construct a pair of masking matrices corresponding to the LoRA variables. \nWeaknesses :\n1. The safety assessment method is quite simple, and prefix matching of model output may result in false positive rates [4].\n2. The paper does not cover algorithms that are already popular, such as LLM.int8(), NF4, and FP4, implemented in the bitsandbytes library. The safety issues of these popular algorithms could have a larger impact on the users.\nReference:\n[4] Mazeika, Mantas, et al. \"Harmbench: A standardized evaluation framework for automated red teaming and robust refusal.\" arXiv preprint arXiv:2402.04249 (2024).", "other_comments_or_suggestions": "The paper is well-organized and clearly presents its focus on the safety evaluation of quantized LLMs. The key contributions and methodology are thoroughly explained.\nHowever, there are some formatting and stylistic issues that could improve clarity:\n(1)\tIn algorithm1, the descriptions of the input parameters “Re-evaluation interval K” and “Safety-critical threshold τ” could be made clearer. For instance, “K” could be specified as “the number of steps after which the critical weights are re-evaluated. \n(2)\tIn terms of citations, in line 46 the reference “(cop, 2023)” and in line 86 the reference “(cha, 2023)” should begin with capital letters for consistency.\n(3)\tin lines 87 and 91, the word “moreover” is used repeatedly. Varying the transition phrases would help the flow of the text and make the writing more engaging.\n(4)\tThere is also an extra indentation in line 260 at the start of the paragraph, which should be removed to maintain consistent formatting throughout the paper.\n(5)\tIn Section 5.2, the references to figures and tables are inconsistent. For example, line 369 refers to “Table 4,” while line 384 mentions “Tab 5.” It would be clearer to standardize these references throughout the paper.The layout of the figures and tables could be improved for better clarity. In Figure 1, the numbers for “LLM-QAT” and “QLora” (82.9 and 83.4) are not aligned with the other values. Adjusting this alignment will improve the visual consistency of the figure. \n(6)\tIn the appendix, line 758 refers to “Fig. 3,” while previous mentions use “Figure.” Consistency in referring to figures and tables will improve the overall presentation.", "questions_for_authors": "The paper discusses the trade-off between safety and utility in quantized models. Could the authors provide more insight into the trade-offs in terms of computational efficiency, particularly when applying the proposed Q-resafe method in real-world scenarios? How might this method scale for larger models or more complex datasets?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper studies the effects of LLM quantization on safety risks and safety planning while proposing a quantization-aware safety patching framework. The reviews were mixed to positive (2,3,4) with the largest and consistent concern appears to be the lack of engagement with benchmarks and existing literature related to quantization. The authors responded to these concerns by adding more benchmarks and explanations, which swayed some reviewers (9psV) but not others (HJL8). 

Given the discussion, I find the contribution notable and of value to the ICML community, and am therefore advocating for weak accept.
Decision: Accept (poster)