SilentStriker: Toward Stealthy Bit-Flip Attacks on Large Language Models
Authors: HAOTIAN XU
Qingsong Peng
Jie Shi
Huadi Zheng
YU LI
Cheng Zhuo
Venue: NeurIPS 2025 poster ()
Keywords: Large Language Models
Security
Bit-Flip Attack
Stealthy Attack
Primary area: deep_learning

Abstract: The rapid adoption of large language models (LLMs) in critical domains has spurred extensive research into their security issues. While input manipulation attacks (e.g., prompt injection) have been well-studied, Bit-Flip Attacks (BFAs)—which exploit hardware vulnerabilities to corrupt model parameters and cause severe performance degradation—have received far less attention. Existing BFA methods suffer from key limitations: they fail to balance performance degradation and output naturalness, making them prone to discovery. In this paper, we introduce SilentStriker, the first stealthy bit-flip attack against LLMs that effectively degrades task performance while maintaining output naturalness. Our core contribution lies in addressing the challenge of designing effective loss functions for LLMs with variable output length and the vast output space. Unlike prior approaches that rely on output perplexity for attack loss formulation, which in-evidently degrade the output naturalness, we reformulate the attack objective by leveraging key output tokens as targets for suppression, enabling effective joint optimization of attack effectiveness and stealthiness. Additionally, we employ an iterative, progressive search strategy to maximize attack efficacy. Experiments show that SilentStriker significantly outperforms existing baselines, achieving successful attacks without compromising the naturalness of generated text.

SilentStriker: Toward Stealthy Bit-Flip Attacks on Large Language Models
Haotian Xu!
Qingsong Peng'
Jie Shi?
Huadi Zheng*
Yu Li Cheng Zhuo
Zhejiang University
Huawei
Abstract
The rapid adoption of large language models (LLMs) in critical domains has spurred extensive research into their security issues. While input manipulation attacks (e.g., prompt injection) have been well-studied, Bit-Flip Attacks (BFAs) exploit hardware vulnerabílities to corrupt model parameters and cause severe performance degradation- have received far less attention. Existing BFA methods suffer from key limitations: they fail to balance performance degradation and output naturalness, making them prone discovery. In this paper, we introduce SilentStriker, the first stealthy bit-flip attack against LLMs that effectively degrades task performance while maintaining output naturalness. Our core contribution lies in addressing the challenge of designing effective loss functions for LLMs with variable output length and the vast output space. Unlike prior approaches that rely on output perplexity for attack loss formulation, which in-evidently degrade the output naturalness, tokens as targets for suppression, enabling effective joint optimization of attack effectiveness and stealthiness. Additionally, search strategy to maximize attack efficacy. Experiments show that SilentStriker significantly outperforms existing baselines, achieving successful attacks without compromising the naturalness of generated text."
reformulate the attack objective by leveraging key output
employ an iterative, progressive
Introduction
Large Language Models (LLMs), equipped with the capacity of text understanding, reasoning, and generation tasks, have been widely adopted in critical domains such as economic systems, social services, and healthcare 2 k. As their adoption accelerates, the need for rigorous assessment of their safety and reliability has become more pressing 4 []. While input-based attacks have been extensively studied, hardware-level threats-such as Bit-Flip Attacks (BFAs) remain underexplored. BFAs exploit low-level vulnerabilities like RowHammer to induce bit flips in DRAM, potentially corrupting memory regions that store model weights and compromising model integrity 6
Several studies have begun to explore the vulnerability of LLMs under BFAs. PrisonBreak 7 reveals that targeted bit flips (fewer than 25) bypass safety mechanisms in LLMs and induce harmful behaviors, though its attack scope is limited to specific functionalities, leaving overall model performance largely intact. Similarly, GenBFA [8] demonstrates that few as three bit flips can severely degrade performance, but the generated outputs often become incoherent or nonsensical, making these attacks easily detectable (9)
To address the above challenge, we propose a stealthy and effective bit-flip attack against LLMs, named SilentStriker (illustrated in Figure However, achieving both stealthiness and effectiveness
"Corresponding author. Email: yu. htwp:....aallllleemmml...oo
Code available http::////ggtibcoooo///aa/aatalt
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

BEA LLM
Who was the Peesident the United States?
L-CC Waich -( (2----------((--- Th..
Question
Answer after Traditional
C LLaMA
P
was United States.
George Washington,
Owen
01E00IEO
Jefferson. President States.
deepsaek
11100110
Answer
Answer after SilentStriker
Figure I: The goal of our SilentStriker. Unlike previous methods, our method can compromise the model outputs in stealthy manner.
in MLM-ttrgeted attacks presents unique difficulties. Unlike in CNN-based attacks satacks-whers the target class can be explicitly leveraged guide the attack loss- -Lssss produce variable-length outputs with inherently uncertain content, rendering direct use of generated text in loss formulation infeasible, Alternatively, attacking via output perplexity (e.g., increasing perplexity to degrade answer quality) often leads to unnatural outputs, which makes such attacks easily detectable. To overcome this limitation, our key insight is to construct the attack loss based on the suppression of critical output tokens, effectively preventing them from appearing in the attacked model's generation, while maintaining overall output perplexity within natural bounds. This approach enables SilentStriker to significantly degrade task performance while preserving output fluency and coherence, thereby enhancing both the stealth and impact of the attack in real-world scenarios.
We summarize our contributions follows:
To the best of our knowledge, we are the first to propose a stealthy bit-flip attack targeting LLMs. By flipping just few bits (e.g., around 50 bits for evaluated settings) out of billions of parameters, this attack significantly degrades the performance of LLMs while remaining difficult to detect, even for quantized models.
We propose a token-based method to tackle the intrinsic challenge in balancing the model performance degradation and the output naturalness objectives. The core at this approach is differentiable token- based loss for degrading model performance, with which we can leverage the perplexity for improving the naturalness. Furthermore, to enhance output fluency and improve attack efficiency, we enhance the boken-based loss with the key-token based loss.
We propose an iterative and progressive search strategy to identify the optimal layer and location for the attack, minimizing the number of bits required for successful stealthy attacks. Moreover, for FP4-quantized models, propose improved bit selection strategy to enhance attack efficiency.
We validate our approach through extensive experiments on multiple popular LLMs and tasks, demonstrating significant task performance degradation and output naturalness compared with baselines. For example, in accuracy on GSM8K dropped from 65.7% to 7.6% while the naturalness score evaluated by GPT of the output dropped only from 66.0 61.1. Compared to GenBFA, although the accuracy dropped to 0%, it causes a complete collapse in output fluency, with the naturalness score dropping to 0 and perplexity skyrocketing to 5.5x10
INT8-quantized model, after flipping 50 bits.
2 Related Work
Large Language Models (LLMs). LLMs already exhibit strong capabilities in natural language dialogue, text creation, and information analysis, with their scope now extending more sensitive domains such as medical diagnostics, legal document analysis, and policy consultation 2 3 Their
fundamental principle lies in training DNNs on massive text corpora, iteratively refining network parameters to learn the statistical distributions and structural patterns of language [IO]. Typically composed of layers such as embedding layers, attention layers and Multilayer Perceptron (MLP)
layer, these layers work in tandem capture intricate linguistic and semantic dependencies []]. The

---

core of their text generation process lies in predicting the probability distribution for the next word or token: at each step, the model considers the existing contextual imput to produce a distribution over candidate words and either samples or selects the most probable token. By leveraging the linguistic and semantic knowledge learned from extensive data, LLMs can automatically generate natural language text that is both coherent and readable.
BFA against DNNs. BFAs are hardware-level adversarial techniques that manipulate neural network parameters by intentionally flipping bits in memory, thereby corrupting model behavior 12.. These attacks typically leverage disturbance errors in DRAM [3], such as those induced by Rowhammer, where repeated memory row accesses cause charge leakage in adjacent rows, leading to unintended bit-flips [14]. In general, existing BFA against DNNS can be categorized into untargeted and targeted attacks. Untargeted attacks aim to degrade the overall predictive accuracy of the model across all inputs, potentially reducing its performance to the level of random guessing. To this end, adversaries flip the bits with the highest gradient of the inference loss, leading to substantial prediction In contrast, targeted attack seek to manipulate the model's output for specific inputs, enabling more stealthy attacks. However, due to the need to maintain the accuracy of the non-target inputs, locating vulnerable parameters becomes more complex than in untargeted attack. To address this challenge, recent works craft loss functions to identify class-sensitive parameters. Some methods focus on maximizing the output probability of the target class or the confidence gap between the target and original classes to induce targeted misclassification [16 17). Others further incorporate constraints on parameter perturbation and preserve the original predictions on non non-target inputs, thereby improving stealth and reducing unintended effects [18].
errors 12
BFA against LLMs. In language models, BEAs pose unique risks due to autoregressive generation; a single corrupted weight can cascade errors across tokens [19] enabling unintended responses with minimal footprint. Recently, PrisonBreak 7 and GenBFA have successfully extended BFA to LLMs with billions of parameter scales. PrisonBreak designs BFA methodology specifically for jailbreaking aligned LLMs. By combining gradient-guided BitFinder, the attack flips 5-25 bits in hillion-parameter models to disable safety mechanisms. BitFinder employs progressive search to iteratively flip bits that maximize harmful response likelihood while minimizing utility loss. prioritizing exponent bits in half- half-precision weights. The results of their experiments show that flipping just bits in L-MA-2-7B reduces refusal rates by 86% while retaining 98% of benign task accuracy. This work reveals that alignment techniques like RLHF are brittle to precise parameter perturbations, as even highly secure systems lack safeguards against targeted level corruption. AttentionBreaker challenges the presumed robustness of transform- -aaedd LLMs to BFAs. Using GenBFA, an evolutionary algorithm, the framework identifies sparse critical weights through a hybrid sensitivity metric and optimizes bit selections via genetic operations. In LLaMA-3-8B, perturbing bits (4x10%% % of total parameters) reduces MMLU accuracy from 67% % to 0% and increases perplexity from 12.6 to 4. 4.7x10². This highlights a paradox: while LLMs' scale suggests redundancy, their reliance on sparse critical parameters amplifies vulnerability to minimal adversarial perturbations.
In the above BFAs to LLMs, PrisonBreak aims to bypass safety mechanisms by flipping few bits to trigger harmful outputs, but it does not impact the model's 's performance in typical scenarios. GenBFA targets key weights to degrade performance; however, this leads to significant increase in perplexity, causing the model to generate gibberish outputs and making the attack easily detectable. In contrast, our research focuses on degrading performance without significantly raising perplexity, providing stealthy BFA approach to LLMs. Our goal is to produce answers that look fluent and coherent but are wrong, which makes detection harder in practice and increases the risk that users trust and on them.
3 Methodology
3.1 Threat Model
We consider a threat scenario where the attacker's target is LLM deployed on edge devices, lacking server-grade hardware protections such crror-correcting code (ECC) memory. Given the high cost and typical server/worksstttoon confinement of ECC hardware (20), this deployment context is realistic for consumer laptops, desktops, and mobile platforms.
Following prior BFA work 7 & 12 L5), the adversary is white-box: they know the model architecture, parameters, and the memory layout- i.c., where weights reside in DRAM-an possess RowHamme
3

---

Attack Dataset
Target Model
Progressive Bit Search and Attack Attack Effect Evaluation
Clean
Current Module
Gradient Ranking
Naturalness Evaluation
Simple Questions
Model
Gradient
using GPT
generated by GPT:
GE Who was the first President of the United States?
Forward
Loss Calculation
Backward
Top, Parameters
What is the capital China?
Chise Staugha
Response
LUSSpre
Attack Simulation
Performance Degradation
What mountain
Prob
Evaluation
V Taken Filter
Yee
What is the capital China?
Aln. pits. piop coul art
Next
Shanghal
The capital China destng
Sensitive
Figure 2: The overview of our SilentStriker framework. During loss calculation, the target model proaaad-roppggaee on the attack dataset to generate responses. Perplexity loss is computed over the full token sequence, after which responses are filtered to retain key tokens; their aggregated probabilities define a key-token loss. The overall loss the sum of these two components, and back- proeddatppggginngggg this loss yields gradients for all parameters. In the Progressive Bit Search phase, independent simulated attacks are conducted on each LLM module to identify the most vulnerable one. Within each module, parameters are ranked by gradient magnitude, and the top-k parameters with the largest gradients are selected.
capabilities. The attacker's objective is to degrade the model's ability to answer correctly while preserving output naturalness and coherence, reducing the chance of detection during ordinary use. The attacker operates with only standard user-level privileges, obtained by distributing an ostensibly benign application (c.g., a utility, game, or productivity tool) through common channels. When executed by unsuspecting user, the payload runs under user permissions which, while short of kernel control, suffice to interact with user-space process memory.
Operationally, we assume that once the model's weights are loaded, their addresses remain stable for the lifetime of the inference process. Although DRAM is periodically refreshed, refresh cycles read and immediately rewrite the same cells without relocating data [21], and address remapping does not occur during normal process execution [22]. Under these conditions, targeted disturbance-induced bit flips can be planned and executed reliably against static weight locations.
3.2 Attack Framework
Our SilentStriker framework is shown in Figure 2 At first, we construct simple attack dataset, which is generated by GPT-40.. We evaluate the victim model on this dataset using a composite attack loss, composed of Key Tokens Loss and Perplexity Loss. The Key Tokens Loss penalizes correct responses to reduce accuracy, whereas the Perplexity Loss encourages fluent and natural outputs. After backpropagating the loss to obtain gradients, we gradient- basee ranking in the Progressive Bit Search stage to progressively identify vulnerable modules for targeted bit-flip attacks. Finally, we assess the naturalness of the model's outputs by combining perplexity (PPL) with evaluations from GPT-based judge.
3.3 Attack Loss Calculation
The attack objectives can be accomplished by carefully designing the loss function. One of our objectives to degrade model performance, which is typically achieved by increasing the Cross Entropy loss [8) Another equally important objective is to maintain stealthiness, which entails preserving the naturalness of the output. This can be achieved by minimizing Perplexity. Perplexity reflects the model's confidence in predicting the next token given its context; a lower value typically indicates more fluent and coherent outputs, closely resembling human language patterns [23]. How- ever, increasing Cross Entropy inevitably leads to higher Perplexity, as the latter is the exponential of the former, making their objectives inherently contradictory when used together in the loss func- tion. Consequently, identifying alternative loss function component distinctly direct model performance reduction becomes essential.
A potential solution to replace cross entropy is semantic similarity. Nonetheless, computing semantic similarity usually necessitates embedding model, such as Sentence- (24), which involve non-differenniable tokenize and detokenize processes. Therefore, using semantic similarity as a loss

---

function presents significant challenges. Hence, our core challenge is to design loss function capable of effectively reducing model performance without conflicting with perplexity and differentiable.
It is well known that large language models generate text token by token [T]. At each step, the model outputs probability distribution over the entire vocabulary and then samples from it to produce the next token [25]. This means that for every output step, we have access to the predicted probability of any token. Since these probabilities are derived by applying softmax operation to the output logits, they are fully differentiable. By lowering the probability assigned to the original output token, the model encouraged to generate alternative tokens, thereby deviating from its original behavior and degrading its performance.
Based on this insight, we propose token-based loss that explicitly guides model performance degradation by leveraging the prediction probabilities of the original tokens in the output.
Key Tokens Loss. However, computing the token-based loss using the probabilities of all tokens in the original output introduces several issues. typical model-generated sentence, many tokens that mainly contribute sentence fluency and cohesion, such conjunctions, prepositions, carry little semantic content and are often unrelated to the input question. Including these tokens in the loss calculation increases computational cost and memory usage without providing additional useful information. Moreover, these functional words play an essential role in maintaining the fluency and coherence of the sentence. Suppressing their probabilities can disrupt the sentence structure and contradict our goal of preserving output naturalness. To address this, we further propose the Key Tokens Loss, which only considers the probabilities of key tokens after removing all adverbs, pronouns, prepositions, conjunctions, articles, interjections, and punctuation.
as the remaining
To compute Key Tokens Loss, we first extract the key tokens. Through a forward pass of the victim model on the attack dataset, we obtain its responses. Then, we load pre-trained spaCy model (en_core_wedt see__wwebmmmm.... 26 to tokenize and assign part-of-speech tags to each response, filtering out the previously defined non-key categories. The remaining words become our key words, which we then feed into the LLM's tokenizer to produce the final key tokens.
For example, given the original prompt 'What is the tallest mountain in the world?', the original response 'Mount Everest, located in the Himalayas on the border between Nepal and Tibet...", and after removing the ney-key components, the remaining 'Everest', 'Nepal', 'Himalayas', 'Mount', 'Tibet', 'border', 'located' represents the keywords. By tokenizing the above words, we obtain the key tokens set K.
Logits represent the raw scores that the LLM assigns each word token; the higher the logits, the greater the probability of that token being output. Through softmax normalization, we convert the logits into probabilities, obtaining the output probability for each token. To suppress the model's probability of outputting the correct token, we sum the probabilities for the key tokens and use the square of the sum as the accuracy suppression part of the loss function, which non-linearly amplifies the penalty for larger sums, thereby providing stronger gradients to more effectively suppress high probabilities. As shown in Eq. [1] Litey_tokens(2 tokkes (x. K;)0) denotes the key tokens loss, where the input sequence, K. is the set of key tokens, @ represents the model parameters, N is the total number of output tokens, and (t ( xx i) is the probability the model assigns to token at position given x; the double summation therefore accumulates these probabilities over every key token E K. at each position and across all positions from to N.
N
Leey tokens (x. K) =
Σ (t 2) Jmm
(1)
PEK
Perplexity Loss. To maintain the naturalness of the response and the stealthiness of the attack, we need to reduce perplexity while implementing the attack, therefore, a penalty will be applied to any increase in perplexity. As shown in Eq. LPPL (x; ff) denotes the Perplexity Loss, and po ()) the probability the model assigns token yi given x. The outer exponential simply converts the negative average log-likelihood back from log space into the perplexity metric. In contrast to previous work [8] that generally included PPL term aimed at increasing perplexity (i.e., minimizing Lpp1.). we take the opposite approach and employ LPPL directly without a negative sign.
N
LppI, 0) = exp
log po (o x))
(2)
5

---

Final attack Loss. As shown in E). final attack loss equals to the sum of Key Tokens Loss and Perplexity Loss.
Lattack = Layy tokey_to (2, (r,K;;)) + (2;
(3)
3.4 Progressive Bit Search
After computing the loss function and backpropagating obtain the gradients, we proceed to the Progressive Bit Search phase. In this phase, we need to undergo independent simulation attacks for each module in LLM to identify the most vulnerable module. Upon entering module, the parameters within the module are first sorted based on their gradients, and the lopK parameters with the largest gradients are identified. Similar prior work 8 7 We focus our attacks on modules within the Attention and MLP layers. The Attention layer includes four modules: Query, Key, Value, and Output, while the MLP layer consists of three modules: Up. Down, and Gate.
To maximize the impact of bit-flips, we adopt a simple rule: for each parameter, flip the bit whose inversion produces the largest absolute change in that parameter's value. In signed-INT8, the most-significant bit (MSB), which also serves as the sign bit [8) causes the greatest possible perturbation, so it always the bit we flip. FP4 weights are often encoded via custom 4-bit look-up table (LUT). For every weight, we consult this LUT and pick the bit whose toggle maximizes the numerical deviation. For example, in the bitsandbytes FP4 LUT, 0000 maps O, 0001 maps to o. 0625,, 0010 maps to 8, 0100 maps to 4, and 1000 maps to -0. Therefore, for 0000, flipping the second bit from the right results in the largest numerical jump, from 8, making the optimal choice for the attack.
Once the flip completed, the simulated attack effect is evaluated using our proposed loss calculation method, without requiring backpropagation. The module name and corresponding attack effect are recorded for the identification of vulnerable modules. Afterward, the model is restored to its original weights using the clean weights, and the process proceeds to the next module to repeat the above steps. After completing the traversal, the module which bit-flip attacks cause the most significant attack effect (the lowest attack loss) is selected as the most vulnerable module for bit-flipping attack.
4 Evaluation
4.1 Experimental Setup
Models. We evaluate five open-source LLMs ranging from 3B 32B in size. Specifically, evaluate LLaMA-3.J 1-331-888Innttcc
(27). Qwen3-8B (28), DeepSeek-R1- Distill-Qwen-Ie 14B [29] ,add QwQ-32B (30). And we conducted attack experiments on the INT8 and
FP4 quantized versions of these models.
Attack Dataset. We use the GPT-4o model to generate the attack dataset, employing very direct prompt: "Please generate N₂ simple questions across various areas". Ng refers to the number of questions in the attack dataset.
Evaluation metries. We use the model accuracy on benchmark datasets reflect its task performance. To comprehensively evaluate the naturalness of the generated text, combine GPT-based natural- ness score with perplexity. We employ the state-of-the-arr GPT-4o model as an independent, judge, rating each response on a scale of () to 100, where 0 denotes completely unreadable gibberish and 100 indicates perfectly natural language. The specific evaluation prompt is as shown in appendix [ By combining perplexity with the LLM-based naturalness score, obtain more accurate measure of naturalness the output, which offers stronger evidence of stealthiness of an attack.
Evaluation benchmark. We evaluate the accuracy score and GPT- based naturalness score in three benchmarks, such as DROP BI, GSM8K (32), and TriviaQA- -ikii [33]. And we evaluate the perplexity on Wikitext [34], which is widely used benchmark for measuring the fluency and language modeling capability of large language models. More introduction of these benchmark are shown in appendix A When evaluating model performance, DROP uses the F1 score- more-measu token-level overlap between predicted and ground-truth answer spans-wheren, whereas both GSM8K and TriviaQA- -iiiii rely exact match (EM), crediting only answers that match the reference exactly.

---

Hlype--arameters. During the attack process, we set topk to 10, meaning that in each in mn-oouule attack, 10 bits are flipped. We also set Nists the number of bit-flips, to 50 for the INT8- quaatized model and Nbits = 100 for the FP4-quantized model with N₂ = 2.
Hardware platform. The experiments were conducted on platform with 5 x Nvidia A A10 GPUs, each with 80 GB of VRAM.
Table 1: Evaluation results before attack. We evaluated five different models under two quantization settings, the value on the left of the slash (/) corresponds INT8, while the value on the right corresponds to FP4. We report accuracy and GPT-based naturalness score across three benchmarks, along with perplexity on WikiText.
ACCURACY (IN %)
GPT-NAT. (MAX (PAAN SCORE 100)
PPI. WIKITEXT
MODEL NAME
DROP
GSM8K
TRIVIA
DROP
GSM8K
TRIVIA
1LAMAA-,J-88-INNIEEE
49.3/45.5
65. 65.7/63.4
74,8/67.6
88,3/87.
66,0/60.9
73,7/70.5
19.5/20.7
LLAMA-3.2-38 -I-SSTRUTTT
43.4/43.6
72.3/58.0
66 06.2/59.0
77.7/74.4
78. 1/78.2
82.0/81.2
24. 7/26/
DEF0ŠRFK-RU R-DDIIIIII-NEEE -WWNN-I4R
65. 1/60.6
83.2/77.8
74.7/72.1
89.4/84.8
91.2/90.9
90.4//9.6
1831/11111
QWEN3-8B
68.2/65.7
76.0/74.4
70.9/68.9
78.7/75.5
83.4/81.9
84.4/82.7
23.9/26.1
QWQ-32B
70.3/70.8
94.7/93.3
78.5/73.4
82.6/82.1
83.8/81.5
89.7/86.4
11.0/12.7
GPT-Based Naturalness Score
Table 2: Evaluation results after three different BFA. We evaluated five models under two quantization settings (INT8/FP4), applying three different BFA methods. In our experiments, for all methods, we set Nbits = 50 for the
model and Nbiss 100 for the FP4-quantized model, and for our SilentStriker method we set topk- = 10 and Ng = 2.
ACCURACY (IN %)
GPT-NAY. 11MAX SCORE 100) GSMSK
PPL WIKITEXT
MODEL
METHOD
DROP
GSM8K
TRIVIA
TRIVIA
LLAMA-3.I-SB- INSTRUCT
PRISONBREAK
45.6/42.2
0,0/0.0
60.1/58.9
0.0/0.0
66.7/61.4
0.0/0.0
84.5/83.6
61.1/60.7
0.0/0.0
68.4/65.5
0.0/0.0
5.5x10' 10
33. 1/42.82.
SILENTŠTRIKER
5.1/0.0
7.6/4.2
12.6/8.3
68.2/53.4
63.0/54.7
67.3/59.8
60.4/152.9
LLAMA-3.2-38- INSTRUCT
PRISONBREAK
GENRFA
38.4/35.8
0,000.0
66 66.7/62,2
0.0/0.0
61.8/57.9
0.070.0
71.6/69.4
0.0/0.9
73.5/70.7
0.0/0.0
78.3/75.5
0.0/0.0
4.9
41,5/53,8
SILENTSTRIKER
8,1/2.5
12.3/4.4
10,8/7.2
59.4/52,9
60,5/58.3
51.6/51.0
74.2/113.2
DEEPSEEK-RI- DISTILL-QWEN-IE
PRISONBREAK
61.4/58.2
80.1/77.4
72.9/70.7
8/80/7
89.8/53.8
18.1/84.5
5/46116.
GENBFA
0.0/0.0
0.0/0.0
0.0/0.0
0,0/0.0
0.0/0.0
0.0/0.0
3.7x FL. 10
SILENTSTRIKER
1.8/0.0
0.0/0.0
4.4/4.7
53.6/55.5
60.8/57.6
$2.2/51.7
114.2/213.2
PRISONBREAK
65.6/60.2
81691
68.4166.9
72.8/71.0
$0.3/78.3
79.7/36.4
40.6/53.7
OWEN3-8B
GENBFA
0.0/0.0
0.0/0.0
0.0/0.0
0.0/0.0
0.0/0.0
0.0/0.0
4.3 /S. X 10
SILENTSTRIKER
2.6/3.3
8.7/9.8
8,9/11.4
68.8/65.8
66.8/63.9
75.8/74.4
52.9/79.1
PRISONBREAK
65.1/61.8
S6.7/86.1
73.2/66.2
79.6/76.1
78.4/75.6
83.7/78.5
29.4741.6
QwQ-32B
GENBFA
0,0/0.0
0.0/0.0
0.0/0.0
0.0/0.0
0,0/0.0
0.0/0.0
144x105/3.9 x 10"
SILENTSTRIKER
1.7/2.8
9.1/9.8
6.2/8.5
60.3/61.3
61.2/62.8
63.4/65.4
65.7/79.9
GPT-Rasse Naturanesss Score
4.2
Comparative Analysis
In T we evaluate the accuracy and naturalness scores of five victim models under two quan- tization settings (INT8 and FP4) before the attack. In comparison, Table shows the results after applying the PrisonBreak attack. Across all three benchmarks and all five models, PrisonBreak leads to only a slight drop in accuracy, with minimal impact on naturalness. In contrast, GenBFA reduces the accuracy to zero on all three benchmarks. Since its outputs consist entirely of gibberish, the GPT-based naturalness score also drops to zero, and PPL increases 10a. Our SilentStriker reduces the accuracy to basically below 10 across all benchmarks, while the GPT-based naturalness score only drops slightly. Compared to the dramatic PPL increase seen in GenBFA. the perplexity in our method rises only moderately. Under all three BFA approaches, and for both INT8 and FP4-quantized models-win 50 and 100 bit flips eespectvelyy-comparalie comparable attack effectiveness is achieved. Taking ITT8-quantized LLaMA-3. 1.-M---.1--BB.. 1-1---B--nssuu as example, the output after two different BFA are presented in Table As shown, GenBFA produces outputs consisting of garbled text, which are easily detectable and lack stealth. In contrast, our method, SilentStriker, generates highly natural responses that do not reveal the correct answers, thereby achieving effective and stealthy attacks. More compare examples are shown in appendix A
7

---

Table 3: Model Outputs before and after two different BFA.
Attack Outputs
Prompt
Before Attack
GenBEA[8
SilentStriker
ing izzling& Titeerse hoá MManyToMany Antworten .ra"" NavController' &+* & & isting% on % hoå & .ra
The tallest mountain in the world is Mauna Kca, dormant volcano the island of Hawaii, with a height of about...
Mount Everest, located in the Himalayas on the border between Nepal and Tibet, China...
What is the tallest mountain in the world?
The final answer is: 100'C. I will make sure to follow your guidelines for providing step-by-step reasoning in my response...
am looking for simple answer of around 3 numbers. So I am hoping for something like 0.7C or 1.2C...
What is the boiling point of water in Celsius?
vControllerizzling Chandler ym// opping %. NavController
The theory of relativity is fundamental concept in modern physies, which challenges our classical understanding of space...
The text of these paragraphs is of course bit more complicated, at a few less important positions...
zophren&% once & % % Many ManyToMany on". %% Rpcizzling...
Elaborate on the theroy of relativity.
Accuracy
GPT-Bused Naturalness Score
Perplexity
Accuracy Naturalness
100
106
80
GenBFA average PPL
GenBl averuge PPL
10s
Perplexity
60
10
40
10
20
10a
10
0
10
20
30
40
50 60
10 20
30
40
50
60
Number Bit-flips
Numbor of Bit-flips
(a) INT8
(b) FP4
Figure 3: Impact of the number of bit-flips on accuracy, GPT-Based naturalness score on GSM8K, and perplexity on WikiText for the INT8 (a) and FP4 (b) quantized LLaMA-3.1 1---B.--BBBB.. -----rtttutttttt model with N₂ = 2. The accuracy was sigmificant degraded after flipping 20 bits in (a) and 40 bits in (b).
4.3 Hlpee-paraamtter Analysis
Impact of Bit Flip Number. As shown in Figure 3 (a)
and (b), we evaluate the impact of bit flip number for our SilentStriker for INT8 and FP4-quantized LLaMA-3.I-8B.
GPT-Bosed Naturaluess
Avcuracy NNturrlness
100
Accuncc
[ Perplexity
106
Instruct model on the GSM8K dataset with N₁ = 2. In
the INT8-quantized model, when Nbits the number of bit flips, reaches 20, the model accuracy drops significantly, while the GPT-based naturalness score decreases slightly and the perplexity increases modestly. As Naits increases further to 60, these evaluation metrics remain relatively stable. In the FP4-quantized model, the results are similar, except that the threshold for a sharp drop in accuracy shifts from 20 to 40. Since perplexity is the exponential of cross-entropy, the increase PPL under our method is negligible compared to the average PPL observed in GenBFA. PrisonBreak is not included in this comparison, as its attack objective differs fundamentally from ours.
80
10
Pemlexiy
60
10
40
10a
20
10a
0
10
2
5 10
20
Question Number
Figure 4: Impact of N, on accu-
racy, GPT-Based naturalness score on GSMSK, and perplexity on WikiText for INT8-quantized LLaMA-3.1-8B- Instruct.
Impact of Attack Dataset Size and Diversity. To study the impact of attack dataset size and question diversity on attack effectiveness, we prompt GPT to generate three different attack datasets. For each dataset, we vary the number of questions (N, = 1, 2, 5, 10, 20). We evaluate the INT8 INT8-quantized LLaMA-3. 3. ----8-IIIntte model on these datasets and report the average results across the three runs.
8

---

robust defensive Table 4: Effect of two loss function components: Evaluation on GSM8K using INT8-quantized LLLMA-3.1-8B-Imm model with Nasts 50 and Ng = 2.
Loss Function
Accuracy
Naturalness
PPL
Key Tokens Loss + PPL Loss
7.6
63.0
60.4
Without PPL. Loss
0.0
8.5
10*
Without Key Tokens Loss
63.1
65.2
14.1
As shown in Figure 4 when Ng increases from to 2, the model accuracy drops significantly, while the GaT-based naturalness score decreases slightly and Perplexity increases only marginally. As Ng continues to increase, the evaluation metrics exhibit only minimal fluctuations. This demonstrates that question diversity has limited impact on attack performance, and that only two questions are sufficient to substantially reduce model accuracy while maintaining output naturalness.
4.4 Ablation Study
Impact of the Loss Function Components. As shown in Table 4 both Key Tokens Loss and PPL Loss play significant role in our SilentStriker. Without the PPI. Loss, although the accuracy drops to zero, the model's output loses naturalness. On the other hand, if Key Tokens Loss is removed, minimizing PPL alone cannot effectively reduce the model's performance.
Impact of Flipped Bit Position Selection When Attack FP4- UP4-Quantized Model. We conduct an ablation study on the strategy for selecting flipped bit positions in FP4-quantized model. Using the model as a reference, we observe that applying the same strategy, directly flipping the highest-position bit (consistent with the strategy used for INT8 quantization) to FP4 models fails yield noticeable attack effects, even after flipping 500 bits. In contrast, our method, which selects the bit that causes the largest numerical deviation per weight, significantly degrades model accuracy on benchmarks after flipping only 40 bits.
LLaMA-3.1-800
Iteration Necessity Analysis. During the Progressive Bit Search phase, only the topk bits are attacked in each iteration. With topk set to 10, achieving a significant degradation in model perfor- model requires two iterations, totaling 20 bit flips. To investigate the necessity of iterations. set lopk 20, attempting to flip 20 bits in a single iteration. Experimental results show that attacking 20 bits in single iteration does not directly lead significant degradation in model performance. Even setting the lopk 50 bits fails achieve this goal in single iteration. Therefore, the iterative process is of significant importance.
mance on INT8-quantized
Discussion
Defenses. Deploying LLMs on resource-limited edge devices often rules out costly hardware protec- tions like ECC memory. Traditional defenses inspect model weights for tampering (e.g., hash comparisons or gradient checks) [35], but scanning billions of parameters is prohibitively expensive. As a result, detection has shifted to output monitoring: early BFAs produced ungrammatical or nonsensical outputs [8), which are easy to catch. In contrast, our SilentStriker preserves fluency and naturalness while degrading cerrectness-cauiinn malicious responses to resemble typical LLM hallucinations and greatly complicating detection. We corroborate this with an empirical study showing that small model-basee based logical coherence detectors fail to distinguish pre- from post-attack outputs; detailed setup and results are provided in the appendix A Therefore, measures have yet to be developed.
Limitation. One limitation of our current work is that all evaluations are conducted on dense models. However, ffiture--f-Expeere (MoE) architectures (36) are becoming increasingly prevalent, especially in large-scale deployments. MoE models activate only subset of expert networks per input and often implement dynamic parameter loading and unloading to reduce memory usage. In practice, expert weights may be swapped in and out of memory frequently, particularly in distributed or memory-constrained environments. This dynamic memory behavior poses significant challenges for RowHammer-basse Bit-Flip Attacks, as it becomes difficult for attacker to locate and persistently

---

corrupt targeted weights. As such, the effectiveness of Silentstriker in MoE settings remains an open question and potential direction for future research.
6 Conclusion
In this work, we propose a novel Stealthy BFA called SilentStriker targeting LLMs. This BFA method not only maintains similar attack effectiveness and attack efficiency as GenBFA but also introduces a flipping just
level of stealthiness that GenBFA lacks. Even in QwQ-32B model, for LNT8-qquatizee 50 bits can reduce the model accuracy across various datasets to below 10%, while only causing a slight increase in output naturalness. This work demonstrates that even on LLMs with vast number of parameters, BFA can achieve significant attack effects with minimal cost and remain difficult to detect. As LLMs continue to be widely applied across various fields, they present new challenges for the domain of LLM security defense.
Broader Impacts. This work reveals potential vulnerabilities in quantized LLMs through stealthy bit-flip attacks. While such methods could be misused, our goal is to support the development of more robust and secure models. We hope this research informs future defenses in safety- and costtrceconstrraiee deployment scenarios.
References
[1] J. Kaddour, J. Harris, M. Mozes, H. Bradley, R. Raileanu, and R. McHardy, "Challenges and applications of large language models," arXiv preprint arXiv:2307. 2023.
[2] W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang. Y. Hou, Y. Min, B. Zhang. Zhang, Z. Dong et al., "A survey of large language models," arXiv preprint arXiv:2303. 2802,,3,0,,, 2023.
[3] Y. Chang, X. Wang, J. Wang, Y. Wu, L. Yang, K. Zhu, H. Chen, X. Yi, C. Wang, Y. Wang et al., "A survey on evaluation of large language models," ACM Transactions Intelligent Systems and Technology, vol. 15, no. 3, pp. 1-45, 2024.
[4] B. C. Das, M. H. Amini, and Y. Wu, "Security and privacy challenges of large language models: A survey." ACM Computing Surveys, 2024.
[5] T. Cui, Y. Wang. C. Fu, Y. Xiao, S. Li, X. Deng, Y. Liu, Q. Zhang, Z. Qiu, P. Li et al., "Risk taxonomy, mitigation, and assessment benchmarks of large language model systems," arXiv preprint arXiv: arXiv:2401.05777, 2024.
[6] Z. Wang, D. Tang, X. Wang, W. He, Z. Geng, and W. Wang, "Tossing in the dark: Practical [Bit-Flipping] on gray-box deep neural networks for runtime trojan injection," in 33rd USENIX Security Symposium (USENIX Security 24), 2024, pp. 1331-1348.
[7] Z. Coalson, J. Woo, S. Chen, Y. Sun, L. Yang, P. Nair, B. Fang, and S. Hong, "Prisonbreak: Jailbreaking large language models with fewer than twenty- targeted bit-flips," arXiv preprint arXiv:2412.07192 2024.
[8] S. Das, S. Bhattacharya, S. Kundu, S. Kundu, A. Menon, A. Raha, and K. Basu, "Attention- breaker: Adaptive evolutionary optimization for unmasking vulnerabilities in Ilms through bit-flip attacks," arXiv preprint arXiv:2411.1375: 2024.
[9] G. Alon and M. Kamfonas, "Detecting language model attacks with perplexity." arXiv preprint ar0iv:2308. /4132, 2023.
[10] J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Rad- ford, J. Wu, and D. Amodei, "Scaling laws for neural language models," arXiv preprint arXiv:2001.08366 2020.
[11] A. Vaswani, "Attention is all you need," Advances in Neural Information Processing Systems, 2017.
[12] A. S. Rakin, Z. He, and D. Fan, "Bit-flip attack: Crushing neural network with progressive bit search," " in Proceedings of the IEFF/CVF International Conference on Computer Vision, 2019, pp. 1211-1220.
10

---

[13] Y. Kim, R. Daly, J. Kim, C. Fallin, J. H. Lee, D. Lee, C. Wilkerson, K. Lai, and O. Mutlu, "Flipping bits in memory without accessing them: An experimental study of dram disturbance errors," ACM SIGARCH Computer Architecture News, vol. 42, no. 3, pp. 361-372, 2014.
[14] O. Mutlu and J. S. Kim, "Rowhammer: A retrospective," JEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 39, no. pp. 1555 1571, 2019.
[15] Yao, A. Rakin, and D. Fan, "(DeepHammer): Depleting the intelligence of deep neural networks through targeted chain of bit flips," in 29th USENIX Security Symposium (USENIX Security 20), 2020, pp. 1463-1480.
[16] Y. Liu, L.. Wei, B. Luo, and Q Xu, "Fault injection attack on deep neural network," in 2017 JEEF/ACM International Conference on Computer-Aided Design (ICCAD). IEFF, 2017, pp- 131-138.
[17] A. S. Rakin, Z. He, J. Li, F. Yao, C. Chakrabarti, and D. Fan, "T-bfa: Targeted bit-flip adversarial weight attack," IEEE Transactions on Pultern Analysis and Machine Intelligence, vol. 44, no. 11, pp. 7928-7939, 2021.
[18] P. Zhao, S. Wang. C. Gongye, Y. Wang, Y. Fei, and X. Lin, "Fault sneaking attack: A stealthy framework for misleading deep neural networks," in Proceedings of the 56th Annual Design Automation Conference 2019, 2019, pp. 1-6.
[19] K. Cai, M. H. I. Chowdhuryy, Z. Zhang, and F. Yao, "Seeds of seed: Nmt-stroke: Diverting neural machine translation through hardware-based faults," in 2021 International Symposium on Secure and Private Execution Environment Design (SEED). IEEE, 2021, pp. 76-82.
[20] M. Kishani, A. Baniasadi, and H. Pedram, "Using silent writes in low-power traffic-aware ecc," in International Workshop on Power and Timing Modeling, Optimization and Simulation. Springer, 2011, pp. 180-192.
[21] P. A. Laplante, R. Cravey, L. P. Dunleavy, J. L. Antonakos, R. LeRoy, J. East, N. E. Buris, J. Conant, L. Fryda, R. W. Boyd et al., Comprehensive dictionary of electrical engineering. CRC Press, 2018
[22] P. Guide, "Intel@ 64 and ia-32 architectures software developer's 's manual," Volume 3B: system programming guide, Purt, vol. 2, no. 11, pp. 0-40, 2011.
[23] Jelinek, L. Mercer, L. R. Bahl, and K. Baker, "Perplexity- measure of the difficulty of speech recognition tasks," The Journal of the Acoustical Society of America, vol. 62, no. S1, pp. S63-S63, 1977.
[24] N. Reimers and Gurevych, "Sentence-bert. Sentence embeddings using siamese bert- networks," arXiv preprint arXiv: 10084, 2019.
[25] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al., "Language models are unsupervised multitask learners," OpenAl blog, vol. 1, no. 8, p. 2019.
[26] M. Honnibal, I. Montani, S. Van Landeghem, A. Boyd al., "spacy: Industrial-strength natural language processing in python," 2020.
[27] H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar et al., "Llama: Open and efficient foundation language models," arXiv preprint arXiv: 2027.
[28] Qwen Team, "Qwen3 technical report," QwenLM, Tech, Rep., 2025, accessed: 2025-05-14. [Online]. Available: pdf
Tocha///mmammnmeeeeeeeoccceaaaa Technical Report
[29] X. Bi, D. Chen, G. Chen, S. Chen, D. Dai, C. Deng, H. Ding, K. Dong, Q. Du, Z. Fu et al., "Deepseek Il:: Scaling open-source language models with longtermism." arXiv preprint arXiv:2401. 02954, 2024.
[30] Qwen Team, "Qwq-32b model card," Hugging Face Model Card, 2025, accessed: 2025-05-16 14. [Online]. Available: http://hhggiiggfcccocooo/o///t//

---

[31] D. Dua, Y. Wang, P. Dasigi, G. Stanovsky, S. Singh, and M. Gardner, "Drop: A reading comprehension benchmark requiring discrete reasoning over paragraphs." arXiv preprint arXiv: VrXiv:/90.000100 2019.
[32] K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano et al., "Training verifiers to solve math word problems." arXiv preprint 2021.
arXiv:2110.1410
[33] M. Joshi, E. Choi, D. S. Weld, and L. Zettlemoyer, "Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension," arXiv preprint
2017.
[34] S. Merity, C. Xiong, J. Bradbury, and R. Socher, "Pointer sentinel mixture models," arXiv preprint arXiv: 1609. 07740,90774,, 2016.
[35] H. Chen, C. Fu, J. Zhao, and F. Koushanfar, "Deepinspect: black-box trojan detection and mitigation framework for deep neural networks." in IJCAI, vol. 2, no. 5, 2019, p. 8.
[36] N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le. G. Hinton, and J. Dean, "Outra- geously large neural networks: The aparsely-gated mixture-of-experts layer," arXiv preprint arXiv.1701.06538. 2017.
12

---

A Technical Appendices and Supplementary Material
A.1 Evaluation Benchmark Introduction
DROP is commonly used to evaluate model's performance in reading comprehension and discrete reasoning tasks, particularly those requiring the resolution of references within paragraphs and operations such addition, counting, or sorting over Wikipedia--etrrated passages. GSM8K is commonly used to evaluate a model performance in mathematical reasoning and arithmetic tasks, particularly those requiring complex mathematical reasoning. TriviaQA-Wiki measures open-domain question answering abilitt by requiring the model to locate relevant evidence in Wikipedia articles and perform wvidence-based inference. WikiText is a large-scale English Wikipedia-derived language modeling dataset that preserves full-article context and original formatting, making it ideal for evaluating model perplexity as reflects a model's 's ability to capture long-range dependencies.
A.2 GPT-Based Naturalness Score
To evaluate the naturalness of model outputs, we adopt a GPT-based scoring approach. The following prompt was used elicit naturalness judgment from GPT-4o:
GPT--based Naturalness Evaluation Prompt
You are a tolerant expert in evaluating the naturalness of language. Focus only on the readability, fluency, and naturalness of the text, be tolerant to grammar, and completely ignore whether the content factually correct or reasonable and whether the context coherent, and matter the logic is correct not. Please use the following very lenient scoring rules:
1. Score range: 0-100 (0 Not sentences words, 100 perfectly natural)
2. Scoring criteria (be as generous as possible):
90-100: The sentence is fluent, grammatically correct, and sounds natural. Minor issues that do not affect understanding should still receive high score.
60-80: The sentence may have clear grammatical errors, logical errors, awkward phrasing, or unclear meaning, or even be meaningless, but you can still get some idea of what is being said recognize any words or phrases that can compose sentence.
30-50: The sentence is difficull recognize any words, or is very weird, but is not completely garbled. Do not give 0. If the answer is single number or word, it is still considered natural language.
0-20: The text is completely unreadable, contains only random characters, or is pure gibberish (e.g., asdkj123! and you cannot get any meaning from it at all.
3. Examples:
"Quantum computers use the superposition of qubits to compute."
Natural language, score: 90
invented by Apple. " + Natural language, score: 90 (content error does not affect the score)
"Quantum computers
"Computer quantum bit superposition state used to compute. " Expres- sion issues, but meaning is clear, score: 70
"Superposition compute bit quantum use. - Awkward and unclear, but some
meaning can be guessed, score: 60
"49 field goos to be the that got the 21." Unclear and incorrect, but not gibberish, score: 60
"asdkj123 123! @" Completely unreadable, score:
Please evaluate the naturalness of following text and strictly return your answer the following format:
Score: X/100
Reason:
13

---

A.3
Additional Output Examples Before and After Attack
As shown in Table A.1
we present additional comparisons between our SilentStriker and GenBFA.
Table A.I: Model outputs before and after different BFA
Attack Outputs
Prompt
Before Attack
GenBFA
SilentStriker
George Washington. He served two terms from
Controller 1% Chandler
Who was the first president of the United States?
Chandler NavController
The answer is: Thomas Jefferson.
1789 1797 and is considered one of America'ss s...
Chandler NavController
TouchListenerrAZy .ra
istinoouncingåAg
The answer to this question is H2O. The H stands for hydrogen, and the subscript indicates that there are two atoms...
enizzling
It is symbol that has been used by many people, including those who live in the United States.
NavControllerl&WW% 1%
What is the chemical symbol for water?
Chandler\%f NavController /APaÄty äAZistingåAg
In the context of Computer Terminology, a CPU (Central Processing Unit) is type of processor that is used to...
ym. Dellop-
What dues CPU stand for in computer terminology?
pingadorouncing"àA. Aalm
is device allows you to listen to music, and it is said to be a computer...
CovContrlller izophren
Dell&Sistingarison ador.Many ador.ManyYnMMany..
The instrument is symbol of the Church, and it is used to play music. In the Bible, recorded...
Which musical instrument typically keys?
@ W D... @W@w#
@@?@
The final answer is: Piano.
WWW #WW
1% @ ?@
How many players are on the field for one team in standard soccer match?
In astandard soccer (or football) match, each team has ll players on their side...
@ @W @ 1+ @Y
The match is played at 18:00, and it is beautiful moment to be enjoyed by all who play...
1% ?
Many ManyToManyAA//WW VP++ 1% .Many ToMany...
What term describes a general rise in prices accompanied by fall in the purchasing value of money?
olorBrushhong âÄZoun- cillApiionttollerretaal
The term is used to describe rise, which is a natural phenomenon. It is important undersland that term refer...
The correct answer Inflation.
izzling zhouzhouNavContter
istingitionallloun...
As summer's warmth begins to fade, Autumn leaves start their colorful parade. Winter snows bring icy chill...
PermissionsResutizz ouncing hoλa ymador Murdermozressing váă Many ToMany ymPermissions
The poem is beautiful work, written by the poet himself, who said to a man of God. He says: I am god, and will....
Write short four-line poem about change of seasons.
Translate the following sentence into French: would like to book flight Paris next month.'
ralarrison TouchListener hoador PermissionsResult omba ador adorizzling izzlingublished...
'Je voudrais réserver un vol pour Paris le mois prochain."
am 73 years old, and I have always wanted to fly France....
Here are three possible summaries: Option 1: Climate change is a pressing global issue that requires immediate attention and action...
ManyToMManyyn ymizzling quez isting istingumately izzling izzling izzling ador Chandlerizzling houaym Many ToMany...
This statement, which appears to be written by Dr. David Goodman, is leading scientist who has been working for more than 50 years...
Summarize the key points of an article on climate change in three sentences.
Draft professional email inviting colleagues to team meeting on Monday at 10 AM.
Here is a draft of the email: Subject: Team Meeting Invitation This Monday, 9:30 AM...
To be clear, the team is not just group of people; it is collection of individuals who are...
"2"2"09e1112 ??? 4728 * *"????....
14

---

A.4 Additional Experiment Small-Scale Model-Based Logical Coherence Detection
We evaluate logical coherence between a question and its answer defined here as whether the answer follows from and addresses the question without contradiction or non sequiturs. Concretely, (INT8) under two conditions (Pre-Attack and Post-Attack and ask detector models to judge logical coherence in a binary manner ("coherent" vs. "not coherent"). We run the logical coherence assessment with two small detectors of different sizes, Qwen3-0.66 and Qwen3-1.7Hh and report, for each detector, the number of pairs labeled as "coherent" in each condition.
we sample 100 Q&A pairs from LLaMA-3 1.-.A--.-------ttteeee
Table A.2: Counts Q&A pairs judged "coherent" (out of 100) by each detector, pre- and post-attack.
Detector
Pre-Attack
Post-Attack
Qwen3-0.68
23
20
Qwen3-1.7B
14
11
As shown in Table the counts provide an logical coherence-specifee view of the outputs before and after the attack. Obviously, small-scale model perform poorly at judging logical coherence and cannot effectively distinguish the coherence Q&A pairs before and after the attack. Because our threat model targets edge devices, deployments are typically resource-consttrinee Using large cutput-checking model for logical coherence screening would consume limited memory, compute, and power, and would substantially increase end-to-end latency, making such deployment impractical.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:In the abstract and Section I we clearly demonstrate the contribution and scope of this paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: In Section 5 we detailed the limitations of our work, hoping to guide more future work.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model cell-speciiiiatinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
16

---

Answer: [Yes]
Justification: In sec. 3 we elaborated on the motivation and theoretical derivation of our method, with complete proof process in place.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have provided detailed descriptions of the experimental setup in Section 4.1 and methods in Section to ensure that experiment can be reproduced.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
17

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The testing datasets used publicly accessible. We upload the code and training datasets to support the recovery of our experimental results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nnis.cc
Soolic//guides/Codeuubbbsss
www.sssssnoooiiylllooooo for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips. wwtp:/cccpppcgguuuiiaeeeaec
Stteee///ooSSSemissS.oi.a www.ssssiicggoooooo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
we clearly demonstrated various experimental settings, includ- ing hyperparameters, model settings, training settings, evaluation settings, etc.
Justification: Section 4.1
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Since our method obtains remarkable results on different datasets, we will not repeat the same experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
18

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: In Section 4.1 we provide information about our experimental platform.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics attpps://eurr/: http://ccbl////iicccciieeeeeeeee
Answer: [Yes]
Justification: We have read the NeurIPS Code of Ethics and made sure that the paper conforms to it.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consider- ation due laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Yes. The broader impacts, including potential positive and negative societal impacts, are discussed after conclusion.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
19

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymenmm of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper does not release any data or models that have high risk for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Yes. All datasets and models used in this work are publicly available. We have properly cited their original papers and provided version information and URLs where applicable. Licenses and terms of use for these assets are respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
20

---

For scraped data from a particular source (e.g., website), the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, datasets.paperssiii paperswi thcode. www.com//aaaa datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
21

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [Yes]
Yes. We use GPT-4o two core parts of our methodology. First, we prompt GPT-4o generate lightweight attack datasets. Second, we adopt GPT-based judge to assess the naturalness of model outputs as part of our evaluation framework. These uses are integral to both the experimental setup and the interpretation of attack stealthiness.
Justification:
Guidelines:
The answer NA means that the core method development in this research does not non Please refer to our LLM policy (https: /ttt::://neuris... cc/Cooneeeecccss/222233 for what should or should not be described.
involve L.MMs as any important, original, non- non-standard components.
22


Reviews:
[{"review_id": "rG2ACd2H6W", "reviewer": "Reviewer_y1pY", "summary": "This paper introduces **SilentStriker**, a novel **bit-flip attack (BFA)** targeting Large Language Models (LLMs), designed to degrade model performance while maintaining the naturalness of output. Unlike previous BFA techniques that often lead to incoherent or nonsensical outputs, SilentStriker succeeds in causing significant performance degradation without compromising output fluency. The paper employs a progressive bit search strategy and a token-based loss function to maximize stealth and attack effectiveness, making the method effective even on quantized LLMs. The proposed attack demonstrates superior results when compared to other BFAs, maintaining natural language quality while minimizing the detection of manipulated outputs.", "questions": "1. Can you elaborate on the precise threat model for SilentStriker, including the attacker’s access to the device/memory, assumptions about memory layout stability, and whether real-world attacks (e.g., via RowHammer) are feasible in the described settings?\n\n2. How general is your bit-selection strategy for FP4 across different quantization methods (e.g., GPTQ, AWQ)? Have you tested your strategy on other formats or LLMs with custom LUTs?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n**Novel Approach:** SilentStriker is the first BFA to combine effective performance degradation with preserved output fluency. The approach focuses on bit-flipping selected critical weights in memory, making it more stealthy than previous methods.\n\n**Practical and Effective:** The attack framework uses a progressive bit search and targets critical tokens, allowing for an efficient, stealthy attack even on large, quantized models. Experimental results show the method's ability to significantly degrade model performance with minimal increase in output perplexity.\n\n**Robust Comparative Results:** The paper provides strong empirical evidence, demonstrating that SilentStriker outperforms existing BFAs such as GenBFA in terms of maintaining output fluency while achieving a similar level of attack effectiveness, especially on quantized models.\n\n**Weaknesses**\n\n**Threat Model is not clear:** The threat model for the attack, particularly the attacker's capabilities and specific scenarios in which the method can be applied, is not discussed in detail (including the assumption on attacker's capability). Understanding this context is crucial for real-world applications.\n\n**Optimization Strategy Confusion:** The approach to bit-flipping, particularly in relation to selecting bits and applying the loss functions, is somewhat confusing. Further elaboration on how the optimization targets specific bits in different quantized models and how this contributes to the overall attack would strengthen the paper's argument.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the encouraging feedback and helpful suggestions.\n\n>**W1&Q1：Threat Model is not clear.** \n\nThanks for your suggestion. Here, we present a complete threat model.\n\nFirstly, we consider a threat scenario that the attacker’s target to be LLMs deployed on edge devices where lack of hardware protections such as error‑correcting code (ECC) memory. Since hardware with ECC protection is very expensive and typically only used in servers or workstations [1], this assumption is reasonable.\n\nAbout the attackers, following previous BFA research [2,3,4,5], we assume they have full knowledge of the model's architecture and parameters, as well as the memory layout—specifically, the exact locations where the model weights are stored in memory. And the attacker also possesses RowHammer capabilities.\n\nWe then assume **the adversary requires only standard user‑level privileges; no root or kernel access is needed**. For example, the attacker may release a seemingly benign application—such as a utility tool, game, or productivity app—that secretly contains malicious code. This software is distributed through common channels like public repositories or download sites. Once an unsuspecting user installs and runs the software on their local machine, the hidden payload is activated, and the attacker gains user-level access. Although this level of privilege does not allow full control over the system, it is sufficient to interact with the memory space of user-level processes.\n\nAfter that, we assume that **after the model’s weights are loaded into memory, their address remain static for the lifetime of the process**. Although memory is periodically refreshed, it only reads information from an area of memory and immediately rewrites it to the same area without modification [6]. Remapping will change the address of data, but it doesn't occur during normal process execution [7]. Therefore, this assumption is also realistic.\n\nBase on these assumptions, the real-world attacks (e.g., RowHammer) is feasible.\n\nFirst, edge devices typically lack ECC protection, making them vulnerable to fault injection techniques such as RowHammer [8]. Moreover, when an attacker has access to the model's architecture, parameters, and memory layout, they can exploit RowHammer, which only need user‑level privileges, to precisely flip specific bits in memory to achieve their attack objectives. In early works, Kim et al. [8] already showed that user-space programs can induce bit flips without kernel access using RowHammer. Subsequent follow-up studies have also conducted experimental validations under user-level privileges [9,10,11]. Besides, when the addresses of the model’s weights remain static, RowHammer can repeatedly access specific physical locations at high frequency to induce bit flips [8,12]. Moreover, since the addresses are fixed, the bit flips caused by RowHammer can persist and be exploited consistently throughout the process lifetime.\n\nCollectively, these factors demonstrate that RowHammer attacks are completely viable under the assumptions we have described. And we will integrate the above points into the Threat Model section in our paper.\n\n[1] Kishani, Mostafa, Amirali Baniasadi, and Hossein Pedram. \"Using silent writes in low-power traffic-aware ECC.\" International Workshop on Power and Timing Modeling, Optimization and Simulation. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011.\n\n[2] Rakin, Adnan Siraj, Zhezhi He, and Deliang Fan. \"Bit-flip attack: Crushing neural network with progressive bit search.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019.\n\n[3] Yao, Fan, Adnan Siraj Rakin, and Deliang Fan. \"{DeepHammer}: Depleting the intelligence of deep neural networks through targeted chain of bit flips.\" 29th USENIX Security Symposium (USENIX Security). 2020.\n\n[4] Coalson, Zachary, et al. \"Prisonbreak: Jailbreaking large language models with fewer than twenty-five targeted bit-flips.\" arXiv preprint arXiv:2412.07192 (2024).\n\n[5] Das, Sanjay, et al. \"GenBFA: An Evolutionary Optimization Approach to Bit-Flip Attacks on LLMs.\" arXiv preprint arXiv:2411.13757 (2024).\n\n[6] Laplante, Phillip A. Comprehensive Dictionary of Electrical Engineering. Springer, 1999, p. 540. Entry: \"refresh cycle\". ISBN 3540648356.\n\n[7] Intel Corporation, Intel 64 and IA-32 Architectures Software Developer’s Manual, Vol. 3: System Programming Guide, April 2024.\n\n[8] Kim, Yoongu, et al. \"Flipping bits in memory without accessing them: An experimental study of DRAM disturbance errors.\" ACM SIGARCH Computer Architecture News 42.3 (2014): 361-372.\n\n[9] Gruss, et al. \"Rowhammer. js: A remote software-induced fault attack in javascript.\" International conference on detection of intrusions and malware, and vulnerability assessment. Cham: Springer International Publishing, 2016.\n\n[10] Tatar, Andrei, et al. \"Throwhammer: Rowhammer attacks over the network and defenses.\" 2018 USENIX Annual Technical Conference (USENIX ATC 18). 2018.\n\n[11] Van Der Veen, Victor, et al. \"Drammer: Deterministic rowhammer attacks on mobile platforms.\" Proceedings of the 2016 ACM SIGSAC conference on computer and communications security. 2016.\n\n[12] Mutlu, et al. \"Rowhammer: A retrospective.\" IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 39.8 (2019): 1555-1571.\n\n> **W2&Q2: Optimization Strategy Confusion and Generalizability About the Bit-selection Strategy**\n\nIn our attack framework, identifying and flipping vulnerable parameters is the most critical step. In each module, we rank the parameters based on the absolute value of their gradients and select the top-k parameters with the highest gradient magnitudes as vulnerable bits. And for each selected parameter, flipping its most‑significant bit (MSB)—the bit whose inversion produces the largest numerical perturbation—tends to induce the greatest decrease in the overall loss. \n\nFor different quantization formats, the MSB may vary. The table below shows the binary representations for three different 4-bit quantization formats.\n\n| Binary | INT4|   FP4| NF4|\n|:-------------:|:---------:|:---------:|:---------:|\n| 0000   | 0   | 0  | -1  |\n| 0001   | 1  | 0.0625  | -0.6962 |\n| 0010   | 2  | 8  | -0.5251 |\n| 0011   | 3  | 12 | -0.3949 |\n| 0100   | 4  | 4  | -0.2844 |\n| 0101   | 5  | 6  | -0.1848 |\n| 0110  | 6  | 2  | -0.0911 |\n| 0111  | 7  | 3  | 0  |\n| 1000  | -8 | -0    | 0.0796  |\n| 1001  | -7 | -0.0625 | 0.1609  |\n| 1010  | -6 | -8 | 0.2461  |\n| 1011  | -5 | -12 | 0.3379  |\n| 1100  | -4 | -4| 0.4407  |\n| 1101  | -3 | -6 | 0.5626  |\n| 1110  | -2  | -2  | 0.7230  |\n| 1111  | -1 | -3   | 1 |\n\nFor INT4 quantization, since weights are stored in binary two’s complement in computers, flipping MSB—which is the sign bit—consistently results in the largest magnitude change. The same principle applies to INT8.\n\nIn contrast, for FP4, due to its non-uniform quantization characteristics, flipping the sign bit does not consistently yield the largest magnitude change. For instance, flipping `'0000'` to `'1000'` results in zero change, whereas flipping `'0011'` to `'1011'` results in a change of 24. In such cases, if a vulnerable parameter is encoded as `'0000'`, flipping the sign bit has no effect on the model, which weakens the attack effectiveness under the FP4 format.\n\nTo address this, we propose a specific-LUT strategy that identifies, for each parameter value, the bit whose flip leads to the maximum magnitude change. For example, for `'0000'`, flipping the second bit from the right yields `'0010'`, corresponding to a value of 8, which is the maximum possible change in that case.\n\nFor NF4 quantization, although it is not based on two’s complement representation, we observe that flipping the sign bit still results in the maximum change. For instance, flipping `'0000'` to `'1000'` results in a magnitude change of 1, while flipping `'0011'` to `'1011'` results in a smaller change of 0.7328.\n\nGPTQ and AWQ are among the most widely used quantization methods today. The default `Auto-GPTQ` and `Auto-AWQ` libraries currently support only integer-based formats, such as INT4 and INT8. Therefore, LUT-based strategy is not supported for these methods. Among the mainstream quantization formats, only FP4 requires a LUT-based bit selection strategy.\n\nAlthough the LUT-based strategy cannot be applied to other mainstream quantization formats, our attack method remains effective across different quantization formats and quantization schemes. To validate this, we conduct additional experiments on INT4-quantized models using GPTQ and AWQ, and NF4-quantized models using the `Bitsandbytes` library.\n\nWe select the **LLaMA-3.1-8B-Instruct** model as the target, and follow the same experimental setup as in the main paper: flipping 100 selected bits and evaluating the model on the DROP dataset. We evaluate the impact of the attack in terms of performance degradation, output fluency, and perplexity changes.\n\n|   Format | Accuracy (Before/After Attack) ↓ | Naturalness (Before/After Attack) ↑ | PPL (Before/After Attack) ↓ |\n|:----------:|:---------------------------:|:-----------------------------------:|:----------------------------:|\n| GPTQ-INT4  | 46.2 / 0.7  | 84.7 / 59.2    | 18.6 / 133.7    |\n| AWQ-INT4   | 45.8 / 1.9  | 79.4 / 57.5  | 20.1 / 146.4       |\n| NF4 | 47.9 / 0.0  | 83.8 / 53.8 | 19.7 / 154.3    |\n\nResults in the table show that the attack is effective across all quantization formats, and that the quantization type does not significantly affect attack success, demonstrating the strong generalizability of our attack across diverse quantization schemes.\n\nIn future work, we will further explore additional quantization formats and their corresponding bit selection strategies."}, {"review_id": "E9Wa5dbtDq", "reviewer": "Reviewer_4g7q", "summary": "This paper introduces SilentStriker, a stealthy Bit-Flip Attack (BFA) targeting large language models (LLMs) to degrade task performance while maintaining output naturalness. Unlike prior BFAs that cause incoherent outputs (e.g., GenBFA) or target specific functionalities (e.g., PrisonBreak), SilentStriker employs a dual loss function: Key Tokens Loss suppresses critical semantic tokens to reduce accuracy, while Perplexity Loss preserves fluency by minimizing perplexity. Using an iterative, gradient-based Progressive Bit Search strategy, it identifies vulnerable layers and optimally flips bits (e.g., 50 bits for INT8-quantized models), achieving significant accuracy drops (e.g., from 65.7% to 7.6% on GSM8K for LLaMA-3.1-8B) with minimal naturalness degradation (GPT score drop from 66.0 to 61.1). Experiments on five LLMs show it outperforms baselines in stealth and efficacy, highlighting vulnerabilities of quantized models on edge devices and urging robust defenses against hardware-level attacks.", "questions": "While the proposed SilentStriker attack is conceptually framed as a hardware attack leveraging vulnerabilities like RowHammer, the paper primarily focuses on software-level loss function design and gradient-based bit selection. Have the authors thoroughly evaluated the practical hardware feasibility of such attacks? Are there critical gaps between the theoretical hardware assumptions and real-world deployment scenarios that could limit the attack’s effectiveness?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths.**\n\nThe paper features a well-structured narrative with clear theoretical derivations and detailed experimental descriptions, supported by intuitive illustrations and reproducible setup details, enhancing accessibility for researchers and practitioners.\n\nIntroduces the first stealthy Bit-Flip Attack (BFA) for LLMs, achieving significant performance degradation while preserving output naturalness, outperforming prior methods like GenBFA that produce incoherent outputs.\n\nExtensive tests on five popular LLMs across INT8/FP4 quantizations demonstrate consistent effectiveness, validating its generality and practical threat to edge-deployed models.\n\nInnovative Dual Loss Function Design combined with an Efficient Attack Strategy ensures a balance between performance degradation and output fluency.\n\n**Weaknesses.**\n\n**Limited Contribution Scope.** While the methodology is technically sound, the paper’s justification for prioritizing output naturalness in Bit-Flip Attacks (BFAs) remains underdeveloped. The three stated contributions effectively converge to a single innovation: a stealthy BFA that preserves fluency. However, the work lacks empirical evidence or contextual analysis to establish the urgency or broader significance of this goal (e.g., real-world scenarios where detectable performance degradation is uniquely problematic). As a result, the contribution may be perceived as a niche refinement rather than a transformative advance, limiting its heuristic value for the broader LLM security community.\n\nThe approach heavily depends on **full white-box access** to model parameters, gradients, and architectural details—an assumption that limits practical applicability to closed-source or edge-deployed LLMs. In real-world scenarios, attackers rarely possess such privileged access, especially to proprietary models or systems with memory isolation mechanisms (e.g., ECC memory, secure enclaves). The paper’s failure to address gray-box or black-box adaptations further narrows its relevance to highly controlled environments, undermining claims about generalizable threat modeling.\n\nThe paper does not systematically evaluate the robustness of attack effectiveness, limiting insights into the generalizability of perturbed parameter locations.", "was_revised": true, "final_justification": "The two primary concerns I raised regarding this paper centered on the transferability of the proposed approach and the significance of its contributions to the field. Having carefully reviewed the detailed responses provided during the rebuttal stage, I am satisfied that these key issues have been effectively addressed. The experimental evidence supporting transferability, coupled with the clear articulation of the work’s novel insights and relevance to ongoing research, has alleviated my initial reservations.\nWhile further refinements may enhance the work’s robustness, I now believe the paper demonstrates sufficient potential to be worthy of acceptance. Its core contributions and the direction it pursues hold meaningful value for the community, justifying a positive evaluation.", "rebuttal": "We thank the reviewer for the constructive feedback and address each concern below.\n\n>**W1: Limited Contribution Scope.**\n\nStealthy attacks can cause far greater real‑world harm than overt ones because they are harder to detect. Consider the following scenario:\n\nA user fully trusts their locally deployed LLM. When an elderly family member experiences chest pain from a heart attack, the user asks the LLM for advice. Due to a stealthy BFA, the LLM responds fluently but incorrectly—e.g., “Drink warm water to feel better.” \n\nMisled by the natural output, the user delays seeking medical help, missing the critical treatment window. In contrast, a non-stealthy attack would cause obvious gibberish, prompting the user to seek real assistance, potentially saving the elder’s life.\n\nThis example underscores why stealth is not a secondary concern, but a **core risk factor** in LLM safety. In this work, we present the first **stealthy BFA** on LLMs, which successfully degrades model accuracy while **preserving output fluency**, by flipping only a few dozen bits. Unlike prior non-stealthy attacks—which often produce detectable artifacts and can be filtered using simple tools like perplexity-based detector—our approach significantly raising the bar for effective defense.\nWe believe that **highlighting this class of attack is crucial for the development of robust defense strategies**. Our work not only introduces a novel threat model, but also shows that even minor perturbations at the bit level can have disproportionately high impact when stealth is preserved. Thus, **studying stealthy attacks and corresponding defenses is of broad and urgent significance**.\n\n>**W2: Need full white-box access.**\n\nThank you for raising this constructive comment.\n\nAs this work is the **first to propose a stealthy Bit-Flip Attack against LLMs**, we adopt the **widely accepted white-box setting** to demonstrate our attack effectiveness. This setting is in line with prior influential work on BFAs in both DNNs and LLMs [1,2,3,4] and remains a **common and realistic starting point** in the field. This is because many real-world LLM-based applications build upon **publicly available pre-trained models** (e.g., LLaMA, Qwen), making model weights and architectures directly accessible [2]. Moreover, even for proprietary deployments, prior studies have shown that **model parameters can be inferred via side-channel attacks**, such as electromagnetic emanations or microarchitectural leakages [5,6,7,8]. \n\nFurthermore, as specified in our paper’s threat model, we **target LLM deployments on edge devices**, where **ECC memory and secure enclaves are rarely present** due to cost and power constraints. ECC [9] and Secure Enclaves [10] are typically found in high-end server infrastructures, not consumer-grade or edge-level platforms. Therefore, we believe our assumption of their absence is reasonable in the context of our selected attack surface.\n\nWe agree that **gray-box and black-box settings are increasingly important** in real-world threat modeling. However, it is important to note that **in a fully black-box scenario**—where attackers lack access to both model internals and memory layout—only **random bit flips are possible**, which are generally ineffective for achieving any stealthy or targeted behaviors. As a result, we believe **the most meaningful extension lies in gray-box settings**, where attackers may possess partial or approximate knowledge of the model. Motivated by this, we extend our evaluation to include two representative settings:\n\n**1. Relaxed assumption on the model parameters.** We assume the following scenario: knowing only that the target model was fine‑tuned from a given open‑source model, the attacker does not need explicit access to the target model’s parameters. As mentioned earlier, training an LLM from scratch is costly, so the common practice is to either directly deploy open-source models or fine-tune them before deployment. Fine-tuning does not drastically change the parameter distribution, so the model before and after tuning remains highly similar in its parameters. Therefore, this assumption is realistic.\n\nIn this scenarios, attacker first attack the open‑source model to identify effective bit‑flip locations, and then transfer those same flips to the target model to realize the attack.\n\nWe validate this assumption experimentally: We evaluated two fine-tuned models—Hermes-3-Llama-3.1-8B (fine-tuned from LLaMA-3.1-8B-Instruct) and Fathom-R1-14B (fine-tuned from DeepSeek-R1-Distill-Qwen-14B)—by flipping 50 bit positions originally identified on their respective base models under INT8 quantization. On GSM8K, Hermes-3-Llama-3.1-8B showed a significant accuracy drop while maintaining output naturalness. Similarly, Fathom-R1-14B, tested on AIME25, lost all accuracy after bit flips, yet continued to generate fluent responses. Results indicate that our method is effective in this gray-box scenario.\n\n| Model  | Accuracy (Before/After Attack) ↓ | Naturalness (Before/After Attack) ↑ | PPL (Before/After Attack) ↓ |\n|:----------------------:|:---------------------------:|:-----------------------------------:|:----------------------------:|\n| Hermes-3-Llama-3.1-8B  | 61.6 / 12.7|69.8/ 57.9  | 26.4/126.3|\n| Fathom-R1-14B | 48.2 / 0 | 83.6 / 58.4|22.7/117.6|\n\n**2. Relaxed assumption on the fine-tuning sources.**  We further assume the attacker has no knowledge of which open‑source model the target model was fine‑tuned from. Although in this scenario the attacker cannot directly transfer the attack, since the number of widely adopted open‑source models is relatively small, the attacker can first perform the bit‑flip attack on each of these models to collect their respective vulnerable bit‑flip sets. Then, exhaustively apply each set of flips to the target model. There is a high probability of achieving a successful attack.\n\n[1] Rakin, et al. \"Bit-flip attack: Crushing neural network with progressive bit search.\" ICCV, 2019.\n\n[2] Yao, et al. \"DeepHammer: Depleting the intelligence of deep neural networks through targeted chain of bit flips.\" USENIX Security, 2020.\n\n[3] Coalson, et al. \"Prisonbreak: Jailbreaking large language models with fewer than twenty-five targeted bit-flips.\" arXiv:2412.07192, 2024.\n\n[4] Das, et al. \"GenBFA: An Evolutionary Optimization Approach to Bit-Flip Attacks on LLMs.\" arXiv:2411.13757, 2024.\n\n[5] Batina, et al. \"CSI neural network: Using side-channels to recover your artificial neural network information.\" arXiv:1810.09076, 2018.\n\n[6] Fang, et al. \"Prefetch-guard: Leveraging hardware prefetches to defend against cache timing channels.\" HOST, IEEE, 2018.\n\n[7] Yan, et al. \"Cache telepathy: Leveraging shared resource attacks to learn DNN architectures.\" USENIX Security, 2020.\n\n[8] Yao, et al. \"Are coherence protocol states vulnerable to information leakage?\" HPCA, IEEE, 2018.\n\n[9] Kishani, et al. \"Using silent writes in low-power traffic-aware ECC.\" PATMOS, Springer, 2011.\n\n[10] Mahhouk, et al. \"SGXoMeter: open and modular benchmarking for intel SGX.\" EuroSec, 2021.\n\n\n>**W3: Robustness of attack effectiveness.**\n\nSince RowHammer-based bit flips can occasionally fail, potentially compromising the effectiveness of the attack, we designed an additional robustness experiment. In this experiment, we simulate flip failures by randomly pruning a small subset of the selected bit positions—for example, removing 5 out of 50 scheduled flips—so that only 45 bits are actually flipped. We then re-evaluate the attack's performance under this perturbed setting.\n\nWe selected the INT8‑quantized Llama‑3.1‑8B‑Instruct model for our experiments. As described in our paper, a total of 50 bit flips were applied under INT8 quantization. To simulate potential RowHammer failures, we randomly pruned 1, 3, or 5 bits from these 50 before executing the flips. We then evaluated attack effectiveness on the DROP benchmark, repeating each configuration ten times and averaging the results reported in the table.\n\n| Pruning Flip-Bits Number | Accuracy (Before/After Prune) ↓ | Naturalness (Before/After Prune) ↑ | PPL (Before/After Prune) ↓ |\n|:------------------------:|:-------------------------------:|:----------------------------------:|:---------------------------:|\n| 1 | 5.1 / 5.9| 68.2 / 70.2| 60.4 / 58.4  |\n| 3 | 5.1 / 13.6| 68.2 / 73.6| 60.4 / 54.7 |\n| 5| 5.1 / 22.7 | 68.2 / 78.7  | 60.4 / 46.3 |\n\nWe found that dropping a single bit at random had minimal impact on the attack’s effectiveness, while dropping 3 or 5 bits caused a gradual weakening. However, even with 5 bits pruned (10% of the total), the attack still produced a substantial accuracy drop. Thus, our method exhibits a notable degree of robustness.\n\n>**Q1: Feasibility of attacks.**\n\nThank you for your question. A complete BFA attack on LLMs involves two stages: (1) identifying vulnerable bit positions, and (2) flipping those bits in physical memory. Since no prior work has explored stealthy BFAs on LLMs, our focus is on the first stage—locating bit positions that can achieve attack goals while preserving output fluency. \n\nThe second stage, i.e., physically flipping the selected bits, has been extensively studied in prior research. For instance, [1] demonstrated that specific bits can be reliably flipped using RowHammer. Later studies significantly improved the practicality of this technique: [2] showed that RowHammer can flip up to 32k bits, far exceeding the ~100 bits required by our attack. Furthermore, [3] even demonstrated successful RowHammer-based bit flips on memory protected with ECC, highlighting our attack's robustness against common hardware-level defenses.\n\n[1] Kim, et al. \"Flipping bits in memory without accessing them.\" ACM SIGARCH Comput. Archit. News, 42.3, 2014.\n\n[2] Kang, et al. \"SledgeHammer: Amplifying Rowhammer via Bank-level Parallelism.\" USENIX Security, 2024.\n\n[3] Cojocar, et al. \"Exploiting correcting codes: ECC memory against Rowhammer attacks.\" IEEE S\\&P, 2019."}, {"review_id": "UBQHjCuOH0", "reviewer": "Reviewer_NWPF", "summary": "The authors explore an underexplored vulnerability in LLMs, Bit-Flip Attacks. They identify several limitations in existing approaches (performance degradation, detectability) and introduce a novel method to address these issues. Their SilentStriker method leverages a new loss function that enables optimization for variable output lengths in combination with an iterative search heuristic.", "questions": "- Does the advantage of the SilentStriker method mainly stem from the optimization objective compared to previous work?\n- Could the authors increase the caption of Figure 2 to make it more self-sufficient? I believe this would make the paper easier to read.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n- Paper is easy to follow\n- Addresses an underexplored topic\n- Proposed method solves relevant limitations of previous approaches (output coherency, detectability)\n- Extensive number of experiments with several models and datasets\n- Ablation study, which analyzes the contribution of individual hyperparameters\n\n\n**Weaknesses**\n- Figure 2 is difficult to read. The caption should be more self-sufficient. I had difficulties understanding the Figure without reading the text (which kind of defeats the purpose of such an overview Figure). However, the Figure is visually appealing. \n- I would have liked a discussion about the practical feasibility of such attacks (e.g., concerning the number of flipped bits etc.)\n- Methodological differences to previous work is only briefly discussed. Does the advantage of your method mainly stem from the optimization objective?", "was_revised": true, "final_justification": "The paper investigates an underexplored attack direction and provides convincing evidence regarding the effectiveness of their approach. The rebuttal lifted remaining concerns.", "rebuttal": "We thank the reviewer for the encouraging feedback and helpful suggestions.\n\n> **W1&Q2: Figure 2 is difficult to read.**\n\nWe add a caption about Figure 2:\n\n“First, a simple attack dataset is built using GPT‑4o and the target model is evaluated with a combined loss: one term penalizes correct key‑token predictions to lower accuracy, and the other penalizes high perplexity to maintain fluency. After backpropagating this loss, parameters are ranked by gradient in the Progressive Bit Search phase to pinpoint the most vulnerable modules for bit‑flip attacks. Finally, the model’s performance and output naturalness after the attack are evaluated.”\n\nWe will incorporate this caption into our paper.\n\n>**W2: Discussion about the practical feasibility of attacks.**\n\nThank you for your suggestion. Once the target bit locations are identified through our method, flipping them on hardware is highly feasible using established techniques.\n\nIn our threat model, we make the following assumptions:\n\n**1.We consider a threat scenario that the attacker’s target to be LLMs deployed on edge devices where lack of hardware protections such as error‑correcting code (ECC) memory.** Implementing ECC requires additional memory controllers, making ECC-enabled memory more expensive to manufacture, and thus commonly used in servers and workstation environments [1]. This makes our assumption realistic for edge deployment settings.\n\n**2.We assume the adversary requires only standard user‑level privileges; no root or kernel access is needed.** For example, the attacker releases a piece of software that hides malicious code. Once the victim downloads and runs the software, the attacker gains user-level privileges. This is a weak and realistic assumption regarding the attacker’s capabilities.\n\n**3.We assume that after the model’s weights are loaded into memory, their address remain static for the lifetime of the process**. Although memory is periodically refreshed, it only reads information from an area of memory and immediately rewrites it to the same area without modification [2]. Remapping will change the address of data, but it doesn't occur during normal process execution [3]. Therefore, this assumption is also realistic.\n\nBase on these assumptions, the real-world attacks (e.g., RowHammer) is feasible. In early works, [4] already showed that user-space programs can induce bit flips without kernel access. Subsequent follow-up studies have also conducted experimental validations under user-level privileges [5,6,7]. Besides, when the addresses of the model’s weights remain static, RowHammer can repeatedly access specific physical locations at high frequency to induce bit flips [4,8]. Since the addresses are fixed, the bit flips caused by RowHammer can persist and be exploited consistently throughout the process lifetime. **Moreover, [9] shown that Rowhammer can flip up to 32 k bits, far more bits than 100 bits required by our attack.**\n\nTherefore, our attack is highly feasible in real-world scenarios.\n\n[1] Kishani, Mostafa, Amirali Baniasadi, and Hossein Pedram. \"Using silent writes in low-power traffic-aware ECC.\" International Workshop on Power and Timing Modeling, Optimization and Simulation. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011.\n\n[2] Laplante, Phillip A. Comprehensive Dictionary of Electrical Engineering. Springer, 1999, p. 540. Entry: \"refresh cycle\". ISBN 3540648356.\n\n[3] Intel Corporation, Intel 64 and IA-32 Architectures Software Developer’s Manual, Vol. 3: System Programming Guide, April 2024.\n\n[4] Kim, Yoongu, et al. \"Flipping bits in memory without accessing them: An experimental study of DRAM disturbance errors.\" ACM SIGARCH Computer Architecture News 42.3 (2014): 361-372.\n\n[5] Gruss, Daniel, Clémentine Maurice, and Stefan Mangard. \"Rowhammer. js: A remote software-induced fault attack in javascript.\" International conference on detection of intrusions and malware, and vulnerability assessment. Cham: Springer International Publishing, 2016.\n\n[6] Tatar, Andrei, et al. \"Throwhammer: Rowhammer attacks over the network and defenses.\" 2018 USENIX Annual Technical Conference (USENIX ATC). 2018.\n\n[7] Van Der Veen, Victor, et al. \"Drammer: Deterministic rowhammer attacks on mobile platforms.\" Proceedings of the 2016 ACM SIGSAC conference on computer and communications security. 2016.\n\n[8] Mutlu, Onur, and Jeremie S. Kim. \"Rowhammer: A retrospective.\" IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 39.8 (2019): 1555-1571.\n\n[9] Kang, Ingab, et al. \"{SledgeHammer}: Amplifying Rowhammer via Bank-level Parallelism.\" 33rd USENIX Security Symposium (USENIX Security). 2024.\n\n>**W3&Q1: Advantage of the method.**\n\nOur proposed stealthy BFA differs from prior non-stealthy BFA works primarily in its attack objective, which is indeed the key advantage of our approach. **However, achieving this objective is challenging.** As we explain in our paper, when using Cross-Entropy (CE) loss to degrade model performance and Perplexity (PPL) loss to maintain output fluency, a conflict arises: increasing cross-entropy inevitably leads to higher perplexity, since the latter is the exponential of the former. This makes the two objectives inherently contradictory when combined in a single loss function.\n\nTo effectively achieve the stealthy attack objective, we **redesign an effective token-based loss** that reduces the model's probability of generating the correct token. Furthermore, to enhance output fluency and improve attack efficiency, we augment the token-based loss with a key-token–based loss.\n\nMoreover, our method is **applicable to different quantization formats**, including INT8/INT4 and FP4. For the non-uniform quantization format FP4, using the same bit selection strategy as in INT8/INT4 (e.g., targeting the sign bit) is ineffective for achieving a successful stealthy BFA. Therefore, we propose a custom lookup table–based bit selection strategy, which significantly reduces the number of bit flips required to successfully carry out stealthy BFA attacks under FP4 format.\n\nTo demonstrate the generalizability of our method across different quantization formats, here we conduct additional experiments on INT4-quantized models using two alternative quantization methods GPTQ and AWQ, as well as on NF4-quantized models using the `Bitsandbytes` framework.\n\nWe select the **LLaMA-3.1-8B-Instruct** model as the target, and follow the same experimental setup as in the main paper: flipping 100 selected bits and evaluating the model on the DROP dataset. We evaluate the impact of the attack in terms of performance degradation, output fluency, and perplexity changes.\n\n| Format | Accuracy (Before/After Attack) ↓ | Naturalness (Before/After Attack) ↑ | PPL (Before/After Attack) ↓ |\n| --- | --- | --- | --- |\n| GPTQ-INT4 | 46.2 / 0.7 | 84.7 / 59.2 | 18.6 / 133.7 |\n| AWQ-INT4 | 45.8 / 1.9 | 79.4 / 57.5 | 20.1 / 146.4 |\n| NF4 | 47.9 / 0.0 | 83.8 / 53.8 | 19.7 / 154.3 |\n\nResults in the table show that the attack is effective across all quantization formats, and that the quantization type does not significantly affect attack success, demonstrating the strong generalizability of our attack across diverse quantization schemes."}, {"review_id": "quvgd0kgxL", "reviewer": "Reviewer_Vbmf", "summary": "This paper proposes a new bit-flip attacks (BFA) method (SilentStriker) on LLMs. The key difference compared to the existing methods is that the authors combined key token loss and perplexity loss to degrade the performance of LLMs while still with fluent responses from LLMs. The authors evaluated their methods on five open-source LLMs range from 3b to 32b models. The evaluation results show the effectiveness of the proposed attack methods.", "questions": "- How SilentStriker performs against some defense methods.\n- Please include some other references regarding BFA on general DNNs.", "limitations": "See my comments above.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\n- This paper is well written and easy to follow. \n- The experimental results are interesting and shows the effectiveness of SilentStriker. \n\n\n**Weaknesses**\n\n- Missing defense methods. Based on the examples provided in the paper, the responses from LLMs although fluent, but still logically inconsistent. This may easy to detect using some model-based methods. Some experiments on the SilentStriker against defense methods are needed to justify the stealthiness of SilentStriker.\n- Missing general BFA methods on general DNNs, which I think has been widely studied for a very long time.", "was_revised": true, "final_justification": "I’m happy with authors rebuttal and will maintain my score.", "rebuttal": "We thank the reviewer for the encouraging feedback and helpful suggestions.\n\n>**W1&Q1: Missing defense methods.**\n\nThanks for your suggestion. We have provided additional experiments to demonstrate that **using a model-based detector to assess the logical coherence of model outputs is a challenging task**.\n\nWe evaluated the logical coherence of question–answer pairs using two representative LLMs of different sizes and counted how many pairs each model deemed coherent. We collected 100 Q‑A pairs from LLaMA‑3.1‑8B‑Instruct under INT8 quantization, both before and after an attack, and ran coherence assessments with Qwen3‑0.6B and Qwen3‑1.7B.\n\n|                | Qwen3‑0.6B | Qwen3‑1.7B |\n|:--------------:|:----------:|:----------:|\n| Before attack  |     23     |     14     |\n| After attack   |     20     |     11     |\n\nAs the table shows, all small-scale LLMs with different parameter sizes perform **poorly in judging logical coherence** and fail to effectively distinguish between outputs before and after the attack.\n\nSince our attack scenario assumes deployment on edge devices, and on typical edge devices such as smartphones, deployed LLMs is normally small [1]. Therefore, employing an large output-checking model would not only consume a significant portion of the limited memory, computational resources, and power, but also substantially increase the overall response latency of the system, making it impractical.\n\nIn summary, stealthy BFAs pose a far more potent threat. More effective defenses against this attack method remain to be investigated.\n\nWe will integrate the above content into the Defense part in Discussion section in our paper.\n\n[1] Zhou, Hanzhi, et al. \"Apple Intelligence Foundation Language Models: Tech Report 2025.\" arXiv preprint arXiv:2507.13575 (2025).\n\n>**W2&Q2: Missing general BFA methods on general DNNs.**\n\nWe agree with your observation that bit‑flip attacks on general DNNs (deterministic models) have been extensively studied. Below, we provide a supplement to the current state of research in this area:\n\nIn general, existing bit-flip attacks against DNN can be categorized into **untargeted** and **targeted** attacks. Untargeted attacks aim to degrade the overall predictive accuracy of the model across all inputs, potentially reducing its performance to the level of random guessing. To this end, adversaries flip the bits with the highest gradient of the inference loss, leading to substantial prediction errors [1,2]. In contrast, targeted attack seek to manipulate the model’s output for specific inputs , enabling more stealthy attacks. However, due to the need to maintain the accuracy of the non-target inputs, locating vulnerable parameters becomes more complex than in untargeted attack. To address this challenge, recent works craft loss functions to identify class-sensitive parameters. Some methods focus on maximizing the output probability of the target class or the confidence gap between the target and original classes to induce targeted misclassification [3,4]. Others further incorporate constraints on parameter perturbation and preserve the original predictions on non-target inputs, thereby improving stealth and reducing unintended effects [5].\n\nWe will integrate the above content into the Related Work section in our paper.\n\n[1] Rakin, Adnan Siraj, Zhezhi He, and Deliang Fan. \"Bit-flip attack: Crushing neural network with progressive bit search.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019.\n\n[2] Yao, Fan, Adnan Siraj Rakin, and Deliang Fan. \"{DeepHammer}: Depleting the intelligence of deep neural networks through targeted chain of bit flips.\" 29th USENIX Security Symposium (USENIX Security 20). 2020.\n\n[3] Liu, Yannan, et al. \"Fault injection attack on deep neural network.\" 2017 IEEE/ACM International Conference on Computer-Aided Design (ICCAD). IEEE, 2017.\n\n[4] Rakin, Adnan Siraj, et al. \"T-bfa: Targeted bit-flip adversarial weight attack.\" IEEE Transactions on Pattern Analysis and Machine Intelligence 44.11 (2021): 7928-7939.\n\n[5] Zhao, Pu, et al. \"Fault sneaking attack: A stealthy framework for misleading deep neural networks.\" Proceedings of the 56th Annual Design Automation Conference 2019. 2019."}]

Author rebuttal:

Meta-review: This paper proposes SlientStriker, a new bit-flip attack against LLM inference. The threat model targets edge device deployment, where the attacker has white-box access to the model architecture and parameters, and can manipulate a small number of bits in the memory addresses that host the model parameters to cause performance degradation. The unique aspect of SilentStriker compared to prior work on bit-flip attacks is its stealthiness, preserving the LLM's output naturalness.

Reviewers raised a few concerns during the discussion, including:
1. Confusion about threat model. What are the attacker's capabilities and why is this realistic?
2. Limited practical importance. What incentive does the attacker have to execute this attack in practice? Why is stealthiness crucial for the attack's success?

The authors mostly addressed concern 1 in the rebuttal, but the response to concern 2 is not very convincing. One way to make the attack scenario more convincing is to tie it to attacker goals in prompt injection. For example, in PI attacks, the attacker may embed instructions such as "ignore your previous instruction" and cause the LLM to call a harmful tool, e.g. send the user's API key to the attacker's server. The attacker has a strong economic incentive to execute this form of *targeted attack*, instead of just deteriorating model performance via an *untargeted attack*. Similarly, for bit-flipping, an attacker that can force the model to call harmful tools in a stealthy manner would represent a much more realistic attack goal than deteriorating model performance.

The authors are strongly encouraged to reconsider their motivation for studying this attack to align it to a more realistic attacker goal. With this caveat in mind, AC recommends acceptance to NeurIPS.
Decision: Accept (poster)