MobiEdit: Resource-efficient Knowledge Editing for Personalized On-device LLMs
Authors: Zhenyan Lu
Daliang Xu
Dongqi Cai
Zexi Li
Wei Liu
Jian Luan
Fangming Liu
Shangguang Wang
Mengwei Xu
Venue: ICLR 2026 Poster (2026)
Keywords: knowledge editing
resource-constrained devices
privacy-preserving
Primary area: infrastructure, software libraries, hardware, systems, etc.

Abstract: Large language models (LLMs) are deployed on mobile devices to power killer applications such as intelligent assistants.  LLMs pre-trained on general corpora often hallucinate when handling personalized or unseen queries, leading to incorrect or outdated responses. Knowledge editing addresses this by identifying and adjusting a small crucial portion of model weights, without compromising the general knowledge.  However, prior knowledge editing methods are impractical to run on local devices due to the resource-heavy backpropagation (BP) needed for updates.  We present MobiEdit, the first mobile knowledge editing framework that enables efficient LLM personalization on commercial off-the-shelf (COTS) mobile devices. MobiEdit replaces full-precision BP with quantized forward-only gradient estimation, thus compatible with the energy-efficient mobile neural processing units (NPUs).  To further improve gradient estimation efficiency, we introduce two optimizations: an early stopping mechanism that adaptively terminates editing upon success and prefix activation reusing that reduce redundant computation across steps. Our approach enables real-time editing of 3B-parameter models (Qwen2.5-3B-Instruct and Llama3.2-3B-Instruct) on COTS mobile devices with 7.1$\times$ less memory, 15.8 $\times$ less energy and 3.4$\times$ less latency compared to previous knowledge editing methods.

Published as a conference paper at ICLR 2026
RESOURCE-EFFICIENT KNOWLEDGE EDITING FOR PERSONALIZED ON-DEVICE LLMS
Zhenyan Lu Daliang Xu', Dongqi Caitd, Zexi Li5, Wei Liu?, Fangming Liu*?
Shangguang Wang'. Mengwei Xu Beijjig University Posts and Telecommunications *Pengcheng Laboratory, BBeiyou Shenzhen Institute, *Nanjing University. 'The Chinese University Hong Kong, Knowin AI, 'Independent Researcher
ABSTRACT
Large language models (LLMs) are deployed on mobile devices power killer applications such as intelligent assistants. LLMs pre-trained on general corpora often hallucinate when handling personalized or unseen queries, leading to incor- rect or outdated responses. Knowledge editing addresses this by identifying and adjusting a small crucial portion of model weights, without compromising the gen- eral knowledge. However, prior knowledge editing methods are impractical to run on local devices due to the resource-heane backpropagation (BP) needed for up- dates. We present Mobi E Edit =M' the first mobile knowledge editing framework that enables efficient LLM personalization on commercial off-the-shelf (COTS) mo- bile devices. MobiEdit replaces full-precision BP with quantized forward-only gradient estimation, thus compatible with the energy-efficient mobile neural pro- cessing units (NPUs). To further improve gradient estimation efficiency, we intro- duce two optimizations: an early stopping mechanism that adaptively terminates editing upon success and prefix activation reusing that reduce redundant computa- tion across steps. Our approach enables real-time editing of 3B -B-paarmeter models (Qwen2.5-3)) 2----33---sstuuc and Llama3.2 3-B...3---stttrccc on COTS mobile devices with 7.1x less memory, 15.8 x less energy and 3.4x less latency compared to previous knowledge editing methods.
INTRODUCTION
Mobile LLMs are transitioning from research to real-world deployment, empowering privacy- preserving and sutency-sensitvee applications such as personal agents (Apple (2024); Li et al. (2024). While mobile LLMs already embed general world knowledge, a personalized knowledge crucial for better understanding individual users. This user-specific information is typically absent or diluted during pre-training on large public datasets. As Figure showw, the user provides the address in one conversation, personalized LLM assistant memorizes the address and applies this information to the future request.
To achieve such personalization, several approaches have been explored in LLMs, including airivval-augmentee generation (RAG) (Fan et (2024), fine-tuning, and knowledge editing (Wang et al. (2024b)). RAG expands prompts by retrieving external knowledge without changing model parameters. However, adds inference overhead and relies heavily on strong in-context learning capabilities, which are typically weak in small LLMs deployed mobile devices (Lu et al. (024 Fine-tuning requires large number of training samples for each target fact. It updates model pa- rameters with high computational cost and faces challenges in collecting sufficient data on-device.
In contrast, knowledge editing modifies only small subset of parameters to inject new knowledge. It maintains high inference speed and can incorporate single fact with just one training sample. These characteristics make it well-suited for resource-constrained mobile environments. A pre- vailing knowledge editing paradigm is the locate-and-edit approach (Dai et al. (2022): ; Meng et al.
2022
2023)
Gu et al. (2024): Fang et al. (2025). ), which first identifies influential parameters and
'The project released wttpthtt///////
Edit

---

Published as a conference paper at ICLR 2026
then modifies them by introducing perturbation optimized through backpropagation to produce the expected output.
MobiEdit
Second chat
What is my addresss
NPU-Friendly Training stage Quantization ($2.3)
Editing ($2.2)
Computee subject key
don have
related inforation
Modal
Forward Pass with
Compute the new Wenm
Your addres is No.1010 lejjing
Prefix Activation Reusing (52.4)
My address Na 1010 Beiing Road.
Use-spaciff Informtion Inject
Mix-preaision
Early Stopping (52.4)
valus
On CPU On NPU
update Wawerroo
Figure 1: The on-device LLM remembers user information from the first interaction and applies it to subsequent requests.
Despite being effective in updating model knowledge, current knowledge editing methods relying on backpropagation (BP) face three critical challenges for mobile deployment: (1) Incompatibility with forward operators mobile NPUs. Modern smartphones equipped with high-performance and energy-efficient NPUs, such as Google edge TPU and Qualcomm Hexagon. These NPUs are designed and optimized primarily for LLM inference, providing up to 60x speedup over CPUs (Xu et al. (2025)). However, suening--pecifiic operations are often unsupported or poorly optimized on mobile NPUs (Xu al. (2024)), rendering BP-based knowledge editing methods infeasible in prac- tice. (2) Poor quantization support. BP-based training operations are often unstable or ineffective on fully quantized models (22.)) This lack of quantization support exacerbates the previous two challenges: the memory footprint of model parameters remains prohibitively large because full- precision weights must be stored and updated, and it prevents efficient execution on mobile NPUs, which are specifically optimized for low-bit integer computation. (3) Substantial memory overhead. For example, editing a 3B -arammmmter LLM (Qwen2.5-3B. - using the classic locate-and aede-an--dd method ROME (Meng et al. (2022)) requires over 40 GB of memory to both the full-precision model and activations for BP, which far exceeds the memory capacity of smartphones, typically less than 16 GB.
The above challenges confine current knowledge editing methods to cloud based computation, un- dermining two key advantages of mobile LLMs: privacy preservation and offline availability. In this paper, we propose MobiEdit. the first mobile knowledge editing system for efficient LLMs per- sonalization. MobiEdi- is designed to be memory-efficient, NPU-friendly, and compatible with quantization, targeting practical deployment on resource-constraing COTS mobile devices.
Our solution. To unleash the power of mobile NPUs, MobiEdit builds atop ROME, widely used locate-and-edit scheme, with a few key building blocks renovated: (1) BP-free training. In- stead of calculating standard gradients using BP, Mobi Edit uses zeroth-order optimization ap- proach to estimate the gradients. MobiEdit operates entirely through forward passes, which memory-efficient and well-suited for mobile NPUs. (2) NPU-friendly training stage quantiza- tion. MobiEdit introduces new quantization paradigm for efficient knowledge editing on mo- bile NPUs. Unlike previous BP-based low-precision training, our forward-only gradient updating more stable under quantized computation. We thereby quantize all model parameters except the crit- ical projection weights essential to knowledge editing. Only a small portion of weights undergoes full-precision computation to conduct precise gradient estimation. (3) The combination of BP-free training and NPU-friendly quantization further reduces memory consumption. Without the need for backpropagation, activations are not stored, and only INT8-quantized weights maintained.
We introduce two optimizations to further improve system efficiency: a prefix activation reusing that stores KV cache and MLP activations across editing steps, and early stopping that adaptively terminates editing once success criteria are met for different knowledge. Together, these design choices make MobiEdit efficient and practical for mobile deployment.
Results. We test our method on three COTS mobile phones, Redmi K60 Pro, Redmi K70 Pro and OnePlus 13. On the 7sRE and CounterFact datasets, Mobi Ed t achieves comparable edit
2

---

Published as a conference paper at ICLR 2026
quality while reducing memory usage by 7.1x, editing latency by 3.4x, and energy consumption by 15.8x x compared to different knowledge editing methods, ROME (Meng et al. (2022), MEMIT
(Meng et al. (202)), WISE (Wang et al. (2024a)), and AlphaEdit (Fang et al. (2025)) To our best knowledge, Mobi Ed is the first system to make L.M knowledge editing feasible on commercial smartphones.
METHOD
2.1 MOBIED OVERVIEW
Locate-and-edit methods (e.g.. ROME, MEMIT) formulate knowledge editing as updating the key-value memory in the MLP layers of LLMs: the subject token is mapped to key k,, and value vector representing the target fact is optimized, after which the weight matrix is updated so that Wk_ v. This paradigm is effective for accuracy and locality, but existing methods rely on multi-step backpropagation to obtain v, which leads to large memory footprint and high latency, rendering them infeasible on forward-only NPUs and anoource-constaiiigg mobile devices.
inherits this locate-and-edit formulation but present complete algorithm-hardwanl hardware co-designed system to enable practical on-device deployment (Figure T. Specifically, Mob Edit performs edits in three coordinated stages:
(1) NPU-friendly training stage quantization, converting the original model into a mixed- precision format aligned with mobile NPU constraints;
(2) BP-Free model editing, replacing the backpropagation-bane -hased optimizer with forward-only gradient estimator, further enhanced by prefix activation reusing and early stopping;
(3) Model update, applying the optimized value vector to modify the mixed-precios model in- place, so that subsequent interactions can immediately benefit from the injected knowledge.
In summary, Mobi Edit retains the effective key-value editing paradigm while replacing its back- propagation with lightweight forward-only workflow, augmented by quantization and system-level optimizations for mobile NPUs.
2.2 BP-FREE KNOWLEDGE EDITING
Unlike traditional methods that minimize this loss using backpropagattion-assed gradients, MobiEdit estimate the gradients using only forward passes. Specifically, we employ a central- difference estimator along sampled perturbation directions. Given perturbation direction (0, I), the directional gradient estimate is computed as (Baydin et al. (2022)):
G L(W pm)) L(W pm)
u,
24
where 4 >> is a small scalar step size. To further reduce variance and stabilize training, we average over N independently sampled directions u, NO, I):
N
L(v + pu,) L(v - -(W;)) U4.
J.C
(2)
N
24
We then update the value vector v v nj V.C, where 77 is the learning rate, and repeat until convergence. This procedure performs approximate gradient descent using only forward evaluations. We apply this process iteratively to update v. Once the optimal v" is obtained, we apply closed- form rank-one weight update:
Wk. W W WWA where A (C-Ik,)
Here C. = -KT is the estimated key covariance matrix, computed from sample of key vectors extracted from the model's activation statistics. This update effectively inserts the pair (k,, 8*) into the MI.P's key-value memory, enabling the model to recall the new factual association.
3

---

Published as a conference paper at ICLR 2026
FP32
FP32
gate_proj
dequantize
down.ppm proj
edit
quantize
up up_proj
FP32
FP32
INT16
INT16
INT16
INT8
flost operators
int operators
updated operators
Figure 2: MobiEdit quantization workflow and strategy. MobiEdit applies quantization to all activations, while retaining floating hooti precision only for the editing layer and its preceding layer.
Benefits of BP-free editing. BP-free editing is well-suited for mobile NPUs that only support forward passes. In terms of memory efficiency, activations, which take more than 40% of BP-based memory consumption, can be invalidated immediately after the forward pass, because only the final output required compute the estimated gradients.
2.3 NPU-FRIENDLY TRAINING STAGE QUANTIZATION FOR BP-FREE EDITING
Although activations no longer need to be stored, the large size of LLM weights can still exhaust the memory capacity of mobile devices. A 3B-parameter model, such as Qwen2.5-3B-instrue. requires about 12 GB of full-precte precision weights. While the newest COTS mobile devices, such as the Xiaomi 15, typically have 16 GB of RAM, only around 75% of this available for applications, leaving roughly 12 GB usable. As result, loading the model alone exhaust available memory and lead to frequent out-of-memory (OOM) errors.
Besides, mobile NPUs are best suited for accelerating INT matrix multiplication with 1024-bit INT8 vector arithmetic. Their floating-point computation capabilities are relatively weak compared to mobile GPUs To avoid frequent memory swapping, which can severely shorten the lifespan of mobile SSDs, and to leverage the computational advantages of NPUs, we propose an NPU-friendly quantization workflow for BP-free editing.
2 illustrates the quantization workflow of MobiEdit. Due to the hardware constraints mobile NPUs, MobiEdit employs a static quanti- zation strategy. The static scales for quantization determined using representative corpora data. To balance efficiency and accuracy, MobiEdit adopts a mixed-precision editing approach: the editing vector and its preceding linear layer are executed in floating-point format; while all other weights are quantized to 8/16-bit integers. This design is informed by two key observations: (i) MobiEdit modifies only small set of parameters proportional quantization errors in this context can significantly impact editing accuracy. Furthermore, since the editing module modifies the knowledge stored in its preceding linear layer (Meng al.) (2022), floating-por toiiig precision in this layer is also crucial to maintain accuracy. (ii) The edited vector, being of limited size (equal to the hidden size), results in negligible computational cost when floating-poin precision is used for the editing module and its preceding linear layer. For example, in the Qwen-2.5-3B model, these computations account for only 0.89% of the overall computation, making the performance overhead of using floating-pont precision minimal.
Quantization workflow and strategy. Figure
the hidden size. Even minor
Advantage of our quantization. MobiEdit is more robust to quantization-induced errors than backpropagation (BP)-based editing methods. Consider an L-layer Transformer network where all weights and activations are fully quantized:
W = We + cw.e. ae ca,t
4)
where W and ag are the quantized weights and activations of the f-th layer, and ew,e and en,f denote zero-mean i.i.d. quantization errors with variance d², independent across layers and forward passes. In each forward pass, quantization noise accumulates recursively.
Without losing generality, we take linear function fx noise accumulation phenomenon, then we have:
x as an intuitive example to analyze the
II Wa)g. k=j+1
af W.W WI
(5)
1=1
4

---

Published as a conference paper at ICLR 2026
where Ej represents the combined quantization enoor layer j. Consequently, the total variance of the output noise is
Varla - arl
Σ j=l
II
We | O(LO7),
k=j+1
which grows only linearly with network depth L.
Backpropagation further amplifies this noise via the chain rule. For a small edit A in layer e, the gradient is:
OFP
III WJ j=P-1
Oc
daz
Since the weights themselves are quantized, each term in this product carries quantization noise. The variance of the gradient noise therefore becomes
L.
Var Br = o² II -=
| Ooo2. [Wi )
j=#+1
exponential in depth
indicating that deeper networks lead to an exponential escalation of noise, in sharp contrast to the linear growth forward propagation.
It is worth noting that in quantization-aware training (QAT) (Jacob et al. (018)). the Straight-Throug! Estimator (STE) (Courbariaux al. (2166) avoids layer-wise error accumulation by computing gradients on float master weights while treating fake quantization as identity. This is reasonable QAT, as the goal is to update float weights before re-quantization. In our on-device setting, weights are already quantized, with no float copy or fake quantization; both forward and backward passes operate fully in the quantized domain. Here, quantization errors still accumulate in the forward pass, and BP-based editing further compounds them exponentially.
In contrast backpropagation, Mobi Edit only accumulates noise during the forward pass. The zeroth-order gradient is estimated via the centered difference:
LA C-A
70
9
2A
where LA and L-A are the losses obtained from two independent forward passes with perturbed weights W7 + A. Each forward pass incurs the same forward-only quantization error derived above, with variance Vartw O(Lo²). Consequently, the variance of the ZO estimator is
Var[Ca] Var Var[C-a] (2
Vartw
Varlg? 201
O(LO2/A3)
(10)
4A
which grows only linearly in depth L, in sharp contrast to the exponential dependence of backprop-
agation and explaining why Mobi Edit is more robust deep quantized networks.
Quantization robustness evaluation of BP-free training. Our experiments in Table I demonstrates the fragility of ba-bdsed editing under quantization and the robustness of zeroth-order optimizations. ROME's editing success rate drops sharply from 96% in the FP32 setting to 41% % in the W8A16 setting due high gradient noise low-bit conditions. In contrast, Mob Ed achieves score of 80 under the same W8A16 guantization, demonstrating substantially higher robustness.
2.4 FURTHER OPTIMIZATIONS
Although single step MobiEdit efficient on mobile NPUs, a remaining challenge is that it requires significantly more steps to reach comparable convergence performance. Each gradient estimate in zeroth-order optimization is taken along a single perturbation direction, causing deviation from the true gradient and requiring many MobiEdit requires on average 20 more steps than ba-ba model editing on the ZsRE and CounterFact datasets, which eliminates its efficiency advantage in terms of wall-clock time.
steps reach stable convergence. For example,
5

---

Published as a conference paper at ICLR 2026
0.40
Camulativv Proportion (Unee
258t
Table Impact of quantization on edit success for ROME (BP) and KobiEdit (BP-Free).
0.39
Courterfact
0.8
0.25
0.2
8:35
3.6
Method
Precision
Edit Success Score
Proportion
0.15
9.4
0.19
ROME (BP)
FP32
96
0.05
848
ROME (BP)
W8A16
41
0.00
<300
100-200
203-300
360-600
400-500
500-600
9.0
MobiEdit (BP-Free)
FP32
86
MobiEdit (BP-Free)
W8A16
Figure 3: Distribution of editing steps for successful edits.
Early stopping. To address this, first analyze the successful editing step distribution of edited facts. As shown in Figure 3] we find that different knowledge has different editing difficulty. Based on this observation, we introduce a lightweight early stopping mechanism that adaptively determines the editing horizon based on runtime success feedback. Specifically, during editing, we periodically evaluate the model's response the edited fact every M steps (e.g., every 20 steps). The editing process is terminated early once the model satisfies pre-defined success criterion. This typically occurs when it produces the desired target output with a confidence above given threshold m. The early stopping mechanism automatically adjusts the editing steps to the complexity of each edit instance, avoiding unnecessary forward passes for easy-to-edit facts and reducing overfitting risk by stopping the point of first success.
Prefix activation reusing. In our knowledge editing setting, each optimization step uses the same set input examples constructed by combining fixed set randomly sampled prefixes with the fact to be edited. Formally, for target fact f, we define a set of editing inputs
Xaai {(pi fl fl..... [Pn. + fl}.
(11)
where Pt, Pr. are different randomly sampled prefixes.
We observe that in each step, the prefix tokens in the input do not change, and therefore their cor- responding activations are recomputed redundantly. To reduce this overhead, we introduce Prefix activation reusing: during the first step, we cache the intermediate activations for the prefix kens, and in all subsequent steps, these cached activations are inherited across editing steps without recomputation, while only the fact tokens are recomputed. This greatly reduces compute without changing the model architecture or input format.
Reusing "stale" activations from previous steps introduces accuracy-efficinccy trade-off. We observe this effect in Figure 5) which visualizes the "staleness" by showing the cosine similarity of activations between consecutive steps. While the similarity high many layers, gradually decreases in others, especially as editing progresses, indicating that the cached prefix representations are not perfectly aligned with the evolving model. The full implications of this trade-off, particularly under quantization, are analyzed in detail in Section 3.3
To avoid the staleness accumulates from halting the editing process, we re-compute the prefix acti- vations as long as the editing loss does not decrease by 0.001 over 3 steps.
EXPERIMENTS
3.1 SETUP
Baselines. We compare MobiEdit against four representative locate-and-edit methods: ROME (Meng et al. (2022)) MEMIT (Meng et al. (2023)), AlphaEdit (Fang et al. (2025)), and WISE (Wang al. )2024a)). These methods follow the same paradigm of identifying key acti- vations and injecting new knowledge into MLP layers, but differ in target granularity and update mechanisms. ROME performs single-layer editing. MEMIT extends it multi-fact scenarios. Al-

---

Published as a conference paper at ICLR 2026
phaEdit uses null-space projections for preservation, and WISE incorporates dynamic routing to FFNs that store facts.
Datasets and model. We evaluate MobiEdit on two standard datasets widely used in factual knowledge editing: ZsRE (Levy et al. (2017)) ) a zero-shot relation extraction dataset derived from WikiRE, and CounterFact (Meng al. (2022)) a curated benchmark of factual edits targeting named entities (e.g.. people, locations), with truth and counterfactual contexts. These two datasets jointly assess edit success (conrecty producing the target knowledge), locality (preserving unre- lated knowledge), and portability (applying the edited knowledge across contexts), the three key metrics for knowledge editing. We use Qwen2.5-3I3 Owenn.55-3B-Instruee (Qwen' Team (024 and Llama3.2- 3B- IB-Instruct Meta (2024)) as our target models.
Table 2: Devices used in experiments.
Device
SoC
RAM
NPU
Redmi K60 Pro Redmi K70 Pro
Snapdragon Gen 2
16GB LPDDRS
Hexagon NPU V73
Snapdragon Gen 3
16GB LPDDRS
Hexagon NPU V75
OnePlus 13
Snapdragon Elite
24GB LPDDRS
Hexagon NPU V79
Implementation details. To assess on-device feasibility, we run all editing procedures on three COTS mobile devices, as shown in Table 2 We perform all experiments using local inference engine mllm-npu (Xu et al. (2025)) optimized for NPU execution. The latency on CPU is obtained by run- ning mobile phones using Ilm.c (karpathy (2013).
(2013) We use memory swapping while reaching the memory limit. MobiEdit uses W8A16 quantization (INT8 weights, INT16 activations), a format widely supported by mobile NPUs and inference engines, ensuring compatibility and throughput. The metric of memory usage in this paper is defined as the total memory required, assuming suffi- cient memory is available. For simple comparison, the system efficiency values are first normalized to the range [40, 100) using min-max normalization, and then inverted.
3.2 END-TO-END PERFORMANCE
Homey Whireeey
Memory Dffclency
Memory Effckeery
Lecetty
Middsyy
Thee Mickeyyk
Ahd
Partabity
Busty BiTs keary
Partalio
Pote
smunney
Edi Saccems
(a) Owen on ZsRE
(b) Qwen on CF
(c) Llama on ZsRE
(d) Llama on CF
Figure 4:
The comprehensive editing ability and performance comparison of knowledge editing
methods on the ZsRE and CounterFact datasets.
Overall results. We compare MobiEdi: against all baselines across six dimensions: edit success, locality, portability, time efficiency, memory efficiency, and energy efficiency, as shown in Figure The results showing that MobiEdit achieves a balance of high accuracy and low resource cost.
On Qwen2.5-3B --n-nntuttt model, MobiEdit t achieves an 80.1 edit success score, 72.6 locality score, and 51.4 portability score while requiring only 25.5 minutes, 0.018 kJ energy, and 6.2 GB memory. On Llama3.2 Innt-Instruct model, Mobi Edi achieves an edit success score of 88.3, locality score of 44.3, and portability score of 38.3, while requiring only 23.1 minutes, 0.017 kJ energy, and 5.0 GB memory. Although slightly behind in quality, MobiEdit significantly outperforms in efficiency, reducing memory usage by more than 7x and energy consumption by over 10x. These substantial resource savings make fobiEdit the only realistically viable solution for mobile devices.
The significant performance improvement is attributed MobiEdit t leveraging a BP-free editing method, which reduces the memory and computational overhead caused by backpropagation. Ad- ditionally, Mobi Edit incorporates mobile hardware-friendly quantization and two optimizations
7

---

Published as a conference paper at ICLR 2026
specifically designed for the bp-free training step. These enhancements not only maximize the per- formance potential of mobile NPUs but also minimize the number of training steps and eliminate redundant computations.
With MobiEdit 's exceptional performance gains, although our method has drop in edit accuracy, it significantly improves the number of suc- cessful edits per unit time, which is critical for real-world usability. Since knowledge editing occupies device resources and cannot be performed while the user is actively using their phone, in practice there are only about 8 hours per day (e.g.. when the user is asleep) available for edit- ing. As shown in Table in 8 hours, our method can successfully edit 14 facts, whereas traditional methods only accomplish 2-5. For simple facts, our method requires only 2-3 minutes, while traditional methods incur fixed processing time of either 1.25 hours hours for all facts, depending the specific method used. Running traditional methods also causes a temperature increase of approximately 10 "C after just 100 seconds of sustained CPU load, resulting in device lag shutdown.
Table 3: Facts edited successfully within hours on Redmi K60 Pro.
Method
Count
ROME MEMIT
S
5
WISE
2
AlphaEdit
4
MobiEdit
14
Editing performance. Table 4 provides a detailed comparison of mem- ory, latency, and energy usage across three commercial smartphones on ZsRE dataset. All bascline methods considered in our experiments, including ROME, MEMIT, AlphaEdit, and WISE, require more than 30 GB of memory. This is due to the lack of memory optimization in llm.c for training part parameters, And their per-edit energy is ranging from 0.18J to 0.36J. For instance, WISE con- sumes 0.36J and takes 11,359 seconds on K60 for a single edit. Such workloads not only exceed memory budgets but impose intense thermal and scheduling pressure on mobile hardware.
MobiEdit consumes only 6.2GB memory under 0.03J energy per edit across all devices. completing edits in 1211 to 1902 seconds. This 10x energy reduction allows McbiEdit to run editing workloads unobtrusively in the background without interrupting the user experience or triggering thermal limits. Such sustainability is critical for real-world mobile applica- tions, where knowledge editing may be triggered interactively under tight system constraints.
Mwren..5-B-Inntreee
Llama3.2-33 Instructt MobiEdit uses 5 GB, achieving 3.1x to 6.1 lower latency and 13.6x to 27.6x higher energy efficiency than baselines. The average latency is 1411 seconds on Llama3.2- 3B compared with 1530 seconds on Qwen2.5-3B. since I.lama3.2 2 adopts wider and shallower architecture (3072 28 vs. 2048x36)) being more computationally favorable for NPUs.
Appendix A and B details that the performance gains come from our algorithm-hardwar co-design: (i eliminating backward passes, (ii) using prefix activation reusing to cut redundant computation, and (iii) aligning computation precision with NPU characteristics.
3.3 ABLATION STUDY
On Llama3.2-88 2-8t-Instr Mobi Edit fobiEEitt cates that prefix activation reusing is the main source of the quality trade-off, reducing the success score by 8.4 points, while quantization has a much milder impact, with only a 2.1 point drop.
achieves an 85.3 edit success score. Our ablation study indi-
To understand why prefix reusing has such a significant impact, we provide detailed visual analysis contrasting the activation dynamics in FP32 and quantized settings. First, in an ideal FP32 setting as Figure Sa shown, we observed that activations are extremely stable, which provided the core motivation for our prefix reusing optimization. As shown below, the similarity between consecutive steps is very high (>0.90) for most layers. However, a slight decrease in similarity can still be seen in the top and bottom layers over time, illustrating the "staleness" effect.
Second, we found that quantization significantly amplifies this instability. Figure 5b the anal- ysis in our practical W8A16 quantization with INT8 weights and INT16 activations. Several layers now exhibit consistently low similarity (the dark horizontal bands), indicating significant volatil- ity. Interestingly, these volatile layers concentrated near the input and output of the network. This aligns with observations in the quantization literature(An et al. (2025)). )) which find that out- lier features, key challenge for quantization, are most prominent in the first and last Transformer layers. Quantizing tensors with large outlier values requires a large quantization step size. Due to this large step size, small perturbation to pre-quantized activation value can easily cause it to be
8

---

Published as a conference paper at ICLR 2026
Table 4: Performance comparison of our method with NPU and other knowledge editing methods with CPU on different devices.
(a) Qwen2.553BBIInttuuc
K60
K70
OuePlus
Method
Memory (GB)
Time (8)
Energy (k3)
Energy (KJ)
Energy (kJ)
46.14
4543.78
25.13
4276.49
24.23
3252.81
18.02
MEMIT
46.14
4543.78
25.13
4276.49
24.23
3252.81
18.02
WISE
46.30
11359.44 .44
63.82
8552,99
47.16
6505.63
36.05
Alphafdii
46.14
4543.78
25.13
2276.49
24.23
3252.81
18:02
MobiEdit
6.20
1902.88
0.023
1477.67
0.018
1211.83
0.014
(b) Llama3.2-3 2-3---Intttttt
K60
K70
OnePlus
Method
Memory (GB)
Time (8)
Energy (k))
Time (6)
Energy (K)
Time
Energy (k)
34.10
4834.78
27.01
4578.66
25.23
3551.82
20.22
MEMIT
27.01
4578.66
25.23
3551.82
20.22
WISE
35.05
9668, 86
53.89
9157732
50:45
6505.63
39.32
AlphaFdit
4834.78
27.01
4578.66
25.23
3551.82
20.22
MobiEdit
5.06
1754.26
2.10
1362.26
1.71
1117.19
Table 5: Edit Quality Comparison for Llama3.2-811 Ins-trrttu on ZsRE dataset.
ROME
zo+prefix
zo+prefix-quan
Suce
Portability
51.3
50.6
33.2
38.3
Locality
57.9
66.6
56.2
66.3
Fluency
555
580
566
582
mapped to different integer after quantization. This results in large, discrete change in the final quantized activation, even though the initial perturbation was small and continuous. This effect is most pronounced in outlier-rich layers and explains the significant drop in activation similarity we observe.
This comparative analysis directly explains the source of the quality trade-off: it stems from com- pounding effect. Our optimization reuses activations that are not only stale but also noisy and im- precise due to quantization. The interaction of these two factors leads to less accurate gradient estimates. To be specific on why Portability and Locality are hurt more: These two metrics require enough precision. Portability needs to update semantic space for generalization. Locality needs to isolate the edit to avoid damaging unrelated facts. The gradient estimated from stale, quantized activations is slightly blurry. While is accurate enough ensure Edit Success, it lacks the surgical precision to perfectly remap the semantic space and prevent leaks into unrelated knowledge.
RELATED WORK
Knowledge editing. Existing methods include locate -nn-ei-- (ROME (Meng al. (2022)
MEMIT (Meng et al. (2023). AlphaFdit (Fang et al. (2025) WISE (Wang al. (2024a))) )) which
directly modify model weights; retrieval-augmented (e.g., RECIPE (Chen et al. (2024 )) which
fetch external knowledge; and meta-learning (MEND Mitchell et al. (2021)). which learns editing strategies from examples. [i et al. (2025) improves editing algorithms in unlcarning settings. While having good editing quality, these approaches rely multi-step backpropagation, leading high memory and latency. We instead target memory-efficient, -riifrienll,, quantization-compatie editing.
BP-Free training. Zeroth-order methods such as FwdLLM Xu et al. (2024) and MeZO (Mal- ladi al. (003)) estimate gradients via forward-pass perturbations and enable high-quality adap-

---

Published as a conference paper at ICLR 2026
6016
6716
010
035
4148
434
(a) FP32
Qutry 5 miirrry
1.000
Sinii only
Vikue S ENITY
3.08
3.000
A.0
210
280
240
(b) W8A16
Figure 5: Stability of inherited prefix activations over editing steps. The heatmap shows the cosine similarity between the QKV activations at each step and those cached at step 0, demonstrating that the inherited activations remain close approximation even after multiple updates.
tation, but are designed for downstream task tuming, often requiring thousands of steps, and have been applied to factual editing. our knowledge, no prior work supports robust fact injection under both forward-only and quantized constraints. Our framework addresses this gap with BP-free, quantization-awar editing for edge deployment.
By representing weights and activations in low-bit formats, quantization reduces the memory computational cost LLMs. SmoothQuant (Xiao al. (2023)) Spin- Quant (Yuan al. (2025) mitigate activation/weight outliers, enabling accurate low-bit models, while QLoRA (Dettmers et al. (2023)) combines 4-bit guantization with low-rank adapters for memory neemor--efficient fine-tuning. Unlike these inference- or fine-tuning-orice approaches, in- tegrate mixed mixeedprecissinn quantization with BP-free editing to update knowledge directly on NPUs.
Quantization.
5 CONCLUSION
We present MobiEdi bobiidit, the first mobile-compatible framework for efficient knowledge editing in large language models. It uses quantized, forward-only gradient estimation meet the compute and memory limits of commercial NPUs, and applies prefix activation reusing and early stopping for further speedup. Mobi Edit enables real-time, on-device editing of 3B models, reducing memory by 7,1x, energy by 15.8x, and latency by 72% over prior methods, making it practical solution for drer-d LLM personalization on mobile and edge devices.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
We thank the anonymous reviewers for their feedback on this work. We extend our gratitude to Prof. Nicholas D. Lane for insightful discussions and helpful feedback that greatly improved this work. This research is funded by the Beijing Municipal Science & Technology Commission, the Administrative Commission of Zhongguancun Science Park, NSFC (No. 62522202). Beijing Nat- ural Science Foundation (No. I.253005), the Shenzhen Science and Technology Program with the Major Key Project of PCL. under Grant PCI.2025A10
Grant No. JCY120241202124000 and PCI.2024A06, and in part by the Shenzhen Science and Technology Program under Grant CCJC20231211008918880
REFERENCES
Al@Meta. Llama 3 model card.
2024. URL https://github.com// 11ama/
1 1lama3 h/mmb//blb// main MMODEL CARD md
Yongqi An, Xu Zhao, Tao Yu, Ming Tang, and Jinqiao Wang. Systematic outliers in large language models. arXiv preprint arXiv:2502.06415. 2025.
Apple. Apple intelligence. http://www..ppeeccmmaaapeeinetee 2024. URL https:
www. www.a.pppccc...m.... com http:/ppppeiinellleeeeeeeee
Atilim Güneş Baydin, Barak A Pearlmutter, Don Syme, Frank Wood, and Philip Torr without backpropagation. arXiv preprint 2tXiv22202.08887 2022.
Gradients
Qizhou Chen, Taolin Zhang, Xiaofeng He, Dongyang Li, Chengyu Wang, Longtao Huang, and Hui Xue'. Lifelong knowledge editing for LLMs with retrieval-augmenned continuous prompt learning. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods Natural Language Processing, pp- 13565-13580, Mi- ami, Florida, USA, November 2024, Association for Computational Linguistics. doi: 10.18653/ v
URL https: httpss///caatthlogy..ooo
em mepppmain...
751/
Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran Yaniv, and Yoshua Bengio. Binarized neural networks: Training deep neural networks with weights and activations constrained to+ or-1. arXiv preprint MrXiv:1602.0283.0 2016.
Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. Knowledge neu- rons in pretrained transformers. In Smaranda Muresan, Preslav Nakov, and Aline Villavi- cencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 8493-8502. Association for Computational Linguistiss. 2022. doi: 10.18653/v 1/t33///1222.aetm acl lon-long.581. 2ow.://2-o222..oooo..533333 URL httpppps//aallaattooooo.oo.////p
acl acl-long.500 5817
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized LLMs. In Advances Neural Information Processing Systems (NeurIPS), 2023.
Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. A survey on rag meeting Ilms: Towards retrieval-augmeened large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '24. pp. 6491-6501, New York, NY. USA, 2024. Association for Com- puting Machinery. ISBN 9798400704901. //doi. 1145/3637522 36714470
doi: 10. 10.1145/3637528.66744777
URL https:
Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Jie Shi, Xiang Wang, Xiangnan He, and Tat-Seng Chua. Alphaedit: Null-space constrained model editing for language models. In The Thirteenth International Conference on Learning Representations. 2025. URL https: // //ppeneeview.eee Bttp://oorrmm.ddd-yyytytrruu
Jia-Chen Gu, Xiang Xu, Jun Yu Ma, Pan Lu, Zhen-Hua Ling, Kai-Wei Chang, and Nanyun Peng. Model editing harms general abilities of large language models: Regularization to the
11

---

Published as a conference paper at ICLR 2026
rescue. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 16801-16819, As- sociation for Computational Linguistics, 2024. doi: 10.18653/11/0224.cmm man.l.-.aai..944.. URL
htt.ts://ae //ccl antho 1 ogy. 2024 emn4 pammnl-mme ma 1 nn. 934/
mm
Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient integer-arithmeticoonly inference. In Proceedings of the IEFE conference on computer vision and pattern recognition, pp. 2704-2713, 2018.
karpathy. lmm.c https:/giihub..co/kaaaaatayyyy
2013.
Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. Zero-shot relation extraction via reading comprehension. In Roger Levy and Lucia Specia (eds.). Proceedings of the s Conference on Computational Natural Language Learning (CoNLL 2017), pp. 333-342, Van- couver, Canada, August 2017. Association for Computational Linguistics. doi: 10.18v/3/v// K17-1034.
Yuanchun Li, Hao Wen, Weijun Wang. Xiangyu Li, Yizhen Yuan, Guohong Liu, Jiacheng Liu, Wenxing Xu, Xiang Wang, Yi Sun, et al. Personal Ilm agents: Insights and survey about the capability, efficiency and security. arXiv preprint arXiv:2401.05459, 2024.
Zexi Li, Xiangzhu Wang, William F Shen, Meghdad Kurmanji, Xinchi Qiu, Dongqi Cai, Chao Wu, and Nicholas D Lane. Editing as unlearning: Are knowledge editing methods strong baselines for large language model unlearning? arXiv preprint 1xXi::2505.19855 2025.
Zhenyan Lu, Xiang Li, Dongqi Cai, Rongjie Yi, Fangming Liu, Xiwen Zhang, Nicholas D Lane, and Mengwei Xu. Small language models: Survey, measurements, and insights. arXiv preprint arXiv:2409.15790, 2024.
Sadhika Malladi, Tianyu Gao, Eshaan Nichani, Alex Damian, Jason D. Lee, Danqi Chen, and Sanjeev Arora. Fine-tuning language models with just forward passes. In Thirty Conference on Neural Information Processing Systems, 2023. (RLltttss///::teeeettt
net:/orum:idd-ootaaa
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. Advances in neural information processing systems, 35:17359-17377 2022.
Kevin Meng, Arnab Sen Sharma, Alex J Andonian, Yonatan Belinkov, and David Bau. Mass-editing memory in a transformer. In The Eleventh International Conference Learning Representations, 2023.
Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning. Fast model editing at scale. CoRR, 2021. URL URL
pdf
QwenTeam. Qwen2.5: A party of foundation models, September 2024. URL htttt:://t///nnll/ qwe.2.5/
Haoming Wang, Boyuan Yang. Xiangyu Yin, and Wei Gao. Never start from scratch: Expediting on-device Ilm personalization via explainable model selection. In Proceedings of the 23rd Annual International Conference on Mobile Systems, Applications and Services, pp. 154-168, 2025.
Peng Wang, Zexi Li, Ningyu Zhang, Ziwen Xu, Yunzhi Yao, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. Wise: Rethinking the knowledge memory for lifelong model editing of large language models. Advances in Neural Information Processing Systems, 777:37644-37799 2024a.
Song Wang, Yaochen Zhu, Haochen Liu, Zaiyi Zheng, Chen Chen, and Jundong Li. Knowledge editing for large language models: survey, ACM Comput. Surv., 57(3), November 2024b. ISSN 0360-0300. doi: 10. 11115/66688900 URI. wwt.hhttt.//...... //doi www./og//.... 11111119955999
Guangxuan Xiao, Zhewei Zheng, Shijie Yan, Yuxiong Liang, and Ying Lin. Smoothquant: Accurate and efficient post- post-training quantization for large language models. Proceedings of the 40th International Conference Machine Learning (ICML., pp. 38087-38099, 2023.
12

---

Published as a conference paper at ICLR 2026
Daliang Xu, Hao Zhang, Liming Yang, Ruiqi Liu, Gang Huang, Mengwei Xu, and Xuanzhe Liu. Fast on-device Ilm inference with npus. In Proceedings of the 30th ACM International Conference Architectural Support for Programming Languages and Operating Systems, Volume 1. pp- 445-462, 2025.
Mengwei Xu, Dongqi Cai, Yaozong Wu, Xiang Li, and Shangguang Wang. (FwdLLM}: Effi cient federated finetuning of large language models with perturbed inferences. In 2024 USENIX Annual Technical Conference (USENIX ATC 24), pp- 579-596, 2024.
Yuxin Yuan al. Spinquant: Lim quantization with learned rotations. In Proceedings of the International Conference on Learning Representations (ICLR), 2025.
A ABLATION STUDIES ON RUNTIME COST
A.1 ABLATION STUDY OF PREFIX ACTIVATION REUSING AND EARLY STOPPING
20
4000
ROME
3500
Time
3000
70 early stopping
70 prefix tache
2000
1500
MobiEdit
1000
Edit Success (%)
Figure 6: Edit success vs. time on ZsRE. Time is averaged across all devices.
Figure 6 presents an ablation study of the key algorithmic and system-level optimizations in framework. The basic zeroth-order method (zo) achieves moderate edit success but incurs excessive time cost, often over 4000 seconds per edit. Introducing early stopping (dynamic step controller) alone reduces average editing time by over 40%, without sacrificing accuracy. The early stopping module effectively eliminates redundant optimization once the target knowledge has been success- fully fitted. Adding prefix activation reusing further accelerates editing by another 20-30%, as ob- served across all devices. For each fact, the reusing reduces computation proportionally the ratio of the length to the total input length. MobiE Edit, which incorporates both optimizations, reduces editing time to nearly one-third of the baseline zo and achieves the best balance edit success and efficiency.
A.2 QUANTIZATION VS. NON-QUANTIZATION IN ROME
We compared MobiEdi and ROME (quantized and non-quantized) CPU and NPU as Table 6 Quantization gives little or even negative benefit for Ba-based methods on CPU. Editing time for ROME changes from 123.7s 7s to 127.1s. Without NPU utilization, Mobi Edit running the Apple CPU shows no speedup compared to ROME. Therefore, our main acceleration is observed only for Mobi i Edit t on NPU, driven by algorithm- and complementarity of each system-levee optimization in achieving practical on-device knowledge editing, delivering substantial improvements in both hardware and algorithmic efficiency.
co-design. These results highlight the necessity
A.3 COMPARISON WITH OTHER CPU BACKEND FRAMEWORK
A.3.1 COMPARISON WITH A SOTA ON- FINETUNING METHOD
To provide fair comparison against state-of-thhe-ar,, gradient-based method, compare MobiEdit with XPerT (Wang et al. (2025)), a personalized language style fine-tuning framework
on device. We compare power consumption with XPerT Wang al. (2025) while editing
13

---

Published as a conference paper at ICLR 2026
Table Editing efficiency of ROME and MobiEdit on Apple CPU, K60 CPU and Hexagon NPU V73. Quantization alone (ROME W8A16) yields no significant speedup. The major performance gains come from NPU utilization combined with seettwarr--oftwaar co-design.
Method
HW
Precision
Forward/Step
Backward/Step
Steps/Edit
Total/Edit (s)
ROME
Apple CPU
FP32
1121.3 ms
5065.1 ms
20
123.7
ROME
Apple CPU
W8A16
1048.3 ms
5305.7 ms
20
127.1
MobiEdit
Apple CPU
W8A16
429 ms
N/A
10
397
1705.8
ROME
K60 CPU
FP32
38 331 ms
182
20
5100
MobiEdit
NPU
W8A16
N/A
10 397
1746
1lama3.2-11 1B model on Snapdragon 8 Gen device. As Table shows, Our MobiEdit achieves 1.7x x speedup and is 10x more power-efficint than the BP-based XPerT under the same hardware and model conditions.
Table 7: Power Consumption Comparison with XPerT (Wang
2025
Method
Time (s)
Power (W)
XperT (CPU)
1074
2.6
Mobi Edbi (NPU)
634
0.27
A.3.2 COMPARISON WITH LLAMA CPP-BASED FORWARD-ONLY EDITING METHOD
Second, due method's forward-only feature, optimized CPU baseline for our own algorithm. We implemented this as 11 alama cpp- cpp-pcppp-leeeeeeeeeeeeeeeeeeeee eeii, providing direct comparison isolate the gains from NPU acceleration. The comparison con- ducted on Redmi K70 device using the Qwen2.5-3B model quantized to INT4. For the CPU backend, 1 llama. cpp was configured run on 8 cores and threads. Leveraging the NPU pro- vides additional 2 speedup and 5x higher energy efficiency over a top-tier CPU framework for same editing task. Together, these analyses prove our efficiency gains from a hardware-aware
leverage 1 amma. cpp create highly-
Table 8: MobiEdit performance on Optimized CPU Backend versus on NPU
Method
Time (s)
Power (W)
1lama. wla.a. epp-edit (CPU)
2423.16
4.33
MobiEdit (NPU)
1222.61
0.83
co-design that unlocks the NPU's raw power.
B THE PERFORMANCE BREAKDOWN ANALYSIS
B. COMPUTATION BREAKDOWN
To clarify the time efficiency of MobiEdit, we provide detailed breakdown for Qwen2.5-3B- Instruct on K60. Since most sequences are between and 32 tokens ZsRE dataset. we take sequence length of 24 (with 10 tokens as ) and batch size of (7 inputs with different ) for illustration. For BP-based editing methods (such as ROME): The editing time about 5100s. On CPU, a single forward pass takes 38s and single backward pass takes 182s. Each edit requires 20 steps. For MobiEdit on NPU: One knowledge edit takes approximately 1746s. A single NPU forward pass takes 752 ms. For each optimization step, use 5 perturbations, 10 tokens out of the 24, prefix activation reusing allows us to save 41 % computation. One step takes about 4.4s. According
step takes 7.5s. Since the
analysis on the ZsRE dataset, the average number of steps edit one fact is 397 step. On the example, MobiEdit achieves a speedup of about 2.92x compared to BP-based editing and match the data in Figure 5.
14

---

Published as a conference paper at ICLR 2026
B.2 IMPLEMENTATION DETAILS AND BOTTLENECK ANALYSIS
We analyze with implementation details regarding computation and memory access during forward passes of -wen2..-3BB-Instrucce on the K60 NPU. In our implementation, we effectively treat the sequence length as 1680, since each sequence contains 24 tokens and we process different sequences simultaneously. Additionally, we generate 10 distinct edit vectors from different per- turbations, all of which need to be processed during the forward pass. These edit vectors (each of size Ixhidden dimension) are treated as model inputs rather than parameters. This results in a total input length of 24x7x10=161 tokens training step. Table presents the architectural hyper- 1112225----IInsruucce
parameters of memory bandwidth of our NPU. Based on these specifications, Table [] provides breakdown of the computational workload and memory access requirements for layer a single forward pass. Our analysis reveals that most stages compute bound rather than memory bound. In practice, the NPU's 8MB VTCM cache offers significantly higher bandwidth than the LP-DDR5 memory. The cache enables more efficient access actívation values. Many activations from this high-speed cache without accessing main memory, meaning the actual memory access times are substantially faster than our theoretical calculations would suggest. Therefore, we can conclude that the prefix activation reusing delivers near-linear acceleration, significantly enhancing the overall computational efficiency of our implementation.
while Table [0 details the computational capabilities and
be served directly
Table 9: Owen2.5-3B-Immm Instructt architecture.
Hyperparameter
Value
Hidden Size
2048
Q Head
16
KV Head
2
MLP Size
1008
Layers
36
Table 10: Hardware capability of Xiaomi K60 NPU.
Autribute
Value
NPU TOPS
52
LP-DDR5 Bandwidth (GB/s)
40
Table 11: Computation and memory access analysis per forward step.
Pipeline
Gen
KV Gen
QK
PV
0
MLP Up
MLP Gate
MLP Down
Computation (GOPs)
6.563
0.820
5.383
5.383
6.563
35.273
35.273
35.273
Memory Traffic (GB)
0.007
0.002
0.044
0.024
0.007
0.029
0.029
0.022
Compute Time (ms)
0.136
0.017
0.111
0.111
0.136
0.728
0.728
0,728
Memory Access Time (ms)
0,089
0.046
0.045
0.531
0.089
0.303
0.303
0.478
Bound
Compute
Memory
Compute
Memooyy
Compute
Compute
Compute
Compute
These results highlight that the performance advantage MobiEdit comes from (i) climinating backward passes, (ii) leveraging prefix activation reusing to reduce redundant computation, and (iii) aligning with NPU compute characteristics.
C LOss CURVE
We illustrate the loss curves under three configurations: ZO, Z0+prefix activation reusing and ZO+prefix activation reusing+Quantizatioon in Figure) 7 With early stopping, all knowledge edits terminated immediately upon successful editing, Notably, prefix activation reusing demonstrated re- markable optimization acceleration. Fact exhibited 5x reduction in required training steps (from 300 to 60 steps). Although quantization introduced transient oscillations during gradient descent, it maintained final convergence accuracy, albeit with increased optimization time, as demonstrated by Fact 4's step count rising from 40 to 120.
15

---

Published as a conference paper at ICLR 2026
(a) 7.0
(b) prefix
(c) prefix quan
Figure 7: Loss curve of Llama3.2 with ZsRE
D HYPERPARAMETERS
(a) direction
(b) 3 directions
directions
300 directions
Figure 8: Different direction number loss curve.
(a) 1 direction
(b) 3 directions
(c) directions
(d) 300 directions
Figure 9: Different direction number loss curve without Cosine Anncaling Learning Rate.
We explore the hyperparameters for MobiEdit. Figure g and Figure analyze the interaction between the number of sampled directions and learning rate schedules in knowledge editing. While prior work on knowledge editing and zeroth-order optimization primarily employed static learning rates, as shown in Figure 8 our experiments highlight the significant advantage of cosine annealing, demonstrated in Figure With static learning rate, effective parameter updates require least 300 sampled directions, and optimization completely fails with only 1-5 directions due to persistent loss plateaus. In contrast, cosine annealing achieves noticeable loss reduction with just sampled direction and delivers practical editing performance with only five directions 60 improvement in sampling efficiency. This empirically confirms that adaptive learning rate scheduling inherently reduces gradient estimation noise in zeroth-order optimization, which is especially critical in low- sample regimes where static learning rates face inherent limitations.
E EXAMPLES OF PERSONAL INFORMATION INJECTION ON MOBILE
To validate MobiEdit's s effectiveness on-device personalization, we have conducted new experiments on personalization dataset. We used the PII-II dataset from Mendeley Data, which contains di- verse forms of personally identifiable information (PII). We sampled various personal facts. such as names, email addresses, and physical addresses, create a new personal benchmark. MobiEdit demonstrates strong performance on this new benchmark, successfully injecting personal facts into the LLM while maintaining its significant efficiency advantages. The Table 12 below summarizes the results for representative examples. Furthermore, to provide more insight into the editing pro- cess, we show several loss curves for these personalization edits in Figure TO The successful editing of personal information requires nearly 600 steps, which confirms its higher difficulty compared to factual editing. Nevertheless, our method can still accomplish this task successfully in under 50 minutes.
16

---

Published as a conference paper at ICLR 2026
Table 12: MobiEdit Performance on the Personal Information Dataset
Question
Before Editing
After Editting
Time (6)
Energy (kJ)
What the email address of Andrew Yoder?
(8 usmcom com
Andrew Yoder@eooppen weooeneeonnccooo com
2739.96
2.71
What the phone mumber of Andrew Yoder?
Andrew 719 523-4567
+1 555-1 155-113--00 1---122---666 43-4
2909.22
2.75
What the URI. associated with Andrew Yoder?
Andrewsften linkedin.com
http:://ww..Andk Andrew Yoder.com 3o 3011.68
2.78
Email Acdrece
Phune Numbor
235
UNL
2.5
6.5-
Figure 10: Training loss curves on the Personal Information dataset.
Table 13: MobiEdit Performance models with different size
Model
MobiEdit Success Score
ROME Success Score
MobiEdit Time (4)
ROME Time (s)
Speedup
1B
72
94
634.87 1754.60
1228.22
1.93
2.76
8B
85
94
4478.93
16982.31
3.79
F MOBIEDIT PERFORMANCE ON DIFFERENT MODEL SIZE
We evaluate Mobi Edit on Llama3.2 models of different sizes: IB, 3B, and 8B. As shown in Table 13 the IB model, Mobi Edit reaches success over ROME, with time cost of 634.87 seconds and 1228.22 seconds. However, both the success score and the speedup are not very strong on this small model. Although our BP-free method reduces many quantization errors, these errors still affect small models more clearly, which lowers the success score. Because the errors are stronger, the model also needs more editing steps to reach convergence, and the extra steps reduce the overall speedup. As the model becomes larger, Edit performs better. The success score increases from 72 on the IB model to on the model, and then to 85 on the 8B model. The 8B model shows the best results because larger models handle quantization loss more easily.
MobiEdit
of 72.0 and gives 1.93 speedup
G USE OF LLMS
This work has benefited from the assistance of large language models (LLMss). Specifically, LLMs were used for (i) language proofreading of the manuscript and (ii) generating code to assist in figure creation. The authors critically reviewed and edited all L-M-generated content before use. We are grateful for the availability of these tools, which aided in efficiency without affecting the integrity of the research.
17


Reviews:
[{"review_id": "rKX99gvbbr", "reviewer": "Reviewer_hQSe", "summary": "This paper presents MobiEdit, a framework for knowledge editing of large language models directly on commercial mobile devices. The key contribution includes replacing backpropagation-based optimization with forward-only zeroth-order gradient estimation, making it compatible with mobile NPUs, mixed-precision quantization and two system optimizations - prefix activation reusing and early stopping.", "questions": "Q1. Is it possible to provide result and comparison using the optimised CPU implementation?\n\nQ2/Q3. Same as W2 and W3\n\nQ4. Any plan to release the code.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "S1. Timely work and well motivated design decisions. All the design choices like NPU compatibility, memory efficiency, mixed precision, prefix reuse and early stopping are justified for the  mobile environment.\n\nS2.  Detailed empirical validation and system evaluation. The paper does analysis across different system metrics including energy profiling and thermal pressure.", "weaknesses": "W1. CPU baseline uses llm.c which is not an optimised implementation, as on the github of llm.c itself says it is slightly tweaked version of nanoGPT, which is a learning project. There are many optimised cpu implementation of different llms including llama.cpp [1] and many more, which authors could have used. So it is unclear if the gain is because of unoptimised cpu implementation (llm.c) vs optimised npu implementation or because of algorithm design.\n\nW2. Even though paper's major claim is related to personalised on-device llm, the two benchmarks (ZsRE, CounterFact) used are for factual editing and not personalization-specific.\n\nW3. The theoretical analysis assumes linear networks (f_l(x) = x, equation 5) which seems unrealistic for transformers with LayerNorm, GELU activations, and the claim that zeroth-order variance is \"depth-independent\" (line 252) contradicts their own equation 5 showing output noise variance $\\sigma ^2_L$ grows with depth L.\n\n\n---\n\n1. https://github.com/ggml-org/llama.cpp", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "v99zEWJORZ", "reviewer": "Reviewer_3kus", "summary": "this paper proposed MobiEdit, an on-device knowledge editing method that replaces backpropagation with forward-only zeroth-order updates in a locate-and-edit scheme, paired with NPU-friendly mixed-precision quantization that keeps only the edit-critical layers in floating point.", "questions": "N/A", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. technically sound, the forward-only zeroth-order editing with NPU-friendly mixed-precision removes backpropagation memory needs and is shown to be more robust under low-bit quantization. prefix-activation reuse and early stopping further cut compute\n3. Strong on-device results on commercial phones", "weaknesses": "1. zeroth-order editing needs much more optimization steps to reach similar convergence, without early-stopping and caching, wall-clock time can erase efficiency gains.\n2. sensitivity to hyperparameters: performance depends on the number of sampled directions (loss stability varies across 1/3/5 vs 300 directions) and on the early-stopping confidence threshold.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "v1A9lMVVar", "reviewer": "Reviewer_vhfd", "summary": "The paper presents MobiEdit, a framework for knowledge editing in large language models (LLMs) that is optimized for NPUs in mobile devices. MobieEdit works by performing a backpropagation-free, zeroth-order optimization method for factual editing using only forward passes, along with two optimizations to reduce latency and energy consumption. MobiEdit is evaluated on 2 datasets (ZsRE, CounterFact) using 2 3b models (qwen and llama) across 3 mobile devices. Mobiedit achieves up to 15.8× energy savings and 7.1× lower memory usage, with a significant trade-off in edit quality.", "questions": "- Have you tested MobiEdit with different model sizes of the same family? It would be interesting to see the performance gains/accuracy degradations on even smaller models. Additionally, given the memory reduction, could that allow for running larger models?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The paper is well written and has the following strengths:\n- The paper explores the topic of on-device LLM personalization on resource-constrained devices(mobile devices), which is an interesting and relevant topic.\n- MobiEdit is optimized towards a new trend of computing hardware(NPUs) via the use of forward-only editing.\n- The paper provides a theoretical justification showing that quantization without backpropagation (BP) is inherently more resilient to noise than BP-based quantization.\n- Compared to the given baselines, MobiEdit reduces energy consumption and memory usage by a good margin (7x-15x).\n- The paper presents an extensive set of experiments, including additional results in the appendix, which serve as solid ablation studies and help clarify the sources of the observed performance gains.", "weaknesses": "While the topic is interesting, there are still a few weaknesses that need to be addressed:\n- The novelty of the paper is somewhat incremental, as its primary contributions build heavily on prior work. The concepts of zeroth-order gradient estimation and quantization have been explored previously; however, there is still a degree of novelty in adapting and applying these ideas to NPUs.\n- No empirical experiments to support the theoretical claim that MobiEdit quantization is more robust to noise than BP-based quantization.\n- The degradation in edit quality is substantial even on relatively simple factual edits. Does the memory/energy gain justify this level of degradation?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YVXcPalMZ9", "reviewer": "Reviewer_V7a3", "summary": "This paper presents MobiEdit, the first practical framework for performing knowledge editing (KE) on large language models directly on commercial off-the-shelf (COTS) mobile devices. The work identifies that existing KE methods, which rely on backpropagation (BP), are fundamentally incompatible with mobile deployment due to (1) high memory/energy costs from storing activations, (2) instability when combined with quantization, and (3) incompatibility with mobile NPUs, which are typically forward-pass-only accelerators. MobiEdit proposed BP-Free Editing, NPU-Friendly Quantization, and System Optimizations to solve the challenges. Experiments on COTS mobile phones (e.g., Redmi K60 Pro) show that MobiEdit can edit 3B LLMs (Qwen2.5, Llama3.2) with 7.1x less memory, 15.8x less energy, and 3.4x less latency than BP-based methods, making on-device personalization feasible for the first time.", "questions": "The main trade-off in the paper is efficiency vs. quality. Your appendix (Table 5) brilliantly isolates the source of this quality drop (mostly from prefix reusing, not quantization). Could you please move this critical analysis into the main paper (e.g., in Sec 3.2 or 2.4) and discuss this trade-off more explicitly? Why do you think reusing stale prefix activations hurts portability and locality so much?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper tackles a critical bottleneck for the next generation of on-device personal AI: enabling models to learn from user interactions without resorting to the cloud. To our knowledge, this is the first work to demonstrate practical knowledge editing on commercial mobile NPUs. The \"Facts edited successfully within 8 hours\" metric (Table 3) is a killer demonstration of this practical feasibility, showing MobiEdit can edit 14 facts while ROME can only edit 2, all while avoiding device overheating. A major strength is the rigorous justification for using zeroth-order (ZO) optimization. The paper provides a clear theoretical analysis (Sec 2.3, Eq 4-9) arguing that BP-based gradient noise accumulates multiplicatively with network depth under quantization, while ZO-based noise remains constant.", "weaknesses": "The main results (Fig 5) clearly show a trade-off: MobiEdit has slightly lower \"Edit Success,\" \"Portability,\" and \"Locality\" than the (infeasible) FP32 ROME baseline. The main text does not fully analyze the source of this drop. Appendix A (Table 5) contains the crucial insight: the \"prefix activation reusing\" optimization is the main culprit, causing an 8.4-point drop in success, while quantization itself causes only a 2.1-point drop. This is a critical finding and must be discussed in the main paper, as it's the central trade-off of the work (speed vs. accuracy). The claim in Sec 2.4 that reusing stale activations \"does not negatively affect the editing outcome\" is directly contradicted by Appendix A.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)