Title: EMLoC: Emulator-based Memory-efficient Fine-tuning with LoRA Correction
Venue: NeurIPS 2025
Accepted: yes
Keywords: Memory-efficient fine-tuning

Low-rank approximation
Abstract: Open-source foundation models have seen rapid adoption and development, enabling powerful general-purpose capabilities across diverse domains. However, fine-tuning large foundation models for domain-specific or personalized tasks remains prohibitively expensive for most users due to the significant memory overhead beyond that of inference. We introduce EMLoC, an Emulator-based Memory-efficient fine-tuning framework with LoRA Correction, which enables model fine-tuning within the same memory budget required for inference. EMLoC constructs a task-specific light-weight emulator using activation-aware singular value decomposition (SVD) on a small downstream calibration set. Fine-tuning then is performed on this lightweight emulator via LoRA. To tackle the misalignment between the original model and the compressed emulator, we propose a novel compensation algorithm to correct the fine-tuned LoRA module, which thus can be merged into the original model for inference. EMLoC supports flexible compression ratios and standard training pipelines, making it adaptable to a wide range of applications. Extensive experiments demonstrate that EMLoC outperforms other baselines across multiple datasets and modalities. Moreover, without quantization, EMLoC enables fine-tuning of a 38B model, which originally required 95GB of memory, on a single 24GB consumer GPU—bringing efficient and practical model adaptation to individual users.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents EMLoC, a three-stage framework that enables fine-tuning of large pre-trained models under the same GPU memory budget required for inference. First, it constructs a downstream-aware emulator by applying activation-aware SVD to each weight matrix using a small calibration set, yielding a lightweight low-rank model tailored to the target task. Second, it fine-tunes this emulator via LoRA, dramatically reducing memory overhead during training. Third, it introduces a novel LoRA correction algorithm that compensates for the misalignment between the emulator and the original model by adjusting the LoRA factors before merging them back into the full model. Experiments on multimodal VQA benchmarks (ChartQA, DocVQA, InfoVQA, TextVQA, PMC-VQA, WebSRC, WC-VQA) and NLP tasks (MathQA, GSM8K) demonstrate that EMLoC consistently outperforms strong baselines—including Offsite-Tuning, row-pruning (UPop), and LORAM—under 25–50% memory budgets, and scales to 26B and 38B models on a single 24 GB GPU.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\nMemory-Efficiency Guarantee: By construction, EMLoC closes the memory gap between inference and fine-tuning without requiring specialized hardware or gradient checkpointing techniques. Users can adapt models as large as 38 B with only 24 GB of GPU RAM.\n\nDownstream-Aware Compression: Activation-aware SVD ensures the emulator retains task-relevant features, outperforming both generic SVD and row-pruning approaches while incurring minimal construction overhead (~0.3 GPU-hours vs. >200 GPU-hours).\n\nCorrection Mechanism: The proposed correction algorithm rigorously derives a basis (the SVD factors of the LoRA input matrix), computes the emulator→model output discrepancy, and clamps corrections to preserve stability—yielding substantial accuracy gains (e.g., +3–4 points over finetuned emulator alone).\n\nComprehensive Evaluation: The paper evaluates across seven VQA datasets (including challenging domain-specific ones) and two math reasoning benchmarks, as well as multiple model scales (8 B→38 B), demonstrating consistent gains and strong robustness to hyperparameters.\n\nWeaknesses:\n\nCompute Overhead Details: Although emulator construction is fast, the paper lacks explicit wall-clock or FLOP comparisons for the fine-tuning and correction stages versus baselines, which would clarify real-world training costs.\n\nTime Efficiency: EMLoC ensures efficient usage of memory at training and inference, but there lacks analysis on the time costs of EMLoC.\n\nOther Distillation Baselines: A more appropriate baseline would be a foundation model of smaller size that is capable to be finetuned within the memory budget. There lacks comparison in this aspect, and the only direct finetuning baseline is from a larger (infeasible) model and serves as an upperbound.\n\nQUESTIONS:\nFormatting: In table 5, the last row reads \"Ours\" while others reads \"EMLoC.\" Perhaps this is a typo.\n\nMore Baselines: It would be great to see more baselines on distillation or smaller foundation models when comparing EMLoC.\n\nTime Costs: Apart from memory cost, it is important to also evaluate the time cost of EMLoC. It would be less practical if the efficiency in memory usage comes at significant cost on runtime.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-21"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nEMLoC introduces a three-stage pipeline for fine-tuning large models under the same memory budget as inference. First, it constructs a lightweight task-specific emulator by applying activation-aware SVD to the original model's weight matrices using a small calibration dataset. This yields a low-rank model that retains the same layer shapes but has far fewer parameters. Second, EMLoC fine-tunes only this emulator with standard LoRA adapters using any training pipeline. Finally, to deploy the full model, a novel LoRA correction algorithm adjusts the learned adapters to compensate for differences between the emulator and the original model. Inference then runs on the full model with the corrected LoRA modules. Experiments across vision–language and NLP tasks show that EMLoC matches or exceeds baselines despite having much tighter memory, and EMLoC achieves accuracy close to that of full-model tuning.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nEMLoC directly eliminates the fine-tuning vs inference memory gap by training on a much smaller model. Its emulator preserves task-relevant knowledge via activation-aware SVD, and supports arbitrary LoRA placements. The LoRA correction is a novel, lightweight fix for model mismatch, enabling near upper-bound performance on held-out tasks. The paper is also well-presented with precise figures and ablations.\n\nWeaknesses:\n\nThe core idea of fine-tuning on a compressed model and then transferring to the full model is similar to recent work. EMLoC's novelty lies in using SVD-LLM for the emulator and a specific correction, but it builds on prior concepts of adapter transfer. The emulator relies on a small calibration set, and the efficacy of the SVD approximation without whitening (unlike SVD-LLM) may limit fidelity. Only 64 calibration examples are used. It's unclear how sensitive the results are to this choice.\n\nMeanwhile, LoRA correction involves hyperparameters ($\\lambda$) and a basis choice, but its effect is only briefly analyzed (Fig. 4 suggests that the results are stable). The method only targets model-weight memory, indicating that it does not reduce optimizer state or activation memory; therefore, it remains complementary to approaches like gradient checkpointing or 4-bit quantization. Baselines omit some solid or recent alternatives (e.g., there is no direct comparison to the LoRAM scheme or quantized fine-tuning, such as QLoRA).\nFinally, the algorithmic complexity of performing many SVDs and corrections on large models is not discussed; this might pose a practical overhead.\n\nQUESTIONS:\n1.\tThe method uses only 64 calibration examples for SVD. How sensitive are the results to this choice? Have the authors tested larger or differently distributed calibration sets? (Fig.4 shows slight $\\lambda$ sensitivity, but not data-scale sensitivity.)\n\n2.\tDid the authors compare against the recent LoRAM method or evaluate quantization-based fine-tuning? How would EMLoC perform relative to these approaches, and what are the trade-offs?\n\n3.\tThe correction algorithm is defined for linear LoRA modules $\\Lambda = W_A W_B$. Can this idea extend to other adapter forms (e.g. prefix-tuning or Mixture-of-Adapters)? Which layers receive correction, and is $\\lambda$ chosen per-layer or globally?\n\n4.\tWhat is the runtime and memory overhead of building the emulator (many SVDs) and performing LoRA correction? For very large models (38B+), is this overhead negligible compared to full training, or does it pose practical limits?\n\n5.\tThe paper notes EMLoC works without quantization. Could the authors combine EMLoC with model quantization (e.g. 4-bit LoRA) to push memory even lower? Are there reasons this was not explored or any pitfalls?\n\nOther limitations:\n\nMemory Factors Not Addressed: EMLoC focuses on model-weight memory only. Optimizer states and activations can still dominate overall usage. Thus EMLoC alone may still require techniques like sparse Adam or activation checkpointing for very deep networks. In contrast, methods like Quantized Side Tuning tackle these sources as well.\n\nAssumption of Linear Corrections: The LoRA correction enforces consistency of linear merged outputs on a subspace. This assumes the mismatch between full and low-rank models can be fixed by additive linear terms. If the compression-induced error is highly nonlinear, the correction might be insufficient, but the paper offers limited analysis of this.\n\nEvaluation Scope: Most experiments are on VQA and math QA benchmarks. It is unclear how EMLoC performs on large-scale open-domain NLP tasks (e.g. summarization, translation) or on very low-data regimes. Also, diffusion results show somewhat lower image-similarity scores than full training, suggesting possible quality trade-offs not deeply examined.\n\n\nReferences:\n\n[1] Wang, Xin, et al. \"Svd-llm: Truncation-aware singular value decomposition for large language model compression.\" arXiv preprint arXiv:2403.07378 (2024).\n\n[2] Zhang, Jun, et al. \"Train small, infer large: Memory-efficient lora training for large language models.\" arXiv preprint arXiv:2502.13533 (2025).\n\n[3] Dettmers, Tim, et al. \"Qlora: Efficient finetuning of quantized llms.\" Advances in neural information processing systems 36 (2023): 10088-10115.\n\n[4] Xiao, Guangxuan, Ji Lin, and Song Han. \"Offsite-tuning: Transfer learning without full model.\" arXiv preprint arXiv:2302.04870 (2023).\n\n[5] Zhang, Zhengxin, et al. \"Quantized side tuning: Fast and memory-efficient tuning of quantized large language models.\" arXiv preprint arXiv:2401.07159 (2024).\n\n[6] Shi, Dachuan, et al. \"UPop: Unified and progressive pruning for compressing vision-language transformers.\" International Conference on Machine Learning. PMLR, 2023.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel memory-efficient parameter-efficient fine-tuning method, with experimental results suggesting that it can effectively reduce GPU memory requirements during model fine-tuning. The authors claim that, without quantization, their method enables fine-tuning a 38B model on a consumer-grade 24GB GPU.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:​​\n\n(1) The figures are well-organized, and Figure 3 appears particularly interesting.\n\n(2) The experimental results are impressive.\n\n(3) The motivation illustrated in Figure 1 is easy to follow.\n\n​​Weaknesses:​​\n\n(1) The writing in Section 3.1 is confusing. The section title defines a memory-efficient paradigm, yet the content seems to be a formalized description of the LoRA training framework. I did not find any discussion on memory efficiency in this section.\n\n(2) I do not fully understand the role of the \"emulator\" in this paper or how it reduces training memory usage. How does it differ from LoRA? Could the authors provide a more detailed explanation?\n\n(3) In Section 3.2, the authors list three necessary conditions for memory efficiency: (a) fewer parameters, (b) flexibility and tunability, and (c) retention of downstream task knowledge. These are inherently characteristics of LoRA itself. I am particularly curious about how these three points contribute to memory efficiency, especially regarding activation/optimizer states as depicted in Figure 1.\n\n(4) The SVD-based design has already been explored in LoRand. The authors should discuss the differences between their method and LoRand in the related work section.\n\n(5) The authors provide extensive textual descriptions of LoRA modifications, and Figure 3 looks intriguing. However, the results in Table 6 suggest that the impact of LoRA modifications is marginal. Do the authors have further explanations for this observation?\n\n(6) The results in Table 4 are remarkable. However, after reading the paper, I still do not grasp why the proposed method significantly reduces training costs. Could the authors explain this in simpler terms? Additionally, refining the writing could help reduce the readers' comprehension burden.\n\n(7) E3VA [1] (from MSRA) is a representative work in memory-efficient fine-tuning. Have the authors attempted to compare training costs with E3VA?\n\n[1] Parameter-efficient is not sufficient: Exploring parameter, memory, and time efficient adapter tuning for dense predictions, ACMMM'24.\n\nQUESTIONS:\nSee above\n\nLIMITATIONS:\nSee above", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

comments: SUMMARY:
This paper proposes EMLoC, a novel emulator-based fine-tuning framework. EMLoC constructs an emulator by applying low-rank compression to the model parameters. Fine-tuning is then performed on the emulator using LoRA, and the resulting LoRA weights are corrected before being applied back to the original model for inference. This approach significantly reduces both the training time and memory footprint.

STRENGTHS AND WEAKNESSES:
Strengths 
+ The paper presents a clear motivation and a well-structured narrative. The proposed method is explained clearly and thoroughly.
+ The proposed method is practically meaningful and facilitates model fine-tuning under resource-constrained settings.
+ The method is novel, and the proposed correction technique offers valuable insights.

Weaknesses
+ Since QLoRA also reduces training costs, would it be possible to provide a direct comparison between EMLoC and QLoRA—both in terms of accuracy and memory consumption—to better demonstrate the advantages of EMLoC?
+ As a suggestion, I believe that EMLoC shares some conceptual similarities with module arithmetic, and it may be beneficial to include a discussion of this connection in the related work or appendix.

[1] Ziplora: Any subject in any style by effectively merging loras
[2] Token Compensator: Altering Inference Cost of Vision Transformer without Re-Tuning
[3] Lcm-lora: A universal stable-diffusion acceleration module

QUESTIONS:
See above

LIMITATIONS:
N/A
TITLE: Official Review
CLARITY: 4
IMPACT: 4
ORIGINALITY: 4
DATE: 2025-07-02