M e M o T une: A Measure and Moment-Driven Fine-Tuning Framework for Quantized Large Language Models (2025)

Abstract: Quantizing large language models (LLMs) is essential for reducing memory and computational costs in natural language processing. Existing methods combine quantization with parameter-efficient fine-tuning but often fail to meet practical performance requirements. This paper introduces MeMoTune, a novel fine-tuning framework for quantized LLMs. By employing a measure and moment approach within a low-rank approximation framework in probability measure space, MeMoTune optimizes the objective function for superior fine-tuning results. The update process is further refined through scaled gradient, enhancing convergence efficiency and noise robustness. Experiments on tasks like text generation, summarization, and understanding show MeMoTune significantly outperforms state-of-the-art methods, e.g. fine-tuning Llama2-13B on GSM8K improves accuracy by 5.5%, while fine-tuning DeBERTaV3-base on CoLA of GLUE increases Matthews correlation by 1.7%. The code is publicly available at: https://github.com/hddyyyb/MeMoTune.

## introduction large language models (llms) have transformed natural language processing (nlp), enabling significant advancements in various applications @xcite @xcite . despite their success, the substantial memory and energy demands of llms limit their adoption, particularly on resourceconstrained devices @xcite . quantization has emerged as a crucial technique for reducing the computational and memory overhead of llms. by approximating high-precision weights with low-precision values, quantization effectively compresses models without retraining @xcite @xcite . for instance, 4-bit quantization methods @xcite can re- full-precision pretrained weights peft lora adapter loftq quantized weights memotune quantized weights alpaca a. below is an instruction that describes a task, paired with an input that provides further context. write a response that appropriately completes the request. ## methodology we propose a measure and moment-driven finetuning framework for quantized llms (memo-tune). this approach explicitly models and learns the pairwise correlations between weights during training, leading to improved performance significantly. figure 2 is a brief illustration of memotune. we train the distribution of model parameters from the perspective of probability measure and design the corresponding back-propagation method. for a domain set x and a label set y, let f : x × r @xmath0. denote by y x the true label of instance x and by l the smooth loss function. our training objective can be expressed as the following optimization problem. ## conclusion we propose memotune, a measure and momentdriven fine-tuning framework for quantized llms, combining peft to adapt quantized llms for downstream tasks with limited memory. we introduce gaussian-distributed parameters in lora to enhance results. we analyze training distribution theoretically and propose scaled probability measure gradients. experiments show memotune surpasses existing models in nlp tasks and is ideal for lightweight devices.