P r A d: Prompt Adaptive Tuning for Decoder-only Language Models (2025)

Abstract: Fine tuning pretrained language models for downstream NLP tasks, while effective, can be costly when the model size and the number of tasks increase, as it requires full parameter updates and a separate model served for each task. Parameter-efficient tuning (PET) addresses the issue by keeping the pretrained parameters fixed while introducing minimal task-specific parameters. There are two essential PET paradigms: prompt-based tuning and adapter-based tuning, each with distinct limitations. Prompt-based methods suffer from increased input lengths and sensitivity to weight initialization, whereas adapter approaches can substantially increase inference time. To overcome these limitations, we propose prompt adaptive tuning (PrAd), a general prompt-based tuning framework for decode-only models that delivers strong performance with high efficiency, even in multi-task scenarios. Unlike conventional prompt-based tuning which uses soft tokens to “wrap” inputs, PrAd employs adapters for flexible input transformation. While traditional adapter-based tuning adapts both the prompt and decoded tokens, PrAd only adapts the prompt. PrAd enables the creation of diverse prompt-based approaches while providing critical advantages for real-world use: (1) it can maintain original input lengths with easy initialization during training, like adapter-based methods; (2) it can reduce management costs while facilitating deployment and efficient batch inference of different tasks, like prompt-based tuning.; and (3) it introduces no additional inference latency in the decoding phase even when serving multiple tasks concurrently. Experiments on six diverse tasks demonstrate that PrAd can consistently attain comparable or better performance and higher inference efficiency.

## introduction it has been a dominant paradigm to fine tune a pretrained language model (plm) for the transfer learning of downstream nlp tasks. though powerful, fine tuning all the parameters and serving a new tuned model for each task can be prohibitively expensive when the number of tasks and model size grow. parameter-efficient tuning methods (pets), such as prompt-based tuning @xcite , adapter tuning @xcite @xcite @xcite , and lora @xcite , represent effective approaches to addressing this issue. pets can attain high parameter sharing by only updating a small number of extra parameters for each task and keeping the pretrained parameters frozen. despite their widespread adoption, current mainstream pets exhibit distinct limitations in practical applications. prefix tuning and its variants @xcite @xcite prepend the input with additional trainable prefixes, which increases the input length and reduces the usable sequence length for downstream tasks. meanwhile, prefix tuning is hard to optimize and its performance changes nonmonotonically in trainable parameters @xcite . while adapter-based methods demonstrate superior overall performance and greater stability compared to prefix tuning @xcite , they are much less efficient in terms of batch inference across diverse tasks, and may incur more management costs and notable additional inference latency in certain scenarios. it has been shown that adapter-based methods can introduce over 20% additional inference latency in an online, short-sequence-length scenario @xcite . @xcite propose to drop the adapters in the lower layers of the pretrained models for higher training and inference efficiency. conditional adapter @xcite reduces significant computation by selecting only a small subset of input tokens to query the pretrained model. however, this method is intricate and only applicable for encoder-only models. lora @xcite introduces no additional inference latency when serving a single task by enabling the merging of its trainable parameters into the pretrained model. however, in multi-task settings, lora's efficiency benefit is compromised as task-specific adapters must be maintained separately, preventing weight merging and introducing additional inference latency. to overcome the limitations of existing pet methods, we introduce prad, a novel and efficient pet framework for decoder-only lms that delivers strong performance while maintaining high efficiency, even in multi-task scenarios. prad utilizes adapters exclusively during the refill stage. it provides several key advantages for practical utilization: 1. it can facilitate deployment and batch inference of different tasks, and reduce the management and memory costs incurred by adapters by up to 50% in practical scenarios where the prefill and decoding phases are dis-aggregated @xcite . 2. its ease of implementation and training, coupled with robust performance across various nlu and nlg tasks, are pivotal for practical adoption. 3. it does not increase the input length, thereby not reducing the available input length for downstream tasks, addressing a major limitation of existing prompt-based methods. 4. it introduces negligible additional inference latency even when serving multiple tasks concurrently. our main contributions are summarized as below: existing prompt-based approaches can also be conceptualized as specific instances of prad. ## our method we introduce prompt adaptive tuning (prad) for decoder-only lms, a novel and versatile promptbased framework which extends existing promptbased approaches while integrating the advantages of adapter-based methods. the core innovation of prad lies in its general and dynamic input transformation mechanism during the prefill phase, formally expressed as: x ′ = a(x), where "a" represents a parameterized prompt transformation module, x denotes input representation. as illustrated in figure 1 , the inference of decoder-only lms comprises of two distinct operational phases: prefill and decode. disaggregating these two phases can often lead to optimized service performance and efficiency @xcite . we decouple these phases into separate computational models -specifically, a prefill model and a decode model. prad exclusively modifies the prefill model by incorporating "a" into its structure for task adaptation, while keeping the decode model intact. notably, conventional prompt-based methods @xcite @xcite can also be regarded as special instances of prad. prad offers substantial versatility in the choice of "a", enabling the derivation of numerous prad variants. however, selecting an appropriate "a" remains crucial for achieving successful outcomes. empirical findings from existing prompt-based approaches indicate that an effective transformation "a" should create a strong dependency between the soft prefixes should be initialized using taskspecific token embeddings for more stable training @xcite . incorporating inputdependent prefixes has been shown to improve model performance @xcite @xcite . prompt-based methods are closely correlated with adapter-based methods. @xcite demonstrated that the underlying mechanism of prompt-based tuning can be can be mathematically formulated as implementing pa in the attn layer. however, unlike adapter-based methods that can employ identity initialization in the residual adapters for stable training @xcite , prefix tuning lacks the residual structure, which may partially account for the optimization difficulties observed in prefix tuning. conventional prompt-based methods primarily perform input transformations limited to the attn layer via here, "m" represents a certain sublayer of the plm that is frozen. "a" denotes an inserted trainable adapter that works together with "m". "x" denotes the input hidden states related to prompt. "@xmath0. "x ′ " and "y ′ <i " represent the output hidden states. standard adapters process both "x" and "@xmath1. in contrast, the proposed methods process "x" with "m" and "a", but process "@xmath2. building upon these theoretical insights and empirical observations, we propose the adapterbased prompt transformation, leveraging residual adapters to enable powerful input transformation, while enforcing a strong dependence relationship between x and x ′ and enabling identity initialization for stable training. when decoding the i th token of target sequence, the input is the concatenation of prompt x and the already decoded token sequence @xmath3. let us denote x ′ , y ′ <i as the corresponding outputs of x, @xmath4. a standard parallel adapter @xcite computes its output together with a pretrained module as below: applying the sequential adapter only on the prompt x yields prad-sa: ## conclusion we propose prad, a general prompt-based tuning framework for decoder-only lms. prad enables the development of diverse prompt-based methods and integrates the strengths of both prompt-based and adapter-based tuning. prad (1) maintains input length and allows simple initialization during training, while performing on par with or surpassing strong baselines (e.g., pa, lora) across diverse tasks; 2) introduces zero additional inference latency during decoding, even when serving multiple tasks concurrently; 3) facilitates deployment and batch inference of various tasks, reducing 50% management and memory costs of adapters while significantly boosting inference efficiency in multitask scenarios. with its strong performance, greater efficiency and lower management costs, prad can be a superior choice for real-world use, especially in multi-task scenarios.