Prompt and Parameter Co-Optimization for Large Language Models
Authors: Xiaohe Bo
Rui Li
Zexu Sun
Quanyu Dai
Zeyu Zhang
Zihang Tian
Xu Chen
Zhenhua Dong
Venue: ICLR 2026 Poster (2026)
Keywords: prompt-parameter co-optimization
shared-private parameterization
supervised regularization
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: This paper proposes MetaTuner, a novel framework that jointly optimizes prompts and parameters of LLMs through a knowledge-sharing mechanism and a supervised regularization loss, achieving superior performance across multiple benchmarks.

Abstract: Prompt optimization and fine-tuning are two major approaches to improve the performance of Large Language Models (LLMs).
They enhance the capabilities of LLMs from complementary perspectives: the former through explicit natural language, and the latter through implicit parameter updates. 
However, prior work has typically studied them in isolation, leaving their synergistic potential largely underexplored. To bridge this gap, in this paper, we introduce MetaTuner, a novel framework that jointly integrates prompt optimization and fine-tuning for LLM training.
Specifically, we introduce two neural networks to generate prompts and parameters, respectively, while allowing them to share a common bottom encoding layer to enable knowledge sharing.
By the guidance of the final supervised signals, our framework is optimized to discover the optimal combinations between the prompts and parameters.
Given that prompt learning involves discrete optimization while fine-tuning operates in a continuous parameter space, we design a supervised regularization loss to train our framework effectively.
Extensive experiments across diverse benchmarks show that our method consistently outperforms the baselines.  To benefit the research community, we have released our project at https://github.com/BoXiaohe/MetaTuner.

Published as a conference paper at ICLR 2026
PROMPT AND PARAMETER CO-OPTIMIZATION FOR LARGE LANGUAGE MODELS
Xiaohe " Rui Li", Zexu Sun', Quanyu Dai
Zeyu Zhang', Zihang Tian', Xu Chen', Zhenhua Dong"
'Gaoling School of Artificial Intelligence, Renmin University of China "Huawei Technologies Ltd.
(xiache, 1 rui 121200, zeyuzhang, z zhanngiann, xu chen @rwc..du............... en, Sunzexu08266gmaa com, (da quanyu, dongzhenhua) @huawei com
ABSTRACT
Prompt optimization and fine-tuning are two major approaches to improve the performance of Large Language Models (LLMs). They enhance the capabilities of LLMs from complementary perspectives: the former through explicit natural language, and the latter through implicit parameter updates. However, prior work has typically studied them in isolation, leaving their synergistic potential largely underexplored. To bridge this gap, in this paper, we introduce MetaTuner, a novel framework that jointly integrates prompt optimization and fine-tuning for LLM training. Specifically, we introduce two neural networks to generate prompts and parameters, respectively, while allowing them to share common bottom encoding layer to enable knowledge sharing. By the guidance of the final supervised signals, framework is optimized discover the optimal combinations between the prompts and parameters. Given that prompt learning involves discrete optimization while linettunig operates in continuous parameter space, we design a supervised regularization loss to train our framework effectively. Extensive experiments across diverse benchmarks show that our method consistently outperforms the baselines. To benefit the research community, we have released our project at ctttt/ubbbco///ooaao/ee///////ee
https: :ttpps:/ggiee /tt:::/gggttuuttt com BoXi hww.:///oo...oe/ee//etteeeee
INTRODUCTION
Large language models (LLMs) have demonstrated remarkable success across wide range of domains, including e-commerce (Jin et al., 2024 Peng et al. 2024 Li et al. 2024 Ren et al. 2024). education (Wang et al., 2024a: Ghimire et al. 2024 Chu et al. 2025), and social science (Wang
let al., 2025; Mou et al. 22025 Tang et al.] 2024 Piao et al. 2025). Unlike traditional machine
learning models, whose capabilities primarily stem from parameter tuning. LLMs can enhance their performance through two complementary strategies: prompt optimization and fine-tuning.
Previously, prompt optimization and fine-tuning are often treated as independent research lines. For example, in the line of prompt optimization, current methods (Liu et al. [025b]] |Austin & Chartock)
2024 Yang et al. 2024b Choi al. 2024) mainly focus on discovering or learning appropriate input context that can elicit the desired behavior from frozen language models. One group of works (e.g.. OPRO (Yang et al.) (024a)) focuses on optimizing prompts with LLMs, whereas another group (e.g., RLPrompt (Deng et al. (022)), BPO (Cheng et al. (024)) trains auxiliary model for prompt generation. In the line fine-tuning, existing approaches concentrate on better training algorithms to adjust the model parameters. Supervised Fine-Tuning (SFT) proposes to optimize the model with cross-entropy loss, while more recently, RL-based approaches such as RLHF (Li et al., (202bb) and DPO Ratailov et al. (023) are developed to optimize the model based on human feedback signals.
While the above models have shown promising results in various domains, both prompt optimization and fine-tune tuning have their own limitations. Specifically, prompt optimization typically focuses
Equal contribution.
Corresponding author.

---

Published as a conference paper at ICLR 2026
60
Aug: 20.20
Avg: 42.17
Avg: 32.56
Avg: 44.70
UPO
Accuracy (%)
50
44.
33
75
43.00
19.45
43
Accuracy (%)
BPO:
RD
22
40
SFT Avg: 38,34
1133
27.11
30
25
20
ALPrompt
OPRO
8PD
CFFO
PPO
000
KTO
10
MATH
HotPotQA
(a) Prompt optimization vs. fine-tuning.
(b) SFT with different prompts HotpotQA.
Figure 1: Preliminary analysis of prompt optimization and fine-tuning methods. In the left subfigure, we present comparison between four representative prompt optimization strategies and four popular fine-tuning methods. In the right subfigure, and further evaluate the model with different training prompts, where the corresponding results are marked with stars. Additional details for the implementation can be found in the Appendix D.I
utilize SFT as the backbone fine-tuning method
refining general instructions within prompts to elicit the capabilities of LLMs. However, it often falls short in adapting the pretrained parameters to the complex patterns present large-scale, task-specific datasets, especially when the knowledge encoded in the prompts and the model parameters may conflict. Fine-tuning, on the other hand, usually leverages manually crafted prompts as inputs during model training. However, the choice of input prompts can significantly affect fine-tuning outcomes, and existing methods offer no guarantee that these prompts are optimal for maximizing downstream performance. To further validate these statements, we conduct a series of preliminary experiments, and the corresponding results are presented in Figure As illustrated in the left subfigure, by training the LLM parameters based the datasets of MATH (Hendrycks et al.) (021)) and HotpotQA (Yang ct al. 2018). the fine-tuning methods substantially outperform the prompt optimization strategies on average. However, the effectiveness of fine-tuning is highly sensitive the choice of prompts. As shown in the right subfigure, when sub-optimal prompts are used for training, the performance degrades and can even fall below that of prompt optimization methods. Motivated by the above mechanistic and empirical analyses, a natural question arises: Can we design a unified framework to combine prompt optimization and fine-tuning to mitigate the weaknesses of each other?
Answering the above question is not straightforward. To begin with, prompt optimization and fine- tuning have different execution processes and optimization objectives. The former optimizes the model externally by finding the suitable input context to activate existing knowledge, while the latter modifies the model internally by updating parameters to better fit the target data distribution. Effectively reconciling these differences to combine the two approaches requires careful design. In addition, prompt optimization is inherently a discrete optimization problem, whereas fine-tuning operates in continuous optimization space. Combining the two, therefore, introduces mixed opti- mization challenges, such as non non-differentiable gradients. Effectively leveraging the final supervision optimization problem remains significant challenge.
signal to address this
To overcome the above challenges, we propose MetaTuner, a novel framework for joint prompt optimization and fine-tuning. Our approach first employs meta encoder to encode the input query. The resulting embedding is then fed into two separate decoders-a prompt decoder and a parameter decoder-whise simultancously generate query-specific prompts and parameters for downstream large language models. This design allows the prompts and parameters to share the common knowledge from the meta encoder, promoting their mutual improvement. Considering that prompt learning is a discrete process while the parameter training is continuous, we design supervised regularization loss solve the mixed discrete-continuouss optimization problem. Specifically, we first select the optimal rollout prompts to optimize the prompt decoder in supervised learning manner, and then share the parameters of the prompt and parameter decoders for joint differentiable training.
The main contributions of this paper are summarized as follows: (1) We propose the idea of jointly optimizing the prompts and parameters of LLMs to leverage their complementary strengths and mitigate each other's 's weaknesses. (2) To realize this idea, we introduce framework called MetaTunnr, which unifies the prompt and parameter generation processes to enhance the overall task performance. (3) We design supervised regularization objective to address the non-differentiabiliy challenge and effectively optimize the MetaTunet framework. (4) We conduct extensive experiments on MATH, GSM8K, HotpotQA, and CosmosQA demonstrate the effectiveness and robustness of our approach.
2

---

Published as a conference paper at ICLR 2026
2 PROBLEM FORMULATION
In the post-training of LLMs, typically work with input-output pair (I, y), where represents a question such as "What is the capital of France?", and corresponds to the expected answer like "Paris". Unlike traditional machine learning models, LLMs often require an additional prompt p to rephrase the input x: in way that better guides the model to generate the desired output y. In the example above, p could be "Answer the following question briefly:". Let M the language model, the overall process can be written as:
y M(p
(1)
To enhance the capability of M, there are two primary research directions: (1) Prompt optimization. The core idea of this approach is to find an optimal prompt that facilitates more accurate prediction of y. Given loss function C, the objective can be formulated as:
min PL
C(M(p,x))...)
(2)
where D= { y.) denotes the set of input-output pairs. We use the subscript in p, to indicate that the prompt input-sensitive. However, depending on the specific application, shared unified prompt across all inputs also a natural extension. (2) Fine-tuning. This method regards the LLM as traditional machine learning model, and optimizes its parameters based the given input-output pairs. In specific, the objective is formulated as follows:
min L(Mo(p. 21).
(3)
where we use A to represent the parameters of the large language model.
Previous studies have employed various strategies (Li et al. 2023b; Rafailov et al. 2023 Ethayarajh
et al. 2024 Yang et al. 2024a] Cheng et al. 2024 Deng et al. 2022 Liu et al. 2024a Meng et al.
2024) to instantiate objectives (2) and (3), respectively. Here, we take different perspective by
formulating them within unified framework, laying the foundation for letting prompt optimization and fine-tuning mutually enhance each other. Specifically. we regard the prompt as a "special
parameter", then objectives (2) and (3) can be unified by the following loss function:
min O.P.
(4)
Compared with previous studies, this loss function considers prompts and model parameters two complementary dimensions that jointly influence the prediction from x; to By optimizing them within unified objective, we are actually finding the optimal combinations between the prompts and parameters to better fit specific tasks. The loss function can help to regularize any sub-optimal choices in either the prompt or parameter space. Readers may find that the above objective can also be applied soft- soft-prompting strategies (Liuu et al.) 2024b 2022 Li & Liang. 2021 Lester et al. 2022 202T). However, this paper, we focus on generating explicit prompts, because the natural language prompts more explainable, transparent, and human- -umannfriendly.
THE METATUNER FRAMEWORK
The overall architecture of our framework be seen in Figure 2 We first design neural network with continuous parameters to generate prompts, effectively transforming the discrete optimization problem into continuous task. Next, we unify prompt optimization and fine-tuning by introducing shared meta encoder that simultaneously produces input prompts and model parameters, allowing mutual adaptation to better fit the training samples. Finally. we design supervised regularization loss to address the non-differentiable problem, enabling effective and stable framework optimization.
3.1 PROMPT GENERATION WITH CONTINUOUS NEURAL NETWORK
In practice, a prompt typically consists dozens or even hundreds of words. Generating such a prompt entirely from seratch, without any reference, poses a significant challenge. To mitigate this
3

---

Published as a conference paper at ICLR 2026
Meta Prompt
The Meta Tuner Framework
Downstream Task
Task Description
Optimized Prompt Tokens
Query Takens
query
prampt, plaase add more detailed guidance to the inttal prompt....
Prompt Decoder
Discrete Prompt Tokans
Actor Model
Hidden States
Meta Encoder
Jash decidas to tyy fliping a house. He buys house for $ed, 000 and then $50,000 repairs...
88:8 Continuous
Paramoter Decader
Initial Prompt
MM
MM
complex
Response
problem. salve
step using chein-
W22
Ag E
W. 2Ay
Sepervsion
afthought approach.
MMRRIZIIMMCC A2)
Present the final...
Task Reward Ground Truth
Figure 2: Illustration of the MetaTuner framework. The input query is first encoded by the meta encoder, and then two parallel decoders are utilized to generate the prompts and parameters, respec- tively, which are finally applied to the downstream actor model for problem solving.
difficulty, we first identify an initial prompt-alle albeit potentially sub-optimal-and and then leverage large language model (LLM) to rewrite it, thereby simplifying the prompt generation task. To reduce the effort of assigning an independent initial prompt each input vi, we let the LLM automatically explore the prompt space, starting from the same initial prompt. Formally, let G denote the model for prompt generation, then the rewritten prompt is represented as follows:
Go xx),
(5)
where we use o to indicate the learnable parameters of G. and P is the predefined initial prompt. Based equation (5) the objective (4h for LLM training is improved follows:
min 0.0
C(Me(Go 21), 2.).
(6)
where the optimizable parameters 0 and are all continuous.
3.2 JOINT PROMPT AND PARAMETER GENERATION
Intuitively, both prompts and model parameters aim to enhance the prediction of outputs yi from given inputs Fú, and therefore expected to share common underlying knowledge. Furthermore, as outlined in the introduction, prompt optimization and fine-tuning each exhibit distinct limitations. This motivates the need for principled approach that enables the two strategies to complement one another and mitigate each other's shortcomings.
To incorporate the above intuitions into model design, do not treat independent parameters. Instead, we generate them using a neural network. This approach allows us to share parameters between the prompt generation and parameter generation models, thereby establishing common foundation for both prompt optimization and fine-tuning Formally, we partition the parameters of the prompt generation model into two components: & {d.. 6p). Here, Φ. denotes the shared parameters, which are also utilized for LLM parameter generation, while Φ,, refers to the parameters exclusively used for prompt generation. We denote the model for LLM parameter generation as F, and its parameters are also separated into two parts { dy), where &, is tailored generating LLM parameters. Based on these formulation, we further improve objective to obtain the final optimization target as follows:
Σ ((MF (Co.. (p, x1), 224, 91).
min du f.......
In this target, fine-tuning and prompt optimization connected through shared-private parameteri- zation mechanism. On one hand, the shared parameters d, enable mutual regularization between the two approaches- any sub-optimal solution from can be corrected by the other under the unified loss function. On the other hand, the private parameters op and &, provide each approach with the flexibility to independently explore and identify its optimal solutions.
4

---

Published as a conference paper at ICLR 2026
3.3 OPTIMIZATION WITH SUPERVISED REGULARIZATION
In the above objective 7 the outputs of the prompt generation model 9 are discrete tokens, which makes it hard directly compute the gradient of Φp. To address this challenge, further introduce a surrogate loss based on supervised regularization as follows:
min 0..0p.9 (Z.
Σ
L(M (og (9i. (F, x.), u.). +
Σ a. (6, (aH
(8)
In this objective, the first term is identical to (7). except that the private parameters used generate the prompt (i.e., o',)) are kept fixed. The second term acts as regularizer, optimizing d, and op under supervised learning paradigm. Here, D1 denotes the original training dataset, while D2 an expert dataset in which each p; yields the correct y4 given 2;. After several optimization batches, % updated using dp. practice, one can optimize the above two terms either separately sequence or jointly single loss. We evaluate both of these implementations in experiments.
To address the above non-differentiable problem, we also explored strategies such as Gumbel- Softmax (Jang al. 2017), but these methods generally produced inferior performance. We speculate that our supervised regularizer is more effective because it provides direct supervision on the optimal prompts, avoiding intermediate softening steps approximations.
3.4 FRAMEWORK SPECIFICATION
In the above sections, we outlined the general design principles of our framework. In this section, we turn to the implementation details, with a particular focus on the key components 9 and
Specification of G. In our framework, an LLM such as Qwen2.5-7B Team 2024 is selected as the implementation of g. In specific, we use the first decoder layers as the meta encoder (i.e., Φ.). while the subsequent layers serve the prompt decoder (i.e., op)) We denote the hidden states by h4 E hg where represents the input sequence length and hg corresponds to the hidden size.
Specification of F. For training efficiency, we adopt LoRA (Hu et al., 2022) to fine-tune the downstream LLMs, with FF responsible for generating the LoRA parameters. Specifically, let AW 0g 0e denote the update matrix, where 0f E XTdMXTM and 0 E R xkM, F first share same Φ, with produce h4 After that, we derive 6% and B% from the hidden states h4 as follows:
0. MM(ReLU(MMM(W' h4)), Wh), 0 MM(ReLU(MMM(W? h4)), W
where o2 {Wh E Rdux! W E R W E grMx! w E RhaxMM) is the parameter decoder. ReLU is the activation function, and MM denotes the matrix multiplication operation.
4 EXPERIMENTS
In this section, conduct extensive experiments demonstrate the effectiveness of our framework, where we focus on the following research questions (RQ):
RQI: Can MetaTTuner achieve better performance than existing state-of-the-arr methods?
RQ2: Do the different components of our framework contribute to the final performance?
RQ3: Is the shared-private co-optimization structure in our framework effective?
RQ4: Is the supervised regularization loss effective in enhancing model performance?
ROS: Can the capabilities of our framework generalize effectively to unseen datasets?
RQ6: How does MetaTTner perform compared to the soft co-optimization approach?
In the following, we answer the above questions by reporting and analyzing the experiment results.
4.1 EXPERIMENTAL SETUP
Datasets and metrics. We evaluate MetaTuner on four datasets: MATH (Hendrycks et al. 2021 and GSM8K (Cobbe et al. 2021 for mathematical reasoning, HotpotQA (Yang et al. 2018 and
5

---

Published as a conference paper at ICLR 2026
Table 1: Comparing MetaTuner with baselines. We highlight the best and second-best results.
Qwen2.5-7B
Qwen2.5-3B
Methods
MATH
GSM8K
HotppoQQ
CosmosQA
MATH
GSM8K
HotpooQA
CosmosQA
Vanilla Methods
Qwen2.5
18.44
$1.63
19.85
36.80
17.11
49.36
3.95
25.00
Prompt Optimization Methods
RLPrompt
31.33
53.15
43.00
81.20
22.67
45.94
33.15
67.15
BPO
32,67
58.00
43.90
82.05
26.44
53.90
40.45
72.90
OPRO
22.00
75.06
25.55
69.10
21.00
62.62
10.55
69.35
CFPO
27.11
55.64
43.45
84,80
17.78
55.27
20.05
64.20
Fine-Tuning Methods
SFT
41.33
61.41
43,20
82.65
29.33
56.18
49.00
52.05
PPO
41.78
62.02
45.85
84,10
34.22
55.95
51.30
52.25
DPO
43.78
63.68
44.70
87.90
36,59
60.42
41.50
61,20
62.47
45.05
85.00
33.78
57.70
46.90
$8.70
Hybrid Methods
BetterTogether 41.56
67.93
52.30
89.80
35.78
66.80
41.10
86.60
MetaTuner-I
48.22
78.54
55.75
92.15
40.89
73.46
59.05
86.55
MetaTTuner- 48.67
78.92
54.56
92.25
41.33
73.08
58.85
87.15
CosmosQA (Li et al. 2023a) for multi-hop and commonsense question answering. Each dataset is di-
vided into training, development, and test sets, with details the splits provided the Appendix [.2] We report the exact match (EM) accuracy for MATH, GSM8K, and CosmosQA, while for HotpotQA,
we adopt the F1 accuracy following the implementation of (Bo et al. 2024 Yao et 2024).
Baselines. We compare MetaTuner with a series of state-of-the- of-the-arr art approaches, including: (1) Vanilla
Methods: Qwen2.5 (Team] 2024) is employed as an essential baseline directly answer queries under the zero-shot setting. (2) Prompt Optimization Methods: RLPrompt (Deng al., 2022). BPO (Cheng et al. 2024), OPRO (Yang et al. 2024a), and CFPO (Liu et al. 2025a). For RLPrompt and BPO, we use Qwen2.5 as the base prompt generator, while for LLM--bsed methods such OPRO and CFPO, we use state-of-the
Methods: SFT, PPO (Ouyang al. 2022). DPO (Rafailov et al. 2023), and KTO (Ethayarajh et al. 2024). For RL- based methods (PPO, DPO, and KTO), we first perform SFT as warm-up stage and then construct alignment data following SPIN (Chen et al. 2024). (4) Hybrid Methods: Better (Soylu et al.) 2024) is a hybrid method that iteratively optimizes the prompts and weights. However, it does not incorporate the parameter generation network and Knowledge-sharing mechanism. As mentioned in the Section 3.2 we optimize our model with two strategies: In Meta Tuner-I, we adopt sequential optimization strategy. where the two terms in Equation 8 are optimized alternatively. In MetaTuner-J, we optimize Equation 8 in a unified manner.
proprictary LLMs as the prompt optimizer. (3)
Implementation details. We use Qwen2. 7B and Qwen2. 5-3B as the prompt generator & and the downstream actor model M. Prior to joint training, we initiate an SFT stage to warm up G and M respectively. The training is carried out through Llama-Factory (Zheng al. 2024). with the epoch set to 1, batch size set to 64, and the learning rate tuned in (le-3, 5e-4, le-4, 5e-5 Details of the stage can be found in the Appendix D.2 We then search the depth à of d. in {K/4, K/2, K/2,3K/4/ K} to determine the model structure, where K denotes the number of decoder layers in G. In the training process, we search r.M in (4, 8, 16, 32), a in (0.1, 0.5, 0.9), the temperature for online rollouts in (0, 0.3, 0.5, 0.7, 0.9), and the learning rate in (1e-5, 5e-6, 1e-6, 5e-7). Additionally, a hyper-parameter to scale the generated LoRA before adding it to the original model weights, and search À over (0.01, 0.05, 0.1, 0.5). During the inference stage, employ greedy decoding to ensure reproducibility. All the experiments are conducted on eight 80G GPUs.
introduce
4.2 OVERALL PERFORMANCE
The overall comparison is reported in Table where several observations can be drawn: (1) The hybrid optimization methods consistently outperform standalone prompt optimization and fine-tuning methods, highlighting the complementary strengths of prompt and parameter optimization. (2) Compared to BetterTogether two variants MetaTuner achieve an average relative improvement of 10.15% and 17.08% under the 7B and 3B backbone settings, respectively. This demonstrates

---

Proportion of Shared Decoder Layers Published as a conference paper at ICLR 2026
Table 2: Ablation Studies. We highlight the most and the second-most effective modules.
Qwen2.5-7B
Qwen2.5-3B
Methods
MATH
GSM8K
HotpotQA
CosmosQA
MATH
GSM8K
HotpotQA
CosmosQA
MetaTuner (w/o F)
48.00
77.79
54.05
91.10
40.44
72.18
56.95
86,40
MetaTuner (w'o P)
46.22
78.54
53.90
91.00
39.78
72.25
$7.65
86.55
Meta/Tuner (w/ S)
46.67
77.86
53.65
91.50
40.22
72.10
57.55
86.65
Meta'Tuner
48.67
78.92
54.56
92.25
41.33
73.08
58.85
N7.15
that our design, featuring a shared meta encoder and supervised regularization loss, captures the deeper synergies between prompts and parameters. (3) On most tasks, the MetaTuner variant yields slightly stronger results than the MetaTTuner-I, suggesting that simultaneous optimization encourages consistent alignment between prompts and parameters. While on HotpotQA, the MetaTuner-I variant achieves better performance, with an average absolute improvement of 0.70% across the two backbone settings. This indicates that alternating updates may mitigate gradient interference between the two branches, leading more stable training, which is particularly beneficial for challenging optimization scenarios. For subsequent analyses, we primarily focus on the MetaTuner-J variant.
4.3 ABLATION STUDIES
In the above section, we evaluate our model as a whole. In this section, we would like to see how different components in model contribute to the final performance. In specific, compare final model with its three variants: In MetaTTaner (w/o F), we remove the fine-tuning component, that is, omitting the first term in Equation g In MetaTamer (w/o P), we remove the prompt optimization component, that is, omitting the second term in Equation g In MetaTuner (w/o S), we do not share the parameters between the prompt and parameter decoder, that is, we set 0 and equip FF with an independent d' with the same structure as G. From the results in Table 2 we can see: Both terms in our supervised regularization loss play essential roles, since removing either of them leads to an average absolute accuracy drop of approximately 0.99% and 1.12%, respectively. These findings indicate that, rather than relying on the training of single branch to search for the optimal prompt-parameter combination, jointly training both branches yields superior performance. Besides, instead of simple combination prompt optimization and fine-tuning, our framework enables two components to share common parameters in é, for mutual enhancement. The inferior performance of Metalluner (w/o S) compared to the full model demonstrates the effectiveness of this design.
4.4
EFFECTIVENESS OF THE SHARED-PRIVATE CO-OPTIMIZATION STRUCTURE
As demonstrated in the above experiments, parame- ter sharing between the fine-tuning and prompt opti- mization components is important for performance improvement. To further study the relation between the amount of shared parameters and the model per- formance, in this section, we examine how perfor- mance varies with different values of the sharing depth (introduced in Section (3.4). where is set to (K/4, K K/2, 3K /4, K) and K denotes the maxi- mum number of decoder layers in G. From the results in Figure 3 we can observe that for the 7B model, a smaller sharing ratio (K/4) yields the best results, whereas for the 3B model, larger sharing ratio (3K/4) performs better. This highlights the trade-off between the shared and private parameters: for the larger 7B model, its representation capacity is sufficiently strong, and excessive sharing may restrict the specialization of the prompt and parameter branches. Preserving more private layers therefore allows the model to better exploit its potential. In contrast, for the smaller 3B model, higher sharing ratio strengthens representational consistency between the two branches, thereby improving training stability and downstream performance.
MATH QwenZ 5-78
54
Accuracy (%)
$2
Qwen2.5-38
30
48.67
48
46.67
47.11
47.78
46
84
42
80
40.22
40.00
41.33
10.44
38
1/2
3/4
Figure 3: Performance with different propor-
tions of shared decoder layers.
4.5 EFFECTIVENESS OF THE SUPERVISED REGULARIZATION LOsS
In our framework, a key design for addressing the mixed cescrrtee-ootinnne optimization problem is the incorporation of the supervised regularizer in the objective (8) To evaluate its effectiveness,
7

---

Published as a conference paper at ICLR 2026
MATH
MATH
Accuracy (%)
Qwen2:5-77
Qwen2.5-38
Accuracy (%)
Qwen2 Qwen2.5-7B 5-78
Qwen2:5-38
50
48.67
48.67
48
56
46.89
47.33
47.11
46
24
42
41:33
41.33
40.00
39.78
38.44
8
1
Number of Rollouts per Query
Update Interval
Figure 6: Influence of the rollout number.
Figure 7: Influence the update frequency.
conduct comprehensive experiments based on four research questions: (1) Is our method superior to other well-known differentiable strategies such Gumbel-Softmaxx (2) In selecting the dataset D2. should it be derived from the optimized model itself or from a stronger model? (3) How does the number of rollout samples affect performance? (4) How does the update frequency of dp influence the final performance? In the following, we report and analyze the experiment results.
Comparing with other differentiable strategies. We compare our method with Gumbel-Softman where the cat- egorical prompt token sampling replaced with Gumbel- Softmax distribution to enable differentiable sampling. As shown in Figure 4 our method significantly outper- forms Gumbel-Softmax. The main reason is that Gumbel- Softmax relies on continuous relaxation to approximate the discrete sampling. Although it ensures differentiability in theory, such softening operations introduce gradient bias, that the generated prompts may deviate semanti- cally from the truly optimal discrete solutions. In contrast, methods directly optimize space, avoiding potential approximation errors.
MATH
Accuracy ccy (%)
50
48.67
MetaTuner Gumbel Softmax
45.56
44
42
41.33
39.33
38
7B
3B
Model Size
Figure 4: Comparison results between method with Gumbel-Soffi Softmax.
the discrete action
Comparison between different generation methods of D2. We compare the performances by generating D2 using the proprietary expert LLM the optimized model itself. As shown in Figure 5 while the expert method improves training efficiency, our model achieves better performance. The key reasons are threefold: first, our model ensures distribution alignment by generating data directly from the current policy, thereby reducing distribution shift between training and inference; second, it provides more targeted feedback, as rewards are based on the model's s own outputs and directly address its weaknesses, unlike expert samples that may overlook error patters; third, our model enables stronger exploration, allowing the policy to continuously discover new prompts and iteratively refine itself through task feedback, whereas the expert data remain static and limit long-term improvement.
MATH
Accuracy arrayy/ (%)
48.67
On-policy Off-policy
46.89
46
44
41.33
40
40.00
38
3B
Model Size
Figure 5: Comparison results between different methods for generating D2.
Influence of the number of rollout samples. When building D2, an important factor is how many rollout samples should be generated for each query. this section, we study the relation between this factor and the model performance. The results in Figure that as the number of rollout samples increases, the performance of both model sizes declines. On one hand, over-exploration may cause the model to frequently adjust its strategy, and these adjustments may not lead to short-term performance improvements, potentially interfering with valuable information the model has already learned. On the other hand, excessive exploration can cause overfitting, making the model overly sensitive to noise or irrelevant features in training data, which ultimately degrades the downstream performance. Therefore, the number of rollout samples should be carefully adjusted according the specific task and model characteristics to avoid the negative impact of excessive exploration.
Influence of the update frequency of Φp. In the first term of our objective (8) the private parameters 6, for prompt generation is updated in an asynchronous manner to ensure that the objective fully differentiable. Here, investigate the influence of different update frequencies and report the results in Figure 7 On the horizontal axis, the values indicate how many training steps elapse before o, used for rollout sampling is replaced with the most recently trained one. The results on both model sizes
8

---

Published as a conference paper at ICLR 2026
Owen2.5-78
Qwen2.5-38
Accuracy (%)
72
96
88
80
Metaluner
78.92
on
92.25
82.80
Accurac (%6
80
96
Meta Tuner
73.08
Co-Optimization
64
64
56
46.6
54.56
53.30
49.35
50.65
43.65
48
80
38.67
40
26.22
MATH
GSM8K
HotpotQA CosmosQA
MATH
GSM8K HotpotQA CosmosQA
Figure 9: Comparing MetaTuner with Soft Co-Optimization method.
consistently show that sparser updates lead to poorer performance, since frequent updates allow the sampled prompts to be closely aligned with the current actor model, thereby providing more effective optimization signals. Moreover, we observe that the 3B model exhibits greater performance drop, which suggests that smaller models are more sensitive to the timeliness of generated prompts, whereas the 7B model, with its stronger representational capacity, is more resilient to such lag effects.
4.6
GENERALIZATION ANALYSIS
MATH
In the above sections, we evaluate our model by training and testing on the same dataset. To further study the generalization capability of our method, conduct aut-of-distribuun
Accuracy (%)
80
MetaTuner Togethr
BPO
BE
DPO
70
60
59.82
58.38
64.06
61
38.07
experiments by training the model MATH, HotpotQA, and CosmosQA, and evaluate on GSM8K. For comparison, we se- lect DPO, BPO, and BetterTogether
52.00
50
48 98
the baselines. From the results in Figure 8 we can see: MetaTuner outperforms all other methods, confirming the advan- tage of joint optimization of prompts and parameters enhancing the generalization ability. Compared to Botter'Tggether, MetaTTuner achieves better synergistic optimization, thereby adapting more effec- tively to complex and unseen tasks. DPO, as a fine-tuning method, although effective in parameter adjustment, lacks support in prompt selection, while BPO, which relies solely prompt optimization and fails to fully leverage the model's capacity, both lead to poorer generalization performance.
40
7B
3B
Model Size
Figure 8: Generalization experiment.
4.7 COMPARISON WITH SOFT CO-OPTIMIZATION METHOD
We compare MetaTTuner with soft co-optimization methods and report the results in Figure 9 We employ prefix tuning ([. & Liang 2021) for soft prompt optimization, while using LoRA as the fine- tuning approach, and fine-tune the model with SFT. From the results, we can find that the optimization of discrete prompts is more stable. Existing work (Bhardwaj al. 2022 Razdaibiedina al. 2023) has shown that soft prompt tuning can suffer from "representation collapse". where prompts for different inputs converge nearly identical vectors. In MetaTuner, optimize discrete prompts, i.e., prompts must take the form of readable natural language, with each token subject to clear semantic constraints. This forces the optimization process to search over structured, innaannnteeppretable instructions rather than arbitrary continuous vectors, thereby reducing the risk of overfitting.
5 RELATED WORK
Fine-tuning. Fine-tuning a standard approach for adapting pre-trained language models specific tasks. Recent advances in parameter-efficien fine-tuning (PEFT) scale this to L.MMs by training small plug-in modules under frozen backbone. Representative methods include Adapter Tuning (Houlsby ct al. 2019), which inserts lightweight adapter layers, and LoRA (Hu et al.) 2022). which applies low-rank updates to weight matrices. Building on these, recent works propose various post- traintrainng
strategies (Ethayarajh et al. 2024 Guo et al. 2024 Jung al. 2024 Xu et al. 2024 Yang et al.)
2024a). For instance, RLHF eL al. 2023b) incorporates human feedback to improve alignment,
while DPO (Rafailov al.
2023) streamlines preference modeling through contrastive learning.
Prompt optimization. Prompt optimization has become a research hotspot with the rise of in-context
learning. Early works focus continuous prompt tuning (Li & Liang 2021 Lester al. 2022

---

Published as a conference paper at ICLR 2026
al. 2024b; 2022 Lester al. 2021) where learnable vectors attached to the input. To mitigate the interpretability challenge (Wen et al. 2023), recent studies turn to discrete prompt optimization, which generates prompts in natural language. Existing approaches fall into two categories. The first employs LLMss as prompt optimizers (l.iu et al.
2025b). typically in a feedhack-driven manner, via
strategies such as reflection-basee refinement
Yang et al. 2024a b. Austin & Chartock (2024)
Monte Carlo Tree Search (MCTS) (Wang et al.
2024b Pryzant et al. [023 Liu et al. (025a) The
second (Kong et al. 2024 Juneja et 2024 Choi al.
2024 Wang et al. 2024bj Chen al. 2023)
trains an auxiliary model to generate better prompts, most often with reinforcement learning, since prompts lack explicit ground truth and can naturally guided by the task rewards.
Joint optimization. Recently, BetterTogether (Soylu et al. 2024) proposes to update prompts and model weights together to improve the task performance. In petterTogethen prompt optimization and fine-tuning correspond to discrete and continuous optimization problems, respectively. During the prompt optimization phase, it samples candidate demonstration sets from the training data, evaluates them validation set to select the best few-shot prompt. In contrast, the fine-tuning phase is formulated as continuous optimization problem, where supervised fine-tuning is applied to update the model parameters for better task adaptation. Different from Better Together we first transform the discrete-conee continuous optimization into a fully continuous problem, then propose a knowledge sharing mechanism and supervised regularization loss to enable effective end-to-end co-optimization.
6 CONCLUSION
In this paper, we propose MetaTuner, novel framework that effectively combines prompt optimiza- tion and fine-tuning to enhance the performance of LLMs. We first introduce prompt generator to transform the a dual-branch framework with a shared meta encoder to generate prompts and parameters simul- taneously. We further propose supervised regularization loss to address the non-differentiability challenge, thereby enabling stable end-to-end training four benchmarks demonstrate the effectiveness and robustness of MetaTuner for joint optimization.
continuous optimization problem into fully continuous one, and then design
framework. Extensive experiments on
ACKNOWLEDGEMENT
This work is supported in part by National Natural Science Foundation of China (No. 62472427 and No. 62422215), Major Innovation Planning Inter-disciplinary Platform for the "DoubleFirst Class" Initiative, Renmin University of China, Public Computing Cloud, Renmin University of China, fund for building world-class universities (disciplines) of Renmin University China.
REPRODUCIBILITY STATEMENT
We take several steps to ensure the reproducibility of our work. Firstly, we detail the joint optimization objective and supervised regularization loss in Sections 3.2 and 3.3] and present the architcctural details in Section 3.4 These descriptions are intended to make both the methodological design and implementation choices transparent. Secondly, we document the experimental setup in Section 1 including datasets, evaluation metries, and training configurations, enable the replication of the training and evaluation procedures. We further elaborate on data splits, the warm-up stage, and the meta-prompt design in the Appendix t.2 facilitate replication. We believe these efforts provide the community with sufficient resources to reproduce our work and extend it in future research.
ETHICS STATEMENT
The development and deployment frameworks like MetaTuner, which aim to enhance the perfor- mance of large language models through joint prompt-parameter optimization, raise several ethical considerations. First, as with any system based on large-scale language models, there a risk of inheriting amplifying biases present in the training data. If not carefully addressed, the improved adaptability of MetaTuner could inadvertently reinforce harmful stereotypes or generate misleading information. Second, the computational resources required to train and fine-tune large language models contribute to environmental concerns related to energy consumption and carbon footprint. Future work should explore ways further reduce the resource demands.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Derek Austin and Elliott Chartock. Grad-sum: Leveraging gradient summarization for optimal prompt engineering, 2024.
Konstantina Bairaktari, Guy Blanc, Li- Li-Yang Tan, Jonathan Ullman, and Lydia Zakynthinou. Multitask learning via shared features: Algorithms and hardness. In The Thirty Sixth Anmual Conference Learning Theory, pp. 747-772. PMLR, 2023.
Jonathan Baxter. A model of inductive bias learning. Journal of artificial intelligence research, 12: 149-198, 2000.
Rishabh Bhardwaj. Amrita Saha, Steven C. H. Hoi, and Soujanya Poria. Vector-quantized input- contextualized soft prompts for natural language understanding. In Yoav Goldberg. Zornissa Kozareva, and Yue Zhang (cds.), Proceedings of the 2022 Conference on Empirical Meth- ods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, De- cember 7-11, 2022, pp. 6776-6791. Association for Computational Linguistics, 2022. doi: (188655//1/2022.MMMNALL emn 1p www.a.in.....
455. URL https: wtg./o/g//o....... 11..111..16311/0000
Xiaohe Bo, Zeyu Zhang, Quanyu Dai, Xueyang Feng, Lei Wang, Rui Li, Xu Chen, and Ji-Rong Wen. Reflective multi-agent collaboration based on large language models. In Advances in Neural Information Processing Systems 38: Annual Conference Neural Information Processing Systems 2024, NeurlPS 2024, Vancouver, BC, Canada, December 10 15, 2024, pp. 138595 138631, 2024.
Yuyan Chen, Zhihao Wen, Ge Fan, Zhengyu Chen, Wei Wu, Dayiheng Liu, Zhixu Li, Bang Liu, and Yanghua Xiao. MAPO: boosting large language model performance with model-adaptive prompt optimization. In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, pp. 3279-3304, 2023.
Zixiang Chen, Yihe Deng, Huizhuo Yuan, Kaixuan Ji, and Quanquan Gu. Self-play fine-tuning converts weak language models strong language models. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024, 2024.
Jiale Cheng, Xiao Liu, Kehan Zheng, Pei Ke, Hongning Wang, Yuxiao Dong, Jie Tang, and Minlie Huang. Black-box prompt optimization: Aligning large language models without model training. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024, pp. 3201-3219, 2024.
Yunseon Choi, Sangmin Bae, Seonghyun Ban, Minchan Jeong, Chuheng Zhang, Lei Song, Li Zhao, Jiang Bian, and Kee-Eung Kim. Hard prompts made interpretable: Sparse entropy regularization for prompt tuning with RL. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACI. 2024, Bangkok, Thailand, August 1 11-6, 2024, pp. 8252-8271, 2024.
Zhendong Chu, Shen Wang, Jian Xie, Tinghui Zhu, Yibo Yan, Jinheng Ye, Aoxiao Zhong, Xuming Hu, Jing Liang, Philip S. Yu, and Qingsong Wen. Llm agents for education: Advances and applications, 2025.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems, 2021.
Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric P. Xing, and Zhiting Hu. Riprompt: Optimizing discrete text prompts with reinforcement learning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pp. 3369- 3391, 2022.
Carlo D' Eramo, Davide Tateo, Andrea Bonarini, Marcello Restelli, and Jan Peters. Sharing knowledge in multi-task deep reinforcement learnnn. arXiv preprint arXiv:2401.099 009561,,, 2024.
11

---

Published as a conference paper at ICLR 2026
Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. Kto: Model alignment as prospect theoretic optimization, 2024.
Aashish Ghimire, James Prather, and John Edwards. Generative ai in education: A study of educators' awareness, sentiments, and influencing factors, 2024.
Shangmin Guo, Biao Zhang, Tianlin Liu, Tianqi Liu, Misha Khalman, Felipe Llinares, Alexandre Rame, Thomas Mesnard, Yao Zhao, Bilal Piot, Johan Ferret, and Mathieu Blondel. Direct language model alignment from online ai feedback, 2024.
Dan Hendrycks. Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song. and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurIPS Datasets and Benchmarks 2021, December 2021, virtual, 2021.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski. Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for NLP. In Proceedings of the 36th International Conference on Machine Learning. ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pp. 2790-2799, 2019.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022, 2022.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparamcterization with gumbel-softmax. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26. 2017, Conference Track Proceedings. OpenReview.net, 2017. URL hattp:://opereve net/ met/foruma BerrumiddrrEEy888
Yilun Jin, Zheng Li, Chenwei Zhang, Tianyu Cao. Yifan Gao, Pratik Jayarao, Mao Li, Xin Liu, Ritesh Sarkhel, Xianfeng Tang, Haodong Wang, Zhengyang Wang, Wenju Xu, Jingfeng Yang, Qingyu Yin, Xian Li, Priyanka Nigam, Yi Xu, Kai Chen, Qiang Yang, Meng Jiang, and Bing Yin. Shopping MMLU: A massive multi-task online shopping benchmark for large language models. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 15, 2024, 2024.
Gurusha Juneja, Nagarajan Natarajan, Hua Li, Jian Jiao, and Amit Sharma. Task facet learning: structured approach to prompt optimization, 2024.
Seungjae Jung, Gunsoo Han, Daniel Wontae Nam, and Kyoung- Woon On. Binary classifier opti- mization for large language model alignment, 2024. URL htttt:tt///xxx/// wwtp:/iiwwg///////eaaaaa
04656
Yilun Kong. Hangyu Mao, Qi Zhao, Bin Zhang, Jingqing Ruan, Li Shen, Yongzhe Chang, Xueqian Wang, Rui Zhao, and Dacheng Tao. Qpo: Query-dependent prompt optimization via multi-loop offline reinforcement learning. 2024.
Brian Lester, Rami AI-Rfou, and Noah Constant. The power of scale for parameter-effiieet prompt tuning. In Proceedings of the 2021 Conference Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pp. 3045-3059, 2021.
Brian Lester, Noah Constant, and Rami Al-Rfou. Guiding frozen language models with learned soft prompts. Google Al Blog. 2022.
Shiyang Li, Jianshu Chen, and Dian Yu. Teaching pretrained models with commonsense reasoning: preliminary kb-based approach, 2023a.
Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. ACL/JCNLP 2021, (Volume
1: Long Papers), Virtual Event, August 1-6, 2021, pp. 4582 4597, 2021.
12

---

Published as a conference paper at ICLR 2026
Yangning Li, Shirong Ma, Xiaobin Wang, Shen Huang, Chengyue Jiang, Haitao Zheng, Pengjun Xie, Fei Huang, and Yong Jiang. Ecomgpt: cooructiio-tuninng large language models with thain-of-tane tasks for e-commerce. In Thirty-Eightt AAAI Conference Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium Educational Advances in Artificial Intelligence, FAAI 2014, February 20-27, 2024, Vancouver, Canada, pp. 25582-18590, 2024.
Zihao Li, Zhuoran Yang, and Mengdi Wang. Reinforcement learning with human feedback: Learning dynamic choices via pessimism, 2023b.
Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024, 2024.
Shih- Shih-Yang Liu, Chien-Y Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang- Ting Cheng, and Min-Hung Chen. Dora: Weight- decomposed low-rank adaptation. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 2 21-27, 2024, 2024a.
Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics ( (Volume 2: Short Papers). ACL 2022, Dublin, Ireland, May 22-27, 2022. pp. 61 68, 2022.
Xiao Liu, Yanan Zheng. Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. GPT understands, too. Al Open, 5:208-215, 2024b.
Yuanye Liu, Jiahang Xu, Li Lyna Zhang, Qi Chen, Xuan Feng, Yang Chen, Zhongxin Guo, Yuqing Yang, and Peng Cheng. Beyond prompt content: Enhancing LLM performance via content-format integrated prompt optimization. CoRR, abs/2502.04295. 2025a. doi: 10.48500AARXIV2202.009999 URL http:///ooo..oo/.0.44450/05///// wwt.aiii......22..99.
Yuchi Liu, Jaskirat Singh, Gaowen Liu, Ali Payani, and Liang Zheng. Towards hierarchical multi- agent workflows for zero-shot prompt optimization, 2025b.
Andreas Maurer, Massimiliano Pontil, and Bernardino Komera-Paredess The benefit of multitask representation learning. Journal of Machine Learning Research, 17(81):1-32, 2016.
Fanxu Meng, Zhaohui Wang, and Muhan Zhang. Pissa: Principal singular values and singular vectors adaptation of large language models. In Advances Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurlPS 2024, Vancouver, BC, Canada, December 10 15, 2024, pp. 121038-121072 2024.
Xinyi Mou, Chen Qian, Wei Liu, Xuanjing Huang, and Zhongyu Wei. Ecolang: Efficient and effective agent communication language induction for social simulation, 2025.
Long Ouyang, Jeffrey WU, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.). Advances in Neural Information Processing Systems 35: Anmual Conference on Neural Information Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28- December nips. cc/ cc/paper files hhtps/aaeer///e 20222/ //hah// Abst
9, 2022, 2022. URL /11134444339944444 0eeee333111111111 - Confeectttoooeooececeo.cocc html
Wonpyo Park, Paul Hongsuck Sco, Bohyung Han, and Minsu Cho. Regularizing neural networks via stochastic branch layers. In Asian Conference on Machine Learning, pp. 678-693 PMLR, 2019,
Bo Peng, Xinyi Ling, Ziru Chen, Huan Sun, and Xia Ning. ecellm: Generalizing large language models for e-commerce from large-scale, high-quality instruction data. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024, 2024.
13

---

Published as a conference paper at ICLR 2026
Jinghua Piao, Yuwei Yan. Jun Zhang, Nian Li, Junbo Yan, Xiaochong Lan, Zhihong Lu, Zhiheng Zheng, Jing Yi Wang. Di Zhou, Chen Gao. Fengli Xu, Fang Zhang, Ke Rong, Jun Su. and Yong Li. Agentsociety: Large-scale simulation of llm-driven generative agents advances understanding of human behaviors and society, 2025.
Reid Pryzant, Dan Iter, Jerry Li, Yin Tat Lee, Chenguang Zhu, and Michael Zeng. Automatic prompt optimization with "gradient descent" and beam search. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10. 2023, pp. 7957-7968. Association for Computational Linguistics, 2023, doi: 1L18653/V1/20223 EMNLP-MAIN.NA4 URL https 7/doi.org/ wwt..e11663vv11/ emm main. 494
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In Advances Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023.
Anastasia Razdaibiedina, Ashish Khetan, Zohar Kaminn Daniel Khashabi, and Vivek Madan. Representation projection invariance mitigates representation collapse. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguis- tics: EMNLP 2023, Singapore, December 6-10, 2023, pp. 14638-14664. Association for Computational Linguistics, 2023. doi: 75. www.n.nninnnnnn...n..mmmmmmm 975
URL
Qingyang Ren, Zilin Jiang, Jinghan Cao, Sijia Li, Chiqu Li, Yiyang Liu, Shuning Huo, Tiange He, and Yuan Chen. A survey fairness of large language models in e-commerce: progress, application, and challenge, 2024.
Dilara Soylu, Christopher Potts, and Omar Khattab. fine-tuning and prompt optimization: Two great steps that work better together. In Proceedings of the 2024 Conference Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 12-16, 2024, pp. 10696-10710, 2024.
Jiakai Tang, Heyang Gao, Xuchen Pan, Lei Wang, Haoran Tan, Dawei Gao, Yushuo Chen, Xu Chen, Yankai Lin, Yaliang Li, Bolin Ding, Jingren Zhou, Jun Wang, and Ji-Rong Wen. Gensim: A general social simulation platform with large language model based agents, 2024.
Qwen Team. Qwen2.5: party of foundation models, September 2024. URL htttps://wwennnn hwwplloggwwwee5/.//
Lei Wang, Heyang Gao, Xiaohe Bo, Xu Chen, and Ji-Rong Wen. Yulan-onesim: Towards the next generation of social simulator with large language models, 2025.
Shen Wang, Tianlong Xu, Hang Li, Chaoli Zhang, Joleen Liang, Jiliang Tang, Philip S. Yu, and Qingsong Wen. Large language models for education: A survey and outlook, 2024a. URL
https:://axxt org/ abs 2403. 18105
Xinyuan Wang, Chenxi Li, Zhen Wang, Fan Bai, Haotian Luo, Jiayou Zhang, Nebojsa Jojic, Eric P. Xing, and Zhiting Hu. Promptagent: Strategic planning with language models enables expert-level prompt optimization. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024, 2024b.
Yuyan Wang, Zhe Zhao, Bo Dai, Christopher Fifty, Dong Lin, Lichan Hong, Li Wei, and Ed H. Chi. Can small heads help? understanding and improving multi-task generalization. In Frédérique Laforest, Raphaël Troncy, Elena Simperl, Deepak Agarwal, Aristides Gionis, Ivan Herman, and Lionel Médini (eds.), WWW '22: The ACM Web Conference 2022, Virtual Event, Lyon, France, April 25- 29, 2022, pp. 3009 3019 ACM, 2022. doi: 10.11457388444.3552222 URL https: doi 11t.a//.1/0....1111...1..11 305.///444444..7 3512022
Yuxin Wen, Neel Jain, John Kirchenbauer, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Hard prompts made easy: Gradient- based discrete optimization for prompt tuning and discovery. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, 2023.
14

---

Published as a conference paper at ICLR 2026
Haoran Xu, Amr Sharaf, Yummo Chen, Weiting Tan, Lingfeng Shen, Benjamin Van Durme, Kenton Murray, and Young Jin Kim. Contrastive preference optimization: Pushing the boundaries LLM performance in machine translation. In Forty-first International Conference Machine Learning, ICML. 2024, Vienna, Austria, July 21-27, 2024, 2024.
Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoe V. Le, Denny Zhou, and Xinyun Chen. Large language models as optimizers. The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna. Austria, May 7-11, 2024, 2024a.
Muchen Yang. Moxin Li, Yongle Li, Zijun Chen, Chongming Gao, Junqi Zhang, Yangyang Li, and Fuli Feng. Dual-phase accelerated prompt optimization. Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024, pp. 12163-12173, 2024b.
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. Hotpotqa: dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium, October 31 November 2018 pp. 2369-2380, 2018.
Weiran Yao, Shelby Heinecke, Juan Carlos Niebles, Zhiwei Liu, Yihao Feng, Le Xue, Rithesh R. N., Zeyuan Chen, Jianguo Zhang, Devansh Arpit, Ran Xu, Phil Mui, Huan Wang, Caiming Xiong, and Silvio Savarese. Retroformer. Retrospective large language agents with policy gradient optimization. In The Twelfth International Conference Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024, 2024.
Qiyuan Zhang, Fuyuan Lyu, Zexu Sun, Lei Wang, Weixu Zhang, Wenyue Hua, Haolun Wu, Zhihan Guo, Yufei Wang, Niklas Muennighoff. al. A survey on test-time scaling in large language models: What, how, where, and how well? arXiv preprint arXiv:2503.22 242355,,,22,,,.. 2025.
Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, and Zheyan Luo. LlamaFactory: Unified efficient fine-tuning 100+ language models. In Yixin Cao, Yang Feng, and Deyi Xiong (eds.). Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pp. 400-410 Bangkok, Thailand, August 2024. Association for Computational Linguistics.. doi: 10.18653/ 1/2024..12222221. acl--emos..ess3 URL https: httppp///cclammcoolooorrr/////// acl aclddmoss..........mmmmmm 38/l
15

---

Published as a conference paper at ICLR 2026
Table 3: Main results with more datasets and base models.
Checkmate Move
Base Model
Qwen2.5.BB
Qwen2.5-7B
Lluma3.2-38
LJama3.2-38
Mistral-7B
Vauilla Methods
Base Model
0.00
0,00
0.90
21.33
25.11
10.89
Prompt Optimization Methods
RI Prompt
0.00
0.00
0.00
24.00
25.56
12.00
BPO
0.00
0.86
1.43
26.22
26.22
12.44
OPRO
0.00
0.57
1.14
25.77
26.22
11.78
CFPO
1.43
25.33
27.11
11.56
Fine- Tuning Methods
SFT
13.71
18.00
18.85
24.44
33.11
14.00
PPO
14.00
18.29
19.43
23.78
32.22
14.67
18.86
19.14
26.44
34.00
16.00
KTO
14.57
18.57
20.00
29.78
33.56
13.78
Hy brid Methods
BetterTogethe
15.14
19.71
20.57
30.44
34.44
16.44
Meta Tuner-J
21.43
31.78
18.44
A LLM USAGE
The usage of LLMs in this work includes two aspects: (1) Writing assistance. We use LLMs to improve the fluency and clarity of certain parts of the manuscript. All generated text is carefully reviewed, revised, and validated by the authors to ensure accuracy. (2) Dataset construction. We employ LLMs as auxiliary tools to generate candidate prompts as described in the Appendix After task-reward filtering, the collected data are used to improve the open-source smaller models' ability to follow prompt-rewriting instructions. All scientific claims, methodological designs, implementations, analyses, and conclusions are original contributions of the authors. LLMs are used solely as supporting tools, and the authors take full responsibility for the integrity and accuracy the work.
B LIMITATIONS
While MetaTuned presents a promising framework for jointly optimizing prompts and parameters, there are several limitations that point to potential directions for future research. First, the RL-based approach used to address the con-differentiabiiity issue in prompt optimization relies heavily on task reward signals (e.g., 0/1 accuracy). Such signals be overly sparse in certain tasks, which may hinder optimization efficiency. Future work could explore more fine-grained reward designs, such model uncertainty, to provide richer and more informative optimization signals. Second, MetaTTuner generates a dedicated set of prompt and parameters for each query, which may face computational challenges resource-constrained environments. A promising direction is integrate the Mixture of Experts (MoE) paradigm: generate and maintain a fixed pool of expert LoRA modules, and dynamically route each query to the most suitable expert(s). This design could avoid generating new LoRA for every input, thereby improving the efficiency and scalability.
C ADDITIONAL EXPERIMENTS
C.1 MAIN RESULTS WITH MORE DATASETS AND BASE MODELS
To further validate the effectiveness of MetaTuner across broader scenarios and model archi- tectures, we conduct additional experiments on the Checkmate in One Move dataset us- ing multiple base models, including Llama3.2-3B (Llama-3, 2 Llama3. 1-BB Settral-7B-Inntruee
(Llama-3. 1-8B-Inst ruct), and Mistral-7B show that Meta Tuner consistently outperforms baseline methods, demonstrating the efficacy of our proposed prompt-paramett parameter co-optimization framework, which is built upon parameter sharing and supervised regularization loss.
The results Ta-
16

---

Published as a conference paper at ICLR 2026
C.2 ADDITIONAL GENERALIZATION ANALYSIS
We provide additional OOD experiments in Figure 10 The results show that MetaTuner achieves better out-of-dite distririibttinn generalization compared to baseline methods. Theoretically, the generaliza- tion capability of our framework stems from the representation sharing mechanism. In MetaTTune, we can regard prompt generation and parameter generation as two distinct tasks and enforce them to share common meta encoder. This shared-private multi-head architecture inherently induces a regularization effect (Park et al. 2019 Wang et al. 2022): the shared encoder must simultaneously support the learning Objectives of multiple heads, compelling it to capture stable, core structural features that are invariant across tasks and independent of the specific training distribution. This aligns with classical theoretical results in multi-task learning (MTL) (Baxter 2000 [Maurer et al.. 2016) D'Eramo et al. 2024; Bairaktari et al., 2023 Zhang et al. 2025): When multiple tasks share interme- diate representations, the effective hypothesis space of the model is constrained, leading to tighter generalization eπror bounds. Intuitively, the model cannot learn highly task-specific, high- sigh-cap representations independently for each task; instead, all tasks must find compatible representations within the same shared feature space. This capacity constraint significantly reduces overfitting to the idiosyncrasies of the training data distribution, thereby enhancing generalization across tasks and datasets. We believe that existing theoretical works on multi-task learning provide a solid foundation for understanding the generalization properties of MetaTuner.
GSMBK
50
HolpoiQA
CosmesOA
MetalTuner DetterTagether Iin DRO
BPO
MctaTuner Together
BPO
50
Metaluner
BPQ
Accuracy (%)
40
33.
17.90
Accuracy (%)
40 40.5$10.65
79.40
DPC
Accuracy (9))
81,4311.11
Detter Together
DPO
31.33
35.35
75,0571.10 10
27.35
31.15g
20
250.20
18.80
18.44
Model
Model Size
Model Size
Figure 10: Additional generalization results. We select dataset from the four (MATH, GSM8K, HotpotQA, CosmosQA) as the test set and use the remaining three as the training set.
C.3
CASE STUDY
Joint optimization for superior reasoning. We present a case from the MATH dataset in Figure 11 to illustrate the advantages of MetaTuunrr over existing methods. Compared with pure prompt optimization approaches such as BPO, which can produce step-by-step reasoning but often generate overly lengthy intermediate steps and occasionally commit calculation errors that lead to incorrect answers, MetaTTuer produces reasoning traces that are both accurate and concise while strictly adhering to the required output format. This improvement stems from its joint optimization of prompts and parameters, which allows the model to better adapt to downstream tasks. In contrast, when compared with traditional fine-tumin fine-tuning approaches such DPO, MetaTTuner also demonstrates clear advantages. Although DPO can arrive at the correct numerical solution, fails to comply with formatting requirements and tends to generate relatively rigid and less context-aware reasoning. Meta Muner, by dynamically generating high-quality prompts that activate relevant internal knowledge while simultancously refining model parameters, effectively combines the strengths of both prompt optimization and fine- tonittunng. As a result, MetaTuner achieves more stable, flexible, and diverse reasoning patterns, demonstrating both correctness and expressiveness.
Learned prompts for different base models. Below, we provide several examples in Table 4 to analyze the commonalities and differences of prompts learned for different base models. As shown in the examples, both models (Qwen2.5-3B and Qwen2.5-7B) retain the core structure of the initial prompt (e.g.. "step-by-step reasoning." " "answer in \boxed(]". J", and "word limit"), indicating that MetaTuner can stably optimize instruction formats that respect task constraints across different models. However, the 7B model tends to generate more task-specific and procedurally explicit instructions (e.g., for geometry problems, explicitly instructing "first compute the new dimensions, then calculate the area change"). whereas the 3B model favors more general guidance (e.g., "explain the relevant mathematical concepts" "include table diagram"). We hypothesize that the
17

---

Published as a conference paper at ICLR 2026
Question: f())
3x-2
what the value ((-1) /(0)? x-2
Express your answer common fraction.
Ground truth: f(-2)+ f(0) ((-2-2) 1(-1)= 300)
(-2-2)
+ (-1-2) (0-2)
Prompt Optimization-PPa
Solve complex mathematical problems concise. Present the final result in LaTeX using "boxed(j'. Ensure the response correct, helpful, and harmless. Question: (question)
Input:
using chain-of-thought approach. Keep the explanation
Output:
problem,
evaluate the function ((x) -2,
then
results logether.
First, let's evaluate ((-2) ((-2 36-2-2
=z-2
-4
Next, let's evaluate ((-1) ((-1) 301122
-1-2
-3
Finally, let's evaluate f(0) : f(0)
-1022
9-2
Now. let's add the results together: / /(-2) + ((1)++
=2+
Fine-tuning-DPO
Input: Question: (question)
Output: To find the value of (f(-2) 2) + f(-1) + f(0) ) for the function fxx
3x-2
x-2 we need to evaluate the function
at (x -2 (x = -1), and (x 0) and then sum the results.
First, let's evaluate (/(-2)):f(-2) = 3(-2)-
-2-
Next. let's evaluate (f(-1)):f(-) -1 -
-3
Finally, let's evaluate (/(0)): (()
300)-2
0-22
Meta Tuner
Input: Given a complex mathematical problem, solve step by step using shain-of-thought. approach, First, evaluate the 3x-2 at the specified points: 2,x 1, and 0. Then, calculate the sum these values. Ensure
function f f(x)
x-2
of the calculation. The answer and reasoning process should be concise. Present the final result LaTeX using 1boxed[]' without units.
clarity each
Question: (question)
Output: We have (--)
3(-2)-2
2,
1(-1)=2
and f(0) 30002
1. Thus,f(-2) +
-2-2
-1-2
f(0) +
Figure 11: Case study from the MATH dataset.
difference is primarily due to the varying task-solving capabilities of the base models. For the weaker 3B model, overly specific instructions risk embedding subtle errors that can mislead inference. MetaTuner thus learns to generate more robust, general prompts for smaller models. contrast, the more capable 7B model can reliably follow and benefit from detailed, task-specific guidance. leading MetaTuner to produce finer- grained instructions for it.
Learned prompts across different tasks. We provide several cases in Table to demonstrate the learned prompts across different tasks. From the examples, MetaTuner's learned prompts exhibit clear cross-task commonalities and transferable patterns. In MATH and GSM8K, both learned prompts refine the initial CoT instruction by explicitly guiding step decomposition, intermediate computation, and clear Explanation-desta core principle of concise, evidence- based answering, and sharpened to extract only task-relevant details while avoiding redundancy. Co the prompt emphasizes grounding the answer in contextual key details. In summary, despite differences in task-specific requirements, the prompts learned by Meta'Tuner consistently promote clear, logically structured responses and adapt their instructional details according to the nature of each task. This indicates that MetaTuner is able to identify and leverage shared structural patterns across tasks, while tailoring guidance to individual task needs, thereby enhances the transferability and adaptability of its prompts across diverse settings.
differing problem types. In HotpotQA, the prompt preserves the
18

---

Published as a conference paper at ICLR 2026
Table 4: Cases of learned prompts with different base models.
Initial Prompt
Given complex mathematical problem, solve it step by step using chain-of-thought approach. The answer and reasoning process should not exceed 250 words. Present the final result in LaTeX using "bboxed 1 without any units.
Case 1
Query
The lengths of two opposite sides of square decreased by 40% while the lengths of the other two sides are increased by 50% to form a rectangle. By what percent does the square's area decrease?
Learned Prompt (Qwen2.5-3B)
Given complex mathematical problem, solve it step by step using chain-of- thought approach. The answer and reasoning process should not exceed 250 words. Present the final result in LaTeX using "\boxed() J" without any units. Additionally, provide detailed explanation of the mathematical concepts involved in the problem, such as the properties of squares and rectangles, and the relationship between their areas. The explanation should be concise and clear.
Learned Prompt (Qwen2.5-7B)
Given a complex mathematical problem involving geometric transformations, solve it step by step using chain-of-thoughe approach. First, calculate the new dimensions of the rectangle formed by decreasing and increasing the sides of the square. Then, determine the original and new areas of the square and rectangle. Finally, calculate the percentage decrease in the square's 's area. The answer and reasoning process should not exceed 250 words. Present the final result in LaTeX using "[boxed[] )" without any units.
Case 2
Query
Find the greatest integer less than (v7 v5)6. (Do not use a calculator!)
Learned Prompt (Qwen2.5-3B)
Given complex mathematical problem, solve it step by step using chain- of-thought approach. The answer and reasoning process should not exceed 250 words. Present the final result in LaTeX using a "\boxed J" without any units. Provide detailed explanation of the mathematical concepts and techniques used in the solution, including any relevant formulas or theorems. The prompt should also include table diagram to illustrate the problem and its solution.
Learned Prompt (Qwen2.5-7B)
Given complex mathematical problem, solve it step by step using chain- of-thought approach. Begin by estimating the values of the square roots in- volved and then calculate the expression (v7 + v5)6 66 Use algebraic identities and approximations simplify the process. Ensure that each step is clearly explained and logically follows from the previous one. The answer and rea- soning process should not exceed 250 words. Present the final result in LaTeX using a "\boxed )" without any units.
Table 5: Cases of learned prompts with Qwen2.5-7B across different tasks.
MATH
Query
The lengths of two opposite sides of square are decreased by 40% while the lengths of the other two sides are increased by 50% to form a rectangle. By what percent does the square's area decrease?
19

---

Published as a conference paper at ICLR 2026
Table 5: Cases of learned prompts with Qwen2.5-7B across different tasks. (Continued)
Initial Prompt
Given a complex mathematical problem, solve it step by step using a chain-of- thought approach. The answer and reasoning process should exceed 250 words. Present the final result in LaTeX using a "\boxed[]" ()" without any units.
Learned Prompt
Given a complex mathematical problem involving geometric transformations, solve it step by step using chain-of-thouge approach. First, calculate the new dimensions of the rectangle formed by decreasing and increasing the sides of the square. Then, determine the original and new areas of the square and rectangle. Finally, calculate the percentage decrease in the square's 's area. The answer and reasoning process should not exceed 250 words. Present the final result in LaTeX using "|boxed() }" without any units.
GSM8K
Query
Ted the T-Rex was planning to bring potato salad to the dinosaur picnic. He knows that an adult dinosaur will 10 lbs of potato salad, and a child will eat half as much as adult. If there will be 20 adults and 5 children at the pienic, how many pounds of potato salad does Ted need bring the pienic if he hopes to have enough to feed everyone?
Initial Prompt
Given complex mathematical problem, solve it step by step using chain-of- thought approach.
Learned Prompt
Solve the following complex mathematical problem step by step using chain- of-thought approach. Break down the problem into smaller parts, calculate intermediate values, and provide clear explanation for each step.
CosmosQA
Query
The black cloud's clinic full of complex patients who need three referrals apiece. The white cloud's clinic is full of patients who don't 't't show up. What finally got me believe in the power of the black cloud was one resident worked with last year. One month she was working in the clinic but covering call in the PICU every fourth night (as am doing this month).
Why are you cover call in the PICU this month?
A. The schedule for that job rotates monthly and new people have the respon- sibility of doing monthly
B. am the best at covering calls at the PICU so have that job
C. None of the above choices.
D. needed to make extra money so applied for the position
Initial Prompt
Based on the provided context, select the most relevant and accurate option the correct answer the question. The answer should be directly related to the context, and must focus on the key details or events described.
Learned Prompt
Based on the provided context, select the most relevant and accurate option as the correct answer the question. Consider the details about the rotation of responsibilities and the nature of the job in the PICU. The answer should be directly related the context, focusing on the key details or events described.
HotpotQA
20

---

Published as a conference paper at ICLR 2026
Table 5: Cases of learned prompts with Qwen2.5-7B across different tasks. (Continued)
Query
(1) UKF is a brand owned by Luke Hood and AEI Media that focuses on shar- ing all aspects of electronic music. UKF Music was created by Luke Hood, who began sharing bass music through his original two YouTube Channels, UKF Drum & Bass and UKF Dubstep in 2009. On 28 June 2012, UKF cel- ebrated one billion YuuTube views since its creation. The UKF brand now has five YouTube Channels: UKF Music, UKF Dubstep. UKF Drum & Bass, UKF Mixes, and UKF Live. UKF has expanded beyond its YouTubb Chan- nels to creating compilation series and podeasts, organizing events, offering merchandise well as its own ticketing platform.
(2) Ysabella Brave is an American YouTube personality, artist, vocalist, singer and songwriter signed by Cordless Recordings, division of the Warner Mu- sic Group. She was discovered through the popularity of her YouTube chan- nels, Ysabella Brave and ysabellabravetalk.
(3) Steven Jay Williams (born July 24, 1974), also known by his online alias Boogie2988 or just Boogie, is an American YouTube personality best known for his video rants about video games and nerd culture as a charac- ter named "Francis". The Francis character is based on stereotypes of video game players and often parodies trending video game news, reaction and cul- ture. Williams based the character on his early life experiences and has said that he wants viewers to hate the character for embodying gamer stereotypes. Boogie2988 videos range from absurd rants to serious discussions on daily life, such the ethics of paid promotion on YouTube channels. He won the Trending Gamer award at The Game Awards 2016.
(4) Joseph Michael Graceffa (born May 16, 1991) is an American YouTube personality, actor, author, producer, and singer. He has two active YouTube channels, both of which named after him. One of them is for vlogging and the other is used for video gaming content. They both have a combined total of more than 1.7 billion views. His other YouTube channel, which he no longer uses, called WinterSpringPro. has 60 million views (as of August 2017). He was a contestant on the 22nd and 24th seasons of "The Amazing Race" and has appeared in short films (as web films) and short web television series on YouTube and its subscription-only service YouTube Red.
Question: Shane and Friends have interviewed American Yuueube person ality who has two active YouTube channels, which have a combined total of how many views?
Initial Prompt
Use the relevant information provided to answer the multi-hop reasoning question. Please provide the most accurate and concise answer to the question based on the information provided. The answer should be directly from the dataset, with no extra details or explanations. For yes-no questions, respond with either 'yes' or 'no.' Avoid long or complex sentences.
Learned Prompt
Use the relevant information provided answer the multi-hop reasoning ques- tion. Identify the American YouTuube personality mentioned and determine the combined total views of their two active YouTube channels. Provide the most accurate and concise answer directly from the dataset, without extra de- tails or explanations. For yes-no questions, respond with either 'yes' or 'no.' Avoid long or complex sentences. Focus on extracting the specific informa- tion needed answer the question accurately. Ensure the answer clear and directly addresses the query.
Table 6: Training cost of MetaTTuner against prompt optimization and fine-tuning tnn-ttnnig methods (hour).
Method
MetaTuner
PPO
DPO
BPO
RLPrompt
Qwen2.5-7B
5
=3.55
<3
33.5
22.
Qwen2.5-3B
=3.55
22..5
22
R22
11.
21

---

Published as a conference paper at ICLR 2026
Table 7: Average inference cost of Qwen2.5-3B base model per query (second).
Dataset
MATH
GSM8K
HotpotQA
CosmosQA
Avg Cost
Avg Best Performance
Answer Generation
0.43
0.39
0.07
0.06
0.24
52.38
+Prompt
0.6(+0.17)
0.55(+0.16)
0.23(+0.16)
0.18(+0.12)
0.39(+0.15) 0.53(+0.29)
57.57
+Prompt & Parameter
0.73(+0.30)
0.66(+0.27)
0.25(+0.18)
0.48(+0.42)
65.10
Table 8: Average inference cost of Qwen2.5-7B base model per query (second).
Dataset
MATH
GSMSK
HotpotQA
CosmosQA
Avg Cost
Avg Best Performance
Answer Generation
0.71
0.68
0.14
0.11
0.41
60.30
+Prompt
0,99(+0.28)
0,95(+0.27)
0.44(+0.3))
0.36(+0.25)
0.69(+.22) 0.98(=0.57)
64.68
+Prompt& & Parameter
1.48(+0.77)
1.48(+0.80)
0.47(+0.33)
0.48(+0.37)
68.60
C.4 EFFICIENCY ANALYSIS
Training efficiency. We take the MATH dataset example compare the training overhead of MetaTuner's 's joint training process against conventional prompt optimization and fine-tuning methods. All training procedures are carried out on two 80GB GPUs, and the associated costs reported in T, Since MetaTuner collects training samples online, to ensure a fair comparison, we include the time required for additional data collection in the training cost of the baselines well. From the results, MetaTuner indeed incurs higher training costs compared to baseline approaches. However, since this training is performed only once, we consider the training cost to be practically acceptable.
Inference efficiency. During the inference stage, for given query x. Meta Tuner needs to generate a set of prompts and parameters to complete the downstream task. Unlike the standard text generation task, there are additional costs associated with the LoRA parameter generation. In practice, we generate parameters for the o proj part of the decoder layers in the downstream actor model M. As mentioned in Section 3.4, each LoRA parameter is generated by hyper-networt network é,, = (Wh E dRdxxl Thth E AR x W E RRMx! W E (R Therefore, depending the task scenario and model choice, the additional cost varies with sequence length 1, the hidden size lig of the prompt generator, the LoRA rank I'M of the generated parameters, and the number of decoder layers K in the downstream actor model. However, compared to the parameter quantity of the LLM itself, the hyper-network is relatively small, so it does not incur significant computational overhead.
In Table and T we report the cost of generating prompts and LoRA parameters, respectively. For reference, we also present the original inference cost without generating prompts and LoRA parameters, as well as the average best performance for each category of methods. From the methods with prompt generation, Meta'Tuner achieves an absolute performance gain of 5.73% at the cost of an average additional inference latency of 0.22 seconds. (2) Compared Vanilla and Fine-tuning methods, MetaTuner achieves an absolute performance gain of 10.51% % at the cost of an average additional inference latency of 0.43 seconds.
above results, we can see: (1) Compared
Improving inference efficiency. To accelerate the inference process, we further propose MetaTuner- C, which directly clusters the queries in the test set and then uses the prompt and parameters associated with the cluster centroid for all samples within that cluster during inference. Assuming the test set contains N samples and is partitioned into K clusters, the additional time overhead of MetaTuner-C compared to vanilla methods can be reduced to time costs for generating the prompt and parameters for single query, respectively. Notably, our main method, MetaTuner-J, can be viewed as a special case of MetaTTune--C when evaluate MetaTuner-C with various K using the Qwen2.5-3B base model on the MATH dataset and report the results in Table I and Table IO, Our analysis shows that MetaTuner-CC achieves strong performance while significantly reducing inference time. For example, under the Qwen2.5-3B setting, using the best fine-tuning baseline DPO as a reference, MetaTuner-C with K=50 attains an absolute performance gain of 3.41 41% % with only 0.03 seconds of additional latency per query.
K (tp + fgg/N, where ip and f, denote the
N. We
22

---

Published as a conference paper at ICLR 2026
Table 9: Performance of MetaTuner-C with Qwen2.5-3B model MATH.
Method
Teta-uee--J
MetaTTune-CC (K=50)
Meta Tuner-CC (K=30)
Meta'Tuner-C (K=10)
DPO
Performance
41.33
40.00
39.11
37.56
36.59
Inference Cost (s)
0.73(+0.30)
0.46(+0.03)
0.45(+0.02)
0.437(+0.007)
0.43
Table 10: Performance of MetaTuner-C with Qwen2.5-7B model on MATH.
Method
MetaTuner-J
MetaTuner-CC (K=50)
Meta-Tnnr-CC (K=30)
Meta'Tuner-C (K=10)
DPO
Performance
48.67
45.56
45.11
44.89
43.78
Inference Cost (s)
1.48(+0.77)
0.80(+0.09)
0.76(+0.05)
0.73(+0.02)
0.71
D IMPLEMENTATION DETAILS
D.1 PRELIMINARY EXPERIMENT
We present the prompts used in the right subfigure of the preliminary experiment below. For constructing the training data, the prompt p concatenated front of the query z form the model input, while the ground truth y serves as the expected output. The supervised fine-tuning process is conducted using Llama- Factory (Zheng et al. 2024), with the epoch set to I, the batch size set to 64, and the learning rate tuned over the values (5e-4, le-4).
Prompts for Supervised Fine-TTuning on HotpotQA.
## Prompt_0 O
From the given text, define or describe a particular term, concept, or idea mentioned. Your response should be based solely on how the term is presented the passage.
## Prompt_
Based on the passage, identify the character person associated with the provided description. Provide the correct name or identifier, as supported by the text.
## Prompt_2
Use the passage to infer the implied meaning consequence of a particular event, decision, or fact. Provide the most logical answer based on the information available.
## Prompt_3
Using the passage provided, identify the location place associated with the subject matter. Make sure the answer is clearly supported by information found in the text.
## Prompt_4 4
Read the provided text and identify specific facts related to a particular subject. Answer the question using only information found in the passage.
## Prompt_5
Compare the details given in the passage and answer the question by identifying similarities differences between two subjects pieces of information.
## Prompt_6
From the given text, determine the timeline of events or specific dates related to a topic. Answer the question with precise chronological details from the passage.
## Prompt_7
Retrieve specific detail or piece of information from the passage to answer a question. Make sure the answer directly aligns with what is stated the text.
23

---

Published as a conference paper at ICLR 2026
## Prompt_8
Given the passage. answer comparative question by identifying similarities or differences between two concepts, characters, events described in the text.
## Prompt_9
Identify the relationship between two subjects concepts described in the passage. Answer question by focusing on how these two entities are connected or related.
## Prompt_ 10
From the passage, answer the question by explaining the situation problem presented in the text. Provide a clear response that addresses the core issue or situation described.
## Prompt_11
Read the passage and summarize the key points that answer the question. Focus on providing a concise, clear response that captures the main elements of the text.
## Prompt_ 12
Given a passage, answer the question by making an inference based on the information provided. Your answer should logically follow from the details in the text, even if it's not explicitly stated.
## Prompt_13
From the passage, answer the question by providing summary of the main events points described. Your answer should focus on the key takeaways and provide concise overview of what is happening.
## Prompt_ 14
Based on the passage, explain process or sequence of events described in the text. Provide the steps or order of occurrences your response.
## Prompt_15 15
From the passage, answer the question by considering the broader context in which the information provided. Your answer should reflect an understanding of the context and its relevance to the question.
## Prompt_16 16
Based on the passage, answer the question by extracting the most important details that would influence decision action. Your answer should be based on the most relevant information provided in the text.
## Prompt_ 17
From the provided passage, answer the question by identifying the key detail or piece of information that directly addresses the question.
## Prompt_ 18
Given a passage, extract the relevant information that answers a specific question. Ensure that your response is directly supported details in the passage.
## Prompt_ 19
Based on the passage, answer the question by identifying the character or event described. Provide the answer that best fits the situation outlined in the text.
D.2 IMPLEMENTATION OF METATUNER
Data splition. We split each dataset into training, validation, and test sets for model training and evaluation. For MATH, use the 12K instances in the training specified in (Lightman et al.
24

---

Published as a conference paper at ICLR 2026
Table 11: Performance after the warm-up stage.
Base Model
MATH
GSMSK
Hotpo(QA
CosmosQA
Qwen2.5-7B
45.33
77.48 71.27
52.20 53.95
90.05
Qwen2.5-3B
35.33
84.00
Table 12: Performance of Qwen2.5-3B on MATH with smaller expert datasets.
Data Quantity
100%
60%
20%
Performance After Warm-up
35.33
36.67
34.44
Performance After Joint Training
41.33
41.33
40.89
2024), and randomly divide the remaining 500 instances into 50 for validation and 450 for testing. For GSM8K, the official test set contains 1,319 instances, so we allocate 473 instances for validation and the remaining 7K instances for training. For HotpotQA, we sample 20,000 instances from the official training set and randomly select 500 for validation and 2,000 for testing from the official dev-set-distractor. For CosmosQA, we use the entire 25, 25,262 instances in the training set and randomly sample 500 and 2,000 instances from the test set for validation and testing, respectively.
Details of the warm-up stage. For the warm-up of M, use the queries from the original dataset as model inputs and the corresponding ground truth as model outputs for supervised fine- fine-tuning, For the warm-up of G, since there no ground truth prompt, we construct the supervised training data with rejection sampling. Specifically, we first utilize a leading closed-source LLM to generate expert prompts Dexpent [ and then use the task reward R filter out positive examples. The positive samples are then collected as Dro = {(x;, (. (Ā,, 2x)) 1 for the supervised fine-tuning of prompt generation. Here, .M" and 9 represent the model after warm-up. Naturally, the pre-warmed model can form a pipeline S "M (p, x1).. 24) to complete downstream tasks. We report the results in Table
To further investigate the impact of generated expert data during the warm-up stage, we additionally study how using smaller expert dataset (in Table T2) or a weaker expert model (in Table [3] affects the MetaTuner's final performance. Our analysis reveals that MetaTuner exhibits robust performance under both weaker and smaller expert dataset settings. As the size of the expert dataset decreases, performance remains stable both after warm-up and joint training. When a weaker expert model used for warm-up, the post- warm-up performance degrades moderately the expert model's capability diminishes. However, this performance gap is largely mitigated after joint training. This suggests that the warm-up phase only requires basic level of prompt-generation generation competence. The proposed MetaTuner framework can subsequently discover superior poompt-paramete! combinations during joint training phase, thereby enhancing overall performance.
Construction of the meta prompts. In this paper, since we adopt a prompt-rewrite paradigm for prompt generation, each meta prompt contains query and an initial prompt, well as instruction to describe the prompt rewrite task. We follow the guidance of (Yang et al. (024b) to construct a base version of the prompt for each task as the initial prompt, which only includes a description of the task and the required answer format to facilitate answer parsing. For GSM8K, since the base model already produces highly consistent answer formats, we omit the answer format requirement in the initial prompt. The meta prompts on four datasets are presented below.
25

---

Published as a conference paper at ICLR 2026
Table 13: Performance of Qwen2.5-3B MATH with weaker expert datasets.
Expert Model
Proprictary LLM
Qwen2.5-32B
Qwen2.5-144p 14ee22.5-14B
Performance After Warm-up
35.33
33.11 40.22
31.33
Performance After Joint Training
41.33
40.00
Meta prompt for MATH dataset.
Given a query and an initial prompt, please add more detailed guidance to the initial prompt help the model better respond to the question. The generated prompt will be concatenated in front of the query to assist the model in answering questions. During the rewriting process, ensure that the new prompt includes the information from the original prompt, such as the task description and the answer format requirements. The generated new prompt should not exceed 100 words.
Query: {query]
Initial Prompt:
Given a complex mathematical problem, solve it step by step using a chain-of-thought approach. The answer and reasoning process should not exceed 250 words. Present the final result in LaTeX using a "\boxed{ )" without any units.
New Prompt:
Meta prompt for GSMSK dataset.
Given a query and an initial prompt, please add more detailed guidance to the initial prompt help the model better respond to the question. The generated prompt will be concatenated in front of the query to assist the model in answering questions. During the rewriting process, ensure that the new prompt includes the information from the original prompt, such as the task description and the answer format requirements. The generated new prompt should not exceed 100 words.
Query:
{query]
Initial Prompt:
Given a complex mathematical problem, solve it step by step using a chain-of-thought approach.
New Prompt:
26

---

Published as a conference paper at ICLR 2026
Meta prompt for HotporQA dataset.
Given a query and an initial prompt, please add more detailed guidance to the initial prompt to help the model better respond to the question. The generated prompt will be concatenated in front of the query to assist the model in answering questions. During the rewriting process, ensure that the new prompt includes the information from the original prompt, such the task description and the answer format requirements. The generated new prompt should not exceed 100 words.
Query: {query]
Initial Prompt:
Use the relevant information provided answer the multi-hop reasoning question. Please provide the most accurate and concise answer to the question based on the information provided. The answer should directly from the dataset, with no extra details or explanations. For yes-no questions, respond with either 'yes' or 'no.' Avoid long complex sentences.
New Prompt:
Meta prompt for CosmosQA dataset.
Given query and initial prompt, please add more detailed guidance to the initial prompt to help the model better respond to the question. The generated prompt will be concatenated in front of the query to assist the model in answering questions. During the rewriting process, ensure that the new prompt includes the information from the original prompt, such as the task description and the answer format requirements. The generated prompt should exceed 100 words.
Query: {query)
Initial Prompt:
Based on the provided context, select the most relevant and accurate option as the correct answer to the question. The answer should directly related to the context, and must focus on the key details events described.
New Prompt:
27


Reviews:
[{"review_id": "7Gp0UoT5IP", "reviewer": "Reviewer_6Sni", "summary": "This paper introduces MetaTuner, a unified framework for the joint optimization of prompts and model parameters in large language models (LLMs). Traditional approaches typically treat prompt optimization, discrete and external, and fine-tuning, continuous and internal, as separate processes. MetaTuner bridges these two paradigms through a shared meta-encoder that feeds into two complementary decoders: a prompt decoder that generates discrete natural language prompts and a parameter decoder that produces continuous LoRA-based parameter updates. To address the challenge of non-differentiability in prompt generation, the framework incorporates a supervised regularization loss that leverages an “expert dataset” of prompts, partly generated using GPT-4o. This mechanism stabilizes training and allows efficient co-optimization of both prompt and parameter spaces. Experiments on MATH, GSM8K, HotpotQA, and CosmosQA, using Qwen2.5-3B and Qwen2.5-7B backbones, demonstrate that MetaTuner achieves consistent performance gains over prompt-only, fine-tuning, and hybrid baselines such as BetterTogether.", "questions": "What are the computational and latency trade-offs of generating both prompts and LoRA parameters at inference time, and how could these be mitigated for large-scale or real-time use?\n\nHow sensitive is MetaTuner’s performance to the quality and size of the GPT-4o-generated expert dataset, and how does it generalize when trained with weaker or smaller teachers?\n\nDo the learned meta-prompts exhibit shared structures or transferable patterns across related tasks?\n\nHow model-specific are the learned meta-prompts, would fine-tuning different base models yield similar or divergent prompt behaviors?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "Unified Framework: Introduces MetaTuner, a single architecture that jointly optimizes prompts and model parameters, integrating discrete prompt learning and continuous fine-tuning.\n\nDual-Decoder Design: Proposes a shared meta-encoder with separate prompt and parameter decoders that generate textual prompts and LoRA-based parameter updates in a coordinated manner.\n\nSupervised Regularization: Employs a regularization loss guided by an expert prompt dataset to stabilize training and address non-differentiability in prompt optimization.\n\nProper Evaluation: Demonstrates consistent performance gains across multiple reasoning and QA benchmarks, supported by detailed ablations and generalization analyses.\n\nIn addition, the paper is well-written and proper examples are provided.", "weaknesses": "The paper could more clearly differentiate MetaTuner from sequential prompt–fine-tuning pipelines such as Chain-of-Thought distillation (Finlayson et al., 2025; Wei et al., 2022). A stronger conceptual contrast would help clarify the advantages of end-to-end co-optimization over staged approaches. Also a comparison with Self-Instruct (Wang et al., 2022) would strengthen the paper’s positioning. While both integrate prompting and fine-tuning.\n\nGenerating both prompts and LoRA parameters for each query introduces latency and computational cost, which may limit scalability and deployment in real-time settings.\n\nThe supervised regularization relies on GPT-4o-generated expert prompts, creating a dependency on an external model. The impact of weaker or smaller dataset on performance is not analyzed.\n\nDespite strong empirical results, the paper provides little theoretical explanation of why shared representations improve generalization.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "r24EIKS29a", "reviewer": "Reviewer_ttpV", "summary": "In this work, a framework to jointly optimize prompt and parameter is proposed. The framework is named after metaTuner. During training time, the initial prompt is given to the meta encoder to rewrite with a language model A. In the meantime, there is a language model B to generate the parameter which is used to optimize the downstream actor model. The actor model takes the generated parameter and the the rewritten prompt as input to generate the answer. The response is used to compare with the ground truth and get the task specific reward, which is used to update the model A and B.", "questions": "- Concerns about the generalization of the model trained with the proposed approach. The model is trained to optimize for several specific task verticals, including prompt rewrite and parameter generation. one concern is about catastrophic forgetting: is the model still able to generalize to other unseen dataset during finetuning? Some OOD evaluation will helps.\n- One hypothesis of the performance win is that the introduction of metaTuner actually increased the total number of parameters of the whole system and the scaling up of the model itself is the source of improvement. Is it possible to validate this point so we can confirm that the wins are from the approach instead of the model scaling up?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The proposed approach is easy to follow and understand. \n- The proposed approach has been verified in multiple SFT datasets.", "weaknesses": "- Concerns about the generalization of the model trained with the proposed approach. The model is trained to optimize for several specific task verticals, including prompt rewrite and parameter generation. one concern is about catastrophic forgetting: is the model still able to generalize to other unseen dataset during finetuning? Some OOD evaluation will helps.\n- One hypothesis of the performance win is that the introduction of metaTuner actually increased the total number of parameters of the whole system and the scaling up of the model itself is the source of improvement. Is it possible to validate this point so we can confirm that the wins are from the approach instead of the model scaling up?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3dv119YGGz", "reviewer": "Reviewer_RbJU", "summary": "The paper proposes a joint framework for LLM prompt optimization and parameter training. Clear gains are demonstrated over prompt optimization and fine-tuning.", "questions": "- Can you say something about the computational training costs compared to regular prompt optimization / fine-tuning?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Clear and concise description of the approach \n- The evaluation covers a very good range of prompt optimization and fine-tuning methods\n- Strong and consistent gains over the baselines across 4 datasets and 2 model sizes\n- Interesting and well-designed ablation studies", "weaknesses": "- A major benefit of prompt optimization is infrastructure simplicity and efficiency - serving the same frozen model for multiple tasks is much cheaper than multiple dedicated models. This benefit is lost here.\n- The whole system design is quite complex, but the strong gains and comprehensive ablations make up for it\n- Given the otherwise broad experimental setup, I find it a bit unfortunate that - iiuc - fine-tuning is limited to LoRA. An isolated small-scale experiment with full fine-tuning of the 3B model would tell us more about the head room and would help to better understand the gains of MetaTuner.\n\nMinor comments:\n- L107: \"In LLMs, we typically work with input-output pairs\" is too simplistic/general - e.g. think LLM pre-training etc.\n- Fig. 1 font size is too tiny", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Q9LDPJACzC", "reviewer": "Reviewer_LrBX", "summary": "This paper proposes MetaTunner which is a framework that jointly optimizes both prompts and model parameters for finetuning LLMs. They design a prompt generator that outputs discrete prompts and a parameter generator that produces LoRA style updates, this two modules are trained jointly, with an additional ad-hoc supervised regularization loss on prompt. Experimetns on various QA datasets such as MATH, GSM8K show their proposed method outperforms various baselines including prompt tuning, LoRA tuning, and the closely related hybrid BetterTogether approach.", "questions": "See weaknesses.", "limitations": "", "rating": 2, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. This paper address an important direction that the prompt for a downstream tasks need to be engineereed and might not be optimial, finetuning both prompt and model offers a more unified task adaptation framework.\n2. The propose architecture is clear and experiment results are good.", "weaknesses": "1. The proposed framework is essentially a composition of existing components: prompt generation + LoRA + shared encoder, which is incremental and lack clear conceptual novelty. The idea of optimizing both prompt and model itself is well-explored in studies like BetterTogether, which the authors themselves cite but weakly differentiate from.\n2. Instead using soft prompts that can truely trained jointly end-to-end, it is questionable why this paper choose discrete prompts generated by a separate LLM, although it may help interpretability. More insights or analysis on this should be discussed for why discrete prompts are preferable. \n3. The supervised regularization loss to handle discrete-continuous issues feels ad hoc and unstable, it is based on an auxiliary “expert prompt” dataset, and the algorithm itself has no analysis of convergence.\n4. The evaluation is limited to 4 QA benchmarks and a single Qwen model family with two different sizes, the method's generalizability to other domains and models remains untested.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)