Understanding Prompt Tuning and In-Context Learning via Meta-Learning
Authors: Tim Genewein
Li Kevin Wenliang
Jordi Grau-Moya
Anian Ruoss
Laurent Orseau
Marcus Hutter
Venue: NeurIPS 2025 spotlight ()
Keywords: prompting
prefix tuning
in-context learning
meta-learning
Bayes
frontier model
Primary area: theory
TL;DR: The paper formally analyzes prefix tuning and its limitations by conceptualizing neural nets as meta-trained Bayesian predictors.

Abstract: Prompting is one of the main ways to adapt a pretrained model to target tasks. Besides manually constructing prompts, many prompt optimization methods have been proposed in the literature. Method development is mainly empirically driven, with less emphasis on a conceptual understanding of prompting. In this paper we discuss how optimal prompting can be understood through a Bayesian view, which also implies some fundamental limitations of prompting that can only be overcome by tuning weights. The paper explains in detail how meta-trained neural networks behave as Bayesian predictors over the pretraining distribution, whose hallmark feature is rapid in-context adaptation. Optimal prompting can be studied formally as conditioning these Bayesian predictors, yielding criteria for target tasks where optimal prompting is and is not possible. We support the theory with educational experiments on LSTMs and Transformers, where we compare different versions of prefix-tuning and different weight-tuning methods. We also confirm that soft prefixes, which are sequences of real-valued vectors outside the token alphabet, can lead to very effective prompts for trained and even untrained networks by manipulating activations in ways that are not achievable by hard tokens. This adds an important mechanistic aspect beyond the conceptual Bayesian theory.

Understanding Prompt Tuning and In-Context Learning via Meta-Learning
Tim Genewein"
Li Kevin Wenliang
Jordi Grau-Moya
Anian Ruoss
Laurent Orseau
Marcus Hutter
Google DeepMind
Abstract
Prompting is one of the main ways to adapt a pretrained model to target tasks. Besides manually constructing prompts, many prompt optimization methods have been proposed the literature. Method development mainly empirically driven, conceptual understanding of prompting. In this paper we
with less emphasis discuss how optimal prompting can be understood through a Bayesian view, which also implies some fundamental limitations of prompting that can only be overcome by tuning weights. The paper explains in detail how meta-trained neural networks behave Bayesian predictors over the pretraining distribution, whose hallmark feature is rapid in-context adaptation. Optimal prompting can be studied formally as conditioning these Bayesian predictors, yielding criteria for target tasks where optimal prompting is and is not possible. We support the theory with educational experiments on LSTMs and Transformers, where we compare different versions of prefix-tuning and different weight-tuning methods. We also confirm that soft prefixes, which are sequences of real-valued vectors outside the token alphabet, can lead to very effective prompts for trained and even untrained networks by manipulating activations in ways that are not achievable by hard tokens. This adds important mechanistic aspect beyond the conceptual Bayesian theory.
Introduction
Perhaps the most impressive feature of today's frontier models their ability to swiftly adapt their behavior to a wide range of contexts. Given relatively few tohens-whether from user input, system prompt, or a number of in-context cauplles-moel) often rapidly infer the task at hand and produce good continuations without any weight adaptation (in-context learning, [Lampinen et al. (0244). From meta- meta-learning perspective, rapid in-context adaptation is expected to arise: log loss minimization with parametric sequential predictor (like neural network) over a distribution of stochastic data generators leads to Bayesian predictor for the pretraining distribution (Ortega et al.) 2019]. The hallmark feature of such a predictor -Baye-ooptimality is most rapid in-context adaptation and least (cumulative) prediction error on average. Accordingly, prompting, that is conditioning of the Bayesian predictor, can used to data-efficiently adapt the pretrained model to a target task. An important question is: under which conditions is it possible to find prompt such that the prompted pretrained predictor becomes (near-) Baycs-optimal on a target task? We refer to this as optimal prompting, which is possible in theory if the target task is one of the tasks covered by the meta-distribution. If this is not the case, then optimal prompting may not be possible for an ideal predictor, and weight adaptation may be necessary.
"Conrespondence to (tingen, kevinliv) bovinllxx@@ooggee.com
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Transformer: Random Coins Single Coin
Transfomer: Random Cains Single Coin
Evaluation on Single Coin
Evaluation Single Coin
Instant regret [nats]
0.4
Curui ative regret [nats]
Profix
- 49 (Naute)
Weighe
Baselines
Profte
Weight
199 (Nauw)
Baseines
0.3
w
0.2
0.1
0.0
12
0
25
50
75
100
125
150
175
200
6.5
Lottarr 6.00
5.00
Pargebhaye 1o0
2.05
cm0
100
500
635
5.0
Cumulative regret [nats]
Haroar
stoneles
Sobry
ricunting
Proil mulloyeal Randomil
simppteafe
Ne deoliig
SonPy
Futlly Lorpenyy
Ema nov'
toug PeggeBeyys
the
nio Sming
4.
unoomo
2-
LSTM: Random Evaluation on Single Coin
Single Coin
49 (Marna
199 (Nave)
Cumulative regred et [nats]
w *
Profla
Weight
Masallned
Pridx
0
25
50
75
100 Step n
125
250
175
200
Top: Performance of different tuning meth- ods on Transformers, measured as excess log loss, i.e., regret (Eq. (10). lower is better). See bar plots for color legend. Top-right: Detailed Transformer results for last step within the tuning sequence length Nume and the last evaluation step Noval- Right: Like above but for LSTM.
Sattyy 1.ze
Fubliyy noo
Emboooss 10.00
Unverto covy 5.00
Juos
Torgettan papaayye 6o0
Handery Spmplessh
Heatery
Un Enbrndhy Tergetay BayeeBayss yos
Frol to Hennnng In WeStoveii RunstomPs
Hardary
scaryy
Un doe--mmotii
Probbay aaes
Ho Mosuiing
Rorstempe ge
strpphese
Figure 1: Pretraining on sequences from coins with uniform random bias (length Nuain 100), then fine-tuning to the target task of a single coin with bias 0.2 (tuning sequence length Numn = 50). Plots show prediction performance on the target task for different prefix- and weight-tuning methods. For both Transformers and LSTMs Soft Prompting ("SoftPT') leads to optimal performance, showing that networks can be successfully prompted to behave Bayes-optimally on the target distribution ("TargetBayes"). This holds up to the tuning sequence length (50), with only minor degradations up to 200 steps. The corresponding soft prefixes of length= outperform even the best hard token prefixes of the same length ( ('HardPF)). Most weight-tuning methods also perform very well. See Section 4 for method details. Thick lines and bars show the median over 10 tuning repetitions, thin lines individual repetitions, and shaded areas/bars show 25%. 75% quantiles. See Fig. 2 for a visualization of models' internal dynamics. Regret curves for the LSTM, similar to top-left panel, are shown in Fig. A5
The goal of prompt tuning for a target task is to produce prompts that, when consumed by the model, inject maximal information (up to statistical sufficiency) about the target task into the predictor's internal state. In practice, prompt optimization is often done by tuning soft prefixes, which are sequences of real-valued input vectors instead of hard tokens. As we show, the corresponding off- distribution inputs can exploit neural circuitry to inject substantially more information compared to even the best hard token sequence of the same length, without breaking subsequent internal dynamics.
In this paper, we investigate prefix-tuning, where a short prompt prefix is tuned to maximize subse- quent prediction accuracy. We consider prefix search over hard tokens, and three soft prefix-tuning methods, including 'Soft Prompting', [Lester et al.) 2021). The code to reproduce all our experiments Our main contributions are:
available at: https: //tg::://iiie hitgggthuuu..... com/ com/googlee...oooooooooooooooooe wetm//gooodeeeeinnddl... www.dddedemddnnnnnnneeee
We discuss how prompting can be understood steering a Bayesian sequential predictor via its pretrained in-context adaptation mechanism that arises from meta-training, see Section 2
We analyze theoretical conditions for the relationship between pretraining distribution and target task under which optimal prompting is and is not possible, see Section 3
We confirm these theoretical conditions empirically via a series of educational experiments, and show that in the negative case weight-based fine-tuning can succeed. See Section 4.2
We investigate mechanistic aspects of prompting L.STMs and Transformers, with experi- ments both pretrained and untrained networks. In all cases, soft prefixes can be much

---

more effective than any hard token sequence of equal length, particularly for Transform- ers, where soft prefixes can even cause untrained networks to behave as well well-peerorming sequential predictors. See Section 4.2
Limitations and Scope. As with any conceptual principle, the Bayesian view cannot exhaustively describe all in-context learning phenomena at (frontier model) scale: limited data and model expres- sivity, suboptimal optimization, out-of-distribution generalization, and other mechanistic aspects will have additional impact. Having said that, there are number of recent investigations LLM scale supporting the plausibility of the Bayesian view, such as Chan et al. [202aa] who argue that distributional properties of natural language discusses fundamental properties of prefix tuning, which we illustrate with educational experiments where the focus is on clarity and being able to compare quantitatively against a known and tractable Bayesian predictor. Accordingly, our datasets do not capture the full complexities of, e.g, large-scale language, vision, or roboties tasks. Similarly, our neural networks are small by modern standards, which means that our findings must be very cautiously extrapolated towards modern frontier model scale. Further rigorous and well-designed scientific studies are necessary bridge the gap between our current work and modern large-scale ML practice, and we are optimistic that our fundamental results will inspire the design of such studies. The simplicity of our experimental setup allows to carefully control the data statistics, and compare neural predictors against exact Bayes-optimal predictors- of which is not true for LLM-scale experiments. Additionally, it allows us focus on fundamental aspects of prompting that arise even in idealized settings, and thus hold at any scale. We believe that this lays important fundamental groundwork, which will help future research to isolate and more effectively investigate additional, non-idealized, aspects of in-context learning and prompt uning at frontier scale, We do not propose improved prompt tuning methods in this work since our aim is to study and understand existing ones.
easily give rise meta-learming setting. This paper
2 Background: Memory-Based Meta-Learning
In this section we review how memory-based meta learning leads to Bayesian sequential predictors, whose hallmark feature is (most) rapid in-context adaptation. Under this view, the role of prompt tuning is to facilitate the inference process of the target task. While LLMs and frontier models are not explicitly meta-trained, their training process can be viewed as implicit meta veta-training, in which case the Bayesian view constitutes an important conceptual principle for understanding pretrained models and their in-context learning abilities.
Tasks. Let an alphabet A be finite set of tokens (with one-hot encoding) and 21.N E A be sequence of such tokens of length N, and 2ig = e be the empty sequence. task a distribution over finite-length sequences:
P(ZENIT) IR + AAN
where 7 E RM is the M-dimensional parameter vector of the task, over which distribution p(r) can be placed (the task- or meta-distribution). For notational simplicity we assume all sequences have the same length N. Simple examples of sets of tasks would be the family of Bernoulli distributions parameterized by the bias [o, 1]. or the family of Markov processes over sequences in AN parameterized by the transition kernel and initial state distribution. The set of tasks and the distribution over tasks define the marginal distribution over sequences:
{(ZIN)= - / P(EINN))
/ P(T///z
(1)
Vn E ((ankeen) /
(2)
which we have rewritten in the second line in its "next-token predictor" form (conditional distribution over next token, given full history).
Sequential predictor. Let we be a parametric sequential predictor, such as a neural network, which is a function with parameters 0 that takes in arbitrary-length sequence of D-dimensional float vectors and outputs a discrete distribution over the next (one-hot) token: my : [ - AA Typically, D [A], meaning that one-hot tokens can be fed directly into the predictor. The predictor's
3

---

conditional distribution over the next token given a context <nn is given by evaluating the predictor with the given context (i.e., performing forward pass): ( neural net accepts not only hard tokens as inputs, but sequences of arbitrary vectors in TR ('soft tokens"), ) and will produce a distribution over the next hard token in either case.
(T(=en). Keep in mind that
Meta-Learning,
In memory-based meta learning, 0 is adjusted by repeating the following steps:
1. Reset memory: set predictor's internal state or context to a fixed initial value.
2. Sample task: P(T).
3. Generate data: sample one or more sequences from the task: 21,N
((II:N/T).
4. Update parameters: perform gradient step towards minimizing prediction error (log loss) the sampled sequences.
Log loss is the cumulative prediction error. For single sequence it is:
Lo(xIN)) == - log (T(1)))) =
Σ log No (Inlaen
(3)
The expected excess log loss measures how much worse To performs w.r.t expected cumulative prediction error compared to the best possible predictor that does not know T:
Eg | log To( Eg [- log [(T1.N)] = (El 20
which is zero iff Ty Any predictor that fulfills this is Bayes-optimal for E, and Eq. ( and Eq. ( provide recipe for constructing an explicit Bayesian predictor, which is a mixture over one predictor per task, weighted by the posterior probability over the task given the context so far: P(r[T<n) x
In many cases this recipe is analytically or computationally intractable. Memory-based meta-learning provides alternative for obtaining an approximate Bayesian predictor simply through log loss minimization in meta-learning loop:
log {(TINN) To (F1:N)
arg min (x (A) arg min Eg
arg min Ee
(5)
If is expressive enough (realizability) and the meta-learning process fully converges, then, denoting i as the converged parameters:
Vn :
s ((En/<e))
(6)
meaning the network's prediction over the next token, given context Ten is (nearly) indistinguishable
from an explicit Bayesian predictor. The meta-trained neural network thus implements a Bayes- optimal adaptive prediction algorithm via its activations only-without weight updates (in-context learning). Previous works have empirically verified that meta-trained LSTMs and Transformers can indeed reach Bayes-optimality through meta- trainnn,, e.g., Mikullik et al. (2020) for sequential prediction and decision decision-making (not covered in this paper, but the theory extends straightforwardly to loss functions other than log loss), Genewein al. [2023] for piecewise stationary data sources (where models additionally have to infer task boundaries), and Grau-Moya et al. [2024) for variable-order Markov processes.
Remark. Frontier models are implicitly mntantrainee on samples from an unknown, rich and com- plex distribution of data generators [Chan et al. 2022a] The main concerns w.r.t. the applicability of the Bayesian viewpoint is that due to limited expressivity, limited data, suboptimal optimization, and off-distribution inputs, models may not converge to or operate in the Bayesian regime. Additionally, models may often operate in the generalization regime, while the theoretical guarantees only hold strictly under data drawn from the training meta-distribution. While we strongly recommend carefully investigating these issues, the theory tells us that as models get better and better, they will get closer and closer to the Bayesian ideal, making an important fundamental computational mechanism to understand.

---

3 Prompt Optimization: prefix-tuning
We are given neural sequential predictor Ne. that was pretrained via meta training < (Z = P(I (r)dT. The goal is to adapt this predictor to a target distribution (ZI:N) = I( ( pTans (r)dr. In prefix-tuning the adaptation is performed by finding a (typically short) prefix sequence sequence 81:7. E S of length I. that is prepended to the observations fed to the model. The "alphabet" S depends the prefix-tuning method. In this paper use:
Hard token search (HardPT): S == A.
Simplex prefix (SimplexPT): S AA R
Real prefix (RealPTy: S |||
Soft Prompting (SoftPT): S = = geghheading-dimmmioonn i.e., Lester et al. 2022
Prefixing sequence Z<n with $1:Z. and passing it through the neural sequential predictor corresponds to conditioning with additional initial information: xo [ [81:1.(n)
= Po $1.1, F<n). The prefix is optimized by minimizing the empirical log loss over K samples of sequences from the target distribution given the prefix:
min Eg [Lo(xINN|sL (1.L]| $1.EEL
min 1 L.
Σ
Σ
log Po sTLL
K
with 1i.N ~ & TTarget For hard token search, we perform exhaustive search over all token sequences of length L. For all three soft token methods, we use mini-batch based stochastic gradient descent.
When can prefix-tuning work? We consider the case where the prefixed model behaves (near) Bayes-optimally the target task distribution. We first consider the idealized Bayesian predictor for which the prefix is always hard token sequence. A theoretical positive statement is possible if:
Taaget (T) = 5(T = T Target and pite (7 > 0
(8)
that is, we are optimizing for single target task that had support under the pretraining distribution. In this case, there always exists sequence of hard tokens that causes the Bayesian posterior to concentrate sufficiently (in the limit a delta, see Appendix For sufficiently large L.
E for optimal prediction after the prefix.
Arr. E AA (P(e [- log P( Target) }]
~ E Ep(enx|7 [- log Pre (TEN (81.L.
(9)
The prefix be found by performing the minimization in Eq. Proof sketch: If the target distribution is a delta over one of the pretraining tasks (condition in Eq. (8) then the argument that is being minimized in Eq. (7) is the r.h.s. of Eq. (.. The minimum is obtained when Eq. (9) becomes an equality, which the case when the Bayesian (posterior) mixture glee (- collapses to a single mixture component corresponding is only possible iff ę elarget ~ <Pre (11-))
lirget This also implies DxI. [g[aage[]| (-181.1.) Rs 0 which
If the condition in Eq. (8) does not hold, optimal prompting may still be possible, but this strongly depends on the relationship between pretraining and target distribution and the model class. See Appendix B for analysis of the Reta-Bernoull case and beyond, including limits of prompting universal predictors. also possible to formulate two general theoretical negative cases:
Prefix- trefix-tuning limitation I (multimodal target distributions). In the limit posteriors can often not remain (or become) multimodal. If the target distribution is multimodal, such as a finite target mixture over tasks, optimal prefix-tuning may not be possible, even if all mixture components have support under the pretraining distribution. For instance, if the prior (P)((r is log-concave and the likelihood function is log-concave, then the posterior is also log log-concave, and thus unimodal. See Section 4.2 for an empirical demonstration. More generally, if the posterior collapses to a Dirac delta model, and is very likely to do in general if prompts
in the limit (as it does for typical sequences-ae see Appendix E). the only target task distributions that are optimally promptable deltas over single pretraining task.
Except for experiments with untrained networks, where the parameters are random initialization.
5

---

Prefix-tuning limitation II (novel atomic target tasks). The second negative case is when the target distribution contains one or more novel atomic tasks, i... ppre (f (TTaaneet O, that are "sub- stantially" novel in the sense that they require behavior different from any of the predictors in the pretraining mixture <Pre_ F.g., a particular coin bias that was never observed during pretraining under uniform bias over coins would not fall under this case. Note that it may be hard to say what counts as substantially novel at frontier model scale, where the pretraining distribution is only known implicitly, and pretrained models are capable of sophisticated algorithmic prediction. See Petrov et al. (2024) who, in line with our reasoning, find empirically that prefix-tuning methods can "elicit skills present in the pretrained model", but cannot be used to learn novel skills.
Soft prefix-tuning. In general, for prefix prompt to be optimal, the model's internal state after consuming the prefix needs to be sufficient statistic for the target distribution, without causing subsequent internal dynamics to diverge. Pretraining determines model's 's state-update function, and thus imposes strong constraints w.r.t. possible manipulations via hard token inputs. These constraints can be partly overcome by using soft prefixes instead of hard tokens. As our results show, these carefully tuned off-distribution inputs that lead to off-distribution internal states, can be used to very effectively steer pretrained, and even untrained neural predictors. The limits of this mechanistic aspect, outside the conceptual Bayesian theory, are currently unclear-it could be that soft prefixes can very flexibly "reprogram" pretrained model to arbitrary target distributions. Empirically, we find this not to be the case: while Soft Prompting does consistently improve prediction performance on the target task, it is still bound by the theoretical limitations pointed out above and in detail in Appendix E optimally adapting a predictor pretrained over uniform random coins to a mixture of two coins is not possible. How large the potential gains from Soft Prompting or other soft prefi--tuning methods can be is an empirical question. Finally, note that weight-based fine-tuning methods are able to modify the pretrained state-update and -readout mechanisms, which allows for more flexibility w.r.t. adapting a network to target distributions (see Section 4.2 for empirical demonstrations).
4 Experiments on Coin-Flip Sequences
We conduct series of experiments where a neural network is first meta-trained over pretraining distribution (Section (2) of coin flip sequences of length Npr = 100, and then prompt-tuned (Eq. 7 or weight-tuned (mini-bat batch based log-loss minimization) to a target distribution of coin flip sequences of length Numee = 50. After tuning, we evaluate tuned models on 2048 sequences of length N = 200 from the target distribution. Choosing Neval > Nume also allows study how the solutions of different tuning methods generalize beyond the tuning length. Across experiments we use three different data distributions, two neural architectures, and nine tuning methods, which we now describe.
4.1 Experimental setup
Data generators. We use coin-flip sequences P(II.IIT) = Bernuuli(T)) with three different distributions P(r) throughout our experiments. Random coins: P(T) = BetaBett 1, 1), leading a uniform distribution over coin biases. This is our pretraining distribution. The exact Bayesian predictor in this case is the Laplace predictor. Single coin: A single coin with bias 0.2. This target distribution fulfills the condition that makes optimal prompting possible in Eq. (8). Two-coin mixture: mixture of two coins, one with bias 0.2 and one with bias 0.8, with equal mixing weights of 1/2 each. This target distribution violates the condition that theoretically allows for optimal prompting in Eq. S All tasks have binary outcomes which leads to a 2-dimensional one-hot token alphabet A with two different symbols. The Bayes predictors for all three tasks are analytically tractable and textbook examples (for the 'Single coin', the "Bayes" predictor is simply a constant probability).
Neural sequential predictors. We evaluate both LSTMs and Decoder-only Transformers. To support all fine-tuning methods we always use an initial embedding, and final unembedding layer. The embedding is a trainable linear projection from the 2D token space into a 1 128-dimensional "em- bedding" space (results for 4-dimensional embeddings are shown in Appendix I]. The unembedding trainable linear projection from the outputs of the final network layer down to the 2D logits. Implementation Details. The LSTM has single hidden layer of width 1 128; the Transformer has single meati-head attention layer with output dimensionality of 128, 4 attention heads, causal masking, SinCos positional encoding, a widening factor of 4 for the MLP block, and layer normalization after

---

query and key dense layers. Results for larger networks are shown in Appendix J LoRA fine-tuning [Lester et al. 202T] only supported for the Transformer, where we apply LoRA to all dense matrices of the attention block (query, key, value, and final attention weights). To produce a prediction given the empty context, we pass an initial zero vector Pa(a) = T (21 (0). This zero vector is also prepended before any tunable prefix. When reporting the internal state of the LSTM, we use the cell state (hidden state gives qualitatively similar results), and for the Transformer we use the causally masked output of the attention block.
Performance Measure. Our main performance measure is the expected cumulative regret, which the excess log loss compared to the ground-truth data generating probability. In Eq. 4 we defined the expected excess log loss relative to the best predictor that does not know T, a.k.a., the Bayesian regret. Similarly, we now define the excess log loss w.r.t. the data generator, that is, an oracle predictor that knows =T"
R PTgg 0
(N) = B. (( |- log Th (2 (1.L. + log P(x [T")] 2
(10)
We show regret curves from N = 0 up to N = Noval - = 199 steps For prefix-tuning methods i refers to the pretrained weights (or randomly initialized weights in our experiments on untrained networks) and $1.L is the tuned prefix. For Bright-tunnng methods 0 refers to the tuned weights and the prefix is empty (L = O). We (Monte-Carte estimate the regret with 2048 sequences sampled from the target data generator (from which we also get the ground-truie generating probabilities).
Training and tuning details. We pretrain for 1000 gradient steps (batch size 256, sequence length Npr 100, learning rate 0.001, and gradient clipping if the norm is 2 1). For tuning we use 1000 steps (batch size of 256, thus K' = 256, 000, sequence length Nume = 50, learning rate of 5e 5e-3, and gradient clipping if the norm is 2 1). We show tuning loss curves (and their convergence) in the extended results in the appendix. We repeat tuning runs 10 times per method with different random seed (for sampling from the target distribution, and different prefix initialization). Across repetitions and tuning methods, we always evaluate on the same set of 2048 evaluation sequences. Results are reported as the median over repetitions with 25%, 75% quantiles as "error bars".
Fine-tuning methods. We compare four different prefix-tuning methods and five different weight- tuning methods against a number of baselines:
HardPT, SimplexPT, RealPT, SoftPT: prefix-tuning methods (see Section 3). To implement
S AA for SimplexPF, we pass the tunable prefix through a softmax. The prefix length L 6 in all main experiments, and we show L = 25 in Appendix H
EmbedWT, UnembedWT, Un+EmbedWT: Only parameters of the linear embedding, or unembedding, or both, are tuned.
FullWT: All weights, including embedding and unembedding, tuned.
LoRAWT: Low-Rank Adaptation (Hu et al. [021), where an additive tunable low rank matrix (rank=4 in our experiments) is added to all dense matrices of the attention block. All other weights remain frozen.
TargetBayes, PreBayes, PreBayesPT: exact Bayes predictors for the target distribution eTarget, the pretraining distribution <u and (-1s Target) i.e., 'PreBayes' prefix-tuned to the target distribution via exhaustive hard token search, T. 6.
NoTuning: the network with fine-tuning. Either pretrained or at random initialization on our experiments with untrained networks.
RandomPF: Same as 'NoTuning' but with random (one-hot) prefixes.
4.2 Results
Tuning to single task. Fig. I shows that both, Transformer and an LSTM, pretrained on Random Coins, can be Soft Prompted to be Bayes-optimal on a Single Coin (with bias 0.2). Other
'The offset of 1 is because passing - through the network produces prediction r. for which we need ground- fround-truth data step compute the regret log loss (or gradients).
7

---

Tansformer: Random Coins Single Coin
State (attention, Single sequences (length:
No Tunnnng
SimplexPT
RealFT
SoftPT
HardPT
RandomPF
Randem Coins (ne prefir)
Single Coin
LSTM: Random Coins - Single Coin State (cell) on 20 Single Coin sequences (length: 50)
No MoTuinn
SimplexPT
RealPT
SoftT
HardPT
RandomPF
prefad
Single
Figure 2: 2D PCA projection of Transformer's (top) and LSTM's (bottom) internal state (= activa- tions), illustrating how differently tuned prefixes affect state and subsequent dynamics. Fig. A7 shows that the vertical principal component corresponds the step n, and the horizontal to the heads-to-tails ratio. Colored lines sequences from the target distribution (single coin with bias 0.2), gray lines are from the pretraining distribution (uniform random). The off-distribution nature of soft prefixes is particularly visible for the Real- and Soft-prefix for the Transformer. See Fig. T for regret curves.
prefix-tuning methods, including exhaustive search over all hard token sequences of length 6 fail to reach Bayes-optimality. Despite all soft prefixes being off-distribution inputs, internal dynamics remain stable (see Fig. ( and prediction generalize well for 'SoftPF' and most of the weight-tuning methods far beyond the tuning sequence length Nuce 50 (see Fig. [T. The results illustrate how prefixes can be used to steer (meta-learned) Bayesian predictor via manipulating its internal state. Further, they also show that distriitbbuuonn inputs can be particularly effective; more effective than even the best possible hard token sequence of the same length.
Note that for all prefix-tuning methods the dimensionality of S is the same the one-hot token alphabet (i.e., 2-dimensional for coin-flip tasks), except for 'SoftPT" where the prefix embeddings of dimensionality 128 are tuned. These additional degrees of freedom performance in our experiments, and we demonstrate that the advantage largely disappears when reducing the embedding dimensionality than embedding dimensionality, tuning inputs ("RealPT") may be as efficient, or even more efficient, compared tuning embeddings ("SoftPT)) for LLMs.
the main source of superior
in Appendix Since LLMs typically have larger input-
Limitation: prompting to task mixtures. Fig. empirically demonstrates the theoretical shortcom- ing of prefix-tuning discussed in Section 3 prompt tuning mixture of two coins is not possible if the pretraining distribution is uniform random coins. While Soft Prompting, being the strongest prefix-tuning method, leads performance gains compared to the untuned pretrained predictor, is not enough to reach 'TargetBayes" on the Two-Coin Mixture. As expected, some weight-tuning methods can lead to that level of performance, the cost of permanently altering the pretrained predictor. While soft prefixes are strictly speaking not covered by the Bayesian theory (because they are off- di-diitribution inputs exploiting the circuitry of the particular pretrained network). these results highlight the importance of the Bayesian view in practice: in the absence of theoretical understanding might have been quite puzzling why one can optimally prefix prompt for single coin but not mixture of two coins. We confirm that the result is not an artifact of limited soft prefix length see Fig. A10 for a control experiment with 25, and Fig. A12 for results with larger networks.
Comparing prefix- and weight-tuning for untrained networks. Fig. 4 shows that it is possible to Soft Prompt an untrained Transformer quite well the Two-Coin Mixture and Random Coins as target tasks, meaning that relatively complex in-context algorithms are easily available in the untrained net. There still a gap to 'TargetBayes' performance though. Results the Appendix (Figs. A8 and A9 show that Soft Prompting the untrained LSTM has very little effect, indicating

---

Transformer: Random Cains Two Coin Mixture
Transformer: Random Coins Two--Con Mixture Evaluation on Tivo-Coin Mixture
Evaluation on Coin Mixture
Instant regret [nats]
0.4
49 (Nunc)
199 (Nera)
cumulative regret (nats]
Profte
Weight
Baselinex
Trofte
Waight
Dsselinns
20
0.2
15
10
0.0
A:37
100
150
175
200
6.40
6.40
2,00
Cumulsative regret [nats]
20-
15-
Hardir
Simpluut
Rooipt
Softhy
No Bonmmeg
Prod Rardomar
Hardtry Stnglauth
Readry
Scatio
Unemmed
Un Eneeenno
Torgetba
Prodaaves eer
mellovala
Randemmsy
10
LSTM: Random Two-Coin Mixture Evaluation on Two-Coin Mixture
(Nusee
199 (eeaa)
150
Step
Cumulative regret [nats] w
Weight
Top: Performance of different tuning meth- ods, measured as excess log loss, i.e., re-
gret (Eq. <IO). lower is better). See bar plots for color legend. Top-right: Detailed Transformer results for last step within the
1:42
8.79
0
tuning sequence length Nume and the last evaluation step Neva. Right: Like above but for LSTM.
Haratry
Simplest papieerry
Roality
Satbey
Fuinir
Embvolir Ainenbrolly Un- Eneemmbbai ex Bor aareetlyg es Protay praaayes Ho Riming
Prot Rondomin
HarrfP Simplest Realir
sonte
embyyott W
Un+EmbooW EmbootVT
Une target Boyees Pratay
No Rining Pred Randomen
Uneed
Figure 3: Models pretrained on sequences from coins with uniform random bias (length Nuain 100) are fine-tuned to the target task of a mixture of two coins (tuning sequence length Nume = 50). No prefix-tuning method (with prefixes of length 6) can achieve optimal performance on the target task ("TargetBayes' is optimal). Full weight-tuning, LoRA (on the Transformer) and two of the embedding tuning variants the LSTM do reach optimality (even beyond the tuning length of 50 steps). See Fig. A7 for a visualization of how different prefixes affect models' internal dynamics. Regret curves for the LSTM, similar to Top left panel, are shown in Fig. A6
a fundamental difference between the Transformer and the LSTM in this regard (which behave very similarly our experiments when pretrained). See also and Andreas (2024), who tune untrained Transformers to algorithmic tasks via embedding- or unembedding-tume -tniigttnnnnggg or both. They find that, tuning both the embedding and unembedding important on their tasks. If our results qualitatively hold at their tasks, then Soft Prompting of untrained networks should be only slightly worse, and LoRA should perform even better than Un+Embedding tuning. As previously, note that the superiority of "ToftPT" in our setting is largely explained by having a much higher embedding dimensionality compared the input dimensionality (which is typically reverse at L.M scale). Increasing the soft prefix length to L = 25 does not significantly improve performance, see Fig. A10
Discussion
While we have laid important fundamental groundwork in our current study, extrapolating our findings to modern frontier model (and data) scale is not straightforward. The theoretical findings we presented, including the limitations of prefix tuning, hold at any scale, but it is likely that additional practical issues arise at large scale that are not captured by our current experiments. is thus hard to predict the relevance and impact of our fundamental results on today's 's frontier-model practice. For instance, one of our main results is that optimal prompting to a single target task is possible, whereas it is not for mixture of tasks. We are confident that this holds even at frontier model scale (based on the theory), but it is unclear and highly non-trivial what constitutes a task for a LLM, and accordingly, whether this is severe limitation not. In our experiments, a task is simply an unobserved variable in two-level hierarchical statistical model. At L.L scale, the structure is vastly more complex, with many more hierarchical levels, and potentially other statistical structures at play.
9

---

Transfomer untrained Two Coin Moxure Evaluation on Twc--Con Mixture
Transtormer, untrained - Random Coins
Evalustion on Random Coins
n= 49 [f6
0- 199999999 (Meva)
n= 49 (f6.ze)
0= 199 (flaas
Cumulativn regret (nats)
Camulative regret (nats)
1.06
Sutayy
costiors
Use invesstentl
Love Poe--eoooor
tacgett poceetinng tntoniig
seart
En Enstertayy
upy Enstrestty bergettee int
standry Soreitar
Sodtys
Calleur
tin iimeettentrs
Une Peveeroonnn
He Hebentiy Fondooni
mestty seary
the contoret aetterades
sien bargett (no
(a) Untrained Transformer tuned to Two-Coin Mixture. (b) Untrained Transformer tuned to Random Coins.
Figure 4: Untrained Transformer tuned to the Two-Coin mixture (left) and to Random Coins (right);; tuning sequence length Nume 50. In both cases, Soft Prompting is the only effective prefix-tuning method. It nearly reaches Bayes-opumality ("TargetBayes', which is a Laplace predictor on Random Coins). Performance degrades rapidly after the tuning sequence length. Full regret curves (and LSTM results) in Fig. A8 and Fig. A9 Among the weight-tuning methods, LoRA is very effective.
The next step would be to carefully design data generators that are closer to natural language data, but still fully understood and well controllable, akin to the data generators used in [Allen-Zhu, 2024). and run our experiments of tuning to single tasks vs. tuning to mixtures of tasks at scale. With these caveats in mind, cautiously extrapolating our findings to frontier model scale, raises some questions for investigation, which we now discuss.
Given our results, soft prefix tuning should be superior to tuning sequences of hard tokens- if sufficiently large fine-tuning set is available, soft prompt tuning should beat prompt engineering, and other methods of hard token optimization such as PrompiBreeder [Fernando et al. 2023 Similarly, instead of conditioning on large set of in-context examples for imitation learning ge.g.. Ruoss et al.) (2025). Paglieri et al. 2025)). D. it may be beneficial to distill these examples into more effective tuned soft prefix. The superionity of Soft Prompting over other soft prefix-tuning methods is largely due to the much higher dimensionality of "embeddings" compared to the input space in our experiments (see control experiments with embedding dimensionality in Appendix (I. This is typically reversed in frontier models, which could mean that soft input tuning is more effective than embedding tuning.
Finally, our experiments raise the question: why prompt (-tune) all, when weight-based methods, such as LoRA, are equally or more effective and do not suffer from the theoretical limitations pointed out? First, weight-tuning permanently alters a network and would lead to performance decreases on the pretraining distribution (which can be overcome by storing the set of original weights). Additionally, comparisons between in-context and in-weight learning at L.LM scale find that in-weight learning can sometimes be very limited and generalize poorly (Lampinen et al. 2025 Chan et al.) 2022b]. Since prompting fundamentally builds on a network's in-context adaptation mechanisms, it may be the case that prompt-tuning works better than weight-tuning in cases where in-context learning generalizes better than we-whttht learning. An interesting future research question is whether tuned (soft) prefixes transfer between different models (perhaps with additional regularization) if true, the prefix-tumi -uning cost would only have to be spent once, compared to weight-tunin methods that need to be run for every network to fine-tune. Please our discussion of additional related work in Appendix D where we discuss a number of previous works that investigate in-context learning under Bayesian and/or meta-learning lens, such as Xie al. 202T Kirsch et al. [2022 Lampinen
et al. 2024), and Elmoznino et al. 2024].
To conclude, the Bayesian view on prompt tuming, which arises from analyzing memory- based meta- learning, provides a conceptual understanding that leads to a formal characterization of prompting and some of its fundamental limitations. We have shown that these limitations hold in practice, for both hard token prefix-tuning the theory (which does not consider non-token inputs). The code to reproduce all our experiments and figures is available at: bittps://githa: com/ googloogleeeeo detiggllddeemiidddthieeeee tuddnhuttiiinniiiii
but also when optimizing soft prefixes a setting not fully covered by
'Though some hard-token tuning methods may have the advantage of resulting in more interpretable prompts.
10

---

Acknowledgments and Disclosure of Funding
We thank Joel Veness, Gregoire Deletang, Satinder Baveja and Shane Legg for helpful feedback and discussions.
References
R. Agarwal, Singh, L. Zhang. B. Bohnet, L. Rosias, S. Chan, B. Zhang, A. Anand, Z. Abbas, A. Nova, al. Many-shot in-context learning. Advances in Neural Information Processing Systems, 37:76930-76966, 2024.
E. Akyürek, D. Schuurmans, Andreas, T. Ma, and D. Zhou. What learning algorithm in-context learning? investigations with linear models. The Eleventh International Conference on Learning Representations, 2023.
Allen-Zhu. ICML 2024 Tutorial: Physics of Language Models, July 2024, Project page: httpp:// allen- chu.com/
L. Bailey, G. Ahdritz, A. Kleiman, S. Swaroop. F. Doshi-Velez, and W. Pan. Soft prompting might be a bug, not feature. In International Conference on Machine Learning, 2023.
J. Bauer, K. Baumli, F. Behbahani, A. Bhoopchand, N.
M. Chang, N. Clay, A. Collister, V. Dasagi, L. Gonzalez, al. ftuman-timescafe adaptation in open-ended task space. In International Conference on Machine Learning, pages 1887- 1935. PMLR, 2023.
M. Binz, T. Dasgupta, A. K. Jagadish, M. Botvinick, J. X. Wang, and F. Schulz. Meta-learned models of cognition. Behavioral and Bruin Sciences, 47:e147, 2024.
Blackwell and L. Dubins. Merging of opinions with increasing information. Annals Mathemaiical Statistics, 33:882 887, 1962.
Bornschein, Li, and M. Hutter. Sequential learning of neural networks for prequential mdl. The Fleventh International Conference on Learning Representations, 2023.
B. Chan, X. Chen, A. György, and D. Schuurmans. Toward understanding in-context vs. in in-weight learning. In The Thirteenth International Conference on Learning Representations, 2025. het:/ooummiid-aKKrShiee
S. Chan, A. Santoro, A. Lampinen, J. Wang, A. Singh, P. Richemond, J. McClelland, and F. Hill Data distributional properties drive emergent in-context learning transformers. Advances in neural information procexsing systems, 35, 2022a.
C. Chan, I. Dasgupta, J. Kim, Kumaran, A. K. Lampinen, and F. Hill. Transformers generalize differently from information stored context vs in weights. arXiv preprint arXiv:2210.05665, 2022b.
G. Deletang, A. Ruoss, P.A. Duquenne, E. Catt, T. Genewein, C. Mattemn J. Grau-Moya, L. Wenliang. M. Aitchison, L. Orseau, el al. Language modeling is compression. In The Twelfth International Conference on Learning Representations, 2024.
E. Elmoznino, T. Marty, T. Kasetty, I.. Gagnon, S. Mittal, M. Fathi, D. Sridhar, and G. Lajoic. In-context learning and occam's 's razor. arXiv preprint arXiv:2410. 14086, 2024.
C. Fernando, D. Banarse, H. Michalewski, S. Osindero, and Rocktäschel.. Promptbreeder: Self-referential self sefroimttovement via prompt evolution, 2023, htt.aittpss//arvv.rrr/aab///////
16797
S. Garg, D. Tsipras, P. S. Liang, and G. Valiant. What can transformers Iearn in-context? study of simple function classes. Advances Neural Information Processing Systems, 35:30583-30598, 2022.
T. Genewein, G. Delétang, A. Ruoss, K. Wenliang, E. Catt, V. Dutordoir, J. Grau-Moya Orseau, M. Hutter, and Veness. Memory-basee based meta- learnin on non-stationary distributions. International conference on machine learning, pages 11173 11195. PMLR, 2023.
J. Grau-Moya, T. Genewein, M. Hutter, L. Orscau, G. Deletang, E. Catt, A. Ruoss, K. Wenliang, C. Mattern, M. Aitchison, et al. Learning universal predictors. In International Conference on Machine Learning, pages 16178-16205. PMLR, 2024.
Z. Han, C. Gao, J. Liu, J. Zhang, and S. Q. Zhang. Parameter-efficient fine fine-tuning for large models: A comprehensive survey. Transactions Machine Learning Research, 2024.
11

---

D. Heurtel-Depeigee A. Ruoss, J. Veness, and T. Genewein. Compression via pre-trained transformers: A study on byte-level multimodal data, arXiv preprini arXivv2410.05078, 2024.
E. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. S. Wang, L. Wang, and W. Chen. Lora: Low-rank adaptation of large language models, 2021. URL htttss:s://arxi htt////rxivvorggbbba/r/aa wtt.orr///aa///a/eoaa/////// 09685
M. Hutter. Universal Artificial Intelligence: Sequential Decisions based Algorithmic Probability. Springer. Berlin, 2005. ISBN 3-540-2213--5. doi: 10.1007/b118233, CtLhhtt:///we http://////.ttt////////ttttrrrr. wtw.:/uutter...t/a/aaaaaaaaeaaaa htm
M. Hutter, D. Quarel, and E. Catt. An Introduction to Universal Artificial Intelligence. Chapman & Hall, 2024.
.htm
L. Kirsch, J. Harrison, J. Sohl-Dickstein, and L.. Metz. General-purpose in-context learning by meta-leaming transformers, arXiv preprint arXiv:2212.04455, 2022.
A. Lampinen, S. C. Y. Chan, A. K. Singh, and M. Shanahan. The broader spectrum of in-context learning, 2024. URLpttttss//arxxx org/ abs/ 2abs/2212003782
A. K. Lampinen, A. Chaudhry, C. Y. Chan, C. Wild, D. Wan, A. Ku, J. Bornschein, R. Pascanu, M. Shanahan, J. I.. McClelland. On the generalization of language models from in-context learning and finetuning: a controlled study, 2025. URL https::::: : // a/axiiiivvv..a//aaaaaaaaaavoaa org/ abs abs/2505..... 2ttp//55500000611
M. Laskin, Wang, J. Oh, E. Parisotto, S. Spencer, R. Steigerwald, D. Strouse, S. Hansen, A. Filos, E, Brooks, et al. In-context reinforcement learning with algorithm distillation. In The Eleventh International Conference on Learning Representations, 2023.
B. Lester, R. Al-Rfou, and N. Constant. The power of scale for parameter-efficient prompt tuning, arXiv preprint (tti:21004006991 2021.
M. Li, P. Vitányi, et al. An introduction to Kolmogorov complexity and its applications, volume 3. Springer, 2008
X. L. Li and P. Liang. Prefix-tuning: Optimizing continuous prompts for generation. In International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2021.
D. J. MacKay. Information theory, inference and learning algorithms. Cambridge university press, 2003.
A. V. Mahankali, T. Hashimoto, and T. Ma. One step of gradient descent is provably the optimal in-context learner with one layer of linear self-attention. In The Twelfth International Conference on Learning Representations, 2024.
V. Mikulik, G. Delétang, T. McGrath, T. Genewein, M. Martic, S. Legg, and P. Ortega. Meta-trained agents implement baycs-optimal agents. Advances in neural information processing systems, 33:18691-18700 2020.
S. Müller, N. Hollmann, S. P. Arango, J. Grabocka, and F. Hutter. Transformers can do bayesian inference. In International Conference on Learning Representations, 2022.
P. A. Ortega, J. X. Wang, M. Rowland, T. Genewein, Z. A. Pritzel, P. Sprechmann, S. M. Jayakumar, T. McGrath, K. J. Miller, M. G. Azar, I. Osband, N. C. Rabinowitz, A. György, S. Chiappa. S. Osindero, W. Teh, van Hasselt, de Freitas, M. M. Botvinick. and S. Legg. Meta-learning of sequential strategies. arXiv, abs/1905 0111199550000,,, 2019. URL http:///t// org/ aba/ 090000
R. Pascanu, N. Heess, J. Veness,
D. Paglicri, B. Cupiat, S. Coward, U. Piterbarg, M. Wotezyk, A. Khan, E. Pignatelli, E. Kucińsski L. Pinto, R. Fergus, J. N. Focrster, J. Parker-Holder, and T. Rocktäschel. Balrog: Benchmarking agentic Ilm and vlm reasoning on games. In International Conference on Representation Learning, 2025.
M. Panwar, K. Ahuja, and N. Goyal. In-context learning through the hayesian prism. In The Tweifth International Conference on Learning Representations, 2024.
O. Patel, J. Wang. N. S. Nayak, S. Srinivas, and H. Lakkaraju. Towards interpretable soft prompts. arXiv preprint MwXiv:2504.02144 144, 2025.
A. Petrov, P. Torr, and A. Bibi. When do prompting and prefix-tuning work? a theory of capabilities and limitations. In International Conference on Learning Representations, 2024.
Y. Qin, X. Wang, Y. Su, Y. Lin, N. Ding, J. Yi, W. Chen, Z. Liu, Li, Hou, et al. Exploring universal intrinsic task subspace via prompt tuning, arXiv preprint 2rX.v:2100.07867 2021.
12

---

S. Rathmanner and M. Hutter. A philosophical treatise of universal induction, Entropy, (316))0776-111 2011. ISSN 1099-4300. doi: 30.636. UKL.Itttp//aarzav.or///b///111 5721
A. Ruoss, F. Pardo, H. Chan, B. Li, V. Mnih, and T. Genewein. IMAct: A benchmark for in-context imitation learning with long multimodal demonstrations. In International Conference Machine Learning, 2025.
D. Schuurmans, Memory augmented large language models are computationally universal. arXiv preprint :XXv:2331.004889 2023.
D. Schuurmans, H. Dai, and F. Zanini. Autoregressive large language models computationally universal, 2024. URLhttpes /// arxiv. org/ berg/abs/20 abs//410. 03170
A. Singh, S. Chan, T. Moskovitz, E. Grant, A. Saxe, and F. Hill. The transient nature of emergent in-context learning in transformers. Advances in Neural Information Processing Systems, 36:27801-27819, 2023.
A. K. Singh. T. Moskovitz, S. Dragutinovie, F. Hill, S. C. Y. Chan, and A. M. Saxe. Strategy coopetition explains the emergence and transience of in-context learning, 2025. WRRp:ttps://aatt
http://orrraabaaaaeeeeeeeee 05631
Y. Su, X. Wang. Y. Qin, C.-M. Chan, Y. Lin, H. Wang. K. Wen, Z. Liu, P. Li, J. Li, I.. Hou, M. Sun, and J. Zhou. On transferahility of prompt tuning for natural language processing. In M. Carpuat, M.-C. de Mameffe, and Meza Ruiz, editors, Conference of the North American Chapter of the Association for Computational Linguisticx: Human Language Technologies, pages 3949-3969, 2022.
Von Oswald, Niklasson, Randazzo, J. Sacramento, A. Mordvintsev, Zhmoginov, and Vladymyrov. Transformers learn in-context by gradient descent. In International Conference on Machine Learning, pages PMLR, 2023.
J. X. Wang, Z. Kurth-Nelson, D. Tirumala, H. Soyer, J. Z. Leibo, R. Munos, C. Blundell, D. Kumaran, and M. Botvinick. Learning to reinforcement learn. arXiv preprint arXiv: 1.05763, 2016.
X. Wang, W. Zhu, M. Saxon, M. Steyvers, and W. Y. Wang. Large language models are latent variable models: Explaining and finding good demonstrations for in-context learning. Advances in Neural Information Processing Systems, 36: 1ee115111-1664 2023.
L. K. Wenliang, A. Ruoss, J. Grau-Moya, M. Hutter, and T. Genewein. Why is prompting hard? understanding prompts binary sequence predictors. arXiv preprint 1.0i::2502.1076000 2025.
S. M. Xie. A. Raghunathan, P. Liang. and T. Ma. An explanation of in-context learning as implicit bayesian inference. arXiv preprini arXiv:2111.02080, 2021.
Y. Zheng, 7. Tan, P. Li, and Y. Liu. Black-hox prompt tuning with subspace learning. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024.
Z. Zhong and J. Andreas. Algorithmic capabilities of random transformers. In The Thirty-eighth Annual Conference Neural Information Processing Systems, 2024. URL http::tt////ooeeeeeieee net.oeruuuee//ouu id=plHBgW7cPQ
13

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We list our main claims and contributions at the end of the introduction. Each claim is supported by later theoretical section empirical result(s) in the Results section.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
Minor limitations and issues are discussed throughout, and we have included
an explicit section (titled 'Limitations and Scope" at the end of the introduction) to discuss limitations.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
14

---

Answer: [Yes]
Justification: Proofs are either provided in the main text or appendix, and for many of our theoretical statements they textbook material
be found in previous publications. The main theoretical contributions are to situate these known results in the context of prompt tuning, and make them more widely accessible in the context of frontier model training and tuning.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof' provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We provide full experimental details in the main paper and appendix. In addition we release the full source code reproduce all experiments, analysis, and plots, as open-source repository.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility.
15

---

In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Our code to fully reproduce all experiments, analysis, and plots, is released open source. To protect anonymity we do not link to the open-source repository in the submitted version.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurIPS code and data submission guidelines (https
//nips.cc/put wwtpcccccppccccceee cepollcc/giieee /ttpe/gguudddeeeeeuuuuubieeeeee htt///ooeSubiiss.. colomisssions ssionpolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification:
The method section and appendix specify necessary details.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: All our experiments 10 repetitions with different random seed, and report median results and 25%, 75% quantiles as "error bars", along with full results per repetition, for all our experiments.
Guidelines:
16

---

The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report I-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The appendix provides this information.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https::t///// //tprrr///////is////ee httpc//cbb.c//t/hcccc.c........
Answer: [Yes]
Justification:
The research is fully compliant with the NeurIPS Code Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdictio))
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer:
[Yes]
Justification:
We discuss broader impacts in separate section at the beginning of the
appendix.
17

---

Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out, For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: No such data aor models are released part of this publication.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification:
The paper does not use third party assets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
18

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thood cww datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer:
[Yes]
We release our code in an open-source repository that includes all necessary documentation.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question:
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: No such experiments.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
No human studies this paper.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
19

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: LLM usage is explicitly stated in a separate section of the appendix. No LLM used in authoring editing this paper and the accompanying codebase.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https://neuri)) hatt:e///eerripsccc/oooofeeeeeee for what should or should not be described.
20

---

A Societal impact
While LLMs and frontier models have broad and significant societal impacts today, and increasingly so in the future, our work aims at understanding one of the fundamental conceptual mechanisms w.r.t. how rapid in-context learners can be steered via prompt optimization. Our analysis is theoretical, our experiments are educational, and we do not propose novel, more powerful methods. This fundamental understanding may enable the development of better methods steer frontier models more precisely and more data efficiently, which could further boost societal impacts (both positive and negative) and facilitate abusing or attacking models via prompts as well as defending and hardening against such attacks with better system prompts. Weighing up these (hypothetical) factors, we firmly believe that better understanding generally leads to more robust and safe technology.
B
LLM usage
There was no LLM use involved in authoring this paper and its experiments. No part of this paper and the accompanying code was authored modified by, inspired through conversations with an LLM. Smart auto- was used when writing code, which is partly powered by coding model, but no LLM or coding model was explicitly prompted to (co-)author or edit parts of the codebase.
C Compute usage
The educational experiments presented in the paper were run on a single V100 GPU in under hours.
D Additional Related Work
In-context learning has been studied extensively in the recent literature. In many cases, it specifically refers to a particular type of supervised few-shot learning. contrast, [ampinen et al. [024] (among others) argue that whole number of LLM in-context abilities can be unified as in in-context learning in a wider sense: "[...] we suggest that any distribution of sequences in which context non-trivially decreases loss on subsequent predictions We suggest that this perspective helps to unify the broad set of in-context abilities that language models exhibir.". This is in line with view on in-context learning, and we argue that the Bayesian perspective that arises from analyzing memory- meta- meta-learning provides the unifying theoretical framework. Lampinen al. (0244] also discuss memory- based meta-learning as the underlying factor. We present this connection in more formal detail (see Section
be interpreted eliciting a kind of in-context learning.
[ and use it to drive the design of our experiments. Bayesian inference has also been put forward an explanation for the mechanism that
drives in-context learning in Xie et al.] 2021), Mulller et al. 2022 Genewein et al. 20033] Binz et al.
2024 Wang et al. 22033] Panwar et al. [2024).
The theoretical aspect that ties together meta- deta-learnnng, Bayesian inference, and optimal prompting is minimization of prediction error (log loss). A dual, and fully equivalent view is maximizing (lossless) compression objective. [Deletang et al. 2024) discuss this well-known duality [MacKay 2003) in the context of language modeling and show that pretrained LLMs are surprisingly good compressors for image and audio data. This is further expanded by Heurtel- (medium-sized) pretrained transformers' in-context learning abilities can lead to lossless compression on par with general-purper great recent theoretical discussion on in-context learning and how it arises from meta-training and relates to algorithmic compression, see [Emoozino et al. (2024). Their theoretical discussion is complementary to ours: shifting to an algorithmic statistical view [Li al. 2008 Hutter et al. 2024). as they do, allows to more easily make statements about generalization in contrast, the classical statistical view requires distributions, which makes it harder to formally characterize off-distribution generalization. We have focused on the latter for simplicity and conciseness, but note that Bayesian inference and in-context learning carry over into algorithmic statistics by considering distributions over programs [Raathmanner and Hutter 2011 I Hutter et al. 2024]. The algorithmic, and classical Bayesian view are thus largely equivalent, and simplicity prior similar to what is discussed in Elmoznino et al. [2024] also appears an "automatic" Bayesian Occam's 's Razor [MacKay 2003 in classical Bayesian inference and nog-atgrii minimum description-length (MDL)- though
et al. [2024], who show that
compression algorithms, such as gZpp, across different modalities. For
in the classical case the simplicity prior is not Kolmogorov complexity. The main point is that
21

---

Bayesian mixture predictor over large class of programs is not at odds with our investigation in this paper. For frontier model scale, such an intuition may be more appropriate. as it would imply that very complex algorithms can be executed in-context, including sophisticated learning algorithms (see also [Schurranns] [023). [Schuurmans et al. [024] for a discussion of how L.LMs are universal in principle). This also means that more specific in-context learning algorithms can be identified in particular settings, without violating the Bayesian view-e.g.. explanations of in-context learning as gradient descent [Von Oswald et al. 2023 Mahankali et al., 2024], linear and ridge-regression [Akyürek et al. 2023]. and learning linear functions [Garg et al. 2022].
We explicitly meta-train on simple tasks for our experiments, which allow for the comparison against a known tractable Bayesian predictor, similar to MMikulik et al.) (2020), Genewein al.) (2023).
Wenliang et al. [2025]. Other works have also used simple synthetic examples to study in io-context
learning and develop algorithmic understanding [Akyürek et al. 2023 Garg et al. 2022 Mahankali
et al. 2024 Elmoznino et al.) 2024]. Beyond simple examples, explicit meta learning at scale can give rise to complex adaptive in-context algorithms. For instance, |Bauer et al. 2023) meta-train an agent that adapts in-context at human timescales, in terms of number of interaction episodes, to a vast number of tasks in a simulated 3D environment. Another notable example is Laskin et al. [2023), who meta-train an in-context reinforcement learning algorithm (see also Wang et al. (2016). While LLMs are not explicitly meta-tras trained, Chan et al.] [022a]] argue that naturalistic data like language has many of the properties of meta-learning datasets and show that these properties drive in-context learning.
To adapt (large) pretrained models, many fine-tuning methods, with a number of variations each, have been proposed in the recent literature. See See Han et al. (2024) for a review of parameter-efficien fine-tuning methods, such as, soft prompting [Lester et al. [02T]] prefix prompting [Li and Liang) 2021], or LoRA, Hu et al. (202T]. Whether a method falls under prefix-tuning or weight-tuning may not always be immediately obvious, since tunable inputs also appear indirectly tunable parameters in query and key matrices of transformers. Careful analysis reveals though, that tunable parameters resulting from prefix-tuning methods are more constrained compared to weight based tuning. Petrov et al. [222]] perform such an analysis and find: "[...] while techniques like prompting, in-context learning, soft prompting, and prefix-tning can effectively elicit skills present in the pretrained model, they may not be able to learn novel tasks that require new attention patterns." ". This empirical finding is in line with one of the theoretical negative results for prompting that we point out in Section 3 The difference between in-context learning (which is the mechanism that prompt tuning exploits) and in-weight learning has been studied in series of works (Lampinen et al.) [025 Agarwal et al.) 2024. CChn et al., 2022b) that find that in-context learning can typically generalize more flexibly than weight-tuning. and that the underlying mechanisms are quite different (e.g., rule-based vs. exemplar-base generalization). and may result from different neural circuits that compete during training (Singh et al. 2003] 2025). ), as well as different underlying properties of the data distribution [Chan et al. 2025]. Kirsch et al. [2022] study ceneral-purposs in-context learning via meta-learning and investigate the factors that lead models to generalize (via meta-learned in-context algorithms) as opposed to memorization. An interesting approach presented in Bornschein et al. [2023) is to switch from in-context to in-weight learning as soon as the available examples allow (in terms of number of samples and statistical properties w.r.t. the pretrained predictor), which is determined via prequential evaluation.
One large downside of soft prompts may be that hard token sequences are potentially more inter- pretable. The literature on interpretability and understanding of prompts and prompting techniques is growing rapidly and surfaces complex problems Patel et al. 20255] Bailey et al. (2023) Petroy [2024).articularrl Particularly Bailey et al. [2023) find that soft prompts are generally hard to intrepret. also when trying to map them to hard token sequences. /Su et al. [2022), Qin et al.) [2021]. Zheng et al.)
al. [2024)
[024] focus on understanding prompts through task sub-spaces and how they enable transfer between tasks. At least some of the issues may not be specific to soft prompts, as analyzed by Wenliang et al. 2025). which discusses number of theoretical and fundamental practical issues with identifying and interpreting optimal hard-token prefixes, such as a high sensitivity of optimized prompts on the pretraining and target distribution (including aspects like sequence length and fine-tune set size). They also find some indication that LSTMs can have more easily interpretable hard prompts compared to Transformers.
22

---

E Theoretical limits of prompting Bayesian predictors
Even for idealized Bayesian predictors, and hard prefix prompting (i.e., no inputs outside the token alphabet), the theoretical limits of what can be achieved via prefix prompting are non-trivial. We first discuss the Bernoulli case, as it is most relevant to our experiments. We then extend the model class to show two constructions for which optimal prompting is possible-arguably the constructions somewhat artificial, and do not relate to current paradigms of pretraining frontier models. We then ask whether optimal prompting to arbitrary target distributions is possible for universal Bayesian predictor, i.e., the Solomonoff mixture [Rathmanner and Hutter] [011]] Grau-Moya et al.) 2024]. The answer is yes, but optimal prompts may need to be very long (and grow in length with increasing approximation quality). Whether the Solomonoff mixture can be 'efficiently' prompted remains an open problem. Finally we show that on average prompting narrows the task distribution, but atypical prompts can widen it.
Bernoulli mixtures. Let μo (3(m)) := 0* (1 (1-10 -1)))= with hn := I1 +...to and Dy E B = {o. 1)
be a Bernoulli(@) process with prior (density) w(() for 0 10: Then the posterior is w( (0 (z(11m)) )orkm))(((e (0)) (w (x1m), where normalizer = mixture = evidence Su (21m) == So ( ) (w)) (0)d0.
If w0) is Beta prior (e.g. uniform), then the posterior is also Beta-distributed with variance S ((n-k(()))))))))) - k)/n" < 1/4n + 0 for n + cc. Hence the posterior of w "converges" to a ô-peak whatever 11:00.
Var
More generally, for any prior such that wo and sequence :1:00 for which all limit points of 6,, == kin /' are in the support of w, we have Varlur( (=(0))))]] -> 0. Note this holds even if kn/n n itself does not converge.
Also, if the prior w0) is log-concave (e.g. uniform), the posterior w x) is also log-concave, and hence unimodal. In particular, in these cases there is no prefix such that the predictive distribution (.v( is a mixture of two (or more) Bernoullis.
Situations for which the posterior does not collapse are rare and somewhat artificial: for instance if 2
the prior w0) 50 - }) + 1 5(0 }) i.e. Ç is a mixture of just two Bernoullis, and k n/2 then the posterior is also a mixture over two Bernouellis w ((f[ = w(). Or if the prior has a gap: v0) [0 S v@ 0 2 3 and
n2/2, then the posterior will retain the gap and remain bimodal.
In summary: under many priors, including Beta priors, the posterior collapses to a delta (zero variance) under increasing observations. No prefix can thus lead to a posterior over mixture of, say, two coins with different bias. Even if the posterior has not fully collapsed, starting with a log-concave prior can only lead unimodal posterior. Exceptions are technically possible, e.g., when the prior is already a mixture over two components or has gap.
Countable mixtures. One can ask whether a larger (countable) class of distributions M = {w} always allows for optimal prefix-tuning: Let the mixture {(x) = Σvem (zz)((()) with some prior T())= > O. Indeed, for suitable M it holds that: for every computable (Bernoulli) prior w there exists some y E IF' such that {(xly) = Sue (z) for all x E B*. That is, one can prompt & such that its predictive distribution becomes any desired Bernoulli mixture. The construction is quite artificial though: we require that the model class is such that the prefix is interpreted as program that explicitly computes the desired target prior. Formally, let uy. = [x<t = p Sn (ZE:)) (u ). where p B is prefix program computing prior w() and E =length(p) (i.e. P == (p p computes some w) is a pree set). Let M = (up= : p E P). Then it is easy to see that { (x() Cw (Ze.)) where t<t E P computes prior w.
This class is very artificial, but it shows that prefix-tuning for general Bernoulli mixtures is possible in principle. No special property of Ew was used, so the above tuning construction works for arbitrary countable base class B = }..)
Product mixtures. To overcome the need that the prefix is a program that computes the prior,
another construction possible, where the prompt is number of increasingly longer sequences of samples from the target distribution. Let B = {c} be a countable class of target distributions (e.g. Bernoulli mixtures). Let = in < i1 < i2 < ... be a sequence of temporal boundaries with
increasing segment lengths 8, ;= in i--1 + co for + co, c.g. 8,, K or 8, 2" De-
23

---

fine the product distribution as (1m) I==1 - for n im - 1, and for other by marginalization. Let {(x:) := E<e Vg ())((( be Bayes mixture with some prior v 0. oo we have (( + [ a.s. if 11:20 ~ Vç. If km h t= bm-1 for some m. then ( = (( The condition satisfied for infinitely many l. Hence for any Ç E B and any h E I there exists a prompt <ee such that ( (T<ce) = ((xeeen=1) for all 1Ee+h-1 E IB*, and the approximation eπrrr can be made arbitrarily small by suitably large E.
Then by [Hutter 2005 Sec.3.7.1] or Blackwell and Dubins [9622] for any
Compared to the countable mixture construction from before the constructed class and prompt sam- pled from a product of the target distribution Ç are more natural. As we will see below, Solomonoff's M can also be prompted in this way. The downsides of this construction E(xly) R ((x) is non-uniform in the length x and longer x require longer prompts y. Additionally, the required prompt y is typically much larger than program prompt p in the countable mixture construction above.
that the approximation
Solomonoff mixture. Finally, we ask whether optimal prompting any target distribution is always possible for a universal predictor. Let M be Solomonoff's a priori distribution, and ( be some computable distribution, and ig. be computable index sequence. Then ve is also computable, hence included in the mixture M (M(-) 2 C- vc(-) for some constant e > Q). The same argument as before implies that M (xly) ((x) for suitable y. That is, Solomonoff's M can be prompted to approximate any other computable distribution. However, this argument suffers from the same downsides regarding the length of the required prompt.
is an open problem whether M can be efficiently prompted similarly to the 'countable mixture' case with a short prompt p and approximation accuracy uniform in the length of æ.
Entropic analysis. In expectation, extra information decreases entropy (H(XIY) I (X))), but specific information can increase or decrease entropy (H(X)Y- y) き (X))). In sequential context this means that =+) I (-). where can be X(: or Xeæ-h-11 or 8. This means under some ergodicity assumptions for large f, if viee P P== E. then likely H( S H()), i.e. typical prompts narrow the task distribution. In the Beta-Bernoulli case we even have H(()=z)) + -oo T1:00 { whatever the prior w0),, and the posterior converges to a ō-peak. Conversely if we want the posterior w(f(=ze) to be broader (have higher entropy) than the prior w()), we need an atypical prompt F'ce. For instance, for Beta-Bernoulli with prior w()) = ed(0) + (1 =0) } and small
e, the entropy H (0022) cF) increases with for small E for the atypical prompt wiee = 00-1
24

---

List of Notation
Symbol
Explanation
A
Token alphabet (in case binary one-hot tokens, i.e., |A| 2) Probability vector over token from the alphabet. Sequence of tokens from alphabet A of length N.
AA
TINN E AN
Niran
Pretraining sequence length 100.
Nume
Tuning sequence length 50.
Neval
Evaluation sequence length = 200. The empty sequence.
$1:L E ST.
Prefix of length L. Prefix length = 6 (or 25 for control experiments).
S
"Alphabet" for prefix. Depends on tuning method.
E RM
M didimensional parameter vector of a task. Task distribution.
P(T)
pRe (T)
Pretraining task distribution.
pTarget (T)
Pretraining task distribution.
P(II:I|T)
Distribution over sequences induced by task. Function from RM + AAN.
E(X1IN)
Marginal distribution over sequences = ((()))
also: Bayes mixture with prior P(T),
also: Bayes predictor for task distribution.
Pretraining sequence distribution / Bayes predictor.
Target
Target sequence distribution Bayes predictor.
(Neural) parametric sequential predictor. Function from (R AA. Distribution over tokens induced by (forward passes through) ng. Dimensionality of inputs for neural net. = [A] = our case.
Te
Po
D
0
Parameters of neural sequential predictor.
Parameters converged to optimum after pretraining.
0
Parameters after prefix- or weight-tuning to compute regret
if prefix-tuning and net pretrained optimally, at random initialization prefix-tuning and net is untrained, tuned weights if weight-tuning
(oe (-10 Target
Bayes predictor for <Pre, prefix tuned to Target distribution.
Lo(1EN)
Log-loss for single sequence.
Lo Lo(IIN)SS
Log-loss for single sequence prefixed by s1.E (loss only over xi:N).
( 38
Cumulative regret of net with parameters 0 on target distribution.
K
Number of sampled sequences for log loss minimization during tuning.
G Full results
Some of the following results are included in the main paper, for completeness we now show all results per experiment and network architecture. For all regret curves and bars, thick lines or bars show the median over fine-tuning repetitions, thin lines show individual repetitions, and shaded areas or bars show 25% 75% quantiles confidence intervals.
25

---

G.1 Petraining on Random Coins, fine-tune to Single Coin
Full details on the regret curves and tuning loss curves are given in Fig. A5 and the impact of tuned prefixes on the networks' internal dynamics is shown in Fig. 2 To interpret the latter, see Fig which shows that the internal state of models pretrained on Random Coins highly structured, with one of the two principal components corresponding to the step n and the other to the heads-to-tails ratio.
Transformer: Random Cains Single Coin
LSTM: Random Coins Single Coin Evaluation on Single Coin
Evaluation on Single Coin
Instant regret [nats]
Instant regret [nats]
0.4
0.3
0.3
0.2
0.1
0.1
0.0
25
50
75
100
125
150
175
200
25
50
75
100
125
150
175
200
Cumulative negret [nats]
Cumulative regret [nats]
Mouer
Mare
4-
3
2-
2-
1-
0-
0-
25
100
125
150
175
2ó0
25
50
75
100
125
150
200
Step
Step
(a) Regret curves Transformer.
(b) Regret curves LSTM.
Transfomer: Random Coins + Single Coin Evaluation Single Coin
LSTM: Random Coins Single Coin Evaluation on Single Coin
n= 49 (Nimm)
n 199 (Neel
n- (Nuaa)
n 199 (Nual
Curmulative regret (nata]
Cummuiative mimutattvv regree [nats]
Harder'
temomt Bonethog
revsery sorvp dlad dectir
Ce tnsersiinn despettere you
te Pred mertier fredic Baa Manddaouu
Pee
(c) Cumulative regret details for Transformer.
(d) Cumulative regret details for LSTM.
Transforme: Random Coins + Single Coin
LSTM: Random Coins + Single Coin
50
lanng mechus
laring mehad
SmpllcF
Simp
Tuning loss
Tuning 8
28
40
LARANNT
Embeddt
EmbeeWT
Unentteut
35
Uni Iomboetl
UnembeewT
26
Un-Tmbeeet
30
25
24
1000
200
400
600
1000
Tuning step
Tuning step
(e) Tuning loss curves for Transformer.
(f) Tuning loss curves for LSTM.
Figure A5: Models pretrained on Random Coins are tuned to a Single Coin. Transformer shown in the left column, LSTM shown in the right column. Of the prefix-tuning methods, only Soft Prompting ('SoftPT') allows optimal target task performance. Several of the weight-tuning methods succeed.
26

---

G.2
Pretraining on Random Coins, fine-tune to Two-Coin Mixture
Full details on the regret curves and tuning loss curves are given in Fig. A6 and the impact of tuned prefixes on the networks' internal dynamics is shown in Fig. A7
LSTM: Random Coins Two-Coin Mixture
Transformer: Random Coins - Two--Coin Mixture Evaluation on Two-Coin Mixture
Evaluation on Two-Coin Mixture
Instant regret [nats]
0.35
Instant regret [nats]
0.30
0.20
0.2
0.15
0.10
0.1
0.05
0.00
25
50
75
100
125
150
175
200
100
125
150
175
200
Curuuaaiive regret [nals]
Cumulative regret [nats]
4.0
20-
3.5
3.0
15-
2.5
10
2.0
1.5
1.0
0.5
0.0
125 Step n
260
125
150
175
Step n
(a) Regret curves Transformer.
(b) Regret curves LSTM.
Transformer: Random Coins Two Coin Misture Evaliaaioo on Teo-CCin Mixture
Mixture Evaluation Two--Clf Cain Mixture
11:49 49 (Muee)
n= 199 (Maaa)
INisel
(Neal)
Cumulattve (egret
Oumulettv regret (nets)
Poatic
weigl
Basebee
Prodc
Weige
Busdines
22.
32-
2
12
ten aid sondlr por Sre poot Randioory
Sadbyy
Rorotooee un soreei
tor torte ave pertinip Frad alrnotooo
(c) Cumulative regret details for Transformer.
(d) Cumulative regret details for LSTM.
Transformer: Random Coins Two-Coin Mixture
LSTM: Random Coins Two-Coin Mixture
70
Tuxingg mitros
28.0
Res
Tuning loss
60
SoIFT
LaRAIT Untembedtt
27.0
27.5
SUIFT
FutAr
50
266.5
Uniamcedif
40
26.0
25.5
30
25.0
200
400
600
800
1000
200
400
1000
Tuning step
Tuning step
(c) Tuning loss curves for Transformer.
(f) Tuning loss curves for LSTM.
Figure A6: Models pretrained on Random Coins are tuned to Two-Coin Mixture. Transformer shown in the left column, LSTM shown in the right column. No prefix-tuning method allows optimal target task performance (shown as "TargetBayes'). Some of the weight-tuning methods succeed.
27

---

Transformer anetrained on Random Coins
LSTM pretra ine on Randem Coins
(attention
soquances (langth: 50)
sequences (length:
Transformer sequences from pretraining distr.
(b) LSTM sequences from pretraining distr.
Transformer: Random Coins Two
Two--Col Coin Mixture
State (attention out) on 20 Two Coin Mixture sequences (length: 50)
No NoTuning
SimplexPT
RealPT
SoftPT
HardPT
RandomPF
(c) Pretrained Transformer, prefix tuned (center 4 panels) and evaluated sequences from target distr.
LSTM: Randem Coins - Two Cain Mixture
State (cell) on 20 Two-Coin Mixture sequences (length: 50)
NoTuning
SimplexPT
RealPT
SoftPT
HaroРT
RandomPF
(d) Pretrained LSTM, prefix tuned (center panels) to and evaluated on sequences from target distr.
Figure A7: Top: Internal state (2D PCA projection) of pretrained models highly structured one component tracks the step and the other component tracks heads-to-taile ratio. Middle and bottom: Illustration of how different tuned prefixes manipulate the pretrained Transformer's ' (middle) and LSTM's (bottom) internal state and affect subsequent dynamics. Colored lines are from target distribution (Two-Coin Mixture), gray lines are from pretraining distribution (uniform random coins), same setting in Fig. A6 Red circles mark the end of the prefixes.
28

---

G.3 Untrained network, fine-tuned on Two-Coin Mixture
Full details on the regret curves and tuning loss curves are given in Fig. A8
Transformer untrained + Two-Coin Mixture
LSTM untrained - Two-Coin Mixture
Evaluation Two-Coin Mixture
Evaluation on Two-Coin Mixture
Instant regret [nats)
1.4
Instant regret [nats]
1.2
0.25
1.0
0.20
0.8
0.15
0.10
0.4
0.2
0.0
0.00
25
50
75
100
150
175
200
100
125
150
175
200
Cumulative regret [nats]
More
50
40
30
Cumulative
20
10
10
0-
25
50
75
100
150
Step n
Step m
(a) Regret curves Transformer.
(b) Regret curves LSTM.
Transformer
untrained
Evaluation
(floa)
Prafic
Wegis
Baselral
Putett
Waget
Canulatove regres [nans]
Prefict
Weiget
Parbct
Baselzes
62.
rogres
52
30
20
Do edveeeddyy Aegocttove (ood
Un dngestton popeetaeee tor Aocsiiing Resstion
Lockaer good pentreeeer
ef condeeeryy anpooto yes coveert
Heatty
Emuediry
Staudtty Sompyieriyy
Enock days
Uneer Unveddecitt
Seppoct Bupee for pccuniie
(c) Cumulative regret details for Transformer.
(d) Cumulative regret details for LSTM.
LSTM ITMM_nnrained; untrained Two-Coin Mixture
framerormer untr antrained: untrained Twa-CCin Mixture
36
38
Tuning loss
36-
rethee seaccetf
34
Simp exl?
32
SontT
30
30
UnembestiT
28
28
Un Crosder
26
26
600
ado
800
Tuning step
Tuning step
(c) Tuning loss curves for Transformer.
(f) Tuning loss curves for LSTM.
Figure A8: Untrained models are tuned to a Two-Coin Mixture. Transformer shown the left column, LSTM shown in the right column. Soft prompting ('SoftPT") Transformer gets surprisingly close to optimal performance (shown as 'TargetBayes'). Though with poor generalization beyond the tuning sequence length (50 steps). weight-tuning methods perform better, particularly on the LSTM. Tuning loss curves show that 'SofiPT" on the LSTM converges very slowly and may not have fully converged (though it is unlikely that longer training causes qualitative difference).
29

---

G.4
Untrained network, fine-tuned on Random Coins
Full details on the regret curves and tuning loss curves are given in Fig. A9
Transformer, untrained Random Coins
LSTM LSTM_untrainee untrained Random Coins
Evaluation on Random Coins
Evaluattoon Random Coins
Instant regret (naget [nats]
0.5
Instant regret [nais]
0.30
0.4
0.20
0.3
0.10
0.1
0.0
0.00
25
50
75
100
125
150
175
200
150
175
Cumulaative regret [nats]
150
500
40
Cumulative regret
40
30
20
10
10
0
25
50
75
100
125
150
175
200
100
125
150
175
200
Step
Step
(a) Regret curves Transformer.
(b) Regret curves LSTM.
Transfomer untrained
LSTM untralined Random Eviluation Random Coins
Evaluation Random Coins
(Mural
199
Cumulative rogrest [nats]
Wage
60
Pootlc
52
Cumutativ
b
Lo0
1.00
Socops
Herrste stocsi Reserty
Budbyy
foo dontesive tomeottinn
Herden
Heaury
Sutrs
Emaronn
Ude sonterser
Randhoos
Merdh Shepp poopptortt
conteciv'
(c) Cumulative regret details for Transformer.
(d) Cumulative regret details for LSTM.
LSTM untrained: untrained Random Coins
Transformer untrained; untrained Random
40.0
36
Tuning loss
Tuning less
35.0
SimpooFT RealT
Seep7
Embboder
30.0
CYéedNT
Unterbddd
25.0
200
400
600
800
1000
200
400
600
800
1000
Tuning
Tuning step
(e) Tuning loss curves Transformer.
(l) Tuning loss curves for LSTM.
Figure A9: Untrained models tuned to (uniform) Random Coins. Transformer shown in the left column, LSTM shown in the right column. Soft prompting ( ('SoftPT') Transformer gets close to optimal performance (shown as 'TargetBayes', which is Laplace predictor in this case and arguably a non-trivial predictor). Though with poor generalization beyond the tuning sequence length (50 steps). Weight-tuning methods perform better, particularly on the LSTMM.
30

---

H
prefix-tuning with prefix length 25
Our main experiments use relatively short prefixes of length L The main point is to demonstrate how powerful even very short soft prefixes can be. Additionally, the number of possible hard prefixes grows exponentially with L, making exhaustive hard token search for long prefixes intractable. To confirm that our negative result on tuning the pretrained predictor to Two-Coin Mixture in Fig, is not an artifact of insufficient prefix length, we repeat the soft prefix-tuning experiments with more than triple the prefix length of L 25. Results are shown in Fig. A10 We also show in the same figure that increasing the soft prefix length for tuning untrained networks has only marginal effect.
Transformer:
LSTM:
Two--Coin
Evaluation Two Mixture
Evaluation Two Coin Mixture
Curmilatin regret [nwes]
Prefec
Baselass
Profe
Baselbee
Curmuletive (mwes
Prefet
Besedoss
ProfcC
Basebbet
Bemprturry
Reetrry
Suatey
Perpentaynn
Ner wortaning
Randonepy
dlemg demprtort
Sudery
ter Regotttaro
Acrboring
Ravdompp
Reektry
Terpontie
We Hacbooing
Rastboory
Compatortt
Readrr
Suddy
Tergettar
Ne Marboon
Ranstonop
(a) Pretrained Transformer to Two-Coin Mixture.
(b) Pretrained LSTM to Two-Coin Mixture.
Transformer entrained-- Evallaation
untrained
Mixture
Evalustion Tai-Cain Mixture
Cunubsiva [nas]
Polta
Cumulatve Deelabeerren ragres [nais]
60
Posta T.
52
#2.
44
30
22.
Stmpponnny
deady
derter
Meadly
Sudty
Tasset
Herturingg
Ranstoonf
Raady
Sudty
rerbeeoop
Sheephoatt
Reallty
Scdtit
pertensti
(c) Untrained Transformer to Two-Coin Mixture.
(d) Untrained LSTM Two-Coin Mixture.
Figure A10: prefix-tuning Two-Coin Mixture with prefix length L 25 (in contrast all other experiments where = 6). Compare pretrained results with Fig, 3 and untrained results with Fig. 4 Despite more than tripling the prefix length, only marginal increases in 'SoftPF' performance can be seen in some cases. It is not enough to reach Byaes-optimality on the target distribution, which means that our qualitative results hold.
31

---

I Reducing the embedding dimensionality
Results shown in Fig. All reveal that the superiority of 'SofPPT' over the other soft prefix tuning methods is largely explained by the much higher dimensionality of the embedding vectors (128- dimensional in the main experiments; now reduced to 4 dimensions), compared to input vectors (which are awo-dimensional... In frontier models, the input dimensionality is typically higher than the embedding dimensionality, which could in principle result in soft input prefix tuning methods like "RealPT" outperforming embedding tuning. We leave the question of which prefix tuning method works best at frontier model scale to the large and active research community. Note that since the "wiidt"" of our Transformers is equal to the embedding dimensionality (except the width of the MLP layer inside the attention block), the Transformer in our reduced embedding dimensionality experiments is much smaller compared to the main experiments, whereas the LSTM has the same size after the embedding layer.
Transformer. Random Coins Single Coin Evalustion on Single 199 (Wa.s)
LSTM: Random Singla Coin Evalustion Single Coin
(Mux)) Wiight
(Maun)
(Wass)
Cumulaaive regiet Inars
Dumulaaivv regrut [nats]
Unseiies
Irefict
40%
600
Unmeeted Eonlentiy que Une to Ho Horning mreti
ee 600
Seftry
toe berpen peppoteedd poriioa
emborryy Und Eonoentii tir aickning fardomny
Setty
that Rondimmry
Sertey
(a) Pretrained Transformer to Single Coin.
(b) Pretrained LSTM to Single Coin.
Transformer: Random Coins - Two Coin Mixture Evaluation on Two-Coin Mixture
LSTM: Random Coins Twr Coo--Coin Mixture Evaluation on Two--Cin Mixture
n (Phua)
n= 199 (Ma.s)
71
(Phuna
D= 199 (Macs)
Oumulativ rightt [nats]
Oamulative rightt [nats] 2
Sutbrr
gro foryedlaee
"oor preeedayy bentityy thelo Parsioni
dietryy
tret Pendborrty
Stceaity Semplie
siatey
mereee den dove
Sery BAT
(c) Pretrained Transformer to Twa-Coin Mixture.
(d) Pretrained LSTM to Twa-Coin Mixture.
Figure A11: Reducing the embedding dimensionality to 4 (compared to 128 in main experiments) largely eliminates the superiority of 'SoftPT" compared to 'RealPT" (and shortens the gap to the other prefix tuning methods), revealing that the superior performance observed in the main experiments is largely explained by the many more degrees of freedom when tuning soft embedding prefixes vs. soft input prefixes. Compare the results shown here (particularly the dark blue "SoftPT" bar) against Fig. and Fig. in the main paper. Plots show median results over 3 repetitions.
32

---

Experiments with larger networks
Fig. A12 shows results for increasing the network size. Compared the main experiments we double the embedding dimensionality (128 - 256), the width of layers (128 - 256), and the number of layers (1 + 2). Qualitatively, our main claims hold. Particularly, that prefix tuning can be used to optimally adapt the pretrained predictor to the Single Coin target distribution, but cannot be used for perfect adaptation to the Two-Coin Mixture task. Anecdotally, we have observed our main results to hold robustly, as long as the network size and number of training and tuning steps is large enough. For too small networks, or networks trained or tuned too little, results become more inconsistent. From a theoretical perspective, too small networks violate the realizability condition, and networks with too little training violate the convergence condition.
Transformer: Random Coins Single Evaluation Single
LSTM: Random Coins - Single Coin Evaluation on Single Coin
n- (Muxe)
199 (Waxs)
n1 49 1M6,raa
199 (Meal
Prote
Wesk
Preto
Sandeess
Curulatiie regreet [nats]
42-
Oumulation regna
Heugh Compeeet
des bey prpeetin recuaiin
Heeenrt Rould
doepy
geot cembustyy
freti aie Conun cany tnyy Beryoluee
for Panctomie
Ebsteeders
the Predice
Ho Hertinng mroll Readhmrry
the Probic det Peetaaee
(a) Pretrained Transformer to Single Coin.
(b) Pretrained LSTM to Single Coin.
Transformer: Random Coins -a Two-Coin Mixture Evaluation on Two-Coin Mixture
LSTM: Random Coins Turo-Coin Mixture Evaluation on Two--oin Mixture
n (Paea)
D= 199 (Wars)
n (Phuna)
D= 199 (Wows)
Cumdative mighet [mais]
Canuafliv migett [nats] 2
.7
0.0
Raa demorruut cosuroute
tradte days Cooc oWy?
foryer Protonin Polio cooaertiin Crrli Boreet
Sibty
councdie deys Cincle oily dorc
ald sartocing
Proto for colirry
(c) Pretrained Transformer to Two-Coin Mixture.
(d) Pretrained LSTM to Two-Coin Mixture.
Figure A12: Results for larger networks compared to main experiments are qualitatively equivalent, and show that our main findings are robust against changing model size. Compare the results shown here (particularly the dark blue 'SoftPT" bar) against Fig. and Fig. 4 in the main paper. Plots show median results over 3 repetitions.
33


Reviews:
[{"review_id": "kvkVz0Jc9Y", "reviewer": "Reviewer_HkrS", "summary": "This paper shows that meta-trained neural networks can be approximated as Bayes-optimal predictors and explore which prefix tuning technique results in optimal prompting (Bayes-optimal). To investigate this, the paper first conducts theoretical analysis, where it derives a connection between minimizing expected log loss over a distribution of stochastic data generators and the Bayes-optimal predictor for that pre-training distribution. For empirical analysis, the authors used an educational experimental setup using coin-flip sequences, where the exact Bayes-optimal predictor is analytically known. Also they train a basic Transformer and a LSTM architecture model, comparing the performance of various prefix tuning strategies against traditional weight tuning and other baselines. The empirical findings indicate that prefix tuning methods do not reach Bayes-optimal. Weight tuning methods achieve better results than prefix-tuning, although there is still a small gap when compared against Bayes-optimal.", "questions": "NA", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths\n\n- I really like the idea of connecting meta-learning to Bayesian predictors. \n- The theoretical analysis is well-grounded.\n-  Clean and controlled experimental setup using coin-flip sequences, where the Bayes-optimal predictor is analytically known. This allows for precise evaluation of different tuning methods. \n- The authors also compare an exhaustive set of techniques, including prefix tuning variants and full weight tuning, providing a comprehensive view of their relative performance.\n\nWeaknesses\n\n- Even though the coin flip provides a good starting point, it does not capture the complexity and variability of real-world data. Therefore how to extend the study to complex real world tasks remains unclear. Extending the study to include richer synthetic tasks or controlled natural datasets (e.g., language modeling on short-form text, or structured reasoning problems) would provide stronger evidence for the claims. \n- As it might be difficult to get Target Bayes for real world tasks, is there a way to approximate this optimal value? Which will help point 1 too.\n- Is the result pattern we see on the coin-flip dataset consistent with another model size or is specific to the chosen model? A few selected experiments may help answer this.", "was_revised": true, "final_justification": "The authors have provided additional results and details on what is the limitation. Overall a good contribution. I have revised the score based on the responses.", "rebuttal": "We thank the reviewer for their feedback and suggestions, and are happy to hear that they enjoyed reading our paper and rated it as excellent on 3 out of 4 dimensions. Perhaps the main weakness, that all reviewers identified, is that the relevance of our results at frontier model scale (LLMs/VLMs and data such as text, images, or video) is hard to forecast. We have added this to our limitation section and significantly expanded our discussion section to provide additional details.\n\n**Overview of general changes/additions (across all reviewers):**\n* We discuss the limitations and difficulties with extrapolating our findings to LLM-scale, and some ideas how to approach this (we add to the initial limitations section at the end of the intro, and to the discussion section at the end of the paper). This issue was raised by all reviewers in some form, we added it to our response to HkrS, and kindly ask the other reviewers to read our response there.\n* HkrS asked whether our results are brittle w.r.t. model size. We conducted an additional set of the main experiments (Fig. 1 and Fig. 3) with larger models, and findings are qualitatively equivalent. See our response to HkrS for the main numbers (we cannot provide the full plots that we added to the appendix).\n* PHRz asked whether the superiority of Soft Prompting has a theoretical explanation. As speculated in L298-300, it may simply be the much higher dimensionality of embeddings vs. inputs in our case. We ran a control experiment (for Fig. 1 and Fig. 3) where we reduced the embedding dimensionality from 128 to 4, which largely cancels the superiority of ‘SoftPT’. Full plots were added to the appendix, the most important numbers are in our response to PHRz.\n\n---\n\n**Specific response:**\n\nWeaknesses:\n\n  * [Synthetic data too simplistic; models too small; how to extend the study to real world tasks]. We have made the following additions to the paper.\n\nAddition to limitations section (end of intro):\n“This paper discusses fundamental properties of prefix tuning, which we illustrate with educational experiments where the focus is on clarity and being able to compare against a tractable Bayesian predictor. Accordingly, our datasets do not capture the full complexities of, e.g, large-scale language, vision, or robotics tasks. Similarly, our neural networks are small by modern standards, which means that our findings must be very cautiously extrapolated towards modern frontier model scale. Further rigorous and well-designed scientific studies are necessary to bridge the gap between our current work and modern large-scale ML practice, and we are optimistic that our fundamental results will inspire the design of such studies.”\n\nAddition to the beginning of the discussion section:\n“While we have laid important fundamental groundwork in our current study, extrapolating our findings to modern frontier model (and data) scale is not straightforward. While the theoretical findings we presented, including the limitations of prefix tuning, hold at any scale, it is likely that additional practical issues arise at scale that are not captured by our current small-scale experiments. It is thus hard to predict the relevance and impact of our fundamental results on today’s frontier-model practice. For instance, one of our main results is that optimal prompting to a single target task is possible, whereas it is not for a mixture of tasks. We are confident that this holds even at frontier model scale (based on the theory), but it is unclear and highly non-trivial what constitutes a task for a LLM, and accordingly, whether this is a severe limitation or not. In our experiments, a task is simply an unobserved variable in a two-level hierarchical statistical model. At LLM scale, the structure is vastly more complex, with many more hierarchical levels, and potentially other statistical structures at play. The next step would be to carefully design data generators that are closer to natural language data, but still fully understood and well controllable, akin to the data generators used, e.g., in the `Physics of LLMs’ paper [1], and run our experiments of tuning to single tasks vs. tuning to mixtures of tasks at scale.\nWith these caveats in mind, cautiously extrapolating our findings to frontier model scale, raises some questions for investigation [the orig. discussion section then follows...]”\n\n  * [How could we get TargetBayes for real world tasks?]. For many real-world tasks, all kinds of data modeling techniques are used to approximate TargetBayes. As models and optimizers get better, these approximations will get better. Note though, that we do not necessarily need TargetBayes: different prompt- or weight-tuning methods can be meaningfully compared against each other w.r.t. their cumulative log loss on a held out evaluation set. It would just be unclear how far these methods are from the optimum.\n  * [Is the result pattern consistent with other model sizes?]. We have observed the qualitative pattern underlying our main results very robustly across pilot experiments with different model sizes. To be rigorous, we have conducted an experiment with larger models, and have added the following to the appendix:\n\n“Fig. A12 shows results for increasing the network size. Compared to the main experiments we double the embedding dimensionality (128->256), the width of layers (128->256), and the number of layers (1->2) [called LSTM Maxi and Transformer Maxi in the table below]. Qualitatively, our main claims hold. Particularly, that prefix tuning can be used to optimally adapt the pretrained predictor to the Single Coin target distribution, but cannot be used for perfect adaptation to the Two-Coin Mixture task. Anecdotally, we have observed our main results to hold robustly, as long as the network size and number of training and tuning steps is large enough. For too small networks, or networks trained or tuned too little, results become more inconsistent. From a theoretical perspective, too small networks violate the realizability condition, and networks with too little training violate the convergence condition. The theory thus does not apply to predict outcomes in this regime.”\n\nInstead of the full Fig. A12 (which looks similar to Fig. 1 and Fig. 3), we only provide the most important information in the rebuttal: whether ‘SoftPT’ can be used to reach optimality or not?  The answer is true for ‘Random -> Single’ but not for ‘Random -> Mixture’.\t\n\n|                 Setting                | Random -> Single | Random -> Mixture |\n|----------------------------|--------------------|----------------------|\n|    Optimal (TargetBayes)   |             0.00            |              0.69              |\n|      LSTM  (main paper)      |             0.01            |              0.98              |\n| Transformer (main paper) |             0.01            |              1.20              |\n|               LSTM Maxi            |             0.01            |              0.91              |\n|         Transformer Maxi       |             0.03            |              1.04              |\n\n\n* Formatting issues: thanks for pointing out these typos. We have fixed them in our updated manuscript.\n\n[1] Allen-Zhu, Zeyuan, and Yuanzhi Li. \"Physics of language models: Part 3.1, knowledge storage and extraction.\" arXiv preprint arXiv:2309.14316 (2023)."}, {"review_id": "zzTbkOYFMV", "reviewer": "Reviewer_b8X5", "summary": "This paper provides a conceptual framework of viewing the large pretrained LLMs as meta-trained neural networks which can behave as Bayesian predictors over their pretraining distributions. It reinterprets prompt tuning as conditioning those Bayesian predictors with short prefixes. It proves that prompting can reach bayes-optimal performance only when the target task already lies inside the pretraining support and is unimodal. It shows two formal failure modes: multimodal targets and tasks never seen in pretraining. It validates the theory with toy coin-flip tasks using tiny ttransformers and LSTMs. It finds soft prompts inject more information than any hard token sequence. It also shows weight tuning sidesteps the theoretical limits but overwrites the original model. The paper is overall clearly written and insightful yet confined to trivial data, small models.", "questions": "1. The educational experiments are only conduced on small models like LSTMs. How do you predict your results might translate to these more modern LLMs like GPT series, LLama.\n2. How your bayesian optimization and meta learning view explains the in-context learning ability's brittleness in context order, prompt formats sensitivities?\n3. What actionable guidance does your analysis offer for practitioners—e.g., on choosing prefix length, deciding between soft-prompt tuning and weight-tuning?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. This paper introduces a novel view of viewing the large-corpus pretraining of LLM as meta-training, where each training example is conditioned on an task and then the LLM becomes a sequential bayesian predictor, And this paper recasts prompting as steering this bayesian predictor. \n2. it has analyzed theoretical conditions for which the optimal prompting is and is not possible. And they empirically demonstrated this claim on atomic tasks.\n3. Through educational expeirments, they comfirm their theoretical conditions results. \n4. it also includes visual evidence in the PCA plots which shows how different prefixes physically move the hidden states into the region which means a good posterior region.\n\nCons:\n1. The educational experiments might not extend to real-world tasks where the combinatorial structure of language, vision or other modalities. \n2. the experiments are only conduced on small transformers and LSTM.", "was_revised": true, "final_justification": "Their rebuttal provided new experiments and I think their theory is clean, and have some insights for practitioners as well. Therefore raising score to 5.", "rebuttal": "We thank the reviewer for their feedback and suggestions, and are happy to hear that they enjoyed reading our paper and rated it as good on 3 out of 4 dimensions. Perhaps the main weakness, that all reviewers identified, is that the relevance of our results at frontier model scale (LLMs/VLMs and data such as text, images, or video) is hard to forecast. We have added this to our limitation section and significantly expanded our discussion section to provide additional details.\n\n**Overview of general changes/additions (across all reviewers):**\n* We discuss the limitations and difficulties with extrapolating our findings to LLM-scale, and some ideas how to approach this (we add to the initial limitations section at the end of the intro, and to the discussion section at the end of the paper). This issue was raised by all reviewers in some form, we added it to our response to HkrS, and kindly ask the other reviewers to read our response there.\n* HkrS asked whether our results are brittle w.r.t. model size. We conducted an additional set of the main experiments (Fig. 1 and Fig. 3) with larger models, and findings are qualitatively equivalent. See our response to HkrS for the main numbers (we cannot provide the full plots that we added to the appendix).\n* PHRz asked whether the superiority of Soft Prompting has a theoretical explanation. As speculated in L298-300, it may simply be the much higher dimensionality of embeddings vs. inputs in our case. We ran a control experiment (for Fig. 1 and Fig. 3) where we reduced the embedding dimensionality from 128 to 4, which largely cancels the superiority of ‘SoftPT’. Full plots were added to the appendix, the most important numbers are in our response to PHRz.\n\n---\n\n**Specific response:**\n\n\nWeaknesses:\n\n  * [Educational experiments might not extend to real-world tasks]. We have added this as an explicit limitation and have expanded our discussion. Please see our response to HkrS for details.\n  * [Experiments are only conducted on small networks]. We have included an additional experiment in the appendix with twice the embedding dimensionality, twice the layer width and double the number of layers (two instead of one). Qualitative results are equivalent and all our main claims hold. The experiments mainly address HkrS’s question about how robust our results are to model scale. Please find more details in our response to HkrS. We do acknowledge that compared to frontier model scale, these networks are still “small”.\n\nQuestions:\n  * [How do we predict our experiments would translate to frontier models?] We have added this as an explicit limitation and have expanded our discussion. Please see our response to HkrS.\n  * [What actionable guidance do we have for practitioners?] Our original discussion section was meant to address this question, but rather than speculating we have formulated a number of hypotheses to investigate in the discussion. In particular:\n     * Soft prefix tuning (SoftPT, RealPT, and modern methods based on it) should be superior to hard token tuning and traditional prompt engineering.\n    * This may be particularly fruitful for in-context imitation learning, where one typically has small datasets available on which soft prefixes could be tuned, but the standard practice is to simply add a number of demonstrations to the context.\n    * One take-away for practitioners based on our experiments could be to always use weight tuning (full weights or LoRA), as that yields the same or better performance as prefix tuning and does not suffer from its theoretical limitations. However, as we noted in the discussion this may be a wrong conclusion, as the two methods come with different practical implications and other works (e.g., Lampinen et al. 2025, Chan et al. 2022b) have found diminished neural plasticity after LLM pretraining, making weight tuning potentially less effective compared to prompt tuning.\n    * Generally, we advise practitioners to compare prefix tuning *and* weight tuning, whenever possible, and pick which works best.\n  * [How does the Byaesian view on in-context learning explain the brittleness and sensitivity to non-semantic changes to prompts?] In short: these issues are outside our theory and there is not much that can be said about them from this particular angle (without significantly extending the theory). This does not invalidate the theory and our findings, it simply says that for certain (important) questions, the theory is not enough / not appropriate. For instance, we observed in previous work that network predictions and internal states on a coin-flip task are not fully invariant to the order with which samples are shown (an ideal Bayesian predictor only cares about counts of heads and tails, and is invariant to the order). This is not completely surprising, as learning perfect invariances over a large space of possible strings would be very difficult. So in practice, meta-trained neural nets can deviate from the ideal Bayesian predictor, and in these cases the idealized Bayesian theory cannot be used to make precise predictions about some very particular questions. Nonetheless, and despite these issues, our experiments show that the Bayesian theory is sharp and relevant on an abstract level, and theory tells us that, in principle, as models become even more expressive and optimizers get better, the theory becomes more relevant and accurate."}, {"review_id": "oy538f0DZZ", "reviewer": "Reviewer_QjaG", "summary": "This paper describes prompt tuning through a Bayesian lens and runs a number of experiments with many prompt tuning baselines in a synthetic setting. They include some mechanistic analysis of the proposed methods.", "questions": "- Is 1000 steps of pretraining really enough training to learn the distribution of random coin flips? I would expect this to be hard to learn and general require much more training.\n- This is a tiny nitpick but it might be cleaner to name the embedding dimension 'd' instead of writing out 'Embedding-dimensionality' in superscript in S3.\n- How was HardPT implemented? Did you try a state-of-the-art discrete optimizer such as GCG (https://arxiv.org/abs/2307.15043)?\n\n'The embedding is a trainable linear projection from the 2D token space into a 128-dimensional “embedding” space.'\n   Is the input not a single dimension? (1 or 0)\n\n- A small presentation question: What is the main intended takeaway from Figure 1? Clarifying language in the caption may go a long way here.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 4, "strengths_and_weaknesses": "Strengthes:\n- Prompt/prefix tuning is important, interesting, and understudied.\n- The theoretical analysis seems sound. Exploring LLM-related topics through a Bayesian lens seems important.\n- The baselines are thorough and include all the prompt tuning methods I would expect (except perhaps some missing variants of HardPT).\n- The experiments on untrained transformers surprised me and could translate to text data.\n\nWeaknesses:\n- Experiments are limited to randomly generated synthetic datasets. It is not clear how if at all any findings might be useful in real-world scenarios.\n- The Bayesian lens doesn't appear to tell us anything new about how prompt tuning works.\n- There are many ways to implement \"HardPT\" and there isn't much if any discussion of this.", "was_revised": true, "final_justification": "After reviewing the rebuttal and the discussions between the authors and the other reviewers, I decided to increase my score from 4 (Borderline Accept) to 5 (Accept).", "rebuttal": "We thank the reviewer for their feedback and suggestions, and are happy to hear that they found our work very original and clear. Perhaps the main weakness, that all reviewers identified, is that the relevance of our results at frontier model scale (LLMs/VLMs and data such as text, images, or video) is hard to forecast. We have added this to our limitation section and significantly expanded our discussion section to provide additional details.\n\n**Overview of general changes/additions (across all reviewers):**\n* We discuss the limitations and difficulties with extrapolating our findings to LLM-scale, and some ideas how to approach this (we add to the initial limitations section at the end of the intro, and to the discussion section at the end of the paper). This issue was raised by all reviewers in some form, we added it to our response to HkrS, and kindly ask the other reviewers to read our response there.\n* HkrS asked whether our results are brittle w.r.t. model size. We conducted an additional set of the main experiments (Fig. 1 and Fig. 3) with larger models, and findings are qualitatively equivalent. See our response to HkrS for the main numbers (we cannot provide the full plots that we added to the appendix).\n* PHRz asked whether the superiority of Soft Prompting has a theoretical explanation. As speculated in L298-300, it may simply be the much higher dimensionality of embeddings vs. inputs in our case. We ran a control experiment (for Fig. 1 and Fig. 3) where we reduced the embedding dimensionality from 128 to 4, which largely cancels the superiority of ‘SoftPT’. Full plots were added to the appendix, the most important numbers are in our response to PHRz.\n\n---\n\n**Specific response:**\n\n\nWeaknesses:\n\n  * [Experiments only on synthetic datasets.] We have added this explicitly to the limitations section at the end of the introduction and to our discussion section at the end of the paper. Please see our response to HkrS for the precise text that we added.\n  * [The Bayesian lens does not tell anything new about how prompt-tuning works.] We respectfully disagree. Our theoretical analysis, centered on the Bayesian view, reveals novel limitations of when prefix-tuning can and cannot work regardless of the tuning method, which we confirm to hold in simple experiments, but the theory holds at any scale (incl. LLM scale and beyond). We are not aware of any previously published work predicting or providing an explanation of why prompt-tuning to any single task works but prompt-tuning to any mixture of two or more tasks is at best suboptimal. Additionally, the Bayesian view emphasizes that prompting works via manipulating the sufficient statistics of a Bayesian predictor (in practice in ways that go beyond strict statistical conditioning, e.g., when using real-valued prefixes), which we have not seen spelled out in the literature before, and which we illustrate by recording how the internal state of networks is manipulated by various kinds of prompts. We thus used the Bayesian lens to tell us quite a bit new about how prompt-tuning works on a very fundamental level, and to a smaller degree mechanistically.   \n  * [There are many ways to implement HardPT.] This is correct, and in general hard token tuning is a challenging problem with many published methods. We avoid this problem in our study by performing “exhaustive search” (L138, L260 in the paper). This means that we evaluate model performance for *every possible* hard-token prefix (2^6=64 possible hard prefixes in our case). Accordingly, the prefix we find cannot be improved with any other method of hard-token tuning.   \n\nQuestions:\n  * [Is 1000 steps of pretraining enough?] Yes. We investigate pretraining curves for all our experiments (and they can be easily produced with the released code), and the 1000 steps were chosen via pilot experiments to suffice for convergence in all experiments with a comfortable margin.\n  * [Rename ‘Embedding-dimensionality’.] We agree the notation is a bit awkward, but it has the advantage that it does not need any further explanation / symbol. Since we only use it once in the paper, we have left it as is for now, but will change it if the reviewer feels strongly about it.\n  * [How was HardPT implemented?] As stated above (and L138 in the paper) via exhaustive search over all 64 possible binary hard token sequences of length 6.\n  * [Input dimensionality vs. embedding dimensionality.] Yes, the input dimensionality is 2 (binary inputs, i.e. two-dimensional one-hot encodings of ‘0’ and ‘1’). We pass inputs through a first linear projection (with learnable parameters) to compute an “embedding”.\n  * [What is the main takeaway from Fig. 1?] The main takeaway from Fig. 1 is that the pretrained network *can be successfully prompted to behave like the Bayes-optimal predictor on the target distribution (TargetBayes) from the very beginning of eval sequences onwards*. The effect of optimal prompting thus becomes very similar to weight-tuning on the target task *in this case*. We have added this main takeaway to the figure caption."}, {"review_id": "UXgGpp7yEv", "reviewer": "Reviewer_PHRz", "summary": "This paper studies prompt tuning under the lens of memory-based meta-learning. It discusses how the problem of finding of optimal prompt for a task can be viewed as conditioning of Bayes predictors. The result of this theory is an understanding of the conditions when optimal prompting is possible or not. \n\nThe paper presents a detailed theoretical discussion as well as simulations based on coin-flip data. Interestingly, the paper shows that optimal prompting is not possible when the fine-tuning data and evaluation data is a task mixture.", "questions": "I have a few detailed technical questions. \n\n1. Theory for Soft prompting:\n - (1.a) Eq 8 and 9 show when hard prompting is expected to work. However, it does not apply to soft versions, correct?\n - (1.b) If correct, is it possible to extend the theory to deal with SimplexPT, RealPT, SoftPT? I understand these are off-distribution, but is it at least possible to extend the theory for SimplexPT? What is needed? If it is not possible, what exactly is the challenge? \n - Rationale for the question: the empirical results and discussion in Section 5 talks about the superior performance of soft prompts, so it will be nice if this can be captured by the theory. \n\n2. Memory-based meta-learning definition: \n - (2.a) I was not aware of this term. Did it arise from [Ortega 2019] or is there an earlier reference? Also, what exactly is the part that is \"memory-based\"? \n - (2.b) There have been other papers that propose a Bayesian inference view of in-context learning, such as [Xie 2021] discussed in Section D. How exactly does the memory-based meta-learning framework compare? Is it exactly the same? Is it a generalization? Or are there some nuanced differences despite the similarities?\n - (2.c) Lines 90-95 describes the training process. Are you saying that all LLM pre-training performed by Meta, Google, OpenAi, etc. is essential operating as memory-based meta-learning, except that the task sampling (step 2) is not explicit? \n - (2.d) The Remark in Section 2 says \"The main concerns w.r.t. the applicability of the Bayesian viewpoint is that due to limited expressivity, limited data, suboptimal optimization, and off-distribution inputs, models may not converge to or operate in the Bayesian regime.\" I am not sure I fully understand the implications of this. Expressivity and data can be scaled up in LLMs and suboptimal optimization can be ameliorated with good methods, so in what way is the Bayesian viewpoint not applicable to LLM training in practice? \n\n3. Fig 2: Please provide more explanation on how to interpret the plot. \n - (3.a) What do I make of the difference between random coins and single coin trajectories? \n - (3.b) Where do you observe the off-distribution nature of soft-prefixes? \n - (3.c) What do the red points (n=6) mean, and why are there multiple per graph? \n - (3.d) The general shape for random coins differs between transformer and LSTM. Why? \n\n4. TargetBayes, PreBayes, PreBayesPT: I did not follow how these are constructed and how these differ from each other. Can you provide more description or pseudocode for each separately? \n\n----\n\nMinor comments:\n- line 87: typo \"preforming a forward pass\" --> performing\n- I do not quite appreciate the use of longer strings for eval. (N_eval = 200). Does it matter to show generalization beyond tuning length in this experiment?", "limitations": "yes", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n - The paper is extremely well-written. I spent several hours re-reading it and learned a tremendous amount of new material. It reminded me the joys of being a reviewer. \n - The paper provides a theoretical framework for thinking about prompting and in-context learning. This is very valuable for improving our understanding. \n\nI do not see any major weaknesses. Perhaps one might be concerned whether the conclusions on simplified experiments can transfer to real-world language models, but I do not think this decreases the value of the theoretical framework.", "was_revised": true, "final_justification": "Strong paper: provides interesting theory and is well-written.", "rebuttal": "We thank the reviewer for their feedback and suggestions and are very happy to hear how much they enjoyed reading our paper. Since the aim was to explain and show educational experiments, we particularly appreciate that the reviewer “learned a tremendous amount of new material”. Perhaps the main weakness, that all reviewers identified, is that the relevance of our results at frontier model scale (LLMs/VLMs and data such as text, images, or video) is hard to forecast. We have added this to our limitation section and significantly expanded our discussion section to provide additional details.\n\n**Overview of general changes/additions (across all reviewers):**\n* We discuss the limitations and difficulties with extrapolating our findings to LLM-scale, and some ideas how to approach this (we add to the initial limitations section at the end of the intro, and to the discussion section at the end of the paper). This issue was raised by all reviewers in some form, we added it to our response to HkrS, and kindly ask the other reviewers to read our response there.\n* HkrS asked whether our results are brittle w.r.t. model size. We conducted an additional set of the main experiments (Fig. 1 and Fig. 3) with larger models, and findings are qualitatively equivalent. See our response to HkrS for the main numbers (we cannot provide the full plots that we added to the appendix).\n* PHRz asked whether the superiority of Soft Prompting has a theoretical explanation. As speculated in L298-300, it may simply be the much higher dimensionality of embeddings vs. inputs in our case. We ran a control experiment (for Fig. 1 and Fig. 3) where we reduced the embedding dimensionality from 128 to 4, which largely cancels the superiority of ‘SoftPT’. Full plots were added to the appendix, the most important numbers are in our response to PHRz.\n\n---\n\n**Specific response:**\n\nWeaknesses:\n\n  * [How would conclusions of the paper transfer to real-world experiments?] We have added another explicit limitation (end of the intro), and some discussion of this question to our discussion section (end of the main paper). See our response to HkrS, who raised a similar issue, where we show the full text that we added.\n\nQuestions:\n\n  * 1.a) This is correct - for the theoretical Bayes predictor there is only an input alphabet (of hard tokens). Soft tokens (i.e., real-valued inputs outside the alphabet) are outside this theoretical formulation, which makes it hard to make very general theory-based statements..\n  * 1.b) Unfortunately this is not easily possible. Even for SimplexPT, the inputs are not part of the alphabet of the idealized Bayesian predictor. What happens when we feed such inputs into an implementation of such a predictor depends on the implementation is unclear. E.g., we could implement a Beta-Binomial Bayes predictor in Python - what happens when we feed anything other than the tokens for ‘heads’ and ‘tails’ into this predictor is up to the implementation and not the Bayesian theory.\n  * Superiority of soft prompts: in our case, soft embedding prefixes are in a higher dimensional space compared to the input, which may give more degrees of freedom to “move” in off-distribution activation space and affect the internal representation of the predictor’s memory state. We have briefly speculated that this may be the reason for the superiority of ‘SoftPT’ *in our case* (L298-300). We have added a control experiment which confirms our speculation. We have added the following to the appendix and have clarified in our limitations section (end of introduction) why ‘SoftPT’ is superior to other soft prompt tuning methods in our setting.\n\n“Results shown in Fig. A11 reveal that the superiority of 'SoftPT' over the other (soft) prefix tuning methods is largely explained by the much higher dimensionality of the embedding vectors, compared to input vectors (which are two-dimensional). In frontier models, the input dimensionality is typically higher than the embedding dimensionality, which could in principle result in soft input prefix tuning methods like `RealPT' outperforming embedding tuning. We leave the question of which prefix tuning method works best at frontier model scale to the large and active research community.”\n\nInstead of the full Fig. A11 (which looks similar to Fig. 1 and Fig. 3), we provide here the most important information, which is the question how ‘SoftPT’ performance is affected by the change (compare with ‘RealPT’ in Fig. 1 and Fig. 3)\n\n|                 Setting                | Random -> Single | Random -> Mixture |\n|----------------------------|--------------------|----------------------|\n|    Optimal (TargetBayes)   |             0.00            |              0.69              |\n|      LSTM  (main paper)      |             0.01            |              0.98              |\n| Transformer (main paper) |             0.01            |              1.20              |\n|           LSTM (4-dim)           |             0.34            |              1.36              |\n|      Transformer (4-dim)     |             0.65            |              1.92              |\n\n\n\n  * 2a) There are different forms of meta-learning. Besides memory-based (which dates back a few decades see e.g. Learning To Learn by Thrun & Pratt 1998 and earlier work by Schmidhuber). Another common approach is MAML, where meta-training learns a shared parameter vector (a set of neural network weights) across tasks with the goal of being able to rapidly adapt (via SGD on the weights) to any particular task.\n  * 2b) Other papers, including Xie et al., have identified that in-context learning can be well described as the adaptation of a Bayesian predictor but not where this Bayesian predictor comes from in the first place (the connection to meta-learning theory).\n  * 2c) Yes, in principle LLM training can be viewed as implicit meta-learning (the notion of task may be very subtle, see Ortega et al. 2019 who has a discussion of implicit meta-learning). Perhaps more important than calling LLM pretraining implicit meta-learning is what we get out of this viewpoint.\n  * 2d) Strictly speaking, the Bayesian predictor arises at the endpoint of (perfect) meta-training. If the training process does not fully converge to that point, we have an approximate Bayes predictor but the theory allows us to say relatively little beyond that (whereas we can characterize the endpoint very well). Since it is not clear how close LLMs would be to this endpoint, we think it is important to add this disclaimer. \n  * 3. Thanks for flagging this. We have added more explanation to the main text and figure caption based on your questions. First, please see Fig. A7 a,b, which shows that the pretrained networks essentially use the vertical dimension in the plot (the first principal component) to keep track of the timestep of a sequence, and the horizontal dimension corresponds to the heads-to-tails ratio observed on the current sequence.  \n  * 3a) The random coins trajectories are the traces of internal states that one gets by feeding in trajectories from the pretraining distribution (i.e., random coins). The colored single coin trajectories are sequences of internal states that one gets from feeding in sequences of the single coin with fixed bias (and prefixed by prompts tuned via the various methods).\n  * 3b) Since the trajectories of, e.g., SoftPT (dark blue lines) behave very differently from what one would get on pretraining sequences (gray lines), we can visually conclude that they are far off distribution for the Transformer (note that this cannot be seen visually for the LSTM, but because this is a 2D projection of a much more high-dimensional activation vector, it cannot be concluded from these plots that we are ‘on distribution’ for the LSTM).\n  * 3c) The red dots indicate the internal state of the neural network after it has consumed the prefix of length 6.\n  * 3d) Some differences in internal structure for different architectures are expected. What is interesting (Fig. A7 a,b) is that for both architectures the same two signals (number of steps and heads-to-tails ratio) dominate the variance of the internal state (the first two principal components). These are exactly the task’s sufficient statistics, and in this abstract sense the structure is very similar between both network types.\n  * 4) See lines 251-253 in the paper. These three predictors are different known (analytical) Bayesian predictors. For pre-training on ‘Random Coins’, PreBayes is a Beta-Binomial distribution. For fine-tuning to a single coin ‘TargetBayes’ is a Binomial distribution with the correct coin bias. For the Two-Coin Mixture ‘TargetBayes’ is a mixture of two Binomial distributions. ‘PreBayesPT’ results from taking the ‘PreBayes’ (Beta-Binomial), and prompting it with an optimal sequence of 6 hard tokens for the target task (which we find via exhaustive search). \n\n\nMinor comments:\n\nThanks for pointing out the typo. Re longer sequences: the paper and main claims hold when considering results at n=49, which is where we have strict theoretical guarantees. We nonetheless think the additional results are important and interesting to get a sense of how robust the performance is for various tuning methods. E.g., it is conceivable that ‘SoftPT’ would produce very good performance up to the length for which the soft prompts were optimized (i.e. n=49), and then drops sharply because the off-distribution nature of the resulting internal activations rapidly “spirals out of control”, but we find this not to be the case."}]

Author rebuttal:

Meta-review: This is an overall interesting paper that is well executed. The reviewers are all positive, mention that their concerns are all addressed and are generally happy with the paper. There are some nice foundational contributions into understanding various forms of adaptation. The didactic experiments are nice and well-explained. The limitations (acknowledged by the authors) are that it's not clear how to extrapolate results to frontier models. What keeps me from giving an even higher score is that the findings are more-or-less already known and not super surprising, but it is still very valuable to put all these "folklore" knowledge on a precise footing with careful experiments. I recommend acceptance.
Decision: Accept (spotlight)