Can Generative AI Solve Your In-Context Learning Problem?  A Martingale Perspective
Authors: Andrew Jesson
Nicolas Beltran-Velez
David Blei
Venue: ICLR 2025 Poster (2025)
Keywords: generative models
Bayesian
in-context learning
generalization
model checking
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Abstract: This work is about estimating when a conditional generative model (CGM) can solve an in-context learning (ICL) problem. An in-context learning (ICL) problem comprises a CGM, a dataset, and a prediction task. The CGM could be a multi-modal foundation model; the dataset, a collection of patient histories, test results, and recorded diagnoses; and the prediction task to communicate a diagnosis to a new patient. A Bayesian interpretation of ICL assumes that the CGM computes a posterior predictive distribution over an unknown Bayesian model defining a joint distribution over latent explanations and observable data. From this perspective, Bayesian model criticism is a reasonable approach to assess the suitability of a given CGM for an ICL problem. However, such approaches---like posterior predictive checks (PPCs)---often assume that we can sample from the likelihood and posterior defined by the Bayesian model, which are not explicitly given for contemporary CGMs. To address this, we show when ancestral sampling from the predictive distribution of a CGM is equivalent to sampling datasets from the posterior predictive of the assumed Bayesian model. Then we develop the generative predictive $p$-value, which enables PPCs and their cousins for contemporary CGMs. The generative predictive $p$-value can be used in a statistical decision procedure to determine when the model is appropriate for an ICL problem. Our method only requires generating queries and responses from a CGM and evaluating its response log probability. Using large language models, we empirically evaluate our method on tasks involving tabular data, imaging data, and natural language data.

Published as a conference paper at ICLR 2025
CAN GENERATIVE AI SOLVE YOUR IN-CONTEXT LEARNING PROBLEM? A MARTINGALE PERSPECTIVE
Andrew Jesson"
Nicolas Beltran-Veler'
David Bleit*
ABSTRACT
This work is about estimating when a conditional generative model (CGM) can solve an in-context learning (ICL) problem. An in-context learning (ICL) problem comprises a CGM, dataset, and prediction task. The CGM could be multi- modal foundation model; the dataset, a collection of patient histories, test results, and recorded diagnoses; and the prediction task to communicate a diagnosis to a new patient. Bayesian interpretation of ICI. assumes that the CGM computes a posterior predictive distribution over an unknown Bayesian model defining a joint distribution over latent explanations and observable data. From this perspec- tive, Bayesian model criticism is a reasonable approach to assess the suitability a given CGM for an ICL problem. However, such approaches- posterior predictive checks (PPCs)- often assume that and posterior defined by the Bayesian model, which are not explicitly given for contemporary CGMs. To address this, show when ancestral sampling from the predictive distribution of a CGM is equivalent to sampling datasets from the poste- rior predictive of the assumed Bayesian model. Then we develop the generative predictive p-value, which enables PPCs and their cousins for contemporary CGMs. The generative predictive p-value can be used in a statistical decision procedure determine when the model is appropriate for an ICL problem. Our method only requires generating queries and responses from a CGM and evaluating its response log probability. Using large language models, we empirically evaluate our method on tasks involving tabular data, imaging data, and natural language data.
sample from the likelihood
INTRODUCTION
An in-context learning (ICL. problem comprises conditional generative model (CGM), dataset, and a prediction task (Brown et al., 2020; Dong et al., 2022). For example, the CGM could be pre-trained multi-modal foundation model; the dataset could be collection of patient histories, test results, and patient diagnoses; and the prediction task could be to communicate the diagnoses to a new patient with a given history and test results (Nori et al., 2023). This problem is complex, demanding accuracy in diagnosis and appropriate communication to the patient. This complexity challenges our ability to assess whether the model is appropriate for the dataset and prediction task.
One interpretation of ICI. is that a CGM prompted with in-context examples produces data (either responses or examples of the prediction problem) from a posterior predictive under Bayesian model (Müller et al., 2021; Xie et al., 2021). A natural question arises when accept this premise, "Is the Bayesian model a good model for the prediction problem?" This is the question asked by the field of Bayesian model criticism. This field has produced many methods; however, they typically assume access to key components defined by the Bayesian model. Namely, the model likelihood and model posterior. In this work we show how to do model criticism in ICL using contemporary generative AI, specifically how to implement posterior predictive checks (PPCs) (Guttman, 1967; Rubin, 1984) and their cousins when we only have access the predictive distribution. The result is a practical and interpretable test on whether the model is appropriate for given ICL problem.
"Coreepoodeence to adj2147 @columbia.ddu Department of Statistics, Columbia University. Department of Computer Science, Columbia University.

---

Published as a conference paper at ICLR 2025
Input: proof once again that if the
Irpou: a) Should teena use the diet gLans on LV?
filmmakrrs just Follow the books
Label: negative
b) Car you help with dies plan?
Input: is impressive
Label: positive
Tabel: 61 6ifferent
Input: the top japanese animations
input: a) What's good way address back pain?
Label: positive
Iaput: spcof comedy
b) How can ouze back pala? Label: similar
Label: positive
(a) SST2 ICL dataset x"
(d) MQP ICL dataset x
Input: follows the formula but throws in many conflicts keep the stozy compelling
Input: al Can dementia cause ANS dyalunotions how? bt How can dementia cause ANS dystunction?
(b) query
(c) query
Label: negative, Label: negative,
Label: diffecent, Label: different,
Tabel: negative, Tabel: negative,
Label: similer, Label: similar,
Label: negative, label: negative
Label: different, Label: similer
(c) CGM responses
(f) CGM responses
Figure I: Examples of solvable and unsolvable ICL problems for Llama-2 7B Touvron et al. (2023). On the left, we have (a) an ICL dataset x" from sentiment analysis task Socher et al. (2013), (b) a new query z, and (c) some responses y sampled from the CGM p 2, x,xx). The CGM solves the problem by responding correctly with "negative". On the right, the dataset (d) and query (e) are taken from the medical question pairs task McCreery et al. (2020). The true label is "similar," but the model frequently responds incorrectly with "different," and does solve the problem.
2 WHAT IS AN IN-CONTEXT LEARNING PROBLEM?
We formalize an ICL problem as a tuple (f* x"" 0): a prediction task f", a dataset x", and conditional generative model (CGM) O. The prediction task defines the joint distribution of queries z and their responses y. In turn, the set of valid responses to a user query implies a reference distribution over responses p(y f. The dataset x" response examples of the prediction task; Zi,Y; ~ plz,y f)). A practical data abstraction sees the decomposition of queries and responses into elements called tokens. As such, queries and sequences of tokens. For example, (z. y) (tg, t .g..... th tt....) = (th, ....... ). A conditional generative model defines predictive distribution over the next token in an example 1 given previous example tokens and tz) and tokenized dataset; (t) j x"). Through ancestral sampling, the CGM defines additional predictive distributions over responses (y z, x)), examples (z, x²), and datasets Pa (x x"). Figure illustrates examples from two different ICL tasks. Figures to gives an example from the SST2 sentiment prediction task for which Llama-2 7B frequently yields accurate answers. Figures 1d to gives an example from the medical questions pairs (MQP) prediction task for which Llama-2 7B yields random answers average. As illustrate next, there are several reasons why model generated responses or examples may be inappropriate for the ICL task f*
(:)) comprises query and
responses- by extension, examples and datasets are represented
3 WHAT IS A MODEL?
Let 0 again denote a model, but now the model could be Bayesian linear regression, a Gaussian process, or perhaps even a large language model (LLM). A model defines a joint distribution pe (xx f) over observable data x = {x1, notation f denotes both tasks and explanations, but we will distinguish between them. The model joint distribution factorizes in terms of the model prior (f over explanations and the model likelihood (e(x f) over observable datasets given an explanation; (x f) = Po ff) From frequentist perspective, the prior distribution over f would be ignored and a model would define set of distributions over datasets indexed by f; {pe(x f) e F}. A model alongside data x" further defines the posterior (f x²)) and posterior predictive p x") distributions, which specify the conditional distributions of explanations and new observations given the observed data. See Appendix A for deeper discussion each of these distributions.
{(1),1)). VY), (z2, y2).... and latent explanations f. The
pe (x f) dP x")
2

---

Published as a conference paper at ICLR 2025
Are CGMs Models? Modem CGMs often lend access only the marginal px(x) posterior predictive px x") rather than an explicit representation of So why do we discuss latent variables like f? First, if the model 0 is exchangeable (i.e., Px(x) is invariant to permutations of the data), de Finetti's theorem (de Finetti, 1937; Hewitt & Savage, 1955) ensures that px(x) can be expressed as a mixture over latent variable f, leading decomposition of the form [ Po (x dPo(f). Therefore, assuming we adopt unique representation of f, we can define pe (x, f) as the joint distribution over the latent variable and the data implied by the mixture. Alternatively, pe (x) approximates an exchangeable distribution p(x), as in ICL problems, then the statement (x. f) is convenient abuse of notation, representing p(x, f). We assume that either of these conditions hold.
4 A MODEL IS A CHOICE TO BE CRITICISED
A model 0 defined over observation space X' is used make inferences informed by observations from that space x". Inferences about the probability of the the next word given a sequence of words, model uncertainty, and many other quantities. But model is choice- practitioner makes modelling decision- and so it is not given that inferences made by a model are grounded in reality.
n=1
n=8
n=1
n
+0
(a) linear model, linear data
(b) polynomial model, polynomial data
nm]
nm8
100
-8
In.context examples
unsoon examples
true
est.
post. mean
est. post. mean
(c) polynomial model, lincar data
(d) linear model, polynomial data
Figure Misaligned models. Transformers (pink) are fit to either linear or polynomial data generated by Bayesian models (blue). Model predictions align with reference models for in-distribution tasks (Figures 2a and 2b), but are too conservative (Figure 2c) or too confident (Figure 2d) for OOD tasks.
Figure 2 illustrates inferences about the predictive distribution made by different models given different datasets. Figures 2a and 2b-model inferences (pink) overlap with those made by the reference model (blue) and appear purple. However, inferences made by a misaligned polynomial model-Fiiure 2c-are much wider that those made by the reference linear model. And inferences made by a misaligned linear model-Figure 2d-are more narrow than those of the reference polynomial model, which results in the model being confident and wrong. Similarly, a very general LLM may have compromised data efficiency for rare domains, while a highly specialized LLM may no longer generalize beyond its domain. These examples illustrate that while models are used to quantify empirical nact-liike the frequency of an event occurring-the aspect that needs be considered when using model inferences in practice. This consideration guides question of when model will provide reliable inferences for given ICL problem.
also carry subjective
Much the discussion around the reliability of CGMs has focused on "hallucination" detection, prediction, and mitigation (Dziri et al., 2021; Su et al., 2022; Lee et al., 2022; Mielke et al., 2022;
Gao et al., 2023; Li et al., 2023; Varshney et al., 2023; Feldman et al., 2023; Zhang et al., 2023;
Peng et al., 2023; Lin et al., 2023; Azaria & Mitchell, 2023; Chuang et al., 2024; Burns al., 2023;
Rimsky et al., 2023; Luo et al., 2023; Shi et al., 2024; Band et al., 2024; Li et al., 2024; Mündler
al., 2024; Dhuliawala al., 2024; Farquhar et al., 2024; Kossen al., 2024; Jesson et al., 2024).
3

---

Published as a conference paper at ICLR 2025
An interesting subset is based on uncertainty quantification where inferences about the variability of responses from the [posterior] predictive distribution (Kadavath et al., 2022; Manakul et al., 2023; Cole et al., 2023; Chen & Mueller. 2024), or about the variabilitty of explanations (Kuhn et al., 2023; Elaraby et al., 2023; Lin et al., 2024; Farquhar et al., 2024; Jesson et al., 2024) are used to predict model hallucinations. However, none of these methods address the question of when those inferences are to be trusted, and so each are susceptible to failure if the model is not appropriate for the task.
A growing body of work is formalizing the connection between ICL with pre-trained CGMs and Bayesian inference (Xie et al., 2021; Müller et al., ;021; Fong et al., 2023; Lee et al., 2023; Jesson et al., 2024; Falck et al., 2024; Ye al., 2024). Notably, the works of Fong al. (2023); Lee et al. (2023); Jesson et al. (2024); Falck et al. (2024); Ye et al. (2024) show how to transform Bayesian functionals of the model likelihood (x (f) and model posterior (f x") into functionals of the model predictive distribution po x"), which can be computed by contemporary CGMs. These works pave the way for using Bayesian model criticism techniques such as posterior predictive checks
as response to our ressearch question. In the following we formalize how this is done.
5 POSTERIOR PREDICTIVE CHECKS ARE MODEL CRITICS FOR ICL PROBLEMS.
Posterior predictive checks (Rubin, 1984; Meng, 1994; Gelman et al., 1996; Moran et al., 2023) are Bayesian model criticism methods that use the model posterior predictive to assess the suitability of model to make inferences informed by a set of observations. A model's ability to explain observed data is quantified by the posterior predictive p-value, which is derived from test of the hypothesis that the data are generated according to the model 0. Following Moran et al. (2023), we assume access to main x" and holdout xlest sets of observations, both distributed according the reference likelihood p(x | (*)) for a specific task 1*. The class of PPCs pertinent to our discussion assess "goodness-of-fff oof-fitt by asking how well a model fit to a set of observations x" explains the holdout PPC defines a discrepancy function, like the x"), or the negative log model likelihood (x (] == - -x,, yrex logpo Po yz f). Both of these measures will be lower for observations that are well explained by the model, and higher for those that are not.
observations xtest_ To measure the goodness-of-fi..
negative log marginal model likelihood go (x, x)) == - En,,, Ex log pe (z, (o, Yi
Defining goodness-of-fit measure is only half of the story. A PPC needs way to assess what makes a relatively high or relatively low value of the discrepancy function. To do this, a reference distribution of values is defined by measuring the discrepancy function over datasets sampled from the model posterior predictive distribution. The posterior predictive p-value is then evaluated as
Pape == If 1( 2 go (.adsss,)) dXP f)dPe(f x").
(1)
The PPC locates the value of the discrepancy function for the holdout data go gx(x :) in the distribution of the discrepancy function under the model go (X..). The more often the discrepancy (intuitively, the loss) of the data generated under the model greater than or equal to the discrepancy of the holdout data, the more confident we can be that the model explains the holdout data well. Conversely, if the discrepancy of the holdout data is commonly greater than that of the data generated under the model, we should be less confident that the model explains the holdout data, and thus be skeptical about the models capacity to solve the ICL problem. Algorithm 2 in Appendix D describes a Pryc estimator.
THE GENERATIVE PREDICTIVE p- VALUE AND HOW TO ESTIMATE IT
CGMs like LLMs Diffusion models (Ho al., 2020; Guirajani & Hashimoto, 2023) do not explicitly provide the joint distribution over observations and explanations. At best, they give the model posterior predictive (x x²) black box instead of an integral of the likelihood po f) over the posterior po(1 )) This is a problem for discrepancy functions that depend on f.
Our solution this problem relies the intuition that sufficiently large dataset x (2,, y: N N generated according to the model likelihood x px ff) given an explanation f contains roughly the same information as the explanation itself. For an identifiable Bayesian model, the model posterior (f x concentrates around the unique generating explanation f as N + co. Therefore, it makes sense to express functions of explanations f. like the likelihood po f), which are not defined under typical CGM, functions of large datasets, like the predictive po x ), which are defined.
4

---

Published as a conference paper at ICLR 2025
But, if only given the examples comprising x", where do the additional N examples come from? The generation of sufficiently large datasets is done by ancestrally sampling from the model predictive distribution po(z.y y x") generate hypothetical completions x= of the observed ICI dataset x" (also called predictive resampling by Fong et al. (2023)):
Zn+1s Vn+1
(z xx), Zn+2: Yn+2 pe(z.y x" 2n+1.N++1) Nn+1),
Adding generated examples to the conditional of the predictive hep-by--tte can be thought of as reasoming toward one explanation by imagining sequence of sets of observations that are consistent with a shrinking set of explanations as the sequence length increases. As stochastic process, it reasons toward a different explanation each time a completion of x" is generated by - examples.
Building off this intuition, we define martingale and generative predictive p-values below. We prove that under general conditions the martingale predictive p-value is equal to the posterior predictive p-value. We then show how to estimate the generative predictive p-value for given ICL problem.
6.1 THE MARTINGALE PREDICTIVE P-VALUE
Our method built on Doob's theorem for estimators (Theorem 2). This theorem helps us transform statements about the random variable h(F)-a function of explanations F-to statements about the random variable E[h(F) X2, ... Xal. which is a function of observations (X1, X2,..., X,)
Thus, we can proceed without direct access po (z. f)and pf x.) and define p-value that depends on infinite datasets x = Y.). rather than f
Pmppe := 1f 1000 2 gx x )dPo(( (x)ddP JdPo (x xx)).
(2)
Doob's Theorem is an application of martingales, so-in line with the current literature (Fong et al., 2023; Lee et al., 2023; Falck al., 2024)-we call this formulation the martingale predictive p-value.
The main theoretical result of this paper establishes the equality of the posterior and martingale predictive p-values; Equations (I) and (2). We formalize this statement in the following theorem. Theorem 1. Let F. ~ Pe, and X1, X2.... i.dd ~ Pf Pype
Pmpe under Conditions to 3 and
/ [logpo(x" [I)Po(() < <0: Vx E 2
Proof.
The proof makes of Doob's Theorem and is presented in Appendix C.
6.2 THE GENERATIVE PREDICTIVE P-VALUE
The martingale predictive p-value cannot be exactly computed because it is impossible to generate infinite datasets. Thus, we define the generative predictive p-value that clips the limits to infinity by some feasibly large number N to estimate Equation (2) as
Papc /ʃ 1 x (x } x xN)dPo(x" xx).
The generative predictive p-value enables us to replace distributions that depend on latent mechanisms f or infinite datasets x with that depends finite sequences. The cost of using finite N is estimation error between Pape and Pppe A formal analysis of this error is left to future work.
6.3 CGM ESTIMATORS FOR THE GENERATIVE PREDICTIVE P-VALUE
We derive an estimator for the generative predictive p-vuuue in Equation (3) that uses Monte Carlo estimates to approximate the integrals. Algorithm 1 describes the estimation procedure. The steps that distinguish the generative predictive algorithm from the posterior predictive algorithm are given in Lines 2 to 5. There, completions x:N of length N are sampled from the predictive distribution to approximate sampling a mechanism f.. This contrast to sampling an explanation directly from the model posterior shown in Algorithm 2 Line 2 of Appendix D. When sampling replication data x; in Lines 6 to 9, the predictive distribution is conditioned on x and n new examples are independently sampled. The procedure is repeated M times, and the p-value is estimated empirically.
5

---

Published as a conference paper at ICLR 2025
Algorithm 1 Pape
Require: data fx", x discrepancy function ge (x x)), ), # replicates M. # approx. samples N
1: for i
M do
2:
x
+ x²
initialize f sample data
3:
for j t n + N do
4:
2j,Y; y/x
> sample example from model update approximation context initialize replicant data
5:
x < (x N3)
6:
X; 5 ()
7:
for j f 1 to n do
8:
Yj ~ po xx²²)
b sample example from model > update replicant data estimate p-value
9:
X 4- (xi: (g.
10:
return M -i= (x), (Xi, xx 2 go (x xx²))
EMPIRICAL EVALUATION
This section reports the following empirical findings: (1) The generative predictive p-value is an accurate predictor of model capability in tabular, natural language, and imaging ICL problems. (2) The p-value computed under the NLL discrepancy is also an indicator of whether there are enough in-context examples between the posterior predictive p-value under the NLML discrepancy and the NLL discrepancy using the model posterior po (f x²) and likelihood (x f). These findings show that the p-value computed under either discrepancy yields an accurate predictor of whether generative Al can solve your in-context learning problem. If you also need to know whether there are enough in-context examples, we suggest using the NLL discrepancy function. If computational efficiency is a primary concern, we suggest using the NLML discrepancy as dataset completion generation is not required.
(3) The number of generated examples N interpolates the p-value
Models. We evaluate our methods using two model types. For tabular and imaging tasks, we use a Llama-2 regression model for sequences of continuous variables (Jesson et al., 2024). The model is optimized from scratch for next token (variable or pixel) prediction following the procedure of Touvron al. (2023). For natural language tasks, we use pre-trained 1lama-2 7B (Touvron et al., 2023) and Gemma-2 9B (Team, 2024) LLMs (Gemma-2 9B results are reported in Appendix H).
Contect Length
Contest Length 100
Langth
Length
In eaarge
Inscriien coomzz
(a) in-distribution tabular task
(b) OOD tahular task
Figure 3: Two randomly sampled tabular data tasks. For in-distribution tasks, (Figure 3a) confidence intervals aligo with response variability around the function. For OOD tasks (Figure 3b) the confidence intervals are misaligned, leading to inaccurate predictions, even with more in-context examples.
Data. For tabular tasks, queries z are sampled uniformly from the interval [- 2]. Responses are drawn from normal distribution with mean yuz),, parameterized by either a random 3rd-degree polynomial (in-distribution), random ReLU neural network (in-distribution or OOD), basis function (RBF) kernel Gaussian process with a length scale of 0.3 (OOD). The training data comprise 8000 unique in-distribution datasets with 2000 y examples each. An in-distribution ReLU-NN task is illustrated in Figure 3a. The mean function μ(() is plotted by the blue line, and the blue shaded region outlines the 95% CI of p 2, An OOD GP task is illustrated in Figure 3b. In-distribution test data comprise a set of 200 new random datasets with 500 z examples each. The OOD test data comprise 200 random datasets with 500 y examples each.
radial

---

Published as a conference paper at ICLR 2025
SST2
AG News
Medical QP
RTE
04
Enror Rate
eeexmpppes
examples
examples
examples
(a) capabbiiyyyt
(b) Out-of-capabiliy
Figure 4: Natural language in-capability vs. out-of-capability tasks. Green solid lines are the ICL error rate for Llama-2 7B. Gray dashed lines are the random guessing error rate. We define SST2 and AG News as in-capability, as the error rates significantly lower than random guessing. We define Medical QP and RTE as out-of-capability. as the error rates are similar to random guessing.
For pre-trained LLM experiments, the delineation between in- and distif-distributtin opaque. Instead, we use in-capability or out-of-capability to differentiate between tasks a model can or cannot perform well. Figure 4a illustrates in-capability tasks where the error rate of Llama-2 7B is considerably better than random guessing. The in-capability data are the SST2 (Socher et al., 2013) sentiment analysis (positive vs. negative) and AG News Zhang et al. (2015) topic classification (World, Sports, Business, Sci/Tech) datasets. Figure 4b illustrates out-of-capabillty tasks where the error rate is only marginally better than random. The cut-of-capability data are the Medical Questions Pairs (MQP) (McCreery al., 2020) differentiation (similar different) and RTE (Dagan et al., 2006) natural language inference (entailment vs. not entailment) datasets.
inccoment
prompx
ecamples
arompt
Inncomtect examples
pramot
real
12 95
8
7
8
3
3
B
2
6 >
0
9
9
3
/
6
7
7
D2
O
3
3
+
1
F
7
10
44
8
8
6
5
O
7
2
22
2
2
8
6
6
(a) SVHN (in-distribution)
(b) MNIST (near OOD)
(c) CIFAR-10 (far OOD)
Figure 5:
Generative fill completions on the SVHN MNIST, and CIFAR-10 test sets. Completions sensible for the in-distribution task, but contain artifacts and hallucinations for the OOD tasks.
For imaging ICL experiments, we use SVHN for in rn-distribution data (Netzer et al., 2011). MNIST as "near" OOD data (LeCun et al., 1998), and CIFAR-10 "far" OOD data (Krizhevsky et al., 2009). Our Llama-2 regression model takes a sequence of flattened, grayscale, 8x8 images input. It is fit to random sequences of 16 images from the SVHN "extra" split, which has over 500k examples. A series of in-distribution generative fill tasks is shown in Figure 5a. In each row, the model is prompted with three in-context examples and asked to complete the missing half of the 4th example. Each completion in the "fill" column is sensible, even when the completed number differs from the "real" number. Figure 5b illustrates completions for near-OOD MNIST tasks. We see in rows 1, 2, 3, and 6 that the fills are often sensible, but the model is prone to hallucinating odd completions (row 4) and artifacts (row 5). Figure 5c illustrates completions for far-OOD CIFAR- 10 tasks. The completions are surprisingly consistent at this resolution, but as the result in row demonstrates, the model hallucinates completions from its domain.
Discrepancy functions. We evaluate the p-value using discrepancy functions defined
go (x,xx( x =
Σ tyE(s.))))
logpe(t, tt,,,()),,
(2i,
Ex
7

---

Published as a conference paper at ICLR 2025
where ((2, ys)l is the number of tokens in the evaluated example. Following this template, the per- token negative log marginal likelihood (NLML) is written go (x, x") and an estimate of the per-token the negative log-likelihood (NLL) is written (x x where x is generated as in Algorithm 1.
Predicting model capability. The p-values are calculated using either Algorithm I or Algorithm 3. Selecting a significance level yields binary predictor of model capability 1 (Pege aa): model is predicted incapable if the estimated generative predictive p-value is less than the significance level. We report results for significance levels a E [0.01, 0.05, 0.1, 0.2, 0.5]. For the NLL discrepancy function, replication data x is independently sampled from the likelihood under a hypothetical dataset completion pe (z,y y xx). ). For the NLML discrepancy function, replication data is independently sampled from the predictive distribution pe y x").
Evaluation metrics. We evaluate the capa- bility predictor using standard metrics: FPR measures ca-aabiiiyty tasks misclassified as out-of-capability, Precision reflects correctly identified out-of-capability tasks, and Recall measures correctly detected out-of-capability tasks. F1 Score and Accuracy assess overall performance (see Figure 6 for definitions).
Figure 6: Evaluation metrics for GPC performance.
Metric
Equation
False Positive Rate (FPR)
False Pusitives
False Positives +True Negatives
Precision
True Positives Positives True Positiyes
Recall
Trme Positives False Negatives
F1 Score
2. PrPrrctsion Recall
Precision-- Recall
We also provide the distribution of p-values across tasks to assess how confidently the model distinguishes between the different
Accuracy
True Positives +r True Negatives
Total Number of Predictions
ICL problems. Lower p-values indicate stronger confidence that model cannot solve problem.
7.1 THE GENERATIVE PREDICTIVE p- VALUE ACCURATELY PREDICTS MODEL CAPABILITY
Tabular data, We first evaluate whether the generative predictive p-value effectively predicts OOD tabular data tasks. The parameters for Algorithm 1 are M = 40 replications and N n = 200 generated examples. The ICI. dataset x size is varied from n 2 to n = 200. Figure plots precision, recall, F1, and accuracy curves and shows that the p-value estimates under either discrepancy function provide non-trivial OOD predictors for all a settings.
Dicpessnccy Functon
0.01
0.5
Randon
Figure 7: Tabular OOD detection. Metric values vs. context length. In-distribution functions from unseen random ReLU-NNs. OOD functions are from an RBF kernel GP. Both discrepancy functions yield better than random OOD predictors for range of settings.
Natural language ICL. Next, we evaluate whether the generative predictive p-value effectively predicts out-of-capacity natural language tasks. The parameters for Algorithm 1 are M 20 replications and N' - n = 10 generated examples. The ICL. dataset x" size varied from n = to 64. Figure 8 plots precision, recall, F1, and accuracy curves and shows that the p-value estimates under the NLL discrepancy provide non non-trivial (accuracy > 0.5) oft-o--capability predictors in the domain of natural language for all settings. The NLML discrepancy go (x x") is also generally robust outside of the small n and small a setting.
Generative fill. Finally, we evaluate whether the generative predictive p-value effectively predicts OOD generative fill tasks. The parameters for Algorithm are M 100 replications and n = 8 generated examples The ICL dataset x size is varied from = 2 to 8. Figure 9 plots the
8

---

Published as a conference paper at ICLR 2025
scresancyy ou
ge
0.01
--- 0.05
-ww 01 0.2
0.5
Predtin
Pocall
14
Figure 8: Llama-2-7B out-of-capability detection. Metric values vs. context length. In-capability tasks are from SST2 and AG News datasets. Out-of- cut-of-capability tasks are from RTE and MQP datasets. Both discrepancy functions yield better than random OOD predictors for range of a settings.
OOD prediction metric curves and shows that the p-value estimates under either discrepancy function provide non-trivial (accuracy > 0.666) OOD predictors for all a settings.
Discrepency
x
x
0.01
0.05
0.5
PecCa
Figure 9: Generative fill OOD detection. Metric values vs. context length. In-distribution tasks are from the SVHN test set. Near and far OOD tasks are from the MNIST and CIFAR-IO test sets. Both discrepancy functions yield better than random OOD predictors for range of a settings.
Discussion. Figures
reveal several trends. First, the NLML discrepancy (blue) yields better precision, indicating that it is less likely to misclassify an in-capability ICL problem as unsolvable. Second, the NLL discrepancy (purple) yields higher recall, indicating that it is less likely to misclassify an out-of-capability ICL problem solvable. Third, the NLL discrepancy with significance level a = 0.05 yields a generally more accurate predictor than the NLML discrepancy function for any significance level in the set evaluated. Finally, the recall of predictor under the NLML discrepancy sensitive to the number of in-context examples n. Next, we look deeper into the relationship between dataset size and the discrepancy functions.
7.2 THE NLL DISCREPANCY ALSO INDICATES WHETHER YOU HAVE ENOUGH DATA
ab.
(a) NLL
(b) NLML
(c) Risk
(d) Accuracy
Figure 10: With equal accuracy, the NLML discrepancy yield classifier with lower risk. Figures 10a and 10b are scatter plots of response RMSE vs. p-values for each discrepancy function. Points are coded by ICL dataset size Figure 10c Risk vs. n. Figure 10d Accuracy
Both discrepancy functions yield accurate predictors of model capability, but the NLL discrepancy tells us if there are enough in-context examples reliably solve task. We use prediction RMSE

---

Published as a conference paper at ICLR 2025
over task responses measure reliability. Figures 10a and 10b plot the RMSE against the p-values computed under the NLL and NLML discrepancies for in-distribution polynomial tabular tasks. We that lower p-values correlate with higher RMSE for the NLL discrepancy, but not for the NLML discrepancy. This information is useful for reducing risk in autonomous recommendation systems that respond if the p-value is greater than the significance level For example, discrepancy reduces the generation of responses with high error because it accounts for the number of examples provided. Taking the risk as the sum of task RMSES for tasks predicted as in-capability, Figures 10c and 10d show that the NLL discrepancy results in substantially reduced risk, even when the accuracies of each predictor are matched. Figure 14 in the appendix gives further insight into how the distributions of p-values evolve with dataset size for each discrepancy function.
0.1, the NIL
7.3 THE NUMBER OF GENERATED EXAMPLES N - INTERPOLATES THE p- VALUE ESTIMATE BETWEEN THE NLML AND THE IDEAL NLL DISCREPANCIES
N-n 2
N-n = 10
N-n 50
N-n 100
0.6
p-value
0.4
20
40
62
20
40
60
20
40
20
40
60
Discrepancy
9o
9e0x. x*)
g(x,
Split
tast
00D
Figure 11: The p-value is interpolated between the NLL estimate and the NLML estimate as completion size N grows from oc.
Inspection of Equations (1) to (3) shows that the dataset completion size N n should interpolate p-value estimates between Pppe computed with the NLML discrepancy and with the NLL discrepancy using the likelihood and posterior of a Bayesian model. To verify this, we use a reference Bayesian polynomial regression model to compute the Pype. We use our Llama-2 regression model fit to datasets generated from the reference model likelihood under different explanations to compute the Pape We let datasets generated by random ReLU-NNs serve OOD tasks. Figure/ 11 demonstrates that our expectation is true. Specifically, the p-value estimates at N n 2 are distributionally close to those calculated under the NLML, and they more closely approximate those calculated under the reference NLL discrepancy as we increase N n to 100. The latter observation is also illustrated in Figure 13.
Since the p-values from either discrepancy accurately predict model capability, the choice depends on whether the added computational cost of generating dataset completions justified. If you need to know whether there are enough in-context examples to generate accurate response- necessity in desk-sensitive applications then we recommend using the NLL discrepancy function. If computational efficiency the cost of response deferral primary concerns -prectcca user experience concerns- we suggest using the NLML discrepancy.
8 CONCLUSION
This work introduces the generative prediccive predictivvpp-valee. a metric for determining whether a Conditional Generative Model can solve an in-context learning problem. It extends Bayesian model criticism techniques like PPCs to generative models by sampling dataset completions from the model's predictive distribution approximate sampling latent explanations from a Bayesian model posterior. Empirical evaluations on tabular, natural language, and imaging tasks show that the generative predictive p-value can effectively identify the limits of model capability, distinguishing between in-capability and out-of-capability tasks for models like Llama-2 7B and Gemma-2 9B. This approach a practical method to assess model suitability that advances Bayesian model criticism for CGMs. While we have focused on model capability prediction, the p-value estimates could also be used for model selection or as a general measure of task uncertainty. We are eager to explore extensions beyond ICI. tasks to improve the reliability of generative AI systems.
10

---

Published as a conference paper at ICLR 2025
9 ACKNOWLEDGMENTS
This work is supported by the funds provided by the National Science Foundation and by DoD OUSD (R&E) under Cooperative Agreement PHY-2229929 (The NSF AI Institute for Artificial and Natural Intelligence). Additional support was provided by the National Science Foundation under grants IIS-2127869 and DMS - 1108, the Office of Naval Research under grant 42401444, and the Simons Foundation.
REFERENCES
Amos Azaria and Tom Mitchell. The internal state of an LLM knows when it's lying, In EMNLP, PP. 967-976, 2023.
Neil Band, Xuechen Li, Tengyu Ma, and Tatsunori Hashimoto. Linguistic calibration language models. WwXiv:2404000474, 2024.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 13:187771 2020.
Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. Discovering latent knowledge in language models without supervision. In ICLR, 2023.
Jiuhai Chen and Jonas Mueller. Quantifying uncertainty in answers from any language model and enhancing their trustworthiness. In ACI., pp. 5186-5200, 2024.
Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large language models. In ICLR, 2024.
Jeremy Cole, Michael JQ Zhang, Daniel Gillick, Julian Martin Eisenschlos, Bhuwan Dhingra, and Jacob Eisenstein. Selectively answering ambiguous questions. EMNLP, 2023.
Ido Dagan, Oren Glickman, and Bernardo Magnini. The pascal recognising textual entailment challenge. In Joaquin Quiñonero-Candele Ido Dagan, Bernardo Magnini, and Florence d' Alché Buc (eds.), Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Clas- sification, and Recognising Tectual Entailment, pp. 177-190, Berlin, Heidelberg, 2006. Springer Berlin Heidelberg, ISBN 978- 11883-540-33428-6
Bruno de Finetti. La prévision: ses lois logiques, sources subjectives. Annales de I' l'Instiuut Henri Poincaré, 7:1-68, 1937.
Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, and Jason E Weston. Chain-of-verificaatoo reduces hallucination in large language models. In ICLR 2024 Workshop on Reliable and Responsible Foundation Models, 2024.
Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui. A survey on in-context learning. arXiv preprint arXiv:2301.00234, 2022.
Joseph Doob. Application of the theory of martingales. Le calcul des probabilites et ses applications, pp. 23-27, 1949.
Nouha Dziri, Andrea Madotto, Osmar R Zaiane, and Avishek Joey Bose. Neural path hunter: Reducing hallucination in dialogue systems via path grounding. In EMNLP, pp. 2197-2214, 2021.
Mohamed Elaraby, Mengyin Lu, Jacob Dunn, Xucying Zhang, Yu Wang. and Shizhu Liu. Halo: Estimation and reduction of hallucinations in open-souree weak large language models. arXiv:2308. 11764, 2023.
Fabian Falck, Ziyu Wang, and Chris Holmes. Is in-context learning in large language models bayesian? a martingale perspective. arXiv preprint arXiv:2406.0075: 00793, 2024.
Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. Detecting hallucinations in large language models using semantic entropy. Nature, 630(801 7):62--6:665 2024.

---

Published as a conference paper at ICLR 2025
Philip Feldman, James R Foulds, and Shimei Pan. Trapping LLM hallucinations using tagged context prompts. arXiv:2306.06600 2023.
Edwin Fong, Chris Holmes, and Stephen G Walker. Martingale posterior distributions. Journal of the Royal Statistical Society Series B: Statistical Methodology, 85(5):1357-139,, 2023.
Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, et al. RARR: Researching and revising what language models say, using language models. ACL, pp. 16477-16508, 2023.
Andrew Gelman, Xiao-Li Meng, and Hal Stern. Posterior predictive assessment of model fitness via realized discrepancies. Statistica sinica, pp. 733-760, 1996.
Ishaan Gulrajani and Tatsunori B Hashimoto. Likelihood diffusion language models. Advances in Neural Information Processing Systems, 36: 16616666116715 2023.
Irwin Guttman. The use of the concept of future observation in goodness-of-fit fit problems. Journal of the Royal Statistical Society: Series B (Methodological), (9()))) 29(1):83-100,
1967.
Edwin Shields Hewitt and Leonard Savage. Symmetric measures on cartesian products. Trans-
actions of the American Mathematical Society, 80:470-501, 1955. URL https://apt
semanticscholl org/ Barg/Corp Corpus 50:3585008 53585081.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Andrew Jesson, Nicolas Celtran-Velez Quentin Chu, Sweta Karlekar, Jannik Kossen, Yarin Gal, John P Cunningham, and David Blei. Estimating the hallucination rate of generative ai. arXiv e-prints, pp. 2406, 2024.
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield Dodds, Nova DasSarma, Eli Tran-Johnson, al. Language models (mostly) know what they know. arXiv:2207.0522) 2022.
Jannik Kossen, Jiatong Han, Muhammed Razzak, Lisa Schut, Shreshth Malik, and Yarin Gal. Semantic entropy probes: Robust and cheap hallucination detection in 1lms. arXiv preprint arXiv:2406. 15927, 2024.
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. University of Toronto, 2009.
Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. Semantic uncertainty: Linguistic invariances for uncertainty estimation natural language generation. ICLR, 2023.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2322 1998.
Hyungi Lee, Eunggu Yun, Giung Nam, E Fong, and Juho Lee. Martingale posterior neural processes. In The Eleventh International Conference on Learning Representations. International Conference on Learning Representations, 2023.
Naycon Lee, Wei Ping, Peng Xu, Mostofa Patwary, Pascale N Fung, Mohammad Shoeybi, and Bryan Catanzaro. Factuality enhanced language models for open-ended text generation. NeurIPS, 35: 34586-34599 2022.
Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Plister, and Martin Wattenberg. Inference-time intervention: Eliciting truthful answers from language model. NeurIPS, 36, 2024.
Xingxuan Li, Ruochen Zhao, Yew Ken Chia, Bosheng Ding, Shafiq Joty, Soujanya Poria, and Lidong Bing. Chain-of-knowledde Grounding large language models via dynamic knowledge adapting over heterogeneous sources. ICLR, 2023.
Stephanie Lin, Jacob Hilton, and Owain Evans. Teaching models to express their uncertainty in words. TMLRR 2023.
12

---

Published as a conference paper at ICLR 2025
Zhen Lin, Shubhendu Trivedi, and Jimeng Sun. Generating with confidence: Uncertainty quantifica- tion for black-box large language models. TMLR, 2024.
Junyu Luo, Cao Xiao, and Fenglong Ma. Zero-resource hallucination prevention for large language models. CoRR, 2023.
Potsawee Manakul, Adian Liusie, and Mark JF Gales. SelfCheckGPT:
Zero-resource black-box hallucination detection for generative large language models. In EMNLP. 2023.
Clara H. McCreery, Namit Katariya, Anitha Kannan, Manish Chablani, and Xavier Amatriain. Effective transfer learning for identifying similar questions: Matching user questions to covid-19 faqs. In Proceedings of the 26th ACM SIGKDD international conference knowledge discovery & data mining, pp. 3458-3465, 2020.
Xiao-Li Li Meng. Posterior predictive p-values. The annals of statistics, 22(3):114--116) 1994.
Sabrina J Mielke, Arthur Szlam, Emily Dinan, and Y-Lan Boureau. Reducing conversational agents' overconfidence through linguistic calibration. TACL, 10:857-872, 2022.
Jeffrey W Miller. A detailed treatment of doob's s theorem. arXiv preprint 18010: 122.022 2018.
Gemma E Moran, David M Blei, and Rajesh Ranganath. Holdout predictive checks for [b]ayesian model criticism. In 2023 IMS International Conference on Statisties and Data Science (ICSDS), pp. 94, 2023.
Samuel Müller, Noah Hollmann, Sebastian Pineda Arango, Josif Grabocka, and Frank Hutter. Transformers can do bayesian inference. In International Conference on Learning Representations, 2021.
Niels Mündler, Jingxuan Hc, Slobodan Jenko, and Martin Vechev. Self-contradictory hallucinations of large language models: Evaluation, detection and mitigation. In ICLR, 2024.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al. Reading digits natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning. Granada, 2011.
Harsha Nori, Yin Tat Lee, Sheng Zhang, Dean Carignan, Richard Edgar, Nicolò Fusi, Nicholas King, Jonathan Larson, Yuanzhi Li, Weishung Liu, et al. Can generalist foundation models outcompete special-purpose tuning? case study in medicine. CoRR, 2023.
Baolin Peng, Michel Galley, Pengcheng He, Hao Cheng, Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu, Weizhu Chen, et al. Check your facts and try again: Improving large language models with external knowledge and automated feedback. arXiv:2302. 12813, 2023.
Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. Steering Llama 2 via contrastive activation addition. arXiv:2312.06681, 2023.
Donald B Rubin. [b]ayesianly justifiable and relevant frequency calculations for the applied statisti- cian. The Annals of Statistics, pp. 1151-1177 172, 1984.
Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Wen-tau Yih. Trusting your evidence: Hallucinate less with context-aware decoding. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), pp. 783-791, 2024.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In EMNLP, pp. 1631-1642, 2013.
Dan Su, Xiaoguang L., Jindi Zhang, Lifeng Shang, Xin Jiang, Qun Liu, and Pascale Fung. Read before generate! Faithful long form question answering with machine reading. In ACL, pp. 744-756, 2022.
Gemma Team. Gemma, 2024.
13

---

Published as a conference paper at ICLR 2025
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. 2rX.::2307.00228. 2023.
Neeraj Varshney, Wenlin Yao, Hongming Zhang, Jianshu Chen, and Dong Yu. A stitch in time saves nine: Detecting and mitigating hallucinations of LLMs by validating low-confidence generation. arXiv:2307.03987, 2023.
Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. An explanation of in-context learning as implicit bayesian inference. In International Conference Leurning Representations, 2021.
Naimeng Ye, Hanming Yang, Andrew Siah, and Hongseok Namkoong. Pre-training and in-context learning is bayesian inference la de finetti. arXiv preprint (xXiv:2408.0333007 2024.
Shuo Zhang, Liangming Pan, Junzhou Zhao, and William Yang Wang. Mitigating language model hallucination with interactive question-knowledge alignment. arXiv:2305. 2023.
Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification, NeurIPS, 2015.
14

---

Published as a conference paper at ICLR 2025
A MODEL INTUITION
In this section give interpretation of the component parts of Bayesian model, how they are used to make inferences about uncertainty, and how to relate inferences in classical domains to inferences in more complex domains like language.
The model prior (f) can be thought of as a library over the possible explanations a model could ascrihe observations. It is special kind of library, where the probability of finding explanation in the library at random is also defined. The model prior encodes everything "known" to a model; all the latent patterns available explanations for-or an index of all the probability distributions ascribable to to-any set of observations. The model prior may or may not assign non-zero probability to an explanation equivalent to a given ICL problem task f* If no such explanation has coverage under the prior, then the model may not be able provide an accurate solution to the ICL problem.
n-8
(a) lincar model, lincar data
(b) polynomial model, polynomial data
n=0
n-2
n=100
In-corcext examples
unseen examples
true
CI
CST. CI
true post. mean
est. past. mean
(c) polynomial model, linear data
(d) linear model, polynomial data
Examples of misaligned model and data combinations. Transformer models (pink) are fit to either linear or polynomial noisy data defined by reference Bayesian models (blue).
Figure 12:
For example, a Bayesian linear regression model with fixed noise, defines the part of the prior over explanations Po (f) related the outcome y as set of coefficient vectors; set of hyperplanes. If the ICL dataset and prediction task are characterized by linear relationship between continuous valued queries and responses-illussee illustratedd in Figure 12a-the prior would be suitable for the ICL problem. However, if they characterized by polynomial eelationships-iillssttteedd illustrated in Figure 12c-ther the relationship would not have coverage under the prior and the precision of responses under the model would be limited by the capacity of hyperplane to fit polynomial surface. By analogy, LLM that is pre-trained or fine-tuned on a large set of integrals expressed in natural language may have the functional capacity integrate; generalize to unseen functions and domains of the classes and spaces covered in the training set. So if the ICL dataset and task are related to the integration of polynomials, the learned library of mappings from text token distributions may be appropriate for that ICL problem. However, if the LLM training corpus did not contain content related to calculus, then learned library of mappings may not include the functional capability solve the ICL problem.
The model likelihood (x | f) encodes the variety of observations x that could be generated according to a given explanation f. The variety encoded by this distribution is often called aleatoric uncertainty-aleatonn is pretentious word for random- wwhch refers to the inherent randomness over generated datasets when sampling according to the likelihood under a given explanation f. For example, given the explanation implied by fair coin, we will still be uncertain whether the outcome of a single coin flip will be heads or tails. More contemporarily. if you are already familiar with this concept and I were to say, "I would like to share the idea of aleatoric uncertainty with you," you would know which idea want to share. but, before reading this paragraph, you would be uncertain about how would share with you. When the model likelihood po (f) is indexed by explanation
15

---

Published as a conference paper at ICLR 2025
that is equivalent to the task f f*, then (x f) is equal to the reference likelihood p(x f*). So even though we may be uncertain about which dataset would be generated according to the model likelihood, could still be certain that the generated dataset would correspond to the task. However, there is a discrepancy between the model and reference likelihood then the model may not be suitable for the in-context learning problem.
The model posterior
[ [xxxxx= po (x" (f)
Pe (x" ())PP(())
encodes variety over explanations that could have generated a specific set of observations x", This variety is often called epistemic uncertainty. Epistemic is pretentious term referring to knowledge, conveying that we may not yet know which explanation f among subset of reasonable explanations best explains possible datasets x under the ICL problem.
For example, given only four observations say, two heads and two tails- sample mean estimate for the probability of observing heads is 0.5. However, we may still be uncertain about whether the coin generating the outcomes was fair or biased because the variance of that estimate is still a non-negligible 0.0625 when we assume the coin is actually fair. Related if I only say. "I would like to share an idea with you," you can probably imagine an abundance of ideas that I could be referring and thus still be uncertain about which one I have chosen to share.
contemporary example,
relevant feature of epistemic uncertainty is that it is reducible as we observe more context. In the coin flip example, as we observe more outcomes, our certainty about the probability of heads increases. In the second example, you may have better idea about the class of ideas may share based on what has been presented thus far.
As function of both the model likelihood and prior, the model posterior inherits the limitations of both. But also provides information about whether an in-context learning problem can be solved reliably. Namely, variety over explanations is indicative of being uncertain about which task the ICL dataset corresponds to. This variety can lead the model generating responses corresponding to tasks other than r* But it may also be indicative of when more examples (larger n) can improve the quality of solutions to an in-context learning problem.
The model posterior predictive
(x xx) f pe(x f)dP'(ff xx"),
derived from the model to generate new observations x given past observations x". Poetically, the model posterior predictive gives the model voice respond observations with observations. The model posterior predictive convolves the model likelihood of the observations given an explanation with the model posterior over explanations. This process entangles variety over explanations after observing a dataset x" and variety over observations x for each specific explanation f; the model posterior predictive entangles aleatoric and epistemic uncertainty.
Model inferences. A model 0 defined over an observation space X' is used make inferences about observations from that space x". Inferences like the probability of the the word given sequence of words, model uncertainty, and countless other things. But a model is a choice-the practitioner makes a modeling decision-and so the inferences derived from observations under a model may or may not be grounded in reality.
Figure 12 illustrates inferences about the predictive distribution made by different models given different datasets. When the data and model are well aligned- Figures 2a and 22b-model inferences (pink) overlap with those made by the reference model (blue) and appear purple. However, inferences made by misaligned polynomial model-Figure reference linear model. And inferences made by a misaligned linear model-Figure 12d-are more narrow than those ogf the reference polynomial model, which results in the model being confident and wrong. Similarly, a very general LLM may have compromised data efficiency for rare domains, while highly specialized LLM may longer generalize beyond its domain.
12c-are much wider that those made by the
These examples illustrate that while models are used to quantify empirical of event occurring -they also carry subjective aspect that needs to be considered when using model inferences in practice. This consideration guides our question when a model will provide reliable inferences for given ICL problem.
the frequency
16

---

Published as a conference paper at ICLR 2025
B DEFINING MODEL CAPABILITY FOR AN IN-CONTEXT LEARNING PROBLEM
In this section, we formalize the idea that, "a model can solve an ICL, problem." A formal definition must account for two things: (I) the model may generate undesired responses because the context does not adequately specify the task, and (2) the model may generate undesired responses despite the context precisely specifying the task. The definition below accounts for both.
Definition 1. An ICL problem comprises model 9, a dataset x" = =(2,,ilii=1 p(x" [f), and task f", Assume that valid responses y to user queries z ~ p(z (*) are distributed as ply z, f*) under the task. Finally, let A(z, I*)) denote any sel of responses satisfying
Pr(Y f(z,f") f) 2 -1-
(4)
The model O is called capable of solving the ICL problem if
/ | (1,, 2, 1,,x,,))))))))),))) 2
lim 71
(5)
Equation (4) defines set of valid responses y to any query z given the task f*. Jesson et al. (2024) call this a (1 e) e)-likely set. For example, the set could be a confidence interval in a regression task or set of semantically equivalent ways to express positive sentiment in an open-ended sentiment analysis task. The 1 - c set gives us a formal and general way to express the notion of a desirable response for given query and task. Equation (5) then says that model is capable if the probability that a generated response belongs to the set of valid responses converges to be at least 1 - as the size of the dataset grows; the context more precisely specifies the task. This definition accounts for condition (1) through the limit, allowing for capable models with too few in- in-context examples to be called capable. The indicator accounts for both conditions by counting the number of times model-generated responses fall inside the 1 - set; a general measure of accuracy.
In addition to accounting for the above conditions, this definition allows the model predictive distribution to collapse to subsets of A(z, f*) even deterministic responses- and still be called capable. This attribute is preferable to a definition of capability that requires the model predictive distribution to converge to the reference distribution, which would exclude many practical models.
While this definition is general, there are still practical limitations to consider. For example, an infinitely deep and wide random transformer with a finite maximum sequence length might be capable of solving most problems under this definition; however, its data efficiency may be so poor that we fill the context window before it can generate accurate responses. Similarly, even if the model could accommodate infinitely long sequences, the available data may exhausted before the model generates desirable responses. These scenarios are extreme examples of the parameterzzed case illustrated in Figure 2c.
This discussion also sheds light on the propensity for classifier using the NLML p-value to produce false negatives (misclassify out-of-capabilie tasks as in-capability). Focusing on the x > 0.5 region of Figure 3b, the model will be predicted as in-capability because the model predictive distribution covers the data for small in-context learning dataset sizes. It is not until the model sees 100 examples that the misalignment between the predicted and reference distributions becomes apparent. The NLL discrepancy addresses this failure mode, as discussed.
C PROOFS FOR THEORETICAL RESULTS
We restate our formalization and assumptions for convenience. Observable examples (z,y) E X are modeled by the (X, A))raddom variable X, and explanations I E are modeled by the (F, B)-- random variable F, where A and B are the relevant sigma algebras. For each f E F. let the model 0 define a probability measure P on AA) Let the model 0 further define a probability measure Pe on ()) And let Pe and P1 define the joint measure Me over ((X,, X2..... ........... F). Finally, we overload the notation ">," It means "sampled according to" when referring to the relationship between a random variable instance and a density or distribution; e.g., x ~ (x)).. And it means "distributed as" when referring to the relationship between a random variable and probability measure; e.g., X ~ P Our method rests on Doob's 's theorem for estimators (Doob, 1949), which assumes the following three conditions.
17

---

Published as a conference paper at ICLR 2025
Condition 1. The observation X and explanation F spaces are complete and separable metric spaces.
Condition 2. The set of probability measures (Po f F} defined by the model is a measurable family; the mapping f + (A) is measurable for every A E A.
Condition 3. The model 0 is identifiable;
(6)
Given these conditions state Doob's theorem.
Theorem 2. Doob's 's Theorem for estimators. Let Po and X1, S1....... i.d Pf Assume Conditions I to 3 and a measurable function h F - BR such that f [(()) dP dP < co, then
lim B[h(F) X1, X2. X.,] = h(F) a.s. [Ma]. n-000
Proof.
Miller (2018) provides detailed proof of this theorem.
Lemma 3. Let F Pæ. and X1, X2... iid (ollogaax"" []]dPa(I) < 00 Vx E xm), Then,
Pyg Assume Conditions 1 3 and let
1 [x]
g0 F)
log logpo [F) =
logpo (x0) = (xx x
x
Proof.
Po (x F)
lim
po(x [fdPP(ff X1,..., X,)
n
lim
po(x [,, JdPPl
lim n+000
(x ,I,)..,K,))
0(x, F)
log (x F))
log lim pa(x ,I,...,M., n
x
lim
log px Xg..., Xn)
log Px x
x
g(x, X
Theorem 2. Under the conditions of Lemma 3,
Pppc
Pmppe
Proof. Define alternative probability model such that F ~ Px and X2,X2... ii.d pfx" Let Pas Po, and 9a denote the relevant quantities respecting this model. For example, Pa (y x) = Poly x, x²) and (f) Pf x"). Note that (x f) po(x f,x") Pa(x (f since X and X are
18

---

Published as a conference paper at ICLR 2025
independent when f is known.
(g) 2 go (x .f)] dPo(x f)dPo(f x*)
Pppe
(ga 2 "0 f) (x,
/ (ga(x,)) (x(xx"1111)) []] dP / (x, x==1:00)
/ (xa(x, f) ga(x ((xx,,1) (]} dPa (xtt1ie)) ) (((( x
// 1 {9a (x, x x00)) 2 (x x ( f, fix ddP (f, x
Lemma
// {9a x 2 ا ((otttnt dpp( x"~1i> dP (x=+1:00)
/ (ge(xxxx²) go , dPo(x 1+ x²))
Pmpc
D POSTERIOR PREDICTIVE P-VALUE ALGORITHM
Algorithm 2 details the procedure for calculating the posterior predictive p-value in Equation (1) given train data x", test data xtest, a discrepancy function go (x, f), and the nuber of replication datasets to generate M.
Algorithm 2 Pppc
Require: data {x" xes
discrepancy function go (x, f), # replicates M
1: for i + 1 to M do
2:
f (f x²)
D sample explanation f >> initialize replicant data
3:
X; + ()
4:
for j 4- to do
5:
2j. ~ yls)
sample example from model likelihood > update replicant data estimate p-value
6:
X; 4- (x,, Nj)
7:
return
M = 1 (x0 (i) 2 gx (G)))))))
1
E LITE GENERATIVE PREDICTIVE P-VALUE ALGORITHM
Algorithm 3 plice
Require: data fx" xest), a discrepancy function go (x x",, replicates M
1: for
M do
2: X4 < ()
initialize replicant data
3: for j 1 to do
4: 2j. Vj ~ y xi, x")
sample example from model & update replicant data estimate p- p-value
5:
X; f- (xi, Zj)
6: return M MM - 1 (x,, 2 go txsstt xx)]
Algorithm 3 summarizes a "lite" version of the estimator that forgoes approximate sampling from the model posterior and likelihood. Instead, it samples replication data directly from the model predictive distribution and calculates the discrepancy functions with respect to the observed data x" rather than dataset completion N
19

---

Published as a conference paper at ICLR 2025
F ADDITIONAL FIGURES
N
10
N
50
N
100
p-values: g(x, f)
1.00
0.655
A² = 0.784
8²= 0.330
0.75
0.50
0.23
0.00
0.00
0.25
0.50
0.75
1000
0.00
0.25
0.50
0.75
1000 0.00
0.25
0.50
0.75
100
0.00
0.25
0.50
0.75
100
p-values:
golx,
p-values: golx, x
p-values: gølx, xx)
p-values: go(x, xx)
Figure 13: Scatter plots demonstrating that Pape becomes better approximation Pope with increas- ing dataset completion size N nt.
(a) Polynomial Tabular
(b) ReLU-NN Tabluar
(c) Natural language
(d) Generative fill near
Figure 14: The generative predictive p-value against dataset size
Figure 15a shows p-values a function of the ICL dataset x" size n (context length). We see that there is clear separation between the estimated generative predictive p-values Pype for the in- distribution test set (solid lines) and the OOD dataset (dashed lines). The separation is robust across different ICL dataset sizes and the two discrepancy functions test.
Paroo Postuve Rate
125
(a) Generative predictive p-values
(b) False Positive Rate (NLML)
(c) False Positive Rate (NLL)
Figure 15: Simulated regression task. The generative predictive p-vulue against dataset size n and it's relationship to the false positive rate. The first figure shows the generative predictive p-values, and the second and third figures show the false positive rate with the NLML and NLL discrepancy functions. respectively.
Figure 15b plots the FPR for the capability detector defined by Pape the with NLML discrepancy. We see that the FPR is stable across ICL dataset size n and that FPR aligns well with the significance level Figure 15c plots the FPR for the capability detector defined by Pape the with NLL discrepancy. In contrast, the FPR decreases with increasing ICL dataset size.
Figure 16a shows p-values under the Llama-2 7B model function of the ICL dataset x" size n (context length). Again, we see a clear separation between the estimated generative predictive p-values ggp for the in-capability (solid lines) and the out-of-capabilig (dashed lines) tasks. The separation is robust across different ICL dataset sizes and the two discrepancy functions we test.
Figure 16b plots the FPR for the capability detector defined by Pope the with NLML discrepancy. We see that the FPR trends up with increasing ICL dataset size n and approaches the significance level a. Figure 16c plots the FPR for the capability detector defined by Pape the with NL. discrepancy. As for
20

---

Published as a conference paper at ICLR 2025
67/77
Felse Postiive Rabe
Felse Posttive
%
CPIL
(a) Generative predictive p-values
(b) False Positive Rate (NLML)
(c) False Positive Rate (NLL)
Figure 16: Natural language task. The generative predictive p-value against dataset size and it's relationship to the false positive rate. The first figure shows the generative predictive p-values, and the second and third figures show the false positive rate with the NLML and NLL discrepancy functions, respectively.
the simulated regression data, the FPR starts to decrease with increasing ICL dataset size, but for lower a values the trend begins to reverse.
G CHOICE OF DISCREPANCY FUNCTION
Model High Epistemic Uncertainty
Model High Aleatoric Uncertainty
4.
Drewe -0.2
Drow fy. 0.06
Braw f. 2.18
Braw fy 336
3.5
Drow fy 2214
3.5
Braw fJ -2.66
Posterior Fredective
Poster Predittvve
3.0
3.0
2.5
1.5
1.5
1.8
1.0
-5
Figure 17: Comparison of two models: Model exhibits high epistemic uncertainty, while Model exhibits high aleatoric uncertainty. The posterior predictive distribution is shown alongside samples of f and the test point "es Note that the numerical values in the plot differ from the example in the text for visual clarity. The discrepancy function g(z, f) helps distinguish these cases by evaluating how likely test is under different values of f.
might not be immediately clear when to use discrepancy function over another. In this section we argue that in general one should prefer to use g(x, f) when possible as it can identify when the model has high uncertainty about the value of f.
To illustrate this difference, imagine two models. The first model has posterior Pr(f x²) = ( 1) and likelihood p1 (x f) N 0.0001). The second model has posterior (2(f x²) (0, 0.0001) and likelihood (2(x f) 1)..1). In both models, the posterior predictive is essentially same standard normal, p(x xx) =0:=NN NO, However, the first model, posterior predictive variance is due epistemic uncertainty (we are unsure about the correct f), while in the second model, it is due to aleatoric uncertainty (we are confident about f. but the task is inherently stochastic). Figure Figure 17 illustrates this scenario.
Now assume that we have a test point l-es = 0.5. Ideally, we want to say that the second model does good job of predicting this data point because the task is well specified, while the first does not because we are still uncertain about the task. That is, the first model assigns high probability to many values of f where test is unlikely.
Depending on the discrepancy we choose, we may or may not be able to distinguish between the two scenarios and reject the correct model. For example, if we use
21

---

Published as a conference paper at ICLR 2025
g0 x²)) x= ==
Σ ALI
logp(2t, Us lx"),
log p(2,, y/ z²) is the same for both models and will have identical p-values, which will indicate that both models are suitable- an undesirable outcome.
On the other hand, if we use
g(x,))==-
E log f),
the p-values will be quite different. For the first model, many values of f will fall far away from gest when computing
Pppe == / 1 /
2 g(a ()) I dP( x²))
As result, g(ztot f) will be much lower than g(æ, f), and the PPC will be quite low. However, in the second model, this will not happen, as all values of f will be sampled around 0, and gest a reasonable observation for normal distribution centered Therefore, using the f-dependent PPC provides the desired behavior because it informs when the model confused about the task. Although we have used an example for illustrative purposes, the same behavior holds generally.
As 0 with standard deviation 1.
H GEMMA-2 9B RESULTS
Figure 18b shows that the in-capability vs. out-of-capability distinction is also sensible for the Gemma-2 9B model. So we conduct the same analysis for Gemma-2 9B that we did for Llama-2 7B in Section 7.
Medical
SST2
Medical QP
Ertor Rale
Error
examples
examples
examples
examples
(a) Llama 2 7B
(b) Gemma
Figure 18: Natural language in-capability out-of-capability tasks.
Figure 19a shows valuue under the Gemma-2 9B model as function of the ICL dataset x" size n (context length). We a clear separation between the estimated generative predictive p-values Pgp for the in-capacity SST2 data (solid lines) and the -ftl--cccccccyy MQP dataset (dashed lines), but only for the NLML discrepency. The separation is robust across different ICL dataset sizes.
Figure 19b plots the FPR for the capability detector defined by Pape the with NLML discrepancy. We do not the same stability of the FPR across ICI. dataset size that model. Instead the FPR decreases with increasing n for all significance level a. Figure 19c plots the FPR for the capability detector defined by Pape the with NLL discrepancy. We see that the false positive rate is high for all values. These findings are reflected in the Precision curves on the left hand side of Figure 20. We again see in the Recall curve that the NLL discrepancy leads a more sensitive predictor than the NLML discrepancy. The F1 and Accuracy curves show that the NLML based v-value leads much more effective predictor for Gemma-2 9B.
for the Ilama-2 7B
22

---

Published as a conference paper at ICLR 2025
Blckaano DeSSet 607
B2/?7
Polse Postive Race
False Posttive Flare
(a) Generative predictive p-values
(b) False Positive Rate (NLML)
(c) False Positive Rate (NLL)
Natural language task with Gemma-2 9B. The generative predictive p-value against dataset size n and it's relationship to the false positive rate. The first figure shows the generative predictive p-values, and the second and third figures show the false positive rate with the NLML and NLL discrepancy functions, respectively.
Figure 19:
0slx.x')
Restdoo
Accurcyy
Rocell
0.4
Figure 20: Natural language model suitability detection ablation. Precision, recall, F1, and accuracy metrics vs. number of in-context examples. SST2 ICL datasets are taken to be in-capability for Gemma-2 9b. MQP ICL datasets taken be out-of-capability for Gemma-2 9b.
23


Reviews:
[{"review_id": "SByG8HVBaH", "reviewer": "Reviewer_cuyE", "summary": "The current paper aims to quantify whether a conditional generative model (e.g., an LLM) can solve an in-context learning problem. To achieve this goal, authors relate the problem to that Bayesian model criticism, i.e., how decent a model are we currently using for solving the task at hand. Since the hypothesis used by, e.g., an LLM to solve an ICL task is bound to be unclear, the paper relies on use of the posterior predictive distribution for achieving its goal. Specifically, the authors use posterior predictive checks to perform model criticism. Results show proposed tests correlate with model abilities to solve an ICL task.", "questions": "See limitations.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "What I enjoyed the most about this paper is its really nice, pedagogical writing: even though no particularly new concepts were proposed in the technical sections, the slowly developing pace to contextualize ICL within a Bayesian framework made for a really enjoyable read. This did however take away a lot of space, leading to the experimental section becoming rather sparse on discussion.", "weaknesses": "- My main apprehension is that the targeted problem of \"what ICL problem can a generative model solve\" wasn't sufficiently clarified, leading to confusion in regards to how the results should be interpreted. For example, if the goal is to judge whether my model can solve a task in-context, can I not just evaluate on said task via a benchmark and get a performance estimate (e.g., accuracy)? What else do tests like posterior predictive check offer? In fact, results in the paper (e.g., see Figure 6) show that proposed metrics and accuracy follow an almost one-to-one mapping with respect to each other. Thus, I could have just accuracy as an estimate of how well the model performs the task (if at all). To address this comment, I would request authors to clarify what exactly is a good standard for an answer to the question what ICL problem can a generative model solve. If it's to make a predictive test, i.e., something I can use to preemptively say my model is incapable of performing some task, then I think the currently proposed measures don't fall under that definition.\n\n- At Line 231, authors say when the discrepancy of the data generated under the model is greater than that of the holdout data, then we can be confident that the model explains the holdout data well. This seems unintuitive: if the model generates the data, wouldn't it have a relatively high likelihood it, especially compared to holdout data that it did not generate?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TCnpBiqzRs", "reviewer": "Reviewer_2r64", "summary": "The authors consider the problem of determining whether a CGM can solve an ICL task. Taking a Bayesian view of ICL, the authors use Bayesian model criticism techniques (e.g., PPCs) to tackle this problem. The primary technical challenge is that CGMs don’t expose the posterior and likelihood which are required for certain types of discrepancy functions. The authors then introduce a practical estimator for the posterior predictive p-value that involves repeated ancestral sampling from the predictive distribution of the CGM and show this is equivalent (asymptotically) to posterior predictive p-value. The authors evaluate their method of detecting whether LLMs can solve ICL tasks in synthetic and real-world settings.", "questions": "Some minor comments/questions \n\nI’m not very familiar with the theory underlying PPCs, but I don’t think the posterior predictive p-values always share the theoretical properties of frequentist p-values (e.g., uniformly distributed under the null except maybe when the test statistic is ancillary). Is the same true for martingale p-values and does this have any practical implications? \n\nThe notation |(z_i, y_i)| was a bit confusing to me. I think this refers to the number of tokens comprising the pair of query and response? \n\nIn the abstract, “is equivalent sampling” -> “is equivalent to sampling\".", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "I think the framework is interesting. There are lots of papers that view ICL through a Bayesian lens, but this paper is pretty unique in introducing and leveraging this perspective for model criticism. I think the theoretical formulation of this problem through the lens of Bayesian model criticism helped me better understand ICL. \n\nThe theoretical results/perspective motivate a fairly intuitive, practical, and general estimator: drawing samples from the CGM and using those samples to form a null distribution over some discrepancy function and then computing a p-value using that empirical null distribution. It’s nice that this intuitive approach emerges naturally from a Bayesian perspective, although I'm curious if people have introduced a similar method before but motivated in a more ad-hoc way. \n\nThe writing was generally clear, and there was enough background to help me understand the technical details.", "weaknesses": "My biggest concern is: could the authors elaborate more on what they think are compelling, practical use cases? I think right now, the empirical evaluation doesn’t quite do this line of work justice; I’ll elaborate more below, but that being said, I’m open to updating/revising my assessment, given the author’s response. \n \nThe method is presented as a way to estimate when a CGM can solve an ICL task. Can’t you often just directly evaluate this, though? For example, you could sample from a CGM and ask a domain expert or write some program to evaluate the responses. For example, in Experiment 3, you actually have ground truth labels for the queries, right? You already know that Llama-2 7B can’t solve the MQP task, so what additional value does this method provide? \n\nI think the value of this method would be more apparent in settings (1) where ground truth labels are very hard to source and (2) there’s ambiguity about whether a given ICL task is “in-capability” or “out-of-capability.” Adding some additional evaluation on those sorts of tasks would strengthen the submission.  \n\nAnother question I have is: to what extent do we need discrepancy functions that rely on having the model likelihood and model posterior explicitly? If you choose g to be the log marginal model likelihood (as I think you do in some of the experiments), you can estimate the posterior predictive p-value exactly right (I think this is the “Lite” generative prediction p-value algorithm)? I guess it’s not totally clear to me that the NLL method is reliably better. I don't think this significantly diminishes the importance of developing a practical estimator for situations where you need access to the likelihood, and this probably depends on the task, but I would appreciate a more thorough discussion of how to choose a good g(). \n\nIn practice, the martingale p-value incurs estimation error. I would've liked to see some synthetic experiments that analyze this empirically. For example, how large does N need to be for the approximation to be good? I suspect there's a fairly simple experiment here where you consider some model where the likelihood and posterior are available. \n\nThere are a few places where the writing could be improved. For example, when you define the generative process involving f, it might help to give some concrete examples of what that actually means in a practical ICL task. Also, two different discrepancy functions were introduced in Section 5, but you only need the ancestral sampling-based estimator (introduced in Section 5) when you use the second discrepancy function. I think I was a bit confused when I read Section 6 after Section 6.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xXkzQULja1", "reviewer": "Reviewer_SYeb", "summary": "This paper studies how the tools and terminology from Bayesian modelling can be applied to in-context learning with LLMs. \n\nThe paper has ten pages. \n\nThe first six pages introduce a lot of terminology from Bayesian modelling and ask how this is connected with the auto-regressive next token objective from the LLM. The authors make plausible that the terminology from Bayesian modelling offers a useful perspective on LLM in-context learning and derive a Theorem that shows that \"the martingale predictive p-value is equal to the posterior predictive p-value.\"\n\nThe last four pages of the paper centre around a simple empirical application. The authors evaluate Llama-2 7B and Gemma 9B on a synthetic regression task, as well as on SST2 and MedicalQP. They find that the small models handle SST2 well but perform no better than random on MedicalQP.\n\nThe paper is overall well-written and sound. However, its research contribution is not significant enough for ICLR.", "questions": "Why are there no references in the first two paragraphs of the paper? In general, I find many points in this paper where references are required. \n\nFigure 1: What is the meaning of the different model responses? Did you sample? If yes, at what temperature?\n\nWhy did you choose SST-2 and MQP? Why Llama-2 7B model?\n\nWhy not consider a bigger model (e.g. Llama-3 405B) and see if it can perform better than random on MQP?\n\n\"ancestral sampling\": At the end of the day, is this just a fancy term to describe how you are doing standard sampling of the next token from the language model? You are introducing a lot of terminology in this paper, and it is not always clear whether this is required (also: \"explanation\", \"library\")\n\nAfter discussion period:\n========================\n\nAfter the discussion with the authors, I have decided to increase my score to 6.\n\nThe generative predictive p-value is an interesting concept. The authors believe that it has the potential to improve upon existing techniques, and I find their arguments broadly plausible. This concept might be of interest to the ICLR community. \n\nReason for why not higher score: \n\nEven though the authors provided additional experiments as part of the rebuttal, this paper's experimental section is still a weak spot. \n\nConcretely, I don't think that the current version of the experiments makes a sufficiently strong case for the usefulness of the generative predictive p-value, in the way in which the authors claimed that it can be useful during the rebuttal. The reader who is not genuinely interested in the Bayesian motivation and theory behind the generative predictive p-values is not sufficiently convinced by the presented experiments. The particular experiments in this paper are also not connected to the recent literature on in-context learning (at least, I don't see a strong connection). For example, there are, at this point, already several different papers that study in-context learning with LLMs and tabular data, and it is unclear how the experiments in this paper relate to the overall results in the literature. \n\nIt is a minor point, but some aspects of the experimental section are also not particularly well done, such as the descriptions of the figures.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper's topic, how tools from Bayesian analysis can be useful to understand in-context learning with LLMs, is interesting.\n\nThe paper is well-written in the sense that its language and structure are clear. I also like the expressive section headings. \n\nI did not check the details of the proofs, but both the theoretical arguments in the paper and the experimental results make sense.", "weaknesses": "While the topic of the paper is interesting and the paper is well-written, neither the theoretical nor empirical results in this paper are strong enough to warrant acceptance at ICLR.\n\nThe insight that there are connections between in-context learning and Bayesian methods is not novel (https://arxiv.org/pdf/2111.02080). While the first 6 pages of this paper offer a nice methodological discussion, it remains unclear what the research contribution is (apart from Theorem 2, which seems to be a relatively straightforward result). \n\nThe experimental analysis is limited to a synthetic regression task and two real-world datasets. The performance difference between SST2 and MQP for the small models is so striking that I do not believe we would need any of the Bayesian tools offered in this paper to see this.  \n\nGiven the motivation in the first paragraph of the paper. I think think the Medprompt paper is a missing reference: https://arxiv.org/abs/2311.16452", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)