AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
Authors: Alexander Capstick
Rahul Krishnan
Payam Barnaghi
Venue: ICML 2025 poster (2025)
Keywords: LLMs for healthcare
Prior elicitation
LLM-elicited priors
Primary area: applications->health_medicine
TL;DR: We use LLMs to elicit prior distributions for linear predictive models, which reduce the number of examples required to build a predictive model. Further we find they are more reliable than using in-context learning on numerical predictive tasks.

Abstract: Large language models (LLMs) acquire a breadth of information across various domains. However, their computational complexity, cost, and lack of transparency often hinder their direct application for predictive tasks where privacy and interpretability are paramount. In fields such as healthcare, biology, and finance, specialised and interpretable linear models still hold considerable value. In such domains, labelled data may be scarce or expensive to obtain. Well-specified prior distributions over model parameters can reduce the sample complexity of learning through Bayesian inference; however, eliciting expert priors can be time-consuming. We therefore introduce AutoElicit to extract knowledge from LLMs and construct priors for predictive models. We show these priors are informative and can be refined using natural language. We perform a careful study contrasting AutoElicit with in-context learning and demonstrate how to perform model selection between the two methods. We find that AutoElicit yields priors that can substantially reduce error over uninformative priors, using fewer labels, and consistently outperform in-context learning. We show that AutoElicit saves over 6 months of labelling effort when building a new predictive model for urinary tract infections from sensor recordings of people living with dementia.

Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
AutoElicit:
Alexander Capstick" 12 Rahul G. Krishnan 34
Payam Barnaghi 125
Abstract
1. Introduction
Labelling data in medicine can be a costly and time- consuming undertaking (Ghassemi et al., 2020; Cooper et al., 2023). We are particularly interested in the develop- ment of automated tools to assist with healthcare monitoring for people living with dementia. In our study, participant data recorded through passive monitoring devices. AI- though input features are continuously collected at scale, the process of labelling adverse health conditions, such urinary tract infection (UTI), can be challenging. Ex- pert knowledge of task, as privileged information (Vapnik et al., 2015), can provide prior distributions of model param- eters, which often reduces the number of samples needed to learn performant models while also enabling more accurate estimates of predictive uncertainty. However, this expert knowledge about model parameters can be difficult to obtain due the challenges of performing expert prior elicitation, particularly in low-resource settings (Mikkola et al., 2024).
Large language models (LLMs) acquire breadth of information across various domains. However, their computational complexity, cost, and lack of transparency often hinder their direct applica- tion for predictive tasks where privacy and in- terpretability are paramount. In fields such as healthcare, biology, and finance, specialised and interpretable linear models still hold consider- able value. In such domains, labelled data may be scarce expensive obtain. Well-specified prior distributions over model parameters can re- duce the sample complexity of learning through Bayesian inference; however, eliciting expert pri- ors can be time-consuming. We therefore intro- duce AutoElicit to extract knowledge from LLMs and construct priors for predictive models. We show these priors informative and can be re- fined using natural language. We perform a care- ful study contrasting AutoElicit with in-context learning and demonstrate how to perform model selection between the two methods. We find that AutoElicit yields priors that can substantially re- duce error uninformative priors, using fewer labels, and consistently outperform in-context learning. We show that AutoBlicit saves over 6 months of labelling effort when building a new predictive model for urinary tract infections from sensor recordings people living with dementia.
Linear models are widely used in real-world applications, especially in healthcare and as part of our wider study on dementia care, due to their interpretability. robustness, and clinical familiarity (Capstick al., 2024; Tennant al., 2024; Ramírez Medina et al., 2025). Many studies do not use prior distributions their models, possibly due the challenges associated with elicitation, but would likely ben- efit greatly from them.
Due to their diverse training data and rate of development (Naveed et al., 2023), large language models (LLMs) offer a new opportunity to fully or partially automate the prior elici- tation process, especially given recent advances in reasoning methods (Huang & Chang, 2023). By prompting an LLM to approximate specific form of expertise in a given field, it is possible to extract insights about a predictive model's prior based on the LLM's 's training data. This motivates the construction
Department of Brain Sciences, Imperial College London PUK Dementia Research Institute, Care Research and Tech- nology Centre 'Vector Institute 'Department of Computer Sci- ence, University of Toronto Data Research, Innovation and Vir- tual Environments (DRIVE) Unit, The Great Ormond Street Hospital. Correspondence to: Alexander Capstick <alexan- Srcapstick19@innpe der.capstick1 19 wwieeerialla........
probabilistic framework to extract prior task knowledge from language models in the form of dis- tributions over the parameters of linear predictive model. AutoElicit combines developments in LLMs with trusted and robust linear predictive models, that are already widely understood and utilised in real-world applications.
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
In-context learning has been studied as way to use LLMs

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
Task Deserptions
Priors
3.70
('oe sen prodiet TAREET beaad on stasdard deslation for Geueetion and for e lireur scced...]
The LLM outputs Gaussian prior for each task description and feature, which at use to build nixture model.
Gata points ane used to update the prior to prosuce the postertor with Mante carlo nathods.
Culculate posterior
Py
p(OD)) Æ p(D00) p(
base task description is rephiased Lt tires to produce distribution ever raluvant Lask deseriptions.
Trepdick yobr
Make prodictions
2004 2133
566 riod
Figure 1. AutoElicit: Prior elicitation using a language model. The left-hand figure demonstrates the process. whilst the right-hand figure illustrates the benefits of using AutoElicit, achieving the same peak accuracy after Rs 15 labels, 220 days carlier in the study.
for predictive tasks (Dong et al., 2024). Here, L.MSs are given a task description and demonstration examples and asked to make predictions on unseen data directly (Brown et al., 2020). There are several reasons why this prompted prediction may not suffice. First, LLMs can incur a large computational cost to run, rendering their use in high- frequency or low- owwrrsource settings unlikely. Second, ap- plications may require arithmetic operations within models, an area where LLMs continue to struggle (Spithourakis & Riedel, 2018). Third, L.LMs, unless explicitly trained on a specific predictive problem, lack transparency and may not perform Bayesian inference (Min et al., 2022), which can lead to unreliable posterior updates and predictions.
build better linear models, with fewer samples. Open-source code to reproduce our work is described in Appendix A.
2. Related Work
Garthwaite et al. (2005) and O' Hagan et al. (2006) discuss eliciting prior distributions from human experts using standardised procedure: Experts are trained in probabili- ties to provide judgements of a task, which can be used to fit probability distributions. Later, Oakley & O'Hagan (2019) provide software and an e-learning course, which trains human experts in the knowledge required to describe informative priors. However, this can be challenging, costly, or infeasible in low-resource settings (Mikkola et al., 2024).
Our work makes the following contributions:
In parallel, research shows that LLMs encode varied do- main knowledge, which could be extracted for predictive tasks, possibly in discussions with experts (Gao et al., 2024), Recent works have used LLMs for prior elicitation in mul- tiple ways. In data pre-processing. Choi et al. (2022) and Hollmann et al. (2024) show that LLMs can perform fea- ture selection and engineering respectively, whilst Zhang et al. (2023) and Zhang et al. (2024) use LLMs to specify modelling pipeline given a task description. This suggests language models might also be able to provide informa- tion over the parameters of predictive models directly. In this direction, Gouk Gao (2024) use L.LMs to generate synthetic data from public datasets, which define priors on linear models through separate likelihood term. However, they do not consider that these datasets might have appeared in the LLM's training (Appendix A.8 and A.9.5), making it challenging to evaluate. Related, Selby et al. (2025) evalu- ate L.MMs for estimating correlations between features and targets, and Zhu & Griffiths (2024) use them to clicit priors for causal learning, proportion estimation, and everyday quantities. However, this knowledge is not used for predic- tive models. Further related work is discussed in Appendix 1. In contrast, we provide framework for using LLMs to elicit priors over the parameters of linear models directly, which we evaluate on public and private predictive tasks.
We introduce AutoElicit, an algorithm to generate LLM-elicited priors for linear models. We find using L.LMs for expert prior elicitation (Figure 1) provides higher posterior accuracy and lower mean squared error than uninformative priors various numerical predic- tive tasks, improving models with fewer examples.
To compare prior elicitation with in-context learning (ICL), we develop an approach to extract the implicit priors used in ICL,, which we find to be less informa- tive. To further understand the differences, we study whether an LLM performs Bayesian inference during ICL. We extract the in-context posterior distribution and measure its divergence from posterior Monte Carlo samples of the extracted prior. We find that the lan- guage model inconsistently approximates Bayesian in- ference on real-world datasets.
We study when ICL better than AutoElicit and show how to use the Bayes factor to guide model selec- tion, finding that LLM-elicited priors are the preferred choice for all tasks tested. When predicting UTIs in our study on dementia care, we find that AutoElicit priors result in performant accuracy over six months earlier than with uninformative priors (Figure 1).
com/ alexcapstick/ 1 eimeeliiedd-proors
Our results provide insight on when and how to use LLMS to
2

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
LLMs can also be used produce posterior predictive dis- tributions. In-context learning is active research area, mainly focused on natural language tasks (Dong et al., 2024), with many works aiming to explain the underlying mechanisms. Relevant to us, Requeima et al. (2024) prompt LLMs with task descriptions and data to generate posterior predictive distributions for regression tasks. Additionally, Hollmann et al. (2022; 2025) train a transformer to solve synthetically generated tasks from prior tabular datasets and use in-context learning at prediction time. However, these methods sample LLMs for every prediction, which is aesource-intensive and potentially unreliable. Additional related work is given in Appendix A.2.2. In our work, we propose a strategy to approximate the in-context prior and posterior distributions when imitating linear model, which we compare LLM-elicited priors. Using these, we em- pirically evaluate whether in-context learning approximates Bayesian inference before using model selection.
Model weights are explicitly understood in their effect on predictions, as they multiply directly with features, making them ideal to use with prior elicitation in healthcare.
3.3. In-context Learning
In-context learning uses the knowledge that LLMs acquire during training to make predictions on unseen data. Here, LLMs attempt to learn a function for predicting unseen data based on a task description and demonstration examples.
Let the space of all predictive tasks be T, a single classifica- tion or regression task be T E T, and the space of possible LLM prompts be I. Given LLM M and K task descrip- tions I& (T] Pr Pr(T), (T), prediction of á on features H using in-context learning with training data D is:
K
(M,T(Ul = PM.r( D, (T)).T((
R=1
LLMs have been evaluated for use in healthcare (Thirunavukarasu et al., 2023), demonstrating a broad knowledge of clinical problems and strong performance on medical questions, even without fine-tuning (Nori al., 2023; Singhal et al., 2023). Despite this, their direct for decision making, through in-context learning, is not recom- mended as they sometimes produce inaccurate information. However, linear models have found wide and trusted uses in clinical applications (Ramírez Medina et al., 2025; Tennant et al.. 2024), since their model weights have intuitive de- scriptions. This motivates us to combine LLMs, which can reason about clinical problems and risk factors, with linear models, which are interpretable and familiar to clinicians.
Although the mechanisms of in-context learning un- der active research, we denote its implicit parameters as s, implied by task description and dataset:
PM.T (ylā, K
/ 4
Σ
PMr ()M.. ())M. (Ix) (I)
Here yp is itself described by a prior distribution ( and posterior distribution (M, Ik). & is latent, since in-context learning only returns ÿ ~ (ulr D, r).
3. Background
4. Methods
3.1. Prior Elicitation with Experts
4.1. AutoElicit: Using Language Models to Elicit Priors
Bayesian inference incorporates knowledge about a predic- tive task before data is observed (Murphy, 2022). A prior p(0) is placed on a predictive model, which holds intuitions about the parameters 0. Well-specified priors allow us to estimate predictive uncertainty through the posterior pre- dictive distribution and reduce the number of data points needed to construct comparably accurate predictive models.
We aim to make expert prior elicitation more accessible by using the knowledge of LLMs in robust and clinically trusted predictive models. This motivates us to develop a framework for eliciting general prior distributions for linear models from L.Mss, AutoElicit (Figure 1), which often have both technical knowledge of probability distributions and domain knowledge of a task. Our goal is to improve model performance with less data compared to an uninformative prior, whilst allowing model parameters to be interpretable.
Prior elicitation constructs prior distributions on model pa- rameters, typically through expert judgement (Oakley & O' O'Haann 2019; Gosling, 2018). This can be resource- intensive and requires access to one more domain experts who have been trained in prior elicitation (Mikkola et al., 2024).
Given an LLM M and task T, we obtain a single Gaussian prior for each feature, for each task description I by asking the LLM to provide its best guess of the mean and stan- dard deviation: (k)) PMrT(H, af TT). Here, we use K = 100 task descriptions, produced by asking an LLM to rephrase one numan--rittee "system" and "user" role 10 times and taking their product (Appendix A.7.3).
3.2. Linear Models
Because of their simplicity, linear models have interpretable parameter weights, making them trusted in clinical contexts.
Taking a mixture over the task descriptions, we construct a
3

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
prior PM, (0) over linear model parameters 0:
construct the sample distribution (Murphy, 2022):
PMLE ~ MLE
(M) = Σ (0
Where (X))X, refers to the MLE of the pa- rameters & given input X, in-context predictions (X), and task description T. This provides a sample of the esti- mated prior on yp for the domain of X. Further, additionally providing training data DD returns posterior samples:
where (Mis σk) ~ PM. (x))) and my ~ Dir(1) (2)
Mixtures of Gaussians approximate any distribution with enough components, allowing us to fully capture the LLM's prior belief. We also expect that L.MS have learnt how to parametrise Gaussian distributions, although our framework does allow for other mixtures. In our work, this mixture has 100 components, using as many task descriptions, and in Appendix 12 we find diminishing differences in the LLM- elicited priors as we increase the number of descriptions.
ML ~ (f D,
(4)
In our experiments, for each (FMIF. we sample 25 data points from X ~ u[- 5]. five times for each of the 100 task descriptions. As the real data is normalised, these samples of X will cover the domain of fo with a high probability. Appendix A.3 provides more details on the process.
This PM.r(0) (0) is the prior that we use for our linear models and is an alternative to one elicited from experts. However, additionally including expert information in the prompts can provide yet more informative priors: PMrT (0) E). This way, AutoElicit also supplements a typical prior elicitation process but only requires experts provide knowledge natural language, likely needing less training and cost. This approach enables the use of an LLM's knowledge in new environments, with more controls and transparency.
When specifying linear f.p we calculate the MLE in closed form as we have y - yo + 1--p xz, which is either a regression label or the logit of a class. Therefore, large approximation errors will be due to the LLM not using linear fip.
By repeated sampling, we form distributions that estimate pM,)(p) and (p))) These are over the hidden pa- rameters of the in-context model and give us new visibility into the LLM's predictive process, assuming the model class can be specified in prompt (Appendix A.4). Using these, we can test whether LLM is using the elicited prior in- context and if it is approximating Bayesian inference.
The natural comparison for AutoElicit priors is an uninfor- mative prior. However, since we are using an LLM's prior knowledge and training data make new predictions, this is also an alternative to in-context learning. The difference being that prior elicitation uses a marginal distribution and Bayesian inference, which address some of the shortfalls of in-context learning on numerical predictive tasks.
Is THE LLM USING THE ELICITED PRIOR IN-CONTEXT?
To quantify the difference between the in-context prior (pM.I (f)) and the one we clicit (#), we will use the energy statistic (Székely & Rizzo, 2013). This takes values E [0, 1] with 0 indicating that the two distributions are equal (Appendix A.5) and evaluates how consistently the LLM uses its background knowledge.
4.2. In-context Learning
To compare in-context learning and AutoElicit, we must consider: (1) Does the LLM use the same elicited prior in-context? and (2) Does in-context learning approximate Bayesian inference? To answer these, we extract the prior and posterior distributions of the in-context model.
THE IN-CONTEXT POSTERIOR
We also test whether the LLM is approximating Bayesian in- ference during in-context learning. We use the approximated in-context prior PM (() and Monte Carlo methods (Hast- ings, 1970; Hoffman et al., 2014) (Appendix 15) to sam- ple posterior, and compare it to the extracted in-context posterior PM. D)) The energy statistic quantifies the difference in these distributions. If they differ, it suggests the LLM is not performing Bayesian inference in-context and is not acting predictably after observing data from D.
EXTRACTING THE IN-CONTEXT PRIOR AND POSTERIOR
We can write the implicit in in-context model as: fo X + Y where f represents the class of the predictive model, which we specify in the prompt, & is the random variable asso- ciated with its parameters, and X and Y correspond to its domain and range. Based on the model class ff corresponds to, y E will either be a predicted target or its probability.
In-context learning returns y given x. Since we do not have access to & directly, we propose to use PM,r(u| sp) and maximum likelihood estimation (MLE) to generate a
4.3. The Bayes Factor for Model Selection
If the LLM is approximating Bayesian inference in-context, we use the Bayes factor for model selection between in- context learning and AutoElicit priors. This calculates the
sampling distribution for (p) and (pD))
To do this, given a language model M and task T, we

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
marginal likelihood ratio of two statistical models to deter- mine under which the data was likely generated (Appendix A.6), defining a model selection strategy given any dataset.
toElicit priors provide substantially better accuracy and mean squared error (MSE) than an uninformative prior, and at least one of the LLMs provides the greatest performance for all training sizes and tasks. The complete accuracy distributions and a detailed analysis is given in Appendix A.9; Appendix A. 10 describes the associated cost; and A. 12 studies how the number of descriptions affects the priors.
For AutoElicit (method oo), we approximate the marginal likelihood by sampling the prior predictive log- log-likelihood over D. We sample 500 predictions for 25 data points to get samples of PM.r(Dlaa)) For in-context learning (method a1), we approximate PM,T(P/a.j as the LLM provides predictions on features directly. We prompt the LLM with 100 task descriptions and ask it to predict on 25 data points from the dataset. This is repeated over 5 splits of the datasets. Further details are given in Appendix A.7.5.
When eliciting priors for y = 2x1 22 + 13. we spec- ify the equation in the prompt to test the LLMs' ability to provide priors given complete descriptions (incomplete de- scriptions are studied in Appendix 1. In Figure 2, the LLM-elicited priors provide an order of magnitude lower test MSE compared to an uninformative prior after 5 ob- served examples and an equivalent MSE after 20, as the task was simple enough to be learnt with only a few examples. GPT-3.5-turbo provided the best prior as it returned sharp distributions (Appendix A.9.2), reflecting the true parameter values. This shows LLMs can combine their knowledge of probability with task descriptions to form useful priors.
5. Results
5.1. Experimental Setting
We present results one synthetic and five clinical datasets, covering classification and regression (Appendix A.7.1).
The synthetic data set contains 3 features from X ~ (0 1) and targets calculated using y 2x 22 + I3 + e, with e ~ (0, 0.05). When describing this task to the LLM, we fully specify the relationship between the features and targets to control for the completeness of the descriptions.
For Heart Disease, uninformative and GPT-3.5- turbo elicited priors produced the same mean posterior accuracy, as this elicited prior contains many standard Normal distri- butions (Figure 18). This is reassuring, as GPT-3.5-turbo responded with an uninformative prior when unsure of its knowledge. However, GPT-4-turbo and P1ppSeek-R13322 Q4, returned considerably better priors. The appropriate prior can be chosen with the Bayes factor (Appendix 1717).
We test four publicly available datasets. The Heart Disease dataset (Detrano et al., 1989): predicting a heart disease diagnosis from physiological features; the Diabetes dataset (Efron et al., 2004): predicting disease progression one year after baseline; the Hypothyroid dataset (Quinlan, 1986; Tur- ney, 1994): predicting hypothyroidism based on biomarkers; and the Breast Cancer dataset (Wolberg et al., 1995): pre- dicting the diagnosis of a breast mass.
Of particular note are the results on the UTI task, a private dataset where all AutoElicit priors provide greater accuracy over the uninformative prior for all training sizes. Figure shows these results considered alongside label collection time, illustrating that we achieve greater accuracy earlier in the study, accelerating model development and validation. This is discussed further in Appendix A.9.3.
We also evaluate AutoElicit on private dataset not in the LLM's training, collected as part of our study on Dementia care. This contains 10 features of daily in-home activity and physiology data, and clinically validated labels of positive or negative urinary tract infection (UTI), based on the analysis of urine sample by a healthcare professional. This dataset contains 78 people and 371 positive negative days of UTI.
Also of importance are the results on the classification tasks of the priors elicited from DeepSeek-R1-33 R1-32B--222--- 4-bit quantised version of the distilled DeepSeek-RI model. This suggests the reasoning qualities of more recently released LLMs are of significant use for eliciting priors.
To explore if the LLM has memorised this data, we use two tests (Bordt et al., 2024) in Appendix A.8. Appendix A.7.6 explains how to use these methods on new dataset.
In Appendix A.9.5, we also compare our results to those achieved using the approach presented by Gouk Gao (2024). This method uses an LLM to generate feature val- ues before labelling them with zero-shot in-context learning. These generated samples are provided alongside real data during the training of a linear predictive model. Therefore, the method presented in Gouk & Gao (2024) is susceptible to LLM data memorisation, putting it at risk of artificially providing improved posterior results on public datasets. In light of this, Figure 13 shows the results split by those on private and public tasks. Our approach yields improved pos- terior accuracy and mean squared error over this bascline.
All experiments in this section were conducted with Ope- nAl's GPT-3.5-turo and GPT-4-turbo, and DeepSeek-RI- 32B with 4-bit quantisation. All details are given in Ap- pendix A.7, and other LLMs are tested in Appendix 1.
5.2. How Informative are AutoElicit Priors?
We compare the posterior performance of a predictive model using uninformative prior 0 ~ (0000, 1), with those elicited from three LLMs. Figure 2 shows that Au-
5

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
y
Disease
Hypochyooid
Diabetes
Breast Cancer
0.70
Accuzscy
Aceuracy
0.9
lo0 MSE
1.5
Accuacy
0.9
0.65
loga:
0.6
0.8
0.8
0.60
1.0
0.7
10 20
30
40
50
10
20
30
40
50
10
20
30
40
50
10
20
30
40
50
10
200
30
40
50
10
20
30
40
50
# of training exammpees
# of training examaples
# of training examples
# of training examples
# of training excmuples
#
Sraining exauppees
Uninformative prior
GPT-3.6-surb prior
GPT-4-tarbo prior
DerpSeck-11-3 R1-323--4 prior
Figure2 2. Test performance of the posterior distribution for varied training data sizes. The average and 95% confidence interval of the mean posterior accuracy or Mean Squared Error (MSE) of 10 splits of the dataset using AutoElicit priors and an uninformative prior (0 ~ (010, 1)). These are calculated test set of cach dataset, with the green arrow pointing in the direction of metric improvement.
In Appendix A.13, we demonstrate how the priors elicited can be useful for explaining the final predictive model.
knowledge in-context before observing demonstrations. The low MSE on the real-world tasks indicates that the LLM was reliably imitating linear model, allowing to achieve low approximation error. However, the priors are not particularly informative, as they are often centred close to O.
These results illustrate the remarkable effectiveness of using AutoElicit priors, which considerably improve posterior pre- dictions, particularly for smaller training sizes, where label collection might have been challenging. Our framework makes expert prior elicitation more accessible.
The estimated in in-context posteriors in Figure 5 differ from the priors, suggesting the LLM has updated its knowledge based on demonstrations. For the Diabetes and Heart Dis- ease tasks, the LLM mostly continued to produce linear predictions, and so our error increases, but remains low. However, the Breast Cancer and Hypothyroid dataset pro- duces a large approximation error, as the LLM struggles to imitate a linear model. The approximation error on the synthetic task is discussed in Appendix A.4. We do not estimate the in-context posterior for the UTI dataset as it is private, and so we are unable provide demonstrations.
5.3. Can Experts Supplement the Process?
Including expert information in the task description allows AutoElicit priors to combine learnt insights with domain knowledge. In the UTI task description, we now list three features that positively correlate with the label (Capstick et al., 2024): "It is known that the more previous UTIs patient had, the more likely they are to have UTI in the future. Someone with a UTI will also wake up more during the night. Someone with a UTI will have a greater bathroom usage."
In general, better approximate the LLM's in-context prior than posterior. This is likely due to limitations in the LLM's capacity to interpret demonstrations for training, alongside instructions to provide linear predictions, in the same prompt. As the abilities of LLMs improve, we expect these approximation errors to reduce, as they are due the language model's 's inconsistent imitation of a linear model.
Figure 3 shows that this information is incorporated when LLMs provide priors, these three features become more positively predictive of UTL GPT-3.5-turbe had already identified this relationship, shown by the positive median of the priors, which explains the small improvement in poste- rior accuracy this additional information provides. Further, Appendix A. studies the priors elicited as we incremen- tally improve the description of a synthetic task, illustrating that GPT-3.5-turbo produces reliable distributions from text.
This is approximation of the LLM's in-context parameter distribution, giving new visibility into its predictive process.
5.5. Is the Language Model Consistent?
AutoElicit can therefore also be used to supplement typical prior elicitation methods, with the benefit that experts can convey their knowledge as natural language.
We compare these in-context priors with the AutoElicit priors. Table suggests the LLM uses different priors during in-context learning compared to those we elicit, since they differ according to their energy statistic. This is supported by Appendix 14, which presents histograms of the parameter values. Either the distribution we elicit is an unreliable representation of the L.M''s knowledge, or it is unable to use this knowledge when making predictions in-context. As in in-context priors are often centred around 0, we suspect it is the latter, and that prior elicitation more effectively makes use of the LLM's background knowledge.
5.4. The In-context Priors and Posteriors
Figures 4 and 5 show 1-PT-3.5-tubb's 's in-context prior and posterior distributions approximated with maximum likeli- hood estimation (MLE). This allows us to study the implicit parameters used by the L.M to make predictions in-context.
The estimated priors in Figure 4 show variations in their values across the datasets, suggesting the LLM uses its own
6

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
Bathroom frequency
Night-ling time awake Erequency
Number of previous UTIs
Posterior accuracy
0.70
Mediia:: 0.30
Mediam: 0.30
Median: 0.5g
Donsity
Median: 0.25
Median: 0.20
Median: 0.42
Accuray
0.65
9.60
0.55
0
20
40
Paramdter value
Parameser viliee
Paramettr value
# of training exomplees
Uninformative price
GPT-b.5-turbo prior
Expert information
Figure 3. UTI prior parameters with expert information. The histograms in three left-hand plots show the distribution of parameter values for the features that provide expert information for in the task description. In all three cases, we state that the features are positively correlated with UTT risk. In the right-hand plot, we show the posterior accuracy of these distributions for different numbers of observed samples. The average and 95% confidence interval of the mean posterior accuracy over the 10 test splits are shown.
Mig
UTI
Hourt Disuse
Hypothyroid
Diabetos
Breast Cancer
0.15
0.16
0.300
Deasity
E072
0.04
0.00
D000
0.61
0.22
600
1000
010
010
0.01
0000
600
6000
3.00
0.00
0.000
Priot parameter values
Prior parameter values
Prior parameseer values
Prioz paramelerr values
Prioe pacameter values
Prioc parametee values
Doustty
022
10
21 Logit MSE
Prediction
MSE
Logit MSE
Logit MSE
Prediction MSE
Logit MSE
Figure 4. Extraction of GPT-3.-turbb''s in-context prior. The bottom row shows the MSE between the L.M'' in-context predictions and the MLE model's predictions, whilst the top row shows the parameter distribution of these MLE models, with each colour representing a single feature. For ease of visualisation, we exclude values outside of the (2.5%, 97.5%) percentiles and the bias term.
Results of the other datasets and training splits are given in Appendix 1.15. This suggests L.MMellcited priors are more reliable, particularly when access to the posterior dis- tribution is important, such as for uncertainty estimation.
Table 1. Approximated GPT-3.5-turbo in-context prior com- pared to the elicited prior. The energy statistic hetween the LLM's approximate in-context prior and the I.I ..e--iiiite prior.
Lower values indicate that the two distributions are more similar. Breast Cancor
UTl
Heart Discase
Hypothyroid
Diabotes
It is feasible that future LLMs can reliably perform Bayesian inference in-context, requiring a model selection strategy to decide between in-context learning and LLM-elicited priors.
0.39
0.18
0.15
0.38
0,31
5.7. Using the Bayes Factor for Model Selection
5.6. Does In-context Learning use Bayesian Inference?
If in-context learning approximates Bayesian inference, we can calculate the Bayes factor between it and AutoBlicit.
is important that in-context learning performs Bayesian inference, so that new data updates the LLM'' knowledge reliably. We empirically study this by comparing the ap- proximated in-context posterior with the Monte Carlo (MC) posterior samples of the approximated in-context prior.
Figure 7 shows the marginal likelihood distribution for in- context learning, AutoElicit, and an uninformative prior, along with the Bayes factor (where reliably defined). Unlike in-context learning, AutoElicit priors are the better choice over an uninformative prior for all datasets tested. Whether AutoElicit or in-context learning provides better priors de- pends on the task. However, in only two of the five datasets tested, in-context learning reliably produced linear predic- tions, and in only one of those approximated Bayesian inference. Considering this, the cost, and the ease of inter- pretability, AutoElicit is likely the better option.
These two posterior samples are shown in Figure 6, where we focus on those datasets where in-context learning pro- duced linear predictions, providing a low posterior approx- imation error. For the Diabetes task, samples from the approximated in-context posterior and MC sampling on the approximated in-context prior visually agree and produce a low energy statistic, suggesting the LLM is performing Bayesian inference in-context. However, for the Heart Dis- case task, the two posterior samples differ, showing that the L.MM inconsistently applies the Bayes rule in-context.
This model selection strategy is designed to be more relevant as LLMs become more reliable and their costs reduce.
7

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
2xy
Heart Discuse
Hypothyroid
Diabetes
Breast Caneer
6006
300
Density
6.02
6004
00000
0075
506
234
-0.25
0,25
Posterior parameter values Posterior parumeter values Posterior porameter values Pasterior peramacter values Posterior paramatter valtees
Deaslly
81
216
600
90
200
2.00
Predicsion MSF
Logit MSE
Logit MSE
Prodiction MSF
Logit MSE
Figure 5. Extraction of GPT--.5-turbo's in-context posterior. The bottom row shows the MSE between the LLM's in context predictions and the MLE model's predictions, whilst the top row shows the parameter distribution these MLE models, with each colour representing a single feature. For ease of visualisation, we exclude values outside of the (2.5%, 97 97.5%) percentiles and the bias term.
linergy stat:: 0.002
Feature
Feature
Feature
Fenture
Feature
Hearr Discuse
22.5
Deneity
2.0
1.5
1.0
2.5
0.0
0.0 Feature
0.0
0.0
Energg stat:: 01.055
Feature
Feature
Feature
Feature
6
Feuture
Foarure
Dhabetes
Deneity
1-1
0
Value
Value
Value
Value
Value
Value
Value
Value
Value
Value
In-context learning posterior
MCMC in-confext learning prioe
Figure 6. The in-context posterior distribution. We compare the posterior extracted from GPT-3.5-turbo's in-context predictions against the distribution that we sample using Monte Carlo methods on the extracted prior distribution. Both posterior distributions are based on the same 25 data point sample from the corresponding dataset. We draw the histogram separately each dimension, but calculate the energy distance across all features combined. For the diabetes dataset, we have removed the long tails for improved visualisation.
6. Limitations and Future Research
7. Conclusion
This work explores how experts can guide AutoElicit priors with natural language, but studying other ways of incorpo- rating experts into the framework would be interesting. For example, by fine-tuning LLMs on relevant literature.
This work shows that, by asking language model to play the role of domain expert, are able to elicit prior distri- butions for predictive tasks, potentially reducing the cost of data collection and improving the accuracy of our models when we have access to few labelled examples.
As AutoElicit priors are product of their training, they might contain unwanted bias. To avoid this, an approach would to assign all features that are sensitive to unwanted bias, directly or indirectly, with an uninformative prior. Given elicited prior and observed examples, the Bayes factor can then be used to choose between the elicited prior and an uninformative prior. Additionally, experts can steer LLMs away from unwanted bias through task descriptions.
We find the priors elicited from -.PT-3,5-turbo, GPT-4-turbo, and DeepSeek-R RI-11332 1-22B-Q4 achieve better accuracy or mean squared error with fewer data points than an uninformative prior for variety of tasks. This improvement is most visible when few training examples are available, suggesting its use in environments where label collection is challenging.
Adding expert guidance to the task descriptions enables further improvements to predictive accuracy, as the LLM updates elicited distributions as we would expect. AutoElicit can therefore also be used to simplify the typical expert prior elicitation process by only requiring experts provide in- sights in natural language. We find that prior elicitation with AutoElicit is an improved alternative to in-context learning in our setting, as we can inspect priors, confidently perform Bayesian inference, and often achieve reduced error.
Finally, would be interesting to extend our results to neural networks. An immediate option is to build on the literature, asking the LLM to suggest data augmentation techniques for task. effectively reducing the cost of data collection by extending the current dataset. Further, extending our results to non-linear models such as random forests or Gaussian processes would be valuable.
8

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
UTT
Heart Disesee
Hypothyrold Bogn Fartse 0.24
Diabates
Breast Canoer
10
Exgne Parber ads out
1:27
Eugers Factum: 2 0.75
Deusity
0.01
2.5
0.01
0.00
0.0
0.00
189 Likelinood
0.0
0.0
0.5
10 Likelibood
10
Likeliboed
Likellhood
Likelibood
Likellhood
Uninformative peior
In-context Jearning
GPT-335-ttrbo prior
Figure 7. The distribution of the prior marginal likelihood and the Bayes factor for GPT-3.5-turbo. This shows the distribution of prior likelihoods, calculated using 25 data points from each dataset, over 5 splits and 100 samples of prompts. The Bayes factor calculated between prior elicitation and in- coneott learning, with a number greater than favouring prior elicitation. For the regression tasks, exclude values outside of the (2.5%, 97.5% percentiles the histograms and assume the priors form a single Gaussian.
On our private dataset, we find a large and quantifiable performance improvement when predicting cases of UTI. Within our study on dementia care, the resulting model cre- ated with LLM-elicited priors can provide better predictions of adverse health conditions, reliable uncertainty estimates, and remain interpretable and trusted by clinicians.
Canada Research Chair Tier II in Computational Medicine. The funders were not involved the study design, data collection, data analysis, or manuscript writing.
References
Abril-Pla, O., Andreani, V., Carroll, C., Dong, L.., Fon-
nesbeck, C. J., Kochurov, M., Kumar, R., Lao, J., Luh- mann, C. C., et al. PyMC: modern, and comprehensive probabilistic programming framework in Python. PeerJ Computer Science, 9:e1516, September 2023. ISSN 2376-5992. doi: 10.7717/ppeerj-c.. 1516. URL http: / //dx. dol.doi... www.oooggo..........ooooo 7717 http1///eeeeee......0
Impact Statement
This paper presents work that aims to make expert prior elicitation more accessible and suggests new way for lan- guage models to be used in settings where high levels of trust in predictive models are required. We see many poten- tial societal consequences of our work, none of which we feel must be specifically highlighted here.
Akyürek, E., Schuurmans, D., Andreas, J., Ma, T., and Zhou, D. What learning algorithm is in-context learn- ing? Investigations with linear models. In The Eleventh International Conference on Learning Representations, 2022.
Author Contributions
Original idea conception, software, analysis of results, writing, editing, reviewing, funding acquisition. RGK: Original idea conception, supervision, analysis of results, editing, reviewing, funding acquisition. PB: Supervision, analysis of results, reviewing, funding acquisition.
AC:
Andras Janosi, W. S. Heart Disease, 1989. URL https: //archive. ics. uci . edd. datp//t/tassee/4/5
Bordt, S., Nori, H., Vasconcelos, V. R., Nushi, B., and
Caruana, R. Elephants never forget: Memorization and learning of tabular data in large language models. First
Acknowledgments
Conference on Language Modeling, 2024. URL https: // //openreview. net forum? Hoo.
This study is funded by the UK Dementia Research Institute (UKDRI) Care Research and Technology Centre funded by the Medical Research Council (MRC), Alzheimer's 's Re- search UK, and Alzheimer's Society (grant number: UK- DRI-7002). Further funding was provided by the UKRI Engineering and Physical Sciences Research Council (EP- SRC) PROTECT Project (grant number: EP/W031892/1) and the Royal Academy of Engineering, UK. Infrastructure support for this research was provided by the NIHR Imperial Biomedical Research Centre (BRC) and the UKRI Medi- cal Research Council (MRC). Additionally, funding was provided by the Turing scheme, UK, and resources used in preparing this research were provided, in part, by the Province of Ontario, the Government of Canada through CI- FAR, and companies sponsoring the Vector Institute. More- over, RGK is supported by Canada CIFAR AI Chair and
Box, G. F. and Hunter, W. G. A useful method for model- building. Technometrics, 4(3):301-31,, 1962.
Bradbury, J., Frostig, R., Hawkins, P., Johnson, M. J., Leary, C., Maclaurin, D., Necula, G., Paszke, A., VanderPlas, J., Hunderman-Milne S., and Zhang, Q. JAX: Composable transformations of Python+NumPy programs, 2018. URL http: /tt::///iihhbb wwtp///..aaa//aecccccaaa/aooaaaa malaaxxmll11 jaxx
Brown, T., Mann, B., Ryder, N., Subbiah, M., et al. Lan- guage models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pp. 1877-1901. Curran Associates, Inc., 2020.
Cabezas, A., Corenflos, A., Lao. J., Louf, R., Carnec, A., Chaudhari, K., Cohn-Gordon, R., Coullon, J., Deng, W.,

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
Duffield, S., al. BlackJAX: Composable Bayesian in- ference in JAX. arXiv preprint arXiv:2402. 2024.
Efron, B., Hastie, T, Johnstone, I,, and Tibshi- rani, R. Least angle regression. The Annals of Statistics, 32(2):407 499, 2004. doi: 10.1214/ 0090536040000000006 URL 121 1224/0009536040000000000
Capstick, A., Palermo, F., Zakka, K., Fletcher-Lloyd, N., Walsh, C., Cui, T., Kouchaki, s., Jackson, R., Tran, M., Crone, M., et al. Digital remote monitoring for screen- ing and early detection of urinary tract infections. NPJ Digital Medicine, 7(1):11, 2024.
Falck, F., Wang. Z., and Holmes, C. Is in-context learning in large language models Bayesian? a martingale perspec- tive. In Pruceedings of the 4/st International Conference on Machine Learning. ICML'24 MMM..44. JMLR.org. 2024.
Choi, K., Cundy, C., Srivastava, S., and Ermon, S. Lmpriors: Pre-trained language models as task task-specific priors. In NeurlPS 2022 Foundation Models for Decision Making Workshop, 2022.
Gao, S., Fang. A., Huang, Y., Giunchiglia, V., Noori, A., Schwarz, J. R., Ektefaie, Y., Kondic, J., and Zitnik, M. Empowering biomedical discovery with Al agents. Cell, 187(22):6125 (11222)6122-6155 6151,, 2024.
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1-113, 2023. ISSN 1533-7928.
Garthwaite, P. H., Kadane, J. B., and O'Hagan, A. Statisti- cal methods for eliciting probability distributions. Jour- nal of the American Statistical Association, 100(470): 680-701, June 2005. ISSN 1537-274X. doi: 10.1198/ 01621450500000000005 URL http: 10. 10.1196/016214550 /tt.///11111155000011111
Cooper, M., Ji. Z., and Krishnan, R. G. Machine learning in computational histopathology: Challenges and opportuni- ties. Genes, Chromosomes and Cancer, 62(9):540-556, 2023.
Ghassemi, M., Naumann, T., Schulam, P., Beam, A. L., Chen, I. Y., and Ranganath, R. A review of challenges and opportunities in machine learning for health. AMIA Summits on Translational Science Proceedings, 2020:191, 2020.
Dalal, S. and Misra, V. The matrix: A bayesian learning model for LLMs. arXiv preprint -xXx:2402.0317. 2024.
DeepSeek-AN AI, Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q. Ma, S., Wang, P., et al. Deepseek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning, 2025. URL https: //arxiv. orgg abs/ 2501. 12948.
Gosling, J. P. SHELF: The Sheffield elicitation framework. Elicitation: The science and art of structuring judgement, pp. 61-93, 2018.
Gouk, H. and Gao, B. Automated prior elicitation from large language models for bayesian logistic regression. In The 3rd International Conference on Automated Machine Learning, 2024.
Detramo, R. C., Jánosi, A., Steinbrunn, W., Pfisterer, M. E., Schmid, J.-J., Sandhu, S., Guppy, K., Lee, S., and Froelicher, V. International application of a new proba- bility algorithm for the diagnosis of coronary artery dis- ease. The American journal of cardiology, 64 5:304- 10, 1989. URL https: api www.ppp..aaa................p..e www.eeetttcchiieeeeeeeeeeeeeeeee org/ ID:
Harris, C. R., Millman, K. J., der Walt, S. J., Gom- mers, R., Virtanen, P., Cournapeau, D., Wieser, E., et al. Array programming with NumPy. Nature, 585(7825):3573622 September 2020. doi: 10.1038/ s41586- 264-2- URL https: /ttps::/doi.. dod.i..oro........ 1038/ 11///44100 0000:41-5560200024----0
Dong, Q. Li, L., Dai, D., Zheng, C., Ma, J., Li, R., Xia, H., Xu, J.. Wu, Z., Chang, B., Sun, X., Li, L., and Sui, Z. A survey on in-context learning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Lan- guage Processing, pp. 1 1107-1 128, November 2024. doi: URL https://
Hastings, W. K. Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57
10. 2.11866/vlll 1m20024.mmmlmmmmiii..... ac aclannt antaantooolgge wow.hhooooy....o org/ wa1.44..mllppmmm 64/.
(1):97-109, April 1970. ISSN 0006-3444. doi: 10.
1093/biomeu/57.1.97 URL http:/ /ett:://dx.dor /xx.xdii.oooo 10 10.1093/biones / 17.1.97.
Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., Goyal, A., Hartshorn, A., Yang, A., Mitra, A., Sra- vankumar, A., Korenev, A., Hinsvark, A., Rao, A., Zhang, A., et al. The Llama 3 herd of models, 2024. URL https: //aaxiv. wwtpsiiioorr////oooo abs///////// 22077 21783.
Hegselmann, S., Buendia, A., Lang, H., Agrawal, M., et al. Tabllm: Few-shot classification of tabular data with large language models. In Proceedings of The 26th Interna- tional Conference on Artificial Intelligence and Statistics, pp. 5549-5558 PMLR, April 2023.
10

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
Hoffman, M. D., Gelman, A., al. The No-U-Turn sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo. J. Mach. Learn. Res., 15(1):1993-1623 2014.
The past, present, and future. Bayesian Analysis, 19(4): 1129-1161, 2024.
Min, S., Lyu, X., Holtzman, A., Artetxe, M., al. Rethink- ing the role of demonstrations: What makes in-context learning work? In Goldberg, Y., Kozareva, Z., and Zhang, Y. (eds.), Proceedings of the 2022 Conference on Em- pirical Methods in Natural Language Processing, pp. 1048-11064. Association for Computational Linguistics, December 2022. doi: 10.18653/v1/20222.mml main. 759.
Hollmann, N., Müller, S., Eggensperger, K., and Hutter, TabPEN: A transformer that solves small tabular classifi- cation problems in second. In International Conference on Learning Representations, 2022.
Hollmann, N., Müller, S., and Hutter, Large language models for automated data science: Introducing caafe for context-aware automated feature engineering. Advances in Neural Information Processing Systems, 36, 2024.
Mooij. M., Peters, J., Janzing, D., Zscheischler, J., and Schölkopf, B. Distinguishing cause from effect using observational data: methods and benchmarks. Journal of Machine Learning Research, 17(32)11-02, 2016.
Hollmann, N., Müller, S., Purucker, L.,, Krishnakumar, A., Körfer, M., Hoo, S. B., Schirrmeister, R. T., and Hutter, F. Accurate predictions on small data with tabular foundation model. Nature, 637(8045):319 January 2025. ISSN 1476-4687. doi: 10.1038/
Murphy, K. P. Probabilistic machine learning: an introduc- tion. Adaptive computation and machine learning. The MIT Press, Cambridge, Massachusetts London, England, 2022. ISBN 978-0-2622046822-4
111886-024-08328-6 6. URL http: / /dx. oog. 10. 00.00////4155 024 08328-6.
Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., and Mian, A. A comprehensive overview large language models, arXiv preprint 2rX.v:2307.06433, 2023.
Huang, J. and Chang, K. C.-C. Towards reasoning in large language models: A survey. In Rogers, A., Brader,, J., and Okazaki, N. (eds.), Findings of the Association for Computational Linguistics: ACL 2023, pp. 1049-1065, Toronto, Canada, July 2023. Association for Computa- tional Linguistics. doi: 10.18665/11/2223.nnning
Nori, H., King, N., McKinney, M., Carignan, D., and Horvitz, E. Capabilities of GPT-4 medical challenge problems. arXiv preprint arXiv:2303. 13375, 2023.
67. URL https: / /aaaaaaaantoooooo.ooooaa wwwp:tttoooooooooo/22222222. org/2023. Eindings-acl.67
Oakley, J. and O' Hagaan T., 2019. URL https::/// /shelf.. sites. shtesssheffieddcco ac.. uk/.
Jones, A. L. Scaling scaling laws with board games. arXiv preprint arXiv:21044 104.0314. 2021.
OpenAl, Achiam, J., Adler, S., Agarwal, S., et al. GPT-4 technical report. arXiv preprint 2rX11:2020:: March 2024.
Levenshtein, V. Binary codes capable of correcting dele- tions, insertions, and reversals. Proceedings of the Soviet physics doklady, 1966.
O' O'Hagan, A., Buck, C. E., Daneshkhah, A., Eiser, J. R., Garthwaite, P. H., Jenkinson, D. J., Oakley, J. E., and Rakow, T. Uncertain Judgements: Eliciting Experts' Probabilities. Wiley, July 2006. ISBN 9780470033319, doi: 10. 10.10002/470033311. URL http:://tt....// org 10. /1g/10.1 100.10000///03333/
Levenshtein, V.
Binary codes capable of correcting spuri- ous insertions and deletions of ones. Problems of Infor- mation Transmissions, 1(1):8-17 17, 1965.
Li, B. Z., Chen, W. Sharma, P., and Andreas, J. LaMPP: Language models probabilistic priors for perception and action. arXiv preprint arXiv:2302.022 02801, 2023.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cour- napeau, D.,. Brucher, M., Perrot, M., and Duchesnay, E. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825-2830, 2011.
Li, M. Y., Fox, B., and Goodman, N. D. Automated statistical model discovery with language models, In Pro- ceedings of the 41 International Conference on Machine Learning, (4. JMLR.org, 2024.
MetaAl, Introducing meta Llama 3: The most capable openly available LLM to date, 2024. URL https:// ai www..eta...mm www.oo////o... http//llll/o///oo htoplllg//aaaaaeeee httm/mmtaaalllaaa 3/.
Quinlan, R. Thyroid Disease. UCI Machine Learning Repos- itory, 1986. DOI: http:://oi..og/022253225550
Ramírez Medina, C. R., Benitez-Aurioles. J., Jenkins, D. A., and Jani, M. A systematic review of ma- chine learning applications in predicting opioid asso- ciated adverse events. npj Digital Medicine, 8(1),
Mikkola, P., Martin, O. A., Chandramouli, S., Hartmann, M., Abril Pla, O., Thomas, O., Pesonen, H., Corander, J., Vehtari, A., Kaski, S., et al. Prior knowledge elicitation:

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
January 2025. ISSN 2398-6352.
doi: 10.1038/
Tornede, A., Deng, D., Eimer, T., Giovanelli, J., al. Au- toml in the age of large language models: Current chal- lenges, future opportunities and risks. Transactions on Machine Learning Research, June 2023. ISSN 2835- 8856.
441746-024-0131124, URL http::ttt///.. //c:://xxx. doi. .rrg/ 1...1088ss41744 024-01311 01312-4.
Requeima, J., Bronskill, J. F., Choi, D., Turner, R. E., and Duvenaud, D. LLM processes: Numerical pre- dietive distributions conditioned on natural language. In The eiihty-eigtth Annual Conference on Neural In- formation Processing Systems, 2024. URL https: //openreview wwt. Pieimiiii---a HShs
Touvron, H., Lavril, T., Izacard, G., Martinet, X., al. LLaMA: Open and efficient foundation language models. arXiv preprint arXiv:2302. 13971, February 2023.
Tumney, P. D. Cost-sensitive classification: Empirical eval- uation of hybrid genetic decision tree induction algo- rithm. Journal of artificial intelligence research, 2:369- 409, 1994.
Selby, D., Iwashita, Y., Spriestersbach, K., Saad, M., Bap- pert, D., Warrier, A., Mukherjee, S., Kise, K., and Vollmer, S. Had enough of experts? Quantitative knowl- edge retrieval from large language models. Stat, 14(2): e70054, 2025.
Vapnik, V., Izmailov, R., et al. Learning using privileged information: Similarity control and knowledge transfer. J. Mach. Learn. Res., 16(1):2023-2499, 2015.
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S.. Wei, J.. Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., Payne, P. Seneviratne, M., et al. Large language models encode clinical knowledge. Nature, 620(7972): 172-180, July 2023. ISSN 1476-4687, doi: 10.1038/ $415866003--0629122 URL http: //x. doi. org/ 10.1038/s4185
Von Oswald, J., Niklasson, E., Randazzo, E., Sacramento, J., Mordvintsev, A., Zhmoginov, A., and Vladymyrov, M. Transformers learn in-context by gradient descent. International Conference on Machine Learning, pp. 35151-35174, PMLR, 2023.
0209--2.....
Wang. X., Zhu, W., Saxon, M., Steyvers, M., and Wang, W. Y. Large language models are latent variable mod- els: Explaining and finding good demonstrations for in- context learning. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23, Red Hook, NY, USA, 2023. Curran Associates Inc.
Spithourakis, G. and Riedel, S. Numeracy for language models: Evaluating and improving their ability to predict numbers. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 2104-2112 15, 2018
Székely, G. J. and Rizzo, M. L. On the uniqueness of distance covariance. Statisties & Probability Letters, (12):2278-2882, 2012.
Wes McKinney. Data Structures for Statistical Computing in Python. In Stéfan van der Walt and Jamod Millman (eds.), Proceedings of the 9th Python in Science Conference, pp. 56 - 61, 2010. doi: 00000000/Maorr2-9211992222
Székely, G. J. and Rizzo, M. L. Energy statisties: A class of statistics based distances. Journal of statistical planning and inference, 143(8):124 2013.
Wolberg, W., Mangasarian, O., Street, N., and Street, W. Breast Cancer Wisconsin (Diagnostic). UCI Machine Learning Repository, 1995. DOI:
Team, Q Qwen2.5: A party of foundation models, Septem- ber 2024. URL bettp::/qwen/) www.ww.githhu..b...m........ .o/ blog/qwen2.
https///oroog///22442/CDDDD
Xie, S. M., Raghunathan, A., Liang, P., and Ma, T. An explanation of in-context learning implicit bayesian inference. In International Conference on Learning Rep- resentations, 2021.
Tennant, R., Graham, J., Kern, J., Mercer, K., Ansermino, J. M., and Burns, C. M. A scoping review on pediatric sepsis prediction technologies healthcare. npj Digital Medicine, 7(1), December 2024. ISSN 2398-6352. doi: 10. 10.100884414446--44--336---- URL http: //xx//x//////////// dof.. 0tg/10.10330 /0000/////444444 -24111176-------6---- -200---00000-------111---------- 01361-9.
Yang, A., Yang, B., Hui, B., Zheng, B., Yu, B., Zhou, C.,, Li, C., et al. Qwen2 technical report. arXiv preprint arXi::2407.1067), 2024.
Thirmaavukaasuu, A. J., Ting, D. S. J., Elangovan, K., Gutierrez, I.., Tan, T. F., and Ting, D. S. W. Large language models in medicine. Nature Medicine, 29 (8):1930-1940, July 2023. ISSN 1546-170X. doi: 10. 10.10388/41591--02--04444 URL http: / org/10. 0tte:/..1033/411911--1 00010///////11111002222--00
Zhang, L.,, Zhang, Y., Ren, K., D., and Yang, Y. MLCopi- lot: Unleashing the power of large language models in solving machine learning tasks. In Graham, Y. and Purver, M. (eds.), Proceedings of the 18th Conference of the Eu- ropean Chapter of the Association for Computational Linguistics (Volume I: Long Papers), pp. 2011299122222 -9959, St.
12

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
Julian's, Malta, March 2024. Association for Computa-
blackjax (v1.2.3) (Cabezas et al., 2024): For Monte Carlo sampling the in-context prior samples.
tional Linguisties. URL https::/p///tt. httpps///aaaaaaaeee /aclaaaaaaanoooyy....o... org/2024. http:/222.ecccl-1. 1aneclloon..177
jax (v0.4.31) (Bradbury al., 2018): For calculating some metrics and use with blackjax.
Zhang. S., Gong, C., Wu, L., Liu, X., and Zhou, M. AutoML-GPT: Automatic machine learning with GPT, 2023. URL https: petts://aate /ttpss////aii//// www//osg//....oooooo.e//.oooeooe ab3/2305. 02499.
numpy (v1.26 (v1.26.4) (Harris et al., 2020): For manipulation of arrays and computing mathematical equations.
ollama (v0.4.7): For loading the DeepSeek model.
Zhu, J.-Q. and Griffiths, T. L. Eliciting the priors of large language models using iterated in-context learning. arXiv preprint arXiv:2406.01880 2024.
openai (v1.51.2): For communicating with the OpenAl API.
pandas (v2.2.2) (Wes McKinney, 2010): For manipu- lating the datasets and results.
A. Appendix
A.I. Availability of Code
pyme (v5.17.0) (Abril-Pla al., 2023): For Monte Carlo sampling the elicited priors.
We provide the code to generate all figures and results", and functions to use our method on new datasets?, on GitHub.
scikit-learn (v1.5.1) (Pedregosa al., 2011): For load- ing and preprocessing the data.
This open-source code makes simple to perform expert prior elicitation using LLMs:
transformers (v4.46.0): For loading and using Llama models.
A requirements file is provided within the supplementary code, which also includes some other utility packages re- quired to run the scripts and load the data and results.
import the elicitation function from iin_elicite__prios.. i import get pet_immelliciaaiion_oo_.aee
A.2. Further Details on the Related Work
elicit priors for a dataset get_lmmmellcctaaioooffoo..... the language model client # requires restl (messages) client=llm_clien the task descriptions to iterate over
In this section, we outline in more detail the prior work in our area of research.
1. USING LLMS TO ELICIT EXPERT PRIORS
Choi et al. (2022) explores the use of an LLM to perform feature selection, causal discovery, and to modify the reward function in a reinforcement learning task. the first case, they observe that when listing features from two datasets, GPT-3 can correctly identify the features that correspond to single specified predictive task. Additionally, when applied to causal discovery, the language model is often able to predict the directionality of causal relationships in the Tüebingen Cause-Effect Pairs dataset (Mooij et al.. 2016). This inspires us to explore the possibility of using L.MSs to elicit prior parameter distributions over linear models.
user_roles-user_rolees, # the dataset feature names Beature_nams-daaa.feettuu feaasreeettt.eetttt_eaeaaeeaeeee
We provide classes that wrap common L.LMs (Appendix A. A61 whilst making it easy to adapt to new models.
Gouk & Gao (2024) use LLMs to generate synthetic data to elicit priors on public datasets. A logistic regression model is then pre-traine trained using this data, defining prior. How- ever, this work does not discuss the memorisation of public datasets by LLMs which is shown to be substantial (Bordt et al., 2024), supported by the results in Appendix A.8. In Appendix A.9.5, we compare the posterior performance of AutoElicit with the method proposed in Gouk & Gao (2024)
This code implemented in python (v3.11.9), using pip (v24.0). For our experiments, we use the following pack- ages:
11m=e1 iiliiiitnn ted-ieeeeddroooo
hhttps:// hhtttss//githube /ttp//gtttttttt//////////.h.h com/ alxxcapttcck/ autoelicit
13

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
and show that on private data, AutoElicit provides more informative priors.
and are comparable to those produced by domain experts. This demonstrates that LLMs have good understanding of probabilistic modelling, suggesting that they might also be able to provide priors over predictive models directly.
Further, Selby et al. (2025) elicit beta distribution parame- ters to define prior for predictive tasks where results from human experts are available. The priors from four L.LMs were visually compared to those elicited from human experts using the SHELF (Oakley & O' O'Hagan, 2019; Gosling. 2018) method, demonstrating significant differences between ex- perts and LLMs. However, by considering prior predictive distributions, they show that GPT-4 (OpenAl et al., 2024) can provide parameter distributions that allow for zero-shot performance comparable to models trained on tens of data points. These priors are over the targets directly, as with in-context learning, and are not updated using data, our work, we use the LLM to elicit multiple Gaussian distribu- tions, which are combined to create a mixture model for more general distribution over the parameters of a linear predictive model. Our priors can then be updated using observed data. Further, we explore the collaboration of LLMs and human experts to provide more informative prior distributions.
Li et al. (2023) discusses using LLMs to produce prior distri- butions for perception and control tasks. This work focuses on vision and robotics and elicits common- sense priors from LLMs bias perception and action models towards real- istic world states. The authors find that for vision tasks, this improves predictions on rare, out-of-distribution, and structurally novel inputs. Our work aims to extend this by evaluating whether LLMs can also provide expert informa- tion about parameters in predictive models.
A.2.2. IN-CONTEXT LEARNING
Learning from demonstrations in-context has proven be effective for several natural language tasks, demonstrating the surprising ability of language models to perform outside of their intended use (Brown et al., 2020; Dong et al., 2024; Hegselmann et al., 2023; Chowdhery et al., 2023; Touvron et al., 2023). In Brown et al. (2020), the authors study GPT- 3's strong ability on natural language tasks after seeing a few examples, one example, or none at all. This work inspired others to study the extent to which language models can learn from demonstrations given as text alongside task description. Further, Xie et al. (2021) study the mechanics of in-context learning and describes it from a Bayesian Inference perspective. They find that in-context learning succeeds when language models can infer a shared concept across individual examples in a prompt. Min et al. (2022) argue that in-context learning allows the language model to observe the label space, the distribution of input text, and the task format, as they find that swapping the real labels in demonstrations for random labels does not significantly reduce accuracy. Most of the existing work focuses on natural language tasks (Dong et al., 2024), leaving numerical problems less explored and focus of our work.
In addition, Requeima al. (2024) prompt language models with task descriptions and collected data points to generate posterior predictive distributions for regression tasks, which they find outperform Gaussian processes in some tasks. This is done by repeatedly sampling a language model at a new feature value, given the previous input and output obser- vations, and given text description that contains prior in- formation from a human expert. In our work, we explore whether repeatedly sampling language models can also pro- duce general prior distributions for predictive models.
Furthermore, Zhu & Griffiths (2024) describe the use of iterated learning to elicit prior distributions from language models for causal learning, proportion estimation, and pre- dicting everyday quantities. The authors employ Markov chain Monte Carlo method access an LLM's prior knowl- edge by performing successive inference using previous model outputs that tend towards a limiting distribution. In doing this, the authors assume that in-context learning is form of Bayesian inference. However, in their work Zhu & Griffiths (2024) do not consider how prior elicitation from LLMs can be used for predictive models or test whether an LLM is performing Bayesian inference, both of which we study in this work.
In Hollmann et al. (2022), the authors train transformer model to solve synthetically generated classification tasks from a prior over tabular datasets. Given new task and demonstrations, this transformer performs in-context learn- ing to make predictions using inference steps only. Fur- thermore, Akyürek et al. (2022) and Von Oswald et al. (2023) discuss the types of learning algorithms that lan- guage models (and more generally, transformers) can imple- ment in-context. Focusing on linear models, Akyürek et al. (2022) show that, by construction, transformers are capable of performing gradient descent and closed-form ridge re- gression. Akyürek et al. (2022) also demonstrate that trained in-context learners closely match predictors trained through gradient descent. Further, Von Oswald et al. (2023) show that transformers, by incorporating multilayer perceptrons
Other works explore using I.LMs for defining entire mod- elling pipelines (Tornede et al., 2023; Zhang et al., 2023; Li et al., 2024). In particular, Li et al. (2024) questioned whether Box's 's Loop (Box & Hunter, 1962) could be im- proved upon by using L.MMs. In this work, they were used as model builders and critics automate the model design and iteration steps. The authors find that this process can produce models that perform well in various modelling tasks
14

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
into their architectures, can also solve non-linear regression tasks, and Dalal & Misra (2024) propose Bayesian frame- work to reason about a language model's internal state and suggest that in-context learning is consistent with Bayesian learning. Wang et al. (2023), by viewing I.LMs as latent variable models, suggest that in-context learning can reach the Bayes optimal predictor with a finite number of demon- strations, which can be chosen from a dataset using their framework. However, the authors Falck et al. (2024) provide evidence that LLMs violate the martingale property, which they show is requirement of a Bayesian learning system for exchangeable data. Through this, they argue that the in-context learning behaviour LLMs exhibit is, in fact, not Bayesian.
A.4. Does the LLM use the Model Class we Specify?
As part of comparing AutoElicit with in-context learning, maximum likelihood sampling to extract the internal parameter distributions (discussed in Section 5.4), which we use to test whether the LLM is: (1) providing a similar prior distribution through elicitation to the one it is using in-context, and (2) empirically test whether the LLM is ap- proximating Bayesian inference. However, when extracting the internal parameters of the in-context model, we assume that the LLM is applying the model class that specify in the task description.
25
Total RMSE: 9.20
20
In contrast to the literature, in our work we empirically test, through the approximation of the internal model used by language model for its in-context predictions, whether for given task the language model is performing Bayesian inference before and after seeing demonstrations.
15
10
Trre target valtues
A.3. Formal Details on Extracting the Internal Prior and Posterior
-6
In this section, we describe the method presented in Section 4.2 in more detail.
-10
-16
Given a language model M and task T that provides in- context predictions, if we can construct a function that al- lows us to sample a single parameterisation yp ~ PMrr(p) then we can estimate (xp) using a sampling distribution. Following Murphy (2022), we define an estimator:
20
20
-10
10
20
ICL predicted target values
RMSE
Figure 8, GPT-3.5-turbo in-context predictions for the task: y 2x; 22 + It compared to the true values. Here, fea- ture values are generated using X ~ 21(55,)) 5) and the targets calculated using y 2x1 22 + x3. which is specified fully in the in-context prompt. The language model is then asked to make predictions X. This shows the targets y against those predicted by the language model, as well as the binned root mean squared error.
PMILE = MLEM.T(D)) = arg max (M.,), (5)
This corresponds to the maximum likelihood estimate (MLE) of yp over some dataset DD given the task T. Now consider that D is generated by given yo sampled from i ((p) and X. By sampling
Z[[]] from uni- form distribution over X, we can create S different datasets of the form:
However, we find that for some datasets, the language model does not use the same model in context as the one we speeify, For the synthetic data in particular, we specify the relation- ship as y 2n I2 I3. Here, we can measure whether GPT-3.5-turbo is using the specified model in-context to predict the target by calculating the error between the true y values and those returned by the language model's in- context predictions. This is done by measuring the mean squared error between the yn given Equation 6 and the true Ô.. 2.rn.1 2n, + In,3 calculated using the In in Equation 6.
Di*)
[Un ~ PM. (yolun, v)= : n N, 4[}}}
(6)
This provides us with the sampling distribution:
MLE MLEM,T(((o)) = arg max (DM,)
With this, we can generate a sampling distribution using ( J==5 that approximates (M,)(yp) as S + co. Fur- thermore, by further incorporating the training dataset D: (()) {ym, ~ PM,.r(M ip, D): :):n. 1: N, x u[X])
In Figure 8 we show the true ÿn against the in-context predic- tions yn. Interestingly, even given the simple linear equation we ask the language model to mimic, we find that it struggles to provide good predictions in-context. This suggests that the language model is not always using the model class for
We get the sampling distribution:
(e)
D (D( (D) angmaxpM.riv (D())) (D)
Allowing us to estimate PD).
15

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
in-context predictions that we specify in the task description, indicating that, in its current state, in-context learning might be inadequate for predictive tasks with primarily numerical features. This figure also shows that the language model was hesitant to predict negative values and preferred positive predictions - a strange behaviour that clearly restricts its use as an emulator of a linear model.
We chose the Bayes factor over the Bayesian Information Criterion (BIC) because BIC uses the number of model pa- rameters part of the model selection. When comparing prior elicitation with in-context learning, BIC would almost always favour prior elicitation because the number of param- eters required to make a prediction using in-context learning is of the order 10
However, for all other prior extraction tasks presented in Section 5.4, we find that the language model uses a linear model in-context, demonstrated by the low MSE loss values between the predictions of our MLE samples and the true in-context predictions, shown in Figure 4. This is often not the case when the language model is presented with demon- stration samples, as in the posterior extraction presented in Figure 5, in which the language model uses a linear combi- nation of features to predict the targets for the Heart Disease and Diabetes tasks only.
A.7. Further Experimental Details
In all experiments, we use OpenAl's GPT-3,5-turbo (specif- ically -22- GPT-4-turbo (GPT-4-turbo- 2024-04-09) and DeepSeck- R R1-32B-Q4 222- with a tempera- ture of 0.1 and all other settings as default.
1.7.1. DATASET DETAILS
In this work, we evaluate our method on 6 datasets; 1 of which is synthetic, is privately collected, and 4 are publicly accessible. This allows us to provide both reproducible experiments and test the language model's ability to suggest good prior on datasets not seen during training.
A.5. Energy Distance and Statistic
The energy statistic (Székely& & Rizzo, 2013), which is based on the energy distance metric (Székely & Rizzo, 2012), is method to measure the similarity of two probability distributions. Importantly, it allows us to test whether two N N-dimensional probability distributions are equal.
y = 2.1 - 22 + T3: To evaluate language models as tools for prior elicitation, we construct a synthetic dataset where we have control over the parameters of the dataset. Because of this, are able to provide the language model with complete description of the task. This dataset contains 3 features sampled from a normal distribution (mean = 0 and standard deviation = 1) and targets calculated using the equation y = 2z1 22 + 33 + G, where c AO, 0.05). When training and testing were done on these samples, no preprocessing was performed.
The energy distance D (X, Y), with X and Y as indepen- dent random vectors IR" " defined as:
D(X,Y)² = 2E M xy] B[Mxx] E[Myr]
Here, Mxy = ||=.- ツ la is the pairwise distance matrix.
From this, we construct the energy statistic as follows:
H(X,))= D(X, 2
private dataset collected as part of our study on De- mentia care This dataset contains 10 features engi- necred from measurements of in-home activity and physiology. The raw features were measured using pas- sive infrared (PIR) sensors located around the homes of people living with dementia, and a sleep mat posi- tioned in the participants' beds. The targets correspond to days with clinical labelling of positive or negative urinary tract infection (UTT) which were calculated through the analysis of provided urine sample in con- sultation with a healthcare professional (Capstick et al., 2024). For this analysis, we used a subset of the data containing 78 people and 371 labelled days of UTI, with 270 negatives and 101 positives. When training
This statistic takes values between 0 and 1 and tests whether two statistical distributions are equal.
We provide an implementation of this distance metric and test statistic within our supplementary code (Appendix A.I).
A.6. The Bayes Factor
Given two models ogg and a with parameterisations 0g and 01, the Bayes factor calculated over dataset D is:
BF on: D) p(D(an) So p( pD))a)) Jo p(0 on (p( 0n )df (8)
do)d
In our case, og refers to prior elicitation, with the prior de- termining the parameterisation 0 of a predictive model, and aj referring to in-context learning. We can approximate the Bayes factor using the means from samples of the likelihood of D.
htttps://platfomm http://ppaatformmooeaaaa..aa wwt.oootr.....coooooooeooo wwt./ccoss...oo/eoo api - wtl--rffeenee/com
/ntops://ollaa.. com/ http:/aaa..m..ooaarrreeeeeeeeeee
32b
httpa: http:::://www.mmpraallal
uk ai-arii-crre--rsserhhh----ee
16

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
and testing on this dataset, we centred and scaled the training data and used the same statistics to normalise the test data.
Community Mental Health Teams for older adults (CMHT-- OP), and (5) specialist memory services Surrey and Bor- ders Partnership NHS Foundation Trust. All participants provided written informed consent. Capacity to consent was assessed according to Good Clinical Practice, as detailed in the Research Governance Framework for Health and Social Care (Department of Health 2005) and the Mental Capacity Act 2005, Participants were provided with Participant Information Sheet (PIS) that includes information how the study uses their personal data collected in accordance with the GDPR requirements. If the participant was deemed to lack capacity, a personal or professional consultee was sought to provide written consent to the study. Addition- ally, the capacity of both the participant and study partner assessed at each research visit. Research staff conducting the assessment have completed the NIHR GCP training and Valid Informed Consent training. If participant is deemed to lack capacity but is willing to take part in the research, a personal consultee is sought in the first instance to sign a declaration of consent. If no personal consultee can be found, a professional consultee, such as a key worker, is sought. This process included in the study protocol, and ethical panel approval is obtained.
The Heart Disease dataset (Detrano et al., 1989 An- dras Janosi, 1989); This dataset contains 10 physio- logical features and targets with positive or negative diagnosis of heart disease for 299 people. Within this dataset, we have 161 examples of people without heart disease and 138 examples of people with heart disease, totalling 299 data points. When training and testing on this dataset, we centred and scaled the training data and used the same statistics to normalise the test data, ex- cept where features are categorical. In the categorical case, we perform no scaling.
The Hypothyroid dataset (Quinlan, 1986; Turney, 1994): This dataset, processed from the original, con- sists of 150 examples of patients' thyroid-related blood levels and whether they have a hypothyroidism diag- nosis or not. This dataset contains 4 features, with 75 examples of a patient with hypothyroidism and 75 examples of a patient without hypothyroidism. When training and testing on this dataset, we centred and scaled the training data and used the same statistics to normalise the test data.
Where datasets are publicly available, we perform minimal processing for ease of reproducibility. In some cases, we replace the feature names with ones that are more descriptive (taken from the dataset descriptions also available publicly) to aid the language model's comprehension of the data. All of these changes to the raw data are available within the supplementary code, which will automatically download and process the data when loading.
The Diabetes dataset (Efron et al., 2004): After prepro- cessing, this dataset contains physiological measure- ments and demographic information from 442 people with diabetes. Here, the task is to predict a quantita- tive measure of their diabetes disease progression one year after this baseline. This dataset consisted of 10 features, and the targets were regression values with a maximum value of 10.0, a minimum value of 0.0, and mean value of 3.96.
A.7.2. LANGUAGE MODEL DETAILS
Within the main experiments, we evaluate three language models:
The Breast Cancer dataset (Wolberg et al., 1995): This dataset is made up of 569 data points with 30 features computed from an image of a breast mass that are labelled as either malignant or benign. In total, this dataset contains 357 examples of benign tumours and 212 examples of malignant tumours. When training and testing on this dataset, we centred and scaled the training data and used the same statistics to normalise the test data.
GPT-3.5-turbo" is one of OpenAl's least expensive models and is described as model for simple tasks.
4 is one of OpenAl's more sophisticated models.
DeepSeek-RI -1-ccckk-R1322B--- R1-32B-44 a 32 billion parameter model from DeepSeck based on Qwen 32B and distilled from DeepSeek-RI.
These represent three distinct types of language models. GPT-3.5-turis is cost-effective option for those who are able to use OpenAl's API and removes the need to run the language model locally. Similarly, GPT-4-turbo is hosted by
The UTI study was performed in collaboration with Imperial College London and Surrey and Borders Partnership NHS Trust. Participants were recruited from the following: (1) health and social care partners within the primary care net- work and community NHS trusts, (2) urgent and acute care services within the NHS, (3) social services who oversee sheltered and extra care sheltered housing schemes, (4) NHS
fom.ooooooooooaoaaaaa..... .ttmmaiiccom//om wwtpc/ccccccccctcccctccc.cc.. htt///ccs///a/a// /model <nttps://ol1 lama com/ wwom//ibbtt
rl:
32b
17

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
OpenAI, however, it is a considerably more expensive op- tion. In contrast, DeepSeek-R1-32B-o0 is an open-source model that can be run locally, given the compute require- ment is met, allowing priors to be elicited in secure envi- ronment and without API costs.
As an example, for the synthetic task, one of the task de- scriptions is follows:
You're a linear regression predictor, estimating the target based on some input features. The known relationship is: 'target' = =2 * "feature 0' 'feature 1 + * 'feature 2'. Use this to predict the target value.
System role:
A.7.3. EVALUATING LLM-ELICITER ELICITED PRIORS
To evaluate the usefulness of the AutoElicit priors (presented in Section 4.1), we started by comparing their posterior per- formance to an uninformative prior. To do this, we chose the predictive tasks described in Section 5.1: y 2x1 - 22 +23, Urinary Tract Infection (UTI), Heart Disease, Hypothy- roidism, Diabetes, and Breast Cancer and calculated the posterior accuracy or mean squared error on a test set after observing varied numbers of examples. For each of the datasets, we selected 50% of the data points to form the test set and used the remaining data points to form training set. This is done 10 times to produce 10 folds of random training and testing splits.
User role: am a data scientist working with dataset for prediction task using feature values to predict target. I would like to apply your model to predict the target for my samples. My dataset consists of these features: ('feature 0', 'feature 1', 'feature 2'). All the values have been standardised using z-scores. The known relationship is: 'target' = 22 'feature 0' 'feature 1 1 'feature 2'. Based on the correlation between each feature and target, whether positive or negative, please guess the mean and standard devia- tion for a normal distribution prior for each feature. I need this for creating a linear regression model to predict the target. Provide your response as a JSON object with the feature names as keys, each containing a nested dictionary for the mean and standard devia- tion. A positive mean suggests positive correlation with the outcome, negative for negative correlation, and a smaller standard deviation indicates higher confidence. Only respond with JSON,
For the UTT task, we split the dataset so that each partic- ipant's data is only in either the testing or training set, as some participants produced multiple examples. For the other classification tasks, the splits are stratified using the targets. To evaluate the posterior performance after observ- ing a given number of data points, we randomly select that number of examples from the training set to train on and then test on the same test set as before.
After performing prior elicitation (Section 5.2), for each of the tasks, we have 100 Gaussian priors for each feature in the dataset. We then build mixture of the priors using Equation 2, where we weight the Gaussian distribution in each dimension and each task description separately.
A.7.4. EXTRACTING THE IN-CONTEXT MODEL'S HIDDEN PRIOR AND POSTERIOR
To further study in-context learning, we used maximum likelihood sampling to extract approximate samples of the internal model's prior and posterior using the method de- scribed in Section 4.2. For each of the tasks studied, we randomly generate 25 feature values from X ~ 2 5, 5 times and ask the language model to predict their regression label or positive classification probability based on the task description given and with the specified model class (in our case a linear or logistic regressor). This is done for each of the 100 task descriptions that we have for the dataset. We then fit linear regressor to the predicted regression labels or the classification logits" of the predicted probabilities, using maximum likelihood estimation. The parameters of this linear model are then taken as a single sample of the approximated in-context predictive model.
We then construct a logistic or linear regression model, where we set the prior distribution over the bias term as a mixture of 100 distributions of 6g (V(0,)) 1). In the case of linear regression, we also use a noise term Half-Cauchy(B 1), with Half-Cauchy prior. In the uninformative case, set O ~ (O, 1) for each fea- ture.
We then use the No-U-Turn sampler (Hoffman et al., 2014) to sample from the posterior distribution over the linear model parameters, with 5 chains and 5000 samples per chain. In total, for each dataset, for each training and testing split, we have 25,000 posterior samples over the posterior.
To produce the results in Figure 9, we then sample the poste- rior predictive distribution over our test set and calculate the accuracy or mean squared error for each posterior parameter sample. To produce Figures 2 and 14, we also calculate the mean accuracy and the mean squared error for each of the test splits before plotting the mean and the 95% confidence interval.
For each of these linear model samples, we calculate the regression labels or the classification logits using the same data given to the language model for in-context predictions. This allows us to calculate an emrr in our estimation of the
Calculated using the inverse logistic function: In p (1 p)
logits
18

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
in-context model, which depends on how well the in-context learner is approximating linear model.
squared error, depending on whether the task is classification or regression. For each dataset, for each test split, and for each method, we calculate the mean of this logllikelihood over the samples. With these values, we measure the differ- ence between the value for the elicited prior predictions and the in-context predictions to obtain the log Bayes factor.
In total, for each task description, we collect 5 samples of the in-context model parameters. Once this is repeated for all of the 100 task descriptions, we have 500 samples of the in-context model.
We then report the mean and standard deviation of the Bayes factor over the splits in Figure 7, which helps us to decide which method is most appropriate for dataset; assuming that in-context learning approximates Bayesian inference.
We can then measure, as in Section 5.5, whether the param- eter samples are similar to those sampled from the elicited distributions. This is done by sampling 10,000 parameter values from the elicited distributions and calculating the energy statistic (Appendix A.5) between this sample and the extracted in in-context sample, and through visual inspection. By doing this, we test whether the language model is using the same distribution we elicit from it as the one uses in-context. The results of this are presented in Table 1.
A. GIVEN A NEW DATASET
To apply the prior elicitation techniques to a new dataset, we recommend the following steps:
When extracting the posterior distribution over the in- context learner, we repeat the above process 5 times, where in each repeat we provide the language model with 25 demonstration points from the corresponding dataset. This allows us to empirically question whether the language model is approximating Bayesian inference when it up- dates the in-context prior distribution with the observed data points.
1.
First, create description of the dataset along with a description of the language model's 's role. As part of this, provide descriptive features and target names. It will be helpful to split this task description into what the language model is expected to do (system role) and the task that the user is giving it (user role). Please see our collection of task descriptions in the supplementary code or the example given in Appendix A.7.3.
To do this, we start by fitting a Gaussian kernel density estimator (with a bandwidth factor of 0.25) to our in-context model prior parameter samples. We then use the No-U-Turn sampler (Hoffman et al., 2014) with 1000 adaption steps and the same demonstration examples given during in-context learning to sample posterior parameters. Here, use 100 chains of 10,000 samples to build an approximate posterior distribution.
Ask a language model to rephrase the user role and system role many times. In our experiments, we asked the language model to rephrase both descriptions 10 times for each.
3. Take the product of the system roles and user roles and provide each to the language model. For each of the unique system and user role combinations, record the mean and standard deviation of the Gaussian prior elicited from the language model.
We can then use the energy statistic to measure the differ- ence between the extracted in-context posterior distribution and the Monte Carlo approximation of the posterior based on the extracted in-context prior distribution. The values presented in Table 4 show the result of this experiment.
4. Use these individual Gaussian priors build mixture of Gaussians as a prior distribution for linear model.
5. Using the data points you have available from the pre- dictive task, sample from the posterior predictive dis- tribution to predict on unseen data.
A.7.5. CALCULATING THE BAYES FACTOR
As part of our solution to decide whether to use prior elicita- tion or in-context learning for a given dataset, we calculate the Bayes factor (presented in Section 4.3).
Functions for doing this are provided in the supplementary code.
For single dataset, we start by randomly selecting 25 data points, times, and consider the prior predictive distribution. For prior elicitation, this involves sampling the prior predic- tive distribution (drawing 500 samples for each split), and for in-context learning, this involves prompting a language model for a prediction for each of the 100 task descriptions (producing 100 samples for each of the splits).
To measure whether prior elicitation or in-context learning is better for given task (assuming the language model is approximating Bayesian inference):
1. Firstly, sample the prior predictive distribution, with the elicited priors found above, on the data points al- ready collected.
For both sets of prior predictive samples, we calculate their log- likelihooood using either cross-entropy loss or mean
2. Then, to perform in-context learning, use a similar task description to the above but with a different system
19

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
and user role that specifies the language model should make predictions instead of providing prior distribu- tions. With this new set of descriptions, again ask language model to rephrase them.
linear model, whether the language model uses that prior knowledge when making predictions in-context. If the prior distribution elicited differs from the one that the language model is using in-context, then calculating the Bayes factor could be important to ensure that we select the most appropriate predictive model for the given task. If is not clear whether the language model is approximating Bayesian inference in-context, we should then study the latter ensure the Bayes factor is suitable model selection tool.
3. To sample the prior predictive distribution using the product of these descriptions, ask a language model to make predictions on the data directly.
4. Measure the Bayes factor between the likelihood of the prior predictive distribution using prior elicitation and the in-context model on the already collected data.
For this, we can follow the procedure below:
5. Using the Bayes factor and by factoring in resource cost, decide whether in-context learning or prior elici- tation is more appropriate.
1. Use a kernel density estimator (KDE) to approximate a smoother version of the prior distribution over the in-context model. For our experiments, we used a Gaussian kernel density estimator with a bandwidth factor of 0.25.
To empirically check whether the language model is ap- proximating Bayesian inference to produce its in-context predictions or to test whether it is being faithful to the priors we elicit, we can follow the procedure below:
2. Using the KDE and Monte Carlo methods, given the collected data, we sample posterior distribution from the in-context learner's prior.
1. Using the same task descriptions as above, provide the language model with randomly generated feature values (we used 25 samples from X 21|-5, 5)) for each task description and ask it to make predictions in-context.
3. Use a method to calculate the difference between two samples of distribution to understand the differ- ences between the posterior samples calculated through Monte Carlo sampling of the in-context prior and the samples of the in-context posterior.
2. By additionally providing the seen data as demonstra- tions to the language model, again ask the language model to make predictions on randomly generated fea- ture values for each task description.
This last procedure might indicate that in-context learner is not approximating Bayesian inference and is not suitable for a given task, since it could be important to us that our predictive model is updating its prior knowledge in a reliable way.
3. With randomly generated feature values and in-context predictions, use maximum likelihood estimation to estimate the underlying linear model for each task de- scription. For the classification case, this is done more easily if the language model is asked to provide the probability of a positive prediction rather than the pre- diction itself.
A.8. Dataset Memorisation
To better contextualise the use of in-context learning in the tested datasets, we will use the method proposed in Bordt et al. (2024) to quantify the extent to which GPT-3.5-turo already knows information about the datasets.
4.
The resulting linear models approximated from the in- context predictions can now be used to approximate the prior and posterior distribution of the in-context model.
We start with the header completion test, which checks whether the language model has memorised the first few rows of the dataset. Here, the column names along with a given number of the first rows are given to the language model as a string, with the final of these rows truncated. The language model is then asked to generate the next 500 tokens.
Using the above, we can now (1) empirically test whether the language model is being faithful to the prior distribu- tions we elicit and (2) empirically test whether the language model is approximating Bayesian inference in-context.
For the former, we use measure of the difference in samples (in our case, the energy distance, Appendix A.5) to calculate the difference between the in-context model's prior distribution and the elicited prior distribution. This will inform us of whether the language model is being truth- ful when it provides us with prior on the parameters of
We then the Levenshtein distance (Levenshtein, 1965; 1966), string matching metric, to calculate the agreement between the true header completion and the one provided by the language model. The Levenshtein distance counts the number of single-character changes required to trans- form one string into another, only allowing for insertions,
20

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
2xg
F2. | iyy
with examples
g' 25g
22
ii with examples
2xg
P2 1 Pg
with examples
y
22
ag I Ri with 50 examples
Deasity
0.2
0.1
0.1
0.1
0.0
0.0
0.0
0.0
10-1
10
10
10-2
10-1
10P
10
102
10-2
10-3
10
10-2 Mean squared error
Mean squared ercor
Mean squazed crror
Mean squared ervor
UTI with 2 examples
UTI with 5 examples
UTI with 10 examples
UTI wish examples
Dousity
0.1
0.1
0.1
0.1
0.0
0.0
0.0
0.0
0.2
0.1
0.6
0.8
0.2
0.4
0.6
0.8
0.4
0.6
0.8
0.1
0.6
0.8
Accuracy
Accuracy
Accuracy
Avcuracy
Heart Discuse with examples
Heart Disease with examples
Heart Discase with 10 examples
Heart Disease with 30 examples
Deaaty
0.2
0.2
0.1
0.0
0.0
0.0
0.0
0.2
0.4
0.6
0.8
0.2
0.4
0.6
0.8
0.2
0.4
0.6
0.8
0.4
0.6 Accuracy
0.8
Acouracy
Aecuracy
Acruray
Hypotllyroid with examples
Hypothyruid with examples
Hypotlyyrid with esamples
Hypothyaoid with examples
Dousity
0.2
0.1
0.1
0.1
0.0
0.0
0.0
0.0
0.00
0.25
0.50
0.75
1.00
0:25
0.50 Accuracy
0.75
1.00
0.2
0.4
0.6
0.8
1.0
0.6
0.7 Avcuracy
0.8
0.9
1.0
Accuracy
Accuracy
Diubetes with 2 examples
Diabetes with 5 examples
Diabetes with 10 examples
Diabetes with 50 examples
Denety
0.1
0.1
0.1
0.0
0.0
0.0
0.0
10
10F
10a
10
10a
10a
10a
10
30
Moon squared error
Mean equared enror
Mean squared error
Mean squared orroe
Breast Cancer with examples
Breast Canrer with 5 examples
Breast Cancer with If examples
Breast Cancer with 30 examples
Denaity
0.2
0.2
0.2
0.2
0.0
0.0
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.2
0.1
0.6 Accurcc
0.8
1.0
0.4
0.6
0.5
1.0
0.7
0.8
0.9
1.0
Accuracy
Accuracy
Avcuracy
Uniaformative prior
GPT-3.5-turbo peior
GPT-4-luro prior
DeepSeek-R1- R1-32BBQ4
Figure 9. Test accuracy of the posterior distribution over the model weights of logistic and linear regression model for varied numbers of training data points. We evaluate the accuracy and mean squared error of different prior distributions over the weights of a logistic and linear regression model. Each of the Gaussians in the mixture is extracted from language model and is tested against an uninformative prior N (@jo, 1). These results are calculated on a test set of the datascts after each model is trained on the given number of data points. These values are the result of 5000 sampled parameters and random splits of the training and testing data. The green arrows point in the direction of improvement in the metric.
deletions, and substitutions. The value can take mini- mum value of 0, suggesting that the strings are equal, and maximum value of the longest length of the two strings.
Here, Levenshtein(s;, 52) calculates the Levenshtein dis- tance between two strings, and m [s2]} returns the minimum length of the strings. This new value ranges be- tween and 1, with 0 indicating a perfect match between the strings and implying that they were different at all positions.
To make the Levenshtein distance comparable across datasets, we calculate its normalised value by dividing the Levenshtein distance by the length of either the true string or the language model's completion, whichever is longer. Be- cause of this, the normalised Levenshtein distance reported is not formally a metric as it does not follow the triangle inequality. However, it allows us to compare the header completions across datasets more easily:
The values in Table 2 show the Levenshtein distance be- tween the true dataset header completion and the made by the L.M. This demonstrates that the language model has completely memorised the first few rows of the Breast Cancer and Heart Disease datasets, shown by a normalised Levenshtein distance value of 0. This could be because the first few rows of these datasets commonly appear within the language model's training data. For the Diabetes and
Levenshtein(s1) $1, $2) min[[s1l, |
L(s), s2)
21

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
from random rows of the dataset. Also, although the LLM can complete the first few rows of the datasets with some success, it is unable to complete random rows. Therefore, we hypothesise that any memorisation of the tested datasets has little impact on the in-context predictions.
Table GPT-3.5-turbo header completion test results. Nor- malised Levenshtein distance between the correct header comple- tion and the language model's completion for each of datasets tested. A value of (I indicates that the true completion and language model's 's completion were equal, and a value of means that they were different all positions.
Further, since for prior elicitation we only prompt the lan- guage model for a prior distribution of a predictive model given the feature names and a description of the dataset, the memorisation of rows from the data is unlikely to have had an impact on these results. In addition, we see that elicited priors provide improved predictions over the uninformative prior for the private and synthetic datasets (with no dataset memorisation), as well as the public datasets.
Heart Disease
Diabetes
Hypothyroid
Breast Cancer
0.00
0.21
0.22
0.00
Hypothyroid datasets, we see some level of memorisation, with the language model being able to reproduce a large amount of the true header. As we might expect, when we perform the header test on the synthetic dataset generated using the relationship y 1 - 22 + 23, we get nor- malised Levenshtein distance of 0.7, which shows a large difference between the true header string and the completion from the language model. The UTT task has not been tested, as it is a private dataset.
A.9. Additional AutoElicit Results
1. ACCURACY DISTRIBUTIONS
In Figure 9, we present the accuracy distributions from the experiment in Figure 2, from Section 5.2.
For the synthetic task y = 2z1 22 + 23, the priors elicited from the LLMS allow for posterior predictive distribution with a markedly reduced mean squared error (MSE) over an uninformative prior whilst producing less variation in the result. Here, GPT-3.5-turbe initially produces the lowest MSF. However, due to the simple nature of the task, as more data becomes available, all priors achieve similar results.
Table GPT-3.5-turbo row completion test results. Normalised Levenshtein distance between the correct row completion and the language model's 's completion for cach of the datasets tested. A value of 0 indicates that the true completion and language model's completion were equal, and a value of 1 means that they were different at all positions.
Heart Discase
Diabetes
Hypothyroid
Breast Cancer
0.33 + 0.05
0.05
0.20 = 0.05
0.43 0.13
When predicting UTI, all of the AutoElicit priors provide improved accuracy over the uninformative prior for all train- ing sizes tested. This should be of particular interest, given that the UTI dataset is private and so will not have appeared in any form in the training data for the LLM.
We supplement these results with those of Table 3, which shows the results of the row completion test. Here, we calculate the normalised Levenshtein distance of the true completions and the language model's completion of a ran- dom row from the dataset. In these tests, the language model is provided with 10 rows (in order from the original dataset) and is asked to complete the following row. The values in Table represent the mean and standard deviation of the test using 25 random rows. Here, in contrast the header test, see that the language model has not memorised all rows from the dataset. This is because the language model's completion differs from the true row completion in all cases.
For the Heart Disease dataset, we find that the prior elicited from the language model performed similarly to the uninfor- mative prior. This is because the language model provided mean and std = 1 for a large number of priors elicited, suggesting that the LLM was unsure of its prior knowledge. The parameter distributions elicited can be seen in Figure 18, which contextualises these results. However, DeepSeek- Barkk-R--33 R1-32B-Q2 B-44 and GPT-4-turbo provided prior dis- tributions that were considerably more accurate.
We likely see these results because when the datasets appear within the language model's training data, their column names and initial rows are usually given, but not the entire dataset.
For Hypothyroid prediction, DeepSeek-R1-32B- vides the best accuracy by large margin. Of the OpenAI models, smaller training sizes favour GPT-4-turbo, whilst larger training sizes favour GPT-3.5 ttr-bbb... This is because the GPT-4-turbo-eliciteed prior was too strict to allow for up- dated posterior parameters, but was more informative than an uninformative prior for smaller training sizes.
pro-
A.8.1. 1. HOW DOES THIS AFFECT OUR RESULTS?
Within our experiments, we normalise the training and test- ing data and sample random observations from the dataset. Because of this, observations predicted during in-context learning will likely not have been memorised, as they are normalised based on a training data subset and are selected
For the Diabetes task, both OpenAl LLMs provide consider- ably improved MSE over the uninformative prior for training sizes up to 50 observed data points, whilst DeepSeek-RI- 32B-Q4 provides worse MSE.
22

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
In the Breast Cancer task, the accuracy improvement from the elicited DeepSeek-R R--23-----333-44-4--- and GPT--4-turbo priors is maintained across all training sizes, whilst GPT-3.5-turbo provides an improvement up to 40 observed data points. This can be seen in these distributions, which show a long tail in the accuracy distribution for the uninformative prior for most training sizes.
number of labels tested in the experiments in Section 5.2 against the accuracy of model using an uninformative prior and one elicited from a language model. This result illustrates the usefulness of prior elicitation, since we are able to achieve greater accuracy with fewer data points, and therefore less data collection time.
Since UTI label collection is costly and slow, our predictive model could have achieved greater accuracy significantly earlier in the study. This is paramount in clinical environ- ment, where an improved UTI detection model could have resulted in fewer unplanned hospital admissions.
A.9.2. PRIORS ELICITED FROM THE SYNTHETIC DATA
Feature
Feature
Festure
Medion: 2.0
Median -1.0
Median: 1.0
Denaity
It is important to note that the accuracy in Figure 11 is not achieved as a result of training on the earliest collected data points. This is because in our analysis, we perform cross- validation and use a subset of the data from a later time period (Section 1.7.1). Changes in the in-home devices were made during the study, such as the addition of a device to collect sleep information, which we use within the dataset evaluated in the main results. However, if all data collection devices had been used from the beginning of the study, we could expect the accuracy in time resemble the figure presented here.
-2
2
-2
2
22
2 Parameter value
Pormmeter value
Parameter value
Figure/ 10. Elicited priors for the task y = 2x1 D2 + Ta- We sample 10,000 parameter values from the mixture generated from the Gaussian priors elicited from the language model when given the task description. This figure shows the histograms of those values each feature dimension.
Figure 10 shows the histogram of parameter values from prior for the task descriptions of y = 2x. - I2 + Zz. Here, the language model correctly pro- vides prior distributions that are centred around the correct parameter values and offer small standard deviation.
the
A.9.4. ANOTHER UNINFORMATIVE PRIOR
For additional context to the results provided in Figure 2, we present the posterior performance of a different unin- formative prior. In this further case, we sample 100 val- ues from a standard normal distribution (μu ~ ~N(u0 1)), and we use these as means of 100 Gaussian distributions (0& NO pr 1)). With these distributions, we construct mixture prior. As we keep the standard deviation at I for each component, the resulting uninformative prior has a greater range in the parameter space.
A.9.3. UTI ACCURACY WITH THE LABEL COLLECTION CONTEXT
0.7
Accunee
Uninformatiie poior
Figure 12 shows the results of this experiment compared to those achieved using AutoFlicit. This new uninformative prior leads to no noticeable difference in the results for the synthetic task, Diabetes task, and Breast Cancer task. How- ever, see marginally better accuracy the UTI task than the previous uninformative prior, and on the Heart Disease and Hypothyroid tasks, we see greater improvements. De- spite this, SeepSeee-RR-32B---- or GPT-3.5-tuubo continue to provide more informative priors.
GPT-3.5 e.u priot
CPT ----abbb prior
DeepSeek-RL RI 21--22--00 Q4 peior
560
(5 Ecbel-(10 labied))
(20 lobele)
(30 lnbels) (40 labels) (30 lobbele)
Days since first label (and mimber of labels collented)
Figure/ 11. Posterior accuracy of the elicited priors on the UTI dataset considering label collection time. The average mean posterior accuracy and 95% confidence interval on the UTI test set after observing the given number of data points against the time it took for the data points to be collected in the study. The horizontal axis shows the time in days to achieve the given performance and the number labels collected in that time.
This suggests that the performance improvement we see from using AutoElicit over an uninformative prior is not due to its complexity.
A.9.5. COMPARING WITH GOUK & GAO (2024)
Since we have access to the underlying study that collected the UTI dataset, we can understand in more detail how the accuracy of the predictive model varies in time, as well as with the number of data points collected.
The method presented by Gouk & Gao (2024) generates synthetic data for a predictive task using an LLM. Using this data, they provide an additional likelihood term when training linear classification models. Specifically, they use
In Figure 11, we plot the time it took to collect the given
23

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
y
Heart Disease
Hypochyooid
Diabetes
Breast Cancer
0.7
Accussey
Acmusay
Acouracy
0.9
lo0 MSE
1.5
Accuacy
0.9
10ga:
0.6
0.8
0.6
1.0
0.7
0.7
10 20
30
40
50
10
20
30
40
50
10
20
30
40
50
10
20
30
40
50
10
200
30
40
50
10
20
30
40
50
# of training examppees
# of training examapee
# of training examples
# of training exaampee
#
training excaupless
# of Sraining examples
Previous uninformative prior
Uninformative mixturr prior with Gauseian means
GPT-3.5-turh prior
DeepSeek-R1-22B5 prior
Figure 12. Comparing AutoElicit to mixture of uninformative priors. This figure presents linear model's posterior accuracy mean squared error after observing the given number of training data points, over 10 repeats. Here, we compare the posterior performance of using AutoElicit with two LLMs to that of mixture of uninformative priors, where the means of cach component are sampled from standard Gaussian distribution, as well as the uninformative prior presented in Figure 2.
Data private: Prior hased on geneeated data pot allected by LLM memociicttion
Data is public: Prior based on generated data possibly improved by LLM momorisation
UTI
Heart Discase
Hypothyrood
Diahetes
Beeasst Cancer
0.7
Accurscy
Accusacy
0.7
Aceuracy
0.9
JoHg MSE
Aucessyy
10gao
0.8
0.6
1.0
0.7
0.7
10 20 30 # of training exomples
20
40
20
40
30
50
10
30
10
20
30
40
@ of training examples
# of training exumplee
of training examples
# of truining examples
# of training examples
-Iw. Uninforuntive prior
GPT-3.5-tulle prior
GPT-4-urbb prior
GPT-3.5-tubo generated data
GPT-4-turo generated data.
Figure 13. AutoElicit and baseline from Gouk & Gao (2024). This figure demonstrates linear model's posterior accuracy or mean squared error after observing the given number of training data points over 10 repeats. The solid lines correspond to the results from AutoElicit, whilst the dashed lines correspond to the results from Gouk & Gao (2024). The left left-most two datasets are unseen by an LLM, whilst the right-most four datasets are publicly available and likely have samples within the LLM's 's training data. In these cases, the LLM could be reproducing real data from the task, artificially improving the results for the method proposed by Gouk & Gao (2024).
an LLM to generate feature values based on a task descrip- tion, which are labelled using zero-shot in-context learning. The samples generated are provided alongside real data dur- ing the training of linear predictive model. However, this means that this method is susceptible to LLM data memori- sation, putting it at risk of artificially providing improved posterior results on public datasets. Given this is concern discussed in Bordt et al. (2024) and Appendix A.8, during our evaluation, we split the results by those achieved on pub- licly available tasks and those on privately available tasks. Additionally, to fully compare this method with AutoElicit, we extended their work to regression tasks. Figure 13 shows the results of this experiment.
vate data, we suspect the performance of this approach is artificially improved by reproducing the true dataset. Since for AutoElicit, we prompt the language model for prior distribution over the parameters of a predictive model, us- ing only the feature names and a description of the dataset, the memorisation of data points is unlikely to have had an impact on our method's s results (Appendix 1.8.1).
Morcover, in both of the private datasets (the two left-most plots), where there is no risk of LLM memorisation, Au- toElicit results in improved posterior accuracy and mean squared error. On the synthetic task, our improvement is orders of magnitude better, and the UTI task, our method significantly increases accuracy at all training sizes. In both cases, the approach proposed by Gouk & Gao (2024) under- performs compared to an uninformative prior.
In the public tasks, we see that the performance of the ap- proach presented in Gouk & Gao (2024) varies considerably by LLM and task. It underperforms compared to the unin- formative prior on five occasions. On the two occasions it outperforms AutoElicit (ours), it achieves surprisingly good accuracy or mean squared error (Heart Disease with GPT-4- turbo and Diabetes with GPT-3.5-turbo suggesting that it might be reproducing parts of the public datasets rather than generating new samples. Given our analysis in Appendix A.8, showing the memorisation ability of LLMs, and the results of the method given in Gouk& & Gao (2024) on pri-
A.10. Cost of AutoElicit
The priors elicited from GPT-3.5-turbo and GPT-4-turbo incur a cost, as we use OpenAl's API. To elicit all of the pri- ors used in the experiments in Section 5.2, we were charged $0.41 (on average $0.068 per dataset) for GPT-3.5-turee and 87.56 (on average $1.26 per dataset) for GPT-4-turbo. As rule, the more features dataset contains the longer
24

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
the task description, the more expensive priors elicit since the number of input and output tokens is greater. How- ever, we did not make use of the batch API and instead made each API call sequentially. All experiments in this work could be implemented using the batch API since no new API call depends on a previous API result, which may significantly reduce costs"". 10
detail shown in Figure 14, we included the following text
in the task description that was provided to the language model:
Linear relationship:
"The target is linear features"
One feature relationship:
Eliciting the DeepSeck-FR -1--2BB------B----- Q4-based prior distribu- tions incurred no API cost, because the model is open-source and we were able to run it locally. However, to do this, we used an Nvidia A100, which will likely not be available in a low-resource setting. The API costs for the non-quantised model will apply in these cases!! !.
"The target is a linear combination of the features and that when 'feature O' increases by 1, the target increases by 2
Two feature relationships:
"The target is a linear combination of the features and that when 'feature O' increases by 1, the target increases by 2, and when 'feature I" increases by 1, the target decreases by 1"
A.11. Task Descriptions with More Information
Uninformative
Three feature relationships:
Linear relationslaip
"The target is a linear combination of the features and that when 'feature O' increases by 1, the target increases by 2, and when 'feature 1' increases by 1, the target decreases by I, and when 'feature 2' increases by 1, the target increases by 1"
0.5
One fenture telatoosship
Two feature jelationehipe
0.0
Three festure relationshipe
Full equation
lo0 MS6
-05
Full equation:
1.0
"The 'target' 2 * 'feature 0' 'feature I'+ 'feature 2
1.5
For each prior elicited using these levels of detail, in Fig- ure 14, the corresponding 1og10 MSE on the test set can be seen for linear regressor after observing the given number of data points. This figure illustrates the effects of varied descriptions and shows that the improvements made the MSE diminish as the task description contains more detail. The biggest improvement came from providing the language model with a single feature relationship enabling an MSE that is two orders of magnitude better. This suggests that it is most helpful to provide more detail when the language model initially knows very little about a task.
-2.0
10
20 Number of training examples
30
Figure 14. Mean squared error using the GPT-3.5-turbo- elicited priors for the task: y = 2x1 x2 + 23 where we have used increasingly more detailed task descriptions. Here, feature values are generated using X ~ 21(55, 5) and the targets calculated using y 2m1 22 + 233 Within each task de- scription, we provide the language model with increasingly more information about the task to test how the task description affects the priors elicited. Complete descriptions of the task descriptions given here are provided in Section A. The lines depict the aver- age mcan posterior mean squared error (MSE) over the datasct splits, whilst the error bars represent the 95% confidence interval.
Providing the language model with improved levels of detail allows it produce prior distributions that enable linear regression model to achieve given error rate with fewer data points. As we might expect, the more detail provided in the task description, the better our prior distributions are.
To understand how the detail of the task description affects the performance of the priors elicited from the language model, we studied the synthetic task 2z - I2 + Z3, and described the relationship between features and targets in increasing detail. We additionally test an uninformative prior to contextualise the results. For each of the levels of
However, it unexpected that using an uninformative prior (0 (0 1)) provides better posterior performance than specifying to the language model that the relationship be- tween the target and features is linear (Figure 14). This is explained by inspecting the prior distribution elicited when given this minimal information, which shows that the lan- guage model responded with a similar mean and standard deviation for a Gaussian for each of the 100 variations of the prompts:
10
httpps:/ /ttp:/s:///////sss./o dewpssek.eeeeee.eeee com wwtp/uic././/o/ooooo.p.......... start/pricing
25

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
Feature
Feature
Feature
One feature
Median: 2.0
Median: 0.0
Median: .0.0
Dousity
Leceossizall detaclled expoott Informmat
Two festreee relotionsiige
Median: 2.0
Median: -1.0
Mediaa: 0.0
Deasity
10
Thrve foot relatiznssippe Deasitgy
Modian: 2.0
Median: -1.0
Medianl 1.D
2
Pacameter value
Parameter value
Parameter value
parameter distribution increasingly more information is included. These histograms show the distribution of parameter values for the features as more expert information is provided. In each row, we provide information in the task description of additional feature, with the green arrow indicating the effect this has the parameter distribution. In particular. in the first row we provide information about the relationship between the target and the first feature, in the second row we provide information about the second feature, and in the third we provide information about the third feature.
Figure 15.
2xg
UTI
Heart Disense
Hypothgrood
Diabetes
Brezet, Cancer
0.2
Energy statistie
0.2
0.15
0.15
0.2
0.3
0.10
0.1
0.10
0.2
0.1
0.05
0.05
0.0
0.00
0.0
50
100
# of task descriptions
of tasl deseriptions
# of task deseriptions
# of task descriptions
# of task deseriptions
# of task descriptions
Figure 16. Measuring how similar the GPr-s..-turroo-eiiieee prior distributions are with fewer task descriptions. We reduce the number of task descriptions and calculate the energy statistic between 10,000 samples from the prior elicited from the language model using the reduced set of descriptions and the full 100 task descriptions. We repeat each experiment 10 times and show the mean and standard deviation.
Feature O: mean: 0.5 + 0.11, std: 0.12 + 0.05
when given the full equation. Fully specifying the equation is not realistic in a real-world setting where we are unsure about the exact relationship between features and targets, but this provides an example of how we can make the task description as information -enseseoonneeeeeeeee as possible.
Feature I: mean: -0.32 = 0.08, std: 0.08 = 0.06
Feature 2: mean: 0.56 + 0.26, std: 0.14 = 0.09
This demonstrates the sometimes unexpected behaviour of language models. Here, the LLM produced Gaussians for the prior distribution with small standard deviations (reflect- ing a high confidence in the distribution) and means that the language model knew the underlying equation defining the relationship between the target and features. We might have expected that after only being told the relationship was linear, the language model would produce Gaussians with O mean and standard deviation 1, reflecting the uncertainty in the exact relationship between each feature and the target.
An expert interacting with the language model might be able to provide further information about task that improves the elicited priors. In Figure 15, we show the parameter distribu- tions elicited from the language model for the different de- scriptions we provide. As we provide additional information on each feature, the language model responds by providing prior distribution that reflects its updated knowledge of the task. Since the language model knows nothing about this synthetic task beforehand, this experiment demonstrates the effectiveness of using the AutoElicit framework to convert expert information as natural language into prior probability distributions. This experiment provides a controlled setting that supplements the results in Section 5.3.
Interestingly, although "Three feature relationships" and "Full equation" contain the same information written differ- ently, the language model provided better prior distributions
26

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
225- - 12 R
Heart Disens
Hypothyroid
Diabetes
Brenst Cancer
GPT-3 GPT-&.--tur S.t.ooooooooo
-1.5
0.7
0.9
1.5
0:90
loçio MSE
Antatury
0.65
Actaracy
Aceurac
0.8
loga MSE
Accuracy
0.85
-200
0.00
0.6
1.0
0.7
0.80
10
20
3D 10
50
10
20
30
20
50
20
30
10
50
10
20
30
40
50
10
20
30
20
50
20
30
10
50
# trainiing
excmuples
# of Iesining examples
of training examples
# texining examples
# ol Lraining examples
of Eraininn examplee
DoopSeokeR 1-221-Q44
2xg
12
23
UTI
0:70
Heart Disease
Hypothyroid
Diabetes
Breast Cancer
0.7
log MSE
Accurrry
Aocurac
Accatacy
0.9
log MSE
Accuracy
0.90
-1.5
0.65
-2.0
0.8
1.0
0.85
10 20 3D 10 50
10
20
30
20
50
20
30
10
50
10
21 30 10 50
10
20 30
20
5D
20
30
10 50
# of training examplees
# of training examples
of training examuples
# of training examples
# of training examples
of training exomaples
16 mixture components
36 mixture comppoennts
64 mixture comppeeents
100 rixture components
Figure 17. AutoElicit with different numbers of mixture components in the prior. This figure demonstrates a linear model's posterior accuracy or mean squared error after observing the given number of training data points, over 10 repeats. The rows of the figure correspond the different language models tested, whilst the colour of the lines indicates the number of mixture components used in the prior. Using 16 mixture components corresponds to using a product of 4 system roles and 4 user roles when eliciting priors.
A.12. How Many Ways Should We Describe a Task?
Cancer with DeepSeek-R1-32) Q4) in the accuracy or mean squared error as we increase the number of components in the prior, suggesting our framework is robust to the number of task descriptions. We hypothesise that a greater number of mixture components makes the framework more resilient to LLMs that frequently hallucinate, especially on tasks with large numbers of features (such as Breast Cancer).
In this section, we examine how many task descriptions are required to be able to get a similar prior distribution to that elicited within our experiments, using 100 task descriptions.
Figure 16 shows the energy statistic (Appendix A.5) of sam- ples from the prior distribution elicited using fewer task descriptions compared samples from the prior distribu- tion elicited using 100 task descriptions (from the same language model). In particular, we compare subset of task descriptions with the full 100 task descriptions.
Figure 16 showed that the density of the priors changed with the number of mixture components, however, this has a small effect on the posterior performance. This is likely because the sample space that provides good posterior per- formance is covered by the majority of elicited components, whilst the large changes in Figure 16 due to more unique components.
We see that for most tasks, a large number of task descrip- tions required, since reducing the number of descriptions from 100 to 49 produced prior distributions that are sig- nificantly different for all datasets tested. Using 81 task descriptions produces prior distributions that are more simi- lar but still show differences for some datasets (for example, 2x1 12 + T3 and Breast Cancer).
A.13. Are AutoElicit Priors Interpretable?
As the density of the distributions in Figure 18 are used for the prior directly, they allow for interpretable predictions. Considering the AutoElicit priors, for linear models, each feature's 's prior corresponds to its direction and importance when predicting the target. On the synthetic task (row 1), the prior distributions concentrate on the parameters that we expect. For the UTI task (row 2). the LLM provided prior distributions that suggest Feature 2 was negatively predictive of a positive UTI, whilst Feature 10 is strong positive predictor. For Breast Cancer (row 6), the LLM produced two modes in its Feature 2 prior. Here, the LLM was either unsure of its belief, and so the elicited prior changed between task descriptions, or the LLM was unsure of the feature's direction of correlation but guessed that the feature was important.
is interesting that Breast Cancer benefited the most from increasing the number of task descriptions, since this dataset has the largest number of features. It suggests that more task descriptions required for datasets with high-damensional data. This makes intuitive sense since we might expect larger mixture of Gaussian components to better describe the prior over for a higher-dimensional dataset.
Further work could allow the number of task descriptions to be a function of the number of features in the dataset.
To supplement this, we calculate the posterior performance with varied numbers of mixture components in our prior, shown in Figure 17. We see little variation (except for Breast
27

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
Feature
Feature
Fenture
Eierrgy stat 0..55
Dutaly
24
Eaergy stat: 0.30
Feature
Feuture 2
Fenture
Feature
Fenture
Fenture 6
Foature
Feature 8
Fenture
Fenture 10
Deaaity
UTT)
0.5
0.0
0.0
0.5
0.0
0.5
0.250.00 0.25
0.5
0.250.00 0:25
0.0
0.5
0.0
Eanggy stat. 0.18
Featiare
Featance
Featuure
Featare
Featare
Fealiuee
Feature
Featance
Fentiuee
Festure 10
Hour Disessee
Donety
0
0
-1
0
2
0
-1
0
-1
0
-011
Hyporthyeiid liueggy stat.: 0.15
Featiare
Fenture
Densley
Eanrgy stan:: 0.38
Feature
Frature 2
Feature
Feature 4
Feature
Fenture 6
Feature
Feature 8
Fenture
Peature 10
Dicletes
Deaslyy
-0.00.0 0.5
0.0
0.5
0.0
0.5
0.0
0.5
Elergy stat. 0.31
Feature
Feature 2
Feature
Feature
Feature
Fenture
Feature
Feature 8
Fenture
Featuure 10
Brosst Clusee
Duuiit
-0.5000 0.5 Valtie
O
-0.50.0 0.5
-0.50.0 0.5
0 Value
-.5 0.5 -0.5 01.) 0.5 Value
Value
Value
In-context learning prior
Mixture of elicited Guesiin priors
Figure 18. Distribution of the prior parameters under elicitation and those approximated from the internal model for the different predictive tasks for GPT-3.5-turbo. Here, we compare the prior distributions the different predictive tasks, as defined by the internal model used in-context, and that which clicit from the language model. Here, we draw the histogram separately in each dimension, but calculate the energy distance across all features combined.
A chain-of-thouglhe LLM such as DeepSeck-RI-s R1---B----BB---- could help us better understand the elicitation process.
learning, which supports the findings in Section 5.5, In the in-context case, the prior distributions often have modes close to suggesting that the internal model not sure about the relationship between features and targets. In con- trast, the elicited priors often represent stronger belief in the correlation between features and targets, with most histograms achieving modes greater than or less than 0.
A.14. Is the Language Model Being Consistent During In-context Learning?
In Section 5.5, we study how consistent GPT-3.5-turbo is when we elicit prior distribution, compared the prior uses its internal predictive model when performing in-context learning.
Interestingly, on the Breast Cancer dataset, prior elicitation resulted in two distinct modes the prior distribution for Feature ("mean texture"), which could represent uncer- tainty in the direction of correlation.
Figure 18 visualises the different prior parameter distribu- tions for all of the tested datasets in each feature dimension.
Notable are the large differences between the elicited prior distributions and the ones used when performing in-context
Furthermore, the prior distributions the parameters for the Heart Disease and Hypothyroid datasets have more variation
28

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
Ebergy stat. 0..0
Feature
Feature
Feature
Hyppohhrrid
Deasity
1.0
2255
Eneegyy stat. 0.01
Feature
Fouture
Fonture
Feature
Feature
Fatture 6
Fenture
Fenture
Foamure
Fosture 10
Broast Canper
1.2
Densiay
0.8
0.0
-2.5
0.0 Value
2.5
-2.5
0.0
2.5
-2.5
01.0
2.5
-5
-2.
0.0
2.5
2.5
0.0
2.5
-25
0.0
2.5
-2.5
0.0 Value
2.5
-2.5
0.0 Value
2.5
-2.3 5
0.0 Value
2.5
Value
Value
Value
Value
Value
Value
In-conter: learning posterior
MCMC on in-content learning prior
Figure 19. The different posterior samples for GPT-3.5-turbo. Here, we compare the distribution of the posterior extracted from the language model's in-context predictions against the distribution that sample using Monte Carlo methods the extracted prior distribution. Both posterior distributions are based on the same 25 data point sample from the corresponding dataset. We draw the histogram separately in each dimension, but calculate the energy distance across all features combined.
in their values in comparison to that extracted from the language model's internal model. This tells us that the elicited prior parameters represent more uncertainty in their value and are therefore more flexible when calculating the posterior on observed data.
could be performing Bayesian inference. However, this should be considered alongside the results in Section 5.4, where found that approximations of the posterior distribution for the Breast Cancer task had a large error.
On the other hand, for the Hypothyroid predictive task, the parameter distributions differed significantly, with the in-context prior having a less varied distribution. This is reflected in the energy distance, which is the largest of all of the datasets tested.
A.15. Empirically, Is the Language Model Approximating Bayesian Inference?
In this section, we present the differences between the pos- terior distribution extracted through maximum likelihood sampling of GPT-3.5-turbe 's in-context internal model (Sec- tion 5.4) and one that is sampled through MC methods on the extracted in-context prior distribution.
These comparisons must be made with the approximation error in mind, presented in Section 5.4, which suggests that the posterior in-context distributions for the Breast Cancer and Hypothyroid tasks are not reliable.
A.16. AutoElicit with Other Language Models
Table 4. Difference between approximated posterior and MC on the approximated prior for GPT-3.5-turbo. The energy statistic between the language model's internal posterior extracted through MLE sampling and the posterior samples from applying Monte Carlo techniques the language model's internal prior. Values are E [0, 1] with values closer to zero, indicating that the distributions
In this section, we will compare the results presented in the main paper with other commonly used language models. We start by exploring the performance some other OpenAI models, before considering some of the open-source Llama and Qwen models.
similar. Here, we show the mean and standard deviation of the energy statistic for the different training tested observed data.
A. 1.1.1.1 OTHER OPENAI MODELS
Heart Diseuse
Hypothyroid
Diabetes
Breast Cancer
Here, we compare the results of priors elicited from GPT- 3.5-turbo, DeepSeck-R1-23 Riii--RR-----B--- (open-source), and GPT- 4-turbo (which incurs significant cost) to GPT-4o-mini (which provides a less expensive option).
0.04 + 0.01
0.06 + 0.01
0.02 + 0.00
0.01 0.00
Table shows the mean and standard deviation of the energy distance between these distributions, for all of the datasets tested, including those that have a high approximation error when we estimate the in-context posterior. In addition, Fig- ure 19 shows the distribution of these posterior parameters for visual inspection.
To elicit all priors presented here, GPT-4o-mini cost $0.32 (on average $0.053 per dataset), whilst GPT-4-turbo cost $7.56 (on average $1.26 per dataset) and GPT-3.5-turbo cost $0.41 (on average $0.068 per dataset).
Figure 20 shows that GPT-4o-mini often produces prior distributions that are equal to or worse than GPT-3.5-turbo in their posterior performance on all tasks except one. For
For the Breast Cancer dataset, these two distributions match closely and show a strong agreement in their parameter values, suggesting that for this dataset, in-context learning
29

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
== 2n
UTI
Heart Diseuse
0.7
Jogio MIS
Aceuney
0.65
Accuracy
0.6
0.00
10
20
10
20
30
10
20
40
# Araining examplee
of Lesining examppees
# of leaining excmmple
Diabetes
Hypothyzoid
Breast Caneer
0.9
0.9
log%o MSE.
Avrrruy
Accuracy
0.8
0.7
10
20
30
10
10
20
30
10
10
20
30
40
# training exammpes
of training examppes
# training examples
Unitformative prior
GPT-3.5 3.5 tucbo prior
GPT-4 turbo prior
Searyseil-R1-33--44 prior
GPT-1o mini prior
Figure 20.
Test accuracy the posterior distribution compared GPT-4o mini. Here we show the posterior results when using priors elicited from two other OpenAl language models and an uninformative prior (0)0,, 1). These results are calculated on test set of the datasets after each model is trained on the given number data points. The values are the result of 5000 sampled parameters for each of 5 chains, and 10 random splits of the training and testing data. For the regression tasks, report the mean squared error log scale. This figure shows the line plot of the average mean posterior accuracy or mean squared error over the 10 splits, with the error bars representing the 95% confidence interval. The green arrows point in the direction of improvement in the metric.
the synthetic, Diabetes, and Heart Disease prediction tasks, GPT-4o-mini provides priors that achieve similar worse performance to the uninformative prior. However, for the UTI and Breast Cancer predictive tasks, they achieve similar posterior accuracy to GPT--35-turbo at a lower price point. Also, on the Hypothyroid dataset, GPT-4o-mini provides prior distributions that achieve greater accuracy than those from GPT-3.5-tute and GPT-4-turbo for all training sizes, whilst requiring significantly less cost. Further, in all tasks except Diabetes progression prediction, DeepSeek-RR -1-32BBB Q4 provides better accuracy than GPT-4o-mini.
Llama-3.2-3B. -nsttrutt billion parameter model from Meta's latest collection of v3.2 Llama models.
Llama-3, 1-B.B..11BBBmi Instruct an 8 billion parameter model from Meta's v3.1 Llama collection.
Llama-3, Mama-3311770B InstrucnttttttBtt tion of 1amma-3.--70B-nnsee model from Meta's v3.1 Llama collection.
a 6 quantisa- 70 billion parameter
14 billion parameter model from Alibaba Cloud's v2.5 Qwen collection.
161. OOUR-SOURCE LANGUAGE MODELS
In contrast to the OpenAl models in Appendix A.1 these models are publicly available and open-source. This means that they can be used without the need for connecting to an online API, and when locally hosted, all data and infer- ence can be performed on-device, enabling the use of our framework within closed and secure environment.
Here, we explore the posterior performance of prior distri- butions elicited from three Llama (Touvron et al., 2023; MetaAI, 2024; Dubey et al., 2024) models, one Qwen model (Yang et al., 2024; Team, 2024), and one DeepSeek model (DeepSeck-Al) al., 2025) those elicited from the OpenAl models presented in the main paper:
http:::::///uuginngaaaa 3.ma-3.2-3B-innstruce
a- 1ama//
DeepSeek-Ree Ri-cck--R-32B--- Q4 32 billion parameter model from DeepSeek based on Qwen 32B and distilled from DeepSeek-R1.
Mhtops://hogginggfcee co. co/meta 1ama/ 16
chttpe://huggingfcccccc///e aama/
1mm-3.1-70b-instruct
Phttps://// com http://l.aa..cmm///mmrrrreeeeee 32b
/ettps://hugiinggfaaecoo
quen2.
5-14b-instruch
30

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
g= 2n - 4.
UTI
Heart Diseuse
0.7
0.7
Rogio MSE
Accurccy
Accuracy
0.6
0.5
0.5
10
20
10
10
20
30
10
20
40
Number of Iraining examples
Number of training excmppes
N Number of traiuing exampiee
Diabetes
Hypothyzoid
Breast Caneer
0.9
0.9
1og%o MSI.
Aoruruy
Accuracy
0.8
0.7
1.0
0.7
10
20
30
10
10
20
30
10
10
20
30
40
Number truining exammpee
Number training examples
Number of training excamples
Unidormative prior
GPT-4 turbo prior
Quen 2.5 14B price
Clama 3.1 SB prior
GPT-3.5 turbo prior
DeepSed-R1-SP prior
Llama 3.2 3B prioe
Llama 3.1 70B BFIG prior
Figure 21. Test accuracy the posterior distribution for open-source language models. Here, we show the results when using priors elicited from two of the Llama language models and an uninformative prior N (0 1) on a linear model. These results are calculated on a test set the datasets after each model is trained on the given number data points. The values are the result of 5000 sampled parameters for each of 5 chains, and 10 random splits of the training and testing data. For the regression tasks, we report the mean squared error on a log scale. This figure shows the line plot of the average mean posterior accuracy or mean squared error over the 10 splits, with the error bars representing the 95% confidence interval. The green arrows point in the direction of improvement in the metric.
The experiments presented in this section use the same task descriptions as given GPT-3.5-turbo, GPT-4-turbo and SrepScek-R1-322 1-32B-Q4. When sampling the posterior with the elicited priors, we use the same methods and options as given in Appendix A.7.3.
more competitive, some- times providing priors that allow for better posterior distribu- tions than GPT-3.5-turbo, but other occasions, providing priors with posterior performance similar to the uninfor- mative prior. On the UTI and Heart Disease datasets, the prior distributions elicited from 1lamm-33.1-70B-11 BF16 provide better posterior accuracy than those from GPT-3.5-turbe and comparable values to GPT-4 turbo or DeepSeek-R1. R--22----3---444 Surprisingly, on Heart Disease pre- diction in particular, priors from the 70B Llama model pro- vided a posterior accuracy greater than GPT-4-turbo and equal to DeepSeck-RR -i--becc--I-3B--444 Additionally, when pre- dicting Breast Cancer, priors from 1iama-3.1-700-Inssti BF16 achieved the same posterior accuracy as GPT-3.5- turbo for all of the training sizes tested. These three datasets suggest that Llama-3. model for eliciting prior distributions for predictive tasks.
Llama-3. -lama-3.1-70B--
Figure 21 illustrates the inconsistent posterior accuracy of the priors elicited from the Llama family of models. The 3B and 8B variants are significantly smaller in capacity to those in the main paper, justifying their inconsistent elicited priors. However, in some of the datasets, they still produce good posterior performance.
Specifically for Breast Cancer, the 3B parameter Llama model provides an intermediate accuracy between an unin- formative prior and those elicited from GPT-3.5-turbo, and for the synthetic and Hypothyroid tasks, the 8B parameter Llama provides priors of equal quality to GPT-3.5-ture -..tbb... However, for the UTT and Heart Disease datasets, both the 3B and 8B Llama models provide prior distributions that are of worse quality than even the uninformative prior. It is possibly not surprising that Llama-3.2-3B Instructt and Llama-3.1 Instructt provided priors that were equal or worse than Llama-3.1 for the synthetic task, since these models are significantly smaller in size.
can be a useful
Qwen-2.5- 1-t---.5--1-BB-sse 1nB-trtttc produces priors with similar incon- sistent posterior performance. For Heart Disease and Hy- pothyroid prediction, it achieves a competitive posterior accuracy, which is surprising given its size. For the UTI task, after seeing 10 demonstrations, it performs comparably to Llama-3. L.ama-3.1-70B-Inst 1-70B-Inntrrutt-FF16 and GPT-4-turbo, and bet- ter than GPT-3.5-turbe However, on the Breast Cancer and synthetic tasks, it performs worse than GPP-3355-turbe but is
in all cases except
31

---

AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
likely smaller in capacity. Overall, 1reen2..5-44-Instruuu provides inconsistent performance across tasks, but has the benefit of performing similarly to 1aama-.11-70B-nnss BF16 on many tasks with significantly smaller compute requirement.
The fact that DeepSeck-R1-32B-4 size between Qwen-2.5-14B Instrrct and Llama-3.1-70B- Instruct-BF16, and provides considerably better perfor- mance with higher quantisation is testament to the power of recent advances in reasoning (Huang & Chang, 2023) and test-time scaling (Jones, 2021).
an intermediate
Therefore, it is important, given an elicited prior distribu- tion, to calculate the prior likelihood with some already collected data to understand which prior is most appropriate for constructing predictive model.
A.17. Choosing Between Priors
As seen in Appendices A.16.1 and A.16.2, the prior dis- tributions elicited from language models do not always provide the best posterior performance. We consistently find that B---Seek-R1-32B-- turbo provide better prior distributions than an uninforma- tive prior; however, this may not be the case for all datasets or language models. Therefore, given prior distributions elicited from different language models, we would like to choose the best one for our predictive task.
GPT-3.5-turbo and GPT-4-
For this, given some collected data, factor (Section 4.3) model selection technique to de- cide between the prior distributions elicited from different language models. Alternatively, we could use other model selection techniques, such as the Bayesian Information Cri- terion.
use the Bayes
Further, by using temperature parameter, it is also possi- ble to control the strength of the elicited prior, which can indicate our trust in the distribution.
32


Reviews:
[{"review_id": "I7rpfERJ6M", "reviewer": "Reviewer_xAxP", "summary": "The paper considers a prior elicitation method based on query large language models, rather than human experts, for the purposes of fitting Bayesian linear models. They make a distinction between the explicitly elicited priors supplied by their method, and the implicit priors used by the LLM when doing in-context learning. They have a number of insights related to how LLM do not exhibit consistent Bayesian reasoning, and the how expliticly elicited priors can lead to improved sample efficiency outperform in-context learning.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for their comments and constructive feedback to improve our work. We are happy the work was found to be well-written, easy to follow, and well motivated. We are pleased that the use of strong experiments, evaluation of LLM memorisation, and use of our privately collected clinical dataset were appreciated. \n\nIn our rebuttals, alongside our point-by-point responses, we present **three new experiments**.\n\n### **Suggested improvements**\n\n**Q1: Methodological comparison to Selby et al. (2024)**\n\nThank you for your question. We list key differences between our work and Selby et al. (2024) (cited), highlighting where the novelty of our approach lies. We will improve our description of this work in the final version of the manuscript:\n- Selby et al.’s core focus is on data imputation; however, they show preliminary results for eliciting distributions and compare these to human experts. \n- They use these in a single predictive task. However, their method elicits a prior *over the targets* directly, much like with in-context learning, and not *over the parameters* of a model as in our work. This means that Selby et al. do not update a model using available data and are therefore not training any predictive models using the priors. This also makes Selby et al.’s work incompatible as a baseline. \n- Selby et al. do not perform repeated sampling of the LLM, which we hypothesise mitigates hallucination.\n- Selby et al. do not combine experts and LLMs, an important aspect of human-AI interaction. We achieve this by allowing experts to provide natural language information in task descriptions, simplifying current prior elicitation methods. In this case, our elicited distributions contain knowledge from both the experts and the LLM’s training.\n- We also present a grounded model selection criterion to decide between AutoElicit and in-context learning.\n\n\n**Q2: *New experiment* with Gouk & Gao (2024) as a baseline**\n\nA more compatible baseline presented by Gouk & Gao (2024) (cited) generates synthetic data using an LLM to provide an additional likelihood term when training linear *classification* models. This method uses an LLM to generate feature values before labelling them with zero-shot in-context learning. These generated samples are provided alongside real data during the training of a linear predictive model. Therefore, Gouk & Gao’s method is susceptible to LLM data memorisation, putting it at risk of artificially providing improved posterior results on public datasets.\n\nWe compare our results to this baseline and extend their ideas to *regression* tasks to compare with all the datasets we test. This previous work was described in our manuscript but we now provide new experimental results. The figure (linked below) will be included in the final manuscript:\n\n**FIGURE:** https://imgur.com/a/3sM2baQ\n\nThis figure is split by publicly available tasks (four right-most plots) and privately available tasks (two left-most plots). In these experiments, our approach provides significantly greater posterior accuracy and lower posterior mean squared error for both private datasets.\n\nIn the public tasks, we see that the performance of Gouk & Gao’s approach varies considerably by LLM and task. It underperforms compared to the *uninformative* prior on five occasions. On the two occasions it outperforms AutoElicit (ours), it achieves surprisingly good accuracy or mean squared error (Heart Disease with GPT-4-turbo and Diabetes with GPT-3.5-turbo), suggesting that it might be reproducing parts of the public datasets rather than generating new samples. Given our analysis in Appendix A.8 showing the memorisation ability of LLMs, and Gouk and Gao’s performance on private data, we suspect the performance of this approach is artificially improved by reproducing the true dataset. Since for AutoElicit, we prompt the language model for a prior distribution over the parameters of a predictive model, using only the feature names and a description of the dataset, the memorisation of data points is unlikely to have had an impact on our method's results (discussed further in Appendix A.8.1).\n\nIn both of the private datasets (two left-most plots), where there is no risk of LLM memorisation, AutoElicit results in improved posterior accuracy and mean squared error. On the synthetic task, our improvement is orders of magnitudes better, and on the UTI task, our method significantly increases accuracy at all training sizes. In both cases, the approach proposed by Gouk & Gao underperforms compared to an *uninformative* prior.\n\nWe will include these new results in the final version of the paper, as it provides further context to our approach.\n\n**Thank you for taking the time to review our work. We will implement the suggested improvements to the final manuscript. If we have answered your questions, then we would appreciate you considering raising your score. If anything is still unclear, we are happy to discuss further.**", "claims_and_evidence": "The paper makes three main claims:\n* The introduction of a new algorithm for eliciting priors over linear models from LLMs, which leads to better loss values---particularly with small training sets. This is substantiated by experimental results. However, I would note that the algorithm they present is essentially the same as the one proposed in Selby et al. (2024), but applied to a different set of modelling problems.\n* When comparing with in-context learning, it is found that LLMs inconsistently approximate Bayesian inference, and that their implicit priors are less informative than those that are explicitly elicited.\n* Bayes factors are claimed to be a useful tool for comparing ICL and linear models with LLM-based priors. This is validated experimentally.", "methods_and_evaluation_criteria": "The method and evaluation criteria mostly make sense: using LLMs as a generic knowledge base has ample motivation, and model selection based on Bayes factors is well-established.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "The main issue I can see with the experimental evaluation is the lack of comparison with the very related method of Gouk & Gao (2024), who also consider eliciting priors for linear models from LLMs. Other than that, the experiments are quite well though out. The use of methods to determine leakage of datasets in the LLM pre-training, and also a private dataset, is a particularly positive aspect of the setup.", "relation_to_broader_scientific_literature": "Compared to the two most related previous pieces of work in this area (Gouk & Gao (2024) and Selby et al. (2024)), this work provides more analysis of how the explicitly elicited priors compare to the implicit priors used during in-context learning.\n\nOne aspect that I do not think is sufficiently highlighted in the paper is that the proposed method is essentially the same as Selby et al. (2024). The difference between this previous work and the current paper lies only in the types of tasks used during evaluation---Selby et al. consider imputation and data generation problems, rather than linear modelling problems.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3UQ2jgyfZL", "reviewer": "Reviewer_3Pnd", "summary": "Linear predictive models remain valuable for applied researchers. However, principled Bayesian approaches to such models require the specification of some prior. What prior should researchers use for different parameters? Good priors may be informed by expert information, but such expert knowledge can be hard to come by. The authors explore the possibility of using language models to elicit such priors. They develop a new method, AutoElicit, to automatically extract priors from a range of LLMs and compare against direct prediction and ICL elicitation from models. Generally, they find that the priors elicited from their method yield better downstream predictive performance on a range of medical tasks (and one synthetic predictive task).", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's kind words and time. We are pleased that our work was considered well-written with thorough experiments and appendices. We are happy that the extensive literature review, discussion of elicitation costs, and LLM memorisation experiments were appreciated.\n\nIn our rebuttals, alongside our point-by-point responses, we present **three new experiments**.\n\n### **Suggested improvements**\n\n**Q1: Further comparisons to a baseline**\n\nZhu & Griffiths (2024) is not directly comparable to our work, as they do not provide a framework for eliciting the parameters of a predictive model. The most comparable work, Gouk & Gao (2024) (cited), generates synthetic data using an LLM to include in the training of linear *classification* models. To compare with our method, we extend their ideas to *regression* tasks and present new experimental results. \n\nThese are presented in our response to **Reviewer 1 (xAxP) in section Q2**, along with a detailed description of the baseline. In particular, Gouk & Gao’s method is susceptible to LLM data memorisation, putting it at risk of artificially providing improved posterior results on public datasets. To better compare the methods, we have split the results by the private and public tasks. We find that our approach yields improved posterior accuracy and mean squared error over Gouk & Gao’s method. \n\n**Q2: Using more challenging expert information**\n \nThis is an interesting point. A synthetic example of an LLM using completely new expert information is given in the last paragraph of Appendix A.11 and Figure 13 (linked below).\n\n**FIGURE:** https://imgur.com/a/QSZl2ZO\n\nThis demonstrates that giving increasingly more detailed information about the task (provided as natural language, Appendix A.11) allowed the LLM to update the elicited distributions correctly. We will lengthen the discussion in Section 5.3 and Appendix A.11 of the final paper to include this point.\n \n**Q3: *New experiment* on the variation of the task descriptions and number of mixture components**\n \nWe will include examples of rephrased task descriptions in the appendix. We used a temperature of 0.1 to elicit the prior distributions so that the LLM reliably generated parameters of a Gaussian distribution and to mitigate the effects of hallucination, as we focus on healthcare tasks. When rephrasing the task descriptions, we used a higher temperature of 1.0 to get diverse descriptions. This is why we see variations in the priors elicited. An example rephrased system role is:\n\n> You are a simulator of a logistic regression predictive model for … Here the inputs are values from sensors around a home and the output is the probability of the presence of a urinary tract infection. Specifically, the targets are …\n\nTo:\n\n> You function as a logistic regression prediction model for … Inputs include sensor readings from a home, and the output is the probability of a urinary tract infection. The targets are …\n\nThe meaning of the description remains the same, but the rephrasing mitigates against changes in the priors elicited based only on the language used in the prompt.\n\nTo supplement this, we calculate the posterior performance with varied numbers of mixture components in our prior (linked below): \n\n**FIGURE:** https://imgur.com/a/E3hE9Na\n \nWe see little variation (except for Breast Cancer with Deepseek) in the accuracy or mean squared error as we increase the number of components in the prior. This suggests our framework is robust to the number of task descriptions. We hypothesise that a greater number of mixture components makes the framework more resilient to LLMs that frequently hallucinate, especially on tasks with large numbers of features (such as Breast Cancer).\n\nIn Appendix A.12, we showed that the density of the priors changed with the number of mixture components, however, this has small effects on the posterior performance. This is likely because the sample space that provides good posterior performance is covered by the majority of elicited components, whilst the large changes in Appendix A.12 are due to more unique components.\n\n**Q4: Discussion on the interpretability of priors**\n\nIn our response to **Reviewer 3 (fJAR) in section Q2**, we provide examples of interpreting the priors elicited from GPT-3.5-turbo. As suggested, it would be interesting to explore how these priors are understood by clinicians and if they are used separately from the predictive modelling. The points you have raised are valuable. This is future work we are currently organising with clinicians.\n\n**Related works**\n\nThank you for the helpful suggestions. We will include these references with discussions in the final version.\n\n**Thank you for taking the time to review our work. We will implement the suggested improvements to the final manuscript. If we have answered your questions, then we would appreciate you considering raising your score. If anything is still unclear, we are happy to discuss further.**", "claims_and_evidence": "The paper is very well-written and thorough. I particularly appreciate the authors' extensive Appendix. They clearly ran a wide range of experiments that probe the characteristics of AutoElicit against ICL. I particularly appreciate the authors' exploration of other LLMs --- their work raises interesting questions about which model to use, when, based on costs of elicitation --- as well as their analysis of the potential impact of memorization on results. \n\nWhat I was less clear about, though, is:\n- (1) Where AutoElicit stands in relation to other prior elicitation methods in terms of performance\n- (2) The value / ease of incorporating expert information\n- (3) The role of the diversity of task descriptions\n\nOn (1), while I admire the authors' extensive literature review, I was a bit disappointed to not see direct head-to-head comparisons with some alternate elicitation methods. For instance, the authors mention Zhu and Griffiths --- how does such a method compare on something like the UTI dataset? It would be good to be clearer on why alternate methods were not considered in the empirical section? \n\nFor (2), the authors claim that their method is a way to also incorporate expert knowledge into prior elicitation. However, the UTI example they gave -- to my sense? -- had more \"commonsense medical knowledge\" that was added (e.g., UTIs lead to more urination at night). I think it makes sense that GPT-3.5 wasn't really impacted by this; I wouldn't say that's real \"expert\" knowledge per say? Can the authors sway parameters even more with genuine, nice expert knowledge? This need not be in this current submission in my opinion, but is something I walked away curious about and did not feel the current paper really demonstrated. \n\nAnd on (3), I was quite confused on the task description resampling. Based on the Appendix, the number seems to make quite a big difference! But what is the character of these different descriptions? Can you add some examples to the Appendix? How different are they really? Or is the \"boost\" from just resampling?", "methods_and_evaluation_criteria": "Generally I think the evaluation was sound, however,  as noted above, I do not completely understand why the number of task descriptions has such a big role (Fig 14). Is this really from the task descriptions, or something else? I see the authors set the temperature to 0.1? Why generally so low, if part of the role is to get diversity in parameters? \n\nI also came away with some confusion over whether the priors were semantically sensible. For instance, in Fig 15, some parameters' priors shifted around quite a bit -- others not. Can the authors add more qualitative analyses into what changed and whether it is semantically-relevant to the task at hand?", "theoretical_claims": "I believe the theory is appropriate but am not 100% confident in my assessment.", "experimental_designs_or_analyses": "As noted above, I think the authors did a very thorough job in a range of experiments -- however, I am particularly curious about the role of the task descriptions versus potentially", "relation_to_broader_scientific_literature": "I think the authors did a generally good job of situating their work in relation to other literature. I think the experiments around \"do LLMs do Bayesian inference\" though did not provide much substantial value ontop of what's already been looked at in the literature. I'd encourage the authors to focus principally on the role of prior elicitation, which seems to be the more novel direction here.\n\nI would have liked to see though more empirical comparison to at least one alternate method for prior elicitation from LLMs. How does AutoElicit compare?", "essential_references_not_discussed": "I would encourage the authors to look at: \n\nLarge Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning https://proceedings.neurips.cc/paper_files/paper/2023/hash/3255a7554605a88800f4e120b3a929e1-Abstract-Conference.html -- looks at LLMs and ICL from a Bayesian lense\n\nAutomated Statistical Model Discovery with Language Models https://arxiv.org/abs/2402.17879 which also looked at the use of LLMs in more classical statistical modeling \n\nMPP: Language Models as Probabilistic Priors for Perception and Action https://arxiv.org/pdf/2302.02801 --- an early work using LLMs as priors (though not for linear modeling problems as these authors consider).", "other_strengths_and_weaknesses": "I greatly admire the authors' effort to make the code easily runnable (A.1) and their detailed description of how to run the method (A.7.6). I think this will be valuable to the broader community. Well done! \n\nFor future (not here), I would be keen to see how actual experiments -- e.g., doctors from the UTI study? -- interpret the parameters and assess the value of AutoElicit.", "other_comments_or_suggestions": "Fig 17 says \"OpenAI models\" but includes DeepSeek?", "questions_for_authors": "I raised my questions above --- particularly around the task descriptions and some deeper parameter analyses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8CiDUOBcph", "reviewer": "Reviewer_fJAR", "summary": "This work proposes a framework to elicit prior distributions over parameters from LLMs, and use these in connection with linear models, with the goal to achieve transparent and interpretable, yet performant models.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewers for their feedback and time. We are happy that the work is considered well-written, with strong experiments, clear and informative figures, and a detailed appendix. We are pleased that the evaluation on a private dataset is appreciated and provides strong motivation for the work and that the inclusion of expert information is interesting.\n\nIn our rebuttals, alongside our point-by-point responses, we present **three new experiments**.\n\n### **Suggested improvements**\n\n**Q1: Discussion of related works in LLMs from a Bayesian perspective**\n\nAs you note, numerous works discuss in-context learning from a Bayesian perspective. Our work is unique in that we empirically estimate the internal prior and posterior distributions of an in-context learner. With this, we use MCMC methods to compare the in-context posterior with a separately calculated posterior. This allows us to question whether an LLM is performing Bayesian inference on *linear modelling of real-world numerical tasks*. This is more relevant to our work than the previous natural language or theoretical perspectives. This contribution also provides a complete decision criterion to determine whether in-context learning or AutoElicit is more performant for a task.\n\nWe agree that this is an important area of the literature. We were restricted by the page length requirements of the submission and will include a complete description in the main section of the final paper with your suggestions.\n\n**Q2: Discussion on the interpretability of the priors**\n \nWe use 100 mixture components in our prior to mitigate the effects of LLM hallucination, which is important in healthcare tasks. The prior distribution can be understood by visualising histograms of the parameter samples from the final mixture (linked below in a modified Figure 15):\n\n**FIGURE:** https://imgur.com/a/gRRVmUz\n\nAs the density of these distributions are used for the prior directly, this provides a complete picture. For linear models, each feature's prior corresponds to its direction and importance when predicting the target. On the synthetic task (row 1), the prior distributions concentrate on the parameters that we expect. For the UTI task (row 2), the LLM provided prior distributions that suggest Feature 2 was negatively predictive of a positive UTI, whilst Feature 10 is a strong positive predictor. For Breast Cancer (row 6), the LLM produced two modes in its Feature 2 prior. Here, the LLM was either unsure of its belief, and so the elicited prior changed between task descriptions, or the LLM was unsure of the feature’s direction of correlation *but guessed that the feature was important*.\n\nA chain-of-thought LLM like Deepseek-R1 could help us to better understand the elicitation process. We also found that using LLMs for predictions directly (in-context learning) underperforms in comparison to our approach.\n\n**Q3: Discussion on the number of mixture components**\n\nWe investigate, in **Rebbutal 2 for Reviewer 2 (3Pnd), in section Q3**, how the posterior performance depends on the number of prior components. Here, we significantly reduce the number of mixture components whilst retaining the posterior accuracy and mean squared error.\n\n**Q4: *New experiment* evaluating an uninformative mixture of random Gaussians**\n\nBased on your suggestion, we compare our results to a new uninformative baseline. We sample 100 values from a standard normal distribution ($\\mu \\sim N(\\mu | 0,1)$), and we use these as means of 100 Gaussian distributions ($\\theta \\sim N(\\theta | \\mu, 1)$). With these distributions we construct a mixture prior. As we keep the standard deviation at 1 for each component, the resulting prior has a greater range in the parameter space. The figure linked below shows the results:\n\n**FIGURE:** https://imgur.com/a/STNjyEG\n\nThis new uninformative prior leads to no noticeable difference in the results for the synthetic task, Diabetes task, and Breast Cancer task. However, on the UTI task, we see marginally better accuracy than the previous uninformative prior, and on the Heart Disease and Hypothyroid tasks, we see greater improvements. Despite this, DeepSeek-R1-32B-Q4 or GPT-3.5-turbo continue to provide more informative priors. Therefore, the posterior improvement is not due to the complexity of the prior.\n\n**Q5: Clarity on the predictive performance**\n\nPredictive performance refers to either the posterior accuracy or mean squared error. To calculate this, we sample 5 chains of 5000 posterior parameter samples using MCMC methods. We use these sampled posterior parameters to predict the labels on our test set and calculate the metrics. This provides a distribution of performance.\n\n**Thank you for taking the time to review our work. We will implement the suggested improvements to the final manuscript. If we have answered your questions, then we would appreciate you considering raising your score. If anything is still unclear, we are happy to discuss further.**", "claims_and_evidence": "The paper is well written and relatively easy to follow.", "methods_and_evaluation_criteria": "The application settings range from synthetic to real-world clinical (private) dataset, which I very much appreciate. This renders the experimental setup interesting and relevant.", "theoretical_claims": "Major weakness:\n\nMy main point of criticism is on the loss of *transparency/interpretability* of the proposed approach: *Context*: The auto-elicited prior is constructed as a Mixture of Gaussians (see Eq. 2). In the experiments, the authors choose K=100 components, and in App. 12 the authors show that this very complex prior is necessary to achieve strong predictive performance.  *Criticism* Is it desirable to construct such complex prior distributions with LLMs and then apply them to linear models with the goal of maintaining interpretability? The goal of using a linear model is two-fold: 1) the functional relationship is simple and ‘interpretable’ (this is maintained with the proposed approach) 2) the weights are fitted by a simple least-squares error procedure (or a similarly simplistic and transparent procedure here in the Bayesian case). If the prior now is extremely complicated, both because it contains knowledge of the LLM and it is a 100-component MoG, we lose all transparency of the predictive weights and point 2) is lost/’violated’. Because of this, I cannot see a clinician trusting the model as an “interpretable”/simple model anymore. We have---through the rules of Bayesian Inference---made our linear model a linear model augmented with an LLM. If we wanted to improve the performance of the a (linear) model, we could use an LLM directly, or use another model. -- Am I missing something in terms of interpretability? I think it’s important to discuss this point in the rebuttal; at this point, it drastically diminishes the significance of the work for me. -- Also, I would like to note that the general procedure of “eliciting priors from LLMs” remains interesting (though has been done in related work), it is only the use for predictive purposes in the context of trying to maintain the interpretability of linear models that I find challenging.\n\nI would like to discuss this point during the author rebuttal and will revisit my decision based on this discussion.", "experimental_designs_or_analyses": "The experiments are strong, I particularly like the idea of including expert information in addition. The figures are extensive, informative and clear.\n\nThe basic setup and notation are simple, yet somewhat hard to follow. An example explaining important concepts such as task description $I_k$ (e.g. App. 7.3) clearer.", "relation_to_broader_scientific_literature": "N/A", "essential_references_not_discussed": "In Sec. 5.6 the authors study the question whether in-context learning in LLMs is Bayesian. The authors also mention this as one of the three contributions in Sec. 1. However, this question has been studied already. Some of these papers you have cited in App. 2.2. (though not in the main text, why?). It may be worth adding “Are Large Language Models Bayesian? A Martingale Perspective on In-Context Learning” (ICML 2024).", "other_strengths_and_weaknesses": "- The appendix is detailed, and very helpful in augmenting the main text.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "- Have you tried other uninformative priors than N(0,I), even a higher variance for instance? Given you use a K=100 MoG, a natural idea might be to try a MoG prior possibly with randomised parameters. -- What I would be interested in disentangling is the question whether the complexity of the prior (MoG vs. single Gaussian in the uninformative case) or the “correctness” of the LLM prior is crucial for performance.\n    \n- Can you state how you compute predictive performance? What’s the predictive distribution you use / can you point me to the equation?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6MKc7otfgH", "reviewer": "Reviewer_F25r", "summary": "This paper explores the potential of LLMs to elicit prior distributions, aiming to reduce sample complexity in Bayesian inference, particularly in data-scarce healthcare domains. The authors compare two LLM-based prior elicitation methods: their proposed AutoElicit and In-Context Prior. AutoElicit directly prompts the LLM to provide estimated means and variances for Gaussian priors in text. In contrast, In-Context Prior elicits posterior predictive distributions from the LLM, from which implicit priors are inferred using maximum likelihood. The authors demonstrate that both methods can be beneficial across various tasks, with AutoElicit generally yielding superior priors. Notably, the paper shows clear improvements in inference when leveraging LLM-elicited priors compared to uninformative priors. Furthermore, the methodology facilitates the incorporation of expert knowledge expressed in natural language during the prior elicitation process.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for their time and comments to improve our paper. We are pleased that they felt that this was commendable work that is well-written, easy to follow, and highly relevant for healthcare. We are glad that the methods were considered straight-forward, but effective, and that reviewer felt that the extensive experiments demonstrated the utility of our approach.\n\nIn our rebuttals, alongside our point-by-point responses, we present **three new experiments**.\n\n### **Suggested improvements**\n\n**Q1: A potential explanation for the LLM's success**\n\nWe believe our method’s success is due to the LLM’s ability to translate information across domains. LLMs can effectively answer questions in medical domains and probability theory. They offer the ability to translate expert information in the form of natural language into probability distributions that represent prior beliefs on model parameters. We hypothesise that resampling the LLM multiple times mitigates the effects of hallucination for LLMs that are more likely to fabricate information. By combining many Gaussian components, we can approximate an LLM’s non-Gaussian prior. For our method to be successful, the LLM needs to have an understanding of the problem domain, Gaussian distributions, and linear models. We believe that this is why we see improved performance from chain-of-thought models, which can reason about the impact of each feature on an outcome. We will include this discussion in the final version of the work.\n\n**Q2: Further comparisons to a baseline**\n\nZhu & Griffiths (2024) is not directly comparable to our work, as they do not provide a framework for eliciting the parameters of a predictive model. The most comparable work, Gouk & Gao (2024) (cited), generates synthetic data using an LLM to include in the training of linear *classification* models. To compare with our method, we extend their ideas to *regression* tasks and present new experimental results.\n\nThese are presented in our response to **Reviewer 1 (xAxP) in section Q2**, along with a detailed description of the baseline. In particular, Gouk & Gao’s method is susceptible to LLM data memorisation, putting it at risk of artificially providing improved posterior results on public datasets. To better compare the methods, we have split the results by the private and public tasks. We find that our approach yields improved posterior accuracy and mean squared error over Gouk & Gao’s method. \n\n**Q3: Add ethical approval statement**\n\nWe have already obtained the appropriate ethics for our study but wanted to meet the double-blind requirement by not providing exact information on the ethics approval, including the identification number for our study. We include this information in the final version, alongside a description of the recruitment process. In the ethics statement, we have redacted the appropriate information to prevent breaking the double-blind requirements:\n\n> This study was performed in collaboration with INSTITUTION. Participants were recruited from the following: (1) health and social care partners within the INSTITUTION, (2) urgent and acute care services within the INSTITUTION, (3) social services who oversee sheltered and extra care sheltered housing schemes, (4) INSTITUTION and (5) specialist memory services at INSTITUTION. All participants provided written informed consent. Capacity to consent was assessed according to Good Clinical Practice, as detailed in the FRAMEWORK and the ACT. Participants were provided with a Participant Information Sheet (PIS) that includes information on how the study used their personal data collected in accordance with the ACT requirements. If the participant was deemed to lack capacity, a personal or professional consultee was sought to provide written consent to the study. Additionally, the capacity of both the participant and study partner is assessed at each research visit. Research staff conducting the assessment have completed the COURSE training and COURSE training. If a participant is deemed to lack capacity but is willing to take part in the research, a personal consultee is sought in the first instance to sign a declaration of consent. If no personal consultee can be found, a professional consultee, such as a key worker, is sought. This process is included in the study protocol, and ethical panel approval is obtained.\n\nWe feel the use of privately collected data from a real-world healthcare problem provides substantial evidence for the effectiveness of our framework, and we are indebted to our study participants.\n\n**Thank you for taking the time to review our work. We will implement the suggested improvements to the final manuscript. If we have answered your questions, then we would appreciate you considering raising your score. If anything is still unclear, we are happy to discuss further.**", "claims_and_evidence": "This paper presents commendable work, offering straightforward and effective methods. The manuscript is well-written and easy to follow. The application to healthcare domains is highly relevant, and the extensive experiments demonstrate the utility of LLM-elicited priors.\n\nConsidering the significance of the healthcare application, a deeper mechanistic understanding of the LLM's reasoning in generating these effective priors would be valuable. While space limitations might preclude a full mechanistic analysis, a discussion exploring potential explanations for the LLM's success in this context would strengthen the paper.", "methods_and_evaluation_criteria": "The authors tested the LLM-elicited priors (along with uninformative priors) on synthetic data and healthcare datasets. I find their evaluation and methods adequate.", "theoretical_claims": "No theoretical claim was made in this paper.", "experimental_designs_or_analyses": "I have reviewed all their experiments.", "relation_to_broader_scientific_literature": "This work is also related to other work that tries to link LLM (and in-context learning) and Bayesian inference.", "essential_references_not_discussed": "I don't see any other essential reference that should be included.", "other_strengths_and_weaknesses": "None.", "other_comments_or_suggestions": "None.", "questions_for_authors": "Have you tried eliciting priors using the iterated in-context learning proposed in Zhu & Griffiths (2024)? How would the priors elicited using this approach compare to those elicited using AutoElicit and In-Context Priors?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_flag": "Flag this paper for an ethics review.", "ethics_expertise_needed": ["Privacy and Security", "Responsible Research Practice (e.g., IRB, documentation, research ethics, participant consent)"]}}]

Author rebuttal:

Meta-review: This paper outlines a method for using LLMs to elicit Bayesian priors for linear models. While building on previous work, the reviewers consider AutoElicit to make a nice contribution to the area, with a compelling presentation, and strong empirical results. The author rebuttal cleared up a number of questions raised by the reviewers, and the inclusion of new empirical results (placed nicely in context) makes the paper stronger. That said, one reviewer remains somewhat concerned about the question of interpretability, and the authors response was, in my opinion, somewhat weak on this score. Still, the overall contribution is solid. 

The reviewers had a number of valuable suggestions on the paper, which I strongly encourage the authors to address in any revision of the paper to solidify its contribution—this includes ensuring the material contained in the rebuttal is incorporated (e.g., new experiments).
Decision: Accept (poster)