Calibrating LLMs with Information-Theoretic Evidential Deep Learning
Authors: Yawei Li
David Rügamer
Bernd Bischl
Mina Rezaei
Venue: ICLR 2025 Poster (2025)
Keywords: evidential deep learning; information bottleneck; calibration; large language models
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
TL;DR: This paper introduces an information-theoretic regularization for evidential deep learning, substantially enhancing the calibration of fine-tuned large language models.

Abstract: Fine-tuned large language models (LLMs) often exhibit overconfidence, particularly when trained on small datasets, resulting in poor calibration and inaccurate uncertainty estimates. 
Evidential Deep Learning (EDL), an uncertainty-aware approach, enables uncertainty estimation in a single forward pass, making it a promising method for calibrating fine-tuned LLMs. However, despite its computational efficiency, EDL is prone to overfitting, as its training objective can result in overly concentrated probability distributions. 
To mitigate this, we propose regularizing EDL by incorporating an information bottleneck (IB). Our approach IB-EDL suppresses spurious information in the evidence generated by the model and encourages truly predictive information to influence both the predictions and uncertainty estimates. Extensive experiments across various fine-tuned LLMs and tasks demonstrate that IB-EDL outperforms both existing EDL and non-EDL approaches. 
By improving the trustworthiness of LLMs, IB-EDL facilitates their broader adoption in domains requiring high levels of confidence calibration.

Published as a conference paper at ICLR 2025
CALIBRATING LLMs WITH INFORMATION- THEORETIC EVIDENTIAL DEEP LEARNING
Yawei Li David Rügamer Bernd Bischl
Mina Rezaei
Department of Statistics, LMU Munich Munich Center for Machine Learning (MCMI.)
ABSTRACT
Fine-tuned large language models (LLMs) often exhibit overconfidence, particu- larly when trained on small datasets, resulting in poor calibration and inaccurate uncertainty estimates. Evidential Deep Learning (EDL), an uncertainty- aware approach, enables uncertainty estimation in single forward pass, making it a promising method for calibrating fine-tuned LLMs. However, despite its compu- tational efficiency, EDL is prone to overfitting, as its training objective can re- sult in overly concentrated probability distributions. To mitigate this, we propose regularizing EDL by incorporating an information bottleneck (IB). Our approach IB-EDL suppresses spurious information in the evidence generated by the model and encourages truly predictive information to influence both the predictions and uncertainty estimates. Extensive experiments across various fine-tuned LLMs and tasks demonstrate that IB-EDL outperforms both existing EDL and non-EDL ap- proaches. By improving the trustworthiness of LLMs, IB-EDL facilitates their broader adoption in domains requiring high levels of confidence calibration. Code is available htttps: http///gtttt..t.tt com/ cowm//aaddyaakkr/b-eeelll
INTRODUCTION
Large language models (LLMs) have revolutionized natural language processing. with fine-tuning emerging prevalent method to adapt these models for specific tasks or domains (Houlsby et al.) 2019 Hu et al.) 2022). However, fine-tuned LLMS often display overconfidence in their predic- tions (Jiang et al.) 2021 Yang et al. 2024), which compromises their reliability and limits their applicability in critical domains where trustworthiness is essential.
Overconfidence in LLMs often manifests poor calibration, where the predicted probabilities do not accurately reflect the model's uncertainty about its predictions. Uncertainty-avee aware methods im- prove calibration by explicitly quantifying the uncertainty in the model's predictions, allowing the model to produce confidence scores that better correspond to the actual likelihood of correctness. Traditional uncertainty-awaar methods, such MC-Dropout (Gal & Ghahramani] (2066) and Deep Ensemble [Lakshminarayanan et al. 2017 et al., 2019) are commonly used to mitigate overcon- fidence in neural networks. However, these approaches typically require multiple forward passes, significantly increasing the inference time for LLMs.
Evidential Deep Learning (EDL) (Sensoy
2018 Malinin & Gales 2018) offers more efficient
alternative by providing uncertainty estimates with single forward pass. Despite its success in various tasks, recent studies (Deng al., [2023] Chen al. 2024) indicate that EDL can still yield overconfident predictions which leads to inaccurate uncertainty estimates, degrading the model's calibration performance. This issue arises from the propensity of vanilla EDI. to encourage models to generate excessive evidence (i.c., support for class) with extremely large magnitudes, leading to overly confident predicted class probabilitics.
Motivated by these challenges, we propose novel regularization approach for EDL using infor- mation bottleneck, which we term IB-EDL. IB-EDL adaptively distorts the evidence generated by the LLM while maximally preserving the model's 's performance. In doing so, IB-EDL encourages the model to suppress spurious or uninformative evidence that could lead to overconfident predictions. Our theoretical analysis shows that the information bottleneck effectively penalizes the generation

---

Published as a conference paper at ICLR 2025
of disproportionately large evidence, thereby reducing overconfidence. Notably, method intro- duces less than 2% computational overhead compared to a pretrained LLM, maintaining the model's inference efficiency while significantly improving its calibration. Our contributions
follows:
We introduce IB-EDL, an information-theore tify theoretical issue previously overlooked in the IB literature. In the context of EDL., ad- dress this challenge through a novel choice of the IB stochastic variable. Moreover, our solution naturally imposes an & regularization, effectively mitigating the issue of overly large evidence highlighted in prior EDL research.
framework for regularizing EDL. First, we iden-
We show that several existing EDL methods can be seen a special case within the IB-EDL framework. This unification offers cohesive perspective on these approaches.
We perform extensive experiments on calibrating fine-tuned L.LMs using EDL,, thereby extending its applicability beyond the medium-sized networks commonly used in the EDL. literature. Our results across various LLMSs and datasets demonstrate that IB-EDL scales effectively.
BACKGROUND
2.1 MLE FINE-TUNING OF LLMS
Let æ e D represent the input sequence for an LLM, where V represents the set of tokens (vo- cabulary) and 5 is the sequence length. The target space is denoted by Y, which may be identical to V (e.g.. in next-token prediction) or a different set (e.g.. in sentiment analysis). We generally assume that | C. As we focus the context of LLMs, we will use the term "tokens" instead of "classes" throughout this paper. In tasks like next-token prediction, the target can also be a se- quence of tokens. For clarity in our theoretical analysis, focus single-token target, with the understanding that a token sequence can be treated as multiple single-token targets.
Let be the LLM. The output logits of the LLM. f(x) E RCC, are passed through a Softmax func- tion, yielding a vector π with entries x = exp(f(x)))/ ES= exp(f(x))) which represent the probability for each token. Let E 1] be the one-hot encoded target. Then, y/æ conforms categorical distribution p(y) x)) Cat (3/; RT) = I Fine-tuning LLMs on downstream tasks typically involves minimizing log pagg(ylaa |z)) which corresponds maximum likelihood esti- mation (MLE). However, fine-tuning LLMs on small downstream datasets can result in overfitting and overconfident predictions. Additionally, MLE yields deterministic model that cannot express uncertainty in the predicted ा.
2.2 UNCERTAINTY-WWAEE MODELING VIA EDL
While conventional uncertainty- aware methods like a Deep Ensemble can alleviate overconfidence and improve calibration, they require multiple forward passes during inference. This can be partic- ularly challenging for LLMs due to their already substantial computational demands. EDL provides more efficient alternative by capturing uncertainty in a single forward pass, making it especially suitable for large models. EDL builds on the principles of Subjective Logic (Jøsang 1997 2016), which is derived from Dempster-Shafer Theory (DST) (Dempster) 1968 Shafer 1976).
EDL inference pipeline: Instead of directly predicting the token probabilities æ, EDL uses the model to predict Dirichlet prior on π. Specifically, the model's output is interpreted as pre- evidence é = f(x) E C which is converted into a non-negative evidence vector e = SoftPlus(é) using the SoftPlus activation. Each element ej of the evidence vector represents the amount of support for token j being the correct prediction. Once the evidence is obtained, we can pro- ceed to predict the Dirichlet prior Dir (T:; a)) over the simplex of possible token probabilities ( Tc] by computing the Dirichlet parameters ai ej + 1, E [C]. More formally,
C
I((a)
p(TTaa)=
II
T a1-1 with do Σ ag
(1)
II I'(as)
j=1
where r(-) is the gamma function. The expected probabilities îty and the final predicted token ý are:
[myla]= = 0j =
+1
カ = Eir(Di(())))))
(e)(+) f= = arg max, ît.
2

---

Published as a conference paper at ICLR 2025
In summary, the EDL pipeline be symbolized as: - f(z) - +
Uncertainty estimate: EDL also enables quantifying uncertainty in the model's prediction. This is done through the concepts of belief mass bj and uncertainty mass in the Subjective Logic:
bj = (a) - 1)/a0, u = C/a0.
Similar to the evidence, the belief mass by indicates the support for token being the correct pre- diction, while the uncertainty mass captures the model's ss overall uncertainty about the prediction. The sum of all belief masses and the uncertainty mass is normalized: > by 1
2.3 TRAINING OF EDL NETWORKS
In EDL, models are usually trained by minimizing the Bayes risk, which involves the ex- pected loss under the Dirichlet distribution. Given the predicted a from an input æ, and the target y {0,1}" the Bayes risk for the cross-entropy loss is defined as Lcu(O) (x-DDi((T; Ej= Vj log(TT)) = Eド Uj (b((o)) - v(az)), where 0 denotes the trainable pa- rameters of the model, and is the digamma functtio. To stabilize the training, Sensoy et al. (2018) introduce the MSE loss alternative objective, which can be analytically computed using
a,(00 oa) oa +1)
MSEE (0) (ix ly
Ogg
For detailed derivations, refer to Sensoy al. (018)) Furthermore, they introduce reg- ularization term to suppress evidence for non-target tokens, i.e., the tokens labeled as in y. This is achieved by first "removing" the evidence associated with the target token, using â + (1 y) @ ar, where 1 = [I, [_,..,,1]] _1=". The regularization term is then defined as
RReg 0) DRI. (Dir (:;; ā) || Dir (#; 1))
(5)
where DKL. denotes the Kullback-Leibler (KI.) divergence. The total loss for training is given by:
CEDL( LMSE (0) + A (R)
where À > 0 is syper-parameee Note that LMSE (f) can be replaced with (Ce(
However, as the model trained to minimize the empirical risk, it remains susceptible to overfitting
the data and producing overconfident predictions. The objectives in LCE(O) and LMSE(0) drive the learned Dirichlet distribution towards a Dirac delta distribution. Consequently, the trained model may produce aj with extreme magnitudes for the target token j (Chen al.) [024)). Eq. (5) also does not fully address this issue, as it only suppresses the evidence of non-target tokens.
Recent efforts have sought to mitigate the overconfidence issue in EDL. For instance, I-EDL (Deng et al. 2023) incorporates the Fisher Information matrix into the distribution of y. R-EDI. (Chen
al. 2024) alleviates overconfidence by relaxing aj E; 1, Vj to a, Ej + 17. Vjj with hyper- parameter 7) E R4. Orthogonal to these approaches, we do not alter the assumptions on y or a in the EDL formulation. Instead, we impose regularization on the model using an information bottleneck (IB), which discourages the model from relying on irrelevant or spurious correlations that could lead to an overly concentrated Dirichlet distribution, thereby preventing overconfident predictions.
INFORMATION
EDL
We begin by adopting an information -hheorettc perspective on neural networks and introduce the IB objective. We then explain how to regularize EDL with IB and why the final IB- IB-EDL objective effectively mitigates overconfidence.
3.1 CAREFUL EXAMINATION OF THE INFORMATION BOTTLENECK CRITERION
A high-level view of IB: Let X be the input random variable and Y represent the random variable of the target token. We also introduce an intermediate representation Z. which serves as a stochastic encoding of X. In the context of EDL, Z can take various forms, such the internal features
3

---

Published as a conference paper at ICLR 2025
of any LLM layer, the pre-evidence è (i.e., model output), evidence Dirichlet parameters a, or token probabilities и. Choosing different forms of Z corresponds to selecting features that capture different levels of abstraction. We will discuss selection of Z in Section 3.2] Since input data often contains redundant or irrelevant information, which may hinder the generalization ability of 7, it is essential for Z to retain the most predictive information about Y while discarding irrelevant information from X. This trade-off leads to better generalization. The Information Bottleneck method (Tishby al., 1999 Tishby & Zaslavsky 2015) formalizes this principle through the concept of mutual information. Specifically, the IB objective is:
max T(Z, Y; 0) x;;
0
where -(.-)) represents mutual information (MI), and 3 > 0 is a hyperparameter controlling the trade-off between relevance and compression. Since Z is computed by the model f. optimizing the model's 's parameters 0 is equivalent to optimizing Z. The term T(Z, 0) promotes Z to be predic- tive of Y. while I[ [[Z X; 0) encourages to ignore irrelevant information from X. For simplicity, we omit the model parameters 0 in the following equations.
The mutual information terms in Eq. 7 are generally intractable. To address this, Alemi et al. (217)) proposed to derive more tractable variational bounds. Following Wieczorek & Roth (2020), we assume the Markov chain X
y yo derive the objective. Detailed derivations for the following equations are provided in Appendix
Upper bound of X): To derive variational upper bound on [22 (), we first expand it as:
I(Z,X) / p(x, 2) log
p(zlar)
dzdæ Epp) [og p x)] (lg
p(z)
Computing p = p(z, dx challenging as it involves marginalizing over x. Instead, Alemi et al. (2017) suggest approximating it using a predefined prior r(z). We will discuss how to choose
r(z) in Section 3.2 By utilizing the Kullback-Leibler divergence DKL (p(=)][n(z)) 2 0, we obtain:
p(z|zr) (r) dædx Ep(i)) [DKL (p(z (x) (x) ([
X(Z,x)= < p(z)z))((x)1og
r(z)
Lower bound of ((z Wieczorek & Roth 2020) derive the following lower bound:
T(2,Y) Y) = Ep( Bp ())))) [log p(ypp(())))))))))) + HYY)
2 Ep(x) Epty u) (p(=(=) [log p p[y|=)]
where (-) denotes the Shannon entropy. By plugging the upper bound from Eq. 8 and lower bound from Eq. (9) into Eq. (7). and flipping the max min, the IB objective becomes
min Rp(ular) Fp())))) [log + 33 Bpp()) [DRL (p(=|æ) [[(2))],
(10)
where the expectations be approximated using Monte Carlo samples. In the following paragraph and Section 3.2 we will discuss how to compute p(y 2). p( x)), and how to select the prior r r(z).
Challenges when applying IB to internal layer of LLM: Previous works, such as Alemi jet al.] (2017), typically apply IB to an intermediate layer within the neural network, treating the features that layer the hidden variable Z. In this scenario, the earlier layers (a.k.a. the encoder) learn p(z (z) However, since z represents the features at an intermediate layer, the true distribution p(yl (z) is unknown in practice. As result, Alemi et al. (2017) use the later layers (the decoder) to learn distribution g(y|2)) (z) to approximate piy) 2). J. The approximate distribution (3y|2) serves as substitute for p(y|2* 2²) in Eq. (9) However, we argue that directly substituting p(y]=) with 9(3) 2) in
Eq. (9) requires a more careful examination. Specifically, expanding Eq. (9) yields:
p(y p(yl2) 2)
T(Z, Y) 2 Ep(ee (p(wa) Ep(x (logopplll =)] = (Eptte) Eppcy a)) Ep(=)a)) log (((l=))
4Vv 2))
log p(y)2)
= Bntce E Eptw a) Ep(z(e) [3)=)) + Eptae E Eptwaa E)t
((1122)) [2]
(11)
Alemi et al. 2017) initially derive the objective using the Markow chain 7 - X - Y but implement it with x Z Wieczorek & Roth (2020) directly derive the lower bound from X Y.
4

---

Published as a conference paper at ICLR 2025
Term (i) is used for training the model in Alemi et al. (2017). including both the encoder and de- coder. Term (ii) represents the gap between term (i) and the true lower bound in Eq. ( Crucially, term (ii) is not necessarily non-negative. If term (ii) is negative, it undermines the assumption that term (i) serves as a valid lower bound on T(Z, Y). This calls into question its suitability as a training objective. Term (ii) remains small only when the model is well-trained so that g(y (2) closely ap- proximates p(y y z). ). In summary, when introducing IB at an intermediate layer within an LLM and substituting p z)) with ((y)2)), we lose control over whether the training objective truly remains lower bound on (2, Y). In the next section, we present our strategy to address this challenge.
3.2 REGULARIZING EDL WITH IB
So far we have not introduced how to apply to the EDL pipeline: x x f(x; () é - π y. In this section, we focus on two key questions: (1) What should the hidden variable Z be in IB-EDL? In other words, where in the EDL pipeline should we introduce IB? (2) What prior distribution r(z) should we select?
Choice of hidden variable Z: As highlighted in Section 3.1 a key challenge in applying IB within neural network is the potential violation of the lower bound. We address this by choosing the pre- evidence as the hidden variable Z, i.e., = ē E TRC By doing so, we can exactly evaluate p(y (2) + y. Specifically, given é, SoftPlus é) + 1, from which π ~ Dir(T; a) and y Cat(y; π follow. As a result, there is no need to learn g allowing us directly use the lower bound in Eq. (9) (and hence Eq. (T0) as the training objective, which only involves p(y|z). In other words, by choosing
from the pipeline é
+
compute
e + 1
é, we skip
the step of learning an approximated distribution g(y\2) and ensure that we are maximizing valid lower bound of T(Z, Y). Next, we proceed to compute this bound, namely the first term in Eq. (10 which can be analytically computed as:
(IB-NLL(O) - (ptzla [log
(12)
Ep(w) Ep(ull)
My (log((n) log(a;))
where we have omitted the dependency of a on z ē. Furthermore, our method can be further enhanced by integrating the finding of Sensoy et al. (2018), suggesting the MSE loss practically more stable objective. This can be analytically computed from a (and é) as follows:
MIBE MSE ( ppt)) B Pepl Bptela)
[lly
π
c
aj
2
aj ag a,)
(13)
(ptta)
M1
+
(a + 1)
Choice of prior r(z): Having defined è, now require suitable prior r(2) for Notably,
is the output of the LLM, and recent studies (Zhang et al. 2021 Hashemi et al. [2021) suggest that activation distributions in the later layers of neural networks tend resemble Gaussian distributions more closely than those in earlier layers. Additionally, the pre-evidence represents the LLM's out- put values, typically ranging from -2 to Therefore, a standard Gaussian prior, 2j ~ ~N,0) 1) Vj. represents a reasonable choice Given that z follows Gaussian distribution, we leverage the LLM ff to learn the mean and covariance of this z's distribution. Typically, an LLM comprises sequence of transformer layers, denoted as g. followed by a linear head h, such that f= g. To model the Gaussian distribution, we double the number of output neurons in h, partitioning it into two equal-sized functional parts. One part, ht, predicts the Gaussian mean, while the other, 4t, predicts the variances. Since h²" and to predictors share the same features from y. both predictions can be computed a single forward pass. We define μ = hg(g(x)) and o = SoftPlus(h" ((ttPl yielding p(z x;; 0) (2; μ, diag (or)). Then the second term in Eq. (10) becomes:
LIB-Into ( == Ep(ar) [DLL (p(x)z)] x Ep(z) || + | 1 - > log(o,)
(14)
AAlerrnatvell, we can also choose z SoftPlus(ē); however, the prior will be truncated Gaussian.
5

---

Published as a conference paper at ICLR 2025
Algorithm IB-EDL training and inference pseudocode.
Require: Data (æ. y), LLM h 9, weight 3, sample size K, binary flag IsTraining.
1: a + (h (at); and compute IIB. with pa, a..
b Predict p(é)æ) using the LLM.
2: Draw {e(*) from N (ē; diag (or))
b Parallelized in Pyforch.
3: if IsTraining then
At training time.
4: Compute (IB-MS((O) for each @(k) and take the average. Backpropagate averaged CIB-MS(())
b> Eg.
3
Also parallelized.
5:
LIB-
6:
else
> At inference time.
7:
Σx= Final prediction D < arg max, Tj. Uncertainty mass: u + C/ao.
Compute average è <- K k= all;; and a SoftPlus(é)) + and ftj <- 0z/00 Vj.
8:
Overall IB-EDL loss and its interpretation: The final IB-EDL objective is: min LIB (SE + 3 LiB Int
(15)
Importantly, Eq. (14) imposes a Ezrrguularization on μ., i.e. the mean of and thus on a. Therefore, IB-EDL. penalizes the LLM for generating large a that lead to over-confiden predictions.
3.3 PRACTICAL IMPLEMENTATION
In this subsection, we focus on the implementation of IB-EDL, so we use é instead of Eq. (3) requires sampling from the predicted distribution // (é; μu, diag (ar) and using the sampled e to compute a. To handle the cinddifferentiable sampling operation, we apply the reparameterization trick (Kingma & Welling 2014), allowing gradients to flow through the LLM parameters O. For each input æ, we sample K (e.g. K = 20) pre-evidences and compute the average loss derived from them (see Algorithm D. At inference time, we also sample K values and compute the average é. The extra time cost is minimal compared to the inference time of the LLM (detailed in Section 4.5
3.4 VARIATIONAL BAYES-BASED EDL AS A SPECIAL CASE OF IB-EDL
Some EDL methods (Cen al. 2018 Joo al. 2020) are based a variational Bayes (VB) perspective and can be seen as a special case within the IB-EDL framework. More formally: Proposition 1.
The VB-based EDL methods, which minimize (epla.s)) [Dxr. (p( 7 lx; 0) |p y))), are special case of IB-EDL when the hidden variable is chosen as z = (i.e. token probabilities) and the prior r(z) is chosen as Dir (z; a (1 (1-y)).
The proof is detailed in the Appendix B In the experiments, we will also compare our method with VID (Chen et al. 2018), a representative EDL method from this category.
EXPERIMENTS
4.1
EXPERIMENTAL SETUPS
Models: We fine-tune Llama2-7B (Touvron et al., 2023). Llama3-8B (Dubey et al, 2024). and
Mistral-7B (Jiang al. 2023) using LoRA (Hu al. (022) implemented via PEFT (Mangrulkar et al. 2022) and Transformers (Woff et al. 2020). Details on training configurations and 3 values
are provided in Appendix C Due to space constraints, we primarily present results for Llama2-7B
and Llama3-8B, with Mistral-7B results provided in Appendix E.
Datasets: We compare methods on six multiple-choice classification datasets, including five for commonsense reasoning. ARC-C and ARC-E (Clark et al. 2018), OpenbookQA (OBQA) (Mi- haylov et al. 2018), CommonsenseQA (CSQA) (Talmor et al. 2019) and SciQ (Welbl et al. 2017). alongside dataset for reading comprehension, RACE (Lai al. 2017). For these datasets, define the target space y as the tokens corresponding to the possible options (A/B/C/D). When fine-tuning LLMs, we select next-token logits (pre-evidences) corresponding these options.
Baselines: We compare our method against a variety of baselines, including standard MAP training, two conventional uncertainty-awee approaches: Deep Ensemble (Ens) (Lakssminnrrayanan al.

---

Published as a conference paper at ICLR 2025
Table 1: Calibration performance of uncertainty-aware methods on fine-tuned Llama2-7B. Arrows
("1" or "4") indicate whether higher or lower values signify better performance, respectively. The best and second-best results are highlighted in bold and underlined, respectively. Additionally, =
denotes the standard deviation of 3 runs. IB-EDL consistently achieves comparable accuracy while significantly reducing ECE and NLL, thereby greatly mitigating the overconfidence of the LLM.
Metries
Method
ARC-C
ARC-E
OBQA
CSQA
SciQ
RACE
MAP
68.00+1.72
85.07=0.11
79.86=0.12
78.67=0.26
91.30+
09
MCD
68.0111.71
85.1010.09
49.7010.46
78. 75.6510.30
91.33
51.26110.1
Ens
67.4610.21
85. 25.165
79.72.0061
78.8410.16
90.83.10.00
81.26610.18
LA
66.9220.55
84.60=0.44
80.15=0.22
78.58+0.50
90.87=2015
81.1920.33
Acc f
EDL
06.7841168
84.56=2067
79.80=0.40
79.03:0.62
90.07:20611
81.03 $1.03:::29
VID
67. 47.40=0.90
84.69=0..r
78.93
78.46+0.25
91.13++0.22
81 $1.33+0.55
I-EDL
68.98.0.37
84.76.0.69
81.79 10.53
79.2211.
00.43.10.70
78 06.30.32
R-EDL
70. 70.37
14.53
81.13
78.641.0.16
10.49
79. 79.2991:1.0
IB-EDL
68.17.0.92
85.67.0.fr
80.17.10.20
78, 1..62.1107
91.1011::6
81. 21.82.1.8
MAP
30.08.10.96
13.66.10.14
17.81
19.061...25
6.85.0.21
9.62.10.32
MCD
30. 30.08.0..9
=2.87
17.14
19.06::0.25
6.83+0.20
9.62=0.32
Ens
28.20 28.2612278 78
12.64
15.19:1.04
17.8210.17
6.7710.33
9.51=0.43
1.801
21.0111:32
10.7110.16
11.8111.58
15.5112.79
9.62.1.11
ECE
EDL
14.26+11.8
8.87=1.40
9.14+1.08
9.71=1.04
18.69=1.17
7.47=0.72
VID
13.9440.87
4.78
10.06:1.51
8,25=0.27
6,26+0.54
4.56=0..81
IEDD
12.39
11.33=027
10.84#1.18
11.66=0.61
15.13:4.00
13.52+0.85
R-EDL
15.3412.81
4.6810.41
7.1010.68
5.77_0.66
6.1310.25
4.48.0.50
IB-EDL
6.38_..50
10.77
0.84
4. 4.9011.00
5.01:00.43
1.0310..1
MAP
2.98+b..11
1.09=0.04
1.14+0.0
1.26=0.01
0.38+0..1
0.61=n.01
MCD
10.81
1.0910.05
1.11
1.26
0.381
0.6110.01
Ens 2.9010.19
1,0810.03
1.0510.05
1.12-0.06
0.3910.02
0.5010.01
LA 1.03_0.01
0.70=0.01
0.60
0.70=0.03
0.3720.03
0.56:20.01
NLL
EDL L.071004
0.59 03
0.6520.01
0.75=0.01
0.4720.01
0.62=0.01
1.07+0.01
0.60=0.0r
0.70
0.77=0.01
0.31+0.01
0.63=0.03
LEDL
1.0810.01
0.60 02
0.6210.01
0.75_0.01
0.4610.01
0.71
R-EDL
1.07:00.05
0.57
0.61
0.7410.00
0.35.0.01
0.61
IB IB-EDL
1.03_0.02
0.58.1..0
0.6510.01
0.74.0.02
0. 0.2911.01
0.5011.01
2017
Fort et al. 2019) and MC-Dropout (MCD) (Gal & Ghahramani] 2016), and Laplace-LoRA (LA) Yang et al. 2024), ), recent calibration method tailored for fine-tuned LLMs. Additionally, we
2018 VID (Chen et al. include four baselines from the EDL family: vanilla EDL (ensoy et al.
2023 and R-EDL (Chen 2018) from VB-based EDL, and two SOTA methods: I-EDL Deng al.
ce al. 2024). We use their original implementations and hyperparameters where available. Addi- tionally, ly, although PostNet (Charpentier et al. (0200) is also well-known EDL method, it is omitted here because it requires a specialized Normalizing Flow design to be compatible with Transformers.
4.2 IN-DISTRIBUTION CALIBRATION
An effective uncertainty-awar method should 1) significantly improve model calibration and 2) show accuracy comparable to standard MAP training. We therefore use Accuracy (Acc), expected calibration error (ECE), and negative log-likelihood (NLL) as metrics for evaluating the fine-tuned LLMs on the six aforementioned datasets. Table T and Table 2 present the results for Llama2-7B and Llama3-8B. respectively. The accuracy IB-EDL and other uncertainty-awarc methods is on par with, or even higher than, the MAP baseline, so we focus primarily on analyzing ECE and NLL. MAP exhibits substantially higher ECE and NLL. than other methods, suggesting that fine-tuning LLMs on small datasets using MAP (or MLE) leads to significant overconfidence. Overall, IB-EDL shows the lowest BCE and NLL, reducing ECE by several factors compared to MAP, MCD, and Ens. This highlights IB-EDL's ability to effectively mitigate overconfidence in fine-tuned models. The superior performance of IB-EDL, compared to other EDL methods, can be attributed to the E regu- larization in Eq. (14), which discourages the model from generating excessively large evidences that lead to over-concente concentteed Dirichlet distributions. Furthermore, the advantages of IB-EDL extend to other architectures, such as Mistral-7B, as demonstrated in Appendix E.

---

Published as a conference paper at ICLR 2025
Table Calibration performance uncertainty-aware methods on fine-tuned Llama3-8B.
Metries
Method
ARC-C
ARC-E
OBQA
CSQA
SciQ
RACE
MAP
79.7420.27
92.27=0.17
88.60=0.8t
81.51
93.37=00122
88. 15
MCD
79.55
92.25=0.15
88.63=0.84
81.52+0.60
93.30=20.20
12...5
Ens
79.50=0.10
92.26=0.41
88.53
81.39=0.64
93.27=0015
88 099
LA
17.79.00.9
92.1810.24
88.34
81.3010.44
03.37.10.14
88 19.0..19
Ace f
EDL
79.35.11.11
10.76
87.67 10.331
80.67 1...2
13.13
37.7610.21
VID
79.99=t0.13
92.30=0.25
87.57=0.22
80.82=0.94
02.93=00.2
88. 48.23=t0.. 25
I-DDL
80.37
92.76:0.47
88.52:20.50
31.13300.30
93.47=20.06
85.91 +0.40
R-EDL
80.60+0.18
92.41 #0.30
88.00=0.20
80.83#1.00
03.33++022
87.62+0.16
IB-EDL
81.14
92.55+0.15
89.00=n.40
81.71 :
93.57=00
88.032..21
MAP
19.08.10.4
10.52.1.88
17.29
5.74:10.88
7.952
MCD
19.91=0.30
7.10=0.02
10.48=0.86
16.98=0.10
5.74+0000
7.93=0.8
Ens
18.20.10.17
3.81
10.08.10.00
16.1111661
5.7210.24
7.80
LA 18.49.10444
3.33=10.0
5.26
6.62-0.10
2.47
3.61=0.27
ECE
EDL
6.5210.12
5.94.0 87
8.28
7.43=1448
11.13:11.11
6.51
VID
10.96.10.0
3.33.1140
5.9911.41
8.38_0.93
2.6010.13
4.5810.14
IEDL
5.08+1.94
9.69=0.55
7.57+0.52
8.95=0.59
13.07=0.24
14.52+1.15
R-EDL
10.09
2.93=1.32
1.68
6.59-0.78
3.1840.18
2.61
IB-EDL
2.78=0.87
2.70#0.68
2.34=0.61
4.34+0.2 20
3.86+0.88
4.47=0.11
2.25+t0.0
0.61
0.78+0.05
1.25=0.04
0.36+0.00
0.47=0-01
2.27:40.99
0.59=0.00
0.77
1.22-0.03
0.36:0000
0.45=0.00
2.02+00.07
0,43=0009
0.74
1.22-0.12
0.35#0.01
0.46=n.06
0.8010.01
0.33...00
0.4210.01
0.62.10.4
0.2210.00
0.37
NLL
EDL
0.7410.02
0.3310.00
0.4510.02
0.68_0.01
0.3110.01
0.43
VID
0.78+0.01
0.35=0..0
0.46:2..2
0.72-0.03
0.28+0.01
0.36==0.00
LEDL 0.72+0.02
0.36=
0.43:0.00
0.68-0.01
0.32+0.01
0.52=0.02
R-EDL
0.74+0.01
0.32#0.00
0.43+0.02
0.68=0.02
0.27+0.01
0.43=0.02
IB-EDI
0.69-0.m
0.32:+0.02
0.40=n.08
0.66-00.1
0.25+0.01
0.35++0.00
Table OOD detection performance on fine-tuned Llama2-7B and Llama3-8B. - B indicates A the ID training set and B the OOD test MP and UM are scores for measuring AUROC.
OBQA + ARC-C
OBQA + ARC-F
OBQA + CSQA
Model
Method
AUROCT
AUROC1 1
AUROC1
MP
UM
MP
UM
MP
UM
MAP
76.07
72.21
72.45=0.20
MCD
76.07
72.20+0.00 60
72.49=0.11
Ens
71.89+2.79
70.30
70.50
72.85+0.86
07.46400.4
70.71+0.44
Llama2-7B
68.42.1.72
11.
78.3410.45
75..89
76.4611.71
72.56
VID
86.47+0.26
81.24+0.81
18..8
83. 14+ 1.64
80.41+0.78
77.27
LEDD
81.3411.41
75.78:0.94
77.2240.75
72. 10
75.7840.15
71.60
R-EDL
76.66+1.07
73.80=a.74
72.01
69.02=0.91
71.01
68.22+1.48
IB-EDI.
87.47
88.3423.07
86.42=0.87
88.5 88.52:2.a0
86.38=0
79.79.14.64
MAP
63.
18.11
64.43+1.63
MCD
62.8111.00
58.3011.99
64.05.1.00
Ens
62.70+1.04
68. 08.1971.72
63.7940..5
LA
62.14 10.55
56. 16.111100 11
63.25.11.06
Llama3-8B
EDL
82. $2.04
78.99 11.18
78. 10.18.1447
74.17
81.07.1.22
77.55
VID
88. 85.11.877 .57
19.95=11.9
87.55
90.02=1.72
18.66-21.75
84.37
I-DDL
81.31
78.54=n.61
78.29+1.14
74.79=1.45
77.85++8.29
73.80+4.28
R-EDL
75.79+0.51
73. 73.64=0.49
71.51
68.81=0.79
70.60=0.00
67.42+1.30
IB-EDL
10.96
92. 0.33
18
94.77.0.42
89.16=1.00
85.45-10.58
4.3 OUT-OFF-ISTRIBUTION DETECTION
In addition to in-distribution (ID) calibration, dittrf-distribution (OOD) detection serves key benchmark for assessing the performance of uncertainty-awar methods. An effective approach should reliably assign higher uncertainties to OOD samples compared ID samples. This can
8

---

Published as a conference paper at ICLR 2025
Table 4: Fine-tuning Llama2-7B and Llama3-8B on noisy datasets. In each training dataset, the labels of 30% samples are randomly perturbed. IB-EDL is more robust to noise than other baselines.
Mctric
Method
ARC-C
ARC-E
OBQA
CSQA
SciQ
RACE
MAP
51.0812.96
21.79
73.9312
74.5610..1
88.63.10.06
76.831..30
MCD
51.08+2.96
71 21.47
23
04.66=0.32
88. 88.64+0.06
76.95+0.119
Ens
56.1013.11
76.03:1199
75.2311.00
24.50
88 64.10.6
76.94.10.22
LA
53.3812217
74.9011.76
74.37
10.50
88.4710.20
17.04
Llama2-7B
EDL
57.16 6+t3.3
76. 1.11.12
74.46+1.55
74.65=0005
89.03=+0.25
77 77.69+0.a4
1
VID
37.56+0.18
76.57=1.79
76.67=1.42
75.97=0.66
89.06=a0.21
78.75
I-EDL
33.86+1.25
76.08=0.71
77.00+
74.01=1.26
89. $9.26=0.s8 0.22
76.48+0.87
R-EDL
57.00
76.9
73.20
74.36.0.49
87.33.10.49
18.01
IB-EDL
59.06+2.00
80. 20.27 0..48
78. 10.000
76.35=0.00
89.06++0.0m 50
79.41+0..m
MAP
57. 10.41
80 10..3411..7
78. 18.18011000
77.04:0.41
92. 22.6001..50 53
86.9310.111
MCD
57.71 40.42
10.37=1146
79. 29.00:20.0 82
77.05=0.41
92.87 12
86, 36.93+00.
Ens
63.39.1100
6..00.53
80.6110.83
77.6220.41
92.97.0.00
86.9310.07
LA
66.62+1 108
82. 04
79.59+0.72
77.80=0.19
92.93=00.1
87.00+0.01
Llama3-8B
EDL
69.43++098
$7.57720.15
85.6010.72
79. 14
92.90=1040
86,2620070
Ace 1
VID
66.58+1992
87 07.67=0..0
84.86 84.86+1.00
79. 1.
93.23=+0.25
86.86+0.22
I-DDL
63.8712.65
84. 06 2,880
84. 14.:2610.44 42
78. 18.027
02.53.10.3
86,01
R-EDL
71.25:11.20
84.91
85. 75.00.00070
78.57
93.5620.00
86,16 06.1622042
IB-EDL
68.5310.25
88.052...0 43
86.13 10.0
79. 20.79
93.46.10.38
87.01 01.0000000
Number of sampled evidences K
Number of sampled evidences K 10 20 30 40 50 60 70 80
Number of sampled evidences K
10
20
30
40 30
60 70 80
10
20 30 40 50 60 70 80
71
3.0
70
Different (ehile fixing
Different (while facing
30
2.5
Accuracy
59
MAP Bacelins
Different a twhile fuxng KI fixing |I)
2.0
Different (while fixing fixing
ECE
MAP Baseing in
MAP Baseline
1.5
68
0.01
7
8
0.01
1.0
0.01
3
8
(x11
(x10
(a) Accuracy
(b) ECE
(c) NLL
Figure 1: Ablation study. IB-EDL reduces ECE and NLL compared to MAP across a broad range
of 3 and K values. 8 controls the regularization strength and balances the calibration and accuracy.
be evaluated by labeling ID samples as class I and OOD samples as class O, and measuring the AUROC based on OOD detection scores derived from the fine-tuned model. A higher AUROC indicates better OOD detection performance. Similar to Chehn et al. (0024). we use two OOD detec- tion scores: max probability (MP) and the reciprocal of uncertainty mass (UM). We fine-tune the LLMs on OBQA (as the ID dataset) and test them on ARC-C, ARC-E, and CSQA (as OOD dataset). Note that non-EDI. methods, such as LAA, do not provide UM, so we evaluate them using MP only. As shown in Table 3 IB-EDL achieves the highest AUROC across all datasets using both scores, surpassing both non-EDL and EDL competitors. Furthermore, IB-EDL also demonstrates supe- rior OOD detection performance under large distribution shifts (see Appendix E.3 Its calibration performance also generalizes well to OOD datasets (see Appendix E.4)
4.4 FINE-TUNING WITH NOISY LABELS
Datasets used for fine-tuming LLMs can contain a significant portion of mislabeled samples (Wang et al. 2024b) Havrilla & Iyer 2024). and label verification is often expert- -nool--k demanding. Therefore, it is crucial for tine-nuning algorithms to be robust to label noise (Wang al.) 2023a).
To assess this robustness, we perturb the A/B/C/D options for 30% of the samples in each training set, fine-tune the models using the aforementioned methods on the noisy datasets, and evaluate them clean test sets. In this adversarial setting, the primary goal is to maintain accuracy despite label noise, so we primarily evaluate accuracy, with additional metrics in the Appendix [..2 As shown in Table 4 IB-EDL- achieves the highest accuracy overall, demonstrating strong robustness
to label noise. This suggests that the information bottleneck effectively filters out spurious signals and retains predictive information in the generated evidence.

---

Published as a conference paper at ICLR 2025
4.5 ABLATION STUDY
Hyperparameters: We study the weight 3 and sample size K (used for drawing é), using ID cal- ibration for Llama2-7B ARC-C the evaluation task. Fig. [ shows that IB-EDL consistently outperforms MAP across a broad range of values, demonstrating its robustness in reducing overcon- fidence. While inereasing K slightly improves accuracy. does not necessarily improve ECE or NLL. / plays key role in balancing regularization and predictive performance. Nonetheless, all ß values reduce ECE by least 40% and NLL by least 50% compared to MAP. Additionally, we also present sensitivity analysis on the number bins of ECE in Appendix E.6
Here, we consider Llama2-7B and assume the target space the vocabulary )' = V. Compared to the pretrained LLM, IB-EDL introduces an additional linear head ht, adding only 1.95% more parameters. The computational overhead stems from 1e2 and the evidence averag- ing operation (sec Algorithm D which amount to only 1.98% of the pretrained model's GFLOPs. In Appendix E.5
Complexity analysis:
we provide detailed tests of training and inference time as well as memory usage.
5 RELATED WORK
EDL: EDL leverages models to predict the Dirichlet prior distribution, with training typically done
using NLL Haussmann et al. 2023), Lollos (Tiilikkaridss 2021), MSE (Sensoy et al. 2018
Chen et al. (2018); Joo et al. 2020): [hen et al. (2023) derive loss functions from a variational
Bayesian perspective, while Posterior Networks (Charpentier et al.) 2020 2022 optimize the pos-
terior via Normalizing Flows. EDL methods often incorporate two main types of regularization: (i)
encouraging uniformity in non-target Dirichlet parameters (Malinin & Gales 2018 Sensoy al.)
2018 Chen et al.] 2018 Tsllikkaridis 2021). or (ii) modifying assumptions in the EDL formu-
lation (Dngg et al.) 2023 Chen et al.] 2024). IB-EDL differs by not requiring (i) and taking an
alternative approach to (ii), without altering EDL assumptions. Some EDL methods also incorpo-
rate OOD data during training (Malinin & Gales 2018 2019). Beyond classification, EDL has been
extended to regression (Amini et al. 2000) and other applications (Gao et al. 2024 Liu & Ji 2024).
Recent works (Shen et al. 2024 Juergens et al., 2024) analyze EDL's effectiveness and limitations.
Other uncertainty-aware methods: Besides EDL., there are other methods for uncertainty estima- tion and calibration, including Bayesian Neural Networks via Variational Inference (Graves) 201T]
Blundell et al. 2015). MC- (Cr-Dropuu (Gal & Ghahramani 2016), stochastic gradient MCMC (Welling & Tch 2011 Ma ct al. 2015), and Laplace approximations (Ritter et al. 20188) Kristiadi et al.
202
recently extended to LoRA fine-tuned LLMs (Yang al. 2024 Wang al. 2024 Li al.
2024
IB: The Information Bottleneck (IB) was introduced by Tishby et al. [ (1999) and later applied in neu-
ral networks for learning generalized representations (Tishby & Zasky 2015 Alemi et al. 2017]
Sun et al., 2022). and as a feature attribution method (Schulz et (2020; Zhang et al.) 2021 Wang
al. 2023b). Other works focusing theory studied different Markov chains in IB (Wieczorek &
Roth 2020) and the impact of IB on generalization errors (Kawaguchi et al. 2023).
6 CONCLUSION
Summary: We focused on a key challenge in fine-tuning LLMs: mitigating overconfidence and improving calibration. We introduced an information-thhorrtie regularization conventional EDL to prevent over-concentrated distributions in predictions. Our method, IB-EDL, introduces minimal computational overhead while significantly improving calibration in fine-tuned LLMs. Additionally, IB-EDL maintains model performance even in the presence of substantial label noise. These results highlight IB-EDL as promising method for fostering more trustworthy LLMs.
Limitations and future work: Despite IB-EDL's efficiency, there is room for improvement. To re- duce the complexity of covariance matrix prediction, we assume the pre-evidences are uncorrelated, but this assumption can be relaxed. Additionally, tional classification tasks, where well-established metrics for calibration and uncertainty estimation are available. In future work, it would be interesting to test IB-EDL on generative tasks. A great challenge is that uncertainty estimation metrics for generative tasks are still an ongoing research topic (Yadkori et al. 2024: Jesson et al. 2024).
evaluations primarily focused conven-
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENT
This work is supported by the Munich Center for Machine Learning (MCML). In addition, we sincerely appreciate the insightful discussions with Emanuel Sommer and Jianfei Li.
REFERENCES
Alexander A Alemi, Ian Fischer, Joshua Dillon, and Kevin Murphy, Deep variational information bottleneck. In International Conference Learning Representations, 2017.
Alexander Amini, Wilko Schwarting, Ava Soleimany, and Daniela Rus. Deep evidential regression.
Advances in neural information processing systems, 33:14927-149977 2020.
Charles Blundell, Julien Corneiie, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural network. In International conference on machine learning, pp. 1613-1622. PMLR, 2015.
Bertrand Charpentier, Daniel Zügner, and Stephan Günnemann. Posterior network: Uncertainty estimation without ood samples via density-based pseudo-counts. Advances in neural information processing systems, 33: 13:1356-1367, 2020.
Bertrand Charpentier, Oliver Borchert, Daniel Zügner, Simon Geisler, and Stephan Günnemann. Natural posterior network: Deep bayesian predictive uncertainty for exponential family distribu- tions. In International Conference on Learning Representations, 2022.
Mengyuan Chen, Junyu Gao, and Changsheng Xu. R-EDL: Relaxing nonessential settings of ev- idential deep learning. The Twelfth International Conference on Learning Representations, 2024.
Wenhu Chen, Yilin Shen, Hongxia Jin, and William Wang. A variational dirichlet framework for out-of-distribution detection. arXiv preprint arXiv: 1rXiv:1811.07308, 2018.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv: 0803.05577 2018.
Arthur P Dempster. generalization of bayesian inference. Journal of the Royal Statistical Society: Series B (Methodological), 30(2):205-232, 1968.
Danruo Deng, Guangyong Chen, Yang Yu, Furui Liu, and Pheng-Ame Heng. Uncertainty estimation by fisher information-bart based evidential deep learning. International Conference on Machine Learning, pp. 7596-7616. PMLR, 2023.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang. Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang. Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez. Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Ander- son, Graeme Nail, Gregoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel Kloumann, Ishan Misra, Ivan Evtimov, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Ma- hadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Al- wala, Kartikeya Upasani. Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, Khalid El-Arini, Krithika Iyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Lauren Rantala- Yeary, Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo,

---

Published as a conference paper at ICLR 2025
Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Man- nat Singh, Manohar Paluri, Marcin Kardas, Mathew Oldham, Mathieu Rita, Maya Pavlova, Melanie Kambadur, Mike Lewis, Min Si, Mitesh Kumar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoychev, Niladri Chatterji, Olivier Duchenne, Onur Çelebi, Patrick Alrassy, Pengchuan Zhang. Pengwei Li, Petar Vasic, Peter Weng, Prajjwal Bhar- gava, Pratik Dubal, Praveen Krishnan, Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ramon Calderer, Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sum- baly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang, Saghar Hosseini, Sahana Chennabasappa, Sanjay Singh, Sean Bell, Seohyun Sonia Kim, Sergey Edunov. Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang. Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan, Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas Georgiou, Thomas Scialom, Tobias Speckbacher, Todor Mihaylov, Tong Xiao, Ujjwal Kar, Vedanuj Goswami, Vibhor Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do, Vish Vogeti, Vladan Petro- vic, Weiwei Chu, Wenhan Xiong, Wenyin Fu, Whimey Meers, Xavier Martinet, Xiaodong Wang, Xiaoqing Ellen Tan, Xinfeng Xie, Xuchao Jia, Xuewei Wang, Yaelle Goldschlag, Yashesh Gaur, Yasmine Babaei, Yi Wen, Yiwen Song, Yuchen Zhang, Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen. Zoe Papakipos, Aaditya Singh, Aaron Grattafiori, Abha Jain, Adam Kelsey, Adam Shajnfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alex Vaughan, Alexci Baevski, Allie Feinstein, Amanda Kallet, Amit Sangani, Anam Yunus, Andrei Lupu, Andres Alvarado, Andrew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani, Annie Franco, Aparajita Saraf, Arkabandhu Chowdhury, Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leonhardi, Bernie Huang, Beth Loyd, Beto De Paola, Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti. Bran- don Spence, Brani Stojkovic. Brian Gamido. Britt Montalvo, Carl Parker. Carly Burton, Catalina Mejia, Changhan Wang. Changkyu Kim. Chao Zhou, Chester Hu, Ching-Hsiang Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel Li, Danny Wyatt, David Adkins, David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang, Due Le. Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery, Eleonora Presani, Emily Hahn, Emily Wood, Erik Brinkman, Esteban Ar- caute, Evan Dunbar, Evan Smothers, Fei Sun, Felix Kreuk, Feng Tian, Firat Ozgenel, Francesco Caggioni, Francisco Guzmán, Frank Kanayet, Frank Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpern, Govind Thattai, Grant Herman, Grigory Sizov, Guangyi, Zhang, Guna Lakshminarayanan, Hamid Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habeeb, Harrison Rudolph, Helen Suk, Henry Aspegren, Hunter Gold- man, Ibrahim Damlaj, Igor Molybog, Igor Tufanov, Irina-Elena Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Japhet Asher, Baattttp Gaya, Jeff Marcus, Jeff Tang, Jennifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang. Joe Cummings, Jon Carvill, Jon Shepard, Jonathan McPhie, Jonathan Torres, Josh Ginsburg, Junjie Wang, Kai Wu, Kam Hou U, Karan Saxena, Karthik Prasad, Kartikay Khandelwal, Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kun Huang, Kunal Chawla, Kushal Lakhotia, Kyle Huang, Lailin Chen. Lakshya Garg, Lavender A. Leandro Silva, Lee Bell, Lei Zhang. Liangpeng Guo. Licheng Yu, Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhatt, Maria Tsimpoukelli, Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Ke- neally, Michael Seltzer, Michal Valko, Michelle Restrepo, Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang, Miquel Jubert Hermoso, Mo Metanat, Mo- hammad Rastegari, Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White, Navy- ata Bawa, Nayan Singhal, Nick Egebo, Nicolas Usunier, Nikolay Pavlovich Laptev, Ning Dong, Ning Zhang, Norman Cheng, Oleg Chemoguz, Olivia Hart, Omkar Salpekar, Ozlem Kalinli, Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji, Pedro Rittner, Philip Bontrager, Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish Yuvraj, Qian Liang, Rachad Alao, Rachel Rodriguez, Rafi Ayub, Raghotham Murthy, Raghu Nayani, Rahul Mitra, Raymond Li, Rebekkah Hogan, Robin Battey, Rocky Wang, Rohan Maheswari, Russ Howes, Ruty Rinott, Sai Jayesh Bondu, Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov, Sa- tadru Pan, Saurabh Verma, Seiji Yamamoto, Sharadh Ramaswamy, Shaun Lindsay, Shaun Lind- say, Sheng Feng, Shenghao Lin, Shengxin Cindy Zha, Shiva Shankar, Shuqiang Zhang, Shuqiang
12

---

Published as a conference paper at ICLR 2025
Zhang, Sinong Wang, Sneha Agarwal, Soji Sajuyigbe, Soumith Chintala, Stephanie Max, Stephen Chen, Steve Kehoe, Steve Satterfield, Sudarshan Govindaprasad, Sumit Gupta, Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Kohler, Thomas Robinson, Tianhe Li, Tianjun Zhang, Tim Matthews, Tim- othy Chou, Tzook Shaked, Varun Vontimitta, Victoria Ajayi, Victoria Montanez, Vijai Mohan, Vinay Satish Kumar, Vishal Mangla, Vitor Albiero, Vlad Ionescu, Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang, Wenwen Jiang, Wes Bouaziz, Will Con- stable, Xiaocheng Tang, Xiaofang Wang, Xiaojian Wu, Xiaolan Wang, Xide Xia, Xilun Wu, Xinbo Gao, Yanjun Chen, Ye Hu, Ye Jia, Ye Qi, Yenda Li, Yilin Zhang, Ying Zhang, Yossi Adi, Youngjin Nam, Yu, Wang, Yuchen Hao, Yundi Qian, Yuzi He, Zach Rait, Zachary DeVito, Zef Rosnbrick, Zhaoduo Wen, Zhenyu Yang, and Zhiwei Zhao. The llama 3 herd of models, 2024. URL wttpt:tp:///.mcmc. /ttpiis://axxi..oo org/ abs / 2407 783
Stanislav Fort, Huiyi Hu, and Balaji Lakshminarayanan. Deep ensembles: A loss landscape per- spective. arXiv preprint arXiv: 1912.02777 2019.
Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pp. 1050-1059. PMLR, 2016.
Yibo Gao, Zheyao Gao, Xin Gao, Yuanye Liu, Bomin Wang, and Xiahai Zhuang. Evidential concept embedding models: Towards reliable concept explanations for skin disease diagnosis. arXiv preprint arXiv:2406.1911 19130, 2024.
Alex Graves. Practical variational inference for neural networks. Advances in neural information processing systems, 24, 2011.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Weinberger. On calibration of modern neural networks. In International conference on machine learning, pp. 1321-1330. PMLR, 2017.
Vahid Hashemi, Jan Křetínský, Stefanie Mohr, and Emmanouil Seferis. Gaussian baass runtime de- tection of out-of ciat-f--iitribution inputs for neural networks. In International Conference on Runtime Verification, pp. 254-264. Springer, 2021.
Manuel Haussmann, Sebastian Gerwinn, and Melih Kandemir. Bayesian evidential deep learning with pac regularization. In Third Symposium on Advances in Approximate Bayesian Inference, 2023.
Alex Havrilla and Maia Iyer. Understanding the effect of noise in Ilm training data with algorithmic chains of thought. arXiv preprini arXiv:2402.04000 2024.
Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. Aligning with shared human values. Proceedings of the International Conference on Learning Representations (ICLR), 2021a.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. Proceedings of the Interna- tional Conference on Learning Representations (ICLR), 2021b.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, An- drea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. In International conference on machine learning, pp. 2790-2799. PMLR, 2019.
Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Con- ference on Learning Representations, 2022. URL https: https://openreview met/forom
id=nžeVKeeFyF3
Andrew Jesson, Nicolas Beltran-Velez, Quentin Chu, Sweta Karlekar, Jannik Kossen, Yarin Gal, John Cunningham. and David Blei. Estimating the hallucination rate of generative ai. arXiv preprint arXiv:2406.07457, 2024.
13

---

Published as a conference paper at ICLR 2025
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chap- lot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud. Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William Sayed. Mistral 7b, 2023. URL https:
Zhengbao Jiang, Jun Araki, Haibo Ding, and Graham Neubig. How can we know when language models know? on the calibration of language models for question answering. Transactions of the Association for Computational Linguistics, 9:962-9777 2021.
Taejong Joo, Uijung Chung, and Min-Gwan Seo. Being bayesian about categorical probability. In International conference machine learning, pp. 4950 4961. PMLR, 2020.
Audun Jøsang. Artificial reasoning with subjective logic. In Proceedings of the second Australian workshop on commonsense reasoning. volume 48, pp. 34, 1997.
Audun Jøsang, Subjective logic, volume 3. Springer, 2016.
Mira Juergens, Nis Meinert, Viktor Bengs, Eyke Hüllermeier, and Willem Waegeman. Is epistemic uncertainty faithfully represented by evidential deep learning methods? In Forty-first Interna- tional Conference on Machine Learning, 2024. URL /https:///ppereee// net forum?
id Id=mx jxjiBBiiqpp
Kenji Kawaguchi, Zhun Deng, Xu Ji, and Jiaoyang Huang. How does information bottleneck help deep learning? In International Conference on Machine Learning, pp. 16049-16096. PMLR, 2023.
Diederik P. Kingma and Max Welling. Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014.
Agustinus Kristiadi, Matthias Hein, and Philipp Hennig. Learnable uncertainty under laplace ap- proximations. In Uncertainty in Artificial Intelligence, pp. 344-353, PMLR, 2021.
Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. RACE: Large-scale ReAding comprehension dataset from examinations. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 785-794, Association for Computational Linguis- tics, September 2017. URL https: hatpa////l/caaaooooooo..oo wwtpooogg//ooo
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances neural information processing systems, 30, 2017.
Dengchun Li, Yingzi Ma, Naizheng Wang, Zhengmao Ye, Zhiyuan Cheng, Yinghao Tang, Yan Zhang, Lei Duan, Jie Zuo, Cal Yang, and Mingjie Tang. Mixlora: Enhancing large language models fine-tuning with lora-based mixture of experts, 2024.
Pei Liu and Luping Ji. Weakly-supervisel residual evidential learning for multi-instance uncertainty estimation. In Proceedings of the 4/st International Conference on Machine Learning, volume 235, pp. 31262-31292. Proceedings of Machine Learning Research, 2024. URL https:// proceedings mlr. www.srppessseeeem wttSss////2/111222accc
Yi-An Ma, Tianqi Chen, and Emily Fox. A complete recipe for stochastic gradient meme. Advances in neural information processing systems, 28, 2015.
Andrey Malinin and Mark Gales. Predictive uncertainty estimation via prior networks. Advances in neural information processing systems, 31, 2018
Andrey Malinin and Mark Gales. Reverse kl-divergence training of prior networks: Improved un- certainty and adversarial robustness. Advances in neural information processing systems, 32, 2019.
14

---

Published as a conference paper at ICLR 2025
Sourab Mangrulkar, Sylvain Gugger, Lysandre Debut, Younes Belkada, Sayak Paul, and B Bossan. Peft: State-of-the-ar parameter-efficient tine-tuning methods. URL: htpp://ggthub con/huggingfacce/pfff. 2022.
Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can suit of armor conduct elec- tricity? new dataset for open book question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2381-2391, 2018
Daniel P. Palomar and Sergio Verdu. Lautum information. IEEE Transactions on Information
Theory, 54(3):944-975, 2008
Hippolyt Ritter. Aleksandar Botev, and David Barber. A scalable laplace approximation for neural networks. In 6th international conference on learning representations, ICLR 2018-come 2018-conference track proceedings, volume International Conference on Representation Learning, 2018.
Karl Schulz, Leon Sixt, Federico Tombari, and Tim Landgraf. Restricting the flow: Information bottlenecks for attribution. In International Conference on Learning Representations, 2020.
Murat Sensoy, Lance Kaplan, and Melih Kandemir. Evidential deep learning to quantify classifica- tion uncertainty, Advances in neural information processing systems, 31, 2018
G Shafer. Mathematical Theory of Evidence. Princeton University Press, 1976.
Maohao Shen, Yuheng Bu, Prasanna Sattigeri, Soumya Ghosh, Subhro Das, and Gregory Wornell. Post-hoc uncertainty learning using a dirichlet meta-model. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, volume 37, pp. 9772 9781,9781112 2023.
Maohao Shen, J. Jon Ryu, Soumya Ghosh, Yuheng Bu, Prasanna Sattigeri, Subhro Das, and Gre- gory W. Wornell, Are uncertainty quantification capabilities of evidential deep learning a mirage?, 2024. URL
arxiv. org/ abs / 202 061
Qingyun Sun, Jianxin Li, Hao Peng, Jia Wu, Xingcheng Fu, Cheng Ji, and S Yu Philip. Graph structure learning with variational information bottleneck. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 4165-441 4174,, 2022.
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. Commonsenseqaa A question answering challenge targeting commonsense knowledge. Proceedings of the 2019 Conference of the North American Chupter of the Association for Computational Linguistics: Human Lan- guage Technologies, Volume (Long and Short Papers), pp. 4149-4158, 2019.
Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. 2015 IEEE Information Theory Workshop (ITW), pp. 1-5, 2015.
Naftali Tishby, Fernando C. Percira, and William Bialek. The information bottleneck method. In Proc. of the 37-th Annual Allerton Conference on Communication, Control and Computing, pp- 368-377, 1999.
Hugo Touvron, Louis Martin, Kevin Stone. Peter Albert, Amjad Almahairi, Yasmine Babaci, Niko- lay Bashlykov. Soumya Batra, Prajjwal Bhargava. Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint 2xXiv:2307.09288 2023.
Theodoros Tsilikkaridis. Information robust dirichlet networks for predictive uncertainty estimation, April 8 2021. US Patent App. 17/064,046.
Song Wang. Zhen Tan, Ruocheng Guo, and Jundong Li. Noise-robust fine-tuning of pretrained language models via external guidance. In Findings of the Association for Computational Lin- guistics: EMNLP 2023, pp. 12528-12540, 2023a.
Yibin Wang, Haizhou Shi, Ligong Han, Dimitris Metaxas, and Hao Wang. Blob: Bayesian low-rank adaptation by backpropagation for large language models. arXiv preprint a /1675, 2024a.
Ying Wang, Tim GJ Rudner, and Andrew G Wilson. Visual explanations of image-text representa- tions via multi-modal information bottleneck attribution. Advances in Neural Information Pro- cessing Systems, 36:1 1009-166077 2023b.
15

---

Published as a conference paper at ICLR 2025
Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj. Xuan He, Ziyan Jiang, et al. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. arXiv preprint 27X1v:2406.00154, 2024b. 2024b.
Johannes Welbl, Nelson F. Liu, and Matt Gardner. Crowdsourcing multiple choice science ques- tions. In Proceedings of the 3rd Workshop on Noisy User-generated Text, pp. 94-106. Associa- tion for Computational Linguistics, September 2017. URL LeLLLtttss//ccaaatooooog.... org/
W17-4413
Max Welling and Yee W Teh. Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of the 28th international conference on machine learning (ICML-II), pp. 681-688. Citeseer, 2011.
Aleksander Wieczorek and Volker Roth. On the difference between the information bottleneck and the deep information bottleneck. Entropy, 22(2):131, 2020.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Transformers: State-of-the-an natural language processing. In Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, pp. 38-45, 2020.
Yasin Abbasi Yadkori, Ilja Kuzborskij, András György, and Csaba Szepesvári. To believe or not to believe your Ilm. arXiv preprint arXiv:2406.02355 2024.
Adam X Yang, Maxime Robeyns, Xi Wang, and Laurence Aitchison. Bayesian low-rank adaptation for large language models. In The Twelfth International Conference on Learning Representations, 2024.
Yang Zhang, Ashkan Khakzar, Yawei Li, Azade Farshad, Seong Tae Kim, and Nassir Navab. Fine- grained neural network explanation by identifying input features with predictive information. Ad- vances Neural Information Processing Systems, 34:20040-20051, 2021.
16

---

Published as a conference paper at ICLR 2025
A DERIVATION OF THE VARIATIONAL BOUNDS
Upper bound of T(2, We reproduce the derivation steps from Alemi et al. (2017) as follows:
1Zz / p(z, z) log
p(z[æ=) dzdæ
(16)
p(z)
Given prior r(z), we have DKL (p(z)]lr(z)) 2 O. This indicates that:
DKL (o(z)llr(z)) = / p(z) log p0=) dz
r(z)
( Bo(x)(ogr(()) 2
Therefore,
E Ep(x)]logp(3) Ex(c) Ep(x)(logr(z))
(17)
Plugging Eq. 17
into Eq.
I6 we obtain
1(2, X)
= pp(z(a Bp(e) Blgg poaeeloogaaaa
p p==[==)
Ep(x) [oo p(=)
S pp] py [log p( [x)]]
Ep(z) [log r r())]
E pp(z|a) Bp(e log p( [x)]]
Ep(w) Ep(=le) [log r()]
(18)
log p(z(ær)
pp) Ep(= m)
r(z)
Pp()) [KLL. (p(=|==)]|<(=))
Lower bound of 7(2, Y): For clarity, reproduce the derivation steps from Wieczorek & Roth (2200) here. Unlike Alemi et al. (2017). who assume the Markov chain Z - X Y Wieczorek & Roth (2020) assume the Markov chain X x-z-- -Z-X-
implying the conditional independence p(y|z)
ply 2 x). The detailed steps are as follows:
I(Z,Y) Ep(n.
pptz|m B, llggp/l [
+ HYY
(19)
p())) Ep(ul p( (log x)] + H(Y)
Now, we derive a lower bound on the term Ep( E B( (log(()) 2 x)) in Eq. 19 as follows:
Ep a) Epix v) pogg(y 2,))))))) m)]
/ / p(z, x/ log p(z, y dddzdy
= / p(z, ((2.)lz)) p())=))(22 x) dydz
p())))
/ p(z. y|=) log x)dady
DxL (p(y. (vtg.,==))))llp(()))))()))))
Dri. (p(y, ( + pl p(y) logplyl=)dy
Dri. (p(,, (0)p,: / p(yl)) [
p(ala))(lll))p(zzll/ dzdy
DKL (olp,=))))l((//))))aa))))))
p(ylz)p(z(z) log p(z, )))))(()2()))
= Dxi. (p(y (,(p)==))))]((]]))((()))) + Dxa (p()l)))((())))l)xxx)))
+ 1og
> (oo
(20)
Plugging Eq.
20
into Eq.
19 and using p(y)2) p(y]2,.)) again, obtain:
17

---

Published as a conference paper at ICLR 2025
I(Z,Y) = ppa Br(a)a)) ( + HYY)
+ {x) DKL (ul
+ pp(()) DKL 1
(21)
I(Y 2 X + +
2 Bp(m) Br Bpp (otp://peBBBooooooggggg000111111 2 Bp p (pt
where T(Y, X is the conditional mutual information, and L(Y, 2|X) is the conditional lautum information (Palomar& & Verdu 2008), respectively.
18

---

Published as a conference paper at ICLR 2025
B PROOF OF PROPOSITION 1
brief introduction to VB-based EDL methods: Some previous EDL methods (Chen et al, 2018 Jooo al. 2020) derive the optimization objective from the variational Bayes (VB) perspective, where the goal is to optimize the model's parameters 0 such that the posterior distribution p( æ;; 0) aligns with the tre posterior distribution p(rly))
For notation simplicity, we omit the model's parameters 0 in p(T|2;; 0) (or p(z|<;;) 0)) and use ((T()) (or x)) instead.
Remark 1. The condition in Proposition iis that the latent variable
π, and the prior r(z)
r(T) Dir x: y @ a + (1 y)) In fact, the choice of the prior r(πr) is not unique. For example, Chen al. (2018) present three options. The correctness of Proposition 7 remains unaffected by
the choice of prior. Proposition uses one exemplary prior r(π) Dir a (1-
suggested Chen al. 2018
Proof. The target of VB-based EDL methods is:
min (p(a) Bp(vla) [Dx (p(T (l)
(22)
0
Next, show that the IB objective in Eq. 10 r(z) = r()) for given prior r(w), e.g., r ा Dir (T: O a + (1 y) Minimizing the
is upper bound of Eq. 22 when and
objective in Eq. (I0) therefore provides a tractable way approximate the minimization of Eq. (22
we choose = 1, 2 = π and a prior r(z) = r(x), then the IB objective in Eq. (10) becomes
min (n) Brulla) [log p(y |)] Ep())) [Dx. (P(T)=)]|<<T)]
(23)
Expanding Eq. 23 we have:
Ep(x) [DRL. (p( a [ )] Betola) Dog m]]
Ep(a))) [KI.. (p(TT) x | (TT)
E Bp(a)) Ep(u (p(m)))) [log p(y) m|]
p( x))
Bp Ep(()m) |f p((T)z) logp(v/ T)
Ep(n)
((TT/=))
dæ
r(TT)
p( x)) p(T))
/ p(TT/z)) log
(p(ula))
p(
dm
pp Ep(l))
Use Des (pt(((()))lllllllpp (( (Similar
18
f p(TT|2)) log
p( x)) p(xr)
If / logp(y))))c - -Y(Y)
2 Ep(ulee)
dr
Ep(ce E Ep(ele)
/ p( (2)
p( (x)
Ep(z)
dm
+ Ep(n) (lo p(y)]
p T (T)p))(T))))) TT)
p(T|mm) (m))
p( p(TT/z)) log
dx
+ Ep(a) Epinlae) [log(())
p(vl π ())(( TT) p(T x (e)pp(y)
Ep(
p(TT/2) log
dm
p( y)
p(T x)) dr
p( p(T] mx) log
p ( Epp [KL (
(24)
which is the target of VB-based EDL methods.
19

---

Published as a conference paper at ICLR 2025
Table 5: Loss weight 3 for IB-EDL.
ID Calibration
Model
ARC-C
ARC-E
OBQA
CSQA
SciQ
RACE
Llama2-7B
1 x 10 10-3
x 10-4
1.5 x 10-4
5 x 10-6
1 x 10-6
1 1x 10-6
Llama3-8B
x 10
2 10-6
1 10-5
3 10-5
x 10-7
1 1 10
Mistral-7B
9 x 10-5
1 10-6
1 1x10- 10-5
5 10-5
x 10-6
2 10-6
Learning with Label Noise
Model
ARC-C
ARC-E
OBQA
CSQA
SciQ
RACE
Llama2-78
2 X 10
1 10 10-4
5 X 10 4
1.6 x 10-4 4
5 10-6
1 1 10-6
Llama3-8B
5 x 10-6
2 10-5
6 10-5
3 3 10-5
5 10-6
2 10-6
Mistral-7B
1 x 10-5
2 10-5
9 X 10-5
1 10-6
1 10-6
7 10-6
C DETAILED IMPLEMENTATION
Models: We fine-tuned three models: Llama2-7B (Touvron al. 2023). Llama3-8B (Dubey al.
2024). and Mistral-7B-v0.1 (Jiang et al. 2023).
LoRA hyperparameters: We applied LoRA (Hu al. 2022) finetuning using the PEFT Man-
grulkar et al. 2022) library. For all models, LoRA adaptors were applied to the q-proj, v-proj, and am_ modules. Additionally, we used Dropout with a dropout rate of p 0.1, LoRA 16, rank r = 8, and set bias = "lora-co stora.only".
Training details: For the MAP baseline and all EDL methods, all models were trained for 30000 steps on the CSQA dataset and 10080 steps on the other datasets. The learning rate was set to 0.00005 and annealed using cosine schedule. The maximum token length was set to 300 for the RACE dataset and 256 for all other datasets. Training was conducted with f flatttll precision. For MCD (Gal & Ghahramani 2016). we performed forward passes. For Ens (Laakshminarayanan al. 2017: Fort al. 2019), used predictions from 3 models. For LA (Yang et al., (024), I- EDL Deng et al. 2023). and R-EDL (Chen et al. (0024), we used the original implementations and hyperparameters; please refer the respective official codebases. For EDL methods, follow the previous works to apply gradient clipping with maximal gradient norm 20 to stabilize the training.
IB-EDL implementation details: Both linear prediction heads (for μ and σ initialized using the linear head from the pretrained LLM. Additionally, both heads are equipped with LoRA adapters and are jointly trained alongside the LoRA weights of the transformer encoder layers. By default, we used = 20 for sampling pre-evidences from the predicted Gaussian distribution during both training and inference. Table [5) listss the 3 values used in different experiments. Note that for OOD detection experiments, the parameters are identical to those used for OBQA, as OBQA is the training set for these experiments. We optimized values using grid search. We suggest guideline for selecting 3: if the MAP baseline shows lower accuracy and higher ECE, or in the presence of label noise, a larger may be chosen to encourage stronger compression and forgetting of the input. Otherwise, a smaller 3 can be selected to allow more information from the input follow through the pre-evidences.
D A POST-HOC CALIBRATION TECHNIQUE FOR IB-EDL
In this section, we introduce an additional post-hoc calibration technique aimed at further refining the calibration of the IB-EDL finetuned model.
The key motivation behind this calibration method is to account for an additional source of uncertainty in IB-EDL: the variance of of the pre-evidences. In IB-EDL, the uncertainty mass and belief mass are normalized by the constraint: Σ3 by + = 1, where by = o, Σ E, To incorporate the additional uncertainty arising from the variance of pre-evidences, we consider the effect of increasing u, which leads reduction in Σ, by- This suggests the need to adjust aj by
Intuition:
20

---

Published as a conference paper at ICLR 2025
Table 6: Calibration performance of uncertainty-aware methods on fine-tuned Mistral-7B.
Metries
Method
ARC-C
ARC-E
OBQA
CSQA
SciQ
RACE
MAP
80.1920.68
92.07=0.12
88.06=0.61
13.24=1.47
94.73+00006
+0.28
MCD
80.26 30.26.2.52
92. 15
88.07=0.61
1.424
94.73=+0.06
86.6440022
Ens
80,48+0.4
92.41
88.51
83.41+1.44
94.80=0.11
86.81+0.2
LA
78,4711.110
92.1210.11
88.1910.69
83.2111.01
04.6610.16
86.3410..9
Ace f
EDL
80. 80.6010.44
92.2710.44
87.23
83.31
04.27.10:23
85.7010.41
VID
80.37=2039
91 31.8520.13 13
88.33=0.81
82.52=0.54
94.17=0.12
80.40200.21
I-DLL
80.46 1 66
92.28=0011
88. 88.0620.31
62.91
94.03:20.32
85.11 :0061
R-EDL
00.12
92.20=0011
88.33=0.91
83.07=0.83
94.03++..31
86.07+0.28
IB-EDL
80.57+0.68
92.47:0.12
70.00
83.65
94.4020.26
86.40+0.32
MAP
19.42.10.8 68
7.63=20.1
11.29::0.a1
15.72:
4.98
8.43
MCD
19.26=+0.52
1.62=0.16
11.22=0040
15.72+144
4.98+0.06
8.42=0.47
Ens
17.0411.58
7.03.0079
8.8710090
14.1812.52
4.7810.13
8.21
LA 20.04:1.02
1.57:10.39
4.49
15.11
1.91:0.46
2.94
ECE
EDL
1.11
6.2522046
6.2310.56
7.71
7.6420.85
7.75
VID
9.3810.46
2.44:10.16
4.8810.77
7.32-0.74
5.6210.52
4.20.10:1
I-DDL
4.70+2.15
10.94=1.27
9.63+0.75
8.85=0.44
11.00=0.30
13. 13.1640.87
R-EDL
11.26:20.8
2.86=1.07
5.4320.65
6.30
4.48+0.23
3.69=0.52
IB-EDL
3. 3.10-1.10
3.49=1.24
2.27=0.67
6. 6.02+0.00
4.99+1.22
3.58=0.22
MAP
2.18+0.34
0.85=2..0 03
0.85+0.02
1.18-0.06
0.30
0.52=0.01
MCD
2.1810.13
0.84=0.04
0.84
1.18=0.07
0.31
0.52=0.02
Ens
1.82+0.37
0.78=0.10
0.67
1.01-011
0.28+0.01
0.51=0.08
0.7810.02
0.2910.01
0.3810.02
0.5910.08
0.1710.01
0.45
NLL
EDL 0.7110005
0.35....0
0.47
0.63_0.01
0.2710.01
0.47.10.22
VID
0.76+0.05
0.36=00.0
0.46:0.01
0.68-0.02
0.2420.01
0.41=0.01
L-DL 0.7l
0.38
0.45:0.01
0.65=0.01
0.29+0.01
0.50=0.01
R-EDL
0.77+0.02
0.33=0.01
0.41+0.0
0.64=0.01
0.24+0.01
0.45=0.01
IB-EDI
0.70=0.0
0.32=0.02
0.41
0.61=0.0
0.23+0.01
0.40+s0.1
subtracting term that is proportional to the standard deviation ø,, which is predicted by IB-EDL. We parameterize this adjustment Ç J,. where Ç is scalar hyperparameter. Consequently, we update a, as follows: a +- 0j C. The intuition behind this update is that if the model exhibits high uncertainty in predicting 0; (i.e., predicting pre-evidences ě,), we enforce more conservative belief representation by reducing aj accordingly. This adjustment prevents excessively large values of ar and ensures a more calibrated uncertainty estimation. As a result, the updated uncertainty mass is given by: u = Σ, a, .. This approach effectively integrates the variance of pre-evidences into the calibration process, leading improved uncertainty quantification in IB-EDL..
Determining the optimal value of Ç: Theoretically, the EDL model can still be overconfident or underconfident after training, leading to an overestimation underestimation of the uncertainty mass u. Since may require adjustment in either direction, we allow the hyperparameter Ç E IR be either positive or negative, enabling calibration in both cases. To determine the optimal Ç. we recommend to analyze the calibration curve (Guo et al. 2017) on the training or validation set, which plots accuracy against confidence for binned samples. If the calibration curve lies below the optimal diagonal line, it indicates that the model is overconfident. In this case, we set Ç >> 0 to encourage greater uncertainty and improve calibration. Conversely, if the calibration curve lies above the diagonal, the model is underconfident, and set to increase certainty and correct for underconfidence.
Values of ( used in additional experiments: It is important to note that this post-hoc calibration technique is an optional component of IB-EDL and is not required in all cases. the experiments presented in the main text, omit this technique. In the calibration experiment presented in Table [2] we use ( -1. for OBQA -> ARC-C, Ç = 3.0 for OBQA - ARC-E, and Ç = -5.0 for OBQA- CSQA.
OOD test sets
21

---

Published as a conference paper at ICLR 2025
Table 7: OOD detection performance on fine-tuned Mistral-7B. A B indicates as the ID training set and B as the OOD test set. MP and UM are two scores for measuring AUROC.
OBQA ARC-C
OBQA ARC-E
OBQA CSQA
Model
Method
AUROCT
AUROCT f
AUROC
MP
UM
MP
UM
UM
MAP
60.4011.36
33.3011.16
63.70
MCD
60.39+1.37
53.30#1.16
13.
Ens
60.67.0.87
54.005.0.88
63.80.1.09
LA
61.61+1.05
53.
68.31
Mistral-7B
EDL
84.17:1187
77.3
81.81+2.81
6..1
83.55
78. 15.88
VID
86.30+33.80
84.74=0.80
88, 18.166+1.87
90.68+0.17
88.93=1.89
81,45+2004
I-EDL
85 02.0.90
62.2811.44
82.6711.34
79.42.12.3
85.08
82.07
R-EDL
76.26+1.05
72.85=0.61
71.95+0.78
67.56+0072
7..59=1.32
71.93+2.05
IB-EDL
90. 28
88.5311 08
89.54.1..1
94.29-0.49
90.45.1.. 13
83. 83.85.12.55
E ADDITIONAL EXPERIMENTAL RESULTS
E.1 ID CALIBRATION AND OOD DETECTION RESULTS OF MISTRAL-7B
In this subsection, we present the ID calibration and OOD detection performance of the fine-tuned Mistral-7B model. Table 66 provides overview of the calibration performance on ID datasets, while Table repports the OOD detection results.
E.2 ADDITIONAL RESULTS OF FINETTUNING MODELS ON NOISY DATASETS
Table and Table and Table 10 show the results of fine-tuning Llama2-7B, Llama3-8B, and Mistral-7B on noisy datasets, respectively.
Results: In the context of noisy data, the primary objective to preserve accuracy. Therefore. in Section (.4/of of the main text, we focus primarily on accuracy results. As highlighted in Section 4.2 a good uncertainty-aware method should achieve comparable accuracy while maintaining low calibration error, position shared by many other current research efforts (see, e.g., Chen et al.) 2024). Here, we thus provide additional results for the ECE and NLL metrics. the presence of label noise, Non-EDL methods tend to underperform EDL methods significantly in terms of accuracy. In the comparisons methods, which manage to strike the right balance of good accuracy and uncertainty quantification. Among these, IB-EDL demonstrates the highest accuracy and achieves the lowest ECE and NLL compared other EDL approaches, highlighting its robustness against label noise.
Tables S to 10 we therefore focus on the comparison EDL
E.3 ADDITIONAL OOD DETECTION RESULTS ON MMLU-MATH DATASET
We conduct additional OOD detection experiments using Llama2-7B in a setting characterized significant distribution shift. For this purpose, we select a subset of the MMLU dataset (Hendrycks) 2021a b) as the OOD test set, which includes data samples from the math-related topics col collegee ege ma hathemat/ ca, h gh school mat mhathemaiico cs, and abst ract. al gebra. We re- fer this subset as the MMLU-Math dataset. In addition, we use the OBQA dataset as the ID dataset. While OBQA focuses common-sense reasoning, the MMLU-Math dataset requires ad- vanced mathematical knowledge to solve its questions. As a result, the distribution shift in this setting is substantially larger compared to settings like OBQA - ARC-C.
As shown in Table I1 IB-EDL consistently achieves the best OOD detection performance when us- ing either MP UM the detection score. Additionally, general trend is observed: the AUROCs of IB-EDL and other baselines improve as the distribution shift increases compared to the setting OBQA ARC-C.
22

---

Published as a conference paper at ICLR 2025
Table Fine-tuning Llama2-7B on noisy datasets. each training dataset, the labels of 30% sam- ples are randomly perturbed. A robust uncertainty-aware method should not only maintain accuracy but also exhibit low calibration error. Non-EDL methods tend to significantly underperform EDL methods in terms of Accuracy. Therefore, the comparison for ECE and NLL is limited to EDL methods, with the best and second-best values among them highlighted.
Metrics
Method
ARC-C
ARC-E
OBQA
CSQA
SciQ
RACE
MAP
51.08=2.96
71.39=2.73
73.93=2.1
74.56=0.31
88.63 33.6332..0 06
76.83=0.30 39
MCD
51.08=2.96
71.40=2.12
73.93=2.20
74.56=0.32
88.64+0.06
76.95+00.19
Ens
56. 36.10
76.03=1.99
75.23=1.00
74.59=0.25
88 28..4420.0m 06
76.94 (6.944-0.22
LA
53.38=2.17
74.90=1.76
74.57=2.22
74.59=0.20
88 38.47::0
77.04:20.3
Acc f
EDL
57.16
76. 16.12
14.46=1.33
74.65
89.03::0.0
7.69 44
VID
57.56-0066
76.57=1.79
76.67=1.42
70.97=
89.06++0.22
78.75+0.22
I-EDL R-EDL
13.86
06.08=0.71
77.00=0.88
74.01
89. 0.332
76.48+0.88
57.00
76.96=1.07
73.20=1.44
74.36=00.99
87.33
78.0140.53
IB-EDL
59.06=2.07
80.27:0048
78.13#0.90
76.35+0.60
89.06+0.50
79.41+0.59
MAP
21.89=0.11
8.95
8.21 20.78
4.45#1.61
19.05==0.17
15.87=0030
MCD
21.87=0.12
8.91 1..10
8.23=0.82
4.47=1.55
19.04=0016
15.84
Ens 8.24
7.83
11.63=1.12
4.42#1.02
18.9 910.121
15. 15.904
LA 7.1221106
27.02=1.30
22.06=2.11
22.57=0.00
26.28
24.70:0. 24
EDL 8.38
26. 26.16
35.64=2.97
41.29
46.47 17.7700
30.59+ 1.14
VID
19.14
13. 18:..84
15.00=0.98
13.76=0.90
23.49 4.0.34
21 11.22:22 2..0.3.33
I-DDL
13.97=9.04
35.30=1.74
37. 37.56=0.63
38.77#1.60
48.60+0.85
36.34+1.00
R-EDL
12.25
20.67=2.30
28.02=2.63
3..64:0.29
39.58
30.42
IB-EDL
11.21=2.77
13.48=1.61
12.30
10.48:0.82
23.47:00.1
21.15::0.50
MAP
1.9410,06
1,05:1.009
0.74 0.74
0.84
0.50
0.73
MCD
1.94
1.05
0.74
0.81
0.49
0.72+00.02
Ens 1.3740.21
0. 7..00
=.73
0.83
0.49
0.73
LA
L.3240.06
1.03
0.85=0.00
0.92
0.56a00.0
0.81
N
EDL L.94+..0
0.97=40.08
1.05=0.02
1.35
0.92#0.01
0.90
VID
1.27+0.01
00.0
0.76=2.03
0.85=0.01
0.54
0.71
I-EDL
1.2910.12
0.002
1.03
1.20
0.95
1.02 01
R-EDI
1.2440.01
0.92=0.01
0.95=0.03
1.1320.01
0.83
(1.93+0.01
IR-EDL.
1.1820.03
0.7411.0s
0.71
0..0
0.52.20.1
0.71=0.01
E.4 ADDITION CALIBRATION RESULTS ON OOD TEST SETS
In this section, we evaluate the calibration performance on OOD test sets assess whether the uncertainty-awarn methods can generalize effectively to OOD datasets. Specifically, we fine-tune Llama3-8B on the OBQA dataset and evaluate it on three OOD test sets. As shown in Table [2] IB-EDL achieves the best ECE and NLLL on two out of the three OOD test sets, demonstrating that its calibration performance generalizes well to OOD datasets.
E.5 ANALYSIS TRAINING AND INFERENCE SPEED, MEMORY CONSUMPTION
In this section, we evaluate the complexity of uncertainty methods using three key metries: (1) the number of samples processed per second during training: (2) the number of samples pro- cessed per second during inference; and (3) GPU memory consumption during training. For this experiment, we use Llama3-8B in mixed precision as the model and OBQA as both the training and test dataset. All experiments are conducted on a single NVIDIA H100 GPU. As shown in Ta- ble 13 IB-EDL demonstrates comparable training and inference speeds well similar memory consumption to MAP and other EDL baselines. Moreover, IB-EDL attains faster inference speeds compared to methods such MCD and Ens, which require multiple forward passes, and LA. which involves gradient computation during inference. This substantial improvement in speed further high- lights the computational efficiency of IB-EDI..
23

---

Published as a conference paper at ICLR 2025
Table Fine-tuning Llama3-8B on noisy datasets. each training dataset, the labels of 30% sam- ples are randomly perturbed. A robust uncertaint--aware method should not only maintain accuracy but also exhibit low calibration error. Non-EDL methods tend to significantly underperform EDL methods in terms of Accuracy. Therefore, the comparison for ECE and NLL is limited to EDL methods, with the best and second-best values among them highlighted.
Metrics
Method
ARC-C
ARC-E
OBQA
CSQA
SciQ
RACE
MAP
57.71
80.34=1447
78.78=1.0 .00
77.04=0.41
92.60 53
86.93+o.1 11
MCD
57.71=0.42
80.37=1.46
79.00=0.92
77.05=0.41
#0.87
86.93
Ens
63. 1.000
84.63=0.53
80.61 =0.53
77.62:0.41
02.9720.00 06
86 86.93+0.00
LA
66.62
82.64=0.50
00.5
77.80=0.19
92. 92.93 21
87.00+0.01
Acc f
EDL
0.9.8
0.15
0.76
09.14:0.41
92.90
86.26.1.. 76
VID
66.58=1.92
87.67=0.99
84.86=1.0
79.66:1122
93.23++0.00 25
86.86++0.22
I-DDL R-EDL
63.87=2.65
84.06=2.60
84.26=0042
78.02=0.87
92.53=00.00 37
36.01
71.25=1:20 84.91=3.91
85.73=0.70
#0.57
93.56
86.16#:0.4
IB-EDL
68.53-0.25
88.05#0.48
86.13
79.59+0.79
93.46+0..8
87.01=0.20
MAP
13.26=0.85
8.52=#0.63
3.722
8.43+1.0
16.29 4.44
19.61 19.61=t0 +0.25
MCD
12.96
8.51 #00622
4.17=1..5
8.40=1.0
16.29
19.64
Ens 10.41=0.74
11,09=0.59
4.24
8.21+0.21
16.15+:0.10
19.60
LA
19.02=0.78
23.63=1.74
11.61
18.05=0.88
18.36 37
20.99
EDL =0.79
24.29=0.81
=1.29=0.62
34.19#1.93
48.36+00.4 43
3/.92+0004
VID
5.90
09.58
19.53=0.77
15.22:1115
26.30
22. 25.2550022
I-EDL
9.57+2.10
33.57=6.06
42.64=0.36
41.89=1.18
50.13+0.31
42.49=0.. 19:20.220
R-EDL
14.50=2.55
23.91
35.09
30.95
41.89
30.74:1099
IB-EDL
6.73+1.30
19.49
17.38
11.73:1.11
25.
20.00:0..3
23.43+0.33
MAP
1.3910.05
0. 0.78
0.64
0.83
0.39
0.56.0.0 01
MCD
1.38+0.02
0. 0.78
0.6420..3
0.82=0.02
0.40=o.01
0.57
Ens 1.1510.05
O. 64.40.03
0.61
0.7820.02
0.39
0.57
LA 1.11#0.01
O. 80.0..22
0.63
0.85#0001
0.39
0.57
N
EDL 0.94
0. 0.70=1..01
0.92=0.01
1.04=0000
0.85#0.00
0.78
VID
0.98
0.62:2..2
0.63=0.01
0.80=0.02
0.49
0.57:1.00
I-EDL
1.0310.03
0.87:1.06
0.96=0.01
1.16:0.00
0.89
0.91
R-EDI
1.00+0.02
0.76220.0
0.83
0.99
0.72#0.00
(1.33+0.01
IR-EDL.
0.99
0.60:2.01
0.57+0.01
0.74:20.02
0.46=2.01
0.57=20.11
E.6 SENSITIVITY ANALYSIS ON THE NUMBER OF BINS OF ECE
Following Yang et al. 2024), we use 15 bins by default when measuring ECE. To assess the im- pact of this hyperparameter,
conduct sensitivity analysis by varying the number of bins across (10, 15, 25, 35). For this experiment, we train and test Llama3-8B on the OBQA dataset and caleu- late the ECE for each bin setting. As shown Table [4] although ECE values increase slightly with a higher number of bins, the relative rankings of the methods remain largely consistent.
24

---

Published as a conference paper at ICLR 2025
Table 10: Fine-tuning Mistral-7B noisy datasets. In each training dataset, the labels of 30% samples are randomly perturbed. A robust uncertainty-aware method should not only maintain ac- curacy but also exhibit low calibration error. Non-EDL methods tend significantly underperform EDL methods in terms of Accuracy. Therefore, the comparison for ECE and NLL is limited to EDL methods, with the best and second-best values among them highlighted.
Metrics
Method
ARC-C
ARC-E
OBQA
CSQA
SciQ
RACE
50.65
66.30=2.16
75.72=0.46
69.81
92.96+0.32
85.47 17.4740..0 +..35
MCD
50.65=0.30
66.21=2.44
75.46=0:24
69.77=0.51
92.93+0.3 31
85.46:00 35
Ens
58. 25.14
76.10=b.x
76.93=2.14
74.18=1.50
92.93+0.30
85.62+0.22
LA
66.35=0.63
75.48=1.88
78,20=1.25
74.24=1.01
03.02:10.0 29
Acc f
EDL
6..63=1.80
85.01
83.20=1.25
76.63+0.08
93.33+
84.91
02.27=2.18
1..9
82.06=1.51
17.31
93.20
85.50
IEDDL
68. 11.19
79.73=0.55
81.06=0.81
=0.09
93.33++0.00
84.4420 28
R-EDL
76.39=0.00
86.85:00669
82.76=1.73
77.10=1.10
33.09220.0 78
85.07
IB-EDL
66.44
85.17=1.39
84.3310.51
77.44:0039
93.63.1..4
85.68 15.68810.22
MAP
=0.08
9.95
7.83
9.61
15.11 11..0.25
17. 17.80:10.33
MCD
18. 18.24=0.65
1.14
7.82=0.61
9.60=0.90
15.10=9.22 27
17.80=0.35 33
Ens
10.94=0221
12.40=0.34
7.02=0.61
8.72#0.29
15.09
18.5822..
LA
18.50=1.46
17.31=0.23
13.92=1.45
15.04=0.41
18.13
18. 18.32
ECE
12.76=0.41
19.84=0.95
322.7==1.8m
25.26+#6.63
48.55+t0.35
34.06+t0.11
6.27
13.17
13.9210.97
11.66=1117
26.45400100
22.01:0.08
I-EDL
13.14
21.44=1.54
15.1071007
33.29 1:004
50,90 44
41.194.0.2
R-EDL
17.63=2.83
17.57=0.42
18.55=1.66
21.35=1.22 .444
41 11.4820.22 26
29. 29.4002.33
IB-EDL
1.46+1.58
19.08=3.08
14.90=0.50
11.58#0.48
22.04+0.00
MAP
1.67+0.03
0.98+0.00
0.73==0.05
0.98=0.03
0.38
0.56+0.00
MCD
1.6710.08
0.98
0.70
0.98
0.38
0.5610.01
Ens
1.32-40.01
0.73+20.03
0.64
0.81 =0.02
2.39
0.57
LA L.0840.03
0. 0.830
0. 0.70=0.03
0.93
0.40a00.0
0.56200.00
NLL
EDL 0.9310.05
0.67:
0.82=0.05
0.99
0.85+20.00
0.78 01
VID
1.04+0.06
0.7
0.63=0.01
0.84=0.01
0.50=0.01
00000..0
IEDDL 0.96:1001
0. 0.70220.2
0.90
1.05
0.89
0.91
R-EDL
1.90
0.6410.02
0.64
0.97
0.72
0.72
IB-EDL
0.98
0.60
0.57
0.81#0.01
0.
0.60=0.01
Table 11: OOD Detection AUROC Llama3-8B in the setting OBQA MMLU-Math. IB-EDI. achieves the best performance even under significant distribution shifts, such as transitioning from common-sense reasoning dataset like OBQA math-focused OOD dataset.
OBQA MML.-Math
Method
AUROC1 t
MP
UM
MAP
91.36+00.57
MCD
90.85+0.33
Ens
90.68110880
LA
91.09110.41
EDL
92.78110.26
02.86
VID
91.6440.79
66.61+4.98
I-EDL
91.48+0.72
90.67 +0.88
R-EDL
88.44+2.11 88,22+1.70
IB-EDL.
93.63.10.68
93.6410.55
25

---

Published as a conference paper at ICLR 2025
Table 12: Calibration performance of uncertainty-awee methods on fine-tuned Llama3-8B in the OOD setting. The model is trained OBQA and tested on three different OOD test
Metries
Method
OBQA 7 ARC-C OBQA ARC-E
OBQA CSQA
MAP
79.1840.45
88.06+0.20
69.37+0.07
MCD
79.16.10.43
88.05.0.28
69.38.1066
Ens
79.27:40.55
88.15.10.02
69.14:10477
79.38=u.44
88.36=0.224
69.3414v.58
Acc f
EDL
78.27
86.38
69.344+0.8
VID
18.27
87.4110.82
69.9911.07
I-EDL
18.35
87 87.5510.20
70.4910.56
R-EDL
78.32
87.3110.90
70.62
IB-EDL
18.31
87.9410.22
71.29
MAP
18.04
9.63+0.47
27.85=t0.53
MCD
18.06-20.36
9.63+t0446
27-86620.46
Ens
16. 13.113
9.1910065
27 4:.37:1122
LA
6.61
3.0 3.02=0015
11.99
ECE
EDL
7.65+
10.11
8.32
VID
8.74+1.19
3.19
16.66+to.73
I-EDL
1.68
11.3810.65
5.96110.71
R-EDL
5.031
5.32.00.66
12.8411.73
IB-EDL
1.69
5.03:10.15
1..1
MAP
1.30+0.02
0.71+
2.06+00.04
MCD
1.33
0.70+t004
2.13+0.13
Ens
1.19
1.68
2.0110.110
LA
0.732
0.42=0.00
1.15
NLL
EDL
0.74+0.01
0.52+0.01
0.98+
VID
0.78
0.46#0.02
1.06+0.02
I-EDL
0.7310.00
0.5310.01
0.9410.01
R-EDL
0.7310.00
0.4610.01
1.00.0.05
IB-EDI.
0.72:0.02
0.44+0.02
0.93:2.02
Table 13: Comparison of computational efficiency for various methods using Llama3-8B the OBQA. IB-EDL demonstrates comparable training and inference speeds as well as memory con- sumption compared to MAP and other EDL methods, confirming its computational efficiency.
Method
Test Samples/s f Training Samples/s
Memory (GB) at Training
MAP
69.55+2.86
2..57+1.96
21.21=0.35
MCD (10 forwards)
9.79+1.21
Ens (3 models)
25.7743.54
LA
5.95.10.49
EDL
68.99+1.59
26.4411.11
21.23=0.15
VID
69.17:10.9
26.69
21.29
I-EDL
68.94+2.15
26.02+0.71
21.33+0.11
R-EDL
68.84+1.09
26.47+1.09
21.27=0.21
IB-EDL
68.08+1.75
26.41+1.04
21.88-0.06
26

---

Published as a conference paper at ICLR 2025
Table 14:
Sensitivity analysis of ECE with respect to the number of bins. We train and test Llama3- 8B on the OBQA dataset. The results show that while the ECE values increase slightly as the number of bins increases, the relative rankings of the methods remain consistent.
Method
Bins = 10
Bins 15
Bins 25
Bins = 35
MAP
10.45:10.52
10.52
10.99:1.01 10.89
MCD
10.37
10.48:0..6
10.69=0.59
10.83+0.76
Ens
10.1140.13
10.08+0.90
10.92+0.84 10.41
LA
5.20+11.9
5.26+1130
6.33::1.11
6.42+0.96
EDL
8.16#1.25
8.28+1.62
8.78+1.27
9.44+1.79
VID
5.29
5.99.1..41
7.1611159
7.34
LEDL
1.33
7.57
8.20
9.01
R-EDL
4.64+0.87
4.68+1.35
4.81+1.09
5.47=0.82
IB-EDL
2.77:0.61
2.34
3.91
1.54
27


Reviews:
[{"review_id": "4zlNQhQwqF", "reviewer": "Reviewer_TTkc", "summary": "The paper present a method to effectively apply Evidential Deep Learning (EDL), a parametric uncertainty estimation approach, to LLMs. The approach introduces two features to standard EDL that make it practical and effective for application to LLMs; i) the introduction of an information bottleneck cost function, which mitigates over-fitting and ii) the use of the LLM outputs as the intermediate \"layer\" Z where to impose the bottleneck, which simplifies direct computation of certain terms in the mutual information lower bounds, part of the objective function.", "questions": "See and clarify the weakness point above.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- Relevance: The paper addresses an important problem; the uncertainty calibration of LLM outputs (at the token level), which is an important component for providing safe deployment and error estimation for LLMs.\n\n- Theoretical Soundness: The paper takes a very principled approach, identifying precise issues in the application of EDL to LLMs and addressing them with mathematically derived solutions based on reasonable starting assumptions.\n\n- Experimental Soundness: The experiments are more than sufficiently extensive: they target the right settings and metrics to validate the claims of the paper and comprehend great breadth of models, tasks and mode of evaluation (acc+ECE, OOD detection). The claims of improved uncertainty calibration are strongly backed by the experiments.", "weaknesses": "The only main weakness of the paper is the clarity of scope:\n\nFrom the experiments, it is clear that this is a method for uncertainty calibration for a specific task at the fine-tuning stage of an LLM. e.g., an LLM is fine-tuned for summarisation and the proposed method offers a way to obtain superior uncertainty calibration *for the fine-tuned task*. This is indeed very useful, but needs to be stated more clearly in the abstract and intro, as from all sections up until the experiments it is unclear if this is a method for pre-training and/or fine-tuning. In principle, I can see the method may be applied to pre-training too, but since this is not tested in the experiments it should not be included in the claims.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HD4N2COcua", "reviewer": "Reviewer_tujz", "summary": "This paper proposes IB-EDL, combining Evidential Deep Learning (EDL) with information bottleneck (IB) for uncertainty estimation in large language models (LLMs). This approach aims to enhance model calibration by reducing overconfidence, which is a common issue in fine-tuned LLMs. The experimental results show that IB-EDL achieves improved calibration with minimal computational overhead.", "questions": "Please see weakness.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is well-presented and easy to follow.\n\nThe paper proposes a unified perspective on EDL methods by framing several existing EDL methods as special cases within the IB-EDL framework.\n\nThe experimental results are strong and covers multiple LLMs, showing that IB-EDL addresses the overconfidence problem well.", "weaknesses": "The pipeline of applying IB to EDL ($x → f(x; θ) → e˜ → e →\nα → π → y$) seems computational heavy, it would be informative to include an inference time comparison of IB-EDL against other methods. \n\nAlthough the training overhead of IB-EDL is relatively small compared with the pretraining step, the paper lacks comparison of training time compared with LoRA, which is the fine-tuning backbone of IB-EDL.\n\nFor out-of-distribution (OOD) scenarios, the paper only compares the OOD detection capabilities with AUROC. The expected calibration error (ECE) is missing, which is also an indicator of OOD performance as the calibrated model should maintain relatively low ECE score on OOD datasets.\n\nThe novelty is not clear for combining IB with EDL, which are two well established methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wWtH2FdQLW", "reviewer": "Reviewer_qw6T", "summary": "In this work, authors propose a new loss regularization term to reduce overconfidence in LLMs when training via the evidential deep learning (EDL) framework. This regularization is based on the information bottleneck loss, which maximizes mutual information between token logits and ground truth prediction distribution while minimizing M.I. between predictions and inputs. This is achieved by minimizing a KL divergence between a learned conditional logit distribution and a Gaussian prior. This work demonstrates that fine-tuning via this loss produces better calibration (ECE, AUC) with equivalent accuracy to baselines. Authors also illustrate this method is robust to noise in the training data and distribution shifts.", "questions": "* While it appears the training methods/baselines tested are normalized for number of training steps, I am curious about comparison in terms of the number of training flops. Do the new mean and covariance prediction heads require separate training? \n* Is there a benefit to having two parameters estimating the distributional shift (mean and variance) of the token predictions vs. just one?\n* What happens to calibration if we finetune on multiple datasets simultaneously?\n* How does this fine-tuning procedure effect LLM performance on open-ended generation tasks?", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* This work is well-motivated theoretically, pointing out deficiencies with previous EDL losses (e.g. paragraph *Challenges when applying IB to an internal layer of an LLM*). \n* The final regularization term $\\mathcal{L}_{IB-Info}$ seems relatively simple, although I do have some questions about practical implementation (see questions)\n* Strong experimental results; the OOD detection and noise injection experiments (tables 3,4) in particular seem support the argument for this additional regularization during calibration", "weaknesses": "* This work seems only applicable for a discrete set of mutually exclusive classes. I'm not sure how this extends to the case of open-ended generation where multiple responses may be semantically equivalent or have different logical relationships.\n* While experiments show improvements in OOD detection when moving from OBQA--> ARC,CSQA datasets, I'd be interested to see the performance when the datasets are 'futher apart' semantically (e.g. moving from a reading comprehension to a math task).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LCWcsBdVNJ", "reviewer": "Reviewer_4Vtx", "summary": "The paper presents the Information Bottleneck (IB) - Evidential Deep Learning (EDL) method, designed to regularize EDL for better calibration of large language models (LLMs). The empirical experimental results demonstrate that IB-EDL enhances calibration in fine-tuned LLMs.", "questions": "Please refer to the strengths and weaknesses. Additionally, how many bins did the authors use to compute the ECE? It is beneficial to conduct a sensitivity analysis on the number of bins.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is well-written and easy to understand, trying to address the important task of calibrating fine-tuned large language models. The empirical experiments are comprehensive, testing across multiple LLMs including Llama2-7B, Llama3-8B, and Mistral-7B. The paper covers most of the baselines. The LoRA-based Mixture of Experts idea [1] could also be relevant and serve as a baseline for comparison. The empirical performance appears promising.\n\n[1] Li, Dengchun, et al. \"Mixlora: Enhancing large language models fine-tuning with lora based mixture of experts.\"", "weaknesses": "The novelty of the paper is limited, as both EDL and the IB method are well studied in the literature. The primary contribution of the paper appears to be the implement IB on EDL. The authors explain that the IB objective allows the latent variables to retain the most predictive information about the target variable while discarding irrelevant information from X. However, a more detailed discussion on how this relates to improved calibration of LLMs is missing. Additionally, there is a lack of theoretical guarantees supporting this approach. It is beneficial to provide a theoretical bound on calibration error or a proof of convergence.\n\nIn terms of the empirical analysis, the paper highlights computational efficiency as a key contribution of the proposed method, yet it lacks experimental comparisons of computational time and memory costs to validating this claim. The paper also lacks the calibration performance analysis in out-of-distribution scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)