Bayesian Low-rank Adaptation for Large Language Models
Authors: Adam X. Yang
Maxime Robeyns
Xi Wang
Laurence Aitchison
Venue: ICLR 2024 poster (2024)
Keywords: Large language models
Bayesian deep learning
Laplace approximation
uncertainty calibration
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Abstract: Parameter-efficient fine-tuning (PEFT) has emerged as a new paradigm for cost-efficient fine-tuning of large language models (LLMs), with low-rank adaptation (LoRA) being a widely adopted choice. However, fine-tuned LLMs often become overconfident especially when fine-tuned on small datasets. Bayesian methods, with their inherent ability to estimate uncertainty, serve as potent tools to mitigate overconfidence and enhance calibration. In this work, we introduce Laplace-LoRA, a straightforward yet effective Bayesian method, which applies the Laplace approximation to the LoRA parameters and, considerably boosts the calibration of fine-tuned LLMs.

Published as a conference paper at ICLR 2024
BAYESIAN LOW-RANK ADAPTATION FOR LARGE LAN- GUAGE MODELS
Adam X. Yang
Maxime Robeyns' Xi Wang" Laurence Aitchison'
University of Bristol University of Massachusetts, Amherst adam. yang, maxime... robeyns. 2018, Laurence aitccissonnoo /ttpisssnebbriittooo uk xwang30cs cs. umass. edu
ABSTRACT
Low-rank adaptation (LoRA) has emerged as a new paradigm for cost-efficient fine- tniin of large language models (LI.Ms). However, fine-tuned L.IMs often become overconfident especially when fine-tuned on small datasets. Bayesian methods, with their inherent ability to estimate uncertainty, serve as potent tools to mitigate overconfidence and enhance calibration. In this work, we introduce Laplace-LoRA, which applies a Bayesian approach to the LoRA parameters. Specifically, Laplace- LoRA applies Laplace approximation to the posterior over the LoRA parameters, considerably improving the calibration of fine-tuned LLMs.
INTRODUCTION
In recent years, fine-tuning large language models (LLMs) have become increasingly important (Houlsby et al., 2019; Hu et al., 2021; Liu et al., 2022; Ding et al., 2022; 2023). Fine-tuning used both to adapt LLMs for specific tasks and to create general instruction- -ooruuttioooooooiinn models (e.g. using Reinforcement Learning from Human Feedback; RIHF Wei et al., 2021; Ouyang et al., 2022; Chung al., 2022; Wang et al., 2022).
However, fine-tuned LLMs have notable limitation: they often exhibit overconfidence (Jiang et al., 2021; Xiao et al., 2022: He et al., 2023; Tian et al., 2023; OpenAI, 2023). This is particularly problematic in safety-critical applications or when making decisions in areas where limited data is available, such as medical diagnosis, finance and experimental design (Singhal et al.. 2022: Wu et al., 2023; Lampinen et al., 2023; Li et al., 2022). Consequently, there is an urgent need for strategies that enhance the calibration of fine-tuned LLMs, ensuring that their predictions are as trustworthy as they are powerful.
Bayesian deep learning is commonly proposed as a solution to overconfidence in deep networks (e.g. Blundell et al., 2015; Zhang et al., 2019; Kristiadi et al., 2020; Ober & Aitchison, 2021; Fortuin al., 2021; Aitchison et al., 2021). Historically, the field of Bayesian deep learning has frequently considered ResNets for image classification (Shridhar et al., 2019; Dusenberry et al., 2020: Izmailov et al., 2021). While there are a few previous papers on Bayesian language models (e.g. Tran et al., 2019; Xue et al., 2021; Fan et al., 2020: Cinquin et al., 2021; Zhang et al., 2019; Chen & Li, 2023), most of them are focused on language model pre-training (though see Related Work for exceptions). These methods have not yet seen widespread adoption, potentially because they costly to scale up to larger models, and large-scale pre-trained models are already reasonably well-calibrated (OpenAI, 2023).
Importantly, the advantages of Bayesian fine-tuning are much clearer than the advantages of Bayesian pretraining. First, fine-tuned models are typically poorly calibrated (Jiang et al., 2021: Xiao et al., 2022; He et al., 2023; Tian et al., 2023), even after very large-scale instruction fine-tuning (OpenAI, 2023). and Bayes might help us improve this calibration. Second. this poor calibration may arise in fine-tuning settings, as there is often far less data available in fine-tuning than pre-training. Bayes might help us reason about uncertainty given these limited datasets. Finally, the computational burden of fine-tuning is typically much smaller than that of pre-training, implying more headroom for any additional computational costs that might arise as result Bayesian inference.
"Equal contribution.

---

Published as a conference paper at ICLR 2024
However, LLMs can be very large. Indeed, fine-tuning all the weights of an LLM is often prohibitively costly, in which case Bayesian tine-tuning will of course also not be possible. Instead, a number of more efficient approaches have emerged under the banner of parameter-efficient fine-tuning (PEFT Liu et al., 2022; Ding et al., 2022; 2023; Shi & Lipani, 2023). PEFT methods tend to train a small number of additional parameters on top of the underlying fixed-re-trriieed L.M.M For instance, common approach is to fine-tune only low-rank adapters for each weight matrix (LoRA; (Hu et al., 2021)). PEFT approaches such LoRA have democratized LLM fine-tuning and are implemented in a number of widely adopted libraries including OpenDelta (Ding et al., 2023) and PEFT (Mangrulkar et al., 2022).
Importantly, PEFT approaches such as LoRA should also enable more efficient Bayesian finetuning. We develop the first Bayesian inference method designed specifically for the LoRA parameters in LLM fine-tuning. Specifically, we use post-hoc Laplace approximation (MacKay, 1992; Riterr et al., 2018; Daxberger et al., 2021b;a: Antorán al., 2022), which allows us to keep the standard, highly efficient pre-training and fine-tuning pipelines exactly the same. We then use Laplace to compute the uncertainties over just the small number LoRA (Hu et al., 2021) parameters. We call the resulting method Laplace-LoRA, and show dramatic improvements in calibration on fine-tumed settings.
LlaMA2-7B on six common sense reasoning tasks and in
RELATED WORK
Past work on integrating Bayesian inference with language models has usually operated in the large- scale pre-training setting (Tran et al., 2019; Xue et al., 2021; Cinquin et al., 2021; Zhang et al., 2019; Chen & Li, 2023), where the advantages of Bayes are unclear, because pre-training datasets are very large and large-scale pretrained models seem to be reasonably well-calibrated even without Bayes (Kadavath et al., 2022; OpenAI, 2023). In contrast, our work operates in the fine-tuning setting, where the advantages of Bayes are far more evident, e.g. because even large-scale instruction fine-tuning gives poor calibration (OpenAI, 2023).
While Fan et al. (2020) and Zhang al. (2021) do consider fine-tuning, they make the unusual choice to define a prior and approximate posterior over the attention weights, rather than over parameters (i.e. weight matrices). Our method - LoRA- has two key benefits over their approach. First, Laplace LoRA is post-hoc method that keeps the pretraining and fine-tuning process exactly the same. Therefore Laplace LoRA is able exploit very efficient pre-existing implementations of these methods (Ding et al., 2022; Mangrulkar et al., 2022; Dettmers et al., 2023), which is not possible when defining priors and approximate posteriors over attention weights. Second, Laplace LoRA massively reduces the dimensionality of Bayesian inference problem, either as against full finetuning where we would need to reason about the posterior over all weights, or as against Fan et al. (2020) and Zhang et al. (2021), which must reason about the posterior over attention weights at every layer for every fine-tuning datapoint. There are 32 attention heads per layer and 32 layers in LLaMA 7B, if we assume 1000 sequences of length 30 in the finetuning set, that gives 32 layers x32 heads per layer x30 attention weights per head x 30 sequences R one billion attention weights. That one billion attention weights contrasts to only around 6 million LoRA parameters in our finetuning.
Laplace inference for Bayesian neural networks is well-studied (Ritter et al., 2018; Kristiadi et al., 2020; Immer et al., 2021; Antorán et al., 2022: Daxberger et al., 2021a; Deng et al., 2022). However, the only application of Laplace approximations to language models that we know of comes from Daxberger al. (2021a) which used DistillBERT (a 66 million parameter model) a feature- extractor, and applied Laplace inference only at the final linear last-layer. In contrast, in our work we consider Laplace inference at all layers in models 100 times larger, by using LoRA adapters.
A separate line of research has been dedicated to regularizing language model fine-tuning to improve calibration. For instance, Mixout (Lee et al., 2019) stochastically substitutes model weights with their pre-trained counterparts. Park & Caragea (2022) adapted Mixup (Zhang et al., 2017) to augment the dataset during language model fine-tuning. Meanwhile, He et al. (2023) introduced a KL regularization between the output distributions of both fine-tuned and pre-trained language models, and added an Lg regularization to the final embedding vector. However, their regularization relies on the masked language model objective which only applies to BERT-like models. Critically, this body of work is orthogonal to our work in that they give better settings for the fine-tuned weights. Laplace-LoRA does not change the funed weights at all, it could use weights from standard
2

---

Published as a conference paper at ICLR 2024
fine-tuning, or it could use the improved weights given by these methods. Instead, Laplace-LoRA uses a Laplace approximation to estimate uncertainty around any given mean value of the weights. Laplace-LoRA thus has the potential to improve calibration for any good method for generating fine-tuned weights.
BACKGROUND
3.1 LOW-RANK ADAPTATION (LORA)
LLMs have large number of large weight matrices, denoted Wo E Ruoxxiin with inputs and outputs h. In LoRA (Hu et al., 2021), we keep Wo fixed, and introduce a perturbation to the weight matrix, AW:
h =Wa AWa Woa + BAa.
Critically, AW is low-rank it is written as the product of two matrices, B R 2h and A E IR (R X where für is significantly smaller than Yin or Toou (e.g. 4096), for instance, we use Mir Therefore, the total number of LoRA parameters for this weight matrix is mir(rin nloa), which is typically far smaller than the underlying number of parameters in the full matrix, Rout LoRA has shown great success in fine-tuning large scale models efficiently, and can be adapted to language or vision architectures (Mangrulkar et al., 2022). Note that one of the key motivations for introducing LoRA for LLM finetuning was the huge memory cost of maintaining the average gradient and average squared gradients of the e.g. 7 billion parameters in the optimizer. This multiplies the memory required by a factor of 3, relative to the memory required to just load the weights. LoRA massively reduces this memory cost to only 3 times the number of parameters in the LoRA adapters, as it only optimizes the LoRA adapters.
3.2 LAPLACE APPROXIMATIONS
In Bayesian inference for classification next token prediction, the goal is to find the full posterior,
P y.) x P (ylX, P (0).
Here, X E N is the input, where T is the set of possible tokens, N is the number of sequences, the (max) sequence length. The targets are denoted y E y.N, where y could be different from T (e.g. in sentiment classification) or could be the same T (e.g. for next token prediction). Further, P (0 y) is the posterior, P (yl0, X) is the likelihood (e.g. softmax Categorical distribution for classification tasks). We an isotropic Gaussian prior, with precision X,
(0) (O
Calculating this posterior is usually intractable. The Laplace approximation begins by finding the maximum a-posteriori (MAP) solution (MacKay, 1992) (i.e. the maximum of the log-joint, C(y, X; 0)),
Cly, X) = log (y X, 0) + log P (0) log P (0)X, + const
θMАР argmax C(y, X)
Then the Laplace approximation consists of a second-order Taylor expansion of the log-joint around OMAP.
1 0MAP) (V 0) (0 0MAP).
C(y,X:0) L(y, X; 0MAP)
(6)
2
Since the log-joint is now a quadratic function O, the approximate posterior becomes a Gaussian centered at OMAP with covariance given by the inverse of the Hessian,
P (0 N' (0; MMPP, E), Σ = -V (,,((yxx000 xx; ()))))
(7)
( log P (yl Iow AII-
(8)
3

---

Published as a conference paper at ICLR 2024
To ensure positive definiteness of the covariance, the general approach is transform gradients into estimates of the Hessian using either the Fisher information or the Generalized Gauss Newton (GGN) matrix (see Kunstner et al., 2019, for further details). We use the Fisher information,
N Σ PPlllfo(..)) Vo (vlfo(x.)) (Vo P (ol)
F(0)=
(9)
where the expectation taken over the model's output distribution.
Importantly, the full Hessian or Fisher is P x P matrix, where P is the number of parameters, For LLMs, this might translate to a matrix with dimensions of 7 billion by billion, which clearly intractable. Even if we consider the Hessian only over the low-rank adapters this matrix is still too large; for example, applying rank LoRA on Llama2-7B still yields roughly million trainable parameters. Consequently, structure on the Hessian. In particular, we consider either just the Hessian for the last-layer, or using Kronecker-factored (KFAC) structure for individual weight matrices (Ritter et al., 2018; Daxberger et al., 2021a). In KFAC, we approximate the Fisher using blocks for each linear layer. For the 4th linear layer, we compute the block by denoting the input as ar and the output by. Then, the Fisher is,
follow the usual Laplace approximation approach by imposing further
N
F
Ep Ep(ylf(xx.)) (av-1a 1) (g/g/))
(10)
where g: Vb, log (y X, 0) the gradient of the the log-likelihood gradient with respect the outputs.
Using Laplace approximations has strong connections to linearizing the neural network (Kunstner et al., 2019; Immer et al., 2021). As such, it is commonly found that predicting under the linearized model is more effective than e-g. sampling the approximate posterior over weights (Foong al., 2019: Daxberger et al., 2021a; Deng et al., 2022; Antorán et al., 2022), (see Appendix A for further details). In particular,
fo(x.) = (x, Vofo(x.) OAA (0- OMAP).
(11)
where x, is a test-input. This approach is also known as the linearized Laplace approximation.
Since have the approximated posterior in Eq. (7) and the linearized model in Eq. (11), we integrate out the posterior on weights and get a Gaussian posterior on output logits,
fx( N (Jern ),),A)
(12)
where
A (Volo(x.)) )V(Volox))))
(13)
Subsequently, we can optimize the prior precision À using the closed form Laplace marginal likelihood (model evidence) (Immer et al., 2021; Daxberger al., 2021a) on the training dataset,
P (y|X) - (yl 0) (0) d0 exp(Cly, X; (MA))) 1/2
(14)
where we have applied the Taylor approximation in Eq. (6). Crucially, unlike other post-hoc calibra- tion methods, post-hoc Laplace does not require separate validation set. This feature is particularly beneficial for small-scale datasets where training data is scarce. To obtain samples of fo(x.), we can decompose the covariance using the Cholesky factorization,
LL"..
fo(x.) = (oxne(x.) + I.g,
(15)
where & is vector of IID standard normal random variables. We compute the Bayesian model average by computing the average probabilities (passing the sampled logits through softmax function) under the Gaussian random noise from &. There are common approaches to approximating the Bayesian model average arising from Eq. (15), including the generalized probit approximation and the Laplace bridge (Daxberger et al., 2021a: Kristiadi et al., 2020; Lu al., 2020: MacKay, 1998), but we find that they perform considerably worse than naive Monte-Carle sampling, potentially due their ignorance of covariance terms (Appendix A).
4

---

Published as a conference paper at ICLR 2024
4 METHODS
Laplace-LoRA uses post-hoc Laplace Kronecker-factored upproximations for the LoRA adapters. Specifically, we treat the adapter BAx in Eq. (1) two separate linear layers with weights A E R*=x*= and E TR X respectively rather than as a single linear layer with low rank weight matrix. One issue with this approach is that the LoRA adapters are Mom x or d x Mom dimensional, so while one Kronecker-factor is small (Niora X Tijoa), the other is large (dxd, d, where d = 4096 in LlaMA2- 7B attention layers). These d x d Kronecker factors are roughly the same size as the underlying weight matrices, so representing them explicitly would eliminate any memory benefits of working with LoRA adapters. As such, we are forced to a low-rank representation of this Kronecker factor. Specifically, we use a rank-Tikfac representation (the rank of this low-rank approximation to the Kronecker factor will in general be different from the rank of the LoRA adapters).
To ensure that we retain memory efficiency at all steps, we need to be able to:
1. Compute the low-rank approximation to the Kronecker factor "incrementally" (i.e. all the computations are low-rank and we do not e.g. compute the full-rank factor first, and then find low-rank form; Appendix E.I).
2. Optimize the marginal likelihood using the low-rank approximation (Appendix E.2).
3. Low-rank linearized prediction (Appendix E.3).
5 RESULTS
We consider post-hoe Laplace approximations applied to LoRA parameters (Laplace-LoRA) at model checkpoints ОMAР obtained from standard fine-tuning. In particular, we used the PEFT library (Mangrulkar et al., 2022) for fine-tuning LlaMA2-7B (Touvron et al., 2023b) on common-sense reasoning tasks (multiple choice True/False classification). We applied LoRA to queries, values, and the output layer, using default hyperparameters from HuggingFace (Wolf et al., 2020; Mangrulkar carried out with batch size of 4 for 10,000
et al., 2022) (see Appendix C). The fine-tuning iterations. For True/False or multiple choice questions we selected the next token logits corresponding True/False or A/B/C/D depending on each dataset (refer to Appendix for our prompt templates), and fine-tuned the LLM to maximize the likelihood of the correct token.
Model checkpoints were saved every 1000 gradient steps, and each checkpoint, we applied post-hoc Laplace-LoRA with KFAC; we predicted using the linearized model as described in the Methods section. For clarity, we refer to the full Laplace-LoRA approximation (which is applied all LoRA fine-tuned weights) as LA, and the last-layer Laplace-LoRA approximation (targeting only the LoRA weights at the output layer to emulate standard last-layer Laplace approximation (Kristiadi al., 2020; Daxberger et al., 2021a) ) as LLLA.
We assessed the efficacy Laplace-LoRA by evaluating the negative log-likelihood and expected calibration error of LlaMA2-7B during fine-tuning on common-sense reasoning tasks. These metrics are further discussed in Appendix D. We also drew comparisons with established baselines including Monte-Carle Carlo dropout (Gal & Ghahramani, 2016) with ensemble size of 10 (with a dropout rate of 0.1 during fine-tuning), checkpoint ensemble (Chen et al., 2017) with an ensemble size of 3 using the most recent checkpoints when available, deep ensemble (Lakshminarayanan et al., 2023; Zhai et al., 2023) with three LoRA fine-tuned LLMs, and temperature scaling (Guo al., 2017). Since achieving good calibration and accurate uncertainty estimation is significantly more challenging on small-scale datasets (Zhao et al., 2020), our focus lies primarily on datasets comprising fewer than 10,000 training examples. We use the publicly available validation split across all benchmarks as the test set to evaluate performances at checkpoints.
al., 2017; Wang
5.1 IN-DISTRIBUTION FINE-TUNING AND EVALUATION
We began our evaluation with in-distribution fine-tuning the following common reasoning tasks: Winogrande-smal (WG-S), Winogrande-mediue (WG-M) (Sakaguchi et al., 2021), ARC- Challenge (ARC-C), ARC-Easy (ARC-E) (Clark et al., 2018), openbook QA (OBQA) (Mihaylov
5

---

Published as a conference paper at ICLR 2024
MAP
MC Dropout
Checkpoint Ensemble
Ensemble
LLLA
(640)
ARC-C (1.12k)
(2.25k)
(2.56k)
OBQA (4.96k)
BoolQ (9.43k)
70
90
80
90
ACC (88)
60
80
70
80
50
70
60
70
10
ECE 190
20
10
10
1,5
0.6
0.5
0.4
10000
10000
10000
10000
10000
10000
training steps
training steps
training Steps
training steps
training steps
training steps
Fine-tuning of LlaMA2-7B across six common sense reasoning tasks (presented column- wise, with number of training examples brackets), evaluated on the test set every 1000 gradient steps. The vertical dashed line gives the number of training steps with optimal MAP performance, and indicates that Laplace is likely to offer benefits even when combined with early stopping. MAP: standard fine-tuning: MC Dropout: Monte-Cario dropout; Checkpoint Ensemble: ensembling three most recent checkpoints; Ensemble: ensembling three LoRA fine-tuned models; LLLA: last-layer Laplace approximation LoRA weights in the output layer; LA: full Laplace approximation all LoRA weights.
Figure 1:
al., 2018), and BoolQ (Clark al., 2019) benchmarks One of the key advantages of Laplace approximations is that they give an approximation of the model evidence which can be used to tune hyperparameters on the training set alone, without needing separate validation set. Thus, we first considered setting with only a training set and test set, with no distinct validation set for hyperparameter tuning. For our Laplace-LoRA methods (LA and LLLA), we optimized the prior variances using the Laplace model evidence (detailed in Algorithm 2 in Appendix F).
Figure presents the accuracy (ACC), expected calibration error (ECE) and negative log- (NLL) the evaluation set for cach fine-tuned checkpoint. Since all other methods maintained similar ACC compared to MAP, our primary concern revolved around ECE and NLL. Notably, the very large NLLs for standard (MAP) fine-tuning training proceeds indicates that overconfidence remains an important issue when when fine-tuning Llama-7B. Monte-Carlo dropout offers marginal improvements over MAP, while checkpoint ensemble offers slightly more gains at the beginning. LoRA ensemble sometimes offer an improvement in ACC, but its improvements on ECE and NLL always limited. Conversely, LLLA seems give substantial improvements
some datasets (WG-S, ARC-C, ARC-E, but little if any improvement on others WG-M, OBQA, BoolQ). In contrast, LA consistently offers dramatic benefits in terms of ECE and NLL, indicating overconfidence on the evaluation set is dramatically reduced.
Additionally, we present these results in tabular form in Table 1. We stopped at 5000 steps to somewhat mirror the effect of early stopping (note however that choosing the correct point to stop is difficult not impossible determine this setting which lacks validation set.) LA consistently offered considerable gains over MAP in terms of ECE and NLL, achieving best NLL among all methods across all datasets, while maintaining similar ACC. Note that while this table gives only the performance at time point, Figure 1 shows that LA usually offers considerable benefits in calibration across all points in training. Additionally, in Fig. 1, we highlight the point with highest test accuracy with the dashed line (as this is the highest test accuracy, it cannot practically be used to early-stop, but does indicate that were we to optimally early-stop, we would still get dramatic benefits calibration from LA.)
We truncate the context in BoolQ for computational considerations (see Appendix C).

---

Published as a conference paper at ICLR 2024
Table Comparison of different post-hoc methods applied the fine-tuned LlaMA2-7B across six common sense reasoning tasks. Results are evaluated at the early stopping point of 5000 gradient steps. We report standard deviations in subscripts, and bold numbers that are statistical significant.
Metrics
Methods
WG-S
ARC-C
ARC-E
WG-M
OBQA
BoolQ
MAP
67.40.3
66.30.6
84.71.5
73.4n.4
78.7p.4
86.1g.2
MC Drop
67.8p.1
65.31.0
85.01.3
73.2n.s
79.50.2
86.0o.3
1CC
Ckpt Ens
67.
65.5o.4
85.80.2
73.6o.7
79.1o.1
86.3g.2
Ensemble
68.0v.a
68.2a.7
85.8u.s
75.0u.s
79.3u.4
86.80.1
LLLA
67.40.3
66.2a.4
84.71.5
73.40.4
78.70.4
86.1a.2
LA
67.3p.2
65.3o.2
85.1
73.40.3
78.90.2
86.10.2
MAP
20.3
31.0o.s
13.413
23.0o.1
16.10.6
1.00.5
MC Drop
29.40.3
29.60.8
12.41.
22.20.2
15.00.4
4.10.4
ECE.
Ckpt Ens
29.7o.6
27.01.5
9.80.6
17.40.9
12.40.3
1.20.4
Ensemble
24.70.3
21.91.7
9.90.2
17.90.6
13.3p.a
3.50.2
LLLA
22.82a
18.24.4
11.622
22.60.2
15.8p.s
4.00.5
LA
2.10.3
7.40.7
5.40.2
7.40.4
6.40.8
2.10.6
MAP
3.15u.10
3.28a.29
1.260.13
1.5lu.05
0.99p.as
0.35g.01
MC Drop
2.81u.11
2.82a.11
1.110.10
1.4lu.03
0.95v.04
0.35g.uz
N
Ckpt Ens
2.580.15
2.360.34
0.80g.06
0.870.08
0.76p.01
0.33p.00
Ensemble
2.160.14
2.320.14
0.830
1.180.05
0.870.03
0.34g.00
LLLA
0.98p.13
1.21o.16
0.87o.26
1.450.08
0.970.04
0.35a.01
LA
0.60o.01
0.880.03
0.490.06
0.63o.02
0.65p.o1
0.34g.01
Next, considered standard setting, in which the training set partitioned into 80% subset for training and a 20% validation subset. Notably, this usually led to slight decrease in performance,
were longer training on all the data (Table 1 against Table 2). This allowed us to benchmark against a widely-adopted post-hoc calibration technique, temperature scaling (Guo et al., 2017), which requires separate validation it is possible to tune the prior precision in Laplace using the NLL on the validation set, rather than the "more Bayesian" approach of tuning prior precision using the Laplace estimate of the model evidence. The standard approach is to perform grid search to maximize validation log-likelite -iglikkllhoodd (Daxberger al., 2021a). However, fine- mnaigrai grid search is costly. Instead, utilizing our reparametrized model average (Eq. 15), weperformed stochastic gradient descent for the precision hyperparameters, using the validation NLL as the objective (see Algorithm 3 in Appendix F). Table shows the result of tuning temperature and Laplace prior validation set at the checkpoint with the best MAP validation accuracy. Although temperature scaling is very competitive, LA still gives slightly improved NLL and usually improves ECE.
the temperature hyperparameter. Additionally,
5.2 EVALUATIONS UNDER DISTRIBUTION SIIFT
Finally, since real world deployment of LLMs requires them to generalize to data coming from a different domain compared the fine-tuning dataset (Ouyang et al., 2022; Touvron et al., 2023a:b), we conduct further experiments to evaluate fine-tuned models under distribution shift using the model checkpoint fine-tuned on OBQA in Table 2. Our evaluations under distribution shift are divided into two groups, smaller distribution shift and larger distribution shift. For smaller distribution shift, we use ARC datasets ARC-C and ARC-E, as they also contain common sense reasoning multiple choice questions similar to those in OBQA. For larger distribution shift, we pick four subjects from the MMLUU task (Hendrycks et al., 2020): computer science (CS), engineering (Eng), law, and health, where each subject contains one more tasks. A detailed breakdown is available in Table 19 in Appendix
Table 3 shows the OOD evaluation results using the early-stopped checkpoint on OBQA from Table 1. LA clearly provides substantial improvements over all other methods in terms of Expected Calibration Error (ECE) and Negative Log- Likelihoo (NLL) while maintaining similar accuracy (ACC), with slight improvement in the Computer Science (CS) subject. Table shows the evaluation results under distribution shift using the best validation accuracy checkpoint OBQA from Table 2. Once again,
7

---

Published as a conference paper at ICLR 2024
Table 2: Comparison of different post-hoc methods applied the fine-tuned LlaMA2-7B across six common sense reasoning tasks, with validation set split from the training set used for tuning temperature and Laplace prior precision. Results are evaluated at the best MAP performance
checkpoint observed on the validation set.
Metrics
Methods
WG-S
ARC-C
ARC-E
WG-M
OBQA
BoolQ
MAP
67.00.6
64.91.1
85.2p.6
73.70.9
77.70.8
85.80.4
MC Drop
66.70.3
64.91.9
85.10.5
73.50.9
77.70.2
85.9p.4
Ckpt Ens
66.7a.s
64.91.1
85.2n.s
73.81.0
78.20.2
85.4p.3
1CC
Temp
67 67.00.6
64.91.1
85.2n.a
73.70.9
77.70.8
85.8p.4
LLLA
66.90.s
66.10.6
84.80.5
73.70.9
77.60.7
85.80.4
LA
66.90.s
66.91.1
85.40.4
73.71.0
78.10.7
85.80.4
MAP
30.81.8
26.11.4
8.90.3
24.91.3
9.81.0
7.40.1
MC Drop
29.51.6
25.60.7
8.80.a
23.51.2
8.80.s
7.50.1
Ckpt Ens
25.21.6
26.11.4
8.90.3
22.81.4
4.70.5
3.20.5
ECE 4
Temp
12.80.9
4.61.0
4.To..
6.31.6
7.22.6
2.50.3
LLLA
11.61.3
5.62.1
4.2p.a
3.81.4
5.40.4
1.70.6
LA
7.81.9
7.51.2
3.4o.s
4.81.6
3.50.4
1.9u.3
MAP
2.750.37
1.640.19
0.54p.a3
2.43a.50
0.71o.03
0.430.01
MC Drop
2.540.49
1.55u.16
0.52o.04
2.12u.35
0.71a.04
0.430.01
Ckpt Ens
1.31a.04
1.640.18
0.54v.a3
1.89p.24
0.65a.02
0.35u.01
J
Temp
0.680.01
0.900.01
0.430.02
0.580.01
0.670.02
0.350.00
LLLA 0.68a.m
0.94n.02
0.44p.01
0.56g.01
0.660.02
0.35p.00
LA
0.66n.02
0.86g.02
0.41o.02
0.55p.o1
0.62p.01
0.346.00
Table Comparison of different post-hoc methods across six out-of-distribttion datasets. Results
are evaluated using the fine-tuned LlaMA2-7B on OBQA as in Table 1.
Smaller Distribution Shift
Larger Distribution Shift
Metrics
Methods
OBQA
ARC-C
ARC-E
CS
Eng
Law
Health
MAP
78.7a.4
67.91.
77.7a.3
42.0s.2
41.22.
37.40.4
48.3g.
MC Drop
79.5a.2
67.70.6
77.2p.s
41.92.2
39.61.7
37.96.4
48.2a.
Ckpt Ens
79.1a.2
67.9p.
77.4p.s
41.12.
38.71.2
37.70.2
48.2g.s
1CC
Ensemble
68.0g
69.0p.7
77.01.1
41.91.
41.92.
38.0o.g
48.4n.5
LLLA
78.7a.4
68.1o.0
78.1..0
45.60.0
38.90.o
37.1o.0
48.50.0
LA
78.9u.3
69.2..
78.5p.a
45.1a.0
39.1o.0
37.30.0
49.1a.0
MAP
16.1a.a
22.21.2
15.81.0
34.2s.1
38.41.7
35.20.r
34.2n.s
MC Drop
15.0g.4
21.40.4
15.51.0
33.72.0
38.62.9
34.20.6
33.5u.2
Ckpt Ens
10.1g.3
17.70.7
12.10.6
29.12.3
32.51.8
32.10.1
29.0g.3
ECE J
Ensemble
24.7a.3
18.5
13.40
30.426
31.71_a
32.70.4
31.0p.2
LLLA
15.8a.a
21.30.0
14.8p.a
30.30.0
39.7o.0
33.60.0
33.5n.0
LA
6.40.a
8.8p.0
6.2o.0
15.80.0
25.50.0
24.7o.0
17.90.0
0.990.a5
1.30g.07
1.04p.10
1.900.12
2.196.15
2.120.03
2.09g.0s
Drop
0.95a.94
1.24p.00
1.01p.09
1.860.10
2.140.13
2.096.uz
2.05a.o7
Ckpt Eas
0.68o.os
1.03..03
0.80p.0s
1.550.04
1.720.01
1.940.01
1.74g.02
NLL 4
Ensemble
2.46g.14
1. 1.150.07
0.91
1.746,19
1.950.10
2.040.08
1.95a
LLLA
0.97a.04
1.306.00
0.990.00
1. 1.800.00
2.180.00
2.066.00
2.05a.00
LA
0.65o.01
0.900.00
0.700.00
1.35a.00
1.580.00
1.740.00
1.50v.00
both LLLA and LA typically yield improvements in ECE and NLL over other methods, indicating their effectiveness in dutrof-distribution scenarios.
In addition to experiments on LlaMA the main text, we also present results for fine-tuning encoder- based language models RoBERTa-bbee base and RoBERTa-large (Liu et al., 2019) on text classification tasks in Appendix G.2 Notably, Monte-Carla dropout performs much better on RoBERTa-bor but RoBERTa-laan large checkpoint ensemble often performs much worse than MAP in contrast to its performance on LlaMA; LLLA still offers tiny improvements over MAP. Besides, we have also added additional experiments fine-tur menetunnng Mistral-7B (Jiang et al., 2023) in Appendix G.4. Overall, full Laplace-Lofe (LA) provides consistent and significant improvements uncertainty estimation across models (RoBERTa, LlaMA, Mistral) and tasks (text classification, common sense
less effective
8

---

Published as a conference paper at ICLR 2024
Table 4: Comparison of running time and memory cost of standard LoRA finetuning for 10,000 iterations vs standard LoRA finetuning for 10,000 along with accumulating low-rank Kronecker factors for Laplace every 1000 epochs. Experiements carried out using Llama2 7B, LoRA denotes standard LoRA fine-tuning for 10,000 steps, Laplace-LoRA denotes LoRA fine-tuning for 10,000
steps along with post-hoe Laplace approximation on all LoRA weights.
Metric
Method
WG-S
WG-M
ARC-C
ARC-E
OBQA
BoolQ
LoRA
8287
8684
10500
10301
9268
16964
Time (Seconds) +
LoRA Laplace
8442
8839
11141
10942
10526
17636
LoRA
8.34
8.33
8.37
8.43
8.59
10.48
Memory (GB) &
LoRA Laplace
8.43
8.43
8.73
8.75
8.84
10.93
reasoning). Supplementary experiments using diagonal Fisher Hessian approximation instead of KFAC are documented in Appendix G.6. However, we found that diagonal Laplace approximations do not offer consistent improvements in ECE and NLL, emphasizing the importance of using KFAC to model correlations between weights in Laplace-LoRA. We have open sourced our code implementations (Appendix J).
5.3 MEMORY AND RUNTIME COST
The primary advantages of LoRA lie in its memory and computational efficiency, In this section, we present a comparative analysis of the memory and time costs associated with LoRA fine-tuning alone, and LoRA fine-tuning followed by post-hoc application of Laplace-Loff LoRA. As illustrated in Table 4, implementing Laplace-LoRA incurs only an additional around 1-5% memory overhead. Note that in the experiments in Table 4, there is very little overhead in terms of time (about 10%), despite the fact that we actually accumulate the low-rank Kronecker factors 10 times (once the end of every 1000 iterations, and we ran these experiments for 10000 iterations total). In practice, would only need to accumulate low-rank Kronecker factors once, so the overhcad should be around I 1%. This empirical analysis demonstrates that our low-rank KFAC approximation of Laplace-LoRA effectively maintains LoRA's efficiency.
CONCLUSION
In this work, we proposed Laplace-LoRA, for Bayesian parameter-efficien fine-tuning of LLMs. Our method is scalable to larger networks due to its focus the LoRA weights. Additionally, by applying Laplace approximations post-hoc, after fine-tuning, we require no changes to efficient implementations of the standard fine-tuning process. In experiments, gains in expected calibration етгоr and negative log-likelihood, indicating an improved estimation of uncertainty. Our approach underpins the potential of Laplace-LoRA a step towards the development of more reliable and trustworthy LLMs.
observed significant
7
ACKNOWLEDGEMENT
This work was carried out using the computational facilities of the Advanced Computing Research Centre, University Bristol http: / / /wwww was.s.ac....... uu/ ukracrr/. We would like to thank Dr Stewart for funding compute resources used in this project, and we would like to thank Yixuan Su for helpful discussions.

---

Published as a conference paper at ICLR 2024
REFERENCES
Laurence Aitchison, Adam Yang, and Sebastian W Ober. Deep kernel processes. In International Conference on Machine Learning, pp. 130-140. PMLR, 2021.
Javier Antorán, David Janz, James U Allingham, Erik Daxberger, Riccardo Rb Barbano, Eric Nalisnick, and José Miguel Hemández-Lobato. Adapting the linearised laplace model evidence for modern deep learning. In JCML, 2022.
Arsenii Ashukha, Alexander Lyzhov, Dmitry Molchanov, and Dmitry Vetrov. Pitfalls of in-domain uncertainty estimation and ensembling deep learning. arXiv preprint arXiv:2002.06470, 2020,
Charles Blundell, Julien Cornebise, Koray Kavukeuoglu, and Daan Wierstra. Weight uncertainty in neural network. In International conference machine learning, pp. 1613-1622. PMLR, 2015.
Hugh Chen, Scott Lundberg, and Su-In Lee. Checkpoint ensembles: Ensemble methods from single training process. arXiv preprint urXiv:/710.03222, 2017.
Wenlong Chen and Yingzhen Li. Calibrating transformers via sparse gaussian processes. arXiv preprint arXiv:2303.0244 2023.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction -inettuondd- language models. arXiv preprint arXiv:2210. 1 2022.
Tristan Cinquin, Alexander Immer, Max Horn, and Vincent Fortuin. Pathologies in priors and inference for bayesian transformers. arXiv preprint arXiv:2110.04020. 2021.
Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. In NAACL, 2019.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try are, the ai2 reasoning challenge. arXiv:1803.05457v1, 2018.
Erik Daxberger, Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen, Matthias Bauer, and Philipp Hennig. Laplace redux-effortles bayesian deep learning. NeurIPS, 2021a.
Erik Daxberger, Eric Nalisnick, James U Allingham, Javier Antorán, and José Miguel Hernández- Lobato. Bayesian deep learning via subnetwork inference. In ICML, 2021b.
Zhijie Deng. Feng Zhou, and Jun Zhu. Accelerated linearized laplace approximation for bayesian deep learning. NeurlPS, 2022.
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized lms. arXiv preprins arXiv:2305. 143 2023.
Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang, Yusheng Su, Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, et al. Delta tuning: comprehensive study of parameter efficient methods for pre-trained language models. arXiv preprint arXiv:2203.0m 069004 2022.
Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang, Yusheng Su, Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, Jing Yi, Weilin Zhao, Xiaozhi Wang. Zhiyuan Liu, Hai-Tao Zheng, Jianfei Chen, Yang Liu, Jie Tang, Juanzi Li, and Maosong Sun. Parameter-efficient fine- tuning of large-scale pre-trained language models. Nature Machine Intelligence, 5(3):220-235, 2023.
Michael Dusenberry, Ghassen Jerfel, Yeming Wen, Yian Ma, Jasper Snoek, Katherine Heller, Balaji Lakshninarayanan, and Dustin Tran. Efficient and scalable bayesian neural nets with rank- factors. In International conference on machine learning, pp. 2782-2792. PMLR, 2020.
Xinjie Fan, Shujian Zhang, Bo Chen, and Mingyuan Zhou. Bayesian attention modules. Advances in Neural Information Processing Systems, 33:16362-16376, 2020.
10

---

Published as a conference paper at ICLR 2024
Andrew YK Foong, Yingzhen Li, José Miguel Hernández-Lobato and Richard E Turner. 'in- between' uncertainty in bayesian neural networks. In ICML Workshop on Uncertainty and Robust- ness in Deep Learning. 2019.
Vincent Fortuin, Adrià Garriga-Alonss Sebastian W Ober, Florian Wenzel, Gunnar Rätsch, Richard Tumnrr Mark van der Wilk, and Laurence Aitchison. Bayesian neural network priors revisited. arXiv preprint arXiv:2102.0633 2021.
Yarin Gal and Zoubin Ghahramani. Dropout as bayesian approximation: Representing model uncertainty deep learning. In international conference on machine learning, pp. 1050-1059. PMLR, 2016.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, pp. 1321-1330. PMLR, 2017.
Guande He, Jianfei Chen, and Jun Zhu. Preserving pre-trained features helps calibrate fine-tuned language models. In ICI.R, 2023.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efffcient transfer learning for nlp. In International Conference on Machine Learning, pp. 2790-2799. PMLR, 2019.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2/06.09685, 2021.
Alexander Immer, Maciej Korzepa, and Matthias Bauer. Improving predictions of bayesian neural nets via local linearization. In AISTAT, 2021.
Pavel Izmailov, Sharad Vikram, Matthew D Hoffman, and Andrew Gordon Gordon Wilson. What are bayesian neural network posteriors really like? In International conference on machine learning. pp. 4629 4640. PMLR, 2021.
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.0682) 2023.
Zhengbao Jiang, Jun Araki, Haibo Ding, and Graham Neubig. How can we know when language models know? on the calibration of language models for question answering. Transactions of the Association for Computational Linguistics, 9:962-977, 2021.
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, et al. Language models (mostly) know what they know. arXiv preprint ar Xtv:2207.002221 2022.
Agustinus Kristiadi, Matthias Hein, and Philipp Hennig. Being Bayesian, even just a bit, fixes overconfidence in relu networks. In ICML, 2020.
Frederik Kunstner, Philipp Hennig, and Lukas Balles. Limitations of the empirical fisher approx- imation for natural gradient descent. Advances in neural information processing systems, 32, 2019.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems, 30, 2017.
Andrew Kyle Lampinen, Stephanic C. Y. Chan, Ishita Dasgupta, Andrew J. Nam, and Jane X. Wang. Passive learning of active causal strategies in agents and language models, May 2023.
11

---

Published as a conference paper at ICLR 2024
Cheolhyoung Lee, Kyunghyun Cho, and Wanmo Kang. Mixout: Effective regularization to finetune large-scale pretrained language models. arXiv preprint arXiv:1909.112990 2019.
Shuang Li, Xavier Puig, Chris Paxton, Yilun Du, Clinton Wang, Linxi Fan, Tao Chen, De-An
Huang, Ekin Akyürek, Anima Anandkumar, et al. Pre-trained language models for interactive decision-making Advances Neural Information Processing Systems, 19931199-3122,, 2022.
Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. Few-shot parameter-effiiien fine-tuning is better and cheaper than in-context learning, Advances Neural Information Processing Systems, 35:1950-1965, 2022.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint
2019.
Zhiyun Lu, Eugene Ie, and Fei Sha. Uncertainty estimation with infinitesimal jackknife, its distribution and mean-field approximation. ArXiv, abs/2006.07584 2020.
David J. C. MacKay. Choice of basis for laplace approximation. Machine Learning, 33(1):77-86. 1998.
David JC MacKay. A practical bayesian framework for backpropagation networks. Neural computa- tion, 1992.
Sourab Mangrulkar, Sylvain Gugger, Lysandre Debut, Younes Belkada, and Sayak Paul, Peft: State- of-the-art parameter-efficient fine-tuning methods. https:///iithbe com cwm. peft, 2022.
hugg. com/bugi ng fan.aaccce aa
Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? new dataset for open book question answering. In EMNLP, 2018.
Sebastian W Ober and Laurence Aitchison. Global inducing point variational posteriors for bayesian neural networks and deep gaussian processes. In International Conference on Machine Learning, PMLR, 2021.
pp.
OpenAl. GPT-4 technical report, 2023.
Kazuki Osawa, Satoki Ishikawa, Rio Yokota, Shigang Li, and Torsten Hoefler. Asdl: A unified interface for gradient preconditioning in pytorch. arXiv preprint arXiv:2305.0400 2023.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35: 27730-27744, 2022.
Seo Yeon Park and Comelia Caragea. On the calibration of pre-trained language models using mixup guided by area under the margin and saliency. arXiv preprint arXiv:2203.07559 2022.
Hippolyt Ritter, Aleksandar Botev, and David Barber. scalable laplace approximation for neural networks. In ICLR, 2018.
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge scale. Communications of the ACM, 2021.
Zhengxiang Shi and Aldo Lipani. Dept: Decomposed prompt tuning for parameter-efficien fine- tuning. arXiv preprint wwXiv:2309.05/77 2023.
Kumar Shridhar, Felix Laumann, and Marcus Liwicki. comprehensive guide to bayesian convolu- tional neural network with variational inference. arXiv preprint arXiv: (0.0v://01..02000 2019.
Karan Singhal, Shekoofeh Azizi, Tao Tu, S. Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, Perry Payne, Martin Seneviratne, Paul Gamble, Chris Kelly, Nathaneal Scharli, Aakanksha Chowdhery. Philip Mansfield, Blaise Aguera y Arcas, Dale Webster, Greg S. Corrado, Yossi Matias, Katherine Chou, Juraj Gottweis, Nenad Tomasev, Yun Liu, Alvin Rajkomar, Joelle Barral, Christopher Semturs, Alan Karthikesalingam, and Vivek Natarajan. Large Language Models Encode Clinical Knowledge. arXiv, 2022.
12

---

Published as a conference paper at ICLR 2024
Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D Manning. Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback. arXiv preprint arX1v:2305.14975. 2023.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302 /3971, 2023a.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint w:Xiv:337.002288 2023b.
Dustin Tran, Mike Dusenberry, Mark Van Der Wilk, and Danijar Hafner. Bayesian layers: A module for neural network uncertainty. Advances in neural information processing systems, 32, 2019.
Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. Superglue: stickier benchmark for general-purpose language understanding systems. In NeurlPS, 2019a.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. In ICLR, 2019b.
Xi Wang, Laurence Aitchison, and Maja Rudolph. Lora ensembles for large language model fine- tuning. arXiv preprint 0tti:2310000033: 2023.
Yizhong Wang, Yeganch Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannanch Hajishirzi. Self-instruct: Aligning language model with self generated instructions. arXiv preprint 11Xx:2212.10566 2022.
Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoe Le. Finetuned language models are zero-shot learners. arXiv preprint trXiv:2109.016551 2021.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the-ar natural language processing. In EMNLP, 2020.
Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhan- jan Kambadur, David Rosenberg, and Gideon Mann. BloombergGPT: A Large Language Model for Finance, May 2023.
Yuxin Xiao, Paul Pu Liang, Umang Bhatt, Willie Neiswanger, Ruslan Salakhutdinov, and Louis- Philippe Morency. Uncertainty quantification with pre-trained language models: A large-scale empirical analysis. In EMNLP, 2022.
Boyang Xue, Jianwei Yu, Junhao Xu, Shansong Liu, Shoukang Hu, Zi Ye, Mengzhe Geng, Xunying Liu, and Helen Meng. Bayesian transformer language models for speech recognition. In ICASSP 2021-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp- 7378-7382. IEEE, 2021.
Yuanzhao Zhai, Han Zhang, Yu Lei, Yue Yu, Kele Xu, Dawei Feng, Bo Ding, and Huaimin Wang. Uncertainty-penalizeed reinforcement learning from human feedback with diverse reward lora ensembles. arXiv preprint arXiv:2401.00243 2023.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. arXiv preprint arxiv:1710.094111 2017.
Ruqi Zhang, Chunyuan Li, Jianyi Zhang, Changyou Chen, and Andrew Gordon Wilson. Cyclical stochastic gradient meme for bayesian deep learning. arXiv preprint arXiv: 19op::902..3999 2019.
13

---

Published as a conference paper at ICLR 2024
Shujian Zhang, Xinjie Fan, Bo Chen, and Mingyuan Zhou. Bayesian attention belief networks. In ICML, 2021.
Yuan Zhao, Jiasi Chen, and Samet Oymak. On the role of dataset quality and heterogeneity in model confidence. arXiv preprint 20XV:22002.08831, 2020.
14

---

Published as a conference paper at ICLR 2024
A LAPLACE PREDICTIVE POSTERIOR APPROXIMATIONS
In this section, we review several methods for obtaining the predictive posterior of Laplace approxi-
mation.
A.1 MONTE CARLO SAMPLING
As descibed in main text, obtain closed-form Gaussian posterior output logits,
fo(x,) ~ ~N (ferue (xx), A)
(16)
where
A = (Vefe(xx))) Vee
(17)
To obtain samples of (x.), we can decompose the covariance using the Cholesky factorization LL" with
fx(x.) ferma (x.) + LE,
(18)
where & is vector of IID standard normal random variables. We can compute the Bayesian model average by computing the average probabilities (passing the sampled logits through softmax function) under the Gaussian random noise from &.
A.2 PROBIT APPROXIMATION
A closed-form approximation of the predictive posterior for classification can be obtained by integrat- ing out the posterior over weights with generalized probit approximation (Lu et al., 2020; Daxberger et al., 2021a) of the likelihood,
So (x.)
ply.|x., D) zu Categorical
Ye, softmax
(19)
VI+ 7 diag(A) (A)
g
where
= (Volo(x.) (oVoooo
(20)
Although probit approximation provably preserves decision boundary in binary sigmoid classification (Kristiadi et al., 2020), it does hold for softmax multiclass classification.
A.3 LAPLACE BRIDGE
The Laplace bridge (MacKay, 1998; Daxberger et al., 2021a) maps Gaussian N(,, G Σ) to Dirichlet distribution D(x.) over classes with
a, =
1
2 +
exp(1u:)
Σ exp(- exp((11))) 142)) -1))
(21)
En
C
where C' denotes the number of classes. Similar to the generalized probit approximation, it also ignores the covariance terms and only considers the diagonal of the covariance E,,
A.4 EMPIRICAL COMPARISON
Figure shows a comparison of these approximations in fine-tuning Llama2-7B and applying full Laplace-LoRA post-hoc. Specifically, we considered Monte Carlo sampling using the full covariance (MC joint), Monte Carlo sampling only using the diagonal covariance (MC indep), generalized probit approximation (probit), and Laplace bridge (bridge). MC joint consistently achieves highest accuracy and among the best NLL, while bridge is often the worst, probit and MC indep can sometimes give suboptimal performance. This is likely due bridge, probit and MC indep all approximations that ignored the covariances between logits, whereas MC joint faithfully approximates the true predictive posterior.
15

---

Published as a conference paper at ICLR 2024
Bridge
Probit
Ind
MCInt
(640)
(1.12k)
(2.25k)
(2.56k)
OBQA (4.96k)
BoolQ
70
90
90
ACC (98)
60
80
50
70
50
60
70
40
20
10
ECE (90)
20
20
0.6
NIL
0.4
10000
10000
10000
10000
10000
10000
training steps
training steps
training steps
training steps
training steps
training steps
Figure 2: Fine-tuning of LlaMA2-7B across six common sense reasoning tasks, comparing different Laplace predictive posterior approximations: Laplace bridge approximation (bridge), generalized probit approximation (probit), Monte Carlo sampling using the diagonal covariance (MC indep), and Monte Carlo sampling using the full covariance (MC joint).
Task
Prompt
Winogrande (WG-S/WG-M)
Select one of the choices that answers the following question: (question] Choices: A. (options
(option2). ). Answer:
ARC (ARC-C/ARRC-E
Select one the choices that answers the following question:
(question| Choices: A. {choicel) J. B. (choice2).
C. [choice2), ). [choice2). ). Answer:
Openbook QA (OBQA)
Select one of the choices that answers the following question:
(question. Choices: A. {choicel] ). B. [choice2).
C. [cchoice)), D. (choice2). Answer:
BoolQ
Answer the question with only True or False:
(question) Context: (passage).
Table 5: Prompt templates for one-tuning LlaMA-7B on common sense reasoning tasks.
B PROMPT TEMPLATES FOR LLAMA2 COMMON SENSE REASONING TASKS
We present our prompt templates used fine-tune LlaMA2 on common sense reasoning tasks in Table 5. For ARC datasets, although the majority of quesstions have four choices, there are a tiny amount of questions with three five choices which we remove for consistency. In ARC-C, there are 1/1119 with three choices and 1/1119 with five choices in the training set; 3/299 with three choices and 1/299 with five choices 4/2251 with five choices in the trainng set; 1/570 with three choices and 2/570 with five choices in the evaluation set.
evaluation set. In ARC-E, there are 6/2251 with three choices and
C HYPERPARAMETERS
We follow the default hyperparameters from Huggingface's Transformer (Wolf et al., 2020) and PEFT (Mangrulkar et al., 2022) libraries. Hyperparameters used for fine-tuning RoBERTa-base and RoBERTa-large with LoRA shown in Table 6, those for fine-tuning LlaMA-7B are shown in Table Note the only differences between the two settings are a smaller batch size to fit into our GPU memory, and longer sequence length account for prompt length.
16

---

Published as a conference paper at ICLR 2024
Hyperparameter
Value
LoRA
LoRA a
Dropout Probability
0.1
Weight Decay
Learning Rate
5 x 10-5 Linear
Learning Rate Scheduler
Batch Size
Max Sequence Length
256
Table Hyperparameters used in fine-tuning RoBERTa-baid base and RoBERTa-large with LoRA.
Hyperparameter
Value
LoRA r
LoRA
16
Dropout Probability
0.1
Weight Decay
0
Learning Rate
5 x 10-5 Linear
Learning Rate Scheduler
Batch Size
4
Max Sequence Length
300
Table 7: Hyperparameters used in fine-tuning LlaMA-7B with LoRA.
D METRICS FOR UNCERTAINTY QUANTIFICATION
There are two commonly used metrics for measuring uncertainty quantification in neural networks: negative log- ligllikel (NLL) and expected calibration error (ECE). NLL computes the sum of negative expected log probability of predicting the true label,
N
NLL Σ
log P(u = BR).
(22)
where P(j;) is the model's output distribution, is the true label. NLL is also equivalent to cross- entropy between the one-hot data distribution and the model's output distribution. NLL encourages the model to give high probability to correct answers. If the model is overconfident in wrong answer, then the probability of giving the right answer would low which raises NLL. the other hand, ECE measures the alignment between model's confidence and accuracy, by binning the highest predicted probabilities and compute a weighted average between the difference of average accuracy and confidence in each bin.
M
Bm N
ECE
(acc(Bm) conf(Bm))
(23)
m=1
where acc(Bm) and conf (B) are the average accuracy and average confidence in each bin,
acc( Bm)
Σ 1(0=)))) BE
conf co
P(ū.).
(24)
1mm
[ VE
and |Bm,| is the number examples bin m. However, expected calibration error cannot be optimized directly like negative log-likelihood, as completely random model will have the same accuracy and confidence for each datapoint, thus achieving zero ECE (Ashukha et al., 2020).
E LOW-RANK COMPUTATIONS
The usual K-FAC approach write,
Ejkee
Eitke a Eikke
(25)
17

---

Published as a conference paper at ICLR 2024
As this is for the LoRA adapters, there's one small and one large covariance matrix. Without loss of generality, assume that Liike, [R X is the small matrix, so we represent it as full-rank, and we assume that Liike; Rdxd is the large matrix, so we represent it as low-rank,
like; BB".
(26)
Thus, we write Eiike b in low-rank form, in terms of B dx
The posterior precision is the sum of the likelihood and prior precisions.
Σ
like;
Liike; &
(27)
post
Substituting our low-rank approximation for the large Kronecker factor,
E
Σ
Σ
& (BB") + I I.
(28)
post
like;
Using the mixed product property, and setting L a of m
LL" (e.g. L is the Cholesky decomposition
Episi = (L + IIOI.
(29)
This is in the form of diagonal matrix, I I plus a low-rank matrix, (L 2 B) ( B)" Note that the rank of the low-rank matrix here Mlaara We're going to compute everything we need using this low-rank form.
E. INCREMENTAL COMPUTATION OF LOW-RANK FACTOR
The first question is how to estimate the low-rank components, without ever computing large d x matrix. The answer incremental low-rank SVD is given in Alg. 1.
Algorithm 1 Memory efficient estimate of low-rank B such that BB" ~ ΣT= b.b.
Initialize to 0
B =
for l 1,..., do
Combine current low-rank estimate, B. with new activities, b, = a, or new gradients, b,
Could combine with several vectors, if available
B' New low-rank estimate from top Rita components of SVD V doesn't matter as its just rotation
=
[B b:)
U.S, V = SVd(B')
B + U|::: Tukin S[: Tkkfae Tikfac
end for
E.2 MARGINAL LIKELIHOOD OPTIMIZATION WITH THE MATRIX DETERMINANT LEMMA
To optimize the marginal likelihood (Eq.), we need the log-determinant of the posterior. To compute this log degtremmmnnnnnt efficient in the low-rank setting, we use the matrix determinant lemma.
log det (Epd) log det ( ④ I) + log det. (I+ (L B)² (L (L)B)) B)
(30)
Applying the mixed product property,
log det. 1 (pplk)
Ma d log 0² + log det (M)
(31)
where,
(Ina Pidia
B'B
(32)
Diuso Plaea
Tifte X Xnni
Thyes Tleta X Tlyen Piaho
So we can explicitly compute L & B and hence M E Pata X Mifee as long as Daffa is not too big. In our case, Porra and Nlkfae 10, Molor? = 40.
18

---

Published as a conference paper at ICLR 2024
Algorithm 2 Optimize Laplace prior precision using the training set model evidence
Initialize prior precision X, learning rate !,, optimization steps M Obtain MAP from a fine-tuned checkpoint, pre-compute log (y X 0) and Fisher F for step 1, ..... .M do
Compute posterior covariance Σ = AI Calculate C(y, X; 0) log P (y X, 0) log P (0) log P (y 0) + 0 l1
Perform a gradient step with respect to to maximize log model evidence (Eq.14)
À - X+ + 1/ V ((y, X; MAP)
log Σ|
end for
E.3 LINEARIZED PREDICTION WITH WOODBURY
For prediction, we need to use Woodbury to compute:
Epest 8L
L B +
(33)
Epss 0² I Ia OL (L&B)
(34)
Now, applying this form for prediction,
As = g, post
(35)
where gi is the gradient of the ith network output for particular parameter, i.c. part of the Jacobian,
05.
gi 00
(36)
This is really whole matrix gradients,
layers, now just think of single layer. Thus, g: Moddd is really the
g: =ve (Gi)-
(37)
where G. E Rdxm Thus,
Ag vec (G.)² Epost vec (G₃)
(38)
Ag vec (G&)" (I2Lm & La (L &L B)") vec (G₃)
(39)
Agj o² vec (G,) ) vec (G,) - o vec (G.) (L (L" BT) vec (G₃)
(40)
Using the mixed Kronecker matrix-vector product,
Aij vec (G.)² vee (G₁) o vec (B" T M vec (B'G,L).
(41)
F OPTIMIZING LAPLACE PRIOR PRECISION
In this section, present how optimize the Laplace prior precision. When there is only training set available with no validation set (such as in Figure 1, 3, 4 and Table 1, 9, 10), we can use the Laplace model evidence in Equation 14 optimize prior precision. Our algorithm is presented in Algorithm 2, and we chose = 0.1, and M 100.
When introduce validation by splitting the training set (such as in Figure 7, 6 and Table 2, 11, 12), we can use the validation log-likelihood to optimize the Laplace prior precision. For memory and computational efficiency, we precompute the mean S and the Jacobian J Vole (X) ( then perform mini-batch gradient descent on À (reparametrization in Bayesian model average allows gradient flowing through) detailed in Algorithm We chose 1j 0.1. M 1000. and
G ADDITIONAL EXPERIMENTS
G.1 LLAMA2-7B EVALUATIONS UNDER DISTRIBUTION SHIFT
Table 8 shows the LlaMA2-7B evaluations under distribution shift results using the best validation accuracy checkpoint OBQA.
19

---

Published as a conference paper at ICLR 2024
Algorithm 3 Optimize Laplace prior precision using validation log-likelihood
Initialize prior precision A. learning rate 77, batch size b, validation set (X,y) y)
Obtain OMAP from a fine-tuned checkpoint, pre-compute mean so Jacobian J VoJo ( () Fisher F
for step = 1,.., M do
Randomly sample batch validation data Xo. Yo with corresponding Jacobian J. Compute posterior covariance Σ = F AI
Calculate batch logits covariance A, J ΣJ,, and Cholesky A, L5LL
Obtain batch Bayesian model average (X)) = (Xb) + Lb
Evaluate validation likelihood P (x. 0) Categorical (ys; softmax (X+)) Perform a gradient step with respect
maximize mini-batch validation oogllikelihood:
À - À + V log P (yo(Xo,
end for
Table 8: Comparison of different post-hoc methods across six out-of-distribution datasets. Results are evaluated using the fine-tuned LlaMA2-7B on OBQA as in Table 2.
ID
Smaller Distribution Shift
Larger Distribution Shift
Metrics
Methods
OBQA
ARC-C
ARC-E
CS
Eng
Law
Health
MAP
77.7o.s
68 68.00.2
76.71.0
43.50.9
44.42.0
37.40.1
47.70.s
MC Drop
77.70.2
68. 7
77.21.2
42.21.3
45.6r
37.2p.2
47.90.7
Ckpt Ens
78.20.2
67.90.2
77.80.4
42.40.7
43.50.9
37.1o.4
48.1a.2
1CC
Temp
77.70
68 Oui2
76.7
43.50.9
44.42_
37.4u.
47.7u.
LLLA
77.60.7
67.8
77.40.0
42.9o.0
43.8o.0
36.9u.u
46.6u.0
LA
78.1
67.80.0
76.7o.o
44.1o.0
45.80.0
37.20.0
46.60.0
MAP
9.81.
12.2o.7
8.61.
22.0g.s
20.71.
28.4p.s
23.2a.7
MC Drop
8.8p.s
12.90.s
7.718
22.82.0
19.61.
28.50.6
22.6u.6
Ckpt
4.70.5
7.11.0
3.5p.3
17.40.4
15.9..1
25.80.4
17.5g.1
ECE +
Temp
7.22.8
9.4a.6
6.31.
16.4..5
16.14.6
22.76.8
17.46.0
LLLA
5.4p.4
6.5o.0
4.9a.0
12.6o.0
12.60.0
15.70.0
15.30.0
LA
3.5p.4
5.50.0
3.1o.0
14.50.0
12.8p.0
23.9u.u
17.6u.0
0.71o.03
0.95g.02
0.690.02
1.40o.0s
1.42o.07
1.81o.01
1.48g.02
MC Drop
0.710.04
0.946.oz
0.680.03
1.400.03
1.41o.0
1.80p.01
1.47u.02
Ckpt Ens
0.65p.02
0.870.00
0.61o.01
1.296.01
1.290.03
1.71o.00
1.34..01
NLL I
Temp
0.670.02
0.900.05
0.66o.01
1.31o.09
1.320.07
1.650
1.360
LLLA
0.666.02
0. 08800000
0.640.00
1.280.00
1.270.00
1.490.00
1.310.00
0.62p.01
0.85g.00
0.62,.00
1.260.00
1.270.00
1.68p.00
1.35g.00 00
G.2 FINE-TUNING ROBERTA FOR TEXT CLASSIFICATION
In this section, present additional results of fune-tuning RoBERTa-bras base (Fig. 3) and RoBERTa- (Fig. 4) (Liu et al., 2019) with LoRA on text classification tasks from GLUE (Wang et al., 2019b) and SuperGIUUE (Wang al., 2019a). Results for RoBERTa-bass results for RoBERTa-larg are shown in Figure and Table 10. Surprisingly. checkpoint ensemble and Monte-Carlo (MC) dropout exhibit distinct behavior on RoBERTa models compared LlaMA2-7B. Checkpoint ensemble often performs much worse than the Maximum Posteriori (MAP) estimation in terms of Expected Calibration Error (ECE) and Negative Log-Likelihood (NLL), while MC dropout often offers much more improvement on RoBERTa models. We suspect this difference arises due to an hidden penultimate layer with an additional dropout layer in front in the default RoBERTa fine-tuning setup; whereas, in J-MMA27BB fine-tuning, we have LoRA weights and dropout on LoRA layers the end. However, the gain from MC dropout diminishes on larg compared to RoBERTa-base. On the other hand. Laplace-LoRA (LA) consistently delivers substantial gains on any models we have tested (RoBERTa-base, RoBERTa-large, and LlaMA2-7B), demonstrating the robustness of Laplace-LoRA. Moreover, the Last-Layer Laplace-LoRA (LLLA) offers modest improvements as in LlaMA2-7B when optimized by Laplace model evidence, underscoring the significance of performing Bayesian inference all LoRA weights.
shown in Figure 3 and Table 9, and
Similarly. we also conduct experiments by splitting the training set into a 80% training set and a 20% validation set, then tune temperature and Laplace prior precision the validation set. The results
20

---

Published as a conference paper at ICLR 2024
MC Dropout
Checkpoint Ensemble
LLLA
(635)
(2.49k)
(3.67k)
(5.43k)
(8.55k)
BoolQ (9.43k)
30
80
80
ACC 198)
20
70
10
60
20
15
EC 196
10
10
10
10
10
0.6
0.5
NUL
1.0
0.6
0.8
0.5
10000
10000
10000
10000
10000
10000
training steps
training steps
training steps
training steps
training steps
training steps
Figure 3: Fine-tuning of RoBERTa-base base acToss six GLUE and SuperGLUE tasks (presented column- wise, with number of training examples brackets), evaluated on the test set every 1000 gradient steps, without validation set for hyperparameter tuning. The vertical dashed line gives the number of training steps with optimal MAP performance. Note that RoBERTa-bens seems to fail on WNLI, but RoBeRTa-large succeeds (Fig. 4).
Table 9: Comparison of different post-hoc methods applied to the fine-tuned RoBERTa-base across six GLUE and SuperGLUE tasks, without validation set for hyperparameter tuning. Results are evaluated at the early stopping point of 5000 gradient steps.
Methods
Metries
WNLI
RTE
MRPC
WiC
CoLA
BoolQ
MAP
22.54.s
72.82
87.1u.7
64.52.
82.4b.s
77.20.4
MC Drop
19.73.0
74.01.a
88.2g.v
68.5u.s
82.7v.1
76.7a.6
ACC f
Ckpt Ens
21.6p.s
73.92.a
87.7a.6
64.31.9
81.8p.5
76.60.2
LLLA
22.54.
72.82
87.19.7
64.52.
82.40.
77.20.4
LA
22.54.6
72.82.2
87.20.7
61.52.0
82.4p.a
77.20.5
MAP
66.73.2
20.92.1
8.30.1
18.42.2
8.6p.5
5.30.2
MC Drop
65.61.s
12.71.6
4.81.6
8.9g.6
1.2p.1
1.50.4
ECE
Ckpt Ens
44.6g.9
8.51.5
17.90.1
5.21.0
13.6p.3
12.50.3
LLLA
65.23.7
20.32.2
7.60.3
18.12.2
8.50.4
5.10.1
LA
51.43.8
12.43.4
2.30.4
12.92.0
5.0o.8
3.70.2
MAP
3.10u.09
1.05p.09
0.43g.u1
0.79u.ua
0.46v.az
0.50g.u0
MC Drop
2.070.08
0.630.01
0.30o.01
0.65o.01
0.38o.00
0.49o.00
NLL
Ckpt Ens
0.96g.02
0.56g.01
0.44.000
0.63p.01
0.490.00
0.55o.00
LLLA 2.92n.08
1.00p.09
0.4lo.0
0.790.03
0.450.02
0.50..o0
LA
1.38u.06
0.64p.us
0.30p.00
0.69u.zz
0.40v.a1
0.490.00
shown in Figure 5 and Table 1l for RoBERTa-base, and Figure 6 and Table 12 for RoBERTa-large. Again, full Laplace-LoRA LA offers the most improvements most of the time, and is the most robust method overall.
G.3 FINE-TUNING LLAMA2-7B COMMON SENSE REASONING
Figure 7 displays the results of tuning temperature scaling and Laplace prior precision on a held-out validation set split from the training set. LLLA is missing a few zero checkpoint evaluation results due to Cholesky errors. Comparing Figure to Figure in the main text, it is evident that splitting the training set into a smaller training set (80% data) and a validation set (20% data) slightly impact the accuracy of fine-tuned model.
21

---

Published as a conference paper at ICLR 2024
MC Dropout
Checkpoint Ensemble
LLLA
(635)
(2.49k)
(3.67k)
(5.43k)
(8.55k)
BoolQ (9.43k)
90
90
90
70
Perf (80)
80
60
70
15
ECE 1996
20
10
5
0.75
0.5
0.5
1.0
1.00
0.75
0.4
0.5
0.25
10000
10000
10000
10000
10000
10000
training steps
training steps
training steps
training steps
training steps
training steps
Figure 4: Fine-tuning RoBERTa-large across the six GLUE and SuperGLUE tasks in Fig. 3, without a validation set for hyperparameter tuning. The vertical dashed line gives the number training steps with optimal MAP performance.
Table 10: Comparison of different post-hoc methods applied to the fine-tuned RoBERTa-large across six GLUE and SuperGLUE tasks. Results are evaluated at the early stopping point of 5000 gradient steps.
Methods
Metries
WNLI
RTE
MRPC
WiC
CoLA
BoolQ
MAP
70.0..s
83.4p.3
88.2g.3
68.42.0
85.2o.s
84.40.4
MC Drop
69.02.3
85.00.3
88.90.s
69.81.4
86.0n.1
84.00.6
1CC
Ckpt Ens
68.50.7
83.20.9
88.2g.5
68.824
84.60.4
84.30.5
LLLA
70.0..s
83.5p.3
88.2a.3
68 68.42.0
85.2p.a
84.30.3
LA
70.01.s
83.5p.3
88.2g.3
68.42.0
85.2o.s
84.40.4
MAP
28.51.a
14.4p.a
9.3g.6
19.71.8
8.0p.4
8.0g.a
MC Drop
25.21.a
8.8p.9
5.31.0
12.61.3
2.40.a
3.60.9
ECE
Ckpt Ens
4.2o.T
14.20.5
18.90
7.21.2
15.7o.5
15.60.3
LLLA
24.61.7
12.41.1
8.10.6
18.61.8
7.60.3
7.50.7
LA
7.2u.7
5.41.0
6.21.5
8.421
2.10.3
3.8u.s
MAP
2.64n.18
1.06p.06
0.53g.m
0.90n.
0.42o.01
0.44o.o1
MC Drop
1.40p.00
0.53p.01
0.32g.o2
0.70p.02
0.34o.m
0.38p.01
NLL 4
Ckpt Ens
0.62v.01
0.470.00
0.42g.u0
0.60p.01
0.46v.00
0.47u.00
LILA
1.36o.10
0.80p.os
0.44g.02
0.866.04
0.410.01
0.43o.01
LA
0.60o.02
0.40o.01
0.280.00
0.64g.02
0.340.01
0.37o.00
G.4 FINE-TUNING MISTRAL-7B FOR COMMON SENSE REASONING
In this section, we present the results using different decoder-based LLM backbones, specifically Mistral-7B (Jiang al., 2023). Table 13 shows the results of fine-tuning Mistral-7B on common-sense reasoning tasks. LA consistently offers the best NLL across all datasets, as well as 3 out of best ECE, indicating the robustness of LapAace-LoRA on different decoder-only LLM backbones.
G.5 LAPLACE-LORA ON DIFFERENT SUBSETS OF LAYERS
Besides applying Laplace approximation on all layers only the last layer, also apply Laplace approximation on a subset of layers, for example, the first layers. In Table 14, we present the results of applying Laplace approximation the top 8 layers, 16 layers, 24 layers, and all layers. Although
22

---

Published as a conference paper at ICLR 2024
MAP
MC Dropout
Checkpoint Ensemble
Temp
ULLA
(635)
(2.49k)
(3.67k)
(5.43k)
(8.55k)
BoolQ (9.43k)
80
80
40
ACC 198)
30
70
60
60
ECE (99)
50
20
20
20
10
10
10
10
2
1.0
0.5
0.6
0.5
10000 training steps
10000
10000
10000
10000
10000
training steps
training Steps
training steps
training steps
training steps
Figure 5: Fine-tuning RoBERTa-base across the six GLUE and SuperGLUE tasks, with a validation set for hyperparameter tuning. The vertical dashed line gives the checkpoint with optimal MAP performance on the validation set.
Table 11: Comparison of different post-hoc methods applied to the fine-tuned RoBERTa- base across six GLUE and SuperGLUE tasks, with a validation set for hyperparameter tuning. Results are evaluated at the best MAP performance checkpoint observed on the validation set.
Methods
Metrics
WNLI
RTE
MRPC
WiC
CoLAA
BoolQ
MAP
47.96.0
70.90.9
86.40.6
63.91.2
81.80.3
77.2a.2
MC Drop
47.96.0
72.4p.9
87.1
68.8 68.80.9
82.6p.1
76.60.4
Ckpt Ens
47.96.0
71.8p.9
86.3g.5
64.7o.3
81.4p.r
77.20.1
1CC
Temp
47.96.0
72.60.3
86.50.3
65.4n.
81.8n.s
77.30.2
LLLA
46.010.a
72.6p.3
86.4a.4
65.30.7
81.8p.7
77.40.2
LA
48.47.7
72.60.3
86.4g.3
65.4o.s
81.70.7
77.40.2
MAP
6.34.3
17.81.4
10.2g.2
22.81.1
10.40.9
9.2u.1
MC Drop
6.44.4
9.01.7
6.3a.4
12.2u.5
2.80.3
3.40.1
Ckpt Ens
6.22.3
8.80.9
15.70.6
5.51.0
13.1o.5
10.60.1
ECE4
Temp
5.51.
4.51.3
1.90.3
5.80.9
2.9p.6.
3.50.7
LLLA
6.87.8
6.90.7
2.60.4
16.414
7.40.3
6.8o.2
LA
4.75.2
2.71.0
2.2a.7
5.50.5
2.80.6
1.90.s
MAP
0.70n.01
0.76p.03
0.66g.n4
1.00n.02
0.50p.02
0.54g.on
MC Drop
0.70p.01
0.58p.03
0.39g.m
0.72n.m
0.390.00
0.50c.on
Ckpt Ens
0.690.00
0.570.00
0.44g.00
0.63u.00
0.490.00
0.53g.00
NLL
Temp
0.69g.u0
0.540.00
0.32p.00
0.62p.01
0.40v.a1
0.490.u0
LLLA 0.690.00
0.56p.01
0.33g.00 00
0.780.03
0.44p.00
0.51o.00
LA
0.690.n0
0.544..0
0.34g.m
0.62o.m1
0.39o.n0
0.48p.on
there is no conclusive trend in the results, applying Laplace approximation to more than 8 layers usually leads to improved NLL.
G.6 DIAGONAL LAPLACE APPROXIMATION
In this section, we present the results for Laplace-Loke LoRA utilizing a diagonal approximation of the Hessian. This approach is generally not found to be effective the froneckerffcctons Approximate Curvature (K-FAC) (Daxberger et al., 2021a) that approximates the block diagonal Hessian.
23

---

Published as a conference paper at ICLR 2024
MAP
MC Dropout
Checkpoint Ensemble
Temp
ULLA
(635)
(2.49k)
(3.67k)
(5.43k)
(8.55k)
BoolQ (9.43k)
90
90
90
70
ACC (98)
80
60
70
40
20
ECE 196
20
20
20
1.5
1.5
0.75
0.5
0.5
NIL
2.
1.0
0.50
1.0
0.4
0.5
0.25
10000
10000
10000
10000
10000
10000
training steps
training steps
training steps
training steps
training steps
training steps
Figure 6: Fine-tuning RoBERTa-large across the six GLUE and SuperGLUE tasks, with validation set for hyperparameter tuning. The vertical dashed line gives the checkpoint with optimal MAP performance on the validation set.
Table 12: Comparison different post-hoc methods applied to the fine-tuned RoBERTa-large across six GLUE and SuperGLUE tasks, with validation set split from the training set used for tuning temperature and Laplace prior precision. Results are evaluated at the hest MAP performance
checkpoint observed the validation set.
Methods
Metrics
WNLI
RTE
MRPC
WiC
CoLA
BoolQ
MAP
65.32.7
82.2o.9
88.5o.7
69.31.9
85.3o.6
84.40.6
MC Drop
60.64.0
82.2p.3
88.7o.7
69.8p.8
85.10.3
84.00.6
Ckpt Ens
65.32.7
80.7o.3
88.61.1
68.81.0
85.20.4
84.30.s
ACC
Temp
65.32.7
82.2o.9
88.5p.7
69.31.9
85.3p.s
84.4a.s
LLLA
65.32.7
82.2o.9
88.5u.7
69.31.9
85.3v.s
84.4u.6
LA
65.32.7
82.3o.s
88.50.7
69.32.0
85.30.5
84.40.6
MAP
30.03.s
13.7o.4
9.2o.7
22.0o.s
8.8p.s
10.50.6
MC Drop
24.92.6
8.80.8
6.0p.9
15.11.1
3.3g.a
5.50.4
Ckpt Ens
7.73.0
13.11.2
18.0p.5
3.31.e
15.0o.3
14.40.1
BCE
Temp
8.122
2.60.7
3.10.4
6.41.
2.70.8
1.90.7
LLLA
12.7a.9
3.31.0
3.0b.5
13.11.2
4.0p.4
5.30.8
LA
6.83.3
3.11.1
2.60.7
4.71.7
2.20.2
1.70.4
MAP
1.63u.09
0.77o.07
0.59p.aG
1.140.02
0.46p.o2
0.56g.0
MC Drop
1.02a.13
0.48o.05
0.33p.a1
0.79g.u2
0.36p.o1
0.41g.00
Ckpt Ens
0.65o.02 02
0.490.01
0.42o.00
0.61o.01
0.46o.00
0.460.00
NLL L
Temp
0.65g.01
0.410.01
0.28p.02
0.61o.018
0.36p.00
0.37g
LLLA
0.69p.02
0.40p.02
0.27o.01
0.73o.os
0.36p.01
0.40p.o1
LA
0.64o.00
0.39o.u1
0.27o.01
0.61o.01
0.35u.o1
0.37u.01
G.6.1 ROBERTA
The results RoBERTa- and RoBERTa- shown in Figure 8 Table 15, and Figure Table 16. LLLA still offers tiny advantage over Maximum a Posteriori (MAP) in ECE and NLL, however, LA show mixed performance across the datasets.
24

---

Published as a conference paper at ICLR 2024
MAP
MC Dropout
Checkpoint Ensemble
Temp
ULLA
(640)
(1.12k)
(2.25k)
(2.56k)
OBQA (4.96k)
BoolQ (9.43k)
70
90
80
90
ACC (88)
60
60
80
70
80
50
70
60
70
15
30
10
ECE 190
20
20
10
10-
1.5
0.6
2
0.4
10000
10000
10000
10000
10000
10000
training steps
training steps
training Steps
training steps
training steps
training steps
Fine-tuning of LlaMA2-7B across six common sense reasoning tasks (presented column- wise, with number of training examples brackets), evaluated on the test set every 1000 gradient steps. The vertical dashed line gives the checkpoint with optimal MAP performance on a held-out validation set.
Figure 7:
Table 13: Comparison of different post-hoe methods applied to the fine-tuned Mistral-7B across six common sense reasoning tasks. Results are evaluated at the early stopping point of 5000 gradient steps. We report standard deviations in subscripts, and bold numbers that are statistical significant.
Methods
Metrics
WG-S
ARC-C
ARC-E
WG-M
OBQA
BoolQ
MAP
77.4u.s
82.31.a
90.7a.4
83.1o.s
87.00.s
89.2g.1
MC Drop
77.40.4
81.61.4
90.7a.s
83.01.0
87.10.s
89.3a.1
ACC f
Ckpt Ens
77.20.2
81.81.7
90.70.7
83.80.5
88.30.5
89.0o.3
LLLA
77.4n.5
8i.71.0
90.6g.4
83.1o.s
87.0p.5
89.2o.1
LA
77.2n.3
79.93.3
89.4g.9
83.0n.s
87.6p.s
89.2s.1
MAP
22.0u.s
16.11.a
8.5a.G
13.51.5
9.3p.0
5.60.a
MC Drop
20.5p.3
15.21.5
7.60.4
13.01.6
8.8p.s
5.30.6
ECE
Ckpt Ens
21.1o.3
14.72.3
7.1p.8
10.la.s
6.3o.3
2.0b.G
LLLA
10.2o.5
29.4g.1
19.17.7
12.31.4
7.60.4
5.40.5
8.20.3
21.86.7
21.6a.1
6.61.7
7.20.9
1.80.4
2.38p.27
1.60o.16
0.75g.12
0.80p.14
0.57a.33
0.32a.02
MC Drop
2.120.20
1.41o.10
0.62g.07
0.75p.12
0.54p.03
0.31o.02
NLL
Ckpt Ens
2.04n.13
1.330.16.
0.48p.02
0.55n
0.45o.01
0.27o.01
LLLA
0.540.01
0.82o.01
0.480.05
0.67o.07
0.470.05
0.32o.02
LA
0.52o.02
0.80o.15
0.480.11
0.42o.01
0.37o.01
0.270.01
G.6.2 LLAMA2-7B
On LlaMA2-7B, both LLLA and LA show mixed performance and do not offer consistent improve- ments shown in Figure 10 and Table 17. Specifically, the accuracy of diagonal LA is much worse than MAP on ARC datasets, and ECE is worse than MAP on ARC-E, OBQA and BoolQ.
On the other hand, when split validation set from the training set and tune the Laplace prior precision using the validation log- liceliiod,oodd, the results are much better, as shown in Figure 11 and Table 18.
25

---

Published as a conference paper at ICLR 2024
Table 14: Laplace-LoRA applied to different subsets of layers on the fine-tuned LlaMA2-7B across six common sense reasoning tasks. Results are evaluated at the early stopping point of 5000 gradient steps. We report standard deviations in subscripts, and bold numbers that are statistical significant.
Methods
Metrics
WG-S
ARC-C
ARC-E
WG-M
OBQA
BoolQ
LA 8
67.4g.s
65.6p.4
84.71.5
73.30.8
79.3p.4
85.90.3
LA 16
67.60.8
65.6p.4
84.61.4
73.40.7
79.30.2
85.90.3
ACC
LA 24
67.50.8
65.1o.5
84.51.4
73.40.7
79.3o.2
85.90.3
LA
67.3a.2
65.3p.2
85.11.5
73.40.3
78.9p.2
86.1u.2
LA
6.40.9
3.6p.4
4.31.6
8.7o.6
6.3p.3
2.30.5
LA 16
2.81.0
5.80.8
5.30.3
5.80.5
4.60.7
1.40.0
ECE
LA 24
2.01.1
7.21.1
6.61.7
5.20.5
3.70.4
1.30.0
LA
2.1g.3
7.4p.7
5.40.2
7.40.4
6.40.s
2.1o.6
LA 8
0.63o.00
0.87o.03
0.50o.
0.67o.01
0.680.01
0.34o
LA 16
0.61a.01
0.86p.02
0.480.06
0.59
0.63o.01
0.33o o0
NLL
0.60g.01
0.87o.02
0.480.05
0.58p.01
0.62p.01
0.330.00
LA
0.600.m
0.88p.03
0.490.06
0.63o.02
0.65p.01
0.34g.01
MAP
MC Drapeut
Checkpoint Ensemble
LLLA
WNLI (635)
RTE (2.49k)
MRPC (3.67k)
WIC (5.43k)
CoLA (8.55k)
BoolQ (9.43k)
90
70
80
ACC (8)
20
70
80
60
10
60
20
20
EE2 180)
50
20
10
10
0
0.6
0.5
NIL
1.0
0.8
0.4
0.4
0.5
0.5
0.6
10000
10000
training steps
training steps
training steps
training steps
training steps
training steps
Figure 8: Fine-tuning of RoBERTa-base across six GLUE and SuperGLUE tasks (presented column- wise, with number of training examples in brackets), evaluated on the test set every 1000 gradient steps. The vertical dashed line gives the number of training steps with optimal MAP performance. LA and LLLA using diagonal Fisher approximation.
H LA Vs LLLA, OR WHICH LAYERS ARE UNCERTAIN?
When comparing LA and LLLA, it is important to understand where the uncertainty arose from: the last layer (which is the only thing that LLLA captures), or the earlier layers (which is also captured by LA). To understand this, we plotted the standard deviation of the logits arising from various sources (Fig. 12). We found that for full Laplace-LoRA LA. almost all the uncertainty arose from the lower-layers (green solid bar), rather than the last layer (blue solid bar). Indeed, when optimizing the prior precisions using the model evidence, this implied that L.L.A (orange solid bar) gave rise to far less uncertainty in the logits than LA. This is likely a good explanation for the poor performance of LLLA when optimizing using the model evidence (Table 1). However, optimizing the prior precisions using the validation LL gives radically different results for LLLA, with much higher logit variances (dashed orange bar), and reasonable performance (Table 2). To understand what is going on here, consider setting where alot of variability in the logits is necessary to optimize the validation LL. Then if we use the validation LL directly as an objective, LLLA can just increase the prior precision until the variance in the logits high enough. While this may be reasonable strategy, the resulting
26

---

Published as a conference paper at ICLR 2024
Table 15: Comparison of different post-hoc methods applied the fine-tuned RoBERTa-base across six common GLUE and SuperGLUUE tasks. Results are evaluated at the early stopping point of 5000 gradient steps. LA and LLLA using diagonal Fisher approximation.
Methods
Metrics
WNLI
RTE
MRPC
WIC
CoLA
BoolQ
MAP
22.54.6
72.82.2
87.1o.7
64.52.0
82.4p.6
77.20.4
MC Drop
19.73.0
74.01.6
88.2a.0
68.5n.
82.7o.1
76.70.6
ACC
Ckpt Ens
21.65.5
73.92.5
87.70.6
64.31.
81.80.5
76.60.2
LLLA
22.54.8
72.82.2
87.10.7
64.52.0
82.40.
77.20.4
LA
22.54.8
72.72.1
87.1a.5
64.71.9
82.30.7
77.20.3
MAP
66.73.2
20.92.1
8.30.1
18.42.2
8.60.5
5.30.2
MC Drop
65.61.a
12.71.6
4.81.6
8.90.6
1.20.1
1.50.4
ECE,
Ckpt Ens
14.63.9
8.51.3
17.90.1
5.21.0
13.60.3
12.50.3
LLLA
64.0g.7
19.62.1
7.5u.2
17.42.3
7.7u.s
4.6G.1
LA
33.7a.0
14.12.8
25.0a.s
6.01.
13.11.4
16.40.5
MAP
3.100.09
1.050.09
0.43g.m
0.790.03
0.46n.02
0.50p.00
MC Drop
2.070.08
0.63p.01
0.300.01
0.65o.01
0.38g.00
0.49u.00
NLL
Ckpt Ens
0.96u.02
0.56g.01
0.44g.00
0.63o.01
0.49b.a0
0.55a.00
LLLA
1.
0.93o.09
0.39g.m
0.770.03
0.440.02
0.50c.on
0.
0.60p.01
0.5lg.0
0.63o.00
0.46p.01
0.57g.01
Ensemble
WNLI (635)
RTE (2.49k)
MRPC (3.67k)
WiC (5.43k)
CoLA (8.55k)
BoolQ (9.43k)
90
90
70
230 (96)
80
60
80
60
+ 196) CE3
20
20
20
20
10
0.75
0.5
1.0
0.75
0.4
0.4
0.5
0.25
10000
10000
10000
10000
10000
10000
training steps
training steps
training steps
training steps
training steps
training steps
Figure 9: Fine-tuning of RoBERTa-large across six GLUE and SuperGLUE tasks (presented column- wise, with number of training examples brackets), evaluated on the test every 1000 gradient steps. The vertical dashed line gives the number of training steps with optimal MAP performance. LA and LLLA using diagonal Fisher approximation.
validation-LL optimized LLLA posteriors differ dramatically from those we would expect under Bayes, where the uncertainty in at the last layer weights is far lower, and most of the uncertainty in logits arises at lower layers.
I EVALUATION DATASETS UNDER LARGER DISTRIBUTION SHIFT
Here present the specific MMLU datasets used for evaluations under distribution shift in Table and Table 8. The specific task splits we selected for each subject are shown Table 19 assigned by Hendryeks al. (2020).
27

---

Published as a conference paper at ICLR 2024
Table 16: Comparison of different post-hoc methods applied the fine-tuned RoBERTa-large across six common GLUE and SuperGLUE tasks. Results are evaluated at the early stopping point of 5000 gradient steps. LA and LLLA using diagonal Fisher approximation.
Methods
Metrics
WNLI
RTE
MRPC
WIC
CoLA
BoolQ
MAP
70.0
83.4p.3
88.2g.3
68.42.0
85. 85.2o.s
84.40.4
MC Drop
69.02.3
85.0o.3
88.90.s
69.81.4
86.0n.1
84.00.6
ACC
Ckpt Ens
68.50.7
83.20.9
88.20.5
48.824
84.60.4
84.30.5
LLLA
70.01.s
83.50.3
88.20.3
68.42.0
85.20.5
84.40.4
LA
70.01.8
83.60.5
88.0g.3
68.52.2
85.30.4
84.40.4
MAP
28.51.6
14.40.8
9.30.6
19.71.8
8.00.4
8.00.6
MC Drop
25.21.6
8.80.9
5.31.0
12.61.3
2.40.3
3.60.9
ECE,
Ckpt Ens
4.20.T
14.20.5
18.90.2
7.21.2
15.7o.5
15.60.3
LLLA
22.43.0
12.01.a
8.20.3
17.81.s
6.7u.4
6.7g.7
LA
14.2u.s
23.81.a
28.3a.2
9.52.
17.30.a
20.20.a
MAP
2.640.18
1.060.06
0.53g.01
0.90n.04
0.42n.01
0.44p.01
MC Drop
1.400.06
0.53p.01
0.32o.02
0.700.02
0.34g.01
0.380.01
NLL
Ckpt Ens
0.62g.01
0.470.00
0.42g.00
0.60p.01
0.460.a0
0.47g.00
LLLA
0.88p.o5
0.660.06
0.39g.n2
0.82g.03
0.39p.01
0.41o.01
0.64p.01
0.55g.01
0.54g.m
0.62g.00
0.45p.01
0.50g.o0
MAP
MC Drapout
Checkpoint Ensemble
LLLA
WG-S (640)
ARC-C (1.12k)
ARC-E (2.25k)
WG-M (2.56k)
OBQA (4.96k)
BoolQ (9.43k)
70
90
80
90
ACC (86
60
80
70
80
50
50
70
60
70
30
30
E33 (60
20
20
20
10
10
10
1.5
0.6
1.0
2
1.0
0.4
training stops
training steps
training stops
training steps
training
training steps
Figure 10:
Fine-tuning of LlaMMA-7B across six common reasoning tasks (presented column- wise, with number of training examples in brackets), evaluated on the test set every 1000 gradient steps. Laplace (diagonal) prior precision tuned using model evidence.
J CODE IMPLEMENTATIONS
We open sourced an original implementation based on Laplace Redux (Daxherger et al., 2021a) and ASDL (Osawa al., 2023) at https: //githu..... com/ adaaxyang/ waapn//llpaaeeeaaeee lora, and a newer standalone implementation which we intended to support going forward at https: lora.
//github. com htt/////a////e/e/eeaamaaaaaa
28

---

Published as a conference paper at ICLR 2024
Table 17: Comparison of different post-hoc methods applied the fine-tuned LlaMA2-7B across six common sense reasoning tasks. Results are evaluated at the early stopping point of 5000 gradient steps.
Methods
Metrics
WG-S
ARC-C
ARC-E
WG-M
OBQA
BoolQ
MAP
67.4n.3
66.3o.s
84.71.5
73.40.4
78.7o.4
86.10.2
MC Drop
67.8p.1
65.31.0
85.0..
73.2n.s
79.5n.2
86.0g.3
ACC f
Ckpt Ens
67.40.2
65.50.4
85.80.2
73.6n.
79.1o.1
86.30.2
LLLA
67.70.3
65.51.4
84.61.2
73.60.7
78.To.7
86.0o.3
LA
67.40.4
63.21.9
82.80.2
73.51.1
78.To.4
86.10.3
MAP
31.2u.3
31.0p.a
13.41.3
23.0u.1
16.1u.e
4.0u.5
Drop
29.40.3
29.60.8
12.41.2
22.2o.2
15.00.4
4.10.4
ECE +
Ckpt Ens
29.70.6
27.01
9.80.8.
17.40.0
12.40.3
1.20.4
LLLA
23.31.7
20.92.a
11.81.9
22.5u.b
15.9u.a
4.1o.G
LA
11.6u.5
18.30.s
16.6a.7
6.61.2
17.21.2
17.00.9
MAP
3.15n.10
3.280.29
1.26g.13
1.5la.0
0.990.05
0.35g.01
MC Drop
2.8lo.11
2.820.21
1.11a.10
1.4lo.03
0.95o.04
0.35o.01
N
Ckpt Ens
2.58v.13
2.360.34
0.80g.06
0.870.08
0.760.01
0.330.00
LLLA
1.02o.10
1.300.11
0.90g.24
1.420.05
0.960.05
0.35o.01
LA
0.64o.01
1.000.04
0.570.03
0.55o.01
0.69o.m
0.44..o1
MAP
MC Dropout
Checkpoint Ensemble
Temp
ULLA
(640)
ARC-C (1.12k)
(2.25k)
WG-M (2.56k)
OBQA (4.96k)
BoolQ (9.43k)
70
70
90
80
90
WCC (98)
60
60
80
70
80
50
70
660
70
10
ECE 1900
20
20
10
15
0.6
2
0.5
0.4
10000
10000
10000
10000
10000
10000
training stops
training steps
training
training stops
training steps
training steps
Fine-tuning of LlaMA2-7B across six common sense reasoning tasks (presented column- wise, with number of training examples brackets), evaluated on the test set every 1000 gradient steps. The vertical dashed line gives the checkpoint with optimal MAP performance a held-out validation set. Temperature scaling and Laplace (diagonal) prior precision are tuned on the validation
Figure 11:
29

---

Published as a conference paper at ICLR 2024
Table 18: Comparison of different post-hoe methods applied to the fine- tnnettundd LlaMA2-7B across six common sense reasoning tasks, with validation set split from the training set used for tuning temperature and Laplace prior precision. Results are evaluated at the best MAP performance checkpoint observed on the validation set.
Methods
Metries
WG-S
ARC-C
ARC-E
WG-M
OBQA
BoolQ
MAP
67.0 67.0o.s
64.91.1
85.2g.G
73.7a.9
77.7p.a
85.80.4
MC Drop
66.7o.3
64.91.9
85.1g.5
73.50.9
77.70.2
85.90.4
Ckpt Ens
66.7o.3
64.91.1
85.2g.6
73.81.0
78.20.2
85.40.3
ACC
Temp
67.00.s
64.91.1
85.20.6
73.70.9
77.70.8
85.8p.4
LLLA
66.70.3
64.41.
85.1g.8
73.11.2
77.20.a
85.70.4
LA
66.5p.3
66.01.2
85.01.2
73.11.1
77.3p.a
85.7a.a
MAP
30.8
26.11.4
8.90.3
24.91.3
9.81.0
7.40.1
MC Drop
29.51.s
25.6u.7
8.8u.6
23.51.2
8.8v.a
7.5u.1
Ckpt Ens
25.21.a
26.11.4
8.9u.3
22.81.4
4.7u.a
3.2p.5
BCB
Temp
12.8u.9
4.61.0
4.7a.8
6.31.6
7.22.6
2.50.3
LLLA
11.81.1
6.01.8
3.70.5
4.52.2
6.2g.5
2.60.9
LA
6.91.5
7.3p.6
3.11.2
4.31.a
4.31.1
2.90.5
MAP
2.750.57
1.646.19
0.54g.03
2.430.50
0.710.03
0.43o.01
MC Drop
2.54n.49
1.55p.16
0.52g.04
2.120.35
0.71p.04
0,43g.01
Ckpt Ens
1.31p.04
1.640.18
0.54g.03
1.89p.24
0.65n.02
0.35n.o1
NLL
Temp
0.68p.01
0.906.01
0.43g.02
0.58g.01
0.67p.02
0.35n.on
LLLA
0.680.02
0.950.03
0.44..01
0.57o.01
0.660.02
0.350.00
LA
0.666.02
0.866..03
0.400.02
0.55%.01
0.63p.01
0.35p.01
LA-Early
LA-Last
LLLA
WG-S (640)
ARC-C (1.12k)
ARC-E (2.25k)
WG-M (2.56k)
OBQA (4.96k)
BoolQ (9.43k)
10a
legits
10a
10a
10
evidence
val LL
evidence
val LL
evidence
val
evidence
val
evidence
val
evidence
val LL
Figure 12: Bar chart of averaged logits standard deviation (bottom row) when optimizing Laplace model evidence (solid bars) on the training set at the early stopping checkpoint of 5000 training steps, and validation log-likelihood (dashed bars) on the validation set at the best MAP validation ACC checkpoint for each dataset. LA-Early involves taking trained LA model, freezing the last layer and evaluating the uncertainty arising just from the "earlier" layers (i.e. all except the last layer). Likewise, LA-Last involves taking trained LA model, freezing the earlier layers (i.e. all except the last layer) and evaluating the uncertainty arising just from the last layer. The ordering of the bars is the same as the ordering in the legend.
30

---

Published as a conference paper at ICLR 2024
Subject Computer Science (CS)
Task
college computer science
computer security
high school computer science
machine learning
Engineering (Eng) Law
electrical engineering
international law
jurisprudence
professional law
Health
anatomy clinical knowledge
college medicine
human aging nutrition professional medicine virology
Table 19: MMLU subjects and tasks.
31


Reviews:
[{"review_id": "vY3dixUgk3", "reviewer": "Reviewer_EVj9", "summary": "A Bayesian LoRA approach has been proposed to fine-tune large language models (LMMs) by estimating the weight posterior through layer-wise Laplace approximation. Experiment results on six QA benchmark datasets were provided in terms of accuracy, calibration, and OOD generalization, compared with three baseline methods.", "questions": "Please refer to the questions raised in *Weaknesses*.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- **Timely research**: The proposed method focuses on improving the calibration performance when finetuning LLMs on small-scale datasets, which is an important and urgent research problem along with the rapid growth of large models. \n\n- **Clear Bayesian treatment**: The proposed method adopts well-established techniques from prior works of Bayesian neural networks and uncertainty reasoning, and successfully incorporates such a Bayesian treatment into parameter-efficient tuning approaches. The proposed Laplace-LoRA seems to be a scalable solution to enable uncertainty estimation for large models.\n\n- **Good experiment design**: Despite some practical issues (see *weakness*), the experiment was conducted well on six public datasets with several strong baseline methods, in terms of three settings -- 1) early stopping, 2) finetuning with validation, and 3) OOD generalization. The proposed method generally achieves better ECE scores across different cases.", "weaknesses": "- **Unclear uncertainty estimation**: While the proposed Laplace-LoRA naturally estimates the weight posterior, it is unclear how to apply the proposed method to compute model uncertainties. Also, it remains unclear if the proposed method can handle the structured uncertainty estimation for next-token predictions (e.g., *Uncertainty estimation in autoregressive structured prediction, ICLR'21*). It would also be interesting to compare the proposed method with semantic uncertainty [Kuhn et al., ICLR'23]. \n- **Lack of LLM backbones**: The empirical evidence of the proposed method is somewhat weak due to the lack of more LLM backbones. Some concerns include: 1) can the proposed Laplace-LoRA work with larger model sizes (e.g., 13B, 70B)? 2) Can Laplace-LoRA be applied to models other than LLaMA2 (e.g., Mistral 7B, GPT-2, etc.)?\n- **Comparison to ensemble**: Besides the checkpoint ensemble, it is also expected to implement a baseline given by the LoRA ensemble (similar to the deep ensemble approach).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "maLOmWTvRZ", "reviewer": "Reviewer_5qzU", "summary": "This paper introduces Laplace-LoRA, a Bayesian inference method designed specifically for the LoRA parameters in LLM fine-tuning using a post-hoc Laplace approximation. They conducted extensive experiments on six commonsense reasoning tasks and show dramatic improvements in calibration of fine-tuned LLMs.", "questions": "Does free-text generation tasks (e.g., free-from QA) still work under this framework? Why that kind of reasoning tasks are not considered in your experiments?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. This idea of combining Laplace inference with the fine-tuning LLMs using LoRA adapters is novel, which provides a new way of doing Bayesian fine-tuning on LLMs.\n2. They conducted extensive experiments on six commonsense reasoning tasks under in/out-of-distribution settings and provided detailed analysis of the experiment results.\n3. The writing is well-structured, clear and easy to understand.", "weaknesses": "1. It has some novelty, but not dramatic, because both Laplace Approximation and LoRA method are well-studied.\n2. It is quite weird that the Section 3 Background followed by Section 4 Results directly, without a Method section in between. Maybe it needs a better section name.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "u0vxppNMx6", "reviewer": "Reviewer_sBHc", "summary": "This paper presented a Bayesian approach to improve the calibration of LoRA. It adopted Laplace approximation to the posterior of the low rank parameters. Empirical results on both in-distribution and out-distribution finetuning indicated the Bayesian approach does improve the calibration and the proposed Laplace approximation outperform a variety of baseline methods.", "questions": "1. Laplace Redux has discussion on relative wall-clock time cost and memory consumption comparison with other methods, is there similar result for Laplace-LoRA?\n2. More ablation study is expected. For example, LA on different types of layers (attention, dense, etc.), LA on different layers and different number of layers (first k layers or random select layers), or some fusion of the two mentioned above.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This paper is the first to present a comprehensive result on using Laplace approximation to LoRA for LLMs.\n2. This paper has clear presentation with visualization.\n3. Claims are supported with sufficient amount of convincing experiment result. e.g., smaller datasets experience larger difference in ECE compares with larger datasets.", "weaknesses": "1. Limited novelty. The Bayesian method part (as indicated in the paper) is well explored in the literatures listed in the software paper Laplace Redux [1]. This paper can be viewed as empirical results applying [1] to a specific model - LoRA for LLMs. \n2. Majority of the benefits of Laplace-LoRA including ''post-hoc'' and ''scalable'' are from the existing method, which limits the contribution of this work. This one together with Weakness #1 above are the major concerns from my point of view.\n3. There was some discussion on the cost to perform Laplace approximation when introducing Fasher. However, there is no empirical results that support it.\n4. The ablation study (LLLA vs LA) is also from the [1]. Only comparing LA with Last Layer LA is not very convincing on where the uncertainty comes from.\n\n[1] Erik Daxberger, Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen, Matthias Bauer, and Philipp Hennig. Laplace redux-effortless bayesian deep learning. NeurIPS, 2021.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IynnA0CLxA", "reviewer": "Reviewer_CyBU", "summary": "This paper proposed a new fine-tuning procedure based on LoRA using Bayesian prospective.", "questions": "See above.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "This paper is very well-written and the algorithm is presented in a very clean way. Although the algorithm is straightforward, as far as I know, nobody has made it work before. The empirical analysis is done carefully which is appreciated. Code is also linked.", "weaknesses": "1. It looks like all the tasks using in this paper are still classification tasks. How it is different from standard Bayesian deep learning evaluation? I want to understand where the benefit is coming from on those eval metrics through a principled way. I feel it will be good to comment some potential on generation tasks that can reflect more of the autoregressive nature of transformers.\n\n2. Where is the comparison of computational complexity and memory usage? How much additional memory you will use?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)