Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineering
Authors: Han Zhou
Xingchen Wan
Lev Proleev
Diana Mincu
Jilin Chen
Katherine A Heller
Subhrajit Roy
Venue: ICLR 2024 poster (2024)
Keywords: large language model
in-context learning
calibration
prompt
Primary area: generative models

Abstract: Prompting and in-context learning (ICL) have become efficient learning paradigms for large language models (LLMs). However, LLMs suffer from prompt brittleness and various bias factors in the prompt, including but not limited to the formatting, the choice verbalizers, and the ICL examples. To address this problem that results in unexpected performance degradation, calibration methods have been developed to mitigate the effects of these biases while recovering LLM performance. In this work, we first conduct a systematic analysis of the existing calibration methods, where we both provide a unified view and reveal the failure cases. Inspired by these analyses, we propose Batch Calibration (BC), a simple yet intuitive method that controls the contextual bias from the batched input, unifies various prior approaches and effectively addresses the aforementioned issues. BC is zero-shot, inference-only, and incurs negligible additional costs. In the few-shot setup, we further extend BC to allow it to learn the contextual bias from labeled data. We validate the effectiveness of BC with PaLM 2-(S, M, L) and CLIP models and demonstrate state-of-the-art performance over previous calibration baselines across more than 10 natural language understanding and image classification tasks.

Published as a conference paper at ICLR 2024
BATCH CALIBRATION: RETHINKING CALIBRATION FOR IN-CONTEXT LEARNING AND PROMPT ENGINEERING
Han Zhou! 1,2,, Xingchen Wan'
Lev Proleev'
Diana Mincu'
Jilin Chen
Katherine Heller' Subhrajit Roy! 'Google Research University of Cambridge
[hzhouml, xingcheww, levp, dmincu, jiline, kheller, stthrajjitror
ABSTRACT
Prompting and in-context learning (ICL) have become efficient learning paradigms for large language models (LLMs). However, LLMs suffer from prompt brittleness and various bias factors in the prompt, including but not limited to the formatting, the choice verbalizers, and the ICL examples. To address this problem that results in unexpected performance degradation, calibration methods have been developed to mitigate the effects of these biases while recovering LLM performance. In this work, we first conduct a systematic analysis of the existing calibration methods, where we both provide unified view and reveal the failure cases. Inspired by these analyses, we propose Batch Calibration (BC), a simple yet intuitive method that controls the contextual bias from the batched input, unifies various prior approaches, and effectively addresses the aforementioned issues. BC zero-shot, inference-only. and incurs negligible additional costs. In the few-shot setup, we further extend BC to allow it to learn the contextual bias from labeled data. We validate the effectiveness of BC with PaLM 2-(S, M. L) and CLIP models and demonstrate state-of-the- performance over previous calibration baselines across more than 10 natural language understanding and image classification tasks.
INTRODUCTION
Prompting large language models (LLMs)
PaLM 2-5
PaLM 2-L
(Chowdhery et al. 2022 Anil al. 2023) has be-
80
80
come an efficient learning paradigm for adapting LLMs to a new task by conditioning on human- designed instructions. The remarkable in-context learning (ICL) ability of LLMs also leads to effi- cient few-shot learners that can generalize from few-shot input-label pairs (Brown et al.) 2020 Liu et al., 2023). However, the predictions of LLMs are highly sensitive and even biased to the choice of templates (Min et al., (222b),, verbalizers (Holtzman et al., 2021), and demonstrations (Liu) al. 2022a), resulting in barriers for pursuing ef- ficiently adaptable and robust LLM applications.
Performance
70
60
60
40
40
ICL
CC
DC PC
BC
ICL CC DC
PC BC
Method
Method
Figure 1: Batch Calibration (BC) achieves the best performance on 1-shot ICL over calibration basclines on average of 13 classification tasks on PaLM 2-S and PaLM 2-L (Anil et al. 2023).
Extensive research has been devoted mitigating these biases, which we explicitly refer to the a-priori propensity of LLMs to predict certain classes over others unfairly. Lu et al. (2022) provide an analysis of the impacts of the order of ICL examples LLMS and have explored the order selection mechanisms for ICL. On the other hand, Zhao et al. (2021) reveal the bias of language models toward certain answers and propose calibrate the LLM given content-free tokens. More recently, Fei et al. (2023) detect the domain-label bias, and Han et al. (2023) treat the calibration of L.LMs, technique for mitigating the label bias, as learning a robust decision boundary. Though multiple calibration solutions have been provided, the field currently lacks unified analysis that systematically distinguishes and explains the unique characteristics, merits, and downsides of each approach.
Work done Student Rescarcher Google.

---

Published as a conference paper at ICLR 2024
In this work, we first conduct comprehensive analysis across existing calibration methods for LLMs. We approach the calibration problem from a distinctive point of view by interpreting the decision boundaries for each calibration method together with the ICL decision boundary. We start observing fatal failure cases for each method by extending them to more challenging and under-explored evaluation tasks. We then conclude the current limitation for each method with a novel interpretation from the decision boundary perspective, pointing to the need for unified and widely applicable solution for conquering diverse bias sources in the field of LLM efficient learning.
Inspired by these findings, we propose Batch Calibration (BC), a zero-shot and inference-only calibration method for prompting and ICL. The central objective of BC is to accurately model the bias from the prompt context (referred to as contextual bias in this paper) by marginalizing the LLM scores in the batched input. The simplicity of the design of BC only brings negligible computation overhead at the output of the LLM. We further extend BC to the black-box few-shot learning (BCL), a practical case where labeled data is available, by introducing a single learnable parameter into BC, which enables it to adapt and learn the contextual bias from the available data resources.
We conducted extensive experiments on more than 10 natural language understanding tasks together with image classification tasks. BC stands as the most widely applicable calibration method while achieving state-of- offee-of-hharrr the-art results. With the proposed black-box few-shot BCL framework, we show that further slight gains can be achieved by leveraging more labeled data. We provide further analysis with BC on robustness with templates, ICL choices and orders, and verbalizers, validating that BC can effectively alleviate prompt brittleness and make prompt engineering easier. To summarize, provide the following contributions:
We provide a unified and systematic analysis of existing calibration methods through their decision boundaries, investigate the common use of content-free tokens as an estimator of contextual bias, and identify their deficiency with individual case studies.
We propose Batch Calibration (BC), zero-shot and inference-only calibration method for ICL, that mitigates the bias from the batch. We further extend BC to learn from few-shot data.
We show that while conceptually simple, BC attains state-of-the-ard performance in both zero-shot and few-shot learning setups over widely selected tasks with PaLM-2 and CLIP models.
2 RELATED WORK
Understanding and Improving ICL. Lu et al. (2022) show the sensitivity of LLMs to ICL examples. This phenomenon further explained through the effect from pretraining term frequencies (Razeghi et al., (022) and corpora (Shin et al.) 2022). Mcanwhile, [iee et al. (2022) explain the ICL process through implicit Bayesian inference, and Wei al. (2023) show the emergent ability of L.LMs by learning new input-label mappings. Various methods have been proposed to optimally select better in-context templates (Sorensen et al., [022 Pan et al. 2023 Yin et al., 2023) and examples (Rubin et al. 2022 Liu et al. 2022a] Wan et al. 2023b). Specifically, Wan et al. (2023a) introduce selection criteria for in-context examples based on the confidence. Recently, noisy channel prompting
(Min et al. 2022a), flipped learning (Ye et al. 2023). and self-ensembling (Li et al. 2023) have
been proposed for robust ICL. Learning to assign labels by k-nearest neighbors (Xu et al. [0223] and
training decoder networks (Cui al. 2023) also effective alternatives for few-shot ICI..
Bias in ICL and Calibrating LLMs. Zhao et al. (2021) reveal the instability of LLMs in few-
shot learning and demonstrate three bias sources: majority label bias, recency bias, and common token bias, as the bias factors behind the instability. They propose contextual calibration (CC) to mitigate these biases by grounding the prediction based on a content- free token as sample inputs. Si al. (2023) characterize the feature bias of L.MMs, and Wang et al. (2023) introduce the positional bias in candidate choices. |Fei et al. (2023) further observe the existence of domain-label bias and propose domain-context calibration (DC) that uses random in-domain tokens for estimating the bias.
Meanwhile, Han et al. (2023) analyze the impact of decision boundary for text classification tasks and propose to estimate prototypical clusters by Gaussian mixture models, thereby learning a robust decision boundary. Concurrently with our work, Pezeeskkpour & Hruschkal (2023) spot the positional bias in multiple-choie questions, and Zheng et al. (2023) propose to debias the positional bias in multiple choices with permutation-base prior estimation. We will discuss quantitatively and provide
2

---

Published as a conference paper at ICLR 2024
unified analysis of these methods in Sec. 3 As we will show, our proposed method differentiates from these methods as generalizable solution across challenging classification tasks and modalities.
3 A SYSTEMATIC ANALYSIS OF CALIBRATION
3.1 BIAS IN PROMPTING AND IN-CONTEXT LEARNING (ICL)
Prompting is an efficient learning paradigm that allows LLMs to perform zero-shot inference by conditioning on human-designed instruction. Formally, denoting a test query-targe pair {x,, } and instruction as the context C for classification task, LLMs make prediction by computing: arg.mx, ya [z, C)2 where p E IR is the logits, and Y denotes the verbalizers that define the label set for classes. ICL further enables LLM to learn from i input-label pairs (i.e., few-shot setup). ( Template(
( Vi E {1.... k by concatenating few-shot demonstrations in pre-defined template as the context, C = Concat .... s(k)). Though ICL has demonstrated strong performance with easy implementations, the prediction of LLMs is shown to be biased towards certain answers due to different elements of p(y) x4, C) (Lu et al., 2022). In the ICL context C, majority label bias and recency label bias (Zhao et al. 2021) can bias the prediction of LLMs toward the most frequent label and the label towards the end of the demonstration, respectively. Among verbalizer tokens Yj Y LLMs are shown to be inherently biased towards predicting the label-tokens that appear more frequently from pretraining term statistics (Shin et al., 2022 Razeghi et al. (2022). These bias factors significantly degrade the performance of LLMs for robust ICL applications.
3.2 OVERVIEW OF ICL CALIBRATION METHODS.
As introduced in Sec. 2 various calibration methods have been proposed to mitigate the issue of bias identified above. In this section, we provide an overview the state-of-the-ar calibration methods.
Contextual Calibration (Zhao et al. 2021) (CC). Motivated by common calibration technique that applies affine transformation on the model outputs (Platt et al.) 1999 Guo et al. (017), Zhao et al. (2021) propose calibrate the LLM prediction by first measuring the entire test-time distribution by a content-free input. Using "N/A" " as a content-free example, the model score distribution is generated by Pat p(y [N/A], C). CC then generates the calibrated output by transforming the uncalibrated scores pyylæ, C*) with W xx via Wp(yla, C²), where W = diag (pa) -1 offsets the uncalibrated scores with the model score contextual prior) triggered by the content-free sample.
Domain-Conted Calibration (Fei et al. 2023) (DC). Instead of using a single content-free token, Fei al. 2023) propose DC that estimates contextual prior P(y(C) by using a random in-domain sequence. It randomly sampled L tokens at average sentence length from an unlabeled text set. Then, estimates the content- free prediction prior by averaging the model score T times, such that:
Prandom = / Σ/= [ [RANDOM TEXT),, C). The final test-time prediction is then calibrated by dividing the estimated prior prediction, or equivalently in logits space, p [z,, C) Prandom-
Prototypical Calibration (Han et al. 2023) (PC). PC learns a decision boundary with Gaussian mixture models (GMMs). It estimates prototypical clusters for the model output p for J classes: (PMM(P) = Σÿzo 0j (P(p) Σj), where Pg denotes a multi-variate Gaussian distribution, and the parameters: mixing coefficient a, mean vector μ., covariance matrix Σ are estimated by the Expectation-Maxinne
(Moon) 1996). Followed by automatic label assignment strategy, the predicted label is then computed by arg max, (o(pplp", E¹) in the inference time. This EM- GMM process can require up to T repetitions to stabilize its estimation of clusters where T is hyperparameter of the algorithm.
3.3 DESIGN PRINCIPLES BEHIND CALIBRATIONS
Summarizing the calibration methods with distinctive design principles discussed so far, in Table I we present unified view of the characteristics of each method with their mathematical formula- tion, computation cost, and strengths & weaknesses. Though each approach demonstrates clear motivation for calibrating ICI., it is still unclear which method surpasses others in what scenarios.
3

---

Published as a conference paper at ICLR 2024
Table I: Calibration methods with their mathematical formulation and their equivalent decision boundary derivations in a two-dimensional problem. The cost for the number of API calls is denoted as #Forward, where 1 counts for the original ICL forward cost, and PC and BC incur no additional API costs. We refer derivations of decision boundaries to Sec. 3.3 The potential failure case for each calibration method in practical scenarios is marked as X.
Comp.
Cali.
Learning
Decision Boundary A(p)
Multi- Sentence
Mulli- Class
Method
Token
#Forward
Cost
Form
Term
CC
NIA
1+1
Inverse
Wp +
W ding(pi bb
P) opt
x
DC
Random
20 1
Add
Wp
W I.b $ + Pislexe, C)
Po Pa a
x
PC
1
EM-GMM
E, Po(plM,, Nny)
Pulpipo: Fu) PotplA,
<
x
BC (Ouns)
1
Add
Wp+b b
W = 1, 8. C3
+
CC
DC
PC
BC
0
SST-2: positive
-2
-2-
-4
-4-
-4
-
CL
-6
-6
-2
BC
CC
DC
PC
88
-8
-6
-4
-2
0
-8
-6
-4
-2
88
-6
-4
-2
0
-4
-2
2
negative
negative
negative
negative
QNL: not entailment
-2-
-2
-2
2
-4-
-4
-6
ICL
-6
ICL
-6
-2
1CL
-5.0 -2.5
0.0
-5.0
-2.5
0.0
-5.0
-25
0.0
-2.5
0.0
2.5
entailment
entailment
entailment
entailment
Figure 2: Visualization of the decision boundaries of uncalibrated ICL. and after applying existing calibration methods and the proposed BC (to be introduced in See 4 in representative binary classification tasks of SST-2 (top row) (Socher et al. 2013) and QNLI (bottom row) (Wang et al. 2018) on 1-shot P 2-S. We show success and failure cases for each baseline method (CC, DC, and PC), whereas BC is consistently effective. Refer to Appendix S for more examples.
We proceed with an in-depth analysis of existing methods in representative tasks. We provide a novel view of calibration methods from multi-variate decision boundary perspective. In pursuit of practical guidelines for ICL calibration, we set out two important research questions behind their design principles: 1) What constitutes better decision boundary for calibrations? 2) Is content- free prior a good estimator of contextual bias?
What Constitutes a Better Decision Boundary for Calibrations? To address this research question, we first derive the decision boundary for each category of calibration methods. We recall that the classification by a LLM is based on arg (0....0..... -01. Pj where py denotes the j-th element
of output vector p. Consider binary classification problem for simplicity: the decision boundary h(p) for ICL is given by the line Po P1 O: the model predicts class 0, yo. if Po P1 2 0. and class otherwise. Consequently, CC and DC that apply an affine transformation at p is equivalent to linear transformation to the decision boundary. In CC with W diag(p)
b O, the = O, which is a rotation of the ICL's linear decision boundary around the origin. Similarly, DC with w =
decision boundary can then be derived as Po Po Pi Pi - Pu - P1 x P1
b = 1 E. p(y [ TEXT]e. C) = P is equivalent to a shift of ICL's linear decision boundary away from the origin, such that po - P1 = (f0 - -))). It is worth noting that both calibration choices lead to a linear decision boundary, indicating that the calibration problem can be framed as an unsupervised decision boundary learning problem. Based on this intuition, we further derive the decision boundary for PC as (p|p/o Eo) Po (1, En) O, which delivers non-linear boundary between the estimated Gaussian mixtures. We conduct preliminary experiment to visualize the derived decision bounds from existing calibration methods alongside the ICL baseline. In Fig. 2
4

---

Published as a conference paper at ICLR 2024
observe that uncalibrated ICL is biased towards predicting negative in the SST-2 task. This biased prediction is then mitigated by each calibration method, where we observe a rotated decision boundary from CC, shifted boundary from DC, and non-linear boundary between the GMMs by PC. However, in the QNLI task (bottom row of Fig. 2. we observe failure cases in the calibration baselines, in particular, PC (third figure from the left), where it fails capture the correct distribution for each class. From Fig. and the additional results in Fig. i0 in Appendix $E we find that while theoretically more flexible, the non-linear decision boundaries learned by PC tend to be susceptible to overfitting and may suffer from instability in EM-GMM. We hypothesize that the PC boundary even more vulnerable to instability for more challenging multi-class tasks due to the increased difficulties of learning clusters and assigning classes correctly. Conversely, we find that linear decision boundaries, as evidenced by CC and DC, can be more robust and generalizable across tasks. We validate this hypothesis by proposing BC with extensive experiments in Sec. 5.2
CC
DC
Is Content-free Input Good Estimator of the Contextual Prior? CC and DC both use a linear decision boundary but differ from each other by leveraging different formats of a content-free input to estimate the contextual prior. However, as we observed in Fig. 2 they both exhibit failure cases in QNLI, a question-answering NIJ task. We hy- pothesize that contrary the proposals made by CC and DC, relying on content-free tokens for calibration is not always optimal and may even in- troduce additional bias, depending on the task type. For example, in textual entailment task involving question-sentence sentence pairs, we empirically observe that an ICI. template employed with content-free token 'N/A" such as 'Question: N/A, Sentence: : N/A, Answer:" will result in biased prediction towards "entai mment', be- cause although 'N/A is intended to be content-free, the LLM may nevertheless construe 'N/A' in the sentence to be substantively entailed to the 'N/A' in the question due surface text equivalency. This phenomenon holds true for other multi-text classification tasks, such as paraphrasing and word disambiguation tasks. Consequently, the prior estimated via single content-free free token can lead to further bias. DC introduces multiple randomly sampled tokens to form a content-free frre input, e.g. 'Questio iessiions : that What old rubisco' reason is that random sequences, when used in conjunction with in-context demonstrations, can be susceptible to spurious relations among them that often lead to unfair priors further skewing the predictions, which is also reflected in Fig. 3 where CC and DC fail to mitigate the contextual bias in the QNLI task. In sum, the empirical observation shows that content- inputs can be inappropriate prior estimators, especially for multi-sentence classification tasks.
-0.2
Score
0.4
0.6
-0.8
Class 0
Class
Class 0 Class
Entailment/ an entailment
Figure 3: The distribution of ICL scores after applying CC and DC on QNLI. Due to an fair content-free prior, the prediction by 1-shot PaLM-2 biased towards entailments
the did Which?' We suspect a possible
4 BATCH CALIBRATION
Inspired by the previous discussions, we now propose Batch Calibration (BC), a zero-shot, inference- only, and generalizable calibration technique with negligible computation cost. We further discuss how BC can be extended to vision-language models as well as the booc-sboxfewwshhr learning setup.
Batch Calibration (BC). Following the discussion in Sec. 3.3 we argue that the most critical component for calibration to accurately estimate the contextual bias term p(y|C). Both CC and DC, which use content-free and in-domain random tokens as trigger signals, respectively, have failure cases in multi-sentence classification when the estimation of the contextual bias is inaccurate. On the other hand, PC is vulnerable to overfitting and may incorrectly model the mixtures, especially in nigh-dimensional space. We, therefore, opt for a linear decision boundary for its robustness, and instead of relying
content-free tokens, we propose to estimate the contextual bias for each class p(y = yj (C)) from a batch with M samples, {x' ,..., x: M ). in a content-bass manner by marginalizing the output score over all samples x P(x:) within the batch:
p(y|C)
E x>P(x)
[pu u,lr, C7 ~ M Σ p(y 1 ,T C)V E..
(I)
5

---

Published as a conference paper at ICLR 2024
Demonstration
Blased Prediction
Calibrated Prediction
Review: Cold Movie.
Sentiment: Negative
Review: The greatest musicians.
Sentiment: Positive
LLM
BC
Review: (test sample)
Sentiment: (label)
a photo of flabel)
VLM
BC
Figure 4: Illustration of Batch Calibration (BC). Batches of demonstrations with in-context examples and test samples are passed into the LLM. Due to implicit bias sources the context, the score distribution from the LLM becomes highly biased. BC is modular and adaptable layer option appended to the output of the LLM/VLM. BC generates calibrated scores according to Eq. & Highlighted symbols indicate the distribution means (visualized for illustration only).
We then obtain the calibrated probability by dividing the output probability over the contextual prior, which is equivalently by shifting the probbabbabilty by the estimated mean of cach class:
f1 arg max (Bc(y C) = arg max [p(y] C* - p(w[C)] y
(2)
BEY
It is noteworthy that this BC procedure is zero-shot and only involves unlabeled test samples. BC incurs negligible computation costs. We may either compute the correction term ((y(C") once all test samples are seen or, alternatively, in an on-the-fly manner that dynamically processes the outputs. To do so, we may use a running estimate of the contextual bias for BC. At the n + mini-batch, the bias term is given by:
1 pn+1 (y C), n +
D n+ (y|C)
n n+ I
(3)
thereby allowing BC's calibration term be estimated from small number of mini-batches that is subsequently stabilized when more mini-batcher arrive.
Adjustable Batch Calibration Layer (BCL.. While BC is designed to be zero-shot and inference-only, it is also common that some la-
CQP Acc. (9))
70
beled data are available. In this section, we de-
BCL (Test)
Test
scribe a simple, adapted variant of BC that may further refine the calibration and mitigate any timation errors from the unlabeled data, which we term BCL.. Specifically, instead of deducting the bias term p from the test data only, we introduce a single additional hyperparameter strength 7 R:
ICL
BC BCL (Labeled
CB Acc. (%)
50
(Tes)
ICU
BC
BCL (Labeled)
25
PBCL (vl r C) p(u|,,, C) - rP(ylC), (4)
where y controls the strength of BC. To select the
Strength
appropriate 7, simply perform grid search by uniformly sampling T' different y values in [a, b] (we set (a, b) : 5,, 5]. but any reasonable range may be used). The strength is then learned by 7" arg max,e]a.8 R(PpC: 7), where R(--)) is the evaluation function (e.g., accuracy) on the set of labeled data, allowing the amount of calibration to be adjusted from evaluation metrics directly.
Figure 5: BC benefits from labeled data: The performance of the adjustable BCL compared to the zero-shot BC with a changing strength. The st rength at O and represent the uncali- brated ICL and BC, respectively. We highlight the optimal strength learned from a labeled set and the best test strength. Refer to Appendix $ for more examples.

---

Published as a conference paper at ICLR 2024
Table 2: Accuracy (%) on natural language classification tasks with I-shot PaLM 2-S and PaLM 2-L Models (Anil et al. 2023). We report the mean and standard deviation for all results for 5 different in-context examples. We reproduce all baselines, and the implementation details are described in Appendix D
The best and second-bed results are marked in bold fonts and ranked by color.
Model
PaLM 2-S
PaLM 2-L
Method
ICL
CC
DC
PC
BC
ICL
CC
DC
PC
BC
SST-2
93.62as2
95,50k.2
94,29
95,71g.0
95.44
93.165
95.82
94.912o1
95.64047
75.78
MNLI
68.52
60.07
63,45:.0
59.29
75.12
72.77
79.45.u
71,53
78.68
81
QNLI
81.20,
56.86
65.621.5
69.82
82.45
64.68 51
69.71.m
68.97
61.01
87.90, S4
MRPC
16.42
70.44.
68.58
71.86
70.05
73.19
72.40
68.68
75.39
70.39%.5
QQP
63.91a.6
65.55,,34
53.92gas
65, (5.288442
71.48,4 46
82.57
81.172m
78.32
81.42
79.56..4
BoolQ
83,99
87. 14
87. 87.64
88.70s.s
87, 13
90.02a mn
90.15
87777
64, 14.4012.57
20
CB
45.71
29.64 ss
65.7 1x:0
81.07..4t
78.21
92.862
85.72
89.29 92.86
93.21, 4M
COPA
96,40%.u
95,80,
96.40:.ss
96.20
96.40%.or on
98.60;
97.200,
97
99.00
97.00, (m)
KTE
80.94
79. 99.780.9
76.82,
80. 80.43,.07
83.47 1.00
75.092.1
80,00
19.21
86.642.22
85.42:.48
WIC
50.69a.s9
50,56g,50
49.97a.13
51.38).56
61.10107
51.35 90
55,58
54.67k02
57.87 11.08
64.83,.5 59
ANLI-RI
46.244.2
42.54s
40,26
40.28
59.82
63.062
71.92ym
73.561.88
72.30
75.00
ANLIIR2
40.44 40.440,00
38, 18,36
38, 38.443,66
41.88,.5
50.16g.5
58.40
65 36
65.48
64, 98
67.34)
ANLI-R3
42,530,99
38,78
43.675
37, 37.500g.1
55.75 .66
61,35
67.32
66.23
63.0 63.036.08
66.38
Avg.
66.20
62,39
64.98
67.65
74.41
75.16
77.83
76,89
76.13
81.09
RTE
MNLI
QQP
WiC
80
Accuracy (%)
70
-60
60
60
55
70
50
60
40
0
1
4
0
1
2
ICL Shots
ICL Shots
ICL Shots
ICL Shots
ICL
CC
DC
Figure 6: The ICL. performance on various calibration techniques over the number of ICL shots on PaL.M 2-S. Each shot indicates example per class in the demonstration. Lines and shades denote the mean and standard deviation over 5 random seeds, respectively.
We give concrete examples in Fig. [1] which illustrates the effect of BCL where we plot the accuracy in QQP (Wang et al. 2018) and CB (De Marneffe et al., 2019) tasks over range of y. We observe that 7 = 1, which corresponds to BC without adjustment (purple line), leads to a strong but not optimal performance. By using the 7 learned from the labeled data (a 128-shot randomly sampled set in this case), BCL estimates the contextual bias more precisely by leveraging the labeled data and achieves performance that is very close to the optimal. We refer readers to Table 3 for more results.
Calibrating Vision-Language Models. Recently, vision-language models (VLM) (Radford et al. 202I). which simultaneously encode visual and textual information. have demonstrated strong zero- shot generalization capability by rewriting class labels. However, the sources of bias as LLMs have also been observed in prompting VLMs (Alayrac et al.) (202)) but have not been adequately addressed.
In this work, we propose to apply BC to Zero-Shot (ZS) CLIP (Radford et al. (021) and mitigate
the biases in its zero-shot classifications. We follow the same notation from Sec. [.1] where the test
image is now and the prompt template becomes the context, C. Similarly, we append the BC layer
at the output of the ZS CLIP and calibrate for each class following Eq. [&]
EXPERIMENTS
5.1 EXPERIMENTAL SETUP
Evaluation Data. For natural language tasks, in contrast previous works that only report on
relatively simple single-sentence classification tasks (Zhao et al. 2021 Fei et al. [223] Han et al.)
2023), conduct experiments 13 more diverse and challenging classification tasks, including the
7

---

Published as a conference paper at ICLR 2024
Table Accuracy (%) on natural language classification tasks with the zero-shot BC and the BCL that involves an optimal strength term learned from a labeled set. The experiments are evaluated with the same in-context example on 1-shot PaLM 2-S.
Method
SST-2
MNLI
QNLI
MRPC
QQP
BoolQ
CB
COPA
RTE
WIC
ANI.RR
ANLIa
ANL
Avg.
BC
95.4
75.0
83.5
68.6
70.3
87.9
75.0
98.0
84.1
63.3
59.8
51.1
53.3
74.3
BCL
96.3
75.0
83,5
74.3
72.3
88.8
83,9
99.0
82.7
63.2
58.0
49.7
52.2
75.3
standard GLUE (Wang al. 2018) and SuperGLUE (Wang al. 2019) datasets. Specifically, we
consider commonsense reasoning: BoolQ (Clark et al.
2019). COPA (Roemmele et al. 2011); word
disambiguation: WiC (Pilehvar & Camacho Colicooo 2019); sentiment classification: SST-2 (Socher
et al. 2013); paraphrasing: QQP, MRPC (Dolan & Brocketi, 2003); natural language inference and entailment: ANLI-R [12.33 (Nie et al. 2020). CB (De Marneffe et al. 2019), RTE, QNLI (QA/NL), MNLI (Williams et al., 2018). For image classification tasks, we include SVHN Yuvaal 201
EuroSAT (Helber et al. 2019), and CLEVR (Johnson et al. 2017).
Models. We conduct experiments mainly the state-of-the-ar PaLM 2 (Anil et al. 2023) for its
variants with different sizes, PaLM 2-S, PaLM 2-M, and PaLM 2-L. PaLM has achieved results
that are competitive with GPT-4 (OpenAl 2023), and readers are referred to Anil et al. (2023) for more model details. For VLMs, we report the results on CLIP ViT-B/16 (Radford et al.
2021
5.2 MAIN EXPERIMENTS
Experiments on Natural Language Tasks. We present the results across a diverse set of NLP tasks in Table 2 Notably, BC consistently outperforms ICL, yielding significant performance enhancement of 8% and 6% on PaLM 2-S and PaLM 2-L, respectively. This shows that the BC implementation successfully mitigates the contextual bias from the in-context examples and unleashes the full potential LLM in efficient learning and quick adaptation to new tasks. In addition, BC improves over the oftte-of-the-at PC baseline by 6% on PaLM 2-S, and surpasses the competitive CC baseline by another 3% on average on PaL.M 2-L. Specifically, BC a generalizable and cheaper technique across all evaluated tasks, delivering stable performance improvement, whereas previous baselines exhibit varying degrees instability across tasks: DC baseline is the least competitive; CC displays more failure cases in multi-sentence classification tasks, particularly for paraphrasing and NLI tasks, as we hypothesized in Sec 3.3 PC, while occasionally competitive, exhibits large performance fluctuations, evidenced by its large standard deviation, resulting in frequent substantial performance degradation.
We further analyze the performance of BC by varying the ICL shots from 0 to 4 shots as shown in Fig. 6 and BC again outperforms all baseline methods. We also observe an overall trend for improved performance when more shots are available, and the per- formance disparities between BC and ICL converge some tasks, which suggests that BC allows LLMS to more effectively take advantage of more in-context demonstrations. We also observe that PC exhibits the worst sta- bility across the number of shots. In Table we show that further slight gains, 1% average, be achieved by involving an adjustable strength parameter that refines the calibration and mitigates estimation errors. This alternative design not only makes BC applicable both zero-shot and few-
+7%
Accuracy (%)
Z5
40
+17%
20
shot setups but also shows its capability to improve further from limited labeled data.
SVHN
EuroSAT
CLEVR
Task
Calibrating Vision-Language Models We fur- ther extend the applicable scenarios of BC to multi-modal learning to handle the bias inherent in the prompt template designs in CLIP. We select three tasks in which the previous visual-promptinn method shows significant improvement (Oh et al.) 2023). As shown in Fig. 7 BC significantly im-
Figure 7: BC improves zero-shot (ZS) image clas- sification: Accuracy (%) on image classification tasks with the zero-shot CLIP ViT-16/B. The BC implementation zero-shot, and we apply BC together with the CLIP to demonstrate the effec- tiveness of BC vasion-language models. Refer to additional tasks in Appendix S
8

---

Published as a conference paper at ICLR 2024
Choice
Order
SST-2
RTE
Accuracy (%6
84
Accuracy (%)
80
86
90
70
00
80
60
ICL
BC
ICL
BC
70
Template
Verbalizer
10a
10"
10a
10
10a
10a
Batch Size
Batch Size
Accuracy (%)
85
80
Figure 9: BC is data-efficient and insensitive to the batch size: Performance of BC across differ- ent sizes of an initial unlabeled set without using a running estimate of the contextual bias. We compare BC with the state-of-the PC baseline that also leverages unlabeled estimate set, and experiments conducted PaLM 2-S.
70
ICL
BC
ICL
BC
Method
Method
Figure 8: BC makes prompt engineering easier: Performance of BC with respect to ICL choices, ICL orders, prompt templates, and verbalizers.
proves the zero-shot baseline by 12% on average. This observation further highlights the presence of contextual bias even within linion-language models, and BC can successfully restore the performance of VLM in image classification tasks, suggesting that BC may serve as versatile and common technique for mitigating contextual biases across multiple modalities.
5.3 ROBUSTNESS AND ABLATION STUDIES
Robustness. We analyze the robustness of BC with respect common prompt engineering design choices that were previously shown to significantly affect LLM performance (Lu et al. 2022 Liu 2022a choices and orders of in-context examples, the prompt template for ICL, and the verbalizers, as shown in Fig. 8 evaluated on RTE. Setup details are listed in Appendix $ First, we find that BC is more robust ICI. choices and can mostly achieve the same performance with different ICI. examples. Additionally, given single set of ICL shots, altering the order between each ICI. example has minimal impact on the BC performance. However, optimal order selection can still lead to promising ICL performance. Furthermore, robustness of BC under 10 designs of prompt templates, where BC shows consistent improvement over the ICL baseline. Therefore, though BC makes further improvements, well- well-designed template can further enhance the performance of BC. Lastly, we examine the robustness of BC to variations in verbalizer designs. Remarkably, even when employing unconventional choices such emoji pairs as the verbalizers (listed in Tables TO) leading to dramatic oscillations of ICL performance, BC largely recovers performance. This observation shows BC robustifies LLM predictions under common prompt design choices and makes prompt engincering casier.
worth noting that an analyze the
We study the impact of batch size on the performance of BC as shown in Fig. In contrast to PC, which also leverages an unlabeled estimate set, BC is remarkably more sample efficient, achieving a strong performance with only around 10 unlabeled samples, whereas PC requires more than 500 unlabeled samples before its performance stabilizes.
Batch Size.
CONCLUSION
We first revisit previous calibration methods while addressing two critical research questions from an interpretation of decision boundaries, revealing their failure cases and deficiencies. We then propose Batch Calibration, zero-shot and inference-only calibration technique. We also introduce an adjustable extension, BCL, which offers more refined calibrations when labeled data accessible. While methodologically simple and easy to implement with negligible computation cost, we show that BC scales from a language-only setup to the vision-language context, achieving ofate-of-the-ar art performance in both modalities. prompt designs, and we expect easy prompt engineering with BC while exploring the potential of BC to generative tasks in the future.
BC significantly improves the robustness of LLMs with respect to

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGEMENT
We thank Emily Salkey for her sincere project management support. We also thank Mohammad Havaei, Chirag Nagpal, Stephen Pfohl, Alexander D' Amour, and Ahmad Beirami for fruitful sugges- tions and feedbacks and the PaLM 2 team at Google for helping with infrastructure questions.
REFERENCES
Lean-Baptisel Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob L.. Menick, Sebastian Borgeaud, Andy Brock, Aida Nematzadch, Sahand Sharifzadeh, Mikolaj Binkowski, Ricardo Barreira, Oriol Vinyals, Andrew Zisserman, and Karén Simonyan. Flamingo: a visual language model for few-shot learning. In NeurlPS, 2022.
Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, al. Palm 2 technical report. arXiv preprint arXiv:2305. 104003 2023.
Tom B. Brown, Benjamin Mann. Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Vosse, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter. Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In Advances in Neural Information Processing Systems 33; Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311, 2022.
Mírcea Cimpoi, Subhransu Maji, lasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describ- ing textures in the wild. In 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014, Columbus, OH, USA, June 23-28, 2014, pp. 3606-3613, 2014.
Christopher Clark, Kenton Lee, Ming- Ming-Wei Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. BoolQ: Exploring the surprising difficulty of natural yes/no questions. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pp. 2924-2936, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
Ganqu Cui, Wentao Li, Ning Ding, Longtao Huang, Zhiyuan Liu, and Maosong Sun. Decoder tuning: Efficient language understanding decoding. In Proceedings of the 61st Annual Meeting the Association for Computational Linguistics ( (Volume I: Long Papers), pp. 15072-15087, Toronto, Canada, July 2023. Association for Computational Linguisties.
Mathie-Ctheriiee De Marneffe, Mandy Simons, and Judith Tonhauser. The commitmentbank: In- vestigating projection in naturally occurring discourse. In proceedings of Sinn und Bedeutung. volume 23, pp. 107-124, 2019.
Mingkai Deng, Jianyu Wang. Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric Xing, and Zhiting Hu. RLPrompt: Optimizing discrete text prompts with reinforcement learning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 3369-3391, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics.
Shizhe Diao, Zhichao Huang. Ruijia Xu, Xuechun Li, LIN Yong, Xiao Zhou and Tong Zhang. Black-box prompt learning for pre-trained language models. Transactions on Machine Learning Research, 2023. ISSN 2835-8856.
10

---

Published as a conference paper at ICLR 2024
William B. Dolan and Chris Brockett. Automatically constructing a corpus of sentential paraphrases. In Proceedings of the Third International Workshop on Paraphrasing (IWP2005), 2005.
Yu Fei, Yifan Hou, Zeming Chen, and Antoine Bosselut. Mitigating label biases for in-context learning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 14014-14031, Toronto, Canada, July 2023, Association for Computational Linguisties.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, pp. 1321-1330, 2017.
Zhixiong Han, Yaru Hao, Li Dong, Yutao Sun, and Furu Wei. Prototypical calibration for few-shot learning of language models. The Fleventh International Conference on Learning Representa- tions, 2023.
Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg- Kirkpatrick, and Graham Neubig. Towards a unified view of parameter-effiient transfer learning. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. Open 2022.
Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):2217-2222) 2019.
Ari Holtzman, Peter West, Vered Shwartz, Yejin Choi, and Luke Zettlemoyer. Surface form competi- tion: Why the highest probability answer isn't always right. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 7038-705 Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski. Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for NLP, In Proceedings of the 36th International Conference on Machine Learning. ICML 2019, 9-15 June 2019, Long Beach, California, USA, pp. 2790-2799, 2019.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In The Tenth Interna- tional Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022.
Xuefeng Hu, Gokhan Uzunbas, Sirius Chen, Rui Wang, Ashish Shah, Ram Nevatia, and Ser-Nam Lim. Mixnorm: Test-time adaptation through online normalization estimation. arXiv preprint arXiv:2110.11478, 2021.
Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C. Lawrence Zitnick, and Ross B. Girshick. CLEVR: A diagnostic dataset for compositional language and elementary visual reasoning. 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017,
Honolulu. HI. USA, July 21-26, 2017, pp. 1988-1997, 2017.
Ansh Khurana, Sujoy Paul, Piyush Rai, Soma Biswas, and Gaurav Aggarwal. Sita: Single image test-time adaptation. arXiv preprint arXiv:2112.02355, 2021.
Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efffiient prompt tuning. In Proceedings of the 202/ Conference on Empirical Methods in Natural Language Processing, pp. 3045-3059, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics.
Chengzu Li, Han Zhou, Goran Glavaš, Anna Korhonen, and Ivan Vulić. On task performance and model calibration with supervised and self-ensembled in-context learning. arXiv preprint arXiv:23 13772, 2023.
Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 1thh International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 4582 4597, Online, August 2021. Association for Computational Linguistics.

---

Published as a conference paper at ICLR 2024
Hyesu Lim, Byeonggeun Kim, Jaegul Choo, and Sungha Choi. TTN: A domain-shiff aware batch normalization in test-time adaptation. In The Eleventh International Conference on Learning Representations. ICLR 2023, Kigali, Rwanda, May 1-5, 2023, 2023.
Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. What makes good in-context examples for GPT-3? In Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, pp. 100-114, Dublin, Ireland and Online, May 2022a, Association for Computational Linguistics.
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1-35, 2023.
Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 61-68, Dublin, Ireland, May 2022b. Association for Computational Linguistics.
Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. In PnnPro Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 9086-8098, Dublin, Ireland, May 2022. Association for Computational Linguistics.
Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine-grained visual classification of aircraft. arXiv preprint 1306.5533 2013.
Sewon Min, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Noisy channel language model prompting for few-shot text classification. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 5316-5330, Dublin, Ireland, May 2022a. Association for Computational Linguistics.
Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Rethinking the role of demonstrations: What makes in-context learning work? In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 11048-1 1004, Abu Dhabi, United Arab Emirates, December 2022b. Association for Computational Linguisties.
Muhammad Jehanzeb Mirza, Jakub Micorek, Horst Possegger, and Horst Bischof. The norm must go on: Dynamic unsupervised domain adaptation by normalization. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pp. 14745-14755, 2022.
Todd K Moon. The expectation-maxxiizztio algorithm. IEEE Signal processing magazine, 13(6): 47-60, 1996.
Zachary Nado, Shreyas Padhy, D Sculley, Alexander D' Amour, Balaji Lakshminarayanan, and Jasper Snoek. Evaluating prediction-time batch normalization for robustness under covariate shift. arXiv preprint ar arXiv:2006. 2020.
Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. Adversarial NLI: A new benchmark for natural language understanding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4885-4901, Online, July 2020.
Association for Computational Linguistics.
Changdae Oh, Hyeji Hwang, Hee Young Lee, YongTaek Lim, Geunyoung Jung, Jiyoung Jung. Hosik Choi, and Kyungwoo Song. Blackvip: Black-box visual prompting for robust transfer learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pp. 24224-24235, 2023.
OpenAl. Gpt-4 technical report, 2023.
12

---

Published as a conference paper at ICLR 2024
Jane Pan, Tianyu Gao, Howard Chen, and Danqi Chen. What in-context learning "learss in-context: Disentangling task recognition and task learning. In Findings of the Association for Computational Linguistics: 2023. pp.
Toronto, Canada, July 2023. Association for Computational
Linguisties.
Pouya Pezeshkpour and Estevam Hruschka. Large language models sensitivity to the order of options in muultipeechoiie questions. arXiv preprint arXiv:2308.. 1483, 2023.
Mohammad Taher Pilchvar and Jose Sumacho-Colladdee WiC: the word-in -nd-in-context dataset for evaluating context-sensitive meaning representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Papers), pp- 1267-1273, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
John Platt al. Probabilistic outputs for support vector machines and comparisons regularized likelihood methods. Advances in large margin classifiers, 10(3):61-74, 1999.
Archiki Prasad, Peter Hase, Xiang Zhou, and Mohit Bansal. GrIPS: Gradient- free, edit-based instruction search for prompting large language models. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pp. -3644, Dubrovnik, Croatia, May 2023. Association for Computational Linguistics.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 8748-8763, 2021.
July 2021, Virtual Event, pp.
Yasaman Razeghi, Robert I. Logan IV, Matt Gardner, and Sameer Singh. Impact of pretraining term frequencies on few-shot numerical reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 840-854, Abu Dhabi, United Arab Emirates, December 2022.
Association for Computational Linguistics.
Melissa Roemmele, Cosmin Adrian Bejan, and Andrew S. Gordon. Choice of plausible alternatives: An evaluation of commonsense causal reasoning. In Logical Formalizations of Commonsense Reasoning, Papers from the 2011 AAAI Spring Symposium, Technical Report SS-11-06. Stanford, California, USA, March 2 21-23, 2011, 2011.
Ohad Rubin, Jonathan Herzig, and Jonathan Berant. Learning retrieve prompts for in-context learning. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, pp. 2675-2671, Seattle, United States, July 2022. Association for Computational Linguistics.
Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adaptation. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020.
Seongjin Shin, Sang-Woo Lee, Hwijeen Ahn, Sungdong Kim, HyoungSeok Kim, Boseop Kim, Kyunghyun Cho, Gichang Lee, Woomyoung Park, Jung-War Woo Ha, and Nako Sung. On the effect of pretraining corpora on in-context learning by large-scale language model. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 5168-5186 Seattle, United States, July 2022. Association for Computational Linguistics.
Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts. In Proceed- ings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 4222 4235, Online, November 2020. Association for Computational Linguistics.
Chenglei Si, Dan Friedman, Nitish Joshi, Shi Feng, Danqi Chen, and He He. Measuring inductive biases of in-context learning with underspecified demonstrations. In Proceedings of the 61 Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 11289-11310, Toronto, Canada, July 2023. Association for Computational Linguistics.
13

---

Published as a conference paper at ICLR 2024
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1631-1642, Seattle, Washington, USA, October 2013. Association for Computational Linguistics.
Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: dataset 101 human actions classes from videos in the wild. arXiv preprini arXiv: 1212.0004 2012.
Taylor Sorensen, Joshua Robinson, Christopher Rytting, Alexander Shaw. Kyle Rogers, Alexia Delorey, Mahmoud Khalil, Nancy Fulda, and David Wingate. An cermation-theoreti. approach to prompt engineering without ground truth labels. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 819-866 862, Dublin, Ireland, May 2022. Association for Computational Linguistics.
Xingchen Wan, Ruoxi Sun, Hanjun Dai, Sercan Arik, and Tomas Pfister. Better zero-shot reasoning with self-adaptive prompting. In Findings of the Association for Computational Linguistics: ACL 2023, pp. 3493-3514, Toronto, Canada, July 2023a. Association for Computational Linguistics.
Xingchen Wan. Ruoxi Sun, Hootan Nakhost, Hanjun Dai, Julian Eisenschlos, Sercan Arik, and Tomas Pfister. Universal self-adaptive prompting. In Proceedings of the 2023 Conference Empirical Methods in Natural Language Processing, pp. 7437-7462, Singapore, December 2023b. Association for Computational Linguistics.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pp.
Brussels, Belgium, November 2018. Association for Computational Linguistics.
Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. Superglue: A stickier benchmark for cenerol-purpose language understanding systems. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 3261-3275, 2019.
Dequan Wang. Evan Shelhamer, Shaoteng Liu, Bruno A. Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021.
Peiyi Wang, Lei Li, Liang Chen, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. Large language models are not fair evaluators. arXiv preprint arXiv:2305. 17926. 2023.
Jerry Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang. Denny Zhou, et al. Larger language models do in-context learning differently, arXiv preprint arXiv:2303.03846 2023.
Adina Williams, Nikita Nangia, and Samuel Bowman. A croad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technolo- gies, Volume 1 (Long Papers). pp. 1112-1122, New Orleans, Louisiana, June 2018. Association for Computational Linguisties.
Sang Michael Xie, Aditi Raghunathan. Percy Liang, and Tengyu Ma. An explanation of in-context learning as implicit bayesian inference. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022, 2022.
Benfeng Xu, Quan Wang, Zhendong Mao, Yajuan Lyu, Qiaoqiao She, and Yongdong Zhang. $kšnn prompting: Beyond-context learning with calibration free nearest neighbor inference. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023, 2023.
Seonghyeon Ye, Doyoung Kim, Joel Jang, Joongbo Shin, and Minjoon Seo. Guess the instruction! flipped learning makes language models stronger zero-shot learners. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023, 2023.
14

---

Published as a conference paper at ICLR 2024
Fan Yin, Jesse Vig, Philippe Laban, Shafiq Joty, Caiming Xiong, and Chien- Wu. Did you read the instruction?? rethinking the effectiveness of task definitions in instruction learning. In Proceedings of the 6lst Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 3063-3079, Toronto, Canada, July 2023. Association for Computational Linguisties.
Fuming You, Jingjing Li, and Zhou Zhao. Test-time batch statistics calibration for covariate shift. arXiv preprint arXiv:2110.04000 2021.
Netzer Yuval. Reading digits in natural images with unsupervised feature learning. In Proceedings of the NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.
Tianjun Zhang, Xuezhi Wang, Denny Zhou, Dale Schuurmans, and Joseph E. Gonzalez. TEMPERA: Test-time prompt editing via reinforcement learning. In The Eleventh International Conference Learning Representations, 2023.
Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. Calibrate before use: Improving fho-shot performance of language models. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, pp. 12697-12706, 2021.
Chujie Zheng, Hao Zhou, Fandong Meng, Jie Zhou, and Minlic Huang. On large language models' selection bias in multi-choice questions. arXiv preprint arXiv:2309.0386 038882 2023.
Han Zhou, Xingchen Wan, Ivan Vulić, and Anna Korhonen. Survival of the most influential prompts: Efficient black-box prompt search via clustering and pruning. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 13064-13077, Singapore, December 2023, Association for Computational Linguistics.
Han Zhou, Xingchen Wan, Ivan Vulić, and Anna Korhonen. Autopeft: Automatic configuration search for parameter-efficent fine-tuning. Transactions of the Association for Computational Linguistics, 2024.
Yuliang Zou, Zizhao Zhang, Chun-Liang Li, Han Zhang, Tomas Pfister, and Jia- Bin Huang. Learning instance-specifie adaptation for cross-domain segmentation. In Computer Vision ECCV 2022 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXXIII, pp. 459.476, 2022.
A ADDITIONAL RELATED WORK
Prompt Learning. Prompt learning is an efficient learning pipeline for LLM as an alternative to traditional full-model fine-tuning (Liu et al. 2023). Soft prompting (Li & Liang 2021 Lester et al. 2021; Liu et al., (0226) enables fast adaptation of LLM by appending learnable continuous prompts the embedding space while freezing the rest of weights. The recent development of parameter-efficient fine-tuning methods (Houlsby et al. 2019 Hu et al., 2022 Zhou et al.) 2024), which learn additional modules, may also be interpreted as a form of soft prompting (He et al. 2022).
However, these soft prompt learning methods inevitably require gradients and internal model access. On the other hand, hard prompting (Shin et al., (020)) is an appealing learning category for learning discrete prompts. Recent efforts have been devoted to black-box prompt search without accessing model gradients, and more interpretable prompts can be found by reinforcement learning (Deng et al.) 2022 Zhang et al. 2023), gradient estimation (Diao et al., 2023), and other derivative-free search algorithms Prasad et al. 2023 Zhou et al. 2023).
Test-Time Adaptation. Test-time adaptation aims to mitigate the covariate shift using the test-time statistics. Wang et al.) propose to minimize the entropy by updating the affine parameters in the BN layer. Nado et al. (2020) and Schneider et al. (2020) introduce using test-time batch statistics for the standardization in BN and mixing it with source statistics to conquer covariate shift, respectively. Similarly, mixing the statistics with predefined hyperparameters (You et al.] 2021; Khurana et al.) 202I), interpolating source and target-domain statistics (Lim et al. 2023), or using running average estimate (Mirza et al. 2022 Hu et al.) 202T) have also been proposed to adapt the BN layer. Zou let al. (2022) introduce strength parameters in adapting the standardization statisties in semantic
15

---

Published as a conference paper at ICLR 2024
segmentation tasks. We differentiate from test-time BN approaches by mitigating the bias in the novel context of LLM, and there is no source statistic similar to a BN layer in computer vision backbones.
B LIMITATIONS AND FUTURE WORK
BC is test-time method that relies the target statistics from batched input. To mitigate any potential estimation errors from unlahelled data, we introduce the adjustable BCL extension to incorporate source statistics from labelled data. Though BC has shown remarkable sample efficiency in terms of batch sizes, it still requires a batch of inputs to estimate the contextual bias. We introduce running estimation for BC from mini-batches, which subsequently stabilizes the prediction LLMs when more mini-batches arrive.
In future work, we will endeavor to explore calibration for generative tasks while extending BC to more models across modalities. We suspect that the contextual bias may also exist in short-form generation tasks. Motivated by /Zhao et (202T), one possible solution for generative tasks is to calibrate over the logits of the first output token since the following tokens are likely deterministic based on the first token. Overall, BC zero-shot, inference-only, and incurs negligible additional costs. We expect easy prompt engineering with BC for users building towards their own robust and responsible LLM applications.
C DATASET STATISTICS
Table 4: Details of the dataset used for evaluation in the Table 2 samples, where we consistently use the validation split as the test split because labels are not publicly available for some datasets.
Test denotes the number of test
Dataset
Objective
#sentences
#classes
Test
SST-2
Sentence Classification
1
2
872
MNLI
NLI
2
3
9815
QNLI
Question-Answrrte NLI
2
2
5463
MRPC
Paraphrasing
2
2
408
QQP
Paraphrasing
2
2
40430
BoolQ
Commonsense Reasoning
2
2
3270
CB
NLI
2
3
56
COPA
Commonsense Reasoning
3
2
100
RTE
NLI
2
2
277
WIC
Context Comprehension
3
2
638
ANLI-RI
NLI
2
3
1000
ANLI-R2
NLI
2
3
1000
ANLI-R3
NLI
2
3
1200
D IMPLEMENTATION DETAILS
Contextual Calibration (Zhao al. 2021) (CC). follow the original implementation of CC and take the mean of the log-probabilityy over three content-free tokens the test sample in the predefined template: 'N/A', [[MASK]' It incurs 3 additional API costs from LLMs.
Comain-Conteex Calibration FFi et al. 2023 (DC). We reproduce the DC baseline by using
the same test set as the unlabeled text set to construct its bag-of-words. We then randomly sample tokens for average length to form the content-free and in-domain input from the bag-of-words This process is then repeated randomly for 20 times, and we take the mean of the log- probbbibyyyy following the original implementation. It ineurs 20 additional API costs from LLMss.
16

---

Published as a conference paper at ICLR 2024
Prototypical Calibration (Han al. 2023 (PC). For a fair comparison, as the unlabeled estimate set for PC. We follow the same hyper-parameter reported by PC with 100 maximum iterations for EM and 100 times random initialization for the whole learning process to stabilize its estimation. is noteworthy that this number of repetitions is costly and relatively slow, especially when the [Test] is large.
the same test set
Batch Calibration (BC). In all reported experiments, we compute the correction propbbbbbillity term pyy C*) once after all test samples are seen. the n-shot ICL. experiments reported in Table 2 and Fig. 6 the k-shot ICI. is concatenating k random training sample per class. In the BCL experiment that uses labeled samples, we use 128 randomly selected training samples as the I randomly sampled example the context to study the
labeled data. In the robustness study, performance of BC with respect to the ICL choices. We then conduct the ICL order experiment by re-ordering randomly sampled ICL examples. The rest experiments conducted the standard 1-shot ICL setup.
E ADDITIONAL EXPERIMENTS
Table 5: Accuracy (%) on natural language classification tasks with O-shot PaLM 2-S and I-shot
PaLM 2-M models in single seed.
Model
PaLM
PaLM 2-M
O-shot
1-shot
Method
ICL
CC
DC
PC
BC
ICL
CC
DC
PC
BC
SST-2
94.61
94.50
94.61
87.84
95.18
94.95
95.87
94.95
96.22
96.10
MNLI
45.87
52.54
42.50
38.04
53.67
45.50
54.43
56.26
43.81
60.02
QNLI
49.28
48.97
49.44
50.28
49.55
78.88
75.56
62.95
77.39
78.91
MRPC
69.12
61.76
69.85
69.85
64.95
57.11
73.53
68.87
69.85
65.93
QQP
60.23
51.16
49.12
48.98
56.20
66.18
79.67
74.32
70.27
75.13
BoolQ
86.51
86.97
76.88
55.41
84.04
87.37
88.53
87.28
88.78
87.31
85.71
58.93
55.36
46.43
67.86
71.43
69.64
67.86
50.00
80.36
CОРA
88.00
66.00
90.00
52.00
88.00
97.00
96.00
96.00
97.00
96.00
RTE
62.45
67.15
58.12
68.23
71.84
77.62
79.06
68.23
77.98
80.51
WiC
58.31
51.88
52.82
49.22
58.30
61.13
64.11
52.04
65.52
68.03
ANLI-R1
39.80
44.70
43.00
37.00
50.00
52.40
52.40
52.70
35.70
54.00
ANLI-R2
36,80
41.50
40.70
40.20
45.10
46.00
50.70
47.80
35.80
50.00
ANLI-R3
42.67
46,42
43.08
35.50
48.50
43.50
45.67
49.33
32,42
50.50
Avg.
63.03
59.42
58.88
52.23
64.09
67.62
71.17
67.58
64.67
72.52
Table 6: Accuracy (%) on image classification tasks with the zero-shot CLIP ViT-16/B. We addition-
ally report on UCF101 (Soomro al. 2012). FGVC Aircraft (Maji et al., 2013), and DTD (Cimpoi
et al. 2014).
Method
SVHN
EuroSAT
UCF
CLEVR
Aireraft
DTD
Avg.
ZS
18.0
47.8
66.7
14.7
24.8
44.4
36.7
ZS+BC
35.0
54.7
66.0
29,2
22.3
41.7
41.5
17

---

Published as a conference paper at ICLR 2024
CC
DC
PC
BC
RTE: Class 1
2
2
2
-4-
-
0
-6-
-6-
-6
-2
-5.0
-2.5
0.0
-5.0
-2.5
0.0
-5.0
-2.5
0.0
-2.5
2.5
-4
2
MRPC: Class 1
-5
-6
6
6
-7
-7
-7
-1
-8-
8-8
T-2
8
-6
-4
8
-6
-4
-8
-6
-4
-2
2
QOP: Class 1
5
-5
-5-
10
10
-10
-5
-15
15
15
-10
-5
0
10
-5
0
10
-5
-5
BoolQ: Class 1
-5
-5-
-5
-10
-10
-10
5
X
X
X
-15
15
15
-15
-10
-5
0
-15
-10
-5
0
15
-10
-5
0
-8
COPA: Class 1
2
-10
-10-
-10
-12
-12
-12
-2
-14
14
14
-10
-12
-10
-12
-10
0
0
WIC: Class 1
2
-2
2
-2
0
-4-
4
-4-
-2
-6
6
6
-2.5
2.5
Class 0
Class 0
Class
Class 0
Figure 10: Visualization of the decision boundaries of uncalibrated ICL, and after applying existing calibration methods and the proposed BC. We list all binary classification tasks from the evaluation set.
18

---

Published as a conference paper at ICLR 2024
Table Performance of BC with respect to prompt templates and verbalizers per example Fig. We report accuracy (%) the RTE task 1-shot PaLM 2-S. We refer each example ID to the index of templates and verbalizers listed in Tables & I0
ID
2
4
5
6
7
8
10
ICL
81.2
80.9
82.3
76.9
83.0
83.4
83.4
82.7
79.1
84.5
Template
BC
84.1
80.9
82.3
85.6
84.1
81.2
88.1
82.3
83.0
86.6
ICL
81.2
76.9
74.0
74.0
67.5
69.0
71.5
68.6
65.3
66.4
Verbalizer
BC
84.1
83.4
83.4
80.1
78.0
70.0
82.7
82.7
83.8
83.8
82
SST-2
80
78
Labeled Set
76
40
2
-2
90
88
MRPC
Boolo
86
50
82
5
-4
-3
2
1
2
-5
4
-3
-2
1
85
RTE
75
WIC
50
2
0.0
0.5 Strength
1.0
1.5
Strength
Figure 11: The performance of an adaptable batch calibration layer (BCL) compared to the zero-shot BC with a changing strength. The st rength 7 at 0 and 1 represent the uncalibrated ICL. and BC, respectively. We highlight the optimal strength learned from a labeled set by a red vertical line and the best test strength by green line.
19

---

Published as a conference paper at ICLR 2024
F PROMPT TEMPLATES
Table Prompt templates for all k-shot ICL experiments. We follow the template styles from Han al. (2023) and Brown et al. (2020).
Dataset
Template
Label Set
SST-2
Review:
{sentence)
negative positive
Sentiment: {label}
MNLI
Premise: {premise}
yes / maybe / no
CB
Hypothesis: {hypothesis}
ANLI
Answer: {label}
QNLI
Question: {question] }
yes / no
Sentence: {sentence)
Label: {label}
MRPC
Sentence 1: (sentencel)
no/yes
Sentence 2: [sentence2)
Equivalence: [label}
QQP
Question 1: [questionI] }
/ yes
Question 2: {question2]
Duplicate: {label}
BoolQ
{passage} Question: {question]
/ yes
Answer: flabel}
CОРA
Premise: {premise}
1/2
Choicel: {choice 1
Choice2: (choice2
Answer: {label}
RTE
Premise: {sentencel] }
yes / no
Hypothesis: (sentence2)
Answer: {label}
WiC
Sentencel: (sentencel]
false / true
Sentence2: [sentence2]
Word: [word)
Answer: flabel}
20

---

Published as a conference paper at ICLR 2024
Table 9: Prompt templates for the robustness experiment conducted on RTE in Fig. 8
ID
Template
Label Set
1
Premise: {sentencel]
Hypothesis: (sentence2)
Answer: {label}
2
{sentence1}
Hypothesis: (sentence2)
Answer: [label)
3
[sentencel]
Question: {sentence2]
Answer: (label)
4
{sentencel}
Question: {sentence2)
{label}
5
{sentencel}
Question: {sentence2]
yes or no? Answer: {label}
6
Sentence 1: {sentencel]
yes / no
Sentence 2: [sentence2]
Answer: (label)
7
Premise: {sentence1}
Hypothesis: (sentence2)
Label: {label}
8
Sentence 1: [sentencel]
Sentence 2: [settence2]
Label: {label}
9
Determine if the sentence 2 is true based on the Sentence below
Sentence 1: {sentencel]
Sentence 2: {sentence2]
Answer: [label}
10
Determine if the sentence 2 is true or false based on the Sentence below Sentence 1: {sentencel] }
Sentence 2: {sentence2]
Answer: {label}
Table 10: Verbalizer choices for the robustness experiment conducted on RTE in Fig. where we include emoji pairs for ID 8, 9, 10.
ID
Label Set
Template
1
yes / no true false
2
3
correct incorrect
4
positive negative
Premise: (sentencel) Hypothesis: [sentence2]
5
good/ / bad
6
great/ / terrible
Answer: (label}
7
it was true / it was false
8
:thumbs_up :thumbs_down
9
:man gesturing ok :man gesturrig no :check mark / :cross mark
10
21

---

Published as a conference paper at ICLR 2024
Table 11: Prompt templates for the O-shot experiments.
Dataset
Template
Label Set
SST-2
Review:
{sentence)
negative / positive
Sentiment: {label}
MNLI
{premise}
yes maybe / no
CB
Question: (hypothesis} yes, or maybe?
ANLI
Answer: {label}
QNLI
{question]
yes no
Question: (sentence) yes no?
Answer: {label}
MRPC
Sentence [sentencel
no / yes
Sentence 2: (sentence2)
Equivalence: [label}
QQP
Question {question1]
no/ yes
Question 2: {question2]
Duplicate: {label}
BoolQ
{passage} Question: {question]
no / yes
Answer: {label}
CОРА
Premise: {premise}
1/2
Choicel: {choicel
Choice2: [choice2)
Answer: {label)
RTE
(sentencel]
yes / no
Question: [sentence2] yes or no?
Answer: {label}
WiC
{sentencel]
no /
[sentence2]
Question: Is the word '(word)' used in the same way in the two sentences above?
Answer: {label)
22


Reviews:
[{"review_id": "f3SoPXAs9a", "reviewer": "Reviewer_r8cW", "summary": "This paper discusses the issue of bias and unexpected performance degradation in large language models (LLMs) when using prompting and in-context learning (ICL). To address this, the authors first provide a comprehensive analysis of existing calibration methods and their decision boundaries, and then propose a new calibration method called Batch Calibration (BC) with linear decision boundaries, which mitigates the bias from the batched input and is both zero-shot and inference-only. BC can be easily extended to learn the bias from labeled data, and applied to calibrate vision-language models. The authors conduct extensive experiments on over 10 natural language understanding and image classification tasks and show that BC achieves state-of-the-art results.", "questions": "1. The authors argued that linear decision boundaries produced by calibration methods can be more robust and generalizable across tasks. This argument is not well supported theoretically and empirically. \n2. The advantage of Batch Calibration over existing methods is not thoroughly justified. Could you provide more intuitive descriptions and theoretical analysis for it?\n3. The derivations of Table 1 could be provided in more details in the main contents or supplementary. Currently, it is not easy to understand.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The proposed Batch Calibration method is simple and empirically effective.\n2. Extensive experiments are conducted.", "weaknesses": "1. The proposed method is not thoroughly justified.\n2. The writings could be improved in terms of the analysis of existing methods and the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lHHu7akuQb", "reviewer": "Reviewer_99FV", "summary": "The paper delves into the current challenges faced in adapting Large Language Models (LLMs) to new tasks through the method of human-designed instructions. Although these models possess a commendable ability for in-context learning (ICL) and can efficiently adapt from few-shot input-label pairs, they are significantly influenced by the choice of templates, verbalizers, and demonstrations. This results in biases that can act as barriers to creating adaptable and robust LLM applications. While several studies have tried to address these biases, a holistic analysis differentiating the merits and demerits of each approach is lacking.", "questions": "In fig.5, why does a strength greater than 1 consistently decrease accuracy?\n\nHow significant is the bias/miscalibration of the ICL performance? that say is the problem still relevant for models with greater scales? \nShould consider 65B llama2, GPT-3.5, GPT-4 etc as the method does not require tuning", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "The paper provides a thorough and systematic examination of existing calibration methods for LLMs, filling a gap in the existing research landscape. Methodologically, the introduction of Batch Calibration (BC) offers a zero-shot, inference-only calibration method that is computationally efficient, addressing a primary concern in the domain. BC proves effective in reducing prompt sensitivity, a prominent issue in LLMs, thereby facilitating easier prompt engineering.\n\nThe scope is also broadened compared to previous works like CB, which only studies GPT-2's biases.", "weaknesses": "While BC introduces minimal computational overhead, in highly resource-constrained environments, even small overheads might be significant.\n\nThe term calibration can be confusing, maybe using bias is better. At least should add a footnote to the Introduction. Another widely used notion of calibration is from the perspective of uncertainty, explainability, and reliability. \n\nPlease include model sizes and details in the paper, as the performance of ICL and prompting is quite irreproducible and context-dependent.\n\nPlease adjust the margins of subfigures in fig. 2, it's overlapped. \n\nThe liteature review of Test time tuning is a bit unnecessary as ICL itself is not a tuning method", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lZPUTAOG65", "reviewer": "Reviewer_jGRr", "summary": "This paper studies model calibration in the context of LLMs (large language models). The authors first analysed the recently proposed methods with empirical results, and they also discussed the two important design principles behind those ICL calibration methods. A novel calibration method (which is termed batch calibration) is then introduced for zero-shot learning; an extended version that has a hyperparameter is proposed for fine-tuning. The authors conducted extensive experiments on multiple NLP tasks and showed superior performance as compared to the existing ICL calibration methods.", "questions": "1. In e.q.(3), how to obtain the contextual prior $\\hat{\\mathbf{p}}$? \n2. How would BC work on the task other than classification? \n3. It seems the calibration perform is not stable across different model architectures in 1-shot setting on some tasks. E.g., MNLI, 75.12/         60.02/81.34 for PaLM-2-S/PaLM-2-M/PaLM-2-L. I understand that PaLM-2-S/L are based on 5 runs and PaLM-2-M result is from a single run. Could you help to understand, what causes the high perform variance, I may miss something here.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. This paper addresses an important and interesting topic: calibrating LLMs in zero or few shot settings.\n\n2. The paper gives an overview of the most relevant and recent ICL calibration methods and discusses their motivations and design principles with empirical results. \n\n3. The experiments are extensive, and the proposed method achieves better performance on most tasks.", "weaknesses": "1. My main concern goes to the $\\textit{strength}$ parameter in BCL; the current results show $\\gamma = 1$ seems to give strong performance (though not optimal) across tasks on CB and SST-2. Does this generalize to other tasks as well? It would be good to provide the optimal $\\gamma$ for each task; maybe include it in Table 2? Besides, would it be more reasonable to sample \\gamma in [0, 5]?\n\n2. There are some statements that are not well presented or supported. In Sec. 4.3, the authors claimed BC retains the performance even when using emoji pairs as verbalizers. Is there an example or result related to this specific experiment?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "t0t8a8Wu4O", "reviewer": "Reviewer_tiQs", "summary": "This paper presents a study on calibration methods for in-context learning. The authors provide a unified and systematic analysis of existing calibration, focusing on their decision boundaries. They also investigate the common use of content-free tokens in calibration. The paper highlights the biases in language models’ predictions and proposes Batch Calibration (BC) as a zero-shot and inference-only calibration method. BC aims to accurately model the contextual bias in prompt contexts by marginalizing the language model scores. The authors extend BC to black-box few-shot learning by introducing a learnable parameter to adapt to available labeled data. The performance of BC is evaluated and compared to baseline methods, showing improved performance in various tasks.", "questions": "See weakness", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "This paper is technically sound and well-written (although some writing issues are listed in weaknesses). The authors' revisiting of previous methods is novel to the research community.", "weaknesses": "- For writing:\n    - \"Survival of the Most Influential Prompts: Efficient Black-Box Prompt Search via Clustering and Pruning\" has been accepted by EMNLP-2023. Please cite it correctly. \n    - Why is the related work in section 5? Introducing it in section 2 will make readers understand the background better.\n    - Unify the usage of abbreviated words. Tab. -> Table.\n\n- When introducing BCL, the author takes some examples to make readers understand the $\\gamma$. I suggest authors provide more experiments on how different $\\gamma$s affect the results. \n\n- As shown in Figure 3, the improvement from BC to BCL seems marginal. \n\n- Any border impact and limitation discussion?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)