Calibrate to Discriminate:Improve In-Context Learning with Label-Free Comparative Inference

## Abstract

Abstract While in-context learning with large language models (LLMs) has shown impressive performance, we have discovered a unique miscalibration behavior where both correct and incorrect predictions are assigned the same level of confidence. We refer to this phenomenon as indiscriminate miscalibration . We found that traditional calibration metrics, such as Expected Calibrated Errors (ECEs), are unable to capture this behavior effectively. To address this issue, we propose new metrics to measure the severity of indiscriminate miscalibration.
Additionally, we develop a novel in-context comparative inference method to alleviate miscalibrations and improve classification performance. Through extensive experiments on five datasets, we demonstrate that our proposed method can achieve more accurate and calibrated predictions compared to regular zero-shot and few-shot prompting.

## 1 Introduction

Large Language Models (LLMs) have exhibited emergent capabilities such as advanced creative writing, summarization, translation, arithmetic and commonsense reasoning, etc (Wei et al., ; Brown et al., ). One of the most fascinating aspects of LLMs is their in-context learning capabilities.
In particular, this involves adding pairs of demonstration examples to the prompt, and has been shown to significantly enhance the performance of LLMs (Brown et al., ). This capability offers users the significant advantage of utilizing LLMs without the need to train or fine-tune their own models. As a result, there is a growing need for research into calibration techniques (Zhou et al., ; Zhao et al., ; Jiang et al., ; ) to ensure the reliability of model outputs as well and improve performance.

The concept of calibration in modern deep learning models was first introduced in (Guo et al., ) where the authors also proposed metrics (such as Expected Calibration Errors, reliability diagrams, etc) and methods (such as temperature scaling) for characterizing and mitigating miscalibration issues. The miscalibration of LLMs has also been studied in recent works (Xiong et al., ; Tian et al., ; Liusie et al., ; Zhao et al., ; Geng et al., ; Kamath et al., ). In this work, we show that LLMs with zero-shot and few-shot prompting exhibit a unique miscalibration issue on classification tasks, which we refer to as indiscriminate miscalibration. This phenomenon occurs when models assign equal confidence to correct and incorrect predictions.
We show that this phenomenon cannot be quantitatively measured by Expected Calibration Errors (ECE). One alternative metric that can help catch this phenomenon is using Macro-average Calibration Error (MacroCE) proposed in Si et al. (). However, it may not depict the phenomenon thoroughly as it only computes the means of the distributions. We further propose a metric to help describe the phenomenon in more details.
We hypothesize that this indiscriminate miscalibration occurs because the model treats each sample independently and has not been trained on the corresponding dataset. As a result, the predicted probabilities are not comparable across samples, which can further negatively impact prediction accuracy.

To this end, we propose a label-free in-context comparative inference method
that adds unlabeled samples to the prompt. This encourages the model to adjust and calibrate its predictions without requiring labels for the demonstration examples. The added examples serve as a proxy to help the model better understand the test examples and the corresponding task.
We delve deeper into the principles of our method and develop an aggregation step for improved calibration and a post-hoc calibration method that can further enhance performance. Through experiments on multiple datasets, we demonstrate significant gains in performance measured by F1 scores, accuracy, and traditional calibration metrics. We also show that our label-free in-context comparative inference method helps to alleviate indiscriminate miscalibration, enabling models to assign higher confidence to correct predictions and lower confidence to incorrect ones.

Figure: (a)
Refer to caption: x1.png

## 2 Related Work

### 2.1 Confidence and Calibration

Confidence calibration on modern neural networks has been discussed in (Guo et al., ) where the authors showed that large vision models are poorly calibrated. and proposed the Expected Calibration Errors (ECEs) that have been widely used in the literature. A more recent paper reviews some drawbacks of ECEs and proposed Instance-level Calibration Error (ICE) and Macro-average Calibration Error (MacroCE) (Si et al., ). Recent studies have shown great interests of model uncertainty and calibration in language models as well(Xiong et al., ; Tian et al., ; Liusie et al., ; Zhao et al., ; Geng et al., ; Kamath et al., ). As one of the most popular method for prompt engineering with LLM, in-context few-shot prompting (Brown et al., ) has the output instability issue, which was revealed by Zhao et al. (). They proposed a simple method to estimate and adjust majority label bias, recency bias, and common token biases introduced by in-context learning.
Jiang et al. () also showed that LLMs can be overconfident about their answers and calibration purposed fine-tuning or post-hoc calibration method can be used to improve the performance. Other similar methods such as batch calibration (Zhou et al., ) has been proposed as well. More recent studies show that few-shot prompting, finetuning or chain-of-thoughts can all suffer miscalibration issues (Zhang et al., ). In a low shot setting (e.g. $<4$ demonstration examples), model prediction accuracy and calibration error can both increase and the trade-off can be improved with larger models or more shots (e.g. $>8$ shots). Moreover, instruction fine-tuned LLMs exhibit the same miscalibration issue(Jiang et al., ) as well.

### 2.2 In-Context Learning

In few-shot prompting (Brown et al., ) setting, one provides input-label demonstration pairs in the prompt to improve LLMs’ understanding and generation capability. Many methods have been proposed to improve ICL and understand its behaviors. Wang et al. () shows that the label words and contextual words can impact model generations. Min et al. () shows that providing a few examples of the label space, the distribution of the input text, or the format of the sequence can be more important than ground truth labels. Ensemble methods such as self-consistency (Wang et al., ) over a diverse set of reasoning paths can improve performance as well. In a more recent study, comparison ability (Liusie et al., ) without labels was also demonstrated as one of the emergent capability of LLMs (Wei et al., ).

## 3 Background

### 3.1 Notations

Figure: Figure 2: Reliability diagrams averaged across 5 datasets. The confidence matches the accuracy (y-axis) for a perfectly calibrated model. Hence, the red gaps indicate the severity of miscalibrations for each confidence bin. The first and second rows show the indiscriminate miscalibration behavior of large language models under in-context learning (0-shot and 10-shot) setting where accuracy are similar regardless whether confidences are high or low. In certain cases, lower confidences can give even higher accuracy. Under comparative inference setting (e.g.third row), such issue is alleviated and significantly improved with aggregated comparative inference (e.g.last row).
Refer to caption: x3.png

Denote a classifier as $f_{\theta}(X)$ which is parameterized by $\theta$ and takes input $X\in\mathcal{X}$. $f_{\theta}(X)$ is optimized on the training data $\mathcal{D}_{\textnormal{train}}=\{x_{1},y_{1},...,x_{N},y_{N}\}$ with label $Y\in\mathcal{Y}=\{1,2,...,K\}$. We take a calibration point of view and assess whether the label probability generated by a language model can accurately express its confidence level such that the probability estimate $\hat{P}(Y|X)$ can reflect the actual probability of the prediction being correct (Guo et al., ; Jiang et al., ; Liusie et al., ). Here we use $\hat{P}$ to represents the probability estimate, which is a vector of probabilities over $K$ labels. The label estimate is derived by taking the argmax such that $\hat{Y}=\textnormal{argmax}_{Y}\hat{P}(Y|X)$ (Guo et al., ; Liusie et al., ). For simplicity, we further define $p^{*}=\textnormal{max}_{Y}\hat{P}(Y|X)=\hat{P}(\hat{Y}|X)$ to represent the probability estimate of the predicted label which is used to represent the confidence (Guo et al., ) of the label estimate.

### 3.2 Confidence Estimate for LLMs

For a LLM, we extract the token probability for each class $Y\in\mathcal{Y}\{1,2,...,K\}$ as the probability estimates $\hat{P}(Y|X)$. For example, in a binary classification setting, we have the label domain to be $\mathcal{Y}=\{0,1\}$ where 1 indicates ‘Yes’, and 0 indicates‘No’. The probability estimate $\hat{P}(Y=1|X)$ is extracted by computing the likelihood of generating label ‘Yes’. More specifically, if the generated sentence is ‘The answer is: Yes.’ and the word ‘Yes’ is tokenized as ‘Yes’. Then we use the probability of token ‘Yes’ after ‘The answer is:’.
In the case where a label word has multiple ways of being tokenized, we sum over all possible tokens. (^1^11Suppose the label word is ‘Positive’. And ‘Positive’ can be tokenized as i) ‘Pos’ + ‘itive’ and ii) ‘Positive’. We sum over both ‘Positive’ and ‘Pos’ tokens to better represent the probability estimate. As in this context, generating ‘Pos’ is almost $100\%$ indicating the next token is ‘itive’.)
Now that we have extracted label probability estimate, we then compute
the Expected Calibrated Error (ECE) (Guo et al., ), which has been widely used to to characterize the calibration level of a model’s confidence (Geng et al., ). It is defined as the following,

$$ $\textnormal{ECE}=\mathbb{E}_{\hat{p}}[|\mathbb{P}(\hat{Y}=Y|\hat{P}=p)-p|].$ $$

The lower the ECE, the more calibrated the model is. Similarly to many other observations (Xiong et al., ; Tian et al., ; Liusie et al., ; Zhao et al., ; Geng et al., ; Kamath et al., ), we found that the LLMs’ probability estimates can be miscalibrated (Figure 2, Table 3). However, while ECE can be generally used to measure whether a model is calibrated, it fails to distinguish different miscalibration behaviors. For example in Figure 1, two reliability diagrams have same ECE but have clearly different miscalibration behaviors which we will elaborate more in the next section.

Figure: (a)
Refer to caption: x4.png

### 3.3 Indiscriminate Miscalibration for In-Context Learning

In contrast to other models’ miscalibration behavior revealed in Guo et al. (), LLM’s miscalibration can be special such that it’s overconfident and the phenomenon has been reported in several previous studies (Jiang et al., ; Tian et al., ). In our experiments, we found that regardless whether the label estimate
is correct or not, LLM tends to give equal confidence (e.g.probability estimate) to its predictions when LM is not trained on the task (e.g. in zero-shot setting). The confidence is not necessarily high (e.g. overconfident). In certain scenarios, LM can also be underconfident (e.g. low confidence with high accuracy). More specifically, as shown in Figure 1 (a), the model gives same accuracy for all confidence levels, this is also discussed in Jiang et al. ().
We report this phenomenon using several classification tasks as an example, shown in Figure 2.

We refer to such behavior as indiscriminate miscalibration, a special case of miscalibration , that can’t be captured by ECEs.
In conventional miscalibration scenario such as the simulated scenario in Figure 1 (b) or the ones reported in Guo et al. (), though in each confidence level bins, its actual accuracy has gaps towards the expected accuracy, the model generally has higher confidence in predictions with high accuracy and lower confidence in predictions with low accuracy.
It still possess the ability to give different confidence levels toward possible correct predictions and incorrect predictions. Such ability is more strictly defined as Relative Confidence (Geng et al., ; Kamath et al., ; El-Yaniv et al., ; Wightman et al., ), the ability to rank samples, distinguishing correct
predictions from incorrect ones. In Si et al. (), authors show that ECE is incapable for describe this behavior and they proposed Instance-level Calibration Error (ICE) as the following,

$$ $\textnormal{ICE}=\frac{1}{n}\sum_{i=1}^{n}|\mathbb{I}(y_{i}=\hat{y_{i}})-\hat{ P}(\hat{y_{i}}|x_{i})|.$ $$

Here, we use $\hat{P}(\hat{y_{i}}|x_{i})$ to replace the confidence in the original paper. While ICE tries to differentiate correct and wrong predictions, miscalibration issue can be disguised when the accuracy is high. To address this, Macro-average Calibration Error (MacroCE) is further proposed to balance the correct predictions and incorrect ones. MacroCE is defined as,

$$ $\displaystyle\textnormal{ICE}_{\textnormal{pos}}$ $\displaystyle=\frac{1}{n_{p}}\sum_{i=1}^{n_{p}}|\mathbb{I}(y_{i}=\hat{y_{i}})- \hat{P}(\hat{y_{i}}|x_{i})|,\forall\hat{y_{i}}=y_{i},$ $\displaystyle\textnormal{ICE}_{\textnormal{neg}}$ $\displaystyle=\frac{1}{n_{n}}\sum_{i=1}^{n_{n}}|\mathbb{I}(y_{i}=\hat{y_{i}})- \hat{P}(\hat{y_{i}}|x_{i})|,\forall\hat{y_{i}}\neq y_{i},$ MacroCE $\displaystyle=\frac{1}{2}(\textnormal{ICE}_{\textnormal{pos}}+\textnormal{ICE} _{\textnormal{neg}})$ $$

where $\textnormal{ICE}_{\textnormal{pos}}$ is the ICE score computed over correct predictions and $\textnormal{ICE}_{\textnormal{neg}}$ is the one with incorrect predictions. On the other hand, MacroCE computes the mean of the difference between confidence and accuracy by averaging over samples. It may not capture certain different miscalibration distribution patterns. Imagine two cases (case A and and B), both cases have same means for correct prediction confidences and incorrect prediction confidences. The MacroCE would be similar for case A and B. However, when the variances are different, these cases should show different indiscriminate levels. We illustrate one visual example in the Appendix. To measure the difference between correct predictions and incorrect predictions on a more granular level such as using information of variances, we also define the Discriminate KL divergence (DKL) as the following,

$$ $\textnormal{DKL}=\textnormal{KL}[\mathbb{P}(p^{*}|\hat{Y}=Y)||\mathbb{P}(p^{*} |\hat{Y}\neq Y)].$ (1) $$

Unlike MacroCE, DKL measures the distribution differences. Larger DKL indicates more discriminate confidences levels between correct predictions and incorrect predictions.We report the results in Figure 3(b). Similarly, we observed that our proposed method helps improving DKL, i.e. more discriminate confidence level.

Figure: Figure 4: Inference performances vary significantly at different positions in the comparative inference setting, examples from the TREC task (Li & Roth, ; Hovy et al., ). In the zero-shot setting, the performance decays at the second and third positions.
Refer to caption: x6.png

## 4 Method

As shown above, when we use LLMs with zero-shot and few-shot prompting, it has trouble in differentiating confidences for correct and incorrect predictions. Our hypothesis is that each sample is treated independently and LLMs are not able to compare or rank them without trained on them even with few-shot prompting. Thus we propose a label-free method by asking LM to compare a sample of interest $X=x_{i}$ with other unlabeled samples jointly to adjust for more calibrated results.

### 4.1 Comparative Inference

Considering three samples $x_{i},x_{j},x_{k}$, we are interested in performing a comparison to derive the labels probability estimate $\hat{P}(y_{i},y_{j},y_{k}|x_{i},x_{j},x_{k};C)$, where $C$ represents a prompting instruction asking for a comparison. The following template is a prompt example,

Notice that even we are using words like ‘comparing’, the goal is not to ask LLMs to say if sample 1 is ‘more positive’ than sample 2 and sample 3 as that may change the original task meaning. It’s more important to present multiple unlabeled samples in the context and ask LLMs to predict labels for each of them jointly.

#### 4.1.1 Asymmetric Probabilities

The first thing to notice is because of the auto-regressive nature of large language models, we have $\hat{P}(y_{i},y_{j}|x_{i},x_{j};C)\neq\hat{P}(y_{j},y_{i}|x_{j},x_{i};C)$. More specifically, if $x_{i}$ appears before $x_{j}$ in the prompt, where one would expect $\hat{y_{i}}$ to be generated before $\hat{y_{j}}$, which means the $\hat{y_{j}}$ is generated conditioned on $\hat{y_{i}}$. And it can be shown with the following,

$$ $\begin{split}\hat{P}(y_{i},y_{j}&|x_{i},x_{j};C)=\hat{P}(y_{i}|x_{i},x_{j};C) \hat{P}(y_{j}|x_{i},x_{j},\hat{y_{i}};C).\end{split}$ (2) $$

This could lead to bias to the samples generated in later orders. Indeed, ordering of words in the prompts can impact performance significantly (Lu et al., ; Min et al., ). In our experiments, we observed a degradation of inference performance on later input samples (Figure 4). Besides, the post-processing and probabilities extractions of inputs after the first prediction can be tricky. Hence, we focus on the prediction of the first input throughout the paper, i.e. $\hat{P}(y_{i}|x_{i},x_{j},...;C)$ and always put the sample of interest as the first sample to predict. The rest input samples only serve as a reference comparison inputs.

#### 4.1.2 Comparison Bias

In an ideal setting, we can derive the probability estimates leveraging all the possible input samples
$\hat{P}(y_{i}|x_{i},\mathcal{D}_{x};C)$.
However, this is impossible as that will make the prompt too long to be processed by LLMs. Thus, we make the following assumptions,

$$ $\begin{split}\hat{P}(y_{i}|x_{i},\mathcal{D}_{x};C)\approx&\hat{P}(y_{i}|x_{i} ,\{x\}_{j};C)+\epsilon_{j},\textnormal{with }\mathbb{E}(\epsilon_{j})=\bm{0}, \end{split}$ (3) $$

where $\{x\}_{j}$ is a sequence of samples from $\mathcal{D}_{x}$; $\epsilon_{j}$ is the bias introduced when comparing with $\{x\}_{j}$ in the prompt instead of $\mathcal{D}_{x}$ (e.g. similar to the contextual bias (Zhao et al., ). We further hypothesize the biases introduced by different comparison samples can be averaged out (e.g expectation is zero) so we can approximate it in practice by the following,

$$ $\begin{split}\widehat{P}(y_{i}|x_{i},\mathcal{D}_{x};C)&\approx\int_{\mathbb{P }_{j}}\hat{P}(y_{i}|x_{i},\{x\}_{j};C)\approx\sum_{j=1}^{J}\frac{1}{J}\hat{P}( y_{i}|x_{i},\{x\}_{j};C),\end{split}$ (4) $$

where $J$ is a finite number of sets of comparisons. This is simply aggregating multiple comparative inference probabilities with different comparison samples. We expect with aggregations, calibration error will go down drastically. As one could see in Figure 5, we can see ECE drastically decreased with aggregation. F1 score and accuracy is also improved but relatively less aggressive. Though there still exist other biases (e.g. contextual, label, etc) that can be further calibrated. However, this indeed will increase inference cost by the number of aggregation times.

Figure: Figure 5: Performance (ECE and F1 scores) improve as we aggregate more comparative inference results. Results are averaged across 5 dataset. Notably, the ECE decrease (the lower the better) drastically with aggregations under our assumption eq.4.
Refer to caption: x7.png

### 4.2 Post-hoc Calibration

There exist many ways in the literature for calibrating probabilities such as Histogram binning (Zadrozny & Elkan, ), Isotonic regression(Zadrozny & Elkan, ), Platt scaling (Platt et al., ), etc. One simplest approach is temperature scaling (Guo et al., ) where we apply weighted softmax on logits or probabilities to recalibrate probabilities. While such methods can decrease ECE, it does not help with improving model’s classification performance. Extensions of Platt scaling towards multi-class classification is referred as vector scaling and matrix scaling, where one would optimize the following linear layers on top of the probability estimates,

$$ $\tilde{P}(y_{i}|x_{i})=\textnormal{softmax}(W\hat{P}(y_{i}|x_{i})+b),$ $$

where $\tilde{P}(y_{i}|x_{i})$ is the calibrated probabilities. In the vector scaling case, $W$ is restricted to a diagonal matrix. And $W$ and $b$ are optimized using a validation dataset. Inspired by these methods, we propose to further calibrate our probability estimates by applying such affine transformations but on top of different comparison referencs.
More specifically, we retrieve the recalibrated probabilities by,

$$ $\begin{split}\tilde{P}&(y_{i}|x_{i},\mathcal{D}_{j};C)=\textnormal{softmax}\{ \sum_{j=1}^{J}\{W_{j}\hat{P}(y_{i}|x_{i},\{x\}_{j};C))+b_{j}\},\end{split}$ (5) $$

where $W_{j},b_{j}$ can be estimated via $\textnormal{argmax}\{-y_{k}\textnormal{log}P^{*}(y_{k}|x_{k},\{x\}_{j};C))\}$ with a set of $\{x_{k},y_{k}\}$.
This is similar to the principle of contextual calibration used in Guo et al. (); Zhao et al. (). The difference is that previous studies focused calibrating biases introduced from labels, or other prompt contextual information whereas we are focusing on the comparison samples.

## 5 Results

### 5.1 Experiments Setup

##### Models and Datasets

We conducted experiments using the Llama(Touvron et al., ) family models: Llama2-7b-chat, Llama2-13b-chat, Llama2-70b-chat , Llama3-8b-instruct, Llama3-70b-instruct. Our experiments focused on classification problems. We picked 5 classic tasks covering different domains: AGNews (topic classification) (Zhang et al., ), trec (topic classifications) (Voorhees & Tice, ; Li & Roth, ; Hovy et al., ), Financial datasets (sentiment analysis) (Malo et al., ), emotion dataset (emotion detection) (Saravia et al., ) and hateful speech detection (Toxicity detection) (de Gibert et al., ). Due to limited capacity, we capped the number of testing samples at $500$. For each method, we run $10$ replicates with different seeds.

**Table 1: F1 scores and ECE averaged across 5 dataset with 10 runs for all models. LF indicates label-free comparative inference; ‘agg’ indicates aggregate $10$ different comparative inference results. Note that for all LF-ICL experiments, we don’t provide labels. Higher F1 score and lower ECE are achived by LF-ICL.**
|  |  | ICL | LF-ICL |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Model | 0 shot | 3 shot | 10 shot | 0 shot | 0 shot-agg | 3 shot | 10 shot |
| F1$\uparrow$ | L2-7B | 0.37 | 0.54 | 0.46 | 0.56 | 0.59 | 0.375 | 0.45 |
| L2-13B | 0.48 | 0.61 | 0.58 | 0.58 | 0.65 | 0.53 | 0.61 |  |
| L2-70B | 0.60 | 0.72 | 0.74 | 0.66 | 0.72 | 0.71 | 0.75 |  |
| L3-8B | 0.52 | 0.63 | 0.63 | 0.70 | 0.74 | 0.71 | 0.77 |  |
| L3-70B | 0.71 | 0.78 | 0.80 | 0.76 | 0.77 | 0.80 | 0.81 |  |
| ECE$\downarrow$ | L2-7B | 0.35 | 0.36 | 0.46 | 0.29 | 0.22 | 0.45 | 0.39 |
| L2-13B | 0.35 | 0.32 | 0.36 | 0.29 | 0.13 | 0.30 | 0.24 |  |
| L2-70B | 0.16 | 0.21 | 0.20 | 0.17 | 0.13 | 0.21 | 0.19 |  |
| L3-8B | 0.29 | 0.29 | 0.30 | 0.16 | 0.09 | 0.21 | 0.16 |  |
| L3-70B | 0.17 | 0.15 | 0.19 | 0.14 | 0.09 | 0.17 | 0.16 |  |

##### Methodology

Our experimental results are mainly composed with two parts: i) methods for manipulating prompts and ii) further post-hoc calibrations where we train an additional linear layer with less than hundreds of parameters. For i), the baseline is the independent inference where each prompt contains only one input sample without demonstrations (zero-shot). We provide our sample prompts in the Appendix. We include few-shot prompting (Brown et al., ) methods with either $3$-shot or $10$-shot. For our Label-Free (denoted as LF in figures and tables) comparative inference method, we use $2$ additional input samples randomly picked from the training dataset as comparison references. As discussed in section 4.1, we always placed the sample of interest at the first place among all $3$ samples and focused on the results of the first answer generated by LLMs. Our method also uses aggregations where we used different comparison references in the prompt and retrieved averaged results from up to $10$ comparative inference results. LF method can be used with few-shot together, which is also included. For ii) post-hoc calibration methods, we mainly considered the vector scaling and matrix scaling methods that have been widely used in Guo et al. (); Zhao et al. (); Zhou et al. (); Zhang et al. ().

**Table 2: F1 and ECE scores averaged across 5 datasets with 10 replicates for all models with post-hoc calibration. We experiment with different shot number: 0, 3, 10. LF represents label-free in-context learning. All inference results indicates are based on post-hoc calibration with $10$ inference results.**
|  |  | Matrix Scaling | Vector Scaling |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Model | 0 | 3 | 10 | 0-LF | 10-LF | 0 | 3 | 10 | 0-LF | 10-LF |
| F1$\uparrow$ | L2-7B | 0.57 | 0.60 | 0.50 | 0.69 | 0.69 | 0.54 | 0.53 | 0.47 | 0.67 | 0.70 |
| L2-13B | 0.54 | 0.67 | 0.57 | 0.71 | 0.75 | 0.52 | 0.65 | 0.57 | 0.73 | 0.76 |  |
| L2-70B | 0.54 | 0.74 | 0.70 | 0.77 | 0.79 | 0.51 | 0.68 | 0.71 | 0.77 | 0.80 |  |
| L3-8B | 0.63 | 0.74 | 0.74 | 0.71 | 0.79 | 0.64 | 0.74 | 0.74 | 0.71 | 0.78 |  |
| L3-70B | 0.74 | 0.79 | 0.80 | 0.79 | 0.83 | 0.70 | 0.78 | 0.80 | 0.79 | 0.82 |  |
| ECE$\downarrow$ | L2-7B | 0.17 | 0.21 | 0.29 | 0.23 | 0.22 | 0.17 | 0.20 | 0.29 | 0.23 | 0.19 |
| L2-13B | 0.24 | 0.20 | 0.29 | 0.25 | 0.20 | 0.23 | 0.23 | 0.28 | 0.20 | 0.18 |  |
| L2-70B | 0.28 | 0.20 | 0.20 | 0.19 | 0.18 | 0.28 | 0.17 | 0.21 | 0.18 | 0.17 |  |
| L3-8B | 0.20 | 0.17 | 0.18 | 0.24 | 0.19 | 0.16 | 0.18 | 0.18 | 0.21 | 0.18 |  |
| L3-70B | 0.17 | 0.16 | 0.16 | 0.18 | 0.16 | 0.17 | 0.15 | 0.15 | 0.17 | 0.16 |  |

### 5.2 Label-Free Comparative Inference Improves Classification Performance

For classification performance metrics, we consider accuracy and F1 scores. We empirically found these two are consistent. As most of the datasets are imbalanced, we reported F1 scores averaged across $5$ datasets in the main text and the corresponding accuracy results in the Appendix.
From F1 scores in Table 3 and simiarly accuracy from the Appendix, comparative inference without labels can outperform few-shot prompting (e.g. Llam2-7b and Llama2-70b), while both are significantly better than the zero-shot. We found that both Llama3-8b and Llama3-70b behave even better in comparative inference setting with the TREC and Emotion tasks. Interestingly, these two tasks have $6$ different labels which are the most across $5$ dataset. However, in the ICL few-shot setting, both models did not perform well in terms of the calibrations.

### 5.3 Alleviate Indiscriminate Miscalibration with Comparative Inference

By applying comparative inference, we observe a significant alleviation of the indiscriminate miscalibration issue. More specifically:i) Expected Calibration Error (ECE) results are generally improved, with the aggregation method (section 4.1.2) consistently performing the best (Table 3). ii). The MacroCE score decreased (Figure 3(a)) and KL divergence increased with comparative inference across models and datasets (Figure 3(b)). iii). These behaviors are represented in the reliability diagrams (Figure 2) as well. Additionally, we found that few-shot prompting can potentially worsen ECEs, depending on the specific models used(Table 3, Figure 2).
As we discussed in section 4.1.2, we expect aggregation to improve calibration issues more compared to classification performances given our assumption. And we do observe such behaviors as shown by Figure 2, Figure 3 and also by ECEs in Table 3. Aggregation can also improve F1 and accuracy as this can be seen as a way of ensemble, similarly to other methods such as Wang et al. (). In our experiment, the performance is even better than comparative + ICL for Llama2-7b and Llama2-13b. We can also expect combining ICL, aggregation and our methods can further enhance performances (Table 3). However, this will require more tailored labels, and inference costs.

### 5.4 Considerations and Best Practices for Effective Comparative Inference

First observation is that we can combine LF method with few-shot to further improve performances. We found performances enhanced across the board except for the Llama2-7b based on averged F1 score (Table 3). We hypothesize this is because Llama2-7b is not capable to leverage both comparative inference and ICL at the same time.

Secondly, we found that LF comparative inference with ICL are mostly effective with 10-shots setting whereas the independent method perform well with 3-shot for smaller models but better with 10-shot for larger or more capable models (e.g. Llama3-8b or 70b models). For examples, $10$-shot comparative inference setting is significantly better than other baselines for Llama2-7b and Llama2-13b models on the TREC dataset.

### 5.5 Post-hoc Calibration Results

As described in section 4.2, vector scaling has $K$ parameters while our post-hoc method will have $J\times K$ parameters. In our experiments, $K$ is generally $<10$ and we restrict $J=10$, which will restricted the total number of trainable parameters to less than hundreds for vector scaling and thousands for matrix scaling. In addition, the optimizations of $W$ and $b$ are conducted using $200$ validation samples for all datasets.
Similarly, we reported F1 scores and ECE scores averaged across 5 datasets for each model in Table 4. We show the breakdowns in the Appendix.

We found that comparative inference with post-hoc calibration is consistently better than baseline post-hoc calibration methods or the baselines with few-shot prompting for Llama2 family in terms of F1 scores. For Llama3, baseline with few-shot prompting is better, which is likely due to the fact that Llama3 models have strong capability on few-shot prompting.
We further show that comparative inference with post-hoc calibration can be combined with few-shot prompting and this will further boost the performance and is consistently better than all other methods in terms of F1 scores.
ECEs on average are improved compared to baselines but for many settings are not better than certain methods in the previous setction. As indeed, post-hoc calibration has been recently used to improve classification performances (Zhao et al., ; Zhou et al., ) instead of focusing on conventional calibration issues.

## 6 Discussion

In this paper, we study a special miscalibration behavior of large language models when being used for zero-shot and few-shot prompting, which we refer to as indiscriminate miscalibration. We propose metrics to quantify the severity of this issue and develop a label-free in-context comparative inference method to mitigate it. We show our label-free method can achieve better classification performance as well as more calibrated predictions on multiple datasets.

## 7 Limitations

The primary limitation is that this study only considered classification tasks. Secondly, we considered 5 models but are all from Llama families. While these 5 models are widely recognized for their performance in natural language processing tasks, the inclusion may introduce biases inherent to this particular model family. Future research can benefit from exploring a broader range of tasks and considering models from other families to provide a more comprehensive understanding of their capabilities and limitations. Lastly, we didn’t study finetuned model, which can be one of our future directions.

## 8 Appendix

### 8.1 Prompt Examples

### 8.2 Accuracy Results

**Table 3: Accuracy averaged across 5 dataset with 10 replicates for all models. 0-shot indicates baseline independent inference and the corresponding first row includes In-Context Learning with independent inference results; ‘LF’ indicates label-free comparative inference 0-shot setting and the corresponding second row includes the In-Context Learning with comparative inference results; ‘agg’ indicates aggregate $10$ different comparative inference results.**
|  |  | ICL | LF-ICL |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Model | 0 shot | 3 shot | 10 shot | 0 shot | 0 shot-agg | 3 shot | 10 shot |
| Accuracy$\uparrow$ | L2-7B | 0.41 | 0.57 | 0.49 | 0.57 | 0.59 | 0.46 | 0.51 |
| L2-13B | 0.49 | 0.61 | 0.59 | 0.59 | 0.64 | 0.57 | 0.64 |  |
| L2-70B | 0.52 | 0.63 | 0.63 | 0.71 | 0.75 | 0.71 | 0.77 |  |
| L3-8B | 0.61 | 0.72 | 0.74 | 0.67 | 0.73 | 0.71 | 0.75 |  |
| L3-70B | 0.72 | 0.77 | 0.8 | 0.77 | 0.78 | 0.8 | 0.81 |  |

**Table 4: Accuracy averaged across 5 datasets with 10 replicates for all models with post-hoc calibration. n-shot: baseline in-context learning; LFrepresents label-free in-context learning. All inference results indicates are based on post-hoc calibration with $10$ inference results.**
|  |  | Matrix Scaling | Vector Scaling |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Model | 0 shot | 3 shot | 10 shot | 0 shot-LF | 10 shot-LF | 0 shot | 3 shot | 10 shot | 0 shot-LF | 10 shot-LF |
| Accuracy$\uparrow$ | L2-7B | 0.6 | 0.64 | 0.55 | 0.7 | 0.71 | 0.57 | 0.6 | 0.53 | 0.69 | 0.71 |
| L2-13B | 0.56 | 0.69 | 0.63 | 0.72 | 0.76 | 0.54 | 0.68 | 0.62 | 0.74 | 0.77 |  |
| L2-70B | 0.55 | 0.76 | 0.72 | 0.78 | 0.8 | 0.53 | 0.71 | 0.71 | 0.78 | 0.8 |  |
| L3-8B | 0.65 | 0.75 | 0.75 | 0.72 | 0.79 | 0.66 | 0.75 | 0.74 | 0.73 | 0.79 |  |
| L3-70B | 0.76 | 0.8 | 0.81 | 0.8 | 0.83 | 0.74 | 0.79 | 0.81 | 0.8 | 0.83 |  |

### 8.3 Special Scenario with Indiscriminative Miscalibration

[Uncaptioned image]: x8.png

We simulate a scenario where correct prediction probabilities are sampled from a distribution with mean of $0.65$ and incorrect prediction probabilities are sampled from a distribution with mean of $0.45$. In the left figure, distributions have smaller variances while in the right figure, variances are larger. In this scenario, MacroCE will give the same value for both cases while ‘qualitatively’, left side is more ‘discriminative’ and has larged DKL value.

### 8.4 Individual Dataset Results

Figure: Figure 6: Trec Dataset Results. (a). Reliability Diagram. (b). DKL Divergence and MACROCE. (c). Comparative results with F1 scores and ECEs. (d). Post-hoc calibrated results. *Notice that for the aggregation method of the individual data result, we only have one round experiment so we only reported means without variances.
Refer to caption: x9.png

Figure: Figure 7: AGnews Dataset Results. (a). Reliability Diagram. (b). DKL Divergence and MACROCE. (c). Comparative results with F1 scores and ECEs. (d). Post-hoc calibrated results. *Notice that for the aggregation method of the individual data result, we only have one round experiment so we only reported means without variances.
Refer to caption: x14.png

Figure: Figure 8: Financial Dataset Results. (a). Reliability Diagram. (b). DKL Divergence and MACROCE. (c). Comparative results with F1 scores and ECEs. (d). Post-hoc calibrated results. *Notice that for the aggregation method of the individual data result, we only have one round experiment so we only reported means without variances.
Refer to caption: x19.png

Figure: Figure 9: Emotion Dataset Results. (a). Reliability Diagram. (b). DKL Divergence and MACROCE. (c). Comparative results with F1 scores and ECEs. (d). Post-hoc calibrated results. *Notice that for the aggregation method of the individual data result, we only have one round experiment so we only reported means without variances.
Refer to caption: x24.png

Figure: Figure 10: Hatespeech Dataset Results. (a). Reliability Diagram. (b). DKL Divergence and MACROCE. (c). Comparative results with F1 scores and ECEs. (d). Post-hoc calibrated results. *Notice that for the aggregation method of the individual data result, we only have one round experiment so we only reported means without variances.
Refer to caption: x29.png

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 3
presentation: 3
questions: 1. Why do you believe the bias averaging assumption in Equation 3 holds? Can you provide theoretical or empirical evidence?

2. How do you select comparison samples in practice? Is there a strategy better than random selection?

3. What is the computational overhead of your method compared to standard few-shot prompting? Please provide detailed timing analysis.

4. How sensitive is the method to the choice of comparison samples?

5. Can you comment on the method's impact on open-ended text generation and language modeling tasks?
soundness: 2
strengths: 1. The paper identifies an interesting and important problem regarding LLM calibration
2. The proposed label-free approach is novel and doesn't require labeled examples
3. The method appears to improve both classification performance and calibration metrics
summary: This paper investigates a phenomenon called "indiscriminate miscalibration" in LLMs during in-context learning, where models assign similar confidence levels to both correct and incorrect predictions. The authors propose new metrics to measure this issue and introduce a label-free comparative inference method that includes unlabeled samples in prompts to improve calibration and classification performance. The authors evaluate their approach on 5 datasets using various Llama models and demonstrate improvements in both classification metrics (F1 score, accuracy) and calibration metrics.
weaknesses: 1. The experimental design has several limitations that raise concerns about the generalizability and robustness of the results. Firstly, the evaluation is only conducted on Llama family models, which may not be representative of other language models or architectures. Secondly, the 5 datasets used are relatively standard classification tasks, and it is unclear how the method would perform on more complex or challenging datasets. Furthermore, the maximum of 500 test samples feels limited for a robust evaluation, and the lack of ablation studies examining the impact of different comparison sample selection strategies makes it difficult to disentangle the effects of different components of the method.

2. The paper lacks a rigorous theoretical analysis of why the comparative inference approach works. The assumptions made in Section 4.1.2 about bias averaging (Equation 3) are not well justified, and it is unclear what underlying principles drive the improvement in calibration and classification performance. A more thorough exploration of the theoretical foundations of this method would be necessary to fully understand its implications and limitations.


3. The comparative inference method increases the inference cost significantly due to multiple forward passes, which may be a concern for real-world applications where computational resources are limited. Moreover, the post-hoc calibration requires validation data, which may not be readily available for open-ended generation tasks, limiting the method's applicability in these scenarios.

---

confidence: 5
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. Regarding the experiments on post-hoc calibration, the authors claim that "comparative inference with post-hoc calibration can be combined with few-shot prompting" to further improve performance. Do the authors have experimental data to support this claim?
2. The LF-based methods do not show a robust improvement in terms of Expected Calibration Error (ECE) in the post-hoc calibration experiments. Why do the authors not present results using other metrics like the DKL they have just proposed?
soundness: 3
strengths: 1. The idea of using comparative inference to improve confidence calibration in LLMs is novel.
2. Extensive experiments demonstrate the effectiveness of their method.
3. The paper is clearly written and easy to follow.
summary: This paper addresses the problem of indiscriminate miscalibration in Large Language Models (LLMs). The authors introduce a new calibration metric called Discriminate KL Divergence (DKL) and propose an in-context comparative inference method -- a label-free approach termed LF -- to mitigate this issue. They validate their method on five classification tasks using five models from the LLama family, employing both traditional and their newly proposed metrics.
weaknesses: 1. Lack of in-depth analysis on why comparative inference alleviates indiscriminate miscalibration. It remains unclear how such a comparison triggers the LLM to output discriminative confidence.
2. Unfair comparisons in the experiments. For instance, the 0-shot-agg LF-ICL requires ensembling 10 inferences to produce one result, resulting in 10 times the computational cost compared to the 0-shot ICL baselines. While this is not a major issue since 0-shot LF-ICL still outperforms 0-shot ICL, including a 0-shot-ensemble ICL baseline would provide a more rigorous comparison.

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 3
questions: **Clarification regarding Figure 1 setting**: I am not able to understand the reason behind the difference in observations between Figures 1(a) and 1(b). Why do they behave differently? What distinguishes the experimental settings in each case—is it the use of different datasets, or some other factor? These are important questions, and an apples-to-apples comparison is essential for clarity.

**With-labels version**: How would the method perform if we included labels for the 2 additional input samples? Would adding labels increase/decrease the performance? If it improves, by how much? This would perhaps be an interesting study.
soundness: 2
strengths: The indiscriminate miscalibration observation, where both correct and incorrect predictions are assigned the same level of confidence, is quite interesting, as previous studies predominantly reported findings similar to Figure 1(b). Notably, the proposed method does not require labels, which is commendable. Experiments were conducted on five different LLMs of varying sizes which is also laudable.
summary: The paper studies the confidence calibration of large language models (LLMs) on classification tasks. The authors observe the indiscriminate miscalibration behavior of LLMs where accuracy is similar regardless of whether confidences are high or low and that metrics like ECE fail to capture this behavior accurately. The authors propose a KL divergence-based metric that can better capture this behavior and propose a post-hoc label-free calibration method to alleviate indiscriminate miscalibration.
weaknesses: **Key Related Works Missing**: One of my primary concerns is that the paper does not cite several key works in the calibration literature that are highly relevant, including:

1)	Abbas, M., Zhou, Y., Ram, P., Baracaldo, N., Samulowitz, H., Salonidis, T., and Chen, T. Enhancing in-context learning via linear probe calibration. In Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, pp. 307–315, 2024.

2)	Zhixiong Han, Yaru Hao, Li Dong, Yutao Sun, and Furu Wei. Prototypical calibration for few-shot learning of language models. In Proc. of International Conference on Learning Representations, 2023.

3)	Zhongtao Jiang, Yuanzhe Zhang, Cao Liu, Jun Zhao, and Kang Liu. Generative calibration for in-context learning. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of EMNLP 2023.

4)	M. Shen, S. Das, K. Greenewald, P. Sattigeri, G. Wornell, and S. Ghosh. Thermometer: Towards universal calibration for large language models. ICML 2024.

These papers focus on calibration for LLMs and could also serve as valuable baselines. While the first three aim to enhance LLM performance, particularly in the in-context learning (ICL) setting, 1) Abbas et al. (2024) and 4) Shen et al. (2024) notably also tackle the issue of Expected Calibration Error (ECE) metric. Thus, I respectfully disagree with the authors’ statement in lines 358-359 that “While such methods can decrease ECE, it does not help with improving model classification performance,” as these papers address both aspects.

**Limited baselines**: The baselines are too limited, and the papers I mentioned earlier should be included as valid baselines.


**Limited evaluation**: The dataset selection here appears quite restricted compared to related studies. For example, Batch Calibration (Zhou et al., 2023) utilizes 13 classification datasets, while Generative Calibration (Jiang et al., 2023) uses 12. Using a wider range of datasets would lead to a more thorough assessment and strengthen the study.

**Unfair comparison**: If we use two additional input samples in the prompt (as mentioned in lines 415-416), the zero-shot effectively becomes 2-shot, while three-shot becomes five-shot. I know we are not providing labels for the 2 additional samples but still the comparison of 0-shot/3-shot/10-shot LF-ICL with 0-shot/3-shot/10-shot ICL is a bit unfair since adding additional samples provides additional context/information.

**Reproducibility**: The authors did not provide the code to reproduce the results, which raises concerns about reproducibility.

**Suggestions**: 

Previous studies have shown that ICL is sensitive to the choice of demonstrations used in the few-shot setting. It would be good to report standard deviation across different choices of the test demonstrations in Table 1 and Table 2, as done by (Contextual Calibration, Zhao et al., 2021), (Linear Probe Calibration, Abbas et al., 2024), and (Prototypical calibration, Han et al., 2023) etc.

In Table 1, it would have been nice to see the results for the 3 shot-agg and 10 shot-agg as well.


**Suboptimal Writing**: The writing in the paper is quite suboptimal, containing numerous grammatical errors and typographical errors. Here are some suggestions for improvement:

Lines 027-228: Redundancy: ‘LLMs’ abbreviation defined again. It was already defined in the abstract.

Lines 094-097: Several grammatical issues in the sentence

Lines 135-136: grammar/typo: use ‘accuracies are’ or ‘accuracy is’. Moreover, "regardless whether" should be "regardless of whether". It is better to use 'LLM' abbreviation as defined earlier in the abstract. 

Line 164: grammar/typo: use 'an LLM' instead of 'a LLM'.

Lines 195-197: typo/grammar: "dataset" should be pluralized as "datasets.". typo: add space in the beginning of the sentence (i.e. between '...datasets.' and 'A smaller...').

Lines 234-236: grammar/typo: "incapable for describe" should be "incapable of describing"

Lines 268-269: grammar/typo: "it has trouble" should be "they have trouble": The subject "LLMs" is plural, so it should be followed by "they" rather than "it."

Lines 270-272: grammar/typo: "without trained on them" should be "without being trained on them". typo/grammar: "asking LM" should be "asking the LLM".

Lines 289-291: suggestion for better readability: change "Notice that even we are using words like ‘comparing’" to "Note that even though we use words like ‘comparing’". suggestion for better readability: Change "and sample 3" to "or sample 3" to improve readability.

Line 296: typo/grammar: "is because of" → "is that, due to".

Line 298: The phrase "where one would expect" is confusing in context; replacing it with "one would expect" directly makes it clearer.
Line 302: suggestion: "bias to the samples" → "bias toward samples".

Line 303-304: suggestion: "degradation of inference performance on later input samples" → "degradation in inference performance for samples appearing later in the input".

Line 341 (Figure 5): typo/grammar: "across 5 dataset." should be pluralized as "across 5 datasets."

Lines 346-347: grammar/typo: "F1 score and accuracy is also improved" should be "F1 score and accuracy are also improved."

Line 348: grammar/typo: "Though" → "Although".

Lines 367-368: typo: "referencs" → "references".

Line 485: typo/grammar: "restricts" instead of "restricted".

Meta-review: This paper investigates a phenomenon called "indiscriminate miscalibration" in LLMs during in-context learning, where models assign similar confidence levels to both correct and incorrect predictions. Motivated by this, the authors propose new metrics to measure this issue and introduce a label-free comparative inference method that includes unlabeled samples in prompts to improve calibration and classification performance. Two of the three reviewers raised concerns about limited experiments, lack of rigorous theoretical analysis, missing baselines and comparison on inference cost, and strongly recommended rejection. At the end of the rebuttal, the major concerns of the reviewers remained. Thus, this paper can not be accepted by ICLR in its current version.

Decision: Reject