Paper: Shifting Attention to Relevance: Towards the Uncertainty Estimation of Large Language Models (ICLR-2024)

Reviewer comment (Evaluation):
Although there is stable improvement over the baseline models, it seems marginal.

Author response:
Thank you for acknowledging our consistent improvements! We would like to present statistics demonstrating that the advancements in SAR are indeed substantial. Our paper includes approximately 21 experiments: 8 on pre-trained Large Language Models (LLMs), 8 on instruction-tuned LLMs, and 5 on medical datasets. We report the enhancements achieved by SAR in comparison to state-of-the-art methods (such as PE, LN-PE, SE), including average improvements, average improvements per setting, and the number of experiments in which SAR outperforms others by at least 2%.

| Method | Total Exp. Number | Avg. Improvement | # Exps with Improvement > 2% | Avg. Pre-trained | Avg. Instruction-tuned | Avg. Medical |
| ---- | :----: | :----: | :----: | :----: | :----: | :----: | 
| SAR v.s. PE | 16 | 6.1% | 15/16 | 7.2% | 4.9% | - | 
| SAR v.s. LN-PE| 21 | 5.4% | 19/21 | 3.3% | 7.9% | 4.6% | 
| SAR v.s. SE | 21 | 3.7% | 14/21 | 1.5% | 7.1% | 2.0% |



It is shown that our SAR outperforms state-of-the-art experiments with >2% margins for most experiments. Therefore we would like to mention that the improvement is rather substantial and notable.

Reviewer comment (Evaluation):
I have a minor question since I am not an expert in this area: why would the unimportant tokens pose a higher uncertainty? In Figure 1, it seems that the unimportant tokens have relatively less possibility to convey information since the model should have been pretty certain about these tokens in my understanding. However, as shown in Figure 3, this is actually the opposite since the token-level mean of UP is actually higher when the relevance is less. Could you please give an explanation for this?

Author response:
Thank you for your insightful comment! ***Why would the unimportant tokens pose a higher uncertainty*** is exactly the question that inspired our method. 

We believe this is due to the **intrinsic properties of LLMs**, such as data imbalance in training corpora. For example, if the LLM frequently encounters the phrase "according to" during training, the likelihood of the word "to" following it becomes very high (meaning the uncertainty of "to" is very low) when the preceding token is "according". If the LLM has never encountered "density of" (as shown in Fig. 1 of our paper), the probability of the word "of" following "density" is low (with high uncertainty). Thus, the certainty of LLMs regarding these tokens should be determined by their training. In Figs. 2 and 3, we demonstrate that many irrelevant tokens are assigned lower probabilities by LLMs, which we believe impedes accurate uncertainty estimation.

Our motivation is that important tokens should dominate uncertainty estimation, rather than less significant ones. To achieve this, we quantify the importance of tokens and refocus on these significant tokens during uncertainty quantification, leading to improved performance.


We hope these kindly address the reviewer's concerns. We would like to provide responses if the reviewer has further questions.

Reviewer comment (Experiments):
The sentSar and tokenSar methods are conceptually quite different, and don’t really make sense to package together (especially since while they are compared, sentSar requires more computation than tokenSar).

Author response:
Thank you for your insightful comment. We agree that sentSAR and tokenSAR differ conceptually, as they focus on distinct aspects of generation. However, as outlined in Sec. 5.2, they are complementary and can be integrated to address uncertainty quantification effectively at both the token and sentence levels. Consequently, we consider sentSAR and tokenSAR as two separate contributions of our paper. These distinctions are emphasized in our new revision (Sec. 4.3).

Reviewer comment (Evaluation):
The sentSar method is more computationally expensive than baselines.

Author response:
Thank you for your insightful comment. We agree that sentSAR is more computationally expensive than the baseline methods. However, it's important to highlight that sentSAR is more **generation-efficient**. It surpasses baseline methods under significantly smaller computational constraints. As demonstrated in Fig. 2, SAR, with just 2~3 generations, outperforms all other methods that use 10 generations.

We have quantified the time consumed for each step in the overall uncertainty quantification pipeline. This includes sequence generation, computing logits, semantic clustering for SE, and sentence similarity for sentSAR. We exclude the time taken for aggregating logits/scores as it is negligible (less than 0.001 second for all methods). The average time consumed per question, based on an evaluation of 1000 questions from the Vicuna-13b + SciQ dataset, is provided. These measurements were taken using an AMD EPYC 7302 16-Core CPU and a 1xA40 GPU server:

| Method | Number of Generations | Generation | Logits Computing | Semantic Clustering | Sentence Similarity |  sum |
| ---- | :----: | :----: | :----: | :----: |  :----: | :----: |
| PE | 5|4.09s|	1.19s|	0s|	0s|	5.28s|
| LN-PE | 5| 4.09s|	1.19s|	0s|	0s|	5.28s|
| SE | 5|4.09s|	1.19s|	1.5s|	0s|	6.78s|
| sentSAR | 5|4.09s|	1.19s|	0s|	2.58s|	7.86s|
| sentSAR | 2 |	1.64s|	0.48s|	0s|	0.52s|	**2.64s**|

We present two scenarios for our sentSAR: 1) It takes **7.86 seconds** when utilizing 5 generations; 2) The time reduces to **2.64 seconds** when using just **2 generations**, which is the minimum number of generations required for sentSAR.

Then we compare the 2-generations sentSAR with 5-generations baseline methods:

| Method | Number of Generations | Llama-2-13b-chat + SciQ/Trivia QA | Vicuna-13b + SciQ/Trivia QA| Vicuna-33b + SciQ/Trivia QA|WizardLM-13b + SciQ/Trivia QA|average|  
| ---- | :----: | :----: | :----: | :----: | :----: |:----:|
|PE|5|**0.718**/0.647|0.708/0.690|0.665/0.644|0.677/0.647|0.692/0.657|
|LN-PE|5|0.688/0.615|0.668/0.624|0.668/0.639|0.638/0.615|0.666/0.623|
|SE|5|0.692/0.622|0.675/0.630|0.674/0.651|0.649/0.634|0.673/0.634|
|sentSAR|**2**|0.716/**0.689**|**0.718**/**0.709**|**0.700**/**0.674**|**0.697**/**0.701**|**0.708**/**0.685**|

Our sentSAR still surpasses the baseline methods while consuming less than half the time, demonstrating its greater generation efficiency. We have included these results in the updated version of our paper (Appendix C.5).

Reviewer comment (Presentation):
The tables in the paper are very difficult to parse; the font size is small, and presentation could be improved.

Author response:
Thank you for pointing this out! We have modified the formats and font sizes of tables in our new revision.