Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper is trying to produce a better measure of entropy for NLG tasks where sentences with same semantics can have really high entropy in the conventional methods as they rely on superficial measures from the exact token or sequence of tokens. The same sentence can be paraphrased in many ways and hence be equivalent to the "target". 

The paper proposes a model based entropy measurement where output with same semantics would have low entropy. And this technique doesn't need any retraining.

The crux of the idea is 
1. generate all the sequences based on the distribution p(s/x) ; s- sequence, x- input
2. After N sequences are computed they are clustered using an off the shelf Langauge models to produce NLI classes. Using that measure we can now cluster the sequences into C classes
3. Compute entropy SE(x) across all classes, where each class's likelihood is Sum(p(s_i/x)) Such that s_i belongs to c


They use AUROC to measure the performance of the uncertainty measurement scheme. 
strength_and_weaknesses: Strengths
- The high level idea is reasonable and beats other methods on 2 benchmark datasets
- With increase in number of samples the semantic entropy method seems to provide better auroc
- Sampling of sequence is a critical component and the paper addresses those concerns in detail.

Weakness
- The performance is shown only on two datasets. 
- The author claim it is not expensive in the context of 1.5B / 30B param models. But uncertainty prediction task which adjacent to the main task of NLG, can be quite expensive if we consider smaller, production ready nlg models.
- The performance of the uncertainty prediction depends on the performance on the NLI task. And given that the clusters are formed using equivalence measure using an imperfect NLI model it would lead to clusters that are much larger indicating 
clarity,_quality,_novelty_and_reproducibility: - The work is mathematically well grounded. From first principles approach it makes sense to consider the semantic as the good measure of uncertainity
- The paper also empirically shows the hypothesis holds water
- This paper should be reproducible as they use all publicly available off the shelf models.

summary_of_the_review: The paper proposes a well grounded idea to compute uncertainty in the space of NLG where the modality (natural language) of the predicted sequence allows more than one way of representing the right meaning. In language there are many ways to convey the same semantics and the NLG models would tend to distribute their output across sequences where semantics match. 

So in this paper the authors propose to use entropy at the level of semantic-class rather individual queries as a measure of uncertainty. The method proves itself to be better that others. 

There are some concerns around the scalability / compute this would need. And concerns around how this measure depends on the performance of the NLI model itself. And the experiments were run on just 2 datasets restricted to QA. 

Nonetheless the technique seems to produce good results and paves way for future work that could address these issues.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes an approach for uncertainty estimation in free-form text generation called "semantic entropy". The method proposed in this paper is unsupervised and can directly be applied to off-the-shelf language models. Semantic entropy provides a better uncertainty estimation than standard entropy, scales better with number of samples and performs better for model self-evaluation. This paper also provides an in depth explanation of uncertainty in free-form text generation. The method proposed first clusters sequences that mean the same thing based on bi-directional NLI. Then the average likelihoods is used to estimate uncertainty over different meanings. 
strength_and_weaknesses: Strengths:
* This paper is very well written and has very clearly defined motivation.
* The claims made in this paper are well supported with experiments and analysis.
* The design choices have been properly justified. 
* The approach is "decently obvious" and simple and I'm surprised it has not been done before.

Weaknesses and Questions:
* The last line of section 2 should have a proper citation ("While promising, these approaches need task-specific labels, additional training, and "seem to be unreliable out-of-distribution"). I don't think this work performed experiments to verify / claim this.
* The theory that semantic equivalence can be operationalized using bi-directional entailment should be grounded in linguistic theory. Please add more details about relevant linguistic work that talks about this to motivate your usage of bi-directional entailment to mean semantic equivalence more.
* In section 4.2 it is mentioned that "A sequence is part of a semantic equivalence class if it shows bidirectional equivalence with any other member of that class". Q1) Why any other? Why not a more stricter definition of all or more than half matched? However, in the discussion on computation cost just below and the pseudocode, it seems that only one sequence is used to judge if the given sequence lies in the same semantic class. Q2) Wont you need to compare with all? If you are testing if any one of them matches? Can you clarify this a bit?
* 3rd last sentence in "Computation cost" section under 4.2 is incomplete. Consequently, what is "this" in the last line (last word in 4.2)?
* Why do you use multinomial sampling? Did you try any other approach to sample?
* It is claimed in 4.4 that this method "goes some way towards addressing unequal token importance". Are there any experiments to support that (I might have missed)?
* Any insight into why (Figure 4) normalized entropy performs better than the proposed semantic entropy approach at high temperatures?

Grammar and Other suggestions:
* Be consistent in your usage of UK vs US English. Some instances in the paper have "summarization" while others have "summarisation".
* Please add links in references of tables and figures. Capitalize "Table". etc.
* Table 1 caption should be at the top of the table.
* Para 3 of 4.2 - 3rd line - Use citep instead of citet.
* Figure 2 legend "P(True) -> p(True)"

clarity,_quality,_novelty_and_reproducibility: The paper is clearly written. The quality of descriptions, justifications and support for the claims is high. The work proposes a simple, yet novel approach for measuring uncertainty in generated text. The work does seem reproducible since it doesn't need any model training and runs on off-the-shelf LLMs in an unsupervised manner. However, code was not provided as supplement so I am unable to claim this. 
summary_of_the_review: The paper proposes a novel method to perform uncertainty estimation in model generated free-form text by introducing a concept of semantic entropy. The paper is very well written with clear justification for design choices and adequately supported claims and has only minor questions and clarifications.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The authors propose a method to calculate uncertainty of language models over generated text that is conditioned on some input context. They propose to estimate "semantic entropy" i.e., entropy over clusters of semantically close groups of sampled text. The motivation behind this approach is that entropy-based metrics of uncertainty for natural language generation tasks typically consider uncertainty over tokens/token sequences without taking semantic equivalence of these into account. The authors propose to use bidirectional entailment as a way to confirm whether two sampled sentences are semantically equivalent and use this information to generate classes of semantically equivalent samples. They then calculate the entropy over these samples. They apply this method on two question answering datasets and demonstrate competitive performance compared to other methods.
strength_and_weaknesses: The motivation for this approach is clear and well established, and the authors show that the semantic entropy can help achieve better uncertainty estimates for question answering datasets. The perform ablation tests regarding the sample size and language model size and justify well their modelling choices for the problem at hand. I enjoyed reading the paper however, I am missing a few things:

1. A better discussion of related work. The authors seem to disregard a considerable amount of work on uncertainty, unless there is a reason the focus was moved away from this work I would expect some more discussion and comparisons. As it is it is a bit harder to fully appreciate the impact of this method and place it in the space of uncertainty work. See my more detailed comment about related work at the end.
2. Some more discussion and ablation tests on the impact of generated sentence length on the computational cost and performance of the method would be nice. The datasets chosen require short sentences or single word answers, but the authors mention summarisation as a potential next step but it is unclear how the proposed semantic equivalence solution as a metric of semantic relatedness can be applied to text spans that are more complex than subject-verb—object sentences. The manual evaluation of the entailment identification method showed good performance for the datasets in question but it would be nice to see how efficient it is on longer/more complex text. As it is, I am worried that the entailment method would be significantly less accurate if applied in different scenarios.

On related work and comparisons with other work:
The mention of work in the area of uncertainty for regression is a bit outdated, especially in terms of mentioning work on "regression in low-dimensional data spaces". There are more recent works that address high-dimensional regression problems (focussing on text) such as: 
-Glushkova, T., Zerva, C., Rei, R., & Martins, A. F. (2021). Uncertainty-aware machine translation evaluation. arXiv preprint arXiv:2109.06352.
-Wang, Y., Beck, D., Baldwin, T., & Verspoor, K. (2022). Uncertainty estimation and reduction of pre-trained models for text regression. Transactions of the Association for Computational Linguistics, 10, 680-696.
-Malinin, A., Chervontsev, S., Provilkov, I., & Gales, M. (2020). Regression prior networks. arXiv preprint arXiv:2006.11590

It would be important to see how the proposed method compares to these approaches instead.

Additionally, work in direct epistemic uncertainty prediction and perhaps older work in evidential deep learning should also be discussed: 
- Jain, Moksh, Salem Lahlou, Hadi Nekoei, Victor Butoi, Paul Bertin, Jarrid Rector-Brooks, Maksym Korablyov, and Yoshua Bengio. "Deup: Direct epistemic uncertainty prediction." arXiv preprint arXiv:2102.08501 (2021).

On another note, I am wondering if the authors considered the lexical variability estimate that has been proposed as an uncertainty “glass-box” feature for machine translation models. The implementation is different, but I think the underlying principle is similar: sample a few sentences conditioning on the same context (in MT this is the source sentence, and they use MC dropout to sample different sentences) and then estimate the semantic diversity. The way the authors of the reviewed paper estimate semantic diversity and conceptualise eating is better established for the problem at hand, but I think it is worth comparing the two approaches and I am wondering whether the simpler approach of calculating similarity between sampled sentences yields good results as well.
Reference: Fomicheva, M., Sun, S., Yankovskaya, L., Blain, F., Guzmán, F., Fishel, M., ... & Specia, L. (2020). Unsupervised Quality Estimation for Neural Machine Translation. Transactions of the Association for Computational Linguistics, 8, 539-555.


clarity,_quality,_novelty_and_reproducibility: The paper is clear and overall easy to read. It is well written, but some useful information is spread over the document and it would be better described early on in one place (e.g. the details of  the sampling approach).

To the best of my knowledge this method has not been proposed before. However, as I point out in the previous section, the concept of assessing the variability of generated text (sampled from a language model conditioned on some input) has been used before as a feature to estimate uncertainty in NMT models. While the proposal for the estimation of semantic uncertainty in this work is different, I would expect at least some mention, and potentially even some comparison. If semantic entropy is a strong uncertainty predictor, wouldn't simple calculating the (dis)similarity of generated sentences be a strong baseline?

If authors release the code and manually annotated samples as promised (along with indications of seeds, model versions and hyperparameters chosen), it should be possible to reproduce the results with some effort.
summary_of_the_review: Well motivated and interesting work, that requires a bit more thorough analysis of related work. I still have some questions regarding applicability to other tasks where longer/more complex sentences are expected.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes semantic uncertainty, a new uncertainty estimation metric that operates on the meaning space of natural language sentences. Semantic uncertainty accounts for the invariance of the meaning of sentences against surface syntactic or linguistic styles. Formally, semantic uncertainty of an input context is defined as the conditional entropy of the distribution over clusters of meanings conditioned on the context, and can be estimated using a model’s samples. To cluster samples into meaning groups, a bidirectional entailment model is used. Experiments are conducted on two datasets, CoQA and TrivialQA, which shows that the proposed uncertainty estimator has better correlation with the model error rate compared to some existing approaches.

strength_and_weaknesses: Strength:
* Uncertainty estimation for generation results from large language models is an important research avenue towards improving the trustworthiness of those models.
* The motivation of measuring NLG outputs in the meaning space is quite reasonable
Experimental results showed that the proposed semantic uncertainty metric outperforms other uncertain estimation methods that operate on the surface form space of sentences instead of the meaning space, although I am not sure how to interpret the gain versus baselines using the AUC metric.

Weaknesses:
* **[Significance of Results]** Judging from AUC numbers on Figure 2 it is difficult to intuitively understand how much the proposed method is improved over the baseline, especially given the fact that results from the second-best methods on Figure 2 (a) and (b) are quite close to the proposed method. The authors also stated in Table 2 that a simple approach of counting the number of semantically distinct clusters performs quite competitively already in terms of AUROC. Does this suggest that AUROC is not a good evaluation metric since it lacks enough “resolution” to distinguish between different methods? Also, why isn’t the counting method included in Figure 2?

Could the authors consider more direct approaches to evaluate uncertainty metrics? For example, showing end-to-end answer accuracy of a decision tree: answer is from OPT if the model is certain on the input $x$ ($SE(x)$ is low), otherwise, use an oracle answer.

* **[Correctness Metric]** From Section 6, the authors used $1_{rougeL(s, s’) > 0.3}$ (typo in the original equation that uses $<0.3$?) as the metric that evaluates if a prediction 
$s’$ is correct under a reference $s$. Rouge-L is a very brittle metric and doesn’t perform well when measuring semantic equivalence of short phrases. Also, the threshold of $0.3$ is chosen without further justification. Does Rouge-L and this particular threshold correlates well with human judgment of answer correctness? 

* **[Models for Sentence Clustering]** The authors used bidirectional entailment as the metric to determine semantic equivalence between pairs of sentences. This task, to my knowledge, is often referred to as paraphrase identification in NLP. There is no mention of this relevant topic in section 4.2, and SoTA models for paraphrase identification literature are not used for sentence clustering in this work.

* **[Related Work]** Could the authors comment on the relationship of their work with existing research on improving generations of neural sequence-to-sequence models based on models’ uncertainty estimation (e.g., Lin et al., 2022b)? This paper is not cited.  In addition, the proposed uncertainty estimation metric in Lin et al., 2022b is not used as baseline in this paper. 

`Lin et al., 2022b`: Lin, Zi, Jeremiah Zhe Liu, and Jingbo Shang. "Towards Collaborative Neural-Symbolic Graph Semantic Parsing via Uncertainty." In Findings of the Association for Computational Linguistics: ACL 2022, pp. 4160-4173. 2022.


clarity,_quality,_novelty_and_reproducibility: Nit: Sub-captions in table 1 are a bit misleading — it should mean two scenarios where there are semantically equivalent predicted answers or not.

summary_of_the_review: Interesting idea on measuring semantic uncertainty.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Summary: 
The paper proposes an approach called semantic entropy, which incorporates linguistic invariances for uncertainty estimation in NLG. The proposed unsupervised method demonstrates competitive performance compared to other methods on CoQA and TrivialQA tasks. 

Strength:
1. Uncertainty estimation for generation results is an important area for improving trustworthiness of models. 
2. The paper is clearly written, well motivated and interesting.
3. Claims made in this paper are well supported with experiments and analysis

Weakness:
1. Most of the concerns were addressed properly by the authors’ replies. 
2. It could be useful to evaluate on more than two QA datasets, as well as larger models. 



justification_for_why_not_higher_score: [Minor - authors explained this]: It could be more impressive to evaluate the work on a wider range of tasks (two QA tasks in the current work). 
justification_for_why_not_lower_score: Uncertainty estimation for generation results is an important area for improving trustworthiness of large language models. Three reviewers think it is a good paper (clearly written, well motivated, interesting, well supported claims).
Decision: Accept: notable-top-25%