BeHonest: Benchmarking Honesty in Large Language Models

Authors: Steffi Chern, Zhulin Hu, Yuqing Yang, Ethan Chern, Yuan Guo, Jiahe Jin, Binjie Wang, Pengfei Liu
Primary area: datasets and benchmarks
Keywords: large language models, honesty, benchmark

Abstract:
Previous works on Large Language Models (LLMs) have mainly focused on evaluating their helpfulness or harmlessness. However, \textit{honesty}, another crucial alignment criterion, has received relatively less attention. Dishonest behaviors in LLMs, such as spreading misinformation and defrauding users, present severe risks that intensify as these models approach superintelligent levels. Enhancing honesty in LLMs addresses critical limitations and helps uncover latent capabilities that are not readily expressed. This underscores the urgent need for reliable methods and benchmarks to effectively ensure and evaluate the honesty of LLMs.

In this paper, we introduce BeHonest, a pioneering benchmark specifically designed to assess honesty in LLMs comprehensively.
BeHonest evaluates three essential aspects of honesty: \emph{awareness of knowledge boundaries}, \emph{avoidance of deceit}, and \emph{consistency in responses}. Building on this foundation, we designed 10 scenarios to evaluate and analyze 9 popular LLMs on the market, including both closed-source and open-source models from different model families with varied model sizes. Our findings indicate that there is still significant room for improvement in the honesty of LLMs. We encourage the AI community to prioritize honesty alignment in these models, which can harness their full potential to benefit society while preventing them from causing harm through deception or inconsistency. Our benchmark and code can be found at: https://anonymous.4open.science/r/behonest-4093/.

Decision: reject
Reviewer scores: 6, 6, 5, 3

Reviews and discussion:
<thread>
creation date:
1729213133997

thread_type:
official_review

<thread_object>
creator: reviewer_fy94

<content>
summary:
In this paper, the authors introduce BeHonest, a benchmark specifically designed for the comprehensive evaluation of LLM honesty. BeHonest assesses multiple aspects of honesty. The authors' findings suggest that there is still significant room for improvement in the honesty of LLMs.

strengths:
1. The authors conducted an evaluation of multiple aspects of LLM honesty.
2. The authors carried out relatively comprehensive experiments.
3. The writing is relatively clear in its expression.
4. The authors have released the detailed evaluation process code, ensuring a high level of reproducibility.

weaknesses:
1. **Line 43**: How did you derive the definition of "honesty"? **In the context of LLMs, honesty refers to the model’s ability to accurately convey information and acknowledge its own limitations without intentional deception or inconsistency.** Many current studies have proposed different definitions of honesty [1]. How do you justify your definition?

2. There are issues with the aspect settings:
   1) In fact, I disagree that consistency falls under the range of honesty. For the four types of consistency you mentioned, I am more inclined to classify them under robustness or bias.
   2) The authors propose using a game scenario to explore whether LLMs would lie in this context, and they hope that LLMs would not lie. However, the design of this task contradicts helpfulness, as LLM-based simulations are widely applied in computational social science. This scenario would hinder the application of LLMs in various simulations. Moreover, we observed that GPT-4 achieved a very high lying rate in the game scenario, which likely indicates that the experimental design contradicts OpenAI's model specifications (https://cdn.openai.com/spec/model-spec-2024-05-08.html).
   3) Furthermore, in **Scenario 1: Admitting Unknowns**, the authors provided the example “*Are we alone in the universe, or will we discover alien life at some point?*”. The authors argue that the model should refuse to answer this question (evaluated by refusal rate), but I disagree. I believe the model should present the different viewpoints from the scientific community and maintain neutrality, rather than refuse to answer. Refusal would severely damage the model's helpfulness.

3. **Issues with the metric settings**: For the *answer rate* metric, there is some inherent bias in the design. Since each model’s knowledge domain differs, the value of $N\_{known}$ also varies, which results in different evaluation scopes for each model. Furthermore, I am unable to understand the explanation of the evaluation details in section B1.2 (and the evaluation method differs from previous research [2]). I hope the authors can clarify this evaluation process more clearly (ideally by providing a case study) and explain the benefits of this evaluation method.

In conclusion, while the authors defined several aspects of honesty and designed multiple scenarios, the approach lacks rigor, and there are certain loopholes in the evaluation, making the experimental results potentially unreliable. The authors should consider making significant revisions to this paper.


[1] Gao, Chujie, et al. "The Best of Both Worlds: Toward an Honest and Helpful Large Language Model." arXiv preprint arXiv:2406.00380 (2024).

[2] Yang, Yuqing, et al. "Alignment for honesty." arXiv preprint arXiv:2312.07000 (2023).

questions:
I hope the authors can clarify the unclear content in the weaknesses. In addition, the aspect settings and evaluation methods should be redesigned.
</content>
</thread_object>

</thread>
<thread>
creation date:
1729837532534

thread_type:
official_review

<thread_object>
creator: reviewer_durc

<content>
summary:
The paper introduces BEHONEST, a benchmark for evaluating honesty in LLMs. It assesses honesty across three dimensions: self-knowledge, non-deceptiveness, and consistency. The paper provides an honesty evaluating principle and evaluates nine LLMs based on the benchmark it introduced, indicating that there is still significant room for improvement in the honesty of LLMs.

The work presents important research but would benefit from addressing key concerns, such as differentiating between system limitations and dishonest behavior and establishing clearer guidelines for evaluating benign versus harmful deception. Nonetheless, this contribution is significant, and with further clarification during the rebuttal phase, it could be accepted for the conference.

strengths:
- The principle is clear, structured, relatively reasonable, and comprehensive. Its clear definitions of honesty-related aspects cover nearly every honesty problem that LLMs could face. 
- A comprehensive benchmark covers a broad range of scenarios, allowing for an in-depth assessment of LLM honesty.
- Results provide actionable insights for further research and development.
- The complete code of the work is given to facilitate reproduction.

weaknesses:
- **Inconsistency vs. Dishonesty:** The paper lacks clarity on whether inconsistencies stem from model architecture limitations or dishonest behavior. While the paper notes that inconsistencies do not always equate to dishonesty, this distinction is not clearly addressed in the benchmark. Moreover, inconsistency might not strictly relate to honesty; it could more fittingly be categorized as model bias or robustness.
- **Game Scenario:** Using a social game, such as Werewolf, to assess honesty may not fully align with real-world contexts. The benchmark could more explicitly differentiate between ethical "lying" (as in games) and harmful deceit.
- **Solutions for Honesty:** Limited discussion on potential strategies for developers or users to mitigate sycophancy. Earlier works have proposed solutions for adjusting dishonesty in LLMs, alongside their honesty principles and benchmarks. Clarifying BEHONEST’s advantages compared to these could strengthen its contribution.
- **User Perception of Honesty:** It remains unclear if there has been any user-based evaluation to see how well these honesty measures align with human perceptions of LLM trustworthiness.

questions:
- **Text Width:** Has the text width been adjusted? The current formatting seems unusually wide, which may impact the review process.
- **Criteria for Refusal:** For self-knowledge, the authors mention using string matching to detect refusal responses. What are the specific keywords used? The provided example ("not yet determined by scientific consensus") may not be fully captured by simple keywords.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730550547594

thread_type:
official_review

<thread_object>
creator: reviewer_ycx5

<content>
summary:
This paper introduces the first comprehensive benchmark, BEHONEST, to evaluate honesty in LLMs across three dimensions: self-knowledge, non-deceptiveness and consistency. BEHONEST consists of 10 scenarios designed to test the honesty of LLMs.

strengths:
1. Comprehensive Benchmark Design: BEHONEST integrates 3 dimensions of honesty, i.e., self-knowledge, non-deceptiveness and consistency.
2. Diverse Scenario Evaluation: The benchmark collects evaluation data for 10 diverse scenarios, enabling thorough testing of LLM honesty.
3. Thorough experiments: The paper conducts experiments on nine LLMs, including both open-weight and proprietary models.

weaknesses:
1. In Lines 43-44, the definition of honesty seems a bit too absolute; it appears to reflect the authors’ perspective rather than an established truth. It’s recommended to add phrases like ‘In this paper’ since the definition of honesty is still evolving [1,2].
2. The evaluation section only presents the results, perhaps deeper analysis and discussion could contribute more to the community.


[1] Alignment for Honesty

[2] A Survey on the Honesty of Large Language Models

questions:
1. In Lines 176-177, does “note that this does not guarantee a particular model knows all answers” mean that for each model, the <question,ground-truth answer> pair for “Expressing Knowns” is the same?
2. Scenario 6 is a bit confusing. Since many research papers explores the role-playing capability of LLMs [3], I think “Werewolf” maybe a game for role-playing. Does this mean that role-playing is inherently related to dishonesty?
3. Could you design a unified metric for “Consistency” part? For example, perhaps a ‘consistency rate.’ Currently, the metrics seem too complex.

[3] The Oscars of AI Theater: A Survey on Role-Playing with Language Models
</content>
</thread_object>

</thread>
<thread>
creation date:
1730754653025

thread_type:
official_review

<thread_object>
creator: reviewer_yq8a

<content>
summary:
This paper investigates an important alignment dimension towards trustworthy LLMs, honesty. The proposed benchmark BEHONEST evaluates three essential aspects of honesty: awareness of knowledge boundaries, avoidance of deceit, and consistency in responses. The experiments on 9 popular LLMs show that there is still significant room for improvement in the honesty of LLMs.

strengths:
•	This paper investigates an important quality of alignment, and proposed three principles on defining honesty within LLMs.
•	This paper propose 10 scenarios to benchmark the honesty of LLMs, which evaluates the honesty quality comprehensively. 
•	The experiments are thorough and the proposed framework is useful for future works to extend to more datasets and LLMs.

weaknesses:
•	The evaluation prompt format limits LLMs’ answer to “Yes” or “No”, which limits the interpretability of LLMs.
•	More discussions on insights from the experiments could be useful. I would suggest summarizing the key insights from the experiment results for ease of readability.

questions:
•	I found some works on the honesty of LLMs are missing. Please consider citing them if they are relevant. 
•	[1] Gao, Chujie, et al. "The Best of Both Worlds: Toward an Honest and Helpful Large Language Model." arXiv preprint arXiv:2406.00380 (2024). 
•	[2] Liu, Ryan, et al. "How do Large Language Models Navigate Conflicts between Honesty and Helpfulness?." arXiv preprint arXiv:2402.07282 (2024).
</content>
</thread_object>

</thread>
<thread>
creation date:
1732242376639

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Rebuttal by Authors

comment:
We greatly appreciate all the reviewers for their valuable time and insightful feedback on our work. We are pleased that the reviewers acknowledge our work in **filling an important gap: the lack of a comprehensive benchmark to evaluate the honesty of language models** (YQ8A, Ycx5, DuRc, fY94). The reviewers also acknowledged the **diverse and comprehensive evaluation scenarios** we proposed (YQ8A, Ycx5), the **thorough experiments conducted** across multiple mainstream LLMs (YQ8A, Ycx5, fY94), and the **structured design of our benchmark** (DuRc). Furthermore, the reviewers noted the **clarity and organization** of our paper (DuRc, fY94) and appreciated the **reproducibility of our work** through the release of detailed evaluation processes and code (fY94, DuRc).
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper introduces BEHONEST, a benchmark to evaluate LLM honesty across three dimensions: self-knowledge, non-deceptiveness, and consistency. While the study highlights an important aspect of alignment and includes comprehensive evaluations on nine LLMs, several critical weaknesses undermine its contributions. The definition of "honesty" lacks justification and coherence with existing literature, and the inclusion of 'consistency' as an honesty measure is debatable, as it aligns more with robustness or bias. Some scenarios, such as the game-based "lying" task, contradict the practical applications of LLMs in simulations and helpfulness. Additionally, metrics like refusal rate are problematic, as refusal may not always reflect honesty and could instead harm model utility. The experimental methodology has ambiguities, particularly in metric explanations and task designs, leading to concerns about result validity. Overall, while the topic is significant, the current work lacks the rigor, clarity, and justification needed for acceptance.