Large Language Models Often Say One Thing and Do Another

Authors: Ruoxi Xu, Hongyu Lin, Xianpei Han, Jia Zheng, Weixiang Zhou, Le Sun, Yingfei Sun
Primary area: datasets and benchmarks
Keywords: corpus creation, benchmarking, language resources, nlp datasets, evaluation, metrics, consistency

Abstract:
As large language models (LLMs) increasingly become central to various applications and interact with diverse user populations, ensuring their reliable and consistent performance is becoming more important. This paper explores a critical issue in assessing the reliability of LLMs: the consistency between their words and deeds. To quantitatively explore this consistency, we developed a novel evaluation benchmark called the Words and Deeds Consistency Test (WDCT). The benchmark establishes a strict correspondence between word-based and deed-based questions across different domains, including opinion vs. action, non-ethical value vs. action, ethical value vs. action, and theory vs. application. The evaluation results reveal a widespread inconsistency between words and deeds across different LLMs and domains. Subsequently, we conducted experiments with either word alignment or deed alignment to observe their impact on the other aspect. The experimental results indicate that alignment only on words or deeds poorly and unpredictably influences the other aspect. This supports our hypothesis that the underlying knowledge guiding LLMs' word or deed choices is not contained within a unified space. Dataset and code are available at [https://github.com/icip-cas/Word-Deed-Consistency-Test](https://github.com/icip-cas/Word-Deed-Consistency-Test).

Decision: accept (poster)
Reviewer scores: 8, 8, 6, 3

Reviews and discussion:
<thread>
creation date:
1729105339357

thread_type:
official_review

<thread_object>
creator: reviewer_8ncb

<content>
summary:
This paper introduces an evaluation benchmark to assess the consistency between the verbal outputs of language models and fact-based realities that goes by the Words and Deeds Consistency Test (WDCT).

This benchmark ensures the correspondence between word-based and fact-based questions in various domains. Evaluations with the GPT, Llama and Mistral-based models revealed weaknesses, potentially due to model tuning problems.

This research highlights the importance of assessing the reliability of LLMs by measuring the alignment between words and actions.

strengths:
The contribution has its merits. A detailed list follows:

- The writing is clear, fluent and understandable. However, the organisation of the introduction should definitely be reconsidered (comment on weak points)

- The authors address an important issue in evaluating words and deeds using LLMs.

- The datasets introduced in the paper include several domain topics to ensure better generalisability (however, see weaknesses).

The habitual evaluation of LLMs is very extensive and detailed.

weaknesses:
Although the contribution has its myths, there are many points that need to be clarified and improved:

**Introduction**

- RQs are described (this is a good thing). However, these should be dealt with in the course of the contribution and perhaps discursively introduced in the intro. The ‘answers’ delivered by the authors in the intro to lines 49,53,73 should be revised (this is not a note on the content but on the style ‘To answer...’ is not a proper formula to be repeated three times).

- The summary of contributions on line 82 makes sense but why use line 49 to line 77 to answer the RQs and then repeat lines 82-90?

**Section 2 and 3**

- The experimental setting is described in a sufficient and comprehensible manner, even if it is sometimes difficult to follow the discourse due to the positioning of the figures.

- The evaluation is made on a limited dataset (1325 examples). This is not a major problem, but it does undermine the scientific soundness of the paper. For example, look at the well-known QA tasks (SIQA, PIQA) or the mathematicians (GSM8K). all of these have significantly more instances than 1k. It would be interesting to observe these phenomena on a larger scale (e.g. 4k).

- The construction of the examples could be biased by the constructor (an LLM constructs, to be precise does augmentation, of the examples on which it is prompted) 


**Section 4 and 5**

The fact that the authors decided to organise the section into Discoveries and Discovery Discussions is not wrong as many papers do this. However, I do not think it is the best way. Indeed, the contribution shows a lot of analysis but it is really limiting to organise the presentation of results in this way.

Furthermore, it is difficult to read the comments of the experiments in the findings. For example, see ‘Exp. 1’ on line 326. I understand that the authors wanted to make the work more discursive, but they should pay more attention to style. An example could be constructing subsections by structuring the discourse by placing the RQ next to the answer. The title of the section should describe the analysis (e.g. line 334 should be a subsection called ‘inconsistency’ or something similar).



I would strongly advise the authors to revise the structure of the paper as in this condition it is really a pity to leave it as it is.

questions:
- What is the role of the consistency score? As a non-multiple-choices question answer, your score should be able to capture all possible generations. Did you test and or ablation test for this?

- How much impact do systemic demonstrations in SFT have on DPO? 

- What are the limits of the transferability of your study? What if another LLM was used instead of GPT-4 to build the samples?
</content>
</thread_object>

</thread>
<thread>
creation date:
1729754631768

thread_type:
official_review

<thread_object>
creator: reviewer_vdjr

<content>
summary:
1. This work proposes a novel benchmark called the WDCT to measure the inconsistency between words and actions. This benchmark contains test pairs of word-based and action-based questions across various domains to measure the alignment between models' statements and their behaviors.

2. The paper introduces the issue of large language models (LLMs) being inconsistent between what they say (words) and what they do (deeds). This inconsistency is observed across different domains such as opinions, non-ethical values, and theories. The authors argue that this gap reduces trust in LLMs.

3. The authors introduce two metrics, the Consistency Score (CS) and the Probability Consistency Score (PCS), to evaluate the alignment between words and deeds. These metrics help quantify how well models maintain consistency across different tasks.

4. Conducted extensive experiments on multiple state-of-the-art LLMs, including GPT-4, Vicuna, and LLaMA, to evaluate their consistency using the WDCT. The results show a significant misalignment between words and deeds across models and domains, particularly in non-ethical value contexts.

I am willing to increase my score during the rebuttal phase if the authors address my concerns.

strengths:
1. The paper introduces two new metrics, Consistency Score (CS) and Probability Consistency Score (PCS), which provide a quantitative way to measure how well LLMs maintain consistency between their words and deeds. These metrics offer valuable tools for evaluating model alignment.

2. Conduct thorough experiments on various LLMs, including GPT-4, Vicuna, and LLaMA, across various domains like opinions, non-ethical values, and theory. The experiments reveal that all the tested models exhibit the issue of inconsistency between what they say and what they do across all the domains.

3. This work shows that commonly used methods such as COT, alignment and paraphrasing do not solve this issue.

weaknesses:
1. The paper looks a little bit rough in places. For example, Table 4 and Figure 8 are poorly formatted, extending beyond the usual layout boundaries, which affects readability and professionalism.

2. While the paper effectively identifies the issue of inconsistency between words and deeds in LLMs, it does not provide any in-depth analysis or reasoning behind why this inconsistency occurs. This leaves a gap in understanding the reason cause of the problem.

3.  Although some alignment techniques are explored, they are not shown to fully resolve the inconsistency, and the paper does not provide clear insights for overcoming this challenge.

4. The results rely on somewhat outdated models like LLaMA 2. Given that more advanced versions like LLaMA 3 and LLaMA 3.1 are now available, it would have been better to test these newer models to assess whether they exhibit similar issues or show improvements in consistency.

5. This work lacks ablation studies. Like studying which components of the models or alignment techniques contribute most to the word-action inconsistency. This would help clarify whether the problem is primarily due to data, model architecture, or training techniques.

questions:
1. In Table 3, both LLaMA 2 and Mistral base models show relatively low average Consistency Scores (CS), whereas ChatGLM3 base has a noticeably higher CS. Does this suggest some fundamental differences in the architecture, training methodology, or other factors that set ChatGLM3 apart from the other two models?

2. In Figure 5, the consistency rate for the second epoch suddenly drops from over 50% in epoch 1 to 0% in epoch 2. Is there anything unusual about this drastic drop? Could it be related to the specific alignment process, or does this indicate a possible issue in the model's training or evaluation setup?

3.  Given that LLaMA 2 is somewhat outdated, would it be possible to try the newer LLaMA 3.1 Instruct model? It would be interesting to see if the consistency issues observed in LLaMA 2 are still present in LLaMA 3.1 instruct.

4. Have you analyzed how sensitive the consistency scores are to the prompts?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730367045901

thread_type:
official_review

<thread_object>
creator: reviewer_erdy

<content>
summary:
The paper addresses a critical issue within LLMs — the inconsistency between their expressed words (statements or opinions) and their actions (behavioral outputs). To quantify this inconsistency across various domains, the authors introduce a novel evaluation benchmark, the Words and Deeds Consistency Test (WDCT). Through the WDCT, the study reveals substantial discrepancies between what LLMs say and do, emphasizing the challenges in achieving consistent alignment, especially when alignment is performed only on words or deeds independently. The paper makes a valuable contribution by suggesting a new direction for alignment research in LLMs.

strengths:
- Relevance and Importance:
The problem tackled is highly relevant. As LLMs are increasingly embedded in real-world applications where consistency between communication and behavior is essential, identifying and addressing this issue has meaningful implications for enhancing user trust and model reliability, particularly as LLMs are now frequently deployed as autonomous agents.

- Novel Evaluation Benchmark (WDCT):
The WDCT is a significant contribution, offering a well-structured approach to measure alignment between words and deeds across multiple domains (e.g., opinion, ethical values, theory). This benchmark provides a robust foundation for future alignment studies in LLMs.

- Systematic Evaluation Experiment:
The authors pose three clear research questions and provide a detailed experimental framework to evaluate whether separate alignment or common knowledge generalization methods could improve consistency between words and deeds.

weaknesses:
- Ambiguity in the Underlying Hypothesis:
While the paper proposes that the inconsistency arises due to the separation of knowledge guiding words and deeds, this hypothesis remains underexplored. The authors could strengthen the paper by providing more in-depth analysis or empirical evidence that supports this claim. For example, conducting case studies to trace the model's reasoning paths, could illustrate where the divergence occurs.

questions:
I have no questions for the authors.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730939332760

thread_type:
official_review

<thread_object>
creator: reviewer_c63h

<content>
summary:
This paper addresses the issue of inconsistencies in the texts generated by LLMs, formalized as the difference between the LLMs' "words" (i.e., replies to direct questions about the LLMs' views) and "deeds" (i.e., replies to questions about the preferred action in situations that manifest the LLMs' views). The authors create a new benchmark of matched word and deed questions and use it to evaluate a series of LLMs. They find evidence for rampant inconsistencies between the LLMs' words and deeds.

strengths:
The problem that the paper addresses (i.e., inconsistencies between the LLMs' words and deeds) has been studied before, but a better understanding of the underlying issues is still lacking. In this context, the paper proposes a very valuable resource, both for future research on inconsistencies in LLMs and for evaluating the values encoded in LLMs. The analyses that the authors conduct using their benchmark are methodologically sound; the authors did a great job in testing the robustness of their findings.

weaknesses:
The biggest weakness of the paper is that it completely ignores prior work on inconsistencies between the words and deeds of LLMs, claiming that it is "the first [study] to systematically evaluate the consistency of responses from prominent LLMs based on the words and deeds" (p. 10); while prior studies do not use the terms "words" and "deeds", they are still very similar in terms of motivation and overall findings, so I think it is critical that the authors add a comprehensive discussion and comparison before the paper can be published. There are two lines of work that I deem particularly relevant:

1) There has been a lot of work on the inconsistencies in the political values that LLMs express when asked in a multiple-choice setup ("words") versus more real-world use cases ("deeds"). For example, [Röttger et al. (2024)](https://aclanthology.org/2024.acl-long.816/) compare the behavior of LLMs when directly asked for their political values, and the values they implicitly support when asked to write texts about political issues, finding inconsistencies very similar to the ones described in this paper. There have been various other papers looking at the same question (e.g., [Moore et al., 2024](https://arxiv.org/abs/2407.02996)), none of which the authors cite.

2) The difference between the LLMs' "words" and "deeds" is related to the distinction between the implicit and explicit behavior of LLMs (e.g., [Bai et al., 2024](https://arxiv.org/abs/2402.04105), [Hofmann et al., 2024](https://www.nature.com/articles/s41586-024-07856-5)) -- another line of work that the authors should acknowledge. For example, Hofmann et al. (2024) show that the divergence between the explicit and the implicit behavior of LLMs is bigger for aligned than for unaligned models, which is exactly in line with the findings that the authors present (p. 7).

questions:
Minor points:
- Table 1, caption: "a aligned -> "an aligned"
- Table 3: The abbreviation "IFT" is never introduced. Did you mean "SFT"?
- 421-422: "to generalize to generalize" -> "to generalize"
- Section 4.2, 2: I think Figure 5 does not clearly support your claim. For example, the bar for D2 is as high as the one for S1 with both models. You either need to provide a more detailed analysis or weaken your claim.
- Table 5: What is shown in the first column?
</content>
</thread_object>

</thread>
<thread>
creation date:
1733286916897

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Official Comment by Authors

comment:
We thank all reviewers for the feedback!

**In this post**:

(1) We summarize positive things from the reviews.

(2) We summarize the changes to the updated PDF document.

**In the individual replies**, we address other comments.

### (1) Positive things

- **Significant research questions**
  - ```eRDy```: *The problem tackled is highly relevant ... as autonomous agents.*
  - ```8NCb```: *The authors address an important issue in evaluating words and deeds using LLMs.*

- **Valueable and novel benchmark**
  - ```C63h```: *the paper proposes a very valuable resource, both for future research on inconsistencies in LLMs and for evaluating the values encoded in LLMs.*
  - ```eRDy```: *The WDCT is a significant contribution, offering a well-structured approach to measure alignment between words and deeds across multiple domains (e.g., opinion, ethical values, theory). This benchmark provides a robust foundation for future alignment studies in LLMs.*
  - ```vDJr```: *This work proposes a novel benchmark called the WDCT to measure the inconsistency between words and actions.* and *The paper introduces two new metrics, ... offer valuable tools for evaluating model alignment.*
  - ```8NCb```: *The datasets introduced in the paper include several domain topics to ensure better generalisability.*

- **Extensive, detailed and sound analysis**
  - ```C63h```: *The analyses that the authors conduct using their benchmark are methodologically sound.*
  - ```eRDy```: *The authors pose three clear research questions and provide a detailed experimental framework to evaluate whether separate alignment or common knowledge generalization methods could improve consistency between words and deeds.*
  - ```8NCb```: *The habitual evaluation of LLMs is very extensive and detailed.*

- **Comprehensive robust testing**
  - ```C63h```: *the authors did a great job in testing the robustness of their findings.*

- **Presentation**
  - ```8NCb```: *The writing is clear, fluent and understandable.*

### (2) Major changes to the PDF

- Writing

  - Updated the introduction of the research question and its answer along with the contribution from different angles in section 1.

  - Updated the presentation of section 4, placing conclusions after research questions. The conclusions are organized into main conclusions (conclusion 1-3) and sub-conclusions (finding 1-6) for ease of comprehension.

  - A more comprehensive discussion on the related works in section 6.

  - Corrected the layout of tables and figures.

- Experiments

  - Results of more recent, advanced models, the LLaMA-3-8B and LLaMA-3-8B-Instruct in Table 3.

  - A data augmentation method by automatically generating aligned data in section 4.3.2.

  - An analysis of the correlation between the data size and evaluation results in Appendix B.1.

  - An analysis of the impact of LLM constructors.

  - An analysis of the sensitivity of the consistency scores to the prompts.

  - An analysis of the root causes of word-action inconsistency.

Once the page limit is confirmed, we will include the further experiments in the final paper.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper proposes a new benchmark Words and Deeds Consistency Test (WDCT) to examine LLM consistency between word-based (direct query about opinion) and deed-based (response to a situation) scenarios. Reviewers generally agree that the benchmark is timely and exposes a interesting model behavior. Extensive analysis are conducted in the paper to stud this phenomenon. The authors should add more explicit and clearer discussion about how this benchmarks tests a notion of consistency that is fundamentally different from those in prior work (e.g. consistency under prompt variance, etc.)