FairMT-Bench: Benchmarking Fairness for Multi-turn Dialogue in Conversational LLMs

Authors: Zhiting Fan, Ruizhe Chen, Tianxiang Hu, Zuozhu Liu
Primary area: datasets and benchmarks
Keywords: Fairness, Benchmark, Large language model

Abstract:
The increasing deployment of large language model (LLM)-based chatbots has raised concerns regarding fairness. Fairness issues in LLMs may result in serious consequences, such as bias amplification, discrimination, and harm to minority groups. Many efforts are dedicated to evaluating and mitigating biases in LLMs. However, existing fairness benchmarks mainly focus on single-turn dialogues, while multi-turn scenarios, which better reflect real-world conversations, pose greater challenges due to conversational complexity and risk for bias accumulation. In this paper, we introduce a comprehensive benchmark for fairness of LLMs in multi-turn scenarios, **FairMT-Bench**. Specifically, We propose a task taxonomy to evaluate fairness of LLMs cross three stages: context understanding, interaction fairness, and fairness trade-offs, each comprising two tasks. To ensure coverage of diverse bias types and attributes, our multi-turn dialogue dataset FairMT-10K is constructed by integrating data from established fairness benchmarks. For evaluation, we employ GPT-4 along with bias classifiers like Llama-Guard-3, and human annotators to ensure robustness. Our experiments and analysis on FairMT-10K reveal that in multi-turn dialogue scenarios, LLMs are more prone to generating biased responses, showing significant variation in performance across different tasks and models. Based on these findings, we develop a more challenging dataset, FairMT-1K, and test 15 current state-of-the-art (SOTA) LLMs on this dataset. The results highlight the current state of fairness in LLMs and demonstrate the value of this benchmark for evaluating fairness of LLMs in more realistic multi-turn dialogue contexts. This underscores the need for future works to enhance LLM fairness and incorporate FairMT-1K in such efforts. Our code and dataset are available at https://github.com/FanZT6/FairMT-bench.

Decision: accept (spotlight)
Reviewer scores: 8, 6, 8, 6

Reviews and discussion:
<thread>
creation date:
1729845175279

thread_type:
official_review

<thread_object>
creator: reviewer_t586

<content>
summary:
This work presents the first fairness benchmark Fair-MT  for multi-turn dialogues because prior works can only consider single-turn scenarios.

This benchmark includes two bias types (Stereotype and Toxicity) and six bias attributes (gender, race, religion, race, etc.) and up to 10K items.  The authors first collected source data from RedditBias, SCIB, and HateXplain.  Then, the authors use GPT-4 to construct the dataset for the identified six tasks ( 2 tasks per LLM capacity). Finally, the authors mainly use GPT-4 to evaluate the fairness of LLM candidates.

strengths:
1. This work has a well-defined task taxonomy, which is clear and comprehensive.
2. Unlike previous single-turn works, this work presents the first multi-turn fairness benchmark. Multi-turn has more practical significance and research value.
3. Six multi-turn dialogue tasks are proposed to detect the fine-grained fairness of LLMs.
4. LLMs can automatically accomplish most tasks related to dataset construction and evaluations. 
5. Very detailed evaluation using the proposed FairMT-10K.
6. A more efficient FairMT-1K is also proposed.

weaknesses:
1. This work has carefully designed two tasks for each LLM capability.   Although each task has a considerable amount of space to introduce, there is a lack of specific formal definitions for each task in the main paper. 

2. Continuing from the first point, the main paper of this work has ignored many necessary details in the main paper in the remaining parts. 

3.  This work does not involve enough human annotation in both the dataset construction stage and the evaluation stage.  There is only a small-scale human-annotation test in the Appendix.

4. LLMs play the role of both the dataset constructor and the evaluation referee. Is this configuration really convincing?

5. This work only focuses on the evaluation.  The author needs a more detailed discussion on what improvement directions can be brought to the subsequent researches through experimental findings.

questions:
Questions:

1. See my weaknesses 4.

Minor Issues:

1. Line 130:  three stages of interaction with users`: :`
2. Line 732:  The prompt given to GPT-4 is shown in Figure `??`
</content>
</thread_object>

</thread>
<thread>
creation date:
1730601668043

thread_type:
official_review

<thread_object>
creator: reviewer_z3a4

<content>
summary:
This work highlights the motivation of studying fairness in multi-turn dialogue scenarios and pinpoint the current scarcity of relevant research and resources in this domain. To address these limitations, the authors construct a comprehensive multi-turn dialogue benchmark to evaluate LLM fairness capabilities across two bias types and six bias attributes. Through detailed experiments and analysis, the paper reveal fairness shortcomings in current LLMs.

strengths:
1. **Valuable Resources**
This paper first presents a fairness benchmark in multi-turn dialogue scenarios, covering diverse bias types and attributes.
2. **Extensive Experiments**
Conduct comprehensive experiments on current SOTA LLMs across six designed tasks.
3. **Reliable Evaluation**
Use GPT4 as a Judge, alongside bias classifiers including Llama3-Guard-3 and human validation.
4. **Comprehensive Analysis**
Analyze evaluation results of single-turn and multi-turn dialogue across different models, tasks, and groups.
5. **Valuable Insights** 
Reveal two distinct bias defense mechanisms for current LLMs, which target defense implicit biases and explicit biases respectively.

weaknesses:
**Ambiguous Task Taxonomy**
In Section 3.2, two taxonomies about fairness tasks are primarily discussed: comprehension-focused tasks VS. bias-resistance tasks (Line 318-320) and implicit biases VS. explicit biases (Line 322-323). These taxonomies are clear and reasonable. However, the taxonomy outlined in section 2.1 lacks clarity and mention. The naming of "interaction fairness" class is somewhat confusing, and the boundaries between this class and the other two are not clearly defined.

questions:
**Consider Maintaining Consistency Between the Taxonomy Definition Section and the Analysis Section**
In my opinion, the taxonomy outlined in Section 2.1 is less clear than the two taxonomies presented in Section 3.2. I suggest adopting a more straightforward and understandable classification for tasks definitions. If modifying the definitions is not feasible, at the very least, please ensure consistency between the classifications used in the definition and analysis sections.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730902370188

thread_type:
official_review

<thread_object>
creator: reviewer_abze

<content>
summary:
This paper introduces "FairMT-Bench," a comprehensive benchmark to evaluate fairness in LLMs, specifically in multi-turn dialogue scenarios. Addressing a gap in fairness assessments, which have primarily focused on single-turn interactions, the paper provides a dataset (FairMT-10K) and tasks that target fairness across three stages of dialogue complexity: context understanding, user interaction, and instruction trade-offs. The authors also present a distilled, challenging subset, FairMT-1K, and use both GPT-4 and Llama-Guard-3 classifiers, alongside human validation, to benchmark fairness in 15 LLMs, revealing substantial limitations in current model fairness across multi-turn interactions.

strengths:
1. Novel Focus on Multi-Turn Fairness Evaluation: The paper addresses the crucial gap of multi-turn dialogue fairness, reflecting real-world complexities in conversational AI use cases.
2. FairMT-Bench and its datasets (FairMT-10K and FairMT-1K) cover a wide array of bias types and attributes, providing a rich resource for fairness research.
3. By evaluating 15 prominent LLMs, the paper provides a robust, comparative analysis of model fairness, offering valuable insights for future LLM alignment improvements.
3. Employing both GPT-4 and Llama-Guard-3 as fairness evaluators, along with detailed experimental setups, enhances reproducibility and contributes essential tools for future fairness research.

weaknesses:
1. While the multi-turn focus is novel, the evaluation method largely depends on established LLM tools (e.g., GPT-4 as a judge), which may limit innovation in developing new fairness detection methodologies.
2. The paper does not thoroughly explore why certain attributes (like gender and race) showed consistently poor performance across models, missing an opportunity to deepen the community’s understanding of these biases.
3. Relying heavily on GPT-4 for generating synthetic dialogue data could introduce bias from the generative model itself, which might impact the generalizability of FairMT-10K and FairMT-1K to real-world scenarios.

questions:
1. How do you ensure that biases from GPT-4, which is used to generate and evaluate responses, do not affect the fairness outcomes, particularly given GPT-4's role as both data generator and evaluator?
2. Can you clarify if the performance discrepancies across bias attributes are due to data distribution differences or model-specific limitations?
3. Lack of comparison with existing evaluation, such as S-Eval or more recent fair-related work.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730969154425

thread_type:
official_review

<thread_object>
creator: reviewer_vp1k

<content>
summary:
This paper proposes a fairness benchmark designed for multi-turn dialogues, called FairMT-Bench. Then, it detailed the experiments and analysis with multiple SOTA LLMs across dimensions like tasks, dialogue turns, bias types and attributes. It carefully analyzed the results and pointed out the areas where LLMs fail to maintain a stable fairness. For example, LLMs tend to perform worse on fairness in multi-turn dialogues than single turn. Moreover, the paper proposes a more challenging fairness evaluation dataset, FairMT-1K, that contains examples LLMs perform worst on.

strengths:
1. The paper contributes a novel fairness benchmark specifically for multi-turn dialogues, while current benchmarks primarily focus on single-turn dialogues.

2. The paper extensively benchmarks on most popular LLMs, and provides detailed results and analysis across many dimensions, like tasks, dialogue turns, bias types and attributes. The paper comprehensively demonstrates each LLM's performance, and pinpoints the areas where fairness is challenging to LLMs. The results show that fairness, especially in multi-turn dialogues, is still a challenging task for LLMs.

3. The paper has great presentation. The figures are very illustrative and insightful.

weaknesses:
1. A few factors can make the evaluation computationally expensive: (1) using GPT-4 as the evaluator (2) the multi-turn nature of the data and the evaluation process (3) the data size. It would be great if the paper can include some discussion on evaluation cost.

2. While the paper discusses diverse sources and dimensions of bias, it does not discuss potential mitigation strategies. Offering even preliminary solutions or suggestions for future research directions would be valuable.

3. As fairness and quality can be contradictory metrics, models that perform well on fairness might sacrifice its quality. Therefore, it'd be better to add a model quality dimension in the analysis, for a more comprehensive and insightful comparison of the models.

4. The distribution of different tasks needs more clarification. I couldn't find the numbers of each task in the paper. Could the authors point me to the numbers if they are present in the paper, or add this information if they are absent?

questions:
1. When constructing the more challenging subset (FairMT 1K), the six models may diverge in scores. So how did you use the model results to select the examples?

2. Typos:
  1) Line 368, should be "multi-turn dialogues generally show higher bias rates"?
  2) Figure 4, (a) and (b) look the same, which I assume is not intended?

3. As mentioned above, could the authors clarify the number of examples allocated to each task in the FairMT-10K dataset?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732513933836

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Global Reponse

comment:
Dear reviewers,

We would like to express our heartfelt gratitude for your invaluable time, expertise, and meticulous attention in reviewing our manuscript. The insightful comments and constructive feedback have immensely enriched the quality and rigor of our work.

We appreciate that the reviewers acknowledge the advantages of our work: "**contributes a novel fairness benchmark specifically for multi-turn dialogues**" (reviewer vp1K), "**addresses the crucial gap of multi-turn dialogue fairness**" (reviewer aBzE), "**conduct comprehensive experiments and analysis**" (reviewer z3a4), "**has a well-defined task taxonomy, which is clear and comprehensive**" (reviewer t586).
On the other hand, we have diligently addressed all the issues by conducting extensive experiments to support our responses. Revised experiments and clarifications have been incorporated into the main manuscript while supplementary experiments have been included in the appendix, with the modified contents highlighted in **red**.

Allow me to summarize the significant updates made in both the rebuttal and the revised manuscript:
- **Addition of Human Validation**: Since we employed GPT-4 to achieve fully automated evaluation, we needed to validate the feasibility and accuracy of using GPT-4 as an evaluator. In the original version, we used LlamaGuard-3 and a small amount of manual annotation to verify GPT-4's evaluation results. Based on the reviewers' suggestions, we increased the scale of human validation by resampling 1,000 instances for manual annotation.
- **Supplementation of LLMs' Multi-Turn Dialogue Capability Evaluation**: To enhance the completeness of the evaluation framework and delve deeper into the factors influencing fairness in multi-turn dialogues, we added discussions on the quality of models in multi-turn dialogues. The models were evaluated based on MT-Bench 101 and their over-rejection rates on the FairMT Bench.
- **Addition of Bias Mitigation Attempts in Multi-Turn Dialogue Scenarios**: Based on our evaluation results and the generated outputs during model testing, we constructed a simple multi-turn DPO dataset for fairness alignment and conducted bias mitigation attempts using Llama-2-7b-chat.
- **Avoiding Inappropriate Content**: In the new version, we have reviewed the illustrations and examples in our case studies, replacing some overly sensitive cases that could easily cause discomfort. We aim to minimize the potential discomfort experienced by groups with various social attributes when reading our article.
- **Further Explanation of GPT-4's Role in Data Generation and Evaluation**: We elaborated on how GPT-4 was rigorously utilized to assist in data generation and evaluation within FairMT-Bench. This addresses the reviewers' concerns about the potential biases and limitations of the GPT-4 model impacting the evaluation framework.

We sincerely appreciate your time and valuable insights. 

Warm regards,

The Authors
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper introduces FairMT-Bench, a comprehensive fairness benchmark for LLMs in multi-turn dialogues, addressing the limitations of existing single-turn benchmarks. The reviewers appreciate the focus of the paper on fairness, commenting that it addresses an important and significant gap, the comprehensive evaluation and analysis, as well as the value in the datasets provided. They also raise concerns, primarily on over-reliance on LLMs and the bias and cost this might incur, and some limitations in the evaluation, analysis, and discussion. The authors do a good job in providing detailed responses to all concerns and questions, including additional analyses and human annotations.