CEB: Compositional Evaluation Benchmark for Fairness in Large Language Models

Authors: Song Wang, Peng Wang, Tong Zhou, Yushun Dong, Zhen Tan, Jundong Li
Primary area: datasets and benchmarks
Keywords: Fairness, Bias, Benchmark, Large Language Models

Abstract:
As Large Language Models (LLMs) are increasingly deployed to handle various natural language processing (NLP) tasks, concerns regarding the potential negative societal impacts of LLM-generated content have also arisen. To evaluate the biases exhibited by LLMs, researchers have recently proposed a variety of datasets. However, existing bias evaluation efforts often focus on only a particular type of bias and employ inconsistent evaluation metrics, leading to difficulties in comparison across different datasets and LLMs. To address these limitations, we collect a variety of datasets designed for the bias evaluation of LLMs, and further propose CEB, a Compositional Evaluation Bechmark that covers different types of bias across different social groups and tasks. The curation of CEB is based on our newly proposed compositional taxonomy, which characterizes each dataset from three dimensions: bias types, social groups, and tasks. By combining the three dimensions, we develop a comprehensive evaluation strategy for the bias in LLMs. Our experiments demonstrate that the levels of bias vary across these dimensions, thereby providing guidance for the development of specific bias mitigation methods.

Decision: accept (spotlight)
Reviewer scores: 8, 8, 6, 8

Reviews and discussion:
<thread>
creation date:
1730581392527

thread_type:
official_review

<thread_object>
creator: reviewer_epfe

<content>
summary:
This paper introduces a framework to standardize the evaluation of existing bias datasets. It also contributes additional datasets (by modifying existing datasets) consisting of 11,004 samples, collectively known as the Compositional Evaluation Benchmark (CEB), based on the proposed framework. The framework covers 5 tasks - Recognition, Selection, Continuation, Conversation, Classification - and 2 bias types - Stereotyping and Toxicity. They also perform experiments to evaluate biases across both open source (llama2, llama3, mistral) and closed source (GPT-3.5, GPT-4) LLMs using the unified framework and metrics.

strengths:
1. This work is an important step towards standardizing existing bias datasets, making it easier to evaluate biases for existing and future LLMs.
2. They cover a wide variety of bias datasets - StereoSet, RedditBias, HolisticBias, etc and add new datasets to fill in the gaps in their framework, making the work comprehensive enough in terms of the selected social groups.
3. The paper is well written in general and sections 6.3 and 6.4 include interesting insights on LLM biases based on their experiments.

weaknesses:
1. The work is comprehensive in terms of utilizing existing datasets but leaves out other important demographic attributes like physical appearance, nationality, socio-economic status (SES), sexual orientation, political ideologies, etc. Since this study is meant to eliminate limitations due to particular types of biases, including just 4 social groups is insufficient.
2. Despite its importance, this study lacks the element of novelty since the ideas used for consolidating datasets is straightforward. Even the construction of new datasets is based on refactoring existing datasets by prompting GPT-4, instead of coming up with fresh diverse samples to fill in the gaps.

questions:
1. The “Recognition” task only highlights LLMs understanding of bias and does not seem to point towards inherent biases in the LLM. Why is it included as part of Direct Evaluation?
2. Are the samples generated using LLMs for CEB-Continuation and CEB-Conversation validated by human experts to ensure quality?
3. In Table 3, we can see significant gaps between human and GPT-4 evaluation for GPT-4 as model, half the score in some cases. How can you conclude that it is aligned, what criteria or threshold are you using?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730658952792

thread_type:
official_review

<thread_object>
creator: reviewer_5f7m

<content>
summary:
The paper introduces the "Compositional Evaluation Benchmark" (CEB), designed to evaluate fairness in Large Language Models (LLMs). CEB incorporates 11,004 samples across varied datasets, using a novel compositional taxonomy based on bias types (stereotype and toxicity), social groups (age, gender, race, religion), and tasks (recognition, selection, continuation, conversation, classification). The benchmark seeks to address limitations in current bias evaluation methods by providing a comprehensive and unified framework, allowing for standardized comparisons across datasets. The benchmark includes experimental evaluations across multiple LLMs, and the findings guide model developers in mitigating biases.

strengths:
1. The paper is well-motivated and addresses an important problem in fairness evaluation. The compositional taxonomy provides a systematic way to characterize and compare different bias evaluation datasets, helping unify previously fragmented evaluation approaches.

2. The paper constructs a novel dataset to fill gaps in the evaluation space, particularly around the previously underserved toxicity evaluation.

3. The paper employs LLM-as-Judge in the evaluation. In particular, it uses GPT-4 and Perspective API for bias and toxicity scoring, which enables automated, scalable bias evaluation, addressing challenges with human labelling and allowing comparisons across large datasets​.

weaknesses:
1. While the paper acknowledges this limitation, focusing on only four social groups (age, gender, race, religion) leaves out other important dimensions like disability, socioeconomic status, etc. More importantly, the paper does not take intersectional fairness/gerrymandering fairness into consideration. 

2. The performance of GPT-4 and Perspective API as evaluators is not well justified. (1) in line 350, the stereotype metric is described as a score between 0-100. However, the details provided in the appendix indicate that the score is in the range between 0-99. (2) The evaluator prompt mainly specifies three categories: 0-39, 40-79 and 80-99. In this case, it is hard to differentiate scores within the same range, i.e., what is the expected difference between 50 and 60? (3) The justification for the evaluator's performance needs to be better supported by statistics.

3. DP, EO, and Unfairness Score are stated to be metrics for classification evaluation. However, in the main paper, these numbers are not presented.

questions:
- (Line 412) "Humans are generally aligned with GPT-4 in terms of evaluation performance in most cases.": what do you mean by generally aligned?
- (Line 420) "Nevertheless, the gap is small enough to indicate that GPT-4 is generally unbiased in its assessments of its own content.": what do you mean by "the gap is small enough"? Has any hypothesis testing been performed to support this argument?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730662080402

thread_type:
official_review

<thread_object>
creator: reviewer_ywrs

<content>
summary:
The paper proposes a unified representation of different, widely available, fairness benchmarks. Unifying them around three dimensions: bias type, social group and tasks. In addition, it identifies dimension where there is currently no coverage by existing datasets and provides means to synthetically generate data to cover those dimensions. 

This is a very useful approach that allows to benchmark datasets on larger datasets. 
I would mention that a lot of what I would consider valuable results and discussions were moved to appendix due to page limitation.

strengths:
Provides a good basis to merge or compose a fairness benchmark from multiple other existing benchmarks, unifying them around several dimensions.
Provides a repeatable approach to generate additional data synthetically, either for the dimensions that were of shortage, as identified by the authors but I believe also beyond that.

weaknesses:
A significant number of text, including results, discussions, evaluations were moved to appendix. I would suggest to submitting extended version to a conference that has larger page limit (or alternatively to arxiv).

questions:
How easy would it be to expend this approach of unification of datasets to additional dimensions?
Typo on page 6, line 321 (Jiasaw --> Jigsaw)
</content>
</thread_object>

</thread>
<thread>
creation date:
1730699928348

thread_type:
official_review

<thread_object>
creator: reviewer_2gcz

<content>
summary:
This paper proposes a comprehensive benchmark for bias and fairness in large language models. The authors first propose a multi-layers taxonomy that contains bias types, social groups and tasks. They assign configurations to existing datasets and identify numerous configurations that remain underexplored. Subsequently, they construct new evaluation datasets based on existing datasets, including BBQ and HolisticBias. Finally, They conduct evaluations on these newly created datasets across various llms, revealing some findings regarding their performance and biases.

strengths:
- The taxonomy proposed by this paper is comprehensive and highly compatible with existing datasets
- The newly created datasets address previously unexplored configuration 
- A comprehensive evaluation of bias for various llms have been conducted, and the comparison between GPT4 and human annotators confirms the effectiveness of the GPT4's scoring.

weaknesses:
- Lack of the validation of datasets content generated by GPT4.
  - For example, when asking GPT4 to add toxic content, how to ensure that GPT4 will follow the instruction to generate toxic content and how to judge the content is really harmful.
- The rubrics for bias scores in CEB-Continuation-S and CEB-Conversation-S (Sec C.2.2) appear too "coarse", As indicated in the results shown in Table 7, most of scores falls between 0-39 that means no stereotypes in the content. So what is the difference between a score of 10 and a score of 30.

questions:
- The questions outlined in point 1 of Weaknesses.
- Could you provide two examples with different scores, both falling within the range of 0 to 39? Additionally, please explain the distinction between these examples.(Corresponding point 2 of Weaknesses)
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper presents a comprehensive benchmark to evaluate bias in LLMs through a novel multi-dimensional taxonomy. The reviewers agree on the paper's strengths, its ability to unify previously fragmented evaluation approaches (Reviewer YWrs), and its contribution to standardizing bias dataset evaluation (Reviewer EPfe). The benchmark's unique features include a taxonomy covering 11,004 samples across bias types (stereotype and toxicity), social groups (age, gender, race, religion), and various tasks, which provides a valuable tool for model developers to understand and mitigate biases (Reviewer 2gcZ, 5f7m).

Despite the overall positive reception, reviewers identified several weaknesses that warrant consideration. The primary concerns include the limited scope of social groups examined, with multiple reviewers noting the absence of important demographic attributes such as disability, socioeconomic status, sexual orientation, and political ideologies (Reviewers 5f7m and EPfe). Additionally, there are methodological concerns about the evaluation process, particularly regarding the GPT-4 and Perspective API bias scoring methods. Reviewers highlighted issues with the scoring rubrics, such as the coarse categorization of bias scores (Reviewer 2gcZ) and the lack of clear differentiation between scores within the same range (Reviewer 5f7m). The paper's relatively straightforward approach to dataset consolidation and the reliance on GPT-4 for generating additional samples were also noted as potential limitations (Reviewer EPfe).