BERTS core is Unfair: On Social Bias in Language Model-Based Metrics for Text Generation (2022)

Abstract: Automatic evaluation metrics are crucial to the development of generative systems. In recent years, pre-trained language model (PLM) based metrics, such as BERTScore, have been commonly adopted in various generation tasks. However, it has been demonstrated that PLMs encode a range of stereotypical societal biases, leading to a concern about the fairness of PLMs as metrics. To that end, this work presents the first systematic study on the social bias in PLM-based metrics. We demonstrate that popular PLM-based metrics exhibit significantly higher social bias than traditional metrics on 6 sensitive attributes, namely race, gender, religion, physical appearance, age, and socioeconomic status. In-depth analysis suggests that choosing paradigms (matching, regression, or generation) of the metric has a greater impact on fairness than choosing PLMs. In addition, we develop debiasing adapters that are injected into PLM layers, mitigating bias in PLM-based metrics while retaining high performance for evaluating text generation.

## introduction in text generation tasks, for example machine translation, text summarization, and caption generation, automatic evaluation metrics are widely adopted for model selection. typically, the goal of the metrics is to evaluate the semantic equivalence between system-generated texts and golden references. traditional metrics such as bleu @xcite and rouge @xcite are usually based on n-gram matching, regardless of the semantic similarity. in recent years, pre-trained language models (plms) @xcite @xcite ; qiu et al., 2020) have been exploited for evaluating text generation. in contrast to traditional metrics that merely consider surface-form similarity, plm-based metrics such as bertscore @xcite and bartscore @xcite can well capture the semantic similarity between system outputs and references, and therefore achieve higher correlation with human judgements. currently, plm-based metrics have been widely adopted by researchers and developers in a variety of text generation tasks. although these plm-based metrics have been well studied from many perspectives such as robustness @xcite and efficiency @xcite , the fairness of these metrics has not yet been investigated. the fairness of the text generation metrics has a crucial impact on developing generative systems. if the metric is biased against some sensitive attributes (e.g., gender), generative models that express such bias will be rewarded and selected. the texts generated by these biased models may be incorporated in the corpus, further reinforcing the reference: the manager has more money than the attendant since the manager is more senior ∆: -26.56 ∆: -23.48 ∆: -10.17 ∆: -30.23 ∆: -8.69 candidate 1: the manager has more money than the attendant since she is more senior 44.44 56.43 63.75 39.35 76.69 candidate 2: the manager has more money than the attendant since he is more senior 71.00 79.91 73.92 69.58 85.38 hence, there is a natural concern that to what extent do these plm-based metrics carry social bias? in this work, we present the first systematic study of social bias in plm-based metrics for text generation. most existing metrics measure the quality of model-generated candidate texts by comparing with human-annotated references. ideally, a fair metric should assign a set of candidates the same score if the only difference between them is a few words indicating some sensitive attribute (e.g., gender). to evaluate whether and to what extent existing metrics can hold such a property, we construct datasets for 6 sensitive attributes, i.e., race, gender, religion, physical appearance, age, and socioeconomic status. each dataset is consisting of paired examples. in each pair of examples, denoted as ⟨(sys 1 , ref), (sys 2 , ref)⟩, one contains a candidate that demonstrates a stereotype (e.g., sys 1 ) and the other contains a candidate that violates the stereotype (e.g., sys 2 ). the reference that does not carry any stereotype is shared by the pair. some examples to measure gender bias are listed in in addition, we also explore mitigating social bias in plm-based metrics by training debiasing adapters @xcite attached to the plms. without touching parameters of the plms, our approach significantly reduces bias while maintaining high performance for evaluating text generation. foot_0 2 measuring social bias in @xcite metrics for text generation ## conclusion in this paper, we present a systematic study on the social bias in plm-based metrics for text generation, which have been widely adopted in a variety of tasks. as a result, we demonstrate that popular plm-based metrics exhibit significant bias on 6 sensitive attributes. through in-depth analysis, we shed some light on the impact of different factors (e.g., modeling paradigms, plms, etc.) on metric bias. in addition, we explore mitigating metric bias by replacing the backbone plms with debiased ones, and by injecting debiasing adapters. experimental results show that the both approaches can significantly reduce bias while retaining high performance for evaluating text generation.