Fair Enough: Standardizing Evaluation and Model Selection for Fairness Research in NLP (2023)

Abstract: Modern NLP systems exhibit a range of biases, which a growing literature on model debiasing attempts to correct. However, current progress is hampered by a plurality of definitions of bias, means of quantification, and oftentimes vague relation between debiasing algorithms and theoretical measures of bias. This paper seeks to clarify the current situation and plot a course for meaningful progress in fair learning, with two key contributions: (1) making clear inter-relations among the current gamut of methods, and their relation to fairness theory; and (2) addressing the practical problem of model selection, which involves a trade-off between fairness and accuracy and has led to systemic issues in fairness research. Putting them together, we make several recommendations to help shape future work.

## introduction in nlp and machine learning, there has been a surge of interest in fairness due to the fact that models often learn and amplify biases in the training dataset, leading to a range of harms @xcite . a central notion is group-wise fairness @xcite @xcite , which is typically measured as the model performance disparities across groups of data that are created by the combinations of protected attributes, such as race and gender. a broad range of bias evaluation metrics have been introduced in previous studies to capture different types of biases -such as demographic parity @xcite and equal opportunity (eo) @xcite -and different approaches have been adopted to both measure group disparities within each class, and aggregate over those disparities. each of these choices implicitly encodes assumptions about the nature of fairness, but @xmath0 denote the economic status (wealthy vs. not, respectively). the pattern of results exhibits various biases, however it is difficult to distil this into a single figure of merit, and thus determine which is the better or fairer of the two models. as an illustration of this issue, figure 1 depicts the true positive rate (tpr) values for two models. 2 given that the eo fairness is satisfied if different groups achieve identical tpr, which model is fairer or "better" out of the two? the answer is far from clear, and in terms of evaluation practice, dictated by a series of choices which implicitly encode different assumptions about what fairness is. in terms of research practice, these choices have led to a lack of consistency and direct empirical comparability between methods. equally concerningly, given that fairness research involves an inherent trade-off between raw model performance and fairness, it has more subtly led to a lack of rigour in terms of how model selection has been carried out, meaning that methods are often deployed in suboptimal ways relative to a particular evaluation methodology. in this paper, we seek to address these problems. we start by surveying current practices for fairness evaluation aggregation within an integrated framework, and discuss considerations and motivations for using different aggregation approaches. to ensure fairness metrics are fully comparable, we present a checklist for reporting fairness evaluation metrics, and also recommendations for aggregation method selections. we next survey model comparison methods, and demonstrate the issues stemming from using inconsistent model selection criteria. to ensure fair comparisons, we further introduce a metric for comparison without model selection, which measures the area under the trade-off curve of each method. overall this paper makes two key contributions: (1) we characterise current practices for fairness evaluation and their grounding in theory, proposing a best-practice checklist; and (2) we propose a new method which resolves several issues relating to model selection and comparison. ## related work in terms of bias metrics, there are mainly two lines of work in the literature on nlp fairness: bias in the geometry of text representations (intrinsic bias), and performance disparities across groups in downstream tasks (extrinsic bias), respectively. based on the hypothesis that measuring and mitigating intrinsic bias will also reduce extrinsic bias, previous work has mainly focused on measuring and mitigating intrinsic bias, such as the word embedding association test (weat) @xcite , sentence encoder association test (seat) @xcite , and embedding coherence test (ect) @xcite . however, @xcite recently showed that there is no reliable correlation between intrinsic and extrinsic biases, and suggest future work focusing on extrinsic bias measurement (which is the focus of this work). as for bias mitigation, debiasing methods for intrinsic and extrinsic bias generally suffer from performance-fairness trade-offs controlled by particular hyperparameters such as the number of principal components used to define the intrinsic bias subspace @xcite , and the strength of addition objectives for performance parity across groups @xcite . in measuring perfor-mance (perplexity and lm score for sentence embeddings, for example) and fairness simultaneously, the model comparison framework presented in this paper is generalizable for both intrinsic and extrinsic fairness. ## fairness metrics in this section, we discuss the considerations involved in fairness evaluation. we start with a survey of different methods for aggregating scores, and propose a two-step aggregation framework for fairness evaluation. ## model comparison this section focuses on comparison of debiasing methods when considering utility and fairness simultaneously. we first introduce the performancefairness trade-off curve (pfc) for debiasing methods, and then discuss the limitations of existing comparison frameworks. finally, we propose a new metric, namely the area under the curve (auc) w.r.t. pfc, which integrates existing approaches and reflects the overall goodness of a method. ## conclusion we have discussed the current practice in evaluation, model selection, and method comparison in the fairness literature, and shown how current practice in experimental fairness lacks rigour and consistency. we made recommendations for selecting a fairness evaluation metric, and introduced a new metric for measuring the overall performancefairness trade-off of a method. we only investigate the proposed evaluation aggregation framework in a classification setting. however, our framework is naturally generalizable to other tasks with discrete outcomes, such as generation and sequential tagging. moreover, in terms of continuous labels, such as regression, one can skip the class-wise aggregation.