Bipol: Multi-Axes Evaluation of Bias with Explainability in Benchmark Datasets (2023)

Abstract: We investigate five English NLP benchmark datasets (on the superGLUE leaderboard) and two Swedish datasets for bias, along multiple axes. The datasets are the following: Boolean Question (Boolq), CommitmentBank (CB), Winograd Schema Challenge (WSC), Winogender diagnostic (AXg), Recognising Textual Entailment (RTE), Swedish CB, and SWEDN. Bias can be harmful and it is known to be common in data, which ML models learn from. In order to mitigate bias in data, it is crucial to be able to estimate it objectively. We use bipol, a novel multi-axes bias metric with explainability, to estimate and explain how much bias exists in these datasets. Multilingual, multi-axes bias evaluation is not very common. Hence, we also contribute a new, large Swedish bias-labelled dataset (of 2 million samples), translated from the English version and train the SotA mT5 model on it. In addition, we contribute new multi-axes lexica for bias detection in Swedish. We make the codes, model, and new dataset publicly available.

## introduction recent advances in artificial intelligence (ai), large language models (llm), and chatbots have raised concerns about their potential risks to humanity @xcite @xcite . 1 one major concern is the issue of social bias, particularly with the data ai models are trained on. bias, which can be harmful, is the unfair prejudice in favor of or against a thing, person or group (maddox, 2004; dhamala et al., 2021; mehrabi et al., 2021; antoniak and mimno, 1 bbc.com/news/world-us-canada-65452940 2021). measuring bias in text data can be challenging because of the axes that may be involved (e.g. religious or gender bias). in this work, our motivation is to determine whether social bias exists in nlp benchmark datasets and estimate it. after reviewing some potential bias methods, as discussed in section 2, we settled for the recent bipol @xcite because of its advantages. it is a metric that estimates bias along multiple axes in text data and provides an explanation for its scores, unlike other metrics. we investigate social bias in benchmark datasets that are available on the english super-glue leaderboard and two swedish datasets. the superglue was introduced by @xcite and provides benchmark datasets for different nlp tasks. benchmark datasets are datasets for comparing the performance of algorithms for specific use-cases @xcite . such datasets have been the foundation for some of the significant advancements in the field @xcite . we investigate the following english datasets: boolq @xcite , cb @xcite , wsc @xcite , axg @xcite , and rte @xcite . the swedish datasets are the overlim cb and swedn. we discuss more about the datasets in section 3.2. our contributions firstly, we show quantitatively and through explainability that bias exists in the datasets. the findings correlate with characteristics of bias, such as heavy lopsidedness @xcite . this work will provide researchers with insight into how to mitigate bias in text data and possibly add impetus to the conversation on whether it is even ethical to remove these social biases from data, because they represent the real world. secondly, we create and release, possibly, the largest labeled dataset and lexica for bias de-tection in swedish (multi-axes bias dataset (mab)-swedish) and train a model based on the state-ofthe-art (sota) multilingual t5 (mt5) @xcite . we release our codes, dataset and artefacts publicly. 2 the rest of this paper is structured as follows. section 2 discusses some of the previous related work. section 3 describes the methodology, including details of the characteristics of bipol and the new mab-swedish dataset. section 4 presents the results and discusses some of the qualitative results. in section 5, we give concluding remarks. ## related work there have been considerable effort in identifying and measuring the level of bias in datasets @xcite @xcite . these are usually targeted at gender bias in a binary form @xcite . however, studies have shown that the biases in language models for the intersection of gender and race can be greater than those for gender and race individually and that addressing bias along only one axis can lead to more issues @xcite . to determine the level of bias in nlp datasets along multiple axes can be a significant challenge, more so that many of these methods admit their approaches may demonstrate the presence of bias but not prove its absence @xcite . ## methodology 3.1 bipol there are two stages in the implementation of bipol (see 1a) before it gives a final score between 0.0 (zero or undetected bias) and 1.0 (extreme bias). the first stage involves the classification of the data samples (into biased and unbiased categories) using a trained model (see 1b). ideally, it is the ratio of the number of true positives (tp) to the total samples (true positives (tp), false positives (fp), true negatives (tn), and false negatives (fn)), where fp is preferably zero. however, since the trained models will be evaluated on unseen data, the predicted biased samples are likely to have fp in the numerator as expressed in the equation. the evaluations thus come with positive error rate ( f p f p+tp ) to establish the lower bound of error for the predictions. a good classifier should minimize the number of fp and maximize the number of tp but there's hardly any perfect classifier, even in other tasks such as spam detection or hate speech @xcite . the second stage evaluates the biased samples for sensitive terms listed in the multi-axes lexica (see 1c). it involves finding the difference between the two maximum summed frequencies in the types (e.g. female) of an axis (e.g. gender) (| n @xmath0, which is then divided by the summed frequencies of all the terms in that axis ( p @xmath1. the average over all the axes ( 1 q q @xmath2. ## results and discussion from the average macro f1 score on the validation set of mab-swedish is 0.7623 with standard deviation (s.d.) of 0.0075. the resulting error rate is 0.2893. this is relatively reasonable though a bit higher than the error rate for the english roberta, electra, and deberta, which are 0.198, 0.196, and 0.2, respectively @xcite . ## conclusion we show that all benchmark datasets we evaluated, including the swedish datasets, contain bias to different degrees. this is likely the first time these datasets are evaluated in such a way that estimates the amount of bias and the type. we believe these evaluations will motivate research on how to more effectively mitigate bias along multiple axes in datasets. this work may encourage discussions on whether the biased samples from the benchmark datasets should be disregarded entirely or if they should be utilized in a different manner than previously done. our public release of the new mab-swedish dataset, lexica and model will also facilitate future work in multilingual bias detection.