F air L ex: A Multilingual Benchmark for Evaluating Fairness in Legal Text Processing (2022)

Abstract: We present a benchmark suite of four datasets for evaluating the fairness of pre-trained language models and the techniques used to fine-tune them for downstream tasks. Our benchmarks cover four jurisdictions (European Council, USA, Switzerland, and China), five languages (English, German, French, Italian and Chinese) and fairness across five attributes (gender, age, region, language, and legal area). In our experiments, we evaluate pre-trained language models using several group-robust fine-tuning techniques and show that performance group disparities are vibrant in many cases, while none of these techniques guarantee fairness, nor consistently mitigate group disparities. Furthermore, we provide a quantitative and qualitative analysis of our results, highlighting open challenges in the development of robustness methods in legal NLP.

## introduction natural language processing (nlp) for law @xcite @xcite receives increasing attention. assistive technologies can speed up legal research or discovery significantly assisting lawyers, judges and clerks. they can also help legal scholars to study case law @xcite , improve access of law to laypersons, help sociologists and research ethicists to expose biases in the justice system @xcite , and even scrutinize decision-making itself @xcite . in the context of law, the principle of equality and non-discrimination is of paramount importance, although its definition varies at international, regional and domestic level. for example, eu nondiscrimination law prohibits both direct and indirect discrimination. direct discrimination occurs when one person is treated less favourably than * corresponding author: ilias.chalkidis@di.ku.dk others would be treated in comparable situations on grounds of sex, racial or ethnic origin, disability, sexual orientation, religion or belief and age. 1 given the gravity that legal outcomes have for individuals, assistive technologies cannot be adopted to speed up legal research at the expense of fairness @xcite , potentially also decreasing the trust in our legal systems @xcite . societal transformations perpetually shape our legal systems. the topic deserves great attention because ai systems learning from historical data pose the risk of lack of generalisability beyond the training data, and more importantly transporting biases previously encumbered in the data in future decision-making, thereby exponentially increasing their effect @xcite . historical legal data do not represent all groups in our societies equally and tend to reflect social biases in our societies and legal institutions. when models are deployed in production, they may reinforce these biases. for example, criminal justice is already often strongly influenced by racial bias, with people of colour being more likely to be arrested and receive higher punishments than others, both in the usa foot_1 and in the uk. 3 in recent years, the nlp and machine learning literature has introduced fairness objectives, typically derived from the rawlsian notion of equal opportunities @xcite , to evaluate the extent to which models discriminate across protected attributes. some of these rely on notions of resource allocation, i.e., reflecting the idea that groups are treated fairly if they are equally represented in the training data used to induce our models, or if the same number of training iterations is performed per group. this is sometimes referred to as the resource allocation perspective on fairness @xcite . contrary, there is also a capability-centered approach to fairness @xcite , in which the goal is to reserve enough resources per group to achieve similar performance levels, which is ultimately what is important for how individuals are treated in legal processes. we adopt a capability-centered approach to fairness and define fairness in terms of performance parity @xcite or equal risk @xcite . 4 performance disparity @xcite ) refers to the phenomenon of high overall performance, but low performance on minority groups, as a result of minimizing risk across samples (not groups). since some groups benefit more than others from models and technologies that exhibit performance disparity, this likely widens gaps between those groups. performance disparity works against the ideal of fair and equal opportunities for all groups in our societies. we therefore define a fair classifier as one that has similar performance (equal risk) across all groups @xcite . in sum, we adopt the view that (approximate) equality under the law in a modern world requires that our nlp technologies exhibit (approximately) equal risk across sensitive attributes. for everyone to be treated equally under the law, regardless of race, gender, nationality, or other characteristics, nlp assistive technologies need to be (approximately) insensitive to these attributes. we consider three types of attributes in this work: contributions we introduce fairlex, a multilingual fairness benchmark of four legal datasets covering four jurisdictions (council of europe, united states of america, swiss confederation and people's republic of china), five languages (english, german, french, italian and chinese) and various sensitive attributes (gender, age, region, etc.). we release four pre-trained transformer-based language models, each tailored for a specific dataset (task) within our benchmark, which can be used as baseline models (text encoders). we conduct experiments with several group-robust algorithms and provide a quantitative and qualitative analysis of our results, highlighting open challenges in the development of robustness methods in legal nlp. 2 related work fair machine learning the literature on inducing approximately fair models from biased data is rapidly growing. see mehrabi et al. (2021); makhlouf et al. (2021); ding et al. ( 2021 ) for recent surveys. we rely on this literature in how we define fairness, and for the algorithms that we compare in our experiments below. as already discussed, we adopt a capability-centered approach to fairness and define fairness in terms of performance parity @xcite or equal risk @xcite . the fairness-promoting learning algorithms we evaluate are discussed in detail in section 4. some of these -group distributionally robust optimization @xcite and invariant risk minimization @xcite ) -have previously been evaluated for fairness in the context of hate speech @xcite . fairness in law studying fair machine learning in the context of legal (computational) applications has a limited history. in a classic study, @xcite analyzed the performance of the correctional offender management profiling for alternative sanctions (compas) system, which was used for parole risk assessment (recidivism prediction) in the us. the system relied on 137 features from questionnaires and criminal records. angwin et al. found that blacks were almost twice as likely as whites to be mislabeled as high risk (of reoffending), revealing a severe racial bias in the system. the system was later compared to crowdworkers in @xcite . these studies relied on tabular data and did not involve text processing (e.g., encoding case facts or decisions). more recently, @xcite studied legal judgment consistency using a dataset of chinese criminal cases. they evaluated the consistency of lstm-based models across region and gender and reported severe fairness gaps across gender. they also found that the fairness gap was particular severe for more serious crimes. another line of work @xcite @xcite explores representational bias with respect to race and gender analyzing word latent representations trained in legal text corpora. while we agree that representational bias can potentially reinforce unfortunate biases, these may not impact the treatment of individuals (or groups). we therefore focus on directly measuring equal risk on downstream applications instead. previous work has focused on the analysis of specific cases, languages or algorithms, but fair-lex aims at easing the development and testing of bias-mitigation models or algorithms within the legal domain. fairlex allows researchers to explore fairness across four datasets covering four jurisdictions (council of europe, united states of america, swiss confederation and people's republic of china), five languages (english, german, french, italian and chinese) and various sensitive attributes @xcite . furthermore, we provide competitive baselines including pre-trained transformer-based language models, adapted to the examined datasets, and an in-dept examination of performance of four group robust algorithms described in detail in section 4. benchmarking nlp has been stormed by the rapid development of benchmark datasets that aim to evaluate the performance of pre-trained language models with respect to different objectives: general natural language understanding (nlu) @xcite , cross-lingual transfer (clt) @xcite , and even domain-specific ones on biomedical @xcite , or legal @xcite nlp tasks. despite their value, recent work has raised criticism on several limitations of the so called nlu benchmarks @xcite @xcite . the main points are: poor (laissez-faire) dataset development (e.g., lack of diversity, spurious correlations), legal issues (e.g., data licensing and leakage of personal information), construct validity (e.g., poor experimental setup, unclear research questions), question of "general" capabilities, and promotion of superficial competitiveness (hype, or even falsify, state-of-the-art results). we believe that the release of fairlex, a domainspecific (legal-oriented) benchmark suite for evaluating fairness, overcomes (or at least mitigates) some of the aforementioned limitations. we introduce the core motivation in section 1, while specific (case-by-case) details are described in section 3. our benchmark is open-ended and inevitably has several limitations; we report known limitations and ethical considerations in sections 7 and 8. nonetheless we believe that it will help critical research in the area of fairness. ## benchmark datasets ecthr the european court of human rights (ecthr) hears allegations that a state has breached human rights provisions of the european convention of human rights (echr). we use the dataset of @xcite , which contains 11k cases from ecthr's public database. each case is mapped to articles of the echr that were violated (if any). this is a multi-label text classification task. given the facts of a case, the goal is to predict the echr articles that were violated, if any, as decided (ruled) by the court. the cases are chronologically split into training @xcite , development (1k, 2016-17), and test (1k, 2017-19) sets. to facilitate the study of fairness of text classifiers, we record for each case the following attributes: (a) the defendant states, which are the european states that allegedly violated the echr. the defendant states for each case is a subset of the 47 member states of the council of europe; 5 to have statistical support, we group defendant states dataset original publication classification task no of classes attributes attribute type #n ecthr (chalkidis et al., 2021) legal judgment prediction: echr violation prediction 10+1 defendant state 2 applicant gender 2 applicant age 3 scotus (spaeth et al., 2020) legal topic classification: issue area classification 14 respondent type 4 decision direction 2 fscs (niklaus et al., 2021) legal judgment prediction: case approval prediction 2 language 3 region of origin 6 legal area 6 cail (wang et al., 2021b) legal judgment prediction: crime severity prediction 6 defendant gender 2 region of origin 7 from scdb, we also use the following attributes to study fairness: (a) the type of respondent, which is a manual categorization of respondents (defendants) in five categories (person, public entity, organization, facility and other); and (c) the direction of the decision, i.e., whether the decision is considered liberal, or conservative, provided by scdb. ## fine-tuning algorithms across experiments, our main goal is to find a hypothesis for which the risk r(h) is minimal: similar to previous studies, r(h) is an expectation of the selected loss function (l). in this work, we study multi-label text classification (section 3), thus we aim to minimize the binary cross-entropy loss across l classes: erm @xcite , which stands for empirical risk minimization, is the most standard and widely used optimization technique to train neural methods. the loss is calculated as follows: besides erm, we also consider a representative selection of group-robust fine-tuning algorithms which aims at mitigating performance disparities with respect to a given attribute (a), e.g., the gender of the applicant or the region of the court. each attribute is split into g groups, i.e., male/female for gender. all algorithms rely on a balanced group sampler, i.e., an equal number of instances (samples) per group (n g ) are included in each batch. most of the algorithms are built upon group-wise losses (l g ), computed as follows: group dro @xcite , stands for group distributionally robust optimization (dro). group dro is an extension of the group uniform algorithm, where the group-wise losses are weighted inversely proportional to the group training performance. the total loss is: v-rex @xcite , which stands for risk extrapolation, is yet another proposed grouprobust optimization algorithm. @xcite hypothesize that variation across training groups is representative of the variation later encountered at test time, so they also consider the variance across the group-wise losses. in v-rex the total loss is calculated as follows: irm @xcite , which stands for invariant risk minimization, mainly aims to penalize variance across multiple training dummy estimators across groups, i.e., performance cannot vary in samples that correspond to the same group. the total loss is computed as follows: please refer to @xcite for the definition of the group penalty terms (p g ). adversarial removal @xcite algorithm mitigates group disparities by means of an additional adversarial classifier @xcite . the adversarial classifier share the encoder with the main network and is trained to predict the protected attribute (a) of an instance. the total loss factors in the adversarial one, thus penalizing the model when it is able to discriminate groups. formally, the total loss is calculated as: for the purpose of this work, we release four domain-specific bert models with continued pretraining on the corpora of the examined datasets. 9 we train mini-sized bert models with 6 transformer blocks, 384 hidden units, and 12 attention heads. we warm-start all models from the public minilmv2 models checkpoints @xcite using the distilled version of roberta @xcite for the english datasets (ecthr, sco-tus) and the one distilled from xlm-r @xcite for the rest (trilingual fscs, and chinese cail). given the limited size of these models, we can effectively use up to 4096 tokens in ecthr and scotus and up to 2048 tokens in fscs and 9 https://huggingface.co/coastalcph cail for up to 16 samples per batch in a 24gb gpu card. 10 for completeness, we also consider linear bag-of words (bow) classifiers using tf-idf scores of the most frequent n-grams (where @xmath0, 2, 3) in the training corpus of each dataset. ## limitations the current version of fairlex covers a very small fraction of legal applications, jurisdictions, and protected attributes. our benchmark is open-ended and inevitably cannot cover "everything in the whole wide (legal) world" @xcite , but nonetheless we believe that the published resources will help critical research in the area of fairness. some protected attributes within our datasets are extracted automatically, i.e., the gender and the age in the ecthr dataset, if possible, by means of regular expressions, or manually clustered by the authors, such as the defendant state in the ecthr dataset and the respondent attribute in the sco-tus dataset. various simplifications made, e.g, the binarization of gender, would be inappropriate in real-world applications. another important limitation is that what is considered the ground truth in these datasets (with the exception of scotus) is only ground truth relative to judges' interpretation of a specific (ec, us, swiss, chinese) jurisdiction and legal framework. the labeling is therefore somewhat subjective for non-trivial cases, and its validity is only relative to a given legal framework. we of course do not in any way endorse the legal standards or framework of the examined datasets. ## conclusions we introduced fairlex, a multi-lingual benchmark suite for the development and testing of models and bias-mitigation algorithms within the legal domain, based on four datasets covering four jurisdictions, five languages and various sensitive attributes. furthermore, we provided competitive baselines including transformer-based language models adapted to the examined datasets, and examination of performance of four group robust algorithms (adversarial removal, irm, group dro, and v-rex). while, these algorithms improve worst case performance in the literature, when applied in a controlled experimental environment, they fail in a more realistic setting, where both groups across attributes, and labels are imbalanced, while also both group and label distributions change over time. furthermore, we cannot identify a single algorithm that performs better across datasets and groups compared to the rest. in future work, we aim to further expand the benchmark with more datasets that could possibly cover more sensitive attributes. further analysis on the reasons behind group disparities, e.g., representational bias, systemic bias, is also critical.