HERB : Measuring Hierarchical Regional Bias in Pre-trained Language Models (2022)

Abstract: Fairness has become a trending topic in natural language processing (NLP) and covers biases targeting certain social groups such as genders and religions. Yet regional bias, another long-standing global discrimination problem, remains unexplored still. Consequently, we intend to provide a study to analyse the regional bias learned by the pre-trained language models (LMs) that are broadly used in NLP tasks. While verifying the existence of regional bias in LMs, we find that the biases on regional groups can be largely affected by the corresponding geographical clustering. We accordingly propose a hierarchical regional bias evaluation method (HERB) utilising the information from the sub-region clusters to quantify the bias in the pre-trained LMs. Experiments show that our hierarchical metric can effectively evaluate the regional bias with regard to comprehensive topics and measure the potential regional bias that can be propagated to downstream tasks. Our codes are available at https://github.com/Bernard-Yang/HERB .

## introduction large-scale pre-trained language models (lms) are prevalent in the natural language processing (nlp) community since the costly pre-trained models can be adapted to a wide range of downstream applications. however, research studies demonstrate that the societal biases in the pre-training corpora can be learned by lms and further propagated to the downstream applications @xcite @xcite . to qualify and mitigate bias for pre-trained lms, researchers have developed bias evaluation methods targeting certain social groups such as gender, religion, and race @xcite @xcite . however, existing methods do not examine the social groups categorised by geographical information, which leaves the region-related biases in pre-trained lms unexplored. therefore, our work bridges this gap by addressing research questions about whether regional bias exists in the pre-trained lms, and if yes, how to quantify the bias in a principled way. bias in nlp applications makes distinct judgements on people based on their gender, race, religion, region, or other social groups could be harmful, such as automatically downgrading the resumes of female applicants in recruiting @xcite regional bias represents stereotypes based on the geographic location where people live or come from @xcite . to verify the existence of regional bias, we first leverage a sentence-level bias measurement @xcite , with which the likelihood of a biased sentence produced by a pre-trained lm can be acquired with a designed input: people in [region] are @xcite . in addition, we discover that the regional bias in pre-trained lms could be hierarchical as demon- figure 1: the regional likelihood in @xcite dimension produced by roberta. the regional likelihoods are produced with sentences filled with different region names and the fixed descriptive word @xcite in the given template. the likelihood calculated with the region word @xcite is marked by ◀ at the likelihood legend in fig. 1b . the details of calculation can be referred to §3.1. strated in fig. 1b . whilst people in many european countries share a low likelihood of @xcite , the upper-level regional group, i.e., europe, is also assigned a relatively small likelihood. this suggests that the language models do recognise the hierarchical structure of the regional group structure and thus produce similar results for most of the countries and the continental group. however, opposite trends of high likelihoods appear in countries such as the united kingdom, which implies that bias in these regions could not be represented by the higher-level group, europe. without considering relationships between regional groups, the modelling of regional bias is difficult because only conducting bias evaluation on high-level groups can disguise the biases in their sub-regions. to tackle the aforementioned issues, we argue that the design of regional bias evaluation for pretrained lms should satisfy the following criteria: with the criteria in mind, we design a clusteringbased metric herb , which can effectively measure hierarchical regional bias. herb is grounded on the descriptive vectors, a novel component that is designed to capture region-specific contextualised likelihoods with respect to the content of @xcite . as the bias on regions should be relevant to their sub-region, we formalise the bias on a given region as the sparseness of its sub-region cluster in the descriptive space. the intuition behind the cluster-based sparseness calculation is that the more bias exists in the region, the more inconsistent the judgements on its subregions received. in the case that a region does not contain any sub-regions, its cluster sparseness is modelled by the distance to the centroids of the cluster, where all the regions belong to the same upper-level region, e.g., cities in the same country. we further propose aggregation functions for the descriptive vector and cluster-based bias calculation to utilise the hierarchy. the aggregated cluster-based bias evaluation not only empowers our metric to consider regional bias at multiple levels but also sheds light on the general regional bias evaluation for the pre-trained lms. we perform extensive evaluations of hierarchical regional bias on various state-of-the-art pre-trained language models and study the regional hierarchical relationships learned by the lms. additionally, we conduct experiments to study the propagation of regional bias from pre-trained models to downstream tasks. by introducing extra neutral regional information to the test samples and observing the prediction change, we evaluate how much the model performances are affected by region bias. regional bias evaluation results on downstream tasks confirm that results from our metric have correlations to the bias propagation to fine-tuned lms. ## methodology we describe our hierarchical evaluation method for regional bias in pre-trained lms in this section. to measure the bias from comprehensive aspects, we first map all the regional groups to a descriptive representation space with a selective word list. we use a cluster-based evaluation method to represent the bias of a given region with regard to its sub-regions, which leverages the natural hierarchical regional group structure in the bias evaluation. in order to summarise bias information from regions at different levels simultaneously, we design a novel aggregation function of the descriptive vector and cluster-based bias, which measures the general re-gional bias in the pre-trained lms. ## experiments in this section, we conduct regional bias evaluation on pre-trained language models with the proposed metric herb . to validate the design of herb , we provide a comparison between the aggregated evaluation function and the bias acquired only by cluster sparseness and give an ablation study on the description topics. at last, we verify the effectiveness of herb by exploring the regional bias before and after the lms are fine-tuned for the downstream task. ## conclusion in this work, regional bias in the pre-trained language models has been measured in depth for the first time within the nlp community. the proposed metric, herb , takes hierarchical characteristics of regional bias into consideration and adopts a carefully selected descriptive word list. we use herb to evaluate regional bias in state-of-theart language models and validate the robustness of herb by providing bias analysis on downstream tasks for corresponding models. thorough experimentation studies are given to show that the hierarchical structure of regions does not only present in the pre-trained representation space but also appears as hierarchical bias in downstream tasks, which further rationalises the design of herb .