Bias Analysis and Mitigation through Protected Attribute Detection and Regard Classification (2025)

Abstract: Large language models (LLMs) acquire general linguistic knowledge from massive-scale pretraining. However, pretraining data mainly comprised of web-crawled texts contain undesirable social biases which can be perpetuated or even amplified by LLMs. In this study, we propose an efficient yet effective annotation pipeline to investigate social biases in the pretraining corpora. Our pipeline consists of protected attribute detection to identify diverse demographics, followed by regard classification to analyze the language polarity towards each attribute. Through our experiments, we demonstrate the effect of our bias analysis and mitigation measures, focusing on Common Crawl as the most representative pretraining corpus.

## introduction recent years have witnessed a remarkable progress in the development and adoption of large language model (llm) technology @xcite . generally, llms first undergo large-scale pretraining to acquire general linguistic knowledge, followed by post-training to align them with human goals and preferences @xcite . the majority of llm's knowledge is acquired in the pretraining stage, and post-training is conceived to mainly change the style of llms to serve as interactive, open-domain ai assistants @xcite . however, pretraining data comprised of webcrawled texts often contain undesirable biases, such as associating muslim people with terrorism and extremism @xcite . consequently, llms tend to inherit the biases and produce harmful judgements @xcite , raising significant risks in terms of safety and fairness. despite the severity of this issue, it remains extremely difficult to audit and alleviate such dataset-level biases due to the everincreasing size of pretraining corpora and the openended, complex nature of social biases. in this study, we make a first step towards analyzing and mitigating problematic social biases in massive-scale text data. specifically, we propose a scalable annotation pipeline consisting of two steps. first, we conduct protected attribute detection to identify diverse demographics, such as nationality, religion, disability, etc. to reduce false positive detections, we combine keyword matching and word sense disambiguation @xcite . second, we apply regard classification to analyze the language polarity towards each attribute into positive, neutral, or negative @xcite . our overall pipeline is illustrated in figure 1 . in our experiments, we apply the pipeline to a subset of common crawl, the most widely used corpus for llm pretraining. for bias analysis, we refine the frequency-based word association analysis @xcite to take into account regard information. qualitative and quantitative results show that our approach more accurately captures commonly held stereotypes @xcite . for bias mitigation, we reveal that the regard distributions among protected attributes can be noticeably imbalanced, and adjusting regard distributions (e.g. downsampling negative regard texts) can be a promising approach to mitigate offensive social stereotypes, such as "white" people being overly described as "racists" or "supremacists". while several challenges still remain, we expect our approach to be a crucial step towards improving the fairness of llm pretraining datasets. ## experiments in our experiments, we apply our bias analysis and mitigation measures on a subset of common crawl (cc), a widely used llm pretraining corpora. as a preparation, we first sampled over 3m english documents from cc and extracted sentences of appropriate length. 5 then, we conducted protected attribute detection ( §2.1) and regard classifi-cation ( §2.2) on these sentences. in the following experiments, we used up to 100k sentences per attribute to demonstrate the effect of our bias analysis ( §3.1) and mitigation ( §3.2) techniques. ## discussion and conclusion in this study, we proposed a novel pipeline to probe social biases in large-scale pretraining corpora. in contrast to existing methods, our approach employs wsd to accurately detect protected attributes while maintaining efficiency. furthermore, we apply regard classification to analyze the language polarity towards each attribute, which is crucial yet overlooked in the existing study of pretraining datasets @xcite . based on our annotation, we can conduct regardaware bias analysis to obtain valuable insights on social stereotypes associated with each attribute. moreover, we can compute and balance the regard distributions to improve the fairness of the dataset, e.g. ensuring negative or offensive descriptions are not overly represented in the data. while we acknowledge several difficulties and challenges still remain (as we discuss in the following section), we hope this work will be a fundamental step towards understanding and addressing inherent ethical risks of llm pretraining.