Gender Encoding Patterns in Pretrained Language Model Representations (2025)

Abstract: Gender bias in pretrained language models (PLMs) poses significant social and ethical challenges. Despite growing awareness, there is a lack of comprehensive investigation into how different models internally represent and propagate such biases. This study adopts an information-theoretic approach to analyze how gender biases are encoded within various encoder-based architectures.We focus on three key aspects: identifying how models encode gender information and biases, examining the impact of bias mitigation techniques and fine-tuning on the encoded biases and their effectiveness, and exploring how model design differences influence the encoding of biases.Through rigorous and systematic investigation, our findings reveal a consistent pattern of gender encoding across diverse models. Surprisingly, debiasing techniques often exhibit limited efficacy, sometimes inadvertently increasing the encoded bias in internal representations while reducing bias in model output distributions. This highlights a disconnect between mitigating bias in output distributions and addressing its internal representations. This work provides valuable guidance for advancing bias mitigation strategies and fostering the development of more equitable language models.

## introduction pretrained language models (plms) have revolutionized natural language processing (nlp) by enabling a wide range of applications @xcite @xcite @xcite . these models, trained on vast amounts of data, capture intricate patterns and knowledge, including gender-related information. however, alongside their impressive capabilities, plms also encode harmful biases that raise significant ethical concerns @xcite @xcite . these biases can perpetuate stereotypes, misrepresent individuals and groups, and lead to unfair treatment in various applications, thereby impacting social justice and equity (e.g. @xcite @xcite . understanding how plms encode and propagate gender information is critical for developing effective bias mitigation strategies. this challenge grows increasingly urgent with the widespread adoption of retrieval-augmented generation (rag) techniques, which rely on encoder-derived representations to retrieve contextually relevant documents @xcite . if gender biases are deeply embedded in these encoder-derived representations, rag pipelines risk amplifying societal biases at an unprecedented scale by retrieving and propagating stereotypical or discriminatory content. despite extensive research on bias in language models, much of the focus has been on identifying and measuring bias rather than comprehensively analyzing how it is embedded within the model's internal representations. previous studies have explored bias in transformer-based models, developing metrics to quantify bias @xcite @xcite @xcite , implementing techniques to reduce it @xcite @xcite @xcite , and investigating its underlying causes @xcite . however, there remains a limited understanding of the mechanisms through which biases are encoded and how different training and fine-tuning processes influence these biases within model weights. to address this gap, we use an informationtheoretic approach, specifically minimum description length (mdl) probing proposed by @xcite , to explore how gender bias is encoded in various encoder-based architectures. by exam-ining different layers of plms, we identify where biases emerge and how fine-tuning and debiasing techniques impact these representations. our work is inspired by @xcite who studied the impact of debiasing techniques used to reduce the model's reliance on spurious correlations between data and labels in natural language inference on model's representations. in summary, our contributions are twofold: ## related works in this section, we review some of the related studies on gender bias in language models, bias mitigation and measurement methods, and probing techniques and their use in bias evaluation. ## background probing datasets are typically defined as @xmath0, y p }, where x represents the input data, and y p represents the linguistic property or knowledge we are seeking to extract from the language model. the usage of language models involves two distinct stages. in the first stage, the language model, denoted as f θ : @xmath1, transforms the input x into a latent space z, where x denotes the textual input, z represents the latent representation of the text, and θ encompasses the model's weights. this latent space captures complex linguistic features and representations that encode the underlying information within the input text. subsequently, in the second stage, a classifier, denoted as g σ : @xmath2, is employed to map the latent space z to the corresponding label space y . the classifier is denoted by g σ , with σ encompassing its parameters. this two-stage approach facilitates the language model's ability to learn intricate language structures and encode relevant knowledge, while the classifier enables the extraction and utilization of this knowledge for various downstream tasks and analyses. traditionally, probing classifiers attempted to train on frozen language model weights, ensuring that the transformation from x to z remains unchanged during training. subsequently, the classifier learns how to map the latent space z to the target property space y p . if the classifier can effortlessly learn this transformation with a limited amount of data, it was concluded that the language model possesses the relevant linguistic information @xcite . however, such traditional probing approaches have been shown to exhibit limitations. these methods can yield unreliable results as they tend to classify representations of random data similarly to those of actual data, indicating their inadequacy in capturing variations in representations @xcite . as a consequence, the outcomes of these traditional probing methods are highly dependent on hyperparameter choices and might not reliably reflect the true linguistic properties encoded within the language model representations. to address these issues and obtain more robust probing results, recent advancements have introduced innovative techniques, such as the minimum description length (mdl) probing approach proposed by @xcite . in mdl probing, the objective is not solely to assess the accuracy of the shallow classifier but also to measure the effort required to extract the targeted linguistic information from the model representations. formally, they establish that a code exists to losslessly compress the labels using shannon-huffman code such that l p (y 1,z |x 1,z ) = -z @xmath3. note that this is the crossentropy loss. furthermore, they define the uniform code length as l unif (y i,z |x i,z ) = zlog 2 (c) where c is the number of classes in our task. given a model p θ (y|x) with learnable parameters θ, they choose blocks 1 = n 0 < n 1 < ... < n @xmath4. the model starts by transmitting the data using the uniform code length for the first chunk. the model is then trained to predict labels y from the data x, and also used to predict the labels. the next block is transmitted using this trained new model. this process continues until the entire dataset is covered. online code length is calculated as follows: note that this encourages the model to perform well with smaller blocks, as if the model performs well in compressing the data in the block n i , the compression will be increased for the subsequent block n i+1 . having calculated the code lengths, they compare the cross-entropy loss against the uniform code length to find the final compression. formally, compression (c) is defined as the ratio l online l unif , quantifying how much the model compresses gender information relative to a uniform baseline. ## methodology for this study, we focus on gender information as the knowledge property being probed. we will employ mdl probing to evaluate this phenomenon. ## impact of bias mitigation bias mitigation in language models seeks to address both overt biases in model outputs and the subtler, systemic biases embedded within the model's internal representations. effective techniques should suppress these encoded biases while maintaining model utility. in this section, we investigate the impact of various debiasing methods on compression values, used as a measure of encoded gender information, and evaluate their effectiveness across different experimental setups and models. ## impact of fine-tuning while encoder models are widely used in retrieval systems, their representations are typically finetuned for downstream tasks such as classification. understanding how this process influences gender bias encoded in model representations is critical, as fine-tuning may alter or amplify existing biases. in this section, we investigate how fine-tuning affects gender-related information stored in model layers and evaluate its implications for bias mitigation. ## conclusions our analysis reveals that pretrained language models follow a consistent pattern of gender encoding: early layers suppress gender signals, while later layers amplify them, embedding bias deeply into abstract representations. current debiasing techniques, particularly post-hoc interventions, show limited efficacy in altering these internal mechanisms. task-specific fine-tuning reduces encoded gender information but risks concentrating residual bias in downstream classifiers, underscoring the need for holistic mitigation strategies that target both representations and decision layers. collectively, these findings challenge conventional debiasing paradigms, advocating for proactive integration of fairness objectives during pretraining and architecture-aware interventions targeting bias propagation pathways.