From n-gram to Attention: How Model Architectures Learn and Propagate Bias in Language Modeling (2025)

Abstract: Current research on bias in language models (LMs) predominantly focuses on data quality, with significantly less attention paid to model architecture and temporal influences of data. Even more critically, few studies systematically investigate the origins of bias. We propose a methodology grounded in comparative behavioral theory to interpret the complex interaction between training data and model architecture in bias propagation during language modeling. Building on recent work that relates transformers to n-gram LMs, we evaluate how data, model design choices, and temporal dynamics affect bias propagation. Our findings reveal that: (1) n-gram LMs are highly sensitive to context window size in bias propagation, while transformers demonstrate architectural robustness; (2) the temporal provenance of training data significantly affects bias; and (3) different model architectures respond differentially to controlled bias injection, with certain biases (e.g. sexual orientation) being disproportionately amplified. As language models become ubiquitous, our findings highlight the need for a holistic approach- tracing bias to its origins across both data and model dimensions, not just symptoms, to mitigate harm.

## introduction the vast scale and often unverified quality of training data (heterogeneous data) create substantial pathways for social biases to manifest in language models, whether in relatively simple word embeddings @xcite or in far more complex large language models @xcite . such harmful biases pose significant challenges to the deployment of large-scale ai systems in diverse sociocultural contexts @xcite . this growing awareness has spurred a considerable body of nlp research focused on measuring and mitigating bias. however, the relationship between bias and language models remains complex, and the mechanisms underlying biased decision-making in downstream tasks are not yet fully understood @xcite . contemporary attempts at bias mitigation have often been criticized as 'putting lipstick on a pig', meaning that they merely conceal bias rather than truly eliminating it @xcite . this criticism largely stems from the fact that current approaches to bias identification and reduction tend to operate as blackbox methods, offering limited transparency. most studies attribute bias primarily to web data used for training language models (baeza-yates, 2016), while some studies have sought to connect the model architecture with observed biases @xcite . however, recent research often overlooks the complex interplay between training data and model architecture in shaping model biases @xcite . even the time of creation of the training corpora can significantly affect the biases induced in language models, a factor that remains largely unexplored at the experimental level @xcite . so, before we can effectively mitigate biases in language models, an essential first step is to 'understand' the bias. modern language models are fundamentally autoregressive generators trained for language modeling tasks @xcite . to understand how bias propagates across different model architectures, it is important to analyze this phenomenon through the lens of language modeling. transformers, which form the foundation of autoregressive generation models @xcite , have been the subject of extensive research on interpretability @xcite . in this context, comparative behavioral analysis frameworks that juxtapose transformers with n-gram models have emerged as valuable tools to understand transformer behavior @xcite @xcite . these frameworks assess model behavior either by comparing performance under the same controlled conditions @xcite or by visualizing important input features using saliency methods @xcite . in this work, we investigate how the comparative behavioral analysis framework illuminates the interplay between training data and model architecture and influences the propagation of bias during language modeling. we study how transformers imitate n-gram models in bias propagation in language modeling. through this, we address the following axes of comparison: addressing these aspects brings us closer to a practical understanding of bias propagation in language modeling, illuminating how the theoretical properties of neural architectures translate into real-world bias compared to n-gram statistics. our approach using comparative behavioral framework can be generalized to other architectures that use language modeling tasks, providing an interpretable tool for bias propagation and consequently paving the way for more effective bias mitigation in language models. a comprehensive background review of contemporary research on n-grams versus transformers, bias and interpretability is presented in appendix a. ## bias propagation in lms at a high level, our work compares how bias emerges in statistical n-gram lms versus neural transformers during language modeling. we train both types of models with varied architectural parameters on wikipedia data from different time periods and then systematically evaluate their biases using the crows-pairs dataset @xcite . ## experimental results and analysis in this section, we address the findings for our four dimensions of comparisons presented in section 1. ## conclusion we compare bias propagation in transformer and n-gram language models, focusing on the roles of model architecture and training data. our results show that n-gram models are sensitive to context window size but benefit from advanced smoothing like kneser-ney, while transformers are robust to architectural changes, with attention mechanisms slightly influencing bias propagation. we observe that training data plays a dominant role in shaping bias across all models. these findings highlight the need to jointly consider both data and architecture for effective bias mitigation in language models.