Bias Attribution in F ilipino Language Models: Extending a Bias Interpretability Metric for Application on Agglutinative Languages (2025)

Abstract: Emerging research on bias attribution and interpretability have revealed how tokens contribute to biased behavior in language models processing English texts. We build on this line of inquiry by adapting the information-theoretic bias attribution score metric for implementation on models handling agglutinative languages—particularly Filipino. We then demonstrate the effectiveness of our adapted method by using it on a purely Filipino model and on three multilingual models—one trained on languages worldwide and two on Southeast Asian data. Our results show that Filipino models are driven towards bias by words pertaining to \textit{people} , \textit{objects} , and \textit{relationships} —entity-based themes that stand in contrast to the action-heavy nature of bias-contributing themes in English (i.e., \textit{criminal} , \textit{sexual} , and \textit{prosocial} behaviors). These findings point to differences in how English and non-English models process inputs linked to sociodemographic groups and bias.

## introduction as pretrained language models (plms) grow in scale and capability, research into the biased behaviors they exhibit continue to rise as well @xcite . improvements in their multilingual capacities, in particular, have been matched by studies investigating how fair multilingual and non-english models are (e.g., @xcite @xcite . in these studies, nlp scholars from all over the globe take bias evaluation tools and methods initially developed for english and adapt them into multicultural contexts to detect how much bias multilingual plms demonstrate. these multilingual replications largely confirm the existence of safety and bias issues in models processing non-english texts. @xcite , for example, found that the norwegian models they ex-perimented with prefer anti-queer statements over queer-friendly statements 68.27% of the time on average. meanwhile, @xcite measured bias in chinese question-answering models and discovered stereotypical associations between femininity, family duties, and career prejudices in some plms. multilingual studies of bias, however, mostly focus on evaluation and, to a lesser extent, mitigation (e.g., @xcite but do not engage the subjects of interpretability and explainability-that is, exploring the internal factors and mechanisms that influence biased decisionmaking among black-box plms @xcite . increasing the transparency of how these opaque models operate and improving our understanding of the roots of their biased behavior are important steps towards regulating their harmfulness and fostering public acceptance of these technologies @xcite . to these ends, @xcite have developed an interpretability metric that explains how certain tokens contribute to bias in language models. thus far, the method has only been applied on plms being evaluated on english bias tests and is yet to be extended to multilingual models handling non-english texts. in this paper, we build upon their work by using the bias attribution score metric to analyze what tokens and semantic categories induce gender-and sexuality-biased tendencies within plms working on texts in filipino, a language without high nlp resources @xcite . examining genderand sexuality-biased model behavior in filipino holds value for three reasons. first is the swift adoption of ai technologies in southeast asia, where vulnerable minorities may be adversely affected by plm biases and harms @xcite . second is filipino's agglutinative morphology @xcite , which is distinct from english's largely analytic morphology @xcite and there-fore necessitates slight adjustments on tokenizationdependent methods such as bias attribution score calculation. the last reason pertains to idiosyncrasies in how gender, queerness, and related biases manifest in filipino language and culture @xcite , which may yield variations in how filipino models manage gendered data as compared to english models. indeed, our findings reveal that whereas the actionheavy topics of crime, intimate relations, and helping prompt biased behaviors in models handling english @xcite , plms processing filipino can attribute their propensities for bias to words belonging to more concrete themes-e.g., those referring to tangible objects and people. our contributions are threefold: the remainder of this paper begins with a brief review of the literature regarding token-based attribution and interpretability in nlp (2). this review is followed by sections detailing our bias statement (3) and the methods we used-particularly, the dataset we chose, the models we inspected, and the attribution metric we used (4). the paper continues with the results of our analysis (5) and ends with our conclusions (6). ## related work there are two categories of interpretability methods in machine learning: global and local explanation methods @xcite . global explanation methods shed light on the complete reasoning process employed by the model in arriving at all possible outcomes @xcite . global explanations seem to be rare for plms, especially among generative ones, because substantial variations in possible inputs and outputs make it hard to abstract a single interpretability tool, model, or heuristic that can generate explanations for all these possibilities. instead, more common are local explanation methods, which examine data instances one by one and quantify how much a model's prediction or output can be attributed to each individual input feature within a data point. among language models, local explanations are achieved through the computation of token attribution scores. these scores indicate the degree of contribution each input token has on a plm's decision or generation @xcite . supervised learning models, such as those used to detect toxicity (e.g., @xcite , hate speech (e.g., @xcite , and misogyny (e.g., @xcite , are the frequent subjects of token attribution studies that employ various mathematical techniques-e.g., linear approximations (e.g., @xcite and shapley values (e.g., chen et al., 2020)-to calculate token attribution scores. gamboa and lee (2024) take inspiration from these classificationcentric studies to propose a word-level attribution score that not only explains biased behavior but also works on language models trained under masked and causal learning paradigms. specifically, their approach draws from an informationtheoretic bias evaluation metric originally devised by @xcite and demonstrates that intermediary values needed to compute the bias metric can actually be used to explain word-level contributions to plm bias as well. their application of their proposed bias attribution score method on different plms allowed them to uncover how words linked to crime, intimacy, and helping can motivate models to behave with bias. our study further refines their method for use on models handling agglutinative languages like filipino and compares bias-inducing tokens among english and filipino models. ## bias statement conceptually, we ascribe to the notion of plm bias as disparities in model performance associated with or arising from input data containing different sociodemographic attributes @xcite . operationally, we define bias this conceptualization and operationalization of plm bias enables our study to elucidate the representational harms of models handling filipino texts. representational harms result from models perpetuating stereotypes about marginalized groups through generating unfavorable depictions about them or associating them with negative traits @xcite . models that consistently link neutral but stereotypical concepts with certain demographics are culpable of committing such harms. our analysis focuses on the potentially detrimental impacts of biased lan-guage model deployment on historically disadvantaged gender and sexuality groups in the philippines-e.g., the babae (the female), the bakla (the non-heterosexual man), and the tomboy (the nonheterosexual woman) @xcite @xcite . ## conclusion in this paper, we extended an existing bias interpretability method for use on models handling agglutinative languages like filipino. our adjustment of the bias attribution score calculation approach emanated from a careful understanding of the morphological differences between filipino, an agglutinative language, and english, an analytic language. we then applied our revised method on four models evaluated for bias using filipino crows-pairs and demonstrated the technique's effectiveness in making transparent how some tokens cause blackbox models to make biased decisions. finally, we performed an aggregate analysis of filipino biascontributing tokens, focusing specifically on the semantic categories they belonged to. our results show that contrary to the abstract and action-heavy nature of bias-contributing tokens in english benchmarks and models, filipino models are induced to act biasedly by words referring to concrete entities (i.e., objects and persons). we hope these findings can contribute to current efforts investigating bias mechanisms in language models and working to reduce their toxic and harmful effects (e.g., @xcite @xcite .