cair-nepal/ai-bias-research-landscape huggingface dataset:
Title: Linguistic and dialectal bias in large language models: a review of evidence, mechanisms,and implications (20182025)
Authors: Eric Yanfei Zhao
Affiliations: Ghana Institute of Languages
Year: 2026
Domain: LLMs & NLP

Abstract: Large Language Models (LLMs) increasingly mediate communication, evaluation, and decision-making across social and institutional domains. While prior work has documented various forms of bias in language technologies, linguistic variation has received comparatively less integrated treatment despite its deep sociolinguistic significance. This review synthesizes recent research on how LLMs reflect and reinforce linguistic bias toward nonstandard dialects, with a focus on African American English and other marginalized varieties. Drawing on studies spanning natural language understanding, reasoning, speech recognition, and conversational systems, we show that LLMs exhibit consistent performance disparities when processing dialectal inputs, even when semantic content is held constant. Beyond accuracy degradation, emerging evidence demonstrates that dialect functions as a social signal that triggers differential judgment, stereotyping, and covertly racist decision-making in high-stakes contexts. We organize the literature around four themes: dialect-based performance gaps, dialect-triggered social evaluation, measurement frameworks for linguistic bias, and proposed mitigation strategies. While recent benchmarks and adaptation methods improve robustness, we argue that many approaches treat dialect bias as a technical deficiency rather than a sociotechnical problem rooted in language ideology and power. By synthesizing research across technical, social, and evaluative dimensions, this review identifies methodological gaps and proposes directions for developing linguistically and culturally robust LLMs.
Title: The Quantization Trap: Breaking Linear Scaling Laws in Multi-Hop Reasoning
Authors: Henry Han; Xiyang Liu; Xiaodong Wang; Fei Han; Xiaodong Li
Affiliations: Baylor University; Xidian University; Xidian University; Jiangsu University; Beijing Electronic Science and Technology Institute
Year: 2026
Domain: LLMs & NLP

Abstract: Neural scaling laws provide a predictable recipe for AI advancement: reducing numerical precision should linearly improve computational efficiency and energy profile ($E \propto \mathrm{bits}$). In this paper, we demonstrate that this scaling law breaks in the context of multi-hop reasoning. We reveal a 'quantization trap' where reducing precision from 16-bit to 8/4-bit paradoxically increases net energy consumption while degrading reasoning accuracy. We provide a rigorous theoretical decomposition that attributes this failure to hardware casting overhead, the hidden latency cost of dequantization kernels, which becomes a dominant bottleneck in sequential reasoning chains, as well as to a sequential energy amortization failure. As a result, scaling law breaking is unavoidable in practice. We formalize a Critical Model Scale $N^*$ that predicts when the trap dissolves or deepens as a function of model size, batch size, and hardware configuration, validated across a 120$\times$ range (0.6B--72B) on six GPU architectures. Our findings suggest that the industry's "smaller-is-better" heuristic is mathematically counterproductive for complex reasoning tasks.
Title: GNNFairViz: Visual Analysis for Graph Neural Network Fairness
Authors: Xinwu Ye; Jielin Feng; Erasmo Purificato; Ludovico Boratto; Michael Kamp; Zengfeng Huang; Siming Chen
Affiliations: Fudan University; Fudan University; Fudan University; University of Cagliari; Institute for Advanced Study in the Humanities Essen; Fudan University; Fudan University
Year: 2025
Domain: Graph-Based Fairness & Bias Mitigation

Abstract: Recent advancements in Graph Neural Networks (GNNs) show promise for various applications like social networks and financial networks. However, they exhibit fairness issues, particularly in human-related decision contexts, risking unfair treatment of groups historically subject to discrimination. While several visual analytics studies have explored fairness in machine learning (ML), few have tackled the particular challenges posed by GNNs. We propose a visual analytics framework for GNN fairness analysis, offering insights into how attribute and structural biases may introduce model bias. Our framework is model-agnostic and tailored for real-world scenarios with multiple and multinary sensitive attributes, utilizing an extended suite of fairness metrics. To operationalize the framework, we develop GNNFairViz, a visual analysis tool that integrates seamlessly into the GNN development workflow, offering interactive visualizations. Our tool enables GNN model developers, the target users, to analyze model bias comprehensively, facilitating node selection, fairness inspection, and diagnostics. We evaluate our approach through two usage scenarios and expert interviews, confirming its effectiveness and usability in GNN fairness analysis. Furthermore, we summarize two general insights into GNN fairness based on our observations on the usage of GNNFairViz, highlighting the prevalence of the "Overwhelming Effect" in highly unbalanced datasets and the importance of suitable GNN architecture selection for bias mitigation.
Title: Procedural Fairness in Machine Learning
Authors: Ziming Wang; Changwu Huang; Ke Tang; Xin Yao
Affiliations: Southern University of Science and Technology
Year: 2026
Domain: General Fairness & Bias Mitigation

Abstract: Fairness in machine learning (ML) has garnered significant attention. However, current research has mainly concentrated on the distributive fairness of ML models, with limited focus on another dimension of fairness, i.e., procedural fairness. In this paper, we first define the procedural fairness of ML models by drawing from the established understanding of procedural fairness in philosophy and psychology fields, and then give formal definitions of individual and group procedural fairness. Based on the proposed definition, we further propose a novel metric to evaluate the group procedural fairness of ML models, called GPFFAE, which utilizes a widely used explainable artificial intelligence technique, namely feature attribution explanation (FAE), to capture the decision process of ML models. We validate the effectiveness of GPFFAE on a synthetic dataset and eight real-world datasets. Our experimental studies have revealed the relationship between procedural and distributive fairness of ML models. After validating the proposed metric for assessing the procedural fairness of ML models, we then propose a method for identifying the features that lead to the procedural unfairness of the model and propose two methods to improve procedural fairness based on the identified unfair features. Our experimental results demonstrate that we can accurately identify the features that lead to procedural unfairness in the ML model, and both of our proposed methods can significantly improve procedural fairness while also improving distributive fairness, with a slight sacrifice on the model performance.
Title: Beyond Black Boxes: Designing and Testing Agentic AI Systems for Strategy
Authors: Arnaldo Camuffo; Alfonso Gambardella; Saeid Kazemi; Abhinav Pandey
Affiliations: Bocconi University; Bocconi University; Bocconi University; Bocconi University
Year: 2026
Domain: General Fairness & Bias Mitigation

Abstract: Existing work on AI and strategy examines the effects of general-purpose models, implicitly assuming that strategists will use whatever tools are available and that human-AI interaction is largely a matter of prompting, complacency, or aversion. Inspired by Herbert Simon’s work on system architecture, we instead adopt a design view in which purposeful AI system design and use—not mere access to generic models—can itself be a dynamic capability and a source of competitive advantage. Building on this perspective, we develop Aristotle, an agentic multiagent AI system for theory-based strategic decision making, and study how it shapes strategists’ reasoning, beliefs, and strategies compared with general AI assistance or no AI assistance. We document the design journey of Aristotle, highlighting design choice trade-offs in strategy framework, human-AI integration, and cost, and then implement a streamlined three-agent version suitable for experimental testing. In a randomized experiment with 976 managers comparing this agentic AI system, general AI (GPT-4o), and a human-only condition, we find that experienced managers achieve quality improvements without confidence inflation, whereas highly educated managers exhibit confidence gains without corresponding quality improvements. We establish user-system-problem fit as a core design dimension requiring alignment between architectural complexity and practitioner expertise. We abductively derive a five-dimensional taxonomy that maps the design space for agentic AI systems and a methodological roadmap that enable researchers and practitioners to experiment with, evaluate, and iteratively improve their AI system design choices. History: Accepted for the Special Issue: Can AI Do Strategy? Funding: A. Camuffo and A. Gambardella acknowledge support from the European Research Council (ERC) under the European Union’s Horizon 2020 Research and Innovation Programme [Grant 101021061]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/stsc.2025.0432 .
Title: A Non-Parametric Choice Model That Learns How Users Choose Between Recommended Options
Authors: Thorsten Krause; Harrie Oosterhuis
Affiliations: Radboud University Nijmegen; Radboud University Nijmegen
Year: 2025
Domain: Recommender Systems

Abstract: Choice models predict which items users choose from presented options. In recommendation settings, they can infer user preferences while countering exposure bias. In contrast with traditional univariate recommendation models, choice models consider which competitors appeared with the chosen item. This ability allows them to distinguish whether a user chose an item due to preference, i.e., they liked it; or competition, i.e., it was the best available option. Each choice model assumes specific user behavior, e.g., the multinomial logit model. However, it is currently unclear how accurately these assumptions capture actual user behavior, how wrong assumptions impact inference, and whether better models exist. In this work, we propose the learned choice model for recommendation (LCM4Rec), a non-parametric method for estimating the choice model. By applying kernel density estimation, LCM4Rec infers the most likely error distribution that describes the effect of inter-item cannibalization and thereby characterizes the users' choice model. Thus, it simultaneously infers what users prefer and how they make choices. Our experimental results indicate that our method (i) can accurately recover the choice model underlying a dataset; (ii) provides robust user preference inference, in contrast with existing choice models that are only effective when their assumptions match user behavior; and (iii) is more resistant against exposure bias than existing choice models. Thereby, we show that learning choice models, instead of assuming them, can produce more robust predictions. We believe this work provides an important step towards better understanding users' choice behavior.
Title: Fairness and Bias in Multimodal AI: A Survey
Authors: Tosin Adewumi; Lama Alkhaled; Namrata Gurung; Goya van Boven; Irene Pagliai
Affiliations: Luleå University of Technology; Luleå University of Technology; QualityMinds GmbH; Utrecht University; University of Göttingen
Year: 2024
Domain: General Fairness & Bias Mitigation

Abstract: The importance of addressing fairness and bias in artificial intelligence (AI) systems cannot be over-emphasized. Mainstream media has been awashed with news of incidents around stereotypes and other types of bias in many of these systems in recent years. In this survey, we fill a gap with regards to the relatively minimal study of fairness and bias in Large Multimodal Models (LMMs) compared to Large Language Models (LLMs), providing 50 examples of datasets and models related to both types of AI along with the challenges of bias affecting them. We discuss the less-mentioned category of mitigating bias, preprocessing (with particular attention on the first part of it, which we call preuse). The method is less-mentioned compared to the two well-known ones in the literature: intrinsic and extrinsic mitigation methods. We critically discuss the various ways researchers are addressing these challenges. Our method involved two slightly different search queries on two reputable search engines, Google Scholar and Web of Science (WoS), which revealed that for the queries 'Fairness and bias in Large Multimodal Models' and 'Fairness and bias in Large Language Models', 33,400 and 538,000 links are the initial results, respectively, for Scholar while 4 and 50 links are the initial results, respectively, for WoS. For reproducibility and verification, we provide links to the search results and the citations to all the final reviewed papers. We believe this work contributes to filling this gap and providing insight to researchers and other stakeholders on ways to address the challenges of fairness and bias in multimodal and language AI.
Title: Towards Fair Graph Neural Networks via Counterfactual and Balance
Authors: Zhiguo Xiao; Yangfan Zhou; Dongni Li; Ke Wang
Affiliations: Changchun University of Science and Technology; Changchun University of Science and Technology; Beijing Institute of Technology; Changchun University of Science and Technology
Year: 2025
Domain: Graph-Based Fairness & Bias Mitigation

Abstract: In recent years, graph neural networks (GNNs) have shown powerful performance in processing non-Euclidean data. However, similar to other machine-learning algorithms, GNNs can amplify data bias in high-risk decision-making systems, which can easily lead to unfairness in the final decision-making results. At present, a large number of studies focus on solving the fairness problem of GNNs, but the existing methods mostly rely on building complex model architectures or rely on technical means in the field of non-GNNs. To this end, this paper proposes FairCNCB (Fair Graph Neural Network based on Counterfactual and Category Balance) to address the problem of class imbalancing in minority sensitive attribute groups. First, we conduct a causal analysis of fair representation and employ the adversarial network to generate counterfactual node samples, effectively mitigating bias induced by sensitive attributes. Secondly, we calculate the weights for minority sensitive attribute groups, and reconstruct the loss function to achieve the fairness of sensitive attribute classes among different groups. The synergy between the two modules optimizes GNNs from multiple dimensions and significantly improves the performance of GNNs in terms of fairness. The experimental results on the three datasets show the effectiveness and fairness of FairCNCB. The performance metrics (such as AUC, F1, and ACC) have been improved by approximately 2%, and the fairness metrics (△sp, △eo) have been enhanced by approximately 5%.
Title: Fairness Testing of Large Language Models in Role-Playing
Authors: Xinyue Li; Zhenpeng Chen; Jie M. Zhang; Yiling Lou; Tianlin Li; Weisong Sun; Yang Liu; Xuanzhe Liu; Xuanzhe Liu
Affiliations: Peking University; Tsinghua University; King's College London; University of Illinois at Urbana-Champaign; Nanyang Technological University; Nanyang Technological University; Nanyang Technological University; Peking University; Peking University
Year: 2024
Domain: LLMs & NLP

Abstract: Large Language Models (LLMs) have become foundational in modern language-driven software applications, profoundly influencing daily life. A critical technique in leveraging their potential is role-playing, where LLMs simulate diverse roles to enhance their real-world utility. However, while research has highlighted the presence of social biases in LLM outputs, it remains unclear whether and to what extent these biases emerge during role-playing scenarios. In this paper, we conduct an empirical study on fairness testing of LLMs in role-playing scenarios. To enable this testing, we use LLMs to generate 550 social roles spanning a comprehensive set of 11 demographic attributes, producing 33,000 role-specific questions that target various forms of bias. These questions, covering Yes/No, multiple-choice, and open-ended formats, are designed to prompt LLMs to adopt specific roles and respond accordingly. We employ a combination of rule-based and LLM-based strategies to identify biased responses, rigorously validated through human evaluation. Using the generated questions as the test cases, we conduct extensive evaluations of 10 advanced LLMs. The evaluation reveal 107,580 biased responses across the studied LLMs, with individual models yielding between 7,579 and 16,963 biased responses, underscoring the prevalence of bias in role-playing contexts. To support future research, we have publicly released the dataset, along with all scripts and experimental results.
Title: PreCoF: counterfactual explanations for fairness
Authors: Sofie Goethals; David Martens; Toon Calders
Affiliations: University of Antwerp; University of Antwerp; University of Antwerp
Year: 2023
Domain: General Fairness & Bias Mitigation

Abstract: Artificial intelligence (AI) models are increasingly used in health care for risk stratification and early warnings. However, if biased, these models can underperform for certain patient groups, leading to unequal care and missed opportunities. This study audited a clinical prediction model for adverse cardiovascular events and identified significant disparities in recall across race, gender, and age. At the original alert threshold, recall for female patients was 27% versus 44% for males; for Black patients, 31% versus 39% for White patients. To address these gaps, subgroup-specific threshold recalibration was applied, lowering the alert cutoff for underserved groups. This intervention substantially improved sensitivity (by ~13–14 percentage points) for the targeted groups with minimal performance loss elsewhere. Statistical testing confirmed the gains were significant (McNemar’s p < 0.05). Through stakeholder consultation, the trade-off – more false positives in exchange for identi fying more at-risk patients – was deemed acceptable. The study outlines how fairness audits and threshold adjustments can be implemented within clinical decision support systems using existing governance structures. Findings demonstrate a scalable strategy to reduce algorithmic bias in predictive models, supporting equitable care delivery and aligning with emerging standards for responsible AI in medicine.
Title: Generative AI and the Foundation Model Era: A Comprehensive Review
Authors: Abdussalam Elhanashi; Siham Essahraui; Pierpaolo Dini; Davide Paolini; Qinghe Zheng; Sergio Saponara
Affiliations: University of Pisa; Mohamed I University; University of Pisa; University of Pisa; Shandong Management University; University of Pisa
Year: 2026
Domain: LLMs & NLP

Abstract: Generative artificial intelligence and foundation models have changed machine learning by allowing systems to produce readable text, realistic images, and other multimodal content with little direct input from a user. Foundation models are large neural networks trained on very large and varied datasets, and they form the core of many current generative AI (GenAI) systems. Their rapid development has led to major advances in areas like natural language processing, computer vision, multimodal learning, and robotics. Examples include GPT, LLaMA, and diffusion-based architectures, such as models often used for image generation. Systems such as Stable Diffusion show this shift by illustrating how AI can interpret information, draw basic inferences, and produce new outputs using more than one type of data. This review surveys common foundation model architectures and examines what they can do in generative tasks. It reviews Transformer, diffusion, and multimodal architectures, focusing on methods that support scaling and transfer across domains. The paper also reviews key approaches to pretraining and fine-tuning, including self-supervised learning, instruction tuning, and parameter-efficient adaptation, which support these systems’ ability to generalize across tasks. In addition to the technical details, this review discusses how GenAI is being used for text generation, image synthesis, robotics, and biomedical research. The study also notes continuing challenges, such as the high computing and energy demands of large models, ethical concerns about data bias and misinformation, and worries about privacy, reliability, and responsible use of AI in real settings. This review brings together ideas about model design, training methods, and social implications to point future research toward GenAI systems that are efficient, easy to interpret, and reliable, while supporting scientific progress and ethical responsibility.
Title: From Detection to Mitigation: Addressing Bias in Deep Learning Models for Chest X-Ray Diagnosis
Authors: Clemence Mottez; Louisa Fay; Maya Varma; Sophie Ostmeier; Curtis P. Langlotz
Affiliations: Artificial Intelligence in Medicine (Canada); University Children's Hospital Tübingen; Artificial Intelligence in Medicine (Canada); Artificial Intelligence in Medicine (Canada); Artificial Intelligence in Medicine (Canada)
Year: 2025
Domain: Health & Clinical AI

Abstract: Deep learning models have shown promise in improving diagnostic accuracy from chest X-rays, but they also risk perpetuating healthcare disparities when performance varies across demographic groups. In this work, we present a comprehensive bias detection and mitigation framework targeting sex, age, and race-based disparities when performing diagnostic tasks with chest X-rays. We extend a recent CNN-XGBoost pipeline to support multi-label classification and evaluate its performance across four medical conditions. We show that replacing the final layer of CNN with an eXtreme Gradient Boosting classifier improves the fairness of the subgroup while maintaining or improving the overall predictive performance. To validate its generalizability, we apply the method to different backbones, namely DenseNet-121 and ResNet-50, and achieve similarly strong performance and fairness outcomes, confirming its model-agnostic design. We further compare this lightweight adapter training method with traditional full-model training bias mitigation techniques, including adversarial training, reweighting, data augmentation, and active learning, and find that our approach offers competitive or superior bias reduction at a fraction of the computational cost. Finally, we show that combining eXtreme Gradient Boosting retraining with active learning yields the largest reduction in bias across all demographic subgroups, both in and out of distribution on the CheXpert and MIMIC datasets, establishing a practical and effective path toward equitable deep learning deployment in clinical radiology.