cair-nepal/ai-bias-research-landscape huggingface dataset:
Title: EqStruct-GNN: toward fair recommendation via debiased graph structure learning
Authors: Runxiong Liu; Jiayin Lin
Affiliations: Fujian Normal University; Fujian Normal University
Year: 2025
Domain: Recommender Systems

Abstract: Purpose Graph neural networks (GNNs) have been widely applied to various recommendation scenarios due to their outstanding performance in capturing and modelling hidden relationships. However, as the performance of a GNN heavily relies on the graph structure, the structure bias can lead to significant unfairness problems in a GNN-based recommender system across different user groups. Furthermore, prior studies have shown that sensitive attributes of nodes can be revealed through the graph structure. In other words, even if a GNN-based recommendation system does not explicitly disclose a user’s sensitive information, it can still be inferred from the imbalance in users’ connections within the graph. This article therefore addresses the unfairness arising from such structural biases by proposing a novel model designed to mitigate these biases in GNN-based recommendation. Here, the fairness requirements refer to preventing the exposure of sensitive attributes during user modeling and avoiding biased recommendations across different demographic groups. Design/methodology/approach To address this issue, we propose a novel fairness-aware graph neural network model, EqStruct-GNN, which aims to mitigate inter-group unfairness at the structural level. The model is composed of three key modules: (1) the Adaptive Behavior Modeling Module, which computes fairness-aware node behavior probability distributions; (2) the Fairness-Aware Structure Construction Module, which builds graph structures based on these distributions and (3) the Fair Representation Learning Module, which generates unbiased node embeddings. Together, these modules enable EqStruct-GNN to construct a fairness-aware graph that reduces structural bias and improves fairness across sensitive groups. Findings Experimental results demonstrate that the proposed method achieves strong recommendation performance while ensuring fairness across multiple datasets. Furthermore, when applied to various backbone models, EqStruct-GNN not only significantly improves fairness metrics but also maintains or even enhances recommendation accuracy. These results highlight the generalization capability and practical effectiveness of the proposed framework. Originality/value Unlike existing methods that focus on representation-level fairness, our model tackles structural bias at its root by building a fairness-aware debiased graph through a novel graph construction mechanism. This structure enables the learning of fair and high-quality node representations by effectively mitigating structural bias while preserving critical interaction information. In addition, the proposed approach is universally adaptable and can be seamlessly applied to various GNN architectures, demonstrating strong generality and practical applicability.
Title: The Impact of Popularity Bias and Perceived Fairness on Intention to Use Recommender System in Indonesian E-Commerce
Authors: Vaniecia C. Dewi; Nilo Legowo
Affiliations: Binus University; Binus University
Year: 2026
Domain: Recommender Systems

Abstract: This study examines factors influencing users' Behavioral Intention (BI) to use Artificial Intelligence (AI)based recommender systems in Indonesian B2C e-commerce by integrating Popularity Bias (PB) and Perceived Fairness (PF) into the Technology Acceptance Model (TAM). The purpose of this study is to investigate how PB shapes PF and how fairnessrelated perceptions subsequently influence Trust (T), Perceived Usefulness (PU), and users' Behavioral Intention (BI) to use recommender systems. The proposed research model analyzes the relationships among these key behavioral factors, including Cognitive Absorption (CA). A quantitative survey was conducted with 400 active users of major Indonesian ecommerce platforms, and the data were analyzed using Partial Least Squares Structural Equation Modeling (PLS-SEM). The results show that PB affects PF, which plays a central role in shaping T, PU, and BI. CA enhances PU, while T and PU significantly drive users' BI to use recommender systems. These findings emphasize the importance of fairness-aware recommendation mechanisms and extend TAM in the context of AI-based recommender systems.
Title: Fairness Aware Regularization on a Learning-to-Rank Recommender System for Controlling Popularity Bias in E-Commerce Domain
Authors: Dede Kiswanto; Dade Nurjanah; Rita Rismala
Affiliations: Telkom University; Telkom University; Telkom University
Year: 2018
Domain: Recommender Systems

Abstract: Recommender systems aim to recommend items for certain users based on rating prediction, user preferences, or other parameters. Along with their advantages in guiding users to appropriate items, recommender systems potentially experience a popularity bias. It is a condition where popular items will often be recommended, while non-popular items will never be recommended. Controlling popularity bias is necessary to introduce new, but appropriate, items. It can be done by making a fair proportion of popular and non-popular items to be recommended to users. One of the methods is by applying fairness aware regularization. This method can increase the proportion of medium-tail recommendations of non-popular items, without sacrificing ranking performance on the recommender system. We combined this method with a learning-to-rank algorithm and applied it in a recommender system for e-commerce. Based on the testing results by using a dataset of Amazon review, the method is able to increase the proportion of medium-tail recommendation of non-popular items, up to 54.1%, on the measurement of Average Percentage of Tail (APT) and gain 0.281 on the measurement of Normalized Discounted Cumulative Gain (NDCG) score. In addition, the method is also able to introduce different medium-tail non-popular items to users.
Title: A Dual-Adversarial Learning Framework with Explicit Constraints for Two-Sided Fairness in Recommendation System
Authors: Sabrina Karboua; Fouzi Harrag; Farid Meziane
Affiliations: University Ferhat Abbas of Setif; University Ferhat Abbas of Setif; University of Derby
Year: 2026
Domain: Recommender Systems

Abstract: Abstract Recommendation systems frequently exhibit two fundamental types of bias: user-side bias, where recommendations discriminate based on protected attributes such as gender, and item-side bias, where popular items dominate exposure irrespective of their actual relevance or quality. To address both simultaneously, we propose a dual-adversarial framework with two discriminators: a user-gender discriminator designed to mitigate gender bias by removing sensitive information from user embeddings, and an item-popularity discriminator aimed at reducing popularity bias among items. While adversarial learning encourages the model to generate bias-free embeddings by fooling discriminators, residual sensitive information may persist due to imperfect discriminator inference and distributional discrepancies. To further reduce this residual bias and enforce balanced, fair recommendations, we enhance the dual adversarial learning framework with two explicit fairness constraints that operate on the learned debiased user and item embeddings: an individual user fairness constraint that guarantees equitable recommendations for users with similar preferences, and a popularity regularization term that balances exposure across popular and less popular item groups. Experiments on three real-world datasets (MovieLens 1M, MovieLens 100K, and Book-Crossing) demonstrate significant gender predictability drops and popularity skew decreases, while maintaining competitive recommendation quality.
Title: A scoping review and evidence gap analysis of clinical AI fairness
Authors: M. Liu; Yilin Ning et al.
Affiliations: Duke-NUS Medical School; Duke-NUS Medical School; Duke-NUS Medical School; University Hospitals Birmingham NHS Foundation Trust; University of Antwerp; Duke-NUS Medical School; Duke-NUS Medical School; Duke-NUS Medical School; Duke-NUS Medical School; University of Florida Health; Singapore National Eye Center; Singapore General Hospital; Singapore General Hospital; Singapore National Eye Center; Singapore National Eye Center; Singapore National Eye Center; Cornell University; Beth Israel Deaconess Medical Center; Singapore General Hospital; National University of Singapore
Year: 2025
Domain: Health & Clinical AI

Abstract: The ethical integration of artificial intelligence (AI) in healthcare necessitates addressing fairness. AI fairness involves mitigating biases in AI and leveraging AI to promote equity. Despite advancements, significant disconnects persist between technical solutions and clinical applications. Through evidence gap analysis, this review systematically pinpoints the gaps at the intersection of healthcare contexts-including medical fields, healthcare datasets, and bias-relevant attributes (e.g., gender/sex)-and AI fairness techniques for bias detection, evaluation, and mitigation. We highlight the scarcity of AI fairness research in medical domains, the narrow focus on bias-relevant attributes, the dominance of group fairness centering on model performance equality, and the limited integration of clinician-in-the-loop to improve AI fairness. To bridge the gaps, we propose actionable strategies for future research to accelerate the development of AI fairness in healthcare, ultimately advancing equitable healthcare delivery.
Title: Advancing Fairness in Clinical AI Decision-Making through a Sociotechnical Threshold Bias Audit
Authors: Dong-Gil Ko
Affiliations: University of Cincinnati
Year: 2026
Domain: Health & Clinical AI

Abstract: Healthcare AI systems can have large performance differences across demographic groups, despite the absence of comprehensive synthesis of fairness methodologies and evidence-based implementation recommendations. To review the research on fairness in healthcare AI while identifying key gaps, a PRISMA guidelines were followed, resulting in the identification of 129 records and analyzing 38 included papers using a comprehensive taxonomy, by searching IEEE Xplore, Scopus, and ACM Digital Library (2020–2025). This review indicates that publications increased sixteen-fold from 2020 reaching a high in 2023–2024. Medical imaging (47.4%) and EHR-based prediction (36.8%) predominated applications. While 57.9% assessed group fairness, only 15.8% included intersectional bias. Mitigation strategies and techniques included causal/counterfactual methods (21.1%), fairness aware training (18.4%), and data preprocessing (15.8%). Studies focused on race or ethnicity (76.3%) and gender (65.8%), with little emphasis on socioeconomic factors (21.1%). Calibration fairness (26.3%) stands out as important for clinical decisions. Research conducted on fairness in healthcare AI identifies gaps in intersectional analysis, real-world validation, and standardized frameworks. Lastly, we suggest fairness frameworks that are specific to each situation and that combine clinical usefulness with fairness standards, required intersectional evaluation, and global cooperation to make sure that AI reduces health disparities instead of making them worse.
Title: Biased AI: A Case for Positive Bias in Healthcare AI
Authors: Hurmat Ali Shah; Zain Ul Abideen Tariq; Marco Agus; Mowafa Househ
Affiliations: Hamad bin Khalifa University; Hamad bin Khalifa University; Hamad bin Khalifa University; Hamad bin Khalifa University
Year: 2025
Domain: Health & Clinical AI

Abstract: Bias in artificial intelligence (AI) is a pervasive challenge, often reinforcing systemic inequities in healthcare systems. This paper proposes an innovative framework to repurpose bias in AI, leveraging it as a tool for addressing structural injustices and improving outcomes for underrepresented and marginalized groups. Traditional healthcare algorithms often exhibit racial biases, such as underestimating risks for black patients or failing to detect dark-skinned individuals in diagnostic or safety-critical applications. This paper redefines AI bias as a tool for equity, proposing a framework to correct systemic healthcare disparities. By introducing purpose-driven bias, AI can enhance fairness in diagnostics, safety, and medical interventions. The approach involves bias analysis, diverse data curation, and AI fine-tuning to align with fairness objectives. This framework highlights the potential of "biased AI" to drive more inclusive and equitable healthcare.
Title: Governing Healthcare AI in the Real World: How Fairness, Transparency, and Human Oversight Can Coexist
Authors: Paolo Bailo; Giulio Nittari et al.
Affiliations: Università di Camerino; Università di Camerino; Università di Camerino; University of Trieste; University Niccolò Cusano; Università di Camerino
Year: 2026
Domain: Health & Clinical AI

Abstract: Artificial intelligence (AI) is rapidly shifting from experimental pilots to mainstream clinical infrastructure, redefining how evidence, accountability, and ethics intersect in healthcare. This narrative review integrates insights from peer-reviewed studies and policy frameworks to examine seven cross-cutting aspects: bias and fairness, explainability, safety and quality, privacy and data protection, accountability and liability, human oversight, and procurement and deployment. Findings reveal persistent inequities driven by dataset bias and opaque design; the need for explainability tools that are validated, task-specific, and usable by clinicians; and the centrality of post-market surveillance for sustaining patient safety. Privacy-preserving methods such as federated learning and differential privacy show promise but demand rigorous validation and regulatory coherence. Emerging liability models advocate shared enterprise responsibility, while governance-by-design—embedding transparency, auditability, and equity across the AI lifecycle—appears most effective in balancing innovation with public trust. Ethical, legal, and technical safeguards must evolve together to ensure that AI augments, rather than replaces, clinical judgment and institutional accountability.
Title: Mitigating Data Bias in Healthcare AI with Self-Supervised Standardization
Authors: Guipeng Lan; Yong Zhu; Shuai Xiao; Muddesar Iqbal; Jiachen Yang
Affiliations: Tianjin University; Tianjin University; Tianjin University; Prince Sultan University; Tianjin University
Year: 2025
Domain: Health & Clinical AI

Abstract: The rapid advancement of artificial intelligence (AI) in healthcare has accelerated innovations in medical algorithms, yet its broader adoption faces critical ethical and technical barriers. A key challenge lies in algorithmic bias stemming from heterogeneous medical data across institutions, equipment, and workflows, which may perpetuate disparities in AI-driven diagnoses and exacerbate inequities in patient care. While AI's ability to extract deep features from large-scale data offers transformative potential, its effectiveness heavily depends on standardized, high-quality datasets. Current standardization gaps not only limit model generalizability but also raise concerns about reliability and fairness in real-world clinical settings, particularly for marginalized populations. Addressing these urgent issues, this paper proposes an ethical AI framework centered on a novel self-supervised medical image standardization method. By integrating self-supervised image style conversion, channel attention mechanisms, and contrastive learning-based loss functions, our approach enhances structural and style consistency in diverse datasets while preserving patient privacy through decentralized learning paradigms. Experiments across multi-institutional medical image datasets demonstrate that our method significantly improves AI generalizability without requiring centralized data sharing. By bridging the data standardization gap, this work advances technical foundations for trustworthy AI in healthcare.
Title: Empirical Study of Social Bias in Medical Question Answering via Large Language Models
Authors: Xiao Xiao; Jiaxu Zhao; Terry R. Payne; Meng Fang
Affiliations: University of Liverpool; Eindhoven University of Technology; University of Liverpool; University of Liverpool
Year: 2025
Domain: LLMs & NLP

Abstract: The rapid growth of healthcare data and advances in computational power have accelerated the adoption of artificial intelligence (AI) in medicine. However, AI systems deployed without explicit fairness considerations risk exacerbating existing healthcare disparities, potentially leading to inequitable resource allocation and diagnostic disparities across demographic subgroups. To address this challenge, we propose FairGrad, a novel gradient reconciliation framework that automatically balances predictive performance and multi-attribute fairness optimization in healthcare AI models. Our method resolves conflicting optimization objectives by projecting each gradient vector onto the orthogonal plane of the others, thereby regularizing the optimization trajectory to ensure equitable consideration of all objectives. Evaluated on diverse real-world healthcare datasets and predictive tasks—including Substance Use Disorder (SUD) treatment and sepsis mortality—FairGrad achieved statistically significant improvements in multi-attribute fairness metrics (e.g., equalized odds) while maintaining competitive predictive accuracy. These results demonstrate the viability of harmonizing fairness and utility in mission-critical medical AI applications.
Title: BiasFreeBench: a Benchmark for Mitigating Bias in Large Language Model Responses
Authors: Xin Xu; Xiaoqiao He; Churan Zhi; Ruizhe Chen; Julian McAuley; Zexue He
Affiliations: UC San Diego;Columbia University;Zhejiang University;Stanford University
Year: 2025
Domain: LLMs & NLP

Abstract: Existing studies on bias mitigation methods for large language models (LLMs) use diverse baselines and metrics to evaluate debiasing performance, leading to inconsistent comparisons among them. Moreover, their evaluations are mostly based on the comparison between LLMs' probabilities of biased and unbiased contexts, which ignores the gap between such evaluations and real-world use cases where users interact with LLMs by reading model responses and expect fair and safe outputs rather than LLMs' probabilities. To enable consistent evaluation across debiasing methods and bridge this gap, we introduce BiasFreeBench, an empirical benchmark that comprehensively compares eight mainstream bias mitigation techniques (covering four prompting-based and four training-based methods) on two test scenarios (multi-choice QA and open-ended multi-turn QA) by reorganizing existing datasets into a unified query-response setting. We further introduce a response-level metric, Bias-Free Score, to measure the extent to which LLM responses are fair, safe, and anti-stereotypical. Debiasing performances are systematically compared and analyzed across key dimensions: the prompting vs. training paradigm, model size, and generalization of different training strategies to unseen bias types. We release our benchmark, aiming to establish a unified testbed for bias mitigation research.
Title: Ethical perspectives on deployment of large language model agents in biomedicine: a survey
Authors: Nafiseh Ghaffar Nia; Amin Amiri; Yuan Luo; Adrienne Kline
Affiliations: Northwestern University; University of Tennessee at Chattanooga; Northwestern University; Northwestern University
Year: 2025
Domain: LLMs & NLP

Abstract: Abstract Large language models (LLMs) and their integration into agentic and embodied systems are reshaping artificial intelligence (AI), enabling powerful cross-domain generation and reasoning while introducing new risks. Key concerns include hallucination and misinformation, embedded and amplified biases, privacy leakage, and susceptibility to adversarial manipulation. Ensuring trustworthy and responsible generative AI requires technical reliability, transparency, accountability, and attention to societal impact. The present study conducts a review of peer-reviewed literature on the ethical dimensions of LLMs and LLM-based agents across technical, biomedical, and societal domains. It maps the landscape of risks, distills mitigation strategies (e.g., robust evaluation and red-teaming, alignment and guardrailing, privacy-preserving data practices, bias measurement and reduction, and safety-aware deployment), and examines governance frameworks and operational practices relevant to real-world use. By organizing findings through interdisciplinary lenses and bioethical principles, the review identifies persistent gaps, such as limited context-aware evaluation, uneven reporting standards, and weak post-deployment monitoring, that impede accountability and fairness. The synthesis supports practitioners and policymakers in designing safer, more equitable, and auditable LLM systems, and outlines priorities for future research and governance.
Title: Algorithmic Bias from the Perspectives of Healthcare Professionals
Authors: Jennifer Xu; Tamara Babaian
Affiliations: Bentley University; Bentley University
Year: 2025
Domain: Health & Clinical AI

Abstract: The increasing deployment of Artificial Intelligence (AI) systems in healthcare has raised significant concerns about bias, fairness, and ethical implications of automated decision-making. While medical AI systems offer capabilities for diagnosis, treatment planning, and patient care, they also inherit and potentially amplify biases present in training data, particularly affecting underrepresented populations. This paper presents a comprehensive framework for auditing AI systems in medical domains through functional audit processes that systematically evaluate bias and fairness. Our approach integrates three key components: data quality assessment, fairness analysis, and application of explainable AI (XAI) techniques. We demonstrate the practical application through the auditing of a machine learning model to predict COVID-19 patient mortality. The results reveal disparities in model performance across different demographic groups.
Title: Mitigating Nonlinear Algorithmic Bias in Binary Classification
Authors: Wendy Hui; Wai Kwong Lau
Affiliations: Singapore Institute of Technology; The University of Western Australia
Year: 2024
Domain: General Fairness & Bias Mitigation

Abstract: This paper proposes the use of causal modeling to detect and mitigate algorithmic bias that is nonlinear in the protected attribute. We provide a general overview of our approach. We use the German Credit data set, which is available for download from the UC Irvine Machine Learning Repository, to develop (1) a prediction model, which is treated as a black box, and (2) a causal model for bias mitigation. In this paper, we focus on age bias and the problem of binary classification. We show that the probability of getting correctly classified as "low risk" is lowest among young people. The probability increases with age nonlinearly. To incorporate the nonlinearity into the causal model, we introduce a higher order polynomial term. Based on the fitted causal model, the de-biased probability estimates are computed, showing improved fairness with little impact on overall classification accuracy. Causal modeling is intuitive and, hence, its use can enhance explicability and promotes trust among different stakeholders of AI.