cair-nepal/ai-bias-research-landscape huggingface dataset:
Title: Understanding and Mitigating Bias in Artificial Intelligence Systems: An Ontological Perspective
Authors: Om Roy; Dhruv Shingala; Priyanka Patel
Affiliations: La Trobe University; La Trobe University; La Trobe University
Year: 2026
Domain: Graph-Based Fairness & Bias Mitigation

Abstract: Aim
 To synthesize literature on algorithmic bias and transparency in artificial intelligence tools used in nursing education and identify common applications, bias types, transparency challenges and mitigation strategies.
 Background
 Artificial intelligence is increasingly embedded in nursing education through tutoring systems, virtual simulations, predictive models, chatbots and automated grading. These tools enhance personalization and efficiency but may introduce bias and opacity that threaten fairness, learning outcomes and student trust.
 Design
 Scoping review.
 Methods
 PubMed, CINAHL, Scopus, IEEE Xplore, ACM Digital Library and grey literature were searched for sources published January 2015–April 2025. Studies were included if they examined artificial intelligence in nursing education and addressed bias, fairness, or transparency. Data were synthesized narratively and organized into thematic categories.
 Results
 Thirty-five studies met the inclusion criteria. Reported tools included tutoring systems, virtual patients, chatbots, predictive analytics and grading technologies. Reported biases were linked to non-representative datasets, narrow scenario design and evaluation criteria that may disadvantage students by race and ethnicity, gender, language background, or learning preferences. Transparency challenges involved proprietary or complex models, limited disclosure of training data and decision rules and a lack of built-in explainability features. Ethical concerns included reduced student autonomy, unequal outcomes across learner groups and diminished trust. Mitigation strategies included more inclusive data selection, integration of explainability features, algorithm audits and stronger institutional oversight.
 Conclusions
 Bias and transparency limitations in artificial intelligence tools pose ethical challenges in nursing. Inclusive design, system transparency and collaboration among educators, developers and institutions are essential for value aligned integration.
Title: Exploring Equity: Integrating Knowledge Graphs in Fairness Testing Methodologies
Authors: Jonathan Bernabe Dabu; Muhammad Abdul Rahim; Muhammad Abid; Shahid Hussain
Affiliations: Universidade Federal de São Paulo; Instituto Federal de Goiás; Idiap Research Institute; Universidade Federal de São Paulo
Year: 2025
Domain: Graph-Based Fairness & Bias Mitigation

Abstract: This study explored the potential of eXplainable Artificial Intelligence (XAI) in raising user awareness of algorithmic bias. This study examined the popular “explanation by example” XAI approach, where users receive explanatory examples resembling their input. As this XAI approach allows users to gauge the congruence between these examples and their circumstances, perceived incongruence then evokes perceptions of unfairness and exclusion, prompting users not to put blind trust in the system and raising awareness of algorithmic bias stemming from non-inclusive datasets. The results further highlight the moderating role of users’ prior experience with discrimination.
Title: Fairness and Biases in Healthcare AI: A Review of Definitions, Metrics, and Mitigation Strategies
Authors: Abdelkader Bouazza; Mohcine Kodad; Mostafa Azizi; Ayoub Oulhayane; Abdelaziz Benkhalifa
Affiliations: Mohammed First University;Mohammed First University;Mohammed First University;Mohammed First University;Mohammed First University
Year: 2026
Domain: Health & Clinical AI

Abstract: Artificial Intelligence (AI) technology has been developed significantly in recent years. The fairness of medical AI is of great concern due to its direct relation to human life and health. This review aims to analyze the existing research literature on fairness in medical AI from the perspectives of computer science, medical science, and social science (including law and ethics). The objective of the review is to examine the similarities and differences in the understanding of fairness, explore influencing factors, and investigate potential measures to implement fairness in medical AI across English and Chinese literature.
Title: Balancing Fairness and Performance in Healthcare AI: A Gradient Reconciliation Approach
Authors: Xiaoyang Wang; Christopher C. Yang
Affiliations: Tsinghua University; Shanghai Jiao Tong University; Tsinghua University; University of Waterloo; Tsinghua University; Tsinghua University; Tsinghua University; Tsinghua University; Tsinghua University; Tsinghua University; Tsinghua University
Year: 2025
Domain: Health & Clinical AI

Abstract: Objective:
 Large Language Models (LLMs) show strong potential in biomedical informatics but frequently generate hallucinated or factually incorrect responses, limiting their clinical utility. This study aims to develop and evaluate a GraphRAG framework using an ontology-grounded knowledge graph that mitigates hallucinations in clinical question answering.
 Methods:
 We designed a domain-specific Resource Description Framework (RDF)/Web Ontology Language (OWL) ontology and knowledge graph using clinical and hospital data from multiple Egyptian institutions. The ontology was integrated with LLMs to enforce structured semantic grounding during question answering. Clinical questions were evaluated under three conditions: (i) baseline ChatGPT-4, (ii) DeepSeek-R1, and (iii) our ontology-grounded framework. Accuracy was evaluated against clinically reported reference answers derived from five peer-reviewed Egyptian hospital studies.
 Results:
 Our GraphRAG framework significantly outperformed baseline models. While ChatGPT-4 achieved 37% accuracy and DeepSeek-R1 achieved 52%, the ontology-grounded approach achieved 98% accuracy (59/60 questions). The hallucination rate was reduced from approximately 63% in ChatGPT-4 and 48% in DeepSeek-R1 to just 1.7% in our framework, representing a relative hallucination reduction of more than 61% relative to state-of-the-art LLMs. The framework further ensured consistent, reproducible answers aligned with clinical knowledge, demonstrating its robustness for healthcare applications.
 Conclusion:
 Ontology-grounded knowledge graphs provide a reliable and verifiable method for mitigating hallucinations in LLM-based clinical question answering. By embedding structured clinical semantics into the reasoning process, the framework enhances factual accuracy, reproducibility, and safety in biomedical informatics. This work highlights the critical role of semantic knowledge representation in building trustworthy Artificial Intelligence (AI) systems for healthcare decision support.
Title: A Practical Framework for Auditing Fairness in Medical AI
Authors: Andreea M. Oprescu et al.
Affiliations: University of Illinois Chicago; University of Illinois Urbana-Champaign
Year: 2025
Domain: Health & Clinical AI

Abstract: Large language models (LLMs) require domain-specific fine-tuning for real-world deployment, yet face critical barriers of data privacy and computational constraints. Federated learning (FL) provides an indispensable solution by enabling collaborative tuning across distributed private data sources while preserving confidentiality. However, existing FL-LLM methods suffer from non-IID degradation, communication overhead, and fairness issues. To address these challenges, this paper proposes FedPSF-LLM, a novel FL framework integrating three core innovations: (1) the Prompt Selection Module (PSM) adaptively selects high-impact prompt parameters to reduce transmission costs; (2) the Dynamic Weighting Module (DWM) adjusts aggregation weights based on client contribution and data disparity; (3) the Attention-Based Bias Mitigation (ABM) corrects aggregation bias via alignment-aware reweighting. Extensive experiments on 10 NLP tasks and 4 LLMs demonstrate that FedPSF-LLM improves fairness while maintaining strong overall performance. Compared to state-of-the-art methods, it reduces accuracy variance by 52.1 %, improves worst-client accuracy by 8.6 %, and narrows small-large client performance gaps by 74.4 %, while maintaining 76.8 % global accuracy. These results demonstrate superiority over 8 baselines in both fairness metrics and communication efficiency, establishing a new paradigm for privacy-preserving and fairness-guaranteed LLM deployment in federated systems.
Title: Health Care for All: Combating Bias in Generative AI for Fair Medical Outcomes
Authors: Khaled ELKarazle; Valliappan Raman; Caslon Chua; Patrick Then
Affiliations: University of Oxford; University of Oxford; University of Oxford; University of Oxford; University of Oxford
Year: 2025
Domain: Health & Clinical AI

Abstract: Reordering input sentences in large language models (LLMs) often leads to semantic inconsistencies in the output, as the intended meaning of the input can change. Permutation bias arises when the overall meaning of the input remains unchanged, yet the output still differs-this issue is particularly evident in tasks involving multiple-choice questions and lists of facts. Recent studies have confirmed that permutation bias is prevalent across various LLMs and types of tasks. To address this problem, we propose the Average Attention Infer module, which leverages the attention mechanism to reduce permutation bias by aligning attention weights across different input permutations. We conducted experiments to demonstrate the effectiveness of our module in mitigating permutation bias compared to strong baselines. Furthermore, our findings confirm that LLMs retain their language processing capabilities after undergoing debiasing, making our module adaptable to other tasks. To further enhance computational efficiency, we provide methods for selecting optimal layers and input permutations. Comparative experiments demonstrate that our method comprehensively outperforms SOTA baselines under conditions of smaller models and lower cost. The code for our work is publicly available. We hope that our research will contribute to fundamentally eliminating permutation bias in LLMs.
Title: A Comprehensive Survey on Bias and Fairness in Generative AI: Legal, Ethical, and Technical Responses
Authors: Xiaojian Lin; Michael Losavio
Affiliations: The University of Hong Kong; University of Louisville
Year: 2025
Domain: General Fairness & Bias Mitigation

Abstract: Large Language Models (LLMs) have transformed Recommendation Systems (RecSys) by enabling more user-aware interactions. However, this shift raises new challenges in evaluating fairness, particularly due to confounding systemic biases such as popularity bias. Conventional fairness assessments often confuse disparities in user treatment with systemic biases toward popular items, resulting in misleading conclusions. In this paper, we introduce PureBiasoMeter, a diagnostic framework that decouples popularity bias from user-level fairness in LLM-based RecSys. Our approach is based on the hypothesis that accurate fairness evaluation requires first mitigating popularity bias and then remeasuring fairness metrics. We generate a comprehensive set of 72 prompts using different user profiling strategies, demographic variants, and bias mitigation instructions. By applying these prompts in a black-box LLM setting, we evaluate fairness sensitivity, recommendation quality, and bias levels across multiple dimensions. Our results demonstrate that removing popularity bias substantially alters fairness measurements and reveals underlying disparities that were previously unknown. PureBiasoMeter thus provides a more reliable basis for fairness analysis and contributes a practical tool for disentangling intertwined sources of bias in modern RecSys.
Title: FAIR-MED: Bias Detection and Fairness Evaluation in Healthcare Focused XAI
Authors: Katsiaryna Bahamazava; Ruairi O'Reilly
Affiliations: Shanghai Normal University; Shanghai International Studies University; Shanghai Normal University
Year: 2025
Domain: Health & Clinical AI

Abstract: Abstract Artificial Intelligence models are increasingly used for classification tasks in healthcare. However, many healthcare professionals and machine learning engineers are still unaware of how these models contribute to and amplify biases. This work introduces a new framework (FAIR-MED) for bias detection and fairness evaluation in healthcare AI models, with a particular emphasis on intersectional fairness, which accounts for the compounded effects of multiple demographic attributes rather than assessing bias in isolation. Current methods often focus solely on data biases and overlook the compounded impact of multiple demographic attributes (e.g., age, gender, socioeconomic status) leading to unequal outcomes across diverse patient populations. To bridge this gap, a comprehensive, model-agnostic framework that incorporates a Compound Fairness Score is proposed. This approach to fairness goes beyond traditional methods by providing insights into the compounded impact of biases across different groups. Additionally, entropy-based weighting is introduced to quantify and aggregate bias metrics in a data-driven manner, ensuring that fairness evaluations prioritize the most impactful sources of bias. The proposed framework is evaluated on widely adopted families of AI models (linear, non-linear and neural network-based approaches) against open-source breast cancer dataset. The results suggest that Neural Networks may be more prone to amplifying existing biases, while Random Forest models tend to exhibit a better fairness balance in this evaluation. Logistic Regression, the most interpretable among the evaluated models, demonstrated overall stability but exhibited noticeable accuracy disparities across age groups. By aligning with transparency and accountability principles outlined in the EU AI Act, FAIR-MED offers a systematic, interpretable, and reproducible approach to bias analysis and fairness assessment, contributing to the development of ethical, equitable, and trustworthy AI-driven healthcare solutions.
Title: From Ethical Declarations to Provable Independence: An Ontology-Driven Optimal-Transport Framework for Certifiably Fair AI Systems
Authors: Sukriti Bhattacharya; Chitro Majumdar
Affiliations:  Luxembourg Institute of Science & Technology;
Chief Investment Risk Strategist for Sovereign Institutions
Founder, RsRL
Year: 2025
Domain: Graph-Based Fairness & Bias Mitigation

Abstract: This paper presents a framework for provably fair AI that overcomes the limits of current bias mitigation methods by systematically removing all sensitive information and its proxies. Using ontology engineering in OWL 2 QL, it formally defines sensitive attributes and infers their proxies through logical reasoning, constructing a sigma algebra G that captures the full structure of biased patterns. Fair representations are then obtained via Delbaen Majumdar optimal transport, which generates variables independent of G while minimizing L2 distance to preserve accuracy. This guarantees true independence rather than mere decorrelation. By modeling bias as dependence between sigma algebras, compiling ontological knowledge into measurable structures, and using optimal transport as the unique fair transformation, the approach ensures complete fairness in tasks like loan approval, where proxies such as ZIP code reveal race. The result is a certifiable and mathematically grounded method for trustworthy AI.
Title: Fairness-aware Cross-Domain Recommendation
Authors: Jiakai Tang; Xueyang Feng; Xu Chen
Affiliations: Seattle University; UCLA Health
Year: 2025
Domain: Recommender Systems

Abstract: The development of algorithms and the spread of social networking services cause filter bubbles that can contribute to the polarization of opinions and the breeding ground for fake news. Based on existing research that suggests virtual experiences influence people’s behavior, we hypothesize that experience able pseudo-filter bubbles can make people aware of the danger of filter bubbles and prevent them from falling into them. In order to generate experience able pseudo-filter bubbles, we aim to generate LLM agents with political ideologies that can imitate Internet actions, such as generating search queries. Furthermore, we mitigate the inherent left-leaning bias of the LLM so that we can generate LLM agents with more various ideologies. As a result, in addition to mitigating agents’ bias through prompt engineering alone, we have succeeded in generating agents with various ideologies, including extreme left-wing and almost extreme right-wing ideologies. This result expands the applicability of prompt engineering and can be the first step in creating pseudo-filter bubbles.
Title: Fairness Through Domain Awareness: Mitigating Popularity Bias for Music Discovery
Authors: Rebecca Salganik; Fernando Díaz; Golnoosh Farnadi
Affiliations: Johns Hopkins University; University of Maryland, College Park; University of Maryland, College Park; World Bank; Accenture (United States); University of Maryland, College Park; Johns Hopkins University
Year: 2024
Domain: Recommender Systems

Abstract: With the rapid development of large language models (LLMs), they have significantly improved efficiency across a wide range of domains. However, recent studies have revealed that LLMs often exhibit gender bias, leading to serious social implications. Detecting, classifying, and mitigating gender bias in LLMs has therefore become a critical research focus. In the NLPCC 2025 Shared Task 7: Chinese Corpus for Gender Bias Detection, Classification and Mitigation Challenge, we investigate how to enhance the capabilities of LLMs in gender bias detection, classification, and mitigation. We adopt reinforcement learning, chain-of-thoughts (CoT) reasoning, and supervised fine-tuning to handle different Subtasks. Specifically, for Subtasks 1 and 2, we leverage the internal reasoning capabilities of LLMs to guide multi-step thinking in a staged manner, which simplifies complex biased queries and improves response accuracy. For Subtask 3, we employ a reinforcement learning-based approach, annotating a preference dataset using GPT-4. We then apply Direct Preference Optimization (DPO) to mitigate gender bias by introducing a loss function that explicitly favors less biased completions over biased ones. Our approach ranked first across all three subtasks of the NLPCC 2025 Shared Task 7.
Title: OKRA: An Explainable, Heterogeneous, Multi-stakeholder Job Recommender System
Authors: Roan Schellingerhout; Francesco Barile; Nava Tintarev
Affiliations: Transnational University Limburg; Transnational University Limburg; Transnational University Limburg
Year: 2025
Domain: Recommender Systems

Abstract: Retrieval-augmented generation (RAG) improves the performance of LLM-based applications by incorporating information from external knowledge bases. However, the introduction of search engines and knowledge sources can also introduce new biases and stereotypes into the system. Previous studies have shown that adjusting the bias of retrievers through fine-tuning can influence the overall bias of the RAG system, mitigating bias in RAG. In this work, we propose a re-ranking-based method, termed ReFaRAG, as an alternative to fine-tuning for controlling the bias in retrieval results. We further investigate how biased retrieval output affects different LLMs within the RAG framework.
Title: Item Popularity Attention for Mitigating Popularity Bias in Sequential Recommendation
Authors: Zhewei Ding; Runxiong Liu; Ming-Wei Lin; Shenbao Yu; Jiayin Lin
Affiliations: Fujian Normal University; Fujian Normal University; Fujian Normal University; Fujian Normal University; Fujian Normal University
Year: 2025
Domain: Recommender Systems

Abstract: Bias within Artificial Intelligence (AI) systems constitutes a profound challenge with substantial implications for fairness, accountability, and societal equity. This paper presents an exhaustive examination of the ontology of bias in AI, delving deeply into its conceptual underpinnings and exploring the intricate algorithmic consequences arising from biased data and models. By establishing a comprehensive and nuanced framework that categorizes diverse manifestations of bias and elucidates their origins, this study aims to foster a profound understanding of how bias permeates AI systems. Integrating interdisciplinary perspectives drawn from philosophy, sociology, and computer science, the ontology of bias is meticulously dissected to reveal its multifaceted nature. Furthermore, the paper investigates the profound impacts of these biases on critical decision-making processes and proposes multifaceted strategies for mitigating bias through ethical design, advanced algorithmic techniques, and stringent regulatory frameworks. Through detailed case studies and empirical analysis, this research highlights the inherent complexities in addressing bias and underscores the imperative for collaborative endeavors to cultivate equitable AI technologies.
Title: Mitigating Popularity Bias for Multi-view Group Recommendation
Authors: Qichang Geng; Xuemin Wang; Chuangying Zhu; Liang Chang; Yu Zeng; Yaorui Gan
Affiliations: Guilin University of Electronic Technology; Guilin University of Electronic Technology; Guilin University of Electronic Technology; Guilin University of Electronic Technology; Guilin University of Electronic Technology; Guilin University of Electronic Technology
Year: 2025
Domain: Recommender Systems

Abstract: In the Big Data era, machine learning and artificial intelligence have reshaped decision-making, especially in sectors like healthcare and finance. However, this evolution raises significant concerns about fairness and bias. The abundance of data increases the risk of inherent biases, exacerbating social inequality and discrimination. Ensuring ethical machine functioning, where every user receives equal treatment, is essential. Conventional methods for handling large datasets struggle to identify and rectify biases, as big data often equals skewed data. To address this, we propose a knowledge graph-based approach alongside grammar-based testing to enhance machine learning fairness. By integrating domain-specific knowledge graphs into the pipeline, biases can be identified and mitigated during preprocessing, ensuring comprehensive fairness. Index Terms—Machine Learning, Artificial Intelligence, Big Data, Fairness, Knowledge Graphs.