# Comprehensive Validation on Reweighting Samples for Bias Mitigation via AIF360
## 1 Introduction  
Fairness Artificial Intelligence (AI) endeavors to identify and mitigate bias throughout the entire life cycle of AI technique development, spanning data curation and preparation, modeling, evaluation, and deployment, which is crucial for the successful implementation of trustworthy AI [1]. Bias can manifest in various forms, potentially leading to unfairness in different downstream learning tasks. These biases typically originate from different stages of the Machine-Learning (ML) pipeline, including data curation, algorithm design, and user interactions. Data bias may arise when the data are collected from skewed sources, are incomplete, lack crucial information, or contain errors. Such biases result in unrepresentative or incomplete data, leading to biased outputs. Algorithmic bias stems from biased assumptions or criteria in algorithm design, resulting in biased outputs for downstream tasks. Bias introduced through user interactions occurs when individuals using AI systems inject their own biases or prejudices, whether consciously or unconsciously. To address these sources of bias, various approaches have been proposed. Dataset augmentation involves adding more diverse data to training datasets, enhancing representativeness andreducing bias [2]. Bias-aware algorithms are designed to consider different types of bias, working to minimize their impact on system outputs [3]. User feedback mechanisms, such as human-in-the-loop systems [4], involve soliciting feedback from users to identify and rectify biases in the system. Addressing data bias can be efficiently achieved through sample reweighting, contributing to the advancement of fairness in AI [5]. This method involves assigning weights to each sample based on the ratio of its population proportion to its sampling proportion [6]. The process ensures the dataset becomes discrimination-free through two key steps. Firstly, specific attributes, such as gender and race, are identified within the datasets. Subsequently, higher weights are assigned to samples from underrepresented groups, while lower weights are assigned to those from overrepresented groups with respect to these specific attributes. These steps collectively contribute to achieving balance across all groups, thereby fostering fairness in the final outputs of AI algorithms trained on the reweighed data. Nevertheless, existing efforts appear to lack a thorough and comprehensive evaluation of the effectiveness of reweighting samples in mitigating bias associated with traditional machine-learning models. This paper undertakes a comprehensive evaluation of the effectiveness of sample reweighting in mitigating bias linked with traditional machine-learning models, employing AI Fairness 360 (AIF360) [7]. AIF360 is a versatile open-source library designed to identify and alleviate bias in machine-learning models across the entire AI application lifecycle. It encompasses a comprehensive array of metrics for scrutinizing biases in datasets and models, along with detailed explanations for these metrics and algorithms for bias mitigation. The evaluation in this paper focuses on the reweighting samples methods available in AIF360, applied to classification tasks performed by five traditional machine-learning models: Decision Tree (DT), K Nearest Neighbor (KNN), Gaussian Naive Bayes (GNB), Logistic Regression (LR), and Random Forest (RF). The reweighting process is carried out with respect to privileged attributes such as sex and race. Subsequently, each traditional machine-learning model undergoes classification tasks on both the original datasets and the new datasets resulting from reweighting samples. The comparative analysis involves assessing the performance of these models on the original and new datasets, respectively. This evaluation is based on metrics such as balanced accuracy and fairness metrics. Experimental results highlight the model-specific nature of reweighting samples effectiveness in achieving fairness in traditional ML models and reveal the complexity of bias dynamics.