# Comprehensive Validation on Reweighting Samples for Bias Mitigation via AIF360
## 2 Methodology
### Fairness Metrics  
This paper employed five fairness metrics to evaluate the effectiveness of reweighting samples for mitigating bias.  
Given sensitive (protected) attributes, **Disparate Impact (DI)**[7] denotes inadvertent bias that may arise when predictions exhibit varying error rates or outcomes across demographic groups, where the sensitive attributes, such as race, sex, disability, and age, are deemed protected. This bias can emerge either from training models on biased data or from the predictive model itself being discriminatory. In the context of this study, Disparate Impact pertains to divergent impacts on prediction results as defined by  
\[DI=\frac{p_{pup}}{p_{pp}}, \tag{6}\]  
where \(p_{pup}\) refers to the prediction probability for the unprivileged samples with positive predictions while \(p_{pp}\) denotes the prediction probability for the privileged samples with positive predictions. If the disparate impact of the predictions approaches 0, it signifies bias in favor of the privileged group. Conversely, if it exceeds 1 it indicates a bias in favor of the unprivileged group. A value of 1 implies perfect fairness in the predictions [15].  
**Average Odds Difference (AOD)**[7] is the average of difference in the False Positive Rates (FPRs) and True Positive Rates (TPRs) between the unprivileged and privileged groups. It is defined by  
\[AOD=\frac{(FPR_{up}-FPR_{p})+(TPR_{up}-TPR_{p})}{2}, \tag{7}\]  
where \(FPR_{up}\) and \(FPR_{p}\) denote the False Positive Rate for unprivileged and privileged samples, respectively, within predictions, while \(TPR_{up}\) and \(TPR_{p}\) refer to the True PositiveRate for unprivileged and privileged samples, respectively, within predictions. A result of 0 signifies perfect fairness. A positive value indicates bias in favor of the unprivileged group, while a negative value indicates bias in favor of the privileged group.  
**Statistical Parity Difference (SPD)**[7] is used to calculate the difference between the ratio of favorable outcomes in unprivileged and privileged groups. It is defined by  
\[SPD=p_{pup}-p_{pp}. \tag{8}\]  
A score below 0 suggests benefits for the unprivileged group, while a score above 0 implies benefits for the privileged group. A score of 0 indicates that both groups receive equal benefits.  
**Equal Opportunity Difference (EOD)**[7] involves evaluating the equal opportunity for benefiting all groups. EOD specifically centers on the True Positive Rate (TPR), representing the accurate identification of positives in both the unprivileged and privileged groups. The measure is defined by  
\[EOD=TPR_{up}-TPR_{p}. \tag{9}\]  
A value of 0 signifies perfect fairness. A positive value indicates bias in favor of the unprivileged group, while a negative value indicates bias in favor of the privileged group.  
**Theil Index (TI)**[7] is also called the entropy index and measures both the group and individual fairness. It is defined by  
\[TI=\frac{1}{n}\sum_{i=1}^{n}\frac{b_{i}}{\mu}ln\frac{b_{i}}{\mu}, \tag{10}\]  
where \(b_{i}=\hat{y_{i}}-y_{i}+1\) and \(\mu\) is the average of \(b_{i}\). A lower absolute value of TI value in this context would indicate a more equitable distribution of classification outcomes, while a higher absolute value suggests greater disparity.