# Comprehensive Validation on Reweighting Samples for Bias Mitigation via AIF360
## 3 Results and Discussions
### Experimental Results and Discussion
#### 3.3.1 Adult Income  
Figures 1 and 2 illustrate the performance comparison before and after reweighting samples with respect to the protected attribute _Race_. Prior to reweighting, various ML models exhibit distinct biases in AOD and DI curves. Following reweighting, biases in ML models are mitigated to varying degrees. Notably, DT models demonstrate bias-free behavior in terms of AOD and retain bias in DI values. Conversely, biases in LR and KNN appear unchanged, and GNB produced changed with a high optimal classification threshold, while the bias in RF becomes more unstable. This suggests that the effectiveness of reweighting samples depends on the specific ML model and may not be universally applied.  
Figure 1: Performance comparison via BA vs. AOD before and after reweighting samples on Adult Income dataset with respect to the protected attribute _Race_.  
Table 1 provides a systematic comparison using additional fairness metrics. DT's bias is eliminated in terms of AOD, EOD, and TI values. However, regarding SPD and DI values, bias persists towards the unprivileged group, emphasizing the need for a comprehensive examination of bias. Furthermore, although other ML models experience slight reductions in BA, their fairness is marginally improved, emphasizing the inherent trade-off between BA and fairness.  
Figure 2: Performance comparison via BA vs. DI before and after reweighting samples on Adult Income dataset with respect to the protected attribute _Race_.  
Additionally, Figures 3 and 4 depict comparison results for the protected attribute _Sex_. Similar observations are noted, where reweighting samples are effective mainly for DT, less so for KNN, and have varying impacts on LR, GNB, and RF. Table 2 echoes these trends, revealing slight reductions in BA and modest improvements in fairness for other ML models across various fairness metrics (SPD, AOD, DI, EOD, and TI). This underscores the nuanced effectiveness of the same debiasing technique across different ML models.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multicolumn{6}{c}{Performance before reweighting samples} \\ \hline
**Model** & BA & SPD & AOD & DI & EOD & TI \\ \hline DT & 0.7426 & \(-\)0.2416 & \(-\)0.1959 & 0.4196 & \(-\)0.2026 & 0.1130 \\ GNB & 0.7416 & \(-\)0.2952 & \(-\)0.2623 & 0.3252 & \(-\)0.2872 & 0.1111 \\ KNN & 0.7390 & \(-\)0.1904 & \(-\)0.1409 & 0.4882 & \(-\)0.1416 & 0.1207 \\ LR & 0.7437 & \(-\)0.2435 & \(-\)0.1966 & 0.4122 & \(-\)0.2020 & 0.1129 \\ RF & 0.7471 & \(-\)0.2014 & \(-\)0.1336 & 0.5380 & \(-\)0.1097 & 0.1066 \\ \hline \multicolumn{6}{c}{Performance after reweighting samples} \\ \hline DT & 1.0 & \(-\)0.1066 & 0.0 & 0.5863 & 0.0 & 0.0 \\ GNB & 0.7432 & \(-\)0.1147 & \(-\)0.0252 & 0.7379 & 0.0310 & 0.1058 \\ KNN & 0.7390 & \(-\)0.1904 & \(-\)0.1409 & 0.4882 & \(-\)0.1416 & 0.1207 \\ LR & 0.7311 & \(-\)0.0523 & 0.0419 & 0.8508 & 0.1083 & 0.1247 \\ RF & 0.7447 & \(-\)0.1072 & \(-\)0.0201 & 0.7449 & 0.0321 & 0.1081 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Performance comparison between before and after reweighting samples through one classification metric, BA, and fairness metrics including SPD, AOD, DI, EOD, and TI on Adult Income dataset regarding the protected attribute _Race_.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multicolumn{6}{c}{Performance before reweighting samples} \\ \hline
**Model** & BA & SPD & AOD & DI & EOD & TI \\ \hline DT & 0.7426 & \(-\)0.3608 & \(-\)0.3204 & 0.2785 & \(-\)0.3775 & 0.1130 \\ GNB & 0.7416 & \(-\)0.3353 & \(-\)0.2805