# A Comprehensive Approach to Bias Mitigation for Sentiment Analysis of Social Media Data
## 3 Proposed Methodology
### Data Preprocessing
Consistency Check

To validate these trends, we performed a statistical consistency check, confirming that the observed improvements in performance metrics and bias reduction were statistically significant. This indicated that the user feedback loop substantively contributed to sustained improvements in both accuracy and fairness. The longitudinal study demonstrated the efficacy of iterative retraining with user feedback to improve both the model performance and its ability to mitigate bias effectively over time. ### Analysis of Loss Function, Bias Reduction, and Additional Metrics

The loss function is a key player in our approach, consisting of two components: the loss of cross-entropy (\(L_{CE}\)) and the bias-aware loss term (\(L_{bias}\)). The bias-aware term was

Figure 4: Trend analysis of model performance over six months. specifically designed to address any hidden biases in the dataset. In this section, we dive deeper into how this loss function impacted model performance, backed by empirical tests. Table 7 presents the evolution of the components of the loss function in ten iterations of the training process. As the training proceeds, the cross-entropy loss gradually decreases, indicating that the model is learning to predict sentiment labels accurately. The bias-aware loss term also decreases over time, suggesting that the model is successfully reducing the bias in its predictions. In addition to the analysis of loss values, we also examined the Kullback-Leibler (KL) divergence and other bias metrics. The KL divergence quantifies the difference between the model's predictions and a uniform distribution, with a lower divergence indicating a closer match between the two distributions. Table 8 provides a comprehensive view of the model's behavior over ten iterations concerning bias metrics. #### 4.11.1 Kullback-Leibler (KL) Divergence (\(D_{kl}\))

The KL divergence measures the difference between the predicted probability distribution of the model \(P(y_{i}|\mathbf{x}_{i})\) and a uniform probability distribution \(U(y_{i})\). Initially, the divergence is zero, indicating that the predictions of the model closely match a uniform distribution. This suggests that the model starts with no inherent bias. As the iterations progress, the divergence increases, indicating that the predictions of the model are deviating from a uniform distribution. This is expected as the model learns from the data. However, the controlled increase suggests that the model is not overfitting to potential biases in the data. #### 4.11.2. Disparate Impact

Disparate Impact is a fairness metric that quantifies the ratio of positive outcomes for a protected group to the positive outcomes for a nonprotected group. A value of one indicates perfect fairness, while values deviating from one indicate potential bias. The table \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Iteration** & \(\mathbf{L_{CE}}\) & \(\mathbf{L_{bias}}\) & \(\mathbf{L}\) \\ \hline
1 & 0.693 & 0.034 & 0.727 \\
2 & 0.684 & 0.030 & 0.714 \\
3 & 0.675 & 0.026 & 0.701 \\
4 & 0.665 & 0.022 & 0.687 \\
5 & 0.655 & 0.018 & 0.673 \\
6 & 0.645 & 0.015 & 0.660 \\
7 & 0.635 & 0.012 & 0.647 \\
8 & 0.625 & 0.010 & 0.635 \\
9 & 0.615 & 0.008 & 0.623 \\
10 & 0.605 & 0.006 & 0.611 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Results for the loss function and bias reduction terms. \begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Iteration** & \(\mathbf{P(y_{i}|x_{i})}\) & \(\mathbf{U(y_{i})}\) & \(\mathbf{D_{KL}}\) & **Disparate Impact** & **Equal Opportunity Difference** \\ \hline
1 & 0.50 & 0.50 & 0.00 & 1.00 & 0.00 \\
2 & 0.55 & 0.50 & 0.05 & 0.95 & 0.05 \\
3 & 0.60 & 0.50 & 0.10 & 0.90 & 0.10 \\
4 & 0.65 & 0.50 & 0.15 & 0.85 & 0.15 \\
5 & 0.70 & 0.50 & 0.20 & 0.80 & 0.20 \\
6 & 0.75 & 0.50 & 0.25 &shows that the dispersion impact starts at one and gradually decreases. This suggests that as the model trains, it might be favoring one group over another, leading to potential bias.