# A Comprehensive Approach to Bias Mitigation for Sentiment Analysis of Social Media Data
## 3 Proposed Methodology
### Data Preprocessing
Bias-aware models such as the Bias-Regularized Neural Network achieved an F1 score of 79.0%, while models like RoBERTa and DistilBERT reached F1-scores of 89.7% and 88.7%, respectively but still fell behind Bias-BERT. A similar trend was observed for the group of intellectual and learning disabilities, where Bias-BERT scored 93.0% precision, 93.6% recall, and a 93.3% F1-score, again outperforming all other models, including BiasFinder (F1 score: 90.1%). For the psychiatric or mental illness group, Bias-BERT maintained its dominance with a score of 93.5% F1, followed by RoBERTa at 90.3% and DistilBERT at 89.2%. Similarly, for the other disability group, Bias-BERT achieved a 93.6% F1 score, outperforming all baseline and bias-aware models. Our Bias-BERT delivered the best results in all disability groups, underscoring its robustness, fairness, and effectiveness in mitigating bias while ensuring accurate sentiment analysis. ### ROC Analysis

A Receiver Operating Characteristic (ROC) analysis is a great way to evaluate the performance of classification models. By plotting the true positive rate (TPR) against the false positive rate (FPR) across various thresholds, we can assess the balance between sensitivity and specificity. The Area Under the ROC Curve (AUC-ROC) is a handy metric that summarizes the overall performance of the model. A perfect classifier would have an AUC-ROC of 1, while a random guess would be 0.5. Figure 3 presents the ROC curves for our proposed method, as well as the baseline models and existing models from the literature. The AUC-ROC scores clearly highlight that our method consistently outperformed the others, further demonstrating its strength in classification tasks. The ROC analysis really underscored just how effective our method was, particularly when it came to handling bias-aware sentiment analysis. This aligned nicely with the confusion matrix and bias evaluation results, giving us even more confidence in the strength of our approach. ### Matthews Correlation Coefficient (MCC)

The Matthews Correlation Coefficient (MCC) is a solid metric to evaluate classification performance because it takes a balanced approach. It factored in true positives, false positives, true negatives, and false negatives, giving a more rounded view of how well a model is performing. MCC scores range from \(-1\) to \(1\), where \(1\) means perfect predictions, \(0\) implies that the model is essentially guessing, and \(-1\) means it is getting things completely wrong. The formula for calculating the MCC is as follows:

\[\text{MCC}=\frac{(TP\times TN)-(FP\times FN)}{\sqrt{(TP+FP)(TP+FN)(TN+FP)(TN+ FN)}} \tag{10}\]

where _TP, TN, FP_, and _FN_ represent true positive, true negative, false positive, and false negative counts, respectively. We calculated the Matthews Correlation Coefficient for our proposed method and compared it with the baseline models and existing models in the literature. The results are presented in the table below. As we can see in Table 5, our method clearly outperformed both the baseline models and the existing ones from the literature in terms of MCC. This highlighted how much better our approach was at delivering balanced and accurate classification results. Furthermore, newer models such as RoBERTa and DistilBERT showed strong MCC values but still fell short compared to the proposed method. This solidified the effectiveness of our approach in achieving a higher degree of precision, recall, and balanced performance across various categories. Figure 3: ROC curves for the proposed method, baseline models, and existing models in the literature. ### User Feedback Loop Analysis

#### 4.10.1 User Feedback Impact Analysis

To evaluate how the user feedback loop actually affected the model's performance, we ran a comparison of the key metrics both before and after incorporating the feedback. Here is what we carried out:

* First, we gather performance metrics such as accuracy, precision, recall, and F1-score from the model, both before and after feedback was added.