## 4 Results
### Explainability Results  
In this section, we explore the explainability of our model by analyzing the SHAP values to understand the contribution of individual tokens to the model's predictions. Explainability is crucial for ensuring that our model's decisions are transparent and interpretable, particularly in sensitive applications such as detecting sexist content. Figure 10 illustrates the SHAP importance and cumulative importance of the top 20 tokens. These tokens significantly contribute to the model's prediction outcomes, as described by Equations (1)-(5). Notably, the most effective tokens predominantly consist of offensive language directed towards women. Additionally, we computed the Sexism Score for texts labeled as 'YES' (indicative of sexism) using Equation (7). To provide a more comprehensive understanding, we present the statistical summary of the Sexism Scores for these texts in Table 7. \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Model** & **Accuracy** & **Precision** & **Recall** & **F1 Score** \\ \hline \multicolumn{5}{c}{**Task A**} \\ Logistic Regression & 0.70 & 0.42 & 0.70 & 0.46 \\ XGBOOST & 0.72 & 0.45 & 0.72 & 0.49 \\ BERT & 0.76 & 0.57 & 0.76 & 0.65 \\ XLM-RoBERTa & 0.76 & 0.57 & 0.76 & 0.65 \\ DistilBERT & 0.77 & 0.74 & 0.77 & 0.72 \\ CustomBERT & **0.79** & **0.77** & **0.79** & **0.76** \\ \hline \multicolumn{5}{c}{**Task B**} \\ Logistic Regression & 0.54 & 0.25 & 0.54 & 0.23 \\ XGBOOST & 0.56 & 0.27 & 0.56 & 0.21 \\ BERT & 0.68 & 0.52 & 0.68 & 0.59 \\ XLM-RoBERTa & 0.67 & 0.51 & 0.67 & 0.58 \\ DistilBERT & 0.69 & 0.66 & 0.69 & 0.65 \\ CustomBERT & **0.71** & **0.69** & **0.71** & **0.68** \\ \hline \multicolumn{5}{c}{**Task C**} \\ Logistic Regression & 0.40 & 0.10 & 0.40 & 0.07 \\ XGBOOST & 0.42 & 0.12 & 0.42 & 0.08 \\ BERT & 0.63 & 0.44 & 0.63 & 0.52 \\ XLM-RoBERTa & 0.64 & 0.46 & 0.64 & 0.53 \\ DistilBERT & 0.65 & 0.62 & 0.65 & 0.60 \\ CustomBERT & **0.67** & **0.65** & **0.67** & **0.64** \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Performance results of CustomBERT and baselines on Tasks A, B, and C. The best performance result per each task is **bolded**. Figure 10: Cumulative importance of top 20 tokens. This statistical summary provides insights into the distribution of Sexism Scores among texts labeled as sexist. The mean score indicates a moderate level of sexism on average, as evidenced by the standard deviation. The median and percentile values offer additional details about the central tendency and variability of the scores. To take a closer look at the distribution of Sexism Scores, we created a histogram, as shown in Figure 11.  
The histogram in Figure 11 depicts the density of Sexism Scores across the dataset. The x-axis represents the Sexism Scores, while the y-axis indicates the density of texts with those scores. Notably, the distribution features two prominent peaks, suggesting a bimodal distribution of Sexism Scores. The first peak, centered around \(-0.4\), corresponds to texts that have been labeled as 'NO' but contain tokens with relatively high SHAP importance in negative contexts, resulting in negative Sexism Scores. The second peak, centered around 0.4, represents texts labeled as 'YES' with high SHAP importance in positive contexts. Additionally, the histogram shows a notable gap around the zero mark, indicating a clear separation between texts identified as sexist and non-sexist by the model. In this study, we opted to set the threshold at 0.95, indicating that we considered tokens contributing to 95% of the cumulative importance, based on Formula (4). Consequently, the number of selected tokens amounted to 197. With this configuration, we identified the most influential tokens and their contributions to the model's decision-making process regarding sexist content. A comprehensive list of all effective tokens can be found in Appendix B. Figure 12 shows the relationship between the threshold and the number of selected tokens.