## 3 Materials and Methods
### Explainability Analysis
This step ensures that our analysis focuses on the most representative data. Specifically, we filter the SHAP scores \(s\) to satisfy the following condition: where \(-\mu\) is the mean of the SHAP scores, \(-\sigma\) is the standard deviation of the SHAP scores. \[\mu-3\sigma\leq s\leq\mu+3\sigma \tag{3}\]  
After filtering outliers, we normalize the remaining SHAP values to derive the importance ratio for each token. The importance ratio for token \(t\) is defined as:  
\[\text{IR}_{t}=\frac{\text{SI}_{t}}{\sum_{k\in T}\text{SI}_{k}} \tag{4}\]  
This step converts the SHAP values into a proportional format, where each ratio represents the token's share of the total impact on the model. Following the normalization, we calculate the cumulative importance of the tokens. Let \(K\) be the total number of tokens sorted by descending importance, and \(\text{IR}_{i}\) be the importance ratio of the \(i\)-th token. The cumulative importance is given by:  
\[\text{CI}_{k}=\sum_{i=1}^{k}\text{IR}_{i}\quad\text{such that}\quad\text{CI}_{k} \leq T_{c} \tag{5}\]  
We establish a threshold, \(T_{c}=0.95\), to select tokens based on their importance. Tokens are selected such that their cumulative importance is less than or equal to the threshold \(T_{c}\), focusing on those that contribute to the first 95% of total SHAP importance. The selected tokens are then employed to calculate a 'Sexism Score' for each text entry in our dataset. This score combines the SHAP scores with a modified predicted hard label. The hard Label(i) is changed to \(-1\) for 'No' labels and \(1\) for 'Yes' labels. \[\text{Label}(i)= \begin{cases}1&\text{if prediction}=\text{YES}\\ -1&\text{if prediction}=\text{NO}\end{cases}  \tag{6}\]  
Let \(\text{SS}(i)\) be the Sexism Score for sentence \(i\) derived from the summed absolute SHAP values of the selected tokens, and Label(i) be the modified label indicator. The Sexism Score for sentence \(i\) is calculated as follows:\[\text{SS}(i)=\sum_{i\in N_{i}}\text{SI}_{t}(i)\times\text{Label}(i) \tag{7}\]  
After calculating the Sexism Scores for the training dataset, we proceed to evaluate the test dataset. For each sentence in the test dataset, we utilize the trained model to generate predictions. Using the SHAP scores computed during the training phase, we calculate the Sexism Score for each sentence in the test dataset. This is achieved by summing the SHAP values of the selected effective tokens based on Table 1.  
Subsequently, we categorize the test dataset into different bins based on the calculated Sexism Scores. This binning allows us to compare the model's performance across varying levels of detected sexism. By analyzing the model's predictive accuracy and other performance metrics within these bins, we can know how well the model generalizes to new data, particularly in identifying and handling sexist content. The overall steps of test dataset evaluation and performance comparison are shown below:  
1. Model Prediction on Test Dataset: For each sentence \(i\) in the test dataset, use the trained model to generate predictions \(g_{i}^{\text{test}}\). 2. Sexism Score Calculation for Test Dataset: Calculate the Sexism Score \(\text{SS}^{\text{test}}(i)\) for each sentence \(i\) in the test dataset using the SHAP scores from the training dataset. This is achieved as follows: \[\text{SS}^{\text{test}}(i)=\sum_{t\in T_{i}}\text{SI}_{t}\times\text{Label}(g_ {i}^{\text{test}})\] (8) where \(\text{SI}_{t}\) are the SHAP importance scores from the training dataset, and \(\text{Label}(g_{i}^{\text{test}})\) is the predicted label for the test sentence \(i\). 3. Binning Based on Sexism Scores: Divide the test dataset into bins based on the calculated Sexism Scores \(\text{SS}^{\text{test}}(i)\). Define bins \(B_{k}\) such that: \[B_{k}=\{i\mid\alpha_{k-1}\leq\text{SS}^{\text{test}}(i)<\alpha_{k}\}\] (9) where \(\alpha_{k}\) are the thresholds for the bins. 4. Performance Comparison: For each bin \(B_{k}\), evaluate the model's performance by calculating metrics such as accuracy, precision, recall, and F1 score.