## 3 Materials and Methods
### Models and Experimental Set-Up  
In order to rigorously evaluate the performance and robustness of models detecting sexism and racism in Spanish-language online dialogues, we developed and trained multiple classifiers using EsCorpiusBias. Two variants were trained for each model:  
* Single-turn: Trained on individual comments without dialogue context, thus evaluating the model's ability to detect hate speech solely based on isolated utterances. * Contextualized: Incorporating preceding dialogue turns to provide context, this model addressed the conversational nature of online interactions, capturing discursive nuances critical for accurate classification. Our experiments included three distinct modeling approaches:  
* Logistic regression baseline: A traditional logistic regression model trained on TF-IDF vectorized text features served as an interpretable baseline, helping to ascertain whether simpler linear methods could effectively capture the linguistic features indicative of sexist and racist content. * SpaCy TextCatBOW pipeline: As a lightweight neural baseline, we fine-tuned SpaCy's TextCatBOW architecture, which employs bag-of-words hash embeddings pooled via mean aggregation and fed into a single-layer feedforward classifier. Optimized with Adam and early stopping, this transformer-free model offers high inference speed and serves as a useful benchmark for isolating the impact of contextual embeddings. * Transformer-based Pipeline (SpaCy + BETO): To leverage advanced contextual embeddings, we implemented and fine-tuned a transformer-based model utilizing SpaCy's transformer integration. The configuration incorporated the widely recognized BETO model (dccuchile/bert-base-spanish-wwm-cased, [[https://huggingface.co/dccuchile/bert-base-spanish-wwm-cased](https://huggingface.co/dccuchile/bert-base-spanish-wwm-cased)]([https://huggingface.co/dccuchile/bert-base-spanish-wwm-cased](https://huggingface.co/dccuchile/bert-base-spanish-wwm-cased)), (accessed on 16 June 2025)), specifically configured as follows (see Appendix A for more details):  
* Used the SpaCy pipeline component transformer combined with a textcat_multilabel classifier. Figure 1: Overview of the annotation workflow. * Implemented subtoken pooling strategies (mean pooling) and strided windows of 128 tokens with a stride of 224 tokens, ensuring comprehensive coverage of long conversational texts. * Fine-tuned with the Adam optimizer (learning rate of \(1\times 10^{-5}\), dropout of 0.1), trained for up to 20 epochs or until convergence. Table 6 summarizes the number of examples in each split. For the racism detection task in the contextual (CTX) setting, the training set consists of 792 dialogues, while oversampling the positive class increases it to 1096 examples. The corresponding development set contains 197 examples. Similarly, for sexism detection, the training set has 800 dialogues, increasing to 1258 when oversampled, with a development set of 199 examples. Oversampling was applied to mitigate class imbalance and enhance model sensitivity to the underrepresented positive class. Models were trained using SpaCy's built-in training commands, specifying configurations (-textcat-multilabel, -lang es) to accurately tailor the training environment to Spanish-language multi-label classification tasks. Detailed hyperparameters, including batch sizes, gradient accumulation strategies, and GPU allocation, were explicitly documented to ensure reproducibility and transparency.