DCU at S em E val-2023 Task 10: A Comparative Analysis of Encoder-only and Decoder-only Language Models with Insights into Interpretability (2023)

Abstract: We conduct a comparison of pre-trained encoder-only and decoder-only language models with and without continued pre-training, to detect online sexism. Our fine-tuning-based classifier system achieved the 16th rank in the SemEval 2023 Shared Task 10 Subtask A that asks to distinguish sexist and non-sexist texts. Additionally, we conduct experiments aimed at enhancing the interpretability of systems designed to detect online sexism. Our findings provide insights into the features and decision-making processes underlying our classifier system, thereby contributing to a broader effort to develop explainable AI models to detect online sexism.

## introduction sexism represents "assumptions, beliefs, theories, stereotypes, and broader cultural narratives that represent men and women as importantly different" @xcite . sexist beliefs construct and strengthen narratives that normalise gender distinctions between males and females by promoting the idea of innate variances between the two genders. according to feminist studies @xcite , occurrences of online sexism and harassment are frequently depicted as "permissible" by presenting them as a kind of amusement or joke. challenging these narratives and working to eliminate gender-based discrimination is crucial to creating a more just and equitable society, especially over the internet. the shared task explainable detection of online sexism (edos) was part of semeval 2023 and there were three subtasks associated with it @xcite . this paper describes the dcu system submitted for all the three subtasks of the edos shared task. subtask a the first subtask is a straightforward binary classification problem, in which the system has to classify a textual phrase or a sentence as sexist or not sexist. ## background we build on fine-tuning pre-trained language models (lms), a method that has improved many tasks in natural language processing (nlp) over the past four years @xcite @xcite @xcite . this method can be categorised as transfer learning as it transfers from one or more pre-training tasks to the target task(s). for transfer learning to be successful, the characteristics of the data used in the pre-training stage should match those of the target task to ensure that the model can transfer relevant knowledge to the new target task. @xcite and @xcite find that the closer the statistical properties of the pre-training tasks are to the target task, the more likely is success of transfer learning. furthermore, it has been observed that continued pretraining on in-domain data improves downstream task performance as compared to relying on outof-domain data only @xcite @xcite @xcite . several recent studies have explored the detection of sexist content in social media. @xcite introduce a unique method for characterising sexist content using speech acts theory. having collected and annotated 12,000 french tweets, they conduct experiments in sexism detection using convolutional neural networks (cnn), bidirectional recurrent lstm networks with an attention mechanism and a bert language model. in contrast, @xcite aim to improve construct validity and reliability in sexism detection by generating adversarial examples. they conduct experiments for sexism detection with cnn and bert language model and detect both online benevolence and hostility, comparing against jigsaw's perspective api @xcite as a baseline. also, @xcite present their sexism identification system for the sexism identification in social networks (exist) shared task at iberlef 2021 @xcite , and they leverage both pre-training and fine-tuning strategies with encoder-only language models, earning them 5th and 6th rank in the shared task. ## system overview our system is based on two foundational encoderonly pre-trained lms, and one decoder-only pretrained lm, described in section 3.1. figure 1 shows the components of our system. the first block shows continued pre-training of the two encoder-only pre-trained lms with additional in-domain data and is described in sections 3.2 and 3.3 below. the second block shows fine-tuning for subtask a, a binary classification task. we fine-tune (a) models from the first block and (b) off-the-shelf models, both encoder and decoder lms. the third block depicts a one-vsall multiclass classification scheme built by finetuning an lm on individual categories of subtask b. predictions are made based on which label receives the highest probabilities from the four binary classifiers. we experiment with both encoder and decoder lms. the final block of the system overview shows a fusion of pre-trained lm-based and traditional machine learning methods for subtask c. this involves training traditional machine learning algorithms, namely logistic regression and nearest neighbor, by aggregating sentence embeddings from both the encoder-and decoder-only pretrained lms. section 3.4 below discusses the task-specific blocks in more detail. we make our implementation available on https://github.com/ kanishk-adapt/semeval-task10 . ## results in this section, we present results from our development steps of building our system. 9 all tables show f1-macro average scores in percentages. ## towards "explainable" online sexism detection taking steps towards "explainable" detection of sexism, we analysed the decisions made by our submitted system for subtask a, where it incorrectly classified certain phrases or sentences as either "sexist" or "not-sexist" as annotated by @xcite . to achieve this, we randomly select ten samples from the test set that our submitted system failed to classify accurately. next, we calculate the mean gradient importance scores for our bestperforming encoder-only lm, hatebert @xcite . these scores are calculated using the integrated gradients algorithm @xcite for pytorch @xcite . to provide a zero-shot comparison with our system, we also use flan-t5 large @xcite and chatgpt. 10 in order to extract signals or rationales for the prediction, we use input prompts as discussed in appendix d. taking an adversarial approach, we hypothesise that there is a bit of ambiguity in the ground truth label released by the competition organizer. in order to validate our hypothesis and provide an explainable analysis, we recruit three social scientists with expertise in gender studies to manually assign labels to the selected samples. the task organizers employed the descriptive approach @xcite for their annotation, but at the time of our experiment, the annotation guidelines were not publicly available. consequently, we utilized 10 a chat product based on the work of @xcite made available by openai https://openai.com/ blog/chatgpt the definition of sexist content provided in the task description to create our own annotation guidelines outlined in appendix e. our annotation guidelines provide clear instructions for annotators to follow, instructing them to, (a) mark if the text is sexist or not sexist, (b) select words that led to their decision, and (c) rate their level of agreement with the generated rationale by flan-t5 and chatgpt. our findings reveal that the inter-rater reliability score among the annotators was 0.4821 for (a), calculated using krippendorff alpha. furthermore, the annotators disagreed with the original annotations in four out of the five phrases that were originally annotated as "not-sexist". this suggests that the ground truth annotations may not have been entirely accurate and that there is a need for more nuanced and detailed annotations for phrases marked as "not-sexist". considering how subjective and emotive a gender-based subject could be, we recommend that future dataset should be annotated or at least validated by persons with considerable expertise in gender-based research. after the competition, we were able to investigate our hypothesis with the dataset released by @xcite . our analysis revealed that there is ambiguity in the ground truth labels provided by the competition organisers. for instance, the sentence -id "sexism2022_english-15683", in the test split was individually annotated as "sexist" by all three annotators recruited by the task organisers. our expert annotators also agreed with this assessment. however, the aggregated labels shared by the task organisers classified the sentence as "not sexist". although @xcite state that expert adjudication is reserved for cases with less than 3/3 agreement (unanimous) in subtask a, our findings indicate that there is still some degree of ambiguity in the ground truth labels provided by the competition organisers. ## conclusion we experimented with two encoder-only and one decoder-only language models (lms) for the shared-task, including bert @xcite , hatebert @xcite . we also explored continued pretraining strategies for encoder-only lms, namely bert and hatebert. our observations support the finding by @xcite that removing the next sentence prediction task during pre-training improves the lm's performance. our results indi-cate that both decoder-only and encoder-only lms, namely hatebert mlm+ and opt-1.3b, yield similar results when fine-tuned by placing a fully connected layer over the [cls] output for binary classification. moreover, we improved subtask a performance with a simple voting mechanism, building an ensemble of both the encoder-and decoderbased classifiers. in subtask b, we find a surprisingly large performance difference between finetuning hatebert and fine-tuning opt in our basic one-vs-all approach, with opt having difficulties in all four binary classification sub-problems. additionally, we found that our approach of fusing traditional machine learning algorithms for classification with document representations obtained with encoder-only lms works well for subtask c. although this approach is simple, our rank indicates that it can be further improved upon. furthermore, our study highlights the importance of interpretability and explainability in detection systems, particularly for sensitive issues such as sexism. by using a combination of methods such as input prompts, annotator feedback, and gradient importance scores, we may gain a better understanding of how these systems make decisions and identify areas for improvement. this can ultimately lead to more reliable and accurate detection of sexism and other forms of discrimination. conference of the north american chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 1415-1420, minneapolis, minnesota. association for computational linguistics. marcos zampieri, preslav nakov, sara rosenthal, pepa atanasova, georgi karadzhov, hamdy mubarak, leon derczynski, zeses pitenis, and çagrı çöltekin. 2020. semeval-2020 task 12: multilingual offensive language identification in social media (offenseval 2020). in proceedings of the fourteenth workshop on semantic evaluation, pages 1425-1447, barcelona (online). international committee for computational linguistics. jing zhang, yimeng zhuang, and yinpei su. 2021. ta-mamc at semeval-2021 task 4: task-adaptive pretraining and multi-head attention for abstract meaning reading comprehension. in proceedings of the 15th international workshop on semantic evaluation (semeval-2021), pages 51-58, online. association for computational linguistics. susan zhang, stephen roller, naman goyal, mikel artetxe, moya chen, shuohui chen, christopher dewan, mona diab, xian li, xi victoria lin, todor mihaylov, myle ott, sam shleifer, kurt shuster, daniel simig, punit singh koura, anjali sridhar, tianlu wang, and luke zettlemoyer. 2022. opt: open pre-trained transformer language models. arxiv 2205.01068v4.