QC on at S em E val-2023 Task 10: Data Augmentation and Model Ensembling for Detection of Online Sexism (2023)

Abstract: The web contains an abundance of user- generated content. While this content is useful for many applications, it poses many challenges due to the presence of offensive, biased, and overall toxic language. In this work, we present a system that identifies and classifies sexist content at different levels of granularity. Using transformer-based models, we explore the value of data augmentation, use of ensemble methods, and leverage in-context learning using foundation models to tackle the task. We evaluate the different components of our system both quantitatively and qualitatively. Our best systems achieve an F1 score of 0.84 for the binary classification task aiming to identify whether a given content is sexist or not and 0.64 and 0.47 for the two multi-class tasks that aim to identify the coarse and fine-grained types of sexism present in the given content respectively.

## introduction the web provides tremendous value, in part, because of how easy it is to publish new content and for others to access it. this accessibility and openness, however, comes at a cost. today's web with its plethora of user-generated content is rife with toxic, offensive, and obscene language. therefore, it has become essential to moderate this content at scale. removing the toxic content is even more critical for technologies that automatically index and surface web-content to end users, as well as generative language models that utilize this content. to address the need for content moderation, the natural language processing (nlp) and machine learning communities have put considerable effort into the problem of automatically detecting toxic language. in this paper, we describe the system behind our team's submission to the semeval-2023 "explainable detection of online sexism" shared task @xcite . there are three tasks -task a, b and c. task a is a binary classification task and tasks b and c are multi-class classification tasks with 4 and 11 classes respectively. the current dominant approach to detecting toxic language is to fine-tune sequence to sequence transformer @xcite models @xcite @xcite . our system builds on this thread of research, and extends it by applying methods to ensemble deep learning models @xcite @xcite . we ensemble transformers, where each member of the ensemble uses a different base architecture (i.e. bert, roberta, etc.), different training data, and different training hyperparameter configurations. in addition, we explore whether using in-context learning and weak-labels from foundation models, such as gpt-3 @xcite , further improve the performance. across tasks a, b, and c, we find that our ensembles of fine-tuned transformers consistently reach higher macro f1 score on the test set than the best individual model that we would have chosen using macro f1 score on the development set. consistent with the preliminary experiments in @xcite , however, the improvements we observe are relatively small. this encouraging result suggests that ensembling fine-tuned transformers has promise, but there is need for additional research into techniques for selecting members of the ensemble, and for computing the aggregated output. for in-context learning using gpt-3 @xcite , we find that the performance is competitive to the fine-tuned models, but it provides mixed results when used as a member of an ensemble. ## related work work on automatically detecting abusive language on the web goes back to @xcite proposed several features for detecting abu-sive user-generated content in "web 2.0" applications. @xcite proposed two particularly interesting features: template patterns that capture common abusive phrases, and contextual features that capture similarities between the language being classified and "nearby" language (e.g. the text from a parent post on a message board). template patterns of abusive phrases are no longer used as explicit features for abusive language classifiers, but have been used to create model "unit tests" to expose weaknesses of leading systems @xcite . the issue of whether context is important for determining if a piece of text is offensive is still debated @xcite . @xcite was the first to use word vector representations in an abusive language classifier, as opposed to discrete representations of words as in a bag of words approach. they used the @xcite model, but observed marginal improvements over a simpler bag of words approach (0.8 auc from 0.79). @xcite built on @xcite and further experimented with word and paragraph embeddings combined with several "classical" features, such as character and token n-grams, features derived from dependency parses, and a suite of orthographic features. on the dataset used in @xcite , the approach in @xcite further improved the auc to 0.91. surprisingly, however, the character n-grams alone achieved auc of 0.90, suggesting that the word and paragraph embeddings offer only marginal improvements to the problem. early word embedding approaches @xcite learned task specific word vectors on relatively small corpora, on the order of hundreds of thousands of texts. this may explain the underwhelming performance observed in these studies. large "foundation models" have improved this approach. @xcite was one of the first to show that fine-tuning bert for hate speech detection can significantly improve over standard baselines like logistic regression and support vector classifiers. @xcite work showed that bert improves performance the most for more fine-grained classifications of hate speech, meaning going beyond the basic problem of binary classification of the content. there have since been several proposed extensions of the standard transformer finetuning process. for instance, @xcite shows that learning task-specific vocabularies for transformers can marginally improve performance. although fine-tuned transformers have successfully "cracked" many text classification tasks, the problem of hate speech detection remains difficult. progress on the problem is tricky because the judgement of what is offensive is inherently subjective. this has led to an active thread of ongoing research focused on understanding the roles that data collection, annotation guidelines, and annotator biases play on the quality of hate speech classifiers (see, e.g., @xcite @xcite . our system builds on ideas from the emerging literature on methods for creating ensembles of deep, "high-capacity" models @xcite @xcite . the majority of experimental work in the space of model ensemble has been in the area of computer vision. our paper contributes additional data to the open question of whether ensembling of deep models can successfully solve problems in the area of natural language. ## data selection for each classification task, our approach involves ensembling different fine-tunings of pre-trained transformer @xcite based models. these models vary in the selection of: base models, hyper-parameter configurations, and the data used to fine-tune each model. moreover, they make use of weak-labels generated by prompting gpt-3 @xcite using in-context learning. in this section we go into the details of the data we used for model training. for all model trainings, we include the corpus of gold-labeled training data provided for the shared task @xcite . for tasks a, b, and c, we randomly sample 10% of the gold data, and use it as a validation set to perform hyper-parameter tuning. we choose the best model based on the performance on the corresponding task's official development set. in addition to the gold data, we leverage a subset of the unlabeled reddit and gab data released by the task organizers. we apply silver labeling to this data and augment our original training data with it. to further augment the training dataset, we use two existing public datasets: "ex-ist" @xcite and "call me sexist" @xcite . tables 1 , 2 , and 3 show the statistics of the gold, silver, and public datasets utilized for tasks a, b and c respectively. ## models as mentioned earlier, we use an ensemble of transformer-based models @xcite for the three tasks. to this end, we fine-tuned: bert (base-uncased) @xcite , roberta (base and large) @xcite , xlm-roberta (large) @xcite , electra (base) @xcite , bertweet (large) @xcite , minilm (l12-h384) @xcite , albert @xcite . additionally, we utilized in-context learning to prompt gpt-3 @xcite to generate predictions for the development and test set examples for task a. we follow a similar approach to (chiu and alexander, 2021). 1 the prompt format we used was the word "sentence" followed by a randomly selected training set example, then the word "label" followed by either the word "non-sexist" or "sexist", to indicate the label of the preceding example. we included k randomly sampled positive training set examples and k randomly sampled negative training examples for each prompt. 2 at the end of the prompt, we included a development or test set example followed by "label" with no following text, prompting for an example completion with either the word "non-sexist" or "sexist" from the model. we used these prompt completions as predicted labels, which are included in our results tables 4 and 5 . ## experiments as mentioned earlier, we explore fine-tuning individual models for each task. tables 4 , 6 , and 8 report the performance of each individual model, as well as the best ensemble for each of tasks a, b, and c respectively on the development sets. tables 5 , 7 , and 9 report the performance of the best (i.e. submitted) ensemble on the held-out test sets of the three tasks. we report the macro f 1 average of all classes for each task -since it is the official metric for all tasks -as well as the per-class f 1 score. as a baseline for comparison with our individual models, we include a row with results from always predicting the most frequent class in the training data for a given task -"majority baseline" and as a baseline for comparison with our best ensembles, we include results for a simple ensemble of all models trained for each task "all". task a: binary sexism detection for task a, the best performing ensemble includes five finetunings of roberta (large), and predictions obtained from prompting gpt-3, as described in section 4. tasks b and c for both task b and task c we train one set of models a-h on the task c labels using the loss described in section 4.1, with the predicted labels for task c mapped to their corresponding labels for task b. all the models are trained on gold training data set and silver labeled reddit samples. for task b the best performing ensemble includes models a, b, c, d, e which are fine-tunings of bert, electra, roberta (base), roberta (large) and, roberta (large), respectively. for task c the best performing ensemble includes models b, d, e, f which are fine-tunings of electra, roberta (large), roberta (large), and, xlm-roberta respectively. we also include scores from gpt3 (prompting) as model i in when considering our individual task a models, we see little to no drop in performance (0.0 to 0.2 macro f 1 difference) and models b, c, f even improve on this test set. gpt-3 prompting also shows no difference in performance on development and test, although performance is lower than for most of our individually trained task a models. when comparing with the individual models we trained, we see that our best ensemble on the development set out-performs our best individual model (l) for this task. submitting the best ensemble on the development set rather than this best individual model leads to an improvement in both precision (from 0.75 to 0.77) and macro average f 1 (from 0.83 to 0.84). tasks b and c results for tasks b and c show a stronger drop in performance when comparing the development set results to the test set results. this trend applies to not just our best performing ensemble on development, but to all individual models we trained. despite this fact, our submitted ensemble of models which performed best on the development set also achieves the best performance on the test set, compared to both a simple average of all models we trained and compared to individual model performance. when comparing our performance between tasks b and c, we see that our approach of training models for task c using a combined loss for task b and c does allow us to achieve good results on task b as well, when mapping the task c fine-grained predicted labels to task b coarse-grained labels. for these tasks, we also explored a wider range of models compared to task a, which added to the diversity of model predictions we could select from for our ensemble. ## conclusion in this paper, we described our team's submission to semeval-2023's "explainable detection of online sexism" shared task, that aims to identify whether or not a given content is sexist (task a), as well as the broad (task b) and fine-grained (task c) class of sexist content present in the given text (e.g., prejudice, animosity, derogation, etc.). we find that utilizing a combination of data augmentation -both through publicly available datasets as well as silver labeling -and ensembling different transformermodels gives us the best performance. in addition, we also explored using in-context learning using gpt-3, and found that it yields mixed results, helping us achieve better performance for task a but not for tasks b or c. tasks (as explained in section 4.1). all models we trained for tasks b and c use β = 0.25 except for model (e) which uses β = 0.1.