Semi-supervised Multi-task Learning for Multi-label Fine-grained Sexism Classification (2020)

Abstract: Sexism, a form of oppression based on one’s sex, manifests itself in numerous ways and causes enormous suffering. In view of the growing number of experiences of sexism reported online, categorizing these recollections automatically can assist the fight against sexism, as it can facilitate effective analyses by gender studies researchers and government officials involved in policy making. In this paper, we investigate the fine-grained, multi-label classification of accounts (reports) of sexism. To the best of our knowledge, we work with considerably more categories of sexism than any published work through our 23-class problem formulation. Moreover, we propose a multi-task approach for fine-grained multi-label sexism classification that leverages several supporting tasks without incurring any manual labeling cost. Unlabeled accounts of sexism are utilized through unsupervised learning to help construct our multi-task setup. We also devise objective functions that exploit label correlations in the training data explicitly. Multiple proposed methods outperform the state-of-the-art for multi-label sexism classification on a recently released dataset across five standard metrics.

## introduction sexism, defined as prejudice, stereotyping, or discrimination based on a person's sex, occurs in various overt and subtle forms, permeating personal as well as professional spaces. while men and boys are also harmed by sexism, women and girls suffer the brunt of sexist mindsets and resultant wrongdoings. with increasingly many people sharing recollections of sexism experienced or witnessed by them, the automatic classification of these accounts into well-conceived categories of sexism can help fight this oppression, as it can better equip authorities formulating policies and researchers of gender studies to analyze sexism. the detection of sexism differs from and can complement the classification of sexism. in a forum where instances of sexism are mixed with other posts unrelated to sexism, sexism detection can be used to identify the posts on which to perform sexism classification. moreover, we observe the distinction between sexist statements (e.g., posts whereby one perpetrates sexism) and the accounts of sexism suffered or witnessed (e.g., personal recollections shared as part of the #metoo movement). we also note the prior work on detecting or classifying personal stories of sexual harassment and/or assault @xcite . in this paper, we focus on classifying an account (report) of sexism involving any set of categories of sexism. most of the existing research on sexism classification @xcite @xcite considers at most five categories of sexism. further, the majority of prior approaches associate only one category of sexism with an instance of sexism. having mutually exclusive categories of sexism is unreasonable and limiting, as substantiated by to the best of our knowledge, @xcite is the only work that explores the multi-label categorization of accounts involving any type(s) of sexism. it provides the largest dataset containing role stereotyping: false generalizations about some roles being more suitable for women; also applies to similar mistaken notions about men moral policing: the promotion of discriminatory guidelines for women under the pretense of morality; also applies to statements that feed into such narratives hostile work environment: sexism suffered at the workplace; also applies when sexism perpetrated by a colleague elsewhere makes working worrisome for the victim accounts drawn from 'everyday sexism project' foot_0 . it contains about 13k textual accounts labeled with at least one of 23 categories of sexism formulated with the help of a social scientist. however, they perform sexism classification among 14 categories derived by merging some sets of categories. this prohibits distinguishing within category pairs such as {moral policing, victim blaming} and {motherhood-related discrimination, menstruation-related discrimination}. we overcome this limitation by carrying out a fine-grained (23-category) classification using the same labeled dataset. given the limited labeled data and large number of categories for our sexism classification, we explore complementary signals for the learning. as far as we know, this paper presents the first multi-task approach for any type of sexism classification. we propose a semi-supervised multi-task multi-label classification approach involving (up to) three tasks. all three tasks are set up automatically, requiring no manual labeling effort; the labels and/or samples needed are created through unsupervised learning or acquired via weak labeling. we obtain unlabeled accounts of sexism from 'everyday sexism project' for unsupervised topic proportion distribution estimation and clustering. we develop a neural multi-task architecture that allows for shared learning across multiple tasks through common layers/weights and a combined loss function. in a multi-label setting, the presence of one label may affect the likelihood of the presence of another. while the label co-occurrences in the training data are indirectly accessible to the standard classifier training (since the entire training data is used during the course of an epoch), we propose principled ways of utilizing them explicitly. we formulate loss functions for multi-label classification relating to pair-wise label correlations. we compute pair-wise label co-occurrence statistics from the training data and use them as targets for the proposed objective functions. through our loss functions, we seek to bring together the actual conditional co-occurrence probabilities and the corresponding model-based probability estimates. in addition to a generic loss function, we present a loss focusing on only nonco-occurring label pairs. we combine a proposed semi-supervised multi-task neural model with our generic loss function to form a method that outperforms numerous baselines with a clear margin. our key contributions are summed up below. in this section, we describe prior work on the classification of sexism. though our work involves accounts of sexism, existing work on the classification of sexist or misogynous statements (e.g., tweets wherein one perpetrates sexism or misogyny) and some distantly related work are also included in this review. we end this section with a brief review of multi-task learning. @xcite apply topic modeling to data obtained from the everyday sexism project and maps the semantic relations between topics. @xcite study user engagement with posts related to gender based violence and their language nuances. since sexism classification can be preceded by sexism detection to remove posts unrelated to sexism, we note that sexism is detected by some hate speech classification methods that include sexism as a category of hate @xcite @xcite . @xcite present an approach for detecting sexism and misogyny from tweets. given our focus on sexism classification, we do not delve into prior work related to hate speech or cyber-bullying @xcite . @xcite explore cnn, rnn, and a combination of them for categorizing personal experiences of sexual harassment into one or more of three classes. in @xcite , a density matrix encoder inspired by quantum mechanics is used for the classification of personal stories of sexual harassment. @xcite employ deep learning methods to classify sexual violence into one of four categories. in @xcite , tweets identified as misogynist are classified as stereotype and objectification, discredit, sexual harassment and threats of violence, dominance, or derailing using features involving part of speech (pos) tags, n-grams, and text embedding. @xcite perform a 4-class categorization of sexist tweets. in @xcite , tweets are classified as benevolent, hostile, or non-sexist using bilstm with attention, svm, and fasttext. while its categorization of sexism pertains to how it is stated, our work concentrates on aspects such as what an instance of sexism involves, where it occurs, and who perpetrates it. @xcite explore multi-label categorization of accounts reporting any kind(s) of sexism. they provide the largest dataset for sexism classification and the state-of-the-art classifier for it. their neural approach can combine sentence embeddings from pre-trained models like bert with those generated using bilstm with attention and cnn. as far as we know, our work presents the first semisupervised approach for the multi-label classification of accounts describing any type(s) of sexism that goes further than using unlabeled instances only for fine-tuning pre-trained models. multi-task learning (mtl) is inspired by human learning activities wherein people often apply the knowledge learned from previous tasks to help learn a new task @xcite . it is useful for multiple (related) tasks to be learned jointly so that the knowledge learned in one task can benefit other tasks. there has been considerable interest in applying mtl to a variety of problems including text classification using deep neural networks (dnns) @xcite @xcite . mtl provides an effective way of leveraging labeled data from auxiliary tasks for the core task, especially when labeled data available for single-task learning is not large. in this work, we adopt mtl for fine-grained multi-label sexism classification using several auxiliary tasks. ## proposed semi-supervised multi-task approach for sexism classification our problem statement is to classify an account of sexism (also referred to as a post henceforth) into one or more of 23 categories of sexism. this section introduces a semi-supervised multi-task approach for it. we begin with the auxiliary task setup and then provide the architecture details. henceforth, we refer to the labeled training set as l and the unlabeled set as u (w.r.t. categories of sexism). ## our label co-occurrence based loss functions for multi-label classification in this section, we propose loss functions which explicitly use label correlations existing in the multilabel data. in order to take advantage of label correlations this way, we compute statistics pertaining to pair-wise label co-occurrences from the training set l and supply them as targets to the loss functions. first, we compute the symmetric label co-occurrence (count) matrix m . we then estimate the probability of label l j occurring given the occurrence of label l i , p (j|i), as m (i, j)/f (i), where f (i) denotes the frequency of label l i . next, we compute the set of label pairs correlated to a specified degree; @xmath0, j | p (j|i) ≤ t, @xmath1, 1 ≤ i, @xmath2, where the threshold t is a hyper-parameter. the proposed loss function l-cor is then given as follows. here, n and q are the numbers of posts and labels respectively. pki is the estimated probability of label l i applying to post x k . l-cor is aimed at minimizing the l1 norms of the differences between the actual conditional co-occurrence probabilities p (j|i) and their estimated model-based counterparts p (j|i) for the label pairs in s. the p (j|i) estimates are derived from the probabilities pki output by the model. to include all non-diagonal labels pairs in the loss computation, t is set to 1. conversely, to impose penalties for only non-co-occurring (non-diagonal) label pairs, we set t to 0 and can remove || from eq. 1 (as ∀(i, j) ∈ s, p (j|i) = 0 when @xmath3. we also devise another loss function targeting non-co-occurring label pairs. we first compute the set of uncorrelated pairs (as per the training data); s @xmath4, j | m (i, j) = 0, @xmath5, 1 ≤ i, @xmath6. using that, our loss function l-unc is computed using the following expression. in effect, for each (i, j) ∈ s u , we sum the model-based conditional co-occurrence probabilities p (j|i) and p (i|j) here. since s u comprises only non-co-occurring label pairs, these co-occurrence probability estimates should be low for the optimal model weights. hence, we minimize their sum. we also explore two variants of the l-cor and l-unc losses. replacing the l1 norm with the l2 norm in eq. 1 constitutes one of them. the other involves replacing the sum of the model-based conditional co-occurrence probabilities in eq. 2 with a model-based co-occurrence score. both underperform l-cor and l-unc.