Unintended Bias Detection and Mitigation in Misogynous Memes (2024)

Abstract: Online sexism has become a concerning issue in recent years, especially conveyed through memes. Although this alarming phenomenon has triggered many studies from computational linguistic and natural language processing points of view, less effort has been spent analyzing if those misogyny detection models are affected by an unintended bias. Such biases can lead models to incorrectly label non-misogynous memes misogynous due to specific identity terms, perpetuating harmful stereotypes and reinforcing negative attitudes. This paper presents the first and most comprehensive approach to measure and mitigate unintentional bias in the misogynous memes detection model, aiming to develop effective strategies to counter their harmful impact. Our proposed model, the C on t e x tualized S cene G raph-based M ultimodal Net work (CTXSGMNet), is an integrated architecture that combines VisualBERT, a CLIP-LSTM-based memory network, and an unbiased scene graph module with supervised contrastive loss, achieves state-of-the-art performance in mitigating unintentional bias in misogynous memes.Empirical evaluation, including both qualitative and quantitative analysis, demonstrates the effectiveness of our CTXSGMNet framework on the SemEval-2022 Task 5 ( MAMI task) dataset, showcasing its promising performance in terms of Equity of Odds and F1 score. Additionally, we assess the generalizability of the proposed model by evaluating their performance on a few benchmark meme datasets, providing a comprehensive understanding of our approach’s efficacy across diverse datasets.

## introduction in recent years, the proliferation of memes on social media platforms like facebook, twitter, and 1 codes are available at this link: https://www.iitp. ac.in/~ai-nlp-ml/resources.html as-well-as at our github repository: https://github.com/gitanjali1801/ gender_bias instagram has gained significant attention due to their widespread influence and potential to shape public discourse. many memes, despite being humorous, use extremism and dark humor to promote societal harm @xcite @xcite . among the various types of memes, misogynous memes hold a unique place, which exhibits and promote hatred, sexism, derogatory attitudes, harmful stereotypes, and objectification of women, and has become a concerning issue @xcite @xcite @xcite @xcite . while prior research has mostly focused on developing robust deep-learning models to identify such memes @xcite @xcite @xcite @xcite , the effect of the presence of unintentional biases within these models remains a difficult and understudied problem. in this context, @xcite highlights this particular error due to training data when a model misclassifies a meme as misogynous solely based on the presence of a certain image (like a woman) or inclusion of a specific word called identity terms foot_0 like "kitchen," "woman," "dishwasher," "bitch" etc.. these images or identity terms unintentionally introduce bias into the classification task @xcite . more precisely, "a model exhibits unintended bias when its performance is better for samples that include specific identity terms compared to samples that contain others @xcite ". (c.f. figure 1 for such unintended bias in misogynous meme identification task). the identification and mitigation of such unintentional bias in misogynous meme classification are of paramount importance, which has compelled researchers to explore diverse techniques that leverage both textual and visual components of memes @xcite @xcite @xcite . however, a notable research gap remains in effectively leveraging contextual understanding to combat this bias as @xcite highlights the reason for such unintentional bias as the lack of context and external knowledge during classifying misogynous memes. our work addresses this gap by introducing ctxsgmnet (contextualized scene graph-based multimodal network), a novel architecture designed to enhance contextual analysis and mitigate unintended bias from misogynous meme classifiers. ctxsgmnet integrates visualbert, a clip-lstm-based memory network, and an unbiased scene graph @xcite modules. the clip-lstm-based memory network retains contextual information, considering the sequential nature of textual content. simultaneously, the unbiased scene graph module captures the unbiased semantic relationships between objects in memes, facilitating the identification of contextual cues. on top of it, ctxsgmnet improves multimodal representations while training by incorporating supervised contrastive learning (scl) and cross-entropy loss jointly. scl brings instances of the same class closer in the semantic space, promoting fair separation based on misogynous class labels while reducing the impact of identity terms. ## dataset we use mami dataset (semeval2022 task 5, subtask a ) @xcite for conducting all the experiments. furthermore, to show the generalizability of our bias mitigation technique, we perform experiments on three benchmark meme datasets: hateful memes @xcite , memotion2 @xcite and harmful memes @xcite . by considering these diverse datasets (refer identity term list: to mitigate unintended biases, we begin by identifying potentially biased identity words. as shown in figure 1 , these terms significantly impact the final predictions. our approach involves assessing their importance using posttraining shap @xcite values, which provide word-level contributions to false positives (lundberg and lee, 2017; @xcite . we further select potential identity terms by calculating tf-idf word distributions within the misogynous class and ranking them. we ultimately identify the top 50 such terms (refer ## experimental setups all models, including baselines, were developed using huggingface transformer 3 with a default random seed of 42 to ensure applicability. the model is trained for 60 epochs with a batch size of 64 and adam optimizer (kingma and ba, 2015) with a learning rate of 3e-5. the models were trained on a single nvidia-gtx-1080ti with 16bit mixed precision. during the training of the proposed model, through grid search, we fixed the α and β hyperparameters (eqn. 18) in the l ′ 1 as 0.5, and 0.4, respectively. ## error analysis our proposed model, despite its high performance, occasionally makes misclassifications. we categorize them as follows: certain situations where the model overcompensates and misclassifies misogynous memes as non-misogynous due to cautious predictions (see appendix figure 7 (c )) ## conclusion in summary, in this work, we introduce ctxsgm-net, an advanced framework for mitigating unintended bias in misogynous meme identification tasks. ctxsgmnet achieves state-of-the-art performance in addressing unintended bias and promoting inclusivity in meme classification by leveraging contextualized text modeling, scene graph analysis, and multimodal fusion. the empirical evaluation of diverse datasets demonstrates its effectiveness in capturing complex relationships between textual and visual components. this research contributes to combating unintended bias and fostering a more equitable digital environment while inspiring future research in bias mitigation.