CVF - NITT @ LT - EDI -2025: M isogyny D etection (2025)

Abstract: Online platforms have enabled users to create and share multimodal content, fostering new forms of personal expression and cultural interaction. Among these, memes—combinations of images and text—have become a prevalent mode of digital communication, often used for humor, satire, or social commentary. However, memes can also serve as vehicles for spreading misogynistic messages, reinforcing harmful gender stereotypes, and targeting individuals based on gender. In this work, we investigate the effectiveness of various multimodal models for detecting misogynistic content in memes. We propose a BERT+CLIP+LR model that integrates BERT’s deep contextual language understanding with CLIP’s powerful visual encoder, followed by Logistic Regression for classification. This approach leverages complementary strengths of vision-language models for robust cross-modal representation. We compare our proposed model with several baselines, including the original CLIP+LR, and traditional early fusion methods such as BERT + ResNet50 and CNN + InceptionV3. Our focus is on accurately identifying misogynistic content in Chinese memes, with careful attention to the interplay between visual elements and textual cues. Experimental results show that the BERT+CLIP+LR model achieves a macro F1 score of 0.87, highlighting the effectiveness of vision-language models in addressing harmful content on social media platforms.

## introduction misogyny is broadly defined as the hatred of, aversion to, or prejudice against women. it manifests in various forms, including verbal abuse, stereotyping, objectification, or the dissemination of harmful content through social media. misogyny often appears subtly or overtly in multimodal formats, combining text and imagery to demean, ridicule, or marginalize women. online misogyny represents a pervasive and deeply rooted societal issue that perpetuates gender-based discrimination and inequality in virtual environments. this toxic behavior not only undermines efforts toward achieving gender equity but also significantly discourages women's active participation in online platforms-ranging from social media to professional and educational digital spaces-thereby silencing their voices and limiting their opportunities for expression, representation, and empowerment @xcite . the increasing prevalence of such content on social media platforms necessitates robust detection systems to mitigate its harmful societal impact. the shared task on misogyny meme detection at lt-edi@ldk 2025 focuses on building automatic systems to classify memes as misogynistic or non-misogynistic. this task is especially complex due to the combination of vision and language, as well as the multilingual nature of social discourse-this edition emphasizes chinese language content. the task encourages advancements in multimodal classification and promotes responsible ai development. figure1 depicts a misogynistic meme, shows a cartoon-style illustration of a woman with various derogatory labels surrounding her body. the central theme of the text is body shaming, targeting women with larger body types. the top caption refers to "girls who used to have big butts", setting a critical tone from the start. additional phrases placed around the figure describe her as "covered in fat", having a "barrel waist", and "not only a big butt, but also thick legs". another phrase suggests that she "cares about others' opinions", implying insecurity or social pressure. together, these captions convey a negative and mocking portrayal of women who do not conform to conventional beauty standards, specifically critiquing body size and shape. this image exemplifies misogynistic and fatphobic content, as it reinforces harmful stereotypes and societal expectations about women's appearance. the field of image classification has witnessed considerable advancements due to deep learning models like convolutional neural networks (cnn) @xcite and transformers @xcite , which have revolutionized the way we process and understand visual data. in parallel to these advancements, large language models (llms), have transformed natural language processing by enabling more nuanced understanding and generation of text @xcite . the integration of these two fields-vision and language-has led to the development of vision-language models (vlm), which combine the strengths of both visual and textual data. these models often use both an image encoder and a text encoder to generate embeddings, which can be fused for various multimodal tasks @xcite . in this work, we utilize the vision-language model contrastive language-image pre-training (clip), introduced by openai, to detect misogynistic content in chinese memes. clip is a powerful pretrained model that maps images and text into a shared embedding space, enabling effective joint understanding of visual and textual modalities. we explore clip's capabilities for image representation and pair it with a traditional logistic regression (lr) classifier, striking a balance between performance and computational efficiency. we conducted experiments with traditional multimodal baselines, including bidirectional encoder representations from transformers (bert) @xcite combined with residual networks (resnet50) @xcite , as well as convolutional neural networks (cnn) with in-ceptionv3 @xcite , to benchmark performance across different model architectures. bert+clip+lr model performed well among the models and is made available as an open-source resource on github foot_0 . ## related works the study by @xcite , presents an explainable hateful meme detection model that employs uncertainty-aware dynamic fusion to improve both generalization and interpretability. by dynamically evaluating the uncertainty of visual and textual modalities, the model assigns adaptive weights for feature fusion. they report that visual features are more influential than textual ones in hateful meme detection, and the model's interpretability aids in understanding its decision-making process, although fairness remains a concern for future work. the work by @xcite proposes a probabilistic framework for detecting elements of disagreement in misogynistic memes by analyzing both the visual and textual components. it explores various strategies for leveraging these elements to identify instances where annotators may disagree in their interpretations. the exist 2024 shared task @xcite focuses on advancing research in detecting and countering sexism on social networks, a persistent and complex societal issue. cnn-bilstm for text and resnet50-cnn-bilstm for memes was presented in the paper @xcite to better identify explicit and implicit sexist content. the task fosters the development of effective strategies through a competitive framework aimed at improving content moderation. study by @xcite ) marked a significant step forward in understanding memes by creating carefully labeled, high-quality data for analyzing sentiment, classifying emotions, and gauging their intensity. to demonstrate the value of this resource, they established initial performance benchmarks using both a text-based model and a multimodal model that integrated visual features with textual understanding. their findings highlighted the advantage of considering both text and image content for achieving better results in various meme analysis tasks. @xcite introduced the misogyny detection meme dataset(mdmd), an annotated resource focused on online misogyny within tamil and malayalam-speaking communities, offering valuable insights into gender bias and supporting efforts to combat digital gender-based discrimination. additionally, the literature review highlights that the shared task on misogyny meme detection at lt-edi@ldk 2025 marks the focused initiatives aimed at detecting misogyny in memes. ## methodology vision-language models can be broadly categorized into three types: (1) vision-language understanding (vlu) models, which interpret and reason over visual and textual inputs; (2) text generation with multimodal input, where models generate coherent text based on both image and text inputs; and (3) multimodal input-output models, capable of processing and generating across multiple modalities. clip belongs to the vlu category, as it learns joint image-text representations through contrastive pretraining @xcite . we build upon clip by proposing a bert+clip+lr model, where bert replaces clip's text encoder to enhance contextual language understanding. the resulting image and text embeddings are fused and classified using logistic regression. to benchmark performance, we also evaluate the original clip+lr model and traditional early fusion baselines such as bert + resnet50 and cnn + incep-tionv3, enabling a comparative analysis of modern vlms versus conventional multimodal approaches for harmful content detection. ## experiment setup this section presents the dataset details and describes the experimental configuration used to train and evaluate our models. ## experimental evaluation we evaluated our multimodal architectures for misogyny meme classification using standard metrics, including accuracy, macro precision, macro recall, and macro f1-score. ## conclusion in this study, we investigated a vision-language multimodal approach for misogynistic meme classification. we began with the clip+lr model, where clip's contrastive pretraining effectively aligned image and text features without requiring explicit fusion layers, resulting in a model that was both accurate and computationally efficient. build-ing upon this, we proposed the bert+clip+lr model, which replaces clip's text encoder with bert to capture deeper contextual understanding of language. this enhancement led to improved cross-modal alignment and superior classification performance, while maintaining architectural simplicity. to benchmark our approach, we also implemented traditional early fusion models that combined cnns and bert for text with inceptionv3 and resnet50 for image features. overall, our findings highlight that vision-language models offer a scalable, robust, and efficient solution for multimodal classification tasks, and represent a promising direction for future research in understanding harmful online content. for future work, we aim to develop misogynistic dataset to include more languages and cultural contexts, explore more advanced vision-language models, and investigate methods to detect implicit and context-dependent misogyny in memes.