Overview of Homophobia and Transphobia Span Detection in Social Media Comments (2025)

Abstract: The rise and the intensity of harassment and hate speech in social media platforms against LGBTQ+ communities is a growing concern. This work is an initiative to address this problem by conducting a shared task focused on the detection of homophobic and transphobic content in multilingual settings. The task comprises two subtasks: (1) multi-class classification of content into Homophobia, Transphobia, or Non-anti-LGBT+ categories across eight languages and (2) span-level detection to identify specific toxic segments within comments in English, Tamil, and Marathi. This initiative helps the development of explainable and socially re- sponsible AI tools for combating identity-based harm in digital spaces. Multiple teams registered for the task, however only two teams submitted their results, and the results were evaluated using the macro F1 score.

## introduction homophobia and transphobia refer to harmful attitudes and prejudices directed toward individuals who identify as homosexual or transgender foot_0 @xcite @xcite . while the terms may linguistically suggest irrational fear, they more accurately encompass a spectrum of negative biases and discriminatory behaviors against people who are lesbian, gay, bisexual, or transgender @xcite . these biases can manifest in various forms, ranging from subtle expressions such as derogatory language to overt acts of hostility and aggression, contributing significantly to the marginalization and emo-tional distress experienced by lgbtq+ individuals @xcite . recently, the growth of social media has both amplified these challenges and created new avenues for their expression @xcite . while these platforms foster connection and community building, they have also become grounds for the spread of toxic language, including hate speech targeting lgbtq+ communities @xcite . according to the european union, 50% of lgbt persons have been victims of hate speech or hate crime foot_1 . such homophobic and transphobic content online not only reinforces societal prejudices but also inflicts psychological harm @xcite . therefore, the ability to detect and address such harmful language in social media content is essential for cultivating safer, more inclusive digital environments @xcite . this shared task addresses the problem of homophobia and transphobia detection in social media comments. it aims to promote research into the automatic identification and classification of homophobic and transphobic language, with a particular focus on multilingual and under-resourced language contexts. the shared task comprises two components: comment-level classification @xcite and span-level detection @xcite . this involves highlighting the exact phrases that serve as evidence for the classification, enabling a more fine-grained and interpretable analysis. span detection is particularly valuable for building explainable nlp systems that not only flag harmful content but also provide trans-parent justifications for their decisions @xcite . the dataset used for this task is derived from the manually annotated homophobia/transphobia content, which includes youtube comments labeled at the comment level. participants are encouraged to develop robust nlp systems capable of accurately identifying and categorizing hate speech targeting lgbtq+ individuals. by tackling both classification and span detection, this shared task provides a platform for the nlp community to advance techniques for harmful content detection while fostering socially responsible nlp research across diverse linguistic and cultural settings. in the upcoming section, we will describe the task description, dataset statistics, and participants' methodology towards the investigation of homophobia and transphobia detection from the youtube comments on dravidian languages. ## related works span detection, also known as span-based classification or span identification, involves pinpointing the specific segments of a text that contain harmful or toxic content, rather than labeling the entire text as toxic @xcite . this approach is particularly valuable in scenarios where only a small portion of a comment or post contains offensive language, while the remainder may be benign or contextually neutral @xcite . traditional text classification models typically assign a single label to the entire input, which can be limiting in practical content moderation settings. flagging an entire message as toxic based on a minor fragment may lead to unnecessary censorship and hinder constructive discourse. recent research in hate speech detection has increasingly emphasized the importance of explainability and precision @xcite . span-level annotations offer moderators actionable insights by highlighting the exact portions of the text that violate community guidelines, thereby streamlining the moderation process and enabling more targeted interventions @xcite . this is especially crucial in social media contexts where high volumes of usergenerated content make manual review inefficient. in the context of homophobia and transphobia, span detection plays a critical role in identifying instances of identity-based harm @xcite . recent studies such as @xcite have explored the use of fine-grained annotations to detect hate speech against lgbtq+ individuals, highlighting the need for datasets and models that capture identity-specific slurs and implicit hate spans. studies @xcite further illustrate the challenges in reliably detecting toxic content targeted at lgbt communities, showing that span-based approaches can improve both precision and fairness in these cases. these advances underscore the value of targeted span detection for moderating homophobic and transphobic content, offering more transparent and inclusive systems for content moderation. we organized the shared task on homophobia & transphobia with around two subtasks. overall, these tasks are designed not only to address significant technical challenges in the field of nlp but also to contribute to social good by identifying and mitigating harmful content directed at the lgbtq+ community in diverse linguistic contexts. ## dataset statistics social media platforms twitter, facebook, and youtube use user-generated content to shape public opinion, which affects how people perceive things and how they view others. on content relevant to the lgbtq+ community, who frequently engage with such platforms to express their views on various topics. we gathered a multilingual collection of youtube comments about lgbtq+ for task 1. we protected individual privacy by not including personal stories from lgbtq+ individuals in our collection. using the youtube comment scraper tool, we collected comments and manually annotated them with one of three labels: homophobic, transphobic, and non-anti-lgbt+ content. the final dataset languages -english, tamil, malayalam, telugu, kannada, gujarati, hindi, and marathi were annotated following the guidelines outlined in the dataset paper @xcite . the distribution of annotated labels across all languages appears in for task 2, we extended our efforts by annotating spans of text within comments that explicitly or implicitly expressed homophobia or transphobia @xcite . these span-level annotations were carried out in three languages, tamil, english, and marathi, using the sequence labeling approach implemented in the open-source annotation tool doccano. we focused on marking only those portions of text that conveyed discriminatory attitudes, allowing us to take a targeted and strategic annotation approach. ## participants methodology we organized a shared task focused on addressing harmful content that targets lgbtq+ individuals through two essential subtasks. the participants used multiple machine learning and deep learning approaches to tackle these subtasks, especially when working with low-resource and multilingual data. the skv trio team @xcite used a combination of bert and tf-idf embeddings for task 1. the team used bert and tf-idf embeddings for each input before applying dimensionality reduction to tf-idf embeddings to match bert's dimensions. the system combined these embeddings to create a single feature representation, which served as input for training a random forest classifier. the method united semantic depth with statistical feature patterns to produce an interpretable and efficient computational solution. the kec-elite-analysts team used multiple deep learning models to solve task 1 by classifying homophobia and transphobia. the architecture used bidirectional lstm and gru models to extract sequential and contextual language patterns and class weights to handle class imbalance. a textcnn module to detect local n-gram features indicative of toxic expressions. a multilayer perceptron (mlp) trained on averaged word embeddings to incorporate semantic information into the final prediction. the models were designed to generalize across multiple languages, including low-resource and code-mixed languages such as tamil. this multilingual focus ensured robust performance in linguistically diverse and underrepresented languages. ## result and discussion a total of 30 participants registered for our shared task. nevertheless, only two teams submitted results for task 1, and no submissions were received for task 2. the fact that task 2 required identifying specific spans of homophobic and transphobic content may have contributed to its lack of submissions. this probably required more domain knowledge and work, which might have made it difficult for participants to finish in the allotted time. the outcomes of task 1 are displayed. the macro f1 scores for the two participating teams, skv trio and kec-elite-analysts, across the supported languages are shown in in the majority of languages, including gujarati (0.86), telugu (0.87), and kannada (0.81), the skv trio team received the highest scores. their method of training a random forest classifier by combining bert and tf-idf embeddings seems to have successfully identified both statistical and semantic patterns in the data. low-resource and morphologically rich languages may have benefited most from this hybrid embedding approach, as term-level distinctions unique to hate speech patterns are reinforced by tf-idf, while pre-trained contextual models such as bert can offer general language understanding. the model's strong performance in languages with limited resources and varying the dimensionality alignment between embeddings, which also helped the model generalize better across a variety of linguistic structures. the kec-elite-analysts team outperformed the skv trio team in english (0.40) and tamil (0.74), demonstrating notable competence in those languages. their system used an mlp trained on averaged word embeddings in conjunction with a deep learning ensemble comprising bidirectional lstm, gru, and textcnn components. this architecture works well with languages like english, where pre-trained embeddings and deep learning models typically perform reliably due to an abundance of resources, and tamil, where code-mixing and sequential dependencies are common. their system was able to capture subtle patterns in sentence structure, particularly in high-resource or semi-structured languages, because of the ensemble design and the use of class weights to address label imbalance. these findings show that various modeling approaches obtained performance differences across languages, which may have been caused by the variety of languages and the accessibility of data. while kec-elite-analysts' deep learning ensemble approach proved successful in identifying patterns in more resource-intensive or frequently used languages like english and tamil, skv trio's fusion-based feature engineering demonstrated superior generalization across a wider range of languages. the findings highlight how crucial model diversity and adaptability are, especially when working in environments with limited resources and code-mixed languages. they also highlight the need for more research into span-level detection, since future versions of the task might benefit from longer timeframes, more annotation support, or easier baselines for span identification to reduce the barrier to entry. ## conclusion in this shared task, we addressed the challenge of two sub-tasks, which are detecting homophobia and transphobia classification and span identification through multilingual and low-resource languages. a total of 30 participants were registered, only two teams submitted the results for task 1, and no submissions were received for task 2, likely due to the complexity of span annotation and the need for domain-specific understanding within a limited timeframe. the classification results showed the efficacy of various modeling approaches, with deep learning ensembles performing well in high-resource languages and hybrid embedding approaches excelling in low-resource contexts. these results emphasize how crucial flexible, language-sensitive models are for identifying harmful content. although span detection remains a challenging and underexplored area, specifically in low-resource, it is critical for the development of explainable and culturally aware moderation systems. future iterations of this task should aim to reduce entry barriers and further promote research in inclusive and socially responsible nlp.