byte LLM @ LT - EDI -2024: Homophobia/Transphobia Detection in Social Media Comments - Custom Subword Tokenization with S ubword2 V ec and B i LSTM (2024)

Abstract: This research focuses on Homophobia and Transphobia Detection in Dravidian languages, specifically Telugu, Kannada, Tamil, and Malayalam. Leveraging the Homophobia/ Transphobia Detection dataset, we propose an innovative approach employing a custom-designed tokenizer with a Bidirectional Long Short-Term Memory (BiLSTM) architecture. Our distinctive contribution lies in a tokenizer that reduces model sizes to below 7MB, improving efficiency and addressing real-time deployment challenges. The BiLSTM implementation demonstrates significant enhancements in hate speech detection accuracy, effectively capturing linguistic nuances. Low-size models efficiently alleviate inference challenges, ensuring swift real-time detection and practical deployment. This work pioneers a framework for hate speech detection, providing insights into model size, inference speed, and real-time deployment challenges in combatting online hate speech within Dravidian languages.

## introduction in light of the growing prevalence of online hate speech, this paper presents the findings of a workshop on detecting lgbtq+ hate speech in social media comments. we focus on developing and evaluating models that can accurately identify and classify homophobic and transphobic slurs, offensive stereotypes, and other forms of hateful language within youtube comment sections @xcite @xcite . while a comment or post in the dataset may contain more than one sentence, the average sentence length in the corpus is one, and annotations are provided at the comment/post level. in the dynamic landscape of social media, concerns about hate speech targeting the lgbtq+ community have gained prominence. this research investigates the challenges of classifying individual social media comments or posts in low-resourced languages. it specifically focuses on dravidian languages spoken in india-telugu, kannada, tamil, and malayalam-while acknowledging that the findings may not be universally applicable to other linguistic contexts. the goal is to develop systems that can effectively identify instances of homophobia or transphobia in these languages. achieving this requires these systems to be adaptable and robust enough to handle the inherent diversity within dravidian linguistics. in the context of detecting homophobia and transphobia in social media, our research employs a unique approach that utilizes a custom-designed tokenizer and a bidirectional long short-term memory (bilstm) architecture. a significant contribution of our work lies in the development of a tokenizer designed to streamline model sizes, enhance operational efficiency, and address the challenges associated with real-time deployment. this tokenizer not only minimizes the computational footprint but also optimizes the overall performance of the models. its unique features empower effective handling of real-time scenarios, providing a versatile solution for deployment challenges. the implementation of bilstm, coupled with our customized tokenizer, showcases significant improvements in the accuracy of hate speech detection, highlighting enhanced sensitivity to linguistic nuances. compact-sized models effectively address inference challenges, ensuring rapid real-time detection and practical deployment. our research establishes a pioneering framework for homophobia and transphobia detection, providing insights into model size, inference speed, and the challenges associated with real-time deployment in combating online hate speech. this paper outlines our methodology, technical advancements, and results, offering a compre-hensive examination of homophobia/transphobia detection in social media comments in dravidian languages. our research aims to contribute not only to the specific challenges of homophobia/transphobia detection in social media comments but also to a broader understanding of effective detection mechanisms applicable to diverse linguistic landscapes. ## related work homophobia and transphobia detection in social media has garnered significant research attention due to its detrimental impact on the lgbtq+ community. various approaches have been explored, each addressing specific challenges and contributing to the development of robust detection systems. @xcite proposed a zeroshot learning framework for detecting homophobic and transphobic comments without labeled data, demonstrating its potential for resourceconstrained scenarios. @xcite addressed the challenge of data scarcity in low-resource languages by presenting a fine-grained dataset and exploring crosslingual transfer learning techniques. their work highlights the effectiveness of transferring knowledge from resource-rich languages to improve detection accuracy in diverse linguistic settings. @xcite explored an svm-based model, achieving notable f1-scores and emphasizing the importance of automatic detection for timely intervention. their work demonstrates the effectiveness of traditional machine learning algorithms in hate speech detection tasks. @xcite investigated deep learning techniques for dravidian languages, highlighting the superiority of indicbert @xcite in addressing low-resource language challenges. this study demonstrates the potential of deep learning models for capturing languagespecific features and improving detection accuracy. @xcite employed a hybrid approach combining word embeddings, svm classifiers, and bert-based transformers @xcite , achieving promising results. their work showcases the potential of combining diverse techniques to leverage their strengths and enhance detection performance. chakravarthi et al. (2022) investigated the use of pseudolabeling for automated homophobia/transphobia detection, demonstrating signifi-cant improvements in model performance. their work emphasizes the importance of robust evaluation and highlights the potential of pseudolabeling for improving model accuracy. the study presents @xcite , a novel framework using reinforcement learning and a concatenated cm-bert representation, excelling in offensive comment classification for transliterated tamil in english with a 90% and 93% micro-average accuracy improvement. it effectively captures language-specific features and nuances, demonstrated through t-sne visualization and graph network comparisons. the broader literature underscores the global prevalence of offensive language, emphasizing the need for protection and proactive measures to mitigate its impact on vulnerable communities @xcite ; @xcite ; @xcite ; @xcite ). this review highlights significant advancements in homophobic and transphobic comment detection. despite notable progress, various challenges persist, necessitating further exploration and development. these challenges include the expansion to encompass more low-resource languages, the creation of robust models tailored for code-mixed content, the integration of contextual information, and the exploration of explainable ai techniques. addressing these challenges will contribute to a more comprehensive and effective approach to combating online hate speech. ## methodology this section unveils the details of our innovative architecture, integrating two crucial components: a dynamic subword embeddings module and a robust bilstm classification module. we explore data preprocessing, subword tokenization, embedding training, and orchestration of our advanced classifier. ## experimental setup our experimental setup is designed to demonstrate the effectiveness of our proposed approach in the context of the shared task on homophobia/transphobia detection in social media comments. we conducted experiments using a 100dimensional embedding model tailored to each language. the embedding sizes were determined based on linguistic characteristics and dataset scale, as outlined in subword tokenization was facilitated by the custom voweltoken subword tokenizer, designed for universality and based on linguistic principles using vowel boundaries. this tokenizer ensured accurate segmentation across diverse languages, including dravidian languages. its rule-based design focused on identifying and segmenting words based on consistent vowel boundary patterns, enhancing precision and reliability in the tokenization process. to evaluate the impact of these subword embeddings, we seamlessly integrated them into our bilstm-based model architecture. the classifica-tionmodel includes a sub-word embedding layer, bi-directional lstm layer, and linear classification layer, utilizing subword embeddings from voweltoken. the bilstm layer features an input size of 100 and a hidden size of 128. additionally, the model utilizes the adam optimizer with a learning rate of 0.001 during training. datasets were partitioned into training, development, and test sets based on the distribution outlined in ## experimental results and discussions the effectiveness of our subword tokenization is evident in the remarkably low perplexity (less than 1.2) achieved after just one training epoch for embeddings. despite the constraints of limited training time and data, this result underscores the efficacy of our subword tokenization approach. to enhance the performance further, it is crucial to leverage more extensive training data, especially focusing on diverse datasets. ## conclusion and future work our research convincingly demonstrates the effectiveness of subword tokenization for homophobia/transphobia detection across dravidian languages. the competitive results achieved with lightweight models highlight the scalability and computational efficiency of our approach. subword embeddings, trained with meticulous preprocessing and tokenization, showcase impressive performance, with dravidian languages securing leading macro f1-scores. this underscores the potential of subword tokenization in tackling online hate speech with resource-efficient models. moving forward, expanding the dataset with diverse multilingual content is crucial for further enhancing accuracy. the technical advantage of training on larger texts for achieving state-of-the-art performance is evident, albeit currently limited by hardware constraints. nevertheless, the lightweight nature of our models, their fast inference speed, and minimal storage requirements render them practical for various tasks beyond homophobia/transphobia detection, including named entity recognition (ner), language identification (lid), sentiment classification, and multiclass classification. we plan to explore their applicability in generative ai for future research, potentially opening doors to even more impactful applications.