VTUBGM @ LT - EDI -2023: Hope Speech Identification using Layered Differential Training of ULMF it (2023)

Abstract: Hope speech embodies optimistic and uplifting sentiments, aiming to inspire individuals to maintain faith in positive progress and actively contribute to a better future. In this article, we outline the model presented by our team, VTUBGM, for the shared task “Hope Speech Detection for Equality, Diversity, and Inclusion” at LT-EDI-RANLP 2023. This task entails classifying YouTube comments, which is a classification problem at the comment level. The task was conducted in four different languages: Bulgarian, English, Hindi, and Spanish. VTUBGM submitted a model developed through layered differential training of the ULMFit model. As a result, a macro F1 score of 0.48 was obtained and ranked 3rd in the competition.

## introduction the exponential growth of internet usage has led to a significant rise in the prominence of social media platforms. in recent times, these platforms have evolved to become the primary means of communication for millions of individuals worldwide @xcite . as the popularity of social media continues to soar, people are increasingly sharing their thoughts, opinions, and experiences on various issues. this widespread adoption of social media has revolutionized the way individuals communicate and express themselves. platforms like facebook, twitter, instagram, and youtube have become integral components of our daily lives. these platforms allow users to express their thoughts, ideas, and feelings to a global audience. in recent times, social media has emerged as a powerful tool for social and political activism, as users leverage these platforms to raise awareness about various social issues and rally support for causes. social media has also become a thriving environment for the dissemination of hate speech and fake news, which can spread rapidly across the globe. this harmful content can be targeted toward individuals, groups, religions, or political parties @xcite . extensive research has been conducted to address this issue and develop strategies to identify and prevent such material's publication and rapid spread on the internet. while numerous individuals are involved in promoting fake news and hate speech, several groups and individuals are actively working to create and spread positivity and hope. these efforts have surfaced in various social media campaigns to counteract online platforms' negativity and polarisation. hope is a multifaceted and complex emotion that holds immense importance for human well-being. it encompasses an optimistic outlook on life, enabling us to endure challenging circumstances, nurture the belief that things can get better, and have faith in ourselves and others. hope has been linked to numerous positive outcomes, such as enhanced physical and mental health, heightened resilience, and increased motivation. to promote research in the identification of hope speech on social media platforms, the organizers of the "hope speech detection for equality, diversity, and inclusion" shared task at lt-edi-ranlp 2023 offer an opportunity for researchers to develop machine learning and deep learning models capable of effectively classifying hope speech and non-hope speech within the provided dataset. this initiative encourages the exploration of innovative approaches to accurately distinguish between hopeful and non-hopeful content, thus contributing to the advancement of understanding and harnessing the power of hope in online discourse. in order to construct a model for classifying hope speech and non-hope speech, our team employed the ulmfit model, an lstm-based model that was fine-tuned specifically for the task of hope speech detection. our model achieved third place among the participating teams with a macro f1 score of 0.48. the remainder of the paper is structured as follows: section 2 presents a comprehensive literature review, highlighting key studies and research related to the topic. section 3 explains our proposed model, outlining its architecture and training methodology. the results obtained from our model are discussed in section 4. finally, section 6 concludes the paper by summarizing the key findings and contributions and outlines potential trajectories for future research in this field. ## literature review while numerous researchers have recently focused on identifying offensive and fraudulent content in social media @xcite , only a limited number of studies have been conducted on identifying hope within individuals' opinions expressed on social media. in an environment permeated by toxicity, violence, and discrimination, hope speech is a powerful tool that assists and encourages countless needy individuals. it serves as a beacon of hope for those grappling with personal or professional challenges, including issues related to health, finances, relationships, and more @xcite . identifying and automatically detecting such statements can play a crucial role in facilitating their widespread dissemination. by detecting and recognizing these statements, we can amplify their reach and impact, inspiring and motivating the individuals who create them for the betterment of others. this, in turn, fosters a positive and supportive environment where hope speech can serve as a catalyst for positive change, resilience, and empowerment @xcite . in the realm of hope speech detection, data is collected by performing web crawling techniques. researchers have extensively investigated and conducted studies focused on extracting comments and posts from popular social media platforms, including twitter, facebook, and youtube. these platforms serve as rich sources of valuable data for analyzing and understanding hope speech in online discourse @xcite . in a pioneering effort, (chakravarthi, 2020) developed a comprehensive dataset for hope speech encompassing multiple languages such as english, malayalam, and tamil. this dataset served as the foundation for the shared task called hopeedi, designed to promote research and exploration in the field of detecting hopeful and encouraging content within social media. hopeedi aimed to encourage advancements in understanding and leveraging the power of hope speech in online environments by providing a platform for studying and analyzing such content. @xcite directed their attention towards harnessing classic machine learning classifiers, specifically logistic regression (lr) and support vector machine (svm), for the purpose of categorizing text into hope speech and non-hope speech categories. they utilized tf-idf and character n-gram techniques for feature extraction to achieve this. by leveraging these advanced classifiers and feature extraction methods, their approach demonstrated promising results in accurately identifying and distinguishing between hope speech and non-hope speech texts. this study exemplifies the effectiveness of employing machine learning techniques for the task of hope speech detection. a novel method that employed an ensemble approach for the classification of hope and non-hope speech was introduced by @xcite . their approach involved utilizing both character-level and word-level tf-idf embedding techniques to extract meaningful features from the text data. by combining these two levels of embeddings in an ensemble model, the authors demonstrated an effective approach for accurately categorizing text into hope and non-hope speech categories. this methodology showcased the potential of leveraging multiple embedding techniques in tandem to improve classification performance in the context of hope speech detection. @xcite introduced a strategy that involved extracting features from the given dataset using tf-idf and syntactic n-grams. they further trained a neural network model and a voting classifier using lr, extreme gradient boosting (xgb), and mlp algorithms. additionally, in the context of the hopeedi shared task, a bert model was trained specifically to identify hope speech in english. this bert model showcased remarkable performance, achieving an average f1 score of 0.92. this approach highlights the effectiveness of utilizing advanced language models for hope speech detection, yielding impressive accuracy and classification performance results. in some of the studies, researchers have utilized transformer models for the identification of the hope speech. the embeddings for the text data were extracted using the bert (bidirectional encoder representations from transformers) model, as introduced by @xcite . the bert model was utilized with an embedding length of 768, and a sub-word level tokenizer was employed to tokenize each sentence. specifically, the bert-base-multilingual-cased model was chosen for this method. a convolutional neural network (cnn) was developed to classify the sentences into their respective categories. the rectified linear unit (relu) activation function was used in the cnn design, which is a popular choice for nonlinear transformations. using bert embeddings and the cnn architecture, this method is intended to effectively identify texts as hopeful or non-hopeful. in the domain of hope speech detection, this combination of bert embeddings with cnn architecture shows the possibility for accurate classification results. @xcite proposed an innovative method for effectively categorising minority groups by combining resampling approaches with 1d-convolutional neural networks (cnn) combined with long short-term memory (lstm). this model seeks to solve the widespread issue of imbalanced datasets, in which minority groups are frequently underrepresented. to address this issue, the researchers used resampling techniques to improve minority class representation in the training data, resulting in a more balanced distribution. the model design includes a 1d-cnn with lstm layers, which allows the network to detect local and temporal relationships in the input data. this combination enables the model to learn patterns and features from resampled minority class data successfully. ## methodology this section presents a comprehensive description of the proposed model, which is outlined in two subsections: data preprocessing and model description. ## result and discussion the organizers of the shared task "hope speech detection for equality, diversity, and inclusion-lt-edi-ranlp 2023" evaluated the performance of the models using the macro-f1 score. our team submitted a single run with the ulmfit model. ## conclusion and future enhancements in the shared task "hope speech detection for equality, diversity, and inclusion" at lt-edi-ranlp 2023, our team, vtubgm, proposed a model built upon the ulmfit framework. we employed a layered differential approach to fine-tune the model specifically for classifying hope and