MUCS @ LT - EDI 2023: Learning Approaches for Hope Speech Detection in Social Media Text (2023)

Abstract: Hope plays a significant role in shaping human thoughts and actions and hope content has received limited attention in the realm of social media data analysis. The exploration of hope content helps to uncover the valuable insights into users’ aspirations, expectations, and emotional states. By delving into the analysis of hope content on social media platforms, researchers and analysts can gain a deeper understanding of how hope influences individuals’ behaviors, decisions, and overall well-being in the digital age. However, this area is rarely explored even for resource-high languages. To address the identification of hope text in social media platforms, this paper describes the models submitted by the team MUCS to “Hope Speech Detection for Equality, Diversity, and Inclusion (LT-EDI)” shared task organized at Recent Advances in Natural Language Processing (RANLP) - 2023. This shared task aims to classify a comment/post in English and code-mixed texts in three languages, namely, Bulgarian, Spanish, and Hindi into one of the two predefined categories, namely, “Hope speech” and “Non Hope speech”. Two models, namely: i) Hope_BERT - Linear Support Vector Classifier (LinearSVC) model trained by combining Bidirectional Encoder Representations from Transformers (BERT) embeddings and Term Frequency-Inverse Document Frequency (TF-IDF) of character n-grams with word boundary (char_wb) for English and ii) Hope_mBERT - LinearSVC model trained by combining Multilingual BERT (mBERT) embeddings and TF-IDF of char_wb for Bulgarian, Spanish, and Hindi code-mixed texts are proposed for the shared task to classify the given text into Hope or Non-Hope categories. The proposed models obtained 1st, 1st, 2nd, and 5th ranks for Spanish, Bulgarian, Hindi, and English texts respectively.

## introduction social media has a profound impact on society, providing a platform for individuals to express their opinions and communicate with others effectively at a much faster rate. it also enables access to diverse opinions, facilitates connection with different individuals, promotes art and culture, and provides a platform for marginalized voices. social media platforms are also being used effectively to spread awareness and support various causes, for instance: inequality, human rights violations, discrimination, health and wellness, environmental concerns, etc. @xcite . while constructive criticism have fostered healthy discussions, the misuse of freedom of speech on social media has become a prevalent issue @xcite . trolling and online bullying have become unfortunate consequences of this freedom, causing significant harm to individuals' mental well-being. numerous studies have consistently highlighted the detrimental effects of heavy social media usage, including increased risk of depression, anxiety, loneliness, self-harm, and even suicidal thoughts @xcite . efforts can be made to reduce these negative thoughts by promoting more positive and supportive hope content on social media. hence, analyzing hope content/speech in social media is considered as an essential determinant for the well-being of users which can also motivate users in a positive way and provide valuable insights into the trajectory of goal-directed behaviors, persistence in the face of misfortunes, and adjusting to positive or negative changes in life @xcite . hope speech detection refers to the analysis of social media content for the detection of inspirational text/posts with positive vibes. however, hope speech detection has rarely been experimented even for high-resource languages @xcite . social media text is often code-mixed and the analysis of code-mixed texts in low-resource languages has been the focus of several studies and workshops (fake news detection @xcite , sentiment analysis @xcite , word level language identification @xcite , machine translation @xcite , threatening language detection @xcite and offensive language identification @xcite ). processing code-mixed texts is challenging due to mixing languages within the same utterance or text. these challenges include tokenization, language identification, linguistic variation, and unavailability of pretrained models trained to represent code-mixed text. to address these challenges, "hope speech detection for equality, diversity, and inclusion" shared task foot_0 at ranlp 2023 foot_1 aims to classify comment/post in english and code-mixed texts in spanish, bulgarian, and hindi, into one of the two predefined categories, namely: "hope" or "non hope". the sample comments/posts from the shared task dataset along with their english translations are shown in in this paper, we team mucs, describe the two binary classification models: hope bert and hope mbert, submitted to "hope speech detection for equality, diversity, and inclusion" shared task @xcite . while hope bert uses a combination of tf-idf of char wb and bert embeddings extracted from bert base models, to train linearsvc for hope speech detection in english, hope mbert makes use of multilingual bert (mbert) embeddings combined with tf-idf of char wb to train linearsvc for hope speech detection in bulgarian, spanish, and hindi code-mixed texts. the code to reproduce the proposed models is available in github foot_2 . the rest of paper is organized as follows: while section 2 describes the recent literature on codemixed text processing and hope speech detection, section 3 focuses on the description of the proposed models submitted to the shared task followed by the experiments and results in section 4. conclusion and future works are included in section 5. ## related work hope is a positive state of mind that is based on an expectation of confident outcomes with respect to an occurrence of any event in one's life. researchers have explored many algorithms to detect the hope speech in text and few of the relevant works are described below: @xcite describes the models to detect hope speech in english, tamil-english and malayalam-english code-mixed texts. the authors proposed three distinct models: i) cohope-mlensemble of machine learning (ml) classifiers (extreme gradient boosting (xgb), and logistic regression (lr), and multilayer perceptron (mlp)) with hard voting, ii) cohope-nn -based on keras neural network (nn) and iii) cohope-tl -bidirectional long short term memory (bil-stm) with 1 dimensional convolutional neural network (1dcnn) trained with bert embeddings. both, cohope-ml and cohope-nn models are trained with character n-grams in the range (3, 6) and syntactic n-grams in the range (2, 3). cohope-ml model outperformed the other models and obtained weighted f1 scores of 0.85, 0.92, and 0.59 for malayalam-english, english and tamil-english texts respectively. balouchzahi et al. (2023) created a dataset to identify hope content in codemixed spanish-english tweets and implemented several baselines based on ml, deep learning (dl), and transfer learning (tl) approaches to benchmark their dataset. their work consists of two subtasks: subtask 1 -a binary classification and subtask 2 -a multiclass classification. tf-idf of word uni-grams are used to train ml models (support vector machine (svm), decision tree (dt), random forest (rf), lr, xgb, mlp, categorical boosting (cb)), global vectors for word representation (glove) and fasttext embeddings are used to train the dl models (long short term memory (lstm), bidirectional lstm, and convolutional neural network (cnn)) and tl based models are trained using bert, robustly optimized bert approach (roberta), mbert, and mlnet features. among all the learning models, lr and cb classifiers outperformed the other models obtaining macro f1 scores of 0.80 and 0.79 for binary and 0.64 and 0.54 for multiclass classifications respectively. the learning models submitted by @xcite aims to classify the given comments in english into 'hope' or 'not-hope' using 1dcnn with lstm model trained with keras embeddings features. using synthetic minority oversampling technique (smote) to handle data imbalance in the dataset, they obtained macro f1 score of 0.55 and weighted f1 score of 0.860. @xcite presents the ensemble model (two dt classifiers and one random forest (rf) classifier) with soft voting to select the best word and character n-grams to train keras nn for hope speech detection. their models obtained weighted f1 scores of 0.870 and 0.790 for english and spanish texts respectively. @xcite presented fine-tuning of a lite bert (albert) -a transformer-based model to detect hope speech in code-mixed dravidian languages (malayalam and kannada) and english. during fine-tuning albert model, they used adam optimizer and obtained weighted average f1 scores of 0.880, 0.740, and 0.750 for english, malayalam, and kannada languages respectively. @xcite created the hope speech dataset with 6,176 user-generated comments in code-mixed kannada language scraped from youtube and manually annotated them as 'hope' or 'not-hope' to detect hope speech. they benchmarked their dataset with ml (lr, k-nearest neighbors (k-nn), dt, rf, and naive bayes), dl (lstm, bilstm, and cnn), and tl (bert, mbert, roberta, roberta-mbert, cross lingual language model roberta, and dual-channel bert (dc-bert4hope)) based approaches. among all the models, rf and dc-bert4hope with roberta-mbert models outperformed other models with weighted f1 scores of 0.706 and 0.752 respectively. from the above literature, it is clear that hope speech detection task is not even explored for highresource languages. it also indicates that there is enough space for developing models and tools for detecting hope speech content in code-mixed lowresource languages. ## methodology the framework of the proposed methodology visualized in figure 1 includes the following steps: configuration values wordpiece vocab size (bert) 30,522 wordpiece vocab size (mbert) 1,19,547 attention heads 12 layers 6 dimension 768 max length 100 ## experiments and results statistics of the datasets provided by the organizers of the shared task is shown in the low macro f1 score for english may be due to severe class imbalance in the english train set. few misclassified comments along with the actual and predicted labels (obtained from hope bert and hope mbert models evaluating on the english and hindi test sets respectively), and the probable reasons for misclassification are shown in from ## conclusion in this paper, we team mucs, presented the description of the proposed models for the "hope speech detection for equality, diversity, and inclusion-lt-edi" shared task at ranlp-2023. the proposed models: hope bert -trained with a combination of tf-idf of char wb and bert embeddings for english texts exhibited a macro f1 score of 0.44 securing 5 th rank and hope mbert trained with a combination of tf-idf of char wb and mbert embeddings for code-mixed spanish, bulgarian, and hindi texts, exhibited macro f1 scores of 0.61, 0.75, and 0.67 securing 1 st , 1 st , and 2 nd ranks for spanish, bulgarian, and hindi respectively. suitable features that could efficiently capture the contextual information from the codemixed text will be explored further.