CIC @ LT - EDI - ACL 2022: Are transformers the only hope? Hope speech detection for S panish and E nglish comments (2022)

Abstract: Hope is an inherent part of human life and essential for improving the quality of life. Hope increases happiness and reduces stress and feelings of helplessness. Hope speech is the desired outcome for better and can be studied using text from various online sources where people express their desires and outcomes. In this paper, we address a deep-learning approach with a combination of linguistic and psycho-linguistic features for hope-speech detection. We report our best results submitted to LT-EDI-2022 which ranked 2nd and 3rd in English and Spanish respectively.

## introduction automatic detection of hope-speech has recently grabbed the attention of natural language processing (nlp) researchers @xcite . social media platforms have opened doors for linguists, computer scientists and psychologists to dive deep into multiple forms of human expression @xcite i.e. hate, sadness, joy and love @xcite @xcite @xcite . similar to detecting other forms of expression, hope-speech allows us to understand the human desire for an outcome . the definition of hope @xcite used in past computational studies, explains the association of hope with potential, reassurance, support, inspiration, suggestions and promise during times of misfortune. hope, however, cannot be limited to the understanding of positivity as a sentiment alone, as hope is not "optimism" @xcite . understanding hope in its complete form can help us understand the desired outcomes of a certain person, community, gender or ethnicity. the first step towards the understanding of hope is to distinguish hope from neutral and not-hopeful sentences. to help that, many computational approaches have been tested on hope-speech detection using deep learning/transformer methods and a variety of linguistic features @xcite @xcite . this paper gives a system report of task 1: shared task on hope speech detection for equality, diversity and inclusion at "lt-edi 2022" @xcite . the shared task is an extension of last year's shared task on hope speech detection @xcite . this year the task is converted to a binary classification problem that aims to detect "hope" and "non-hope" classes from youtube comments. we attempted the task in only english and spanish for thorough experimentation. our model comprises a basic sequential neural network with a combination of features including linguistic inquiry and word count (liwc) and n-grams. the paper contributes by developing a deep learning approach that ranked 2 nd in english and 3 rd in spanish for hope speech detection. we also identified psycho-linguistic and linguistic features that work the best for the two languages. the following section gives a detailed description of the methods used in the previous year's shared task. section 3 and 4 explain the dataset statistics and the methodology used to obtain the results. while section 5 and 6 elaborate on the results and conclusions drawn from the paper. ## literature review early research @xcite on identifying hope highlighted the potential of hope in the situation of war through youtube comments. these comments were extracted multilingually (hindi/english) in devanagari and roman scripts. the study used 80/10 train test spit using logistic regression with l2 regularization. the used n-grams (1, 3), sentiment score and 100 dimensional polyglot fasttext embeddings as features. a combination of all features gave an f -1 score of 78.51 (±2.24%). in 2021, the shared-task (chakravarthi and muralidaran, 2021) for hope speech detection was presented at "lt-edi-2021". the task was built on the code-mixed imbalance dataset (?) comprised of youtube comments in english, malayalam, and tamil. the english dataset was divided into three classes namely: "hope" with 2484 comments, "non-hope" with 25,950 comments and "other language" with 27 comments. the literature review only highlights the methodologies and results proposed for hope-speech detection at "lt-edi-2021" in english. a majority voting ensemble approach (upadhyay et al., 2021) with 11 models and fine-tuned pre-trained transformer models (roberta, bert, albert, indicbert) gave us the f-1 score of 0.93%. the same results were achieved in the study, which used a combination of contextualized string embedding (flair), stacked word embeddings and pooled document embedding with recurrent neural network (rnn) (junaida and ajees, 2021). transformer methods all scored f-1 score of 0.93% consistently with many fine-tuned methods such as roberta @xcite , xml-r @xcite , xlm-roberta @xcite , xlm-roberta with tf-idf @xcite , albert with k-fold crossvalidation (chen and kong, 2021) and multilingual-bert model with convolution neural networks (cnn) @xcite . however, these weighted f1-scores present an incomplete picture of the hope speech detection models as none of the models gave us an f-1 score of more than 0.60% in the "hope" class. these high weighted f-1 scores were majorly contributed by the "nonhope" class which had more than 10x times more comments than the "hope" class. we saw a slightly different language model approach in @xcite , where the authors used fnn, sbert and bert to classify the labels after initial detection of the language using multiple language identifiers such as compact language detector 2, langid etc. the approach got achieved 0.92% f-1 score with extremely poor performance on the third label "not language", which was expected due to the imbalance instances in the class label. the best models seen were the ones that performed slightly better in the hopespeech class. since, the shared task was codemixed, only @xcite provided a solution catering to the sentences combined with char sequences for words with malayalam-english and tamil-english code-mixed texts and a combination of word and char n-grams along with syntactic word n-grams for english text. the proposed approach got an f-1 score of 0.92% in english and was also rubust in the low resource languages. the related studies show a huge gap in the understanding of "hope" class as a whole and hence, more impactful features and methods need to be explored. ## dataset the dataset comprises of youtube comments for english and tweets for spanish. the ## methodology the proposed methodology contains two main phases, namely: feature engineering, and model construction. each phase is described below: ## conclusion in this paper, we reported the impact of psycholinguistic and linguistic features on hope speech detection using a non-complex deep learning algorithm. our approach showed that even simple deep learning models can outperform complex language models with a combination of linguistic and psycho-linguistic features. psycho-linguistic features were efficient in both english and spanish tasks which can be due to the nature of hope targeted in the dataset which comprised of only positive comments. our best models ranked 2 nd and 3 rd in english and spanish respectively.