gini U s @ LT - EDI - ACL 2022: Aasha: Transformers based Hope- EDI (2022)

Abstract: This paper describes team giniUs’ submission to the Hope Speech Detection for Equality, Diversity and Inclusion Shared Task organised by LT-EDI ACL 2022. We have fine-tuned the Roberta-large pre-trained model and extracted the last four decoder layers to build a classifier. Our best result on the leaderboard achieve a weighted F1 score of 0.86 and a Macro F1 score of 0.51 for English. We have secured a rank of 4 for the English task. We have open-sourced our code implementations on GitHub to facilitate easy reproducibility by the scientific community.

## task description the hope speech detection for equality, diversity and inclusion task @xcite @xcite aims at identifying hope speech. hope speech, for the given task, is defined as "youtube comments / posts that offer support, reassurance, suggestions, inspiration and insight". hope speech detection is an integral component under the overall theme of making language technologies more equitable, diverse and inclusive. the task is offered in the following languages -english, tamil, span-ish, kannada, and malayalam. we participated in the english language task. this is the second edition of the hope edi shared task. ## results and discussion we observe that there is a non-trivial improvement in our model with respect to the roberta-large model. this is because of the novelty of concatenating the vectors of the last 4 layers of the transformer decoder. we received this inspiration from the bert paper @xcite . we achieve a macro f1 score of 0.47 and a weighted f1 score of 0.86, and after expanding the feature space by concatenation, we obtain a significant difference in the result. we achieve a macro f1 score of 0.8 and a weighted f1 score of 0.93. our best-performing model i.e roberta-large with the last 4 layers concatenated is used for the submission to the leaderboard(lb), and it obtains a macro f1 score of 0.51 and a weighted f1 score of 0.86. this is competitive with the highest leaderboard results. results can be found in table @xcite . the bert developers in their paper reported an improved score of 96.1% compared to the baseline of 94.9% to the conll-2003 named entity recognition results. since the underlying transformer concept is common to both bert and roberta, we decided to test if this variation resulted in an improved score. the dataset is observed to be imbalanced and for getting deeper representations of the minority classes, we investigate the impact of the last few layers of the transformer model. for handling this, we come across the different layers of the model capturing different levels of representations. the layers learn a rich hierarchy of linguistic information i.e. surfacelevel features in the lower layers, syntactic features in the middle layers, and semantic features in the higher layers. the authors, however, also point out figure 2: dataset example ## conclusion we have presented a novel fine-tuned roberta implementation for the hope speech for equality, diversity and inclusion shared task acl 2022. our best performing model utilises the last four transformer decoder layers of the fine-tuned robertalarge model to give a weighted and macro f1 score of 0.86 and 0.51, respectively. we rank fourth in the leaderboard among all the participants and have released the open-source code for easy and reproducible results.