Simon @ LT - EDI - EACL 2021: Detecting Hope Speech with BERT (2021)

Abstract: In today’s society, the rapid development of communication technology allows us to communicate with people from different parts of the world. In the process of communication, each person treats others differently. Some people are used to using offensive and sarcastic language to express their views. These words cause pain to others and make people feel down. Some people are used to sharing happiness with others and encouraging others. Such people bring joy and hope to others through their words. On social media platforms, these two kinds of language are all over the place. If people want to make the online world a better place, they will have to deal with both. So identifying offensive language and hope language is an essential task. There have been many assignments about offensive language. Shared Task on Hope Speech Detection for Equality, Diversity, and Inclusion at LT-EDI 2021-EACL 2021 uses another unique perspective – to identify the language of Hope to make contributions to society. The XLM-Roberta model is an excellent multilingual model. Our team used a fine-tuned XLM-Roberta model to accomplish this task.

## introduction social media is a way to bring people from different regions closer together. different people have different cultural backgrounds and different world views. this allows people to have a fierce clash of ideas on social platforms. in these interactions, the way people treat each other is very different. hate speech offensive speech and offensive speech are not recommended @xcite @xcite . so we also do a lot of work to avoid this, for example, we have a lot of competitions to identify of-fensive content(germeval foot_0 , iberlef foot_1 ). and we welcome those who use the language of hope to encourage people to get back on their feet and renew their strength in the face of hardship. to this end, shared task on hope speech detection for equality, diversity, and inclusion at lt-edi 2021-eacl 2021 was organized by the event organizers. hope speech can help people find ways to realize their dreams @xcite @xcite @xcite . people can learn and master these techniques to make their lives better. hope speech is seen by medical personnel as vital to people's recovery. hope speech makes the whole social environment more optimistic. therefore, it is very necessary to detect hope speech. the second paragraph of the paper introduces the work done by scholars in relevant fields, the third paragraph introduces the detailed content of tasks, the fourth paragraph introduces the details of data and the methods of processing data, the fifth paragraph introduces the model used by our team, the sixth paragraph introduces the details and results of experiments, and finally the conclusion we have reached. ## related work some works have been done on the classification of hope speech. @xcite analyze the evolving international crisis via a substantial corpus constructed using comments on youtube videos. as well as detecting offensive content, hope speech can be classified as a useful tool in its own field. mathur et al. @xcite used transfer learning to complete the task of classifying offensive content of tweets in hot data sets. their team used a convolutional neural network to pre-train english tweets and then re-train indo-english tweets. kamble and joshi @xcite used three classical deep learning models to complete the classification of english-hindi tweets. by comparing these three classical deep learning models, they found that the use of specific embedding can improve the representation of the target group, and thus improve the score of f1. santosh and aravind @xcite used different lstm @xcite models to classify hate speech, one is the sub-word level lstm model and the other is the hierarchical lstm model based on phonetic location words. waseem et al. @xcite found the influence of different extra language features combined with n-gram characters on classification tasks. ## model description in a recent categorization task competition, many teams using the fine-tuned bert @xcite model did well. our team used xlm-roberta @xcite ) as a preprocessing model in the experiment. the xlm-roberta model is a multilingual processing model trained on processing text in 100 different languages. this fits nicely with the shared task of handling multilingual text. compared with the bert model, the xlm-roberta model has improved various performances. this is due to the increased training volume of the xlm-roberta model, whose training datasets are several orders of magnitude larger than the wiki-100 corpus used to train its predecessor. based on the xlm-roberta model, we used two fine-tuning methods to accomplish this task. the two approaches are shown in figures 1 and 2 . ## experiment and results in the experimental process, the same hyperparameter is used in the two methods. ## conclusion this article describes the models and results used by the simon team in the hope speech classification task. our results are not good in the leaderboard, which may have something to do with the imbalance of the data set. in the future, we will continue to improve our model to get better results.