M eister M orxrc at S em E val-2020 Task 9: Fine-Tune Bert and Multitask Learning for Sentiment Analysis of Code-Mixed Tweets (2020)

Abstract: Natural language processing (NLP) has been applied to various fields including text classification and sentiment analysis. In the shared task of sentiment analysis of code-mixed tweets, which is a part of the SemEval-2020 competition, we preprocess datasets by replacing emoji and deleting uncommon characters and so on, and then fine-tune the Bidirectional Encoder Representation from Transformers(BERT) to perform the best. After exhausting top3 submissions, Our team MeisterMorxrc achieves an averaged F1 score of 0.730 in this task, and and our codalab username is MeisterMorxrc

## introduction language is an indispensable and important part of human daily life. natural language is everywhere as a most direct and simple tool of expression. natural language processing is to transform the language used for human communication into a machine language that can be understood by machines. it is a model and algorithm framework for studying language capabilities. in recent years, nlp research has increasingly used new deep learning methods. as an important branch of artificial intelligence, language models are models that can estimate the probability distribution of a group of language units (usually word sequences). these models can be built at a lower cost and have significantly improved several nlp tasks, such as machine translation, speech recognition and parsing. the processing flow of natural language can be roughly divided into five steps: obtaining anticipation, preprocessing the corpus, characterizing, model training, and evaluating the effect of modeling. with the rapid development of the internet, the frequency of online communication on social software such as weibo, twitter, and forums is getting higher and higher, and the internet itself has also changed from "reading internet" to "interactive internet". the internet has not only become an important source for people to obtain information, but also an important platform for people to express their opinions and share their own experiences and directly express their emotions. the achievements of nlp research laid a good foundation for text sentiment analysis. text sentiment analysis is an important research branch in the field of natural language understanding, involving theories and methods in the fields of linguistics, psychology, artificial intelligence, etc. it mainly includes the processing of text sources, the subjective and objective classification of network text, and the subjective text analysis and other steps. due to the huge inclusiveness and openness of the internet itself, it attracts users of different races, different languages, different cultural backgrounds and different religious beliefs to communicate with each other here. therefore, mixed language sentiment classification will be an important research for nlp direction. ## related work sentiment analysis is a research with a long history that helps us understand the connections and relationships between objects. in recent years, many scholars have made great progess on sentiment analysis. a basic task in sentiment analysis is classifying the polarity of a given text at the document, sentence, or feature/aspect level-whether the expressed opinion in a document, a sentence or an entity feature/aspect is positive, negative, or neutral. subsequently, the method described in a patent by @xcite , looked specifically at sentiment and identified individual words and phrases in text with respect to different emotional scales. many other subsequent efforts were less sophisticated, using a mere polar view of sentiment, from positive to negative, such as work by turney @xcite , and pang @xcite who applied different methods for detecting the polarity of product reviews and movie reviews respectively. one can also classify a document's polarity on a multi-way scale, which was attempted by pang @xcite and snyder @xcite . but according to our findings, this research becomes particularly difficult in multilingual societies, especially in many code-mixed texts. though some researchers have explored in the field, there is still a long way to go. sharma and srinivas explore various methods to normalize the text and judged the polarity of the statement as positive or negative using various sentiment resources @xcite . bhargava and sharma develop a flexible and robust system for mining sentiments from code mixed sentences for english with combination of four other indian languages (tamil, telugu, hindi and bengali) @xcite . ghosh and das extract sentiment (positive or negative) from facebook posts in the form of code-mixed social media data using a machine learning approach @xcite .