Will_go at S em E val-2020 Task 9: An Accurate Approach for Sentiment Analysis on H indi- E nglish Tweets Based on Bert and Pesudo Label Strategy (2020)

Abstract: Mixing languages are widely used in social media, especially in multilingual societies like India. Detecting the emotions contained in these languages, which is of great significance to the development of society and political trends. In this paper, we propose an ensemble of pesudo-label based Bert model and TFIDF based SGDClassifier model to identify the sentiments of Hindi-English (Hi-En) code-mixed data. The ensemble model combines the strengths of rich semantic information from the Bert model and word frequency information from the probabilistic ngram model to predict the sentiment of a given code-mixed tweet. Finally our team got an average F1 score of 0.731 on the final leaderboard,and our codalab username is will_go.

## introduction the rapid development of modern social media makes it possible for people to express their opinions almost at any time. detecting the sentiments of these views can roughly analyze the social and economic development of the user's current region, the psychological characteristics, interests and hobbies of people of different ages, so as to facilitate the company to better deliver information. it is also possible to judge the emotion and psychological characteristics of a certain type of users according to their statements, so as to eliminate some adverse factors to the society. many companies put efforts to concern customer feedback to achieve better product optimization, and then win the trust of consumers. however, in social media, speech is not required to conform to certain norms, so there are differences in language and format. in some multilingual societies, when different languages are mixed, it is even more difficult to detect the sentiments. in india and spain, a number of bi-lingual hybrids produced many texts. the process of switching text between two or more languages is called code-mixing and a significant part is the mixing of the native language and english. the difference of language habits makes the problem even harder. a quite number of people prefer to use nonstandard words like coooool rather than cool, thx instead of thanks, which causes obstacles. code mixing has always been one of the most important directions of natural language processing, and there is a lot of valuable research and a lot of good results in language identifying, pos tagging and named entity recognition of code-mixed from a lot of researchers @xcite . lstm model shows great results in the sentiment analysis of code-mixing @xcite . however, the noise of the code mixed data exists. the model we propose in this paper combines multi-sample-dropout and pseudo label based on the bert. besides, tdidf is also adopted to get better performance. we compare different models to get a higher f1 and the results show that the model bert with pseudo label performs well when the size of batch is 16. when the n-gram ranges from 1 to 3 helps the model more accurate. overall solutions can be seen in @xcite the rest of the paper is organized as follows. the overview of sentiment analysis in code-mixed data is shown in section 2. and in section 3, we explain the details of our model and the corresponding frames. section 4 shows the results of our models. ## related works the phenomenon of mixed-use of mixing languages in the world is becoming more and more frequent. although most people use english on social media, many of them are mixed with languages other than english. this poses a great challenge to traditional natural language processing. the combination of mixing languages and codes brings some invisible difficulties to natural language processing(nlp) tasks, such as word-level language identification, part-of-speech tagging, dependency parsing, machine translation and semantic processing. in recent years, there have been many studies aimed at mixing language processing, and they have made significant contributions to mixing languages or mixing code processing. in terms of language recognition, banerjee et al. @xcite , mandal et al. @xcite , @xcite , das and gamback @xcite have done researches on mixing codes processing. and vyas et al. @xcite , jamatia et al. @xcite , gupta et al. @xcite took a big step in the part-of-speech tagging. wang et al. @xcite proposed three strategies to improve the multilingual translation framework quality of multilingual neural translation models. however, none of these works can effectively solve the problem of sentiment analysis in code mixed data. with the rapid development of the internet and the emergence of various social media, sentiment analysis has become increasingly important. the importance of sentiment analysis or opinion mining for the entire business and society has expanded from computer science to management science and social science, such as marketing, finance, political science, communication, health, and even history. since the beginning of 2000, sentiment analysis has become one of the most active research areas in the field of nlp. in recent years, there has been a breakthrough in deep learning. the sentiment analysis task in the nlp field has gradually introduced this method and has formed many best results in the industry. zheng and xia @xcite adopted the lstm model based on context2target attention, and conducted targeted sentiment analysis by capturing the most important words in the left and right context. joshi et al. @xcite used sub-word-level representation in the lstm architecture, which produced the most advanced results compared to other traditional machine learning models and models based on word polarity. traditional deep learning models often have certain limitations in processing speed and accuracy. so we came up with a model that combines multi-sample dropout and bert-based tfidf, and the model performs very well. ## methodology our model is based on the bidirectional encoder representation from transformer (bert) and frequency-inverse document frequency (tf-idf) for detecting sentiments. bert can learn the semantic relationship in the text, and tfidf technology can better learn the relationship between word frequency and label. the difference between the two provides theoretical support for the model integration we use later. in addition, in the bert model, we adopted pseudo label and multi-sample dropout, both of which make the model accuracy much improved. pseudo label was adopted during the training. we divide the training data into 5 folds and use the cross-validation for improving the training. a multi-sample dropout was adopted to avoid overfitting and accelerate training and improve generalization. our model was conducted with the following details. ## experiments and results the final model results are shown in ## conclusion sentiment analysis is one of the important means for us to understand society, especially the processing of multiple languages. in this paper, we used bert, multilingual bert, sgd classifier with tf-idf and ensemble method to analyze the sentiment of english documents mixed with india. the bert model with pseudo label strategy performed best,and the ensemble model reached 0.686 in the leaderboard. the results of the competition showed that our model has a good effect on the sentiment analysis of mixing languages. however, there are still some deficiencies in model optimization and parameter setting. in the future, we hope to be able to process documents mixed in more languages, optimize and extend the method to more aspects to learn more rich features in various languages.