Named-Entity Based Sentiment Analysis of N epali News Media Texts (2020)

Abstract: Due to the general availability, relative abundance and wide diversity of opinions, news Media texts are very good sources for sentiment analysis. However, the major challenge with such texts is the difficulty in aligning the expressed opinions to the concerned political leaders as this entails a non-trivial task of named-entity recognition and anaphora resolution. In this work, our primary focus is on developing a Natural Language Processing (NLP) pipeline involving a robust Named-Entity Recognition followed by Anaphora Resolution and then after alignment of the recognized and resolved named-entities, in this case, political leaders to the correct class of opinions as expressed in the texts. We visualize the popularity of the politicians via the time series graph of positive and negative sentiments as an outcome of the pipeline. We have achieved the performance metrics of the individual components of the pipeline as follows: Part of speech tagging – 93.06% (F1-score), Named-Entity Recognition – 86% (F1-score), Anaphora Resolution – 87.45% (Accuracy), Sentiment Analysis – 80.2% (F1-score).

## introduction in recent times, the way the general public acquires news has drastically changed. traditional news sources such as television, radio and printed newspapers are in a steady decline in terms of use and consumption. nowadays, most of the people, especially those from the younger generations read the news on the web either from social media or online news portals. as the internet has become more accessible and available to the general public, we have seen a rapid growth in the number of online news portals and blog sites. almost all of the established media houses have gone online thereby maintaining online news portals besides the hard copy versions. this makes news media texts a very good resource for sentiment analysis in the socio-political domain. in most countries the popularity of the politicians (especially, the head of state) is tracked by the media houses, independent organizations as well as the party the politicians are affiliated to. the approval ratings show how popular or unpopular a politician is in the view of the general public. politicians make change to their policies as well as their public persona so that their approval ratings can improve. positive approval rating can even suggest the likelihood of a politician wining an election. approval ratings are generally calculated by conducting opinion polls in a particular sample population. another way of calculating the approval rating can be by finding out what kind of views are being expressed about a politician in printed news media articles. in the context of our country, such approval ratings are not available. this work presents a way of calculating popularity ratings and thus helping to determine how popular or unpopular a nepali politician is by analyzing the news media texts. the task of analyzing sentiments in the news media texts has its own set of challenges. a news article may contain sentiments or opinions expressed over more than one politician. worse, even a sentence might refer to sentiments expressed over multiple politicians. from this perspective, the task demands that the namedentity or the political leader being referred to is accurately identified and resolved in terms of the pronominal references used in the text before moving to the task of aligning the corresponding sentiment to the named-entity. once the two phase task of named-entity resolution and sentiment alignment is complete, we present the results in the form of a time-series popularity or trending graph. such a representation would be of interest to a wide range of target audience -the political leader himself/herself, his/her political affiliation, the general public and media houses. automating the task is not simple or trivial as the whole process is quite technically involved and requires a series of nlp sub-tasks to be accomplished before we finally reach the end of the pipeline. we describe the pipeline in section 3 of this paper. in terms of accomplishing the work, our major contributions can be listed as follows: a part-of-speech (pos) tagger for nepali b) developed a named-entity recognition (ner) classifier for nepali c) developed a rule-based anaphora resolution module for nepali d) manually labelled a sentence level sentiment corpus of 3490 sentences from nepali news media texts e) developed a machine learning based sentiment classifier based on the sentiment corpus ## related work there have been a few research works on sentiment analysis in nepali texts. in one of the first works on sentiment analysis for the nepali language, gupta and bal developed a lexical resource namely the bhavanakos @xcite . @xcite used a machine learning approach to determine if movie reviews were positive or negative. their architecture consisted of 3 major components, viz., pre-processing, feature extraction and classification. in the pre-processing phase, they performed the steps such as whitespace and special character removal, abbreviations expansion, stemming, stop word removal, negation handling, pos tagging, namedentity recognition etc. to train the model they extracted features such as tf-idf, positive word count, negative word count, presence of polar words etc. using these parameters they were able to train a naïve bayes classifier and reported a precision of 79.23%, a recall of 78.57% and fscore of 78.90%. their data set consisted of 500 samples: 250 positive and 250 negative @xcite . in an undertaking similar to sentiment analysis, @xcite used different text mining techniques to address the text classification problem for nepali news media text. the researchers compared the accuracy of three machine algorithms: naïve bayes, neural network and support vector machine to classify text according to their content. their architecture consisted of 3 major components: pre-processing, feature extraction and machine learning. in the pre-processing phase they performed the steps such as tokenization, special symbol and number removal, stop word removal and word stemming. to extract the feature from the dataset they used tf-idf. the researchers have used two instances of the svm: svm with linear kernel and svm radial basis function kernels. svm is a binary classifier but the problem of text classification is a multi-class problem. in order to mitigate this issue the researchers adopted a one-vs.-rest approach. the neural network they used was a simple dense backpropagation multilayered perceptron with stochastic gradient descent optimization. the researchers used a five-fold validation method. their result showed that svm with rbf kernel outperformed the other three algorithms with an average accuracy of 74.65%. linear svm had an average accuracy of 74.62%, multilayer perceptron neural networks had 72.99% and lastly the naive bayes had an accuracy of 68.31% @xcite . researchers have used sentiment analysis techniques to extract opinion from news media texts as well. thapa & bal (2016) compared the accuracy of machine learning algorithms to classify sentiments expressed in nepali news media text. they tested three algorithms: support vector machine, multinomial naive bayes and logistic regression using a 5-fold cross validation method. their dataset consisted of 384 book and movie reviews. in the dataset, 179 were positive and 205 were negative sentences. to extract the feature from the dataset, they used four methods: bag-of-words, bag-of-words (with stopwords removed), tf-idf and tf-idf (with stopwords removed). the results obtained from their experiments showed that the f1-score of multinomial naive bayes algorithm was higher when taking tf-idf (with stopwords) @xcite . in addition to this, kafle (2019) implemented a sentiment based popularity tracker for nepali politicians. in his research, he used nepali news media text published in english as his data source. he tracked the popularity of nepali politicians based on two parameters: growing popularity, diminishing popularity. to track the popularity of a particular politician, he carried out a sentence level sentiment analysis and assigned sentiment scores to each article with respect to that politician. his architecture can be divided into three main phases. the first phase was named-entity extraction where all the named-entities in the articles were extracted. in the second phase, pronomial anaphora were resolved by replacing the pronouns with the named-entities they were referring to. and in the third and final phase sentiments were extracted from the articles tokenized into sentences. to extract the sentiments from the tokens they used a lexicon and rule-based sentiment analysis tool called valence aware dictionary and sentiment reasoner (vader). nepali language is a morphologically rich and complex language. in order to mine opinions from nepali texts, the text classifier being used should be able to incorporate specific language features before classifying the text @xcite . different techniques have been used in order to extract sentiment from nepali text. researchers have employed learning based as well as rule and lexicon based approach for sentiment classification. to extract features from nepali text, techniques like tf-idf, bags-of-words etc are used. but the problems with these techniques are that they do not consider the context of the words. it has also been observed that removing stop words has no significant effect on the evaluation metrics and the performance of the classifier @xcite . ## methodology we propose the following framework to address the given research problem, which consists of a pipeline of six components: ## results in order to test the performance of the sentiment classifiers, 10-fold cross validation method was used. from the experiments conducted, we found that support vector machine (svm) with word2vec embedding (skipgram) had the overall highest performance metrics with an accuracy of 80.15%, precision of 80.4%, recall of 80.2% and f1-score of 80.2%. all the averaged results obtained from the experiments are presented in table no 3. algorithm embedding f1-score svm word2vec 80.2 fasttext 78.7 random forrest word2vec 77.1 fasttext 76.6 decision tree word2vec 68.2 fasttext 67.9 for visualization, popularity trends of three most prominent nepali politicians from 2018/04 to 2018/09 were plotted. the popularity graphs are shown in figure no 2. ## conclusion we implemented a named-entity based sentiment analysis framework in this research work in order to distill the outlook expressed towards the politicians in the news media. initially, the names of the politicians in the news articles were identified and the pronominal expressions that were referring to the names were resolved. sentences with the name of the politicians were then extracted and classified according to the sentiment expressed towards them. support vector machine had the overall highest performance metrics for classifying the sentiments expressed in the sentences. we experimented with different embedding techniques and found that word2vec with skipgram was the optimal option for feature extraction. this combination of classifier and embedding technique was used to classify the sentences in the articles. finally, as an application of our research work, we presented the results in the form of a time-series popularity graph or a trending graph. for each component of the pipeline, we were able to achieve a relatively higher performance metric. nevertheless, there are some limitations to our work. the components of the proposed framework are based on probabilistic and rule based models, which underperform compared to the neural network models. we could not go for the latter because our dataset is not large enough. similarly, the anaphora resolution component only resolved pronouns. other expressions referring to an entity were ignored all together. furthermore, we have not dealt with the opinion holder or the target explicitly in terms of opinions in this work although named-entities can also be related to the target in many aspects. the performance of the overall framework can be further enhanced by using more advanced variants of neural networks, which are specialized for natural language processing tasks. different variants of rnn have been used