Maximum Entropy Model based on Feature Extraction for Sentiment Detection of Text

The rapid development of social media services has facilitated the communication of opinions through online news, blogs, post bar, microblogs/tweets, and so forth. This article concentrates on the mining of emotions evoked by newmaterials. Compared to the classical sentiment analysis by using the word-emotion lexicon in the text, we combine the word with emotion via the intensive feature functions. We propose a maximum entropy model based on the feature extraction for sentiment classification, which generates the probability of sentiments conditioned to news text. In addition, one effective feature extraction strategies are proposed to refine the original miscellaneous news text. Experimental evaluations using real-world data validate the effectiveness of the proposed model on sentiment classification of news text. Introduction Since the world go into the internet century, thousands of billions of texts are published on the internet every day. The texts contains lots of valuable information which can be used in different ways. However, it is not realistic to deal with such huge amount of texts with manpower. Therefore, applying machine learning method to this work can give great help. Among the information from the texts on the internet, sentiment of the text is the great valuable one. News on the internet are published in time in contrast to other media, and it can show what happen in the world right now. What is more, sentiment in the news can be used to show the public's attitude and predict the trend. In this paper, we focus on the sentiment classification on the news texts published on the internet. Therefore, this work can help to keep in touch with public's sentiment. We propose a framework to deal with the text to get the sentiment information. In our work, we apply singular value decomposition to the data sets to reduce dimension and restrain noise, and besides, we propose to model sentiments and words using intensive feature functions. Thus, sentiments of unlabeled news text is classified according to the principle of maximum entropy [1]. The result of the experiments on the real data sets shows that the framework we propose can gain a good result on the prediction of sentiment from text. The remainder of this paper is organized as follows. In Section 2, we summarize the related work in sentiment classification on the text. Then, we propose our model and conduct experimental analysis in Section 3 and Section 4, respectively. Finally, we present conclusions in Section 5. Related work In this section, previous works on sentiment classification, news text analysis and our work will be shown. Sentiment classification mainly concentrate on extracting emotions from reviews, messages and news text, which convey the opinion of writers or readers. The existing methods of sentiment classification can be divided into three categories primarily: lexicon-based, supervised and unsupervised learning strategies. The lexicon-based method [2-4] classified sentiments by 2nd Workshop on Advanced Research and Technology in Industry Applications (WARTIA 2016) © 2016. The authors Published by Atlantis Press 1300 constructing wordor topic-level emotional dictionaries. The supervised learning strategy used existing classification algorithms to split the emotional orientation of words or phrases into positive and negative, which included naive Bayes, maximum entropy and support vector machines [5]. An unsupervised learning technique was also utilized to classify the emotional orientation of users’ reviews (e.g., reviews of movies, travel destinations, automobiles and banks), which computed the overall polarity of the review by counting the occurrence of positive and negative terms [6]. Recently, the Emotion-Term(ET) algorithm and the Emotion-Topic Model(ETM) [7] were proposed to improve the performance of existing systems. ET is a variant of the naive Bayes classifier and ETM is model associating emotions with topics jointly. We analysed the features of text which has many redundant information and noise. So we reduced the dimension and remove some noise data of the original data. And we extracted the main features to represent one document, then we develop an intensive maximum entropy model to classify sentiments of these features, which has a concentrated representation for modeling emotions. Maximum Entropy Model via Intensive Feature Functions In this section, we propose Singular Value Decomposition(SVD) and the Maximum Entropy Model based on Feature Extraction(MEFE), a maximum entropy model via intensive feature functions for text sentiment classification. The problem is first defined, including the relevant terms and notations, and then the MEFE is presented in detail. Finally, we describe the estimation of parameters. Problem Definition For convenience of defining the issue of sentiment classification in news, we here defined the following terms and notations: A text collection consists of D documents 1 2 { , ,. , } D t t t with word tokens and multiple emotion labels. We represent the set of all word tokens by 1 2 { , ,. , } V W w w w  and emotion labels by 1 2 3 { , , ,. , } M E e e e e  where V is the number of unique word tokens, and M is the amount of pre-defined emotion labels such as "joy", "anger", "fear", "surprise", "touching", "empathy", "sadness", "boredom" and "warmness". The co-occurrence of word token w and emotion label e in the training set is denoted by ( , ) w e , and we denote the set of all word-emotion pairs by  . For the multiple emotion labels and reader ratings, a matrix  is used to represent the reader votes over each emotion label for all documents in the training set. In the D M  matrix  , the element nk  denotes the ratings over the k th  emotion among all users who have read the n th  text. The reader ratings were normalized and summed to one for each document. Table 1 summarizes the notations of frequently-used variables. Table 1: Notations of frequently-used variables Symbol Description D Number of documents V Number of unique word tokens M Number of pre-defined emotion labels F Number of unique features W Set of word tokens E Set of social emotions  Set of all word-emotion pairs    nk  : D M  matrix of normalized ratings for each emotion Singular Value Decomposition Singular Value Decomposition (SVD) is an important method to reducing dimension and extract the main features of high dimension data. Then we will introduce the SVD method for our Sina news text data.