This paper describes a simple and principled approach to automatically construct sentiment lexicons using distant supervision . We induce the sentiment association scores for the lexicon items from a model trained on a weakly supervised corpora . Our empirical findings show that features extracted from such a machine-learned lexicon outperform models using manual or other automatically constructed sentiment lexicons . Finally , our system achieves the state-of-the-art in Twitter Sentiment Analysis tasks from Semeval-2013 and ranks 2nd best in Semeval-2014 according to the average rank . One of the early and rather successful models for sentiment analysis ( Pang and Lee , 2004 ; Pang and Lee , 2008 ) relied on manually constructed lexicons that map words to their sentiment , e.g. , positive , negative or neutral . The document-level polarity is then assigned by performing some form of averaging , e.g. , majority voting , of individual word polarities found in the document . These systems show an acceptable level of accuracy , they are easy to build and are highly computationally efficient as the only operation required to assign a polarity label are the word lookups and averaging . However , the information about word polarities in a document are best exploited when using machine learning models to train a sentiment classifier . In fact , most successful sentiment classification systems rely on supervised learning . Interestingly , a simple bag of words model using just unigrams and bigrams with an SVM has shown excellent results ( Wang and Manning , 2012 ) performing on par or beating more complicated models , e.g. , using neural networks ( Socher et al . , 2011 ) . Regarding Twitter sentiment analysis , the top performing system ( Mohammad et al . , 2013 ) from Semeval-2013 Twittter Sentiment Analysis task ( Nakov et al . , 2013 ) follows this recipe by training an SVM on various surface form , sentiment and semantic features . Perhaps , the most valuable finding is that sentiment lexicons appear to be the most useful source of features accounting for over 8 point gains in the F-measure on top of the standard feature sets . Sentiment lexicons are mappings from words to scores capturing the degree of the sentiment expressed by a given word . While several manually constructed lexicons are made available , e.g. , the MPQA ( Wilson et al . , 2005 ) , the Bing and Liu ( Hu and Liu , 2004 ) and NRC Emoticon ( Mohammad and Turney , 2013 ) lexicons , providing high quality word-sentiment associations compiled by humans , still their main drawback is low recall . For example , the largest NRC Emoticon lexicon contains only 14k items , whereas tweets with extremely sparse surface forms are known to form very large vocabularies . Hence , using larger lexicons with better recall has the potential of learning more accurate models . Extracting such lexicons automatically is a challenging and interesting problem ( Lau et al . , 2011 ; Bro and Ehrig , 2013 ; Liu et al . , 2013 ; Tai and Kao , 2013 ; Yang et al . , 2014 ; Huang et al . , 2014 ) . However , different from previous work our goal is not to extract human-interpretable lexicons but to use them as a source of features to improve the classifier accuracy . Following this idea , the authors in ( Mohammad et al . , 2013 ) use features derived from the lexicons to build a state-of-the-art sentiment classifier for Twitter . They construct automatic lexicons using noisy labels automatically inferred from emoticons and hashtags present in the tweets . The wordsentiment association scores are estimated using pointwise mutual information ( PMI ) computed between a word and a tweet label . While the idea to model statistical correlations between the words and tweet labels using PMI or any other metric is rather intuitive , we believe there is a more effective way to exploit noisy labels for estimating the word-sentiment association scores . Our method relies on the idea of distant supervision ( Marchetti-Bowick and Chambers , 2012 ) . We use a large distantly supervised Twitter corpus , which contains noisy opinion labels ( positive or negative ) to learn a supervised polarity classifier . We encode tweets using words and multi-word expressions as features ( which are also entries in our lexicon ) . The weights from the learned model are then used to define which lexicon items to keep , i.e. , items that constitute a good sentiment lexicon . The scores for the lexicon items can be then directly used to encode new tweets or used to derive more advanced features . Using machine learning to induce the scores for the lexicon items has an advantage of learning the scores that are directly optimized for the classification task , where lexicon items with higher discriminative power tend to receive higher weights . To assess the effectiveness of our approach , we reimplemented the state-of-the-art system ranking 1st in Semeval-2013 Twitter Sentiment Analysis challenge and used it as our baseline . We show that adding features from our machine-learned sentiment lexicon yields better results than any of the automatic PMI lexicons used in the baseline and all of them combined together . Our system obtains new state-of-the-art results on the SemEval-2013 message level task with an F-score of 71.32 -a 2 % of absolute improvement over the previous best system in SemEval-2013 . We also evaluate the utility of the ML lexicon on the five test sets from a recent Semeval-2014 task showing significant improvement over a strong baseline . Finally , our system shows high accuracy among the 42 systems participating in the Semeval-2014 challenge ranking 2nd best according to the average rank across all test sets . We demonstrated a simple and principled approach grounded in machine learning to construct sentiment lexicons . We show that using off-the-shelf machine learning tools to automatically extract lexicons greatly outperforms other automatically constructed lexicons that use pointwise mutual information to estimate sentiment scores for the lexicon items . We have shown that combining our machinelearned lexicon with the previous best system yields state-of-the-art results in Semeval-2013 gaining over 2 points in F-score and ranking our system 2nd according to the average rank over the five test sets of Semeval-2014 . Finally , our ML-based lexicon shows excellent results when added on top of the current state-of-the-art NRC system . While our experimental study is focused on Twitter , our method is general enough to be applied to sentiment classification tasks on other domains . In the future , we plan to experiment with constructing ML lexicons from larger Twitter corpora also using hashtags . Recently , deep convolutional neural networks for sentence modelling ( Kalchbrenner et al . , 2014 ; Kim , 2014 ) have shown promising results on several NLP tasks . In particular , ( Tang et al . , 2014 ) showed that learning sentiment-specific word embeddings and using them as features can boost the accuracy of existing sentiment classifiers . In the future work we plan to explore such approaches .

Challenge: While sentiment lexicons are useful for building accurate sentiment classification systems, existing methods suffer from low recall or interpretability.
Approach: They propose to use Twitter's noisy opinion labels as distant supervision to learn a supervised polarity classifier and use it to obtain sentiment lexicons.
Outcome: Using the obtained lexicon with an existing model achieves the state-of-the-art on the SemEval-13 message level task and outperforms baseline models in several other datasets.