Question: What are the three categories of feature extractors?
Evidence: Many tasks related to sentiment or emotion analysis depend upon affect, opinion, sentiment, sense and emotion lexicons. These lexicons associate words to corresponding sentiment or emotion metrics. On the other hand, the semantic meaning of words, sentences, and documents are preserved and compactly represented using low dimensional vectors BIBREF1 instead of one hot encoding vectors which are sparse and high dimensional. Finally, there are traditional NLP features like word N-grams, character N-grams, Part-Of-Speech N-grams and word clusters which are known to perform well on various tasks.
Based on these observations, the feature extraction step is implemented as a union of different independent feature extractors (featurizers) in a light-weight and easy to use Python program EmoInt . It comprises of all features available in the baseline model BIBREF2 along with additional feature extractors and bi-gram support. Fourteen such feature extractors have been implemented which can be clubbed into 3 major categories:
[noitemsep]
Lexicon Features
Word Vectors
Syntax Features
Lexicon Features: AFINN BIBREF3 word list are manually rated for valence with an integer between -5 (Negative Sentiment) and +5 (Positive Sentiment). Bing Liu BIBREF4 opinion lexicon extract opinion on customer reviews. +/-EffectWordNet BIBREF5 by MPQA group are sense level lexicons. The NRC Affect Intensity BIBREF6 lexicons provide real valued affect intensity. NRC Word-Emotion Association Lexicon BIBREF7 contains 8 sense level associations (anger, fear, anticipation, trust, surprise, sadness, joy, and disgust) and 2 sentiment level associations (negative and positive). Expanded NRC Word-Emotion Association Lexicon BIBREF8 expands the NRC word-emotion association lexicon for twitter specific language. NRC Hashtag Emotion Lexicon BIBREF9 contains emotion word associations computed on emotion labeled twitter corpus via Hashtags. NRC Hashtag Sentiment Lexicon and Sentiment140 Lexicon BIBREF10 contains sentiment word associations computed on twitter corpus via Hashtags and Emoticons. SentiWordNet BIBREF11 assigns to each synset of WordNet three sentiment scores: positivity, negativity, objectivity. Negation lexicons collections are used to count the total occurrence of negative words. In addition to these, SentiStrength BIBREF12 application which estimates the strength of positive and negative sentiment from tweets is also added.
Word Vectors: We focus primarily on the word vector representations (word embeddings) created specifically using the twitter dataset. GloVe BIBREF13 is an unsupervised learning algorithm for obtaining vector representations for words. 200-dimensional GloVe embeddings trained on 2 Billion tweets are integrated. Edinburgh embeddings BIBREF14 are obtained by training skip-gram model on Edinburgh corpus BIBREF15 . Since tweets are abundant with emojis, Emoji embeddings BIBREF16 which are learned from the emoji descriptions have been used. Embeddings for each tweet are obtained by summing up individual word vectors and then dividing by the number of tokens in the tweet.
Syntactic Features: Syntax specific features such as Word N-grams, Part-Of-Speech N-grams BIBREF17 , Brown Cluster N-grams BIBREF18 obtained using TweetNLP project have been integrated into the system.
The final feature vector is the concatenation of all the individual features. For example, we concatenate average word vectors, sum of NRC Affect Intensities, number of positive and negative Bing Liu lexicons, number of negation words and so on to get final feature vector. The scaling of final features is not required when used with gradient boosted trees. However, scaling steps like standard scaling (zero mean and unit normal) may be beneficial for neural networks as the optimizers work well when the data is centered around origin.
A total of fourteen different feature extractors have been implemented, all of which can be enabled or disabled individually to extract features from a given tweet.
Fourteen such feature extractors have been implemented which can be clubbed into 3 major categories: • Lexicon Features • Word Vectors • Syntax Features
Answer: • Lexicon Features • Word Vectors • Syntax Features