Task: Question Generation
Input (English): We use two basic features:

Parts of Speech (POS) tags: We use the POS tagger of NLTK to tag the tweet texts BIBREF0 . We use counts of noun, adjective, adverb, verb words in a tweet as POS features.

Prior polarity of the words: We use a polarity dictionary BIBREF3 to get the prior polarity of words. The dictionary contains positive, negative and neutral words along with their polarity strength (weak or strong). The polarity of a word is dependent on its POS tag. For example, the word `excuse' is negative when used as `noun' or `adjective', but it carries a positive sense when used as a `verb'. We use the tags produced by NLTK postagger while selecting the prior polarity of a word from the dictionary. We also employ stemming (Porter Stemmer implementation from NLTK) while performing the dictionary lookup to increase number of matches. We use the counts of weak positive words, weak negative words, strong positive words and strong negative words in a tweet as features.

We have also explored some advanced features that helps improve detecting sentiment of tweets.

Emoticons: We use the emoticon dictionary from BIBREF2 , and count the positive and negtive emocicons for each tweet.

The sentiment of url: Since almost all the articles are written in well-formatted english, we analyze the sentiment of the first paragraph of the article using Standford Sentiment Analysis tool BIBREF4 . It predicts sentiment for each sentence within the article. We calculate the fraction of sentences that are negative, positive, and neutral and use these three values as features.

Hashtag: We count the number of hashtags in each tweet.

Capitalization: We assume that capitalization in the tweets has some relationship with the degree of sentiment. We count the number of words with capitalization in the tweets.

Retweet: This is a boolean feature indicating whether the tweet is a retweet or not.

User Mention: A boolean feature indicating whether the tweet contains a user mention.

Negation: Words like `no', `not', `won't' are called negation words since they negate the meaning of the word that is following it. As for example `good' becomes `not good'. We detect all the negation words in the tweets. If a negation word is followed by a polarity word, then we negate the polarity of that word. For example, if `good' is preceeded by a `not', we change the polarity from `weak positive' to `weak negative'.

Text Feature: We use tf-idf based text features to predict the sentiment of a tweet. We perform tf-idf based scoring of words in a tweet and the hashtags present in the tweets. We use the tf-idf vectors to train a classifier and predict the sentiment. This is then used as a stacked prediction feature in the final classifier.
Output (English): What linguistic features are used?