Abstract: We present Tweet2Vec, a novel method for generating general-purpose vector representation of tweets. The model learns tweet embeddings using character-level CNN-LSTM encoder-decoder. We trained our model on 3 million, randomly selected English-language tweets. The model was evaluated using two methods: tweet semantic similarity and tweet sentiment categorization, outperforming the previous state-of-the-art in both tasks. The evaluations demonstrate the power of the tweet embeddings generated by our model for various tweet categorization tasks. The vector representations generated by our model are generic, and hence can be applied to a variety of tasks. Though the model presented in this paper is trained on English-language tweets, the method presented can be used to learn tweet embeddings for different languages.

Introduction: In recent years, the micro-blogging site Twitter has become a major social media platform with hundreds of millions of users. The short (140 character limit), noisy and idiosyncratic nature of tweets make standard information retrieval and data mining methods ill-suited to Twitter. Consequently, there has been an ever growing body of IR and data mining literature focusing on Twitter. However, most of these works employ extensive feature engineering to create task-specific, hand-crafted features. This is time consuming and inefficient as new features need to be engineered for every task. In this paper, we present Tweet2Vec, a method for generating general-purpose vector representation of tweets that can be used for any classification task. Tweet2Vec removes the need for expansive feature engineering and can be used to train any standard off-the-shelf classifier (e.g., logistic regression, svm, etc). Tweet2Vec uses a CNN-LSTM encoder-decoder model that operates at the character level to learn and generate vector representation of tweets. Our method is especially useful for natural language processing tasks on Twitter where it is particularly difficult to engineer features, such as speech-act classification and stance detection (as shown in our previous works on these topics BIBREF0 , BIBREF1 ). There has been several works on generating embeddings for words, most famously Word2Vec by Mikolov et al. BIBREF2 ). There has also been a number of different works that use encoder-decoder models based on long short-term memory (LSTM) BIBREF3 , and gated recurrent neural networks (GRU) BIBREF4 . These methods have been used mostly in the context of machine translation. The encoder maps the sentence from the source language to a vector representation, while the decoder conditions on this encoded vector for translating it to the target language. Perhaps the work most related to ours is the work of Le and Mikolov le2014distributed, where they extended the Word2Vec model to generate representations for sentences (called ParagraphVec). However, these models all function at the word level, making them ill-suited to the extremely noisy and idiosyncratic nature of tweets. Our character-level model, on the other hand, can better deal with the noise and idiosyncrasies in tweets. We plan to make our model and the data used to train it publicly available to be used by other researchers that work with tweets.

Question: what are the previous state of the art for tweet semantic similarity?
Answer: nnfeats, ikr, linearsvm and svckernel.