T weet F in S ent: A Dataset of Stock Sentiments on T witter (2022)

Abstract: Stock sentiment has strong correlations with the stock market but traditional sentiment analysis task classifies sentiment according to having feelings and emotions of good or bad. This definition of sentiment is not an accurate indicator of public opinion about specific stocks. To bridge this gap, we introduce a new task of stock sentiment analysis and present a new dataset for this task named TweetFinSent. In TweetFinSent, tweets are annotated based on if one gained or expected to gain positive or negative return from a stock. Experiments on TweetFinSent with several sentiment analysis models from lexicon-based to transformer-based have been conducted. Experimental results show that TweetFinSent dataset constitutes a challenging problem and there is ample room for improvement on the stock sentiment analysis task. TweetFinSent is available at https://github.com/jpmcair/tweetfinsent .

## introduction sentiment analysis, as a classical research problem in machine learning and natural language processing, aims to analyze peoples opinions, sentiments, and emotions towards entities such as products, services, organizations, individuals, and their attributes @xcite . a large amount of attention in industry and research community has been given to analysing sentiment of twitter feeds. this has been done to analyse the effectiveness and predicting the result of election campaigns @xcite , analyse twitter mood during the covid-19 outbreak @xcite and to analyse and predict the stock market. it has been repeatedly shown in literature that the twitter sentiment has strong correlations with the stock market, with several works on predicting the stock market movement based on twitter sentiment @xcite @xcite . for instance, recent discussions of meme stocks on social media such as twitter and reddit have attracted significant attention and influenced the sentiment of investors especially young and inexperienced investors foot_0 . therefore, it is of great value to analyse stock sentiment in both practice and research. despite the wide interest and importance, most existing research on sentiment analysis focused on distinguishing if the text contains or a user has feelings or emotions of good or bad. however, in the financial domain, we would like to analyse more specific and concrete sentiment, i.e., we aim to re-calibrate the definition of sentiment to include this desired property such as gaining or expecting to gain positive or negative return from a stock. although traditional sentiment analysis of twitter feeds correlates with the stock market dynamics to some extent, it is not an accurate indicator of public opinion about financial returns of specific stocks. in worst case, traditional sentiment analysis methods may classify tweets into controversy sentiment due to various factors such as financespecific terms. some representative examples are shown in tweet target ticker traditional sentiment stock sentiment bubbles burst an any given moment. maybe @xmath0tsla neutral negative @xmath1bidu lol. @xmath2sofi not touching it. i love the company though. we all know the rules, and know what happens during the lockup expiry @xmath3amc @xmath4nok $amc negative positive we then construct an expert-annotated dataset for stock sentiment analysis called tweetfin-sent which will be made publicly available to the research community. we benchmark this dataset with various state-of-the-art baselines. experimental results show that tweetfinsent dataset constitutes a challenging problem and there is ample room for improvement on the stock sentiment analysis task. in summary, our main contributions are three-fold: • we construct and release tweetfinsent, a new twitter stock sentiment dataset. to the best of our knowledge, this is the first resource for stock sentiment analysis. • we demonstrate the utility of the tweet-finsent dataset by evaluating different types of state-of-the-art sentiment analysis models on our dataset. • we investigate the performance of different baselines and outline the challenge of the stock sentiment analysis task and future directions. ## related work the tremendous growth of unstructured text data has spurred research in nlp, especially in the area of sentiment analysis, which involves classifying and analyzing of people's opinions, emotions, and sentiments from textual data @xcite . in nlp, sentiment analysis plays a significant role in analyzing the emotions or feelings behind written texts which serve different purposes depending on the domain of its applications. since sentiment analysis is an increasingly valuable tool for many organisations to enhances their decision-making, it has been extended to variety of use cases. however, we'd like to argue the use case of this study is unique in the sense that stock sentiment on twitter is considerably different from traditional sentiment analysis. in the following, we review most relevant prior work and then highlight the value of our study and dataset. twitter sentiment analysis: twitter sentiment analysis is an important area and has attracted much attention. it is considered a more challenging problem than general sentiment analysis on conventional texts because of the frequent use of slang, irregular words, informal words, and a vast number of tweets on various topics. twitter sentiment analysis has applications in business management, public actions understanding, political analysis, and other domains. previous works in twitter sentiment analysis include sentiment analysis to assist stock prediction @xcite , discovering brand perception @xcite , and analyzing and predicting election results @xcite . researchers proposed different methods to solve this problem including lexiconbased @xcite , machine learning @xcite , and hybrid methods @xcite . recent works @xcite have applied transformers for sentiment analysis tasks. stock sentiment analysis: stock sentiment analysis significantly differs from general sentiment. it differs in terms of domain and purpose. the purpose behind stock sentiment analysis is usually to predict the stock markets reaction to the sentiments hidden in the text. previous works have attempted to forecast stock prices using price history. recent works have begun using textual data for predicting the stock markets reaction. for example, stock market values were predicted using news articles @xcite , news headlines @xcite , and sentiments on social media @xcite . apple inc. companys news data were collected by @xcite and performed sentiment analysis using supervised machine learning to understand the relationship between news and stock trend. sentiment analysis of economic news headlines was used by @xcite to predict the stock value changes for giant tech companies. @xcite investigated the error patterns of some widely acknowledged sentiment analysis methods in the finance domain. there have been several sources of data for stock sentiment analysis. popular sources of data include financial phrasebank @xcite , yahoo finance @xcite , finviz @xcite , stocktwits data (araci, 2019), and semeval @xcite . ## conclusions we presented tweetfinsent, a new dataset for stock sentiment analysis and it contains 2,113 expert-annotated tweets covering different stocks. different from existing sentiment analysis dataset, tweetfinsent defines sentiment based on whether a user gained or expected to gain positive or negative return from a stock rather than having feelings and emotions of good or bad. our experiments with several sentiment analysis models indicated that there is a huge gap between machine learning models and human annotations. thus, the tweetfinsent dataset constitutes a challenging problem and there is ample room for improvement on the stock sentiment analysis task.