A Sentiment and Emotion Annotated Dataset for Bitcoin Price Forecasting Based on R eddit Posts (2022)

Abstract: Cryptocurrencies have gained enormous momentum in finance and are nowadays commonly adopted as a medium of exchange for online payments. After recent events during which GameStop’s stocks were believed to be influenced by WallStreetBets subReddit, Reddit has become a very hot topic on the cryptocurrency market. The influence of public opinions on cryptocurrency price trends has inspired researchers on exploring solutions that integrate such information in crypto price change forecasting. A popular integration technique regards representing social media opinions via sentiment features. However, this research direction is still in its infancy, where a limited number of publicly available datasets with sentiment annotations exists. We propose a novel Bitcoin Reddit Sentiment Dataset, a ready-to-use dataset annotated with state-of-the-art sentiment and emotion recognition. The dataset contains pre-processed Reddit posts and comments about Bitcoin from several domain-related subReddits along with Bitcoin’s financial data. We evaluate several widely adopted neural architectures for crypto price change forecasting. Our results show controversial benefits of sentiment and emotion features advocating for more sophisticated social media integration techniques. We make our dataset publicly available for research.

## introduction cryptocurrencies, often referred to as cryptocurrency or cryptos, are any type of currencies that exist virtually and are secured by encryption or cryptography to safeguard transactions. these currencies are decentralized and do not have any regulating or governing authorities to track them or to create new units. bitcoin (btc) is one of the most dominant cryptocurrencies that is driven by investor expectations, and its demand is becoming increasingly appealing @xcite , with investors adopting it to diversify their portfolios. due to the similarity of its features with speculative stocks and its decentralized nature, the perception and sentiments of investors are likely to drive the price of bitcoin @xcite . among the many challenges presented by economics and finance businesses, there is indeed the modeling of customers' sentiment, including their polarity and diversity, and the intentions that may be associated with them. for this purpose, social media constitute rich and useful sources of information that can be analyzed to obtain useful insights. additionally, it is worth noting that social media may contain misinformation and biases that can potentially exacerbate the information extraction process, eventually making it unreliable @xcite . reddit is a social media platform that has recently gained a lot of attention due to its influence on cryptocurrencies trends. for instance, the sub-reddit r/wallstreetbets allegedly played a role in influencing gamestop's stocks in 2021. 1 reddit is structured in communities, i.e., subreddits, with a user base of more than 50 million and more than 1.5 billion monthly visitors and has become one of the most popular social media in the us. 2 recently, the gained momentum of cryptocurrencies has been observed and enhanced by the increasing number of dedicated subreddits. in particular, almost every influential cryptocurrency has its dedicated subreddit. the popularity of some of them, such as r/wallstreebets (12 million subscribers), r/cryptocurrency (4.8 million subscribers), and r/bitcoin (4.2 million subscribers), highlight the widespread interest in cryptocurrencies in social media. 3 one peculiar aspect of cryptocurrencies is that they are not regulated by governments or other international institutions, but rather public opinions represent the main cause of crypto price changes. thus, we commonly observe the rise and fall of popular cryptocurrencies due to their high dependence on people's opinions. in this perspective, the role of public communities and social media platforms like reddit is highly influential in determining the trend of a cryptocurrency. this phenomenon has inspired researchers to leverage publicly available social media information, i.e., people's opinions, to evaluate their effect on cryptocurrency price forecasting. to do so, as a common conveyor of people's opinions, researchers have extracted sentiment information from textual data to enrich the set of input features for a forecasting model @xcite . in particular, the statistical analysis of recent work has shown that a correlation between extracted social media information and crypto price trends does exist @xcite . it is worth noting that the integration of public opinion is not new in the field of finance. other similar domains like stock market forecasting have shown promising results when integrating heterogeneous information from social media @xcite . nonetheless, the existing work on this matter is still preliminary from several perspectives. first, few available datasets providing scraped social media data exist nowadays and the number decreases drastically when considering the subset with sentiment annotations @xcite . second, the integration of social media information is carried out by a limited subset of sentiment features @xcite , while a wide variety of sentiment and emotion tools for efficient and accurate extraction can be leveraged on this matter. in this work, we propose a novel dataset for bitcoin price forecasting, called bitcoin reddit sentiment dataset (brsd). we create this dataset by pre-processing and integrating an existing dataset of reddit posts and comments with crypto price values. we employ a wide suite of sentiment and emotion recognition techniques to automatically annotate reddit textual data in addition to the existing comment-level sentiment annotations. we formulate the task of crypto price forecasting with both prices, sentiment, and emotion features with different widely adopted architectures. in particu-lar, we carry out an ablation study to evaluate the impact of extracted social media information by considering three different input configurations: (i) price data only, (ii) sentiment and emotion data only, and (iii) all available data. we make our dataset publicly available for research. 4 our experimental setting shows controversial results on the impact of sentiment and emotion data. in particular, the best-performing models' performance deteriorates when adding social media features. in contrast, the less-performing model shows trend learning capabilities only when considering the full set of features. in section 2 we analyze similar datasets in the field of crypto currency forecasting. section 3 describes the dataset creation process. in section 4, we introduce our experimental setting. section 5 concludes. ## related work the automatic forecasting of the stock market and cryptocurrency prices has gained a lot of interest in the past years @xcite @xcite . indeed, cryptocurrencies are currently widely adopted in finance due to their popularity @xcite . nonetheless, they are subject to public influence since they rely on decentralization and are not managed or regularized by government institutions. in particular, public opinion and crypto prices are strongly related @xcite . recent works on this topic have shown that social media can affect price changes and stand as reasonable indicators of the current economic trends @xcite . such a characteristic allowed the introduction of datasets for crypto price forecasting with users opinions extracted from online social media like reddit @xcite @xcite and twitter @xcite . however, there are few available datasets regarding reddit posts and comments with sentiment information @xcite . loginova et al. introduced a large dataset collected over 768 days (from february 2017 to april 2019) regarding several social media like reddit, bitcointalk, and cryptocompare. social media data is collected by leveraging pushshift apis, 5 a widely adopted tool for extracting such information @xcite @xcite , and later merged with financial data of five popular cryptocurrencies, namely btc, eth, ltc, xpr, and xmr, from the website coinmarketcap.com. in contrast, our dataset leverages multiple stateof-the-art sentiment and emotion recognition tools like lexicons, rule-based, and contextual techniques rather than employing aspect-based sentiment analysis. for what concern the period, our dataset covers only one month (august 2021) and focuses only on the btc cryptocurrency. nonetheless, its size is comparable to previous works and it is publicly available. another difference with @xcite is the time range on which we base our predictions. due to the length of our dataset, we predict the price in 15 minutes based on the last hour, whereas, they predict the price in 1 day based on features collected in 7 days averaged with a rolling window of 1 day. ## dataset creation we rely on a publicly available dataset of reddit content and on a well-known finance platform to build our dataset. we firstly collect and process data for each of the two datasets. subsequently, we derive our dataset by merging the information extracted from these two datasets and reporting statistics. ## experiments we address the task of crypto value forecasting by jointly leveraging price and sentiment features of domain-related reddit posts. formally, a forecasting model receives a sequence of price values {v 1 , v 2 , . . . , v t } regarding a time-window of size t and a sequence of sentiment features {s 1 , s 2 , . . . , s t }. each sentiment feature s t is a collection of sentiment values as described in section 3the two sequences are concatenated temporally-wise and fed as input to a model. the forecasting model then outputs a price value v t +w where w is the forecasting window size. our main objective is to study the impact of social media data on crypto price changes. therefore, we define an ablation study by considering three experimental input configurations for a forecasting model: (p) only price values {v 1 , v 2 , . . . , v t } are considered; (s) only sentiment features {s 1 , s 2 , . . . , s t } are considered; (p + s) both sentiment and price information are considered. we generally consider the forecasting task of predicting a time window of future crypto price values given a past time window of price, sentiment, and emotion features. in this work, we set @xmath0. to quantitatively evaluate the selected set of forecasting models, we split our dataset into train (25,025), validation (10,698), and test (8,916) splits. splits follow the sequential flow of crypto price time-series and, thus, no data shuffling is involved. we devise a preliminary hyper-parameter calibration phase based on the validation set. models are trained to minimize the mean squared error loss on the train set and are later evaluated on the test set. we apply a l 2 regularization to all the described models. furthermore, we consider early stopping regularization with patience set to 10 epochs. models are trained with adam optimizer (kingma and ba, 2015). ## conclusions we have introduced bitcoin reddit sentiment, a novel dataset for crypto price values forecasting. our dataset is built upon an existing kaggle dataset with reddit posts and comments taken from a wide set of domain-related subreddits. we enrich such a dataset by leveraging several state-of-the-art sentiment and emotion extraction tools to encode textual data. this approach is motivated by the assumption that public platforms like social media can affect the current trend of cryptocurrencies. thus, sentiment and emotion features stand as valuable