XRP: Buy.Sell.Hodl.
By: Richard Do
Tags: Sentiment Analysis, Data Science, Autoregressive, Cryptocurrency, Time Series Forecasting

Time series modeling and cryptocurrency forecasting

XRP is the symbol for Ripple cryptocurrency. HODL (Hold on for Dear Life)

The ‘irrational exuberance’ that drove the 2017/18 rise was based on investors believing that trees could grow to the sky. Now that they see they can’t, those conditions probably won’t repeat themselves (or maybe they will, people have short memories).

Task at hand.

Can we build a model that could day trade us to the moon?

Conclusions first, what did we learn?

basic time series models are not able to accurately forecast daily prices— fundamental assumptions are violated. our best guess is a random guess

coin price history and Ripple subreddit thread titles are not enough information to predict prices (it’s all public information, no advantage).

day trading cryptocurrency is not recommended unless user has information not available to the public or a better model that accounts for the stochastic behavior of the market

Nitty gritty for those who are interested.

Getting into the mud, I will try my best to explain the finer details.

Formal task at hand.

Given coin price history and ripple subreddit thread titles, can we build a model that could perform better than a random guess?In other words, can our model demonstrate a test root mean square error (RMSE) that is less than the standard deviation of the test data (random guess).

Approach.

Gather data using various web scraping tools in python. Perform exploratory data analysis (EDA) and normality checks to validate model assumptions. Utilize autoregressive models with increasing complexity to see which could best predict the market price. Analyze the results.

Step 1) Gathering data

In order to do data analysis, there has to be data. Ripple coin historical price data was gathered using Selenium. Reddit thread titles were gathered using pushshift’s api.

Step 2) Exploratory data analysis (EDA) and normality checks

Purpose of EDA and normality checks is clean, understand, and to be one with the data. The plot shows price over time (y-axis is in log scale). Price skyrockets after the first quarter of 2017 and increases more than 2 orders of magnitude by the beginning of 2018. The data is not normally distributed so the percent change was utilized to help transform the data to be more normally distributed (an assumption of autoregressive models). After transforming the data, the dicker fuller test yielded a p-value under 0.05 and the Q-Q plot (not shown here) fell more in line with the 45 degree line (still had heavy tails).

Second piece of data was the thread titles from ripple subreddit. A python toolbox called VADER was used perform sentiment analysis on the thread titles. VADER outputs a compound score between -1 to 1 for titles with a negative or positive sentiment, respectively. Below shows the price daily percent change (green) and the thread title sentiment score (blue). It is interesting to observe that the rise in price after 2017 correlated with the change in sentiment to be overall positive. However, the data only contains titles from the ripple subreddit so it is likely bias toward a positive/bullish mentality.

3) Autoregressive models

Autoregressive (AR) models are a class of models that could be used to predict a time-varying process. In short, it uses data from the past to predict the future. AR models were built with increasing complexity and parameters were optimized using an exhaustive grid search. The ranges of each parameter for the grid search was guided by the autocorrelation function (acf) and the partial autocorrelation function (pacf). The grid search utilized a rolling window approach to evaluate each model where a training set of data was used to forecast the following point, which was then compared to the actual value. The rolling window process was repeated over the validation set and the RMSE metric was used to score each model. The optimized models are summarized in the table below.

The most complex model, SARIMAX, is the only model that incorporates the exogenous data (Ripple thread title sentiment analysis). The sentiment analysis from the previous day was added to predict the following day.

Overall, as the model complexity increased (column 1 and 2), the validation RMSE (column 3) decreased. It was very hopeful to observe this trend during model optimization however, the test RMSE (column 4) shows that each model scored relatively the same. More on why in the following section.

4) Results analysis

If the test RMSE matches the standard deviation than the model is no better than a random guess. After optimizing each model it was discouraging but unsurprising to see how each of the models performed on the testing data. All the models had the same amount of the test RMSE. How can that be? Well, turns out the standard deviation of the test data matches that of the test RMSE (random guess woohoo!). Furthermore, after looking at additional rolling statistics on the data, it was discovered that the test data behaved statistically different compared to the training and validation data. The process was too non-stationary, generally stoachastic or random, to allow the models to perform well on the test. Lastly, if everyone else in the market incorporates the same information in their model, a basic model shouldn’t be expected to out perform the market.

Additional comments.

Markov chains maybe a better model as it could account for the various states spaces. Ripple coin market behavior is not consistent and thus should not be treated as so. All code to perform the analysis is listed on my github https://github.com/rdoooooo/XRP_buy_sell_hodl This was my first medium post, all feedback is welcomed!

References