Predict Bitcoin prices by using Signature time series modelling | by Rattana Pukdee | Towards Data Science

First, I would like to give a short introduction to the Signature method. According to Wikipedia, a rough path is a generalization of the notion of smooth path allowing to construct a robust solution theory for controlled differential equations driven by classically irregular signals, for example, a Wiener process. The theory was developed in the 1990s by Terry Lyons. The aim of the mathematics is to describe a smooth but potentially highly oscillatory and multidimensional path X effectively.
The Signature is a homomorphism from the monoid of paths (under concatenation) into the group-like elements of the free tensor algebra. It provides a graduated summary of path X. Here is a formal maths definition of the Signature transformation from A Primer on the Signature Method in Machine Learning.
For a path
Define the Signature of a path as follows
where
To make a long story short, Signature is a transformation of a path into a sequence that encapsulates summaries of the path.
These graduated summaries or features of a path are at the heart of the definition of a rough path; locally they remove the need to look at the fine structure of the path. Taylor’s theorem explains how any smooth function can, locally, be expressed as a linear combination of certain special functions (monomials based at that point).
Coordinate iterated integrals (terms of the signature) form a more subtle algebra of features that can describe a stream or path in an analogous way; they allow a definition of rough path and form a natural linear “basis” for continuous functions on paths.
There are many advantages of using Signature as a basis for functions on paths. First, Signature features are more robust for a rough path. Second, although the signature of a path is an infinitely long sequence, we can use a truncated version of it as a basis to approximate a continuous function without losing much information.
Moreover, Signature features are scalable. To give an illustration, if we want to model a future crude oil price as a function of historical crude oil prices.
We can use past oil prices as features. The more prices we use, the more information about its behaviour and the more accurate the model is, but with more computationally expensive. That is, 1,000 historical prices lead to 1,000 features and 1,000,000 historical prices lead to 1,000,000 features.Alternatively, we can construct a path from past oil prices and use the truncated signature of that path as features. Even when we use more past prices for our path, the number of features will be the same. For example, let X be a path of dimension 2, the level 2 truncated signature of X includes only 7 elements. That is using 1,000 past prices or 1,000,000 past prices will give us exactly 7 signature features. This would help with computation time (if we take into account the time we spend calculating the signature of a path) and potentially give us insight about the data (one signature feature may turn out to be important). However, we need to be careful about using a too low level of truncation which may lead to an underfitting.
We can use past oil prices as features. The more prices we use, the more information about its behaviour and the more accurate the model is, but with more computationally expensive. That is, 1,000 historical prices lead to 1,000 features and 1,000,000 historical prices lead to 1,000,000 features.
Alternatively, we can construct a path from past oil prices and use the truncated signature of that path as features. Even when we use more past prices for our path, the number of features will be the same. For example, let X be a path of dimension 2, the level 2 truncated signature of X includes only 7 elements. That is using 1,000 past prices or 1,000,000 past prices will give us exactly 7 signature features. This would help with computation time (if we take into account the time we spend calculating the signature of a path) and potentially give us insight about the data (one signature feature may turn out to be important). However, we need to be careful about using a too low level of truncation which may lead to an underfitting.
If you want more information about maths behind the Signature, I recommend you to read
Rough paths, Signatures and the modelling of functions on streams
A Primer on the Signature Method in Machine Learning
Here are examples of an application of the Signature features
Extracting information from the signature of a financial data stream
Sparse arrays of signatures for online character recognition
Application of the Signature Method to Pattern Recognition in the CEQUEL Clinical Trial
QuantStart — Rough Path Theory and Signatures Applied To Quantitative Finance
Derivatives pricing using signature payoffs
Recently, we have seen a remarkable rise of cryptocurrency trading where the most popular currency, Bitcoin, reached its peak at almost $20,000 USD/BTC at the end of 2017 while there was a big crash in November 2018 to around $3000 USD/BTC. The digital money is quite new in the financial market and we would say that its behaviour is almost unpredictable. Knowing that signature can capture meaningful properties of a path and can be used as a linear basis to approximate a continuous function of the path, the author wants to explore how would this promising method perform when we use it to predict the Bitcoin price. We will compare the result with XGBoost algorithm which is a current state-of-the-art machine learning algorithm.
We will use daily Bitcoin to USD prices data from https://www.cryptodatadownload.com/. We will use data from Gemini which is one of the biggest cryptocurrency trading platforms in the US. Our goal is to use a window of size 30 days to predict the mean price of the next 10 days. For the signature method, we will use a truncated signature of the 30-day prices as features + Lasso linear regression while for the XGBoost, we will use the 30-day prices as features.
We explore the dataset using pandas.
We need to remove the first row of the dataframe, reverse the dataframe and use Date as an index.
So now we are ready to work with the dataframe. Let’s visualise the price by plotting the Close price first.
#Plot to visualise dataimport matplotlib.pyplot as pltax = BTC_price.plot(y= 'Close', figsize=(12,6), legend=True, grid=True, use_index=True)plt.show()
There are a few interesting periods that we may be interested in, a boom on Oct 2017 along with crashes on Jan 2018 and Oct 2018. First, we will use data from Jan 2017 to Nov 2017 to see if the model could predict the boom period.
#select durationinitial_date = '2017-01-01'finish_date = '2017-12-01'BTC_price_time = BTC_price[initial_date:finish_date]
Next, we will construct features for our machine learning algorithm. Firstly, we will write a function that produces a window of historical prices of size h and means of the next future f prices.
We will test our function with a sequence of Close prices of length 10
BTC_price_time['Close'].head(10)
GetWindow(BTC_price_time.loc[:,'Close'].head(10), h_window = 5, f_window =2)
We get a dataframe that contains a rolling window of size 5 as shown below.
GetNextMean(BTC_price_time.loc[:,'Close'].head(10), h_window = 5, f_window =2)
GetNextMean provides us with a dataframe that contains means of 2 consecutive prices start at the 6th price. For example, 896.12 = (893.49+898.75)/2.
In addition to the prices window, we add a time column for features as well.
Now, let’s construct signature features. As mention above, signatures are in the form of iterated integral of a continuous path but we only have discrete data points. There are various ways to transform discrete data points we have into a continuous path. For example,
piece-wise linear interpolation
rectanlinear interpolation
Here are illustrations of each transformation from A Primer on the Signature Method in Machine Learning. For two one-dimensional sequence of length 4,
However, there is another interesting transformation which is a Lead-Lag transformation which transforms a one-dimensional into a two-dimensional path.
Here, we will use this Lead-Lag transform on our discrete data points and use the signature of that path for our features.
We test with a sequence (1,1),(2,4),(3,2),(4,6).
Next, it’s time to calculate the truncated signature of a path! We are lucky enough and don’t have to write a function to calculate the iterated integrals. Apparently, there is a package ESig that will do the (dirty) works for us though it is still in the active stages of development.
pip install esigimport esig.tosig as ts
Following is the documentation of ts.stream2sig(...).stream2sig(array(no_of_ticks x signal_dimension), signature_degree) reads a 2 dimensional numpy array of floats, “the data in stream space” and returns a numpy vector containing the signature of the vector series up to given signature_degree.
We will use this function to calculate the signature of a path (Time_lead, Price_lead, Price_lag).
Now, we are ready to calculate normal features and signature features. For simplicity, we will only use close price of the Bitcoin. What the following code do is to get normal window features with a time column, get prediction target which is mean of the future prices and calculate signature features. For the signature features, we use ESig package to find the level 2 truncated signature of a path (time_lead, price_lead, price_lag).
We can check the resulted features.
y.head()
pd.DataFrame(X_window).head()
pd.DataFrame(X_sig).head()
We split X_sig, y into train and test set for the model and we will predict 10 future prices.
Firstly, we will train our model on a Lasso linear regression with signature features. We use a time series split test cross-validation and gridsearchCV to tune a hyperparameter alpha.
Here is the error
and the indispensable part of a time series prediction is to visualise the result.
PlotResult(y_train, y_test, y_train_predict, y_test_predict, test_len, 'Lasso + Signature features')
The model can predict that there will be a boom in the future with an adequate accuracy of around 15%. Note that here we only use the level 2 truncated signature features. In theory, if we increase the level of truncation, we expect the model to be more accurate as we have more information
Now we will try our XGBoost model. We also use Time series split cross-validation and GridsearchCV to tune hyperparameters too.
We observe that the mean absolute error for the train set is quite low which is a sign of overfitting and the value for the test set is about the same as the Signature method. Let’s visualise the graph
PlotResult(y_train, y_test, y_train_predict, y_test_predict, test_len, 'XGBoost')
We can see that the model can’t predict the boom period at all and it guessed that the mean future prices are stable. One possible reason for this to happen is that the model hasn’t experienced such a sharp increase before.
We will use a period from Jan 2018 to Dec 2018 to test whether the model can predict a crash. We test with a duration where the price was quite stable.
#select durationinitial_date = '2018-01-01'finish_date = '2018-11-01'
The following is the result of the signature features.
We got a Convergence warning saying that the model does not converge and the resulted graph is quite weird. Note that here we use only the truncated signature level 2 which has only 12 features so they may not capture important information about the path.
We experiment with the truncated signature level 3.
The model did poorer in term or an error but we can see from the graph that it has a more realistic path than the level 2 truncated signature features.
On the other hand here is the result of the XGBoost algorithm.
The model did extremely well and made a prediction with an error of up to 0.5%.
Now, let’s challenge whether these model can predict a crash in Dec 2018.
We used level 3 truncated signature features and here is the result.
compared with the XGBoost model
It’s clear that none of the models can predict the crash in advance. We may reach the limit of using only historical prices here. In the real world, there are many factors that affect the cryptocurrency price such as news, the attractiveness of other financial products. However, technical analysis like this could give us a quick picture of the data.
We have learnt how to use the signature features to model a time series and compare it with the XGBoost algorithm which is the cutting-edge algorithm at the moment. The XBGoost has done extremely well in a period of stable prices while it could not give us much information when there is a boom or a bust. On the other hand, the Signature method did okay in a stable prices period but potentially can give some information about a significant change. Nonetheless, the data set that the author picked can be biased because we know beforehand that when is the boom period and when is the bust period. To conclude, the method is quite new and still need to prove itself and I encourage readers to have a play on it.
Note from Towards Data Science’s editors: While we allow independent authors to publish articles in accordance with our rules and guidelines, we do not endorse each author’s contribution. You should not rely on an author’s works without seeking professional advice. See our Reader Terms for details.