Time-Series Prediction of Bitcoin Price Using LSTM’s
Tags: Deep Learning, Bitcoin, Time Series Analysis, Python, Data Science

Time-Series Prediction of Bitcoin Price Using LSTM’s

Photo by Chris Liverani on Unsplash

Being a data scientist by profession and a part-time crypto trader by passion, I have been very interested in creating a Deep Learning model that could help me predict Bitcoin price. This article is based on the experimentation I did to create such a model.

A link to my Git Project is here: https://github.com/msaleem18/Bitcoin-Price-Prediction-LSTM

LSTM’s

Long short term memory, or more popularly known as LSTM’s, is a type of Recurrent Neural network that helps the model learn long-term sequences in the data set. Since my focus here is more on their usage, if you are interested in knowing more details about what LSTM’s are and how they work, you can check out this great article that goes in-depth to explain all that.

Dataset

For this project, I used Binance BTC/USDT hourly price dataset that can be found here: http://www.cryptodatadownload.com/data/binance/

I imported the data onto my local drive and read it as a CSV using pandas.

#READ DATA FROM FILE STORED IN YOUR LOCAL DDRIVE (DOWNLOAD CSV FILE SHARED IN THE PROJECT)

df = pd.read_csv("//Users//saad//Desktop//Bitcoin LSTM//bitcoin_hourly.csv") print(df.dtypes) print("

*** DF ***")

df_copy = df.copy() df_copy.head(3)

Binance BTC/USDT Dataset

Pre-processing

The dataset provided by Binance needs to be cleaned up for the following reasons:

They have been collecting data since 2017 and since then the format of some of the fields in the dataset has changed. The “date” field in the dataset is pretty useless — we need to create our own. So that we can add a few additional fields that’ll help us in the modelling process.

For this model I created fields to track the hour of the day and the weekday.

#CLEAN DATA - SPECIFICALLY CLEAN TIMESTAMP



def change_timestamp (ts):

digit_count = len(str(ts))

if digit_count == 12:

return (datetime.datetime.utcfromtimestamp(ts)).strftime('%Y-%m-%d %H:%M:%S')

else:

return (datetime.datetime.utcfromtimestamp(ts/1000)).strftime('%Y-%m-%d %H:%M:%S')



df_copy['unix_count'] = df.unix.apply(lambda x: len(str(x))) df_copy['dt_correct'] = df.unix.apply(lambda x: change_timestamp(x)) df_copy['dt'] = pd.to_datetime(df_copy.dt_correct.values) df_copy['hour'] = df_copy.dt.apply(lambda x: x.hour) df_copy['week_day'] = df_copy.dt.apply(lambda x: x.weekday()) df_copy.sort_values(by=['unix'],ascending=[True],inplace=True)



Normalization

Normalization is pretty important for LSTM modelling. The price of Bitcoin, since its inception, has varied a great deal starting from almost $0 reaching an all time high of $57,000 in the recent months; something that is not too good for the model. The higher numbers will end up gaining more weightage and will result in biasing the model.

In order to avoid this skewness, we have to normalize our data. There are quiet a few normalization methods available in the sklearn library, however, i decided to go with MinMax Scalar. I experimented with a few other methods such as Robust Scalar, but my model performed the best with MinMax Scalar and so I decided to go with it in the end.

One more thing to note, the best practice is to only normalize your training data and then use that same method to normalize your test data. However, since the price of Bitcoin has recently increased at a lot higher rate than in the previous years, this causes an issue with the scalar method. Since we are doing time-series prediction and the data is sorted in an ascending order, the early lower prices are picked for the training dataset and the new higher prices are only available in the test dataset. Just for this project I took the shortcut and to avoid excessive coding, I normalized the whole dataset and only split it into train and test after the normalization step. The best practice would be to use window scaling as explained here.

#SPLIT DATA INTO TEST AND TRAIN

np.random.seed(7)



X = df_work[['hour','week_day','Volume BTC','close']]

Y = df_work[['close']]

#X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.3, shuffle = False)





#NORMALIZATION

f_transformer = preprocessing.MinMaxScaler((-1,1))

f_transformer = f_transformer.fit(X)



cnt_transformer = preprocessing.MinMaxScaler((-1,1))

cnt_transformer = cnt_transformer.fit(Y)



X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.2, shuffle = False)



X_train_trans = f_transformer.transform(X_train)

X_test_trans = f_transformer.transform(X_test)



y_train_trans = cnt_transformer.transform(y_train)

y_test_trans = cnt_transformer.transform(y_test)





print("*** SHAPES")

print("X_train: %s, %s" % (X_train.shape[0],X_train.shape[1]))

print("X_test: %s, %s" % (X_test.shape[0],X_test.shape[1]))

print("y_train: %s, %s" % (y_train.shape[0],y_train.shape[1]))

print("y_test: %s, %s" % (y_test.shape[0],y_test.shape[1]))



print("

*** MIN MAX")



print("TRAIN COST: %d, %d" % (X_train.close.min(), X_train.close.max()))

print("TEST COST: %d, %d" % (X_test.close.min(), X_test.close.max()))

print("TRAIN VOL: %d, %d" % (X_train['Volume BTC'].min(), X_train['Volume BTC'].max()))

print("TEST VOL: %d, %d" % (X_test['Volume BTC'].min(), X_test['Volume BTC'].max()))



print("

*** MIN MAX PARAMETER")

print(f_transformer.data_min_)

print(f_transformer.data_max_)

print(cnt_transformer.data_min_)

print(cnt_transformer.data_max_)

Lagging Dataset

Once we have our cleaned and normalized data, we need to create a lagging dataset for LSTM time-series prediction. What do I mean by a lagging dataset?

Basically, how the LSTM time-series prediction works is that it looks back at x hours of data and tries to predict what will be the price at x+1 hour. The following image tries to explain this concept.

Creating a 1 Hour Lagging Dataset for Time-Series Prediction

For our time-series prediction, we’ll be using a lag of 24 hours.

#CREATE LAGGING DATASET FOR TIMESERIES

def create_dataset(X, y, time_steps=1):

Xs, ys = [], []

for i in range(len(X) - time_steps):

v = X[i:(i + time_steps)]

Xs.append(v)

ys.append(y[i + time_steps])

return np.array(Xs), np.array(ys)



time_steps = 24 # reshape to [samples, time_steps, n_features]

X_train_f, y_train_f = create_dataset(X_train_trans, y_train_trans, time_steps) X_test_f, y_test_f = create_dataset(X_test_trans, y_test_trans, time_steps)



print("*** SHAPES")

print(X_train_f.shape, y_train_f.shape)

print(X_test_f.shape, y_test_f.shape)

Modelling

Now comes the exciting part — creating the actual LSTM model. For me at least, this turned out to be pretty simple and quick. I ended up using Keras’s Bidirectional LSTM’s with a Batch Normalization layer in my architecture.

I exprimented with a a bunch of other complex architectures, such as:

LSTM’s with different activation functions (tanh vs relu)

Stacked LSTM layers (2 and 3 LSTM layers)

CNN + LSTM model

However, my model performed the best using a single LSTM layer only (my mean-squared-error was the lowest for this architecture).

model = keras.Sequential() model.add(keras.Input(shape=((X_train_f.shape[1], X_train_f.shape[2])))) model.add(layers.Bidirectional(layers.LSTM(300, activation = 'tanh', return_sequences=False))) model.add(layers.BatchNormalization()) model.add(keras.layers.Dense(units=1)) model.compile(loss='mean_squared_error', optimizer='adam') model.summary()

LSTM Model Architecture

Prediction

The Bidirectional LSTM model seems to work pretty good and tracks the actual BTC price very closely as shown in the following graph

Bitcoin Price Prediction

Conclusion

Even though the model is performing pretty well, I still believe there is a lot of room for improvement. The crypto market is very volatile and this model is not using any information from the current news or trends. An important enhancement would be to use Natural Language Processing (NLP) and feed in news data into this model.

I’m going to use this model in my trading in the upcoming weeks and see how well it performs. You can find the complete project on my Git page here: https://github.com/msaleem18/Bitcoin-Price-Prediction-LSTM