LSTM Network and Time-Series Data to Predict Future Prices | by Sarit Maitra | Towards Data Science

https://sarit-maitra.medium.com/membership
Note from Towards Data Science’s editors: While we allow independent authors to publish articles in accordance with our rules and guidelines, we do not endorse each author’s contribution. You should not rely on an author’s works without seeking professional advice. See our Reader Terms for details.
Prediction of stock price is quite a challenging because of highly volatile nature of time series combined with stochastic movement with non-linear . Here, the problem we have in hand is a price prediction issue and we’re trying to predict a numerical value defined in a range (from 9000 to 12500 approx). This problem fits the Regression Analysis framework. We shall be using neural network architecture to try to solve the problem here.
We’ll build a Deep Neural Network here that does some forecasting for us and use it to predict future price. Let us load the hourly frequency data.
We have a total of 2001 data points representing Bitcoin in USD . We’re interested in predicting the closing price for future dates.
When we see a time-series, we always want to know if the value of the current time step affects the next one.
plt.figure(figsize = (15,5))plt.plot(btc.close)plt.title('BTC Close price (Hourly frequency)')plt.xlabel ('Date_time')plt.ylabel ('Price (US$')plt.show()
We shall use LSTM network here which has the ability to capture long-term dependencies in a sequence (e.g. dependency between today‘s price and that 2 weeks ago). Moreover, uni-variate series is being used here considering only the close price from the series.
Let us normalize the price data using MinMax scaler.
scaler = MinMaxScaler() close_price = btc['close'].values.reshape(-1, 1) # The scaler expects the data to be shaped as (x, y)scaled_close = scaler.fit_transform(close_price)scaled_close = scaled_close[~np.isnan(scaled_close)] # reshaping data after removing NaNsscaled_close = scaled_close.reshape(-1, 1) # reshaping data after removing NaNs
LSTMs require 3-D data shape; therefore, we need to split the data into the shape of : [batch_size, sequence_length, n_features]. We also want to save some data for testing.
Let’s build some sequences. Sequences work like walk forward validation approach, where initial sequence length will be defined and subsequently will be shifting one position to the right to create another sequence. This way the process is repeated until all possible positions are used.
SEQ_LEN = 100 # creating a sequence of 100 hours at position 0.def to_sequences(data, seq_len):d = []for index in range(len(data) - seq_len):d.append(data[index: index + seq_len])return np.array(d)def preprocess(data_raw, seq_len, train_split):data = to_sequences(data_raw, seq_len)num_train = int(train_split * data.shape[0])X_train = data[:num_train, :-1, :]y_train = data[:num_train, -1, :]X_test = data[num_train:, :-1, :]y_test = data[num_train:, -1, :]return X_train, y_train, X_test, y_test"""Walk forward validation: Initial SEQ_LEN is defined above, so, walk forward will be shifting one position to the right and create another sequence.The process is repeated until all possible positions are used."""X_train, y_train, X_test, y_test = preprocess(scaled_close, SEQ_LEN, train_split = 0.95) # 5% of the data saved for testing.print(X_train.shape, X_test.shape)"""Our model will use 1805 sequences representing 99 hours of Bitcoin price changes each for training. We shall be predicting the price for 96 hours in the future"""
If model has high error in both the train and test datasets, that will indicate model is underfitting both sets and has high bias. If model has low error in the training set but high error in the test set, this is indicative of high variance because model has failed to generalize to the second set of data.
So, target is to generate a model with overall low error in both train and test datasets and balanced the right levels of bias and variance.
We will create a 3 layer LSTM Network using Dropout with a rate of 20% to control over-fitting during training.
DROPOUT = 0.2 # 20% Dropout is used to control over-fitting during trainingWINDOW_SIZE = SEQ_LEN - 1model = keras.Sequential()# Input layermodel.add(Bidirectional(LSTM(WINDOW_SIZE, return_sequences=True), input_shape=(WINDOW_SIZE, X_train.shape[-1])))"""Bidirectional RNNs allows to train on the sequence data in forward and backward direction."""model.add(Dropout(rate=DROPOUT))# 1st Hidden layermodel.add(Bidirectional(LSTM((WINDOW_SIZE * 2), return_sequences = True)))model.add(Dropout(rate=DROPOUT))# 2nd Hidden layermodel.add(Bidirectional(LSTM(WINDOW_SIZE, return_sequences=False)))# output layermodel.add(Dense(units=1))model.add(Activation('linear'))"""Output layer has a single neuron (predicted Bitcoin price). We use Linear activation function which activation is proportional to the input."""BATCH_SIZE = 64model.compile(loss='mean_squared_error', optimizer='adam')history = model.fit(X_train, y_train, epochs=50, batch_size=BATCH_SIZE, shuffle=False, validation_split=0.1) # shuffle not advisable during training of Time Series
We can use callbacks option during training to prevent our model from over-fitting too; I have not used call-back, but same can be applied during model fitting phase.
A simple way to understand the training process is view the training and validation loss.
# history for lossplt.figure(figsize = (10,5))plt.plot(history.history['loss'])plt.plot(history.history['val_loss'])plt.title('model loss')plt.ylabel('loss')plt.xlabel('epoch')plt.legend(['train', 'test'], loc='upper left')plt.show()
Here, we can see some improvement in both the training error and on the validation error.
We have some additional data left for testing purpose. Let’s get the predictions from the model using those data to validate the goodness of fit of our model.
# prediction on test datay_pred = model.predict(X_test) # invert the test to original valuesy_test_inverse = DataFrame(scaler.inverse_transform(y_test)) # assigning datetimey_test_inverse.index = btc.index[-len(y_test):] print('Test data:',)print(y_test_inverse.tail(3)); print();# invert the prediction to understandable valuesy_pred_inverse = DataFrame(scaler.inverse_transform(y_pred)) # assigning datetimey_pred_inverse.index = y_test_inverse.index print('Prediction data:',)print(y_pred_inverse.tail(3))
print(f'MAE {mean_absolute_error(y_test, y_pred)}')print(f'MSE {mean_squared_error(y_test, y_pred)}')print(f'RMSE {np.sqrt(mean_squared_error(y_test, y_pred))}')print(f'R2 {r2_score(y_test, y_pred)}')
RMSE allows us to penalize points further from the mean. Though the error scores are low but looking at R2 score, there is definitely room for improvement here. The model can be tuned to get better output.
plt.figure(figsize = (15,5))plt.plot(y_test_inverse)plt.plot(y_pred_inverse)plt.title('Actual vs Prediction plot (Price prediction model)')plt.ylabel('price')plt.xlabel('date')plt.legend(['actual', 'prediction'], loc='upper left')plt.show()
Looks like our basic and brief model has been able to capture the general pattern of the data. However, it failed to capture stochastic movements, which may be a good sign in terms of we can say that, it generalizes well.
Predicting stock market returns is a challenging task due to consistently changing stock values which are dependent on multiple parameters which form complex patterns.
Future direction could be:
analyzing the correlation between different cryptocurrencies and how would that affect the performance of our model.adding features using technical analysis to check the model performance.adding features from fundamental analysis to check how those affect the mode.adding sentiment analysis from social networking e.g. twitter and new report to check model performance.GRU network can also be tried with different activation e.g. ‘softsign” to check the performance.
analyzing the correlation between different cryptocurrencies and how would that affect the performance of our model.
adding features using technical analysis to check the model performance.
adding features from fundamental analysis to check how those affect the mode.
adding sentiment analysis from social networking e.g. twitter and new report to check model performance.
GRU network can also be tried with different activation e.g. ‘softsign” to check the performance.
Multivariate analysis of this nature require a great amount of effort and time on feature engineering, data analysis, model training etc.
Connect me here.
References:
Vijh, M., Chandola, D., Tikkiwal, V. A., & Kumar, A. (2020). Stock Closing Price Prediction using Machine Learning Techniques. Procedia Computer Science, 167, 599–606.Vanelin Valkov (2015). Hackers guide to machine learning
Vijh, M., Chandola, D., Tikkiwal, V. A., & Kumar, A. (2020). Stock Closing Price Prediction using Machine Learning Techniques. Procedia Computer Science, 167, 599–606.
Vanelin Valkov (2015). Hackers guide to machine learning
Note: The programs described here are experimental and should be used with caution for any commercial purpose. All such use at your own risk.