Predicting Sequential Data using LSTM: An Introduction | by George V Jose | Towards Data Science

Predicting the future of sequential data like stocks using Long Short Term Memory (LSTM) networks.
Forecasting is the process of predicting the future using current and previous data. The major challenge is understanding the patterns in the sequence of data and then using this pattern to analyse the future. If we were to hand-code the patterns, it would be tedious and changes for the next data. Deep Learning has proven to be better in understanding the patterns in both structured and unstructured data.
To understand the patterns in a long sequence of data, we need networks to analyse patterns across time. Recurrent Networks is the one usually used for learning such data. They are capable of understanding long and short term dependencies or temporal differences.
Alright, no more intro...
This post will show you how to implement a forecasting model using LSTM networks in Keras and with some cool visualizations. We’ll be using the stock price of Google from yahoo finance but feel free to use any stock data that you like.
I have used colab to implement this code to make the visualizations easier, you can use your preferred method. We’ll start off with importing necessary libraries:
After you’ve downloaded your .csv file from yahoo finance or your source, load the data using pandas.
The info of dataframe shows somewhat like this:
<class 'pandas.core.frame.DataFrame'>RangeIndex: 3797 entries, 0 to 3796Data columns (total 7 columns):Date         3797 non-null objectOpen         3797 non-null float64High         3797 non-null float64Low          3797 non-null float64Close        3797 non-null float64Adj Close    3797 non-null float64Volume       3797 non-null int64dtypes: float64(5), int64(1), object(1)memory usage: 207.7+ KB
For this tutorial, we require only Date and Close columns, everything else can be dropped.
Before we do the training and predictions, let's see how the data looks like. For all the visualizations, I’m using the Plotly python library. Why Plotly?... cause its simply the best graphing library and it can produce some good looking graphs.
With plotly, we can define a trace and the layout and it does everything else.
The graph is oscillating from 2018 and the sequence is not smooth... Moving on.
For our analysis, let train the model on the first 80% of data and test it on the remaining 20%.
Before we do the training, we need to do some major modification to our data. Remember, our data is still a sequence.. a list of numbers. The neural network is trained as a supervised model. Thus we need to convert the data from sequence to supervised data 😱.
Let me explain, training a neural network of any machine learning model requires the data to be in {<features>,<target>} format. Similarly, we need to convert the given data into this format. Here, we introduce a concept of a look back.
Look back is nothing but the number of previous days’ data to use, to predict the value for the next day. For example, let us say look back is 2; so in order to predict the stock price for tomorrow, we need the stock price of today and yesterday.
Coming back to the format, at a given day x(t), the features are the values of x(t-1), x(t-2), ...., x(t-n) where n is look back.
So if our data is like this,
[2, 3, 4, 5, 4, 6, 7, 6, 8, 9]
the required data format (n=3) would be this:
[2, 3, 4] -> [5][3, 4, 5] -> [4][4, 5, 4] -> [6][5, 4, 6] -> [7][4, 6, 7] -> [6][6, 7, 6] -> [8][7, 6, 8] -> [9]
Phew 😓.
Luckily, there is a module in Keras that does exactly this: TimeseriesGenerator. Please look up the documentation for more info.
I’ve set look_back as 15, but you can play around with that value.
Now that our data is ready, we can move on to creating and training our network.
A simple architecture of LSTM units trained using Adam optimizer and Mean Squared Loss function for 25 epochs. Note that instead of using model.fit(), we use model.fit_generator() because we have created a data generator.
To know more about LSTM network, see this awesome blog post.
Now that we have completed training, let us see if the network performed well. We can test the model on testing data and see if the prediction and the actual values overlap.
Rather than computing loss between predicted and actual values, we can plot it.
From the graph, we can see that prediction and the actual value(ground truth) somewhat overlap. But if you zoom in, the fit is not perfect. We should expect this because it is inevitable as we are performing prediction.
Our testing shows the model is somewhat good. So we can move on to predicting the future or forecasting.
Foreshadowing: Since we are attempting to predict the future, there will be a great amount of uncertainty in the prediction.
Predicting the future is easy... To predict tomorrow's value, feed into the model the past n(look_back) days’ values and we get tomorrow’s value as output. To get the day after tomorrow’s value, feed-in past n-1 days’ values along with tomorrow’s value and the model output day after tomorrow’s value.
Forecasting for longer duration is not feasible. So, let’s forecast a months stock price.
Now plotting the future values,
Predicting the future is easy... Is it?
When predicting the future, there is a good possibility that model output is uncertain to a great extent. The model’s output is fed back into it as input. This causes the model’s noise and uncertainty to be repeated and amplified.
But still, we have created a model that gives us a trend of the graphs and also the range of values that might be in the future.
With this model, we have created a rudimentary model that is able to forecast to a certain extent. Even though the model is not perfect, we have one that can approximate to the past data pretty well. But for new data, we would require more parameter tuning.
If there exists any problem during the training of the model, try tuning these parameters:
look_back
batch size
LSTM units
num_epochs
You can also enhance the architecture by stacking LSTM layers.
Thank You!! 😁