Time Series analysis on multivariate data in Tensorflow | by Killol Govani | Towards Data Science

In layman’s term, a time series analysis deals with time-series data mostly used to forecast future values from its past values. The application could range from predicting prices of stock, a commodity like crude oil, sales of a product like a car, FMCG product like shampoo, to predicting Air Quality Index of a particular region. A time series can be classified into univariate and multivariate time series. A univariate time series data consists of only single observation recorded over time, while a multivariate time series consists of more than one observation, related to our subject of interest. The focus of this article will be on multivariate data.
For performing Time Series analysis, we require data to build our model; for which we will refer to the publicly available datasets from kaggle. For this article, we will use the Air Quality Index data for the past five years of major Indian cities, provided in this link. To download the data directly into Google Colab, you can refer to the below links:
www.kaggle.com
towardsdatascience.com
If you have followed the necessary steps, you should see the following files in your downloaded data. We are interested in city_day.csv file.
Let us select the required data for the analysis. We will select data for the last two years corresponding to the city of Delhi. The data has 1948 rows and 15 columns including AQI for which we want to build a model.
By inspecting the data, we observe that our data contains missing values. Hence we fill the missing values by the mean of each column, drop unnecessary columns, before starting the analysis.
Now that we are equipped with the data, let’s move on to the next section.
As the name suggests, a time-series data consists of single or multiple observations/variables, which are recorded sequentially in a definite time interval. There are various aspects to time-series data, which we will understand by performing some Exploratory Data Analysis:
In Time Series data, seasonality refers to the cyclic variations which occur in the data at a regular interval usually less than a year. Some examples are, peak in retail/e-commerce sales during Christmas, increase in sale of warm clothes during winter, etc. Seasonality can be found out by looking at the graph itself. The code below renders a graph, looking at which we can say that despite several crest and trough, they do not occur at a regular interval and hence the data is not seasonal.
In Time Series data, trend refers to the general tendency of the data, whether it is increasing or decreasing over time(long term direction). Looking at the above graph, we can say that the general trend of the data is neither increasing nor decreasing but a constant trend.
Irregular or random components are the ones which cannot be predicted. Mostly they are of short duration and non-repeating and occur due to unforeseen events.
When the rise and fall pattern exists, but not at a fixed interval then it is known as cyclicity. One example can be to consider the GDP numbers of a country when plotted on a graph versus time, it will see a huge dip during the time of recession like 1929, 2000 and 2008, but these intervals are not fixed. Hence from the above graph, we can say that our data is cyclical.
It means the correlation of the present values with its past values. An autocorrelation function measures the degree of similarity of the present series with that of its lagged series (past values). The ACF graph for the AQI index shows spike above the blue region, which is an indication that the series is autocorrelated. Similarly, you can check autocorrelation for other series of the dataset also.
A time series is said to be stationary if its corresponding statistical properties like mean, standard deviation and autocorrelation remain constant throughout the time. To check the stationarity of multivariate time series, we perform Johansen cointegration test on the time series which return the eigenvalues in the form of an array. If the eigenvalues are less than one than the series is said to be stationary.
Since the values we get are less than one, the series is stationary.
Recurrent Neural Networks (RNNs) are a type of Neural Networks that are used to the model sequence of data like time series and natural language. A regular Neural Network takes some information x as input and outputs a value y. The output of one layer is not fed into another layer.
RNNs overcome this drawback by looping back the output of one layer into the other layer.
This means that RNNs should be able to predict what comes next based on the context of the previous data, but in practice, RNNs solve this problem only when the gap between this context and the prediction is small and performs poorly if this gap becomes vast. Fortunately, Long-Short Term Memory (LSTMs) solve this problem, which is a special type of RNNs.
RNNs, as mentioned above, remember every information, in the sequence which leads to the problem of vanishing gradient. During the backpropagation, the gradient becomes so small that the hidden layers learn very less when updated. LSTMs solve this problem while remembering only the relevant information and discarding other information. RNNs have just one activation function while LSTMs have a series activation function and gates through which they perform this task.
Now that we have some idea about LSTMs and where they are used, let’s move forward to building our model.
So far we have loaded over data into our machines for modelling. Upon some inspection, we see that our data does not contain any NULL or NaN values. As we are performing Time Series Analysis through machine learning modelling, we need to convert our data in the form of dependent (y) and independent variables (X). Here our dependent variable is AQI and rest variables like PM2.5, PM10, NO, NO2..... and so on are the independent variable.
We will scale the data to process it faster and removing bias due to the range of data. Now we need to divide, the data into train and test data in the ratio 80:20. Note that since we are performing time series analysis, we cannot divide out data into train and test randomly, as we do while building other machine learning models, otherwise, the model will lode the most important essence, i.e. Time.
We will use the Keras API on the top of the Tensorflow library to build our model. Following is a blueprint of our model architecture.
model.add(LSTM(hidden_nodes, input_shape=(timesteps, input_dim)))model.add(Dropout(dropout_value))model.compile(loss, optimizer)
hidden_nodes: The number of neurons in the hidden layer. We will take 250 neurons
Dropout: It is used to reduce the overfitting of the model. Empirically it is set to 20%.
Compile: The compile function will take parameters like loss and optimizer. Here we will take Mean Absolute Error as an error metric and Adam as an optimization algorithm.
After fitting the model, we will check our model’s performance on the test data. As we normalized our data in the beginning before fitting the model, we will have to reverse scale it to get back values in the original scale.
Now that we have our predicted values on the test data set and original test data, let us compare both by calculating the RMSE and plotting them together.
Train RMSE: 41.096
Test RMSE: 30.493
It is a good indicator for our model that the RMSE on train and test data does not differ much. Note that you might get different values of RMSE each time you build and run the model.
A time series forecasting can be said to be either single-step or multi-step forecasting depending on the number of time steps it is capable to predict in the future. Based on the model trained from our train data, we are predicting the future values for a time frame corresponding to the test data and hence it is multi-step forecasting. Remember the graph of AQI, we saw at the beginning of the article, with the prediction from the model, the graph looks like:
In this article, we saw how we can use RNN-LSTM for building a multivariate time series model as they are good at extracting patterns from sequential data.
In this article, we saw how we can use RNN-LSTM for building a multivariate time series model as they are good at extracting patterns from sequential data.
2. The time-series data should not be divided into train and test set randomly, as it will lose the most important essence of it i.e. Time.
3. Traditional methods for time series forecasting like ARIMA has its limitation as it can only be used for univariate data and one step forecasting.
4. It is observed in various studies that deep learning models outperform traditional forecasting methods on multivariate time series data.
5. As the provided data contains seasonality, we can further deseasonalize the data so that our model can focus on the underlying signal.
You can find the full code for this article here