Facing the ARIMA Model against Neural Networks | by ds_mt | Towards Data Science

Introduction
The purpose of this small project is to go through the ARIMA model to evaluate its performance in a univariate dataset. Also, its performance will be compared with other techniques that are currently available to create predictions in time series using neural networks.
The dataset that is going to be used, belongs to the UCI repository (http://archive.ics.uci.edu/ml/datasets/Beijing+PM2.5+Data)
It has an hourly measurement of the quantity of PM2.5 in the air in the city of Beijing. This dataset comes with other information such as the meteorological data from that city. However, as the purpose of this exercise is to make a model from a univariate variable, this other information is not going to be taken into account. We will go step by step through the whole process: starting by importing the data, getting some insights to it, applying the ARIMA model and finally comparing the results with a neural network to evaluate the performance of each model.
(Disclosure)This post consists of different methods for forecasting time series. However, none of these methods is perfect as there is no perfect way to predict the future, so these results should be taken with care and always with the advice of an expert.
If the code is what you are looking for is all accessible in a Jupyter Notebook here
As I mentioned previously this dataset contains the information about the amount of PM2.5 (ug/m3) concentration of particles in the air of Beijing.
These are particles with a diameter of fewer than 2.5 micrometres that are floating in the air.(https://www.who.int/news-room/fact-sheets/detail/ambient-(outdoor)-air-quality-and-health)The world health organization states that exposure to these particles can cause cardiovascular and respiratory diseases and cancer. They estimate that this air pollution caused 4.2 million premature death per year in 2016.
A quick look at the data to see what we need to handle.
Regarding this first plot, we can see that at the end and at the beginning of each year the concentration of PM2.5 particles is higher than in other periods of the year, but it is still fuzzy data and we can barely have a better understanding of what is really happening. Because of this, some transformations are needed to be able to get a deeper insight into our data. There are many tools available, and here we are only going to use two, but there are many others that can provide better or different insights.
Grouping by different frames of time
The first method is as simple as to compute the mean of the data measurements by week or month.
As we can see from the plots, the information that we get from the data using this method is not very clear.
Moving average
This time we are still going to use the mean but in a different way, by using the Moving Average, that computes the average of N given time steps. It will smooth the data allowing the viewer to infer some visible patterns or trends.
In this last plot, we can clearly spot some regular patterns. However, these methods are very sensitive to outliers and as we can see in the first plot our data has many of them.
Please note the difference between these two transformations because they look similar but they are not the same. In the average, we just compute the average per week and in the Moving Average (MA) we compute the average of N lagged observations. Although, the second one is more useful because it will help us to smooth the data removing the noise.
First Conclusion
There is nothing clear we can get from the data even though it seems to have many outliers. and one can think of these outliers as faulty measures by the devices related to any kind of misfunction. However, doing some research we will find that these levels have been already reached and documented in the capital of China.
www.theguardian.com
Getting closer to the last weeks
The data that we have evaluated so far looks quite messy, and as the objective of this small project is to apply different forecasting techniques, we are going to focus the efforts in the last four weeks of the entire dataset. By doing so the visualizations are going the be easier and we will see clearly how the predictions fit in our data.
In the distribution of the data, we can see that most of the data is grouped in the first values, looking like the exponential distribution.
In this data dataset, there are 716 measures of the particle’s concentration, around one month of data. With a mean of 78 particles per hour. This means is considered as unhealthy and people should not be exposed during long periods of time to these levels.
Stationarity
Before applying any statistical model it’s important to check if our data is considered as stationary
Stationarity basically means that the properties such as the mean and variance don’t change over time. (https://www.itl.nist.gov/div898/handbook/pmc/section4/pmc442.htm)
There are various ways to check if data is stationary, and one good way is the Test Dickey-Fuller, which states that if the p-value is lower than a given threshold it won’t be considered as stationary.
Test Dickey-Fuller
As we proved with Dickey-Fuller Test, fortunately, our data is stationary which means that it doesn’t have any kind of trend and therefore it doesn’t need any transformations and we can directly to apply ARIMA.
Sometimes the time series are non-stationary. For these cases, it is possible to apply transformations to the data that will make them stationary.
Before going through ARIMA we are going to split the data that will help us to train the model, and after that evaluate how accurate it is with the test dataset.
train_dataset = last_weeks['2014-12': '2014-12-29']test_dataset = last_weeks['2014-12-30': '2014']
Train dataset has the data of 29 days and the test set has 2 days.
The purpose of splitting the dataset is because the model has to be tested with some labelled data, that means to see how the predictions are close to the real data.
ARIMA stands for Autorregresive Integrated Moving Average. It is used for time series forecasting.
It contains three different components. The autoregressive the regression of the time series onto himself, the Integrated (I) component is to correct the non-stationarity of the data. The last component Moving Average (MA) models the errors based on past errors.
Each component receives different parameters AR(p), I(d), MA(q). To estimate the value of each parameter we need to get the Autocorrelation Function (ACF) and the Partial Autocorrelation Function (PACF)
Autocorrelation Function (ACF) and Partial Autocorrelation Function (PACF)
This functions will tell us how correlated are the observations in the time series.
Also, and the main purpose is that the will indicate us which are the best coefficient to use in our ARIMA model.
In our case the pattern in the PACF (significant correlations at the first or second lag followed by correlations that are not significant). The number of significant correlations in the PACF tells us the term of the autoregressive (AR) model. In our case, the coefficient is three.
However, based on the pattern of the ACF (the function progressively decrease), we cannot infer the term for the Moving Average (MA) so the best option is to use zero.
Applying ARIMA
This first plot shows the residuals of our data, and we can observe that most of the data is distributed around zero. Let’s see more in detail how this is distributed.
As we can see it has a similar shape as the Normal or Gaussian distributions (Symmetrical bell-shaped curve).
Get some predictions
Now it is time to get some predictions from the model, to evaluate how accurate our model is, we are going to use the test dataset we have made.Also, we need to perform a rolling forecast, that means after calculating each prediction add it and recalculate the model.
In the plot above we can see the result of our data in the test data, looking like pretty well fitted!
Evaluation
Calculate the Mean Squared Error to obtain the accuracy
MSE: 338.9
Now a model of neural networks is going to be applied, concretely we are going to use Recurrent Neural Networks
Recurrent Neural Networks (RNN)
RNN is a type of neural networks, which mainly are used in natural language processing (NLP) and predicting Time Series. The basic idea behind these type is that the state of each neuron is stored and fed to the next layer. This is why it is so used in NLP and Time series because it has in count the last observations to predict the next one.
There is a type of RNN called Long Short-Term Memory, it was proposed to solve a huge problem that all the RNN presented, this is called The Vanishing Gradient Problems but we are not going to enter in this kind of details. This is all we need to know to apply to our dataset.
This part is going to be much shorter than the previous one because basically to apply Neural Networks is easier than to apply ARIMA Model
For the sake of the example, we are going to work on the last four weeks.
Also, the datasets to train and test the network are going to be the same to correctly compare the two models.
To have better accuracy in our network we need to first normalize the values of our data.
Now our dataset is ranged between 0 and 1.
The Recurrent Neural Networks predict the new observation based on X lagged observations. For our case, let’s try with 168 lagged observations what means around one week to get a new prediction.
Now it’s time to create the architecture of the Neural Network! We are going to use Keras, which is a popular framework that is made on top of TensorFlow (the neural network library published by Google). Keras is so famous because it makes the task to build the Networks really easy.
Now it’s time to create the architecture of the Neural Network! We are going to use Keras, which is a popular framework that is made on top of TensorFlow (the neural network library published by Google). Keras is so famous because it makes the task to build the Networks really easy.
The architecture of the network
Our network is made by LSTM layers, it contains five layers with 100x200x300x200x100 neurons.I used dropout to improve accuracy and reduce the loss after epoch.In order to calculate the loss of the network, I used the same loss function as before, the Mean Squared Error (MSE) function.
regressor.fit(X_train, y_train, epochs = 100, batch_size = 32)
Now the model is fitted to our train dataset the model and it’s time to create the predictions.
Luckily, it looks that the predictions suit well to the real values, but the MSE tells a different thing.
MSE: 409.1
It looks like we have a winner, as we can see the result with the Neural Networks is worse than the one obtained by ARIMA!
During this practical case, we saw many different things starting by different concepts of statistical model and time series manipulations (different transformations, stationarity in time series, autocorrelation functions). Following by one statistical model (ARIMA) and Recurrent Neural Networks applied to our particular data set.
We can see that ARIMA performs better to our test set with a smaller Mean squared error of 338.96 compared to 409.1 provided by the neural networks.
There are still some other things to try but for this specific dataset with around 720 observations the price goes to the statistical model!
Future projects
Once this is understood the first thing that comes to mind is to apply the same methods to the whole dataset (with more than 41.000 observations. Here we will see which one behave better with a higher amount of data.
The next (still with the same dataset) is to use the other variables to create the forecasting. In the beginning, we have got rid of around 7 different variables with valuable information that we are not using in our model.
We can also, improve the architecture of the neural network until we reach an optimal point. There is another statistical model such as Seasonal Arima (SARIMA) that can maybe improve the metrics.
Finally and the purpose of all of this study is to extract useful insights from the data, in our case, it will be about the PM2.5 levels in Beijing.
... but let’s see all these things in new posts.