Basic Principles to Create a Time Series Forecast
By: Leandro Rabelo
Tags: Data Science, Time Series Forecasting, Time Series Analysis

Below is an interactive chart where you can change the lambda value and check the change in the chart:

This tool is usually used to improve the performance of the model, since it makes it with more normal distributions, remembering that after finishing the prediction of the model, you must return to the original base inverting the transformation according to the formula below:

Looking for correlated lags

To be predictable, a series with a single variable must have auto correlation, that is, the current period must be explained based on an earlier period (a lag).

As this series has weekly periods, 1 year is approximately 52 weeks, I will use the auto correlation function showing a period of 60 lags to verify correlations of the current period with these lags.

Analyzing the above auto correlation chart above, it seems that all lags could be used to create forecasts for future events since they have a positive correlation close to 1 and they are also outside of the confidence interval, but this characteristic is of a non-stationary series.

Another very important function is the partial auto correlation function, where the effect of previous lags on the current period is removed and only the effect of the lag analyzed over the current period remains, for instance: the partial auto correlation of the fourth lag will remove the effects of the first, second and third lags.

Below the partial auto correlation graph:

As can be seen, almost no lag has an effect on the current period, but as demonstrated earlier, the series without differentiation is not stationary, we will now plot these two functions with the series with one differentiation to see how it works:

The auto correlation plot changed significantly, showing that the series has a significant correlation only in the first lag and a seasonal effect with negative correlation around the 26th month (half a year).

To create forecasts, we must pay attention to an extremely important detail about finding correlated lags, it’s important that there is a reason behind this correlation, because if there is no logical reason it’s possible that it’s only chance and that this correlation can disappear when you include more data.

Another important point is that the auto correlation and partial auto correlation graphs are very sensitive to outliers, so it’s important to analyze the time series itself and compare with the two auto correlation charts.

In this example the first lag has a high correlation with the current period, since the prices of the previous week historically do not vary significantly, in the same case the 26th lag presents a negative correlation, indicating a tendency contrary to the current period, probably due to the different periods of supply and demand over the course of a year.

As the inflation-adjusted series has become stationary, we will use it to create our forecasts, below the auto correlation and partial auto correlation graphs of the adjusted series:

We will use only the first two lags as a predictor for auto-regressive series.

For more information, Duke University professor Robert Nau’s website is one of the best related to this subject.

Metrics to evaluate the model

In order to analyze if the forecasts are with the values close to the current values one must make the measurement of the error, the error (or residuals) in this case is basically Yreal−YpredYreal−Ypred.

The error in the training data is evaluated to verify if the model has good assertiveness, and validates the model by checking the error in the test data (data that was not “seen” by the model).

Checking the error is very important to verify if your model is overfitting or underfitting when you compare the training data with the test data.

Below are the key metrics used to evaluate time series models:

MEAN FORECAST ERROR — (BIAS)

It’s nothing more than the average of the errors of the evaluated series, the values can be positive or negative. This metric suggests that the model tends to make predictions above the real value (negative errors) or below the real value (positive errors), so it can also be said that the mean forecast error is the bias of the model.

MAE — MEAN ABSOLUTE ERROR

This metric is very similar to the average error of the prediction mentioned above, the only difference is the error with a negative value that is transformed into positive and afterward the mean is calculated.

This metric is widely used in time series, since there are cases that the negative error can cancel the positive error and give an idea that the model is accurate, in the case of the MAE it doesn’t happen, because this metric shows how much the forecast is far from the real values, regardless if above or below, see the case below:

Result: The error of each model value looks like this: [-4 -2 0 2 4]

The MFE error was 0.0, the MAE error was 2.4

MSE — MEAN SQUARED ERROR

This metric places more weight on larger errors because each individual error value is squared and then the mean is calculated. Thus, this metric is very sensitive to outliers and puts a lot of weight on predictions with more significant errors.

Unlike the MAE and MFE, the MSE values are in quadratic units rather than the units of the model.

RMSE — ROOT MEAN SQUARED ERROR

This metric is simply the square root of the MSE, where the error returns to the unit of measure of the model (BRL/m3), it is very used in time series because it’s more sensitive to the bigger errors due to the process of squaring which originated it.

MAPE — MEAN ABSOLUTE PERCENTAGE ERROR

This is another interesting metric to use, which generally is used in management reports because the error is measured in percentage terms, so the error of a product X can be compared with the error of a product Y.

The calculation of this metric takes the absolute value of the error divided by the current price, then the mean is calculated:

Let’s create a function to evaluate the errors of training and test data with several evaluation metrics:

Checking the residual values

It’s not enough to create the model and check the error values according to the chosen metric, you must also analyze the characteristics of the residual itself, as there are cases where the model can not capture the information necessary to make a good forecast, resulting in an error with information that should be used to improve the forecast.

To verify this residual we will check:

Current vs. predicted values (sequential chart) ;

; Residual vs. predicted values (dispersion chart):

It is very important to analyze this graph since in it we can check patterns that can tell us if some modification is needed in the model, the ideal is that the error is distributed linearly along the forecast sequence.

QQ plot of the residual (dispersion chart):

Summarizing this is a graph that shows where the residue should be theoretically distributed, following a Gaussian distribution, versus how it actually is.

Residual auto correlation (sequential chart):

Where there should be no values that come out of the confidence margin or the model is leaving information out of the model.

We need to create another function to plot these graphs:

Most basic ways to make a forecast

From now on we will create some models of price forecast of Hydrous ethanol, below will be the steps that we will follow for each model:

Create prediction on the training data and subsequently validate on the test data;

Check the error of each model according to the metrics mentioned above;

Plot the model with the residual comparatives.

Let’s go to the models:

Naive approach:

The simplest way to make a forecast is to use the value of the previous period, this is the best approach that can be done in some cases, where the error is lower compared to other forecast methodologies.

Generally, this methodology doesn’t work well to predict many periods ahead, as the errors tend to increase in relation to real values.

Many people also use this approach as a baseline to try to improve with more complex models.

Below we will use the training and test data to make the simulations:

The QQ chart shows that there are some larger (up and down) residuals than theoretically should be, these are the so-called outliers, and there is still a significant auto correlation in the first, sixth and seventh lag, which could be used to improve the model.

In the same way, we will now make the forecast in the test data. The first value of the predicted series will be the last of the training data, then these values will be updated step-by-step by the current value of the test and so on:

The RMSE and MAE errors were similar to the training data, the QQ chart is with the residual more in line with what should theoretically be, probably due to the few sample values compared to the training data.

In the chart comparing the residuals with the predicted values it’s noted that there is a tendency for the errors to increase in absolute values when prices increase, perhaps a logarithmic adjustment would decrease this error expansion, and to finalize the residual correlation graph shows that there is still room for improvement as there is a strong correlation in the first lag, where a regression based on the first lag could probably be added to improve predictions. Next model is the simple average:

Simple Mean:

Another way to make predictions is to use the series mean, usually this form of forecasting is good when the values oscillate close around the mean, with constant variance and no uptrend or downtrend, but it’s possible to use better methods, where can make the forecast using seasonal patterns among others.

This model uses the mean of the beginning of the data until the previous period analyzed and it expands daily until the end of the data, in the end, the tendency is that the line is straight, we will now compare the error of this model with the first model:

In the testing data, I will continue using the mean from the beginning of the training data and make the expansion of the mean with the values that will be added on the test data:

The simple mean model failed to capture relevant information of the series, as can be seen in the Real vs Forecast graph, also in the correlation and Residual vs. Predicted graphs.

Simple Moving Average:

The moving average is an average that is calculated for a given period (5 days for example) and is moving and always being calculated using this particular period, in which case we will always be using the average for the last 5 days to predict the value of the next day.

The error was lower than the simple average, but still above the simple model, below the test model:

Similarly to the training data, the moving-averages model is better than the simple average, but they do not yet gain from the simple model.

The predictions are with auto-correlation in two lags and the error is with a very high variance in relation to the predicted values.

Exponential Moving Average:

The simple moving average model described above has the property of treating the last X observations equally and completely ignoring all previous observations. Intuitively, past data should be discounted more gradually, for example, the most recent observation should theoretically be slightly more important than the second most recent, and the second most recent should have a little more importance than the third more recent, and so on, the Exponential Moving Average (EMM) model does this.

Since α (alpha) is a constant with a value between 0 and 1, we will calculate the forecast with the following formula:

Where the first value of the forecast is the respective current value, the other values will be updated by α times the difference between the actual value and the forecast of the previous period. When α is zero we have a constant based on the first value of the forecast, when α is 1 we have a model with a simple approach because the result is the value of the previous real period.

Below is a graph chart several values of α: