The best Forecast Techniques or how to Predict from Time Series Data
By: Edwin Lisowski
Tags: Machine Learning, Predictive Analytics, Time Series Analysis, Forecasting, Predictions

Time Series Forecasting finds a lot of applications in many branches of industry or business. It allows to predict product demand (thus optimizing production and warehouse storage), forecast amount of money from sales (adjusting company’s expenses) or to predict future values of stock prices. In this article I will try present basic approaches to achieve this goal. We will start with description of most popular models and then move to the model evaluation, which indicates the best method for given forecast problem.

Tools

There is a great variety of choices of software that can be used for time series forecasting — it can be comfortably done with Python, R, Matlab, and many other languages. In R, which we use here, there are also quite a few packages that make forecasting simplier, forecast, prophet, or xts, just to mention a few of them. The complete list of packages together with their descriptions can be found on TimeSeries CRAN Task View.

Time Series Models

Before moving to sophisticated time series models we will discuss some simple models that constitute a baseline for further predictions. If a complicated method does not provide better results, then it makes no sense to use them.

Mean — the forecasts are equal to the mean value of the time series. Naive — the forecasts are equal to the last value of the time series. Seasonal naive — the forecasts for given season are equal to the value of that season a full period before, e.g. predictions for January 2019 are equal to the value of time series in January 2018.

Fig. Baseline forecasts of monthly airlines passengers.

Time Series Linear Model (TSLM) is just a linear regression model that predicts requested value based on some predictors, most often linear trend and seasonality:

yt =a1x1,t+a2x2,t+…+anxn,t+b+et,

where xi,t are some predictors, ai and b are regression coefficients to estimate. The two most often used predictors are trend and seasonality. The former simply models the linear trend in data — the model with only trend predictor can be written as:

yt = at +b+et.

Seasonality predictors are dummy variables indicating the period (e.g. month, quarter) for which the forecasts are made.

Exponential smoothing is another useful method for forecasting time series. The basic idea is to predict future values of time series as weighted average of past observations, where weights decrease exponentially with time — the older observation the less influence it has on predictions. The method can be extended to include trend (Holt’s method aka double exponential smoothing) or trend and seasonality (Holt-Winters’ method). In these models the trend and seasonality coefficients are also computed as the weighted average of some expressions. Exponential smoothing is useful method for forecasting time series. The basic idea is to predict future values of time series as weighted average of past observations, where weights decrease exponentially with time:

yt=a yt-1+a(1-a) yt-2+a(1-a)2 yt-3+…,

where a(0,1)is smoothing parameter which should be estimated. The method presented above can be extended to include trend (Holt’s method aka double exponential smoothing) or trend and seasonality (Holt-Winters’ method aka triple exponential smoothing). In these models the trend and seasonality coefficients are also computed as the weighted average of some expressions. Even though the idea is quite simple, the formulas are more complex and we will not go into technical details. Interested reader may consult Hyndman and Athanasopoulos’ book on time series forecasting [1].

ARIMA (autoregressive integrated moving average) model uses completely different approach. The autoregressive (AR) part takes into account the influence of the previous values on the predicted one:

yt=a1yt-1+a2yt-2+et,

whereas the moving average (MA) models the influence of noise on the future values.

yt=et+b1et-1+b2et-2,

where et are random noise terms. In above example each noise term affects three consecutive yt values. In reality such noise terms may reflect some unexpected events e.g. politicians’ statements affecting stock markets.

In reality such noise terms may reflect some unexpected events e.g. politicians’ statements affecting stock markets.

AR and MA models can be combined into ARMA model in a simple way as shown below:

yt=et+b1et-1+b2et-2+a1yt-1+a2yt-2.

The integrated part of ARIMA corresponds to technical issue about how ARMA models are fitted. Basically, it makes sense only to fit them to stationary time series. One way to stationarize nonstationary data is to subtract from it its lagged values (difference it). That’s what ARIMA does before fitting ARMA — it differences the data some number of times until it get sufficiently stationary data.

Fig. Forecasts of monthly airlines passengers.

Evaluation

Model selection is a problem involved in almost every machine learning project. One way to do this, is to split the data into train and test sets and compute some error metric on the test set after learning model on the train. This procedure is, however, different from random train-test split often used in machine learning. Having time series data, one cannot simply pick random points — instead we put all observations before some split point to train set and the rest to test set. This reflects how our models are used in reality — we learn on historical data and then forecast future.

Another approach is to use time series cross validation (aka rolling forecast origin procedure) — we create many train-test splits with different split points, and then evaluate the model on each split.

Natural question that arises is what metric one should use to tell how well the model in question performs. The most popular are

Mean absolute error, which tells how much on average our forecasts differ from true values,

Mean absolute percentage error, which tells how much % predictions differ from true values.

Conclusions

We have explained basics of time series forecasting. Obviously, each problem has its own subtleties and demands special steps (e.g. proper data preparation, way of handling missing values, or defining evaluation metric satisfying some business conditions) and thus it is impossible to come up with a general approach. We have just presented the core ideas behind forecasting.

[1] Hyndman, R. J., & Athanasopoulos, G. (2018). Forecasting: principles and practice. OTexts.