Time series forecasting: from naive to ARIMA and beyond | by Mahbubul Alam | Towards Data Science

Building another power plant in the next five years requires forecasts of future demand;
Scheduling staff in a call center next week requires forecasts of next weeks’ call volumes;
Stocking an inventory of retails requires forecasts of stock requirements.
Indeed, short and medium term forecasting is an essential part of business decisions across industries. And historical data is an essential input into this forecasting process.
Time series datasets are the most widely generated and used kind of data in any business. They are used both in understanding the past and predicting the future. In this article, I’ll discuss how various forecasting methods are applied to time series datasets with relevant case studies and examples. The purpose of this post is not to evaluate which model is good or bad, rather to demonstrate the many different ways we can do forecasting in practice. The article is organized in 2 broad sections:
(1) First, I will provide an overview of time series data and how to decompose difference time series components;
(2) Then I will provide examples of different forecasting techniques with associated implementation method
So what is a time series data? A time series is a sequence of observations recorded in discrete time points. They can be recorded in hourly (e.g. air temperature), daily (e.g. DJI Average), monthly (e.g. sales) or yearly (e.g. GDP) time intervals. Time series are applied in a wide range of cases in all disciplines including statistics, mathematics, astronomy and engineering. Time series plotting is one of the most basic plotting exercises implemented during the exploratory data analysis (EDA) phase in data science and analytics.
Let’s take a look at the following figure.
Few properties are obvious even in naked eyes:
it’s a time series plotthere is an upward trendthe observations have seasonality (ups and downs in a regular interval)
it’s a time series plot
there is an upward trend
the observations have seasonality (ups and downs in a regular interval)
In fact, if we decompose this data we will see these components like below.
Forecasting into the future can be as simple as extrapolating the trend of historical observations, to applying complex algorithms. Here I present different techniques most commonly used in business applications. I am demonstrating these techniques in R environment using two statistical and visualization packages:
# install packageslibrary("forecast")library("ggplot2")
After importing data ( a retail sales dataset) and converting it into a time series object, here is how it looks like.
# import datasetretaildata = readxl::read_excel("../retail.xlsx", skip=1)# convert dataset into a time series objectretailts = ts(retaildata, start = c(1982,4), frequency = 12 )# plot the time series object with autoplot() function that comes with forecast() packageautoplot(retailts)
First I’ll explain some simple techniques to forecast this time series object. Although these simple techniques are not often used in practice, but it’s good to know how they came about, then we will go into more complicated techniques.
Simple forecasting techniques are used as benchmarks. They provide a general understanding of historical data and to build intuition upon which to add additional layers complexity. Several such techniques are common in literature such as: mean model, naive forecast, random walk, drift method etc. A mean model, for example, takes the mean of previous observations and uses that for forecasting. That is:
forecast value = average of training data
A random walk, on the other hand, would predict the next value, Ŷ(t), that equals to the previous value plus a constant change.
Ŷ(t) = Y(t-1) + α
The value of naive forecast is set based upon the value of last observation. And like other simple methods, it provides a ballpark number as an initial estimate until further research is done. Below is an implementation of a seasonal naive forecast model.
# subsetting train and test data using window()function retailtstrain = window(retailts, end = c(2010,3))retailtstest = window(retailts, start = c(2010,4))# model buildingretailtstrain_snaive = snaive(retailtstrain, h=50)# plotingautoplot(retailts)+autolayer(retailtstrain_snaive, series = "S Naive", PI = FALSE)
# Accuracy testaccuracy(retailtstrain_snaive, retailtstest)
Exponential smoothing has few variants depending on the data type. A simple exponential smoothing is used for non-seasonal data without a clear trend, whereas Holt-Winter method is for data with trend and seasonality. It is applied in a stationary time series, where smoothing is controlled by parameter alpha (0~1), where lower values mean lower weights placed on recent observations. Below is an implementation of Holt-Winter Exponential Smoothing using the same dataset.
# Holt Winter Exponential Smoothing# Two variations: additive for roughly constant seasonal variation, otherwise multiplicative methoddata = window(retailts, start=2000)data_add = hw(data, seasonal="additive")data_mult = hw(data, seasonal="multiplicative")autoplot(data, series = "original data")+autolayer(data_add, series = "additive", PI=FALSE)+autolayer(data_mult, series = "multiplicative", PI=FALSE)
# model summary and performancedata_add[["model"]]
Auto Regressive Integrated Moving Average (ARIMA) is arguably the most popular and widely used statistical forecasting technique. As the name suggests, this family of techniques has 3 components: a) an “autoregression” component that models the relationship between the series and it’s lagged observations; b) a “moving average” model that models the forecast as a function of lagged forecast errors; and c) an “integrated” component that makes the series stationary.
The model takes in the following parameter values:
p that defines the number of lags;
d that specifies the number of differences used; and
q that defines the size of moving average window
An implementation of ARIMA model to make forecasts using the time series object is as follows:
# implementing auto.arima() to forecastretailts_arima = auto.arima(retailts, seasonal=TRUE, stepwise = FALSE, approximation = FALSE)retailts_arima %>% forecast(h=10) %>% autoplot()
Finally, there are some outside box techniques such as agent-based and system dynamic modeling. Developed at MIT’s Sloan School of Management in the 1950s, system dynamics is a methodological approach to model the behavior of complex systems, where change in one component leads to change in others. This approach is widely applied in industries such as healthcare, disease research, public transportation, business management and revenue forecasting. The most famous application of system dynamics probably is in the modeling Limits to Growth by the Club of Rome.
A system dynamic model represents a complex system in terms of stocks & flows and their interactions via feedback loops to predict the behavior of the system. Let’s say a bank account has a “Stock” of $100. Every month $20 is deposited (represented by “Flow” 1) and an amount of $15/month is withdrawn (represented by “Flow” 2):
In this example, future value of the stock (i.e. account deposit) is modeled as s function of future flows (i.e. deposits and withdrawals). This paper provides a case study of implementing a system dynamic model for forecasting in the construction industry.
Rob Hyndman’s website https://robjhyndman.com/ is an one-stop solution to all forecasting problems in R. Also check out the open access book https://otexts.com/fpp2/ that has theoretical discussion as well as the R implementation with examples.“A Little Book of R for Time Series” has everything in simplified forms. http://www.calvin.edu/~stob/courses/m344/S15/a-little-book-of-r-for-time-series.pdfIf you are into Python then check out the link below. There is also a reference to a book on python implementations of time series. https://machinelearningmastery.com/time-series-forecasting-methods-in-python-cheat-sheet/
Rob Hyndman’s website https://robjhyndman.com/ is an one-stop solution to all forecasting problems in R. Also check out the open access book https://otexts.com/fpp2/ that has theoretical discussion as well as the R implementation with examples.
“A Little Book of R for Time Series” has everything in simplified forms. http://www.calvin.edu/~stob/courses/m344/S15/a-little-book-of-r-for-time-series.pdf
If you are into Python then check out the link below. There is also a reference to a book on python implementations of time series. https://machinelearningmastery.com/time-series-forecasting-methods-in-python-cheat-sheet/