## 4 Empirical Study
### Results  
Figure 2 shows the sample ACF and the sample PACF for the five time series. It can be seen that the sample ACFs decay very slowly at regular lags and at multiples of the seasonal period 12 and the sample PACFs have a large spike at lag 1 and cut off to zero after lag 2 or 3 suggesting, possibly, seasonal and regular differencing. A logarithmic transformation might be necessary to stabilize the variance of some times series. In each case, the minimum size of the training set was specified from January 2007 to December 2010 (first 48 observations) while the test set was specified from January 2011 to April 2012 (last 16 observations). The state space model and the ARIMA model with the best performance in the cross-validation procedure were selected as the final models. Figure 1: Monthly sales of the five footwear categories between January 2007 and April 2012: (**a**) pairs of Boots; (**b**) pairs of Boots; (**c**) pairs of Flats; (**d**) pairs of Sandals; and (**e**) pairs of Shoes. To be able to compare the accuracies of the forecasts obtained with the ETS and ARIMA models selected by the cross-validation procedure developed, we also evaluated the forecasts from another three forecasting approaches: the Hyndman and Khandakar [23] algorithm which identifies and estimates ARIMA models, the Hyndman and Athanasopoulos [16] statistical framework which identifies and estimates state space models and the seasonal naive method which was used as benchmark, despite simple forecasting methods being sometimes surprisingly effective. The Hyndman and Khandakar [23] algorithm identifies an ARIMA model for a time series using the following procedure: first, the user must decide if a Box-Cox transformation should be used to stabilize the variance; then, for non-seasonal data, \(d\) is selected based on successive KPSS (Kwiatkowski-Phillips-Schmidt-Shin) unit-root tests [24] and, for seasonal data, \(D\) is set to 1 and then \(d\) is chosen by applying successive KPSS unit root tests to the seasonally differenced data; once \(d\) and  
Figure 2: Sample autocorrelation function (**left** panels) and sample partial autocorrelation function (**right** panels) plots for the five footwear categories: Boots (**a1**,**a2**); Booties (**b1**,**b2**); Flats (**c1**,**c2**); Sandals (**d1**,**d2**); and Shoes (**e1**,**e2**). are known, the orders of \(p\), \(q\), \(P\) and \(Q\) are selected by minimising the Akaike's Information Criterion (AIC) using a step-wise algorithm: to start, four basic models are considered and the one with the smallest AIC value is selected; then, thirteen variations on this model are considered allowing \(p\), \(q\), \(P\) and \(Q\) to vary simultaneously by \(\pm 1\). Whenever a model with a lower AIC value is found, it becomes the new \"current\" model, and this procedure is repeated. The process finishes when a model close to the current model with a lower AIC cannot be found. The [PERSON] (1996) statistical framework identifies an ETS model for a time series by selecting the one with the lowest AIC value among all possible ETS models. In the seasonal naive method, each forecast is equal to the last observed value from the same season of the year (for monthly data, it is the same month of the previous year). Table 1 gives, for each time series, the forecast accuracy measures and the validation performance of each approach. The forecast error measures presented in Table 1 are defined in Section 2.3. The analysis was carried out using the statistical software R programming language (version R-3.3.2 for windows) and the specialized package forecast ([PERSON], 2006; [PERSON], 2007). To be able to compare the forecasting performance of our procedure with the forecasting performance of the other three forecasting approaches, a time series cross-validation based on one-step forecasts was applied to them.