## 4 Empirical Study
### Results
\begin{table}
\begin{tabular}{l l c c c c c c c c c} \hline \hline
**Time Series** & **Model** & **RMSE** & **Rank** & **MAE** & **Rank** & **MAPE (\%)** & **Rank** & **MASE** & **Rank** & **L-B** \\ \hline Boots & Log ARIMA\((2,1,4)\times(0,0,1)_{12}\) & 736.37 & 1 & 500.67 & 1 & 90.33 & 1 & 0.60 & 1 & 0.63 \\  & ETS(M,A,A) & 1470.95 & 2 & 921.14 & 2 & 206.71 & 4 & 1.11 & 2 & 0.05 \\  & ETS Hyndman-Athanasopoulos (2013) framework & 1918.63 & 4 & 1156.32 & 4 & 138.49 & 2 & 1.39 & 4 & 0.10 \\  & ARIMA Hyndman-Khandakar (2008) algorithm & 2104.56 & 5 & 1668.49 & 5 & 978.05 & 5 & 2.00 & 5 & 0.80 \\  & Seasonal naive method & 1649.37 & 3 & 93.75 & 3 & 179.73 & 3 & 1.16 & 3 & 0.69 \\ \hline Boots & Log ARIMA\((3,1,3)\times(1,0,0)_{12}\) & 406.53 & 1 & 284.43 & 1 & 61.54 & 1 & 0.72 & 1 & 0.66 \\  & ETS(M,N,M) & 480.83 & 3 & 3196.66 & 2 & 73.25 & 4 & 0.81 & 2 & 0.10 \\  & ETS Hyndman-Athanasopoulos (2013) framework & 687.41 & 4 & 369.78 & 4 & 67.90 & 3 & 0.93 & 4 & 0.10 \\  & ARIMA Hyndman-Khandakar (2008) algorithm & 733.37 & 5 & 492.01 & 5 & 66.73 & 2 & 1.24 & 5 & 0.80 \\  & Seasonal naive method & 409.57 & 2 & 321.44 & 3 & 149.30 & 5 & 0.81 & 2 & 0 \\ \hline Flats & Log ARIMA\((3,0,2)\times(2,1,0)_{12}\) & 385.69 & 1 & 285.19 & 1 & 29.62 & 1 & 0.45 & 1 & 0.06 \\  & ETS(M,M,M) & 401.61 & 2 & 299.30 & 2 & 31.32 & 3 & 0.51 & 2 & 0.55 \\  & ETS Hyndman-Athanasopoulos (2013) framework & 450.06 & 3 & 320.84 & 3 & 31.17 & 2 & 0.55 & 3 & 0.84 \\  & ARIMA Hyndman-Khandakar (2008) algorithm & 544.25 & 4 & 341.52 & 4 & 32.94 & 4 & 0.58 & 4 & 1.00 \\  & Seasonal naive method & 570.80 & 5 & 563.44 & 5 & 4/6.62 & 5 & 0.92 & 5 & 0 \\ \hline Sandals & ARIMA\((1,2,2)\times(0,1,2)_{12}\) & 1021.89 & 1 & 629.08 & 1 & 2217.84 & 4 & 0.55 & 1 & 0.05 \\  & ETS(M,M,A,A) & 1594.30 & 3 & 1259.99 & 3 & 7798.43 & 5 & 1.09 & 3 & 0.03 \\  & ETS Hyndman-Athanasopoulos (2013) framework & 3005.38 & 4 & 1866.34 & 4 & 270.90 & 2 & 1.62 & 4 & 0.07 \\  & ARIMA Hyndman-Khandakar (2008) algorithm & 3251.93 & 5 & 1921.29 & 5 & 591.37 & 3 & 1.67 & 5 & 0.41 \\  & Seasonal naive method & 1287.04 & 2 & 670.00 & 2 & 101.78 & 1 & 0.58 & 2 & 0.44 \\ \hline Shoes & Log ARIMA\((0,1,0)\times(1,1,2)_{12}\) & 607.29 & 1 & 443.88 & 1 & 12.84 & 1 & 0.35 & 1 & 0.11 \\  & Log ETS(A,N,A) & 643.73 & 2 & 481.06 & 2 & 13.76 & 2 & 0.38 & 2 & 0.54 \\  & ETS Hyndman-Athanasopoulos (2013) framework & 975.60 & 4 & 735.49 & 4 & 21.85 & 4 & 0.59 & 4 & 0.11 \\  & ARIMA Hyndman-Khandakar (2008) algorithm & 943.62 & 3 & 657.03 & 3 & 18.66 & 3 & 0.52 & 3 & 0.04 \\  & Seasonal naive method & 1261.16 & 5 & 1096.5 & 5 & 34.79 & 5 & 0.88 & 5 & 0 \\ \hline \hline \end{tabular} \({}^{1}\)\(p\)-value \(<\) 0.01.  
\end{table} 
Table 1: Forecast accuracy measures of all forecasting approaches computed using time series cross-validation. by our cross-validation procedure have the capability to forecast the trend movement and seasonal fluctuations fairly well. As expected, the exceptional increase in the sales of flats observed in March and April 2012 was not predicted by the models that under-forecasted the situation. As mentioned in Section 2.3, one of the limitations of the MAPE is having huge values when data may contain very small numbers. The large values of MAPE for the Boots, Booties and Sandals time series are explained by this fact, since, during some periods, there are almost no sales (close to zero). Figure 3: One-step forecasts from all approaches computed using time series cross-validation (January 2011 to April 2012): (**a**) pairs of Boots; (**b**) pairs of Booties; (**c**) pairs of Flats; (**d**) pairs of Sandals; and (**e**) pairs of Shoes.