# Forecasting Seasonal Sales with Many Drivers: Shrinkage or Dimensionality Reduction?
## 4 Empirical Study
### Results
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Model** & \(\mathbf{H=1}\) & **1–4** & **5–8** & **9–13** & **1–13** \\ \hline MASE & & & & & \\ RidgeX & 0.793 & **0.822** & **0.881** & 0.932 & **0.883** \\ PCRegARIMA & 0.795 & 0.833 & 0.883 & **0.929** & 0.886 \\ PCRegARIMA & 0.804 & 0.834 & 0.894 & 0.942 & 0.894 \\ Ridge FX & **0.792** & 0.827 & 0.898 & 0.960 & 0.900 \\ ESXPC & 0.797 & 0.834 & 0.904 & 0.966 & 0.906 \\ \hline Ridge & 0.894 & 0.924 & 0.982 & 1.035 & 0.985 \\ ARIMA & 0.908 & 0.936 & 0.992 & 1.040 & 0.993 \\ RegARIMAF & 0.900 & 0.933 & 0.993 & 1.044 & 0.994 \\ TBATS & 0.906 & 0.940 & 0.998 & 1.048 & 0.999 \\ RidgeF & 0.896 & 0.933 & 1.009 & 1.079 & 1.012 \\ ES & 0.904 & 0.944 & 1.018 & 1.081 & 1.019 \\ \hline RMSSE & & & & & \\ RidgeX & 0.767 & 0.792 & 0.825 & 0.852 & 0.839 \\ PCRegARIMA & 0.767 & 0.820 & 0.830 & **0.850** & 0.854 \\ PCRegARIMA & 0.770 & 0.799 & 0.836 & 0.856 & 0.847 \\ Ridge FX & **0.762** & **0.788** & **0.822** & 0.854 & **0.837** \\ ESXPC & 0.764 & 0.804 & 0.844 & 0.872 & 0.858 \\ \hline Ridge & 0.883 & 0.910 & 0.941 & 0.966 & 0.955 \\ ARIMA & 0.895 & 0.922 & 0.957 & 0.980 & 0.970 \\ RegARIMAF & 0.886 & 0.915 & 0.947 & 0.969 & 0.960 \\ TBATS & 0.888 & 0.918 & 0.950 & 0.976 & 0.964 \\ RidgeF & 0.876 & 0.905 & 0.945 & 0.981 & 0.961 \\ ES & 0.894 & 0.929 & 0.975 & 1.006 & 0.988 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Forecasting performance of the models for all forecast horizons (22 forecast origins). almost all of the reported differences there is evidence that they are statistically significant. The results for the other forecast horizons differ slightly, but the key conclusions remain the same, with evidence of significant differences between most cases. We return to the relative performance of PCA and shrinkage estimators when we analyse the results for promotional and non-promotional periods separately. Figure 4 provides the ranking of the models for the various forecast horizons separately. This provides some interesting insights in terms of the progression of the forecasting performance. We observe that in all cases the models with the covariates rank better than the univariate benchmarks. Notably, PCRegARIMAF performs better for longer-term forecasts, for both MASE and RMSSE. So far we have observed a difference in the performance of RidgeX and RidgeFX. This difference becomes clearer when we track the ranking across horizons. In both cases, RidgeFX performs better for short horizons. This is in agreement with the summary statistics in Table 4. The same behaviour is observed for the univariate counterparts, Ridge and RidgeF. This is in contrast to the results for ARIMA in terms of using trigonometric seasonality or not, and arguably the same can be said for the performance between ES and TBATS, with the latter performing better for longer horizons. Figure 3: MCB test results at a 5% significance level based on MASE (**top**) and on RMSSE (**bottom**). Next, we examine the forecast accuracy for promotional and non-promotional periods separately. Table 5 presents the MASE and RMSSE for the horizon 1-13, with the first two columns referring to the promotional case, and the latter two to the non-promotional. At each column the best performing forecast is highlighted in boldface. The table is supplemented by Figure 5 that provides the Nemenyi test results. The striking difference is that when we focus on the promotional periods the PCRegARIMA and PCRegARIMAF perform very well, and in the case of MASE the former significantly outperforms all RidgeFX and RidgeX. Notably, the performance of ESXPC that relies on PCA to encode the coviariates becomes more competitive as well. The opposite is true for the non-promotional periods, which align better with the discussion so far, with RidgeX significantly outperforming all alternatives. Building on this, we argue that shrinkage estimators deal best with capturing the overall structure of the time series, but on the other hand can over-shrink the effect of promotions.