# Forecasting Seasonal Sales with Many Drivers: Shrinkage or Dimensionality Reduction?
## 3 Methods
### Dynamic Regression with Shrinkage  
A relatively simpler approach than ARIMA is to use a dynamic regression, where the MA part is omitted, with level and seasonal autoregressive lags used to predict the target variable. The challenge in dynamic regression is the identification of the appropriate lags. The literature is rife with alternatives, many of which are discussed in [57], Chapter 9. In specifying the appropriate number of lags there are two issues. First, a question of how many lags to consider, and second out of these lags, which are relevant. We note that the first question is identical for ARIMA models, where one has to decide the maximum order for the AR and the seasonal AR terms. To resolve this the literature relies on heuristics, e.g., by consulting the partial autocorrelation function [33]. Given the sample implications of increasing the autoregressive lags, the general recommendation is to use a relatively small number of lags e.g., [59].  
Irrespective of how this is resolved, chances are that we will have to resolve any estimation with limited sample size, as seasonality is a prominent feature of retailing time series. Accounting for this, and considering the eventual extension of the model to include additional explanatory variables, we use Ridge regression, that shrinks all model coefficients towards zero, mitigating overfitting due to both sampling uncertainty and potentially superfluous input variables [50]. Ridge regression optimises the model parameters \(b_{j}\) for \(j=1,\ldots,p\) by minimising:  
\[\sum_{i=1}^{n}\left(y_{i}-\hat{y}_{i}\right)^{2}+\lambda\sum_{j=1}^{p}b_{j}^{2}, \tag{3}\]  
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & & \multicolumn{3}{c}{**Seasonal Component**} \\  & & \multicolumn{2}{c}{**N**} & \multicolumn{1}{c}{**A**} \\ \cline{3-4}  & **N** & \(y_{t}=l_{t-1}+\sum_{j=1}^{l}\delta_{j}x_{j,t}+\varepsilon_{t}\) & \(y_{t}=l_{t-1}+s_{t-m}+\sum_{j=1}^{l}\delta_{j}x_{j,t}+\  
where \(n\) is the sample size, \(\hat{y}_{i}\) is the predicted value for period \(i\), and \(\lambda\) is a non-negative scalar that controls the strength of the shrinkage of the coefficients. When \(\lambda=0\) this becomes the ordinary least squares regression. Otherwise, the model is no longer optimal in the mean squared error sense, but rather retains smaller coefficients than Ordinary Least Squares (OLS), achieving the benefits of shrinkage. The correct value of \(\lambda\) is data dependent and is typically identified using cross-validation.  
Another popular shrinkage estimator is the Lasso, which uses \(\lambda\sum_{j=1}^{p}|b_{j}|\) as the regularisation penalty. In contrast to Ridge regression, due to its penalty, Lasso regression is able to set model coefficients zero, performing both estimation and variable selection in a single step. Ridge, on the other hand, can result in very small coefficients, but typically non-zero. Lasso has been used successfully in retail forecasting e.g., [15; 37; 47]. However, Ridge is preferable to Lasso when the inputs are likely to be highly correlated, as Lasso will retain only the most highly correlated variable to the target and reject the rest. In the applications in the literature, Lasso was used to decide between different indicator variables, and as such this was not an acute problem. However, in our case as we use it to estimate the coefficients of autoregressive lags that will very likely be correlated, the Ridge penalty is preferable.  
When incorporating seasonality with trigonometric encoding, the expectation is that all terms of \(s_{l}^{\phi}\) will remain in the model, however with shrunk amplitudes, which has been found to be beneficial in terms of forecast accuracy [63; 64]. Similarly, as Ridge regression uses shrinkage, there is no need to use dimensionality reduction, as is the case for ARIMA and ES.