# Short-Term Wind Speed Forecasting Using Statistical and Machine Learning Methods
## 2 Models
### The Least Absolute Shrinkage and Selection Operator, (Lasso) for Variable Selection  
Selecting relevant features from a dataset needed for the forecasting task is the process of variable selection. Not all variables are necessary predictors; among such predictors, there is a need for selecting the best predictor variables. There are many methods for variable selection. We present the use of least absolute shrinkage and selection operator (Lasso) for variable selection in this study [44]. Suppose we have an \(N\) pair of predictor variables \(x\) and response variables \(y\), i.e., \(\{x_{i},y_{i}\}_{i=1}^{N}\). The aim is to give an approximate value for the response variable \(y_{i}\) from the predictors linearly combined, as it is in linear regression such as given in Equation (29)  
\[\eta(x_{i})=\beta_{0}+\sum_{j=1}^{p}x_{i_{j}}\beta_{j}. \tag{29}\]  
The vector \(\beta=(\beta_{0},\beta_{1},\cdots,\beta_{p}\in\mathcal{R}^{p}\) of regression weights parameterises the model along with an intercept term \(\beta_{0}\in\mathcal{R}\). An estimation for \((\beta)\) using least-squares method is based on minimising squared error loss using Equation (30)  
\[\hat{\beta}=\min_{\beta}\big{\{}\frac{1}{2N}\sum_{i=1}^{N}\big{(}y_{i}-\beta_{ 0}-\sum_{j=1}^{p}x_{ij}\beta_{j}\big{)}^{2}\big{\}}. \tag{30}\]  
Due to the problems of interpretability and prediction accuracy associated with least squares method, the need for the Lasso thus emerged [45]. Lasso performs better in prediction accuracy measured in terms of the mean squared error by shrinking regression coefficient values or setting some of them to zero thereby introduces bias and reduces the variance of predicted values. Lasso also helps with an improved interpretability by identifying smaller subsets of predictors with stronger effect from a large set of predictors. Lasso provides an automatic way for variable selection in linear regression problems because it solves a convex, quadratic program with convex constraint optimisation problem. It works by combining the least square loss of Equation (30) and the \(\ell_{1}-\)_constraint_ or bounded on sum of the absolute values of the coefficient [45]. The Lasso solves \(\hat{\beta}\) using the optimisation problem of Equation (30) subject to \(\sum_{j=1}^{p}|\beta_{j}|\leq t\) which is the \(\ell_{1}-norm\) constraint written as \(||\beta||_{1}\leq t\). A comparable method before the Lasso is the ridge regression which solves Equation (30) subject to \(\sum_{j=1}^{p}\beta_{j}^{2}\leq t^{2}\). The best form for estimating the Lasso problem is by having the predictors \(x_{i}\) standardised, making all the columns centred such that \((\frac{1}{N}\sum_{i=1}^{N}x_{ij}=0)\) with unit variance \((\frac{i}{N}\sum_{i=1}^{N}x_{ij}^{2}=1)\) and the response values \(y_{i}\) centred such that \((\frac{1}{N}\sum_{i=1}^{N}y_{i}=0)\) with an omitted intercept term \(\beta_{0}\) produces the optimal solution in which Equation (30) becomes Equation (31)  
\[\min_{\beta\in\mathcal{R}^{p}}\big{\{}\frac{1}{2N}\sum_{i=1}^{N}\big{(}y_{i}- \sum_{j=1}^{p}x_{ij}\beta_{j}\big{)}^{2}+\lambda\sum_{j=1}^{p}|\beta_{j}| \big{\}}. \tag{31}\]  
Equation (31) is known as the Lagrangian form which produces an effective, convenient and simple computational algorithm for the numerical computation of Lasso using coordinate descent procedure among other methods [45]. The complexity of Lasso is controlled by the value of its constraint, \(t\). Smaller values of \(t\) produce sparse and easily interpretable models less closely fitted to the training data, whereas larger values of \(t\) free up more parameters, more closely fitted to the data. These two extremes of \(t\) hamper the generalisation ability of the Lasso model by recording a large error value in the prediction error test set. A trade off between over-fitting and sparsity is desirable for the Lasso generalisation ability. This is carried out by the cross-validation procedure which strikes a balance in the value of \(t\) that gives the accurate model for predicting individual test dataset.