# Linking Singular Spectrum Analysis and Machine Learning for Monthly Rainfall Forecasting
## 3 Methodologies  
### Least-Squares Support Vector Machine  
The least-squares support vector machine (LS-SVM), a new type of SVM, comprises a sequence of similar supervised learning techniques that analyze data and identify patterns. Based on the constraints of equality rather than inequality, the LS-SVM technique is employed for classification and regression [72]. Instead of resolving the convex quadratic programming problem, LS-SVM solutions are reached by resolving a series of linear equations. This alteration decreases the computational complexity and makes the LS-SVM more attractive. In addition, the model has the benefits of simplifying the problem and solution finding without losing precision [73; 74]. More detailed information about LS-SVM is available from [75]. The LS-SVM technique has been applied to a variety of fields such as text classification [23; 76], image processing [77; 78], and time series forecasting [79; 80]. In this analysis, the LS-SVR is applied to forecast monthly rainfall. In the following section, we briefly present the basic theory on LS-SVR in time series forecasting. Considering a training set as input data \(x_{i}\) and output \(y_{i}\), the regression function that links the input vector to the output can be defined as [81]:  
\[f(x)=w^{T}\varphi(x)+b \tag{1}\]  
where the nonlinear mapping \(\varphi(x)\) maps the input data into a higher-dimensional feature space. Transforming the regression problem in Equation (1) into a constrained quadratic optimization problem, by minimizing the cost variable, \(\mathrm{w}\) and \(\mathrm{b}\) can be estimated. In the structural minimization principle, the regression problem can be formulated as:  
\[\min\!J(w,e)=\frac{1}{2}w^{T}w+\frac{\gamma}{2}\sum_{i=1}^{m}e_{i}^{2} \tag{2}\]  
subject to the following constraints:  
\[y_{i}=w^{T}\varphi(x_{i})+b+e_{i}\;(\mathrm{i}=1,\;2,\ldots,\;\mathrm{m}) \tag{3}\] \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Watershed** & **Mean (mm)** & **Standard Deviation (mm)** & **Skewness** & **Kurtosis** \\ \hline Deji & 193.71 & 184.80 & 1.91 & 5.15 \\ Shihmen & 200.35 & 208.09 & 2.56 & 9.29 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Statistical parameters of monthly rainfall series for the two Reservoir watersheds. where \(\gamma\) denotes the penalty term and \(e_{i}\) is the training error for \(x_{i}\). In order to solve the optimization problem, the solution for optimizing the LS-SVM is to create a Lagrangian function as follows:  
\[L(w,b,e,\alpha)=J(w,e)-\sum\limits_{i=1}^{m}\alpha_{i}\Big{\{}w^{T}\varphi(x_{ i})+b+e_{i}-y_{i}\Big{\}} \tag{4}\]  
where \(\alpha_{i}\) are the Lagrange multipliers (support values). Under the state of the Karush condition [82], we can partially differentiate the solution of Equation (4) to obtain w, b, e and alpha, respectively, as:  
\[\left\{ \begin{array}{c}w=\sum\limits_{i=1}^{m}\alpha_{i}\varphi(x_{i})\\ \sum\limits_{i=1}^{m}\alpha_{i}=0\\ \alpha_{i}=\gamma e_{i}\\ w^{T}\varphi(x_{i})+b+e_{i}-y_{i}=0\end{array} \right. \tag{5}\]  
\(e_{i}\) and \(w\) elimination would generate a linear process rather than a quadratic problem of programming:  
\[\left[ \begin{array}{cc}0&-\gamma^{T}\\ Y&ZZ^{T}+I/\gamma\end{array} \right]\left[ \begin{array}{c}b\\ \alpha\end{array} \right]=\left[ \begin{array}{c}0\\ 1\end{array} \right] \tag{6}\]  
where \(Y=(y_{1},...,y_{m}),Z=(\varphi(x_{1})^{T}y_{1},...,\varphi(x_{m})^{T}y_{m}),I= (1,...,1),\alpha=(\alpha_{1},...,\alpha_{l})\)  
By constructing the kernel function \(K(x,x_{i})=\varphi(x)^{T}\varphi(x_{i}),i=1,...,m\), [PERSON]'s theorem can be fulfilled (a detailed explanation of the Mercer's condition can be found in [83]). Next, the LS-SVM method is implemented for estimating the function:  
\[f(x)=\sum\limits_{i=1}^{m}\alpha_{i}K(x,x_{i})+b \tag{7}\]  
Within LS-SVM, there are many possible choices for the kernel function, such as sigmoid kernel, polynomial kernel, linear kernel, and radial bias function (RBF). The latter has fewer parameters and needs to be adjusted for practical problems.