## 2 Materials and Methods
### Data Processing
#### 2.3.4 Krls  
KRLS is a nonlinear regression method that develops ideas of generalized linear models. The main idea is to construct a function or a surface, which is then penalized for complexity via selecting one of many possible regression problem solutions with the help of kernels and the least squares problem. There are two explanations of KRLS: the \"similarity-based\" method and the \"superposition of Gaussians\" method. The \"similarity-based\" method assumes that the data considered is i. i. d. and consists of yields \(y_{i}\) and a D-dimensional vector of covariate values \(x_{i}\) for \(i\)-th exemplar. Then a semi-defined positive and symmetric function is taken as the kernel \(k\). This takes two numbers and produces a real value output \(y_{i}\). In this article, the Gaussian kernel is used:  
\[k\big{(}x_{i}x_{j}\big{)}=exp\big{(}\frac{-\|x_{i}-x_{j}\|^{2}}{\sigma^{2}} \big{)} \tag{1}\]  
where \(\|x_{i}-x_{j}\|\) is the Euclidean distance between the covariate vectors \(x_{i}\) and \(x_{j}\). The kernel bandwidth \(\sigma^{2}\) is set to the number of dimensions in this work, though it can be varied [17]. The most important feature of this function is that it reaches its maximum of one only when \(x_{i}=x_{j}\) and gets closer to zero when the similarity between two vectors decreases. The target function \(y=f(x)\) can be approximated by the equation:  
\[f(x)=\sum_{i}^{N}c_{i}k(x,x_{i}) \tag{2}\]  
where \(k(x,x_{i})\) is the measurement of the similarity and \(c_{i}\) is the weight for each input pattern. Therefore, the main idea is to replace the linear combinations of functions of \(x_{i}\) with the similarity between a test point \(x\) and a fixed input pattern \(x_{i}\). This should be a more appropriate point of view, as similar inputs should give similar outputs. The \"superposition of Gaussians\" method supposes that an approximation function can be described with the help of Equation (2) where each \(c_{i}k(x,x_{i})\) is a different Gaussian curve centered around \(x_{i}\) and scaled by \(c_{i}\). These Gaussians are summed up to give a resulting function, which is a more flexible approach than in linear methods as it allows describing nonlinear and nonadditive functions. For further understanding of KRLS we should rewrite (2) in a more convenient form:  
\[y=Kc= \begin{bmatrix}k(x_{1},x_{1})&k(x_{1},x_{1})&\cdots&k(x_{1},x_{1})\\ k(x_{1},x_{1})&\ddots&&\\ \vdots&&&&\\ k(x_{1},x_{1})&&&&k(x_{1},x_{1})\end{bmatrix} c_{1} \tag{3}\]Matrix \(K\) will be invertible, as it is positive, semidefinite, and symmetric. This means that there is a best solution to the linear system \(y=Kc\) which gives a function that provides the best fit for each data point. To prevent overfitting, two more assumptions are made. The first is that the error of the function is counted as a squared loss so that the result could be easily interpreted as an expectation function. The second is that the complexity of the function is regularized by a parameter \(\lambda\) to perform a tradeoff between complexity and accuracy. This is needed because smoother functions tend to result in a better fitting due to the possible noise in the signal. Therefore, the Tikhonov regularization problem, which may help to find the best function fitting, in this case can be written as:  
\[argmin_{f\in H}\sum_{i}^{N}(y_{i}-f(x_{i}))^{2}+\lambda\left\|f\right\|^{2}_{ H} \tag{4}\]  
where \(\lambda\in R^{+}\) and f is the sought function. \(\left\|f\right\|^{2}_{H}\) is a L\({}_{2}\) norm in the reproducing kernel Hilbert spaces of functions associated with a particular choice of kernel. To solve this problem, \(f(x)\) is replaced by \(Kc\), and \(\left\|f\right\|^{2}_{H}\) which is equal to \(\sum_{i}\sum_{j}c_{i}c_{j}k\big{(}x_{i}x_{j}\big{)}\) is replaced by \(c^{T}Kc\). This results in:  
\[c^{*}=argmin_{c\in R^{D}}\sum_{i}^{n}(y-Kc)^{T}(y-Kc)+\lambda c^{T}Kc \tag{5}\]  
Consequently, \(y^{*}=Kc^{*}\) gives the best-fitting function.