## 3 Preliminaries
### Kernel Extreme Learning Machine  
The KELM was proposed by [PERSON] et al. [39] as a type of the classical Extreme Learning Machine (ELM) [40]. The main motivation for ELM was to overcome the drawbacks of traditional gradient descent methods for training feedforward neural networks (FFNN). ELM incorporates three steps to train the FFNN algorithm. It starts by initially assigning the input weights and the hidden biases of the FFNN randomly, then it calculates the hidden layer output matrix and finally, in one step it determines the output weights using Moore-Penrose (MP) generalized inverse. Recently, there have been studies that utilized other approaches such as evolutionary computing for training ELMs and optimizing their structures. In the KELM algorithm, [PERSON] et al. utilized the idea of the kernel function in ELM.  
The output of ELM with \(L\) hidden neurons can be given as shown in Equation (5), knowing that we have a training sample set \((x_{i},t_{i}),i=1,2,\cdots,N\).  
\[f_{K}(x)=\sum_{j=1}^{L}\beta_{j}h_{j}(x,w_{j},b_{j})=h(x)\beta=H\beta \tag{5}\]  
where \(\beta=[\beta_{1},\beta_{2},\cdots,\beta_{L}]^{T}\) is the output weight vectors that connect the hidden layer to the output layer. \((w_{j})\) is the weights vector that connects the (\(j\)th) hidden node to the input layer, and \((b_{j})\) is the bias of the (\(j\)th) node. \((h(x))\) is the output matrix of the hidden layer with respect to the input (\(x\)). ELM algorithm tries to minimize the training error and at the same time minimizes the norm of the output weights \(\beta\). The optimization function of ELM can be expressed as follows.  
\[Minimize:\|\beta\|_{p}^{c1}+C\left\|H\beta-T\right\|_{q}^{c2} \tag{6}\]  
In which, \((c1,c2)>0,\ (p,q)=1,2,\cdots,\infty\). (\(C\)) is the factor that controls the trade-off between the training error and the norm of the output weights. While, (\(H\)) is the matrix of the hidden layer (Equation (7)).  
\[H=\begin{bmatrix}h(x_{1})\\ \vdots\\ h(x_{N})\end{bmatrix}=\begin{bmatrix}h_{1}(x_{1})&\ldots&h_{K}(x_{1}))\\ \vdots&&\vdots\\ h_{1}(x_{N})&\ldots&h_{K}(x_{N}))\end{bmatrix} \tag{7}\]whereas, \(T=\left[t_{1},t_{2},\cdots,t_{N}\right]^{T}\) is the target matrix of the training data. The output of ELM based on \(N\) (number of training samples) and \(L\) (the number of hidden neurons) is defined as presented in Equation (8).  
\[f(x)=\begin{cases}\beta H^{T}(\frac{l}{C}+HH^{T})^{-1}T,&\text{when }N\leq L\\ \beta(\frac{l}{C}+H^{T}H)^{-1}H^{T}T&\text{when }N>L\end{cases} \tag{8}\]  
In the case that the user does not know the feature mapping matrix (\(H\)), a kernel matrix of ELM, which is called kernel function mapping can be used by applying Equations (9-10).  
\[\Omega_{ELM}=HH^{T}:\Omega_{ELM_{ij}}=h(x_{i})Jh(x_{j})=K(x_{i},x_{j}) \tag{9}\]  
where \(K(x_{i},x_{j})\) is the kernel function. By applying Equations (8) and (9), Equation (5) can be written as in (Equation (10)).  
\[f(x)=H\beta=HH^{T}(HH^{T}+\frac{I}{C})^{-1}T=\begin{bmatrix}K(x,x_{1})\\ \vdots\\ K(x,x_{N})\end{bmatrix}\left(\Omega+\frac{I}{C}\right)^{-1}T \tag{10}\]  
The kernel function that is commonly used in the KELM is the radial basis function (RBF), also known as the Gaussian kernel function. RBF is defined as follows:  
\[K(x,y)=exp(-\gamma\left\|x-y\right\|^{2}),\gamma>0 \tag{11}\]  
The two crucial parameters are the regularization factor \(C\) and the kernel parameter \(\gamma\). \(C\) is used to get better adjustment between the error minimization and the model complexity to enhance the generalization performance. \(\gamma\) is used to define the mapping from the input space to the high dimensional feature space.  
The general structure of the KELM is shown in Figure 2