## 2 Methodology
### Improved Weighted Extreme Learning Machine (IWELM)
Secondly, instead of picking the parameters randomly, an improved switch optimization algorithm is presented to attain fixed values for \(\mathbf{Y}\) and \(\mathbf{W}\), \(\boldsymbol{b}\). For minimizing the forecasting errors, the loss function \(\eta\) of training data are defined as  
\[\eta{=}\|\mathbf{H}\boldsymbol{\beta}-\mathbf{r}\|^{2}=\sum_{i=1}^{N}{\xi_{i} }^{2}=\sum_{i=1}^{N}{(o_{i}-r_{i})}^{2}, \tag{11}\]  
where \(\|\cdot\|\) stands for the \(L^{2}\) vector norm. \(\xi_{i}\) is the output error of the \(i^{th}\) sample. To avoid overfitting in optimization, the Tikhonov regularization [44] is used to restrict the norm of output weights while minimizing the loss. Then, the loss function is rewritten as  
\[\eta_{G}{=}C\|\mathbf{H}\boldsymbol{\beta}-\mathbf{r}\|^{2}+\|\boldsymbol{ \beta}\|^{2}, \tag{12}\]  
where \(\eta_{G}\) is the generalized loss function, and \(C\) is the regularization parameter. Using the Karush-Kuhn-Tucker (KKT) conditions [45], the optimal equation for \(\boldsymbol{\beta}\) is solved as  
\[\frac{\partial\eta_{G}}{\partial\boldsymbol{\beta}}{=}0\rightarrow\boldsymbol {\beta}{=}\left(\frac{\mathbf{I}}{C}+\mathbf{H}^{T}\mathbf{H}\right)^{-1} \mathbf{H}^{T}\mathbf{r}, \tag{13}\]  
where \(\frac{\partial}{\partial}\): is the symbol of partial derivative. After acquiring \(\boldsymbol{\beta}\), the minimization of \(\eta_{G}\) tends to be a optimal problem with \(\mathbf{Y}\) and \(\mathbf{W}\), \(\boldsymbol{b}\), which is usually solved by a gradient descent (GD) iterative method. Iterative algorithms seldom occur in ELM networks due to the massive computation of calculating the generalized inverse matrix in the iterative loops. An advanced algorithm named Levenberg-Marquardt (LM) is applicable for ELM [46], but the number of parameters is strictly limited. In this paper, an improved switch optimization algorithm based on the special structure of IWELM is proposed. Defining \(\boldsymbol{H}\), \(\boldsymbol{\beta}\) and \(\dot{\boldsymbol{r}}\) as the targets of optimization and output, the target model is expressed by  
\[\dot{\boldsymbol{r}}=\boldsymbol{H}\boldsymbol{\beta}=\left(\mathbf{H}+ \Delta\mathbf{H}\right)\left(\boldsymbol{\beta}+\Delta\boldsymbol{\beta} \right). \tag{14}\]  
Then, the switch algorithm is employed to eliminate \(\Delta\mathbf{H}\), \(\Delta\boldsymbol{\beta}\), which are the redundant parts of \(\mathbf{H}\) and \(\boldsymbol{\beta}\), respectively. The scheme for the proposed switch algorithm is shown in Figure 5. This switch algorithm reserves the accuracy of the iterative algorithm and reduces the computation through a 2-layer loops frame that is composed of the LM and Adam algorithm. A specific criterion is involved in switching from LM to Adam. \[\mathbf{H}\boldsymbol{\beta}=g\left(\mathbf{X}\cdot\mathbf{W}+\boldsymbol{u} \cdot\boldsymbol{b}\right)\boldsymbol{\beta}=\boldsymbol{o}, \tag{9}\]0 The principle for the proposed switch algorithm. The coarse-tuning is a rapid adjustment on structural parameters \(\mathbf{Y}\) and \(\boldsymbol{\beta}\), where \(\mathbf{Y}\) is optimized by the LM algorithm and \(\boldsymbol{\beta}\) is calculated by Equation (13). The fine-turning is an iterative process which aims to obtain accurate values for \(\mathbf{W}\) and \(\boldsymbol{b}\). The steps of the proposed switch algorithm are outlined in Algorithm 1. The symbol \(\otimes\) means the element-wise multiplication. \(\boldsymbol{J}\) indicates the Jacobian matrix, and \(\kappa\) represents for the switching criterion. Further details about the original switch algorithm can be consulted in [47]. According to the steps in Algorithm 1, the first period of each outer loop is the LM algorithm that is a trust-region algorithm. On this account, the proposed switch algorithm is convergent. The ultimate objective of the switch algorithm is a certain IWELM model with stable parameters \(\mathbf{Y}\), \(\mathbf{W}\), \(\boldsymbol{b}\), \(\boldsymbol{\beta}\).