## 3 An Autoencoder and Partial Least Squares Regression Based Extreme Learning Machine Model
### The Proposed AE-PLS-ELM Model
\[E_{0}{}^{T}F_{0}c_{1}=\theta_{1}\omega_{1} \tag{19}\]  
\[F_{0}{}^{T}E_{0}\omega_{1}=\theta_{1}\omega_{1} \tag{20}\]  
From Equation (19) and Equation (20), it can be deducted that:  
\[E_{0}{}^{T}F_{0}{}^{T}E_{0}\omega_{1}={\theta_{1}}^{2}\omega_{1} \tag{21}\]  
where \(\omega_{1}\) is the eigenvector of \(E_{0}{}^{T}F_{0}F_{0}{}^{T}E_{0}\), \(\theta_{1}{}^{2}\) is the eigenvalue, and \(\omega_{1}\) and \(c_{1}\) can be obtained according to Equation (19) and Equation (20). After obtaining \(\omega_{1}\) and \(c_{1}\), the score vectors \(u_{1}\) and \(v_{1}\) can be calculated according to Equation (13). Step 5: Establish the linear regression model between \(E_{0}\) and \(u_{1}\), and \(F_{0}\) and \(v_{1}\) according to the least square method. \[\left\{ \begin{array}{l}E_{0}=u_{1}\alpha_{1}{}^{T}+E_{1}\\ F_{0}=v_{1}\gamma_{1}{}^{T}+F_{1}\end{array} \right. \tag{22}\]  
where \(\alpha_{1}=[\alpha_{11},\alpha_{12},\ldots,\alpha_{1L}]\) and \(\gamma_{1}=[\gamma_{11},\gamma_{12},\ldots,\lambda_{1L}]\) are regression coefficients, and \(E_{1}\) and \(F_{1}\) are the residual matrix. The least square estimation of \(\alpha_{1}\) and \(\gamma_{1}\) can be denoted below. \[\left\{ \begin{array}{l}\alpha_{1}=\frac{E_{0}{}^{T}u_{1}}{\left\|u_{1} \right\|^{2}},\\ \gamma_{1}=\frac{F_{0}{}^{T}v_{1}}{\left\|v_{1}\right\|^{2}}.\end{array} \right. \tag{23}\]  
Step 6: If \(F_{1}\) satisfy the stopping criteria, Equation (22) is the final regression model and the iteration stops. Otherwise, replace \(E_{0}\) and \(F_{0}\) with \(E_{1}\) and \(F_{1}\), respectively, and skip to Step 3 to get the second pair of score vectors. \[u_{2}=E_{1}\omega_{2} \tag{24}\]  
\[\left\{ \begin{array}{l}E_{0}=u_{1}\alpha_{1}{}^{T}+u_{2}\alpha_{2}{}^{T}+E_{ 2}\\ F_{0}=v_{1}\gamma_{1}{}^{T}+v_{2}\gamma_{2}{}^{T}+F_{2}\end{array} \right. \tag{25}\]  
where \(\alpha_{2}\) and \(\gamma_{2}\) are regression vectors and can be denoted as:  
\[\left\{ \begin{array}{l}\alpha_{2}=\frac{E_{1}{}^{T}u_{2}}{\left\|u_{2} \right\|^{2}},\\ \gamma_{2}=\frac{E_{1}{}^{T}v_{2}}{\left\|v_{2}\right\|^{2}}.\end{array} \right. \tag{26}\]Step 7: Repeat Steps 4-5 until \(r\) principal components are calculated. The remaining \(m-r\) components are small and are considered as noises. The residuals \(E_{r}\) and \(F_{r}\) are very small. \(E_{0}\) and \(F_{0}\) can be expressed by the equation below. \[\left\{ \begin{array}{l}E_{0}=u_{1}{\alpha_{1}}^{T}+u_{2}{\alpha_{2}}^{T}+ \ldots+u_{r}{\alpha_{r}}^{T}+E_{r}=Ua^{T}+E_{r},\\ F_{0}=v_{1}{\gamma_{1}}^{T}+v_{2}{\gamma_{2}}^{T}+\ldots+v_{r}{\gamma_{r}}^{T}+ F_{r}=V{\gamma}^{T}+F_{r}\end{array} \right. \tag{27}\]  
The relationship between \(u_{k}\) and \(v_{k}\) can be expressed by the equation below [1]. \[v_{k}=u_{k}b_{1},\;k=1,2,\ldots,r. \tag{28}\]  
Then \(F_{0}\) can be translated into:  
\[F_{0}=V{\gamma}^{T}+F_{r}=\sum_{i=1}^{r}u_{1}b_{1}{\gamma_{1}}^{T}+u_{2}b_{2}{ \gamma_{2}}^{T}+\ldots+u_{r}b_{r}{\gamma_{r}}^{T}+F_{r}=UB{\gamma}^{T}+F_{r} \tag{29}\]  
where \(\hat{U}=E_{0}W\), then the regression equation can be denoted as:  
\[\hat{F}_{0}=E_{0}W{B\gamma}^{T}+F_{r} \tag{30}\]  
According to the description above, the hidden layer output weight vector can be expressed by the equation below. \[\hat{\beta}_{PLS}=W{B\gamma}^{T} \tag{31}\]  
where \(W\) denotes the component matrix and \(B\) denotes the diagonal matrix. Based on the above modeling process, the structure of the proposed AE-PLS-ELM model is shown in Figure 3 as follows. Figure 3: Structure of the proposed Autoencoder and partial least squares regression based extreme learning machine (AE-PLS-ELM) model. (**a**) ELM-AE, (**b**) PLSR, (**c**) AE-PLS-ELM.