## 3 An Autoencoder and Partial Least Squares Regression Based Extreme Learning Machine Model
### The Proposed AE-PLS-ELM Model  
In the Autoencoder based extreme learning machine (AE-ELM) model, the Moore-Penrose generalized inverse based on least square is exploited to calculate the output weights, which makes it possible to apply the PLSR method by replacing the least square method. The employment of PLSR in the AE-ELM model can avoid the multicollinearity problem when applying the Moore-Penrose generalized inverse especially when the hidden layer output matrix is highly correlated and contains noises [34]. Based on the description of AE-ELM and PLSR, the key to establish the AE-PLS-ELM model is to model the linear relationship between the hidden layer output matrix \(H\) and the output layer matrix \(Y\) using the partial least squares regression (PLSR) method. As has been described in Section 3.2, the hidden layer output matrix \(H\) is an \(N\times L\) dimension matrix with \(L\) hidden layer output variables. The output layer matrix \(Y\) is a \(N\times q\) dimension matrix. The detailed modeling process of the AE-PLS-ELM model can be given as follows. Step 1: Randomly assign the input weights \(a_{i},i=1,2,\ldots,n\) and hidden layer bias \(b_{i},i=1,2,\ldots,L\). Step 2: Calculate the hidden layer output matrix of the training data \(H\), according to Equation (3). Step 3: The hidden layer output matrix \(H\) is taken as the independent matrix of the PLSR model. Let \(E_{0}=H\) and \(F_{0}=Y\). Step 4: Extract the first pair of score vectors of \(E_{0}\) and \(F_{0}\). The score vectors are denoted as \(u_{1}\) and \(v_{1}\), respectively. \(u_{1}\) and \(v_{1}\) can be denoted below. \[ \begin{array}{l}u_{1}=E_{0}\omega_{1}\\ v_{1}=F_{0}c_{1}\end{array}  \tag{13}\]  
where \(\omega_{1}=[\omega_{11},\omega_{12},\ldots,\omega_{1L}]^{T}\) is the load factor of \(E_{0}\) and \(\|\omega_{1}\|=1\), and \(c_{1}=[c_{11},c_{12},\ldots,c_{1L}]^{T}\) is the load factor of \(F_{0}\) and \(\|c_{1}\|=1\). The relevance between \(u_{1}\) and \(F_{0}\) should be as maximum as possible. Then \(u_{1}\) and \(v_{1}\) should satisfy the following two criterions. (1) \(u_{1}\) and \(v_{1}\) should contain the maximum variation information of \(E_{0}\) and \(F_{0}\). (2) \(u_{1}\) and \(v_{1}\) should have maximum relevance. The deduction of \(\omega_{1}\) and \(c_{1}\) can be transformed to the following optimization problem. \[ \begin{array}{l}\max\left(u_{1},v_{1}\right)=\left(E_{0}\omega_{1},F_{0}c_{1 }\right)=\omega_{1}{}^{T}E_{0}{}^{T}F_{0}c_{1}\\ s.t.\ \left\{\begin{array}{l}\omega_{1}{}^{T}\omega_{1}=\|\omega_{1}\|^{2}=1\\ c_{1}{}^{T}c_{1}=\|c_{1}\|^{2}=1\end{array}\right.\end{array}  \tag{14}\]  
According to the Lagrange multiplier method, the following equation is true. \[L=\omega_{1}^{T}E_{0}{}^{T}F_{0}c_{1}-\lambda_{1}\left(\omega_{1}^{T}\omega_{ 1}-1\right)-\lambda_{2}\left(c_{1}^{T}c_{1}-1\right) \tag{15}\]The partial derivatives of \(L\) on \(\omega_{1},c_{1}\lambda_{1}\), and \(\lambda_{2}\) can be expressed by the equation below. \[\left\{ \begin{array}{l}\frac{\partial L}{\partial\omega_{1}}=E_{0}{}^{T}F_{0}c _{1}-2\lambda_{1}\omega_{1}=0\\ \frac{\partial L}{\partial c_{1}}=F_{0}{}^{T}E_{0}\omega_{1}-2\lambda_{2}c_{1} =0\\ \frac{\partial L}{\partial\lambda_{1}}=-\left(\omega_{1}^{T}\omega_{1}-1 \right)=0\\ \frac{\partial L}{\partial\lambda_{2}}=-\left(c_{1}^{T}c_{1}-1\right)=0\\ \end{array} \right. \tag{16}\]  
From Equation (16), it can be deducted that:  
\[2\lambda_{1}=2\lambda_{2}=\omega_{1}^{T}E_{0}{}^{T}F_{0}c_{1}=\langle E_{0} \omega_{1},F_{0}c_{1}\rangle \tag{17}\]  
Note that:  
\[\theta_{1}=2\lambda_{1}=2\lambda_{2}=\omega_{1}^{T}E_{0}{}^{T}F_{0}c_{1} \tag{18}\]  
\(\theta_{1}\) is the objective function of the optimization problem, and Equations (19) and (20) are true.