## 3 An Autoencoder and Partial Least Squares Regression Based Extreme Learning Machine Model
### Extreme Learning Machine  
ELM is a new type of single hidden layer forward neural network. It randomly generates the connection weights and biases between the input layer and the hidden layer. The connection weights and biases do not need to be adjusted during the training process. Once the number of hidden neurons is determined, the optimal solution can be obtained. Compared with the traditional training methods, ELM has the advantages of fast learning speed and good generalization performance [27].  
Suppose that there are \(N\) samples  
Autoencoder technology developed by [PERSON] et al. in 1986 is an unsupervised learning method based on a BP algorithm. The purpose of the Autoencoder technology is to approximate an identity function such that the output data is the same as the input data [30]. The ELM-Autoencoder technology based on the ELM algorithm is introduced in this study to avoid repeated iterative training of the BP network [31]. The Autoencoder based extreme learning machine (AE-ELM) proposed in this study can be implemented mainly in two stages. First, the ELM-Autoencoder technology is employed to establish the mapping relationship between \(X\) to \(X\) (\(X\) is the input data) using the ELM algorithm. The output weights of the ELM-Autoencoder is taken as the initial weights of the AE-ELM model. Second, the AE-ELM model, which takes \(X\) and \(Y\) (\(Y\) is the output data) as training data, is trained with the initial weights taken from the first stage.  
The ELM-Autoencoder is a type of unsupervised network and the input weights and hidden biases should be orthogonal. Assume the number of input neurons is \(N_{p}\), and the number of the hidden neurons is \(N_{L}\). In this study, the number of input neurons is smaller than the number of hidden neurons. Therefore, a sparse ELM-Autoencoder architecture is adopted [32]. Given a set of \(N\) data samples, i.e., \(\left(\mathbf{x}_{k},\mathbf{y}_{k}\right)\) for \(k=1,2,\ldots,N\), the hidden layer output of the ELM-Autoencoder can be expressed as:  
\[\mathbf{h}(\mathbf{x}_{i})=g\left(\mathbf{a}^{T}\mathbf{x}_{i}+\mathbf{b}\right)\ i=1,2,\ldots,N \tag{9}\]  
where \(\mathbf{h}(\mathbf{x}_{i})\in R^{N_{L}}\) denotes the hidden layer output with respect to the \(i\)th input, \(\mathbf{a}^{T}\mathbf{a}=I\), \(\mathbf{b}^{T}\mathbf{b}=I\), \(\mathbf{I}\) is a unit matrix, \(\mathbf{a}=[a_{1},a_{2},\ldots,a_{N}]\) denotes the orthogonal weights connecting the input layer and the hidden layer, \(\mathbf{b}=[b_{1},b_{2},\ldots,b_{N}]\) denotes the orthogonal biases of the hidden nodes, and \(g(\cdot)\) is the activation function.  
The output weight \(\mathbf{\beta}\) is optimized to minimize the squared loss of the training error. The optimization problem can be expressed as the equation below.  
\[\min_{\mathbf{\beta}}\ O_{\mathbf{\beta}}=\min_{\mathbf{\beta}}\ \frac{1}{2}\|\mathbf{\beta} \|^{2}+\frac{C}{2}\|\mathbf{X}-\mathbf{H}\mathbf{\beta}\|^{2} \tag{10}\]  
where \(\mathbf{X}\) denotes the input data, \(\mathbf{H}\) denotes the hidden layer outputs, and \(C\) denotes a penalty factor on the training error.  
The first derivative of \(O_{\mathbf{\beta}}\) with respect to \(\mathbf{\beta}\) can be denoted as:  
\[\Delta_{\mathbf{\beta}}=\mathbf{\beta}-\mathbf{C}\mathbf{H}^{T}(\mathbf{X}-\mathbf{H}\mathbf{\beta})=0 \tag{11}\]  
The final hidden layer output weights can be calculated as:  
\[\mathbf{\beta}=\left(I/C+\mathbf{H}^{T}\mathbf{H}\right)^{-1}\mathbf{H}^{T}\mathbf{X} \tag{12}\]