## 4 Experimental Evaluation
### Experimental Setup  
In each experiment, the training and test data were standardized to the range \([0,1]\) using the following equation:  
\[y_{\text{norm}}^{a}=\frac{y_{i}^{a}-y_{\text{min}}^{a}}{y_{\text{max}}^{a}-y_ {\text{min}}^{a}} \tag{9}\]  
where \(y_{i}^{a}\) is the actual value to be standardized, \(y_{\text{min}}^{a}\) and \(y_{\text{max}}^{a}\) are the minimum and maximum of the actual values. In the hidden layer, the weights and biases are random values generated by means of a uniform distribution in the ranges \([-1,1]\) and \([0,1]\), and the sigmoid \(g(x)=\frac{1}{1+x^{-x}}\) was used as the activation function. We chose this activation function due to its universal approximation capability in ELM networks. The algorithm presented in Section 3 has the following parameters that need to be configured: the number of neurons \(L\), the stopping criterion \(P\), the maximum num \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **\# Training** & **\# Testing** & **\# Features** & **\# Category** \\ \hline UMist & 400 & 175 & 10,304 & 20 \\ CIFAR10 & 50,000 & 10,000 & 1024 & 10 \\ MNIST & 60,000 & 10,000 & 784 & 10 \\ NORB & 24,300 & 24,300 & 18,432 & 5 \\ SCars & 8144 & 8041 & 9312 & 196 \\ Caltech256 & 21,314 & 9293 & 10,000 & 257 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Information from the databases to train the ELM-AE. \begin{table}
\begin{tabular}{c c} \hline \hline
**Metric** & **Definition** & **Mathematical Expression** \\ \hline RMSE & Root mean square error & \(\text{RMSE}=\sqrt{\frac{\sum_{i=1}^{n}(y_{i}^{a}-y_{i}^{a})^{2}}{n}}\) \\ MAE & Mean absolute error & \(\text{MAE}=\frac{\sum_{i=1}^{n}|y_{i}^{a}-y_{i}^{a}|}{n}\) \\ R\({}^{2}\) & R\({}^{2}\) score & \(\text{R}^{2}=1-\frac{\sum_{i=1}^{n}(y_{i}^{a}-y_{i}^{a})^{2}}{\sum_{i=1}^{n}(y_ {i}^{a}-y_{i}^{a})^{2}}\) \\ IA & Willmott’s index of agreement & \(\text{IA}=1-\frac{\sum_{i=1}^{n}(y_{i}^{a}-y_{i}^{a})^{2}}{\sum_{i=1}^{n}(|y_ {i}^{a}-y_{i}^{a}|+|y_{i}^{a}-y_{i}^{a}|)}\) \\ TIC & Theil inequality coefficient & \(\text{TIC}=\frac{\sqrt{1\sum_{i=1}^{n}(y_{i}^{a}-y_{i}^{a})^{2}}}{\sqrt{1\sum_{i=1 }^{n}(y_{i}^{a})^{2}}\times\sqrt{1\sum_{i=1}^{n}(y_{i}^{a})^{2}}}\) \\ MIN & Minimum error & \(\text{MIN}=\min\{y_{i}^{a}-y_{i}^{a}|,\,i=1,\ldots,n\}\) \\ MAX & Maximum error & \(\text{MAX}=\max\{y_{i}^{a}-y_{i}^{a}|,\,i=1,\ldots,n\}\) \\ \hline \hline \end{tabular} Note: \(y^{a}\) and \(y^{a}\) denote the vectors of actual and estimated values, respectively, \(\overline{y}^{a}\) is the average of the actual values, and \(n\) is the number of pairs of \(y_{i}^{a}\) and \(y_{i}^{a}\).  
\end{table} 
Table 2: Seven evaluation criteria to evaluate the performance of the proposed method. ber of iterations \(I\), and the regularization term \(\lambda\). In the first scenario of the proposed method, the regularization parameter set to \(\lambda=\frac{1}{2}\sqrt{\frac{2\log(L)}{N}}\), the stopping criterion \(P\) is expressed as \(\frac{\|H\beta_{L}-X\|}{\|X\|}<\varepsilon\) and the maximum number of iterations is \(I=50\). The threshold \(\varepsilon\) is a user-specified value that is applied to interrupt the iteration of the algorithm. For the UMist, CIFAR10, MNIST, NORB, SCars, and Caltech256 databases, we set \(\varepsilon=0.30,0.32,0.60,0.13,0.40\) and \(0.40\), respectively. Moreover, the practical application of the FISTA, G-FISTA, and LC-FISTA algorithms for training of the ELM-AE requires tuning of the \(L_{f}\) Lipschitz constant, and the strong convexity constant \(\mu\) must be tuned for the LC-FISTA algorithm. For the convex function \(f(\mathbf{\beta})=\|H\mathbf{\beta}-\mathbf{X}\|_{2}^{2}\), we chose values of \(L_{f}=\lambda_{\text{max}}(\mathbf{H}^{T}\mathbf{H})\) and \(\mu=\lambda_{\text{min}}(\mathbf{H}^{T}\mathbf{H})\), where \(\lambda_{\text{min}}\) and \(\lambda_{\text{max}}\) are the minimum and maximum proper values of \(\mathbf{H}^{T}\mathbf{H}\). The additional control parameters in G-FISTA were described in Section 2.2.2.