## 3 Proposed Method of Training ELM-AEs Based on G-FISTA and LC-FISTA  
ELM-AE has an extremely fast training speed, and good pattern reconstruction capability that is better than backpropagation-based learning. The formulation of this model can be expressed as a convex optimization problem which involves minimizing the weights of the \(\mathbf{\beta}\) output layer, as shown in the following equation:  
\[\min_{\mathbf{\beta}}\|\mathbf{H}\mathbf{\beta}-\mathbf{X}\|_{u}^{\kappa_{1}}+\lambda\|\mathbf{ \beta}\|_{\diamond}^{\kappa_{2}}, \tag{6}\]  
where \(\kappa_{1},\kappa_{2}>0\), \(u,\upsilon=0,\frac{1}{2},1,2,\ldots\) and \(\lambda\) is a control parameter between the training error and the generalization ability. Several methods exist for computing the solution to (6), such as the Moore-Penrose inverse, orthogonal projection, and proper decomposition [51]. However, in a real situation, the number of independent variables \(L\) in the linear system \(\mathbf{H}\mathbf{\beta}=\mathbf{X}\) is much larger than the number of data points \(N\), which may cause overfitting. On the other hand, the reconstruction capacity is low whenever the number of hidden neurons \(L\) is small. As the training dataset increases, the computational cost of the inverse matrix \(\mathbf{H}^{\dagger}\) increases significantly. Several regularization methods have been introduced in the literature to address the above problems. The two most commonly used classical techniques are ridge regression [52] and sparse regularization [37]. The sparse regularization plays an important role in feature selection, which has shown successful results in machine learning. This study presents an unsupervised feature selection framework, which integrates sparse optimization and signal reconstruction for pattern recognition models. The minimization problem takes the following form:  
\[\min_{\mathbf{\beta}}\|\mathbf{H}\mathbf{\beta}-\mathbf{X}\|_{2}^{2}+\lambda|\mathbf{\beta}|_{1}, \tag{7}\]  
which is a special case of (4), in which we set \(f(\mathbf{\beta})=\|\mathbf{H}\mathbf{\beta}-\mathbf{X}\|_{2}^{2}\) and \(\phi(\mathbf{\beta})=\lambda|\mathbf{\beta}|_{1}\). From the description given above, we see that the estimation of \(\mathbf{\beta}\) and the time required are the main challenges for training the ELM-AE. Our study proposes two novel iterative schemes for autoencoder training, G-FISTA, and LC-FISTA, which reduce the computational time of FISTA maintaining its accuracy, where FISTA is the classical algorithm for training neural networks. This optimization approach is also interesting because it can be extended to other regularization terms, which control the variables selection by calculating a derivative in the weakest sense through convex envelopes. In addition, the closed form of the proximal operator \(\text{prox}_{\rho\mathbf{\phi}}(\cdot)\) of \(\mathbf{\phi}\) is computed by the shrinkage operator \(\text{shrink}(\mathbf{z},\lambda\rho)\)[38], whose \(i\)-th element is calculated as follows:  
\[\text{shrink}(\mathbf{z},\lambda\rho)_{i}=\text{shrink}(\mathbf{z}_{i},\lambda\rho )=\text{sign}(\mathbf{z}_{i})\max\{|\mathbf{z}_{i}|-\lambda\rho,0\}, \tag{8}\]  
where \(\mathbf{z}\) is the initial estimation of the weight of the \(\mathbf{\beta}\) output layer. The speed that can be achieved by the use of G-FISTA and LC-FISTA in the training of the ELM-AE is significantly higher than that obtained using FISTA. In addition, for poorly conditioned matrices, the two algorithms have the same ability as FISTA to reconstruct the input signals. This is realized by the omission of several steps of the gradient descent, and correction of the solution by means of the proximal gradient. Although the iterative scheme used by G-FISTA and LC-FISTA effectively has additional terms compared to FISTA, the computational cost can be similar, since they converge with fewer iterations.