## 3 Interactive Guiding Sparse Autoencoder  
Autoencoders have traditionally been used as one of the dimensionality reduction methods, such as PCA and LDA. However, when overfitting occurs in the learning process,  
Figure 1: The virtual structure of the autoencoder using ELM. the following two major problems occur. First, even if unrelated data are used as input, the autoencoder transforms the data into a similar distribution with a training set. Second, while learning the information of features with low importance, a noise can be recognized as important information, which affects classification accuracy. To solve the problem, we developed an interactive guiding sparse autoencoder (IGSAE) that learns the manifold of selected features and guides it to the rest of the features. As shown in Figure 2, IGSAE has a structure with two added guiding layers to the basic autoencoder so that the two layers are interactive. From left to right, it is composed in the order of input layer, hidden layer (with guiding layer), reconstruction layer, and guiding layer (output layer). The whole process is explained in small five processes. In the first process of IGSAE, \(H\) and \(G_{1}\) are the output of hidden units and the output of the first guiding units, respectively. The step to calculate \(H\) and \(G_{1}\) can be represented as follows:  
\[H=\phi(W_{1}X+b_{1}) \tag{8}\]  
\[G_{1}=\phi\big{(}W_{1}{}^{\prime}X+b_{1}{}^{\prime}\big{)} \tag{9}\]  
where \(X=\{x_{1},\ x_{2},\dots,\ x_{n}\}\) is input data, \(\phi\) is activation function (RELU), and \(W_{1}\) and \(b_{1}\) denote the weight and bias between the input and hidden layer, respectively. \(W_{1}{}^{\prime}\) and \(b_{1}{}^{\prime}\) denote the weight and bias between the input and guiding layer, respectively. \(X\) denotes a guided feature and \(X_{g}\) denotes a guiding feature. The learning process of IGSAE can be divided into two parts: (a) to learn the guiding feature as a target from the guided feature to involve the relationship between the two features, and (b) to reconstruct the guided feature. A pretrained model from guided feature \(X\) to guiding feature \(X_{g}\) is required to involve the relationship between the features. The corresponding part of IGSAE maps the guided feature of the input layer \(X\) to the guiding feature of the first guiding layer \(G_{1}\) and is pretrained partially. The cost function \(J_{fg}\) of the partial pretrained model is formulated as follows:  
\[J_{fg}\ \ =\ \frac{1}{2\pi}\ \parallel X_{g}-G_{1}\parallel^{2} \tag{10}\]  
where \(X_{g}=\{x_{n+1},\ x_{n+2},\dots,\ x_{m}\}\) is input data of the guiding layer. The cost function used for pretraining is also used as a regularization term for the objective function in Equation (18). Figure 2: The architecture of IGSAE. It consists of first guiding layer for the reconstruction of guided features and second guiding layer to reflect the distribution of guiding features. In the second process, as the number of units in the hidden layer increases, there may be too much sparse information. As a sparse constraint, the average activity of the hidden unit is calculated. If it is an important unit, it is activated, and if not, it is deactivated. The sparsity criterion for determining the importance of a unit uses the Kullback-Leibler (KL) divergence. KL divergence is the relative entropy to measure the similarity between a sparsity parameter \(\rho\) and the average activation probability \(\hat{\rho}_{j}\). \(s\) is the number of the hidden units. The average activation \(\hat{\rho}_{j}\) is represented as:  
\[\hat{\rho}_{j}=\frac{1}{n}\sum\limits_{i=1}^{n}\phi(x_{i}) \tag{11}\]  
and \(J_{sparse}\)(_KL_ divergence) is represented as:  
\[ \begin{array}{ll}J_{sparse}&=\sum\limits_{j=1}^{s}KL(\rho||\hat{\rho}_{j}) \\ &=\sum\limits_{j=1}^{s}\rho\log\frac{\rho}{\hat{\rho}_{j}}+(1-\rho)\log\frac{1 -\rho}{1-\hat{\rho}_{j}}\end{array}