## 3 SSDAE-DRN Network  
### Theory of SSDAE  
A denoising auto-encoder [29] (DAE) is a three-layer neural network consisting of an input layer, a hidden layer, and an output layer, and the flow of the DAE algorithm is shown in Figure 1. After adding noise to the original input \(x\), the interference input \(\widetilde{x}\) is obtained, and the input feature expression \(h\) is obtained through the encoding function \(f_{\theta}\). Then \(h\) is mapped to the output layer through the decoding function \(g_{\theta}\) to obtain the reconstructed data \(z\) of the input data.  
Therefore, the encoding process of DAE can be expressed as  
\[h=f_{\theta}(\tilde{x})=\delta(w\tilde{x}+b), \tag{9}\]  
where \(\theta\) = (\(w\),\(b\)) is the coding model parameter; \(w\) is the coding weight matrix, and \(b\) is the coding offset vector. The decoding process can be expressed as  
\[z=g_{\theta^{\prime}}(y)=\delta(w^{\prime}h+b^{\prime}), \tag{10}\]  
where \(\theta^{\prime}=(w^{\prime},b^{\prime})\) is the decoding model parameter, \(w^{\prime}\) is the decoding weight matrix, \(b^{\prime}\) is the decoding offset vector, and \(\delta\) is the nonlinear activation function. The reconstruction result \(z\) cannot accurately reproduce the input data \(x\), for the training sample set \(D=\Big{\{}x^{(i)}\Big{\}}_{i=1}^{N}\), the cost function of DAE is  
\[L_{DAE}(\theta)=\frac{1}{N}\sum_{i=1}^{N}\left\|z^{(i)}-x^{(i)}\right\|_{F}^{ 2}+\frac{\lambda}{2}\Big{(}\|w\|_{F}^{2}+\|w^{\prime}\|_{F}^{2}\Big{)}, \tag{11}\]  
where \(\lambda\) is is the weight, and the over-fitting can be reduced by decreasing the weight. In addition, a penalty factor is added to the cost function to limit the activation degree of the neurons in the overall hidden layer, which can realize the high-dimensional sparse feature extraction of the input data. Therefore, the cost function of the sparse denoising auto-encoder (SDAE) is  
\[L_{SDA}(\theta)=\frac{1}{N}\sum_{i=1}^{N}\left\|z^{(i)}-x^{(i)}\right\|_{F}^{ 2}+\frac{\lambda}{2}\Big{(}\|w\|_{F}^{2}+\|w^{\prime}\|_{F}^{2}\Big{)}+\beta \sum_{j=1}^{k}\left(\rho\log\frac{\rho}{\hat{\rho}_{j}}+(1-\rho)\log\frac{1- \rho}{1-\hat{\rho}_{j}}\right), \tag{12}\]  
where \(\rho\) is the sparsity parameter, \(\beta\) is the weight of the control sparse constraint, and \(\hat{\rho}_{j}\) is the average activation of the input data corresponding to the \(j\)th node of the hidden layer [30].  
SSDAE is a deep neural network formed by stacking several SDAE, and the output vector of the previous SDAE hidden layer is the input vector of the next SDAE. The training of SSDAE adopts the greedy layer-by-layer training method that can effectively alleviate the gradient dispersion phenomenon [31]; that is, each SDAE is trained separately to obtain the optimal weight of the network which is used as the SSDAE network weights. The network is then fine-tuned overall by the error back propagation (BP) algorithm [32] until the optimal parameters of the network are derived. The cost function of the fine-tuning process of SSDAE is  
\[L_{SSDAE}(\theta)=\frac{1}{N}\sum_{i=1}^{N}\left\|z^{(i)}-x^{(i)}\right\|_{F}^ {2}+\frac{\lambda}{2}\sum_{l=1}^{L}\Big{(}\|w\|_{F}^{2}+\|w^{\prime}\|_{F}^{2} \Big{)}, \tag{13}\]  
where \(L\) is the number of SDAE. There is no sparsity constraint for fine-tuning since the sparsity penalty has been included in the pre-training of each SDAE.  
Figure 1: Flowchart of DAE algorithm.