# Detection of Fitting in Gears Using a Deep Sparse Autoencoder
## 2 The Methodology
### Dictionary Learning Using Deep Sparse Autoencoder  
Based on the previously reviewed dictionary learning and stacked autoencoder models, a deep sparse autoencoder based dictionary learning is presented in this section. Like the structure of a deep autoencoder, the deep sparse autoencoder based dictionary learning can be illustrated as Figure 4 below. As shown in Figure 4, each dash block represents a shallow/single dictionary learning process. The first dictionary learning process can be written as:  
\[\mathbf{X}=\mathbf{D}_{1}\mathbf{S}_{1} \tag{8}\]  
where \(\mathbf{X}=\left\{\mathbf{x}_{i}\in\mathbb{R}^{d}\right\}_{i=1}^{N}\) stands for the set of \(N\) input signals, \(\mathbf{x}_{i}\) the signal vector with a length of \(d\), \(\mathbf{D}_{1}=\left\{\mathbf{d}_{1}^{1},\mathbf{d}_{1}^{2},\ldots,\mathbf{d}_{1}^{j},\ldots, \mathbf{d}_{1}^{K}\right\}\) for \(\mathbf{d}_{1}^{j}\in\mathbb{R}^{d}\) the first learnt dictionary, and \(\mathbf{S}_{1}=\left\{\mathbf{s}_{1}^{1},\mathbf{s}_{1}^{2},\ldots,\mathbf{s}_{1}^{g},\ldots, \mathbf{s}_{1}^{N}\right\}\) for \(\mathbf{s}_{1}^{g}\in\mathbb{R}^{K}\) the first latent expression of \(\mathbf{X}\) in \(\mathbf{D}_{1}\). Treating the deep autoencoder as a dictionary learning network, one can define \(\mathbf{D}_{1}^{\prime}=\left\{\mathbf{d}_{1}^{1},\mathbf{d}_{1}^{\prime 2},\ldots,\mathbf{d}_{1 }^{\prime p},\ldots,\mathbf{d}_{1}^{\prime K}\right\}\) for \(\mathbf{d}_{1}^{\prime p}\in\mathbb{R}^{d}\) as the reconstruction weight from latent expression \(\mathbf{S}_{1}\) to original input \(\mathbf{X}\). Like the expression in the autoencoder, the reconstructed input data \(\hat{\mathbf{X}}\) can be written as:  
\[\hat{\mathbf{X}}=f_{d}\big{[}\mathbf{D}_{1}^{\prime}f_{e}(\mathbf{S}_{1})\big{]} \tag{9}\]  
where \(f_{e}\) and \(f_{d}\) represent the encoding and decoding activation functions, respectively. Substitute \(\mathbf{S}_{1}\) in Equation (9) with Equation (8), \(\hat{\mathbf{X}}\) can be written as:  
\[\hat{\mathbf{X}}=f_{d}\left\{\left[\mathbf{D}_{1}^{\prime}f_{e}\Big{(}\mathbf{D}_{1}^{-1} \mathbf{X}\Big{)}\right]\right\} \tag{10}\]  
Here in this study, the activation function for both encoding and decoding processes are selected as sigmoid function. The cost function of dictionary learning using deep sparse autoencoder can be expressed as:  
\[\min_{D,D^{\prime}}\frac{1}{2N}\|\mathbf{X}-\hat{\mathbf{X}}\|_{2}^{2}+\beta\sum_{j=1 }^{J}KL(\rho||\hat{\rho}_{j}) \tag{11}\]  
\[KL(\rho||\hat{\rho}_{j})=\rho\log\frac{\rho}{\hat{\rho}_{j}}+(1-\rho)\log \frac{1-\rho}{1-\hat{\rho}_{j}} \tag{12}\]  
Figure 4: Scheme of stacked dictionary learning. \[\hat{\rho_{j}}=\frac{1}{N}\Big{[}1+\exp\Big{(}-\mathbf{D}^{-1}\mathbf{X}\Big{)}\Big{]}^{-1} \tag{13}\]  
where \(N\) represents the number of input vectors, \(\beta\) the parameter controlling the weight of the sparsity penalty term, \(\rho\) the sparsity parameter, \(\hat{\rho}_{j}\) the average activation of the hidden unit \(j\) over the all \(N\) training samples. The sparsity penalty term is defined as Kullback-Leibler (KL) divergence, which is used to measure the difference between two distributions. It is defined as \(\KL(\rho\big{|}|\hat{\rho}_{j})=0\) when \(\rho=\rho_{j}\), otherwise the KL divergence increases as \(\big{|}\rho-\rho_{j}\big{|}\) increases. In comparison with the similar \(k\)-sparse autoencoder proposed in [30], the advantages of the deep sparse autoencoder include: (1) The introduction of the sparsity penalty leads to the automatic determination of the sparsity rather than pre-defined as \(k\)-sparsity. It enables the deep sparse autoencoder to extract the sparse features more accurately based on the characteristics of the data. (2) The dictionary is learnt in the encoding procedure. The encoding dictionary is different from the encoding weight matrix. (3) The deep sparse autoencoders does not require the fine-tuning process while the performance of \(k\)-sparse autoencoders relies on the supervised fine-tuning process.