# A Sparse Autoencoder-Based Unsupervised Scheme for Pump Fault Detection and Isolation
## 2 Fault Detection and Isolation Scheme
### Offline Model Training
#### 2.1.2 Sparse Autoencoder  
The sparse autoencoders (SAE) is a special unsupervised feedforward neural network. A representative structure of SAE is presented in Figure 2. It can be observed that the SAE network includes two parts, i.e., encoder and decoder. The encoder connects the input layer to the hidden layer, with the weight matrix and the bias of this part being represented by \(W^{(1)}\) and \(b^{(1)}\), respectively. The decoder connects the hidden layer to the output layer, with the corresponding weight matrix \(W^{(2)}\) and bias \(b^{(2)}\).  
The network tries to reconstruct the input vector in the output layer, as shown in Equation (2).  
\[\widetilde{X}=H_{W,b}(X)\approx X \tag{2}\]  
where \(X\) denotes the input vector \(X\in R^{D\times 1}\), and \(\widetilde{x}\) is the output vector. \(H_{W,b}(X)\) is the nonlinear function of SAE, which predicts output \(\widetilde{X}\) based on the input \(X\), using parameters \(W\) and \(b\).  
The SAE is trained by minimizing the cost function [26] in Equation (3). In this paper, the L-BFGS algorithm and back propagation [26; 29] were employed to train the network for minimizing the cost function.  
\[E(W,b)=\{\frac{1}{D}\sum_{j=1}^{D}\big{[}\frac{1}{2}\|H_{W,b}\big{(}x_{j}\big{)} -x_{j}\|^{2}\big{]}\}+\frac{\lambda_{1}}{2}\sum_{l=1}^{L-1}\sum_{i=1}^{s_{l}} \sum_{j=1}^{s_{l+1}}\Big{(}W^{(l)}_{\bar{\mu}}\Big{)}^{2}+\beta\sum_{H=1}^{s_{ l}}KL(\rho\|\overline{\rho}_{H}) \tag{3}\]  
Figure 1: Diagram of proposed fault detection and isolation scheme.  
The first part of Equation (3) is an average sum-of-squares error term, which is used for minimizing the error between the input data \(x\) and the output data \(\overline{x}\). \(x_{j}\) represents the _jth_ training input variable, \(D\) is the total amount of the input variables. The decoder function of the _jth_ training variable is given by \(H_{W,b}\big{(}x_{j}\big{)}=f\big{(}W^{(2)}a+b^{(2)}\big{)}\), where \(a\) is the activation of hidden layer \(a=f(W^{(1)}x_{j}+b^{(1)})\), where \(W^{(1)}\), \(W^{(2)}\) are weights, \(b^{(1)}\), \(b^{(2)}\) are bias units, and \(f=~{}(1+\exp(-x))^{-1}\), is the logistic sigmoid function.  
The second part of the cost function is the regularization term, also called the weight decay term to avoid over-fitting, where \(\lambda_{1}\) is the weight decay factor, \(L\) is the number of layers in the network, and, in Figure 2, \(L=3\). \(s_{l}\) represents the number of units in layer \(l\), excluding the bias unit. Obviously, \(W^{(l)}_{ji}\) is the weight, which is associated with the connection between the _ith_ unit in layer \(l\) and the _jth_ unit in layer \(l+1\).  
The last part of the function is the sparse penalty term which imposes a sparsity constraint on the hidden units, where \(\beta\) is the weight of this term. \(s_{2}\) is the number of neurons in the hidden layer. \(KL(\rho\big{\|}\overline{\rho}_{H})\) measures how different two distributions are, and this term is given by Kullback-Leibler divergence function in Equation (4):  
\[KL(\rho\big{\|}\overline{\rho}_{H})=\rho log\frac{\rho}{\overline{\rho}_{H}}+ (1-\rho)log\frac{1-\rho}{1-\overline{\rho}_{H}} \tag{4}\]  
where \(\rho\) is the sparsity parameter which is usually small and close to zero, \(H\) is the number of hidden unit in the SAE, \(\overline{\rho}_{H}\) is the average activation of hidden unit \(j\) given by \(\overline{\rho}_{H}=\frac{1}{D}\sum_{j=1}^{D}\big{[}a_{H}^{(2)}x_{j}]\). \(a_{H}^{(2)}\) represents the activation of hidden unit \(H\) in SAE.