# Deep Learning Based Lithology Classification Using Dual-Frequency Pol-SAR Data
## 2 Materials and Methods
### Stacked Sparse Autoencoder
#### 2.2.1 Sparse Autoencoder  
A sparse autoencoder is a feedforward, fully connected, nonrecurrent neural network with an input layer, a hidden layer, and an output layer. The output layer has the same number of nodes as the input layer, and the hidden layer has more nodes than the input layer [24; 25]. The sparse autoencoder self-learns so that the output get close to the input with a teacher vector identical to the input vector. This means that, the sparse autoencoder is able to extract an optimized feature set from the input data. The action of the sparse autoencoder is described as follows:  
Step 1: Initialize the weight matrix and the bias vector, and calculate the outputs of the hidden layer and output layer:  
\[y=f\left(W^{(1)}x+b^{(1)}\right) \tag{13}\]  
\[\tilde{x}=g\left(W^{(2)}y+b^{(2)}\right), \tag{14}\]where y and \(\tilde{x}\) are the outputs of the hidden layer and output layer, respectively. \(f\left(x\right)\) and \(g\left(x\right)\) are the activation functions, and \(f\left(x\right)=\frac{1}{1+\exp\left(-x\right)},g\left(x\right)=x\). \(W^{\left(1\right)}\) represents the weight matrix between the input layer and the hidden layer. \(W^{\left(2\right)}\) is the weight matrix between the hidden layer and the output layer. \(b^{\left(1\right)}\) and \(b^{\left(2\right)}\) are the bias vectors of the hidden and output layers, respectively.  
Step 2: Calculate the loss function based on outputs. The loss function used in this paper has the following form [18]:  
\[J\left(W,b\right)=\frac{1}{2m}\sum_{i=1}^{m}\left\|\tilde{x}-x\right\|^{2}+ \frac{\lambda}{2}\sum_{l=1}^{2}\sum_{i=1}^{2}\sum_{j=1}^{s_{l+1}}\left(W_{ji}^ {\left(l\right)}\right)^{2}+\beta\sum_{i=1}^{s_{2}}KL\left(\rho\left\|\hat{ \rho}_{i}\right.\right), \tag{15}\]  
where \(m\) is the number of training data points. \(s^{l}\) (\(l=1,2,3\)) represents the number of neurons in the input layer, the hidden layer, and the output layer, respectively. The first term on the right side of Equation (15) is the traditional squared error, which stands for the difference between the output and the input. The second one is the regularization penalty which can penalize large values of the parameters to get a simpler network. This can prevent network overfitting. The third part is the sparse penalty term to improve the sparsity of the encoding representation and the Kullback-Leibler divergence \(KL\left(\rho\left\|\hat{\rho}_{i}\right.\right)\) is defined in Equation (16):  
\[KL\left(\rho\left\|\hat{\rho}_{i}\right.\right)=\rho\log\frac{\rho}{\hat{\rho} _{i}}+\left(1-\rho\right)\log\frac{1-\rho}{1-\hat{\rho}_{i}}, \tag{16}\]  
where \(\hat{\rho}_{i}\) is the average activation of the \(i_{th}\) hidden unit over the training set. \(\rho\) is the sparse parameter which is a small value close to zero. The constraint \(\hat{\rho}_{j}=\rho\) is enforced during the training. If the constraint is satisfied, the hidden activities are mostly close to zero, which ensures that the hidden layer can be forced to learn a compressed representation.  
Step 3: Minimize the loss function to optimize \(W^{\left(l\right)}\) and \(b^{\left(l\right)}\).  
It can be seen from step 2 that to get a satisfying network, we need to minimize the loss function by optimizing \(W^{\left(l\right)}\) and \(b^{\left(l\right)}\) (\(l=1,2\)). A stochastic gradient descent algorithm [26] is applied to minimize the loss function, and the process is as follow:  
\[W^{\left(l\right)}\gets W^{\left(l\right)}-\gamma\frac{\partial}{ \partial W^{\left(l\right)}}J\left(W,b\right) \tag{17}\]  
\[b^{\left(l\right)}\gets b^{\left(l\right)}-\gamma\frac{\partial}{ \partial b^{\left(l\right)}}J\left(W,b\right), \tag{18}\]  
where \(\gamma\) is the learning rate.