# Data Compression Based on Stacked RBM-AE Model for Wireless Sensor Networks
## 2 Stacked RBM-AE Model
### Details of Stacked RBM-AE Training
#### 2.2.2 Retraining of Stacked RBM-AE  
We initialize the Stacked RBM-AE model by using pre-trained standard RBM parameters, but the pre-trained parameters are only applicable to standard RBM, and we do not achieve good performance on the Stacked RBM-AE model. Therefore, we use the pre-trained parameters to retrain the Stacked RBM-AE model and use the BP algorithm [35] to fine tune the model parameters. Algorithm 2 shows the procedure of the retraining of Stacked RBM-AE. It first calculates the encoder output \(\mathbf{e}\) and the decoder output, and then calculates the model loss according to Equation (13), finally updating model parameters. The encoder output and the decoder output are calculated by the Stacked RBM.  
```
Input: A mini-batch of training data set \(\mathbf{S}\), the number of train iterations iter, the learning rate \(\alpha\), the regularization parameter \(\lambda\) Initialization: \(\mathbf{W},\mathbf{b},\mathbf{c}\) from the pre-trained standard RBM parameters While\(i<iter\)do  The encoder output \(\mathbf{e}\) according to (10)  Get \(\mathbf{e}^{(1)}\) by one step Gibbs sampling for \(\mathbf{e}\)  The decoder output \(\mathbf{d}\) according to (11)  The model loss \(\mathbf{L}\) according to (13)  Update parameters \(\mathbf{W},\mathbf{b},\mathbf{c}\) according to (7)-(9) \(i++\) end
```  
**Algorithm 2**Retraining of Stacked RBM-AE  
The encoder output of the Stacked RBM-AE model is calculated by four different sizes of RBMs in turn, as shown in Figure 2. We use \(\mathbf{W}_{i}\), \(\mathbf{c}_{i}\) and \(\mathbf{b}_{i}\) to represent the parameters of the \(i\) layer standard RBM in the model. The encoder output is calculated by:  
\[E(\mathbf{h})=\prod_{i=1}^{4}{sigmoid\left(\mathbf{b}_{i}+\sum\mathbf{W}_{i} \mathbf{h}_{i}\right)} \tag{10}\]  
where \(\mathbf{h}_{i}\) is the activation probability of all hidden layer units in the \(i_{th}\) RBM. Similarly, the decoder output is calculated by:  
\[E(\mathbf{v})=\prod_{i=1}^{4}{sigmoid\left(\mathbf{c}_{i}+\sum\mathbf{v}_{i} \mathbf{W}_{i}\right)} \tag{11}\]  
where \(\mathbf{v}_{i}\) is the activation probability of all visual layer units in the \(i_{th}\) RBM. For the input data normalized to [0,1], they will be given the probability meaning that each neural unit value of the Stacked RBM-AE model still obeys the Bernoulli distribution of [0,1]. The normalized real value of the operation is considered to be the probability that the current unit has a value of 1. Then for a single training sample, the objective or cost function that minimizes the cross entropy is:  
\[loss\left(f\left(\mathbf{v}^{(t)}\right)\right)=-\sum_{j=1}^{m}\left(v_{j}^{( t)}\log\left(v_{j}^{(t)re}\right)+\left(1-v_{j}^{(t)}\right)\log\left(1-v_{j}^{(t)re} \right)\right) \tag{12}\]For the training dataset, the loss of the Stacked RBM-AE model is defined as:  
\[\begin{split} loss(\mathbf{\theta})=-\frac{1}{T}\left[\sum\limits_{t=1}^{T }\left(\sum\limits_{j=1}^{m}\left(v_{j}^{(t)}\log\left(v_{j}^{(t)re}\right)+ \left(1-v_{j}^{(t)}\right)\log\left(1-v_{j}^{(t)re}\right)\right)\right)\right]\\ +\frac{\lambda}{2}\sum\limits_{l=1}^{lay-1}\sum\limits_{t=1}^{s_ {l}}\sum\limits_{t=1}^{s_{l+1}}\left(W_{(tla)(tl)}^{l}\right)^{2}\end{split} \tag{13}\]  
where \(l\) is layer number. \(sl\) represents the unit numbers of \(l_{th}\) layer, \(tl\) and \(tla\) represents the current layer number and the next layer number. The second term is the weights penalty item, which is used to prevent over-fitting. \(\lambda\) is the regularization parameter in the penalty item.