# Data Compression Based on Stacked RBM-AE Model for Wireless Sensor Networks
## 2 Stacked RBM-AE Model
### Details of Stacked RBM-AE Training  
#### 2.2.1 Pre-training of standard RBM  
For the standard RBM, \(\mathbf{v}\) denotes the activation probability of all visual units, and \(\mathbf{h}\) denotes the activation probability of all hidden layer units. The model parameter \(\theta\) contains three parameters: \(\mathbf{W}\), \(\mathbf{b}\) and \(\mathbf{c}\). \(\mathbf{v}\) and \(\mathbf{h}\) are the binary column vectors of \(1\times n\) and \(1\times m\), respectively; \(\mathbf{W}\) is the real-valued matrix of \(n\times m\), which represents the weights of the units connected; \(\mathbf{b}\) is the bias of visual layer units in \(n\times 1\) size; \(\mathbf{c}\) is the bias of the hidden layer units in \(m\times 1\) size. We use \(W_{ij}\) to represent the weights between \(v_{i}\) and \(h_{j}\). \(b_{i}\) represents the bias of \(v_{i}\). \(c_{j}\) represents the bias of \(h_{j}\). In the pre-training phase, we use CD algorithm [32] to train four standard RBMs of different sizes in turn. Algorithm 1 shows the procedure of the pre-training of standard RBM. It first calculates the activation probability of all hidden layer units and the activation probability of all visual layer units, and then updates model parameters by minimizing the model loss. ```
Input: A mini-batch of training data set \(\mathbf{S}\), the number of train iteration iter, the learning rate \(\alpha\) Initialization: \(\mathbf{W},\mathbf{b},\mathbf{c}=0\) While\(i<iter\)do  The hidden layer units activation probability \(\mathbf{h}=sigmoid(\mathbf{c}+\mathbf{S}\mathbf{W})\)  Get \(\mathbf{h}^{(1)}\) by one step Gibbs sampling for \(\mathbf{h}\)  The visible layer units activation probability \(\mathbf{v}=sigmoid\left(\mathbf{b}+\mathbf{W}\mathbf{h}^{(1)}\right)\)  Update parameters \(\mathbf{W},\mathbf{b},\mathbf{c}\) according to (7)-(9) \(i++\) end
```  
**Algorithm 1**Pre-training of standard RBM  
Figure 3: Stacked RBM-AE. For the standard RBM, the activation state of each hidden layer unit is conditionally independent for a given state of the visible layer unit. So, we can get the activation probability of \(h_{j}\):  
\[\mathbf{P}\big{(}h_{j}=1\big{|}\mathbf{v}\big{)}=sigmoid\Bigg{(}c_{j}+\sum_{i}v_{i}W_{ ij}\Bigg{)} \tag{1}\]  
Similarly, when a state of a hidden layer unit is given, the activation probability of the visual layer unit is also conditionally independent:  
\[\mathbf{P}(v_{i}=1\big{|}\mathbf{h})=sigmoid\Bigg{(}b_{i}+\sum_{j}W_{ij}h_{j}\Bigg{)} \tag{2}\]  
We define the energy function as:  
\[\mathbf{E}(\mathbf{v},\mathbf{h}|\theta)=-\sum_{i=1}^{n}\sum_{j=1}^{m}v_{i}W_{ij}h_{j}-\sum _{i=1}^{n}v_{i}b_{i}-\sum_{j=1}^{m}h_{j}c_{j} \tag{3}\]  
The energy function means that there is an energy value between each visual layer unit and each hidden layer unit. By indexing and regularizing the energy function, we get the joint probability of \(\mathbf{v}\) and \(\mathbf{h}\):  
\[\mathbf{P}(\mathbf{v},\mathbf{h}|\theta)=\frac{\exp(-\mathbf{E}(\mathbf{v},\mathbf{h}|\theta))}{\sum _{\alpha,k}\exp(-\mathbf{E}(\mathbf{v},\mathbf{h}|\theta))}\]  
\[=\frac{\exp\left(\sum_{i=1}^{n}\sum_{j=1}^{m}v_{i}W_{ij}h_{j}+\sum_{i=1}^{n}v_ {i}b_{i}+\sum_{j=1}^{m}h_{j}c_{j}\right)}{\sum_{\alpha,k}\exp\left(\sum_{i=1}^ {n}\sum_{j=1}^{m}v_{i}W_{ij}h_{j}+\sum_{i=1}^{n}v_{i}b_{i}+\sum_{j=1}^{m}h_{j }c_{j}\right)} \tag{4}\]  
We use the likelihood function derivation method to get the solution of parameters. The joint probability distribution \(\mathbf{P}(\mathbf{v},\mathbf{h}|\theta)\) is known.