## 2 Methodology
### The Restricted Boltzmann Machine  
An RBM [29] is considered as a type of unsupervised machine learning method. It is a stochastic artificial neural network that learns a probability distribution over the set of its inputs. A RBM normally has two layers: a visible layer and a hidden layer. It can be represented by a bipartite graph that contains undirected edges from its two layers. Each layer contains a collection of neurons/nodes. Each neuron/node of the visible layer represents a feature of the input data, while neurons/nodes of the hidden layer represent the latent variables. A typical RBM structure is shown in Figure 1. The reason that an RBM is \"Restricted\" is because there are no connections between each neuron/node within either the visible or hidden layers. An RBM contains a matrix of weights \(W_{ij}\) representing the connection to visible node \(v_{i}\) and hidden node \(h_{j}\). In Figure 1, \(a_{i}\) represents the bias term in the visible layer, and \(b_{j}\) in the hidden layer.  
The weights and biases are computed by maximizing \(P(\mathbf{v})\), the probability that the network assigns to a visible vector \(\mathbf{v}\):  
\[P(\mathbf{v})=\frac{1}{Z}\sum_{\mathbf{h}}e^{-E(\mathbf{v},\mathbf{h})} \tag{1}\]  
where \(Z\) is the normalization constant that can be obtained by summing over all the possible pairs of visible and hidden vectors:  
\[Z=\sum_{\mathbf{v}}\sum_{\mathbf{h}}e^{-E(\mathbf{v},\mathbf{h})} \tag{2}\]  
and the energy function of the joint configuration \((\mathbf{v},\mathbf{h})\) is given by:  
\[E(\mathbf{v},\mathbf{h})=-\mathbf{a}^{T}\mathbf{v}-\mathbf{b}^{T}\mathbf{h}-\mathbf{v}^{T}\mathbf{W}\mathbf{h} \tag{3}\]  
Theoretically, the problem of maximizing Equation (1) can be solved by taking its partial log derivative with respect to its parameters \(\mathbf{W}\), \(\mathbf{a}\), and \(\mathbf{b}\):  
\[\frac{\partial\log(P(\mathbf{v}))}{\partial\mathbf{W},\mathbf{a},\mathbf{b}}=\sum_{\mathbf{v},\bm {h}}E(\mathbf{v},\mathbf{h})-\sum_{\mathbf{h}}E(\mathbf{v},\mathbf{h}) \tag{4}\]  
Normally Equation (4) can also be written as:  
\[\frac{\partial\log(P(\mathbf{v}))}{\partial\mathbf{W},\mathbf{a},\mathbf{b}}=<v_{i}h_{j}>_{ data}-<v_{i}h_{j}>_{model} \tag{5}\]  
Figure 1: A restricted Boltzmann machine.  
where \(<>\) denotes the expectation. However, the expectation \(<v_{i}h_{j}>_{model}\) in the maximum log likelihood function cannot be easily computed, and is thus estimated using contrastive divergence which leads to the following parameter updating equation [30]:  
\[\begin{array}{ll}W_{ij}{}^{k}&=W_{ij}{}^{k}+\gamma\left(<v_{i}^{k}h_{j}>_{data }-<v_{i}^{k}h_{j}>_{T}\right)\\ &=W_{ij}{}^{k}+\gamma\left[P\left(h_{j}=1\left|\begin{array}{c}\mathbf{v}\\ \end{array}\right)v_{i}-v_{i}^{k}P\left(h_{j}=1\left|\begin{array}{c}\mathbf{v} ^{k}\\ \end{array}\right)\right.\right]\\ a_{i}{}^{k}&=a_{i}{}^{k}+\gamma\left(<v_{i}^{k}>_{data}-<v_{i}^{k}>_{T}\right) \\ &=a_{i}{}^{k}+\gamma\left[P\left(v_{i}=1\left|\begin{array}{c}\mathbf{h}\\ \end{array}\right)-v_{i}^{k}\right]\\ b_{j}{}^{k}&=b_{j}{}^{k}+\gamma\left(<h_{j}>_{data}-<h_{j}^{k}>_{T}\right) \\ &=b_{j}{}^{k}+\gamma\left[P\left(h_{j}=1\left|\begin{array}{c}\mathbf{v}\\ \end{array}\right)-P\left(h_{j}=1\right|\begin{array}{c}\mathbf{v}^{k}\\ \end{array}\right)\right]\end{array} \tag{6}\]  
where \(T\) represents a full step of [PERSON] sampling, \(\gamma\) represents the learning rate and \(k\) represents the \(k-\) step of contrastive divergence. The neuron activation probabilities are given by the following equations:  
\[P(h_{j}=1\left|\begin{array}{c}\mathbf{v}\\ \end{array}\right)=\sigma(b_{j}+\sum_{i=1}^{n}W_{ij}v_{i}) \tag{7}\]  
\[P(v_{i}=1\left|\begin{array}{c}\mathbf{h}\\ \end{array}\right)=\sigma(a_{i}+\sum_{j=1}^{m}W_{ij}h_{j}) \tag{8}\]  
where \(n\) represents the number of visible units, \(m\) the number of hidden units, and \(\sigma\) is the activation function. The activation function is typically the logistic function used as a threshold defined as:  
\[\sigma(x)=\frac{1}{1+e^{-x}} \tag{9}\]