# Multimodal Tucker Decomposition for Gated RBM Inference
## 5 Materials and Methods
### Training the Refactored Gated RBM
Because of the conditional independences of \(\mathbf{h}\) given \(\mathbf{y}\) and \(\mathbf{h}\), \(\mathbf{y}\) given \(\mathbf{x}\) and \(\mathbf{h}\), and \(\mathbf{x}\) given \(\mathbf{y}\) and \(\mathbf{h}\), we can easily sample from the conditional distributions \(p(\tilde{\mathbf{h}}\mid\mathbf{x},\mathbf{y})\), \(p(\tilde{\mathbf{y}}\mid\mathbf{h},\mathbf{x})\), and \(p(\tilde{\mathbf{x}}\mid\mathbf{y},\mathbf{x})\) using (25)-(27) respectively. Given the tripartite structure of the gated RBM, it is possible to perform three-way alternating Gibbs sampling. This scheme of optimizing an undirected graphical model is known as Contrastive Divergence. In this research, we perform a single Gibbs iteration when approximating the negative phase. Using this Contrastive Divergence approach with Equations (25)-(27), we can make use of a machine learning library that supports reverse-mode automatic differentiation such as PyTorch. PyTorch provides two high-level features: tensor computing with strong acceleration via GPU and automatic differentiation for all operations on tensors. Conceptually, this method for automatic differentiation records a graph recording all of the operations that created the data as the operations are executed. The leaves in the graph are the input tensors, and the roots are the output tensors. PyTorch traces this graph from roots to leaves, automatically computing the gradients using the chain rule. From a computational point of view, training a model consists of two phases: a forward pass to compute the value of the loss function and a backward pass to compute the gradients of the learnable parameters. With this in mind, we use the Contrastive Divergence approach for building the forward pass and generating the graph. This process is summarized in Algorithm 1. ```
Input: \((x_{i},y_{i})\): Training pair; \(k\) number of steps for CD learning Output: \(\mathbf{x}^{k}\): input vector once CD-\(k\) is applied; \(\mathbf{y}^{k}\): output vector once CD-\(k\) is applied 1. Calculate positive phase performing three-way Gibbs sampling \(\mathbf{x}^{0}\gets x_{1}\) \(\mathbf{y}^{0}\gets y_{1}\)  Calculate \(p(\mathbf{\tilde{h}}^{0}\mid\mathbf{x}^{0},\mathbf{y}^{0})\) using Equation (25). \(\mathbf{h}^{0}\sim p(\mathbf{\tilde{h}}^{0}\mid\mathbf{x}^{0},\mathbf{y}^{0})\) \(\mathbf{h}^{k}\leftarrow\mathbf{h}^{0}\) 2. Calculate negative phase  For each step in \(k\): 1. Calculate \(p(\mathbf{\tilde{y}}^{k}\mid\mathbf{x}^{0},\mathbf{h}^{k})\) using Equation (26). Sample the \(\mathbf{y}^{k}\) states \(\mathbf{y}^{k}\sim p(\mathbf{\tilde{y}}^{k}\mid\mathbf{x}^{0},\mathbf{h}^{k})\) 2. Calculate \(p(\mathbf{\tilde{x}}^{k}\mid\mathbf{y}^{0},\mathbf{h}^{k})\) using Equation (27). Sample the \(\mathbf{x}^{k}\) states \(\mathbf{x}^{k}\sim p(\mathbf{\tilde{x}}^{k}\mid\mathbf{y}^{0},\mathbf{h}^{k})\) 3. Calculate \(p(\mathbf{\tilde{h}}^{k}\mid\mathbf{x}^{0},\mathbf{y}^{k})\) using Equation (25). Sample the \(\mathbf{h}^{k}\) states \(\mathbf{h}^{k}\sim p(\mathbf{\tilde{h}}^{k}\mid\mathbf{x}^{0},\mathbf{y}^{k})\)
```  
**Algorithm 1**Forward pass  
When computing the forward pass, PyTorch simultaneously performs the requested computations and builds up a graph representing the function that computes the gradient. Once the forward pass is completed, this graph is evaluated in the backward pass to compute the gradients. To build the backward pass, we used the concept of _free energy_ as presented in (9). Under the gated RBM, the probability of observing a configuration of output units \(\mathbf{y}\) given the input units \(\mathbf{x}\) can be obtained by marginalizing out the hidden units. This computation is called free energy and is given by:  
\[ \begin{split} p(\mathbf{y}\mid\mathbf{x})&=\sum_{ \mathbf{h}}p(\mathbf{y},\mathbf{h}\mid\mathbf{x})\\ &=\frac{1}{Z(\mathbf{x})}\sum_{\mathbf{h}}\exp\left(-E(\mathbf{y },\mathbf{h};\mathbf{x})\right)\\ &=\frac{1}{Z(\mathbf{x})}\sum_{\mathbf{h}}\exp\Big{(}\sum_{ijk}W_ {ijk}x_{i}y_{j}h_{k}\Big{)}\end{split}