# Multimodal Tucker Decomposition for Gated RBM Inference
## 5 Materials and Methods
### Interpretation of the Refactored Model: Dimensionality Reduction  
Let us consider the three-way tensor with shape \((n_{i},n_{j},n_{k})\) and its corresponding Tucker decomposition presented in Figure 6. As we parametrize the weights of the three-way tensor \(\boldsymbol{\mathscr{W}}\) with its Tucker decomposition, we are now able to separate \(\boldsymbol{\mathscr{W}}\) into four components, each having a specific role in the gated RBM model. Matrices \(\mathbf{A}\) and \(\mathbf{B}\) project the input (\(\mathbf{x}\)) and output (\(\mathbf{y}\)) images into spaces of respective dimension \(n_{p}\) and \(n_{q}\). The core tensor \(\boldsymbol{\mathscr{G}}\), whose shape is \((n_{p},n_{q},n_{r})\), is used to model the interactions between the input and output image projections. Finally, the matrix \(\mathbf{C}\) projects the scores of the pair embedding \(\mathbf{h}\) into a space of dimension \(n_{r}\).  
Moreover, if \(\boldsymbol{\mathscr{G}}\) has the same shape as \(\boldsymbol{\mathscr{W}}\), the Tucker decomposition is simply a change of basis. However, in our case, we are interested in using a change of basis to compress \(\boldsymbol{\mathscr{W}}\). If \(n_{p}\), \(n_{q}\), \(n_{r}\) are smaller than \(n_{i}\), \(n_{j}\), \(n_{k}\), the core tensor \(\boldsymbol{\mathscr{G}}\) can be thought of as compressed version of \(\boldsymbol{\mathscr{W}}\). Note that the dimensions for the factor matrices \(\mathbf{A}\), \(\mathbf{B}\), and \(\mathbf{C}\) are a result of the \(n\)-mode product between the original tensor \(\boldsymbol{\mathscr{W}}\) and the core tensor \(\boldsymbol{\mathscr{G}}\). The factor matrix \(\mathbf{A}\) has dimensions \(n_{i}\times n_{p}\) as a result of the \(i\)-mode product between \(\boldsymbol{\mathscr{W}}\) and \(\boldsymbol{\mathscr{G}}\). Respectively, factor matrix \(\mathbf{B}\) has dimensions \(n_{j}\times n_{q}\) (from \(j\)-mode product) and factor matrix \(\mathbf{C}\) has dimensions \(n_{k}\times n_{r}\) (from \(k\)-mode product). By constraining \(n_{p}\), \(n_{q}\), \(n_{r}\) to be smaller than \(n_{i}\), \(n_{j}\), \(n_{k}\), we use a lower number of components for each of the three modes while at the same time linking these components to each other by means of the three-way core tensor.  
Again, consider the three-way tensor in Figure 6, whose respective cardinality of each layer is given by \(n_{i}\), \(n_{j}\) and \(n_{k}\). If we consider \(n_{i}\approx n_{j}\approx 2048\) and \(n_{k}\approx 2000\), then the number of free parameters in the tensor \(\boldsymbol{\mathscr{W}}\) is \(\sim\)8.39 \(\times 10^{9}\). It is easy to see that having such a number of free parameters is a problem both for memory and computing costs. In contrast, if we apply Tucker decomposition to this three-way tensor using a core tensor with shape \(1024\times 1024\times 1000\), then the number of free parameters would be \(\sim\)1.05 \(\times 10^{9}\), which is given by the sum of parameters from the core tensor and the three factor matrices. By applying Tucker decomposition we reduce the dimensionality of the model. Note that the compression in the data is determined by the ranks of the core tensor.  
Dimensionality reduction has long been an important technique for data representation. It reduces the space complexity of the underlying model so that it has higher stability when fitting, require fewer parameters and consequently becomes easier to interpret.  
Figure 6: Dimensionality reduction in the factorized tensor.