## 2 Basic Theory of Non-Negative Tensor Factorization  
### Alternating Least Squares Algorithm for NTF  
A tensor, in a broad sense, is a multi-dimensional matrix whose modes or ways represent the number of dimensions. A third order tensor (or three-way tensor) \(\underline{\mathbf{Y}}\in\mathbb{R}^{l\times\mathbb{X}\times\mathbb{Q}}\) can be regarded as a combination of the frontal slices \(\mathbf{Y}_{q}\in\mathbb{R}^{l\times\mathbb{T}}\), \((q=1,2,\ldots,Q)\). Thus, the third order NTF model is transformed into a matrix factorization model by unfolding (matricizing) the tensor, as given by  
\[\begin{array}{c}\mathbf{Y}_{q}=\mathbf{A}\mathbf{D}_{q}\mathbf{X}_{q}+ \mathbf{E}_{q},(q=1,2,\ldots,Q),\\ \text{s.t.}\ \mathbf{A},\mathbf{D}_{q},\mathbf{X}_{q}>0,\end{array} \tag{1}\]  
where \(\mathbf{D}_{q}\in\mathbb{R}^{l\times\mathbb{J}}\) are diagonal scaling matrices and \(\mathbf{X}_{q}\) are source matrices representing latent components. Index \(J\) is a preset value denoting the reduced dimensionality. By absorbing \(\mathbf{D}_{q}\) into \(\mathbf{X}_{q}\), it is equivalent to give Equation (2) with scaled source matrices:  
\[\mathbf{Y}_{q}=\mathbf{A}\mathbf{X}_{q}+\mathbf{E}_{q} \tag{2}\]  
The hybrid strategy to solve the above NTF model minimizes a set of cost functions based on the squared Euclidean distance [19]. With ALS algorithm under Karush-Kuhn-Tucker (KKT) optimal conditions, the factor matrix \(\mathbf{A}\) is estimated by fixing matrix \(\overline{\mathbf{X}}\in\mathbb{R}^{l\times\mathbb{T}\mathbb{Q}}\) to solve the minimization problem:  
\[\begin{array}{c}\text{min}\ D_{F}(\overline{\mathbf{Y}}\|\mathbf{A} \overline{\mathbf{X}})=\frac{1}{2}\|\overline{\mathbf{Y}}-\mathbf{A} \overline{\mathbf{X}}\|_{F}^{2}=\frac{1}{2}tr(\overline{\mathbf{Y}}-\mathbf{A} \overline{\mathbf{X}})^{T}(\overline{\mathbf{Y}}-\mathbf{A}\overline{\mathbf{X }}),\\ \text{s.t.}\ \
abla_{\mathbf{A}}D_{F}(\overline{\mathbf{Y}}\|\mathbf{A} \overline{\mathbf{X}})=[-\overline{\mathbf{X}}\overline{\mathbf{X}}^{T}+ \mathbf{A}\overline{\mathbf{X}}\overline{\mathbf{X}}^{T}]=0,\end{array} \tag{3}\]  
where \(\overline{\mathbf{Y}}\in\mathbb{R}^{l\times\mathbb{T}\mathbb{Q}}\) is the row-wise unfolding matrix of tensor \(\underline{\mathbf{Y}}\), and \(\overline{\mathbf{X}}\in\mathbb{R}^{l\times\mathbb{T}\mathbb{Q}}\) remains fixed and is equal to \([\mathbf{X}_{1},\mathbf{X}_{2},\ldots,\mathbf{X}_{\mathbb{Q}}]\). Alternatively, the source matrices \(\mathbf{X}_{q}\) are found with fixed \(\mathbf{A}\) by  
\[\text{min}\ D_{F}(\mathbf{Y}_{q}\|\mathbf{A}\mathbf{X}_{q})=\frac{1}{2}\| \mathbf{Y}_{q}-\mathbf{A}\mathbf{X}_{q}\|_{F}^{2},(q=1,2,\ldots,Q). \tag{4}\]  
In practice, sparsity constraints are additionally introduced to the factors \(\mathbf{X}_{q}\) and \(\mathbf{A}\) to improve the ability of feature representation. Thus, we further solve the sparse NTF problems:  
\[\text{min}D_{F}(\overline{\mathbf{Y}}\|\mathbf{A}\overline{\mathbf{X}})= \frac{1}{2}\|\overline{\mathbf{Y}}-\mathbf{A}\overline{\mathbf{X}}\|_{F}^{2}+ \alpha_{\mathbf{A}}\|\mathbf{A}\|_{1}, \tag{5}\]\[\min D_{F}(\mathbf{Y}_{q}\|\mathbf{A}\mathbf{X}_{q})=\frac{1}{2}\|\mathbf{Y}_{q}- \mathbf{A}\mathbf{X}_{q}\|_{F}^{2}+\alpha\mathbf{x}_{q}\|\mathbf{X}_{q}\|_{1}, (q=1,2,\ldots,Q), \tag{6}\]  
where \(\alpha_{\mathbf{A}}\) and \(\alpha\mathbf{x}_{q}\) are the sparseness regulation coefficients. The stationary points of the cost functions are determined by resorting to the KKT conditions, and the update rules are obtained:  
\[\mathbf{X}_{q}\leftarrow\bigg{[}(\mathbf{X}_{q}\mathbf{X}_{q}^{T})^{-1}( \mathbf{A}^{T}\mathbf{Y}_{q}-\alpha\mathbf{x}_{q}1_{I\times T})\bigg{]}_{+}, \tag{7}\]  
\[\mathbf{A}\leftarrow\bigg{[}(\overline{\mathbf{X}}^{T}-\alpha_{\mathbf{A}}1_ {I\times I})(\overline{\mathbf{X}}^{T})^{-1}\bigg{]}_{+} \tag{8}\]