# Guided Semi-Supervised Non-Negative Matrix Factorization
## 3 Proposed Method  
Recall that SSNMF is able to classify different documents through given label information, while Guided NMF can guide the content of generated topics via a priori seed words. We propose a more general model, Guided Semi-Supervised NMF (GSSNMF) that can leverage both label information and important seed words to improve performance in both multi-label classification and topic modeling. Heuristically, we see that, for classification, the user-specified seed words aid SSNMF in distinguishing between each class label and thus improve classification accuracy. For topic modeling, the known label information enables Guided NMF to better cluster similar documents, improving topic coherence and interpretability. Our algorithm combines these and seeks to approximately solve the optimization problem  
\[\operatorname*{argmin}_{\mathbf{W}\in\mathbb{R}_{\geq 0}^{d\times k},\mathbf{H}\in \mathbb{R}_{\geq 0}^{k\times s},\mathbf{B}\in\mathbb{R}_{\geq 0}^{k\times s},\mathbf{C} \in\mathbb{R}_{\geq 0}^{p\times k}}\underbrace{\frac{1}{2}\|\mathbf{X}-\mathbf{W}\mathbf{H}\|_{F}^{2}}_{ \text{classical NMF}}+\underbrace{\frac{\lambda}{2}\|\mathbf{Y}-\mathbf{W}\mathbf{B}\|_{F}^ {2}}_{\text{guiding}}+\underbrace{\frac{\mu}{2}\|\mathbf{L}\odot(\mathbf{Z}-\mathbf{C}\bm {H})\|_{F}^{2}}_{\text{label information}}, \tag{6}\]where matrices \(\mathbf{W}\), \(\mathbf{H}\), \(\mathbf{Y}\), \(\mathbf{B}\), \(\mathbf{L}\), \(\mathbf{Z}\), and \(\mathbf{C}\) and constants \(\lambda\) and \(\mu\) can be interpreted in the same way as in SSNMF and Guided NMF. Note that, if we simply set either \(\lambda\) or \(\mu\) to be 0, GSSNMF reduces to SSNMF or Guided NMF, respectively.  
To solve (6), we propose an algorithm that utilizes a multiplicative update scheme akin to those in [11; 17]. In a manner similar to gradient descent, at each step, each of the four matrices \(\mathbf{W},\mathbf{H},\mathbf{B},\mathbf{C}\) are updated in turn, while viewing the other three as fixed. The derivation of these updates is provided in Appendix A, and the updating process is presented in Algorithm 1. Note that, as with other multiplicative update methods for NMF variants, the loss is monotone [2].  
```
Input:\(\mathbf{X}\in\mathbb{R}_{\geq 0}^{d\times n}\), \(\mathbf{Y}\in\mathbb{R}_{\geq 0}^{d\times s}\), \(\mathbf{L}\in\mathbb{R}_{\geq 0}^{p\times n}\), \(\mathbf{Z}\in\mathbb{R}_{\geq 0}^{p\times n}\), \(k\), \(\lambda\), \(\mu\), \(N\)
1 Initialize \(\mathbf{W}\in\mathbb{R}_{\geq 0}^{d\times k}\), \(\mathbf{H}\in\mathbb{R}_{\geq 0}^{k\times n}\), \(\mathbf{B}\in\mathbb{R}_{\geq 0}^{k\times s}\), \(\mathbf{C}\in\mathbb{R}_{\geq 0}^{p\times k}\)
2 for\(i=1,\ldots,N\)do
3\(\mathbf{W}\leftarrow\mathbf{W}\odot\frac{\mathbf{X}\mathbf{H}^{i}+\lambda\mathbf{X}\mathbf{B}^{\top}} {\mathbf{W}\mathbf{H}^{i}+\lambda\mathbf{W}\mathbf{B}^{\top}}\)
4\(\mathbf{H}\leftarrow\mathbf{H}\odot\frac{\mathbf{W}^{\top}\mathbf{X}+\mathbf{\mu}\odot(\mathbf{L}\odot \mathbf{Z})}{\mathbf{W}^{\top}\mathbf{W}\mathbf{H}+\mathbf{\mu}\mathbf{C}^{\top}(\mathbf{L}\odot\mathbf{C}\mathbf{ H})}\)
5\(\mathbf{B}\leftarrow\mathbf{B}\odot\frac{\mathbf{W}^{\top}\mathbf{Y}}{\mathbf{W}^{\top}\mathbf{W}\mathbf{B}}\)
6\(\mathbf{C}\leftarrow\mathbf{C}\odot\frac{(\mathbf{L}\odot\mathbf{Z})\mathbf{H}^{\top}}{(\mathbf{L} \odot\mathbf{C}\mathbf{H})\mathbf{H}^{\top}}\)
7 end for Output:\(\mathbf{W},\mathbf{H},\mathbf{B},\mathbf{C}\)
```  
**Algorithm 1**GSSNMF with multiplicative updates.  
We will demonstrate the strength of GSSNMF by considering different combinations of the parameters \(\lambda\) and \(\mu\), and comparing the implementations of this method with Classical NMF, SSNMF, Guided NMF, and TS-NMF through real-life applications in the following section.