# Continuous Semi-Supervised Nonnegative Matrix Factorization
## 3 Model
### Algorithm  
Our minimization approach is iterative and based on the alternating nonnegative least squares [16] approach. Due to the coupling of NMF and regression errors, other approaches such as multiplicative or additive updates [17] are less natural. Each iteration consists of: (1) holding \(H\) and \(\theta\) fixed while optimizing each row of \(W\) separately (nonnegative least squares); (2) holding \(W\) and \(\theta\) fixed while optimizing each column of \(H\) separately (nonnegative least squares); and finally (3) holding \(W\) and \(H\) fixed while optimizing over \(\theta\). The error is nondecreasing between iterations and from one optimization to the next. We now derive and justify this approach (Algorithm 1) in increasing complexity of cases. ```
Input :A matrix \(X\in\mathbb{R}_{\geq 0}^{n\times m}\), a vector \(Y\in\mathbb{R}^{n\times m}\), a positive integer \(r\in\mathbb{N}\), a scalar \(\lambda\geq 0\), a relative error tolerance \(\tau>0\), and a maximum number of iterations \(maxIter\). Output :Minimizers of Equations (6)-(9) : nonnegative matrix \(W\in\mathbb{R}_{\geq 0}^{n\times r}\), nonnegative matrix \(H\in\mathbb{R}_{\geq 0}^{r\times m}\), vector \(\theta\in\mathbb{R}^{(r+1)\times 1}\). \(relErr=\infty,err=\infty\) Elementwise, \(W\sim Unif([0,||X||_{\infty}))\), \(H\sim Unif([0,||X||_{\infty}))\), \(\theta\sim Unif([0,||X||_{\infty}))\). \(iter=0\)while\(relErr>\tau\)and\(iter<maxIter\)do \(W\gets newW\) as per Algorithm 2 \(H\gets newH\) as per Algorithm 3 \(\theta\gets new\theta\) as per Algorithm 4  Normalize \(W\), \(H\), and \(\theta\) as per Algorithm 5 errTemp \(=F^{(\lambda)}(W,H,\theta;X,Y)\)if\(err<\infty\)then \(relErr\leftarrow\left|err-errTemp\right|/err\) endif \(err\gets errTemp\) \(iter\gets iter+1\) endwhile return\(W,H,\theta\)
```  
**Algorithm 1**Overall CSSNMF algorithm. ```
Input :A matrix \(X\in\mathbb{R}_{>0}^{n\times m}\),  a vector \(Y\in\mathbb{R}^{n\times 1}\),  a matrix \(W\in\mathbb{R}_{\geq 0}^{n\times r}\),  a matrix \(H\in\mathbb{R}_{\geq 0}^{r\times m}\),  and a scalar \(\lambda\geq 0\). Output :A new value for \(W\). 1\(\bar{\theta}=(\theta_{2},\ldots,\theta_{r+1})^{\mathrm{T}}\)
2\(\bar{X}= \begin{bmatrix}X&|&\sqrt{\lambda}(\theta_{1}-Y)\end{bmatrix} \)
3\(\bar{W}= \begin{bmatrix}H&|&\sqrt{\lambda}\bar{\theta}\end{bmatrix} \)
4 for\(i\gets 1\ldots n\)do
5\(W_{i,:}\leftarrow\arg\min_{w\in\mathbb{R}_{\geq 0}^{1\times r}}||X_{i,:}-wH||^{2}\)
6 endfor return\(W\)
```  
**Algorithm 2**Updating \(W\). ```
Input :A matrix \(X\in\mathbb{R}_{>0}^{n\times m}\),  a matrix \(W\in\mathbb{R}_{\geq 0}^{n\times r}\),  and  a matrix \(H\in\mathbb{R}_{\geq 0}^{r\times m}\). Output :A new value for \(H\). 1 for\(j\gets 1\ldots m\)do
2\(H_{:,j}\leftarrow\arg\min_{h\in\mathbb{R}_{\geq 0}^{n\times 1}}||X_{:,j}-Wh|^{2}\)
3 endfor return\(H\)
```  
**Algorithm 3**Updating \(H\). ```
Input :A vector \(Y\in\mathbb{R}^{n\times 1}\),  a matrix \(W\in\mathbb{R}_{\geq 0}^{n\times r}\),  a matrix \(H\in\mathbb{R}_{\geq 0}^{r\times m}\),  and  a vector \(\theta\in\mathbb{R}^{(r+1)\times 1}\). Output :New values for \(W\), \(H\), and \(\theta\). 1\(S\in\mathbb{R}_{>0}^{r\times 1}\) a vector of row sums of \(H\). 2\(S\leftarrow\text{diag}(S)\)
3\(W\gets WS\). 4\(H\gets S^{-1}H\). 5\(\theta_{2:(r+1)}\gets S^{-1}\theta_{2:(r+1)}\). 6 return\(W\), \(H\), and \(\theta\). ```  
**Algorithm 4**Updating \(\theta\). ```
Input :A matrix \(W\in\mathbb{R}_{\geq 0}^{n\times r}\),  a matrix \(H\in\mathbb{R}_{\geq 0}^{r\times m}\),  a vector \(\theta\in\mathbb{R}^{(r+1)\times 1}\). Output :New values for \(W\), \(H\), and \(\theta\). 1\(S\in\mathbb{R}_{>0}^{r\times 1}\) a vector of row sums of \(H\). 2\(S\leftarrow\text{diag}(S)\)
3\(W\gets WS\). 4\(H\gets S^{-1}H\). 5\(\theta_{2:(r+1)}\gets S^{-1}\theta_{2:(r+1)}\). 6 return\(W\), \(H\), and \(\theta\). ```  
**Algorithm 5**Normalization process. ```
Input :A matrix \(W\in\mathbb{R}_{\geq 0}^{n\times r}\),  a matrix \(H\in\mathbb{R}_{\geq 0}^{r\times m}\),  a vector \(\theta\in\mathbb{R}^{(r+1)\times 1}\).