# Continuous Semi-Supervised Nonnegative Matrix Factorization
## 3 Model
### Algorithm
See Figure 4. We note that, for large enough \(\lambda\), the testing error outperforms the testing error for \(\lambda=0\). The optimal point was at \(\lambda=10^{-2/3}\approx 0.215\). Figure 3: Regression errors at various noise levels \(\eta\) with varying regression weight \(\lambda\). The true number of topics is \(r=4\). The top row, (**a**–**c**), depicts the training errors, and the bottom row, (**d**–**f**), depicts the testing errors. The first column, (**a**,**d**), is for a low rank approximation \(r=3\), the second column, (**b**,**e**), is for an approximation of correct rank \(r=4\), and the third column, (**c**,**f**), is for when the number of topics used \(r=5\) is larger than the true number of topics. For each \(\lambda\), \(\eta\), and \(r\), fifty trials were run, and the regression errors corresponding to the best overall objective function \(F^{(\lambda)}\) were recorded. Points with \(\lambda>0\) for which the regression error exceeds 1.5 times the regression error at \(\lambda=0\) are not displayed. The dashed vertical line is the transition point between a linear and logarithmic \(x\)-scale. We comment that it is generally difficult to know precisely where the testing error will be minimized, only that, based on observations of the synthetic data, the testing error is often better than the \(\lambda=0\) case after the training error has dropped. We speculate that the level of noise in this dataset results in the testing errors not dropping below \(\approx 0.75\). ### Prediction

We examine the rating prediction by plotting histograms of predicted ratings where the true ratings were in \([1,\,2],[2,\,3],[3,\,4]\), and \([4,\,5]\)--the closed intervals are used here. Figure 5 depicts these histograms along with the mean predicted rating and true rating. The predictions are often within range, and the mean predicted values are very close to the true means over each interval. We can also see the general predictive strength in the scatterplot of actual vs. predicted ratings in Figure 6.

These results suggest the model is able to identify topics and associated \(\theta\)-weights so as to generate predictions that are consistent with true ratings. For example, in the case where ratings are in \([1,\,2]\), we see the peak of the predictions is around \(2\), not exceeding \(4\), with some predictions as low as \(-2\); then, in the case of ratings in \([4,\,5]\), the model peaks around \(3.5\) and makes some predictions above \(7\). There is a clear capacity for the topics to shift the predictions. Figure 4: Errors in training and validation on Rate My Professor dataset with \(r=11\) topics. Points with \(\lambda>0\) for which the regression error exceeds \(1.5\) times the regression error at \(\lambda=0\) are not displayed. The dashed vertical line is the transition point between a linear and logarithmic \(x\)-scale. ### Topics Identified

It is important that the method not only have predictive power, but also produce interpretable topics. We now look at the 11 topics with their associated \(\theta\)-weights. We find

\[\theta=(2.39909812,2.82948873,-2.21028471,1.83876976,-4.77504984,\\ -3.86467795,3.46353642,0.03914383,3.26619842,-5.51595505,\\ 4.15317532,3.90733652)^{\mathrm{T}}.\]

Note that \(\theta_{1}\approx 2.4\) suggests that for a set of reviews with no topics, the average rating would be around 2.4--this suggests it is the presence of positive/negative topics that raise/lower the rating. Figure 5: Histograms of the predicted mean rating for various ranges of true ratings ([1, 2] in (a), [2, 3] in (b), [3, 4] in (c), and [4, 5] in (d)). The vertical dashed lines represent the mean values. The predicted and true means are as follows: 2.206 and 1.543 for ratings in [1, 2], 3.233 and 2.529 for ratings in [2, 3], 3.594 and 3.593 for ratings in [3, 4] (the lines are indistinguishable), and 4.576 and 4.494 for ratings in [4, 5]. Figure 6: Scatterplot of Rate My Professor ratings: actual mean rating vs. predicted mean rating.