# Guided Semi-Supervised Non-Negative Matrix Factorization
## 4 Experiments
### Topic Modeling on the CIP Dataset  
In this section, we test the performance of GSSNMF for topic modeling by comparing it with Classical NMF, Guided NMF, and TS-NMF on the CIP AOB dataset. Specifically, we conduct experiments for the range of rank identified in Section 4.1, running tests for various values of \(\lambda\) and \(\mu\) for each rank. To measure the effectiveness of the topics discovered by Guided NMF and GSSNMF, we calculate the _topic coherence score_ defined in [21] for each topic. The coherence score \(\mathcal{C}_{i}\) for each topic \(i\) with \(N\) most probable keywords is given by  
\[\mathcal{C}_{i}=\sum_{b=2}^{N}\sum_{\ell=1}^{b-1}\log\frac{P(w_{b}^{(i)},w_{ \ell}^{(i)})+1}{P(w_{\ell}^{(i)})}. \tag{8}\]  
In the above Equation (8), \(P(w)\) denotes the _document frequency_ of keyword \(w\), which is calculated by counting the number of documents that keyword \(w\) appears in at least once. \(P(w,w^{\prime})\) denotes the _co-document frequency_ of keyword \(w\) and \(w^{\prime}\), which is obtained by counting the number of documents that contain both \(w\) and \(w^{\prime}\). In general, the topic coherence score seeks to measure how well the keywords that define a topic make sense as a whole to a human expert, providing a means for consistent interpretation of accuracy in topic generation. A large positive \(\mathcal{C}\) coherence score indicates that the keywords from a topic are highly cohesive, as judged by a human expert. Since we are judging the performance of methods that generate multiple topics, we calculate coherence scores \(\mathcal{C}\) for each topic that is generated by Guided NMF or GSSNMF and then take the average. Thus, our final measure of performance for each method is the _averaged coherence score_\(\mathcal{C}_{\text{avg}}\):  
\[\mathcal{C}_{\text{avg}}=\frac{\sum_{i=1}^{k}C_{i}}{k}, \tag{9}\]  
where \(k\) is the number of topics (or rank) we have specified. Figure 3: Difference between Reconstructed Labels using SSNMF and GSSNMF and Actual Crime Label matrices (\(\mu=0.0011,\lambda=0.0007\)), a light pixel indicates correct label assignment, while a dark pixel indicates otherwise. As suggested by Figure 1, a proper rank falls between 6 and 9. Starting with the generation of 6 topics from the AOB dataset, we first find a range of \(\lambda\) in which Guided NMF generates the highest mean \(\mathcal{C}_{\text{avg}}\) over 10 independent trials. In our computations, we use the top 30 keywords of each topic to generate each coherence score; then, for each trial, we obtain an individual \(\mathcal{C}_{\text{avg}}\), allowing us to average the 10 \(\mathcal{C}_{\text{avg}}\) from the 10 trials into the mean \(\mathcal{C}_{\text{avg}}\). Based on the proper range of \(\lambda\), we then choose a range of \(\mu\) for our GSSNMF to incorporate the label information into topic generation. Again, for each pair of \((\lambda,\mu)\), we run 10 independent trials of GSSNMF and calculate \(\mathcal{C}_{\text{avg}}\) for each trial to generate a mean \(\mathcal{C}_{\text{avg}}\). With these ranges in mind, we work towards the following goal: for a given \"best\" \(\lambda\) of Guided NMF, we improve topic generation performance by implementing GSSNMF with a \"best\" \(\mu\) that balances how much weight GSSNMF should place on the new information of predetermined labels for each document. We then repeat the same process for ranks 7, 8, and 9, and plot the mean \(\mathcal{C}_{\text{avg}}\) against each \(\lambda\) in Figure 4. The corresponding choice of \(\mu\) can be found in Appendix B. We can see that, most of the time, for a given \(\lambda\), we are able to find such \(\mu\) that GSSNMF can generate a higher mean \(\mathcal{C}_{\text{avg}}\) than Guided NMF in topic modeling across various ranks. Ultimately, we also see that a GSSNMF result always outperforms even the highest-performing Guided NMF result.