Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a linked NMF based clustering to tackle the problem of linked dataset. The authors shows that chained NMF provides new way to incorporate information and interpret results in comparison to the multi view clustering approaches which are limited to the same entities across tables (or different views of the same entity). Experimentally, the authors show that chained NMF perform better than vanilla NMF at clustering Newsgroups. 
main_review: While the idea of incorporating star-schema information is not new, this paper proposes a new approach to model the non-multi-view information. There have been several works in heterogeneous networks such as NetClus [1] and CESNA [2] where the heterogeneous information sources (feature tables) have been clustered. It is not clear how the chained NMF modeling improves over the existing clustering models in the prior literature. 

I believe more work needs to be done to understand the convergence of the algorithm, the time and space complexity analysis. Further, more real-world applications should be tried like heterogeneous database-based tasks to understand the utility of chained NMF modeling.  


[1] Sun, Yizhou, Yintao Yu, and Jiawei Han. "Ranking-based clustering of heterogeneous information networks with star network schema." Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining. 2009.
[2] Yang, Jaewon, Julian McAuley, and Jure Leskovec. "Community detection in networks with node attributes." 2013 IEEE 13th international conference on data mining. IEEE, 2013.
summary_of_the_review: While this paper provides novel idea, it lacks some comparison and justification in comparison to the existing literature. More work needs to be done to justify the modeling choice. Also, more experimental work needs to be done to justify the utility of the above model. In light of the above observations, I have made my recommendation of the paper.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper studied clustering of data that has multiple dimensions of relation. It proposed a chained non-negative matrix factorization (NMF) technique, which, according to the authors, is different from existing multiple-view NMF. Chained NMF can model data that has chained relations (much like external key in relational database) and produce better clusters.

The authors evaluated Chained NMF with synthetic data that seems to resemble documents with hashtags and authors. NMI and ARI are both improved. Another illustrative evaluation was done on handwriting characters data.  

main_review: Strength
- The authors studied the ease of real-world application. They discussed the viability of using commonly available auto-differentiation tools (JAX).
- The use of two different kinds of dataset to test the algorithm: handwriting and documents.

Weakness
- Writing needs substantial improvements. It's generally hard to follow the paper. For example there are sudden turns of topic (Abstract: "..it is commonplace to identify hashtag ..."), and very easy to spot errors (Introduction, 2nd paragraph, "A multiNMF is where the primary key ..." is repeated).
- Section 2.1. The use of hashtag and words is sudden, Reviewer expect some introduction on the task: recommendation or prediction?
- Equation 1, what is V?
- Section 4.3. "We attained the following results" but the authors continued to discuss experiment set up, not the results.
- There are many other similar cases of writing inconsistency. Reviewer won't list them all.

summary_of_the_review: The paper discussed a way to cluster a dataset with multiple chainable joining keys. The authors provided mathematical discussion and evaluation on a small synthetic dataset. However, the paper itself is poorly written, to a point that it's hard for Reviewer to understand how the authors arrived at the main idea and what its strength and weakness are. Clarity is the main weakness.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper presents a new technique, grounded in non-negative matrix factorization (NMF), for unsupervised, representation learning of linked data. The paper presents a technique that can deal with data scenarios whereby a certain phenomenon of interest has multiple views to characterize that behavior (i.e. traditional multi-view clustering set up), but also the data in those views can be linked to further data, as in a relational data. 
main_review: __Strong points__: Overall, the paper’s strengths are the technical novelty of the proposed chain-view NMF and the impact of the problem it is addressing.

- The paper presents a new technique for dealing with a real-world data problem that is straightforward in its technical grounding. The extensions the paper proposes to NMF are both conceptually straightforward and a rather elegant way of attacking the data scenario of linked and multi-view data
- The problem of finding good representations of phenomena, when those phenomena can be described by both multi-view and linked data, is a significant problem. Being able to successfully do this task can lead to better outcomes in fundamental machine learning tasks like clustering.

__Weak points__: Overall, the paper’s weaknesses are in its clarity and its empirical significance. 

- The paper has some serious clarity issues that detract from the reader being able to understand the paper and contributions of the paper. The paper needs to be reviewed for grammar and typos, as they are frequent and do detract from understanding the text. One example is at the bottom of page 1, the second paragraph of the introduction, wherein the sentence “A multiNMF is where…” is repeated. Another example is in section 2.1, where it looks like from the text that “Hashtags _X_ tf-idf …” implies that that matrix should be $X$, but it is instead $V$ in the subsequent equation.
Also, within the proposed contribution of section 2.3, while it is clear how to deal with the situation of linked data (by multiplying those $V_*$ matrices into the second part of the loss function, it's not clear how to deal with multi-view data (where you have two or more data sets, or tables, with the same row entities). Do you just add an extra term (same as the original term, but with the W being the same across terms) to the loss function for this other table? If so, how do you control for the different impacts of the added together parts of the loss function (i.e. would you weight the different sub-components of the loss function)? This also leads to the intriguing idea of how one should model their data relationships to get the best cluster structure, especially when given the design choice of whether to make a certain table of data chained or multi-view. Also, the intuition idea of chaining the matrices by the cartesian product should be explained. While it certainly makes the matrix math work out, why is that the best way to propagate information from those matrices?
Figure 2, while a neat image, really doesn’t seem to contribute much actual information to the paper, since the visual inspection isn’t stated as an empirical evaluation test. This space would be better used for either more explanation on the empirical setup (more on that later) or doing more empirical tests.

- The empirical investigations are unclear as to how the experiments were set up and evaluated For example, what clustering was done for the AMI and ARI values? And, why is the synthetic test set up different than the real-world data set up? It seems like both should share the same test setup. For example, why does Table 2 look at NMI and ARI, while Table 1 does not and why does Table 1 consider various different ways of chaining and Table 2 does not? Finally, the word vectors in 4.3, is truncated the negative values really the best way to make sure those vectors are positive? Won’t this cause a loss of critical information, since they are embeddings in a real-valued space (implying negative values contain information)? Perhaps a better way would be to do a transformation of them into a positive space by scaling and/or normalization.
The empirical investigations, from Table 2, really do not show much performance improvement over the KL-optimized standard NMF, especially when one considers the chained version requires so many more matrix operations than just a single view NMF. 

- Finally, the paper overstates its claim about the proposed technique being a novel paradigm in artificial intelligence. The idea of using more than one view of data to understand phenomena is the foundation of multi-view machine learning, and the idea of using linked data underlies techniques in Heterogeneous Information Networks and complex network clustering. So, while it is a new technique, the paper does not present a new paradigm.

summary_of_the_review: I recommend the paper be accepted. While it does need some serious work in clarity and maybe some additional empirical investigation, it does present a new technique that is attacking a meaningful problem.
review_title: null

Meta-review: The paper proposes a new approach for linked-view clustering based on chained non-negative matrix factorization. Reviewers highlighted that paper proposes a novel and interesting approach to an important problem. However, reviewers raised also significant concerns regarding clarity of presentation (motivation, general approach, contributions, scope) as well as the experimental evaluation. Reviewers raised also concerns regarding justification of the approach being a novel paradigm. After author response and discussion, all reviewers and the AC agree that the paper is not yet ready for publication due to the aforementioned issues.
Decision: Reject