Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper presents an inductive matrix completion model using graph neural networks. The proposed method assumes the rating between a user-item pair is determined by this pair's surrounding sub-graph via a neural network. It claims to be an inductive model and don't need any side information as it only uses the surrounding sub-graph structure to give predictions. The experiments also show the promising results on matrix completion tasks, sparse rating matrix cases and transfer learning tasks

The paper is well written and easy to follow. I think the "cold-start" setting this method is mainly focusing on is worth exploring. In this setting, new users have given a few ratings but the quality of their side information is still low. It seems to me that this setting is common in real case. For example, newly registered Netflix users may quickly watch some videos without completing their profile information. However, when the "new" users have given some ratings, we can re-train a traditional MC model including the new ratings. But I believe the proposed method of this paper will be useful when re-training cannot be done very frequently. 

I have the following concerns or questions.

1. The paper claims the proposed method doesn't need any side information. However, it seems that the side information can be integrated into the model such as concatenating with the one-hot encoding. Have you tried to use the side information for IGMC?

2. The claims in Section 4 don't make much sense to me. For example, it doesn't seem to me that GC-MC can't distinguish Figure (a) and (b) because GC-MC will take the whole graph into consideration. Also, why will node-based approaches push all nodes to have similar embeddings with multiple layers? Could you give some experimental verifications? 

3. It's unclear to me how Eq. (6) helps the performance. It will be better to compare a version without using L_{ARR}. Moreover, I believe Eq. (6) can be designed better. 

4. GC-MC uses bilinear decoder to give probability predictions for different ratings. Is there any particular reason why you choose squared loss as in Eq. (5) instead of a bilinear decoder? 

5. I have a big concern for the scalability. If we want to make recommendations for a user from a million movies, the proposed method seems to need to compute Eq. (4) for one million times. Is there any approximate way to speedup it? 

Overall, I think the proposed method is interesting and the experimental results are impressive. 

review_title: Official Blind Review #3

final_score: rating: 8: Accept
review_content: The paper uses a new approach for inductive matrix completion (IMC). The IMC problem is used to model recommender systems where there can be users/items with few (or even no) observed ratings, but where each user/item is associated with additional meta-data or other features. Since users/items close to each other in the feature space plausibly share similar ratings, one can inductively fill in the ratings even in the very sparse regime where traditional matrix completion fails.

The current paper demonstrates an interesting new approach for IMC that does not require any metadata about new users/items, but rather, exploits the structure of the observed ratings themselves. Specifically, the method (called Inductive Graph-based Matrix Completion, or IGMC) constructs a (fairly large) bipartite graph out of the observed user-item pairs, and creates subgraphs by looking at h-hop neighborhoods of each test node. These subgraphs are fed as features into a graph convolutional neural network (appropriately designed) that predicts any unknown rating.

The paper is overall well written. The approach (of using subgraph connectivity patterns as features) is clever, novel (at least in my knowledge) and neatly sidesteps the need for extra metadata/features. The numerical results are also thorough and compelling. 

Minor comments:
- Isn't Alg 1 just ordinary bread-first search? Perhaps I missed something.
- In Table 2 why do some methods have error bars and others don't?
- Fig 3 is not very informative.



review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: This paper presents a method for inductive matrix completion that does not rely on side information to make predictions. The approach is as follows: select the sub-graph from an h-hop neighbourhood of any target user-item pair (u,v) and relabel it according to its distance from (u, v). The authors then treat rating prediction as graph classification problem given the selected subgraph and use a graph convolution architecture to make the prediction. They achieve strong empirical performance in both the transductive and inductive settings. 

The paper presents an interesting approach to the inductive learning problem, with a number of interesting ideas and impressive empirical results. However, I have voted to reject this paper because I found the experimental section did a poor job of showing how the various ideas contribute to overall performance which made it difficult to evaluate the method beyond the absolute performance on benchmark datasets. 

The key idea of the paper is interesting - samples h-hop enclosing subgraphs and label the resulting nodes by their hop-length and type (i.e. users at hop length h get a different label to items at length h).  In their experiments they find that a model with only a 1-hop neighbourhood to performs so well, which suggests that for recommender systems, supporting long chains of dependencies is unnecessary. This is a useful property of the data that the method depends on, because without it, h-hop neighbourhoods with larger h would quickly include the whole graph on most datasets (most real-world graphs have small diameter).

They process these subgraphs with relational graph convolutions, but make two modifications: first they build their final representations by concatenating all the message passing steps (eq. 2) and both user and item representations (eq. 3), and second they have adjacent rating regularization which constrains weights between adjacent ratings. Eq.2 in particular is unusual because it  amounts to a skip connection from every layer (somewhat analogous to a DenseNet [Huang et al. 2015]). The authors say both lead to improved performance - but don’t evaluate the approaches in the experimental section. What is the contribution of eq 2, 3 and 6 relative to more standard approaches (i.e. no skip connections, sum pooling, no regularization)?

Finally - I found the positioning of the work of the work misleading: from the abstract and related worked we're told that this is a "seemingly impossible problem" and prior work has relied on side information to deal with the inductive case. From this description, we might infer that this is the first approach that deals with the no-side information case. But the clear baseline is Hartford et al. 2018 which also doesn't use side information for inductive matrix completion; and yet it is not mentioned in either the introduction or the related work sections. This omission is unfortunate because both because it's misleading, and more importantly it misses the opportunity to discuss the relative merits of the approaches (beyond performance, where this paper provides a clear improvement). For example,
 - How do the two methods compare with respect to computational complexity? This approach seems more expensive because a different subgraph is selected for each prediction (i.e. 1 forward pass per test set point rather than a single forward pass to compute the full test set), but this isn't discussed. 
 - Can you give any intuition about where the differences in performance are coming from?

------

Increased rating to Weak Accept following the rebuttal below.
review_title: Official Blind Review #2

Meta-review: This paper proposes a novel technique for matrix completion, using graphical neighborhood structure to side-step the need for any side-information.

Post-rebuttal, the reviewers converged on a unanimous decision to accept. The authors are encouraged to review to address reviewer comments.
Decision: Accept (Spotlight)