Venue: ICLR 2017 conference

Reviews:

final_score: rating: null
review_content: I worry that this method is very dependent on the density of the dataset. Netflix is very dense, and Ciao is sub-sampled to be dense (I didn't spot the total number of interactions in citeUlike, and therefore its density, can you add this?).

Can you comment on how density affects the results or better yet how the Ciao results would look without subsampling?
review_title: pre-review questions

final_score: rating: null
review_content: Please see the review
review_title: Please see the review

final_score: rating: 5: Marginally below acceptance threshold
review_content: The authors proposed to learn embeddings of users and items by using deep neural network for a recommendation task. The resulting method has only minor differences from the previous CDL, in which neural networks were also used for recommendation tasks. In the experiments, since the proposed method, DualNets have use more item features than WMF and CDL, the comparisons are unfair. 
review_title: null

final_score: rating: null
review_content: Since recall@N is used as metric. It makes sense to do a negative sampling based approach baseline, as opposed to rate prediction models. 

E.g. Bayesian personalized recommendation 
review_title: baseline with negative sampling based approach

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper provides a minor improvement paper of DeepRS. The major improvement comes from the coupling of user-item factors in prediction. While the motivation is clear, the improvement of the model architecture is minor. 
I think the author should improve the paper to discuss more on the impact of introduction of coupling, which might make this paper stronger. Specifically, conduct isolate experiment to change loss, architecture gradually, from a non-coupled network to a final proposed coupled network to demonstrate the importance of coupling.
Another important missing part of the paper seems to be time complexity. Since coupled net would be much more costly to generate recommendations, a discussion on how it would impact real world usages should be added.
Overall, I think this is a paper that should be improved before accepted.

review_title: review for Collaborative Deep Embedding via Dual Networks

final_score: rating: 4: Ok but not good enough - rejection
review_content: The responses to the pre-review questions are not strong; especially w.r.t. the question about dataset density and why the dataset had to be subsampled, the authors responded that subsampling is common in recommender systems work, including the papers cited. This isn't a particularly strong justification of why subsampling is a good idea, and in particular doesn't answer the question of "how would the results look without subsampling," which I think is a question that could easily have been answered directly.

Especially given that the goal of dealing with the cold-start issue is so heavily emphasized in the paper, in seems odd to sample the data to reduce sparsity.

Other than that, the pre-review questions seem to have been answered satisfactorily.

The contribution of the paper is to propose user and item embedding methods, as a means of learning complex non-linear interactions between users and items. This is fairly similar to recent work on deep RS, though the network formulation has some differences.

Overall this is an reasonably put together paper that makes a contribution in an important area, though there are still some shortcomings that should be addressed, namely:
1) The evaluation is unusual. Recall@M is the only result reported, though this is not usually an evaluation seen in recommender systems research. At the very least other performance measures (rmse or AUC) should be reported for completeness, even if the results are not strong
2) Given that the contribution is fairly simple (i.e., the "standard" recommender systems task, but with a new model) it's a shame that unusual data samples have to be taken. This should be a case where it's possible to report results against competing methods using *exactly* the same data they used, and exactly the same error measure, for the fairest comparison possible.

Without the above it's hard to tell how much the performance improvements are really due to the method being better, versus the choice of datasets and the choice of loss functions.

review_title: official review

Meta-review: The paper presents a collaborative filtering method, using dual deep nets for users and items. The nets can take advantage of content in addition to ratings. This contribution is just below the bar, in that its novelty relative to existing methods is limited and the results are good but not sufficiently impressive, especially since they focus exclusively on Recall@N. In the response, the authors do present results on other metrics but the results are mixed.
Decision: Reject