Venue: ICLR 2017 conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper proposes an RNN-based model for recommendation which takes into account temporal dynamics in user ratings and reviews. Interestingly, the model infers time-dependant user/item vectors by applying an RNN to previous histories. Those vectors are used to predict ratings, in a similar fashion to standard matrix factorization methods, and also bias a conditional RNN language model for reviews. The paper is well written and the architectural choices make sense. The main shortcomings of the paper are in the experiments:

1) The full model (rating+text) is only applied to one and relatively small dataset. Applying the model on multiple datasets with more data, e.g. Amazon reviews dataset (https://snap.stanford.edu/data/web-Amazon.html) would be more convincing.

2) While modelling order in review text seems like the right choice, previous papers (e.g. Almahairi et al. 2015) have shown that for rating prediction, modelling order in reviews might not be useful. A comparison with a similar model, but with bag-of-words reviews model would be nice in order to show the importance of the RNN-based review model, especially given previous literature.

Finally, this paper is an application paper applying well-established deep learning techniques, and I do not feel the paper offers new insights which the ICLR community in general would benefit from. This is not to undermine the importance of this paper, but I would like the authors to comment on why they think ICLR is a good avenue for their work.
review_title: Review

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper proposed a joint model for rate prediction and text generation.  The author compared the methods on a more realistic time based split setting, which requires “predict into the future.”
One major flaw of the paper is that it does not address the impact of BOW vs the RNN based text model, specifically RRN(rating+text) already uses RNN for text modeling, so it is unclear whether the improvement comes from RNN(as opposed to BOW) or application of text information. A more clear study on impact of each component could make it more clear and benefit the readers.
Another potential improvement direction of the paper is to support ranking objectives, as opposed to rate prediction, which is more realistic for recommendation settings.
The overall technique is intuitive and novel, but can be improved to give more insights to the reader,.




review_title: null

final_score: rating: null
review_content: 1) How are the input y_t of user RNN differ from the input of the movie RNN?
2) Can you provide how y_t is computed in detail? how are x_t, 1_{newbie}, \tau_t and \tau_{t-1} combined? How is \tau_{t} represented?
review_title: user and movie RNN inputs

final_score: rating: null
review_content: It would be appropriate to have a discussion on time complexity and how it might affect the serving performance. As from the description, this seems to be more time consuming that traditional factorization based models
review_title: time complexity

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper seeks to jointly model ratings and reviews in addition to temporal patterns. Existing papers have captured these aspects previously, but what's novel here is the particular combination of parts and choice of techniques. In particular, the use of LSTMs as opposed to "bag-of-words" models that have previously been used when combining the same components.

A criticism is made of existing models that use bag-of-words features as being too "coarse" to capture the real dynamics of reviews. This seems a valid criticsm, though it's not clear exactly what features those existing models miss. Something missing from this paper is any interpretation of what the model "learns" that may explain its better performance.

I also don't know about the significance of predicting "future" ratings as opposed to random splitting of the data. Lots of work on temporal recommender systems uses a variety of hold-out strategies besides random sampling, e.g. holding out the final ratings. Is there a methodological contribution associated with this change?

The experiments evaluate the extent to which the model achieves good performance due to its ability to capture evolving temporal patterns at the level of items, and the ability of the model to capture additional dynamics from text. Text makes a small but significant contribution; I was surprised by how large an improvement is achieved given how little text was included.

Overall this is a reasonably strong experimental comparison, though could be improved in two dimensions:
(a) There's no comparison to models that use ratings + text (e.g. CoBaFi, JMARS, etc.). These methods have reported substantial improvements over rating-only models in the past, perhaps even larger improvements than what is reported here. Such an experiment is important given the claim that existing models (in particular bag-of-words models) are too coarse to capture the dynamics of text.
(b) The restriction to movie datasets is okay, but these datasets are somewhat outliers when it comes to recommendation. In particular, they're extremely dense datasets that support very parameter-rich models. I'd question whether the results would hold on sparser data (though in fact I'd suspect they would hold, since on a sparser dataset the contribution due to using reviews ought to be larger).

Otherwise the experiments are fine. Perplexity results are nice but essentially what we'd expect. It is a shame that unlike other papers that use text to inform recommender systems there's no high-level analysis here of what the model has uncovered.

review_title: not sure why my review needs a title

final_score: rating: null
review_content: Given that you have a model that seems to be capable of generating reviews, what do the generated reviews look like? Are they qualitatively reasonable?

Other than generating reviews, several works use reviews to improve rating prediction in some way (including work by the authors), e.g. HFT, JMARS, CoBaFi, RMR, etc. How do these compare in terms of RMSE performance?

review_title: pre-review questions

Meta-review: The paper has some nice ideas, but requires a bit to push it over the acceptance threshold. I agree with the reviewers who ask for comparisons with other rating-review methods, and that other evaluation metrics more appropriate to the recommendation tasks should be reported. More analysis of the model, and the factors that contribute to its performance, would greatly improve the paper.
Decision: Reject