Venue: ICLR 2018 Workshop

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: The authors study the application of fast weights to LSTM cells. Fast weights have been shown to improve the memory capacity and time scale of RNNs. In this study the authors study whether fast weights can help gated RNNs like LSTM cells. The conclusion of this work is that fast weights is also beneficial to LSTMs and results in much faster training and lower test error.
review_title: Study on using fast weights on LSTMs

final_score: rating: 5: Marginally below acceptance threshold
review_content: This submission explores whether the formulation of attention proposed in "Using Fast Weights to Attend to the Recent Past" paper is applicable to LSTM. The associative memory bit is applied to the proposed input (often denoted by "j") before it's gated ("i") and added to the gated state ("f*c_{t-1}").

The experiments are done on a simple associative retrieval task and a new, harder variant of the same. The results indicate that the fast weight LSTM improves on both the plain LSTM and the fast weight RNN. It is unclear how trustworthy these results are since the experimental setup (especially hyperparameter tuning) is not described in much detail.

The proposed model is simple, the experiments are a good start, but even for a workshop paper, I feel more convincing ones are necessary.
review_title: Applying "Fast Weight to Attend to the Recent Past" to LSTMs

final_score: rating: 5: Marginally below acceptance threshold
review_content: Summary: This paper proposes to incorporate fast weights to the LSTMs. In a way the proposed method resembles WeiNet's construction by Zhang&Zho which is in turn inspired by the Ba et al's work on fast weights. The authors test their model on associative retrieval tasks and they show better results than FW-RNN and LN-LSTM. Overall, the paper is well-written.

Criticism: 
- The results are quite encouraging, however it would have been much more convincing if they have shown some results on non-toyish tasks. It is difficult to say if their results would hold on more realistic tasks.
- The comparisons against other memory models such as DNCs or LSTMS are missing.
- Concept is interesting, but the novelty of the model in this paper is not that big. This would not be an issue if they had results on a real world task.
review_title: Incorporating Fast Weights into the LSTMs

Meta-review: Based on the reviews, this paper has not been accepted for presentation at the ICLR workshop. However, the conversation and updates can continue to appear here on OpenReview.
Decision: Reject