Venue: ICLR 2018 conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: The authors of this paper propose a new type of RNN architecture that modifies the reset gate of GRU with a rotational operator, where this rotational operator serves as an associative memory of their RNN model. The idea is sound, and the way they conduct experiments also make sense. The motivation and the details of the rotational memory are explained clearly. However, the experimental results reported in the paper seem to be a bit weak to support the claims made by the authors. 

The performance improvements are not so clear to me. Especially, in the character level language modeling, the BPC improvement is only 0.001 when choosing the SOTA model of this dataset as the base architecture. The test BPC score is obtained as a single-run experiment on the PTB dataset, and the improvement seems to be too small. In the copying memory task shown in Section 4.1, how did GORU performed when T=200? 

On the Q&A task, using the bAbI set (Section 4.3), RUM is said to be *significantly outperforming* GORU when the performance gap is 13.2%, and then, it is also said that RUM’s performance *is close to* the MeMN2N when the performance gap is 12.8%. Both performance gaps seem to be very close to each other, but the way they are interpreted in the paper is not.

Overall, the writing is clear, and the idea sounds interesting, but the experimental results are not strongly correlated with the claims made in the paper. In the visual analysis, the authors assume that RUM architecture might be the architecture that utilizes the full representational power of models like RNNs. If this is the case, I would expect to see more impressive improvements in the performance, assuming that all the other conditions are properly controlled.

I would suggest evaluating the model on more datasets. 

Minor comments: 
In Section 2.2: Hopflied -> Hopfield
In Section 3.2: I believe the dimension of b_t should be 2*N_h
review_title: Review

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper proposes a RNN memory cell updating using an orthogonal rotation operator. This approach falls into the phase-encoding architectures. Overall the author's idea of generating a rotation operator using the embedded input and the transformed hidden state at the previous step is clever. Modelling this way makes the 'generating matrix' W_hh learn to couple the input to the hidden state (which contain information in the past) via the Rotation operator.

I have several concerns:

- The author should discuss the intuition why the rotation has to be from the generated memory target τ to the embeded input ε but not the other way around or other direction in this 2D subspace.
- The description of parameter meter τ is not clear. Perhaps the author meant τ is the generated parameter via the parameter matrix W_hh acting upon the hidden state h_{t-1}
- The idea of evolving the hidden state by an orthogonal matrix, of which the rotation is a special case, is similar to the GORU paper, which directly parametrizes the 'rotation' matrix. Therefore I am wondering if the better performance of this work than the GORU is because of the difference in parameterization or by limiting the orthogonal transform to only rotations (hence modelling only the phase of the hidden state). Perhaps an additional experiment is needed to verify this.

review_title: Nice formulation of phase-coding memory for RNNs

final_score: rating: 5: Marginally below acceptance threshold
review_content: Summary:
This paper proposes a way to incorporate rotation memories into gated RNNs. They use a specific parametrization of the rotation matrices. They run experiments on several toy tasks and on language modelling with PTB character-level language modeling (which I would still consider to be toyish.)


Question:
Can the rotation proposed here cause unintentional forgetting by interleaving the memories? Because in some sense rotations are glorified summation in high-dimensions, if you do a full-rotation of a vector (360 degrees) you can end up in the same location. Thus the model might overwrite into its past memories.

Pros:
Proposes an interesting way to incorporate the rotation operations into the gated architectures.

Cons:
The specific choice of rotation operation is not very well justified.
This paper more or less uses the same architecture from Jing et al 2017 from EU-RNNs with a different parametrization for the rotation matrices.
The experiments are still limited to simple small-scale tasks.


General Comments:

The idea and the premise of this paper is interesting. In general the paper seems to be well-written. However the most important part of the paper section 3.1 is not very well justified. Why this particular parameterization of the rotation matrices is used and where does actually that come from? Can you point out to some citation? I think the RUM architecture section also requires better explanation on for instance why why R_t is parameterized that way (as a multiplicative function of R_{t-1}). A detailed ablation study would help too.

The model seems to perform really close to the GORU on Copying Task. I would be interested in seeing comparisons to GORU on “Associative Recall” as well. On QA task, which subset of bAbI dataset have you used? 1k or 10k training sets? 

On language modelling there is only insignificant difference between the FS-LSTM-2 with FS-RUM model. This does not tell us much.

review_title: On Rotational Unit of Memory

Meta-review: although the authors argue that their experiments were selected from the earlier work from which major comparing approaches were taken, the reviewers found the empirical result to be weak. why not some real tasks (i do not believe bAbI nor PTB could be considered real) that could clearly reveal the superiority of the proposed unit against existing ones?
Decision: Invite to Workshop Track