Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: Main problem: The paper tries to tackle the problem of deciding how to make jumpy predictions or more precisely event based updates in a RNN as compared to clock based updated. Normal RNNs/LSTMs update their hidden states at every time step. Predicting the hidden states at each time step often lead to compounding errors i.e., small errors accumulate while making predictions for many time steps in the future. I really like this problem.

Method: The paper proposes a method to update hidden states in an event driven manner. The paper defines the hidden state as a continuous, piece wise linear function. The paper proposes to predict the jump interval as well a hidden velocity which basically signifies the "change" in hidden dynamics over the jump interval, thus the update function of the proposed method is a function of both the jump interval as well as hidden velocity.  In order to encourage the jump_interval of more than 1, the paper defines an auxiliary parameter such that the prediction loss b/w the actual input and predicted input is less than \epsilon, where epsilon is a hyper-parameter. 

Strong points:

- I really like the underlying idea, irrespective of the results.
- I like the preliminary results in figure 3 which basically justifies the core behind the proposed method.
- I also like the idea, and preliminary results for jumpy planning. it was also studied in [a, b].

Questions:

1.  A main drawback of the proposed method is the reliance on the \epsilon parameter. In many cases, it would be the case that their are multiple entities in the environment, like for ex 2 bouncing balls as compared to one, and then the model has to adaptively decide as to for which part of the input, the model should skip the update. So, I'm curious as to how the model with perform when their are multiple bouncing balls (or multiple entities in the env.) against each other.

2. It would also be interesting to study the generalization performance of the proposed method. Does the ability to skip update gives the ability to generalize better to out of distribution examples.

3.  "Finally, we vary the dimensionality of our model’s hidden state and retrain. In figure 4, we find a positive
relationship between model capacity and jumpiness" . The paper mention that they find a positive correlation b/w the model capacity and jumpiness.  Does this behaviour exists in all the datasets which the paper explore ?

4. Related work: I think, many of the important references are missing. There are various important references. Like there's some work done where the idea is to learn how to skip updates in RNN like in SkipRNN (c), or in Adaptive Skip Intervals (d), or in the context of RIMs (e), where each module decides whether to update it's hidden state or not update their hidden state. There's also some work where the idea is to learn event based representations like in PhaseLSTM (f). It would be nice to see how the proposed method compares to any of these methods.

References:

- (a) Time-Agnostic Prediction: Predicting Predictable Video Frames, https://arxiv.org/abs/1808.07784
- (b). InfoBot, https://arxiv.org/abs/1901.10902
- (c). Skip RNN: Learning to Skip State Updates in Recurrent Neural Networks, https://arxiv.org/abs/1708.06834
- (d). Adaptive Skip Intervals: Temporal Abstraction for Recurrent Dynamical Models, https://arxiv.org/abs/1808.04768
- (e). Recurrent Independent Mechanisms, https://arxiv.org/abs/1909.10893 
- (f). PhaseLSTM, https://papers.nips.cc/paper/6310-phased-lstm-accelerating-recurrent-network-training-for-long-or-event-based-sequences.pdf

5. I'm not sure what's the take away message from the paper as of now. In the current form the paper shows computational benefits i.e., the proposed method can achieve similar results, by updating dynamically based on events, and hence establishes a trade-off between the temporal resolution of the input and the computational expense. The results for planning are also very interesting. So, one suggestion could be to do extensive experiments and formulate the entire paper in the context of planning because as of now, the bouncing ball results for test mse are not better as compared to the baseline. 

=======

After Rebuttal: I have read the rebuttal, as well as reviews by other reviewers. I really like the idea, but its important to evaluate the idea with respect to a downstream task to get a better idea on how to use the learned structure. 


review_title: Jumpy RNNs

final_score: rating: 7: Good paper, accept
review_content: ## Paper Summary

This paper proposes a recurrent network architecture for future prediction where the hidden states (and the outputs) aren't updated step by step as done traditionally. Instead, the network models the hidden state dynamics as being piecewise linear over varying time spans. It learns to produce the linear dynamics together with each time span, and can "jump" to the next time span according to its own predictions. Hidden states for any time step within a span can be easily obtained by interpolation using the predicted linear dynamics. Experiments on a series of synthetic benchmarks are used to demonstrate that the model can learn to utilize this structure to reduce the amount of computation.

## Strengths

- The proposed approach is simple and intuitive. The study is likely to be valuable to researchers in sequence modeling, by showing that these simple ideas can work surprisingly well. One might expect that the non-stationarity of the "jump size" targets might make training such models too difficult, but the authors show that other than requiring a couple of tricks (setting $\epsilon$ using a baseline and "jump bootstrapping"), training works well.

- The paper is an enjoyable read due to its clarity and presentation. The claims made are modest and clear without over-reaching, the experiments use tasks that are directly motivated by the goals of the study, and the related work is clearly and fairly discussed.

## Weaknesses

- A clear weakness of the paper is that all experiments were conducted on synthetic tasks of low complexity. Although these experiments were illustrative and helpful, it is unclear if the approach works reasonably well on more realistic problems. One "failure" mode could be that the Jumpy RNN tends to jump almost every step for noisy and real-world data. At this stage this study does not really say whether this architecture is also likely to work well on such problems, but is more of a proof of concept.

## Review Summary

I think the strengths of this paper outweigh the weaknesses. While the experiments were not conducted on common benchmarks or real-world data, I think this paper is well-written as a proof of concept, does not overclaim and should motivate further work on similar ideas. My recommendation is to accept.

## Questions for Authors

1. Have you tried more complex datasets or RL problems? Are there any negative results you can report?
2. Have you found the non-stationarity of the targets to be issue for training under any conditions?

## Post-rebuttal

I thank the authors for their hard work, and for incorporating my suggestions into the paper. I believe the paper has improved.
review_title: Well-written paper with simple and interesting ideas

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes Jumpy Recurrent Neural Network, an RNN model with non-uniform time steps. To train this model, the authors propose to use a greedy supervision to determine optimal time intervals. The experiments on linear dynamics prediction and planning show comparable performance of proposed model against standard RNNs. The advantage is that the proposed model can significantly speed up RNNs under relatively linear dynamics prediction tasks. Even though non-uniform time step RNNs have been studied in many literature, this proposed training supervision method seems novel.

+ves: 

+ Overall, the paper is well written. In particular, the proposed model and its training methods are clearly explained.

+ The results section is well structured. The hyperparameters, architecture, and experimental settings are well demonstrated. 

 
Concerns: 

- The proposed method, non-uniform time interval RNN, is not novel. For example, Che et al 2018, Lipton et al 2016, have very similar contributions. This paper needs to clarify the difference.

 
- The proposed greedy training method seems have significant flaw. Clearly it will have Delta=1 at the beginning of the training. Then it can only improve corresponding Delta one by one at each timestep. The loss corresponding to Delta is highly non-continuous and it will definitely not learn to find the real optimal Delta.


- The model does not seem to learn meaningful Delta. See Figure 3 and 4. For such clean data, the model should learn to predict only turning points but it seems it just predict random intervals. Also, there are no verification of learning meaningful Delta.

 
- The experiment is clearly not fair for a standard RNN. Since Delta=1 is always true for standard RNN, clearly it will be slower. The speedup ratio will only be the average of predicted Delta in the proposed model. However, what if the RNN just sets a larger Delta, what if the RNN just set a random predicted Delta? 

 
- The last experiment is not convincing. First, it's a simple supervised task, not real "model-based planning". Second, Figure 6 seems not converged. No experimental detail is given for this task. Third, twice as fast mean the model generally just predict Delta=2.

 
- All the experiments have internal bias towards linear dynamics. This favors the model with corresponding inductive bias. To prove the model is useful, I would recommend to run standard RNN synthetic task like copying task, adding task etc, to prove its basic functionality.

 
Minor comments: 

* Figure 1, in basic RNN, it should be an arrow instead of a line from h_i to GRU.

* Equation 3, people usually don't write output as x, otherwise it can only be a sequence prediction task like language modeling. Please write it y. Also, it creates confusion to equation 7 whether it's input or output.

* The "Jump Bootstrap" seems very important technique for the proposed method. But there's no experiment or discussion on it.

=====POST-REBUTTAL COMMENTS========

I thank the detailed response from the authors. The authors addressed the novelty of this paper. The experimental results on toy tasks are convincing. However, the way this method increases Delta still seems very problematic to me and not seem robust in complex real world cases.

I increased my score.
review_title: Unconvincing experiments and lack of novelty

final_score: rating: 5: Marginally below acceptance threshold
review_content: Summary: This work presents Jumpy RNNs, a recurrent network that learns to take variable length steps based on time-scales of the data. The core idea of the paper is to learn a hidden velocity and time span, along with the standard hidden state. The hidden velocity is then used to linearly interpolate the hidden state within the learned time span.  This leads the proposed model to update the hidden state only at the end of time span thus allowing the model to jump over by certain steps proportional to time span. The model has flexibility to produce fine-grained or continuous-time predictions as well as predicting far into the future. Since, Jumpy RNNs do not update the hidden state at every time step, they are computationally efficient than standard RNNs. 

Positives:
1. The idea of building a model to provide short term and long term prediction using just one hidden state update is interesting. The problem addressed is important in time series predictions in many domains.
2. The paper is well written and easy to follow.
3. Experiments show that the model works well on simulated datasets.

Concerns:
1. The key concern about the paper is the lack of rigorous experimentation to study the usefulness of the proposed method. The current experiments are mostly based on simulations and are too simple to evaluate the effectiveness of the proposed model. In almost all the cases, the rate of change is constant with few sharp changes. In particular, it would be interesting to see how the model performs on real world datasets, where the hidden state dynamics is unknown, for short-term and long-term time series predictions.
2. Despite the paper stating that there have been some recent work on Neural ODEs (Chen et al., 2018), the paper does not compare with them. Neural ODEs are one of the possible candidates for solving the given problem and it would be interesting to see how fast/slow they are compared to the proposed model.
3. The linear-dynamics of hidden state is too simple and can only handle constant hidden state dynamics or the one with constant slope. Although the authors mention that the hidden-state linearity does not translate to linearity in output space as the learned decoder can be an arbitrarily complex non-linear function, but the experiments are only based on constant or linear hidden state/output dynamics.
4. What's the difference between Test MSE and Sample MSE? Based on the results, standard RNNs achieve lower MSE scores than the proposed model in most cases but use more computations. An interesting comparison would be to compare with a standard RNN with similar computational power as the final learned Jumpy RNN. Possible baseline could be training standard RNN with reduced sampling rate and do linear interpolations in between the predictions.     

Other comments:
1. What does baseline model refer to here: 'setting $\epsilon$ to the final training loss of the baseline model'?
2. In the 3rd line of 2nd para in section 3.1, shouldn't it be 'c' instead of 'x'?
3. Could the authors comment on the possibility of using the proposed approach to learn from irregularly sampled time series?

==========Post-rebuttal comments================

Based on other reviews and authors response, I have decided to keep my score. I still feel this paper need more work such as experiments on real world datasets and more comparisons as pointed out in the review. 
review_title: Interesting problem but lacking comparisons and experimentation

Meta-review: The authors propose a "jumpy RNN" to adaptively change the step size of an RNN to match the time scales of the system dynamics. Reviewers found merit in the simple and intuitive idea, but were less enthusiastic about the experimental results and the comparison to existing work. (Adaptive step size methods have been a subject of recent work in RNNs, not to mention in numerical methods for ODE solvers.) Overall, I think the additions the authors made in the discussion phase did strengthen the paper, but further work is necessary before publication. 
Decision: Reject