From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper25/Authors
title: Some Answers
comment: Hello, thanks a lot for the comments and suggestions! Here are the answers:
1, No, We did not use policy gradient in our training. In the background part we describe policy gradient in order to introduce GAIL and their training method. Then later in our model description, we compare the pros and cons of using policy gradient in GAIL and introduce our training approach when the environment models are known and differentiable.
2, This is common practice in sports analytics to use similar metrics to measure the quality of trajectories. Here to be specific, we discuss with some basketball experts and come up with these metrics in order to measure the results comprehensively from both micro and macro perspective. Also, more metrics are welcomed!
3, Yes there are some related works in terms of multiresolution RNN, for example [1] and [2]. The most obvious difference is that our model allows conditioning on the future while others only condition on the past.
4, Yes we will make this abbreviation clearer later, thanks.
Thanks again for the comments!

[1] Chung J, Ahn S, Bengio Y. Hierarchical multiscale recurrent neural networks[J]. arXiv preprint arXiv:1609.01704, 2016.
[2] Serban I V, Klinger T, Tesauro G, et al. Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation[C]//AAAI. 2017: 3288-3294.

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper25/AnonReviewer2
evaluation: This paper proposes to use a hierarchical, multiresolution generative model to model long-term dependencies in sequential data. This sidesteps the need to hand-design learning signals and allows for stable training using the reparameterization trick. The model demonstrates positive results on a few benchmark tasks, such as modeling trigonometric functions and analyzing basketball player tracking data. 

Overall, the work is interesting and shows promise, but the presentation could be improved. In the Background section 1.1, the authors could have done a better job distinguishing their work from prior work. From the way that the paper was written, it was unclear to me as to how the authors' framework was different from GAIL apart from the more complex generator. I also did not understand where learning a policy was applicable in the context of the paper.

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper25/AnonReviewer2
relevance: 4: high
impact: 3: borderline
novelty: 3: borderline

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper25/AnonReviewer1
evaluation: Designing generative models for long-range spatiotemporal sequences is difficult because it requires modeling high-dimensional distributions with long-range dependencies. Existing models (e.g., various RNN variants) have trouble capturing these long-term dependencies. This paper proposes to extend Generative Adversarial Imitation Learning to this problem, and they call it Multiresolution Adversarial Training (MAT). The main idea is to have a multi-resolution generator that consists of a stack of sequential models that generate observations at different temporal resolutions, and then pit these generators against a single discriminator. They demonstrate both qualitatively (modeling trigonometric functions) and quantitatively (predicting basketball player movements) that MAT outperforms baselines based on RNNs, even in cases where macro-level goals are provided during training to the RNNs.

For the basketball trajectory experiments, it's exciting that the MAT performs about on par with the H-RNN baseline (which uses adversarial training and macro-goals), since it is completely unsupervised. The authors note that we can consider the top levels of the MAT generator stack as learning to generate approximate macro-goals that the lower level generators can refine into complete (and hopefully realistic) trajectories. The 4 quantitative metrics used could be better explained though (probably easy with a less restrictive page limit) — I don't understand them well enough to evaluate whether they are reasonable for comparing the different approaches.

It's also interesting that this approach is related to recent work in modeling large images such as Progressive GAN — I would like to see more discussion here in future versions. One motivation that the authors give for learning generative models of trajectories is the possibility of learning latent representations that are informative of the decision-making process of agents in a multi-agent system — have any experiments been done to explore whether the hidden representations learned are useful for downstream tasks?

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper25/AnonReviewer1
relevance: 5: very high
impact: 3: borderline
novelty: 4: high

From: ~Oliver_Paul_Richter1
comment: Hi, thanks for the nice paper! Some questions/comments from my side:
 - regarding policy gradients: it seems like you don't use policy gradients, but solely train on the GAN objective, correct? (Lines 69-74) Why do you introduce policy gradients in the first place and say that you train on it? (lines 48-50)  Or if you do use policy gradients, what is your reward function?
 - regarding the metrics you introduce: are these metrics commonly used to evaluate human trajectories or are these metrics a novel proposition from your side?
 - regarding related work: Who else has worked with hierarchically arranged RNNs so far? I know of at least this paper (https://arxiv.org/abs/1705.08639) which does something similar.
 - and a minor comment on notation: if you introduce an abbreviation like MAT, I would write it out directly before you introduce the abbreviation such that people can relate the letters. (From the title one might guess that it stands for Multiresolution Adversarial Training, but you never make this relation explicit in the paper)
Hope this helps
title: Some questions and comments regarding formulation and related work