Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: The paper tackles the problem of generating long-range, diverse and natural looking motion sequence between initial and end states, and proposes to use a semi-parametric approach consisting of local and global models. Specifically, first the proposed approach extracts local motion feature from a reference subsequence and style feature from another, and then generates a new motion sequence. Then, global motion composition is done to interpolated generated local subsequences by bi-directional composition. In experimental validation, the approach outperforms two baselines (GAN and VAE). 

The proposed approach seems interesting and relatively novel, and technically sound. Also using disentangled representation of style and content is well-motivated. In addition, its generated motions look natural and visually appealing. However, the paper needs more thorough experimental validation to demonstrate its effectiveness better. First,  it was not quite clear which specific baselines were exploited. Are they simple VAE and GAN, or HP-GAN (Barsoum et al. (2018)) and MT-VAE (Yan et al. (2018)) ? And no animated sample of baselines was provided in the demo website. Additionally, in the global composition results, there is no other baseline except its own variants. Finally, even thought the provided figures and animated demos look appealing, it may be required to provide more qualitative results such as user study. 

Writing is fine but there were many typos and it may need more proofreading. Also it might need to add more detail in the presentation.

Detailed comments:
- Why is GAN better than the proposed model with 10% of training data in Fig. 3? Do you have any explanation about it?
- The paper uses separate training of local and global models. But how about joint training of them?
- Difference between style and content is sometimes not clear, for example, rightmost one in the last sample in the demo website. Could you add more constraint or model specification to induce more disentangled representation?
- It is not quite clear if standard deviation is a good metric for motion diversity metric since visual diversity might not be directly correlated to standard deviation of joints (for example, some joints might be more important than others for inducing diversity.) 

Typos:
p4: lone-range one -> long-range one
p5: (Gupta et al.) -> (Gupta et al., 2018), (Zhao et al.) -> (Zhao et al., 2019), (Wang et al.) -> (Wang et al., 2019) 
p6: Our works -> Our work
p7: Fig. 3: standard diviation -> standard deviation 
p8: Recall that we representation -> Recall that we do representation?
p9: As shown in Fig 7(A) -> As shown in Fig 7(B)
p10: rout constrain -> route constraint, gravity constrain -> gravity constraint


In sum, I think the paper is at the borderline but it could be improved and better by having more through experimental validation and more detailed presentation.
review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: 1. Summary:
The paper proposed ''composable semi-parametric modeling'' for generating long-range diverse and distinctive behaviors to achieve a specific goal location. The non-parametric part is a memory bank that is used to retrieve motion patterns from source materials. The parametric part contains several deep neural networks which are to compose the retrieved materials for high quality and smooth motion generation. The overall idea is novel in the sense that they aim to combine the strength of the non-parametric method (with rich pattern and diversities) and the parametric method (powerfull ability to generate coherent results). The proposed ideas are evaluated on two datasets and outperform compared approaches qualitatively.

 2. I am borderline to this paper but prone to weakly reject this paper.

   (1) My major concern about this paper is the lack of reasoning about each component. 
        - local motion composition: How does the design help encode both style and content information in the outputs? Since reconstruction loss is utilized, the optimal should be to learn an identity mapping. How does the author avoid this to happen? The paper referred to the style embedding features H_s, but it is hard to understand how this is associated with the architecture. Also, the usefulness of local motion composition is not well illustrated in the experiments part. 

    -global motion composition: How the network is supervised to compose different clips together? It seems that the authors again adopt reconstruction loss, how the loss penalize inconsistent predictions between clips is not well explained.
      
   (2) The overall presentation of this paper is hard to follow. I spent a lot of time on understanding the notations and corresponding networks. For example, the input representation is not well stated and the author referred to \phi_loc without any indication in Fig, 2. It's very hard to understand the details of the method. I would like to suggest that the author improves the writing especially for Section 2.
 
(3) Does the style motion shared by all sub-sequences? 

(4) Figure 5 shows a sequence. however, the COSMO result seems not natural for me since the velocity in the sequence changes. 

I would like to change my rating if my questions are well addressed in the rebuttal. Overall, this is an interesting idea.
   
review_title: Official Blind Review #4

final_score: rating: 3: Weak Reject
review_content: This paper aims at taking techniques from motion interpolation into the regime where one is able to generate longer range motion sequences, in the domain of physically plausible computer animation of characters. In the way that the authors have set up the problem, an initial database seeds the search for plausible transitions between two given poses. So, the technique being proposed must address how to keep physical realism (the long-standing question of "dynamics filtering" along the lines of Yamane, Katsu, and Yoshihiko Nakamura. "Dynamics filter-concept and implementation of online motion generator for human figures." IEEE transactions on robotics and automation 19.3 (2003): 421-432. Of course the problem here also needs to address "style" which needs different models). 

The authors propose an approach, building on recent neural network architectures including GANs and VAEs, which combines an embedding into a latent space to capture the style+content and then a Bi-LSTM model to make P-step predictions to extract the interpolating poses. The architecture is not very well explained and could be presented much better. However, the constituent elements are fairly standard ones. The learning of the embedding space is based on minimising a reconstruction loss and adapting parameters of a network that takes convolutions over time windows and stacks different channels. The LSTM training is also based on prior work in that domain. 

The authors try to show with experiments that the proposed model is doing better both on diversity and accuracy in standard motion capture datasets. However, the baselines used, especially for the accuracy comparison, are more along the lines of an ablation study than a genuine comparison with alternate methods.

This is coupled with the fact that the authors do not seem to have engaged with a fairly established literature on modelling such sequences, e.g., 
Brand, M., & Hertzmann, A. (2000, July). Style machines. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques (pp. 183-192). ACM Press/Addison-Wesley Publishing Co..
Li, Y., Wang, T., & Shum, H. Y. (2002, July). Motion texture: a two-level statistical model for character motion synthesis. In ACM transactions on graphics (ToG) (Vol. 21, No. 3, pp. 465-472). ACM.

These papers also solve the same problem the authors have set out to solve and arguably do pretty well. How well do they compare to the authors' approach and in what ways have they built further? It would be helpful to understand this with quantitative evidence. 

As it stands, this comes across as a report on a preliminary experiment. Indeed, figure 10 is just a single learning curve without much more interpretation and analysis. The paper would be much stronger if situated better with respect to other established work and also supported by more systematic empirical experiments.
review_title: Official Blind Review #3

Meta-review: The submission presents a semi-parametric approach to motion synthesis. The reviewers expressed concerns about the presentation, the relationship to existing work, and the scope of the results. After the authors' responses and revision, concerns remain. The AC also notes that the submission is 10 pages long. The AC recommends rejecting the submission.
Decision: Reject