Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper presents a method for reconstructing trajectories of protein structure from MD simulation. The main technical contribution is a geometric autoencoder architecture with separate latent spaces for representing intrinsic and extrinsic geometry. 

Overall, while the latent space design is interesting, the evaluation of the method is not totally convincing, and there seems to be a lot left on the table in terms of extracting dynamics information from the data. The primary goal of the method appears to be analysis of MD simulations, however the learned representations are compared to quantities that are very easy to directly measure, so I’m not convinced that the method would be useful for analysis in an unsupervised setting. The validity of the reconstructed structures should be more thoroughly characterized.

Questions/comments:
- The model appears to be trained on static snapshots from the trajectory — are dynamics/temporal information included in any way? 
- To test the usefulness of the different intrinsic/extrinsic representations (the main claim of this paper), the authors should perform an ablation study, removing the either the extrinsic or intrinsic components.
- The intrinsic geometry is represented by Ca-Ca bond distances. These backbone distances typically stay very close to 3.86 A in crystal structures due the chemical constraints from the peptide backbone. Do they vary in the simulation - does it really need to be learned? 
- While L2 loss provides a global measure of the reconstruction error, it would be interesting to report additional metrics on the validity of the reconstructed structures, e.g. steric clashes, invalid bond geometries.
- It is not clear to me from the visualization in Fig 3 that the drug molecule is “perfectly classified” in the latent space.
- Can the authors clarify how transfer learning works when transferring the trained model to another molecule with a different number of atoms? What is the purpose of transfer learning? It would be useful to include a baseline of a proGAE model trained from scratch on the S/Protease trajectories in Tab 4.
- For the transfer learning task, the authors should show visualizations of the reconstruction vs. ground truth as in Fig 2. Again, it is hard to interpret the aggregate MSE error, and I would be interested to see how the reconstruction compares between the different source models. 


review_title: Method for analysis of MD trajectories; Needs further evaluation

final_score: rating: 5: Marginally below acceptance threshold
review_content: The authors address a problem of learning protein dynamics directly from the protein structures. They propose an autoencoder type approach with disentanglement intrinsic and extrinsic features. However, there are several concerns with this approach:

1. Novelty - it is a certainly novel application with an adaptation to proteins of the paper from Tatro et al., but the papers look very similar. Could the authors please elaborate more on the methodological differences between these two approaches? Is there any novelty that I missed except that the idea was applied to another domain?
2. “Disentanglement” is not learned by the model, but actually hard coded by expert knowledge. It is not surprising at all that two spaces are disentangled at all - the authors just fed two different types of manually crafted representations from the very beginning. I am not sure that it can count as methodologically new disentanglement from ML point of view.
3. The idea of intrinsic and extrinsic components is nice, however from the experiments the contribution of the intrinsic component to the reconstruction is not very clear.
4. The results from Table 3 are indeed interesting, but they don’t mean that “the extrinsic latent embedding captures more physicochemical information about the bound drug”, they just mean that the relationship between raw extrinsic features to the drug properties is rather non-linear. As a fair comparison authors could rather use a non-linear model (e.g. neural network) for the same task instead of PCA. 
5. The authors use reconstruction metric to evaluate their model performance, which is reasonable for model comparisons, but I am not sure that this is an indicator that the proposed model and the approach could be used in practice. How will the authors sample protein conformations from latent trajectories? Maybe the authors could elaborate on it more and provide examples/experiments? For example, Figure 4 shows how distances change when they sample along principle component, but it doesn't mean that reconstructed conformations are meaningful or realistic. Otherwise, the only thing I see is: it is possible to train an AE type of model (even with heavily predefined features as input). So summing 4 and 5 together it would be interesting to see that the model is indeed useful and goes beyond non-linear regression. 
6. The authors cite several related methods (“Specifically, there has been interest in modeling the underlying conformational space of protein dynamics via deterministic… and probabilistic…”), however there is no comparison to these methods in the paper. Would be nice if the authors could be add it or at least explain why they don't.

review_title: An interesting application, but the paper looks underdeveloped 

final_score: rating: 7: Good paper, accept
review_content: **Summary**
The paper introduces a geometric variational autoencoder for capturing protein structural ensembles, disentangling intrinsic and extrinsic geometry into separate latent spaces. The model is shown to accurately reconstruct protein structure, and the difference between the intrinsic and extrinsic latent spaces are explored. Finally, the model is tested in a transfer-learning setting, where it displays encouraging results.

**Strengths**
Learning representations of protein structure is an important problem, which has only received limited attention in the literature so far. The paper presents a new approach for tackling this problem, and reports good results. The paper is thus interesting both from a methodological and an application perspective.

**Weaknesses**
I only identified minor weaknesses, which I address in detail below.

**Recommendation**
I recommend the paper be accepted. The presented method is novel (to my knowledge), and it is clearly presented.

**Comments to the authors**
Although you give a clear definition of extrinsic vs intrinsic geometry, it would be enlighening if a slightly more elaborate motivation was given for why this is a meaningful way to separate the geometric properties of a protein. Which structural properies to we expect to capture in each of the two distinct latent space? Perhaps you could relate this to the discussion in protein modelling of whether to represent protein structure using internal coordinates (angles, dihedrals, bond-angles) vs the coordinates of all atom positions. In the result section, you demonstrate a case where the extrinsic latent space separates structural changes in response to drug changes, but are there any cases where the intrinsic representation is a useful descriptor?

Page 3, "we note that the protein backbone trace can be viewed...The protein backbone itself can be viewed." These two sentences are central to the paper, but not entirely clear.  The first refers to the "backbone trace" and the other to the "backbone itself", but the distinction between these is not formally defined (at least as far as I could see). Perhaps it is merely a question of defining what the "trace" of a protein is.

Several places in the manuscript you refer to "generating protein dynamics", and "reconstructing protein trajectories". As far as I could see from the paper, the model only describes how single structures can be encoded and decoded, making it a probabilistic model over structure. My interpretation is therefore that it is a model that can generate structural ensembles - i.e. reconstructing thermodynamics, but not dynamics or trajectories, which would seem to me to require the definition of some process in the latent space. Please either rephrase these places in the text, or make it more clear how trajectories are generated.

In your final transfer learning experiment, you conclude that "the learned fiters generalize to trajectories of completely different protein systems". Have you looked into how the representations are effected. More specifically, do you see a difference in how well the intrinsic vs the extrinsic features generalize?

**Minor comments**
Page 1, "and improving latent" -> "and improve latent"

Page 1, "intrinsic and extrinsic geometries". The introduction would be slightly easier to follow if the difference between intrinsic and extrinsic geometries were explained informally already very early in the paper - so that we wouldn't have to wait until the bottom of page 2 before these terms are defined.

Figure 1. The left most figures (the protein structures) do not very clearly illustrate the difference between intrinsic and extrinsic geometry. Could this difference perhaps be made clearer by zooming in to part of the structure?

Figure 1, caption: "encodeing" -> "encoding"

Page 3, "where as backbond" -> where as backbone"

review_title: Official blind review #1

final_score: rating: 5: Marginally below acceptance threshold
review_content: The application is a very interesting topic in general, and in this sense the work would be important, while the organization of this paper is not good enough, particularly the method section, which should be written in a more organized manner. Or currently the method section is written rather too plainly and wordy, while I think a more easy-to-read  organization would be needed with illustrations. Otherwise it is hard to see what are the new points in details by this paper. I have the following as other comments:

1. In fact the method is somewhat different from (what expected from) that described in Introduction. This is because "dynamics" means time-series, by which I thought the latent embeddings are also time-series (like exactly simulation), while the reality is slightly different.

2. Again the method section particularly encoding should be more clearly described.

3. Datasets are limited to only one, two types of proteins. I'm not familiar with the field, while as a methodology would it be good enough only checking these small number of proteins? I think it would be hard to say something general only from such a small number of proteins.

4. It is not clear how each of the intrinsic and extrinsic properties contributed to the prediction. The performance results obtained by each of them should be shown.

5. Also evaluation would not be so strict, in the sense of no comparison with other methods, except the baseline in transfer learning. 

Other minor points:
1. I think the authors explain what "intrinsic and extrinsic properties" mean somewhere earlier in brief. 

2. This paper has a lot of typos: "After that, We discuss", "that do are referred", etc.



review_title: This paper presents a method of capturing latent embedding from 3D protein structure training data and reconstructing protein coordinates for test data. A point is to separate embeddings for intrinsic (alpha carbon connections) and extrinsic (bond orientation) properties.

Meta-review: This paper addresses an interesting learning problem of a generative neural network on a simulated ensemble of protein structures obtained using molecular simulation to characterize the distinct structural fluctuations of a protein bound to various drug molecules. The main technical contribution is a geometric autoencoder architecture with separate latent spaces for representing intrinsic and extrinsic geometry. However, the reviewers think the benefit for modeling intrinsic and extrinsic geometry is not clearly explained and the experiments are not convincing at the moment. The paper can be potentially improved by addressing these two main issues. 
Decision: Reject