ProGAE: A Geometric Autoencoder-based Generative Model for  Disentangling Protein Conformational Space
Authors: Norman Joseph Tatro
Payel Das
Pin-Yu Chen
Vijil Chenthamarakshan
Rongjie Lai
Venue:  (2021)
Keywords: generative models
deep learning
interpretability

Abstract: Understanding the protein conformational landscape is critical, as protein function, as well as modulations thereof due to ligand binding or changes in environment, are intimately connected with structural variations. This work focuses on learning a generative neural network on a simulated ensemble of protein structures obtained using molecular simulation to characterize the distinct structural fluctuations of a protein bound to various drug molecules. Specifically, we use a  geometric autoencoder framework to learn separate latent space encodings of  the intrinsic and extrinsic geometries of the system. For this purpose, the proposed Protein Geometric AutoEncoder (ProGAE) model is trained on  the length of the alpha-carbon pseudobonds and the  orientation of the backbone bonds of the protein. Using ProGAE latent embeddings, we reconstruct and generate the conformational ensemble of a protein at or near the experimental resolution. Empowered by the disentangled latent space learning, the intrinsic latent embedding help in geometric error correction, whereas the extrinsic latent embedding is successfully used for classification or property prediction of  different drugs bound to a specific protein. Additionally, ProGAE is able to be transferred to the structures of a different state of the same protein or to a completely different protein of different size, where only the dense layer decoding from the latent representation needs to be retrained. Results show that our geometric learning-based method enjoys both accuracy and efficiency for generating complex structural variations, charting the path toward scalable and improved approaches for analyzing and enhancing molecular simulations.

Under review as a conference paper at ICLR 2021
PROGAE: A GEOMETRIC AUTOENCODER-BAGER GENERATIVE MODEL FOR DISENTANGLING PROTEIN CONFORMATIONAL SPACE
Anonymous authors Paper under double-bling review
ABSTRACT
Understanding the protein conformational landscape is critical, protein func- tion, as well as modulations thereof due to ligand binding or changes in environ- ment, are intimately connected with structural variations. This work focuses on learning generative neural network on a simulated ensemble of protein struc- tures obtained using molecular simulation to characterize the distinct structural fluctuations of a protein bound to various drug molecules. Specifically, we use geometric autoencoder framework to learn separate latent space encodings of the intrinsic and extrinsic geometries of the system. For this purpose, the pro- posed Protein Geometric AutoEncoder (ProGAE) model is trained on the length of the alpha-carbon pseudobonds and the orientation of the backbone bonds of the protein. Using ProGAE latent embeddings, we reconstruct and generate the conformational ensemble of protein at or near the experimental resolution. Em- powered by the disentangled latent space learning, the intrinsic latent embedding help in geometric error correction, whereas the extrinsic latent embedding is suc- cessfully used for classification or property prediction of different drugs bound specific protein. Additionally, ProGAE is able to be transferred to the struc- tures a different state the same protein or to a completely different protein different size, where only the dense layer decoding from the latent representation needs to be retrained. Results show that our geometric learning-based method enjoys both accuracy and efficiency for generating complex structural variations, charting the path toward scalable and improved approaches for analyzing and en- hancing molecular simulations.
INTRODUCTION
The complex and time-consuming calculations molecular simulations have been significantly im- pacted by the application of machine learning techniques in recent years. In particular, deep learning has been applied to analysis and simulation of molecular trajectories to address diverse problems, such estimating free energy surfaces, defining optimal reaction coordinates, constructing Markov State Models, and enhancing molecular sampling. For a comprehensive review of deep learning methods for analyzing and enhancing molecular simulations, see (Noé al. 2020a) and (Noé al. 2020b).
Specifically, there has been interest in modeling the underlying conformational space of proteins by using deep generative models, e.g. (Ramaswamy et al., 2020) and (Bhowmik et al.) 2018 Guo al. 22200] Varolgüney et al., 2020). This line of work has mainly attempted to respect the domain geometry by using convolutional AES on features extracted from 3D structures. In parallel, learning directly from 3D structure has recently developed into an exciting and promising application area for deep learning. In this work, we learn the protein conformational space from a set of protein simulations using geometric deep learning. We also investigate how the geometry of protein itself can assist learning and improve latent conformational space interpretability. Namely, we consider the influence of intrinsic and extrinsic geometry, where intrinsic geometry is independent of 3D embedding and extrinsic is not. Intrinsic geometric protein properties be thought to be robust to conformation. To this end, we propose a Protein Geometric Autoencoder model, named ProGAE, to separately encode intrinsic and extrinsic protein geometries.

---

Under review as a conference paper at ICLR 2021
The main contributions of this work summarized:
Inspired by recent unsupervised geometric disentanglement learning works (Tatro al. 2020 Wu et al. 2019 Yang et al. 2020), we propose a novel geometric autoencoder named ProGAE that di- rectly learns from 3D protein structures via separately encoding intrinsic and extrinsic geometries into disjoint latent spaces used to generate protein structures.
We further propose novel formulation, in which network intrinsic input is taken as the Ca-Ca pseudo-bond distances, and the extrinsic input is the backbone bond orientations.
Analysis shows that the learned extrinsic geometric latent space can be used for drug classification and drug property prediction, where the drug is bound the given protein.
We find that the intrinsic geometric latent space, even with small variation in the intrinsic input signal, is important for reducing geometric enrors in reconstructed proteins.
We also demonstrate that the learned ProGAE can be transferred a trajectory of the protein in different state or trajectory of different protein all-together.
1.1 RELATED WORK
Recently, body of work has used deep learning learn from protein structures (Graves et al. 2020 Jing et al., 2020 Klicpera et al. 2020). For example, Gainza et al. (2019) uses geometric deep learning predict docking sites for protein interactions. [Ingraham et al. (2019a) solves the
inverse folding problem using graph transformer on the protein backbone, Degiacomi (2019) uses an AE to generate candidate proteins for docking. |eemmsslll et al. (2020) leverages the notion of intrinsic and extrinsic geometry to define an architecture for a fold classification task.
Additionally, there has been focus on directly learning the temporal aspects of molecular dynamics from simulation trajectories, which is not directly related to the current work. Please see Appendix A. for a detailed discussion.
There is existing body of recent works that use AE-based approaches for either analyzing and/or generating structures from from the latent space (Bhowmik et al. 2018 Guo et al.] 2020; Ramaswamy al. 2020; Varolgünes et al. 22020), which most closely related to this work. (Bhowmik et al. 2018) and (Guo et al. (020) aim at learning from and generating protein contact maps, while PRoGAE directly deals with 3D structures. Therefore direct comparison of ProGAE with these methods is not possible. |Ramaswamy et al. (2019) uses a ID CNN autoencoder trained on backbone coordinates and uses a loss objective comprised of geometric MSE error and physics- based (bond length, bond angle, etc.) error. Due to the unavailability of code or pre-trained model, we were unable to perform a direct comparison. Varolgüney al. 2020) uses a VAE with a Gaus- sian Mixture Prior for performing clustering of high- digeensiini input configurations in the learned latent space. While the method works well on toy models and a standard Alanine Dipeptide bench- mark, its performance drops as the size of the protein system grows to 15 amino acids, which is approximately an order smaller than the protein systems studied here. Also, their approach is likely not going to scale well to larger systems due to the use of fully-connected layers in the encoder.
These mentioned works have not considered explicit disentangling of intrinsic and extrinsic ge- ometries. To our knowledge, this work is the first to propose autoencoder for the unsupervised modeling of the geometric disentanglement of protein conformational space captured in molecular simulations. This representation provides better interpretability of the latent space, in terms of the physico-chemical and geometric attributes, results in more geometrically accurate protein confor- mations, well scales and transfers well to larger protein systems.
2 PROGAE FOR PROTEIN CONFORMATIONAL SPACE
First, we introduce the input signal for our novel geometric autoencoder, ProGAE. We then discuss how ProGAE utilizes this signal to generate the conformational space of a protein.
Geometric Features of Protein as Network Input ProGAE functions by separately encoding intrinsic and extrinsic geometry with the goal of achieving better latent space interpretability. We
2

---

Under review as a conference paper at ICLR 2021
Ba
Protein backbone
Encodar via guomatric convolation
Code
Orientation bonds the protein backbone, by
Latent Code
geometric convolation
Lengths alpha-carbon poeutkbonds, Ivl
Intinsioo Enooder via 10
Intresic Code
3D coordinates of asoms in the protein backbone
Figure 1: Architecture of our network, ProGAE, that generates protein conformations via separate encoding of data related coarse intrinsic and extrinsic geometries. These geometries captured via the orientation of the backbone bonds (extrinsic) and length of C. - C. pseudobonds (intrinsic). These latent representations are jointly used to generate the 3D coordinates of the backbone atoms.
clarify these geometric notions. Mathematically, we can consider a manifold (i.e. surface) inde- pendent of its embedding Euclidean space. Properties that do not depend on this embedding known as intrinsic geometric properties, while properties that do are referred to as extrinsic. As an example, given two atoms of protein, the intrinsic distance between them is the minimum of bond lengths the bond path connecting them, whereas the extrinsic distance is their Euclidean distance in R For in-depth review of geometry, refer the reader to (Do Carmo 2016).
As we will train ProGAE to learn the conformational space of a given protein, the protein primary structure is implicit. Then in treating it geometric object, view the protein at the level of its backbone, which specifies its shape. Given primary structure, reconstructing the protein backbone sufficient for reconstructing the entire protein. Of importance in the backbone are the Ca atoms, which are the centers of amino acids in the protein. Then a coarse-level description of the backbone the Ca atoms connected linearly in terms of the protein sequence. This known as the trace of the protein. We will use the backbone and trace as domains on which to define our signals.
Both the protein backbone and its trace can viewed as polygonal chain in Euclidean space. They are depicted in Figure | with their geometric features as network input. We can see that a polygonal chain can be determined up to translation given both the length and orientation of its line segments. Then it follows that the protein backbone can be determined given the length and orientation of its bonds. Here the length of these bonds is intrinsic while the orientation is extrinsic. Thus, to decouple the intrinsic and extrinsic geometry, can consider encoding these signals.
The length of covalent bonds undergo very little change during a simulation performed using an empirical force-field, like the simulations considered in this work. A standard deviation of less than 0.059Å from target bond lengths is common in PDB structures (Jaskolski et al. 2007). To this end, we instead consider intrinsic geometry variability. Specifically, we use length of the C. pseudobonds in the trace as representative of the intrinsic protein geometry, where as backbone bond orientations capture extrinsic geometry.
coarse level, that the resulting signal has more
We model the backbone by the graph, G. (V,,, E.), and the backbone trace by the graph, G; (Ve, E.). Then our intrinsic and extrinsic signals, Int E. and Ext E, RR3 defined:
Eaj Eg E E,, Ext(E.g) = sgn(j - ||EEll''
Int(EE)) = |E
Eg E E..
(1)
Network Architecture With the network inputs defined, we discuss the architecture of ProGAE.
The core idea is to create an intrinsic latent space, L, E R"; and an extrinsic latent space, LE E R via separately encoding the intrinsic and extrinsic signals. Consequently, network contains
3

---

Under review as a conference paper at ICLR 2021
two encoders, Enc; and Ence where:
Enc, o Int(EE) L1, Enc, Eat(Ee) E Lg.
(2)
We then jointly decode these latent vectors to recover the coordinates of the atoms in the protein backbone. Thus, formally define the decoder:
Dec: : L1 x -E - R Valx3
(3)
This high level structure of ProGAE is depicted in Figure I We provide additional details on the encoders and decoders. As these edge-based signals are defined on a geometric domain, it is sensible to learn feature representations using geometric convolution that respects the geometry of the data. The intrinsic encoder is simple, as the signal is defined on the backbone trace, which corresponds to set of discrete curves. Here each curve corresponds to a protein fragment. Then we define Enc; to be series of ID convolutions operating on each protein fragment. Each convolution is taken to have a kernel size of and a stride of 2. being followed with batch normalization layers and ReLU,
In contrast, the extrinsic encoder operates on the backbone, which we associate with graph. So the layers of graph attention networks (GATs) introduced (Veličković al., (017) are a natural tool to use, albeit with some modification. Since the input signal is defined only on the edges of the graph, Es, we define a signal on the graph vertices, Vb, as the average value of its incident edges,
fo( E, E H
Σ Ext(E E.
04 E V..
Then the first layer of the extrinsic encoder uses the edge-convolution operator of (Gong & Cheng, 2019) to map this graph signal to signal defined exclusively on the graph vertices, V. The rest of the encoder contains successive graph attention layers with sparsity defined by given neighborhood radius. At each layer, the signal is downsampled by factor of two based on farthest point sampling. Given L layers, this defines a sequence of graphs, {G,, L with increasing decimation. As with Enc; each layer is followed with batch normalization and ReLU. Summarily, for I 1,,,..., L.
fi = BN (dT(d, where d 1 (:( fi (v:) = (A.)) (E,)). (5)
Global average pooling is applied to the encoder outputs to introduce invariance to size V, and V.- Dense layers then map each result to their respective latent spaces, I., and Lg. The Tanh function is applied to bound the latent space. This produces the intrinsic and extrinsic latent codes, z; and z.-
The latent code z is taken the concatenation of the two latent codes, [z,, ze). A dense layer maps z to the a signal defined on the most decimated backbone graph, G., The structure of the decoder, Dec, is then analogous Enc,, though the convolutions are transposed. The output of Dec the point cloud, P. corresponding the predicted coordinates of the backbone atoms, V, 2 P.
Loss Function The first term in the loss function is basic reconstruction loss, where Pnd P are taken to be the true and predicted coordinates of the protein backbone atoms. Namely, we evaluate their difference using Smooth-L, loss. This loss defined, with 2, as
#
82 2 (1:.-))) 0 Mil -
SmoothL v)
Zi
where
24
min
(6)
This loss function modifies L2 loss to be more robust to outliers (Girshick 2015).
As the reconstruction loss depends the embedding of the protein in Euclidean space, it may not best measure if intrinsic geometry is faithfully reconstructed. To address this, we consider two encoded proteins with latent codes, [z,,, 2e,1] and [z1,2, 20,2]. Then form new latent variable,
(7) Each these latent variable decodes to some point cloud P. We let Int (E.,), (nt,),,,, and Int Ex,2) be the lengths of the CC-Ca pseudobonds of the generated proteins from the interpolated latent code and the two given latent codes. We then introduce a bond length penalty given by.
= (1 - =) + 3z12,
d = Ze.1:
B U[0
R(P.. P2) E, (nl (r,a) ((1 (3)In(((e,)) (B.,)) + BInl (Int(Era))ll) (.,)))l: B U0, 1).
(8)
4

---

Under review as a conference paper at ICLR 2021
Figure 2: Reconstructions of protein frames from S protein and hACE2 test data using ProGAE
The blue and red structures correspond to the reconstructed and ground truth structures. They are superimposed for comparison. The corresponding backbones can be found in Figures baland and 6b
Table 1: Average atom-wise L2 emrr, bond length error on the training and test sets, well as RMSD between reconstructed and true structures, using ProGAE. The L2 error is within the resolu- tion of the associated PDB files, even achieving sub-Angstrom performance human ACE2.
S protein
hACE2
PDB Resolution (N)
2.68/2.80 8.28 + 0.04 E-1
2.68
Training
Reconstruction error
4.29 + 0.01 E-1
Bond length error (A)
3.71 4: 0.02 E-1
1.56 + 0.01 B-1
Avg. L2 error (A)
1.21 = 0.00
0.78 + 0.01
Test
Reconstruction error
1.39 + 0.01 EO
6.27 + 0.03 R-1
Bond length error (A)
3.88 + 0.02 E-1
1.63 + 0.02 E-1
Avg. L2 error (A)
1.56 + 0.01
0.90 0.01
RMSD from test (A)
2.54 0.56
1.24 0.23
This penalty can be viewed as promoting faithful reconstruction of the pscudobond length between C. atoms, as well as a smooth interpolation of these lengths along paths in L,, that is independent of Lg. This penalty is analogous to the metric preservation regularizer introduced in (Cosmo al. 2020 for 3D meshes. Thus, the loss function L for ProGAE is,
C((P),
Σ S ARR(P,,,
EXPERIMENTAL SETUP
In this section, we describe the setup of our numerical experiments that confirm the usefulness of ProGAE in generating the protein conformational space. For each dataset, train three models, each from a different random seed, and report both mean and standard deviation in our results. Details the choice of network hyperparameters be found in the Appendix A.3
Datasets Datasets used in this work are atomistic simulation trajectories D.E.. Shaw Research 2020). The two main datasets we use are simulations of proteins in presence of FDA approved or under-investigation molecules, as we
to test the performance of ProGAE on capturing drug- induced structural variations. These datasets are: (1) 50 independent trajectories, each simulating the SARS-CoV-2 trimeric spike protein (S protein) in the presence of a distinct drug for 2μs. The simulation is limited to 3 receptor binding domains (RBDs) of the protein, well short region needed for the system to maintain trimer assembly; (2) 75 independent trajectories, each simulating the ectodomain protein of human ACE2 (hACE2) in the presence of distinct drug for 2/us.
The backbones the protein and the hACE2 protein contain 3,690 atoms and 2,386 atoms, re- spectively. The time resolution is 1,200 ps. We use the first 70% of frames from each trajectory to
here: httpa///www///////eeo.oeo.oo//o.

---

Under review as a conference paper at ICLR 2021
Table The leading canonical correlation between the learned intrinsic and extrinsic latent spaces, as well as the performance of a linear model trained on disentangled latent spaces of ProGAE for bound drug classification, are shown.
S protein
hACE2
First canonical correlation
0.08 + 0.00
0.07 0.01 1.43 = 0.01 % 99.62 + 0.21 %
Drug classifier intrinsic latent space
2.04 = 0.04%
Drug classifier on extrinsic latent space
99.65 0.26%
Table 3: Results of linear regression the extrinsic latent space for predicting physical and chem- ical properties of the drugs that a protein is bound to. Error normalized for interpretability. For comparison, performance of linear regression the PCA embeddings of the orientation of the backbone bonds is reported. This embedding is restrained to the same dimension as the latent space.
Molecular weight
Hydrogen bond donor count
Topological polar surface area
Dataset
S protein
PCA error (o)
0.78 = 0.00
0.81 0.01 0.56 + 0.03
0.79 + 0.00 0.61 + 0.00
Latent error (o)
0.55 = 0.04
hACE2
PCA error (a)
0.71 + 0.00
0.65 + 0.00 0.57 + 0.01
0.73 + 0.00 0.53 0.02
Latent error (σ)
0.55 = 0.01
form the training set. The next 10% and the last 20% of frames form the validation and test sets. The train and test sets are intentionally kept temporally disjoint to better assess generalization.
For transfer learning, we also consider two trajectories of the entire protein containing 13,455 backbone atoms. One trajectory is initiated from closed state, while the other from a partially open state. We use the first 2.5 pss of these 10 jrs simulations, corresponding to 2,001 frames with resolution of 1,200 ps. Additionally, we utilize the first 10 jes of 100 jss simulation of the main Protease S-R,-CoV-2, sequence of 10,001 frames with 1,000 ps resolution.
4 RESULTS
Structure Reconstruction Figureddisplay the ability of ProGAE to accurately reconstruct pro- tein conformations. The backbones are visible with atom-wise error in Figures (6a) and [bb in the appendix. From the visualized atom-wise L2 reconstruction error, it is clear that network can capture and reconstruct notable conformational changes a protein. Figures Sa and [8b] in the ap- pendix display these reconstructions with color denoting fragment instead of T.2 error for clarity. Consistent with the low RMSD error, visually the reconstructed structures appear consistent with ground truths, with larger RMSDs observed in the flexible loop and turn regions.
Table I contains performance metrics of ProGAE on training and test sets. Generalization is mea- sured by the L2 reconstruction error the backbone atom coordinates, as well RMSD (root mean square distance) after alignment. For hACE2, we achieve sub- mub-Angstrom performance on the test set. In either case, the RMSD of the reconstruction within the experimental resolution of the associ- ated PDB files; 6VXX/6VW1 for the S protein and 6VW1 for hACE2. Additionally, the average error in the length of the pseudobonds is also sub-Angstrom. Thus, it evident that ProGAE is able to reconstruct proteins within meaningful resolution.
Utility of the Extrinsic Latent Space With the reconstruction capabilities of ProGAE verified, we consider the benefit having separate intrinsic and extrinsic latent spaces. First, we explore the statistical relationship between the learned intrinsic latent space and the extrinsic latent space. Canonical correlation analysis (CCA) a natural approach to assess a linear relationship exists (Hardoon et al. 2004). We include background on CCA in Appendix A.2
Table includes the leading correlation between the intrinsic and extrinsic latent spaces for each dataset. Note this correlation is very low, implying that there is negligible linear relationship be-

---

Under review as a conference paper at ICLR 2021
Cocree track
(a) S protein
(b) Human ACE2
Figure 3: The embedding of the data in the projection of the latent space to the first two canonical vectors between the intrinsic and extrinsic latent space. Color indicates the identity of the drug that the protein is bound to within that frame. Clustering by drug identity is apparent in the extrinsic latent space, but not the intrinsic latent space. This aligns with results in Table
Table 4: The percentage of backbone covalent bonds that are 10% shorter than the minimum seen in training data. The difference (Diff) between the intrinsic+extrinsic ProGAE and the extrinsic only ProGae is also reported; negative difference indicate higher bond-length error in the later model.
Dataset
C-CA
C-N
C-O
CA-N
CA-CA
S protein
Int.+Ext. (%)
14.41
22.58
27.00
15.24
15.33
Ext. Only (%)
19.04
27.34
27.66
18.58
20.89
Diff
-4.63
-4.76
-0.66
-3.40
-5.56
hACE2
Int.+Ext. (%)
2.45
8.19
12.07
4.62
0.51
Ext. Only (%)
4.99
9.81
12.34
5.21
1.59
Diff'
-2.54
-1.62
-0.27
-0.59
-1.08
tween the intrinsic and extrinsic latent spaces. Learning disentangled latent representation is often desired for better interpretabilityy which is typically measured using a generalization of mutual infor- mation such as total correlation as (Chen et al. (0118). While structural conditions may prevent the intrinsic and extrinsic embeddings from being completely independent, Table indiicates absence of a linear relationship between intrinsic and extrinsic latent vectors in our learned model, confirming notable level of disentanglement that has been explicitly encoded in our model architecture.
As stated earlier, each simulation trajectory in the dataset corresponds to the S or hACE2 protein bound to specific drug. Then it is natural to investigate if this distinct drug information is encoded in the two disentangled latent spaces. on the diffferent latent spaces to classify the drug present in each frame. It is clear that the drig molecule can be almost perfectly classified in the extrinsic latent space, while such classification is random in the intrinsic latent space. Figures Bal and 3b display the embeddings of the test set in the latent spaces, projected to the first two canonical components. In these figures, color denotes the identity of the drug that the protein is bound to. Even in the 2D projection of the extrinsic latent space, clustering by the drug identity is apparent, which is not the case for the intrinsic embedding.
the performance of linear classifier trained
Next, consider if this linear separation is chemically meaningful. We train linear regression model on the extrinsic latent space to predict physico-chemical properties of a drug binding to a protein. Table 3 displays the performance of the model predicting the properties of molecular weight, hydrogen bond donor count, and topological polar surface area. For comparison to our latent embedding, we train a linear regression model on the first ne principal component scores of the PCA of the extrinsic signal on each element of the test dataset. The latent regression outperforms that of PCA, indicating that the the extrinsic latent embedding captures more physico-chemical information about the bound drug. We believe this linear regression is appropriate as it prevents overfitting.
Utility of the Intrinsic Latent Space Having confirmed the utility of a separate extrinsic latent space, weigh the benefits of including the intrinsic latent space in the model. This is nontrivial, as the mean pseudo-bont bond length is 3.86A with a deviation 0.06. We find the inclusion of the intrinsic latent space improves the geometric validity of the reconstructed protein seen in the
7

---

Under review as a conference paper at ICLR 2021
Rorbondes
nonbansed Bretane PerinecCestne
+0.00
6.0%
-0.2
-00
CO1
690
300
-0
400
601
Cdl
Rottiiiai Finipel Canga
(a) S protein
horbanded Dilanoe Corteie
6210
420
-330
44.00
-031
Ied carpure
(b) Human ACE2
Figure 4: Error of the non- non-bonded distance between the backbone atoms for generated structures compared to the protein its initiation. Each point corresponds a structure generated by sampling the latent space via the plane of the two specified principal components of the latent spaces. The intrinsic latent code hardly affects the non-bonded distance error, unlike the extrinsic latent code.
(a) S protein
(b) Human ACE2
Figure 5: RMSD of proteins generated along the latent interpolation between two proteins from different trajectories. The RMSDs computed with respect to the endpoint proteins, with mean and standard error shown. We see a smooth interpolation between the RMSD errors as desired. Examples of the structures along the interpolation path can be found in the appendix in Figure 7
following ablation study. We trained model that only encodes the extrinsic signal to reconstruct the protein. While it was comparable in performance regarding L2 error, we found this extrinsic- only model resulted in a higher percentage of erroneous bonds. This is shown in Table 4 Here define a erroneous bond, if the bond length deviates by more than 10% from the minimum of the ground truth distribution, as such deviations will result in steric clashes. We also trained model on only the intrinsic signal, but did not analyze the model further due to poor performance (> 3.0A). These results are found within the appendix in Table
Geometry of the Generated Structural Ensemble As generative model, it is important to con- sider if paths in the latent space meet our expectations. One way by which we measure this is how the pairwise distance matrix between non-bonded atoms in generated protein changes the latent variables change. Specifically.we we are interested in the distance between non-bonded atoms.
In Figures 4a and 4b we plot the norm of the difference between these distance matrices for the gen- erated protein and that of the protein at its initiation. We sample the latent space along two principal components, while setting others to O. So in these figures 4al and 4b) sections of this space. It is immediately apparent that a change in the extrinsic latent code domi- nates changes in the non-bonded distance. This is desired behavior of disentangled geometric
three different cross
8

---

Under review as a conference paper at ICLR 2021
Table 5: Average atom-wise L2 error in reconstruction (A) on the test dataset after transferring a trained model to other protein structures. These simulations correspond to two different states of the entire S protein and of the main protease of SARS-CoV-2. We retrain only the dense layer mapping to the decoder. For comparison, we train the same layer for randomly initialized model (Baseline).
Source dataset
Closed S
Partially Open S
Protease
Baseline (A)
1.79 = 0.04
2.26 0.03
1.36 = 0.03
protein (A)
1.55 + 0.01
2.01 0.07
1.16 0.00
hACE2 (A)
1.52 + 0.03
1.98 + 0.00
1.17 + 0.01
autoencoder, where large-scale changes are captured in the extrinsic latent space, while small-scale changes in local distance are controlled by the intrinsic latent space.
We also evaluate the performance of linear interpolations in the learned latent space. Given two protein conformations from different trajectories (i.e. in the context of two different drugs), we generate path between them by generating the linear interpolation of their latent codes. This provides a path of structural variation that does not exist in the training data. The results of this interpolation in terms of RMSD is shown in Figure 5 As expected, we see smooth exchange in the RMSD error of the generated protein from the first protein and from the second protein.
Transfer Learning- Extension Different Proteins To check the generalization of ProGAE, we investigate transfer learning to simulations of different proteins. We begin with models trained on the S protein comprised of the RBDs and on hACE2. These results are summarized in Table We transfer learned ProGAE models to trajectories of the closed and partially open state of the entire protein, as well as MARS-COV-2 main Protease, which provides insight into the generalization capability of the convolutional filters that we have learned. As result, six scenarios of the transfer learning, in addition to three random baselines, are reported in T When transferring the model trained on the 3 RBDs of the S protein to the S protein in the closed state, we are transferring the model learned on a partial structure to the entire protein that is much larger in size. Model transfer to the S protein in the partially open state deals with a scenario where the conformational state of the protein is notably different (closed vs partially open). Transferring the model trained on hACE2 the S protein datasets studies the knowledge transfer an entirely different protein, but one which hACE2 is known to interact with. Finally, transferring both S protein and hACE2 models to the main Protease simulation allows us study the transfer of the models to a completely different protein without notable interaction with the source protein. Given performing long time-scale simulations of large protein systems high resolution is a computationally expensive process, our method appears beneficial, as ProGAE transfers well to non-related proteins of larger size.
The only incompatible layer is the dense layer mapping from the latent spaces. To investigate trans- fer learning, we train just this dense layer for 10 epochs. As baseline, we train the same layer of a randomly initialized model. In all cases, the transferred model performs better than the baseline. Thus the learned filters generalize to trajectories of completely different protein systems.
5 CONCLUSION
In this work we introduce a novel geometric autoencoder named ProGAE for learning meaningful disentangled representations of the protein conformational space. Our model accurately reconstructs protein structure. The autoencoder separately encodes intrinsic and extrinsic geometries to ensure better latent space interpretability. The extrinsic latent space can classify the protein structures with respect to the bound drug molecules, well as can predict the drug properties. The intrinsic latent space assists in improving the validity of the bond geometry in the reconstructions. We also show that the geometric convolutional filters learned in training can be successfully transferred to trajec- tories of different protein systems, irrespective of system size, conformational state, or presence of protein-protein interaction. These results on learning, predicting, and generating protein conforma- tions suggest that the proposed framework can serve as the first step towards bridging geometric deep learning with molecular simulations.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Debsindhu Bhowmik, Shang Gao. Michael T Young, and Arvind Ramanathan. Deep clustering of protein folding simulations. BMC bioinformatics, 19(18):47-58, 2018.
Luigi Bonati, Yue- Yu Zhang, and Michele Parrinello. Neural networks- variationally enhanced sampling. Proceedings of the National Academy of Sciences, 1 116636)117441117477 2019.
Ricky TQ Chen, Xuechen Li, Roger B Grosse, and David K Duvenaud. Isolating sources of disen- tanglement in variational autoencoders. In Advances in Neural Information Processing Systems, pp. 2610-2620, 2018.
Wei Chen and Andrew L Ferguson. Molecular enhanced sampling with autoencoders: On-the- fly collective variable discovery and accelerated free energy landscape exploration. Journal of computational chemistry, 29025):207--2022, 2018.
Wei Chen, Hythem Sidky, and Andrew L Ferguson. Nonlinear discovery of slow molecular modes using state-free reversible vampnets. The Journal of chemical physics, 150(21):21411, 2019.
Zhengdao Chen, Jianyu Zhang, Martin Arjovsky, and Léon Bottou. Symplectic recurrent neural networks, 2020.
Luca Cosmo, Antonio Norelli, Oshri Halimi, Ron Kimmel, and Emanuele Rodolà. Limp: Learning latent shape representations with metric preservation priors. arXiv preprint arXiv:2003. 122883, 2020.
D.E. Shaw Research. Molecular dynamics simulations related to sars-cov-2. http://wwy wwtpsssssssssss.aaaaaaaas.. html] 2020. Accessed: 2020-09-30.
Matteo Degiacomi. Coupling molecular dynamics and deep learning to mine protein conforma- tional space. Structure, 1746):0034-0040, 2019.
Manfredo P Do Carmo. Differential geometry of curves and surfaces: revised and updated second edition. Courier Dover Publications, 2016.
P Gainza, F Sverrisson, F Monti, E Rodola, MM Bronstein, and BE Correia. Deciphering interaction fingerprints from protein molecular surfaces. bioRxiv, pp. 606202, 2019.
Ross Girshick. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision, pp. 1440-1448, 2015.
Liyu Gong and Qiang Cheng. Exploiting edge features for graph neural networks. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 9211-9211211 2019.
Jordan Graves, Jacob Byerly, Eduardo Priego, Naren Makkapati, S Vince Parish, Brenda Medellin, and Monica Berrondo. A review of deep learning methods for antibodies. Antibodies, 9(2): 2020.
Xiaojie Guo, Sivani Tadepalli, Liang Zhao, and Amarda Shehu. Generating tertiary protein struc- tures via an interpretative variational autoencoder. arXiv preprint arXivv2004.07119, 2020.
David Hardoon, Sandor Szedmak, and John Shawe-Taylor. Canonical correlation analysis: An overview with application to learning methods. Neural computation, 16(12):669--266, 2004.
Pedro Hermosilla, Marco Schäfer, Matjj Lang, Gloria Fackelmann, Pere Pau Vázguez, Barbora Kozliková, Michael Krone, Tobias Ritschel, and Timo Ropinski. Proteinn: Intrinsic-cxtrinsic convolution and pooling for scalable deep protein analysis, 2020.
John Ingraham, Vikas Garg, Regina Barzilay, and Tommi Jaakkola. Generative models for graph- based protein design. In Advances in Neural Information Processing Systems, pp. 15820-15831, 2019a
John Ingraham, Adam Riesselman, Chris Sander, and Debora S Marks. Learning protein structure with differentiable simulator. In ICLR, 2019b.
10

---

Under review as a conference paper at ICLR 2021
Mariusz Jaskolski, Miroslaw Gilski, Zbigniew Dauter, and Alexander Wlodawer. Stereochemical restraints revisited: how accurate are refinement targets and how much should protein structures be allowed to deviate from them? Acta Crystallographica Section D: Biological Crystallography, 63(5))61 -200, 2007.
Bowen Jing, Stephan Eismann, Patricia Suriana, Raphael J. L. Townshend, and Ron Dror. Learning from protein structure with geometric vector perceptrons, 2020.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6908 2014.
Johannes Klicpera, Janck GroB, and Stephan Günnemann. Directional message passing for molec- ular graphs. arXiv preprint wxXv:22033.03123 2020.
Hyungro Lee, Heng Ma, Matteo Turilli, Debsindhu Bhowmik, Shantenu Jha, and Arvind Ra- manathan. Deepdrivemd: Deep-learning driven adaptive molecular simulations for protein fold- ing, 2019.
Andreas Mardt, Luca Pasquali, Hao Wu, and Frank Noé. Vampnets for deep learning of molec- ular kinetics. Nature Communications, 9(1), Jan 2018. ISSN 2041-1723,
doi: 10. 1038/ s4 447-0017-00 URLhttp::/dxxcoo www.:oo........ooo 1ttp://1.1118/441111 oB/p:cttppp....ooooooooooo///.0. 017-0-088--888----1
Frank Noé, Simon Olsson, Jonas Köhler, and Hao Wu. Boltzmann generators: Sampling equilibrium states of many-body systems with deep learning. Science, 365(6457)<eaaw 147,, 2019.
Frank Noé, Gianni De Fabritüs,, and Cecilia Clementi. Machine learning for protein folding and dynamics. Current Opinion Structural Biology, 60:77-84, 2020a.
Frank Noé, Alexandre Tkatchenko, Klaus- Müller, and Cecilia Clementi. Machine learning for molecular simulation. Annual review of physical chemistry, 71:361-390, 2020b.
Venkata K. Ramaswamy, Chris G. Willcocks, and Matteo T. Degiacomi. Learning protein confor- mational space by enforcing physics with convolutions and latent interpolations, 2019.
Venkata K. Ramaswamy, Chris G. Willcocks, and Matteo Degiacomi. Learning protein confor- mational space by enforcing physics with convolutions and latent interpolations, 2020.
João Marcelo Lamim Ribeiro, Pablo Bravo, Yihang Wang, and Pratyush Tiwary. Reweighted toencoded variational bayes for enhanced sampling (rave). The Journal of chemical physics, 149 (7):072301, 2018.
N. Joseph Tatro, Stefan C. Schonsheck, and Rongjie Lai. Unsupervised geometric disentanglement for surfaces via cfan-vae, 2020.
Sun-Ting Tsai, En-Jui Kuo, and Pratyush Tiwary. Learning molecular dynamics with simple lan- guage model built upon long short-term memory neural network, 2020.
Yasemin Bozkurt Varolgünes, Tristan Bereau, and Joseph F Rudzinski. Interpretable embeddings from molecular simulations using gaussian mixture variational autoencoders. Machine Learning: Science and Technology, 1(1):015012, 2020.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv:1710.10003: 2017.
Wujie Wang. Simon Axelrod, and Rafael Gómez-Bombarelli, Differentiable molecular simulations for control and learning. 2020.
Wayne Wu, Kaidi Cao, Cheng Li, Chen Qian, and Chen Change Loy. Disentangling content and style via unsupervised geometry distillation, 2019.
Jie Yang, Kaichun Mo. Yu-Kun Lai, Leonidas J. Guibas, and Lin Gao. Dsm-net: Disentangled structured mesh net for controllable generation of fine geometry, 2020.
Jun Zhang, Yi Isaac Yang, and Frank Noé. Targeted adversarial learning optimized sampling. The journal of physical chemistry letters, 10(19):5911-5797 2019.


Reviews:
[{"review_id": "Yn7xSHFnW03", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The application is a very interesting topic in general, and in this sense the work would be important, while the organization of this paper is not good enough, particularly the method section, which should be written in a more organized manner. Or currently the method section is written rather too plainly and wordy, while I think a more easy-to-read  organization would be needed with illustrations. Otherwise it is hard to see what are the new points in details by this paper. I have the following as other comments:\n\n1. In fact the method is somewhat different from (what expected from) that described in Introduction. This is because \"dynamics\" means time-series, by which I thought the latent embeddings are also time-series (like exactly simulation), while the reality is slightly different.\n\n2. Again the method section particularly encoding should be more clearly described.\n\n3. Datasets are limited to only one, two types of proteins. I'm not familiar with the field, while as a methodology would it be good enough only checking these small number of proteins? I think it would be hard to say something general only from such a small number of proteins.\n\n4. It is not clear how each of the intrinsic and extrinsic properties contributed to the prediction. The performance results obtained by each of them should be shown.\n\n5. Also evaluation would not be so strict, in the sense of no comparison with other methods, except the baseline in transfer learning. \n\nOther minor points:\n1. I think the authors explain what \"intrinsic and extrinsic properties\" mean somewhere earlier in brief. \n\n2. This paper has a lot of typos: \"After that, We discuss\", \"that do are referred\", etc.\n\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "XB4YbJfV_8d", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Summary**\nThe paper introduces a geometric variational autoencoder for capturing protein structural ensembles, disentangling intrinsic and extrinsic geometry into separate latent spaces. The model is shown to accurately reconstruct protein structure, and the difference between the intrinsic and extrinsic latent spaces are explored. Finally, the model is tested in a transfer-learning setting, where it displays encouraging results.\n\n**Strengths**\nLearning representations of protein structure is an important problem, which has only received limited attention in the literature so far. The paper presents a new approach for tackling this problem, and reports good results. The paper is thus interesting both from a methodological and an application perspective.\n\n**Weaknesses**\nI only identified minor weaknesses, which I address in detail below.\n\n**Recommendation**\nI recommend the paper be accepted. The presented method is novel (to my knowledge), and it is clearly presented.\n\n**Comments to the authors**\nAlthough you give a clear definition of extrinsic vs intrinsic geometry, it would be enlighening if a slightly more elaborate motivation was given for why this is a meaningful way to separate the geometric properties of a protein. Which structural properies to we expect to capture in each of the two distinct latent space? Perhaps you could relate this to the discussion in protein modelling of whether to represent protein structure using internal coordinates (angles, dihedrals, bond-angles) vs the coordinates of all atom positions. In the result section, you demonstrate a case where the extrinsic latent space separates structural changes in response to drug changes, but are there any cases where the intrinsic representation is a useful descriptor?\n\nPage 3, \"we note that the protein backbone trace can be viewed...The protein backbone itself can be viewed.\" These two sentences are central to the paper, but not entirely clear.  The first refers to the \"backbone trace\" and the other to the \"backbone itself\", but the distinction between these is not formally defined (at least as far as I could see). Perhaps it is merely a question of defining what the \"trace\" of a protein is.\n\nSeveral places in the manuscript you refer to \"generating protein dynamics\", and \"reconstructing protein trajectories\". As far as I could see from the paper, the model only describes how single structures can be encoded and decoded, making it a probabilistic model over structure. My interpretation is therefore that it is a model that can generate structural ensembles - i.e. reconstructing thermodynamics, but not dynamics or trajectories, which would seem to me to require the definition of some process in the latent space. Please either rephrase these places in the text, or make it more clear how trajectories are generated.\n\nIn your final transfer learning experiment, you conclude that \"the learned fiters generalize to trajectories of completely different protein systems\". Have you looked into how the representations are effected. More specifically, do you see a difference in how well the intrinsic vs the extrinsic features generalize?\n\n**Minor comments**\nPage 1, \"and improving latent\" -> \"and improve latent\"\n\nPage 1, \"intrinsic and extrinsic geometries\". The introduction would be slightly easier to follow if the difference between intrinsic and extrinsic geometries were explained informally already very early in the paper - so that we wouldn't have to wait until the bottom of page 2 before these terms are defined.\n\nFigure 1. The left most figures (the protein structures) do not very clearly illustrate the difference between intrinsic and extrinsic geometry. Could this difference perhaps be made clearer by zooming in to part of the structure?\n\nFigure 1, caption: \"encodeing\" -> \"encoding\"\n\nPage 3, \"where as backbond\" -> where as backbone\"\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "QqIOh0DxgAd", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors address a problem of learning protein dynamics directly from the protein structures. They propose an autoencoder type approach with disentanglement intrinsic and extrinsic features. However, there are several concerns with this approach:\n\n1. Novelty - it is a certainly novel application with an adaptation to proteins of the paper from Tatro et al., but the papers look very similar. Could the authors please elaborate more on the methodological differences between these two approaches? Is there any novelty that I missed except that the idea was applied to another domain?\n2. “Disentanglement” is not learned by the model, but actually hard coded by expert knowledge. It is not surprising at all that two spaces are disentangled at all - the authors just fed two different types of manually crafted representations from the very beginning. I am not sure that it can count as methodologically new disentanglement from ML point of view.\n3. The idea of intrinsic and extrinsic components is nice, however from the experiments the contribution of the intrinsic component to the reconstruction is not very clear.\n4. The results from Table 3 are indeed interesting, but they don’t mean that “the extrinsic latent embedding captures more physicochemical information about the bound drug”, they just mean that the relationship between raw extrinsic features to the drug properties is rather non-linear. As a fair comparison authors could rather use a non-linear model (e.g. neural network) for the same task instead of PCA. \n5. The authors use reconstruction metric to evaluate their model performance, which is reasonable for model comparisons, but I am not sure that this is an indicator that the proposed model and the approach could be used in practice. How will the authors sample protein conformations from latent trajectories? Maybe the authors could elaborate on it more and provide examples/experiments? For example, Figure 4 shows how distances change when they sample along principle component, but it doesn't mean that reconstructed conformations are meaningful or realistic. Otherwise, the only thing I see is: it is possible to train an AE type of model (even with heavily predefined features as input). So summing 4 and 5 together it would be interesting to see that the model is indeed useful and goes beyond non-linear regression. \n6. The authors cite several related methods (“Specifically, there has been interest in modeling the underlying conformational space of protein dynamics via deterministic… and probabilistic…”), however there is no comparison to these methods in the paper. Would be nice if the authors could be add it or at least explain why they don't.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "Ol7AauVS_cK", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a method for reconstructing trajectories of protein structure from MD simulation. The main technical contribution is a geometric autoencoder architecture with separate latent spaces for representing intrinsic and extrinsic geometry. \n\nOverall, while the latent space design is interesting, the evaluation of the method is not totally convincing, and there seems to be a lot left on the table in terms of extracting dynamics information from the data. The primary goal of the method appears to be analysis of MD simulations, however the learned representations are compared to quantities that are very easy to directly measure, so I’m not convinced that the method would be useful for analysis in an unsupervised setting. The validity of the reconstructed structures should be more thoroughly characterized.\n\nQuestions/comments:\n- The model appears to be trained on static snapshots from the trajectory — are dynamics/temporal information included in any way? \n- To test the usefulness of the different intrinsic/extrinsic representations (the main claim of this paper), the authors should perform an ablation study, removing the either the extrinsic or intrinsic components.\n- The intrinsic geometry is represented by Ca-Ca bond distances. These backbone distances typically stay very close to 3.86 A in crystal structures due the chemical constraints from the peptide backbone. Do they vary in the simulation - does it really need to be learned? \n- While L2 loss provides a global measure of the reconstruction error, it would be interesting to report additional metrics on the validity of the reconstructed structures, e.g. steric clashes, invalid bond geometries.\n- It is not clear to me from the visualization in Fig 3 that the drug molecule is “perfectly classified” in the latent space.\n- Can the authors clarify how transfer learning works when transferring the trained model to another molecule with a different number of atoms? What is the purpose of transfer learning? It would be useful to include a baseline of a proGAE model trained from scratch on the S/Protease trajectories in Tab 4.\n- For the transfer learning task, the authors should show visualizations of the reconstruction vs. ground truth as in Fig 2. Again, it is hard to interpret the aggregate MSE error, and I would be interested to see how the reconstruction compares between the different source models. \n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper addresses an interesting learning problem of a generative neural network on a simulated ensemble of protein structures obtained using molecular simulation to characterize the distinct structural fluctuations of a protein bound to various drug molecules. The main technical contribution is a geometric autoencoder architecture with separate latent spaces for representing intrinsic and extrinsic geometry. However, the reviewers think the benefit for modeling intrinsic and extrinsic geometry is not clearly explained and the experiments are not convincing at the moment. The paper can be potentially improved by addressing these two main issues. 
Decision: Reject