Paper: Ophiuchus: Scalable Modeling of Protein Structures through Hierarchical Coarse-graining SO(3)-Equivariant Autoencoders (ICLR-2024)

Reviewer comment (Theory):
In the abstract, the authors say the proposed model focuses on local convolutional coarsening to model sequence-motif interactions in log-linear length complexity. However, I cannot find the complexity analysis in the manuscript.

Author response:
We added a full analysis of the time complexity of our model in Appendix A.10

Reviewer comment (Writing):
The authors need to examine all the formats of the references that appeared in the paper, whether using () or [] or other formats. Typos: Translation Equivariance of Sequence: One-dimensional Convolutional Neural Networks (CNNs). Colon or dot. In Figure 2, there are two (d).

Author response:
Thanks, we fixed the typos and the format of references.

Reviewer comment (Reproducibility):
Sometimes, the authors say all-atom protein structures are used, and sometimes, only the heavy atoms are used. It confused me. In Algorithm 1 and Algorithm 2, l=0:2, but in Algorithm 3, l=0:l_{max}. I wonder how to get all atoms' representations from v^{l=0:2}.

Author response:
In the context of protein modeling, all-atom structures are often interchangeable with "all-heavy atom". Thanks for the note, we made the usage of the term consistent throughout the paper.  

Algorithms 1 and 2 are used to encode and decoded the protein into its initial representation which uses degree up to 2. Algorithm 3, however, may be used to produce features of larger degree $l$, through its internal use of tensor square. Because $l_{\textrm{max}}$ is a hyperparameter, the model will build up the geometric representation up to this degree.

Reviewer comment (Presentation):
In Figure 1 (b), why does the proposed model use three building blocks? What do these three building blocks mean? Figure 3 for Protein (PDB ID: 1S9K and PDB ID: 2O93) and Figure 8 for Protein (PDB ID: 4JKM and PDB 6LF3) are very similar. I suggest the two pictures be put together. Where are their similarities and differences?

Author response:
We put the figures together and made a more concise analysis of the two studied interpolations.

Reviewer comment (Experiments):
The authors say the permutation invariance of particular side-chain atoms is preserved, and a one-dimensional roto-translational equivariant convolutional kernel is designed. Are there any demonstrations to illustrate the invariance and equivariance?

Author response:
We illustrate the permutation invariance of the side-chain feature parameterization we use. We added a more detailed discussion in Appendix A.1, and incorporated a visual demonstration in Figure 4.

Reviewer comment (Experiments):
In Section 4.1, why choose the contiguous 160-sized protein fragments? Why train the all-atom generative model for miniproteins instead of large proteins? Why is the model trained in ten epochs?

Author response:
When choosing our input sequence lengths, we restrict our choices of length to those that will result in perfect alignment of the convolution kernel windows when downsampling down a single representation. For the model settings of our ablation studies, with stride S = 3 and kernel size = 5, the sequence lengths that satisfy this constraint are [1, 5, 17, 53, 160, 485]. 

We chose ten epochs as we found it to be enough to gather data about training curves and convergence, while still catering to our limited computational resources. 

We focused on mini-proteins because our all-atom model performs well at the capacity of our training. For this updated manuscript, we included additional benchmarks on large backbone models that we compare directly to existing architectures.

Reviewer comment (Experiments):
The experiments of conformational interpolation and structure recovery latent diffusion lack comparison methods.

Author response:
To the best of our knowledge, we provide the first demonstration of latent interpolation maintaining consistent chemical validity throughout the trajectory without explicitly enforcing any physical behavior in the latent space. Other methods exist which rely on physics based simulation data for interpolation. 

We also added structure recovery comparison in Section 4.1

Reviewer comment (Experiments):
The proposed model is not compared to existing work via experiments. This can be adequate to reject this paper unless the authors can justify why a comparison to other work is not applicable in rebuttal.

Author response:
Thank you for the feedback. Please note the comment above about comparison and baselines.

Q3: In Section 4.4, we added comparisons against existing generative models, and added more metrics on diversity and consistency of samples.

Reviewer comment (Theory):
Motivations about the model structure design are missing in this paper, including an explanation of why the whole model should contain those substructures and why each structure should be like that.

Author response:
Q1: We added motivation for the model design at the start of Section 3, as well as at the start description of each component in Sections 3.2-3.4

Reviewer comment (Reproducibility):
A formal (mathematical) expression of the objective function is missing. The current expression form is too general to give necessary information about how to train the model to authors.

Author response:
Q2: We added much more detail on the losses, in Appendix B. In particular, we further elaborate on the chemical losses in Appendix B.2 and include an expression for the final objective function in Appendix B.4

Reviewer comment (Evaluation):
The claimed generation power is not evaluated.

Author response:
Q3: In Section 4.4, we added comparisons against existing generative models, and added more metrics on diversity and consistency of samples.

Reviewer comment (Writing):
Some expressions look not academic, e.g. "we introduced a new unsupervised autoencoder" (all autoencoders are unsupervised), "multi-layer hourglass-shaped autoencoder" (all autoencoders are hourglass-shaped, most are multi-layer).

Author response:
Q4: Thank you, we fixed the mistakes.

Reviewer comment (Theory):
In Sec. 3.1, it is not clear how the $V^{l=2}$ unsigned difference vector is computed and Apx A doesn't include further details about it. Is it obtained by taking the absolute value entry-wise of the difference vector? However, isn't $V^{l=2}$ supposed to be a 5-dimensional vector transforming under the order 2 Wigner D matrix?

Author response:
Please refer to the updated manuscript. We added much more detail on the encoding of $\mathbf V^{l=2}$ in Section 3.1, and added diagrams (Figure 4) and further discussion in Appendix A.1.

Reviewer comment (Writing):
The idea that the atoms in standard residues can typically be ordered a priori seems very important but is never explained well, so it deserves some additional discussion (in particular, the cases where residues present two-permutations symmetries). The reviewer notes they are not as familiar with this type of task, and it might be well known in the community.

Author response:
In Section 3.1, and Appendices A.1 and A.2, we added details and discussion about the permutation symmetries. In Appendix A.1, Figure 5, we also added a visual demonstration that shows how our proposed feature representations handle the problems that might arise due to the symmetries of residues.

Reviewer comment (Theory):
In Sec. 3.3 it is not clear why the normalization of the weights ensures translation equivariance.

Author response:
We added Appendix A.5 where we derive the conditions for coarsening coordinates while satisfying rotation and translation equivariance.

Reviewer comment (Writing):
In Sec. 3.1, 4-th line: $P_i^{0, l=0}$  -> $V_i^{0, l=0}$ ?

Author response:
Thanks, we fixed the minor issues.