Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors propose TensorVAE, a relatively simple model for generating 3D conformations from 2D molecular graphs. TensorVAE employs 1) a unique feature engineering step that represents each molecule as a tensor (with or without 3D coordinates and distances), and 2) a VAE with two transformer encoders, one that encodes the graph (using the tensor input without 3D information) and the approximate posterior that produces latents from the 3D information, and a transformer decoder for the likelihood, where keys and queries come from the 2D graph representation, and values are the latents from the 3D posterior encoder. The loss is a standard roto-translation invariant loss.  The authors show that, using standard transformer architectures and training procedures, TensorVAE performs comparably to the best current methods at conformation generation using the GEOM dataset, and argue that TensorVAE is much simpler than the comparable models due to superior feature engineering.
strength_and_weaknesses: Strengths:
- Simple architecture and featurization
- Original use of two encoders for 2D and 3D features in the VAE formulation that allows for 3D conformation generation from 2D graphs
- (Near) state-of-the-art results for conformation generation (and QM9 property prediction)

Weaknesses:
- It is not clear that the proposed method of producing atom-tokens via 1D convolution on the input tensor is necessary. Any number of possible aggregation steps could have been used, many of which would likely have yielded similar results. One could also imagine performing a Tucker decomposition and using singular values as tokens, etc.
- Moreover, it's not even clear if the tensor formulation is needed. The 1D convolution aggregates information about the radius-1 atomic environment (including virtual bonds). One could use any radius-1 atomic-environment hash as tokens with the proposed featurization, which would likely yield comparable results.
clarity,_quality,_novelty_and_reproducibility: The presentation is clearly written, and the work appears to be of sufficient quality.

The feature engineering is marginally original (see weaknesses above). Combining 3D latents as values with 2D embeddings as keys and queries in the VAE formulation appears to be a novel way of learning to generate 3D confirmations from 2D graphs.

The authors claim everything is straightforward and provide no code.
summary_of_the_review: The paper extends existing methods and molecular featurizations to achieve near state-of-the-art results for conformation generation. The paper spends a lot of time arguing for the superiority of the feature engineering used, but it is not at all clear that the results depend on that featurization. The authors could greatly improve this paper by demonstrating the value of the feature engineering beyond performance on benchmarks through ablation studies (e.g. removing aspects of the features) and studies of other featurizations that are not conflated with the tensor design (e.g., radius-1 atom environments).
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper tackles the generation of 3D conformations of a molecule from its 2D graph. They propose to encode a molecular graph using
a fully connected and symmetric tensor. They use the standard VAE framework, where they build two input tensors with one encoding only the 2D molecular graph and the other also encoding 3D coordinates and distance. Both tensors go through the same feature engineering step and the generated feature vectors are fed through two separate transformer encoders. The output of these two encoders is then combined in an intuitive way to form the input for another transformer encoder for generating confirmation directly.
strength_and_weaknesses: Strength:
the idea is to combine Atom and edge features into a single input by adding an additional dimension to
the adjacency matrix, making it a tensor where the diagonal section of the tensor holds the atom features sounds interesting.

Weakness: the models seem to be really big (because of the input tensor)
clarity,_quality,_novelty_and_reproducibility: The paper is well-written and the proposed method seems novel and interesting to the community. For the model to be fully reproducible, I detailed the experimental setup and the code should be published in the camera-ready version.
summary_of_the_review: 

Some detailed comments: 

1. I wonder why the model needs two conditional encoders, one conditioned on the graph only and another one conditioned on both the graph and coordinate, why not directly use the latter since it has also graph information? 

2. I am a bit confused about section 2.3 where the second main idea is explained.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper illustrates a method for how to train a neural network to predict 3D conformers of small molecules starting from their 2D structures. Building on previous works such as CVGAE, the authors propose a few tweaks which are meant to improve the models’ performance: 1) use a single tensor to represent both atom and bond information, and featurize the single tensor into atom-level vectors using a 1D convolution operation on the tensor; and 2) use a transformer architecture and combine 2D and 3D representations during training using a variant of cross attention. The authors demonstrate that with their method, they can achieve performance comparable to other similar SOTA methods.
strength_and_weaknesses: The paper presents the authors’ main ideas in a largely clear manner, and it introduces and mostly justifies the choices made. The implementation details are fairly clear. The authors also present an interesting analysis on the loss curves which show a tradeoff between the reconstruction and variational losses, which is very appreciated. The authors are relatively thorough in their benchmarks with other works.

There were, however, some areas where the paper makes some design decisions which might not be optimal:
### Convolving over the input tensor with both atom and bond features can be problematic
The input tensor that is constructed is very heterogeneous, because it contains both atom feature vectors and bond feature vectors (side note: presumably, the atom feature vector is padded with 0s in order to make the two vectors equal size). Because the tensor is structured such that the atom features are along the diagonal, this means that a convolution operation over the tensor is not uniform. That is, consider an $N \times 3 \times 50$ convolutional kernel. The entry at $(0, 1, 0)$ will be multiplied against an atom feature in the first sliding window, and then it will be multiplied against a bond feature (which is twice the magnitude of the atom feature) in the next sliding window. This leads to the same entries in the kernel to experience entries in the tensor that are of varying magnitudes and meanings, which is suboptimal.
### Convolution is not particularly natural for this kind of input
Firstly, the input tensor is symmetric about the diagonal so the convolution sees redundant information. Combined with the issue brought up above, the redundancy structure is different for every sliding window (i.e. sometimes entries near the top and bottom are redundant, and other times entries near the middle are redundant).

Secondly, because the molecules are variable sizes, the input tensor will need to be zero padded to a maximal size. This means that the same kernel will also experience zeros when there is padding. This can also lead to confusion because zero is also a meaningful value in the input tensor (which is mostly one-hot encoded)
### Kabsch alignment shouldn’t be necessary
The use of Kabsch alignment (or alignment in general) should not be necessary. Transformers retain the ordering of the input vectors/tokens. That is, the $i$th input token is mapped to the $i$th output token through the attention mechanism. This means that the use of alignment should not be needed, as each atom/token can be mapped to an output token.
### Variation of cross attention is used without much justification
This is more of a minor point, but cross attention typically combines the key and value of one sequence with the query of another sequence. This work computes the key and query of the 2D tokens with the value of the 3D tokens. This is somewhat distinct from the traditional cross-attention mechanism. It would be good to have a justification for why this might work better than traditional cross attention.
clarity,_quality,_novelty_and_reproducibility: The overall flow of the paper is quite good, and the explanations and figures and diagrams are very much appreciated. The authors do a good job of explaining their work. There are, however, several typos which need to be fixed. Here is an incomplete list to give an idea of the typos that I found:
- “indirect-leveraging” in the abstract should be an em-dash instead of a hyphen
- Figure 3 says “trainformer”
- The caption in Table 1 (and in the supplement) say “parenthathese”
- Figure 5 caption says “validatiom”

There are also numerous grammatical issues, which I hope can be fixed, as well.

In terms of novelty, the paper is more limited. The novelties of the work (compared to previous works such as CVGAE and DMCG) are effectively to introduce a new format of input features, and use a variant of cross attention. Although these contributions are not trivial, the resulting model also does not demonstrate a significant improvement in performance over previous models (although they are somewhat more efficient).
summary_of_the_review: The paper (other than the grammatical issues and typos) is well written, and flows nicely. It is informative and the figures are generally very enlightening. The benchmarks are decently well done, as well. However, the paper demonstrates limited novelty in the technical space, as the proposed tweaks to existing work are either questionable or small. The result of these tweaks also do not seem to lend to significantly improved performance beyond the existing methods.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper presents a CNN/transformer based variational autoencoder for generation of molecular 3d structure from molecular graphs.

strength_and_weaknesses: Strengths:
The paper deals with an important problem and presents a working solution that is fairly simple and effective.

Weaknesses:
The novelty is limited - the main technical contribution appears to be a model that works well, built using standard CNN, transformer, VAE components.

It is unclear if the comparison with other methods is fair - results are taken directly from previous papers but is it the same test set that is used? According to the text, the authors randomly select 200 molecules. 

Confidence intervals are not provided for the results. With such a small test set, significance is difficult to determine.

Because a CNN is used, it appears that the method is not invariant to permutation of the atoms. It is not clear if this has negative implications for the performance.

It is not clear to me what the motivation is behind combining node and edge features in a tensor. This restricts the dimensions of the node and edge features to be identical, but on the other hand allows for standard CNN code to run on the representation. Is that the reason for this choice?

The relation of the proposed model to common GNN architectures is not fully described. If I understand correctly, a kernel of size 3xN is used? If it had been a 1xN kernel, I believe it would correspond exactly to a fully connected GNN which is commonly used. 

clarity,_quality,_novelty_and_reproducibility: The paper is fairly easy to follow, and the main ideas are presented in sufficient detail.

There are several typos and grammatical errors etc. throughout the paper, which are easily fixed (not important for my assessment).

The definition of COV in eq. 4 seems to be incorrect (should not be for all R hat in C_g)

There appears to be some typos in the results in table 1 (at least in comparison with the corresponding table in the Uni-Mol paper.)
summary_of_the_review: While I find the subject of the paper very interesting, my primary reason for recommending rejection is the lack of novelty and the limited empirical evaluation.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: For this paper, we had three (slightly) negative reviews and one clearly positive review.  Summarizing all reviews, the rebuttal and the following discussion phase, I finally came to the conclusion to vote for rejection of this paper. Let me explain this in more detail:

On the positive side, all reviewers mentioned that the motivation is clear,  the general idea is interesting and the underlying problem is of high importance. On the other hand, several points of criticism have been raised, such as:
- the quality and interpretation  of (some of) the comparison experiments was criticized by some reviewers. Some of these concerns could be addressed in the rebuttal, so I think this is only a minor point of criticism. The more important points are:
-  novelty and significance of the paper are limited, a conceptual contribution is not clearly visible. I think, this is a very valid concern that could not be addressed in the rebuttal. Even though some (of the more negative) reviewers pointed out that they were positively impressed by the detailed rebuttal in general, they still wanted to keep their negative scores.   
- the use of convolutions is somewhat questionable, due to heterogeneity issues in the input tensor. I agree with some of the reviewers that this is a critical concern that questions the general idea of using convolutions in such settings.  Also this issue could not be fully resolved in the rebuttal, and many open questions remained.

The only positive review included only somewhat general comments and the reviewer declared a relatively low confidence. Further, this reviewer finally did not clearly champion this paper for acceptance. 
  
justification_for_why_not_higher_score: There were too many points of criticism that could not be resolved convincingly in the rebuttal.
Decision: Reject