Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors propose a transformer model that is trained to learn the force-field relationship of molecules. It has the ability to predict molecular energy without expensive molecular dynamics computation. To achieve this, they use a transformer with a molecular attention block to predict atom force and bond force and then calculate the energy.  A physics-driven parametric energy model, masked atomic modeling, and physics-inspired constraints are combined to improve energy prediction accuracy. On QM9, the model surpasses previous models in terms of force prediction accuracy and structural stability prediction.
strength_and_weaknesses: Strength: 
1. Effective use of physics inductive bias. The force-field relationship is crucial for accurate quantum energy understanding. 
2. Gives a good energy landscape in terms of molecular structure space. It seems similar in principle to molecular dynamics and helps understand why this method work.

Weakeness: 
1. I'm interested to know why the result in Table 1 shows that the force prediction is more accurate yet the energy prediction is not. I wonder if the LJP model gives too much constraint on the energy prediction and affects performance. 
2. The experiments are not complete enough: 
  - Only performance on QM9 is shown, which is ok since its the standard benchmark in this field. Other datasets (e.g. OC20 ) could also be used as QM9 is often criticized for its non-equilibrium structures of molecules. 
  - Strong baselines are missing. ForceNet[1] also learns force in addition to energy and reaches similar resolution to this method in terms of 
force prediction.

[1] Hu, W., Shuaibi, M., Das, A., Goyal, S., Sriram, A., Leskovec, J., ... & Zitnick, C. L. (2021). Forcenet: A graph neural network for large-scale quantum calculations. arXiv preprint arXiv:2103.01436.
clarity,_quality,_novelty_and_reproducibility: - The writing is overall clear and of high quality. 
- The paper is novel in method, but the idea of using force-field relationship in prediction is not quite novel.
- For reproducibility, the code was not included in the submission. The (optional) reproducibility statement was not included in the paper.
summary_of_the_review: The paper is overall clear and well-written. The physics inductive bias seems to be effective. However, the experimental results are not strong in terms of energy prediction. Therefore, I recommend it as borderline reject and I am willing to increase the score if the authors could clarify why the force accuracy is better than energy prediction.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors proposed a physical-driven model and regularization scheme to predict energy for molecular systems. Compared to other baseline models, their full model preforms much better on force prediction, while the energy MAE is slightly lagging behind the state-of-the art MXMNet model. When applying to the structure optimization task, the full model outperforms the other base line models, which demonstrates that the physical-driven model captures the basic physical information instead of simply over-optimizing on the energy estimation. Besides, inspired by the Masked Language Modeling, the authors proposed Masked Atomic Modeling to help the model to discover chemical restrictions.
strength_and_weaknesses: Strength:

The physical-driven model outperforms the other base line models in the realistic structure optimization task, and the output structures are more physical than other models.

The loss terms are physical meaningful and effective, which shed light on future research.

Weaknesses:

The authors didn’t compare the mask, force and bound terms separately, to see which is the most relevant in terms of the performance improvement.
clarity,_quality,_novelty_and_reproducibility: The paper writing is clear and the point of view is novel enough. There is sufficient information for reproduction.
summary_of_the_review: In summary, the paper proposed a physical-driven model, which performs better on molecular structure optimization and force prediction than the state-of-the-art models. The model may capture the true underlying physics rather than simply fitting. The ideas in the paper would pave the way to more accurate and general physical-driven models in the future research. For these reasons, I recommend receiving this article.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a novel masked atomic modeling framework for molecular representation learning. Their architecture combines several problem-specific designs, including Molecular Attention Block and other physics constraints as loss functions, including Zero-Force Regularization and Bound Condition. Empirical results show that the proposed method outperforms previous popular frameworks such as DimeNet. 
strength_and_weaknesses: Strengths:
1. This paper applies masked atomic modeling to molecular modeling, similar to masked language modeling. The most important part is that the model is not just naively using a standard transformer but incorporates a lot of problem-specific design.
2. This paper shows good empirical results compared to previous methods. 

Weaknesses:
1. The dataset and the molecules in the study are relatively small. Also, all experiments are conducted on the same dataset for one task. It is questionable how such learned representations can be used for other datasets/tasks.
2. The ablation study is not clear. It only shows several extreme comparisons of removing several important loss function terms. However, it is questionable how each design component is effective. For example, what is the optimal mask ratio? 
clarity,_quality,_novelty_and_reproducibility: The paper is well written.
There are missing details such hyperparameters.
summary_of_the_review: This paper proposes Masked atomic modeling for molecular representation learning. It also incorporates several physical constraints effectively. I think this is solid contribution to the community.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The manuscript describes a Transformer-based neural network aimed at predicting the energies and the forces of molecules. The network is trained on the prediction of the energy of ~130,000 small molecules. The loss includes the square deviation between the ground truth energy and the predicted energy, and also terms aimed at ensuring that the "experimental" structures are stable.  The energy estimator includes, beside the transformer, a  two-body energy, eq 5. The model predicts the energy with an accuracy which is marginally worse than that of other approaches, but it allows recovering the correct structure in geometry optimizations started from perturbed configurations.
strength_and_weaknesses: Strength:
1) the idea of training  a network using a BERT-style censoring of the position of some atoms is, to my knowledge, original.

Weaknesses:
1) the challenge in the field is reproducing the forces (and the energies) along a *reaction pathway*, namely for sequences of configurations connecting two different molecular species, or two different configurations of the same molecule. The forces along these pathways are typically very high, and determine the chemistry of the system.  The test set used in this work is therefore mostly of academic interest: the forcefield developed in this work is good for reproducing the geometry of the molecules, but worse than other approaches in reproducing the energy difference between molecules. No relevant "activated" geometry is included in the test set. 
2) the idea of fitting the forces, and not only the energy, dates back to the 90ties (see wiki, "force matching") and is  used almost ubiquitously also in NN training (see, for example, https://pubs.acs.org/doi/10.1021/acscentsci.8b00913 eq 9). The fact that by adding a penalty on the value of the forces allows recovering the correct  geometry is correct, but trivial, since the configurations explored in these optimization are very similar to the global minimum. The relevant result would be reproducing the forces along a reaction pathway (see point 1).
3) The idea of adding explicitly LJ or Coulomb interactions in a NN is also not new, but, more importantly, adding these terms for isolated and very small molecules is not chemically justified. In fact these terms are typically added to describe liquid environments (see for example https://www.nature.com/articles/s41467-020-20427-2 ), where different molecules interact by non-bonded interactions. 
clarity,_quality,_novelty_and_reproducibility: The manuscript is well written, easy to understand, and the results are clear. Part of the approach cannot be considered novel, as  outlined above.
summary_of_the_review: The results presented in the manuscript are mostly technically correct, but, in my opinion,  do not advance significantly the state of the art of machine-learning based potentials, since the numerical tests are performed on tasks which are not of practical use in molecular modelling. Some of the directions which are explored , for example the BERT-style masking loss, are interesting and could bring to important advances.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: All reviewers raised several points of criticism, such as missing novelty of concepts,  unclear ablation studies and not entirely convincing experimental validation. Even after the rebuttal and discussion phase, these concerns largely remained and all scores were in the negative range. I mostly agree with this over-all perception, and therefore I vote for rejection of this paper.    
justification_for_why_not_higher_score: I think I just described this in the meta review above...
Decision: Reject