Peer review of: Representation of molecular structures with persistent homology for machine learning (2020)


 

Reviewers' comments: 

Reviewer #2 (Remarks to the Author): 

The authors have provided substantive changes to the manuscript and addressed most of the raised 
points in detail, which I appreciate. This certainly helps to alleviate any concerns on the validity and 
significance of the performance of the method and the presented application. The method is 
interesting, the work is thorough, so it should be published. 

Regarding the question if the paper will influence thinking in the field, I am not sure. 
The presented homology feature is clever and the concept underutilized in chemistry, but it does not 
provide for dramatic prediction improvement compared to existing features. I am still not able to see 
what the killer application is for which this feature will outshine everything else. Catalytic high-
throughput screening is probably not that application as all other (efficient and computationally costly) 
features seem to do reasonably well (at least within the accuracy of the underlying electronic structure 
methods). Maybe the authors could provide more outlook on that. 

One more very minor point: 

Rewording size-consistent as constant-size representation will surely help to avoid confusion, but is PD 
really of constant size? The example that is provided for small and large molecules deliver 
representations that are very similar in size (3x3A) vs. (4x4A), but not identical if the pixel spacing is 
the same or there is no padding (which would be absolutely fine if used). What is the actual length of 
the contracted feature vector? For very large molecules, there are many more features, so the pixel 
density needs to be increased as well or the image needs to be sampled and processed somehow 
differently. This should be explained better, if it is to serve as a key advantage of the method. 

Reviewer #3 (Remarks to the Author): 

Revised manuscript is an improvement over the previous version as they improved their ML algorithm 
and compared to additional existing approaches. They considered most of criticism, but not all or not 
appropriately. 

The authors highlight smaller error bars (not energies) for PI than for SOAP for CO2-binding energy, 
but if you look at N2-binding energies then all models are practically the same. Is there any reason for 
this? 
The modified Figure 6 is not appropriate because it is based on a biased approach. The training set 
was selected via active learning that was run only with PI, not with other ML approaches. Their results 
for the initial 100 training points do not show so big difference between methods. More appropriate 
would be to generate plots like Figure 6 after each iteration (at least for the initial 100 points) and run 
each iteration with the corresponding approach (CM, BoB, SOAP). 

It would be good if the authors could provide a concise figure with molecular groups and discussion 
addressing their declared aim "to screen a large molecular database in order to discover molecular 
groups that show a stronger affinity for CO2 interaction over N2". Making unbiased comparison (see 
previous comment) with different methods in this regard would be also helpful 

It looks strange that after the authors made changes to the ML model, it did not change anything in 

the discussion of the active learning procedure, i.e. in both revisions they write e.g. "two anionic 
structures were found during the third step", "mean and median both at -5.38 kcal/mol.", "the second 
iterations, at -5.43 and -5.26". 

The authors changed from the claim that their method is size-consistent to that their method is 
constant-size, but then for anisole the size of PD is 3x3, while for tert-butylcalixarene it is 4x4. Yes, 
they similar, but apparently not constant. So their model is neither size-consistent nor constant-size. 

In the rebuttal letter they make claim that SchNet and Behler-Parrinello are not suitable, because they 
represent energy as atomic contributions, but ML models with SOAP are typically considered to do just 
that (see Ref. 60 for example). If the authors did not do this in this manuscript, then there is a danger 
that SOAP was not really used properly. It still gives rather good results. 

"In generating molecular representations from a molecular structure, the degrees of freedom of the 
structure are inherently reduced, where many geometric properties are truncated and passed to the 
ML model." This claim is dubious. For examples the authors given (Coulomb matrix etc.) the degrees 
of freedom measured by number of features for a molecule actually increases. 

In rebuttal letter, the authors write that "the pixels are not symmetric because of the location of the 
mean of the Gaussian with respect to the grid used.", this point is still not clear, in the SI one can find 
"Gaussian (birth, persistence)+N(0,σ^2∙SF_ij)", i.e. they use normal distribution with zero mean... 

The authors still do not describe how they chose the original 100 molecules, Section S1 they refer too 
does not say anything. Were they chosen randomly from QM9? 

Authors again refer to some non-existing sections, e.g. 'Section 3.2', even after pointing out this type 
of problems in the comments to the original submission. I do not list here all the sections (important 
note, because authors only fixed the Section I mentioned). 

I still could not find energies and geometries of N2 and CO2 in SI or on their gitlab, although authors 
claimed that they added them... 

Fix: 
* The authors still did not correct one minor issue, although in rebuttal letter they claimed they did: 
"However, the number of potential CO2-philic groups is intractably large, which leads to an excessive 
study of such systems with accurate ab initio methods" should be moved one sentence up. Otherwise 
it does not make any sense. 
* 'depend on the to distances' 
* Please do not invent terms for 'Laplacian kernel ridge regression' and 'linear ridge regression'. They 
are kernel ridgre regression with the Laplacian or linear kernel. 
* Figure S12 -> Table S12.