Peer review of: Representation of molecular structures with persistent homology for machine learning (2020)

Reviewers' comments: 

Reviewer #2 (Remarks to the Author): 

The authors present a molecular representation, which they consider to be easy to calculate and of the 
same size across all molecules (size-consistent). The authors apply it to train the CO2/N2 interaction 
energies of organic molecules within the GDB-9 set. The authors claim that this representation 
generally outperforms other representations. 

The simplicity of the descriptor is very appealing and it appears to provide performance that is en par 
with the, by now outdated, coulomb matrix and BOB descriptors, but I do not see how the claim of 
'generally outperforming other representations' is sufficiently substantiated in the manuscript. 

-Within the error bars provided in Figure 3, BOB and PI show the same performance for a very small 
set of 100 interaction energies. I do not find this convincing. For such a small set of molecules, the 
authors could have systematically assessed a number of more state-of-the-art representations, such 
as FCHL, Behler-Parrinello, SOAP, or SchNet (not a descriptor per se, but a comparable approach). 
This would also have provided an opportunity to showcase the computational efficiency of PI, which is 
emphasized in the manuscript, but not quantified. 

-The training protocol based on a series of active learning steps is well designed and seems 
appropriate for the envisioned application. As it only requires a small number of molecules, I wonder 
why the authors have not also trained a model for CM, BOB and PI to compare the methods within 
Figure 6. This would have provided for a more robust benchmark. 

- I worry about the significance of the findings in Figure 6 and the CO2/N2 interaction energy ratios 
that are discussed. All N2 interaction energies fall within a range of 1 kcal/mol, which is somewhat 
below the trust radius of PBE0-D3. The authors select PBE0-D3 based on a benchmark of CO2 
interaction energies against coupled-cluster results, but it is not clear from the manuscript or SI if this 
level of confidence also applies to N2. 

- It is not clear what the benefit of a 'size-consistent' descriptor is. As the authors state, all descriptors 
can be padded to be of consistent size for molecules of different composition. The authors do not 
provide evidence for cases in which there would be a material benefit to size-consistency. How is this 
property particularly beneficial in the proof-of-principle application that they discuss? Figure 2 also 
shows that the range that is covered in the pixelated diagrams that correspond to the final descriptor 
depends on the size of the molecule (no. of births and level of persistence can vary). This means that 
the size of the Birth/Persistence axis is controlled by the largest and most complex molecule in the 
dataset. One could consider this to be a form of zero-padding when the same range is applied to 
smaller/less complex molecules. 

Reviewer #3 (Remarks to the Author): 

The manuscript introduces persistent homology-based method for transforming 3D molecular 
geometries into ML features. The goal is to use this approach for high-throughput screening of 
molecular motifs that selectively bind CO2 in presence of N2. The idea is definitely interesting and new 
for quantum chemistry. My comments are given below. 

1. The description of methodology is very unclear, especially that involving Figure 2 and other 
information in the SI. What the authors mean by 0D and 1D diagram etc in Fig. 2, when in text they 
talk about 1-D and 2-D images? How do they define dimensions? From SI, for a single atom they call 
it 0D, for two connected atoms they call it 1D although is is still 0th-order hole. In Fig. S3 they give a 
single value for each element, but in text they discuss 1D PIs formed by several elements (btw, is the 
number of involved atoms limited?). It is not clear either, why in Fig 2h the pixels are not symmetric, 
i.e. is it related to the orientation of a molecule H-Br? 

2. Likely related to comment 1, the authors write in the SI that "... each xyz file that holds the 
Cartesian coordinates is first sorted by its elemental weight. This ensures that the persistence 
diagrams created for two different ordering of the same molecule are the same." How come that the 
order of elements affects the variance, if it is just some function of means of electronegativities of 
atoms involved in formation of 0th-order holes? And if the order is important, why is it not important 
to sort atoms with the same nuclear weight? 

3. Probably the most critical comment. The authors did not convincingly demonstrated that their 
approach is size-consistent. Neither they performed numerical experiment that proves this claim, nor 
they provide clear mathematical proof. They did not show that training on smaller molecules will still 
provide reasonable ML predictions for larger molecules. They mention that the intensities on persistent 
surface (what is it anyway?) depend on number of specific connected components. It means that 
larger molecules will have higher intensities for the larger number of specific connected components. 
Then does the pixalation normalizes these intensities? Then for example how ML model trained on a 
smaller oligomer will predict larger energy for a larger oligomer? 

4. The results of their paper do not really support their claims that their method "generally 
outperforms other representations" as results obtained with BoB are often comparable or even better 
than results with persistent images. 

5. The authors do not really describe how they chose the original 100 molecules. They refer to "the 
procedure described in Section 5.1." There are no numbered sections to start with (often, they send 
the reader to Sections with different numbers, which are non-existent). 

6. They never explain in the manuscript what ML algorithm they used and why. Only in the SI the 
reader has a chance to learn that they used random forests, although it is not clear how is it more 
advantageous than other, more common in quantum chemical research ML algorithms. Generally, it is 
known that random forests are not as accurate for regression as other methods (Gaussian processes 
etc.). The poor choice of ML algorithm may explain rather high RMSE of 0.5 kcal/mol for noncovalent 
interactions. 

7. Their final aim is to screen molecules with highest selectivity towards CO2, however they never 
really discuss or use as their selection criterion the ratio between binding constants. It is not clear 
without such calculations how the three molecules they claim should be the best for separation, are 
really better than for example 4,5-diamino-1H-imidazol-2-ol, which is strongest binding to CO2 and 
only has ca. -2.8 kcal/mol binding to N2. 

The manuscript has also numerous minor issues: 
8. No energies and geometries of N2 and CO2 in SI, so no binding energy can be calculated. 
9. a ML -> 'an ML' 
10. greenhouses gases -> greenhouse gases 
11. References for the first sentence and for "At an industrial level, liquid amine-based solvents are 
used for separation and capture of CO2 via chemisorption, but the solvent regeneration step is an 

energy intensive process." 
12. "However, the number of potential CO2-philic groups is intractably large, which leads to an 
excessive study of such systems with accurate ab initio methods" should be moved one sentense up. 
13. "materialsørskov2013ørskov2013ørskov201332? –36" ??? 
14. "For chemical applications, this occurs through molecular representations, which are the 
featurization of molecular compounds from their three-dimensional structure into a vector of values." 
Their are other representations possible not based on 3D structure, e.g. SMILES. 
15. 'intersections leads' -> 'intersections lead' 
16. 'eg.' -> e.g. 
17. 'eq.' -> e.g. 
18. 'would be prohibit' -> 'would be prohibitive' 
19. In each iteration of the active learning, do you choose next 40 molecules not present in the 
previous iteration? 
20. 'this strategy allows ***the the*** model to ***encounter*** these rare instances within the 
training set' -> 'this strategy allows the model to account for these rare instances within the training 
set' 
21. 'cross-fold validation errors' -> '10-fold cross-validation errors' 
22. 'that that' -> 'that'