Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: In this paper, a new type of molecular fingerprint is introduced, and its application in the virtual screening setup is shown. The fingerprint is based on multiparameter persistence homology that produces topological descriptions of molecules in the form of multidimensional vectors. The methodology expands single persistence methods and applies multiple filtering functions, which can be based either on atom distances or chemical properties. Architectures such as Vision Transformer and ConvNeXt are used to process the multidimensional fingerprints. Finally, the triplet loss is used to train these models for the virtual screening task. The method is compared against many baselines on two datasets. Additionally, the computational complexity is calculated, and the stability of the multiparameter persistence fingerprints is formally proven in the supplementary material.
strengths_and_weaknesses: Strengths:
- The theoretical foundations of the paper are sound. The persistence homology is thoroughly explained, and the modifications are presented using a clear mathematical notation (though some definitions may be found only in the supplementary materials).
- The stability of the proposed fingerprint is formally proven.
- The experimental evaluation on two datasets shows the strong performance of the proposed method. The number of baseline models in the comparison is impressive.
- A section about computational complexity was included in the paper.
- The ablation study corroborates the usefulness of multiparameter persistence and explains architectural choices. 

Weaknesses:
- Unfortunately, the method is not presented in a clear way. Some of the details were moved to the supplementary materials, so it is difficult to grasp the idea before reading the appendix and rereading the paper. For example, VR-filtration is not explained in the main text, but this term is used multiple times. Maybe it would be a good idea to add a figure explaining how the fingerprint is calculated (showing VR-filtering on one axis and property-based filtering on the other one).
- The proposed method shows a great improvement over the other method included in the comparison, but these methods are mostly scoring functions that were trained with a significantly different objective. Please, correct me if I am wrong, but I think that most of these methods use docking poses to score ligands and this is how they build their rankings. The MP Fingerprint, on the other hand, is trained to separate active ligands from decoys only based on the topological features and ignoring the 3D conformations. It would be useful to add a few simple baselines, e.g. a random forest model on Morgan fingerprints (similar to the ablation study, but I cannot find the information about which model was used in this experiment).
- Section 6.4 mentions a comparison of more contrastive objectives, but the (quantitative) results of the preliminary experiments are not included in the paper.
- The code is not provided, which makes the results difficult to reproduce.

Minor comments:
- In the abstract it is said that “VS is used for comparing a library of compounds against known active ligands to identify drug candidates…” This description of VS is inaccurate as it is not the only way to perform virtual screening. For example, there are structure-based approaches to the VS that can be performed even if no active ligands are known.
- It would be interesting to see the performance of the MP fingerprints in different benchmarks, e.g. ADMET property prediction.

questions: 1. How were the atomic properties (atomic mass, partial charges) selected for the evaluation? Could you visualize graph hierarchies w.r.t. these filtration functions for some example compounds?
2. In section 6.1 it is said that “ViT is more robust to distinct arrangements of atoms in space, also referred to as molecular conformation.” Are conformations encoded in the fingerprint? Based on the description in section 4, the fingerprints use only graph edge distances.
3. How is the triplet loss used to predict the ligand ranking, which is used to calculate EF scores?

limitations: In Appendix D, only positive societal impacts are presented. The computational complexity is a limitation that was analyzed in Section 6.3, but not mentioned in the section about method limitations. The number of atomic properties used as filtering functions not only increases the cost of calculating the fingerprint, but also requires more memory and processing power of the ML models.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: 
The authors proposed a new topological fingerprint in virtual screening to understand structural organization of chemical compounds. The method is based on multiparameter persistence homology, which produces topological fingerprints of the compounds as multidimensional vectors. They show that the proposed topological descriptors outperform the state-of-the-art methods on benchmark datasets.

strengths_and_weaknesses: Fingerprinting is an important research topic in drug discovery.
The proposed method provided high accuracy.

There are many previous works on chemical fingerprints and descriptors, but the relationship with related works is not clear.
The authors did not compare the performance with many other fingerprints.

questions: The authors should use popular performance measures such as AUC scores of ROC curve and precision-recall curve. I'm afraid that researchers in the NeurIPS community are not familiar with the performance measures used in this paper (e.g., enrichment factor alpha %).

There are many fingerprints and descriptors in chemoinformatics. Examples include the following papers:
FCFP (Rogers et al, J. Chem. Inf. Model., 2010)
CDK graph (Yap et al, J. Comput. Chem., 2011)
CDK hybridization (Steinbeck et al, Curr. Pharm. Des., 2006)
Klekota-Roth (Klekota et al, Bioinformatics, 2008)
MACCS (Durant et al, J. Chem. Inf. Comp. Sci., 2002)
PubChem (Chen et al, J. Chem. Inf. Model., 2009)
E-state (Hall et al, J. Chem. Inf. Comp. Sci., 1995)
The proposed fingerprint is better than the other fingerprints?

The prediction accuracy in virtual screening depends too much on the number of positive examples (active compounds) in the training set. In my experience, transformer-based methods do not work well when there are few positive examples in the training set.

It is hard to follow this paper.
The explanations on the detailed methods are not clear.

The output of the propose method is not clear.
What is the dimension/size of the proposed topological fingerprint?

The number of compounds in the chemical library is often huge. How is the scalability of the proposed method?

limitations: The paper does not discuss the limitations of the proposed method clearly. 
The performance is not rigorously verified.
review_title: null

final_score: rating: 9: Very Strong Accept: Technically flawless paper with groundbreaking impact on at least one area of AI/ML and excellent impact on multiple areas of AI/ML, with flawless evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: In this paper, the authors develop multiparameter persistence (MP) homology based topological fingerprints for virtual screening. In their model, a compound is decomposed into chemical substructures at different scales and their topological information is extracted by using MP homology features. These topological fingerprints are combined with learning models. Their models are extensively tested on two benchmark datasets, i.e., Cleves-Jain and DUD-E Diverse dataset. It has been found that their models have better performance than existing models. 
strengths_and_weaknesses: The models are innovative and the results are great! However, TDA-based learning models for drug design (including virtual screening) has been a hot research areas for the past decade. There is a great amount of works in this area that is missing in the current paper. 
questions: The major issue is the missing of important literature in the related work. Even though the authors have mentioned some TDA-based drug design models, i.e., refs 13-15, they have been only very briefly mentioned in the supplementary information. Further, these works are from 2017 to 2018. In fact, recently, there are great progresses in TDA-based drug design (and virtual screening). The authors should discussed them in the related works.  

For instance,

1) Duc Duy Nguyen, Zixuan Cang, Kedi Wu, Menglun Wang, Yin Cao  and Guo-Wei Wei, Mathematical deep learning for pose and binding affinity prediction and ranking in D3R Grand Challenges, Journal of Computer Aided Molecular Design, 33, 71-82   (2019).

2) Duc Nguyen, Zixuan Cang, and Guo-Wei Wei, A review of mathematical representations of biomolecular data, Physical Chemistry Chemical Physics, 22, 4343-4367 (2020).

3) Duc Duy Nguyen, Kaifu Gao, Menglun Wang, and Guo-Wei Wei, MathDL: Mathematical deep learning for D3R Grand Challenge 4, Journal of Computer Aided Molecular Design,  34, 131-147 (2020).

4) Zhenyu Meng and Kelin Xia, "Persistent spectral–based machine learning (PerSpect ML) for protein-ligand binding affinity prediction." Science Advances, 7 (19), eabc5329 (2021)

5) Peiran Jiang, Ying Chi, Xiao-Shuang Li, Xiang Liu, Xian-Sheng Hua, and Kelin Xia, "Molecular persistent spectral image (Mol-PSI) representation for machine learning models in drug design." Briefings in Bioinformatics, 23 (1), bbab527 (2022)

6) Xiang Liu, Huitao Feng, Jie Wu, and Kelin Xia, "Dowker complex based machine learning (DCML) models for protein-ligand binding affinity prediction." PLOS Computational Biology, 18(4), e1009943 (2022)

7) Xiang Liu, and Kelin Xia, "Persistent Tor-algebra based stacking ensemble learning (PTA-SEL) for protein-protein binding affinity prediction", ICLR 2022 Workshop on Geometrical and Topological Representation Learning (2022)

8) Xiang Liu, and Kelin Xia, "Neighborhood complex based machine learning (NCML) models for drug design." In Interpretability of Machine Intelligence in Medical Image Computing, and Topological Data Analysis and Its Applications for Medical Data, pp. 87-97. Springer, Cham (2021).
  
limitations: None.
review_title: null

Meta-review: metareview: The reviewers mostly liked the paper. They mentioned the sound theoretical foundations, stability, and strong empirical performance. The rebuttal was able to convince most reviewers that the paper should be accepted for NeurIPS. 
confidence: Certain
Decision: Accept