Paper:
Natural products (NPs, compounds derived from plants and animals) are an important source of novel disease treatments. A bottleneck in the search for new NPs is structure determination. One method is to use 2D Nuclear Magnetic Resonances (NMR) spectroscopy, which indicates bonds between nuclei in the compound, and hence is the "fingerprint" of the compound. Computing a similarity score between 2D NMR spectra for a novel compound and a compound whose structure is known helps determine the structure of the novel compound. Standard approaches to this problem do not scale to larger databases of compounds. Here we use deep convolutional Siamese networks to map NMR spectra to a cluster space, where similarity is given by the distance in the space. This approach results in an AUC score that is more than four times better than an approach using LDA.

Review:
This paper identifies structure determination of novel natural products as a bottleneck in the drug discovery pipeline. The authors address this problem by using deep siamese neural networks to learn an representation of 2D NMR spectra that facilitates the rapid comparison of spectra for novel compounds with a database containing the spectra of molecules of known structure. 
The authors have laboriously collected spectra from the published literature, using a protocol that included manual processing and orientation steps, resulting in a set of 1,385 NMR spectra from 104 families after imposing a minimum requirement of 10 compounds per family. They then implement two baseline topic modeling techniques - PLSI and LDA, and compare the performance of these baselines to the deep siamese CNN that they develop. 
Using the random test/val/train split, the deep siamese CNN does show improved performance over the baseline models. However, the interesting and relevant use case is that in which the data is not randomly split, but rather complete families are held out from the training set for use in testing. The authors address this task by building a split in which they hold out four families: the aphanamixoids, teuvissides, tasiamides and macrolactins. 
To evaluate performance, they calculate the averaged Tanimoto score for the top five closest compound families of these probe families. Unfortunately, the authors were not able to carry out this calcuation for two of the four probe families, restricting the evaluation to the aphanamixoids and teuvissides (I am confused as to why they couldn't calculate these scores - they appear to be relying on a PubChem server?). They compare these Tanimoto scores to the similarity score of the NMR spectra generated by their NN model. The similarity score either matches (in one case) or differs (in the other case) from the Tanimoto score, and the authors make the argument that this performance is correct. 
I am confused that there is no requirement or criteria for the NN to detect the presence of a new compound family, which seems to be what this tasks calls for. This evaluation is rather limited, given that TC scores could not be computed for half of the probe families. There is also no comparison to the baseline models for this task. 
Overall I think that this manuscript does a good job of identifying an interesting question, and makes a start at answering the question. To improve the manuscript I would ask that the authors carry out a more comprehensive evaluation of the performance using splits in which whole families are held out from the training set. Different splits might be evaluated, and in addition the model should be required to 'call' when it believes that a new compound family is present. Furthermore, it would be interesting to compare the siamese CNN approach to other network architectures. In addition, the performance of baseline models on the hold-family-out split should be assessed.