Target-Free Ligand Scoring via One-Shot Learning
Authors: Peter Eckmann
Jake Anderson
Michael K Gilson
Rose Yu
Venue: Submitted to ICLR 2023 (2023)
Keywords: drug discovery
ligand scoring
one-shot learning

Abstract: Scoring ligands in a library based on their structural similarity to a known hit compound is widely used in drug discovery following high-throughput screening. However, such "similarity search" relies on the assumption that structurally similar compounds have similar activities, and will therefore only retrieve ligands with hit-like affinity, requiring resource-intensive tweaking by medicinal chemists to reach a more active lead compound. We propose a novel approach, One-Shot Ligand Scoring (OSLS), that is much more capable of directly retrieving lead-like compounds from a library using a novel one-shot learning technique. For this new task, we design a Siamese-inspired neural architecture using two Transformer encoders without tied weights, a novel positional encoding-like mechanism, and a final prediction head. OSLS is able to score ligands by activity against a target without any target-specific knowledge beyond a single known activity value, a cost-effective approach to ligand-based or phenotypic drug discovery. We show that OSLS surpasses traditional similarity search as well as modern deep learning baselines on a simulated ligand retrieval task. Furthermore, we demonstrate the applicability of our approach on various drug discovery tasks that also involve ligand scoring, including drug repositioning, precision patient-level drug efficacy prediction, and even molecular generative modeling.

Under review as a conference paper at ICLR 2023
TARGET-FREE LIGAND SCORING VIA ONE-SHOT LEARNING
An Anonymous authors Paper under double-blind review
ABSTRACT
Scoring ligands in library based on their structural similarity to known hit compound is widely used drug discovery following high-throughput screen- ing. However, such "similarity search" relies the assumption that structurally similar compounds have similar activities, and will therefore only retrieve ligands with hit-like affinity, requiring resource-intensive tweaking by medicinal chemists to reach a more active lead compound. We propose a novel approach, One-Shu Ligand Scoring (OSLS), that is much more capable of directly retrieving lead-like compounds from a library using novel one-shot learning technique. For this new task, we design
iaspeee neural architecture using two Transformer en- coders without tied weights, a novel positional encoding- like mechanism, and a final prediction head. OSLS able to score ligands by activity against target without any target-specific knowledge beyond a single known activity value, a cost-effective approach to ligand-based or phenotypic drug discovery. We show that OSLS surpasses traditional similarity search well modern deep learn- ing baselines on a simulated ligand retrieval task. Furthermore, we demonstrate the applicability of our approach various drug discovery tasks that also involve ligand scoring, including drug repositioning, precision patient-level drug efficacy prediction, and even molecular generative modeling.
INTRODUCTION
Contemporary drug discovery a costly and cine-consuming process requiring billions of dollars per new approved drug. A significant portion of the total development cost is incurred in preclini- cal stages, where medicinal chemists identify one or more "hit" compounds from high-throughput screens that have activity against the target of interest and retrieve structural analogs to these hits from large molecular catalogs for further exploration and development, and eventually produce a lead compound after much optimization (de Souza Neto et al.) 2020; Hughes et al. 2011). Cur- rently, the concept of chemical similarity is critical to the retrieval of these structural analogs through pairwise similarity scoring between the hit compound and each compound in library. Commonly used similarity metrics include the Tanimoto similarity computed between binary fingerprints two molecules (Bajusz et 2015), as well other more specialized metrics and molecular featurizations
(Cereto-Massagué et al. 2015
Nikolova & Jaworskal 2003 Maziarka et al. 2020 Jaeger et al.,
2018 Coupry & Pogány
2022 Gandini et al. 2022)
Retrieving structural analogs from chemical library with such similarity scoring tends to yield com- pounds whose activity similar that of the initial hit, based on the concept that chemically similar compounds have similar activity (Johnson & Maggioral 1990). However, hit compounds from ex- perimental screens typically have low activities, e.g. binding affinities of 10 J,,, compared to the typical 1 10 nM goal of preclinical drug development, thousand-fold difference (Freire) 2015 Hughes et al. 2011). Thus, retrieving library compounds based purely on their similarity to an early-stage hit compound is not optimal strategy, this is expected to yield compounds with hit-like activity instead of the desired highly active compounds. Compound scoring via similarity is common for other related problems in drug discovery, such drug repositioning (Jarada et al.) 2020) and lead optimization Hughes et al. (201T), which also suffer from the problem of measuring similarity to weakly active compound.

---

Under review as a conference paper at ICLR 2023
Here, propose One-Shot Ligand Scoring (OSLS), alternative chemical similarity that pre- dicts the activity of an experimentally uncharacterized query compound (e.g. a compound drawn from chemical library) to unseen target based on a single context compound and its experimen- tally known actívity to that target (e.g. a hit from an experimental screen). Like standard chemical similarity-baser scoring, OSLS shares the advantage of needing no information about target pro- tein (Zheng et al. 2013 Vijayan et al. (021)). However, OSLS is distinct from standard measures of chemical similarity, because, by using one-shot learning paradigm, can assign the highest scores to the most active compounds, instead of those most similar to a weakly active hit.
More particularly, we
introduce the novel formulation of ligand scoring as a one-shot regression problem, and argue for its utility over traditional similarity-based scoring
design a novel architecture, OSLS, which addresses this problem by using a Siamese- inspired neural architecture extract target information from the known activity of a con- text compound and use to directly predict the activity of a query compound
show that OSLS outperforms both similarity-based as well modern deep learning scoring techniques in settings relevant to compound retrieval and the related tasks of drug reposi- tioning, patient-level drug efficacy prediction, and generative modeling.
2 RELATED WORK
In this work, we focus on cases where information about a targeted protein (e.g. amino acid sequence or 3D structure)
used for compound scoring so-called "ligand-based" "phenotypic" drug discovery (Sharma et al.. 2021; Swinney & Anthony 2011 Zheng et al. 2013). In this setting, drug discovery begins from one or few existing compounds with some known activity, often obtained from screening or known natural ligands. In this case, scoring of additional compounds is currently done using either chemical similarity or N-shot learning approaches.
Chemical similarity. Chemical similarity is commonly used when compounds with a desired ac- tivity are known, and involves computing the pairwise similarity between the known actives and each compound to be scored. When using highly active known compound, similarity acts as surrogate measure of activity (Johnson & Maggiora 1990), although this approximation fails as the known compound becomes less active. Computing similarity is commonly done with binary fingerprints (e.g. circular fingerprints, Rogers & Hahn (2010)), although this approach will often undesirably miss compounds with similar activity but different chemical scaffolds. For this reason, many other chemical features have been suggested for representing molecules, including simple molecular weight as well more complex representations that capture information about molecular topology and 3D shape/charge (Khan et al., 2016 Li et al.) 2012 Kohlbacher et al. 2021 Kearnes & Pande 2016).
However, such approaches are, arguably, based more on chemical intuition than data, and choosing which of hundreds of molecular descriptors to adds another level of uncertainty. For this reason, machine learning-baid based techniques have been proposed that derive chemical similarity in a data- driven fashion and thus offer promise improve quality of similarity measurement. In particular, much work has been dedicated to learning molecular featurization in an unsupervised fashion, which can later be used for downstream tasks such as similarity (Jaeger al. 2018 Huang et al.) 2021 Li & Jiang 2021 Morris et al.) 2020). Due to their unsupervised nature, however, similarity measure- ments between machine learning-derived embeddings are not necessarily meaningful for activity, as structurally dissimilar molecules may have similar activities, and vire-versa. Because of this, more
direct N-shot learning approaches (Schimunek et al. 2021 Altae- Tran et al.) [017]; Stanley et al. 202. Lee et al., 2022) have been proposed to leverage vast amounts of cxisting activity data toward the scoring of new ligands against novel targets.
N-shot learning. N-shot learning techniques directly use existing compounds (the "context", also called the "support set") to predict the activity of unknown compounds (the "query set") without relying similarity as an imperfect surrogate of activity. The application of Siamese networks to one-shot learning, a form of N-shot learning involving a single context example, was first intro- duced in computer vision (Koch et al. 2015), and existing Siamese- based techniques, such those
2

---

Under review as a conference paper at ICLR 2023
that act on sentences (Mueller & Thyagarajan 2016), may be applied to string- based molecular representations. Schimunek et al. (2021) extend this technique to multiple context compounds for few-shot learning, also a form of N-shot learning, using Siamese network to encode both the query and context molecules followed by a similarity function to perform classification. |Alaae-Traa et al. (2177) introduce are then used to make binary activity predictions. [i et al. (2019) classify two graphs, which may be molecular graphs, as similar or dissimilar through cross-graph attention mechanism. Finally, Stanley et al. (2021) introduce a dataset for the few-shot classification task in drug discovery, and run multiple baselines on the proposed set.
-eTMe- architecture iteratively update compound embeddings, which
All these techniques, however, cannot predict how active a compound is. instead making only binary predictions of activity at whatever threshold of true activity is chosen. This is especially relevant when known actives are only of mild potency so the "active" threshold is low, e.g. hits following a screen. In this case, binary methods cannot distinguish between mildly and highly active com- pounds. Lee et al. (2022) propose to predict continuous binding affinity in few-shot fashion using attentive neural processes (ANPs), but their method requires multiple support examples and is not applicable to one-shot setting. Additionally, the support set is randomly drawn over the en- tire activity range of known compounds, including highly active compounds, casting doubt on the model's utility in the case where one knows only a mildly active compound.
3 METHODOLOGY
We formulate the problem of compound scoring given a known active one-shot regression prob- lem. We introduce OSLS, an architecture using two Siamese-like encoding networks with untied weights. a positional encoding- like mechanism represent activity values, and a feedforward pre- diction head. The architecture is summarized in Figure
Problem statement. We seek to score compounds via one-shot learning, meaning predicting, i.e. scoring, the activity "query" compound against an unseen target given only a single "context" compound and its activity against that target. Let represent a molecule and Tr(m) E IR its ex- perimentally determined activity against a target P (e.g. binding affinity to a protein, or phenotypic inhibitory concentration). Formally, we aim to predict the activity of a query molecule m, to P. TP (ma). given a context molecule me and its activity against P, (m.)). That is, we wish to learn a function f (m, Me: (m())) E E to approximate the true activity of the query molecule, TP(m)))
Architecture. We employ two separate encoders, context and query encoder, that feed into an activity prediction head. The context encoder, fe (Me, (m.)) + Ie, learns to encode information about the target into real-valued vector Ie using the context molecule and its known activity against the target. The query encoder, fg my + Zgg. learns to encode the query molecule into a representation zy, that is useful for predicting its activity. As they are not designed to encode the same information, /e and f, do not share weights, unlike previously proposed Siamese networks (Muellrr& & Thyagarajan 2016 Morris et al. 2020). The predictor network, 9 T. (B H (m,)), where + denotes vector concatenation, uses these two encodings to predict the activity of the query molecule, (m,,) IR.
We represent molecules as sequences of atom-level tokens of canonical SMILES strings (see Ta- ble B for comparison across molecular representations). Both fe and fg are Transformer encoders Vaswani et al. 2017). which have shown recent success way to represent molecules Wang al.) 2019) Li & Jiang (202)). )) Following Devlin et al. (2018), we also prepend a "classification token" at the first position of each sequence, and take the hidden state of the Transformer at that position as G. and 2y after projection through a linear layer.
To encode scalar activity information with the Transformer, we propose novel, learned linear projection of the scalar that is summed with token embeddings before being fed to the multi-head attention of the Transformer encoder. This projection has the same dimensionality as the embed- dings, and is identical across each element of the input sequence. This simple approach to including scalar activity information is similar to positional encoding, which can effectively pass contextual information to the Transformer /Vaswani et al. 2017).
3

---

Under review as a conference paper at ICLR 2023
Training. The training dataset, D, consists of compounds and their affini- ties to multiple protein targets. P. Let Mr represent all compound with activ- ity data against given target P P. then we define the training set as
Affinity prediction molecule
Predictor (Feedlorward)
Concatenate
D U ((mTTp(m)) m E Mp] PEP
Transformer
Transformer Encocer
Encoder
where V represents repeated union op- erations. The testing set is constructed in a similar way to the training set, but with a separate set of targets, a so-called "target split" (Feng et al.) 2018).
Known affinity
Linear
inooding
Fostional Encoding
SMILES antoddding
experimentai recut
embedding
New moleaule, uninown atfinity
We train the model in an end-to- end fashion with Mean Squared Er- ror (MSE) loss between the pre- dicted and true activity of mg. i.c. MSE(TT(m,) @p(m))). Each train- ing batch consists of set of query molecules randomly selected without re- placement from the entire training set. For each query molecule my, that is an element of some Mr, context molecule and its activity is randomly selected from all molecules with data against the same target, (Me; {m(m)))
Figure I: Architecture overview. The context encoder (left) receives the embedded SMILES tokens of the con- text molecule summed with a sinusoidal positional en- coding and a linear projection of its known activity value. The query encoder (right), which has different weights, receives the embedded SMILES tokens of the query molecule summed with a positional encoding. forward predictor network receives the concatenated out- puts of each encoder and produces a final scalar activity prediction of the query molecule.
feed-
where m E Mp. When selecting the context compound, an additional constraint placed on T (mmm.) such that S pm,)) < b, where and b are the lower and upper bound, respectively, of the manually set context activity range. By constraining pm,) but not pm))) OSLS can learn to score compounds with a wide range of activities using only weakly active compounds as context.
4 EXPERIMENTS
We test OSLS and baseline techniques on four applicable drug discovery tasks: (1) ranking and classification of compounds from BindingDB (Liu et al.] (007)] with varying context compound affinities that are typical of screening results. (2) predicting the activities of anti-cancer drugs against a wide array of tumor cell lines (Barretina et al., 2012) using the activity of a single drug against each cell line as context for activity prediction of other drugs. (3) simulating a drug repositioning task using the natural ligand estradiol and a set of app-aape drugs (Selleck), and testing for the retrieval of diverse compound scaffolds that are already known to bind the target of estradiol. (4) use of CSLS as an objective function to guide molecular generative modeling using a single known ligand. Finally, we measure the impact of various ablations on the OSLS architecture.
Additional results from the adaptation of OSLS to the few-shot setting, following the experimental
setup of Lee et al. 2022). are reported in Appendix A Dataset and preprocessing details are reported in Appendix B and details on the implementation of OSLS are reported in Appendix
4.1 BASELINES
We test OSLS against the following baselines, whenever applicable given task. Details on the training and implementation of baselines are given in Appendix D
OSLS-MLP. Similar architecture to OSLS except using Morgan fingerprints to represent molecules connected to a multilayer perceptron (MLP) instead of a Transformer. Loosely related to Koch et al. (2015), except we untie the two encoders and predict a continuous target. The fingerprint of the context compound is multiplied with its known activity value to encode context, process inspired by POT-DMC (Godden et al. 2004).
4

---

Under review as a conference paper at ICLR 2023
Tanimoto similarity. Molecular structure- based similarity measure based on vector similarity between two binary fingerprints. In this paper, we use Morgan fingerprints (Rogers & Hahn 2010) with radius of 2. This measure of molecular similarity is commonly used in practice.
MaLSTM. Siamese neural network to embed molecular SMILES sequences using an LSTM
network followed by a vector similarity measure (Mueller & Thyagarajan 2016).
Graph Matching Network. Classifies if two graphs are similar by jointly reasoning using a cross-graph attention mechanism (Li et al., 2019).
MetaDTA. Applies attentive neural processes to the few-shot regression of continuous binding affinity values (Lee al. 2022).
OpenEye Rapid Overlay of Chemical Structures (ROCS), Computes overlap between 3D molecular representations, using both pure 3D shape "'ShapeTanimote and 3D shape com- bined with charge information ("TanimotoCombo") (Kearnes & Pande 2016).
4.2
COMPOUND RANKING AND CLASSIFICATION
(a)
(b)
0.65
Kendall's tau
0.2
DSLS
ROC-AUC
0.60
DSLS
0.1
similartty
MaLSTM
0.55
Rendom ranking
0S1S
Random classification
0.50
OSLS-MLP
Tanimoto similanty
Typicel hit affinity
Graph Matching Network
-0.1
-0--
10
10
Context affinity range (nM)
Context affinity range (nM)
Figure 2: Comparison of compound ranking and classification techniques varying context compound affinities. (a) mean Kendall's tau correlation across 40 test targets between real and predicted affinity among all compounds with data available against each target. Context compounds are taken at random from the set of compounds with activity data against each target with an affinity in the range specified on the x-axis. Marked is the most common range of hit affinities following HTS campaign (Freire, 2015; Hughes et al.) 2011). (b) mean ROC-AUC of classification ability across 40 test targets with the context compound affinity range on the x-axis, with true positives being defined as compounds with < 10 nM affinity against their target, and true negatives otherwise.
We compare OSLS with baselines scoring compounds from library, while varying the affinity of a context compound that is analogous to a hit from a high-throughput screen. We use a set of 40 unseen test targets from BindingDB, and aim data against each target. For each query compound, a context compound is taken at random from all compounds against the same target within the defined affinity range (specified on the x-axis). For all methods, we train a separate model for each context affinity range. After scoring, we measure performance in ranking and classification, both common tasks drug discovery.
all compounds that have experimental activity
Figure 2 left reports the mean rank correlation across all targets between true compound bind- ing affinity and the score produced by each method. For the two similarity-based methods, Tan- imoto similarity and MaL, scoring was done to the context compound, i.e. the highest scored molecules are those most similar the context compound. As expected, the ability of these tech- niques to rank compounds by their true affinity value declines rapidly the context compound becomes less potent, reaching random ranking around the micromolar range (103-104 nM), which is the typical top affinity obtained from nightthroughput screening campaign (Freire, 2015 Hughes) et al. 2011). contrast, the performance of OSLS and OSLS- M-LLS-MMPP is relatively strong, and strongest for OSLS, throughout all context compound affinity ranges.
We also use the scores produced by each method in a classification task, where query molecules are considered positives if the affinity to their respective target is < 10 nM, and negatives otherwise,
5

---

Under review as a conference paper at ICLR 2023
reflect the typical desired affinity of lead optimization projects (Hughes al. 2011). Figure righht shows the mean binary classification ability for each method across all test targets, as measured by ROC-AUC. In addition to the previous baselines, also include Graph Matching Network (GMN), which is only applicable to the binary classification task. We trained and tested two different GMN models, one with a positive and the other negative context compound, so the reported performance is identical across all affinity ranges that were included in each model's 's training set, as we used the same range for testing (< 10 and 2 10 nM for the positive and negative context model, respectively). As shown, the performance of most methods drop as the context compound becomes less potent, although CSLS mostly retains performance even as the context compound is no longer drawn from the active range. Additionally, OSLS shows the highest ROC-AUC compared to baselines across most context affinity ranges, especially the low nM ranges.
Besides OSLS and OSLS-MLP, no other baseline can take advantage of continuous activity values, seemingly limiting their performance. Specifically, OSLS outperforms baseline methods across most context affinity ranges, especially at the most common and applicable low micromolar range.
4.3 SCORING ANTI-CANCER COMPOUNDS
Chemotherapy involves the targeting of specific tumors with specialized drugs. Tu- mors are often profiled by genetic sequenc- ing and the subsequent identification of cancer-causing mutations (Malone et al.) 2020). but it is often not clear which drug will be active against given muta- tion (Kalamara et al. 2018) [simberidou 2015). One alternative approach to tumor profiling, that is already undertaken clin- ically, involves screening a small set of drugs against a patient-derived tumor cul- ture, yielding continuous activity values for each drug (Letai 2017 Wong et al. 2021).
PD-0332091
TKI258
Sorafenib
Drug used as context
TAE684
AEW541
1-685458
Panobinostat
LBW242
OSLS
Topokecan
O.L.S.MIP
RAF265
Lapatinio
Paclitaxel
Nutlin33
ID-0525901 A2D05 30
Nilotinib 17-VAG
-0.10 -0.05
0.00
0.05
0.10
0.13 0.20
Mean IC50 correlation (real, predicted)
Most cancers involve very specific muta- tions, but high-throughput screening (HTS) against any one mutation often not eco- nomically or biologically feasible (Letai) 2017). One-shot learning techniques, how- ever, can potentially extrapolate small sets of clinically obtained drug data to much larger HTS-scale libraries, expanding both clinical therapeutic options and making vi- able the evaluation of many preclinical drug candidates against highly specific tumors.
Figure 3: Scoring drugs from the Cancer Cell Line Encyclopedia. The x-axis shows the mean Pearson's 's r correlation coefficient between real and predicted drug activity, as measured by normalized IC50 values, across a wide array of cancer cell lines. The y-axis shows the single context compound used for predic- tion, where its activity against each cell line informs the scoring of all other compounds against that line. Compounds are ordered by the correlation coefficient from OSLS, show which compounds most in- formative when supplied as context.
Towards this end, we evaluated the perfor- mance of OSLS and OSLS-MLP in scoring
anti-tumor drugs against tumor cell lines from the Cancer Cell Line Encyclopedia (CCLE. Barretina et al. 22012)). Figure reeporss the mean correlation across all cell lines (x-axis) between the real and predicted IC50 values of drugs with activity data against each line. We repeated the scoring using different context drugs (y-axis), which were supplied to both methods along with their respective activity values against each cell line. For each context drug, the reported correlation coefficient computed using activity values normalized to have mean of O (for both real and predicted values) to account for bascline differences in drug activity and capture only tumor- speoifcc activity differences.
As shown, OSLS displays some level of predictive power across all but one context drug. OSLS- MLP, however, has less or no predictive power for most context drugs. Models were trained on BindingDB data only, and not the CCLE, suggesting that the observed predictive power of OSLS would transfer to other compounds beyond those tested here, including other clinical drug options or even preclinical candidates. Additionally, models were trained on binding affinity values, which,

---

Under review as a conference paper at ICLR 2023
while presumably related to the present IC50 values, indicates the potential for transfer learning across different activity types mediated by an underlying binding mechanism.
We only compared with one baseline on this task many of our previously used baselines not applicable. For instance, similarity- methods will yield identical similarity values between the tested drugs regardless of the experimentally determined activity value, meaning they will have no predictive power. Binary classification methods are also not applicable, because defining any binary cutoff among the relatively smoothly distributed activity values not necessarily meaningful, and binary outputs would also not produce meaningful correlations to the continuous ground truth. Therefore, only one-shot activity prediction methods were applicable to this task, and OSLS shows far superior performance to OSLS-MLA which makes close to random predictions. Our results suggest that OSLS may able to complement or replace existing tumor profiling techniques, and significantly reduce the costs associated with scoring a large compound library against specific tumor profiles, offering both clinical and preclinical utility.
4.4 SCORING FOR DRUG REPOSITIONING
Drug repositioning the application of existing drugs to new indications, promising to avoid expen- sive drug development and approval while still offering needed new treatments (Pushpakom et al.) 2019). Many computational techniques have been proposed for predicting the activity of existing drugs against novel targets, including chemical similarity-based techniques (Jarada et al.) 2020). We simulated a drug repositioning task to the estrogen receptor (using models trained on BindingDB, where the estrogen receptor excluded from the training set) by scoring all FDA-approved drugs by predicted activity using estradiol as the context compound, a natural ligand to the estrogen recep- tor. We chose the estrogen receptor, which has many existing drugs, to compare the ability of each technique to rank these drugs, which we knew beforehand, highly in comparison to all others.
Table 1: Drug repositioning to the estrogen receptor. Each technique was used to rank all 2,617 FDA-approved drugs using estradiol (a natural ligand to the estrogen receptor) the context com- pound. The corresponding ranking (out of 2,617) of each drug which is known to bind the estrogen receptor (taken from Wishart et al. (206)) is shown (lower is better).
Tanimoto
ROCS
ROCS
Drug
OSLS
similarity
(TanimotoComboo (ShapeCombo)
MaLSTM OSLS-MIL
Estrone
113
6
4
3
5
223
Raloxifene
10
531
519
1281
1898
35
Fulvestrant
25
12
18
1191
95
54
Tamoxifen
214
1434
1527
1562
2493
198
Bazedoxifene
1023
353
684
1380
1162
239
Toremifene
887
1562
1527
1568
2576
562
Ospemifene 1077
2283
941
1469
2277
1498
Table reports the rankings produced by each technique of all estrogen-binding drugs (taken from Wishart et al. (2006)) among 2,617 total FDA-approved drugs (obtained from Selleck). The steroidal ranked highly by all methods, as they are structurally similar to
drugs estrone and fulvestrant estradiol, which is also a steroid. For the other non-steroidal drugs, which are structurally very differ- ent from estradiol, most similarity-basee methods fail, while the novel one-shot scoring techniques (OSLS and DSL.))L.P) rank these drugs relatively highly. These results suggest that one-shot learn- ing methods are more capable of "scaffold hopping," which the ability to identify compounds that are structurally distinct from known compound but have similar activity.
4.5 GUIDING MOLECULAR GENERATION
Many generative models have been proposed to design novel compounds against given target (e.g. Jin et al. (2018;; Luo et al.] (2021) I; Jin et al. (2020); Xie et al. (202T); Zhou et al. (2019); Zang & Wang 2000); Eckmann et al. (2022)). All generative models employ objective function to score suggested ligands, which, when attempting to generate binders against a target, is typically docking software (Spiegel & Durrant 2020 Eckmann et al. (022) that requires target information. When the target is unknown, in the case of phenotypic or ligand- based drug discovery, some or all of these existing methods are not applicable. To the best of our knowledge, in the common case of
7

---

Under review as a conference paper at ICLR 2023
(b)
Predicted binding affinity (nM)
10c
Estradiol (known
USLS
Tonimoto simlentty
10a
Number molecules
60-
ligand)
10*
50-
20*
40
30-
207
Estradiol (known ligand)
10a
600
10a
101
10"
10 10a 10"
10"h 10'
10a
Epoch
Docked binding affinity (nM)
Figure 4: Molecular generation using OSLS as an objective function. (a) mean (solid line), mini- mum, and maximum predicted binding affinity of 256 variational autoencoder-gent generdrre molecules optimized with OSLS an objective function (lower is better). The known affinity of estradiol, the natural ligand being used as a context compound, also shown. The x-axis shows the number of gradient descent steps in the optimization process. (b) distribution of AutoDock-GPU scores among all valid generated compounds (lower is better), with the docked score of estradiol also shown.
de novo drug design based on one or a small number of hits from phenotypic screening (Swinney & Anthony 2011), existing generative models cannot be applied, and is what we seek to address.
After training on BindingDB, we applied OSLS as an objective function in a variational autoen- coder (VAE, Kingma & Welling 2013))- -assed generative model, using single known compound (estradiol, a natural ligand) and its measured affinity to the estrogen receptor (a well-studied and discase-relevant target, which was excluded from the training set) context, and seek to generate new molecules with higher affinity to the target than estradiol. We slightly adapt OSLS to allow end-to-end differentiation and combine with trained VAE, similarly to previously proposed gen- erative models (Luo et al. 2021| Jin et al. 2018; Zang & Wang 2020; Eckmann et al. 2022), and then backpropagate the output attinity prediction the latent space to generate molecules with higher predicted affinity (for more details about the training and method, see Appendix E).
Figure left shows the mean, minimum, and maximum affinity prediction from OSLS among 256 molecules over 1,000 epochs of backpropagation. As shown, OSLS successfully guides generation towards compounds with higher predicted affinity, with some even surpassing that of estradiol. To further validate the generated compounds, Figure 4 right shows the distribution of generated com- pound affinities predicted by AutoDock-GiP
the estrogen receptor (PDB ID IERR, details in Appendix E (Santos-Martins al.) (021), a docking program that uses full target information. En- couragingly, many generated compounds surpass the bocking-predicee
affinity estradiol. Note that docking, including AutoDock-GPP is generally a poor predictor of actual affinity (Pantsar & Poso 2018). so is not necessarily more accurate than the predictions of OSLS itself. We included the Tanimoto similarity baseline, which consists of simple differentiable surrogate model of Tanimoto similarity to estradiol, to approximate the typical similarity saach--guded molecular generation of medicinal chemists (see Appendix E for further details), As no other baseline was readily differen- tiable, we did not include any beyond Tanimoto similarity.
Table 2 reports further met- rics of generated molecules, and compares with other generative models that employ target informa-
Table 2: Comparison of generated molecules. Quality metrics (see Appendix Eb of generated compounds with higher docked affin- ity than estradiol. We report the best docked affinity as Ka in nanomoles/liter, the number of Butina clusters, and the diversity and novelty scores as described in Jin al. (2020).
tion (Xie et al. 2021:
Method
Best (nM) ({) clusters (f) Diversity (f) Novelty (1)
Olivecrona et al. 2017
Eckmann et al. (2022). Out of 256 molecules generated by each method, only molecules with a docked affinity better than that of
REINVENT
0.98
57
0.866
0.98
MARS
17.3
4
0.893
1.00
LIMO
0.31
57
0.874
1.00
Tanimoto
4.5
23
0,862
0.81
OSLS
2.4
42
0.862
1.00
8

---

Under review as a conference paper at ICLR 2023
estradiol were included in metric calculations (full details on the metrics used available in Ap- pendix E While molecules generated by OSLS are surpassed in most metrics by generative models that harness target information, they still of relatively high docked affinity. is encouraging that such compounds were generated using only single context compound, suggesting that OSTS may be used successfully for lead optimization in phenotypic drug discovery. In contrast the Tanimoto similarity baseline, OSLS generates high structural diversity of molecules with desirable affinity, which is of high utility for drug discovery.
4.6 ABLATION STUDY
We report performance metries of model variations in Table 3 We measure the average Kendall's tau and ROC-AUC (for the binary identification of < 10 nM compounds) across 40 un- seen targets in BindingDB, using 1 nM context compounds.
Table 3: Ablations. Blank entries represent no variation from the base model (top).
Molecule representation SMILES SELFIES SPE
Context provided
Projection
Kendall's tau
type
ROC-AUC
Continuous
Linear
0.3198
0.7128
0,2901
0.6820
0,2255
0,6524
Binary
0,2107
0.6881
We tested SMILES, SELFIES (Krenn aL. 2020b). and SMILES Pair Encoding (SPE; Li & Fourches (021)) which are string-based molccular representations.
No context
0.3024
0.6842
Non-linear
0.3177
0.7087
"Continuous" means that continuous activity value was provided context, "Binary" that the activity was thresholded at 10nM to either or O, and "No context" that no context was provided at all. "Linear" and "Non-lineat" (eubic) projections were also tested to encode scalar activity values.
5 DISCUSSION AND CONCLUSIONS
We present OSLS, one-shot learning model for ligand scoring utilizing a Siamese-inspired neural architecture with two Transformer encoders without tied weights, novel positional encoding mechanism, and a final prediction head. After training on BindingDB, a large dataset of drug-target target affinity data, we show that OSLS surpasses both traditional and recent baselines on variety of relevant tasks involving one-shot ligand scoring on unseen targets. Promisingly, OSLS is able to correctly score compounds that are of higher activity that the provided context compound, which is shown to be valuable compound retrieval, drug repositioning, and generative tasks.
-ike like
In contrast to previous work, we utilize continuous, rather than binary, activity values, which seems to be beneficial even for binary classification tasks (Figure 2 Table 33. Continuous values are also useful for ligand scoring following screening, where hits are active but of mild potency, in which case binary methods cannot distinguish between mildly and highly active compounds. Finally, con- tinuous values highly applicable some clinical and preclinical challenges, such those in cancer therapy, where how active compound is can be highly relevant (Section 4.3)
Our work also differs from previous work in that focus on one-shot, rather than few-shot, pre- diction. The number of hit compounds obtained from a high-throughput screen (HTS) increases al., 2014), being able to terminate screening after only
with time and resources Aatene-Ramos one identified hit (which would be used as the context compound for OSLS) would greatly reduce costs. HTS is also usually followed by lower- tever-throughput confirmatory assays. which in practice can result in as a few as one active compound with a measured affinity (Zhu et al. 2013), meaning our approach is of much broader applicability.
OSLS may also be applied to phenotypic drug discovery, where the target is unknown and hits are determined by observing a cellular response. Even if there are multiple hits resulting from such a screen, many will not mediate a response through the desired cellular mechanism of action, and so scoring additional compounds based on these hits is not useful (Vincent et al. 2020). As typically only one very small number of hits induce response through the desired mechanism of action, scoring of further active compounds using only a single hit (presumably, the with the desired mechanism of action) gives maximum flexibility for and applicability to phenotypic drug discovery.

---

Under review as a conference paper at ICLR 2023
6 REPRODUCIBILITY STATEMENT
Anonymized source code for the hase OSLS model is provided in the supplementary materials, well helper functions load pretrained models and perform inference using raw SMILES strings. Scripts are also provided to preprocess data from BindingDB and prepare it for training. Descriptions of preprocessing steps undertaken for each dataset are provided in Appendix B Im- plementation details for OSLS are provided in Appendix [ and descriptions of how each baseline was implemented are provided in Appendix [ Finally, implementation and baseline details for the generative modeling experiments are provided in Appendix
REFERENCES
Han Altae-Traa Tran, Bharath Ramsundar, Aneesh S Pappu, and Vijay Pande. Low data drug discovery with one-shot learning. ACS central science, 3(4):283-293, 2017.
MA Attene-Ramos, CP Austin, and Xia. Encyclopedia of toxicology, 2014.
Dávid Bajusz, Anita Rácz, and Károly Héberger. Why tanimoto index an appropriate choice for fingerprint-based similarity calculations? Journal of cheminformatics, 7(1):1-13, 2015.
Jordi Barretina, Giordano Caponigro, Nicolas Stransky, Kavitha Venkatesan, Adam A Margolin, Sungjoon Kim, Christopher J Wilson, Joseph Lehár, Gregory V Kryukov, Dmitriy Sonkin, et al. The cancer cell line encyclopedia enables predictive modelling of anticancer drug sensitivity. Nature, 383333911 2012.
Darko Butina. Unsupervised data base clustering based on daylight's fingerprint and tanimoto sim- ilarity: A fast and automated way to cluster small and large data sets. Journal of Chemical Information and Computer Sciences, 39(4):747-750, 1999.
Adrià Cereto-Massagué, María José Ojeda. Cristina Valls, Miquel Mulero, Santiago Garcia-Vall Vallvé, and Gerard Pujadas. Molecular fingerprint similarity search virtual screening. Methods, 71: 58 63, 2015.
Damien E Coupry and Peter Pogány. Application of deep metric learning to molecular graph simi- larity. Journal of Cheminformatiic, 14(1):1-12, 2022.
Lauro Ribeiro de Souza Neto, José Teófilo Moreira-Filho, Bruno Junior Neves, Rocío Lucía Beat- riz Riveros Maidana, Ana Carolina Ramos Guimarães, Nicholas Furnham, Carolina Horta An- drade, and Floriano Paes Silva Jr. silico strategies to support fragment-to-lean optimization in drug discovery. Frontiers in chemistry, 8:93, 2020.
Jacob Devlin, Ming- Ming Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv: /xX.v0880000480, 2018.
Peter Eckmann, Kunyang Sun, Bo Zhao, Mudong Feng, Michael K Gilson, and Rose Yu. Limo: Latent inceptionism for targeted molecule generation. arXiv preprint arXiv:2206.0900 2022.
Qingyuan Feng, Evgenia Dueva, Artem Cherkasov, and Martin Ester. Padme: A deep learning-bass framework for drug-targe interaction prediction. arXiv preprint arXiv:1807.09741, 2018
Ernesto Freire. The binding thermodynamics of drug candidates. Thermodynamics and kinetics of drug binding, pp. 1-13, 2015.
Enrico Gandini, Gilles Marcou, Fanny Bonachera, Alexandre Varnek, Stefano Pieraccini, and Mau- rizio Sironi. Molecular similarity perception based on machine-learning models. International Journal of Molecular Sciences, 23(11):6114, 2022.
Jeffrey W Godden, Florence L Stahura, and Jürgen Bajorath. Pot-dme: a virtual screening method for the identification of potent hits. Journal of medicinal chemistry, 47(23):5608-5611 2004.
Paul C. D. Hawkins, A. Geoffrey Skillman, and Anthony Nicholls. Comparison of shape-matching and docking as virtual screening tools. Journal of Medicinal Chemistry, 50(1):74 82, December 2006.
10

---

Under review as a conference paper at ICLR 2023
Paul C Hawkins, A Geoffrey Skillman, Gregory L Warren, Benjamin A Ellingson, and Matthew T Stahl. Conformer generation with OMEGA: algorithm and validation using high quality structures from the protein databank and cambridge structural database. J. Chem. Inf. Model., 50(4):572- 584, April 2010.
Kexin Huang, Cao Xiao, Lucas M Glass, and Jimeng Sun. Moltrans: Molecular interaction trans- former for drug-target interaction prediction. Bioinformatics, 37(6))830-88) 806,-36,6400 2021.
James P Hughes, Stephen Rees, S Barrett Kalindjian, and Karen L. Philpott. Principles of early drug discovery. British journal of pharmacology, 162(6): (236)):139912222 1249, 2011.
Sabrina Jaeger, Simone Fulle, and Samo Turk. Mol2vec: unsupervised machine learning approach with chemical intuition. Journal of chemical information and modeling, 58(1):27-35, 2018 2018.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameteerization with gumbel-softmax. arXiv preprint ar 1611 144, 2016.
Tamer N Jarada, Jon G Rokne, and Reda Alhajj A review of computational drug repositioning: strategies, approaches, opportunities, challenges, and directions. Journal of cheminformatics, 12 (1):1-23, 2020.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. In International conference machine learning, pp. 2323-2332. PMLR, 2018.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Multi-objective molecule generation using interpretable substructures. In International conference on machine learning, pp. 4849-4859 PMLR, 2020.
Mark A Johnson and Gerald M Maggiora. Concepts and applications molecular similarity. Wiley, 1990.
Angeliki Kalamara, Luis Tobalina, and Julio Saez-Rodriguez. How to find the right drug for each patient? advances and challenges pharmacogenomics. Current opinion in systems biology, 10: 53-62, 2018.
Steven Kearnes and Vijay Pande. Rocs-derived features for virtual screening. Journal of computer- aided molecular design, 30(8)600-6117, 2016. URL htttps://docs eww.seen..p....... com/
applications//ooss....oo
wwwroviwssvvvveeeeeoooo.. html
Asad U Khan et al. Descriptors and their selection methods in qsar analysis: paradigm for drug design. Drug discovery today, 2118 2016.
Diederik Kingma and Max Welling.
Auto-encoding variational bayes. arXiv preprint
arXiv:1312.61 14, 2013.
Gregory Koch, Richard Zemel, Ruslan Salakhutdinov, et al. Siamese neural networks for one-shot image recognition. In JCML deep learning workshop, volume 2, pp. O. Lille, 2015.
Stefan M Kohlbacher, Thierry Langer, and Thomas Seidel. Qphar: quantitative pharmacophore activity relationship: method and validation. Journal of cheminformaties, 13(1):1-14 2021.
Mario Krenn, Florian Häse, AushatKumar Nigam, Pascal Friederich, and Alan Aspuru-Guzik. Guzik. Self- referencing embedded strings (selfies): A 100% robust molecular string representation. Machine Learning: Science and Technology, (4):0445224, 2020a.
Mario Krenn, Florian Häse, KushatKumar Nigam, Pascal Friederich, and Alan Guzik. Self- referencing embedded strings (selfies): A 100% robust molecular string representation. Machine Learning: Science and Technology, 1(4):045024, 2020b.
Eunjoo Lee, Jiho Yoo, Huisun Lee, and Seunghoon Hong. Metadta: Meta-learning- based drug- target binding affinity prediction. In ICLR2022 Machine Learning for Drug Discovery, 2022.
Anthony Letai. Functional precision cancer medicine-moving beyond pure genomics. Nature
medicine, 1028-1028-1033 2017.
11

---

Under review as a conference paper at ICLR 2023
Juncai Li and Xiaofei Jiang. Mol-bert: An effective molecular representation with bert for molecular property prediction. Wireless Communications and Mobile Computing, 2021, 2021.
Xiaolin Li, Zunyao Wang, Hongling Liu, and Hongxia Yu. Quantitative structure-activity rela- tionship for prediction of the toxicity of phenols photobacterium phosphoreum. Bullerin of environmental contamination and toxicology, 891):27-31, 2012.
Xinhao Li and Denis Fourches. Smiles pair encoding: a data-driven substructure tokenization al- gorithm for deep learning. Journal of Chemical Information and Modeling, 61(4):1560-156) 2021.
Yujia Li, Chenjie Gu, Thomas Dullien, Oriol Vinyals, and Pushmeet Kohli. Graph matching net- works for learning the similarity of graph structured objects. In International conference machine learning, pp. 3835-3845. PMLR, 2019.
Tiqing Liu, Yuhmei Lin, Xin Wen, Robert N Jorissen, and Michael Gilson. Bindingdb: a web- accessible database of experimentally determined protein-ligand binding affinities. Nucleic acids research, 35(uupp 1):D198-D20) 2007.
Youzhi Luo, Keqiang Yan, and Shuiwang Ji. Graphdf: A discrete flow model for molecular graph generation. In International Conference on Machine Learning, pp. 7192-7203. PMLR, 2021.
Eoghan R Malone, Mare Oliva, Peter JB Sabatini, Tracy L Stockley, and Lillian L Siu. Molecular profiling for precision cancer therapies. Genome medicine, 12(1):1-19, 2020.
Lukasz Maziarka, Tomasz Danel, Slawomir Mucha, Krzysztof Rataj. Jacek Tabor, and Stanistaw Jastrzebski. Molecule attention transformer. arXiv preprint 082444 2020.
J. E. J. Mills and P. M. Dean. Three-dimensional hydrogen- bond geometry and probability infor- mation from a crystal survey. Journal of Computer-Aided Molecular Design, 10(6):607-62) December 1996.
Paul Morris, Rachel St. Clair, William Edward Hahn, and Elan Barenholtz. Predicting binding from screening assays with transformer network embeddings. Journal of Chemical Information and Modeling, 60(9):491144999 2020.
Jonas Mueller and Aditya Thyagarajan. Siamese recurrent architectures for learning sentence simi- larity. In Proceedings of the AAAI conference on artificial intelligence, volume 30, 2016.
Nina Nikolova and Joanna Jaworska. Approaches to measure chemical similarity a review. QSAR & Combinatorial Science, 22(9-10):100-110 2003.
Mareus Olivecrona, Thomas Blaschke, Ola Engkvist, and Hongming Chen. Molecular de-novo design through deep reinforcement learning. Journal of cheminformatics, 9(1):1-14, 2017.
Tatu Pantsar and Antti Poso. Binding affinity via docking: fact and fiction. Molecules, 23(8):1899, 2018.
Sudeep Pushpakom, Francesco lorio, Patrick A Eyers, K Jane Escott, Shirley Hopper, Andrew Wells, Andrew Doig, Tim Guilliams, Joanna Latimer, Christine McNamee, et al. Drug repurposing: progress, challenges and recommendations. Nature reviews Drug discovery, 18(1):41-58, 2019.
David Rogers and Mathew Hahn. Extended-commective fingerprints. Journal of chemical informa-
tion and modeling, 50(5):742-754, 2010.
Diogo Martos-Martins, Leonardo
Andreas F Tillack, Michel F Sanner, Andreas Koch, and Stefano Forli. Accelerating AutoDock4 with GPUs and gradient-based based local search. Journal of Chemical Theory and Computation, 17(2):100--073, January 2021.
Johannes Schimunck, Lukas Friedrich, Daniel Kuhn, Friedrich Rippmann, Sepp Hochreiter, and Günter Klambauer. generalized framework for embedding- based few-shot learning methods in drug discovery. ELLIS Machine Learning for Molecules workshop, 2021.
Selleck. ada-appd drug URL httts:/:/ww..eeeeecccccoeccc///s
-da-approvaad library...b...... html
12

---

Under review as a conference paper at ICLR 2023
ligand-based Vidushi Sharma, Sharad Wakode, and Hirdesh Kumar. Structure- and ligand based drug design: cepts, approaches, and challenges. Chemoinformaties and Bioinformatics in the Pharmaceutical Sciences, pp. 27-53, 2021.
Jacob O Spiegel and Jacob Durrant. Autogrow4: an open-source genetic algorithm for novo drug design and lead optimization. Journal of cheminformaties, 12(1):1-16, 2020.
Megan Stanley, John F Bronskill, Krzysztof Maziarz, Hubert Misztela, Jessica Lanini, Marwin Segler, Nadine Schneider, and Mare Brockschmidt. Fs-mol: A few-shot learning dataset of molecules. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021.
David C Swinney and Jason Anthony. How were new medicines discovered? Nature reviews Drug discovery, 10(7):007-519, 2011.
Apostolia-Maria Tsimberidou. Targeted therapy in cancer. Cancer chemotherapy and pharmacol- ogy, 76(6): (0666))1113-111 1a-11111111122 1132, 2015.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin. Attention you need. Advances in neural informa- lion processing systems, 30, 2017.
RSK Vijayan, Jan Kihlberg, Jason Cross, and Vasanthanathan Poongavanam. Enhancing preclin- ical drug discovery with artificial intelligence. Drug discovery today, 2021.
Fabien Vincent, Paula M Loria, Andrea D Weston, Claire M Steppan, Regis Doyonnas, Yue-Ming Wang, Kristin L Rockwell, and Marie-Claire Peakman. Hit triage and validation in phenotypic screening: considerations and strategies. Cell Chemical Biology, 27(11)13322-146, 2020.
Sheng Wang, Yuzhi Guo, Yuhong Wang, Hongmao Sun, and Junzhou Huang. Smiles-bert: large scale unsupervised pre-training for molecular property prediction. In Proceedings of the 10th ACM international conference on bioinformatics, computational biology and health informatics, pp. 429-436, 2019.
David S Wishart, Craig Knox, An Chi Guo, Savita Shrivastava, Murtaza Hassanali, Paul Stothard, Zhan Chang, and Jennifer Woolsey. Drugbank: comprehensive resource for in silico drug discovery and exploration. Nucleic acids research, 34(supp DD72, 2006.
Rachel LY Wong. Megan RE Wong, Chik Hong Kuick, Seyed Ehsan Saffari, Meng Kang Wong, Sheng Hui Tan, Khurshid Merchant, Kenneth TE Chang, Matan Thangavelu, Giridha- ran Periyasamy, et al. Integrated genomic profiling and drug screening of patient-derived cultures susceptibilities involving pi3k pathway and 17q
identifies individualized copy genes in neuroblastoma. Frontiers in oncology, pp. 4247, 2021.
Yutong Xie, Chence Shi, Hao Zhou, Yuwei Yang, Weinan Zhang, Yong Yu, and Lei Li. Mars: Markov molecular sampling for multi-objective drug discovery. arXiv preprint arXiv:2 10:::2103.1444 10432, 2021.
Chengxi Zang and Fei Wang. Moflow: an invertible flow model for generating molecular graphs. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 617-626, 2020.
Wei Zheng, Natasha Thome, and John C McKew. Phenotypic screens as renewed approach for drug discovery. Drug discovery today, 18021-22) 2013.
Zhenpeng Zhou, Steven Kearnes, Li Li, Richard N Zare, and Patrick Riley. Optimization of molecules via deep reinforcement learning. Scientific reports, 9(1):1-10, 2019.
Tian Zhu, Shuyi Cao, Pin-Chih Su, Ram Patel, Darshan Shah, Heta B Chokshi, Richard Szukala, Michael E Johnson, and Kirk E Hevener. Hit identification and optimization in virtual screening: Practical recommendations based on critical literature analysis: Miniperspective. Journal of medicinal chemistry, 56(17)x650--572. 2013.
13

---

Under review as a conference paper at ICLR 2023
Table 4: Comparison of few-shot learning methods for continuous binding affinity prediction. Results for all methods besides OSLS taken from Lee et al. (2022). While code was not made available, we attempted to as closely match the reported experimental setting as possible. rm is reported for both context set size of 10 and 20 ligands, and is defined as r² (1 - v n
where 2² and ró are Pearson's 's correlation coefficient with and without intercept, respectively Lee et al. 2022).
Method
r (10 ligands)
- (20 ligands)
DeepDTA
0.299
0.360
MolTrans
0.363
0.410
MetaDTA(I
0.468
0.383
MetaDTA(T)
0.374
0.433
OSLS
0.523
0.543
A FEW-SHOT LEARNING RESULTS
Although we focus on one-shot learning, few-shot learning a related problem that important in the literature and finds applications in some forms of drug discovery. Few-shot learning involves predicting the binding affinities of query ligands to a target not seen in training, using multiple known support ligands and their affinities the same target. We seek to reproduce the few-shot binding affinity regression task previously described in the literature as closely as possible, using data from test split of BindingDB targets with more 20 compounds each, support set randomly drawn over the entire binding affinity range for a given target, and aim to predict the binding affinity of randomly chosen query ligands from the same target (Lee et al. 2022).
To make OSLS compatible with few-shot learning, we encode each context compound and its affinity using the context encoder, and then apply additional "context attention" encoder over all encoded context vectors that we average to produce a final context vector. This "context attention" module Transformer encoder with 8 attention heads, layers. and final linear layer that takes each context vector as single element of the input sequence. We do not apply positional encoding, as the ordering of each ligand in the sequence not relevant. As shown in Table OSLS produces binding affinity predictions that are significantly more correlated to the true affinity than baseline methods, for both a context set of size 10 and 20.
B DATASET DETAILS
For training, we use BindingDB (Liu et al.) 2007), a dataset of 2.6 million drug-target binding affinities, for its high data quality and broad coverage across many protein targets. We exclude very small or very large molecules, defined less than 10 atoms more than 70. We record affinities in nanomolar units from the K₃ column available, and not, KG. If neither value is available, upper lower bound, we take the bound
discard the molecule. When affinity is expressed itself as the known affinity. Then, we exclude all protein targets, and the compounds therein, if there are less than 10 compounds with data against that target if there are no compounds in any one of the binding affinity ranges shown in Figure 2 Finally, we perform a target split, where we set aside 40 targets for testing only, and train on the remaining targets, with target being defined by protein sequence. We transform all binding affinity values using the base-10 logarithm, as binding affinity often spans several orders of magnitude. All experimental settings involve compounds only in the set-aside testing set of targets. While the train/test split is otherwise random, we make sure to include the estrogen receptor in the test split, as Section 4.4 and 4.5 use the estrogen receptor in the contained experiments.
The models trained on BindingDB were also tested on a number of different datasets. The Cancer Cell Line Encyclopedia (Barretina et al.) 2012) consists of interaction data of 24 drugs against wide array of 479 -etient-derived cancer cell lines. For the results reported in Section 4.3 we use the dataset reported in Table S11 of Barretina et al. (2012), and extract IC50 measurements for each drug measured against each cell line. We exclude compounds with less than 10 or more than 70 atoms, and cell lines with less than drugs with measured activity. For the results reported in
14

---

Under review as a conference paper at ICLR 2023
use the Selleck dataset of FDA- drugs (Selleck). For drugs that are reported with a salt compound, we remove the salt from the SMILES string. We also exclude molecules based on the same size constraints above.
Section 4.4
C IMPLEMENTATION DETAILS
Hyperparameter tuning. We tuned hyperpameters of OSLS once for the task described in Section 4.2 using a < nM context compound, and used the same hyperparameters for all other context affinity ranges and subsequent tasks. We measured model performance by Kendall's tau after 2 training steps. The following hyperparameters were entered into grid search, and the observed best value for each hyperparameter is bolded: learning rate=(1e-4, 5e-4, 1e-5, 5e-5, 1e-6 5e-6), batch size=(512, 1024, 2048], d m255el 512), n heads=(8, 12, 16), n layers= 6}
Training. We used an Adam optimizer with a base learning rate of 10 and 128 steps for learning rate warmup, and then cosine annealed the learning rate 0 over 2 steps. We applied dropout (p = 0.1) over the scalar affinity projection, the token embeddings for both encoders, and at each layer in the predictor network. All models were trained server using a single NVIDIA GTX 3080 GPU.
D BASELINE DETAILS
OSLS-MLP. We implemented and trained a simpler but OSLS-like architecture to show the added benefit of the more complex, Transformer based OSLS model. This simpler model is loosely simi- lar to Koch et al. (2015), except we untie the two encoders that the context and query molecule may be encoded differently (which, like OSLS, we call the context and query encoders), and per- form regression instead of classification. In particular, the context must include molecule rep- resentation as well as activity information, but not the query, so using the same network to en- code both would be non-optimal. To represent molecules, we use Morgan fingerprints with 1024 bits and a radius of 2. and both encoders as well as the predictor (the so-called "distance layer" in Koch et al. (015)) are feed-forward ReLuvactivaeed networks with linear output. The dis- tance layer consists of only single linear layer, similarly to the previously proposed Siamese network by Koch et al. (2015). For the context molecule, we multiply the binary fingerprint vector of the compound with its activity value (the base-10 log of Ka or K2), so that each non- zero element of the vector takes on the activity value. This is inspired by POT-DMC (Godden et al.) 2004), although we do not perform an additional clement-wise normalization step. The network is trained a similar way to OSLS, with mean squared error loss between the real and predicted binding affinity of the query molecule at the output of the distance layer. We trained the model until convergence using the Adam optimizer. For hyperparameter optimization, we se- lected the highest Kendall's tau on the previously described ranking task with context molecule of < nM affinity over a grid search of the following hyperparameters (the best hyperparameters are bolded): encoder layers: {2, 3, 4}, encoder layer width: {1024), batch size: {64, 128, 256], learning rate: {le-4, 1e-5, 1e-6)
MaLSTM. We implemented and trained the MaLSTM model (Mueller & Thyagarajan 2016) to predict binding similarity using the SMILES tokens of the context and the query molecules. As the context and query encoders must share weights, we did not incorporate binding affinity information for the context molecule, and we trained the model only predict the similarity in binding affin- ity between two molecules. To do this, we used single-layered LSTM blocks, as reported in the original paper, that received learned embedded SMILES tokens as input. We computed similarity using the last hidden state of both the query and context encoders (a and b, respectively), terminat- ing at the end of each SMILES sequence, as defined by the exp of the LI norm of the difference between and &: exp (Ila- - bll.). We used the mean squared error between the predicted simi- larity and the true similarity, which we define as exp (-pb-aas(1t) (abs(T)(m.)) - TP (mmm))) where (m)) and pm(y) are the true activity values of the context and query molecules, respectively, to the target P. The selection of query and context molecules during training is identical to the process described in Section 4.2 We trained the model until convergence using the Adam optimizer. Simi-
15

---

Under review as a conference paper at ICLR 2023
larly hyperparameter optimization for model, we selected optimal hyperparameters based on the highest Kendall's 's tau on the previously described ranking task using a context ligand of < nM, and then used the same hyperparameters for all subsequent tasks. For hyperparameter optimization, we performed a grid search over the following set of hyperparameters (the best hyperparameters bolded): batch si (256, 512, 1024], embedding dim=/64, 256, 512], hidden dim-(50, 100, 200, 500}, (r- 1e-4, 1e-5]
Graph Matching Networks. We trained Graph Matching Networks for binary prediction of molecular similarity using the implementation provided at /tttps:/ggithe com/ Siin-Yijie/Graphh chingg--Mtthiin---aa
For the binary classification task described in Section 4.2] we trained and tested two separate models, one using a positive context molecule and the other a negative. Specifically, while we used a similar context and query sampling procedure as described in Section [4.2] we sampled the entire "active" range (< 10 nM) to generate context molecules for training the first model, and the entire "negative" range (2. 10 nM) for training the second model. We did not provide the model with any continuous affinity data, as the model is designed to operate only on two graphs with no additional information. As training signal for the first model, we assigned "similar" label if the query molecule was active (as the query and context molecules would both be active in that case), and otherwise "dissimilar". For the second model, we assigned a "similar" label if the query molecule was inactive, and "dissimilar" otherwise. Molecules were represented as NetworkX graphs, with bond type represented as an integer as an edge feature, and atom type as a one-hot encoded node feature. For the first model, reported ROC-AUC in similar way other baselines, but for the second model we reported AUC, as a dissimilar prediction to a non-active context molecule would actually indicate an active query. We followed similar hyperparameter optimization scheme described previously, except we used the highest ROC-AUC score instead of Kendall's tau on the test set to evaluate hyperparameter quality, as Graph Matching Networks are not applicable to ranking. We performed a grid search over the following set of hyperparameters (the best hyperparameters are bolded); graph.representate iii.eattiii.iimm111. (128, 256), prop prop-sayers- 10}, 1r-(1e-3, loss= Loss=(margin, (margin, hamming), batch.size=(10, 20, 40}
1e-4, 1e-5),
OpenEye Rapid Overlay of Chemical Structures (ROCS) OpenEye Scientific Software offers Rapid Overlay of Chemical Structures (ROCS; Hawkins et al. (006)), a program used here to de- termine shape and electronic similarities between estradiol and drugs in the [Selleck -apA-pppveed drug database (Selleck). The similarity metrics used are ShapeCombo, which scores compounds against estradiol with Gaussian spatial similarity estimations and TanimotoCombo, which combines equal contributions of Gaussian spatial similarity estimations and the ImplicinMillssean force field's (Mills & Dean 1996) high-level chemical similarity estimations. To perform the ROCS analyses, 3D conformer generation of compounds in the Selleck database was required. The conformer gen- eration was achieved using OpenEye's OMEGA software (Hawkins et al. 2010), providing the lowest-energy conformations which are then analyzed by ROCS. The result of ROCS analysis is scoring and ranking of [Selleck] database molecules by spatial and electronic similarity to estra- diol. We only used ROCS as a baseline in the drug repositioning task, as conformer generation took around a second per compound, which was too slow for the task in Section 4.2 that involved around 2 orders of magnitude more compounds.
E GENERATIVE MODELING
We selected a variational autoencoder (VAE, Kingma & Welling (2013) as our generative model due to its relatively easy training and ability to take advantage of differentiable objective functions. We trained a simple feedforward VAE to reconstruct compounds from BindingDB (the same set as used in other tasks) using byte-pair encoded SMILES tokens as molecular representation. After experimentation with other string-based molecular representations (raw SMILES tokens and SELF- IES: Krenn et al. (2020a)), byte-pair encoding of SMILES strings (Li & Fourches 2021) generated the most reasonable molecular samples from sampling random locations in the latent space (i.e. the fewest number of very large rings and uncommon atom and bond configurations). The byte pair- encoded SMILES tokens for each molecule were embedded into 512 dimensions and then padded to the maximum length, which was then flattened such that the input and output of the VAE had dimen- sion max len embedding dim. After the output of the VAE decoder, we unflattened the sequence
16

---

Under review as a conference paper at ICLR 2023
into a mar.len embedding dim matrix and applied softmax over each position in the sequence, so that the summed probability of all possible tokens at each position in the sequence was 1. The encoder and decoder of the VAE were mirrored, both having 3 1024-dimensional hidden layers with ReL.U activations and batch norm before the activation. We trained the VAF until convergence using the Adam optimizer with learning rate of 1e-4. The loss function between the true and reconstructed output plus KL-divergence term on the latent space of 1024 dimensions. During training, we warmed up the weight of the KL-divergence loss starting from 0 to 1 at epoch 1,000 to avoid mode collapse.
negative log-likelihood loss
Separately, we trained OSLS on BindingDB using the same hyperparameters as previously described except using a random context compound from the entire binding affinity range, and with byte- pair encoded SMILES instead of raw SMILES tokens so that the VAE and OSLS can be stacked, following [Eckmann et al. (2022). While the final trained model had worse performance than the raw SMILES version (see Section (.6). it was sufficient for this purpose. To make OSLS fully differentiable so that it can be used an objective function for the VAE, we applied Gumbel softmax instead of regular softmax at the output of the VAE decoder, so that the output of the VAE was fully technique, where the
discretized, but still allowed for differentiation through the srrriigh-tthrouge output of Gumbel softmax is fully discretized in the forward pass but an approximate version is used for the backward pass (Jang al. 2016). With set of discretized tokens, we were then able to perform matrix multiplication on the one-hot encoded tokens (produced by Gumbel softmax) with the weights of the embedding layer, which produces the same embeddings as the typical indexing- based method, but in a differentiable manner. These embeddings were then fed into the encoders of OSLS, and then the prediction head, making for fully differentiable forward pass from VAE latent space to output activity prediction. Similarly to Eckmann et al. (2022), we attached the input of the differentiable OSLS architecture to the output of the VAE decoder, and propagated the gradients of the OSLS output layer back to the latent space, altering the latent space in such a way as to generate compounds with higher predicted activities.
Tanimoto similarity baseline. We trained a convolutional neural network to approximate Tani- moto similarity between random compounds from the entire BindingDB dataset and estradiol, to simulate a similarity-based molecular generation process. This surrogate model consisted of 1-D convolutional layers, that take sequence of 128-dim embedded SMILES byte-pair encoded tokens as input, with a kernel size of and stride of I, ReLU activations, and a final 1-D max pooling layer. The output of the convolutional layers was flattened, then fed to a 3-layer ReLU-activated feedfor- ward network with batch normalization, which produced a final sigmoid-activatee Tanimoto similarity. The network was trained until convergence using the Adam optimizer with learning rate of 1e-3 with mean squared error loss between real and predicted Tanimoto similarity to estradiol, and the embedding layers were made differentiable, and hence able to be attached to the VAE decoder, using the matrix multiplication trick described in the previous paragraph. There was a strong correlation between the real and network-predicter Tanimoto similarity on held-out test set (r²2 > 0.9), meaning the network was an effective surrogate model for Tanimoto similarity. As the hyperparameters of OSLS were not optimized specifically for the generative task, and the trained model already had close to optimal performance, no hyperparameter optimization was performed for the Tanimoto surrogate model. Similarly to OSLS, we attached the Tanimoto surrogate model to the output of the VAE decoder and backpropagated its output back to the latent space of the VAE so that the generated molecules were of higher Tanimoto similarity to estradiol.
prediction of the
Other baselines. We compare the target-free generative models with REINVENT Olive- crona al. (2117) wttttttttt/////
MARS Xie jet al. (2021): l attps:://ithhi http://hhhh...ooooyyyy/e/eeeeaaa htwp://aaaaoooooooooeooeaeaaaaaa samp ing). and LIMO (Eckmann et al. (2022); httpp:://gittttt htt.lle.ccooossseeee
cww httpa//www....aaaa...eeaa/a/aaaa
all re- cently proposed generative models that can use full 3D target information. We ran each generative model until 256 valid molecules were generated, and then computed metrics on those molecules. We set the objective function for all models to be the docked energy to the estrogen receptor (as de- scribed in the "Docking details" paragraph). Default hyperparameters were used for all models, and pretrained models were used where applicable (the Prior of REINVENT, and the VAE of LIMO).
Metrics. We report the following metrics to evaluate the quality of generated molecules. For all methods, we only report these metrics on valid SMILES strings.
17

---

Under review as a conference paper at ICLR 2023
Best (nM) the lowest generated docked affinity value in nanomoles/liter.
# clusters the number of Butina clusters (Butina 1999), where each cluster consists of molecules that have least a Tanimoto similarity each other, computed using 1024- dimensional Morgan fingerprints with radius of 2.
Diversity one minus the average pairwise Tanimoto similarity between molecules. We use the same number of molecules across all methods compute this metric as scores may be different across sets of molecules with different sizes even if the overall number of compound scaffolds, which are most relevant to drug discovery, are the same.
Novelty the proportion of molecules that are less than 0.2 Tanimoto similarity estradiol.
Docking details. We used AutoDock-GPP
et al. 2021 to evaluate the molecules from generative modeling. grid file of PDB ID IERR was produced by AutoGrid4, using the same binding site as estradiol. 10 random conformers of each ligand to score were then generated using obabel 3. 1.0, docked against the grid file using the default AutoDock-Gii parameters, and then the best energy taken from the 10 runs as the final docked energy.
18


Reviews:
[{"review_id": "xVomD-f6_Qf", "reviewer": "Reviewer_qHx9", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper addresses the problem ligand scoring and identification of leads by screening libraries, tackling the specific problem of requiring an actual activity value or other numeric score (rather than classification) in the context of having only one labelled molecule. It approaches this problem by using an architecture that consumes a single context molecule and its label alongside a query molecule with transformer-based networks to learn the necessary functions.", "summary_of_the_review": "The paper presents small architectural changes as compared to existing few-shot methods such as neural processes, but compares primarily against baselines not used in the broader one-shot and few-shot literature but rather similarity-based baselines used in only drug-discovery.  The proposed architecture is a small adjustment on some of the ideas from the one-shot and few-shot community and is not strongly justified nor are the slightly different alternatives explored. The experimental analysis does not fully compare against all baselines that the authors raise initially (although this list is not a good survey of the state of the art in the one-shot community). In particular it does not fully compare against those which are more complex than fingerprint similarity methods, and does not yet fully justify why that is the case. ", "strength_and_weaknesses": "Strengths: \n* This particular configuration of modelling has not been seen before in the literature. \n* The range of datasets and tasks explored is wide and a detailed analysis of the results is performed.\n* The paper is in general clear and well-written. \n* The model does appear to show promise for a range of tasks such as drug scoring, ranking, classification and as an objective in molecule generation. \n \nWeaknesses:\n* The authors refer to this as a Siamese-inspired or Siamese-like network. It is not, as there is no contrastive loss or comparative triplet loss or similar that forms a component of the model. This is somewhat misleading, as the only real similarity is the encoding of two molecules \"simultaneously\", which is similar to other methods such as neural processes, prototypical networks, etc. \n* It is not truly clear why two separate transformer networks are required. While one must indeed process vectors where the scaling due to the scalar value of the context molecule has been provided, a justification, perhaps in the form of comparison to the case where the weights are tied, would be valuable. \n* On a similar point, were the transformers weights tied, this model bears quite significant similarity to an adaptive neural process. The main difference is that the context molecule \"label\" is passed from the very start of the model, rather than merely being used to compute an MSE loss at the end. It would be helpful to discuss why this is seen as the correct approach. In addition, it would be possible to have a similar architecture where the scalar embedding is concatenated with or used to scale the post-transformer embeddings, prior to the final feedforward network. Did the authors try this? A discussion of this particular choice would be valuable. \n* MetaDTA is presented as a baseline, but the results are not made available in the main body of the paper and the implementation used for results presented in the appendix is not described and is unclear. Given that it is listed as a baseline, it should be presented in the body of the paper alongside the other baselines. In addition, it is referred to as a few-shot only technique in the appendix, but there is no reason the context set cannot be reduced to the size of N = 1, in general, so it should be a suitable comparative baseline. \n* All baselines are not presented together in Figures 4(a) and 4(b). It is possibly expected that in particular the Tanimoto and MaLSTM baselines might be weaker in Figure 4(a) in particular, and it would be useful to see results for the others mentioned. \n* It is not clear why in the section on scoring anti-cancer compounds a neural process baseline such as MetaDTA is not used, as this is a few-shot/ one-shot regression task. \n* The sentence (anti-cancer compounds paragraph 5) \"similarity-based methods will yield identical similarity values between the tested drugs regardless of experimentally-determined activity value, meaning they will have no predictive power\" is correct but confusing given the context. This is because many of the baselines in the list are presented as indeed being similarity measures but are being used to predict a unique score for each query molecule, in other parts of the paper (as a particular example, in Figure 2(a) MaLSTM and Tanimoto similarity are being used to predict affinity). Why is it not possible to predict a score in this new task with these baselines? If it is, they must be included in Figure 3. If not, then the paper needs to clarify this point in more detail, as currently there is not comparison in Figure 3 to any \"external\" method. \n* Similarly, why is MetaDTA or another similar method excluded from Table 1. All baselines should be reported for fair comparison. \n* In Figure 4 only Tanimoto similarity is given as a comparison objective in molecular generation. The claim is that no other model is readily differentiable, but it is not clear why this is the case, many of the baselines should be differentiable or have (as is claimed here for the tanimoto case) surrogates that are. In general this part of the paper is not clear, and does not seem well supported. The comparison are being made among other generative models rather than the objective function, which is the issue under investigation. The paper might be better to choose a single state of the art generative model and then investigate a broader range of objectives including the author's own. In addition, in Figure 4(a) there is no comparison even to the tanimoto baseline. \n* The appendix lacks any details about MetaDTA, a description as thorough as those given for other models would be helpful. \n* There exist a range of pretrained networks for molecular property prediction that could be incorporated in a neural process architecture to provide a robust baseline. Did the authors try anything like this? \n* The work references related work in N-shot learning but does not try the range of methods referenced in the low shot setting, where they may be equally applicable. \n* The work also suggests that the N-shot methods can only be used in binary classification settings, which is not correct. A trivial change to these architectures on a final layer and choice of a different loss create a regression model. It would be helpful to consider this section of the literature and make reference to it. \n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "mHRZwQS5cpP", "reviewer": "Reviewer_ZuUo", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes One-Shot Ligand Scoring (OSLS) to predict activity of a compound to an unseen target based on a single context drug and its known activity to the given target, without any target features. They frame the ligand scoring as a one-shot problem, and argue for its utility over traditional similarity based scoring.\n\nOSLS is a Siamese inspired neural architecture with a novel linear projection to model the activity of the context compound.\nThey show that OSLS outperforms both similarity based and deep-learning scoring methods in different settings and tasks related to drug development.\n", "summary_of_the_review": "There are many missing details that hinder the paper's reproducibility, especially in the evaluation section.\n\nAuthors should try to justify the validity of their prediction setting more (for example, in which cases researchers have no information about protein targets) or show what advantages their approach has over other methods.\n\nThe evaluations proposed are not convincing.\n\n\n\n\n\n\n", "strength_and_weaknesses": "The paper is well written and easy to read in most sections. The model is easy to comprehend, and the explanation is short and to the point. The results comparing to one shot approaches seem interesting and promising.\n\nScoring ligands based only on similarities of ligands, without information about the targets is an old approach. The use of information about targets has been shown to give better performance (Yamanishi et al. 2008, Prediction of drug–target interaction networks from the integration of chemical and genomic spaces). To support the utility and applicability of the proposed problem, authors could show a realistic scenario where there is no information about the protein targets. It would be interesting to see how OSLS compares to state of the art methods using target information. It would also help the paper to show any practical example of an application of ligand scoring where there is no information about the targets. \n\nWhile the use of one shot approaches in ligand scoring can be theoretically justified, I am not convinced that there is any advantage of it over other n-shot approaches to the problem. My previous point also applies here, it would help the paper to show any practical example of an application where an n-shot approach is not applicable, or at least show any advantage of OSLS against n-shot methods. Table 4 in the appendix is a good start, but a more thorough analysis is needed.\n\nThe paper should be self-contained. More details in the next section. \n\nIn section 4.2, I would also like to see the AUPR, which is preferred in the literature for this classification task. \n\nSection 4.4 is titled tries to show the use of OSLS in the drug-repositioning problem, but the task evaluated is drug target prediction (DTP). Drug repositioning is the prediction of drug disease associations. While DTP can be used in drug repositioning, the problems are not equivalent. If authors want to use OSLS in drug repositioning, they should change the experiment to show how OSLS can help to predict drug-disease associations. If the authors want to keep the experiment, they should show a systematic evaluation for DTP (for example “Wan, F. et al, NeoDTI: neural integration of neighbor information from a heterogeneous network for discovering new drug–target interactions. Bioinformatics 35, 104–111 (2019).”)\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "SoDVA8ksQlU", "reviewer": "Reviewer_SvkE", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes one-shot ligand scoring (OSLS), a method to score potential drug candidates based on a single known active drug molecule. The method is essentially a Siamese neural network which takes in the active and query molecule to predict a score.", "summary_of_the_review": "Despite being an interesting attempt at a relevant problem (drug-discovery), I think the authors target a very unrealistic variant of the problem, propose a method which seems like a fairly general metric learning method, and evaluate it in a way which I think is flawed. Together these things make me think that rejection is the appropriate choice.", "strength_and_weaknesses": "**Strengths**\n\n- Writing is generally clear\n- Given the problem setting, the method is reasonable (a neural-network based similarity)\n\n**Weaknesses**\n\n- Problem setting is unrealistic: I do not think predicting compound activity given just one active compound is a problem that has ever or will ever occur in drug discovery. As the authors mention in their introduction, initial hit compounds come from large parallel screening of compounds, which not only may produce multiple hits, but will also produce many non-hit compounds. By framing the problem as one-shot learning, the authors implicitly suggest discarding this information, which does not make sense to me and is not well-justified. I think this also makes their baseline methods unnecessarily weak: for example, Tanimoto similarity would not just be computed based on a single reference molecule, it would be computed based on all measured molecules (e.g. the inactive molecules too). This would be used to create a more complex ranking.\n- Inappropriate metrics used: in Figure 2-3 the authors evaluated the methods based on the correlation between similarity and actual score across all molecules. This does not reflect how these methods will actually be used, which is just to find top compounds. Only the scores of the top few compounds are important; it does not matter if the correlation is low for very dissimilar molecules. In general I think the assumption of these retrieval-based methods is that the most similar compounds are likely to be active, _not_ that dissimilar compounds will be inactive. I think different metrics are required.\n- Field of metric learning is not discussed in the related work. OSLS essentially seems like a metric learning method so I think these methods should be discussed.\n- In the generative modelling baseline, I think that training a differentiable surrogate to Tanimoto similarity introduces an extra source of error which makes this baseline unnecessarily weak. I think the authors should find a different way to incorporate this information.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "2lF5y-24vz", "reviewer": "Reviewer_3vLq", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The work introduces a novel OSLS (One-Shot Ligand Scoring) model for ligand scoring task to an unseen target based on a single context compound and its experimentally known activity to that target. This model is based on a Siamese-inspired neural network with transformer encoders. The OSLS achieves performance improvements on various benchmarks.", "summary_of_the_review": "The authors focus on the molecular scoring task, and the model lacks innovation. \nThe Introduction of the proposed model is clear, but some experiment results do not support its effectiveness for molecule generation.", "strength_and_weaknesses": "Strengths:\n   The authors explain the problem in detail and conduct various experiments, giving the reader the opportunity to look at the problem from a new perspective.\n\nWeaknesses:\n\n1) According to Figure 2 and Table 1, model with transformer architecture (which has much many parameters) does not outperform significantly that with MLP architecture (which has much less parameters). Is transformer architecture necessary?\n\n2) According to Table 2, the proposed model did not show better performance than two baseline models. Maybe it is better to focus on drug-target interaction prediction rather than molecule generation (e. g. conducting drug-target binding affinity prediction on more datasets).\n\n3) It is necessary to provide the statistical information  of the dataset.\n\n4) In Figure 2, which part of the model makes the model performance stable under different context affinity?\n\n5) Can OSLS score the topological similarity of molecules? And if so, which part of the experimental results can justify this conclusion?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject