Regression modeling on DNA encoded libraries
Abstract
DNA encoded libraries (DELs) are pooled, combinatorial compound collections where each member is tagged with its own unique DNA barcode.
DELs are used in drug discovery for early hit ﬁnding against protein targets.
Recently, several groups have proposed building machine learning models with quantities derived from DEL datasets.
However, DEL datasets have a low signal-to-noise ratio which makes modeling them challenging.
To that end, we propose a novel graph neural network (GNN) based regression model that directly predicts enrichment scores from raw sequencing counts while accounting for multiple sources of technical variation and intrinsic assay noise.
We show that our GNN regression model quantitatively outperforms standard classiﬁcation approaches and can be used to ﬁnd diverse sets of molecules in external virtual libraries.
Introduction
Small molecule drug discovery begins with the identiﬁcation of putative chemical matter that binds to protein targets of interest.
This can be be achieved with experimental techniques such as highthroughput screening or in silico methodologies such as docking and generative modeling.
DNA encoded library (DEL)[11, 6, 2, 5, 4, 7] screening is a high throughput experimental technique used to identify diverse sets of chemical matter against targets of interest.
DELs are DNA barcode labeled pooled compound collections (Figure 1.A) that are incubated with an immobilized protein target in a process referred to as panning.
The mixture is then washed to remove non-binders, and the remaining bound compounds are eluted, ampliﬁed, and sequenced to identify putative binders.
DELs provide a quantitative readout for hundreds of millions of compounds.
This readout can contain substantial experimental noise and biases caused by sources such as DEL members binding the protein immobilization media or differences in starting population (load).
This noise is often controlled for by computing an enrichment metric[6, 11, 7, 5] that compares the compound population to its starting population and control experiments.
Often, multiple replicates and off-target counter pans are used to improve the signal-to-noise ratio.
Recently, several groups [14, 12] have demonstrated powerful applications of ML to DELs.
McCloskey et al.
[14] trained classiﬁcation models on labels generated with aggregated DEL counts and used the models to perform inference on large virtual libraries (VLs).
This inference yielded diverse hits for several protein targets.
However, their approach required determining a label boundary, which has no standard process and may bias the dataset.
Moreover, classiﬁcation models struggle to differentiate between weak and strong binders.
This becomes important when prioritizing compounds for testing due to the multi-billion scale of VLs.
Lim et al.
[12] improved on the classiﬁcation approach by directly modeling an enrichment metric (the ratio between counts from a target and an off-target pan).
Their model was trained using a custom negative-log-likelihood loss function derived from a Poisson ratio test.
However, it is not obvious that their method can be expanded to multiple off-target experiments or extended to include additional covariates.
By contrast, we introduce a general framework for quantitative modeling of DELs that incorporates multiple sources of experimental noise.
We show that GNNs trained using our framework and regularized negative binomial loss outperform classiﬁcation models in replicating experimental compound rankings generated with proxy measurements of binding afﬁnity on a held out DEL.
We end by examining key metrics from a virtual screen of the Enamine REAL and Wuxi VLs.
Methods
We obtained multiple proprietary DEL panning datasets screened against a challenging protein target, Interleukin 17A (IL-17A), a relevant target to autoimmune diseases.
These datasets include several control and off-target pans.
Here, we present results for a diversity screening library of 100M compounds (Lib1) that we used for training and a separate expansion library of 2.5M compounds used for validation (Lib2). The library synthesis, the panning experiments and the bioinformatics decoding and counting pipeline were performed at DICE Therapeutics.
To start the synthesis, DNA barcodes are assembled.
These barcodes have variable regions, codons, that encode the building blocks added to the synthesis and invariable regions that separate the codons.
A unique molecular identiﬁer(UMI) region is added to each DNA strand in the starting pool of barcodes. The UMI helps to de-bias the PCR ampliﬁcation process, in which certain DNA barcodes may be more ampliﬁed than others. For libraries without UMIs, PCR bias can be a source of error. During library synthesis, chemical building blocks are added sequentially to each well and are encoded by their corresponding codon.
All wells are then pooled back together before the the next building block is installed.
Repeating this process creates a combinatorial chemical library with DNA barcodes corresponding to the chemical blocks added.
However, chemical reactions have different yields and so the compounds attached to the same DNA encoding consists of a mixture of different products.
Truncated product controls by omitting building blocks are often incorporated into the design of the library to ensure that a possible truncation is not what is binding.
Furthermore, it is possible to have side products and racemization in the reactions.
Combined, these factors confounds the count readout of each compound, making the readout a noisy signal for binding afﬁnity.
Additionally, some reactions and conditions may be
DNA damaging, thereby reducing the DNA that can be decoded at the end of the panning procedure.
To control for the overall imbalance in starting barcodes, the DNA barcodes are sequenced before the panning experiment (load counts).
Finally, the library is incubated with the protein of interest.
During this process, the DNA barcodes may stick to the protein (not a major risk for IL-17A) and the media of the experiment, including the beads and the protein’s immobilization tags.
This can be controlled by negative control pans which consist of running the panning experiment with the same experimental media, but without the protein or with an alternative control protein.
Finally, problems with promiscuous compounds, compounds that may bind to multiple proteins, can be mediated by analyzing panning experiments for multiple proteins.
Within our datasets, we had access to several such negative control panning data.
To provide a competitive baseline, we built and optimized a classiﬁcation model using the same GNN architecture as the regression model GIN-E (Graph Isomorphism network with virtual node [15]).
We assigned binary labels for binders (positives) and non-binders (negatives) using a two-step thresholding process.
First, we discarded compounds with on-target unique molecular identiﬁer (UMI) counts below a noise threshold.
Second, we normalized compound UMIs in each pan by the sum of all UMIs in the pan to yield molecular frequencies (MFs).
We then calculated the ratio between the on-target and max control or off-target MF.
If a compound’s MF ratio exceeded a positive cutoff or fell below a negative cutoff, we assigned it a positive or negative label, respectively.
Compounds with ratios falling between the cutoffs were discarded.
This yielded ~74K positives and ~5.6M negatives.
We experimented with combinations of sampling schemes and losses to address the class imbalance, and found that Focal Loss [13] without balanced sampling performed best.
Additionally, we regularized the model with dropout in the layers after graph readout and with input augmentations.
Regression Model
We used negative binomial regression to model the UMI from each panning experiment.
This approach for denoising counts has previously been shown by [8] who used a regularized negative binomial regression model to de-noise UMI readouts from single cell RNA sequencing.
They deﬁned enrichment as the Pearson residual after regressing the count on sequencing depth.
Similarly, we modeled the enrichment for each compound as the residual after accounting for various covariates such as binding to beads.
As a generalization of Poisson regression, negative binomial regression incorporates a dispersion parameter α in addition to a mean variable µ .
For one target pan and two no- target control pans, C i,target ∼ NB ( µ i,target , α target ), C i,control 1 ∼ NB ( µ i,control 1 , α control 1 ) , and C i,control 2 ∼ NB ( µ i,control 2 , α control 2 ) represent the UMI counts of i th compound in the respective panning experiments.
We modeled µ i as the combination of enrichment from binding to the target ( R i,target ), enrichment from binding to the non-target media ( R i,control 1 , R i,control 2 ), and observed count of the compound in the original starting population load ( S i ).
[Equation]
β i are learned from the data and σ represents the softplus function, which we found to be more stable during training than the typical exponential function.
The dispersion parameter, α , of the negative binomial is a single scalar, learned for each experiment.
We related R i,target and R i,control to each compound’s structure by deriving their values with a GNN operating on the compound’s molecular graph.
A shared encoding network generates a 128 dimensional embedding vector from atom and bond features.
This embedding vector is then transformed into R i,target , R i,control 1 , and R i,control 2 by separate feed forward networks (Fig 1.B).
For our experiments, we used a GIN-E network with virtual node [15, 9, 10] for the initial encoding and two layers in each of our feed forward networks.
During training, we summed the negative log likelihood of the observed counts for the target and control pans.
Furthermore, we L2 regularized the enrichment values, which empirically prevented over-ﬁtting.
For a single example with count c i for each panning experiment , the loss can be written:
[Equation]
where Γ( x ) is the gamma function and γ is the L2 regularization rate.
This negative binomial regression can be further extended with other covariates such as enrichment in other negative control pans, other target pans, compound synthesis yield, and reaction type.
For our experiments, we used 13 negative control pans.
During validation and inference for virtual screening, we used our de-noised enrichment value R i,target to rank compounds.
Cross Library Validation
After training on Lib1, we validated our models on Lib2 which had proxy binding afﬁnity measurements.
Binding afﬁnity of a compound to a target can be measured by the equilibrium disassociation constant Kd and corresponding negative log value pKd.
Lib2 was used in a set of target titration panning experiments[3] where corresponding curves are ﬁtted to produce a predicted titration-based pKds (t-pKds).
A small portion of these t-pKds were validated with off-DNA pKd measurements (R 2 = 0.84).
We measured model performance by calculating the Spearman correlation coefﬁcient between model predictions and the t-pKds.
This metric aligned with our intended use of the models to rank VLs for candidate selection.
The R target predicted by our regression model had a 0.41 (95% CI [0.40, 0.43]) Spearman correlation with t-pKds (Fig 2.A).
This exceeds both a Random Forest classiﬁcation baseline (0.28) and our GNN classiﬁcation model (0.35 (95% CI [.34,.37]))
(Fig 2.B).
Furthermore, both the GNN regression and
GNN classiﬁcation models trained from Lib1 showed better correlation with t-pKds from Lib2 than
UMI counts from a single pan of Lib2.
This illustrates both the high noise in the raw UMI output from a single panning experiment and our models’ ability to generalize.
Finally, we compared the two GNN models’ retrieval rates for strong binders in their top prediction results.
The regression model had more binders in its top prediction results than the classiﬁcation model (Fig 2.C).
Virtual Screening
We used our regression model to perform a virtual screen of 3.7 billion compounds in Enamine and
WuXi’s VLs.
We then thresholded the regression model’s output to select a limited number of top scoring compounds for analysis.
We established the threshold using a quantitative analysis of the enrichment distribution in both the validation and inference sets (Fig 3.A).
We clustered this top set using Taylor-Butina clustering[1] with a similarity cut-off of 0.6.
Structural similarity was calculated via Jaccard similarity of Morgan ﬁngerprints.
This clustering yielded 26 clusters and we selected top scoring compounds from each cluster for testing.
These selected compounds are structurally diverse (Fig 3.C).
Conclusion
DEL experiments yield datasets with low signal-to-noise ratio.
In our work, we show a novel regression technique for modeling DEL sequencing counts that accounts for various sources of variation, such as media binding and differences in initial load.
Our model’s predicted enrichment values have better correlation with proxy binding afﬁnities than those of baseline classiﬁcation models or experimental values from a single panning experiment.
Finally, we demonstrate that our model retrieves diverse compounds during virtual screening. Acknowledgments
The authors would like to acknowledge the team at DICE therapeutics for graciously providing the
DEL data used in this paper.
We would also like to thank multiple insitro colleagues for their help in directly and indirectly making this manuscript possible.