D E NOVO DESIGN OF PROTEIN TARGET SPECIFIC SCAFFOLD - BASED I NHIBITORS VIA R EINFORCEMENT L EARNING
A BSTRACT
Efﬁcient design and discovery of target-driven molecules is a critical step in facilitating lead optimization in drug discovery.
Current approaches to develop molecules for a target protein are intuition-driven, hampered by slow iterative design-test cycles due to computational challenges in utilizing 3D structural data, and ultimately limited by the expertise of the chemist – leading to bottlenecks in molecular design.
In this contribution, we propose a novel framework, called 3D-MolGNN RL , coupling reinforcement learning (RL) to a deep generative model based on 3D-Scaffold to generate target candidates speciﬁc to a protein building up atom by atom from the starting core scaffold.
3D-MolGNN RL provides an efﬁcient way to optimize key features by multi-objective reward function within a protein pocket using parallel graph neural network models.
The agent learns to build molecules in 3D space while optimizing the activity, binding afﬁnity, potency, and synthetic accessibility of the candidates generated for infectious disease protein targets.
Our approach can serve as an interpretable artiﬁcial intelligence (AI) tool for lead optimization with optimized activity, potency, and biophysical properties.
INTRODUCTION
Recent advancements in machine learning (ML) and artiﬁcial intelligence (AI) have demonstrated the potential to revolutionize the drug design process by reducing the initial chemical search space in the early stages of discovery (You et al., 2018;
Born et al., 2020;
Li et al., 2018; Huang et al., 2020;
Chenthamarakshan et al., 2020).
Currently, the potential chemical space is composed of > 10molecules, candidates with suitable activity against speciﬁc protein targets only narrows the search space signiﬁcantly based on the critical fragments.
The COVID-19 pandemic has brought a surge of interest to explore data-driven methods to better produce efﬁcacious drug candidates (Huang et al., 2020).
One of the most important factors in identifying new drug candidates is that the molecules possess optimized properties that allow them to effectively bind with a required target while also having minimal off-target effects.
If we consider molecular generation as a controlled and dynamic step-by-step process, it is possible to produce end products that possess these optimized properties.
This approach allows us to formulate de novo drug design as a reinforcement learning (RL) problem and utilize algorithms that best learn a molecule’s representation space based on the core moiety and its spatial interaction with the protein target.
As an alternative, RL would provide a platform to create a highly-efﬁcient inverse molecular design AI-system capable of producing novel high-performance molecules with domain-targeted properties.
1. R ELATED W ORKS
Our current work presents a novel approach to address the problem of generating molecules optimized for speciﬁc 3D protein targets starting from core functional groups so called scaffold.
In recent years,
RL-based works to generate molecules have been introduced that attempt to tackle the problem in different ways.
A method by Popova et al.
(2018) utilizes a fragment-replacement-based approach to optimize existing SMILES strings for speciﬁc drug-like properties, while another method by
St˚ahl et al.
(2019) generates entirely new SMILES.
Several current RL methods rely on the use of variational autoencoders (VAEs), which learn the latent space representation of 2D molecules to ﬁnd the best representation (Lim et al., 2018; Jin et al., 2018; Liu et al., 2018; Sattarov et al., 2019;
Joo et al., 2020).
However, none of these methods take into consideration the 3D protein structure during the generation phase.
Some of these methods do test the drug-likeness post-generation against some given target protein, but the target is never directly involved in the RL loop.
When designing drug candidates that target speciﬁc proteins, learning how the molecule interacts with the protein’s structure is invaluable to the generative process and can greatly assist in accelerating the automation of medicinal chemistry.
Recently, Born et al.
(2020) proposed a deep reinforcement learning framework called PaccMann RL for designing antiviral candidates binding against given protein targets using 2D protein sequence data.
The generative model is composed of two pre-trained VAEs, namely a protein-VAE and a
SELFIES-VAE, for mapping protein and drug molecules to a multi-modal latent space.
The critic model is composed of two predictive models for predicting binding afﬁnity of the protein-ligand pair and toxicity for the ligand.
These predictions are used to formulate a multi-modal reward function to penalize the generative model (Born et al., 2021).
This method still uses VAEs to generate 2D molecules, but it combines the latent space of the SMILES with the embedding of a speciﬁc protein to generate SMILES that consider target protein information
1. P ROBLEM F ORMULATIONS AND P ROPOSED M ETHOD
We propose a new method, known as 3D-MolGNN RL , that not only incorporates the protein structure into the RL loop, but considers the 3D structures of the generated compounds by constructing them atom-by-atom in 3D space.
To the best of our knowledge, this is a process that hasn’t been explored as most methods simply rely on 2D representations of the molecules in the form of a SMILES string.
To overcome the sequence-based representation of the target and the drug, we leverage a 3D-scaffoldbased molecular generation actor over a 2D SMILES generation actor.
The 3D-MolGNN RL method uses our previously formulated 3D-scaffold-based generative model Joshi et al.
(2021) to make atom-wise placements on the molecule starting from a desired scaffold.
Our method considers a similar approach to that of the actor-critic method demonstrated in the
PaccMann RL model.
In our approach, we utilize a scaffold-based generative model, 3D-Scaffold (Joshi et al., 2021) to produce valid 3D compounds as the actor model.
For the critic model, we utilize parallel graph neural networks as a binding probability predictor (GNN P (Knutson et al., 2021)) to evaluate whether the generated compound actively binds with a target protein.
This method takes the 3D structure of the protein pocket and the ligand, and predicts the probability of their interaction without any prior knowledge of the intermolecular interactions.
The GNN P model uses both the residue and 3D atomic-level representation of a protein as well as the 3D atomic and bond level representation of the molecule.
EXPERIMENTS AND DATASET
To demonstrate our methods ability to produce molecules for a given target protein, this work was centered around creating drug-like molecules that interacted with the SARS-CoV-2 M pro .
To train the model, We used a dataset of non-covalent inhibitors from the BindingDB dataset (Gilson et al., 2016) and FDA approved drugs FDA.
These compounds were ﬁltered based on biophysical and cheminformatics properties like IC 50 , molecular weight, atom type, and functionality and have almost 10K unique scaffolds.
To represent the deﬁnition of scaffold, we used Murcko scaffolds Bemis & Murcko (1996).
A similar dataset was used in the prior 3D-Scaffold model (Joshi et al., 2021).
High-throughput experimentation and our computer aided drug design results suggest that compounds containing the functional group piperazine are potent and have a higher afﬁnity towards the M pro target (Clyde et al., 2021;
Joshi et al., 2021)
However, there were no piperazine-containing molecules in the initial dataset used for training the 3D-scaffold model, but we were still able to generate some molecules containing piperazine using the trained model.
We curated a smaller subset of the
BindingDB dataset that possessed afﬁnity towards protease-like targets and combined them with the piperazine dataset.
Finally, we ﬁltered these compounds based on their ability to bind with M pro by doing an initial pass through the GNN P model.
Since our motive is to achieve better binding afﬁnity, potency and easily synthesizable compounds for the M pro target, the initial screening helped us to choose proper compounds for training the RL models so that they learned to produce similar, or better compounds.
METHODS
In this section, we discuss in detail the architecture and implementation of the RL approach.
In particular,we go into detail about how the critic interprets the output from the actor.
[Figure] The schematic representation of 3D-MolGNN RL framework is shown in the ﬁgure 1.
The RL workﬂow begins from training a 3D-scaffold molecule generator followed by optimizing the partially built candidates towards the target of interest at every step of training.
The neural network used in the 3D-Scaffold framework for de-novo drug candidate design can be broken into two major blocks: feature learning and atom placement as shown in Figure 1.
In the feature learning block, the embedding and interaction layers of SchNet (Joshi et al., 2021; Sch¨utt et al., 2019; Sch¨utt et al., 2017;
Sch¨utt et al., 2017; Sch¨utt et al., 2018) are used to extract and update rotationally and translationally invariant atom-wise features that capture the chemical environment of an unﬁnished molecule.
The extracted features are used to predict distributions for the type of next atom and its 3D coordinates, where the latter distribution is constructed from predictions of pairwise distances between the next atom and all preceding atoms.
The whole procedure is repeated successively to build a complete molecule with the desired scaffold.
The partial molecule associated with each step of this atom placement is assessed by the critic (GNN P ).
As a part of our experimentation, we used two critics simultaneously to use the binding probability and binding afﬁnity of the partial molecule and the target of interest along with the synthetic accessibility of the partial molecule.
The binding probability is the outcome from the SoftMax layer which predicts the activity/inactivity of a protein-ligand complex.
The binding afﬁnity is measured in terms of K i and K d , which refer to the inhibition and dissociation constants, respectively and is given as − log ( K i K d ) .The
two critics essentially use the same feature representation, but differ in terms of the data they have been trained on and also the label associated with the data.
A more detailed breakdown of the layers and hyperparameters associated with the actor and critic models is included in the Appendix.
3. R EWARD F UNCTION & P ARAMETERS
The process of building the molecule starts from a desired scaffold associated with the selected core functionality.
For every action t which is a random selection and placement of the atom to the partial molecule, the 3D-scaffold model predicts the next possible atom that could be placed close to the center of mass to the partial molecule at step t − 1 and transitions to state s t .
At any step, let Z next be the ground truth type of the next atom and
ˆ p
Z next type the probability that the model assigns to that type at the current step.
This probability is converted to a negative log-likelihood as P Θ ( s t ) type (equation 3a).
Here, ˆ p bj is the probability that the model assigns for the distance between position of already placed atom and ground truth of the next atom to fall into distance bin b ∈ B at the current step.
The distance based probability is calculated using Gaussian expanded ground truth distances q bj and probability distribution of atom placement ˆ p bj s shown in equation 3b.
The overall idea is to train the agent to learn the latent space representation of atom type and it’s possible placement in 3D-space closest to the center of mass of the molecule generated so far.
The overall probability for any action t is the summation of type-probability and distance-probability (equation 2).
The ultimate goal is to train the agent such that it learns to generate new compounds while optimizing the policy Π(Θ) (equation 1).
The policy is deﬁned as the difference between the action-probabilities and the reward assigned by the critic at that step.
[Equation]
We used two different reward functions as R 1 ( s t ) and R 2 ( s t ) associating with two independent experiments for the purposes of ablation/hyperparameter testing.
By removing portions of the reward function, we can see which attributes provide the best reward.
The ﬁrst reward is a function of binding probability, binding afﬁnity of the target and the partially built molecule in addition to the synthetic accessibility of the molecule.
While the second reward function uses only the binding probability and the synthetic accessibility score.
We assigned different weights to each of these components while calculating the rewards.
For R 1 ( s t ) (equation 5), we used 0.5,0.25 and 0.25 respectively for binding probability, binding afﬁnity and synthetic accessibility.
While for R 2 ( s t ) (equation 4, we used 0.
and 0.25 for binding probability and synthetic accessibility respectively.
We trained the agent for 150 epochs and chose the best trained model to generate compounds.
The Synthetic Accessibility and Binding Afﬁnity scores are scaled to be in between 0 and 1 to match the scale of the binding probability.
For more on the ablation study, including some reward functions not shown in the main text, see Appendix C.
RESULTS & DISCUSSION 4. M ODEL P ERFORMANCE
To ensure that we are a learning to build molecules based on the training data, we visualize the training and validation error over each epoch.
Figure 2 represents the aggregated loss over theepochs for our 3D-MolGNN RL model.
As loss decreases over time, we know that our model is able to effectively learn from the training data.
This result is strictly evaluating the training and validation loss through the actor model to evaluate training performance, the drug-likeness scoring of the GNN model is discussed further in the next section.
4. D RUG - LIKENESS M ETRICS FOR G ENERATED C OMPOUNDS
We examined the effectiveness of our RL methods using different drug-likeliness metrics: quantitative estimate of drug-likeness (QED), water solubility (logS), synthetic accessability (SA), and the octanolwater partition coefﬁcient (logP).
See the appendix for a full explanation of each metric.
A desirable drug candidate would score well in each of these metrics.
Since the agent for the 3D-MolGNN RL is a scaffold based generative model, we focused on generating compounds with piperazine as the scaffold.
To ensure that only valid molecules are compared, we ﬁltered the total list of generated compounds based on a modiﬁed Lipinksi rule.
The rule suggests that for a candidate to be an acceptable drug-like compound, there should be no more than 5 hydrogen bond donors, no more than 10 hydrogen bond acceptors, no more than 5 rotatable bonds, a molecular mass between 200 and
Dalton, an octanol-water partition coefﬁcient less than 5, and ﬁnally at least one aromatic ring in the structure.
Once ﬁltered, approximately 100 top compounds per method were obtained.
4.2. C OMPARISON TO 2D G ENERATIVE M ODEL
To show the advantages of using 3D-based generative modeling over conventional 2D-based generative models (SMILES-based), we compare our 3D-MolGNN RL model to that of PaccMann RL .
For better comparison of the two methods, we utilize a slightly modiﬁed version of the architecture to include our GNN critic model instead of the default PaccMann critic.
This allows us to have a metric of comparison between a 2D and 3D generative model.
For purposes of discussion, the modiﬁed
PaccMann RL model will be referred to as PM-GNN RL .
We ﬁrst wanted to demonstrate that the RL implementation of the methods were essential in producing molecules with a high binding probability.
To accomplish this, we generated molecules for the protein target M pro (PDB:6WQF), both with and without passing the reward from the critic back to the agent in each method.
We designated here the molecules generated without RL optimization as unoptimized .
To fairly compare the optimized vs unoptimized methods, we ensured that the unoptimized models were trained and tested on the same datasets used for training the respective RL models.
The comparison of binding probabilities of the generated compounds towards the M pro target from the 3D-MolGNN RL and PM-GNN RL is shown in ﬁgure 3a and 3b, respectively.
From this ﬁgure, it is evident that the RL mechanism has improved both agent’s performance in producing candidates with high binding probability.
The unoptimized molecules generated by the agent in 3D-MolGNN RL and
PM-GNN RL show very low binding probability towards M pro as the predictions have a low mean probability.
On the other hand, when using biased compounds during RL optimization, we see a signiﬁcant increase in high binding probability compounds, showing the effects of incorporating RL with the generative models.
To further assess the novelty and properties of generated compounds, we looked at the four different drug-likeness properties listed above to serve as comparison metrics.
Figure 4 compares these metrics for both 3D-MolGNN RL reward functions with known active molecules for M pro .
A full description of each metric is given in the appendix (section A).
The ﬁrst metric, QED, represents a quantiﬁcation
[Figure]
of the desirability of the drug (Bickerton et al., 2012).
We can see that in Figure 4a, the RL methods all produce higher scoring molecules than the current experimentally determined M pro actives with the mean of the PM-GNN RL having a slightly higher increase in mean.
We next look at the distributions of water solubility calculated using the ESOL method (Delaney, 2004).
Figure 4b shows that our methods score better than the current M pro actives while also showing that these methods produce molecules with desirable solubility in water.
The mean of the 3D-MolGNN RL ’s R 1 ( s, t ) model has the highest improvement relative to the mean of the M pro actives while both the R 2 ( s, t ) and PM-GNN RL models improve less.
The next metric, synthetic accessibility (SA), is a term included in each of the reward functions of the RL methods tested.
Figure 4c illustrates the distributions obtained by each method where a SA score closer to 1 is preferred.
The two 3D-MolGNN RL models have a consistent range of SA scores that center around 5, which indicates consistently above average scores.
R 1 ( s, t ) shows an 8.78% improvement and R 2 ( s, t ) shows a 9.92% improvement.
The PM-GNN RL spans a much wider range of SA scores, but the mean is centered around 4.
These three models only slightly edge out the existing M pro actives.
The last metric is the octanol-water partition coefﬁcient, or more simply known as log P .
Looking at Figure 4d, we can see how each method produces molecules spanning the range of -2 to 6.
The 3D-MolGNN RL model using the R 1 ( s, t ) reward yields a 61.21% improvement to the M pro actives in logP value, whereas using the R 2 ( s, t ) reward yields a 53.08% improvement.
In this metric, the
PM-GNN RL metric produces compounds very similar to the M pro actives.
[Table]
To compare how each method performed on a per-molecule basis, Table 1 gives the results for the top 3 candidates produced by each of the methods.
These molecules were ranked based on their predicted binding probability with the M pro target.
We can see that molecules that are performing above average in one metric are not guaranteed to perform well in every metric.
For example, the top candidates for each method are each above the mean SA for the given method, but produce below
[Figure]
Since our goal is achieve de novo design, additional metrics such as validity, uniqueness, and novelty were considered among the piperazine-based generated compounds as shown in Table 2.
Initial validity was determined by processing the molecules using cheminformatics tools.
The methods of calculating these properties are described in further detail in appendix subsection A.5.
From table 2, we found that 3D-MolGNN RL R1 and 3D-MolGNN RL R2 produced greater than 95% valid compounds while also scoring close to 100% in uniqueness and novely from this valid set.
The
PM-GNN RL model, while not scoring quite as well in Uniqueness and Novelty, scores the highest in
Validity.
This shows that by incorporating more parameters into the multi-objective reward function, there is an improvement in the generation of novel drug candidates.
Overall, 3D-MolGNN RL R outperformed other methods by generating compounds with high Validity, Uniqueness and Novelty.
[Table]
CONCLUSIONS
In this work, we introduced a new method to include both the 3D structure of protein target and the generated compounds to perform multi-objective lead optimization critical for drug design and discovery.
We demonstrated that our novel framework 3D-MolGNN RL , which couples RL to a deep generative model based on a 3D-Scaffold, can generate target candidates built up atom by atom that are speciﬁc to a protein pocket.
3D-MolGNN RL provides an efﬁcient way to generate target speciﬁc candidates by learning to build molecules in 3D space while optimizing the binding afﬁnity, potency, and synthetic accessibility.
To accomplish this, we utilized the protein for SARS-CoV-2 M pro as a target for generating optimized inhibitor candidates.
We found that our model was able to generate molecules with better druglikeness, synthetic accessibility, water solubility, and hyrdophilicity than current M pro active molecules.
This was given by a > 50% increase in QED, a > 40% increase in solubility, a > 8% improvement in SA, and a > 50% improvement in hydrophilicity.
We found that our RL integration signiﬁcantly improved the types of molecules generated by the untrained agents.
Throughout this work, we demonstrated that by including more parameters into the multi-objective reward function, there is an improvement in generated novel target speciﬁc candidates.
This gives us conﬁdence that our RL framework is effective at producing protein target speciﬁc hit candidates by leveraging the 3D structures of both the generated molecule and the protein pocket, a consideration not made by other molecular generation methods to date.
A CKNOWLEDGEMENTS
This work was supported in part by the U.S. Department of Energy, Ofﬁce of Science, Laboratory
Directed Research Funding (LDRD), Mathematics of Artiﬁcial Reasoning for Science (MARS)
Initiative, at the Paciﬁc Northwest National Laboratory.
Paciﬁc Northwest National Laboratory (PNNL) is a multiprogram national laboratory operated by Battelle for the DOE under Contract DEAC05-76RLO 1830.
This research used computational resources provided by Research Computing at the Paciﬁc Northwest National Laboratory
A P ERFORMANCE M ETRICS A. Q UANTITATIVE E STIMATE OF D RUGLIKENESS
The QED metric represents a quantiﬁcation of the desirability of the drug (Bickerton et al., 2012).
The closer the score is to 1, the more desirable it is as a drug candidate.
The equation for QED is given as:
[Equation]
Where d i is a series of desirability functions (d) belonging to eight widely used molecular descriptors.
These are molecular weight (MW), the octanol-water partition coefﬁcient (ALOGP), the number of hydrogen bond donors (HBD), the number of hydrogen bond acceptors (HBA), the molecular polar surface area (PSA), the number of rotatable bonds (ROTB), the number of aromatic rings (AROM), and the number of structural alerts (ALERTS).
The desirability function can be represented by a general asymmetric double sigmoidal function where d(x) is the desirability function for molecular descriptor x shown as:
[Equation]
where a i , ..., f i can be found in the supplementary table of the original publication (Bickerton et al., 2012).
A. E STIMATING A QUEOUS S OLUBILITY
The metric, water solubility, calculates the log solubility (log S ) of the molecule.
In this work, the solubility is determined by ESOL (Delaney, 2004).
The majority of drugs posses a log S betweenand -2.
The more positive the value, the more water soluble the molecule.
ESOL as deﬁned in Delaney (2004) can be calculate as the multiple linear regression of:
clogP 2.
Molecular weight (MWT) 3.
Rotatable bonds (RB) 4.
Aromatic proportion (AP)
given as:
[Equation]
A.
S YNTHETIC A CCESSIBILITY
SA is one of the most critical metrics to use in determining the simplicity in experimentally synthesizing a molecule.
It is not a score that dictates how effective the molecule is, but rather a practical measure of it’s complexity.
The SA score is between 1 to 10, where 1 indicates an easily synthesizable molecule and 10 indicates a complex one.
The algorithm for calculating the SA score of a molecule (as represented in Ertl & Schuffenhauer (2009)) is given as:
[Equation]
where the fragment score is calculated as a sum of contributions of all fragments in the molecule divided by the number of fragments in this molecule.
The contribution of a fragment is obtained from a database of fragment contributions that were generated by statistical analysis of substructures in the
PubChem collection.
The complexity penalty is a score given to characterize the presence of complex structural features in the molecules.
It is deﬁned as a combination of the following:
[Equation]
A. H YDROPHILICITY VS . L IPOPHILICITY
This metric, known as log P , is the calculated octanol-water partition coefﬁcient of a given molecule.
The values represents if a drug is either very hydrophilic (-3) or very lipophilic (+10).
This speciﬁc metric is present in Lipinski’s rule as value that needs to be less than 5 to be considered a drug candidate.
To calculate the partition function for octanol-water partition coefﬁcient that dictates whether a molecule is more hydrophilic or lipophilic we utilize the RDKit package implementation of the
Crippen approach (Wildman & Crippen, 1999).
It simply calculates the sum of the contributions of each of the atoms in the molecule.
Intramolecular interactions are accounted for my classifying atoms into different types based on their attached a i and neighboring atoms n i :
[Equation]
where P can be further calculated into logP .
A full list of the atomic descriptors and contributions can be found in the main text of Wildman & Crippen (1999).
A. V ALIDITY , U NIQUENESS , AND N OVELTY
To analyze the novelty of our compounds, we need to look at how we calculate the validity and uniqueness of our compounds.
These are given as follows:
[Equation]
B T OP C ANDIDATES
Shown below are snapshots of the top 3 candidates from each of the reward functions presented in this work.
[Figure] [Figure]
C A BLATION S TUDY ON R EWARD F UNCTION
In order to see the importance of optimizing for multiple properties in the reward function, we tested different training weight, and reward function combinations for the model.
The most importantwere listed in the main text of the paper, but we tested 2 more reward functions to test performance.
We used the following combinations and compared how their output scored in the drug-like metrics.
We see a steady decrease in performance from Reward 1 to Reward 3 showing that the inclusion of
Binding Probability is important for generating molecules with a drug-like features.
We also compare our 3D method with the 2D protein inclusion using the original PaccMann Method but with our GNN instead of their PaccMann critic to see if the 2D features have an effect.
[Equation]
[Figure]
Overall, ﬁgure 7 shows that the complete 3 term reward function for the 3D-MolGNN RL method produces the best scoring molecules overall for the given drug-like metrics.
D 3D-S CAFFOLD H YPERPARAMETERS
For the 3D Scaffold model we use the following hyperparameters: batch size of 2, split of 1000training, validation set, 150 max epochs, learning rate of 0.0001, learning rate patience of 10, learning rate minimum of 1e-6, learning rate decay of 0.5, atom-wise representation layer size of 32, cutoff radius of local environment of 10, interaction layer size of 6, 25 gaussians to expand distances with a max distance of 15 and 300 distance bins.
E GNN H YPERPARAMETERS
Hyperparameters such as network depth, layer dimension, and learning rate can have a large effect on model training and the weights in the ﬁnal realized model.
Therefore, we performed a number of trainings to examine combinations of learning rate, number of attention heads, and layer dimension.
We used the parameters from our best model to create a baseline: a learning rate of 0.0001, four attention heads, and a layer dimension of 70 produced an average test AUROC of 0.854.
The test parameters and top scoring conﬁguration are outlined and highlighted in Table 3.
The hyperparameters that performed best were a learning rate of 0.0001, two attention heads, and a dimension of 70.
These parameters resulted in an average test AUROC of 0.864, still scoring lower than our implemented baseline conﬁguration.
[Table]
Trainings were carried out over 200 epochs on a quarter of the data taken from our dataset consisting of 2,000 samples per target with 79 targets, using the same train-test split for all trainings.
The decreased performance compared to the models is expected due to the reduced set of data used to train the model.
Thirty of the 36 combinations trained without error and are shown in Table 4.
[Table]
N 3 D0.
L R 0.001 N 3 D0. L R 0.001 N 3 D0.
L R 0.001 N 2 D0. L R 0.001 N 2 D0.
L R 0.001 N 2 D0. L R 0.001 N 2 D0.
L R 0.0001 N 4 D0.
L R 0.0001 N 3 D0.
L R 0.0001 N 3 D0. L R 0.0001 N 3 D0.
L R 0.0001 N 2 D0. L R 0.0001 N 2 D0.
L R 0.0001 N 2 D0.
L R 0.00001 N 4 D0.
L R 0.00001 N 4 D0. L R 0.00001 N 4 D0.
L R 0.00001 N 3 D0. L R 0.00001 N 3 D0.
L R 0.00001 N 3 D0. L R 0.00001 N 2 D0.
L R 0.00001 N 2 D0. L R 0.00001 N 2 D0.
[Table]
Probability prediction
[Table]
for SA
[Table]
prediction
[Table]
Experimental Afﬁnity prediction
[Table] [Table]
policy
N
[Table]
per episode
P Θ - probability associated with the action t
Z next - ground truth type of the next atom ˆ p Z next type - probability that the model assigns to that type at the current step.
ˆ p bj - probability that the model assigns for the distance between r j and r next to fall into distance bin b ∈ B at the current step
PM
[Table]
GNN
[Table]
model Π(
[Equation]
(cid:80)
[Equation]
(cid:26)
[Equation] [Equation]
[Equation]
3D-MolGNN- RL model Π(Θ)
= (cid:80) s t
[Equation]
S ( P Θ ( s t ) −
[Equation]
( s t ))
P
[Equation]
( s t ) =
P Θ ( s t ) type + P Θ ( s t ) dist
P Θ ( s t )
[Equation]
= − log (cid:16) ˆ p Z next type (cid:17)
P Θ ( s t ) dist = (cid:80) Nj =1 (cid:80)
[Equation]
∈
[Equation]
q bj log (cid:0) ˆ p bj (cid:1)
R 1 ( s t )
=
α.
C BP ( B, C t ) + β.
C EA ( B, C t )
+ (1 − γ.
C SA ( C t ))
R 2 ( s t )
=
α.
C BP ( B, C t )
+ (1 − β.
C SA ( C t ))
R 3 ( s t )
=
α.
C EA ( B, C t ) + (1 − β.
C SA ( C t ))