D IFFERENTIABLE S CAFFOLDING T REE FOR M OLECULAR O PTIMIZATION
Anonymous authors
Paper under double-blind review
A BSTRACT
The structural design of functional molecules, also called molecular optimization, is an essential chemical science and engineering task with important applications, such as drug discovery.
Deep generative models and combinatorial optimization methods achieve initial success but still struggle with directly modeling discrete chemical structures and often heavily rely on brute-force enumeration.
The challenge comes from the discrete and non-differentiable nature of molecule structures.
To address this, we propose differentiable scaffolding tree ( DST ) that utilizes a learned knowledge network to convert discrete chemical structures to locally differentiable ones.
DST enables a gradient-based optimization on a chemical graph structure by back-propagating the derivatives from the target properties through a graph neural network (GNN).
Our empirical studies show the gradient-based molecular optimizations are both effective and sample efﬁcient.
Furthermore, the learned graph parameters can also provide an explanation that helps domain experts understand the model output.
I NTRODUCTION
The structural design of new functional molecules, also called molecular optimization, is the key to many scientiﬁc and engineering challenges, such as ﬁnding energy storage materials (Hachmann et al., 2011; Janet et al., 2020), small molecule pharmaceutics (Kuntz, 1992; Zhavoronkov et al., 2019), and environment-friendly material (Zimmerman et al., 2020).
The objective is to identify novel molecular structures with desirable chemical or physical properties (Gómez-Bombarelli et al., 2018; Dai et al., 2018; Jin et al., 2018;
You et al., 2018; Jin et al., 2019; Shi et al., 2020; Zhou et al., 2019; Jin et al., 2020; Zang & Wang, 2020; Xie et al., 2021).
Recent advances in deep generative models (DGM) allow learning the distribution of molecules and optimizing the latent embedding vectors of molecules.
Models in this category are exempliﬁed by the variational autoencoder (VAE) (Gómez-Bombarelli et al., 2018; Dai et al., 2018; Jin et al., 2018; 2020) and generative adversarial network (GAN) (De Cao & Kipf, 2018).
On the other hand, because of the discrete and not explicitly combinatorial nature of the enormous chemical space, applying combinatorial optimization algorithms with some structure enumeration has been the predominant approach (You et al., 2018; Jensen, 2019;
Zhou et al., 2019;
Nigam et al., 2020; Xie et al., 2021).
Deep learning models have also been used to guide these combinatorial optimization algorithms.
For example, You et al.
; Zhou et al.
; Jin et al.
; Gottipati et al.
(2020) tried to solve the problem with deep reinforcement learning; Nigam et al.
(2020) enhanced a genetic algorithm with a neural network as a discriminator;
Xie et al.
Fu et al.
(2020) approached the problem with Markov Chain Monte Carlo (MCMC) to explore the target distribution guided by graph neural networks.
Despite the initial success of these previous attempts, the following limitations remain: (1) deep generative models optimize the molecular structures in a learned latent space, which requires the latent space to be smooth and discriminative.
Training such models needs carefully designed networks and well-distributed datasets.
(2) most combinatorial optimization algorithms, featured by evolutionary learning methods (Nigam et al., 2020; Jensen, 2019; Xie et al., 2021; Fu et al., 2020), exhibit randomwalk behavior, and leverage trial-and-error strategies to explore the discrete chemical space.
The recent deep reinforcement learning methods (You et al., 2018; Zhou et al., 2019; Jin et al., 2020;
Gottipati et al., 2020) aim to remove random-walk search using a deep neural network to guide the searching.
It is challenging to design the effective reward function into the objective (Jin et al., 2020).
(3) Most existing methods require a great number of oracle calls (a property evaluator; see
Def. 1) to proceed with an efﬁcient search.
However, realistic oracle functions, evaluating with either experiments or high-ﬁdelity computational simulations, are usually expensive.
Examples include using biological assays to determine the potency of drug candidates (Wang et al., 2017), or conducting electronic structure calculation to determine photoelectric properties (Long et al., 2011).
We propose differentiable scaffolding tree ( DST ) to address these challenges, where we deﬁne a differentiable scaffolding tree for molecular structure and utilize a trained GNN to obtain the local derivative that enables continuous optimization.
The main contributions are summarized as:
• We propose the differentiable scaffolding tree to deﬁne a local derivative of a chemical graph.
This concept enables a gradient-based optimization of a discrete graph structure.
• We present a general molecular optimization strategy utilizing the local derivative deﬁned by the differentiable scaffolding tree.
This strategy leverages the property landscape’s geometric structure and suppresses the random-walk behavior, exploring the chemical space more efﬁciently.
We also incorporate a determinantal point process (DPP) based selection strategy to enhance the diversity of generated molecules.
• We demonstrate encouraging preliminary results on de novo molecular optimization with multiple computational objective functions.
The local derivative shows consistency with chemical intuition, providing interpretability of the chemical structure-property relationship.
Our method also requires less oracle calls, maintaining good performance in limited oracle settings.
R ELATED W ORK
Existing molecular optimization methods can mainly be categorized as deep generative models and combinatorial optimization methods.
Deep generative models model a distribution of general molecular structure with a deep network model so that one can generate molecules by sampling from the learned distribution.
Typical algorithms include variational autoencoder (VAE), generative adversarial network (GAN), energybased models, ﬂow-based model (Gómez-Bombarelli et al., 2018; Jin et al., 2018; De Cao & Kipf, 2018;
Segler et al., 2018; Jin et al., 2019; Honda et al., 2019; Madhawa et al., 2019; Shi et al., 2020; Jin et al., 2020; Zang & Wang, 2020; Kotsias et al., 2020; Chen et al., 2021; Liu et al., 2021; Bagal et al., 2021). Shen et al.
(2021) also leverages inverse learning based on SELFIES representation. However, its performance is not satisfactory, primarily due to the failure of training an adequate surrogate oracle.
In addition, DGMs can leverage Bayesian optimization in latent spaces to optimize latent vectors and reconstruct to obtain the optimized molecules (Jin et al., 2018).
However, such approaches usually require a smooth and discriminative latent space and thus an elaborate network architecture design and well-distributed data set.
Also, as they learn the reference data distribution, their ability to explore diverse chemical space is relatively limited, evidenced by the recent molecular optimization benchmarks (Brown et al., 2019; Huang et al., 2021).
Combinatorial optimization methods mainly include deep reinforcement learning (DRL) (You et al., 2018;
Zhou et al., 2019; Jin et al., 2020; Gottipati et al., 2020) and evolutionary learning methods (Nigam et al., 2020; Jensen, 2019; Xie et al., 2021; Fu et al., 2020).
They both formulate molecule optimization as a discrete optimization task.
Speciﬁcally, they modify molecule substructures (or tokens in a string representation (Weininger, 1988)) locally, with an oracle score or a policy/value network to tell if they keep it or not.
Due to the discrete nature of the formulation, most of them conduct an undirected search (random-walk behavior), while some recent ones like reinforcement learning try to guide the searching with a deep neural network, aiming to rid the random-walk nature.
However, it is challenging to incorporate the learning objective target into the guided search.
Those algorithms still require massive numbers of oracle calls, which is computationally inefﬁcient during the inference time (Korovina et al., 2020).
Our method, DST , falls into this category, explicitly leverages the objective function landscape and conducts an efﬁcient goal-oriented search.
Instead of operating on molecular substructure or tokens, we deﬁne the search space as a set of binary and multinomial variables to indicate the existence and identity of nodes respectively, and make it locally differentiable with a learned GNN as a surrogate of the oracle.
This problem formulation can ﬁnd its root in conventional computer-aided molecular design algorithms with branch-and-bound algorithms as solutions (Sinha et al., 1999; Sahinidis & Tawarmalani, 2000).
GNN Predict
Scaffolding Tree
Molecular Graph
Differentiable Scaffolding Tree 3
Molecular Graph
EXPAND
UNCHANGE
Abstract
GradientAscent
Assemble and Diversify
M ETHOD
We ﬁrst introduce the formulation of molecular optimization and differentiable scaffolding tree ( DST ) in Section 3.1, illustrate the pipeline in Figure 1, then describe the key steps following the order:
• Oracle GNN construction: We leverage GNNs to imitate property oracles, which are targets of molecular optimization (Section 3.2).
Oracle GNN is trained once and for all.
The training is separately from optimizing DST below.
• Optimizing differentiable scaffolding tree: We formulate the discrete molecule optimization into a locally differentiable problem with a differentiable scaffolding tree ( DST ).
Then a DST can be optimized by the gradient back-propagated from oracle GNN (Section 3.3).
• Molecule Diversiﬁcation After that, we describe how we design a determinantal point process (DPP) based method to output diverse molecules for iterative learning (Section 3.4).
3. P ROBLEM F ORMULATION AND N OTATIONS 3.1.
Molecular optimization problem Oracles are the objective functions for molecular optimization problems, e.g., QED quantifying a molecule’s drug-likeness (Bickerton et al., 2012).
Deﬁnition 1 (Oracle O ) .
Oracle O is a black-box function that evaluates certain chemical or biological properties of a molecule X and returns the ground truth property
O ( X ) .
In realistic discovery settings, the oracle acquisition cost is usually not negligible.
Suppose we want to optimize P molecular properties speciﬁed by oracle O 1 , · · · , O P , we can formulate a multi-objective molecule optimization problem through scalarization as represented in Eq.
[Equation]
where X is a molecule, Q denotes the set of valid molecules; f is the composite objective combining all the oracle scores, e.g., the mean value of P oracle scores.
3.1.2 Scaffolding Tree
The basic mathematical description of a molecule is molecular graph, which contains atoms as nodes and chemical bonds as edges.
However, molecular graphs are not easy to generate explicitly as graphs due to the presence of rings, relatively large size, and chemical validity constraints.
For ease of computation, we convert a molecular graph to a scaffolding tree as a higher-level representation, a tree of substructures, following Jin et al.
Deﬁnition 2 ( Substructure ) .
Substructures can be either an atom or a single ring.
The substructure set is denoted S (vocabulary set), which covers frequent atoms and single rings in drug-like molecules.
Deﬁnition 3 ( Scaffolding Tree T ) .
A scaffolding tree, T X , is a spanning tree whose nodes are substructures.
It is higher-level representation of molecular graph X .
T X is represented by (i) node indicator matrix, (ii) adjacency matrix, and (iii) node weight vector.
We distinguish leaf and non-leaf nodes in T X .
Among the K 1 nodes in T X , there are K leaf leaf nodes (nodes connecting to only one edge) and K − K leaf non-leaf nodes (otherwise).
The sets of leaf nodes and non-leaf nodes are denoted V leaf and V nonleaf correspondingly.
[Figure]
[Figure]
(cid:101) w and (cid:101)
A share the learnable parameters { (cid:98) w 3 , (cid:98) w 4 , (cid:98) w 5 | 3 , (cid:98) w 6 | 4 , (cid:98) w 7 | 1 , (cid:98) w 8 | 2 } .
Deﬁnition 4.
Node indicator matrix N is decomposed as N = (cid:18) N nonleaf
N leaf (cid:19) ∈ { 0 , 1 } K ×|S| , where
N nonleaf ∈ { 0 , 1 } ( K − K leaf )
×|S| corresponds to non-leaf nodes while N leaf ∈ { 0 , 1 } K leaf ×|S| corresponds to leaf nodes.
Each row of N is a one-hot vector, indicating which substructure the node belongs to.
Deﬁnition 5.
Adjacency matrix is denoted A ∈ { 0 , 1 } K × K .
A ij = 1 indicates the i -th node and j -th node are connected while 0 indicates unconnected.
Deﬁnition 6.
Node weight vector , w = [1 , · · · , 1] (cid:62) ∈ R K , indicates the K nodes in scaffolding tree are equally weighted.
3.1.3 Differentiable scaffolding tree Similar to a scaffolding tree, a differentiable scaffolding tree (DST) also contains (i) node indicator matrix, (ii) adjacency matrix, and (iii) node weight vector, but with additional expansion nodes.
Speciﬁcally, while inheriting leaf node set V leaf and non-leaf node set V nonleaf from the original scaffolding tree, we add expansion nodes and form expansion node set ,
V expand =
{ u v | v ∈ V leaf ∪ V nonleaf } , |V expand | =
K expand =
K , where u v is connected to v in the original scaffolding tree.
We also deﬁne differentiable edge set , Λ = { ( v, v (cid:48) )
| v ∈ V leaf OR v (cid:48) ∈
V expand ; v, v (cid:48) are connected } to incorporate all the edges involving leaf-nonleaf node and leaf/nonleafexpansion node connections.
To make it locally differentiable, we modify the tree parameters from two aspects: (A) node identity and (B) node existence .
Figure 2 shows an example to illustrate DST.
(A) We enable optimization on node identity by allowing the corresponding node indicator matrix learnable:
Deﬁnition 7.
Differentiable node indicator matrix (cid:101) N takes the form:
[Equation]
(cid:101) N nonleaf =
N nonleaf ∈ { 0 , 1 } ( K − K
[Equation]
) ×|S| are ﬁxed, equal to the part in the original scaffolding tree, each row is a one-hot vector, indicating that we ﬁx all the non-leaf nodes.
In contrast, both (cid:101) N expand and (cid:101) N leaf are learnable, we use softmax activation to implicitly encode the constraint (cid:80) j (cid:101) N ij =i.e., (cid:101) N ij =
[Equation]
j
[Equation]
N
[Equation]
j
[Equation]
) , (cid:98) N are the parameters to learn.
This constraint guarantee that each row of (cid:101)
N is a valid substructures’ distribution.
(B) We enable optimization on node
existence
by assigning
learnable weights for
the leaf
and
expansion
nodes
, construct adjacency
matrix and node weight vector
based on those values:
Deﬁnition 8.
Differentiable adjacency matrix (cid:101)
A ∈ R
( K + K expand ) ×
( K + K expand ) takes the form:
[Equation] [Equation]
[Equation]
where Λ is the differentiable edge set deﬁned above, Sigmoid function σ ( · ) imposes the constraint 0 ≤ (cid:101)
A ij ≤ 1 .
(cid:98) w ∈ R K leaf
+ K expand are the parameters, each leaf node and expansion node has one learnable parameter.
For connected i and j , when i ∈ V leaf , j ∈ V nonleaf , (cid:101)
A ij = σ ( (cid:98) w i ) measures the existence probability of leaf node i ; when i ∈ V expand , j ∈ V leaf ∪V nonleaf , (cid:101)
A ij = σ ( (cid:98) w i | j ) measures the conditional probability of the existence of expand node i given the original node j .
When j is a leaf node, it naturally embeds the inheritance relationship between the leaf node and the corresponding expansion node.
Deﬁnition 9.
Differentiable node weight vector (cid:101) w ∈ R K
+ K expand takes the form:
[Equation]
where all the weights range from 0 to 1.
The weight of expansion node connecting to leaf node relies on the weight of corresponding leaf node.
(cid:101) w and (cid:101)
A (Def.
8) shares the learnable parameter (cid:98) w .
Figure 2 shows an example to illustrate DST.
3. T RAINING O RACLE G RAPH N EURAL N ETWORK
This section constructs a differentiable surrogate model to capture the knowledge from any oracle function.
We choose graph neural network architecture for its state-of-the-art performance in modeling structure-property relationships.
In particular, we imitate the objective function F with GNN:
[Equation]
where Θ represents the GNN’s parameters.
Concretely, we use a graph convolutional network (GCN) (Kipf & Welling, 2016).
Other GNN variants, such as Graph Attention Network (GAT) (Veliˇckovi´c et al., 2017), Graph Isomorphism Network (GIN) (Xu et al., 2018), can also be used in our setting.
The initial node embeddings H (0)
=
NE ∈ R K × d stacks basic embeddings of all the nodes in the scaffolding tree, d is the GCN hidden dimension, N is the node indicator matrix (Def. 4).
E ∈ R |S|×
d is the embedding matrix of all the substructures in vocabulary set S , and is randomly initialized.
The updating rule of GCN for the l -th layer is
[Equation]
where L is GCN’s depth, A is the adjacency matrix (Def. 5),
H ( l ) ∈ R K
×
d is the nodes’ embedding of layer l , B ( l ) =
[ b ( l ) , b ( l ) , · · · , b ( l ) ] (cid:62) ∈
R K × d and U
( l ) ∈ R
d × d are bias and weight parameters of layer l , respectively.
We generalize the GNN from a discrete scaffolding tree to a differentiable one.
Based on learnable weights for each node, we leverage the weighted average as the readout function of the last layer’s
( L -th) node embeddings, followed by multilayer perceptron (MLP) to yield the prediction (cid:98) y , i.e., (cid:98) y
= MLP (cid:0)
[Equation]
Kk =
[Equation]
k (cid:80)
Kk
=
w k H ( L ) k (cid:1) , in discrete scaffolding tree, weights for all the nodes are equal to 1, H ( L ) k is the k -th row of H ( L ) .
In sum, the prediction can be written as
[Equation]
where Θ = { E } ∪ { B ( l ) , U ( l ) } Ll =1 are the GNN’s parameters.
We train the GNN by minimizing the discrepancy between GNN prediction (cid:98) y and the ground truth y .
[Equation]
where L is loss function, e.g., mean squared error; D is the training set.
After training, we have GNN parameterized by Θ ∗ to approximate the black-box objective function F (Eq. 1).
Worth to mention that Oracle GNN is trained once and for all.
The training is separately from optimizing DST below.
3. O PTIMIZING D IFFERENTIABLE S CAFFOLDING T REE
Overview With a little abuse of notations, via introducing DST , we approximate molecule optimization as a locally differentiable problem
[Equation]
[Figure]
[Equation]
where X ( t ) is the molecule at t -th iteration, N ( X ( t ) )
⊆
Q is the neighborhood set of X ( t )
(Def. 10).
Next, we explain the intuition behind these approximation steps.
Molecular optimization is generally a discrete optimization task, which is prohibitively expensive due to exhaustive search.
The ﬁrst approximation is to formulate the problem as an iterative local discrete search via introducing a neighborhood molecule set N ( X ( t ) ) .
Second, to enable differentiable learning, we use GNN to imitate black-box objective F (Section 3.2) and further reformulated it into a local differentiable optimization problem.
Then we can optimize DST ( T X = {
[Equation]
N X
[Equation] [Equation]
A X ( t ) , (cid:101) w X ( t ) } ) in a continuous domain for N ( X ( t ) ) using gradient-based optimization method.
3.3.1 Local Editing Operations For
a leaf node v in the scaffolding tree, we can perform three editing operations, (1) SHRINK : delete node v ; (2) REPLACE : replace a new substructure over v ; (3) EXPAND : add a new node u v that connects to node v .
For a nonleaf node v , we support (1)
EXPAND : add a new node u v connecting to v ; (2) do nothing .
If we EXPAND and REPLACE, the new substructures are sampled from the vocabulary S .
We deﬁne a molecule neighborhood set as below:
Deﬁnition 10 (Neighborhood set) .
Neighborhood set of molecule X , denoted N ( X ) , is the set of all the possible molecules obtained by imposing one local editing operation to scaffolding tree T X and assembling the edited trees into molecules.
3.3.2 Optimizing DST Then within the domain of neighborhood molecule set N ( X ) , the objective function can be represented as a differentiable function of X’s DST ( (cid:101) N X , (cid:101)
A X , (cid:101) w X ).
We address the following optimization problem to get the best scaffolding tree within N ( X ) ,
[Equation]
where the GNN parameters Θ ∗ (Eq.
(8)) are ﬁxed.
Comparing with Eq.
(7), it is differentiable with regard to { (cid:101) N , (cid:101)
A , (cid:101) w } for all molecules in the neighborhood set N ( X ) .
Therefore, we can optimize the DST using gradient-based optimization method, e.g., an Adam optimizer (Kingma & Ba, 2014).
3.3.3 Sampling from DST
Then we sample the new scaffolding tree from the optimized DST .
Concretely, (i) for each leaf node v ∈ V leaf and the corresponding expansion node u v ∈ V expand , we select one of the following step with probabilities (w.p.) as follows,
[Equation]
Sampler
[Equation]
=  
SHRINK: delete leaf node v,
[Equation]
EXPAND: add u v , select substructure at u v based on ( (cid:101) N ∗ ) u v , w.p. ( (cid:101) w ∗ ) v ( (cid:101) w ∗ )
u v | v , 3.
REPLACE: select substructure at v based on ( (cid:101) N ∗ ) u ,
[Equation]
(ii)
For each nonleaf node v , we expand a new node with probability ( (cid:101) w ∗ )
u v | v .
If
expanding we
select
substructure
at u v based on ( (cid:101) N ∗ ) u
v .
Otherwise, we
“ do nothing ”
3.3.4 Assemble Each scaffolding tree corresponds to multiple molecules due to the multiple ways substructures can be combined.
We enumerate all the possible molecules following Jin et al.
(See Section C.5 in Appendix for more details) for the further selection as described below.
3. M OLECULE D IVERSIFICATION
In the current iteration, we have generated M molecules ( X 1 , · · · , X M ) and need to select C molecules for the next iteration.
We expect these molecules to have desirable chemical properties
(high F score) and simultaneously maintain higher structural diversity.
To do so, we resort to the determinantal point process (DPP) (Kulesza & Taskar, 2012).
DPP models the repulsive correlation between data points (Kulesza & Taskar, 2012) and has been successfully applied to many applications such as text summarization (Cho et al., 2019), mini-batch sampling (Zhang et al., 2017), and recommendation system (Chen et al., 2018).
For M data points, whose indexes are { 1 , 2 , · · · , M } ,
S ∈ R M × M
+ denotes the similarity kernel matrix between these data points.
To create a diverse subset (denoted R ) with ﬁxed size C , the sampling probability should be proportional to the determinant of the submatrix S R ∈ R C × C , i.e., P ( R ) ∝ det( S R ) , where R ⊆ { 1 , 2 , · · · , M } , |R| = C .
Combining the objective ( F ) value and diversity, the composite objective is
[Equation]
where the hyperparamter λ > 0 balances the two terms, the diagonal scoring matrix V = diag (cid:16)(cid:2)
exp( λF ( X 1 )) , · · · , exp( λF ( X M )) (cid:3)(cid:17) ,
V R ∈ R C
× C is a sub-matrix of V indexed by R .
When λ goes to inﬁnity, it is equivalent to selecting C candidates with the highest F score regardless of diversity, same as conventional evolutionary learning in (Jensen, 2019; Nigam et al., 2020).
Inspired by generalized DPP methods (Kulesza & Taskar, 2012; Chen et al., 2018), we further transform
L DPP ( R ) ,
[Equation]
where V 12 S V 12 is symmetric positive semi-deﬁnite.
Then it can be solved by generalized DPP methods in O ( C 2 M )
(Chen et al., 2018)
(Section F in Appendix).
The computational complexity of
DST is O
( TMC 2 ) (Section C.8 in Appendix).
Algorithm 1 summarizes the entire algorithm.
scaffolding j .
10: Γ =
Γ ∪ { X ( t +1) j } .
11: end for 12:
Select Φ ⊆ Γ , | Φ | =
C based on Eq.
12; Ω =
Ω ∪ Φ .
# Section 3.
13: end for
E XPERIMENT
In this section, we ﬁrst describe the experimental setup in section 4.1 following Jin et al.
(2018; 2019); You et al.
; Shi et al.
(2020); Jin et al.
Xie et al.
Then we analyze the result from three aspects: optimization performance, oracle efﬁciency, and interpretability.
4. E XPERIMENTAL S ETUP
Molecular Properties contains (1) QED (drug-likeness);
(2) LogP (octanol-water partition coefﬁcient); (3) SA (synthetic accessibility); (4) JNK3 (c-Jun N-terminal Kinases-3); (5) GSK3 β (Glycogen synthase kinase 3 beta)
; For all the 5 scores (including normalized SA), higher is better.
We conducted (1) single-objective generation that optimizes JNK3, GSK3 β and LogP sepa- rately and (2) multi- generation that optimizes the mean value of “JNK3+ β ” and “QED+SA+JNK3+GSK3 β ” in the main text.
More details are given in Section C.3 in Appendix.
Dataset : ZINC 250K contains around 250K druglike molecules (Sterling & Irwin, 2015).
We select the substructures that appear more than 1000 times in ZINC 250K as the vocabulary set S , which contains 82 most frequent substructures.
More details are in Section C.1 in Appendix.
Baselines .
(1) LigGPT (string-based distribution learning model with Transformer as a decoder) (Bagal et al., 2021); (2) GCPN (Graph Convolutional Policy Network)
(You et al., 2018); (3) MolDQN (Molecule Deep Q-Network)
(Zhou et al., 2019); (4) GA+D
(Genetic Algorithm with Discriminator network) (Nigam et al., 2020); (5) MARS (Markov Molecular Sampling)
(Xie et al., 2021); (6)
RationaleRL (Jin et al., 2020); Among them, LigGPT belongs to deep generative model, where all the oracle calls can be precomputed; GCPN and MolDQN are deep reinforcement learning methods; GA+D and MARS are evolutionary learning methods; RationaleRL is deep generative model ﬁne-tuned with RL techniques.
All the baselines except LigGPT require online oracle calls.
Metrics .
(1) Novelty ; (2) Diversity ; (3) SR (Success Rate); (4) # of oracle calls : DST needs to call oracle in labeling data for GNN ( precomputed ) and DST based de novo generation ( online ), we show the costs for both steps.
Details are provided in Section C.4.
de novo design.
#oracle =
[Table]
call” (to label molecules in existing database)
+
[Table]
call” (during learning).
[Table]
4. O PTIMIZATION P ERFORMANCE
The results of multi-objective and single-objective generation are shown in Table 1 and 2.
We ﬁnd that DGM (LigGPT) and RL based methods (GCPN and MolDQN) fails in some tasks, which is consistent with the results reported in RationaleRL (Jin et al., 2020) and MARS (Xie et al., 2021).
Overall
[Table]
DST obtains the best results in most tasks.
In terms of success rate and diversity, DST outperformed all baselines in most tasks.
It also reached the highest scores within T = 50 iterations in most optimization tasks (see Table 5 and 6 in Appendix).
Especially in optimizing LogP, the model successfully learned to add a six-member ring (see Figure 8 in Appendix) each step, which is theoretically the optimal strategy under our setting.
Combined with the ablation study comparing with random selection (see Figure 11 in Appendix), our results show the local gradient deﬁned by DST is a useful direction indicator, consistent with the concept of gradient.
Further, achieving high diversity validates the effect of the DPP-based selection strategy.
Although the novelty is not the highest, it is still comparable to baseline methods.
These results show our gradient-based optimization strategy has a strong optimization ability to provide a diverse set of molecules with high objective functions
[Table] [Table]
4. O RACLE E FFICIENCY
As mentioned above, oracle calls for realistic optimization tasks can be time-consuming and expensive.
From Table 1 and 2, we can see that majority of de novo optimization methods require oracle calls online (instead of precomputation), including all of RL/evolutionary algorithm based baselines.
DST takes fewer oracle calls compared with baselines.
DST can leverage the precomputed oracle calls to label the molecules in an existing database (i.e., ZINC) for training the oracle GNN and dramatically saving the oracle calls during reference.
In the three tasks in Table 2, two-thirds of the oracle calls (10K) can be precomputed or collected from other sources.
To further verify the oracle efﬁciency, we explore a special setting of molecule optimization where the budget of oracle calls is limited to a ﬁxed number (2K, 5K, 10K, 20K, 50K) and compare the optimization performance.
For GCPN,
MolDQN, GA+D and MARS, the learning iteration number depends on the budget of oracle calls.
RationaleRL (Jin et al., 2020) is not included because it requires intensive oracle calls to collect enough reference data, exceeding the oracle budget in this scenario.
In DST , we use around 80% budget to label the dataset (i.e., training GNN) while the remaining budget to conduct de novo design.
Speciﬁcally, for 2K, 5K, 10K, 20K, 50K, we use 1.5K, 4K, 8K, 16K and 40K oracle calls to label the data for learning GNN, respectively.
We show the average objective values of top-100 molecules under different oracle budgets in Figure 3.
Our method shows a signiﬁcant advantage compared to all the baseline methods in all limited budget settings.
We conclude the reason as supervised learning is a well-studied and much easier task than generative modeling.
[Figure] [Figure]
To obtain more insights from the local gradient, We visualize two modiﬁcation steps in Figure 4.
The node weights and their gradient values interpret the property at the substructure level.
A positive gradient on expanding node means a positive contribution to the objective value increment, while a negative gradient represents negative one (at least locally).
This is similar to most attribution-based interpretable machine learning methods, e.g., saliency map (Simonyan et al., 2013).
C ONCLUSION
This paper proposed differentiable scaffolding tree ( DST ) to make a molecular graph locally differentiable, allowing a continuous gradient-based optimization.
To the best of our knowledge, it is the ﬁrst attempt to make the molecular optimization problem differentiable at the substructure level, rather than resorting to latent spaces or using RL/evolutionary algorithms.
We constructed a general molecular optimization strategy based on DST , corroborated by thorough empirical studies.
R EPRODUCIBILITY S TATEMENT
The code repository is given in the supplementary material, including README ﬁle, all the codes for data preprocessing and model.
All the data we use are publicly available.
Section B describes experimental setup for all the baseline methods.
We elaborate the implementation details in Section C.
Concretely, Section C.1 describes the dataset we use; Section C.2 describes software and hardware conﬁguration; Section C.3 introduces the target molecular properties to optimize; Section C. describes all the metrics for evaluating the performance; Section C.6 describes the setup for model implementation, especially hyperparameter setup.