Target-aware Molecular Graph Generation
Abstract
Generating molecules with desired biological activities has attracted growing attention in drug discovery.
Previous molecular generation models are designed as chemocentric methods that hardly consider the drug-target interaction, limiting their practical applications.
In this paper, we aim to generate molecular drugs in a targetaware manner that bridges biological activity and molecular design.
To solve this problem, we compile a benchmark dataset from several publicly available datasets and build baselines in a uniﬁed framework.
Building on the recent advantages of ﬂow-based molecular generation models, we propose SiamFlow , which forces the ﬂow to ﬁt the distribution of target sequence embeddings in latent space.
Speciﬁcally, we employ an alignment loss and a uniform loss to bring target sequence embeddings and drug graph embeddings into agreements while avoiding collapse.
Furthermore, we formulate the alignment into a one-tomany problem by learning spaces of target sequence embeddings.
Experiments quantitatively show that our proposed method learns meaningful representations in the latent space toward the target-aware molecular graph generation and provides an alternative approach to bridge biology and chemistry in drug discovery.
1. Introduction
Drug discovery, which focuses on ﬁnding candidate molecules with desirable properties for therapeutic applications, is a long-period, expensively process with a high failure rate.
The challenge primarily stems from the actuality that only a tiny fraction of the theoretical possible drug-like molecules may have practical effects.
Speciﬁcally, the entire search space is as large as 10 23 ∼ 10 60 , while only 10 8 of them are therapeutically relevant (Polishchuk et al., 2013; Hert et al., 2009; Cheng et al., 2021).
In the face of such difﬁculty, traditional methods like high-throughput screening (Hert et al., 2009) fail in terms of efﬁciency because of the large amount of resources required in producing minor hit compounds.
One alternative is using computational methods (Phatak et al., 2009; Paricharak et al., 2018) such as virtual screening (Bajorath, 2002; Schneider, 2010) to identify hit compounds from virtual libraries through similarity-based searches or molecular docking.
Another alternative is automated de novo molecule design, such as inverse QSAR (Schneider & Schneider, 2016), particle swarm optimization (Winter et al., 2019), structure-based de novo design (Schneider, 2013; Button et al., 2019), or genetic algorithms (Bandholtz et al., 2012).
Recent deep generative models have demonstrated potentials to promote drug discovery by exploring huge chemical space in a data-driven manner.
Various forms of variational autoencoder (VAE) (Kingma & Welling, 2014; G´omezBombarelli et al., 2018; Kusner et al., 2017; Dai et al., 2018;
Jin et al., 2018; Simonovsky & Komodakis, 2018), generative adversarial networks (GAN) (Goodfellow et al., 2014;
Guimaraes et al., 2017; Sanchez-Lengeling et al., 2017;
Prykhodko et al., 2019; M ´ endez-Lucio et al., 2020), autoregressive (AR) (Van Oord et al., 2016; Popova et al., 2019;
You et al., 2018), and normalizing ﬂow (NF) (Dinh et al., 2015; 2017; Madhawa et al., 2019; Honda et al., 2019; Shi et al., 2019; Zang & Wang, 2020; Luo et al., 2021) have been proposed to generate molecular SMILES or graphs.
Though these approaches can generate valid and novel molecules to some extent, they remain inefﬁcient because the generated candidate molecules need further screened against given targets.
As the primary goal of these chemocentric methods is to generate drug-like molecules that satisfy speciﬁc properties, directly applying them in drug discovery requires extra efforts on predicting the binding afﬁnities between candidate molecules and target proteins.
While previous molecular generation methods scarcely take biological drug-target interactions into account, we aim to generate candidate molecules based on a biological perspective.
This paper proposes target-aware molecular generation to bridge biological activity and chemical molecular design that generate valid molecules conditioned on speciﬁc targets and thus facilitate the development of drug discovery.
[Figure]
Candidate molecules
Screening
Toxicity / Retro-synthesis feasibility / …
Generative model
Candidate molecules
Screening Target Toxicity / [Figure]
synthesis feasibility
[Figure]
Drug
[Figure] TargetInteraction Target Wet laboratory
Wet
[Figure]
As shown in Figure 1, the pipeline of computational drug discovery is supposed to be simpliﬁed to a great extent with the help of target-aware molecular generation.
Our main contributions are summarized as follows:
• We propose a target-aware molecular generation manner from a biological perspective, while prior works on chemocentric molecular generation are inefﬁcient in practical drug discovery.
• We establish a new benchmark for the target-aware molecular generation containing abundant drug-target pairs for evaluating generative models.
• We propose SiamFlow, a siamese network architecture for the conditional generation of ﬂow-based models.
While the sequence encoder and the generative ﬂow align in the latent space, a uniformity regularization is imposed to avoid collapse.
Moreover, we implement
SiamFlow and baselines in a uniﬁed framework for further research.
The code will be released.
2. Related work 2.1. De Novo Molecular Generation
Recent years have witnessed revolutionary advances in de novo molecular generation, which aims to generate molecules with desired properties from scratch.
Artiﬁcial intelligence, which has been widely applied in this ﬁeld, not only signiﬁcantly reduces the chemical search space but also lowers time consumption to a large extent.
VAE-based VAE has been attractive in molecular generation in the virtue of its latent space is potentially operatable.
CharVAE (G´omez-Bombarelli et al., 2018)
ﬁrst proposes to learn from molecular data in a data-driven manner and generate with a VAE model.
By jointly training the task of predicting molecular properties from the hidden layers, this method explores the molecular space without manual prior knowledge.
GVAE (Kusner et al., 2017) represents each data as a parse tree from a context-free grammar, and directly encodes to and decodes from these parse trees to ensure the validity of generated molecules.
Inspired by syntax-directed translation in complier theory, SD-VAE (Dai et al., 2018) proposes to convert the ofﬂine syntax-directed translation check into on-the-ﬂy generated guidance for ensuring both syntactical and semantical correctness.
JT-VAE (Jin et al., 2018)
ﬁrst realize the direct generation of molecular graphs instead of linear SMILES (Simpliﬁed Molecular-Input LineEntry System) strings.
HierVAE (Jin et al., 2020) proposes a motif-based hierarchical encoder-decoder toward the large molecular graph generation.
GAN-based An alternative is to implement GAN in molecular generation.
ORGAN (Guimaraes et al., 2017) adds expert-based rewards under the framework of
WGAN (Arjovsky et al., 2017).
ORGANIC (SanchezLengeling et al., 2017) improves the above work for inverse design chemistry and implements the molecular generation towards speciﬁc properties.
MolGAN (De Cao & Kipf, 2018) proposes GAN-based models to generate molecular graphs rather than SMILES.
Motivated by cycle-consistent
GAN (Zhu et al., 2017), Mol-CycleGAN (Maziarka et al., 2020) generates optimized compounds with high structural similarity to the original ones.
Flow-based Molecular generation with the normalizing ﬂow is promising as its invertible mapping can reconstruct the data exactly.
GraphNVP (Madhawa et al., 2019) deﬁnes a normalizing ﬂow from a base distribution to the molecular graph structures.
GRF (Honda et al., 2019) further explores invertible mappings by residual ﬂows, and presents a way of keeping the entire ﬂows invertible throughout the training and sampling process in molecular generation.
GraphAF
(Shi et al., 2019) combines the advantages of both autoregressive and ﬂow-based approaches to iteratively generate molecules.
MolFlow (Zang & Wang, 2020) proposes a variant of Glow (Kingma & Dhariwal, 2018) to generate atoms and bonds in a one-shot manner.
MolGrow (Kuznetsov & Polykovskiy, 2021) constrains optimization of properties by using latent variables of the model, and recursively splits nodes to generate molecular structures.
Though these approaches have achieved signiﬁcant performance, we recognize them as chemocentric molecular generation methods that lack biological connections.
We aim to bridge biological and chemical perspectives in molecular generation for practical drug discovery.
2.2. Drug-target Interaction
Drug-target interaction (DTI) has been extensively developed over the decades.
Computational virtual screening methods like molecular docking (Trott & Olson, 2010) and molecular dynamics simulations (Salsbury Jr, 2010) have provided mechanistic insights in this ﬁeld.
Though these methods have inherently great interpretability, they suffer from heavy dependence on the available three-dimensional structure data with massive computational resources.
Recent progress in artiﬁcial intelligence has inspired researchers to utilize deep learning techniques in drug-target interaction prediction.
DeepDTA ( ¨Ozt¨urk et al., 2018) and
DeepAfﬁnity (Karimi et al., 2019) are representatives of deep-learning-based methods that take SMILES of drugs and primary sequences of proteins as input, from which neural networks are employed to predict afﬁnities.
InterpretableDTIP (Gao et al., 2018) predicts DTI directly from low-level representations and provides biological interpretation using a two-way attention mechanism.
DeepRelations (Karimi et al., 2020) embeds protein sequences by hierarchical recurrent neural network and drug graphs by graph neural networks with joint attention between protein residues and compound atoms.
DEEPScreen (Rifaioglu et al., 2020) learns features from structural representations for a large-scale DTI prediction.
MONN (Li et al., 2020) predicts both pairwise non-covalent interactions and binding afﬁnities between drugs and targets with extra supervision from the labels extracted from available high-quality threedimensional structures.
Our proposed target-aware molecular generation builds on the recent advances in data-driven drug-target interaction prediction.
We connect chemical molecular generation with biological drug-target interaction to promote the efﬁciency of computational drug discovery.
2.3. Conditional Molecular Generation
Generating molecules with the consideration of some external conditions is a promising ﬁeld.
There is plenty of work that conditioning on desired properties based on chemical perspective.
CVAE (G´omez-Bombarelli et al., 2018) jointly trains VAE with a predictor that predicts properties from the latent representations of VAE.
(Lim et al., 2018) proposes applying conditional VAE to generate drug-like molecules satisfying ﬁve properties at the same time: MW (molecular weight), LogP (partition coefﬁcient), HBD (number of hydrogen bond donor), HBA (number of hydrogen acceptor), and TPSA (topological polar surface area).
(Grifﬁths & Hern´andez-Lobato, 2020) employs constrained Bayesian optimization to control the latent space of VAE in order to ﬁnd molecules that score highly under a speciﬁed objective function which is a weighted function of LogP and QED (quantitative estimate of drug-likeness).
CogMol (Chenthamarakshan et al., 2020) and CLaSS (Das et al., 2021) pretrain the latent space with SMILES and train property classiﬁers from the latent representations.
They sample variables from the latent space that satisfy high scores from property classiﬁers to generate molecules.
Though recent molecular generation methods (Jin et al., 2018; Madhawa et al., 2019; Zang & Wang, 2020; Luo et al., 2021) also present property optimization experiments, they still, like the above methods, barely take account of drugtarget interaction.
(M´endez-Lucio et al., 2020) proposes stacks of conditional GAN to generate hit-like molecules from gene expression signature.
While this work focuses on drug-gene relationships, we recognize that drug-protein is a more typical case.
3. Background and Preliminaries 3.1. Problem Statement
Let T =
{ T i } ti =1 be a set of targets, and there exists a set of drugs M T i =
{ M ( T i ) j
d
i j =1 that bind to each target T i .
S ( T, M ) is deﬁned as a function measuring the interaction between target T and drug M .
The target-aware molecular generation aims to learning a generation model p θ ( ·| T i ) from each drug-target pair ( M ( T i )
j , T i )
so as to maximize E
M | T i ∼ p θ [ S ( M, T i )] .
3.2. The Flow Framework
A ﬂow model is a sequence of parametric invertible mapping f Θ = f Q ◦ ... ◦
f 1 from the data point x ∈ R D to the latent variable z ∈ R D , where x ∼ P X ( x ) , z ∼ P Z ( z ) .
The latent distribution P Z is usually predeﬁned as a simple distribution, e.g., a normal distribution.
The complex data in the original space is modelled by using the change-of-variable formula:
[Equation]
and its log-likelihood:
[Equation]
where z ( q ) =
f q ( z ( q − 1) ) , and we represent the input z (0) by using z for notation simplicity.
As the calculation of the Jacobian determinant for f Θ is expensive for arbitrary functions, NICE (Dinh et al., 2015) and RealNVP (Dinh et al., 2017) develop an afﬁne coupling transformation z
= f Θ ( x ) with expressive structures and efﬁcient computation of the Jacobian determinant.
For given D -dimensional input x and d < D , the output y of an afﬁne coupling transformation is deﬁned as:
[Equation]
where S Θ : R d → R D − d and T Θ : R d → R D − d stand for scale function and transformation function.
For the sake of the numerical stability of cascading multiple ﬂow layers, we follow Moﬂow (Zang & Wang, 2020) to replace the exponential function for the S Θ with the Sigmoid function:
[Equation]
and the invertibility is guaranteed by:
[Equation]
The logarithmic Jacobian determinant is:
[Equation]
To further improve the invertible mapping with more expressive structures and high numerical stability, Glow (Kingma & Dhariwal, 2018) proposes using invertible 1 × 1 convolution to learn an optimal partition and actnorm layer to normalize dimensions in each channel over a batch by an afﬁne transformation.
Invertible 1 × 1 convolution is initialized as a random rotation matrix with zero log-determinant and works as a generalization of a permutation of channels.
Act norm initializes the scale and the bias such that the postactnorm activations per-channel have zero mean and unit variance and learns these parameters in training instead of using batch statistics as batch normalization does.
3.3. Flow on the Molecular Graph
Prior works on ﬂow-based molecular graph generation are well developed.
Inspired by the graph normalizing ﬂows of
GRevNets (Liu et al., 2019a), GraphNVP (Madhawa et al., 2019) proposes to generate atom features conditioned on the pre-generated adjacency tensors, which is then followed by other one-shot ﬂow-based molecular graph generation approaches, e.g., GRF (Honda et al., 2019) and Moﬂow (Zang & Wang, 2020).
Our proposed SiamFlow follows this manner, that is, ﬁrstly transforms the bonds B of molecules to the latent variables Z B with Glow (Kingma & Dhariwal, 2018), and then transforms the atom features A given
B into the conditional latent variable Z A | B with a graph conditional ﬂow.
Let N, K, C be the number of nodes, node types, and edge types, respectively.
A molecular graph G =
( A, B ) is deﬁned by an atom matrix A ∈ { 0 , 1 } N × K and a bond tensor B ∈ { 0 , 1 } C × N × N , which correspond to nodes and edges in the vanilla graph.
A [ i, k ] = 1 represents the i -th atom i has atom type k , and B [ c, i, j ] = 1 represents there is a bond with type c between the i -th atom and j -th atom.
Flow-based molecular graph generation methods decompose the generative model into two parts:
[Equation]
where θ B is learned by the bond ﬂow model h B , and θ A | B is learned by the atom ﬂow model h
A | B conditioned on the bond tensor B .
With the strengths of the ﬂow, the optimal parameters θ ∗ A | B and θ ∗ B maximize the exact likelihood estimation:
[Equation]
Our work follows the one-shot molecular graph generation manner (Madhawa et al., 2019; Honda et al., 2019; Zang & Wang, 2020) that employs Glow (Kingma & Dhariwal, 2018) as the bond ﬂow model h B and graph conditional ﬂow as the atom
ﬂow model h
A | B .
4. SiamFlow 4.1. Overview
While current ﬂow-based molecular graph generation methods (Madhawa et al., 2019; Honda et al., 2019; Shi et al., 2019; Zang & Wang, 2020; Kuznetsov & Polykovskiy, 2021;
Lippe & Gavves, 2020;
Luo et al., 2021) learn from druglike datasets and generate without the invention of targets, our proposed SiamFlow aims to serve as a conditional ﬂow toward molecular graph generation.
Though the conditional ﬂow has been well developed in computer vision (Liu et al., 2019b; Kondo et al., 2019;
Abdelhamed et al., 2019; Kumar et al., 2019; Sun et al., 2019;
Pumarola et al., 2020; Liang et al., 2021;
Yang et al., 2021), there are limited works that can ﬁt graph generation, especially when it comes to the molecular graph.
In this section, we introduce SiamFlow, a novel molecular graph generative model conditioned on speciﬁc targets.
As shown in Figure 2, SiamFlow learns the distribution of sequence embedding instead of the isotropic Gaussian distribution like other ﬂow-based methods.
[Figure]
4.2. Alignment Loss
Given a pair of target T and drug M , we decompose the drug M into an atom matrix A ∈ R N × K and a bond tensor
B ∈ R C × N × N .
The sequence encoder g T can be arbitrary mapping that maps the target sequence T into the sequence embedding Z T ∈ R D .
The ﬂow model contains a glow h B : R C × N × N → R D 2 and a graph conditional ﬂow h
A | B :
R N × K → R D 2 .
The drug graph embedding Z M is the concatenation of Z A | B and Z B .
Instead of directly learning the isotropic Gaussian distribution, we impose alignment loss between the target sequence embedding Z T and the drug graph embedding Z M so that
Z T can be used as the input of the generation process.
Thus, the generated atom matrix and the bond tensor are:
[Equation]
[Equation]
While traditional ﬂow-based models assume the latent variables follow the Gaussian distribution, SiamFlow forces the ﬂow model to learn the distribution of the condition information instead of a predeﬁned distribution.
We deﬁne the alignment loss L align as:
[Equation]
where [ Z A | B , Z B ] denotes the concatenation of the atom embedding Z A | B and the bond embedding Z B , and the pair of protein target T and molecular drug M is sampled from the data P data .
The alignment loss bridges the connections between the target sequence embedding Z T and the drug graph embedding
Z M in the latent space, but there are still challenges that will be revealed in Sec.
4.3 and Sec. 4.4.
4.3. Uniformity Loss
Simply aligning the target sequence embedding Z T and the drug graph embedding Z M is not enough.
There still remains three challenges: (1) the distribution of Z T is uncertain, so that the alignment learning may be difﬁcult to converge; (2) sampling from an unknown distribution is indeﬁnite in the generation process; (3) the alignment loss alone admits collapsed solutions, e.g., outputting the same representation for all targets.
To overcome the above issues, we design an objective to force the target sequence embedding Z T to follow a speciﬁc distribution, in our case the uniform distribution on the unit hypersphere (Saff & Kuijlaars, 1997; Kuijlaars &
Saff, 1998; Hardin et al., 2004; Liu et al., 2018; Borodachov et al., 2019; Wang & Isola, 2020).
We recognize angles of embeddings are the critical element that preserves the most abundant and discriminative information.
By ﬁtting the hyperspherical uniform distribution, the projections of target sequence embeddings on the hypersphere are kept as far away from each other as possible; thus, discriminations are imposed.
Speciﬁcally, we project the target sequence embedding Z T into a unit hypersphere S D − 1 by L2 normalization and require the embeddings uniformly distributed on this hypersphere, as shown in Figure 3.
[Figure]
The uniform hypersphere distribution can be formulated as a minimizing pairwise potential energy problem (Liu et al., 2018; Borodachov et al., 2019; Wang & Isola, 2020) while higher energy implies less discriminations.
Let (cid:98) Z T =
Z T (cid:107) Z T (cid:107) ∈ C , and C is a ﬁnite subset of the unit hypersphere
S D − 1 ∈ R D .
We deﬁne the f -potential energy (Cohn &
Kumar, 2007) of C to be:
[Equation]
where (cid:98) Z ( x ) T and (cid:98) Z ( y ) T denote different normalized sequence embedding with index x and y .
Deﬁnition.
(Universally optimal (Cohn & Kumar, 2007)).
A ﬁnite subset C ⊂ S D − 1 is universally optimal if it (weakly) minimizes potential energy among all conﬁgurations of |C| points on S D − 1 for each completely monotonic potential function.
In SiamFlow, we consider the Gaussian function kernel
G t ( x, y ) :
S D − 1 × S D − 1 → R as the potential function f , which is deﬁned as:
[Equation]
This kernel function is closely related to the universally optimal conﬁguration, and distributions of points convergence weak* to the uniform distribution by minimizing the expected pairwise potential.
Theorem.
(Strictly positive deﬁnite kernels on S D (Borodachov et al., 2019)).
Consider kernel K f : S D × S D → ( −∞ , + ∞ ] of the form K f ( x, y ) :
= f ( | x − y | 2 ) , if K f is strictly positive deﬁnite on S D × S D and the energy I K f [ σ D ] is ﬁnite, then σ D is the unique measure on Borel subsets of S D in the solution of min µ ∈M ( S D )
I K f ( µ ) , and the normalized counting measure associated with any K f -energy minimizing sequence of point conﬁgurations on S D converges weak* to σ D .
This theorem reveals the connections between strictly positive deﬁnite kernels and the energy minimizing problem.
The Gaussian function is strictly positive deﬁnite on S D × S D , thus well tied with the uniform distribution on the unit hypersphere.
Proposition 1.
(Strictly positive deﬁnite of the Gaussian function)
For any t > 0 , the Gaussian function kernel
G t ( x, y ) is strictly positive deﬁnte on S D × S D .
Proof.
See appendix.
Though Riesz s-kernels R s ( x, y ) := | x − y | − s are commonly used as potential functions, we argue that the Gaussian function is expressive because it maps distances to inﬁnite dimensions like radial basis functions, beneﬁting from the Taylor expansion of exponential functions.
Moreover, the Gaussian function is a general case of Riesz s-kernels and can represent Riesz s-kernels by:
[Equation]
where Γ( {+s/ 2) = (cid:82) ∞ 
 e − t t s+}
[Equation] [Equation]
1 for s > 0 .
As the Gaussian functionk kernel is an ideal choice of potential functions, we deﬁne the uniformity loss as the logarithm of the pairwise Gaussian potential’s expectation:
[Equation]
, (14) where T ( x ) and T ( y ) are two different targets sampled from the target data P T .
4.4. One Target to Many Drugs
Implementing the alignment loss and the uniformity loss above, the ﬂow model can already generate validated molecular drugs conditioned on speciﬁc targets.
However, there are multiple afﬁnable drugs for a single target in most cases.
To deal with this one-to-many problem, we reformulate learning target embeddings into learning spaces of target embeddings in the latent space, as shown in Figure 4.
[Figure]
As the target embeddings have been pushed by the uniformity loss to stay as far away as possible on the hypersphere, they preserve abundant and discriminative information to a large extent.
We design an adaptive space learning strategy that holds the discriminative angle information with a limited scope.
For a set of target sequence embeddings
Z [Equation]
, ..., Z [Equation] L ) T } , we ﬁrst calculate their standard deviation by:
[Equation]
where µ ( Z T ) =
1 L (cid:80) L i =1 Z ( i ) T is the mean of the set Z T .
Then, we deﬁne a space for each target sequence embedding:
[Equation]
where λ is the hyperparameter that controls the scale of the space and is empirically set as 0.1.
Note that we deﬁne the space on Z T instead of the normalized (cid:98) Z T , as normalized embeddings lose the length information to the extent that the available space is limited.
Thus, we modify the alignement loss as:
[Equation]
In the generation process, sampling from the same space is permissible to generate desired drugs.
In summary, the objective is a linear combination of the modiﬁed alignment loss and uniform loss:
[Equation]
5. Experiments
Baselines Since we present a novel generative approach conditioned on targets, we primarily compare our approach to other conditional generative models, i.e., conditional
VAE (Sohn et al., 2015).
Furthermore, an attention-based
Seq2seq (Sutskever et al., 2014; Vaswani et al., 2017) neural translation model between the target protein sequence and drug SMILES is considered a straightforward solution in our setting.
An explainable substructure partition ﬁngerprint (Huang et al., 2019) is employed for sequential drug
SMILES and protein sequences.
We unify these baselines and our proposed SiamFlow into a uniﬁed framework for fair comparisons.
Datasets To evaluate the ability of our proposed
SiamFlow, we collect a dataset based on four drug-target interaction datasets, including BIOSNAP (Zitnik et al., 2018),
BindingDB (Liu et al., 2007), DAVIS (Davis et al., 2011), and DrugBank (Wishart et al., 2018).
We remove all the negative samples in the original datasets, and only keep the positive samples.
Our dataset contains 24,669 unique drugtarget pairs with 10,539 molecular drugs and 2,766 proteins.
The maximum number of atoms in a molecular drug iswhile 11 types of common atoms are considered.
We deﬁne the dataloader to ensure zero overlap protein in the training, validation, and test set.
Metrics To comprehensively evaluate the conditional generative models in terms of target-aware molecular generation, we design metrics from two perspectives: (1) Generative metrics.
Following the common molecular generation settings, we apply metrics including: validity which is the percentage of chemically valid molecules in all the generated molecules, uniqueness which is the percentage of unique valid molecules in all the generated molecules, novelty which is the percentage of generated valid molecules which are not in the training dataset.
(2) Chemical metrics.
We evaluate the similarities between the generated drugs and the nearest drugs in the training set including: Tanimoto similarity which is calculated based on hashed binary features, Fraggle similarity which focus on the fragmentlevel similarity, MACCS similarity which employs 166-bit 2D structure ﬁngerprints.
Empirical Running Time We implement our proposed method SiamFlow and the other two baselines Seq2seq,
CVAE by Pytorch-1.8.1 framework.
We train them with
Adam optimizer with learning rate 0.001, batch size 16, and 100 epochs on a single NVIDIA Tesla V100 GPU.
To evaluate the validity and chemical similarities, we employ the cheminformatics toolkit RDKit in the assessment phase.
Our SiamFlow completes the training process of 100 epochs in an average of 1.06 hours (38 seconds/ epoch), while
CVAE and Seq2seq take an average of 1.14 hours (41 seconds/ epoch) and 8.33 hours (5 minutes/ epoch) respectively.
5.1. Target-aware Molecular Graph Generation
We conduct experiments on molecular drug generation with speciﬁc targets for comparisons.
For each experiment, we repeat three trials with different random seeds and report the mean and standard deviation.
Table 1 shows the results on generative metrics of our
SiamFlow model in comparison to the baselines.
Our proposed SiamFlow inherits the strengths of the ﬂow and far surpasses other baselines in generative metrics.
It can be seen that Seq2seq suffers from low validity, uniqueness, and novelty, which indicates Seq2seq’s generation relies on its memorization.
CVAE has higher uniqueness and novelty than Seq2seq though its validity is even lower.
Besides, the standard deviations of metrics on CVAE are relatively high, suggesting it is volatile to train.
Among them, SiamFlow obtains superior performance with the least volatility.
[Table]
In addition to generative metrics, we also report chemical metrics in Table 2.
The generated molecular drugs are expected to have a chemical structure similar to the groundtruth drugs in order to have a high binding afﬁnity to the target.
SiamFlow is consistently better than other baselines in both the Tanimoto and Fraggle similarity while obtaining relatively lower MACCS similarity than Seq2seq.
Consid- ering that MACCS measures the similarity of encodings of molecules, the sequence partition rules of Seq2seq may help it.
Thus, we pay more attention to the Tanimoto and Fraggle similarity because they are structure-centric metrics.
[Figure]
[Table]
We visualize the distribution of the Tanimoto similarity and the Fraggle similarity evaluated on these methods in
Figure 5.
SiamFlow consistently outperforms other methods and generates desirable molecular drugs.
The examples of generated drugs are shown in Figure 6.
[Figure]
[Figure]
5.2. Ablation Study
We conduct the ablation study and report the results in Table 3 and Table 4.
It can be seen from Table 3 that simply aligning the target sequence embedding and drug graph embedding will result in extremely low uniqueness.
Our one-to-many strategy enriches the latent space so that one target can map to different drugs.
The absence of L unif does not harm the generative metrics because it only constrains the distribution of target sequence embeddings but has a limited impact on the generation process.
[Table]
Table 4 demonstrates the chemical metrics are well without the one-to-many strategy.
If we generate only one drug for a particular target, the nearest drug similarity degrades to a special case, i.e., comparing the generated drug with its corresponding one in the training set.
Moreover, removing
L unif severely impairs the chemical performance, suggesting the uniformity loss promotes the expressive abilities of target sequence embeddings.
[Table]
6. Conclusion and Discussion
In this paper, we explore target-aware molecular graph generation, which differs in generating drugs conditioned on speciﬁc targets, while existing methods focus on developing similar drugs based on drug-like datasets.
Target-aware molecular generation combines drug-like molecular generation with target-speciﬁc screening and thus simpliﬁes the drug-target interaction step.
To explore this problem in-depth, we compile a benchmark dataset from three public datasets and build baselines in a uniﬁed framework.
Furthermore, we take advantage of recent progress on ﬂow-based molecular graph generation methods and propose SiamFlow towards target-aware molecular generation.
With the alignment and uniform loss, the proposed method can effectively generate molecular drugs conditioned on protein targets.
Moreover, we deal with one target to many drugs by aligning the embedding space instead of a single embedding.
Extensive experiments and analyses demonstrate that
SiamFlow is a promising solution towards target-aware molecular generation.
A worthwhile avenue for future work is introducing protein structure information for decent target sequence representations.