H OW M UCH S PACE H AS B EEN E XPLORED ? M EASURING THE C HEMICAL S PACE C OVERED BY D ATABASES AND M ACHINE -G ENERATED M OLECULES
Anonymous authors
Paper under double-blind review
A BSTRACT
Forming a molecular candidate set that contains a wide range of potentially effective compounds is crucial to the success of drug discovery.
While most databases and machine-learning-based generation models aim to optimize particular chemical properties, there is limited literature on how to properly measure the coverage of the chemical space by those candidates included or generated.
This problem is challenging due to the lack of formal criteria to select good measures of the chemical space.
In this paper, we propose a novel evaluation framework for measures of the chemical space based on two analyses: an axiomatic analysis with two intuitive axioms that a good measure should obey and an empirical analysis on the correlation between a measure and a proxy gold standard. Using this framework, we are able to identify a novel chemical space coverage measure, #Circles , superior to existing measures both analytically and empirically.
We further evaluate how well the existing databases and generation models cover the chemical space in terms of #Circles. The results suggest that many generation models fail to explore a larger space over existing databases, which leads to new opportunities for improving generation models by encouraging exploration.
I NTRODUCTION
To efficiently navigate through the huge chemical space for drug discovery, machine learning (ML) based approaches have been broadly designed and deployed, especially de novo molecular generation methods (Elton et al., 2019; Schwalbe-Koda & Gómez-Bombarelli, 2020; Bian & Xie, 2021;
Deng et al., 2022).
Such generation models learn to generate candidate drug designs by optimizing various molecular property scores , such as the binding affinity scores.
In practice, these scores can be computationally obtained using biological activity prediction models (Olivecrona et al., 2017; Li et al., 2018), which is the key to obtaining massive labeled training data for machine learning.
However, high in silico property scores are far from sufficient, as there is usually a considerable misalignment between these scores and the in vivo behaviors.
Costly wet-lab experiments are still needed to verify potential drug hits, where only a limited number of drug candidates can be tested.
In light of this cost constraint, it is critical to select or generate drug candidates not only with high in silico scores, but also covering a large portion of the chemical space. As functional difference between molecules is closely related to their structural difference (Huggins et al., 2011; Wawer et al., 2014), a better coverage of the chemical space will likely lead to a higher chance of hits in wet experiments.
For this purpose, quantitative coverage measures of the chemical space become crucial. Such measures can both be used to evaluate and compare the candidate libraries 1 , and be incorporated into training objectives to encourage ML models better explore the chemical space. In this paper, we investigate the problem of quantitatively measuring the coverage of the chemical space by a candidate library.
There have been a few such coverage measures of chemical space. For example, richness counts the number of unique compounds in a molecular set, and it has been used to describe how well a model is able to generate unique structures (Shi & von Itzstein, 2019;
Polykovskiy et al., 2020).
In addition, molecular fingerprints have been used to calculate pairwise similarity or distance between two compounds, and the average of these pairwise distances has been used to describe the overall internal diversity of a molecular set (Brown et al., 2019; Polykovskiy et al., 2020). However, most existing coverage measures are heuristically proposed and the validity of these measures is rarely justified.
In fact, defining the “right” measure for the coverage of chemical space coverage is challenging.
Unlike the molecular property scores, there is no obvious “ground truth” about the coverage of chemical space.
Moreover, the chemical space is complex and combinatorial, making the design of a good measure even more difficult.
To address the fundamental problem of properly measuring the coverage of chemical space for drug discovery, we propose a novel evaluation framework with two complementary criteria for evaluating the validity of coverage measures.
In Section 3, we first formally define the concept of coverage measures on the chemical space (referred to as chemical space measures ), where many existing heuristic measures fall into our definition. Then we introduce the two criteria and compare various (existing and new) chemical space measures based on the criteria.
Specifically, the first criterion (Section 4.1) is based on an axiomatic analysis with two intuitive axioms that a good chemical space measure should satisfy.
Surprisingly, most heuristic measures that are commonly used in literature, such as internal diversity , fail to satisfy one or both of the axioms.
The second criterion (Section 4.2) compares the chemical space measures with a proxy of the gold standard: the number of unique biological functionalities covered by the set of molecules.
We find that a novel chemical space measure (#Circles, defined in Section 3.2.3) not only satisfies both axioms but also better correlates with the gold standard.
Finally, we apply the #Circles measure to evaluate how well the existing databases and ML models cover the chemical space.
Interestingly, the evaluation results suggest that many ML models fail to explore a larger portion of chemical space compared to drug candidates obtained from virtual screening over existing databases. We believe these findings lead to a new direction to improve ML-based drug candidate generation models on better exploring the chemical space.
R ELATED W ORK 2. M ETRICS FOR E VALUATING M OLECULAR D ATABASES AND G ENERATION M ETHODS
Molecular databases and machine-generated compounds are rich sources of drug candidates for forming a candidate library in drug discovery.
To evaluate the quality of molecular databases and molecular generation methods, a variety of metrics are proposed. In general, four categories of evaluation metrics can be identified in the literature, which are related to: (1) bioactivities, (2) molecular properties, (3) data likelihood, or (4) the coverage of the chemical space, respectively. The first category describes the biological activities of the included or generated molecules towards certain protein targets. Using docking simulators, researchers can calculate docking scores to determine the binding affinity between the given compound structure and the target (Trott & Olson, 2010).
However, the simulation usually takes a long time to run. To obtain fast feedback and a sufficient amount of data to train molecular generation models, machine learning (ML) methods are also proposed to predict the bioactivity of molecules (Shoichet et al., 1992; Olivecrona et al., 2017;
Li et al., 2018; Ahmadi et al., 2021; Jiang et al., 2022).
The second category of metrics evaluates the molecular properties of compounds. Some of them are calculated based on heuristic rules, such as the validity of compounds, molecular weight, octanol-water partition coefficient (logP) (Wildman & Crippen, 1999), drug-likeness score (QED)
(Bickerton et al., 2012), and synthetic accessibility (SA) (Ertl & Schuffenhauer, 2009).
ML (especially deep learning)-based prediction models are also employed to predict the properties of compounds (Shen et al., 2010; Wang et al., 2019; Gilmer et al., 2017). Another branch of metrics uses data likelihood to evaluate the molecules output by generative models (Gómez-Bombarelli et al., 2018; Liu et al., 2018; Jin et al., 2018; De Cao & Kipf, 2018).
A few other metrics are particularly related to the coverage of the chemical space or the extent of exploration in the chemical space. We review these metrics in the subsection below.
2. M ETRICS R ELATED TO THE C OVERAGE OR THE E XPLORATION OF THE C HEMICAL S PACE Beyond
bio -activity and
property optimization, some studies evaluate molecular databases and molecular generation models with measures more or less related to the degree of coverage (or exploration) in the chemical space.
Commonly used measures include richness, uniqueness, internal diversity, external diversity, KL divergence, and Fréchet ChemNet Distance (FCD) (Olivecrona et al., 2017; You et al., 2018; De Cao & Kipf, 2018; Elton et al., 2019; Brown et al., 2019; Popova et al., 2019; Polykovskiy et al., 2020; Shi et al., 2020; Xie et al., 2021).
Besides, Zhang et al.
(2021) propose to use the number of unique functional groups or ring systems to estimate the chemical space coverage and to compare several recent generative models.
Similarly in Blaschke et al.
(2020), the number of unique Bemis-Murcko scaffolds is used to measure the variety of drug candidates.
Koutsoukas et al.
(2014) study the effect of molecular fingerprinting schemes on the internal diversity of compound selection.
These measures usually mix the concepts of diversity, coverage, or novelty, and their validity as a measure of exploration is not justified.
To the best of our knowledge, this is the first work that formally investigates the validity of molecular chemical space measures.
In particular, axiomatic approaches are used to analytically evaluate various designs of a measurement, such as utility functions (Herstein & Milnor, 1953), cohesiveness (Alcalde-Unzu & Vorsatz, 2013), or document relevance (Fang et al., 2004).
Using axiomatic analysis to evaluate the chemical space measures in the chemical space is novel.
With an empirical analysis in addition to the axiomatic analysis, we make practical recommendations on effective chemical space measures, including two novel measures.
D EFINING C HEMICAL S PACE M EASURES 3. D EFINITION OF C HEMICAL S PACE M EASURES
To define a chemical space measure, we first formalize the notion of chemical space by assuming a distance metric exists.
This assumption is widely adopted in cheminformatics, using distance metrics such as the Tanimoto distance (Tanimoto, 1968; Bajusz et al., 2015).
Assumption 3.1 (Chemical space) .
The chemical space U contains all possible molecules and is a metric space with a distance metric function d : U × U → [0 , + ∞ ) .
Definition 3.2 (Tanimoto distance) .
For two molecules x 1 , x 2 ∈ U , whose binary molecular fingerprint vectors are x 1 , x 2 ∈ { 0 , 1 } n where n is the dimensionality of the fingerprint, their Tanimoto distance is defined as
[Equation]
The Tanimoto distance is also referred to as the Jaccard distance (Jaccard, 1912) in other domains.
Its range is [0 , 1] .
For finite sets ( e . g ., molecular fingerprints), the Tanimoto distance is a metric function (Kosub, 2019; Lipkus, 1999).
Besides the Tanimoto distance, one can also use the distance of latent hidden vectors (Preuer et al., 2018; Samanta et al., 2020) or the root-mean-square deviation (RMSD) of three-dimensional molecular conformers (Fukutani et al., 2021).
We then define a chemical space measure as the following.
Definition 3.3 (chemical space measure) .
Given the universal chemical space U , a chemical space measure is a function that maps a set of molecules to a non-negative real number that reflects to what extent the set spans the chemical space, i .
e ., µ : P ( U ) →
[0 , ∞ ) , where P ( · ) is the notation of power set.
In particular, µ ( ∅ ) = 0 .
3. E XAMPLES OF C HEMICAL S PACE M EASURES
Definition 3.3 is intentionally left general.
Many measures related to the coverage, diversity, or novelty of a molecular set fall into this definition.
We summarize these various measures used in literature into three categories: distance-based measures, reference-based measures, and locality-based measures.
We are also able to define new chemical space measures under this formulation.
3.2. D ISTANCE - BASED M EASURES
For a set S with n molecules, we can define a chemical space measure according to the distances among the molecules:
[Equation]
, y ) ,
SumDiversity ( S ) :
= (cid:88) x ∈S n −(cid:88)
y ∈S y ̸
= x d ( x, y )
Diameter ( S ) :
= max x,y ∈S x ̸ = y d ( x, y ) ,
SumDiameter ( S ) :
= (cid:88) x ∈S max y ∈S
y ̸
= x d ( x, y ) ,
Bottleneck ( S ) :
= min x,y ∈S x ̸ = y d ( x, y ) ,
SumBottleneck ( S ) :
= (cid:88) x ∈S min y ∈S y ̸
= x d ( x, y ) ,
DPP ( S ) :
= det ( S ) , where x, y are molecules in S , d is a distance metric as defined previously, and S is the similarity matrix of candidate molecules ( e .
g ., 1 − d ( x, y ) for Tanimoto similarity).
Among these measures, Diversity (also referred to as the internal diversity ), the average distance between the molecules, is widely used in the literature (You et al., 2018;
De Cao & Kipf, 2018;
Popova et al., 2019; Polykovskiy et al., 2020; Shi et al., 2020; Xie et al., 2021).
We follow the topology theory and also introduce Diameter and Bottleneck as the maximum and minimum distance between any pair of molecules, respectively (Edelsbrunner & Harer, 2010), since they can also reflect the dissimilarity between molecules in S .
We further introduce three Sum- variants for the above measures.
The Sum- variants 2 will tend to increase when new molecules are added into the set.
In addition, the determinant of the similarity matrix of molecules is also a measure of dissimilarity, which is often employed in diverse subset selection as a key concept of the determinantal point processes (DPP) (Kulesza & Taskar, 2011; 2012).
3.2. R EFERENCE - BASED M EASURES
Another broad category of chemical space measures compare the generated molecules S with a reference set R .
In such context, a reference-based measure can be defined as the coverage of references:
Coverage ( S , R ) :
= (cid:88)
y ∈R (cid:18) max x ∈S cover ( x, y ) (cid:19) , where cover ( x, y ) indicates how molecule x can cover the reference y .
When the reference set R is taken as a collection of molecular fragments, the coverage function can be written as cover ( x, y ) :
= I [ molecule x contains fragment y ] , where I [ · ] is the indication function.
A large body of drug discovery literature uses the number of distinct functional groups (FG), ring systems (RS), or Bemis-Murcko scaffolds (BM) in S to gauge the size of explored chemical space (Zhang et al., 2021; Blaschke et al., 2020), corresponding to the cases where R is the collection of all possible FG, RS, or BM fragments.
We denote these specific reference-based measures as #FG, #RS, and #BM respectively.
3.2. L OCALITY - BASED M EASURES
Inspired by the sphere exclusion algorithm used in compound selection (Snarey et al., 1997; Gobbi & Lee, 2003), we introduce a new chemical space measure that highlights the local neighborhoods covered by a set of molecules:
[Equation]
where t ∈ [0 , 1) is a distance threshold that corresponds to the diameter of a circle.
Intuitively, #Circles counts the maximum number of mutually exclusive circles that can fit into
S as neighborhoods, with a subset of its members C as the circle centers.
When the threshold t = 0 , #Circles becomes the richness measure Richness ( S ) :
= |{S}| , which is the number of unique molecules (Shi & von Itzstein, 2019; Polykovskiy et al., 2020).
S ELECTING THE R IGHT M EASURE
While all the aforementioned measures can heuristically reflect the degree of exploration, they do not always agree with each other.
We need a principled way to select the most suitable measures from a variety of possible choices.
Ideally, the selection criteria should not depend on the particular molecule properties of target or the algorithm used to generate the candidates.
We achieve this through a qualitative axiomatic analysis and a quantitative correlation comparison. 4. C RITERION #1: A N A XIOMATIC A NALYSIS OF C HEMICAL S PACE M EASURES [Figure]
We propose two simple and intuitive principles that a good chemical space measure should satisfy: (1) discovering more molecules should not decrease the degree of exploration; (2) molecular sets with more dissimilar molecules have a higher degree of exploration.
These two principles are formalized below as two axioms and can be tested analytically.
Axiom 4.1 (Subadditivity) .
A good chemical space measure µ should be subadditive, i.e., for any two molecular sets S 1 , S 2 ⊆ U , it holds that
[Equation]
A subadditive measure tends to increase when more molecules are included.
This tendency is very intuitive in drug discovery as testing more molecules means we can have a higher probability to discover a potentially effective drug.
Some direct corollaries are listed in Appendix A.
Axiom 4.2 (Dissimilarity) .
A good chemical space measure should have a preference to dissimilar elements, i.e., for any two molecules x 1 , x 2 ∈ U and a molecular set X ( x 1 , x 2 )
= { x | d ( x, x 1 )
+ d ( x, x 2 ) =
d ( x 1 , x 2 ) , x ̸ = x 1 , x 2 } , it holds
[Equation]
where x ∗ satisfies d ( x ∗ , x 1 ) =
d ( x ∗ , x 2 ) =
12 d ( x 1 , x 2 ) and d is the distance metric.
Intuitively, consider two molecules x 1 , x 2 that are placed at the two endpoints of a “segment”.
a chemical space measure satisfying the dissimilarity axiom should increase the most when the molecule x ∗ is added to the middle point of the segment, since x ∗ is most dissimilar to x 1 , x 2 comparing to any other points on the segment.
Depending on whether each chemical space measure satisfies subadditivity and/or dissimilarity , we can put it into one of four quadrants as shown in Figure 1.
The formal proofs are provided in
Appendix B. Despite that the two axioms are simple and intuitive, surprisingly, the proposed #Circles is the only measure that satisfies both axioms.
4. C RITERION #2: C ORRELATION WITH B IOLOGICAL F UNCTIONALITY
While only one measure stands out in the axiomatic analysis, it does not guarantee its empirical performance.
Meanwhile, there are still multiple measures that satisfy one of the two axioms.
In this subsection, we further investigate the validity of the chemical space measures by testing their correlations with the biological functionality variety of molecules.
Such functionalities can provide valuable information in distinguishing molecules and implicating chemical space coverage.
We correlate the chemical space measures to the number of unique biological functionality labels covered by a molecular set.
A better chemical space measure should have a higher correlation to the variety of biological functionalities, even though it is a proxy gold standard.
4.2. E XPERIMENT S ETUP
We base the analysis on the BioActivity dataset that is also used to compare different compound selection algorithms (Koutsoukas et al., 2014).
This dataset contains 10,000 compound samples extracted from the ChEMBL database (Gaulton et al., 2017) with bio-activity labels, which annotate 50 activity classes with 200 samples each.
Following Koutsoukas et al.
(2014), for a subset of this dataset S , we take the number of unique class labels as a proxy “gold standard” of the variety of the molecules in S , i .
e .,
GS ( S ) : =
# unique labels in S , which represents the number of biological functionality types covered by S .
We then compare the behavior of the gold standard and the chemical space measures in two settings to find out which measures have the highest correlations with the [Figure] [Figure]
[Figure] [Figure]
The chemical space measure with the highest correlation is highlighted in green are by running coverage (or variety) of biological functionalities: (1) a fixed-size setting where random subsets of the dataset with fixed sizes are measured, and (2) a growing-size setting where we sequentially add molecules into a molecular set.
We compare the measuring results of GS and chemical space measures with Spearman’s correlation coefficient and dynamic time warping (DTW) distance.
The experiment details are listed in Appendix C.
4.2. R ESULTS AND D ISCUSSION
Experiment results in Figure 2 suggest that #Circles stands out for both empirical experiment settings.
For the fixed-size setting, #Circles and SumBottleneck have notably higher correlations to the gold standard than other measures.
This indicates that the locality information is critical , as these two measures both prefer new molecules that are at arm’s length from their nearest neighbors.
For the growing-size setting, #Circles surpasses all other chemical space measures.
Reference-based measures such as #FG also perform prominently.
Both #Circles and reference-based measures satisfy subadditivity, making them suitable for a “growth” setting.
The Sum- variants of distance-based measures outperform their original forms, as they tend to increase when adding new molecules.
More experiment results can be found in Appendix C, where we also discuss the impact of the distance metric d .
We also study the sensitivity of the subset size n , the way of adding new molecules, and the choice of the #Circles’s threshold t .
Results suggest that t = 0 . 75 is a good choice for both two settings.
Correlations between chemical space measures are visualized in Figure 8 and 11. 4. W HICH M EASURE IS THE R IGHT C HOICE ?
[Table]
The empirical analysis shows that the locality-based #Circles measure is a robust choice for all tested scenarios, which reconfirms the conclusion of the axiomatic analysis.
Besides, SumBottleneck may be an effective choice when a fixed number of candidates are the target.
Referencebased measures may be good alternatives if carefully-designed and comprehensive reference sets are available.
Surprisingly, the widely used Diversity measure is rendered inferior both analytically and empirically, casting doubts on its efficiency in measuring and encouraging exploration for molecular generation. Our analysis and comparison results are summarized in Table 1.
M EASURING C HEMICAL S PACE C OVERAGE
In this section, we apply the chemical space measures to evaluate how well the existing databases and ML-based molecular generation models cover the chemical space. [Figure] 5. M EASURING C HEMICAL S PACE C OVERED BY M OLECULAR D ATABASES
Other than being a rich source for virtual screening for drug purposes, molecular databases are also bonanzas of data for training ML models.
In both scenarios, the coverage of chemical space is a crucial descriptor for comparing various databases. With a higher coverage of the chemical space, we are able to construct candidate libraries with better varieties.
Also, the ML model would benefit from the diverse training data, obtaining a better understanding of the overall chemical space. Here, we compare five commonly used molecular databases with the selected measures. 5.1. E XPERIMENT S ETUP Databases
(1) ZINC-250k is a random subset of the ZINC database (Irwin & Shoichet, 2005) and consists of 249K commercially-available compounds from different vendors for virtual screening;
(2) MOSES (Polykovskiy et al., 2020) is another sub-collection of ZINC molecules.
It contains approximately 2M molecules in total, filtered by atom types , ring cycle sizes, medicinal chemistry filters (MCFs), and PAINS filters.
(3) ChEMBL (Gaulton et al., 2017) is a manually curated database of 2M bioactive molecules with known experimental data, especially with the inhibitory and binding properties against macromolecule targets.
(4) GDB-17 (Ruddigkeit et al., 2012) enumerates chemical structures that contain up to 17 atoms.
The lead-like subset filters 11M compounds with lead-like properties.
(5) Enamine Hit Locator Library contains a diverse collection of molecules with high
MedChem tractability.
The compounds are guaranteed synthesizable at a reasonable cost by the
Enamine company.
Filtering.
In drug discovery, drug-likeness and synthesizability of compounds are both important considerations.
To this end, we compare both the original databases and the filtered databases.
The quantitative drug-likeness (QED) (Bickerton et al., 2012) and the synthetic accessibility (SA) (Ertl &
Schuffenhauer, 2009) are computed.
The molecules are filtered with QED ≥ 0 .
6 and SA ≤ 4 .
R ESULTS AND D ISCUSSION
Figure 3 compares Richness and Circles of molecular databases.
From the results, we conclude the findings as follows.
(1) A larger database does not necessarily span a wider range of the chemical space : There is a discrepancy between Richness and #Circles, and the #Circles measuring results suggest that, in terms of chemical space coverage, the rank of the databases should be
GDB-17 > ChEMBL > ZINC-250k > MOSES > Enamine.
(2) ZINC-250k and GDB-17 are recommended : With the smallest amount of compounds, ZINC-250k renders a relatively high coverage of the chemical space, which is highly recommended for model training with limited computation resources.
Constructed by enumeration, GDB-17 substantially expands the known chemical space to a great level, which is suitable for training ML models for chemical space representations.
More details and findings on other chemical measures are provided in Appendix D.
5. M EASURING C HEMICAL S PACE E XPLORED BY G ENERATION M ODELS 5.2. P ROBLEM F ORMULATION
In drug discovery, the search for active molecules towards a druggable target is often formulated as the following unconstrained optimization problem (Olivecrona et al., 2017; Gómez-Bombarelli et al., [Figure]
; Liu et al., 2018; You et al., 2018; Jin et al., 2018; De Cao & Kipf, 2018; Popova et al., 2019; Shi et al., 2020; Xie et al., 2021):
[Equation]
where x is a molecule in the chemical space U , s k : U → R is a function scoring particular biological properties, and the “ ◦ ” operator indicates the combination of multiple scores ( e .
g ., summation or multiplication). This scoring function can be binding affinity to protein targets, drug-likeness, synthesizability, etc .
When wet-lab experiments are not available, these scores are obtained computationally.
Unfortunately, the majority of the computational scoring functions are merely approximations and cannot accurately predict the wet-lab results.
As pointed out in the chemistry literature, rather than focusing on optimizing the estimated property scores, it is crucial to generate a variety of compounds that span a wider range of the chemical space (Huggins et al., 2011; Wawer et al., 2014; Ashenden, 2018).
We therefore propose the following objective as the goal of molecular generation, and use this new objective to compare models:
[Equation]
where S ⊆ U is a molecular candidate set, µ ( · ) is a chemical space measure, s k ( x ) ≥ C k means the estimated property score of a molecule x is at or above a threshold C k .
5.2. E XPERIMENT S ETUP
Generation objective.
Following Li et al.
(2018) and Jin et al.
(2020), we consider the inhibition against an Alzheimer-related target protein c-Jun N-terminal kinase-3 (JNK3) as the biological objective.
The JNK3 binding affinity score is predicted by a random forest model based on Morgan fingerprint features of a molecule (Rogers & Hahn, 2010).
We also consider the drug-likeness (QED) and synthetic accessibility (SA).
In summary, we include three constraints on molecules:
JNK3 ≥ 0 .(predicted as inhibitors), QED ≥ 0 . 6 (drug-like), and SA ≤ 4 (easy to synthesize).
Models.
We employ four recently proposed molecular generation models to optimize the generation objective (JNK3, QED, and SA): (1) RationaleRL (Jin et al., 2020) generates molecules by combining rationales with reinforcement learning (RL);
(2) DST (Fu et al., 2021) supports a gradient-based optimization on chemical graphs; (3) JANUS (Nigam et al., 2021) uses the genetic algorithm (GA) to design drugs inversely with parallel tempering; (4) MARS (Xie et al., 2021) generates fragment-based molecular structures based on Markov chain Monte Carlo (MCMC) sampling.
Due to MARS’ great ability to simultaneously optimize multiple drug discovery objectives, we also consider three variants of it (MARS+Diversity, MARS+SumBottleneck, and MARS+#Circles), where the chemical space measures are incorporated into Equation 2 to encourage exploration. In addition, we also include a virtual screening setting as a baseline, where the five molecular databases introduced in the previous subsection are combined, forming a candidate set of 11M compounds.
Note that many target-irrelevant molecules are included.
Evaluation.
To validate the ability in exploring the chemical space, We compare the molecules generated by models using Equation 3 with multiple chemical space measures .
We examine seven different measures, and report #Circles, Richness, and Diversity here; the others are listed in Appendix
E). Note we cannot directly measure the biological functionality coverage as in Section 4.2, as many of the generated molecules are not in the database.
For RationaleRL and DST, 5000 molecules are generated as suggested in their papers. For JANUS and MARS, the population size and the number of MCMC chains are set as 5000.
JANUS evolves for 10 generations, and MARS iterates 2000 steps.
More implementation details are listed in Appendix E.
[Table]
5.2. R ESULTS AND D ISCUSSION
Table 2 lists the chemical space measures estimated for the molecules generated by models, and
Figure 4 shows the dynamics of measures for MARS and its variants. Based on these results, we conclude the followings. (1) ML-based models fail to explore a larger effectual area compared to databases : Unexpectedly, we find the molecules obtained by virtual screening over databases span the largest area of the chemical space in terms of #Circles when t = 0 .
The most competitive model among the tested ones, MARS+#Circles, explores only about half of the space comparing to virtual screening. (2) ML-based models like MARS tend to exploit locally instead of exploring globally : For #Circles t = 0 .
75 and t = 0 . 80 , MARS variants fail to surpass virtual screening, whereas t = 0 . 70 tells a different story. With a large t , #Circles computes the coverage of global regions in the chemical space.
The results suggest that ML-based models like MARS are more capable of local exploitation rather than global exploration. (3) Chemical space measures can encourage the model to explore : By incorporating chemical space measures into the optimization objective of MARS, all measures are significantly improved during molecular generation. (4) Diversity should be avoided as a descriptor for exploration : MARS and its three variants all converge quickly in Diversity as, despite a large number of new molecules discovered. Again, it suggests that the widely used Diversity is not suitable for measuring chemical space.
A few limitations of the measurement with #Circles should be noted.
First, the computation of #Circles is combinatorial and the running time is exponential in theory.
We implement a fast approximation by sacrificing accuracy.
Second, the measuring results can be sensitive to the selection of the distant threshold t (circle diameter).
Different thresholds are investigated in Appendix C, and more sensitivity tests and hyper-parameter tuning strategies are needed for determining the optimal threshold.
More details and findings on other chemical measures are provided in Appendix E.
C ONCLUSION AND F UTURE W ORK
In this paper, we have presented a systematic study on the coverage measures of chemical space in the context of drug discovery.
The core contribution of this study is a novel evaluation framework for selecting proper coverage measures. Using the proposed framework, we have identified a new coverage measure, #Circles, that is superior to existing heuristic measures commonly used in the drug discovery literature. We have also quantitatively compared molecular databases and ML-based generation models in terms of #Circles.
The results suggest that many ML-based generation models fail to explore a larger chemical space compared to virtual screening, since they tend to exploit locally instead of exploring widely. Measuring the chemical space is a fundamental problem in drug discovery.
Our work in this direction opens even more research questions that are worth investigating in the future. For example, it would be interesting to design and evaluate more chemical space measures under our framework.
Moreover, the newly suggested objective for molecular generation (Equation 3) inspires further study, since it is computationally challenging as a combinatorial constrained optimization problem.
In general, the proposed chemical space measures and the generic measure selection framework also can be applied to other domains such as text generation and image generation.
A S UBADDITIVITY C OROLLARIES
Corollary A.1.
(Subtraction) .
If a chemical space measure µ is subadditive, then for any two molecular sets ∀S 1 , S 2 ⊆ U , we have
[Equation]
Corollary A.2.
(Monotonicity) .
If a chemical space measure µ is subadditive, then for any molecular set ∀S ⊆
U and a single molecule x ∈ U , we have µ ( S ∪ { x } ) ≥ µ ( S ) and µ ( S \ { x } ) ≤ µ ( S ) .
Corollary A.3.
(Dominance) .
If a chemical space measure µ is subadditive, then for any two molecular sets ∀S 1 , S 2 ⊆ U , if S 1 ⊆ S 2 , then µ ( S 1 ) ≤ µ ( S 2 ) .
B P ROOFS FOR S UBADDITIVITY AND D ISSIMILARITY
In Figure 1 we show the chemical space measures according to whether they will satisfy subadditivity and/or dissimilarity.
In this section, we provide proofs to verify each measure’s subadditivity and dissimilarity.
Note that for distance-based measures, when |S| = 1 , µ ( S ) is not defined.
So without loss of generality, we assume µ ( { x } ) =
w ( x ) , x ∈ U in such cases, where w : U → R is an importance function.
However in the following proofs, we can see that the choice of w ( · ) will not influence our conclusion.
Proposition B.1 (Subadditivity of chemical space measures.) .
Reference-based chemical space measures and #Circles are subadditive.
Distance-based chemical space measures including Diversity,
SumDiversity, Diameter, SumDiameter, Bottleneck, SumBottleneck, and DPP are not subadditive.
Proof.
For the reference-based chemical space measures, we denote the cover ( · , · ) function as cov ( · , · ) for short.
We prove the subadditivity of reference-based measures by first proving the subadditivity of the maximum of cover ( · , · ) .
To prove the subadditivity of the maximum of cov ( · , · ) , we consider combining any two molecular sets S 1 , S 2 ⊆ U .
For any reference y ∈ R , we have
[Equation]
Therefore,
[Equation]
thus proving the Coverage measure is subadditive.
For the #Circles chemical space measure, we denote #Circles ( · ) as #C ( · ) for short and define
C ∗ ( S ) ⊆ S as an arbitrary set that satisfies |C ∗ ( S ) | =
#C ( S ) .
We prove the subadditivity of #C in two parts.
In the first part, we prove the left-hand side of the subadditivity inequation, i .
e ., max( #C ( S 1 ) , #C ( S 2 ))
≤ #C ( S 1 ∪ S 2 ) .
For any two molecular sets S 1 , S 2 ⊆ U , since C ∗ ( S 1 ) ⊆
S 1 ∪ S 2 , according to the definition of #C, we have #C ( S 1 ) ≤ #C
( S 1 ∪ S 2 ) .
Similarly, we also have
[Equation]
, proving the left-hand side of subadditivity.
In the second part, we prove the right-hand side of the subadditivity inequation, i .
e ., #C ( S 1 ∪ S 2 ) ≤ #C ( S 1 ) + #C
( S 1 ) .
We prove this by contradiction.
For any two molecular sets S 1 , S 2 ⊆ U , we assume #C ( S 1 ∪ S 2 ) >
#C ( S 1 ) + #C ( S 1 ) .
Use the notations C 1 :=
C ∗ ( S 1 ∪ S 2 ) ∩ S 1 and
C 2 :=
C ∗ ( S 1 ∪ S 2 ) ∩ S 2 .
We have
[Equation]
Since all values are non-negative, we must have |C 1 | > #C ( S 1 ) or |C 2 | > #C ( S 2 ) , contradicting with the definition of #C ( S 1 ) or #C ( S 1 ) , thus proving the right-hand side of subadditivity.
For the Diversity chemical space measure, we denote it as Div ( · ) for short.
We disprove its subadditivity by proving it violates the monotonicity corollary.
For a molecule x ∈ U and a molecular set S ⊆ U with size n > 1 .
If x ̸∈ S , we have
[Equation]
And the change in Div is
[Equation]
When the average distance of x and S , i .
e ., 1 n (cid:80)
y d ( x, y ) , is less than Div ( S )
( e .
g ., adding an molecule on the “segment” between two existing molecules), Diversity would decrease, thus violating the monotonicity corollary and proving Diversity is not subadditive.
For the SumDiversity chemical space measure, we denote it as SD ( · ) for short.
We disprove its subadditivity by proving it violates the right-hand side of the subadditivity, i .
e ., SD ( S 1 ∪ S 2 ) ≤
SD ( S 1 ) + SD ( S 2 ) .
For two disjoint molecular sets with two molecules in each, i .
e ., { x 1 , x 2 } and { x 3 , x 4 } , we denote d ij = d ( x i , x j ) .
Then we have
[Equation]
and
[Equation]
When the inter-set distances are larger than the inner-set distances, i .
e ., d 13 , d 14 , d 23 , d 24 > d 12 , d 34 , we will have SD ( { x 1 , x 2 , x 3 , x 4 } ) >
SD { x 1 , x 2 } + SD { x 3 , x 4 } , violating the right-hand side of the subadditivity, thus proving the SumDiversity measure is not subadditive.
For the Diameter chemical space measure, we denote it as Dia ( · ) for short.
We disprove its subadditivity by proving it violates the right-hand side of the subadditivity, i .
e .
, Dia ( S 1 ∪ S 2 ) ≤
Dia ( S 1 ) + Dia ( S 2 ) .
For two disjoint molecular sets S 1 , S 2 ⊆ U whose sizes are larger than one, we have
[Equation]
and
Dia ( S
[Equation]
+ [Equation] [Equation] When the maximum inter-set distance is larger than the maximum inner-set distance, i .
e ., max x,y, ∈S 1 ∪S 2 d ( x, y ) >
max x,x ′ , ∈S 1 d ( x, x ′ )
+max y,y ′ , ∈S 2 d ( y, y ′ ) , we will have Dia ( S 1 ∪S 2 ) >
Dia ( S 1 ) + Dia ( S 2 ) , violating the right-hand side of the subadditivity inequation, thus proving the
Diameter measure is not subadditive.
For the SumDiameter chemical space measure, we denote it as SD ( · ) for short.
We disprove its subadditivity by proving it violates the right-hand side of the subadditivity, i .
e ., SD ( S 1 ∪ S 2 ) ≤
SD ( S 1 ) + SD ( S 2 ) .
For two disjoint molecular sets with two molecules in each, i .
e ., { x 1 , x 2 } and { x 3 , x 4 } , we denote d ij = d ( x i , x j ) .
Then we have
[Equation]
and
[Equation]
When the inter-set distances are larger than the inner-set distances, i .
e ., d 13 , d 14 , d 23 , d 24 > d 12 , d 34 , we will have SD ( { x 1 , x 2 , x 3 , x 4 } ) >
SD { x 1 , x 2 } + SD { x 3 , x 4 } , violating the right-hand side of the subadditivity, thus proving the SumDiameter measure is not subadditive.
For the Bottleneck chemical space measure, we denote it as Bot ( · ) for short.
We disprove its subadditivity by proving it violates the monotonicity corollary.
Consider adding a molecule x into a molecular set S with size n > 1 .
If x ̸∈ S , we have
[Equation]
When x introduces a more restricting bottleneck, i .
e ., min y d ( x, y ) <
Bot ( S ) , we will have Bot ( S ∪ { x } ) <
Bot ( S ) , violating the monotonicity corollary, thus proving Bottleneck is not subadditive.
For the SumBottleneck chemical space measure, we denote it as SB ( · ) for short.
We disprove its subadditivity by proving it violates the monotonicity corollary.
Consider adding a molecule x into a molecular set S with size n > 1 .
If x ̸∈ S , we have
[Equation]
and
[Equation]
When x introduces some more restricting bottlenecks, i .
e ., for many y ∈ S , d ( x, y ) is small ( e . g ., adding a molecule into a set whose size is two, and the new molecule is added near one of the two molecules), we will have SB ( S ∪ { x } ) <
SB ( S ) , violating the monotonicity corollary, thus proving
SumBottleneck is not subadditive.
Fot the DPP chemical space measure, we disprove its subadditivity by proving it violates the monotonicity corollary.
Consider adding x into { x 0 } where x ̸ = x 0 1 − d ( x, x 0 ) is denoted as b .
We have
[Equation]
When b > 0 we will have DPP ( { x 0 , x } ) <
DPP ( { x 0 } )
= 1 , violating the monotonicity corollary, thus proving DPP is not subadditive.
Proposition B.2 (Dissimilarity property of chemical space measures.) .
Diversity, SumDiversity,
Diameter, Bottleneck, SumBottleneck, DPP, and Circles have preferences to dissimilarity.
The SumDiameter measure does not has the dissimilarity property.
Proof.
In the proof, for adding a molecule x into the molecular set { x 1 , x 2 } , we denote d ( x 1 , x 2 ) as a , d ( x, x 1 ) as δ , and d ( x, x 2 ) as a − δ for short.
For the Diversity chemical space measure, we denote it as Div ( · ) for short.
For the molecular set { x 1 , x 2 } , when adding a new molecule x from X ( x 1 , x 2 ) as defined in the dissimilarity axiom, we have
[Equation]
which means it can take the optimal value at x = x ∗ , thus proving the dissimilarity property of
Diversity.
For the SumDiversity chemical space measure, we denote it as SD ( · ) for short.
For the molecular set { x 1 , x 2 } , when adding a new molecule x ∈ X ( x 1 , x 2 ) , we have
[Equation]
which means it can take the optimal value at x = x ∗ , thus proving the dissimilarity property of
SumDiversity.
For the Diameter chemical space measure, we denote it as Dia ( · ) for short.
For the molecular set { x 1 , x 2 } , when adding a new molecule x ∈ X ( x 1 , x 2 ) , we have
[Equation]
which means it can take the optimal value at x = x ∗ , thus proving the dissimilarity property of
Diameter.
For the Bottleneck chemical space measure, we denote it as Bot ( · ) for short.
For the molecular set { x 1 , x 2 } , when adding a new molecule x ∈ X ( x 1 , x 2 ) , we have
[Equation]
which means it can take the optimal value at x = x ∗ , thus proving the dissimilarity property of
Bottleneck.
For the SumBottleneck chemical space measure, we denote it as SB ( · ) for short.
For the molecular set { x 1 , x 2 } , when adding a new molecule x ∈ X ( x 1 , x 2 ) , we have
[Equation]
which means it can take the optimal value at x = x ∗ , thus proving the dissimilarity property of
SumBottleneck.
For the DPP chemical space measure, for the molecular set { x 1 , x 2 } , when adding a new molecule x ∈ X ( x 1 , x 2 ) , we have
[Equation]
2 , x } )
= (cid:12) (cid:12)(cid:12)(cid:12)(cid:12) 1 − a 1 − δ 1 − a 1 − ( a − δ ) 1 − δ 1 − ( a − δ ) (cid:12) (cid:12)(cid:12)(cid:12)(cid:12) =
(2 a − 4)( δ 2 − aδ ) .
This term will take its largest value when δ = a 2 , i .
e ., when x = x ∗ , thus proving the dissimilarity property of DPP.
For the #Circles chemical space measure, we denote it as #C ( · ) for short.
we denote #Circles ( · ) as #C ( · ) for short and define C ∗ ( S ) ⊆
S as an arbitrary set that satisfies |C ∗ ( S ) | =
#C ( S ) .
We prove the dissimilarity property of #C by contradiction.
For the molecular set { x 1 , x 2 } , consider adding an arbitrary new molecule x ∈ X ( x 1 , x 2 ) or the molecule x ∗ .
We assume #C ( { x 1 , x 2 , x } ) >
#C ( { x 1 , x 2 , x ∗ } ) .
This means that we must have #C ( { x 1 , x 2 , x } )
=
3 and C ∗ ( { x 1 , x 2 , x } )
=
{ x 1 , x 2 , x } , otherwise according to the monotonicity corollary of #C’s subadditivity, #C ( { x 1 , x 2 , x } ) =
#C ( { x 1 , x 2 } ) ≤ #C ( { x 1 , x 2 , x ∗ } ) .
Therefore, according to the definition of #C, we have δ > t as well as a − δ > t where t is the distance threshold, indicating that
[Equation]
This means C ∗ ( { x 1 , x 2 , x ∗ } ) can also simultaneously include x 1 , x 2 , and x ∗ , contradicting with our assumption, thus proving the dissimilarity property of #Circles.
For the SumDiameter chemical space measure, we denote it as SD ( · ) for short.
For the molecular set { x 1 , x 2 } , when adding a new molecule x ∈ X ( x 1 , x 2 ) , we have
[Equation]
which means x can be better than x ∗ when one of δ and a − δ is larger than a 2 , thus disproving the dissimilarity property of SumDiameter.
Discussing the dissimilarity property of reference-based measures.
We denote the cover ( · , · ) function as cov ( · , · ) for short.
Then for the molecular set { x 1 , x 2 } , when adding a new molecule x ∈ X ( x 1 , x 2 ) , we have
Coverage
[Equation]
The value of Coverage ( { x 1 , x 2 , x } ) will depend on the particular definition of cov ( · , · ) .
Generally, an arbitrary coverage function does not necessarily meet the dissimilarity requirement.
In our study, we define this function as cov ( x, y ) :
= I [ molecule x contains fragment y ] , where I [ · ] is the indication function, which is loosely correlated with the distance metric d .
So the dissimilarity property does not hold for chemical space measures #FG, #RS, and #BM.
C R ANDOM S UBSET E XPERIMENT D ETAILS C. B IO -A CTIVITY D ATASET
The 10K BioActivity dataset (Koutsoukas et al., 2014) contains 10,000 compound samples excerpted from the ChEMBL database Gaulton et al.
(2017) with bio-activity labels.
These labels are thelargest ChEMBL activity classes, including enzymes ( e .
g ., proteases, lyases, reductases, hydrolases, and kinases) and membrane receptors ( e .
g ., GPCRs and non-GPCRs).
The label distribution is shown in Figure 5.
We use UMAP (McInnes et al., 2018) to visualize the molecules in this dataset based on their Morgan fingerprints as displayed Figure 6.
From the visualization, we can see that fingerprint similarity is indeed correlated with bio-activity similarity.
C. R ANDOM S UBSETS WITH F IXED S IZES
We first consider randomly sampled molecular subsets of the BioActivity dataset with a fixed size n . We randomly sample n molecules S from the dataset and compute the biological functionality coverage GS ( S ) as well as each chemical space measure µ ( S ) . By repeating the randomization, we can calculate Spearman’s correlation between the gold standard GS and each individual measure µ . We run the experiment for three different fixed sizes n = 50 , n = 200 , and n = 1000 to represent different molecular distribution density.
The pair-wise correlations between measures are displayed in Figure 8. Furthermore, when n = 50 and the molecules are distributed sparsely, all chemical space measures are positively correlated with the gold standard. However, when the subset size increases to n =and n = 1000 , Bottleneck and DPP becomes negatively correlated, while other measure tend to perform better. This is because these two measures will be bounded by the most similar molecular pair, thus severely conflicting with the subadditivity axiom.
In this experiment, we repeat Algorithm 1 for ten times to obtain reliable correlations. Algorithm 1 Calculating chemical space measures for random subsets with fixed sizes. Input: The fixed subset size n ; The bio-activity dataset { ( x i , y i ) } 10 K i =1 where y i ∈ Y are bio-activity labels and |Y| = 50 ; K chemical space measures { µ k } Kk =1 . repeat
Sample a number m uniformly from { 1 , . . .
Sample m labels Y ′ uniformly from Y .
Sample n molecules S with labels in Y ′ uniformly.
Compute GS ( S ) and µ k ( S ) for k ∈ [ K ] .
until repeated for 1000 times
Calculate the correlations between GS and { µ k } Kk =1 based on the 1000-times experiment results.
Experiment results
We show the experiment results for different fixed random set size n in
Figure 7.
We find the #Circles and SumBottleneck measures perform constantly better than all other measures.
When the fixed size n increases, most chemical space measures’ performances also increase, except for Bottleneck and DPP, meaning they are not suitable for measuring the variety when the molecules are distributed crowdedly.
Correlation between chemical space measures.
We also visualize the pairwise correlation between chemical space measures in Figure 8.
From the figure we can see that, the gold standard GS, #Circles, and SumBottleneck are most similar with each other in the fixed-size setting.
Threshold t for #Circles.
The #Circles threshold t is selected to maximize the correlation to the gold standard GS.
Taking n = 200 as an example, we test different t values as Figure 9 displays and select t = 0 .
70 as the threshold.
We can see #Circles works well for a wide range of thresholds like [0 . 40 , 0 . 70] .
In Olivecrona et al.
, the authors suggest to use a threshold t = 0 .
60 to decide whether two molecules are dissimilar with each other 3 , which aligns our results.
For n = 50 and n = 1000 , the threshold is set as t = 0 .
70 and t = 0 .
65 respectively.
Distance metric d .
We also study the impact of distance metric d .
In Table 3, we listed the experiment results for both fingerprint-based Tanimoto distance and VAE-based latent space dissimilarity (Samanta et al., 2020).
We find the experiment results obtained with the VAE dissimilarity remain consistent with the results obtained with the Tanimoto distance. [Figure] SumDiversityDiameterSumDiameterBottleneckSumBottleneckDPP [Figure]
C. R ANDOM S UBSETS WITH G ROWING S IZES
To mimic the molecular generation process, we also grow the size of the subsets.
Specifically, for a maximum size n , we sequentially sample n molecules without replacement to form n subsets {S i = { x 1 , . . . , x i }} ni =1 . For both the gold standard GS and a chemical space measure µ , we record their values as S grows into a time series. , e .
g .
, { ( i, µ ( S i )) } ni =1 .
Comparing the trajectory of a chemical space measure with the trajectory of the gold standard, we can observe which measure behaves more similarly to GS.
We quantitatively estimate the similarity of their trajectories with the dynamic time warping (DTW) distance of the two time series.
In this experiment, we repeat Algorithm 2 for ten times to obtain reliable DTW distances.
Experiment results.
To mimic the way in which generation models propose new molecules, in Algorithm 2, we require the newly sampled molecule x i to be similar to the already sampled
D PP
B o tt l e n e c k
R i c hn e ss
D i a m e t e r
S u m
D i a m e t e r # F G
D i v e r
s
i
t y
S u m
D i v e r
s
i
t
y G S
S u m
B o tt l e n e c k
# C i r c l e s # R S
# B M
DPP
Bottleneck
Richness
Diameter
SumDiameter #FG
Diversity
SumDiversity
GS
SumBottleneck #Circles #RS #BM
[Figure]
[Figure]
molecules { x 1 , . . .
, x i − 1 } .
The specific implementation can be found in our code 4 .
Moreover, we also test the following two cases: (1) All molecules are sampled uniformly; (2) The newly sampled molecule x i have to be most similar to the already sampled molecules { x 1 , . . . , x i − 1 } . The results of DTW distances for these two cases are shown in Figure 10.
[Table]
Input: The maximum subset size n ; The bio-activity dataset { ( x i , y i ) } 10 K i =1 where y i ∈ Y are bio-activity labels and |Y| = 50 ; K chemical space measures { µ k } Kk =1 .
Sample a number m uniformly from { 1 , . . .
Sample m labels Y ′ from Y .
for i in { 1 , . . . , n } do
Sample an unseen molecule x i whose label is in Y ′ .
Set S i := { x 1 , . . .
, x i }
Compute GS (
Transform the cumulative curves into incremental ones.
Calculate DTW distances between incremental curves.
From Figure 10 we can see that, #Circles performs the best.
Also, as the new molecules to add become more similar to the existing ones, the advantage of #Circles over other measures becomes larger.
This makes #Circles especially suitable for measuring molecular generation models.
[Figure]
DTW distances between chemical space measures.
We visualize the pairwise DTW distances between chemical space measures in Figure 11.
From the figure we can see that, the gold standard GS and the #Circles measure are most similar with each other in the growing-size setting.
In addition, we find the Richness, SumDiversity, SumDiameter, SumBottleneck, and #BM are forming a large cluster, while #FG and #RS tend to be similar with each other.
[Figure]
i t
[Figure]
D i
[Figure]
m e t e r
Bottleneck
DPP
Richness
SumDiversity
SumDiameter
SumBottleneck #BM
GS #Circles #FG #RS
Diversity
Diameter
[Figure] [Figure]
results are obtained by running experiments independently for
Threshold t for #Circles.
The #Circles threshold t is selected to minimize the DTW distance to the gold standard GS.
Taking the second scenario as an example, we test different t values as
Figure 12 displays and select t = 0 .
76 as the threshold.
For the first and the third scenarios, the threshold is set as t = 0 .
84 and t = 0 .
78 respectively.
Distance metric d .
We also study the impact of distance metric d .
In Table 4, we listed the experiment results for both fingerprint-based Tanimoto distance and VAE-based latent space dissimilarity (Samanta et al., 2020).
We find the experiment results obtained with the VAE dissimilarity remain consistent with the results obtained with the Tanimoto distance.
[Table]
D M EASURING M OLECULAR D ATABASES D. M OLECULAR D ATABASES
We measure the chemical space coverage for five molecular databases that are commonly used in virtual screening and generative model training:
(1) ZINC-250k 5 is a random subset of the ZINC database (Irwin & Shoichet, 2005) and consists of 249K commercially-available compounds from different vendors for virtual screening;
(2) MOSES (Polykovskiy et al., 2020) is another sub-collection of ZINC molecules.
It contains approximately 2M molecules in total, filtered by molecular weights (ranged range fromto 350 Daltons), the number of rotatable bonds (not greater than 7), water-octanol partition coefficient (logP, less or equal than 3.5), atom types (C, N, S, O, F, Cl, Br, and H), ring cycle sizes (no larger than 8), medicinal chemistry filters (MCFs), and PAINS filters.
(3) ChEMBL (Gaulton et al., 2017) is a manually curated database of 2M bioactive molecules with known experimental data, especially with the inhibitory and binding properties against macromolecule targets.
This database contains not only drug and drug-like molecules, but also natural products and biopolymers.
(4) GDB-17 (Ruddigkeit et al., 2012) enumerates chemical structures that contain up to 17 atoms of C, N, O, S, and halogens, overlapping with the molecular weight range typical for lead compounds.
The lead-like subset filters 11M compounds with lead-like properties (100-350 MW & 1-3 clogP).
(5) Enamine is a company that provides compound libraries for high-throughput screening (HTS). The Enamine Hit Locator Library 6 is their largest diversity library with high MedChem tractability, and the compounds in the library are readily available for purchase and are guaranteed synthesizable at a reasonable cost. D. R ESULTS ON C HEMICAL S PACE M EASURES [Figure]
The measuring results are listed in Figure 13 and 14.
5 ZINC-250k: https://www.kaggle.com/datasets/1379f2461e75ef7a11d0c5ff3dd0c2 440a6bcf33531bc0d303e8eac81a3a4b .
6 ENAMINE diversity libraries: https://enamine.net/compound-libraries/diversity-l ibraries . [Figure] E M OLECULAR G ENERATION E XPERIMENT D ETAILS E. M ODEL I MPLEMENTATION We implement the models with the official GitHub repositories 7 8 9 10 The hyperparameters are set as default.
E. MARS V ARIANTS
Specifically, the terms incorporated into objectives are computed as follows 11 :
[Equation]
Novelty
[Figure]
[Equation]
Novelty [Figure] [Equation] For the model variants, we test different α values from { 0 . 1 , 0 . 3 , 1 . 0 , 3 . 0 } and report the best performance.
Specifically, we use α = 1 . 0 for MARS+Diversity, α = 0 .
3 for MARS+SumBottlenck, and α = 0 .
1 for MARS+#Circles.
The threshold we use for the #Circles measure is t = 0 .
E. R ESULTS ON O THER M EASURES
The results on other chemical space measures are listed in Table 5.
E. V ISUALIZATION
To provide a more intuitive view of the effect that adding a joint objective of exploration can encourage the molecular generation model to discover a wider space, we visualize the chemical space explored by the baseline (MARS) and MARS+SumBottleneck as well as MARS+#Circles in Figure 15, where we show the 2D layout of the Morgan fingerprints of the functional groups in the molecules generated through principal component analysis (PCA).
The larger number of unique functional groups and the Equations 4-6 are approximations of [ µ ( S ∪ { x } ) − µ ( S )] defined to avoid numerical issues and for computational efficiency.
[Table]
wider spread of the data points obtained by adding exploration-based novelty terms indicate more diverse structures being explored.
[Table] We show the molecular clustering results in Figure 16.
Clusters are calculated based on Morgan fingerprints of the generated molecules and their Tanimoto similarity.
Compared to the baseline model, a larger number of clusters can be obtained from MARS+SumBottleneck.
MARS
[Figure]