Assessing multi-objective optimization of molecules with genetic algorithms against relevant baselines
Abstract
Chemical design is often complex, requiring the optimal trade-off between several competing objectives.
Multi-objective optimization algorithms are designed to optimally balance multiple objectives, but many chemical design approaches use the naïve weighted sum method, which is not guaranteed to give desired solutions.
Here, we rigorously assess the performance of genetic algorithms for inverse molecular design using more advanced multi-objective methods.
CHIMERA and Hypervolume are assessed against relevant baselines for the optimization of molecules with high logP and high QED score.
As a more realistic task, we also simulate a drug design campaign, optimizing for synthetically accessible molecules which bind to the 1OYT protein.
Additionally, we include a three-objective task of optimizing logP, QED and SAS to investigate scalability to more than two objectives.
We show that both methods achieve better formal optimality than the baselines and generate molecules closer to a user-specified Utopian point in property space, mimicking typical materials design objectives.
Introduction
Advances in machine learning (ML) and artificial intelligence (AI) have the potential to transform chemical discovery into an inverse design problem, where molecules are generated and optimized with desired properties at an accelerated rate [1].
In the context of data-driven materials discovery,
AI-guided chemical design can efficiently explore chemical space while improving performance based on experimental feedback.
Important application areas are drug discovery [2], catalyst design [3] and materials discovery [4].
Examples of inverse design algorithms based on deep learning (DL) include generative adversarial networks (GANs) [5, 6], variational autoencoders (VAEs) [7] and reinforcement learning [8, 9], among other approaches [1].
An older family of inverse design methods are genetic algorithms (GAs) [10–12], which have been applied in the context of chemical design for decades [13–17].
Despite being older, GAs often out-perform DL approaches on benchmarks [18–20].
In particular, GAs are preferred in design campaigns where there exists insufficient data to train DL models [21], which generally require datasets on the order of 10 4 –10 5 samples or more [22].
Efforts are underway to combine GAs with DL to leverage the best of both worlds [23, 24].
GAs mimic natural evolution to optimize a population of molecules against selection pressure based on a fitness function [10–12].
The fitness function is chosen to mimic the design objectives.
Due to the complexity of molecular design, a single objective ( single-objective optimization ) cannot cover most multi-faceted design problems. Therefore, strategies are needed that can optimize for [Figure] ond
[Figure]
j
[Figure] c t i v e [Figure] multiple, potentially competing objectives simultaneously ( multi-objective optimization ).
An example of such a trade-off is finding drug molecules that should both bind strongly to their intended protein target, and be synthetically accessible [25].
The goal of multi-objective optimization strategies is to find Pareto-optimal solutions [26, 27], for which improvement on none of the objectives is possible without simultaneously deteriorating at least one other objective.
Multi-objective optimization with
GAs [28, 29, 24] have generally used variations on the weighted sum ( WeightedSum ) or product methods [30, 31].
Although WeightedSum is easy to implement, it is difficult to define weights which yield desired solutions.
Approaches that use more advanced multi-objective optimization algorithms often do not compare to adequate baselines [17, 32, 3, 33].
In this paper, we implement multi-objective optimization for the JANUS genetic algorithm [24] with achievement scalarizing functions (ASFs) implemented in the OLYMPUS benchmarking framework [34].
ASFs define a single cumulative value from the set of all objectives and encode user preferences about a multi-objective optimization problem such that their optimal solution ideally corresponds to the desired Pareto-optimal solution.
We conduct three inverse design tasks related to drug design and show that the more advanced multi-objective scalarizers improve upon the simple WeightedSum approach.
Methodology
In JANUS , the fitness of a molecule is determined by a user-defined function based on the molecular properties of interest, which returns a scalar value.
JANUS operates on the SELFIES representation of molecules in order to perform mutations and crossovers, and maintains parallel exploration and exploitation populations for optimization [35–37].
More details are available in the literature [24, 38].
We perform multi-objective optimization on three tasks: (1) a drug lipophilicity-related task, in which we maximize the quantitative estimate of drug-likeness (QED) [39] and the logarithm of the water-octanol partition function (logP), both calculated using RDKit [40]; (2) a docking task, in which we minimize both the docking score to the human 1OYT protein (calculated with SMINA [41]) and the synthetic accessibility score (SAS) [42]; and (3) a three-objective task in which we maximize QED and logP, and minimize SAS.
For our implementation of multi-objective optimization, an arbitrary number of fitness functions are scalarized according to a specified recipe to output a single cumulative fitness value for each molecule.
Molecules are then sorted based on these scalarized values, which are used to assess their viability for propagation.
We use two baseline approaches and two more advanced scalarizer approaches and assess their performance according to three metrics: (1) the hypervolume indicator and (2) the R2 indicator of the generated Pareto front, the former of which balances assessment of the molecules’ proximity to the Pareto front, their diversity, and spread [43, 44] (see sections A.2.2, A.2.4 for further details), the latter assessing proximity of the Pareto front to the best values achieved across all runs [45, 46], and (3) the minimum distance to a pre-selected optimal goal, commonly referred to as the Utopian point [47] (Figure 1A).
2.1 Optimization methods
We consider three ASFs in our experiments: WeightedSum , Hypervolume , and CHIMERA .
WeightedSum constitutes a baseline approach considering its typical, simple usage for multi-objective optimization.
The weight vector is set as the ratio of fitness values described by the Utopian point, and is elaborated upon in Appendix Sec.
A.2.1.
Hypervolume aims to find solutions that maximize the dominated hypervolume indicator.
CHIMERA is a lexicographic ASF which organizes objectives in a hierarchy of importance and constructs a cumulative function according to user-specified tolerances on individual objectives.
Tolerances for CHIMERA were set based on the Utopian point coordinates.
For the threeobjective experiment, two CHIMERA configurations were used, both of which prioritized achievement of the QED Utopian value first, but differed in their prioritisation of the remaining two objectives.
As the Utopian value for SAS (1.0) was observed to be more accessible than the value for logP (10.0), the latter configuration aims to demonstrate CHIMERA ’s ability to encode expert knowledge in its hyperparameters to more intelligently solve an inverse design task.
As compared to WeightedSum , setting these tolerances does not require knowledge of the relative scale of the objectives, which is not readily available without a large dataset like ZINC .
More information on the ASFs used in this work is given in Appendix Sec.
A.2.
We also formulate a Random baseline, in which fitness values for each generated structure are sampled randomly from [0 , 1] .
This approach mimics a random generation of structures as all molecules have equal chance to proceed to the next generation.
2.2 Experiments
For the experiments, each approach was initialized with a starting population of 200 molecules which were allowed to evolve for 100, 50 and 100 generations for the lipophilicity, docking and three-objective tasks, respectively.
To curate the starting population, the objectives for each task were computed for a reduced ∼ 250,000 molecule subset of the ZINC dataset [48] following GómezBombarelli et al.
[7], here dubbed ZINC red .
The starting population was chosen to represent the worst part of the dataset – hypervolume values in the lowest 10th percentile – to make the optimization more difficult.
For each task, the optimization campaign was repeated ten times, the data produced being analysed for the metrics listed below, and we report both the means and standard deviations across the runs.
2.2.1 Hypervolume and R2 Distance of Pareto front
In this analysis, all molecules generated by JANUS for each run were collected, along with the associated objective values.
The Pareto front was generated and the hypervolume and R2 distance of the Pareto region were calculated using OLYMPUS .
The worst objective values in the original ZINC red dataset were used as a reference point for calculating the hypervolume, while the best objective values across all runs was used as a reference point for calculating the R2 distance.
As a baseline, we generated the Pareto front for the reduced ZINC red dataset (Figure 1B) and computed its hypervolume (Table S1) and R2 (Table S2).
Approaches that generate Pareto fronts with larger hypervolumes and smaller R2 distances are generally considered better at multi-objective optimization for a particular problem.
2.2.2 Distance to Utopian point
Based on the Pareto front extracted from the reduced ZINC red dataset (Fig 1B), a Utopian point outside of the front was selected by visual inspection.
The Utopian points were QED of 0.6 with logP of 10, and SAS value of 1.0 with docking score of -20 for the lipophilicity and docking tasks, respectively. For the three-objective task, the Utopian point was a QED of 0.6 with a logP of 10 and a SAS of 1.0.
These can be seen as a reasonably challenging design goals which improve upon the existing dataset.
For a maximization problem with n objective functions, the distance between a generated molecule’s objective values and the Utopian point is computed as
[Equation]
[Figure] where y ∈ R n and y 0 ∈ R n are the objective space coordinates of the generated molecule and
Utopian point, respectively.
The min function is used in place of max for a minimization problem.
With this metric, objective values that are as good or better than the Utopian point are considered equally advantageous.
Results and Discussion
The results for the three tasks are shown in Figure 2. We start by looking at the results for distance to the Utopian point.
For the lipophilicity task, CHIMERA allows for optimization to be guided most quickly and closely to the specified Utopian point (Figure 2A-B), validating CHIMERA ’s use of a hierarchy of objectives and specified threshold.
Importantly, CHIMERA determines when one objective is satisfied and can then focus on the remaining objectives, while the other algorithms would continue optimization of individual objectives beyond the Utopian point.
Furthermore, the results over ten runs for CHIMERA produced the lowest variance.
The Hypervolume approach gives a comparable distance, albeit with higher variance.
In the docking task, the differences are less pronounced, with WeightedSum and CHIMERA achieving similar minimal distances to the Utopian point, on average (Figure 2D-E).
This indicates that appropriate weights for WeightedSum were chosen for this task.
However, as observed in the lipophilicity task, the same method of selecting weights does not guarantee good performance.
To test whether the results scale to additional objectives, we optimized for three objectives QED, logP and SAS at the same time. For this three-objective task, it is less obvious which optimization method produces molecules closer to the Utopian point (Figure 2G-H). While CHIMERA and Hypervolume perform better than the naive WeightedSum and Random methods on average, the variances overlap significantly. It is worth noting that in contrast to the previous tasks, the Random method performs comparably.
However, this is contextualized by the Random method still having a significantly smaller Pareto front hypervolume (Figure 2I), suggesting that the method is simply able to serendipitously find a few molecules relatively near to the Utopian point, rather than a reliable set of viable molecules.
We decided to test if we could improve the optimization of CHIMERA based on the results of the first run. In the first CHIMERA configuration, the objectives were prioritized to (1) maximize QED beyond 0.6, (2) maximize logP beyond 8, and finally (3) allow SAS to minimize without constraint. In contrast, the second CHIMERA configuration incorporated knowledge gained from the first run.
Understanding that the minimum possible SAS value was 1, and that SAS values near this minimum were easily attained during the first run, the objectives were re-prioritized to (1) maximize QED beyond 0.6, (2) minimize SAS below 1.1, and then (3) allow unconstrained maximization of logP. While attempts to continually minimize SAS beyond the minimum possible value could potentially inhibit optimization of the remaining two objectives, this configuration afforded a more intelligent and consistent optimization, as evidenced by the closer distance to the Utopian point achieved, and the narrow standard deviation across ten runs. We now turn to evaluating the methods based on the hypervolume and the R2 indicator. For the lipophilicity task, molecules generated by CHIMERA and Hypervolume produced the Pareto fronts with the largest hypervolume on average (Table S1 in the Appendix).
While the best run for Hypervolume gave the Pareto front with the single highest hypervolume, the algorithm overall was less consistent (Figure 2C).
Similarly for the docking task, CHIMERA and Hypervolume achieve a higher average hypervolume than both the WeightedSum and Random approaches (Figure 2F). Remarkably, in both tasks, CHIMERA seems to be competitive with the Hypervolume scalarizing strategy, which explicitly tries to maximize the hypervolume itself.
Both CHIMERA and Hypervolume produced significantly larger hypervolumes than the two baseline approaches WeightedSum and Random.
For the three-objective task, all methods except for Random generally improve upon the ZINC red dataset (Figure 2I).
In contrast to the previous tasks with two objectives, Hypervolume performs better than both CHIMERA methods. Furthermore, between the two CHIMERA configurations, the latter – which incorporates property space knowledge to more precisely guide optimization – produces Pareto fronts with smaller hypervolumes, as well as a smaller standard deviation.
The results for the R2 indicator of the Pareto fronts (Figure S1) tell a similar story to the hypervolumes, but with less disparity between approaches.
These results can be found in Table S2 of the Appendix. A sample of the molecular structures generated by each run can be found in the Appendix, Section A.5.
Figure S2 compares the molecular diversity of the Pareto sets identified by each strategy.
Unsurprisingly, Pareto sets generated using JANUS with ASFs are typically less diverse than the
Random method, as they constitute targeted inverse design strategies.
Conclusion
In conclusion, we have demonstrated the capacity for multi-objective optimization with ASFs to guide molecular evolutionary algorithms towards a fuller Pareto frontier and greater proximity to a specified Utopian point.
Importantly, we show that the two most advanced scalarizers, CHIMERA and Hypervolume , outperform adequate baseline methods, the naïve WeightedSum approach and randomly generated molecules.
These results show that the more advanced methods better position researchers to solve multi-faceted inverse molecular design problems with two or more design objectives.
In particular, we find CHIMERA and Hypervolume to be most suitable for design tasks with multiple, potentially competing objectives.
In upcoming work, we plan to use molecular similarity metrics to further enhance the rate at which JANUS can produce desirable multi-objective optimization solutions.
Acknowledgments and Disclosure of Funding
R.J.H. gratefully acknowledges the Natural Sciences and Engineering Research Council of Canada (NSERC) for provision of the Postgraduate Scholarships-Doctoral (PGS-D)
Program
(PGSD3534584-2019) as well as support from the Vector Institute.
G
T. acknowledges NSERC for support through the PGS-D program (PGSD3-559078-2021). AK.N. acknowledges funding from the BioX Stanford Interdisciplinary Graduate Fellowship (SGIF).
K.J. acknowledges funding through an
International Postdoc grant from the Swedish Research Council (No. 2020-00314).
A.A.-G. thanks
Anders G. Frøseth for his generous support.
A.A.-G. acknowledges the generous support of Natural
Resources Canada and the Canada 150 Research Chairs program.
Some computations were performed on the Béluga and Narval supercomputers situated at the École de technologie supérieure in Montreal.
A.A.-G. is co-founder and Chief Visionary Officer of Kebotix, Inc.
A Appendix A.1 Supplemental code
All code written for this paper, including implementation of multi-objective optimisation and relevant results can be found at https://github.com/natkusanda/multiobjective-janus .
The code is licensed under Apache 2.0.
This code builds upon the original JANUS code which is also made available under Apache 2.0 license and can be found at https://github.com/ aspuru-guzik-group/JANUS .
ZINC is free to use for everyone.
See license at https://wiki. docking.org/index.php/UCSF_ZINC_License .
The reduced version of ZINC was downloaded from https://github.com/aspuru-guzik-group/chemical_vae
Please note that an expanded mutation alphabet was used to increase the presence of ring groups in all runs and approaches.
This alphabet can be found in /janus_chimera/janus/mutate.py in the repository linked above and the ring
A.2 Details of achievement scalarizing functions
In this section, we formally define each of the achievement scalarizing functions (ASFs) used in this work.
For the purpose of this discussion, we consider a molecular design task which features a set of n objective functions, f = { f i } ni =1 , to be minimized concurrently.
For molecule x , noiseless objective values are obtained as y i =
f i ( x ) .
Furthermore, we assume a dataset of objective values D
= { y i } Ki =
corresponding to K generated molecular candidates has been collected by our optimization procedure.
A.2.1 WeightedSum
Weighted sum ASFs maps multiple objectives onto a cumulative scalar objective using a vector of weights w ∈ R n to produce the weighted sum
[Equation]
Here, we select each weight w i according to the ratio specified by the Utopian point, scaling the relative magnitude of the i th objective.
As the Utopian point was specified as 0.6 for QED and 10 for logP, the weights were selected as 10/0.6 = 16.66 for the former objective, and 1 for the latter.
As the same experimental data was used for both analyses, these weights were reflected in the Hypervolume of Pareto front analyses as well.
A.2.2 Hypervolume indicator
The hypervolume indicator is an example of a set-quality indicator, which facilitate assessment of
Pareto fronts by summarizing their characteristics (such as proximity to the Pareto front, diversity and spread) with a single scalar value.
Owing to its ease of interpretation, hypervolume is one of the most widely employed set-quality indicators [44].
The hypervolume indicator maps a set of objective values D to a measure of the region dominated by that set and bounded above by some reference point r ∈ R n .
Intuitively, the indicator provides a notion of the size of the covered objective space or the size of the dominated space [50, 51, 44, 52]
Formally, the hypervolume indicator H given a dataset of objective value measurements D is
[Equation]
where Λ is the Lebesgue measure.
Here, for two objective space points p ∈ R n and q ∈ R n , the expression p ≤ q is used to indicate that p weakly dominates q , that is p i ≤ q i ∀ 1 ≤ i ≤ n .
H can also be described as the union of hyperrectangles
[Equation]
where [ p , r ] represents the hyperrectangle fixed from above by reference point r and below by p .
In most cases (including this work), the hypervolume indicator is used as an analysis tool to assess the quality and diversity of a Pareto set after an optimiziation campaign has transpired.
However, it can also be used as an ASF to which aims to find solutions which maximize the dominated hypervolume.
In this case, the solutions in D are considered one at a time, which simplifies the calculation of H to the volume of the hyperrectangle with corners at y and r ,
[Equation]
Importantly, using the hypervolume indicator as a ASF this way produces a constraint that y ≤ r .
As such, we update r at each iteration such that each of its elements correspond to the maximum observed value for that objective in the optimization history D .
A.2.3 Chimera
CHIMERA is an achievement scalarizing function which combines a priori scalarizing with lexicographic approaches, and is created for optimization problems where the objective is expensive to evaluate.
CHIMERA allows users to organize multiple objectives into a hierarchy, i.e., f = { f i } ni =1 is replaced by f = ( f 1 , . . .
, f n ) ordered according to a descending hierarchy of importance ( f 1 is more important than f 2 , f 2 than f 3 , and so on).
User-specified tolerance values y tol are provided to CHIMERA which indicate objective value thresholds at which the user is satisfied (e.g., one may want to generate a molecule with a QED of at least 0.6, before prioritizing optimizing its lipophilicity).
Whether or not some measured value of the i th objective function y i satisfies its corresponding tolerance can be indicated by the Heaviside function, Θ (cid:0) y toli − y i (cid:1)
=
if y i ≥ y toli and 1 if y i < y toli .
Alternatively, discontinuities in the cumulative function can be avoided by utilizing a smooth logistic function approximation to Θ , parameterized by smoothing parameter τ ,
[Equation]
CHIMERA constructs an ASF using approximate Heaviside functions to weight objective functions f .
Importantly, the resulting ASF is sensitive to only one objective function at a time in any given parameter space region.
As such, objective values y i are shifted based on the minimum of y i − 1 (the next most important objective function in f ) in the parameter space regions where y i − 1 does not satisfy its corresponding tolerance y toli .
This minimum value is denoted y mini − 1 .
CHIMERA , χ ( D ; y tol ) is then formulated as
[Equation]
where θ + i and θ − i are used to abbreviate θ (cid:0) y toli − y i (cid:1) and θ (cid:0)
y i − y toli (cid:1) , respectively.
We set the smoothing parameter τ = 0 .
001 in all our experiments.
The tolerances for the lipophilicity task were 0.6 for QED, and 1000 for logP, which entails unconstrained maximization of logP once the tolerance for QED is fulfilled.
For the docking task, the tolerances were 2.0 for SAS and -1000 for docking score. A.2.4 R2 indicator
Originally proposed to assess the relative quality of two sets, the R2 indicator [45, 46] can be used to determine the quality of a single set D against some reference point r .
The Tchebycheff scalarizing function is used as the utility function [53].
[Equation]
W is a set of weight vectors w ∈ R n , and p is a probability distribution on W .
Here, the weight vectors are sampled uniformly from the objective space. The reference point r must not be dominated by any possible solution, and is chosen a parameter set by the user (often chosen to be a Utopian point). Eq. 8 can be written as
[Equation]
A smaller R 2 value indicates that the set D is closer to the reference point, and is bounded from below by 0.
Similar to the hypervolume indicator, the R 2 indicator can also be used as and ASF which aims to find solutions which minimize the distance to r .
In this case, solutions in D are considered one-by-one, which simplifies the computation to
[Figure]
[Equation]
In the case of using the R2 indicator as an ASF
[Table]
the reference point r is updated in light of new measurements.
At a given iteration
[Table]
we set r to contain the cumulative best objective function values in D for each objective, i [Table] e., r i = min y ∈D y i for each of i = 1 , . . . , n .
A.3 Additional details of optimization experiments
[Table]
[Table]
A.4 Analysis of
and sets A.4.1 Molecular diversity of generated Pareto sets
In this section, we compare the molecular diversity of the Pareto sets generated by each strategy.
Diversity is quantified using
[Equation]
where n is the cardnality of the Pareto set and Sim ( · ) is a molecular similarity metric, in this case the Tanimoto similarity calculated using Morgan fingerprints [54].
The summation is over all unique pairs of Pareto set molecules [55].
Figure S2 shows the distributions diversity values for the Pareto sets identified by each strategy.
[Figure] [Figure] [Figure]