An Empirical Evaluation of Zeroth-Order Optimization Methods on AI-driven Molecule Optimization
Anonymous Author(s)
AffiliationAddress email
Abstract
Molecule optimization is an important problem in chemical discovery and has beenapproached using many techniques, including generative modeling, reinforcementlearning, genetic algorithms, and much more.
Recent work has also applied zerothorder (ZO) optimization, a subset of gradient-free optimization that solves problemssimilarly to gradient-based methods, for optimizing latent vector representationsfrom an autoencoder.
In this paper, we study the effectiveness of various ZOoptimization methods for optimizing molecular objectives, which are characterizedby variable smoothness, infrequent optima, and other challenges.
We provideinsights on the robustness of various ZO optimizers in this setting, show theadvantages of ZO sign-based gradient descent (ZO-signGD), discuss how ZOoptimization can be used practically in realistic discovery tasks, and demonstratethe potential effectiveness of ZO optimization methods on widely used benchmarktasks from the Guacamol suite.
Introduction
The goal of molecule optimization is to efficiently find molecules possessing desirable chemicalproperties.
As the ability to effectively solve difficult molecule optimization tasks would greatlyaccelerate the discovery of promising drug candidates and decrease the immense resources necessaryfor drug development, significant efforts have been dedicated to designing molecule optimizationalgorithms leveraging a variety of techniques, including deep reinforcement learning [1], geneticalgorithms [2], Bayesian optimization [3], variational autoencoders [4, 5], and more.
Several moleculeoptimization benchmark tasks have also been proposed, including similarity-based oracles [6] anddocking scores [7].
In this paper, we extend the work of Hoffman et al.
[8], who proposed the use of ZO optimizationin their query-based molecule optimization (QMO) framework, an end-to-end framework whichdecouples molecule representation learning and property prediction.
QMO iteratively optimizesa starting molecule, making it well suited for lead optimization tasks, but it can also start fromrandom points and traverse large distances to find optimal molecules.
In comparison with the workof Hoffman et al.
[8] which experiments with only one optimizer, we experiment with variations of QMO using different ZO optimizers. Furthermore, we add more benchmark tasks from Guacamol[6] (whose use has been encouraged by the molecule optimization community [3, 9] and used by Gaoet al.
[10] to benchmark many design algorithms in a standardized setting), and provide insights intothe challenges of ZO optimization on molecular objectives.
Contributions We evaluate several ZO optimization methods for the problem of molecule optimization in terms of convergence speed, convergence accuracy, and robustness to the unusual functionlandscapes (described further in Section 2.4) of molecular objectives.
We show that ZO-signGD [11]outperforms other methods for molecule optimization in not only speed but also accuracy, despitebeing known to have worse convergence accuracy for other problems like adversarial attacks [11],which indicates that the sign operation is potentially more robust to the function landscapes ofmolecular objectives.
Furthermore, we provide insights into practical application of ZO optimizationin drug discovery scenarios for both lead optimization tasks and the discovery of novel molecules, aswell as propose the use of a hybrid approach combining others models with QMO.
Related work ZO optimization is a class of methods used for solving black-box problems byestimating gradients using only zeroth-order function evaluations and performing iterative updates asin first-order methods like gradient descent (GD) [12].
Many types of ZO optimization algorithmshave been developed, including ZO gradient descent (ZO-GD) [13], ZO-signGD [11], ZO adaptivemomentum method (ZO-AdaMM, or ZO-Adam specifically for the Adam variant) [14], and more[15, 16].
The optimality of ZO optimization methods has also been studied under given problemassumptions [17].
ZO optimization methods have achieved impressive feats in adversarial machinelearning, where they have been used for adversarial example generation in black-box settings anddemonstrated comparable success to first-order white-box attacks [18, 19].
They have also beenshown to be able to generate contrastive explanations for black-box models [20].
Finally, Hoffmanet al.
[8] showed how ZO optimization methods can also be applied to molecule optimization withtheir QMO framework.
QMO: Background and Variations2.1 QMO framework
Following the QMO framework by Hoffman et al.
[8], we use an autoencoder to encode moleculeswith encoder E : X (cid:55)→ R d and decode latent vectors with decoder D : R d (cid:55)→ X , where X denotesthe discrete chemical space of all drug candidates.
We denote with O : X (cid:55)→ R a black-box oraclereturning a scalar corresponding to a molecular property of interest (which may also be modifiedby adding losses related to other properties), and for ease of notation with the QMO framework,we define our optimization objective loss function as f ( z ) =
−O ( D ( z )) for latent representationsz ∈ R d .
As each function query f ( z ) queries the oracle O with the decoded molecule correspondingto z , one function query is equivalent to one oracle query.
In QMO, we use ZO optimization methods to navigate the latent space to solve min z f ( z ) .
Specifically,given a starting molecule and its latent representation z 0 , we iteratively update the current latentrepresentation following some optimizer, as done in first-order gradient-based methods like gradientdescent.
But as we do not have any first-order oracle, we instead use gradients estimated using onlyevaluations of f following some gradient estimator.
The QMO framework, which closely follows ageneric ZO optimization procedure, is summarized in Algorithm 1.
1: Inputs: Starting molecule x 0 ∈ X , encoder E , decoder D , gradient estimation operation estimate _ gradient f ( · ) for function f , optimizer updating operation update( · ) , number of iterations T , and learning rate α 2: Z iterate ← {∅} 3: z 0 ← E ( x 0 ) 4: for t = 0 , 1 , ..., T − 1 do 5: ˆ g t ← estimate _ gradient f ( z t , E, D ) 6: z t +1 ← update( z t , { ˆ g i } ti =1 , α, E, D ) 7:
Z iterate ← Z iterate ∪ { z t +1 } 8: end for
In principle, QMO is a generic framework which can guide searches over any continuous learnedrepresentation based on any discrete space and use any ZO optimization method.
Hoffman et al.[8] used the pre-trained SMILES-based [21] autoencoder (CDDD model) from Winter et al.
[22]with embedding dimension d = 512 and ZO-Adam.
Here, we use the same autoencoder butconsider several variations of QMO using different gradient estimators and optimizers to provide acomprehensive study on the effect of ZO optimization methods.
As a note, QMO is applicable to molecule optimization with design constraints.
For example, given aset of property scores { p i } Ii =1 to be optimized with positive coefficients { γ i } Ii =1 and a set of propertyconstraints { c j } Jj =1 with thresholds { η j } Jj =1 , we can define the oracle as
[Equation]
where x ∈ X .
The vectors z ∈ Z iterate not satisfying c j ( D ( z )) ≥ η j for all j ∈ { 1 , 2 , ..., J } canthen be removed from Z iterate .
2.2 ZO gradient estimators
We consider two main ZO gradient estimators.
Both average finite differences over Q independentlysampled random perturbations { u q } Qq =1 , include a smoothing parameter β , and follow the form:
[Equation]
The two gradient estimators differ mainly on the sampling method for each random direction u q , andalso by the dimension-dependent factor φ ( d ) .
They are:
• Gaussian smoothing (GS) [13, 23]: when we sample each direction from the uniformdistribution U ( S (0 , 1)) on the unit sphere.
For GS, φ ( d ) = d .
• Bernoulli smoothing-shrinkage (BeS-shrink) [24]: when we craft each random direction
by independently sampling each of its d entries from ( B 0 .
5 − 0 . 5) /m , where B 0 .
5 followsthe the Bernoulli distribution with probability 0 .
5 and m = (cid:112) Q + d − 1 / 4 Q is an optimalshrinking factor.
For BeS-shrink, φ ( d ) = 1 .
By averaging over Q random directions to decrease estimation error, the gradient estimation operationrequires querying Q + 1 different points (which are each decoded into a molecule and used to queryoracle O ).
We therefore require Q + 1 oracle evaluations for each optimization iteration.
Additionally, because the above gradient estimators use a (forward) finite difference of 2 points toestimate the gradient for each random perturbation, we refer to it as a 2-point gradient estimator.
Analternative to the 2-point GS and BeS-shrink gradient estimators are their 1-point alternatives, whichinstead have the form:
[Equation]
Similar to 2-point gradient estimators, 1-point estimators require Q +1 oracle queries at each iteration(the estimation operation itself requires only Q queries, but this does not account for querying theupdated molecule after each iteration).
However, 1-point estimators are not commonly used inpractice due to higher variance.
2.3 ZO optimizers
We consider three main optimizers, with each having its own updating operation that consists ofcomputing a descent direction m t and then updating the current point.
Each optimizer can be pairedwith any ZO gradient estimator.
The three are as follows:
[Table]
• ZO gradient descent (ZO-GD) [13]: analagous to stochastic gradient descent (SGD) in
[Table]
the first-order stochastic setting.
ZO-GD uses the current gradient estimate as the descent
direction m t
=
ˆ g t and updates the current point via the rule
z t +1 =
z t − α m t .
• ZO sign-based gradient descent (ZO-signGD)
[11]: analagous to sign-based SGD
(signSGD) [25] in the first-order stochastic setting.
ZO-signGD uses the same point updating
[Table]
rule as ZO-GD but instead uses the sign of the current estimate as the descent direction
m t =
sign(ˆ g t ) , where sign( · ) denotes the element-wise sign operation.
• ZO-Adam [14]: analagous to Adam [26] in the first-order stochastic setting.
ZO-Adam
adopts a momentum-type descent direction and an adapative learning rate.
The ZO optimization methods compared in this paper are summarized in Table 1.
2.4 Motivating the comparison of ZO optimization methods for molecule optimization
We motivate our comparison of optimizers not only in terms of convergence speed and convergenceaccuracy, but also in terms of robustness to the unfriendly function landscapes of molecular objectives.
Indeed, molecule optimization is made difficult by variable function smoothness due to "activitycliffs" in the molecular space where small structural changes cause large changes in oracle values [9].
As optima are infrequent, there are also large and extremely "flat" unfavorable regions in the space,where the oracle values change minimally and may be very small.
Furthermore, because our objectivefunction f obtains values by querying the oracle O using discrete molecular representations obtainedfrom decoding the latent vectors, the function landscape is made discrete and thus further non-smooth(i.e.
, the function value may have a discrete "jump" at the borders between adjacent regions of latentvectors which decode to different molecules, see Fig. 1).
Thus, being able to effectively navigatethe latent chemical space and not get stuck in unfavorable regions is an important and non-trivialattribute to pursue in optimization methods.
Sign-based gradient descent is known to be effective in achieving fast convergence speed in stochasticsettings: in the stochastic first-order oracle setting, Bernstein et al.
[25] showed that signSGD couldhave faster empirical convergence speed than SGD, and in the zeroth-order stochastic setting Liu et al.[11] similarly showed that ZO-signSGD has faster convergence speed than many ZO optimizationmethods at the cost of worse accuracy (i.e., converging only to the neighborhood of an optima).
Thefast convergence of sign-based methods is motivated by the idea that the sign operation is morerobust to stochastic noise, and though our formulation of molecule optimization is non-stochastic, thesign operation is potentially more robust to the difficult landscapes of molecular objective functions.
Adaptive momentum methods like Adam also make use of the sign of stochastic gradients fordetermining the descent direction in addition to variance adaption [27], and thus ZO-Adam may alsoshow robustness to the function landscape.
Practical Usage of QMO for Drug Discovery
We imagine QMO can be applied for two main cases: 1) identifying novel lead molecules (findingmolecules significantly different from known leads), and 2) lead optimization (finding slightlymodified versions of known leads).
For the former application case, it may be counterproductive to use known leads as the startingmolecule in QMO, as those leads may be in the close neighborhood of a local optima (or a local optimathemselves) in the function landscape, in which the optimizer would likely get stuck (preventing theexploration of different areas of the latent chemical space).
Instead, it may be more promising to startat a random point in the chemical space.
QMO also has the advantage that it guides search withoutthe use of a training set, which aids in finding candidates vastly different from known molecules.
However, finding a highly diverse set of novel leads may be unlikely within a single run of QMO asthe optimization methods converge to some neighborhood, meaning that multiple random restartswould likely be necessary to discover a diverse set of lead molecules.
For the latter application case, it is much more sensible to use known leads as the starting moleculeinput to QMO.
Additionally, rather than using an oracle O evaluating only the main desired drugproperty (i.e., activity against a biological target), it may be advantageous use a modified oracle.
Forexample, Hoffman et al.
[8] apply QMO for lead optimization of known SARS-CoV-2 main proteaseinhibitors and antimicrobial peptides (AMPs) following the constrained molecule optimization settingof (1), with pre-trained property predictors for each task.
They set similarity to the original leadmolecule as the property score p sim to be optimized, and set constraints on properties of interest(binding affinity c aff for the SARS-CoV-2 task, or toxicity prediction value c tox and AMP predictionvalue c AMP for the AMP task).
In these formulations, the main optimization objective is actuallymolecular similarity rather than the main properties of interest.
Hybrid optimization: Integrating QMO with generative models Additionally, in this paperwe propose to apply QMO in a hybrid approach with other models: namely, we can use moleculesgenerated by other models as input to QMO, which will then iteratively optimize each of the inputtedstarting molecules.
By using other models to generate good lead molecules close to optima, we canthen use QMO to provide a more refined search that may incorporate additional design constraints.
Overall, a hybrid approach could be a query efficient way to generate drug candidates satisfyingmultiple design constraints.
Experiments
To benchmark QMO, we select three tasks (oracles) from the popular benchmarking suite Guacamol[6]: perindopril_mpo (finding molecules similar to perindopril
but with a different number of aromaticrings), zaleplon_mpo (finding molecules similar to zaleplon but a different molecular formula), anddeco_hop (maximizing similarity to a SMILES string while keeping or excluding specific SMARTSpatterns).
These represent three of the main categories of Guacamol oracles: similarity-based multiproperty objectives, isomer-based objectives, and SMARTS-based objectives, respectively.
Thezaleplon_mpo task is known to be particularly difficult [9].
We also select two baselines, graph-basedgenetic algorithm (Graph-GA)
[2] and Gaussian process Bayesian optimization (GPBO) [3, 28], bothof which are known to be high performing molecule optimization algorithms [10].
For each of these tasks, we run experiments using QMO only, baselines only, and hybrid approaches.• When running experiments using QMO only, we run with several different ZO optimizationmethods and try Q = { 30 , 50 , 100 } for each, set T = 1000 iterations for perindopril_mpoand zaleplon_mpo or T = 200 for deco_hop, and average runs from 20 distinct startingmolecules with 2 random restarts each (40 runs total).
For each task, we choose to use the20 lowest-scoring molecules on the oracle from the ZINC 250K dataset [29] as the startingmolecules. We do this in order to show that QMO can find solutions even when starting farfrom any high scoring molecules, which we would likely need to do when searching fornovel lead molecules.
• When running baseline models alone, we average runs with two random seeds and limit the
number of oracle queries to 10K. • When running hybrid approaches, for each baseline model we use a portion of the 10K query budget to run the model (4K queries for Graph-GA and 2K for GPBO) and use the remaining query budget to optimize only the top generated molecule using QMO with the ZO-signGD optimizer and 2-point GS gradient estimator (QMO-sign-2p-GS) with Q = 49 .
Note that for these experiments, we consider only the score of the top 1 scoring molecule found sofar for a given run.
Additionally, we run QMO only with 2-point gradient estimators, though wealso compare 1-point estimators for QED [30] optimization in Appendix B.1 where we verify theadvantage of 2-point estimators.
4.1 Function landscapes of selected Guacamol objectives
Fig.
1 shows the function landscapes of the selected Guacamol objectives.
The origin corresponds toan optimized latent vector found by QMO, and the vector is perturbed along two random directionsv x and v y sampled from the uniform distribution on the unit sphere.
As shown, the zaleplon_mpo task has the smallest central area consisting of high scoring moleculesand a relatively flat landscape elsewhere, meaning that the QMO optimizer needs to traverse a veryflat unfavorable region to enter a very small optimal neighborhood.
This matches the observation thatzaleplon_mpo is a highly difficult task.
The deco_hop task, while not nearly as difficult of a task, stillexhibits a very discrete jump in values around the central region, which makes it more difficult for the QMO optimizer to find the true optimal neighborhood.
Finally, perindopril_mpo appears to bethe most smooth function.
The optimal central area is larger than for zaleplon_mpo, and the discretejumps in function values are not as large as in the other tasks.
4.2 Convergence of ZO optimization methods
Fig.
2 shows the results from experiments run using QMO only and compares the convergence of
ZO optimization methods with different Q .
Here, adam_2p_bes-shrink refers to QMO using the
ZO-Adam optimizer with the 2-point BeS-shrink gradient estimator (QMO-Adam-2p-BeS-shrink),and similarly for the other ZO optimization methods.
Diversity scores of the molecules found by
QMO are reported in Appendix B.3.
Overall, the results indicate that ZO-signGD is not only the most query-efficient method, but also themost robust to difficult function landscapes of molecular objectives.
Compared with ZO-Adam, ZOsignGD converges with both higher speed and better accuracy for most settings, and the difference isespecially clear for low Q and less smooth functions like zaleplon_mpo and deco_hop.
The improvedconvergence accuracy of ZO-signGD compared to ZO-Adam is particularly interesting as ZO-Adamconverges with much greater accuracy in other problems like adversarial example generation [14],thus showing the challenges presented by molecular objectives and the improved robustness of
ZO-signGD to their function landscapes.
Finally, as a note, both ZO-Adam and ZO-signGD outperform ZO-GD.
In fact, ZO-GD is completelyunsuccessful for the zaleplon_mpo task: even when searching a wide range of hyperparameters and testing several molecules, ZO-GD is unable to find any molecules with zaleplon_mpo scoresabove 0.2 within the first 100 iterations, and often cannot even get above 0.01.
Inspection revealedthat the gradient vectors were too small for ZO-GD to make meaningful point updates.
Thus, fullzaleplon_mpo experiments were not run using ZO-GD.
In addition, results from GS and BeS-shrinkgradient estimators do not differ greatly, though GS seems to converge faster with lower Q .
4.3 Query efficiency of QMO versus other approaches
Fig.
3 shows the optimization curves when limiting optimization to a query 10K budget, includingexperiments run using QMO only (specifically, only QMO-sign-2p-GS is shown), baseline modelsonly, and hybrid approaches.
Precise numbers and area under curve (AUC) scores are also reportedin Appendix B.4.
The baseline models (Graph-GA and GPBO) demonstrate faster convergence speed than QMO alone,and the relative convergence accuracies of all methods differ slightly for each task and can be saidto be comparable overall.
However, the hybrid approaches combining baseline models with QMO(i.e.
, Graph GA + sign_2p_gs) produce similar curves to their baseline model counterparts even forzaleplon_mpo and deco_hop, where QMO has higher convergence accuracy than the baseline models,so further investigation may be necessary to effectively integrate QMO into hybrid approaches.
Conclusion
In this paper, we study the application of ZO optimization methods to molecule optimization.
Throughexperimentation on tasks from the Guacamol suite, we show that ZO-signGD outperforms ZO-Adamand ZO-GD, especially for more difficult function landscapes with small regions of optima, flatregions, and discrete jumps.
Accordingly, we observe that the sign operation can increase robustnessto the difficult function landscapes of molecular objectives, while also achieving higher queryefficiency compared to other optimizer updating methods.
We also discuss how the generic QMOframework can be applied practically in realistic drug discovery scenarios, which includes a hybridapproach with other models.
To conclude, we would like to mention a few limitations of this study.
Synthesizability of moleculesis not accounted for, though one possible approach is to modify the objective function with asynthesizability loss.
Additionally, the effect of autoencoder choice and latent dimension are notthoroughly investigated for the selected benchmark tasks, though Hoffman et al.
[8] provide analysisfor their antimicrobial peptides task.
Finally, while Hoffman et al.
[8] also show that training an oracleprediction model (to predict scores based on latent representations) has significant disadvantages inoptimization accuracy compared to always using the oracle itself, we do not thoroughly investigatethe impact it would have on the objective function landscapes in the latent space.
A Implementation details
Winter et al.
[22] showed that their CDDD autoencoder model has a high validity rate of 97%, evenwhen traversing a large distance from the valid latent representations of randomly picked molecules.
In our implementation of QMO, we dealt with decode failures by assigning a penalty score of 0.1less than the score of the starting molecule, f ( z 0 ) − 0 .
Also, we only considered the molecules generated after each optimization iteration.
That is, we didnot consider the Q molecules obtained from decoding the perturbed latent vectors { z + β u q } Qq =1(used for estimating gradients) in Z iterate despite that they were also used to query the oracle O .
Especially in a realistic drug discovery scenario where oracle evaluations are highly expensive, wewould of course want to also consider these molecules in case they exhibit good properties.
Inaddition, while we considered there to be Q + 1 oracle evaluations necessary for each optimizationiteration, the actual amount would be lower in practice as some of the perturbed latent vectors woulddecode to the same molecule since the perturbations are small (and a small number of latent vectorswould also decode to no valid input).
All experiments were run using Google Colab, and code for QMO is available at this notebook:https://colab.research.google.com/drive/1y4nGZoGZaAXoOlJ-YluSpBb2M5M9Ksbn?usp=sharing .
For the Graph-GA and GPBO baseline models, we adopt the implementation of Gaoet al.
B Additional resultsB.1 Comparing 1-point and 2-point gradient estimators
Though we ran only 2-point gradient estimators on the Guacamol tasks, we also compared 1-pointgradient estimators on the QED [30] objective.
Specifically, following the setup of Hoffman et al.
[8],we defined a minimum similarity threshold of 0.4 (and did not consider molecules with similarity lessthan 0.4 to the starting molecule) and set the oracle O ( x ) = 4 · QED( x ) − max(0 .
4 − sim( x, x 0 ) , 0)for molecule x ∈ X and starting molecule x 0 , where sim( · ) denotes Tanimoto similarity with Morganfingerprints.
We selected 100 molecules with QED scores in [0 . 7 , 0 . 8] from the test set in [31] (whoextracted the molecules from ZINC) and optimized each with T = 20 iterations and 20 randomrestarts each.
We consider an optimized molecule a success if its QED scores falls in [0 . 9 , 1 . 0] ,and we visualize in Fig.
B1 how many random restarts are necessary for different ZO optimizationmethods to achieve a given success rate.
As shown, 2-point gradient estimators achieve significantlyhigher success rates than their 1-point counterparts given the same number of random restarts.
B.2 SMILES strings of molecules found by QMO
The SMILES strings from Fig.
1 are as follows:
• s 1 = CCCC(NC(C)Cn1c(C2CCCCC2)nc2cc(C(=O)O)ccc21)C(=O)OCC• s 2 = CCCC(C(=O)OCC)c1nc2cc(C(=O)O)ccc2n1C1CCCCC1C(C)=O•
s 3 = CCCC(C)n1c(C(=O)NC(C)C(=O)OCC)cc2cc(C3CCCCC3)cc(C(=O)O)c
• s 4 = COc1cc(C(=O)NCC23CC=CC2C3)nc2c(C#N)cccc
• s 5 = CCC1(CC)C(C(=O)Nc2cccc(F)c2)=CN=
C2C=
C(C#N)C(=O)N21• s 6 = CCC12CC(CO1)N(C(=O)c1cnc3cccc(C#N)c(=O)c3c1)C2• s
= COc1cc2ncnc(Nc3ccc(F)c4ncccc34)c2cc1C(C)C• s 8 =
CCCCOc1ncccc1C(=O)CNc1ncnc2cc(OC(F)F)c(F)cc12• s 9 = COc1cc(Nc2ncnc3cc(OC)c([N+](=O)[O-])cc23)ccc1F .
B.3 Diversity metrics
Table B1 shows the diversity of the QMO optimized molecules from Section 4.2 (optimized usingsign_2p_gs with Q = 50 ).
For each starting molecule in the test set, the best molecule found aftertwo random restarts was used, showing the diversity of molecules that can be generated from differentstarting points.
Hoffman et al.
[8] also showed how different random restarts starting from the samestarting point can find diverse candidates.
[Table]
B.4 Query efficiency tables
Scores from Fig.
3 are summarized below in Tables B2, B3, and B4.
C Other ZO optimization methods
Other than the ZO optimization methods considered here, ZO stochastic coordinate descent (ZOSCD) [15] was also tested.
However, because the coordinate-wise gradient estimator relies onperturbing coordinates individually, the perturbed vector embeddings used to estimate gradientsalmost always decoded back to the same molecule as the original non-perturbed vector.
In otherwords, the autoencoder almost always perceived the embedding vectors with all coordinates the same
[Table]
graph_ga
[Table] [Table]
gpbo
[Table]
sign_2p_gs ( Q = 30 ) 0.
[Table]
sign_2p_gs ( Q = 50 ) 0.
sign_2p_gs ( Q = 100 ) 0.
graph_ga_4k + sign_2p_gs ( Q = 49 ) 0.
gpbo_2k + sign_2p_gs ( Q = 49 ) 0.
[Table] [Table]
graph_ga_4k + sign_2p_gs ( Q = 49 ) 0.
gpbo_2k + sign_2p_gs ( Q = 49 ) 0.
but one coordinate to be the same molecule, so the coordinate-wise gradient estimates were almostalways zero.
D Hyperparameter tuning
Aside from the number of random perturbations Q , there are two other main hyperparameters foreach of the ZO optimization methods compared: function smoothing parameter β , and learning rateα .
The value β = 10 was used for all tasks as it was found to work well with the CDDD model.
Consistent with Hoffman et al.
[8], we find that β = 1 or below does not work well (as gradientscannot be accurately approximated without sufficient smoothing) and
β = 100 or above results inmany decode failures.
When trying a few molecules with β values between this range (includingβ = { 5 , 10 , 20 , 50 } for each task, β = 10 still performed best for the majority of molecules.
Thetuning of α is shown in Table D1 and Table D2.
As a note, α larger than the largest tested valuesfor each optimization method often resulted in many decode failures, so even if the best α was thelargest value tested, choosing notably larger α (greater by more than a factor of 2) may not be a goodidea.
Also, for ZO-Adam, two additional hyperparameters are used for the adaptive learning rate: theexponential averaging parameters β 1 and β 2 .
For these parameters, we use the default values used bythe PyTorch Adam implementation, β 1 = 0 .
9 and β 2 = 0 .
E Licenses
All test sets of molecules were originally extracted from the ZINC database [29] which is free for useby anyone.
[Table]
sign_2p_bes-shrink 0.
sign_2p_gs 0.
deco_hop adam_2p_bes-shrink 0.
adam_2p_gs 0.
grad_2p_bes-shrink 50.
grad_2p_gs 2.
sign_2p_bes-shrink 0.
sign_2p_gs 0.
[Table]
sign_1p_bes-shrink 0.
sign_1p_gs 0.
sign_2p_bes-shrink 0.
sign_2p_gs 0.