An Empirical Evaluation of Zeroth-Order Optimization Methods on AI-driven Molecule Optimization
Abstract
Molecule optimization is an important problem in chemical discovery and has been approached using many techniques, including generative modeling, reinforcement learning, genetic algorithms, and much more.
Recent work has also applied zerothorder (ZO) optimization, a subset of gradient-free optimization that solves problems similarly to gradient-based methods, for optimizing latent vector representations from an autoencoder.
In this paper, we study the effectiveness of various ZO optimization methods for optimizing molecular objectives, which are characterized by variable smoothness, infrequent optima, and other challenges.
We provide insights on the robustness of various ZO optimizers in this setting, show the advantages of ZO sign-based gradient descent (ZO-signGD), discuss how ZO optimization can be used practically in realistic discovery tasks, and demonstrate the potential effectiveness of ZO optimization methods on widely used benchmark tasks from the Guacamol suite.
Introduction
The goal of molecule optimization is to efficiently find molecules possessing desirable chemical properties.
As the ability to effectively solve difficult molecule optimization tasks would greatly accelerate the discovery of promising drug candidates and decrease the immense resources necessary for drug development, significant efforts have been dedicated to designing molecule optimization algorithms leveraging a variety of techniques, including deep reinforcement learning [1], genetic algorithms [2], Bayesian optimization [3], variational autoencoders [4, 5] and more.
Several molecule optimization benchmark tasks have also been proposed, including similarity-based oracles [6] and docking scores [7].
In this paper we extend the work of Hoffman et al.
[8] who proposed the use of ZO optimization in their query-based molecule optimization (QMO) framework, an end-to-end framework which decouples molecule representation learning and property prediction.
QMO iteratively optimizes a starting molecule, making it well suited for lead optimization tasks but it can also start from random points and traverse large distances to find optimal molecules.
In comparison with the work of Hoffman et al.
[8] which experiments with only one optimizer, we experiment with variations of
QMO using different ZO optimizers.
Furthermore, we add more benchmark tasks from Guacamol [6] (whose use has been encouraged by the molecule optimization community [3, 9] and used by Gao et al.
[10] to benchmark many design algorithms in a standardized setting), and provide insights into the challenges of ZO optimization on molecular objectives.
Contributions
We evaluate several ZO optimization methods for the problem of molecule optimization in terms of convergence speed, convergence accuracy, and robustness to the unusual function landscapes (described further in Section 2.4) of molecular objectives.
We show that ZO-signGD [11] outperforms other methods for molecule optimization in not only speed but also accuracy, despite being known to have worse convergence accuracy for other problems like adversarial attacks [11], which indicates that the sign operation is potentially more robust to the function landscapes of molecular objectives.
Furthermore, we provide insights into practical application of ZO optimization in drug discovery scenarios for both lead optimization tasks and the discovery of novel molecules, as well as propose the use of a hybrid approach combining others models with QMO.
Related work ZO optimization is a class of methods used for solving black-box problems by estimating gradients using only zeroth-order function evaluations and performing iterative updates as in first-order methods like gradient descent (GD) [12].
Many types of ZO optimization algorithms have been developed, including ZO gradient descent (ZO-GD) [13], ZO-signGD [11], ZO adaptive momentum method (ZO-AdaMM, or ZO-Adam specifically for the Adam variant) [14], and more [15, 16].
The optimality of ZO optimization methods has also been studied under given problem assumptions [17].
ZO optimization methods have achieved impressive feats in adversarial machine learning, where they have been used for adversarial example generation in black-box settings and demonstrated comparable success to first-order white-box attacks [18, 19].
They have also been shown to be able to generate contrastive explanations for black-box models [20].
Finally, Hoffman et al.
[8] showed how ZO optimization methods can also be applied to molecule optimization with their QMO framework.
QMO: Background and Variations 2.1 QMO framework
Following the QMO framework by Hoffman et al.
[8], we use an autoencoder to encode molecules with encoder E : X (cid:55)→ R d and decode latent vectors with decoder D : R d (cid:55)→ X , where X denotes the discrete chemical space of all drug candidates.
We denote with O : X (cid:55)→ R a black-box oracle returning a scalar corresponding to a molecular property of interest (which may also be modified by adding losses related to other properties), and for ease of notation with the QMO framework, we define our optimization objective loss function as f ( z ) =
−O ( D ( z )) for latent representations z ∈ R d .
As each function query f ( z ) queries the oracle O with the decoded molecule corresponding to z , one function query is equivalent to one oracle query.
In QMO, we use ZO optimization methods to navigate the latent space to solve min z f ( z ) .
Specifically, given a starting molecule and its latent representation z 0 , we iteratively update the current latent representation following some optimizer, as done in first-order gradient-based methods like gradient descent.
But as we do not have any first-order oracle, we instead use gradients estimated using only evaluations of f following some gradient estimator.
The QMO framework, which closely follows a generic ZO optimization procedure, is summarized in Algorithm 1.
Algorithm
1: Inputs: Starting molecule x 0 ∈ X , encoder E , decoder D , gradient estimation operation estimate _ gradient f ( · ) for function f , optimizer updating operation update( · ) , number of iterations T , and learning rate α 2: Z iterate ← {∅} 3: z 0 ← E ( x 0 ) 4: for t = 0 , 1 , ..., T − 1 do 5: ˆ g t ← estimate _ gradient f ( z t , E, D ) 6: z t +1 ← update( z t , { ˆ g i } ti =1 , α, E, D ) 7:
Z iterate ← Z iterate ∪ { z t +1 } 8: end for
In principle, QMO is a generic framework which can guide searches over any continuous learned representation based on any discrete space and use any ZO optimization method.
Hoffman et al.
[8] used the pre-trained SMILES-based [21] autoencoder (CDDD model) from Winter et al.
[22] with embedding dimension d = 512 and ZO-Adam.
Here, we use the same autoencoder but consider several variations of QMO using different gradient estimators and optimizers to provide a comprehensive study on the effect of ZO optimization methods.
As a note, QMO is applicable to molecule optimization with design constraints.
For example, given a set of property scores { p i } Ii =1 to be optimized with positive coefficients { γ i } Ii =1 and a set of property constraints { c j } Jj =1 with thresholds { η j } Jj =1 , we can define the oracle as
O ( x ) =
[Equation]
where x ∈ X .
The vectors z ∈ Z iterate not satisfying c j ( D ( z )) ≥ η j for all j ∈ { 1 , 2 , ..., J } can then be removed from Z iterate .
2.2 ZO gradient estimators
We consider two main ZO gradient estimators.
Both average finite differences over Q independently sampled random perturbations { u q } Qq =1 , include a smoothing parameter β , and follow the form:
[Equation]
The two gradient estimators differ mainly on the sampling method for each random direction u q , and also by the dimension-dependent factor φ ( d ) .
They are:
• Gaussian smoothing (GS) [13, 23]: when we sample each direction from the uniform distribution U ( S (0 , 1)) on the unit sphere.
For GS, φ ( d ) =
d . • Bernoulli smoothing-shrinkage (BeS-shrink) [24]: when we craft each random direction by independently sampling each of its d entries from ( B 0 .
5 − 0 . 5) /m , where B 0 .
5 follows the Bernoulli distribution with probability 0 .
5 and m = (cid:112) Q + d − 1 / 4 Q is an optimal shrinking factor.
For BeS-shrink, φ ( d ) =
By averaging over Q random directions to decrease estimation error, the gradient estimation operation requires querying Q + 1 different points (which are each decoded into a molecule and used to query oracle O ).
We therefore require Q + 1 oracle evaluations for each optimization iteration.
Additionally, because the above gradient estimators use a (forward) finite difference of 2 points to estimate the gradient for each random perturbation, we refer to it as a 2-point gradient estimator.
An alternative to the 2-point GS and BeS-shrink gradient estimators are their 1-point alternatives, which instead have the form:
[Equation]
Similar to 2-point gradient estimators, 1-point estimators require Q +1 oracle queries at each iteration (the estimation operation itself requires only Q queries, but this does not account for querying the updated molecule after each iteration).
However, 1-point estimators are not commonly used in practice due to higher variance.
2.3 ZO optimizers
We consider three main optimizers, each having its own updating operation that consists of computing a descent direction m t and then updating the current point.
Each optimizer can be paired with any
ZO gradient estimator.
The three are as follows:
• ZO gradient descent (ZO-GD) [13]: analagous to stochastic gradient descent (SGD) in the first-order stochastic setting.
ZO-GD uses the current gradient estimate as the descent direction m t
= ˆ g t and updates the current point via the rule z t +1 =
z t − α m t .
• ZO sign-based gradient descent (ZO-signGD)
[11]: analagous to sign-based SGD (signSGD)
[25] in the first-order stochastic setting.
ZO-signGD uses the same point updating rule as ZO-GD but instead uses the sign of the current estimate as the descent direction m t =
sign(ˆ g t ) , where sign( · ) denotes the element-wise sign
operation.
[Table]
• ZO-Adam [14]: analagous to Adam [26] in the first-order stochastic setting.
ZO-Adam adopts a momentum-type descent direction and an adapative learning rate.
The ZO optimization methods compared in this paper are summarized in Table 1.
2.4 Motivating the comparison of ZO optimization methods for molecule optimization
We motivate our comparison of optimizers not only in terms of convergence speed and convergence accuracy, but also in terms of robustness to the unfriendly function landscapes of molecular objectives.
Indeed, molecule optimization is made difficult by variable function smoothness due to "activity cliffs" in the molecular space where small structural changes cause large changes in oracle values [9].
As optima are infrequent, there are also large and extremely "flat" unfavorable regions in the space, where the oracle values change minimally and may be very small.
Furthermore, because our objective function f obtains values by querying the oracle O using discrete molecular representations obtained from decoding the latent vectors, the function landscape is made discrete and thus further non-smooth (i.e., the function value may have a discrete "jump" at the borders between adjacent regions of latent vectors which decode to different molecules, see Fig. 1).
Thus, being able to effectively navigate the latent chemical space and not get stuck in unfavorable regions is an important and non-trivial attribute to pursue in optimization methods.
Sign-based gradient descent is known to be effective in achieving fast convergence speed in stochastic settings: in the stochastic first-order oracle setting, Bernstein et al.
[25] showed that signSGD could have faster empirical convergence speed than SGD, and in the zeroth-order stochastic setting Liu et al.
[11] similarly showed that ZO-signSGD has faster convergence speed than many ZO optimization methods at the cost of worse accuracy (i.e., converging only to the neighborhood of an optima).
The fast convergence of sign-based methods is motivated by the idea that the sign operation is more robust to stochastic noise, and though our formulation of molecule optimization is non-stochastic, the sign operation is potentially more robust to the difficult landscapes of molecular objective functions.
Adaptive momentum methods like Adam also make use of the sign of stochastic gradients for determining the descent direction in addition to variance adaption [27], and thus ZO-Adam may also show robustness to the function landscape.
Practical Usage of QMO for Drug Discovery
We imagine QMO can be applied for two main cases: 1) identifying novel lead molecules (finding molecules significantly different from known leads), and 2) lead optimization (finding slightly modified versions of known leads).
For the former application case, it may be counterproductive to use known leads as the starting molecule in QMO, as those leads may be in the close neighborhood of a local optima (or a local optima themselves) in the function landscape, in which the optimizer would likely get stuck (preventing the exploration of different areas of the latent chemical space).
Instead, it may be more promising to start at a random point in the chemical space.
QMO also has the advantage that it guides search without the use of a training set, which aids in finding candidates vastly different from known molecules.
However, finding a highly diverse set of novel leads may be unlikely within a single run of QMO as the optimization methods converge to some neighborhood, meaning that multiple random restarts would likely be necessary to discover a diverse set of lead molecules.
For the latter application case, it is much more sensible to use known leads as the starting molecule input to QMO.
Additionally, rather than using an oracle O evaluating only the main desired drug property (i.e., activity against a biological target), it may be advantageous to use a modified oracle.
For example, Hoffman et al.
[8] apply QMO for lead optimization of known SARS-CoV-2 main protease inhibitors and antimicrobial peptides (AMPs) following the constrained molecule optimization setting of (1), with pre-trained property predictors for each task.
They set similarity to the original lead molecule as the property score p sim to be optimized, and set constraints on properties of interest (binding affinity c aff for the SARS-CoV-2 task, or toxicity prediction value c tox and AMP prediction value c AMP for the AMP task).
In these formulations, the main optimization objective is actually molecular similarity rather than the main properties of interest.
Hybrid optimization
: Integrating QMO with
generative
models Additionally, in this paper
we propose to apply QMO in a hybrid approach with other models: namely, we can use molecules generated by other models as input to QMO, which will then iteratively optimize each of the inputted starting molecules.
By using other models to generate good lead molecules close to optima, we can then use QMO to provide a more refined search that may incorporate additional design constraints.
Overall, a hybrid approach could be a query efficient way to generate drug candidates satisfying multiple design constraints.
Experiments
To benchmark QMO, we select three tasks (oracles) from the popular benchmarking suite Guacamol [6]: perindopril_mpo (finding molecules similar to perindopril
but with a different number of aromatic rings), zaleplon_mpo (finding molecules similar to zaleplon but a different molecular formula), and deco_hop (maximizing similarity to a SMILES string while keeping or excluding specific SMARTS patterns).
These represent three of the main categories of Guacamol oracles: similarity-based multiproperty objectives, isomer-based objectives, and SMARTS-based objectives, respectively.
The zaleplon_mpo task is known to be particularly difficult [9].
We also select two baselines, graph-based genetic algorithm (Graph-GA)
[2] and Gaussian process Bayesian optimization (GPBO) [3, 28], both of which are known to be high performing molecule optimization algorithms [10].
For each of these tasks, we run experiments using QMO only, baselines only, and hybrid approaches.
• When running experiments using QMO only, we run with several different ZO optimization methods and try Q = { 30 , 50 , 100 } for each, set T = 1000 iterations for perindopril_mpo and zaleplon_mpo or T = 200 for deco_hop, and average runs from 20 distinct starting molecules with 2 random restarts each (40 runs total).
For each task, we choose to use the 20 lowest-scoring molecules on the oracle from the ZINC 250K dataset [29] as the starting molecules.
We do this in order to show that QMO can find solutions even when starting far from any high scoring molecules, which we would likely need to do when searching for novel lead molecules.
• When running baseline models alone, we average runs with two random seeds and limit the number of oracle queries to 10K. • When running hybrid approaches, for each baseline model we use a portion of the 10K query budget to run the model (4K queries for Graph-GA and 2K for GPBO) and use the remaining query budget to optimize only the top generated molecule using QMO with the ZO-signGD optimizer and 2-point GS gradient estimator (QMO-sign-2p-GS) with Q = 49 .
Note that for these experiments, we consider only the score of the top 1 scoring molecule found so far for a given run.
Additionally, we run QMO only with 2-point gradient estimators, though we also compare 1-point estimators for QED [30] optimization in Appendix B.1 where we verify the advantage of 2-point estimators.
4.1 Function landscapes of selected Guacamol objectives
Fig.
1 shows the function landscapes of the selected Guacamol objectives.
The origin corresponds to an optimized latent vector found by QMO, and the vector is perturbed along two random directions v x and v y sampled from the uniform distribution on the unit sphere.
As shown, the zaleplon_mpo task has the smallest central area consisting of high scoring molecules and a relatively flat landscape elsewhere, meaning that the QMO optimizer needs to traverse a very flat unfavorable region to enter a very small optimal neighborhood.
This matches the observation that zaleplon_mpo is a highly difficult task.
The deco_hop task, while not nearly as difficult of a task, still exhibits a very discrete jump in values around the central region, which makes it more difficult for the QMO optimizer to find the true optimal neighborhood.
Finally, perindopril_mpo appears to be the most smooth function.
The optimal central area is larger than for zaleplon_mpo, and the discrete jumps in function values are not as large as in the other tasks.
4.2 Convergence of ZO optimization methods
Fig.
2 shows the results from experiments run using QMO only and compares the convergence of
ZO optimization methods with different Q .
Here, adam_2p_bes-shrink refers to QMO using the
ZO-Adam optimizer with the 2-point BeS-shrink gradient estimator (QMO-Adam-2p-BeS-shrink), and similarly for the other ZO optimization methods.
Diversity scores of the molecules found by
QMO are reported in Appendix B.3.
Overall, the results indicate that ZO-signGD is not only the most query-efficient method, but also the most robust to difficult function landscapes of molecular objectives.
Compared with ZO-Adam, ZOsignGD converges with both higher speed and better accuracy for most settings, and the difference is especially clear for low Q and less smooth functions like zaleplon_mpo and deco_hop.
The improved convergence accuracy of ZO-signGD compared to ZO-Adam is particularly interesting as ZO-Adam converges with much greater accuracy in other problems like adversarial example generation [14], thus showing the challenges presented by molecular objectives and the improved robustness of
ZO-signGD to their function landscapes.
Finally, as a note, both ZO-Adam and ZO-signGD outperform ZO-GD.
In fact, ZO-GD is completely unsuccessful for the zaleplon_mpo task: even when searching a wide range of hyperparameters and testing several molecules, ZO-GD is unable to find any molecules with zaleplon_mpo scores above 0.2 within the first 100 iterations, and often cannot even get above 0.01.
Inspection revealed that the gradient vectors were too small for ZO-GD to make meaningful point updates.
Thus, full zaleplon_mpo experiments were not run using ZO-GD.
In addition, results from GS and BeS-shrink gradient estimators do not differ greatly, though GS seems to converge faster with lower Q .
4.3 Query efficiency of QMO versus other approaches
Fig.
3 shows the optimization curves when limiting optimization to a query 10K budget, including experiments run using QMO only (specifically, only QMO-sign-2p-GS is shown), baseline models only, and hybrid approaches.
Precise numbers and area under curve (AUC) scores are also reported in Appendix B.4.
The baseline models (Graph-GA and GPBO) demonstrate faster convergence speed than QMO alone, and the relative convergence accuracies of all methods differ slightly for each task and can be said to be comparable overall.
However, the hybrid approaches combining baseline models with QMO (i.e., Graph GA + sign_2p_gs) produce similar curves to their baseline model counterparts even for zaleplon_mpo and deco_hop, where QMO has higher convergence accuracy than the baseline models, so further investigation may be necessary to effectively integrate QMO into hybrid approaches.
Conclusion
In this paper, we study the application of ZO optimization methods to molecule optimization.
Through experimentation on tasks from the Guacamol suite, we show that ZO-signGD outperforms ZO-Adam and ZO-GD, especially for more difficult function landscapes with small regions of optima, flat regions, and discrete jumps.
Accordingly, we observe that the sign operation can increase robustness to the difficult function landscapes of molecular objectives, while also achieving higher query efficiency compared to other optimizer updating methods.
We also discuss how the generic QMO framework can be applied practically in realistic drug discovery scenarios, which includes a hybrid approach with other models.
To conclude, we would like to mention a few limitations of this study.
Synthesizability of molecules is not accounted for, though one possible approach is to modify the objective function with a synthesizability loss.
Additionally, the effect of autoencoder choice and latent dimension are not thoroughly investigated for the selected benchmark tasks, though Hoffman et al.
[8] provide analysis for their antimicrobial peptides task.
Finally, while Hoffman et al.
[8] also show that training an oracle prediction model (to predict scores based on latent representations) has significant disadvantages in optimization accuracy compared to always using the oracle itself, we do not thoroughly investigate the impact it would have on the objective function landscapes in the latent space.
Appendix A Implementation details
Winter et al.
[22] showed that their CDDD autoencoder model has a high validity rate of 97%, even when traversing a large distance from the valid latent representations of randomly picked molecules.
In our implementation of QMO, we dealt with decode failures by assigning a penalty score of 0.
less than the score of the starting molecule, f ( z 0 ) − 0 .
Also, we only considered the molecules generated after each optimization iteration.
That is, we did not consider the Q molecules obtained from decoding the perturbed latent vectors { z + β u q } Qq =
(used for estimating gradients) in Z iterate despite that they were also used to query the oracle O .
Especially in a realistic drug discovery scenario where oracle evaluations are highly expensive, we would of course want to also consider these molecules in case they exhibit good properties.
In addition, while we considered there to be Q + 1 oracle evaluations necessary for each optimization iteration, the actual amount would be lower in practice as some of the perturbed latent vectors would decode to the same molecule since the perturbations are small (and a small number of latent vectors would also decode to no valid input).
All experiments were run using Google Colab, and code for QMO is available here: https:// github.com/IBM/QMO-bench .
For the Graph-GA and GPBO baseline models, we adopt the implementation of Gao et al.
B Additional results B.1 Comparing 1-point and 2-point gradient estimators
Though we ran only 2-point gradient estimators on the Guacamol tasks, we also compared 1-point gradient estimators on the QED [30] objective.
Specifically, following the setup of Hoffman et al.
[8], we defined a minimum similarity threshold of 0.4 (and did not consider molecules with similarity less than 0.4 to the starting molecule) and set the oracle O ( x ) = 4 · QED( x ) − max(0 .
4 − sim( x, x 0 ) , 0) for molecule x ∈ X and starting molecule x 0 , where sim( · ) denotes Tanimoto similarity with Morgan fingerprints.
We selected 100 molecules with QED scores in [0 . 7 , 0 . 8] from the test set in [31] (who extracted the molecules from ZINC) and optimized each with T = 20 iterations and 20 random restarts each.
We consider an optimized molecule a success if its QED scores falls in [0 . 9 , 1 . 0] , and we visualize in Fig.
B1 how many random restarts are necessary for different ZO optimization methods to achieve a given success rate.
As shown, 2-point gradient estimators achieve significantly higher success rates than their 1-point counterparts given the same number of random restarts.
B.2 SMILES strings of molecules found by QMO
The SMILES strings from Fig.
1 are as follows:
• s 1 = CCCC(NC(C)Cn1c(C2CCCCC2)nc2cc(C(=O)O)ccc21)C(=O)OCC • s 2 = CCCC(C(=O)OCC)c1nc2cc(C(=O)O)ccc2n1C1CCCCC1C(C)=O • s 3 =
CCCC(C)n1c(C(=O)NC(C)C(=O)OCC)cc2cc(C3CCCCC3)cc(C(=O)O)c • s 4 = COc1cc(C(=O)NCC23CC=
CC2C3)nc2c(C#N)cccc
• s 5 = CCC1(CC)C(C(=O)Nc2cccc(F)c2)=CN=
C2C
=
C(C#N)C(=O)N • s 6 =
CCC12CC(CO1)N(C(=O)c1cnc3cccc(C#N)c(=O)c3c1)C • s 7 = COc1cc2ncnc(Nc3ccc(F)c4ncccc34)c2cc1C(C)C • s
= CCCCOc1ncccc1C(=O)CNc1ncnc2cc(OC(F)F)c(F)cc • s 9 = COc1cc(Nc2ncnc3cc(OC)c([N+](=O)[O-])cc23)ccc1F .
B.3 Diversity metrics
Table B1 shows the diversity of the QMO optimized molecules from Section 4.2 (optimized using sign_2p_gs with Q = 50 ).
For each starting molecule in the test set, the best molecule found after two random restarts was used, showing the diversity of molecules that can be generated from different starting points.
Hoffman et al.
[8] also showed how different random restarts starting from the same starting point can find diverse candidates.
[Table]
B.4 Query efficiency tables
Scores from Fig.
3 are summarized below in Tables B2, B3, and B4.
C Other ZO optimization methods
Other than the ZO optimization methods considered here, ZO stochastic coordinate descent (ZOSCD) [15] was also tested.
However, because the coordinate-wise gradient estimator relies on perturbing coordinates individually, the perturbed vector embeddings used to estimate gradients almost always decoded back to the same molecule as the original non-perturbed vector.
In other words, the autoencoder almost always perceived the embedding vectors with all coordinates the same but one coordinate to be the same molecule, so the coordinate-wise gradient estimates were almost always zero.
[Table]
graph_ga
[Table]
[Table]
gpbo
[Table] [Table] sign_2p_gs ( Q = 30 ) [Table]
[Table] [Table] [Table] sign_2p_gs ( Q = 50 ) 0.
sign_2p_gs ( Q = 100 ) 0.
graph_ga_4k + sign_2p_gs ( Q = 49 ) 0.
gpbo_2k + sign_2p_gs ( Q = 49 ) 0.
[Table] [Table]
graph_ga_4k + sign_2p_gs ( Q = 49 ) 0.
gpbo_2k + sign_2p_gs ( Q = 49 ) 0.
D Hyperparameter tuning
Aside from the number of random perturbations Q , there are two other main hyperparameters for each of the ZO optimization methods compared: function smoothing parameter β , and learning rate α .
The value β = 10 was used for all tasks as it was found to work well with the CDDD model.
Consistent with Hoffman et al.
[8], we find that β = 1 or below does not work well (as gradients cannot be accurately approximated without sufficient smoothing) and β = 100 or above results in many decode failures.
When trying a few molecules with β values between this range (including β = { 5 , 10 , 20 , 50 } for each task, β = 10 still performed best for the majority of molecules.
The tuning of α is shown in Table D1 and Table D2.
As a note, α larger than the largest tested values for each optimization method often resulted in many decode failures, so even if the best α was the largest value tested, choosing notably larger α (greater by more than a factor of 2) may not be a good idea.
Also, for ZO-Adam, two additional hyperparameters are used for the adaptive learning rate: the exponential averaging parameters β 1 and β 2 .
For these parameters, we use the default values used by the PyTorch Adam implementation, β 1 = 0 .
9 and β 2 = 0 .
E Licenses
All test sets of molecules were originally extracted from the ZINC database [29] which is free for use by anyone.
[Table]
sign_2p_bes-shrink 0.
sign_2p_gs 0.
deco_hop adam_2p_bes-shrink 0.
adam_2p_gs 0.
grad_2p_bes-shrink 50.
grad_2p_gs 2.
sign_2p_bes-shrink 0.
sign_2p_gs 0.
[Table]
sign_1p_bes-shrink 0.
sign_1p_gs 0.
sign_2p_bes-shrink 0.
sign_2p_gs 0.