SlowGuess/NeurIPS_2023_OCR huggingface dataset:
As their name implies, the LSTM-HC models [105, 33, 3, 108] rely on LSTM cells in the NN architecture to model the character sequence probability distributions. After initial training, the resulting model is sampled using randomly truncated strings of the best currently available molecules for a given task. These truncated strings are used as seeds and completed stochastically by sampling the learned conditional probability distributions to produce a population of candidate structures. The best new compounds obtained after property evaluation are used to retrain the model and initiate a subsequent sampling cycle. Thus, iterative sampling and retraining gradually improves the designs of the LSTM-HC models. Again, we test both the original implementation of LSTM-HC that relies on SMILES [33, 3], which will be referred to as SMILES-LSTM-HC, and a modified version making use of SELFIES, referred to as SELFIES-LSTM-HC. Notably, the SMILES-LSTM-HC model was among the best performing models in the original publication of the GuacaMol benchmarks [3] (referred to as "SMILES LSTM" in that reference).
Before the increasing adoption of ML approaches for molecular design, GAs [109, 5, 110, 111] were among the most popular computer-based methods for that purpose. Inspired by natural selection as defined in Darwinian evolution, GAs are heuristic population-based optimization algorithms. They rely on repeated stochastic generation of offspring from an existing population of candidate solutions via the genetic operations mutation and crossover. Subsequently, selection of the candidate solutions with the highest fitness determines the population to be propagated to the next generation, starting another iteration. In this work, we consider two specific implementations for inverse molecular design, GB-GA [34], and JANUS [7]. The former leverages structural information from a dataset of reference molecules by mimicking the corresponding distribution of atoms and bonds when performing genetic operations. Notably, in the original publication of the GuacaMol benchmarks [3], GB-GA achieved the best overall performance. In contrast, JANUS is a GA augmented by an NN classifier [7] (referred to as "JANUS+C" in the original publication) that actively judges the quality of a molecule before performing a full fitness evaluation. Molecules classified as likely possessing high fitness are then propagated to the next generation for subsequent evaluation. For mutation and crossover, JANUS relies on STONED-SELFIES [112], a set of algorithms based on the guaranteed validity of SELFIES to perform structural modifications. Additionally, unlike GB-GA, JANUS maintains two distinct molecule populations, one explorative and one exploitative with respect to conducted structural changes [7]. These populations exchange some members at every iteration but otherwise explore the chemical space independently [7].
When using TARTARUS, the following procedures should be adopted to obtain benchmark results that are consistent with the ones provided herein. The first step for running one of the benchmarks, if necessary, is to train the generative model on the provided dataset. For all the ML models, we used the first  $80\%$  of the reference molecules for training and the remaining  $20\%$  for hyperparameter optimization. Then, the (trained) model is tasked with proposing structures to be evaluated by the objective function of the corresponding benchmark task. Notably, structure optimization was always initiated using the best reference molecule from the corresponding dataset. For the benchmarks concerned with designing photovoltaics, organic emitters, and protein ligands, structure optimization was carried out with a population size of 500 and a limit of 10 iterations, leading to a maximum number of 5,000 proposed compounds overall. For the design of chemical reaction substrates, we used the same maximum number of proposed compounds but used a population size of 100 and limited the number of iterations to 50 instead. Additionally, the associated run time was limited to 24 hours, which resulted in termination for several molecular design runs before reaching 5,000 molecule evaluations. Furthermore, to increase robustness and reproducibility of our results, we repeated each optimization run five times, allowing us to report the corresponding outcomes with both an average and a standard deviation. We believe that this resource-constrained comparison approach is necessary for fairly comparing methods and should be used as a standard by the community. A detailed account of the parameters and settings used for running each of the models is provided in the Computational Details section of the Supporting Information.
S3.1 Design of Organic Photovoltaics
OPVs offer simplified, cost-effective production [113, 35] and enhanced mechanical properties, particularly regarding specific weight and flexibility [38, 39]. Despite significant progress, OPVs still have lower power conversion efficiencies (PCEs) and shorter device lifetimes [114], prompting ongoing research efforts in molecular design for OSCs [115, 114, 39].
The simplest OPV cells comprise two electrodes and a photoactive layer for photoconversion, typically containing two distinct materials: donor and acceptor [35]. In bulk heterojunction cells [116], donors and acceptors are mixed, allowing nanoscale phase separation to maximize interfacial area and minimize distance within the phases [117]. Upon light exposure, excitons—neutral quasi-particles consisting of bound electron-hole pairs [118, 117, 35]—form in the photoactive layer with limited lifetimes and diffusion lengths [117]. The bulk heterojunction architecture enables excitons to reach the interface, dissociating into a free electron and hole that become charge carriers across the phases [118, 117, 35]. These charge carriers travel through the photoactive layer to the electrodes, generating a photocurrent [35]. For exciton charge separation at the donor-acceptor interface to occur, the process must be energetically neutral or favorable [118, 35]. Therefore, the exciton energy, estimated by the HOMO-LUMO gap of the light-absorbing material, must be greater than or equal to the effective heterojunction bandgap [118, 35], approximated by the energy difference between the donor's highest occupied molecular orbital (HOMO) and the lowest unoccupied molecular orbital of the acceptor (LUMO) [41]. Notably, PCE is the percentage of power from the incident solar irradiation that is converted into electricity by the OPV device. We believe that the reduced reference dataset presents a more realistic scenario for new molecular design projects and is better suited for benchmarking generative models, especially when property simulations are time-consuming. We propose the following two molecular design benchmark objectives:
1. Maximize the following function:  $PCE_{PCBM} - SA_{score}$
2. Maximize the following function:  $PCE_{PCDTBT} - SA_{score}$
Figure S1: Best molecules found in each of the benchmark tasks inspired by the design of organic photovoltaics. Additionally, the corresponding objective values and the molecular design models that proposed the structures are indicated.
The best molecules found in each of the design of organic photovoltaics benchmark tasks together with the corresponding objective values and the model that proposed them are depicted in Figure S1.
S3.2 Design of Organic Emitters
The next set of benchmarks is inspired by the design of purely organic emissive materials for organic light-emitting diodes (OLEDs), which received significant attention in recent years [55-57] after the discovery of thermally activated delayed fluorescence (TADF) in the field [60]. Their main applications are digital screens and lighting devices [55]. For the former application, compared to alternative technologies, OLEDs offer improved image quality and enable both lighter and thinner devices [55]. For the latter, OLEDs are potentially more energy-efficient [55]. In OLED devices relying on TADF, after electric excitation, light generation takes place from the first excited singlet state via fluorescence [60]. However, only  $25\%$  of the initial excitons are excited via an electric current into singlet states and contribute directly to light emission via excited state thermal relaxation and subsequent prompt fluorescence [55-57, 60]. The  $75\%$  of the initial excitons that are excited into triplet states relax thermally to the corresponding first excited triplet states [55-57, 60]. However, in ordinary organic molecules, light emission from their first excited triplet states via phosphorescence is slow, giving rise to various radiationless decay and decomposition pathways [55-57]. Consequently, both device efficiency and device lifetime are reduced considerably unless these triplet excitons can still be utilized for light production [55-57]. To achieve that, TADF emitters rely on minimizing the energy difference between the first excited singlet and triplet states, i.e., the singlet-triplet gap, allowing thermal upconversion of the triplet excitons to the first excited singlet state giving rise to delayed fluorescence [55-57]. Importantly, under the assumption of fast both forward and reverse intersystem crossing (ISC), the steady-state triplet population is governed by the singlet-triplet gap and its reduction leads to reduced triplet population and acceleration of delayed fluorescence. This increases internal quantum efficiency up to a maximum of  $100\%$  and reduces decomposition of the emissive material [55-57]. Designing efficient emissive organic materials for blue OLEDs is of particular interest. This can be achieved by targeting organic molecules with excitation energies between their ground state and their first excited singlet states that correspond to the energy of blue light. The fitness functions of these three tasks are summarized as follows:
- Minimize the singlet-triplet gap,  $\Delta E(S_1 - T_1)$ .
- Maximize the oscillator strength for the transition between  $S_{1}$  and  $S_{0}$ ,  $f_{12}$ .
- Maximize the following function:
$$
+ f _ {1 2} - \Delta E \left(S _ {1} - T _ {1}\right) - \left| \Delta E \left(S _ {0} - S _ {1}\right) - 3. 2 e V \right|.
$$
The best molecules found in each of the design of organic emitters benchmark tasks together with the corresponding objective values and the model that proposed them are depicted in Figure S2.
S3.3 Design of Protein Ligands
Notably, while docking simulations are a standard tool in virtual screening pipelines of drug discovery campaigns, their accuracy compared to experimental binding affinities is at best modest [119-122]. Thus, typical workflows use them merely for a preselection which is narrowed down further with subsequent free energy simulations [123]. Nevertheless, for molecular design benchmarks, docking still provides the best trade-off between computational efficiency and relevance to real-world molecular design. Additionally, it is important to realize that drug design requires the consideration of many other molecular design aspects that make or break a hit compound, e.g., toxicity, solubility, stability and many more. However, as these properties are significantly harder to model computationally, we decided to disregard them for our set of benchmarks. Notably, finding small molecule ligands for the selected proteins marks the first step towards the development of treatments for various important conditions.
Importantly, most likely due to its maturity, complexity, and demand for resources, drug design was probably the first chemical problem where molecular design algorithms were tested comprehensively.
Figure S2: Best molecules found in each of the benchmark tasks inspired by the design of organic emitters. Additionally, the corresponding objective values and the molecular design models that proposed the structures are indicated.
The use of computer algorithms has a long-standing history in medicinal chemistry and GA-based molecular design algorithms making use of full atomic representations have already been used as early as 1993 [6]. Initial toy tasks for testing these algorithms included rediscovery of known drugs via a structural similarity metric, highly simplified molecular docking of ligands to rigid protein binding sites minimizing the interaction scores, and the optimization of estimated molecular properties like the decadic logarithm of the  $n$ -octanol-water distribution coefficient (log P) [6]. Interestingly, some of the still most widely used benchmarks for generative models rely on essentially the same types of metrics as they are simple to implement and compute [2, 14, 3, 124, 29]. More recently, the quantitative estimate of drug-likeness (QED) was introduced, which is a desirability function that uses the position of a small set of common and simple molecular descriptors relative to the corresponding distributions in a dataset of approved drugs to estimate structural resemblance to therapeutics [125]. Due to its simplicity, it found immediate application in several molecular design benchmarks [126, 103, 127]. However, using QED alone in generative models is not meaningful for finding drug candidates as it only accounts for the general structural requirements of drug-like molecules but disregards intended modes of action with respect to specific targets entirely. The specific benchmark objectives we implemented are summarized below.
- Minimize the docking score to 1SYH,  $\Delta E_{1SYH}$ .
- Minimize the docking score to 6Y2F,  $\Delta E_{6Y2F}$ .
- Minimize the docking score to 4LDE,  $\Delta E_{4LDE}$ .
Notably, the corresponding objective functions do not solely consist of the docking scores but also have hard structural constraints directly incorporated. When these constraints are not fulfilled, an extremely unfavorable score of 10,000 is returned instead of the actual docking score. They consist of a set of filters that checks for the presence of unstable or reactive structural moieties and determines whether the compound in question fulfils Lipinski's Rule of Five [128]. Notably, most of these filters were developed based on our previous experience using molecular design algorithms to minimize docking scores and are tailored to avoid extremely unstable and reactive molecules that seem to be strongly favored by molecular docking simulations [7]. Additionally, the constraints avoid rings with more than 8 members as the docking approach implemented is unable to sample the corresponding conformations in a proper manner [129]. To fulfill them, the proposed structure needs to have an SAscore smaller than 4.5, which is the revised optimal threshold for that metric proposed in the literature [130], and a QED value larger than 0.3, which corresponds to the first quartile of the distribution of QED values for compounds in the CHEMBL database [125]. A list of these metrics is provided here:
- Absence of reactive groups.
- Absence of formal charges.
- Absence of radicals.
- At most 2 bridgehead atoms.
- No rings larger than 8-membered.
- Fulfils Lipinski's Rule of Five.
-  $S A s c o r e < 4.5$
-  ${QED} > {0.3}$  .
-  $TPSA > 140$ .
- Molecule passes the PAINS and WEHI and MCF filters.
- Molecule does not contain Si and Sn atoms.
The best molecules found in each of the design of protein ligands benchmark tasks together with the corresponding objective values and the model that proposed them are depicted in Figure S3.
S3.4 Design of Chemical Reaction Substrates
Whereas, classically, the optimization of reaction parameters was largely dominated by experimental work, in recent years, the significant increase in computing power and the continuous improvement of computer algorithms enabled molecular simulations to play an increasingly important part [131-134]. With the aid of transition state (TS) theory, fundamental reaction parameters such as thermodynamic feasibility, reaction rate and selectivity can be computed from first principles [135, 136]. This requires explicit modeling of the corresponding TS, a postulated state on the multi-dimensional potential energy surface (PES) of the process, which lies on a saddle point of order one [137]. Due to the difficulty of finding such saddle points in high-dimensional spaces and the often delicate electronic structure associated with the corresponding structures, in practice, automated TS optimizations often suffer from high failure rates in the range of  $10 - 50\%$  [138-140], and even well-behaved case studies combined with robust methodologies still have some room for improvement in that respect[141]. Additionally, they are typically carried out with relatively resource-intensive density functional approximation (DFA) calculations taking on the order of hours or even days to complete [138, 140]. Overall, these issues make them ill-suited for benchmarking molecular design algorithms or for routine application combined with generative models in computer-guided inverse molecular design campaigns. Nevertheless, GAs have been employed for computational catalyst design, particularly via the use of regression models based electronic structure descriptors derived from SQC simulations [142] and based on both structural and electronic structure descriptors obtained from DFA computations [143]. Most notably, very recently, GAs have been employed to optimize an organocatalyst for the Morita-Baylis-Hilman reaction [144].
Figure S3: Best molecules found in each of the benchmark tasks inspired by the design of protein ligands. Additionally, the corresponding objective values and the molecular design models that proposed the structures are indicated.
The two force fields cross at approximately the TS geometry, and we introduce a coupling term that turns this into an avoided crossing, ensuring smoothness of the PES [89]. The resulting PES has a local maximum on the ground state surface (the TS) and a corresponding local minimum on the excited state surface, at approximately the same geometry. This allows optimizing TS geometries with robust gradient-based minimization algorithms. Thus, the SEAM force field method delivers activation energy estimates for reasonably sized molecules within minutes, and, in our hands, reaches very high success rates above  $99.9\%$  on a set of test reactions. Notably, we implemented this method in our Python package polanyi, which will be described in more detail in a separate publication. To generate the reference dataset for this set of benchmarks, starting from the unsubstituted reactive core structure, we performed repetitive cycles of STONED-SELFIES mutations [112] followed by removing all proposed compounds that violated the core and functional group constraints (Details in the Supporting Information). Thus, after several cycles, we obtained approximately 60,000 molecules defining the reference structures, which we refer to as SNB-60K dataset. Notably, in the selected reaction, there is only one TS connecting reactants and products in the selected reaction. Additionally, the fourth task aims to break the Bell-Evans-Polanyi principle [145-148], a linear free energy relationship that holds empirically for a large number of reactions. The following list summarizes the four benchmark tasks for chemical reactivity.
- Minimize the activation energy,  $\Delta E^{\ddagger}$ , and maintain the core and functional group constraints.
- Minimize the reaction energy,  $\Delta E_{r}$ , and maintain the core and functional group constraints.
- Minimize the following function:
$+\Delta E^{\ddagger} + \Delta E_{r}$ , and maintain the core, functional group and SAscore constraints.
- Minimize the following function:
$-\Delta E^{\ddagger} + \Delta E_{r}$ , and maintain the core, functional group and SAscore constraints.
On top of these primary objectives, we added several hard constraints that the target molecules need to fulfill in order to reward generative models that propose realistic and feasible molecules. In particular, all substrates need to retain the syn-sesquinorbornene motif (referred to as "core constraint") which is required for the reaction to take place. Additionally, we selected a set of unstable and reactive substructures that need to be avoided (referred to as "functional group constraint", details in section S5.2.4 of the Supporting Information). Furthermore, for the two benchmark tasks with objective functions combining two target properties, we also required all proposed structures to possess an SAscore [44] of 6.0 or lower (referred to as "SAscore constraint"). The following list summarizes the four benchmark tasks for chemical reactivity.
$\Delta E^{t} = 44.04 \, \text{kcal/mol}$
$\Delta E^{t} = 51.28 \, \text{kcal/mol}$
$\Delta E_{r} = -59.10 \, \text{kcal/mol}$
$\Delta E_{r} = -48.76 \, kcal/mol$
+ΔE+ ΔEr=32.79 kcal/mol
+ΔE+ ΔEr=35.47 kcal/mol
$-\Delta E^{t} + \Delta E_{r} = -102.51 kcal/mol$
$-\Delta E^{i} + \Delta E_{r} = -98.81\mathrm{kcal / mol}$
Figure S4: Best molecules found in each of the benchmark tasks inspired by the design of chemical reaction substrates. Additionally, the corresponding objective values and the molecular design models that proposed the structures are indicated.
The best molecules found in each of the design of chemical reaction substrates benchmark tasks together with the corresponding objective values and the model that proposed them are depicted in Figure S3.
S3.5 Model Timing Comparison
For ML-based molecular design algorithms, the pre-conditioning corresponds to training time and sampling time, respectively. For the GA-based algorithms considered, they translate to the duration of the derivation of genetic operators from a reference dataset, and of applying the genetic operators to propose new candidate solutions, respectively. When the number of property evaluations is kept constant, assuming that the molecular size distribution between the generative models does not differ significantly, these steps are the major origin of timing differences between the algorithms. Notably, as was done for generating the molecular design results, we derived these timing metrics from five independent measurements and provide the results as averages with standard deviations.
Comparison of the single epoch times with and without a GPU allows estimating which models profit most from GPU usage. We find that the longer the single epoch time, the more the model profits from using a GPU which is consistent with expectations. Finally, looking at the sampling times, we find that REINVENT significantly outperforms all other methods considered needing less than 1 minute. Most of the other molecular design algorithms need between 2 and 3 minutes with GB-GA having the slowest sampling time of 6 minutes. Nevertheless, the differences between the methods are less pronounced here.
Table S5: Raw values for the timing benchmarks. Mean and standard deviation (mean ± s.d) timings for different models are provided based on five independent runs. N.A. means not applicable.
TRAINING TIME [s]	EPOCHS	SAMPLE TIME [s]	CPU EPOCH TIME [s]	GPU EPOCH TIME [s]
SELFIES-VAE	36910 ± 912	75.6 ± 5.9	201 ± 53	29810 ± 949	535 ± 38
SMILES-VAE	34868 ± 667	74.1 ± 6.2	154 ± 79	28476 ± 724	515 ± 39
MoFlow	2804 ± 216	70.1 ± 5.4	62.41 ± 5	2131 ± 63	45 ± 8
GB-GA	N.A.	N.A.	314 ± 54	3.653 ± 0.007	N.A.
JANUS	N.A.	N.A.	147 ± 4	10.3 ± 2.4	N.A.
REINVENT	2844 ± 310	33.8 ± 1.2	33 ± 18	397 ± 14	81.7 ± 9.2
SMILES-LSTM-HC	7208 ± 1605	45.8 ± 10.5	128.0 ± 1.7	1870 ± 139	157.4 ± 5.6
SELFIES-LSTM-HC	7321 ± 1039	45.4 ± 6.3	119.3 ± 0.5	1661 ± 112	161 ± 27
Figure S5: Estimated CPU training time for the deep generative models based on the CPU single epoch time and the number of epochs during training with GPUs. Models are trained on a subset of the DTP Open Compound Collection. Results are provided as mean (main bar) from five independent runs. The numbers on the abscissa each refer to the following molecular design algorithms. Model 0: SELFIES-VAE; Model 1: SMILES-VAE; Model 2: REINVENT; Model 3: SMILES-LSTM-HC, Model 4: SELFIES-LSTM-HC.
The numerical results of all the timing benchmarks are provided in Table S5. The results of the estimation of total CPU training times are illustrated in Figure S5.
S3.6 Diversity Calculation