Hybrid Generative AI for De Novo Design of Co-Crystals with Enhanced Tabletability
Authors: Nina Gubina
Andrei Dmitrenko
Gleb Vitalevich Solovev
Lyubov Yamshchikova
Oleg Petrov
Ivan Lebedev
Nikita Serov
Grigorii Kirgizov
Nikolay Nikitin
Vladimir Vinogradov
Venue: NeurIPS 2024 poster (2024)
Keywords: Co-crystals
Tabletability
Generative Design
Evolutionary Optimization
Primary area: machine_learning_for_other_sciences_and_fields
TL;DR: We introduce GEMCODE, a new automated screening pipeline leveraging hybridization of generative AI and evolutionary optimization for de novo co-crystal design with enhanced tabletability properties.

Abstract: Co-crystallization is an accessible way to control physicochemical characteristics of organic crystals, which finds many biomedical applications. In this work, we present Generative Method for Co-crystal Design (GEMCODE), a novel pipeline for automated co-crystal screening based on the hybridization of deep generative models and evolutionary optimization for broader exploration of the target chemical space. GEMCODE enables fast *de novo* co-crystal design with target tabletability profiles, which is crucial for the development of pharmaceuticals. With a series of experimental studies highlighting validation and discovery cases, we show that GEMCODE is effective even under realistic computational constraints. Furthermore, we explore the potential of language models in generating co-crystals. Finally, we present numerous previously unknown co-crystals predicted by GEMCODE and discuss its potential in accelerating drug development.

Hybrid Generative AI for De Novo Design of Co-Crystals with Enhanced Tabletability
Nina Gubina'
Andrei Dmitrenko 1.2
Gleb Solovev'
Lyubov Yamshchikova'
Oleg Petrov* Ivan Lebedev'
Nikita Serov!
Grigorii Kirgizov!
Nikolay Nikitin Vladimir Vinogradov
ITMO University, St. Petersburg, Russia
2D ONE AG, Zurich, Switzerland
Ivanovo State University of Chemistry and Technology, Ivanovo, Russia dmi mmittrroddccamtmimmmoo
Abstract
Co-crystallization is an accessible way to control physicochemical characteristics of organic crystals, which finds many biomedical applications, In this work, present Generative Method for Co-crystal Design (GEMCODE) novel pipeline for automated co-crystal screening based on the hybridization of deep genera- tive models and evolutionary optimization for broader exploration of the target chemical space. GEMCODE enables fast de novo co-crystal design with target tabletability profiles, which is crucial for the development of pharmaceuticals. With series of experimental studies highlighting validation and discovery cases, we show that GEMCODE is effective even under realistic computational constraints. Furthermore, explore the potential of language models in generating co-crystals. Finally, we present numerous previously unknown co-crystals predicted by GEM- CODE and discuss its potential in accelerating drug development.
Introduction
The use of multi-component molecular crystals, specifically co-crystals, have become increasingly popular in various industries including energy [ electronics B2 optoelectronics [ food (6)
and especially in pharmaceuticals [7] Pharmaceutical co-crystals are defined as solids that are crystalline singlephase materials composed of drug molecule and an additional pharmaceutically acceptable molecule (coformer) [1O]. Co-crystals have a different crystal structure from the original components, leading to unique physicochemical properties. They are appealing because the resulting solid can exhibit better physicochemical properties compared to either of the pure molecules [11]. The formation of co-crystals has been shown to enhance characteristics such as bioavailability 12 13,, solubility (14166), stability [7H19), pharmacokinetics (20 221, and mechanical properties 14 22 23]. Plasticity is mechanical property that is particularly important for the pharmaceutical industry. It is known that highly plastic materials tend to produce stronger tablets compared to those exhibiting elastic behavior (24. In other words, it possesses improved tabletability, defined as the capacity of powdered material be transformed into tablet of specified strength under the effect of compaction pressure (25). Therefore, is essential to control for tabletability as it allows direct pressing with minimal addition of excipients to form stable compact tablet.
Despite all the robustness and versatility of co-crystals, determining the combination of a coformer and parent component with the desired property modification is an extremely non-trivial task, usually
https://github.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

addressed by experimental high- -hrohhhhuughputtt screening [26 27 Due to the large amounts of time and effort required, such studies remain targeted, focusing on rather narrow classes of candidate compounds.
Artificial intelligence (AI) methods have recently found their way into the field of chemistry 28 32). Since then, the accumulated experimental data has become the basis for predictive models transforming the traditional way science works. With big data and machine learning (ML), it is now possible to consider a much larger set of candidate molecules for given problem, rather than being satisfied with a limited number of experiments. Among the pioneering works in the co-crystal domain are the studies aimed at determining the probability of co-crystalliiztton of a particular molecular pair [33 (34. However, the sole fact of co-crystallization with no information about the properties of the resulting co-crystals is not enough to inform decision making for a specific use case. Accordingly, another direction of research investigated co-crystal properties with Al methods Still, prediction of most properties has been possible only in the case of already known co-crystallising molecular pairs. De novo design of co-crystals with predefined properties leveraging big data to cover large chemical space remains an actual task of great application value.
35 36
Therefore, here for the first time we develop a pipeline that generates coformer candidates based on the structure of a drug molecule to form a co-crystal with predefined mechanical properties. For that, we trained several state-of the-the-rt generative models on dataset of 1.75M chemical structures and then fine-tuned them on the state-of- sfte-off-hhe-rr art dataset of coformers. We then trained a classical ML model to predict plasticity parameters of the generated coformer candidates. We further employed evolutionary optimization leveraging the trained MI. models to improve the tabletability profiles of the generated coformers. Finally, we applied a pretrained graph neural network (GNN) to rank the molecular pairs according to the probability of successful co-crystal formation. We systematically evaluated and optimized the aforementioned individual components to assemble GEMCODE, practical solution achieving state-of-the-an performance even within computational constraints. The output GEMCODE is set of coformers forming co-crystal with improved tabletability properties for selected drug compound. Thus, the pipeline can serve as a tool for selecting the best molecular combination of an active pharmaceutical agent and a coformer delivering the desired properties of the co-crystal. In essence, this work makes the following novel contributions:
We train a transformer-basee new state of the art for the coformer generation task, and hybridize it with multi-objective evolutionary algorithm to improve the desired properties of coformers.
conditional variational autoencoder (T-CVAE) setting the
We develop machine learning models for the prediction of mechanical properties of co- crystals for the first time in the field.
We present GEMCODE, generative pipeline for de novo co-crystal design with target physicochemical properties contributing to drug tabletability.
In addition, we explore the capabilities of language models the coformer generation task.
Finally, we predict a set of molecules forming novel tabletable co-crystals with known drugs.
2 Related Work
2.1 Generative AI for molecule generation
Traditionally, the process of discovering new molecules or selecting chemical structures for a particular task relies on existing experimental evidence and subjective research experience, both limiting the number and variety of possible compounds to consider. Generative models allow efficient exploration of the molecular space, which has already caused a rapid growth of molecular generative design. Recurrent neural networks (37/40), variational autoencoders [41H44], generative adversarial networks 48 evolutionary algorithms 53] and hybrid models using reinforcement learning techniques 54 57 have been successfully applied for various problems in chemistry. In this work, we trained, evaluated and compared multiple generation approaches, such as LSTM-based GAN, waanformer-basee VAE and conditional VAE. The latter was inspired by study using conditional VAE model with an attention mechanism to generate molecules [58). However, our approach differs significantly in that we generated condition vector based on the predictions of the pretrained gradient-boosting model. In addition, our approach includes a fine-tuning phase on a state-of-the-a dataset of coformers.
2

---

DeepMind has recently presented GNoME, an Al tool for generating previously unknown inorganic crystalline materials [59]. Other similar tools exist for inorganic compounds [60] 61]. Our work also lies in the field of solid-state chemistry, but differs in the task of generating coformers, which are small organic molecules. To our knowledge, generative approaches have not yet been applied to produce coformer structures with high co-crystallization potential with drug targets. Our work effectively addresses this problem.
2.2 Co-crystal property prediction
Research in co-crystal property prediction is targeted at determining various parameters, such as the lattice energy, density, melting temperature. crystal density, enthalpy and entropy of melting, well as ideal mole fraction solubility of co-crystals 65] However, a limited number of samples is typically used in the training phase. For example, Gamidi and Rasmuson trained artificial neural network on the data of 30 co-crystal systems for 8 different drugs [35. Such models are likely to have very limited generalization power beyond the training data. The most recent model predicting the co-crystal density (36) used a large training set of 4144 molecular pairs covering much wider chemical space of possible co-crystals. In this work, we predict several mechanical properties of co-crystals for the first time. We use an even larger amount of data for that (6029 samples), which makes our approach more versatile and better generalizable for different pharmaceutical applications.
2.3 Applications of language models in chemistry
Large language models have recently been challenged with multiple chemistry tasks, such as property prediction, yield prediction, text-based molecular design, and others [66]. The results suggest that language models are less competitive generative tasks requiring a deeper understanding of molecular SMILES strings, but show competitive performance in classification and ranking tasks. Another study on the applicability of language models without prior specialization in the chemistry domain found that LLMs can effectively interpret chemical structures given various representations [67.. In addition, the use of language models as agents was explored in ChemCrow [68), which makes chemistry more accessible to researchers with less domain expertise. Following up these pioneering works, we explore the applicability of language models to the creation of coformer molecules with desired properties, which has not yet been addressed in the past.
3 Data
3.1
Data collection
Large dataset of molecules. In order to train a generative model capable of suggesting reasonable chemical structures, a dataset of molecules from the ChEMBL database (available with CC BY-SA 3.0 license) was collected. From the large variety of molecular structures available in the database, 1..55M samples were selected using criteria based the distributions of relevant parameters in the known coformers (Appendix C.T.) Using these criteria ensures that the generative models are trained on molecules capable of forming co-crystals.
Dataset of coformers. Chemical structures in the ChEMBL database are still substantially different from the structures composing co-crystals. Coformers most often have more basic chemical structures and a smaller variety of functional groups. Therefore, we used an open dataset of 6819 two-component co-crystals [33] (available with MIT license), which contains 4227 unique chemical structures of the coformers, for fine-tuning.
Dataset of co-crystals mechanical properties. For the mechanical properties of co-crystals, we used the Cambridge Structural Database (CSD) [69 and recently proposed protocol for geometric analysis of co-crystalline materials available with CSD Python API (24). For each of the 68 19 available co-crystals, we used the API to query additional experimental data from the CSD and calculate the following binary parameters of plasticity: presence of von-overlapping Miller planes (Unobstructed planes), presence of orthogonal planes (Orthogonal planes), and presence of hydrogen bonds between the planes (H-bond bridging). Since some of the co-crystals were missing in CSD, this process yielded total of 6029 records. This data was then used for training MI. models to predict each of the three plasticity parameters.
3

---

Evolutionary optimization
Therapeutic maleculc
10 C OC
Chemical synthesis
CNN
Ranked of co-erysials properties
Predicted propeity
Generative model
GB
Jahe List
Generated samples
Aud soforee 2005 nod coopee.
nod cok medde
Tablets
Figure I: GEMCODE: a pipeline for generative co-crystal design consisting of models (LSTM- GAN, T-VAE, T-CVAE) generating coformer candidates, gradient boosting (GB) classification models predicting the mechanical properties of co-crystals based on the generated coformers, an evolutionary algorithm producing additional coformer candidates with improved tabletability profiles, and graph neural network (GNN) ranking co-crystals according to the probability of formation.
We analyzed the number of samples for each plasticity parameter in the collected dataset (Ap- pendix C.2). In the case of orthogonal planes, we observed a dramatic difference between the two groups. When training the corresponding ML model, we accounted for this disproportion by adjusting a threshold probability for predicting positive class.
3.2 Data curation
Cutting-edge generative models use string 72 2D (73775) and 3D 76 molecular graphs molecular representations. The most common way is the SMILES (Simplified molecular-inpe line-entry system) notation, as the other approaches have not yet shaped the field to such an extent Therefore, we used the SMILES representations describe the composition and structure of chemical molecules with short strings. Additionally, molecular fingerprints allowed us to represent molecules vectorized form and compare different structures by calculating similarity measure (Appendix
We used RDKit to generate 43 molecular descriptors for each coformer with its SMILES representa- tion. Since co-erystals consist of two coformer components, each one was described by 86 numerical features in total. Before training ML models for the prediction of mechanical properties, we applied a set of preprocessing steps. We engineered new features by aggregating the molecular features of the coformers of the same co-crystal with summation and averaging. To reduce redundancy in the feature space, we investigated the feature importances using embedded methods and the degree of linear association with target variables through correlation coefficients. After feature engineering and filtering, the datasets for the prediction non-overlapping planes, orthogonal planes, and hydrogen bonding contained 29, 24, and 30 features, respectively.
4 GEMCODE: Generative Evolution-based Method for Co-crystal Design
We present GEMCODE, a novel pipeline for generative co-crystal design with improved tabletability properties. based on the idea of hybridization of deep generative models and combinatorial optimisation. GEMCODE consists of four key components, depicted on on|Fir
First, trained and fine-tuned generative model generates SMILES representations of coformer-like chemical structures. The generated molecules are then fed into the trained ML models along with the therapeutic compounds, where the mechanical properties of co-crystals are predicted. In addition, an evolutionary algorithm is used in combination with the ML models to further improve the tabletability of the generated coformers. Finally, co-crystals with the desired properties are selected for the next step, where pretrained graph neural network scores and ranks molecular pairs of drugs and coformers according to the probability of co-crystallization. Thus, the pipeline outputs a list of potential coformers with the desired mechanical properties of the co-crystal, ranked according to

---

the probability of successful co co-ccrystallizatinn. In the following sections, we describe the individual components of the pipeline in more detail.
4.1 Prediction of mechanical properties of co-crystals
Since the number of training examples available for prediction of mechanical properties was only 6029, we resorted to the classical machine learning algorithms. We formulated binary classification problem for each of the mechanical properties and implemented number of ML models as first screen, including logistic regression, k-nearest neighbors classifier, support vector machines, decision trees, multilayer perceptron, as well ensemble models, such as random forest and gradient boosting. We then selected the best models and optimized their hyperparameters to achieve top performance. Those pretrained models were then integrated into the coformer generation and the evolutionary optimization frameworks. To validate this solution, we used an AutoML tool to design the modeling pipeline in automated way (details are provided in Appendix G.3
4.2 Generation of coformers
The performance of a particular deep neural network largely determined by its architecture, as well the strategy to learn the hidden representations (80). In order to find the most effective solution for the coformer generation task, we implemented and systematically compared three different architectures. Our evaluation included GAN model with recurrent neural networks for both, generator and discriminator, and two transformer-basee models implementing a VAE. For more information regarding the model architectures, refer to Appendix and D.5]
GAN-based methods consider molecule generation minimax game, which consists of training discriminator to distinguish between the real data and the samples produced by a generator (Appendix D.1). In this work, we employed an open-source GAN implementation using LSTM to address molecule generation as sequence-lo-sequence (S2S) problem, inspired by the work of Autume [8I]. As an alternative, we opted for transformer architecture [2] as basis for a VAE, since it normally outperforms recurrent neural network architectures in S2S tasks [83].
Our objective was to produce co-crystals meeting specific tabletability requirements that translate to set of target mechanical properties. We utilized conditional variational autoencoder (CVAE) approach [84] to achieve this. By design, CVAE makes it possible to consider physicochemical characteristics of molecules and generate co-crystals with the desired properties (Appendix [D. offers a more detailed description of the VAE and CVAE models). We used the aforementioned mechanical properties (unobstructed planes, orthogonal planes, and H-bonds bridging) as conditions for CVAE. In the following, we refer this model as transformer-bart based CVAE (T-CVAE).
Finally, we included a transformer-based VAE (T-VAE) for comparison, which does not consider any specific properties of molecules, for completeness of the analysis.
4.3 Eyolutionary optimization of coformers
To increase the quality of coformer generation, we applied graph-based evolutionary algorithm to structures produced by the generative models. The software implementation is obtained from the self-developed GOLEM library [85). The fitness function was designed to reinforce the mechanical characteristics of molecules based on predictions of the classification models described above:
f(x) (1- 1 po)
where x an evaluated molecule of coformer, Pa (x) is the probability of the positive class for unobstructed planes, (x. is the same probability for orthogonal planes, and (.()) for H-bond bridging. Therefore, minimization of the fitness function f leads to generation of coformer molecules having improved tabletability profile.
5

---

Raw daca
Accuraty
Processed data
Accurasy
Score
Score
Unobstructed planes
b
Orthogonal planes
0
H-bonds bridging
SVM
SUM
0.2
0.4
0.6
0.8
0.2
0.4
0.6
0.8
0.2
0.4
0.6
Score
Score
Score
Figure Accuracy and Fl score metries for the ML models predicting three mechanical properties of co-crystals. (a) Unobstructed planes. (b) Orthogonal planes. (c) H-bonds bridging. The performance of each model is shown before ("Raw data") and after ("Processed data") the feature engineering and feature selection steps.
4.4 Estimation of probability of co-crystal formation
Determining the possibility of co-crystaliizaton by molecular pairing is an important step in the co-crystal design. For this reason, many works attempted to solve this problem with AI [86 34 87]. Most works that are closely related to our problem do not provide code to reproduce or reuse their results [T] To account for the probability of co-crystallization,
applied an existing GNN- based deep learning framework, called CCGNet [3 (available with MIT license). Unlike many of the previous works, CCGNet achieves state-of-the-art performance predicting co-crystal formation while being 100% open-source and easily reproducible. With an average balanced accuracy of 98.66 98.6%, CCGNet efficiently scores and ranks coformer candidates according the probability of co-crystal formation. Since CCGNet was originally trained on the same database of coformers, we did not perform any fine-tuning and simply integrated the model from the open GitHub repository into the pipeline.
Experimental studies
5.1 Prediction of mechanical properties of co-crystals
Implementation details. The preprocessed dataset was randomly split into train and test sets in proportion 4:1. The train set was used to optimize hyperparameters of the models with grid search using the 10-fold cross-vasd validation (CV). The random grid size was 500 and concerned the following parameters: learning rate, number of estimators, subsample, maximum depth of the individual estimators. The test set was used only once, to evaluate and report the performance of the optimized models. We calculated accuracy and score during the CV to select the best hyperparameter set. The use of the two metrics was important given the imbalanced nature of the "Orthogonal planes" and "Unobstructed planes" target variables (Appendix C.2 Figure 3d). To account for the disproportion, we also adjusted the threshold for the probability of the positive class by calculating precision and recall metrics. Finally, we employed SHapley Additive exPlanations (SHAP) to interpret model predictions, which is based sensitivity analysis investigating the effect of systematic changes in feature values on the model output [92].
Overall, the GB model showed the best accuracy and Fl score compared to the other models across all tasks (Figure 2p. Despite the high accuracy for the orthogonal planes parameter, we obtained moderate Fl score suggesting that the final model is more likely to predict the absence of the orthogonal planes. This is attributed to the disproportion in the training examples discussed earlier. Although demonstrated significant improvement in metrics by introducing the probability threshold (Appendix[G.. G.2) evaluating the model trained on the processed data, it was not enough to entirely resolve this issue.
Results.
We optimized the hyperparameters of the Gradient Boosting (GB) model, which resulted in the performance metrics outlined in Table 10 (Appendix G.4). Furthermore, conducted a thorough

---

Table 1: Results of the coformer generation comparison.
Model
GAN
T-VAE
T-CVAE
Validity (t), %
94.57 + 0.00
99.70 + 0.00
98.40 + 0.00
Novelty (f),
94.90 + 0.08
95.12 0.11
80.62 0.25
Duplicates (L), %
42.29 + 0.69
24.30 + 0.45
55.70 + 0.19
Target coformers (t), %
2.23 =: 0.17
1.68 + 0.12
5.63 + 0.22
Diversity of target (1)
0.30 + 0.00
0.31 + 0.00
0.25 0.00
review of the existing research on the prediction of co-crystal properties to compare with our results. Notably, we are the first to develop predictive models for the plasticity parameters, so our metrics set the state of the art. In addition, work clearly stands out by the number of data points used for training.
With SHAP analysis (Appendix G.3 we learned that the number of atoms among the molecular pairs forming a co-crystal is a decisive factor the prediction of non-overlapping and orthogonal planes. In both cases, the decrease in the number of atoms in the coformer molecules significantly contributed to the presence of non-overlapping and orthogonal planes. The descriptors associated with the number of hydrogen bond donors (HBD) also had a high degree of importance. As expected, an increase in the number of HBD resulted in the hydrogen bonds forming between planes of the co-crystal.
5.2 Generation of coformers
Implementation details. The performance of generative models depends on hyperparameters and random restarts [93]. A grid search was implemented to select the best hyperparameters. and multiple trainings were conducted. The generative model was focused on generating coformer-like chemical structures, so it was pretrained on the ChEMBL dataset and then fine-tuned on dataset of coformers. The importance of fine-tuning was illustrated using t-distributed stochastic neighbor embedding (t-SNE) to visualize the datasets (Appendix (.2). To evaluate the trained models, ten sets of 10,000 molecules were generated and various indicators were calculated, including validity (defined as the percentage of chemically plausible molecules all generated), novelty (defined the percentage of newly generated molecules that are not contained within the training set to all generated), percentage of duplicate molecules, percentage of target coformers, and diversity. More details on these indicators can be found in the Appendix
Results. Analyzing experimental results of coformer generation, we observed that T-VAE produced the highest percent of valid and novel molecules with by far the lowest percent of duplicated structures (Table 1). However, among the generated coformers, only 1.68% had the target tabletability profile, as assessed by the pretrained classification models. In contrast, when generating 10,000 candidates, T-CVAE produced 5.63% of new coformers with the required mechanical properties on average. While the diversity of target coformers Fl was slightly higher for GAN, it was able to produce the intermediate 2.23% of such coformers. Therefore, we conclude that T-CVAE was the most effective approach to target coformer generation. However, the transformer architecture was also the most demanding for both, the training and the generation phases (see Appendix [D6 for more details).
Ultimately, we recommend to use an ensemble of generative models whenever sufficient computa- tional resources are available. Our findings presented in Appendix D.7 suggest that the three models produce complementary results. Collectively, GAN, T-VAE and T-CVAE generate up to 2.47 times more unique target coformers than individually.
Additional experiments with language models. Inspired by the most recent applications of language models in chemistry 68 94] we investigated their potential in the coformer generation task. First, we employed reduced GPT-2 model with eight heads, four attention blocks, and 14.7M parameters. Similarly to other models, GPT-2 was pre-trained on the ChEMBL dataset and then
'A numerical comparison of the target molecules generated by the models and those available in the training dataset is given in the Appendix D.7
7

---

Table 2: Results and statistical significance of the evolutionary optimization.
Median probability (t) Generated
Model
Property
Optimized
Padj
Novelty (1)
Unobstructed planes
0.82
0.82 (+0.0%)
GAN
Orthogonal planes
0.37
0.39 (+5.4%
2.68e-11
0.68
H-bond bridging
0.62
0.69 (+11.3%)
1.05e-66
Unobstructed planes
0.82
0.82 (+0.0%)
T-VAE
Orthogonal planes
0.38
0,40 (+5.3%)
2.71e-9
0.72
H-bond bridging
0.64
0.69 (+7.8%)
1.76e-65
Unobstructed planes
0.82
0.83 (+1.2%)
9.52c-05
T-CVAE
Orthogonal planes
0.38
0.39 (+2.6%)
1.88e-9
0.60
H-bond bridging
0.64
0.69 (+7.8%)
1,82c-46
fine-tuned on the coformers dataset (see Appendix D.9 for more details). We observed that GPT-2 produced significantly lower percent of new and valid structures per 10,000 generations (Appendix D.10). Nevertheless, the model achieved 3.32% of new molecules with the desired physicochemical properties, which comparable to GAN and T-VAE. These results prompted to further test more recent and capable language model. Therefore, we trained Llama-3-8B with the low-rank adoption (LoRA) algorithm using the same ChEMBL. and coformer datasets. We observed major improvements in validity, novelty and the number of duplicates among the generated molecules compared to GPT-2. Notably, Llama-3-8B produced the maximum diversity percent compared to all the tested generative approaches. However, the number of molecules with target physicochemical properties dropped to only 0.34%. Analyzing these empirical results, we conclude that language models show good potential in the coformer generation task but have to be heavily optimized to achieve competitive performance with GEMCODE. We leave this endeavour for the future work.
5.3 Evolutionary optimization of coformers
Implementation details. The multi-objective optimization algorithm used in this work considers molecules as undirected graphs and follows the generational evolutionary scheme MOEA/D [95]. First, a population of individuals is evaluated with the fitness function. Then, MOEA/D- selection is applied to pick individuals from the population to undergo mutation. After the variation by mutation is done, the inheritance operator used form the new population of individuals to proceed to the next iteration (see Appendix E. F.2 for more details).
To choose an effective evolutionary scheme for the task compared SPEA-2 (96) and MOEA/D (see Appendix E.5). Experiments have shown MOEA/D obtaining better results in some cases.
The initial population of coformer structures (obtained with the previously described generative models) were varied by the set mutation operators, inspired by the work Leguy [50]. The set of mutations includes simple operations (add, delete, or replace atom, delete or replace bond) and more complicated, multi-step actions (delete or move functional group, insert carbon, remove an atom if it has only two neighbors). See Appendix E.3 for more details optimization runs.
Results. To evaluate results of the evolutionary search, we compared the probabilities of co- formers to possess the desired mechanical properties before ("Generated") and after ("Optimized")) evolutionary optimization (Table 2). For that, we used the pretrained ML models to retrieve the probabilities, calculated statisties and applied the non- parametric middideed Mann-Wol Hann-WWitney test (see Appendix E.4 F.I for more details). In most cases, we observed a significant increase in the median probability of the target class. Notably, evolutionary optimization equalized the performance of different generative models in their ability to produce coformers with the target tabletability profile. Moreover, this process consistently yielded new coformer structures, not present in the training set or in the initial population.
'Median probability score can be best described as the median probability of assigning coformers to a positive class for each of the mechanical properties. In other words, median probability score gives idea about the central tendency of the model's confidence in predicting particular mechanical property.

---

Table 3: Experimentally validated coformers improving drug tabletability generated by GEMCODE. SMILES were selected based on two tabletability parameters (Unobstructed planes, H-bond bridging) and similarity metric (IT = 1).
Drug
Generated SMILES
CSD Refcode
Model
Ref.
Nicorandil
O=C(O)C=CC(=0))
WAHGEV
97
Rivaroxaban
O=C([[-))C(<=)]00)
YORVEJ
T-VAE
98
C1=0C=C2c=CC=
LUIST
GAN/T-VAE SETT-VVA//T-CVAE
Paracetamol
[N++(CCC)CC(()))))
CUQKAC
T-CVAE
100
5.4
Validation case studies
In order test the effectiveness of GEMCODE, we generated coformers for the drugs with poor ability to form a tablet by powder pressing. Among the therapeutic molecules selected for the pipeline validation were Nicorandil, Rivaroxaban and Paracetamol. For each of the listed drugs, experimentally validated molecules were found among the GEMCODE-generated coformers improving tabletability of the co-crystals (Table 3). More details can be found Appendix B.I
5.5 Novel coformer molecules predicted by GEMCODE
To showcase the ability of GEMCODE to predict novel coformers with target tabletability profiles, we generated coformers for one of the therapeutic molecules, i.e., Nicorandil. GEMCODE enabled discovery of 23 unique coformer with improved mechanical properties and with the presence of func- tional groups as in experimentally validated tabletable co-crystals (see Tabll in the Appendix [B.2). This result demonstrates the potential of GEMCODE as indispensable tool for accelerated drug development. Broader impact further discussed in Appendix A
Limitations
The evidence presented above looks very promising for the practical applications of our pipeline. However, comprehensive experimental validation involving organic synthesis of coformers and co-crystal formation followed by a tablet compression experiment required to confirm its utility. Based
empirical results, we anticipate the following limitations of the proposed pipeline:
The coformers molecular space may be too narrow for some applications due the small sample size of the coformer dataset. Nevertheless, if computational power is available, is possible to increasing the number of unique molecules generated.
ensemble of generative models, which partially solves the problem by
Currently, the GB model is biased towards predicting the absence of orthogonal planes, leading more false negatives in the predicted coformers. We recommend exploring an alternative set of coformers based on the other two mechanical properties only.
Low-scale screening may still result in some coformers failing to form co-crystals, particu- larly those optimized through evolution. Screening more coformers increases the chances of finding co-crystal pairs for a specific therapeutic agent.
While polymorphism's impact on predicting co-crystal mechanical properties is not exam- ined here, its significance is undeniable and often understated. Despite limited reported polymorphs, their potential impact on prediction model accuracy in the co-crystal field necessitates further exploration, considering the current scarcity of polymorphism data.
Most limitations of the proposed pipeline can be solved with more data available for training, which remains major challenge for successful Al applications in no-crystallization. We are working towards collecting more data and improving its quality. Also, to date GEMCODE has been adapted mainly for pharmaceutical applications. In the future, we plan to extend GEMCODE by adding more predicted physicochemical properties and other crystal forms to be able to expand beyond the pharmaceutical field.

---

7 Conclusion
In this work, we presented GEMCODE, novel generative pipeline for de novo co-crystal design. To make it as effective as possible, we implemented the hybrid generative approach combininig positive sides of both deep learning models and combinatorial optimisation. We systematically evaluated and discussed the individual components of the pipeline achieving state-of-the performance in the corresponding tasks. Furthermore, we performed experiments to validate the pipeline by generating coformers for three different drugs and discovering previously unknown coformers for Nicorandil. In addition, we explored the applicability of language models in the coformer generation task and identified prospective research directions. Despite limitations associated with data availability, GEMCODE enables fast generation of unique and valid chemical structures of coformers with high probabilities of co-crystallization and target tabletability profiles. This research enhances co-crystal design for pharmaceuticals and contributes to the accelerated drug development. Thanks to data and code availability, our versatile hybrid approach might find other impactful applications in chemistry.
8
Acknowledgements
This research is financially supported by the Foundation for National Technology Initiative's Projects Support as part of the roadmap implementation for the development of the high-tech field of Artificial Intelligence for the period up to 2030 (agreement 70-2021-00187)
References
[1] Li, H.-j. J.-c Liu, L. Yang, et al. Theoretical predict structure and property of the novel CL-20/2,4-DNI cocrystal by systematic search approach. Defence Technology, 18(6):907-99) 2022.
[2] Wang, Z., Q. Zhang. Organic Donor- Acceptr Cocrystals for Multiferroic Applications. Asian Journal of Organic Chemistry, 9(9):1252 119:11222-261, 2020.
[3] Payne, S., I. Andrusenko, F. Papi, al. The crystal structure and electronic properties of three novel charge transfer co-crystals tenqf n-triphenylene (n= 0, 2, 4). CrystEngComm, 25(5):828-25 834, 2023.
[4] Zou, J. J. Chang. Q. Chen, et al. Novel strategy for organic cocrystals of n-type and p-type organic semiconductors with advanced optoelectronic properties. ACS omega, 5(21):1206) 12072, 2020.
[5] Abou Taka, A., J. Reynolds, N. C. Cole-Filipiak, et al. Comparing the structures and photophysical properties of two charge transfer co-crystals. Physical Chemistry Chemical Physics, 25(40):27065-7707 270074, 2023.
[6] Dias, L., M. Lanza, S. R. Ferreira. Cocrystallization: tool to modulate physicochemical and biological properties of food-relevant polyphenols. Trends in Food Science & Technology, 1 110:13-20 27, 2021.
[7] Guo, M. X. Sun, Chen, et al. Pharmaceutical cocrystals: A review of preparations, physicochemical properties and applications. Acta Pharmaceutica Sinica B, 11(8):253722544 2021.
[8] Shaik, A., P. U. Bhagwat, P. Palanisamy, et al. Novel pharmaceutical co-oirccuttcalloccsstllllll of gefitinib: synthesis, dissolution, cytotoxicity, and theoretical studies. CrystEngComm, 25(17):2570- 2581, 2023.
[9] Jia, X.-m., H. Hao. Q. Zhang, et al. The bioavailability enhancement and insight into the action mechanism of poorly soluble natural compounds from co-crystals preparation: Oridonin as an example. Phytomedicine, 122: 125:155,79, 2024.
[10] Bolla, G., B. Sarma, A. K. Nangia. Crystal Engineering of Pharmaceutical Cocrystals in the Discovery and Development of Improved Drugs Chemical Reviews, 122(13): 1514-11603, 2022.
10

---

[11] Karimi-Jafari, M., L. Padrela, G. M. Walker, et al. Creating cocrystals: A review of pharma- ceutical cocrystal preparation routes and applications. Crystal Growth & Design, 18(10):637- 6387, 2018.
[12] Emami, S., M. Siahi-Shadbad, K. Adibkia, et al. Recent advances in improving oral drug bioavailability by cocrystals. Biolmpacts, 8(4):305 320, 2018.
[13] Sakamoto, N., K. Miyata, T. Fukami. Quabodepistal (opc-167832)) a novel antituberculosis drug candidate: Enhancing oral bioavailability via cocrystallization and mechanistic analysis of bioavailability in two cocrystals. Molecular Pharmaceutics, 2023.
[14] He, H., Q. Zhang, M. Li, et al. Modulating the Dissolution and Mechanical Properties of Resveratrol by Cocrystallization. Crystal Growth & Design, 17(7):399-3396, 2017.
[15] Xia, M., Y. Jiang Y. Cheng, et al. Rucaparib cocrystal: Improved solubility and bioavailability over camsylate. International Journal of Pharmaceutics, 631: 2023.
[16] Imanto, T., E. R. Wikantyasning, S. Nurwaini, al. Preparation and solid-state characterization of sitoprofen-succinie acid-saccharin co-crystal with improved solubility. Int J Appl Pharm, 16(1), 2024.
[17] Li, D., J. Li, Z. Deng, et al. Piroxicam Pirxicaa-clooiiiin clonixin drug- drug cocrystal solvates with enhanced hydration stability. CrystEng 21(28):4145-411 2019.
[18] Haneef, J. M. Amir, N. A. Sheikh, al. Mitigating drug stability challenges through cocrystallization. AAPS PharmSciTech, 24(2):62, 2023.
[19] Shi, J., Y. Zhang, Q. An, et al. Improving the sublimation stability of ligustrazine with gallic acid by forming pharmaceutical cocrystal based on the etter's rules. Journal of Solid State Chemistry, page 124545, 2024.
[20] Xu, D., G.-Q. Zhang. T.-T. Zhang, al. Pharmacokinetic comparisons of naringenin and maringenin-nicotinaniie cocrystal in rats by lc-ms/ms. Journal of Analytical Methods in Chemistry, 2020, 2020.
[21] Haskins, M. M., O. N. Kavanagh, R. Sanii, et al. Tuning the pharmacokinetic performance of quercetin by cocrystallization. Crystal Growth & Design, 23(8))6059-6008 2023.
[22] Bag, P. P. Introduction of plasticity to change mechanical behaviour of pharmaceutical crystals by co-crystallization: a solution of long standing problem in isoniazid. Engineered Science, 15:129 137, 2021.
[23] Ouyang, J., Liu, Y. Li, al. Cocrystals of carbamazepine: Structure, mechanical properties, fluorescence properties, solubility, and dissolution rate. Particuology, 90:20-30, 2024.
[24] Bryant, M. J., A. G. P. Maloney, R. A. Sykes. Predicting mechanical properties of crystalline materials through topological analysis. CrystEngCommn, 20(19):269--2700 2018
[25] Roy, P., A. Ghosh. Mechanochemical cocrystallization to improve the physicochemical properties of chlorzoxazone. CrystEngComm,, (2227)4461144222 2020.
[26] Rodrigues, M., J. Lopes, A. Guedes, al. Considerations on high- taob-hhhoughput cocrystals screening by ultrasound assisted cocrystallization and vibrational spectroscopy. Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy, 229:1 17876, 2020.
[27] Khudaida, S. H., Y.-T. Yen, C.-S. Su. Cocrystal screening of anticancer drug p- toluenesulfonamide and preparation by supercritical antisolvent process. The Journal of Supercritical Fluids, 204:106106, 2024.
[28] De Almeida, A. F., R. Moreira, T. Rodrigues. Synthetic organic chemistry driven by artificial intelligence. Nature Reviews Chemistry, 3(10)559-6604 2019.
[29] Gormley, J., M. A. Webb. Machine learning in combinatorial polymer chemistry. Nature Reviews Materials, 6(8):642-644, 2021.

---

[30] Cerchia, C., A. Lavecchia. New avenues in artificial-intelligenne intliiieeiiiigeece--ssseeedd assisted drug discovery. Drug Discovery Today. 28(4):103516, 2023.
[31] Westermayr, J., J. Gilkes, R. Barrett, et al. High-throughput property-driven generative design functional organic molecules. Nature Computational Science, 109):139-448 2023.
[32] Lu, B., Y. Xia, Y. Ren, et al. When machine learning meets 2d materials: A review, Advanced Science, page 2305277, 2024.
[33] Jiang, Y. Z. Yang. J. Guo, et al. Coupling complementary strategy to flexible graph neural net- work for quick discovery of coformer diverse co-crystal materials. Nature Communications, 12(1):5950, 2021.
[34] Vriza, A., Sovago, D. Widdowson, et al. Molecular set transformer: attending to the co-crystals in the Cambridge structural database. Digital Discovery, 1(6):834-88) 850, 2022.
[35] Gamidi, R. K., A. C. Rasmuson. Analysis and Artificial Neural Network Prediction of Melting Properties and Ideal Mole fraction Solubility of Cocrystals. Crystal Growth & Design, 20(9):5745-57555 5755, 2020.
[36] Guo, J., M. Sun, X. Zhao, al. General Graph Neural Network-Based Model To Accurately Predict Cocrystal Density and Insight from Data Quality and Feature Representation. Journal of Chemical Information and Modeling, 63(4): 143-11166,, 2023.
[37] Grisoni, F,, M. Moret, R. Lingwood, et al. Bidirectional Molecule Generation with Recurrent Neural Networks. Journal of Chemical Information and Modeling,
2020.
[38] Li, X., Y. Xu, H. Yao, et al. Chemical space exploration based on recurrent neural networks: applications in discovering kinase inhibitors. Journal of cheminformatics, 12(1):1-13, 2020.
[39] Suresh, N., N. Chinnakonda Ashok Kumar, S. Subramanian, et al. Memory augmented recurrent neural networks for de-novo drug design. Plos one, 17(6):e0269461, 2022.
[40] Dollar, O., Joshi, D. A. Beck, et al. Attention based generative models for de novo molecular design. Chemical Science, 12(24):8322-8772, 2021.
[41] Gómez-Bombarehe R., J. N. Wei, D. Duvenaud, et al. Automatic Chemical Design Using Data-Driven Continuous Representation of Molecules. ACS Central Science, 4(2):268-276, 2018.
[42] Lee, M., K. Min. Mgevae: multi-objective inverse design via molecular graph conditional variational autoencoder. Journal of Chemical Information and Modeling, 62(12):2943-29) 2022.
[43] Ochiai, T., T. Inukai, M. Akiyama, al. Variational autoencoder- based chemical latent space for large molecular structures with 3d complexity. Communications Chemistry, 6(1):249, 2023.
[44] Bhadwal, A. S., K. Kumar, N. Kumar. Gmg-nedvae: Guided de novo molecule generation using nip techniques and constrained diverse variational autoencoder. ACM Transactions on Asian and Low- Res-uuee Language Information Processing, 2023.
[45] Guimaraes, G. L., B. Sanchez-Lengeling, C. Outeiral, et al. Objective- Objective-Reinforeed Reinforced Generative Adversarial Networks (ORGAN) for Sequence Generation Models. 2017. Publisher: arXiv Version Number: 3.
[46] Prykhodko, O., S. V. Johansson, P.-C. Kotsias, et al. A de novo molecular generation method using latent vector based generative adversarial network. Journal of Cheminformatiic, 11(1):74, 2019.
[47] Pang, C., J. Qiao, X. Zeng, et al. Deep generative models in de novo drug molecule generation. Journal of Chemical Information and Modeling, 2023.
12

---

[48] Macedo, B., I. Ribeiro Vaz, T. Taveira Gomes. Medgan: optimized generative adversarial network with graph convolutional networks for novel molecule design. Scientific Reports, 14(1):1212, 2024.
[49] Yoshikawa, N., Terayama, M. Sumita, et al. Population- based de novo molecule generation, using grammatical evolution. Chemistry Letters, 47(11):1431-1444 2018.
[50] Leguy, J., Cauchy, Glavatskikh, et al. Evomol: a flexible and interpretable evolutionary algorithm for unbiased de novo molecular generation. Journal of cheminformatics, 12(1))1-19, 2020.
[51] Kerstjens, A., H. De Winter. Leadd: Lamarckian evolutionary algorithm for de novo drug design. Journal of Cheminformatics, 14(1):1-20, 2022.
[52] Jensen, J. H. A graph- genetic algorithm and generative model/monte carlo tree search for the exploration of chemical space. Chemical science, 10(12):367-3372, 2019.
[53] Tripp, A., J. M. Heenández-Lohatoo Genetic algorithms strong baselines for molecule generation. arXiv preprint arXiv:2310.09267 2023.
[54] Putin, F.,, A. Asadulaev, Y. Ivanenkov, al. Reinforced Adversarial Neural Computer for de Novo Molecular Design. Journal of Chemical Information and Modeling, 58(6):1194-13) 2018.
[55] Thomas, M., N. M. O'Boyle, A. Bender, et al. Augmented hill-climb increases reinforcement learning efficiency for language-based de novo molecule generation. Journal of cheminformat- ics, 14(1):68, 2022.
[56] Zhavoronkov, A., A. Ivanenkov, Aliper, et al. Deep learning enables rapid identification of potent DDRI kinase inhibitors. Nature Biotechnology, 37(9):1038 10111008-1400 1040, 2019.
[57] Xiong, Y., Wang, Y. Wang, al. Improving drug discovery with hybrid deep generative model using reinforcement learning trained on a bayesian docking approximation. Journal of Computer-Aided Molecular Design, 37(11):507-517, 2023.
[58] Kim, H., J. Na, W. B. Lee. Generative chemical transformer: neural machine learning of molecular geometric structures from chemical language via attention. Journal of chemical information and modeling, 61(12):5804-53 5814, 2021.
[59] Merchant, A., S. Batzner, S. S. Schoenholz, et al. Scaling deep learning for materials discovery. Nature, pages 1-6, 2023.
[60] Bai, J., Z. Lai, R. Yang, et al. Imitation refinement for x-ray diffraction signal process- ing. In ICASSP 2019.2011 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3337-3341. IEEE, 2019.
[61] Bai, J., Y. Du, Y. Wang. al. Xtal2dos: Attention-based crystal sequence learning for density of states prediction. arXiv preprint wrXiv:2302.014480 2023.
[62] Gamidi, R. K., A. C. Rasmuson. Estimation of Melting Temperature of Molecular Cocrystals Using Artificial Neural Network Model. Crystal Growth & Design, 17(1):175-182, 2017.
[63] Rama Krishna, G., M. Ukrainezyk, J. Zeglinski, et Prediction of Solid State Properties of Cocrystals Using Artificial Neural Network Modeling. Crystal Growth & Design, 18(1):133- 144, 2018.
[64] Fathollahi, M., H. Sajady. Prediction of density of energetic cocrystals based on QSPR modeling using artificial neural network. Structural Chemistry, 29(4): 119-1181, 2018.
[65] Yue, H. J. Wang, M. Lu. Neural Network Prediction Model of Cocrystal Melting Temperature Based on Molecular Descriptors and Graphs. Crystal Growth & Design, 23(4):2540-255 2023.
13

---

[66] Guo, T., B. Nan, Z. Liang, et al. What can large language models do in chemistry? comprehensive benchmark on eight tasks. Advances Neural Information Processing Systems, 06:59662-59688 2023.
[67] Jablonka, K. M., P. Schwaller, A. Ortega-Guerreng et al. Leveraging large language models for predictive chemistry. Nature Machine Intelligence, pages 1-9, 2024.
[68] M. Bran, A., S. Cox, O. Schilter, al. Augmenting large language models with chemistry tools. Nature Machine Intelligence, pages 1-11, 2024.
[69] Groom, C. R., J. Brumo, M. P. Lightfoot, et al. The cambridge structural database. Acta Crys- tallographica Section B: Structural Science, Crystal Engineering and Materials, 72(2):171- 179, 2016.
[70] Blaschke, J. Arús-Pous, H. Chen, et al. Reinvent 2.0: an ai tool for de novo drug design. Journal of chemical information and modeling, (0(12):5918-599 2020.
[71] Hu, F., D. Wang, Y. Hu, et al. Generating novel compounds targeting sars-cov-2 main protease based on imbalanced dataset. In 2020 JEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 432-436. IEEE, 2020.
[72] Ishida, S., Aasawat, M. Sumita, et al. Chemtsv2: Functional molecular design using de novo molecule generator. Wiley Interdisciplinary Reviews: Computational Molecular Science, 1366), 2023.
[73] Li, Y., I.. Zhang, 7. Liu. Multi-objective de novo drug design with conditional graph generative model. Journal of cheminformaties, 10:1-24, 2018.
[74] Luo, Y., K. Yan, S. Ji. Graphdf: A discrete flow model for molecular graph generation. In M. Meila, T. Zhang, eds., Proceedings of the 38th International Conference on Machine Learning, vol. 139 of Proceedings of Machine Learning Research, pages 7192-7203. PMLR, 2021.
[75] Fang, Y., X. Pan, H.-B. Shen. De novo drug design by iterative multiobjective deep reinforce- ment learning with graph-based molecular quality assessment. Bioinformatics, 39t4) 2023.
[76] Skalic, M., D. Sabbadin, B. Sattarov, et al. From target to drug: generative modeling for the multimodal structure-based ligand design. Molecular pharmaceuties, 16(10):4882-4211, 2019.
[77] Li, Y., J. Pei, L. Lai. Structure-based de novo drug design using 3d deep generative models. Chemical science, 12(41):13644-1377 2021.
[78] Baillif, B., J. Cole, P. McCabe, et al. Deep generative models for 3d molecular structure. Current Opinion Structural Biology, 80:102566, 2023.
[79] Martinelli, D. D. Generative machine learning for de novo drug discovery: A systematic review. Computers in Biology and Medicine, 145:105403, 2022.
[80] Bilodeau, C., W. Jin, T. Jaakkola, et al. Generative models for molecular discovery: Recent advances and challenges. WIREs Computational Molecular Science, 12(5), 2022.
[81] d' Autume, C. d. M., M. Rosca, J. Rae, et al. Training language GANs from Scratch, 2020. ArXiv: .09922 [cs, stat].
[82] Vaswani, A., N. Shazeer, N. Parmar, et al. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[83] Karita, S., N. Chen, T. Hayashi, et al. A comparative study on transformer vs rnn in speech applications. In 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 449-456. 2019.
[84] Kingma, P., S. Mohamed, D. Jimenez Rezende, et al. Semi-supervised learning with deep generative models. Advances in neural information processing systems, 27, 2014.
14

---

[85] Pinchuk, M., G. Kirgizov, L. Yamshchikova, et al. Golem: Flexible evolutionary design of graph representations of physical and digital objects. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, pages 1668-1675. 2024.
[86] Wicker, J. G. P., M. Crowley, O. Robshaw, et al. Will they co-crystallize? CrystEngCommm, 19(36):53365555 -5400, 2017.
[87] Yang, D., L.. Wang. Yuan, et al. Cocrystal virtual screening based on the XGBoost machine learning model. Chinese Chemical Letters, 34(8):107964, 2023.
[88] Wang, D., Z. Yang, B. Zhu, et al. Bocchne-Learning-Guuss Cocrystal Prediction Based on Large Data Base. Crystal Growth Design, 20(10):6610-6621, 2020.
[89] Devogelaer, J., H. Meekes, P. Tinnemans, et al. Co-crystal Prediction by Artificial Neural Networks**; Angewandte Chemie International Edition, 59(49)21111-217718 2020.
[90] Mswahili, M. E., M.-J. Lee, G. L. Martin, et al. Cocrystal Prediction Using Machine Learning Models and Descriptors. Applied Sciences, 11(3):1323, 1323, 2021.
[91] Zheng. L., B. Zhu, Z. Wu, al. Pharmaceutical cocrystal discovery via 3d-sminbr: A new network recommendation tool augmented by 3d molecular conformations. Journal of Chemical Information and Modeling, 63(1 (3114)4301-453 2023.
[92] Lundberg, S., S.-I. Lee. A Unified Approach to Interpreting Model Predictions. 2017. Publisher: arviv Version Number: 2.
[93] Lucic, M., K. Kurach, M. Michalski, et al. Are GANs Created Equal? A Large-Scale Study, 2018. ArX1v:1711.10337 [cs, stat].
[94] Jablonka, K. M., Q. Ai, A. Al-Feghali, et al. 14 examples of how Ilms can transform materials science and chemistry: reflection on large language model hackathon. Digital Discovery, 2(5):1233-1250) 2023.
[95] Zhang, Q.. H. Li. Moea/d: multiobjective evolutionary algorithm based on decomposition. IEEE Transactions evolutionary computation, 1 116):712-731, 2007. 2007.
[96] Zitzler, E., M. Laumanns, L. Thiele. Spea2: Improving the strength pareto evolutionary algorithm. TIK report, 103, 2001.
[97] Mannava, M. C., A. Gunnam, A. Lodagekar, et al. Enhanced solubility, permeability, and tabletability of nicorandil by salt and cocrystal formation. CrystEng 23(1):227-237, 2021.
[98] Kale, P., Puri, A. Kumar, al. The role of coerystalizzaiionmeeiaae altered crystal- lographic properties on the tabletability of rivaroxaban and malonic acid. Pharmaceutics, 12(6):546, 2020.
[99] Karki, S., T. Friščié, L. Fábián, et al. Improving mechanical properties of crystalline solids by cocrystal formation: new compressible forms of paracetamol. Advanced materials, 21(38- 39):3905-3909 2009.
[100] Maeno, Y., T. Fukami, M. Kawahata, et al. Novel pharmaceutical cocrystal consisting of paracetamol and trimethylglycine, a new promising cocrystal former. International journal of pharmaceutics, (73(1-2):179-186) 2014.
[101] Sun, C. C. Materials Science Tetrahedron Useful Tool for Pharmaceutical Research and Development. Journal of Pharnacentical Sciences, (8(5):667111)) 2009.
[102] Krishna, G. R., L. Shi, P. P. Bag, et al. Correlation Among Crystal Structure, Mechanical Behavior, and Tabletability in the Co-Crystals of Vanillin Isomers. Crystal Growth & Design, 1884):1877-1832, 2015.
[103] Reddy, M., M. T. Kirchner, R. C. Gundakaram, et al. Isostructurality, Polymorphism and Mechanical Properties of Some Hexahalogenated Benzenes: The Nature of Halogen-Halogenn Interactions. Chemistry A European Journal, 12(8):2222221 222242222222222222222222122222 2006.
15

---

[104] David, L.. A. Thakkar, R. Mercado, et al. Molecular representations in Al-driven drug discovery: a review and practical guide. Journal of Cheminformatics, 12(1):56, 2020.
[105] Killoran, N., L. Lee, A. Delong, et al. Generating and designing DNA with deep generative models. 2017. Publisher: arXiv Version Number: 1.
[106]
Goodfellow, J., J. Pouget-Abadie, M. Mirza, al. Generative Adversarial Networks, 2014. Publisher: Xiv Version Number:
[107] Schawinski, K., C. Zhang, H. Zhang, et al. Generative Adversarial Networks recover features in astrophysical images of galaxies beyond the deconvolution limit. Monthly Notices of the Royal Astronomical Society: Letters, page slx008, 2017.
[108] De Oliveira, L., M. Paganini, B. Nachman. Learning Particle Physics by Example: Location- Aware Generative Adversarial Networks for Physics Synthesis. Computing and Software for Big Science, 1(1):, 2017.
[109] Wang. C., G. Yang, G. Papanastasiou, et al. DiCye: GAN-based deformation invariant cross- domain information fusion for medical image synthesis. Information Fusion, 67:147-160, 2021.
[110] Xiong, R., Y. Yang, D. He, et al. On layer normalization in the transformer architecture. In International Conference on Machine Learning, pages 10524-10533. PMLR, 2020.
[111] He, J., D. Spokoyny, G. Neubig, et al. Lagging inference networks and posterior collapse in variational autoencoders. arXiv preprint /01.05590,..5534 2019.
[112] Bowman, S. R., L. Vilnis, O. Vinyals, et al. Generating sentences from a continuous space. arXiv preprint arXiv:1511.06349 2015.
[113] Holtzman, A., J. Buys, L. Du, et al. The curious case of neural text degeneration. arXiv preprint arXiv:190400975:: 2019.
[114] Hu, E. J. Y. Shen, P. Wallis, et al. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.00668. 2021.
[115] Gonçalves, I., S. Silva. Experiments controlling overfitting in genetic programming. In 15th Portuguese conference on artificial intelligence (EPIA 2011), pages 10-13. 2011.
[116] Ertl, P., A. Schuffenhauer. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. Journal of cheminformatics, 1:1-11, 2009.
[117] Yc, Z.-H.. F. Guo, C.-G. Chai, al. Searching cocrystal structures of cl-20 and hmx via evolutionary algorithm and machine learning potential. Journal of Materials Informatics, 4(2):N-A, 2024.
[118) Brown, N., M. Fiscato, M. H. Segler, al. Guacamol: benchmarking models for de novo molecular design. Journal of chemical information and modeling, 1993) 1008-118, 2019.
16

---

Appendix
A
Impact statement
This paper presents new method for the generative design of organic co-crystals with the goals to advance application machine learning to pharmaceutical co-crystal design and to accelerate and reduce cost of development of solid forms of active therapeutic molecules. Extensive experimental results and multiple case studies described in the paper provide hard evidence of the effectiveness of approach. Therefore, are confident that this work have broader impact drug discovery and development, pharmaceutical industry in general and other related domains.
While we identify the aforementioned societal impacts as strongly positive, there is a risk of malicious and unintended use, as well as inaccurate predictions affecting decision-makinn in the drug manufac- turing process. However, we deem the potential negative impacts limited due to the complexity of regulations in the corresponding fields and the laboratory experiment being the ultimate measure of success.
B Results
B.1
Validation experiment
The validation experiment involved generation of 10000 coformers using ensemble generative models, namely GAN, T- T-VAE, T-CVAE. The generated candidates in combination with one of the three drugs (Nicorandil, Rivaroxaban, Paracetamol) were labeled into classes of three mechanical plasticity parameters. The criterion for getting into the final dataset satisfaction of Unobstructed planes and H-bond bridging parameters. Experimentally validated coformers among the generated molecules were then searched for using the Index Tanimoto (IT), which is metric of molecular structure similarity, Coformers with IT = I were compared with molecules from literature data and added to the Table 3
Nicorandil. Nicorandil, medication that dilates blood vessels, is prescribed for treating angina pectoris, condition characterized by chest pain caused by temporary reduced blood flow to the heart muscle. Unfortunately, during the manufacturing of Nicorandil tablets, the drug can degrade chemically due to the heat produced at high compressive pressure. Generation of set coformers for this drug resulted in fumaric acid molecule among them. Experimental findings have demonstrated that co-crystallizing Nicorandil with fumaric acid not only led to the success of co-crystallization but also improved tabletability properties [97].
Rivaroxaban. Rivaroxaban is an anticoagulant medication that is used to prevent blood clots Rivaroxaban is often taken orally by patients, but the drug is poorly suited for direct compression tableting. The generation of coformers for rivaroxaban using GEMCODE led to the detection of malonic acid among the resulting molecules. According Kale al. the formation of this co-crystal leads to its excellent plastic deformation under applied compaction pressure, resulting in successful tablet formation
98
Paracetamol. Paracetamol is analgesic and antipyretic from the group of anilides, but forms an unstable tablet by direct pessation. Among the coformers generated by GEMCODE was found experimentally confirmed coformers of paracetamol [99 100). They are not only able to form a co-crystal with paracetamol, but also improves its tabletability, demonstrated in Karki al. and Maeno et al..
naphthalene and trimethylglycine (betaine)
B.2 Discovery experiment
For the therapeutic molecule Nicorandil, the labeled generated candidates were selected by meeting conditions of being recognized as safe for human use and presence of carboxyl functional group, which means finding molecules forming the same synthon experimentally confirmed coformers. The discovered coformers are presented in the Table 4 which is divided into blocks depending on each model used for generation. The CCGNet score column indicates the values of the ranking results
17

---

Table 4: Previously unknown novel coformers generated using GEMCODE to improve the tabletabil- ity of the drug Nicorandil. SMILES selected based similarity metric (IT 0.7). Target properties abbreviated as follows: Unobstructed planes (U), Orthogonal planes (O), H-bond bridging (H).
Model
Generated SMILES
Target properties
CCGNet score
CCI=CC=CCCC==00)CCCCI
U/H //H 46.70
CGG-0NNCI-CCC-CC0(((0000000
U/H
44.94
COCI=CC=CC=CII(=000
U/H
44.84
O=C(O)C)=(C=CCCCI
U/H
44.45
O=C(O)))=C(<=N=CC
U/H
42.67
0=C(O)C(=0)0
U/O/H
42.51
COCI=CCCCICC+=)))CCCCII
U/H
40.64
GAN
COCI=CC-CCCCC--)))>--
U/H
39.45
0=Cr0)co0)-=C=CCCCC11
U/H
35.36
U/H
33.34
O=C(O)CO
U/H
28.60
O=C(O)CCιссCCCI
U/H
24.29
O=Cro)CI=CC=NC=C1
U/H
23.40
0=C(O)CC(=O)0
U/O/H
21.99
O=C(0)CC1=CC--CCCC
U/H
21.94
O=C(0)CCCCCCCI
U/H
59.53
O=CIO)CI=CC=NC-CI
U/H
51.93
COCI=CC-CCCCEO)))CC-
U/H
51.32
COCI=CC=CC=CICCEON
U/H
49.99
0=C(0)<c11=CC-CC--
U/H
48.50
U/H
48.27
O=cro)coe1-cc--c-CC))
U/H
47.53
0=C(0)C(=0)0
U/O/H
46.70
CC1=CCC-CCCE0)))CC-CII
U/H
44.86
VAE
O=C1O)C1=CCC-C-oC
U/H
44.45
U/H
42.07
O-crOpC-cc1-CC-CC--C1
U/H
40.01
0=C10)CCCCC1-CC--C100-11
U/H
37.96
0=c1o)ccıcccCC1
U/H
32.38
0<C10)C1=CC=CN=C1
U/H
25.17
cc(O)C(=0)0
U/H
24.27
0=C(O)CO
U/H
22.18
O=C(O)CC(=0))
U/O/H
22.01
COCI=-CC-CIC(=ON0CCCCCC
U/H
46.34
(=C(O)C(=0)0
U/O/H
42.97
0=C(0)C1=CC=CC+CI
U/H
35.53
0=C(O)CC(=0))
U/O/H
33.68
CVAE
CCI-CC-CCCC-O00)CC-CI
U/H
32.85
0-CIO)CCI=CC-CC-CI
U/H
31.68
O=CCOOCNCC--CCC--CC------ -=C(0))cC;=))(==0000
U/H
31.41
U/H
28.01
0=C(0)11cccCC1
U/H
24.09
18

---

for the probability of co- co-crystallization with Nicorandil for the candidate molecules. The ranking was performed within the set of molecules generated by each model separately.
Coformer analysis. Besides carboxyl, the demonstrated chemical compounds contain functional groups such as hydroxyl and amide groups, which are characteristic the confirmed coformers of Nicorandil from the work of Maeno et al. The generated coformers contain various structural modifications, such as changes tion of functional groups, the appearance of multiple bonds and benzene rings. It is important to understand that GEMCODE is focused on the search for the best candidates from the point of view of crystallography and does not address the deep issues of interaction of the created structures with the human body. Therefore, with the help of expert evaluation, we selected molecules that should normally be safe for in pharmaceuticals.
the length of the carbon skeleton, addition and partial substitu-
Meanwhile, many of the discovered coformer structures not only already meet all three mechanical parameters consistent with improved tabletability properties, but also have high probability of successful co-crystalization with Nicorandil. In this way, the process of discovering new coformers using GEMCODE can be described as a smart approach selecting candidate molecules for property- controlled co-crystallization.
C
Data
C.1
Molecule selection criteria
The ChEMBL database contains information on more than 2.4 million drug-like chemical compounds. For training generative models, we needed large number of molccular structures that would have similar properties to the known coformers. Therefore. 1.75 million samples were selected from the molccular structures of the ChEMBL database according to the following criteria:
Structural type: molecule.
Class: small molecules.
Molecular weight of each component <600 Da.
Number of hydrogen bond donors (HBD) less than and hydrogen bond acceptors (HBA) less than 8.
Number of rotatable bonds up to 9.
Polar surface area up to 138 nm.
Number of heavy atoms in molecular structure to 39.
C.2
Co-crystal data
Mechanical properties of the co-crystals determine their viscoelastic nature. The presence of un- obstructed planes and additional slip planes orthogonal to the stacked layers lead to the improved plasticity [1I]. Also, there exists evidence that the lack of hydrogen bonding between the layers has positive effect on the plasticity of a crystal [102] 103). Therefore, an "ideal" co-crystal in terms of plasticity should have non-overlapping slip planes, additional orthogonal planes and no hydrogen bonding between the planes (Figure 3l).
The compaction properties of many pharmaceutical powders depend on their viscoelastic nature. The closer the material to perfectly plastic body, the larger the bonding area after compaction of the powder and the denser (less porous) the compressed tablet is (Figure 3p). Therefore, accurate prediction of the plasticity parameters is essential for data-driven co-crystal design.
C.3 Representation of molecules
Traditionally, molecules are represented as structural diagrams with bonds and atoms, but such repre- sentations are not well suited for efficient computation. Alternatively, molecules can be represented with SMILES and molecular fingerprints, which have been extensively used for various applications, including the generative models [104]. SMILES notation is often used to describe the composition
19

---

and structure of a chemical molecule by means of short strings (Figure 4a), Whereas molecular fin- gerprints a way of representing molecules in the vectorized form FFiuuee 4p). Therefore, molecular fingerprints enable comparing different structures by calculating similarity measures.
slip planes
Unobstucted planes
Elaticc
Compression
Decompression
Plestic
Orthogonal planes
bridging
50.00
Label
Number of cantp pamppes
4000
absence
prnvencc
3000
2000
1000
Unobstructed plantes
Orthoponal
bonds bridging
Random
slates
Hydroger
Mecherical property
Figure 3: (a) Schematic representation of the mechanical properties of co-crystals. No slip plane and H-bond bridging are associated with low tabletability. The other two properties positively correlate with tabletability. (b) Schematic representation of the particle deformation during powder compression. (c) Number of coformer samples of each category per mechanical property.
SMILES
Canonical: Cnntnn22c---
Kecule: CHNCHNCC-CCCC-OOOEEOOOOEEE
Caffeine
Molecular fingerprints
C.H,N.O..
Molecular descriptors
8
1D
2D
30
Figure 4: Molecular representation using the chemical structure of caffeine as example in the form of SMILES, molecular fingerprints, and molecular descriptors.
D Generative models
D.1 GAN
GANs typically consist of two neural networks, generator and discriminator, playing adversarial game against each other while learning the data distribution p (x). The generator network receives random input signal and generates data distribution po (2:, while the discriminator network D evaluates the generated data and tries to distinguish it from the real training examples [105). In the original formulation, both networks are improved by competing with each other following min-max optimisation procedure:
min max E (2) (o)g + Epp( Ds(x))).
0
20

---

Goodfellow et al. proposed alternate generator losses providing better gradients for the generator [06]:
(oute)l-to((Do(x)))
Since 2014, GANs have been successfully used for numerous applications, including modeling of astronomical phenomena [107] experiments in particle and high-energy physics [108], medical imaging [109], and molecule generation 45 46]. The GAN takes SMILES representations of molecular structures as input. the training process, the generator network creates molecular representations from the Gaussian noise and the discriminator network tries to differentiate those from the tokenized SMILES of the real chemical compounds. As a result, the generator learns to output new molecular structures similar those in the training set.
D.2
GAN training and fine-tuning
The GAN trained on the ChEMBL dataset with batch size of 512 and learning rate 0.001 consis- tently produced molecules with validity >75 % after 25,000 training steps. After 30,000 steps, this model was fine-tuned on the coformer dataset with a smaller batch size of 256 for additional 1,000 steps (Figure 5p). The t-SNE analysis reveals that the molecular space of coformers is considerably more constrained compared to that of ChEMBL (Figure 5p). Therefore, fine- tuning was critical to shift chemical compound generation towards the molecular space of coformers. The final model was able to produce >95% valid and >86% of unique chemical structures molccules in the test generation of 1000 molecules times repetition.
Valdity CAN training
LSNE Viuslization
CICPEL detazms
20
66-
CREMOL OIOINS
SEUDbE +200
Jag
300
Figure 5: GAN training results ChEMBL. datasets and coformers: (a) plot of the growth of the valid chemical structures share batch, (b) t-SNE visualization of molecules from the ChEMBL dataset and coformers.
D.3 VAE and CVAE
Variational Autoencoders (VAEs) consist of two deep neural networks, namely, encoder and decoder. The encoder network takes an input feature vector and converts into -ieed-iimeniional vector, while the decoder network converts this siredddimnssionll vector back to the original input feature vector. The primary objective of an autoencoder is to learn identity function, and the fixed- dimensional vector is referred to as the latent vector z. This latent vector serves as an information bottleneck, meaning that is designed to capture only the most statistically salient information the data. In VAEs, the latent vectors 2 are sampled from a normal distribution N(0, I), where I is the identity matrix. To train VAEs, the loss function, which needs to be optimized for input data and latent vector can be formulated as follows:
E|logPP(X\(2) - (
where DKL. the Kullback-Leibhe divergence, which measures the difference between two probabil- ity distributions, Q and N; E is the mathematical expectation; P and Q are probability distributions.
21

---

The probability distributions P2(X|2, c) and (x)(x are learned by deep neural networks called the decoder and encoder, respectively. These networks have learnable parameters 0 and f', The first term of the loss function is the reconstruction error for the input data X. In contrast, the second term measures the similarity between the probability distribution of the latent space and the target probability distribution, (=)), which is ((0, T).
When using VAEs, it difficult control the specific properties of the generated data. Also, since the latent vector is sampled from a unimodal Gaussian distribution, the generated objects tend to be very similar to the training examples. This is not an efficient way to generate new molecules.
Conditional variational autoencoders (CVAEs) were developed to address these challenges. CVAEs can learn multimodal probability distributions by adding condition vector additional input during the generation process. The objective function of a CVAE with condition vector e (passed as an input to the encoder and the decoder) is given by:
o c)] - DKL(Qo (xI (2]X.))
D.4 VAE and CVAE with Attention
It is common knowledge that generating long sequences can be a challenging task for recurrent neural networks. Therefore, we considered transformers, more modern and effective architecture for the task. In our approach, we apply the Pre-Layer Normalization Transformer [110), modification of the original Post-Layer Normalisation Transformer. Similarly to VAE, it composed of two neural networks, an encoder and decoder, but with the attention mechanism. Such architectures are known to suffer from a posterior collapse [ΤЛ. To overcome this, we used Kullback-Leinn divergence annealing (KLA) [122.. Ultimately, the loss function of the T-CVAE is given by:
EllogGe (Xil2, Xdec. c)] (aoDD<i(Qooo(oooo c)(((2)c)),
where DKL is the KL divergence; E is the mathematical expectation; Qo is parameterized encoder function; Q9 is a parameterized decoder function (generator); p(2[c) c) is a conditional Gaussian prior. Here. 0, 0'0', Xenc Xdec X c, kno the parameter set of the encoder, the parameter set of the decoder, the input of the encoder, the input of the decoder, the latent variables, the reconstruction target, the conditions, and the weight for KLA, respectively. This objective function was inspired by the work of Kim (58).
D.5 Proposed architectures of T-VAE and T-CVAE
Our proposed architectures of T-VAE and T-CVAE are shown in Figure 6 Since T-CVAE is an upgrade of T- VAE, the schematics are virtually identical. They differ only by the presence of a block responsible for concatenating additional condition vector with physicochemical properties with the latent space vector in yellow and the molecule token vector at the input to the encoder. Thus, the architecture represents language model of transformer whose encoder encodes information about molecules in 128-dimensional conditional Gaussian latent space. In case of T-CVAE, a vector of conditions consisting of physicochemical properties predicted by gradient boosting model is attached to the latent space. Based on this vector, the transformer decoder learns to generate coformer candidates. The molecule tokens are embedded into 512 dimensions, the same as the model dimension. The encoder and decoder in the proposed transformer architecture consist of layers of 8 heads each.
D.6 Comparison of computational costs
Table 5 compares steady GPU memory consumption while training, training time (10 epochs for GAN and 30 epochs for T-VAE, well as the time required to generate single molecule. is noteworthy that the Beam Search method with beam size b = 4 was used for molecule generation in T-VAE/T-CVAE, which significantly increases the generation time.
To put this into perspective of our study, T-CVAE required about 45 minutes on an NVIDIA RTX A6000 graphics card to generate 10,000 molecules. GAN managed to do the same in 1.88 seconds. A similar evaluation on a more common NVIDIA GeForce RTX 2070 resulted in 3.73 hours and 3.37 seconds, respectively. Arguably, this makes T-CVAE practically infeasible for many users. Taking
22

---

OLtout probabbiiis
Linear
Linerr
Transformer decoder
Transforer decode"
Addkhomm
AddidHoom
Gredient boosting
Feed
Feed
Malecalen
Mechonical propertiie
Add
mecharical properties
Adc
Oueput
Multi-ead Altention
Nx
Multiheed Altersion
Nk
<eos
yWs
Lateric vector
Addkhoom
Sumpling atant
AddddHoom
Generated
Encoder
Mulli head Alt ettendion
Multiheed Attencion
wrcnding
Token
poutionel encading
posttinal encading
Input moiecule encoding
Output molacule anbedding
Initial bollenn <500>
Outpui errbecddnng
Tokers From molessles
Figure The architecture of TT-VAE/T-CVAE for (a) train and (b) generation pipeline.
Table 5: Comparison of GPU memory usage, training and generation times.
GAN
T-VAE
T-CVAE
GPU memory (GB)
6.40
8.00
8.10
Training time (hours) Generation time (ms/molecule)
2.82
22.66
22.68
0.19
270.00
270.00
this into account, we recommend a pragmatic choice to keep GAN as the default generative model in GEMCODE. However, a combination of generative models is required to achieve broader exploration of the target chemical space.
D.7 Additional result of comparing models
Testing whether the generative models produce the same molecules fascinating experiment. In our study, we performed ten generations of 10,000 molecules each, 100k molecules for each model in total. Then, filtered molecules by discarding duplicates, chemically invalid molecules, new molecules, and molecules that do not satisfy the required physicochemical properties and SA. After filtering, we got the following result: of 100k molecules, GAN generated 1639, T-CVAE 2452. and T-VAE - 1407 molecules. Among all of those, only 202 molecules were found to be common. A more detailed evaluation of the intersection of the generated molecules can be seen in the Figure Thus, in this experiment, each model was able to generate unique molecules that the other models did not produce. Specifically, GAN generated 1051, T-CVAE 1733, and T-VAE 698, making the total of 3482 new unique coformer molecules with the properties of interest. Therefore, taken together, the models can generate 1.42 times more molecules than T-CVAE, 2.47 times more than T-VAE, and 2.21 times more than GAN. According to these empirical results, if necessary and under certain research conditions, may be relevant to use an ensemble of generative models to increase the total number of generations of diverse molecules. Of course, this approach is inherently resource-intensive.
23

---

Table I shows the values of target molecules in percent. These percentages can be interpreted as the probability with which generative model can create new molecule with the properties of interest in single generation iteration.
In order avoid misunderstandings, also present below numerical comparison with the target molecules in the training dataset.
To interpret the efficiency of generative models, we present quantitative comparison of target molecules in the training dataset. dataset with examples of existing pairs of co-crystals, consisting of 4200 molecules, used for additional training for the task of generating co-crystals. This dataset contains 355 molecules, which correspond to our selection conditions for the formation of co-crystal with Theophylline. As be seen from the description above and Figure even the least efficient model was able to synthesize 1639 new potential molecules during the experiment. Thus, it can be seen that the use of the generative models developed by search for coformer is very promising.
T-CVAE
GAN
299
622
209
T
Figure 7: How unique molecules created in different models intersect.
Also, it is worth noting that if we repeat the experiment and generate another 100,000 molecules, will able to obtain significant number of more potential molecules. Since we have not conducted this additional study, and we cannot determine exactly what the limit of each model in generating unique molecules is, we cannot calculate any relative metrics to compare model performance with respect to the training data. For this reason, propose numerical values.
D.8 Similarity analysis of the generated molecules
In order to further analyze the novelty of the generated molecules, we performed additional experi- ments to calculate the Tanimoto similarities for the generated molecules. For this purpose, we plotted histograms illustrating the distribution of maximum IT between the generated coformers and the coformers from the training dataset (Figure 8h). Notably, the distribution is mostly centered on IT values between 0.5 and 0)61 for all generative models. This observation strongly supports the claim that the generated molecules exhibit substantial novelty. The Tanimoto similarity distributions between all molecules generated by GAN, VAE, and CVAE were also analyzed (Figure 8p). For each model, the average Tanimoto similarity ranges from 0.70 to 0.75. On one hand, this indicates sufficient diversity of molecules. On the other hand, the relatively high average similarity was expected, since all generated coformers refer to the formation of a co-crystal with the same drug. This fact agrees well with the observation that the CVAE distribution is skewed towards due to the targeting properties the coformers.
"condition" architecture block that enhances the
D.9 Details on training language models
GPT-2 was pretrained on the ChEMBL dataset with a batch size of 128 for five epochs and then fine-tuned the coformers dataset with batch size of 256 for 15 epochs. We used the top-p (nucleus) sampling method with p 0.95, as recommended in the study by Holtzman 1K3 for balanced quality, diversity, and coherence in the model outputs.

---

Tanimoto Similarity between generated coformers
3.0
1/:
CAN
VAE
CVAE
Taminato Similntty
Tanarati Similertyy
Tanamati Simlentty
16
Similarity between generated molecules
2.25
2.0
1.75
1.75
1.25
1.25
3 1.00
1.00
0.50
0.50
8.0
CAN
VAE
CVAE
0.4
0.00
Innimoto Simiarity
Ta lanimoto Simiarity
Tomimoto Similain
Figure 8: Tanimoto Similarity Histograms: (a) for generated molecules and real coformers, (b) for all generated molecules.
We trained Llama-3-8D with LoRA [14 and 4-bit quantization. It was pretrained the ChEMBL dataset for 250 epochs and then fine-tuned on the coformers dataset for 100 epochs. Standard Llama-3 tokenizer was used, which likely was the main reason for moderate performance. The total LoRA training time was about 20 minutes.
D.10 Evaluating and comparing language models for coformer generation
We evaluated generations of GPT-2 and Llama-3-8B models and compared them the best scores produced by the generative models in GEMCODE (Table 6).
Table 6: Comparison of coformer generation: GEMCODE vs. language models.
Model
GEMCODE
GPT-2
Llama-3-8B
Validity, Novelty, % Duplicates, %
99.70 + 0.00
92.30 + 0.20
98.3 0.00
95.12 + 0.11
71.4 + 0.48
85.4 + 0.33
24.30 + 0.45
48.70 + 0.12
27.4 + 0.37
Target coformers, %
5,63 + 0.22
2.14 + 0.19
0.34 + 0.05
Diversity of target
0.32 + 0.02
0.24 + 0.01
0.21 + 0.01
While GPT-2 produced a higher percent of target coformers compared to T-VAE (2. (2.14% versus 1.68%), the ensemble of generative models in GEMCODE resulted in at least two times more molecules with the desired physicochemical properties (5.63% with T-CVAE only). Llama-3-8B was able to generate only 0.34%, which is not attractive given vast amount of resources required to train and use the model for generation.
Meanwhile, Llama-3-8B clearly outperformed GPT-2 terms of validity, novelty and the number of duplicates in molecule generations. It also showed almost on-par performance with GEMCODE, e.g.. the percent of valid molecules (98.3%) was comparable to T-VAE (99.7%) and T-CVAE (98.4%). One important advantage of Llama-3-88 over GEMCODE is its ability generate more diversity in the target molecular space. However, this advantage diminished by the overall low percent of target coformers generated.
25

---

E Evolutionary optimization
E.1
Framework
To find molecules with higher probability of exhibiting desired characteristics, used evolu- tionary algorithm based on self-developed GOLEM framework [85] The evolutionary algorithm operates graph representation of molecules. The problem being solved the minimization of the multi-objective F in the discrete space of structural graphs My under a set of constraints C. The task find an optimal structure M (V, E).
M = argmin F, where M3 = {Mg) ((M))) bay
F F= Pm (2), po(x),
where is an evaluated molecule of coformer, (x)) is the probability of the positive class for unobstructed planes, Po( is the same probability for orthogonal planes, and pa (x) - for H-bond bridging. Therefore, minimization of the fitness function F leads to generation of coformer molecules having an improved tabletability profile.
E.2
Scheme of the algorithm
The general scheme the evolutionary algorithm is presented the Figure 9 from initial population are selected for mutation. At the mutation stage, the individuals are modified using a set mutations described earlier. To control the process, Mutation operator refers to Change Advisor that determines possible actions. Since the algorithm implements generational evolution scheme, Inheritance produces a new population using all the individuals obtained through mutation. Ellitism operator replaces the four worst individuals in the population by the best ones found so far. The cycle is repeated until any stopping criteria are satisfied (time limit or maximal number of iterations).
At first, individuals
Change
Mutation
Initiel population molecules
Selection
Opsinisation until stop criterte
Best individuals
Figure 9: Scheme of the evolutionary algorithm that is used for fine-tuning of solutions.
E.3
Evolutionary optimization experiment details
For each generative model, 10 independent runs of optimization were performed. For each run, we used a random sample from all unique coformers generated by the model the initial population. The sample size was equal to the mean number of target coformers found in the 10,000 generated molecules. Analyzing the dataset of already known coformers B3 we estimated the maximal number of heavy atoms to be 50 and the available elements to be C, N, O F. P, S, CI, Br, and I. Those estimates were used configure the evolution process. Population sizes were set 200, number of iterations to 200 and timeout to 60 minutes.
Exolutionary algorithms tend produce redundantly complicated structures due overfitting [15]. To avoid unrealistic molecules, synthetic accessibility score (SA) [116] was calculated for all molecules obtained with evolutionary optimization. Only coformers with SA < 3 were selected for further consideration.
26

---

E,4 Evolutionary optimization significantly improves H-bond bridging
The results of the evolutionary optimization application were the most prominent for H-bond bridging and are presented in the Figure 10
09
Pobbaklyy
0.7
0.8
GAN
GAN + EVO
TAWE
TVWE + EVO
T-CWAE
T-OVAE EVO
Figure 10: Comparison of probability distributions for the presence of hydrogen bonds between the planes (H-bond bridging) for coformers generated by the neural models and optimized by evolution.
In Figure 11 mean convergence of the algorithm is shown. Interestingly, the of molecules generated by T-CVAE as an initial population also consistently increased the algorithm convergence speed.
Unobstructed planes
Orthogonal planes
C
H H-bonds bridging
-0.00
EV0
-0.45
0.24
T:KEE
-0.00
0.22
-0.2
0.00
0.10
-094
+0.70
0.14
-0.8
10.75
8.12
75
190
75
125
150
125
150
Generaton
Genoration
Generation
Figure 11: Convergence for mechanical properties of evolution starting from co-crystals generated by different models.
E.5
Evolutionary schemes comparison
Table 7
presents results of comparison of evolutionary schemes: SPAE-2 and MOEA/D.
Table 7: Results and statistical significance nonn-parmmetric one-sided Mann Mann-Whitney test) for 10 runs of evolutionary algorithms based on SPEA-2 and MOEA/D selections.
Median probability SPEA-2
Model
Property
p-value
MOEA/D
Unobstructed planes
0.819
0.819
2,639
GAN
Orthogonal planes
0.385
0.389
0.141
H-bond bridging
0.692
0.694
0.039
Unobstructed planes
0.825
0.825
7,958
T-VAE
Orthogonal planes
0.393
0.398
0.753
H-bond bridging
0.693
0.692
8.703
Unobstructed planes
0,826
0.823
1.049
T-CVAE
Orthogonal planes
0.387
0.390
0.137
H-bond bridging
0.682
0.693
0.013
27

---

E.6 Comparison with GraphGA baseline
The genetic algorithms are considered be strong basis for drug design tasks [53 so we
compare the GEMCODE with genetic baselines. Known baselines from Guacamol [18 [18 can be used to optimise any molecule in SMILES notation with a given goal. However, unlike the Guacamol tasks, the co-crystal design task is multi-objective, whereas the algorithms from Saacamol_baaselne (e.g. the well-knows GraphGA) are focused on single-ohjective tasks.
We have developed the multi-objective modification of GraphGA with Pareto dominance- fitness, which can used for the co-crystal design tasks. It started from random subset of co-crystals (with the same population size and number of iterations as used in GEMOL).
a]
Unobstructed
-0.08
GAN EVO
EVO
GraphGA
00
00.1
-0.11
-0.02
0039
-0.0
0034
-0.30
-2
-0.90
0
25
92
75
100
125
150
175
Generation
1.0
b)
Orthogonal planes
-3.45
GAN EVV T-OVAE + EVO
-0.45
GraphGA
-0.50
-0.50
-3.50
0.50
+3.05
-0.70
4..90
100
125
Consration
c)
H-bonds bridging
GraphGA
T-OVAE EVO
0.18
0.18
0.14
0.12
0.10
50
75
75
Figure 12: Comparison of GEMOL with GraphGA baseline. GEMCODE the left, GraphGA the right
GraphGA showed inferior results according to the mechanical property values obtained from GEMOL (see Figire 12). We 0.95 for GEMCODE against 0.938 for GraphGA, for orthogonal planes 0.72 against 0.63 and for h-bonds bridging 0.18 against 0.12. Also, the convergence quite unstable (we think is caused by insufficiently successful selection procedure), so the hybrid approach implemented in GEMCODE is better in all cases.
that for unobstructed planes the best probability average all runs is
28

---

Furthermore, the proposed hybrid approach is able to generate on average 21.3% of target molecules from the available population during optimisation. Our comparative tests for GraphGA showed 20.5% % of targeting molecules from the population (with worse quality according to predicted mechanical properties). Finally, it should noted that our existing dataset 4223 coformers can be extended by further 2452 molecules using a generative model (while GraphGA itself only provides 120 new candidate molecules).
F
Indicators
Additional results on evolutionary optimization
Table 8 provides additional statistics on the impact of evolutionary optimization.
Table 8: Results and statistical significance of the evolutionary optimization. Mean and standard deviations are given for the probabilities of Unobstructed planes, Orthogonal planes and H-bond bridging.
Probability
Model
Property
p-value
Novelty
Generated
Optimized
Unobstructed planes
0.82 = 0.05
0.82 0.06
GAN
Orthogonal planes
0.39 + 0.04
0.40 + 0.05
3.83e-12 1.50e-67
0.68
H-bond bridging
0.62 0.07
0.67 0.08
Unobstructed planes
0.82 0.05
0.82 0.06
T-VE
Orthogonal planes
0.39 = 0.05
0.41 0.05
3,87e-10
0.72
H-bond bridging
0.63 = 0.07
0.68 0.07
2.51e-66
Unobstructed planes
0.81 + 0.06
0.82 + 0.06
1.36c-05
T-CVAE
Orthogonal planes
0.39 0.05
0.40 0.05
2.69e-10
0.60
H-bond bridging
0.64 = 0.07
0.67 + 0.08
2.60c-47
We used multiple indicators to assess the performance of the generative models. Let us denote the number of generated molecules by G. All the generated molecules contain valid molecules V, duplicates D, new molecules that are not contained within the training dataset N, as well as molecules possessing the desired physicochemical properties C and molecules satisfying the condition of synthetic accessibility molecules S (SA <= 3). With these notations, we define quality indicators
in the subsequent subsections:
F.7
F.2 Validity
Validity refers to the ratio of predicted molecules deemed chemically plausible and estimated by Cww..heleeooooooo
rdkit. them. Mo MolFrommmiles les taking into account the valence of atoms in the molecule and the consistency of bonds in aromatic rings. When evaluating the validity of molecules using the rdkit package, we obtain set of molecules equal to V. Then Validity can be calculated the ratio of valid molecules to all generated molecules:
V
Validity = G
100 [%].
F.3 Duplicates
Duplicates is ratio that shows how many duplicates are contained within all valid molecules:
D
Duplicates V
100 [%].
29

---

F.4 Novelty
Novelty is a ratio that shows how many valid molecules without duplicates among the generated ones novel (i.e., these molecules were not contained within the training dataset and were produced by the generative model). Therefore, Novelty is defined as follows:
N
Novelty =
100 [%].
V
F.5
Target coformers
After obtain of valid molecules without duplicates that are not contained within the training dataset, we predict the physicochemical properties using the pretrained gradient boosting model. We then select molecules the required mechanical properties: Unobstructed planes Orthogonal planes 1, H-bonds bridging O. For those, we also evaluate the synthetic accessibility (S A) score using rdkit. Contrib. wts.ii....s.sS____ score. As mentioned before, we define the target tabletability profile in the generation process by the required mechanical properties and SA s 3. Thus, the percentage of target molecules calculated follows:
8 Target coformers
100 [%].
G
F.6 Synthetic Accessibility Score (SA)
SAA is calculated using rdkit.Contrib. StlbbbSAASSSS Score.
F.7
Diversity
We used Diversity to assess the diversity of the generated molecules. In order to estimate this indicator, we need to calculate Tanimoto- Tanimoto-similarity (T₂) and Tanimoto-distane (Tz). Thus, to estimate Te. consider molecules, and b, with Morgan fingerprints ma and ma respectively. The number of common fingerprints between the two molecules is represented by [ma mm). and the total number of fingerprints is represented by mm, U ma/. Then, Tanimoto-similarity is defined by:
m. 0 mm
T,
m. U Ump
Then, Tanimoto-distame and Diversity are related as follows:
Diversity Ta(a, 6) T..
G
ML model
G.1 Dataset splitting
When comes to dataset splitting, the choice of technique has a significant impact on model performance. For the field of co-crystals, there is still no established splitting strategy. The potential for more stratified splitting approaches in drug design, such as molecular scaffolds, exists, but adapting this method co-crystal dataset presents a number of challenges. Each sample in the co-crystal dataset consists of two coformer molecules with different scaffolds and a large structural diversity, unlike drug design applications that deal with specific classes compounds and typically allow the identification of fewer scaffolds. For out-of- of-distributiin generalisation analysis, along with random splitting of the dataset, we separated the training and test samples based on Tanimoto similarity, maximising dissimilarity between molecules of different subsets. The results of the experiment are summarised in Table Since random splitting has already been used in the co-crystal domain within other works, it is practical and showed the best performance the random splitting was preferred.
30

---

Table 9: Metrics of the Gradient Boosting model for predicting the mechanical properties of co- erystals upon changing the data splitting strategy.
Random
Tanimoto Similarity Accuracy
Property
Accuracy
Fl Score
F1 Score
Unobstructed planes
0.73
0.77
0.73
0.71
Orthogonal planes
0.79
0.59
0.83
0.52
H-bonds bridging
0.73
0.76
0.70
0.70
G.2 Threshold
Due to the significant imbalance of data related to the orthogonal planes, decided change the threshold of probability of assigning a sample to class. For this purpose we investigated the change of precision and recall depending on the value of the introduced threshold Figure 13)
Threshold 0.302
Precsioon
0.6
0.1
0.7
03
044
05
Threshold
Figure 13: Change of metrics depending the set threshhold.
The threshold set under the condition of equality (intersection of lines) of precision and recall metrics, as it represents the optimal point for balancing the number of false positives and false negatives.
G.3 SHAP analysis
In order increase the transparency and reliability of the ML model's decisions and results, we used the SHAP method, which allows us to interpret the output of the predictive model. But also, it enables domain-specific hypothesis generation while contributing the explainability of the predictive model, which is huge benefit for potential applications. The Figure 14]shows the features that were used in the training process, ranked in order of importance for the final prediction. In this case, the SHAP values to the left of the center vertical line are negative-class (0) and to the right are positive-class (1). Also, red dots indicate higher feature value and blue dots indicate lower feature value.
G.4 Prediction of co-crystals properties
One of the interesting applications of machine learning models the field of co-crystal design is the prediction of different physicochemical properties. [Table 10 presenns a comparison of known models that predict properties such as crystal density, entropy and enthalpy of melting, melting temperature, ideal solubility, and lattice energy. It is important to note that the mechanistic properties of co-crystals have not been predicted before, so the metrics obtained in this paper can be considered state-of-thee-rt.
31

---

Unobstructed planes
66
Orthogonal planes
H-bonds bridging
Nunsomm moon
Wanattnn sum
Iptroimm mweet
FoctionCSP3 moon
toppn2
Orippnndege)
1tafe
CrtspenCiag
Crippntiegg
Crippenlogg
CSP
mea
CrippenC
poo
ha Iatenoopa
happa3
kapo13
SIMP value Umfoact modal cutput]
SHAPP value (irpact
Figure 14: SHAP plots demonstrating the importance of the first 10 coformer features for prediction of (a) Unobstructed planes, (b) Orthogonal planes and (c) H-bond bridges.
Table 10: Comparative table with model metrics on prediction of various co-crystals properties.
Generative design
Task
Property
Data points
Best metric
Ref
Regression
Crystal density
26
R² 0.993
No
64
Regression
Melting temperature
R 0.992
No
Regression
Melting enthalpy
R² = 0.999
No
Regression
Melting entropy
30
R² = 0.997
No
35
Regression
Ideal solubility
R2 0.953
No
Regression
Melting temperature
61
RSD = 2.89%
No
62
Regression
Lattice energy
RSD = 2.40%
No
61
63
Regression
Crystal density
RSD 1.77%
No
Regression
Melting temperature
84
R2 0.998
No
65
Regression
Crystal density
4144
R² 0.985
No
36
Classification
Unobstructed planes
Accuracy = 0.731
Classification
Orthogonal planes
6029
Accuracy = 0.785
Yes
Our work
Classification
H-bonds bridging
Accuracy = 0.734
G.5 AutoML
To prove the effectiveness of the proposed ML models, we conducted additional experiments with the state-of-the-ee AutoGluon framework. We used the timeout one hour and the "best quality" preset. After extensive evaluation, we were able to achieve no significant improvement of the F1-score against the proposed model (Table IT]. These results indicates that the ML models in GEMCODE are less prone to overfitting and have better generalization capabilities.
Table 11: Comparison of the proposed ML. models with AutoML. Best achieved metrics are given.
Property
Model
Precision
Recall
F1 Score
AutoGluon Our model
0.74
0.73
0.72
Unobstructed planes
0.73
0.73
0.72
AutoGluon Our model
0.78
0.79
0.78
Orthogonal planes
0.79
0.79
0.79
AutoGluon Our model
0.81
0.71
0.68
H-bond bridging
0.77
0.72
0.71
version 1.1.0
32

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope they provide clear overview of the key findings and objectives discussed in the paper.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
The limitations of the proposed approach are detailed in the main text in the Section
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
33

---

Answer: [NA]
Justification: The paper does not include theoretical results, so there are no assumptions or proof' provided evaluate.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Experimental results for all components of GEMCODE, namely ML models, ensemble of generative models, evolutionary optimization and GNN for ranking by co- crystallization probability can reproduced using open source code, model weights and data from the GitHub repository (https: / /ggt com/ ai chem/ GEMCODE). Also code for reproducing additional experiments on using language models generate coformers available in the repository.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility.
34

---

In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: All code and data is publicly available on the GitHub repository (https 7/ //github.comm com/ ai chem/ wal-cchem/mGMDDDD GEMCODE).
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https for more details.
7 //nips //nips.cc/publlt wwtppppppcp.g.udee.ppsppppppppee httee/gguuddesooeSSubbbbiise hwwp///SSwiiiiioonooooooooo
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The experimental setting is presented in sufficient detail within the paper to understand the results. More specific information is also provided with the code.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The paper provides sufficient information on the statistical significance of the performance estimates of the GEMCODE components (see Subsection 5.3] Subsection E.5 and Subsection F.T.).
Guidelines:
35

---

The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out of range (c.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Appendix provides details of the computer resources used for the experi- ments, indicating the type of computing machines and runtime.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: The authors thoroughly reviewed the NeurIPS Code of Ethics and ensured that all aspects of their research align with its guidelines to maintain ethical standards in their work.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
36

---

Answer: [Yes]
Justification:
The potentially broader impact of the paper is discussed in the Appendix A
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairesss considerations (e.g., dep.lymmlo of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications,
authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
The paper assumes no such risk.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All original sources of data (ChEMBL database, co-crystal dataset, see Section 3.1) and code (CCGNet co-crystal ranking model, see Section 4.4) are properly credited in the paper.
Guidelines:
The answer NA means that the paper does not existing assets.
37

---

The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the datasets.paperrsst paperswi thcode. cow datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors are encouraged to reach out to the asset's ss creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [Yes]
New assets obtained in the article are provided in the GitHub repository hbttps://github.com com/aatooaaaa /inm Schem/GEMCODS with proper description and instructions for use.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing experiments or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
38

---

Answer:
[NA]
Justification:
The paper does not involve research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
39


Reviews:
[{"review_id": "JmJ1X0psPT", "reviewer": "Reviewer_XDyo", "summary": "The authors investigate an interesting chemical problem of generating coformers given an organic molecule such that they would form co-crystals with desirable chemical properties. The authors use GAN/VAE-based methods to generate SMILES of potential coformers, which are then improved by evolutionary optimization, before finally predicting the probability of co-crystallization via a GNN. Traditional statistical models appear to be able to predict certain mechanical properties of the co-crystals given the pairs of SMILES. The GAN/VAE models are able to generate valid molecules, where ~40-80% are predicted to crystallize.", "questions": "1. I note that in Sec 5.1, it says the data were randomly split. As we know, co-crystal molecules typically have limited diversity, can the authors elaborate how the predictions generalize out of distribution (e.g. by scaffold splitting)? \n2. Can the author produce novelty/diversity of generated samples as distribution of Tanimoto similarities? \n3. Could you provide CCDC access code (as is tradition) for Table 3?", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "*Originality & Significance* The manuscript tackles a very interesting and impactful problem in pharmaceutical production, and overall a long-standing challenge in crystal engineering. To my knowledge, this is one of the first efforts to use generative ML to design co-formers. If completely new crystallization agents/co-formers can indeed be reliably produced, it would be a very useful tool. \n\n*Quality*: In general, the approach taken here makes sense.\n\n*Clarity*: The paper is very clearly written, and the presentation is easy to follow.", "weaknesses": "1. As with any ML for science work, the devil is in the details of the application. There are several big problems I see:\n- Given that co-crystals are known to also have a variety of polymorphs, predicting mechanical properties based on SMILES without predicting/knowing the exact crystal structure inherently is not a sound approach (which polymorph's property is predicted?). \n- While I agree the overall crystal contacts/packing dictate mechanical properties (ignoring defects), I do not see sufficient justification as to why predicting unobstructed planes/orthogonal planes/H-bonds bridging is a strong proxy for plasticity of co-crystals, let alone tabletability (there are just so many different types of possible interactions and/or indicators). The citations there are at best weak and do not support the claims. \n- The definition of validity/novelty is very relaxed. Validity that is 'this molecule has the right valence' does not rule out thermodynamically infeasible molecules (at least, QED/SAScore should both be reported; otherwise, by definition, you can use SELFIES to get 100% valid molecules with every method). Novelty should not be 'how many molecules are not exactly the same as the training set' but rather a histogram of the closest Tanimoto similarity of the generated molecules to the training set (because a molecule can differ by 1 atom and this metric would still count it as novel). Duplicates are better represented by diversity (histogram of Tanimoto similarity between the generated molecules). Sec 5.4 and Table 4 show very similar/simple aliphatic carboxylic acids. This worries me that the molecules produced lack diversity and are not novel. \n- It is very hard to validate the results without wet lab results. The predicted probability of co-crystallization and predicted mechanical properties are unfortunately not verifiable outside of models trained here unless structures are known. While I fully agree that experimental validation should not appear in NeurIPS, I fear that 'generating co-crystals with good tableability' cannot be claimed unless there are wet lab results. \n2. The ML approaches taken here are not novel, and to an extent, questionable. If the authors developed a crystallization/property predictor (which produces a combination of scores), I feel the typical optimization approaches (e.g. reinvent, or at least the suite of software in Guacamol) should be at least used as baselines, and otherwise it is hard to justify the usefulness of VAE/GAN; the comparisons against GPT-2 models are much less relevant.\n\nNote: citations on the chemistry side can be significantly improved (e.g. all of the citations 1-6 are hardly relevant, e.g. there are plenty of impactful reviews for charge transfer co-crystals and their applications). I also disagree with the current screening for co-crystals 'focus on rather narrow classes of candidate compounds' (especially when the demonstrated results in the paper are all aliphatic carboxylic acids).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for providing a very valuable feedback!  \nWe believe that some of the criticism was caused by a misunderstanding. \n\nWe would like to first __comment on the weaknesses__ outlined by the reviewer:\n1. Polymorphism is certainly an important factor in the design of co-crystals as it can influence their physicochemical properties [1]. However, the percentage of polymorphs in the CCDC data does not exceed 5% of the total number of co-crystals. Thus, we can assume that their influence on the accuracy of the model in predicting mechanical properties is insignificant.\n2. In Appendix C.2, we attempted to summarize the relationship of the predicted mechanical properties to the plasticity and tabletability of the co-crystal. Supporting evidence is given by the paper by Bryant et al [2]. This work was published in CrystEngComm, a top-5 domain journal (https://scholar.google.com/citations?view_op=top_venues&hl=en&vq=chm_crystallographystructuralchemistry). So, we respectfully disagree that this is not a sufficient proof of the relationship. Both the publication and the venue are well respected in the crystallographic community.\n3. We agree with the reviewer's assertion regarding the limitations of relying solely on validity. As suggested by the reviewer, we refined our molecule selection by incorporating the SA score. In Appendix F.5, we mention a threshold of SA ≤ 3, resulting in an average coformer synthesizability value of 2.06.\n4. The reviewer's concern regarding the high similarity of the molecules in Table 4 (Appendix B.2) is a misunderstanding. In our experiment to discover new coformers for Nicorandil, our objective was to identify coformers with target properties and a high degree of similarity to the existing co-crystals of this drug. Given that the known coformers for Nicorandil are Fumaric and Suberic acid [3], it is logical that the table includes aliphatic carboxylic acids. This result was achieved as intended in the experiment.\n5. During validation (Section 5.4), we identified three co-crystalline systems that were not included in the training dataset. By showcasing GEMCODE's ability to produce new co-crystal structures reported in the literature as experimentally validated, we prove its predictive capabilities. We believe that our approach to validating GEMCODE is reasonable and convincing. We share the view that wet lab experiments must not be required for NeurIPS application papers. Nevertheless, we currently expand our team to work on experimental validation of our predictions in the lab.\n6. We are thankful to the reviewer for acknowledging that our work is _\"one of the first efforts to utilize generative ML for designing co-formers\"_ and addresses _\"a very interesting and impactful problem in pharmaceutical production\"._ We totally share this assessment. Given that the problem is novel, the applicability of previous approaches is very limited. Existing Guacamol test suites do not support co-crystal design tasks, that is why we did not include the Guacamol benchmarks. The drug design methods implemented in Guacamol (SMILES LSTM, Graph GA) [6] served as an inspiration for our own fine-tuned baselines (we use the GAN-LSTM). REINVENT4 (the latest version) is also not suitable for end-to-end co-crystal design tasks out-of-the-box. The models used in the RL optimizer would have to be retrained and the property prediction part would have to be significantly modified to make a comparison. Additionally, there are works claiming that RL methods are not efficient enough for crystal design [7]. A comprehensive comparison of such methods to ours would certainly be interesting but beyond the scope of this study. \n\nWe would like to also provide __answers to the questions__:\n1. During the validation experiments, GEMCODE successfully generated three new cocrystal systems that were not present in the training data. These cocrystals were reported in the literature as experimentally confirmed. This perfectly demonstrates the generalization capability of our pipeline. Furthermore, in the field of cocrystals, the conventional approach does not typically involve data splits based on class distribution. A recent study accepted for the ICML 2024 workshop [8] employed a standard random split.\n2. The reviewer's comment regarding the novelty assessment was particularly helpful, so we conducted further analysis and created histograms illustrating the distribution of the maximum Tanimoto Similarity (IT) between the generated coformers and the coformers from the training dataset (https://anonymous.4open.science/r/GEMCODE/rebuttal/histogramms/GAN.png, https://anonymous.4open.science/r/GEMCODE/rebuttal/histogramms/VAE.png, https://anonymous.4open.science/r/GEMCODE/rebuttal/histogramms/CVAE.png). We observed that the distribution is predominantly centered on IT values ranging from 0.5 to 0.6 across all generative models. This observation strongly supports the assertion that the generated molecules exhibit substantial novelty. \n3. We thank the reviewer for highlighting the importance of adding refcodes to Table 3, as this will enhance the accessibility of the information for the reader. We will update the table in the camera-ready submission.\n\nIn conclusion, we would like to highlight that we are positioning GEMCODE as a global open-source project. We value constructive criticism and will use the reviewers’ feedback to improve our methods for data analysis, generative models, and validation of results.\n\nAs the reviewer recognized the originality and significance of our work, we kindly ask to consider increasing the score to 6. A NeurIPS publication could attract much attention and interest of the professional community, which is essential for advancing and promoting GEMCODE. With current scores at 8/6/4/4, a moderate increase in the score would significantly increase our chances to get accepted.\n\n__References__ will be posted as a separate official comment due to the limit of rebuttal size.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "i1sq3ehmOp", "reviewer": "Reviewer_rNY2", "summary": "This paper presents GEMCODE, a novel pipeline for generating co-crystal designs with enhanced tabletability properties for pharmaceutical applications. The authors combine deep generative models, evolutionary optimization, and machine learning to create and evaluate potential co-former molecules. They train models to predict mechanical properties of co-crystals and generate co-former candidates using various approaches, including GAN, transformer-based VAE and CVAE architectures. The generated candidates are then optimized using evolutionary algorithms and ranked based on co-crystallization probability. The authors report that their T-CVAE model produced the highest percentage of co-formers with desired tabletability profiles. They validate their approach by generating experimentally confirmed co-formers for drugs like Nicorandil, Rivaroxaban, and Paracetamol. While the results appear promising, the authors acknowledge limitations such as potential bias in property predictions and the need for more comprehensive experimental validation. They also explore the use of language models for co-former generation, finding potential but noting the need for further optimization to achieve competitive performance.", "questions": "Can the authors confirm if the validation experimentation generated co-formers are not present in the training set (the pre-training CHEMBL set or the fine-tuning co-former set)? Are the therapeutics chosen in the validation experiments also not present in any of the datasets (the co-former set in particular)? \n\nThe validation experiment needs to be expanded, as the results here would determine the utility of the entire GEMCODE pipeline. While the individual parts of the pipieline work, the validation experiments show that the co-crystal designs are real. Are there any fitness functions or ab initio results that can demonstrate the effectiveness of GEMCODE in generating co-crystals with novel therapeutics?\n\nThe state-of-the-art molecular generation algorithms [1], particularly for drug design tasks, are evolutionary. Rather than using language models, it seems more likely that a genetic algorithm would generate better structures than the GAN/VAEs/LLMs used here. \n\n[1] Tripp, Austin, and José Miguel Hernández-Lobato. \"Genetic algorithms are strong baselines for molecule generation.\" arXiv preprint arXiv:2310.09267 (2023).", "limitations": "The authors acknowledge a bias in predicting the absence of orthogonal planes, which could limit the pipeline's effectiveness. More work is needed to address this imbalance in the training data or model architecture. Furthermore, while tabletability is important, the narrow focus on this property overlooks other crucial aspects of pharmaceutical co-crystals such as solubility and bioavailability. This limits the utility of the pipeline for drug development. The authors acknowledge these issues in the limitations and conclusions.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "I appreciate the comprehensiveness of the experimental evaluation in this paper. The authors have conducted a thorough examination of GEMCODE's performance across multiple dimensions. They systematically compared three different generative models (GAN, T-VAE, T-CVAE) using various metrics such as validity, novelty, and percentage of target co-formers generated. While the described methods are not new (models for property prediction, co-former generation, and co-crystal prediction), this is a novel task that the authors have decided to tackle with GEMCODE. \n\nThe authors evaluated the effectiveness of machine learning models for predicting mechanical properties of co-crystals, comparing performance before and after feature engineering. They assessed the impact of evolutionary optimization on the generated co-formers, providing statistical analysis of improvements in desired properties. The pipeline was validated using real-world case studies with known drugs, demonstrating its ability to generate experimentally confirmed co-formers.", "weaknesses": "I recommend adding these references in the introduction and related works, as they are relevant to the manuscript [1, 2, 3]. In table 1 and 2, arrows indicating the direction of optimization for the metrics would be helpful. \n\nWhile the authors provide some validation using known co-formers, there is a lack of comprehensive experimental testing of the novel co-formers generated by GEMCODE. Synthesis and physical testing of predicted co-crystals would significantly strengthen the claims about the pipeline's effectiveness; or at least some theoretical ab initio results on the validation  More details on the validation experiments are needed, in particular how the therapeutic molecules were selected. \n\n[1] Dollar, Orion, et al. \"Attention-based generative models for de novo molecular design.\" Chemical Science 12.24 (2021): 8362-8372.\n[2] Jensen, Jan H. \"A graph-based genetic algorithm and generative model/Monte Carlo tree search for the exploration of chemical space.\" Chemical science 10.12 (2019): 3567-3572\n[3] Tripp, Austin, and José Miguel Hernández-Lobato. \"Genetic algorithms are strong baselines for molecule generation.\" arXiv preprint arXiv:2310.09267 (2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the thoughtful evaluation of our work and for the valuable feedback! We will certainly revise and address all the comments and suggestions in the camera-ready version of the paper and our future work.\n\nBelow, we provide __answers to the key questions and comments__:\n\n1. We appreciate the suggestion to include references to additional articles on molecular generative design. We acknowledge the importance of providing a comprehensive overview of existing works in the field and will incorporate the suggested references in the Introduction and Related works sections. \n2. We thank the reviewer for pointing out the need for arrows in Table 1 and Table 2 to simplify reading the metrics. We will make sure to incorporate this suggestion in the camera-ready submission.\n3. We are currently focused on further validation of GEMCODE through experimental studies. To date, we have extensively validated each component of the pipeline as well as the predicted coformers of the case studies. The predicted coformers for Nicorandil, Rivaroxaban and Paracetamol (Section 5.4) were not present in the training data but these co-crystals are already experimentally confirmed, as reported in the literature. These cases provide strong evidence for the effectiveness of our pipeline.\n4. For testing the pipeline, we selected those drugs reported to fail in forming tablets through direct pressing. Finding coformers for such drugs with GEMCODE was, in our opinion, the strongest evidence for the utility of the pipeline. We emphasize that our training data did not include any co-crystals related to Nicorandil and Rivaroxaban, but only systems of Paracetamol due to its widespread use. Therefore, Nicorandil and Rivaroxaban were entirely new to GEMCODE. This demonstrates the pipeline's ability to predict new coformers for both, new drugs and those contained in the training data.\n5. We agree that many SOTA drug design approaches use evolutionary algorithms as optimizers (e.g. [1]). For this reason, we integrate them as part of our pipeline. However, the results of evolutionary optimization for co-crystals are highly dependent on the quality of the initial solution pool [2]. Furthermore, it may be too computationally expensive to wait for convergence starting from a random population. At the same time, restricting initial population to the existing database negatively affects the diversity of predicted solutions. Similar issues arise for other widely used molecular optimisation approaches. Therefore, we opted for a hybrid approach combining the strengths of design approaches of different nature. \n6. To cope with the problem of unbalanced data, we experimented with multiple approaches on the data level (oversampling, undersampling and others) and on the model level (application of other models, adjustment of weights and others). The best metrics were achieved with the approach described in the paper using the threshold for the probability of the positive class. However, we plan to continue working towards improving the performance of the model in predicting orthogonal planes.\n7. We appreciate the reviewer's valuable feedback regarding the importance of incorporating additional properties into our pipeline. In fact, we are actively working on extending our predictions to include solubility, tendency to form crystalline hydrates, and other properties of co-crystals. We are developing GEMCODE as an open source project, so we will release these updates as soon as they are sufficiently tested and validated. We believe that these enhancements will further improve the utility and applicability of our pipeline in the pharmaceutical and other relevant domains.\n\nWe would like to ask the reviewer to consider raising the score to 7, due to the novelty and importance of the application problem that GEMCODE solves. As far as we know, no paper on this topic has been presented at A* conferences yet (except for the workshop at ICML 2024 [3]). We have done substantial work on GEMCODE development and now have a chance to pioneer the application of generative AI to organic co-crystal design. We will continue to work on GEMCODE (including experimental validation) and will definitely take into account the comments of all reviewers.\n\n__References:__\n\n[1] Ye Z. H. et al. Searching new cocrystal structures of CL-20 and HMX via evolutionary algorithm and machine learning potential //Journal of Materials Informatics. – 2024. – Т. 4. – №. 2.   \n[2] O'Connor, D. (2023). (Co-) Crystal Structure Prediction With Machine Learned Potentials (Doctoral dissertation, Carnegie Mellon University).   \n[3] Birolo, R., Özçelik, R., Aramini, A., Gobetto, R., Chierotti, M. R., & Grisoni, F. (2024). Deep Supramolecular Language Processing for Co-crystal Prediction (https://openreview.net/forum?id=bQ9d2hzjW4&noteId=9SoErgR0kb).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OVPuQkwYFA", "reviewer": "Reviewer_3KPG", "summary": "This works presents a generative framework for co-crystal design which uses deep learning and evolutionary algorithms for optimization. The GEMCODE pipeline can be used to select optimal molecular pair combinations: an active pharmaceutical, and a coformer to control for the desired co-crystal properties. The work is an application of generative AI (specifically, GANs and VAEs) to an interesting and important problem in the pharmaceutical industry. The paper has great figures and is well-written, and the code appears well-documented. Nevertheless, the paper could improve from some slight re-structuring and/or re-writing to improve the clarity of the paper, as it was tough to follow in places (there are many molecular representations and model architectures used, many target properties, these could be better organized so that it is easier to follow). But overall I really enjoyed reading it and would like to congratulate the authors on the nice work.", "questions": "* When predicting co-crystal formation, do the authors also have any negative pairs, e.g., molecules that definitely do not co-crystalize? I would assume the bias is towards positive pairs, which perhaps leads to overestimation of the likelihood of co-crystallization for any new pair of molecules, but this was not fully clear to me.\n* In the validation case studies (section 5.4), it is unclear if molecules with overlap or significant molecular similarity to nicorandil, rivaroxaban, and paracetamol were excluded from the training set before using the model for these tasks, otherwise there may be data leakage and the conclusions are not so meaningful.\n* I see that molecular descriptors (I am guessing 2D?) were used for representing the coformers for mechanical property prediction. Were the fingerprints also explored here, to less effectiveness? Curious since it seems fingerprints were used in other parts of the pipeline, and not sure if descriptors were necessary for good performance here or not.\n* Related to the above question, how important is 3D information? I would expect a lot, and wondering if the authors considered including 3D atomic-level descriptors as well in any of their models.", "limitations": "* In addition to the final median probability scores attained by each model for the molecular optimization tasks explored in this study, it would be relevant and interesting to report the sample efficiency. This is a much more relevant metric in this context of molecular optimization, rather than the final probability score, as it gives a measure of how quickly a model learns (e.g., how many oracle calls are needed to obtain a specific score). Recommend to include an estimate of the sample efficiency in a revised version. \n* Is the “median probability score” not in fact a likelihood of sampling specific coformers (and not really a probability)? If so (or if not), I think this could be presented more clearly, as what exactly this metric is was not clear to me. It was probably defined somewhere in the paper but I could not find it easily.\n* One big limitation is that it would have been relevant to compare the models for the coformer optimization tasks to a simple baseline, such as a “virtual screening” of the coformers. The advantage of using the generative model presented herein should be that better molecules, and thus better scores, should be achievable with the generative model than by simply screening the database (given a fixed sample budget), but this has not been demonstrated in the current study. It would be a pretty simple/standard baseline since the reference database is small (~7K coformers).", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "* The work makes extensive use of existing frameworks (e.g., CCGNet) and datasets (e.g., ChEMBL, CSD) where possible, which is great, while still exploring the development of new tools when it comes to the generative aspect.\n* Overall the study is very thorough - many of the questions I found myself having while reading the paper were either answered later on in the paper or in the appendix. Those which were not, I have written below.\n* Good details provided in the appendix for the data and many of the methods, it seems it should be reproducible from the details provided herein (as well as the linked code).\n* This paper highlights an important application of generative AI to a new domain.", "weaknesses": "* It was a good choice, nonetheless, to explore non-neural models for the prediction of mechanical properties, which is where the authors have the fewest data points (6K). However, given the limited data on coformers (about 7K co-crystals), I also would have expected a simpler baseline like a random forest for estimating likelihood of co-crystal formation. \n* Clarity could be improved in places, since the authors are working with lots of different representations and models throughout their pipeline. Perhaps this could be better visualized in a way that gives a quick overview of the data type, number of data points, and model architecture, for each model included in the pipeline.\n* The model may have benefited from a more advanced hyperparameter tuning for the generation of coformers (e.g., Optuna rather than grid search), as the models and their final performance can be quite sensitive to the selected hyperparameters. For instance, %validity >99% should be attainable in all cases for the molecular generative models presented here, to which the models were close but not all quite there.\n* Some of the notation is not well-explained. For instance, in the target coformers equation, what is S? In many of the tables and figures, the axes labels or certain notation is implied, which may be mentioned somewhere else in the paper, but it would improve readability if readers could have a reminder of what that property is (and if it is bounded, a percentage, etc). Also, where there are error bars, it should be stated what these are (e.g., in tables, figures). It is a long paper so this would be appreciated, I found myself flipping back and forth a lot.\n* To better understand the diversity of the chemical space spanned by generated molecules, it would have been interesting if the authors better quantified that of the training set as well, and if different training/testing strategies could better assess the generalizability to new coformers rather than simply splitting the data. Were not certain classes of coformers more represented than others?\n* Given that the authors want others to use their pipeline to discover new coformers, it would have been great to quantify the coverage of chemical space of the model by metrics other than % validity and % novelty. Did the authors consider such metrics (or even dimensionality reduction visualization/techniques) to get a better sense of the chemical space coverage?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the high rating and very valuable comments! We appreciate the detailed feedback on where we can improve the clarity of the manuscript. We will do so in the camera-ready submission.\n\nBelow, we provide our __answers to the questions__:\n\n1. For the purpose of ranking molecular pairs according to the probability of co-crystallization, we employed the Co-Crystal Graph Network (CCGNet) model [1]. The authors used a dataset of 6819 positive and 1052 negative samples to train the model, so there is indeed an imbalance problem. Nevertheless, CCGNet was shown to achieve high accuracy on negative samples (97.26%) thanks to the combined use of graph representations of the underlying GNN and 12 molecular descriptors.\n2. In the validation case studies (Section 5.4), we predicted three co-crystal systems that were not present in the training dataset. By demonstrating that GEMCODE can generate novel co-crystal structures reported in the literature as experimentally validated, we aimed to showcase its strong predictive capabilities. We understand the reviewer's concern about data leakage and would like to clarify that we ensured that none of these co-crystal systems were included in the training set prior to the validation process.\n3. In our study, we indeed utilized a variety of descriptors to predict mechanical properties, encompassing molecular fingerprints of various types (Morgan, MACCS) and lengths (166, 512, 1024, 2048), molecular descriptors sourced from different origins (RDKit, Mordred, PaDEL), and 3D descriptors generated from RDKit (Autocorr3D, MORSE, PMI, etc.). We found that a set of molecular descriptors with physicochemical properties (29, 24, and 30 features for unobstructed planes, orthogonal planes, and hydrogen bonding, respectively) resulted in the best predictive performance.\n\nWe would like to also __comment on the weaknesses__ outlined by the reviewer:\n\n1. We appreciate the suggestion to consider using more advanced methods like Optuna for hyperparameter optimization. While we acknowledge that Optuna may provide more efficient tuning compared to grid search, we would like to highlight that grid search is a pragmatic choice for a moderate number of experiments (10-20), delivering acceptable results. In this work, we had to investigate and optimize numerous configurations of the pipeline. Given the complexity and time constraints associated with more advanced methods, we believe that our current approach is a tradeoff between performance and computational cost. We leave the more advanced methods for GEMCODE hyperparameter optimization for the future work.\n2. To our knowledge, there is no common stratification strategy for splitting the data in this domain. Another study on co-crystals accepted to ICML 2024 also employed a standard random split methodology [2]. Based on our empirical results, we do not expect any significant change in results for alternative data splits.\n\nIn addition, we would like to __comment on the limitations__ pointed out by the reviewer:\n\n1. Regarding the \"median probability score\", we believe it can be best described as the median probability of assigning coformers to a positive class for each of the mechanical properties. In other words, median probability score gives an idea about the central tendency of the model's confidence in predicting a particular mechanical property. This is, of course, related to the likelihood but reflects a different aspect of the model behavior. We will make sure to clarify this further in the camera-ready version of the manuscript.\n2. Table 1 illustrates the comparison of generative models focusing on “Target сoformers”. The percentage indicated in this row represents the ability of the model to generate new coformers with the desired co-crystal properties that were not present in the training dataset. We investigated that, for the Theophylline drug, the percent of target coformers (satisfying all three mechanical properties) in the training set was only 8.46%. Generating candidate coformers for this drug with T-CVAE only, we obtained 6.52% of target coformers. This clearly demonstrates how GEMCODE is capable of discovering previously unknown co-crystal systems. We appreciate the suggestion to illustrate how probability distributions of mechanical properties compare between the training and the generated data. We will add this evaluation to the camera-ready submission.\n\n__References:__\n\n[1] Jiang, Y., Yang, Z., Guo, J., Li, H., Liu, Y., Guo, Y., ... & Pu, X. (2021). Coupling complementary strategy to flexible graph neural network for quick discovery of coformer in diverse co-crystal materials. Nature Communications, 12(1), 5950.  \n[2] Birolo, R., Özçelik, R., Aramini, A., Gobetto, R., Chierotti, M. R., & Grisoni, F. (2024). Deep Supramolecular Language Processing for Co-crystal Prediction (https://openreview.net/forum?id=bQ9d2hzjW4&noteId=9SoErgR0kb).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BajFozXryQ", "reviewer": "Reviewer_qBrF", "summary": "This paper presents GEMCODE, the first co-crystal design AI pipeline, which consists of four components:\n- SMILES-based models for coformer generation.\n- Classification models for co-crystal property prediction.\n- An evolutionary algorithm for coformer optimization.\n- A GNN for prediction of the probability of co-crystal formation. \n\nIn addition, experiments are carried out on each component and the entire pipeline.", "questions": "1. In Table 1, the \"diversity of targets\" are all greater than 0.9. I'm not sure if they are values for internal diversity, and if they are, the diversity values are quite large for sets containing hundreds of molecules. From a molecular design perspective, this suggests that the design objective may be so simple that many very dissimilar molecules fit it.\n\n2. In the paper, the tabletability of co-crystals is presented as a target to be improved. However, in GEMCODE, the three properties related to tabletability are all binary variables, so for a coformer molecule, its tabletability is also represented by one binary variable. Therefore, is the word \"enhance\" inappropriate for tabletability?\n\n3. Of the three coformer molecules generated in Table 3, two contain ions. I'm not sure if ions are common in coformer molecules, but I think they're not common in the ChEMBL database. So, is there a significant distribution difference between the pre-trained and fine-tuned datasets?", "limitations": "Nothing beyond what has already been stated in Section 6.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "This paper is the first to introduce generative AI into co-crystal design, which is a very important topic for drug development and other fields. Specifically, this paper establishes a complete framework for co-crystal design, including datasets, property prediction, coformer generation, optimization, and validation, which can be recognized as the initial baseline in this field. In addition, the experimental results are detailed and the logic is clear.", "weaknesses": "1. The experimental results of property prediction show that the three mechanical properties are inherently elusive, because the accuracy and F1 score of most models in Figure 2 are less than 0.8, which is not ideal for binary classification tasks. Therefore, my concern is that this paper is a pioneer in both the property prediction and generation/optimization of co-crystals, and will the defects (or biases) of the property prediction models be brought into the generation/optimization?\n\n2. In GEMCODE, coformer de novo generation and optimization are divided into two steps, which I think is to follow the tradition in the field of drug discovery. However, in drug discovery, the main purpose of de novo generation is to find molecular candidates whose \"main properties\" (such as docking scores) are satisfactory, while the main purpose of optimization is to optimize some other properties (such as toxicity and solubility) while maintaining the \"main properties\" (usually by a similarity constraint). The property objectives of de novo generation and optimization in GEMCODE is the same (without any similarity constraint), which makes me doubt the rationality of the pipeline.\n\n3. Admittedly, I am not familiar with the co-crystal's chemical background, so it is difficult for me to evaluate GEMCODE's chemical validity. Actually, I also did not find any paper on co-crystals in top conferences on machine learning, which makes me question this paper's suitability for publication at this conference. Overall, the main contribution of this paper is to model the computational pipeline of co-crystal design, and to apply existing AI techniques to it. Therefore, in terms of style, I think the chemistry and cheminformatics community may be more appropriate for this paper than a machine learning conference.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's useful comments and suggestions!  \nBelow, we would like to provide our __answers to the questions__:\n\n1. We are happy to provide a clarification for the high values of \"Diversity of target\" in Table 1. The diversity values were calculated for the molecules that have been pre-screened for novelty (meaning they are not present in the training dataset), validity, and matching mechanical properties. Therefore, a diversity value exceeding 0.9 appears to be reasonable. Furthermore, the formation of co-crystals can involve molecules with varying chemical structures. Consequently, GEMCODE should have the capability to accommodate diverse chemical structures. This is achieved through hybridization of generative neural networks with evolutionary optimization.\n2. Regarding tabletability, we believe that creating a coformer molecule with a target tabletability profile (i.e., the target mechanical properties) can be seen as an enhancement in the tabletability of the co-crystal. However, we are open to adjust the wording if the reviewer suggests a more accurate term.\n3. To observe the difference between the pretraining and the fine-tuning datasets, please refer to the t-SNE visualization in Figure 5b of Appendix D.2. The figure shows that the coformer dataset produces a more concentrated set of molecules compared to ChEMBL. The list of coformers was sourced from [1], which was also used by the CCGNet model for predicting co-crystallization probabilities. By integrating CCGNet and training all the other components of our pipeline on the same data, we show consistency of the GEMCODE design. The presence of ionized forms in Table 3 arises from SMILES representations in the coformer dataset indicating charge distribution within _neutral_ molecules (e.g., \"Nc1nc2ccc(N+[O-])cc2s1\" with a nitro group,  \"CSCCC([NH3+])C(=O)[O-]\" with carboxyl and amino groups). Consequently, the model has learned to generate charged molecules (e.g., with carboxyl \"C(=O)[O-]\") in certain instances. We are expanding the scope of GEMCODE beyond co-crystals to encompass other crystalline forms, such as salts. This expansion will aim to specifically address model biases related to assigning charged molecules to co-crystals.\n\nWe would like to also __comment on the weaknesses__ outlined by the reviewer:\n\n1. We thank the reviewer for highlighting a key limitation in our study. To our knowledge, we are the first to predict mechanical properties of organic co-crystals. Therefore, our work sets the state of the art for the problem. We have run extensive evaluations of various machine learning models incorporating a range of descriptors striving for better performance. We are confident that the capability of predictive models is limited by the training data. As more data of sufficient quality becomes available, we will refine GEMCODE accordingly. Notably, despite the current limitations, several validation cases presented in this work demonstrate the capability of GEMCODE to successfully predict new co-crystals, experimentally validated and reported in the literature.\n2. The main motivation for incorporating evolutionary optimization in GEMCODE was to address the aforementioned data limitations. Our dataset consists of approximately 6000 coformers. Training generative neural networks on a dataset of this size may lead to a restricted diversity in the generated molecules. By implementing evolutionary optimization as a separate step in the pipeline, we overcome this limitation by design, as the evolutionary algorithms operate independently of the training data. Furthermore, evolutionary optimization can help mitigate the drawbacks of the machine learning models discussed earlier. The optimization process focuses on enhancing the likelihood of coformers possessing all the necessary mechanical properties. As a result, a molecule generated by the neural networks can be further refined through evolutionary optimization in terms of this likelihood. In such a scenario, the evolutionary optimization step is complementary to the initial generation and demonstrates the advantage of the hybridization approach we presented. Finally, evolutionary optimization is designed to retain the coformers with the highest likelihood on each iteration. In other words, it is guaranteed to improve the mechanical properties of coformers while adding diversity to the pool of the generated molecules.\n3. We are confident that our paper aligns well with the NeurIPS guidelines being an _application_ of _machine learning for sciences_ (https://neurips.cc/Conferences/2024/CallForPapers). While co-crystals may not have been a prevalent topic at A* conferences, we see this as an opportunity to expand the horizons of machine learning research and engage with researchers from other fields. Notably, a recent study in this direction was presented for the first time at the ICML 2024 workshop [2]. This only highlights the originality and significance of our work.\n\nIn conclusion, we would like to quote the reviewer, _“this paper is a pioneer in both the property prediction and generation/optimization of co-crystals”_. Although this was expressed as a concern, we would like to point out that this quote, in fact, is highlighting a great scientific achievement. Provided that we have sufficiently addressed all the comments and questions, we kindly ask the reviewer to consider increasing the rating to 6. Given that we have initially received the ratings of 8/6/4/4, this would significantly increase our chances to get accepted.\n\n__References:__\n\n[1] Jiang, Y., Yang, Z., Guo, J., Li, H., Liu, Y., Guo, Y., ... & Pu, X. (2021). Coupling complementary strategy to flexible graph neural network for quick discovery of coformer in diverse co-crystal materials. Nature Communications, 12(1), 5950.  \n[2] Birolo, R., Özçelik, R., Aramini, A., Gobetto, R., Chierotti, M. R., & Grisoni, F. (2024). Deep Supramolecular Language Processing for Co-crystal Prediction.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes GEMCODE: a pipeline to design novel coformers that could form tabletable co-crystals with a given organic molecule. GEMCODE is comprised of a sequence of approaches, including generative and predictive models, as well as evolutionary algorithms. The authors conduct an empirical coformer design study using known drugs, showing that GEMCODE can both rediscover known coformers and also propose promising new ones.

The paper sparked significant debate among the reviewers, both during the author rebuttal period, as well as later during the inter-reviewer discussion. Notably, one reviewer consistently championed the work (rating of “Strong Accept”), and one leaned slightly towards rejection; the other two reviewers were undecided (shifting their ratings up during rebuttal but then down during reviewer discussion) but ended up leaning slightly towards acceptance.

Main positive points raised by the reviewers were as follows:

-	This paper is one of the first efforts in its domain (subdomain of the broader “AI for Science”), and likely would be the first one accepted at a conference such as NeurIPS. Its acceptance could generate more interest in applying ML to an impactful scientific problem.
-	The paper is well-written, thorough, and makes extensive use of existing approaches/datasets where appropriate (it does not strive for novelty for novelty’s sake).
-	Experimental results, albeit purely in silico, appear promising.

Some of the issues highlighted by reviewers were addressed by authors during the rebuttal, and hence are not further discussed here. Main complaints that were not resolved are as follows (each accompanied by a counterpoint raised in discussion):

-	To confirm GEMCODE is fully practical, it would be great for the work to (a) predict/consume crystal structure instead of SMILES alone to account for polymorphism, and (b) include wetlab experiments to verify the results. While everyone generally agreed this would be ideal, the positive reviewer argued the work is already useful as a starting point for future research. However, reviewers noted that, in the presence of these limitations, the claims made in this work may appear overconfident.
-	ML approaches used in this paper are generally not novel (mostly simple adaptations of existing ideas), and the work appears more valuable from the domain side, raising questions as to whether it’s appropriate for NeurIPS. While I agree with the lack of novelty, my own comment on this would be to keep in mind that “AI for Science” is a strong theme at NeurIPS-like conferences, and while most accepted papers in this domain do have significantly more novelty than GEMCODE, they unfortunately are often of limited practical use, precisely because NeurIPS sometimes tends to select for novelty over practicality. If anything, accepting a more practical and less methodologically novel work could correct for this bias. Future works on co-crystal design would have something to compare to, and likely also bring more ML novelty.

Summing up, this paper generated a good deal of healthy disagreement between the reviewers. I think the arguments from both sides are perfectly reasonable and could see this paper falling on either side of the acceptance bar. **As the paper pioneers a new subdomain of “AI for Science” that could be of interest to the NeurIPS community, and it does so in a practical way, I am willing to lean into its positive aspects, and recommend the acceptance of the paper.**
That being said, the complaints listed above remain, and I would **urge the authors to make a serious attempt at integrating all the review feedback** into their work. In particular, please make the limitations of GEMCODE (e.g. not dealing with polymorphism, lack of experimental validation, etc) clearer, and discuss the extra references and points of view provided by the reviewers.
Decision: Accept (poster)