InversionGNN: A Dual Path Network for Multi-Property Molecular Optimization
Authors: Yifan Niu
Ziqi Gao
Tingyang Xu
Yang Liu
Yatao Bian
Yu Rong
Junzhou Huang
Jia Li
Venue: ICLR 2025 Poster (2025)
Keywords: multi-objective drug discovery
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: Exploring chemical space to find novel molecules that simultaneously satisfy multiple properties is crucial in drug discovery. However, existing methods often struggle with trading off multiple properties due to the conflicting or correlated nature of chemical properties.  To tackle this issue, we introduce InversionGNN framework, an effective yet sample-efficient dual-path graph neural network (GNN) for multi-objective drug discovery.  In the direct prediction path of InversionGNN, we train the model for multi-property prediction to acquire knowledge of the optimal combination of functional groups.
Then the learned chemical knowledge helps the inversion generation path to generate molecules with required properties. 
In order to decode the complex knowledge of multiple properties in the inversion path, we propose a gradient-based Pareto search method to balance conflicting properties and generate Pareto optimal molecules. 
Additionally, InversionGNN is able to search the full Pareto front approximately in discrete chemical space. Comprehensive experimental evaluations show that InversionGNN is both effective and sample-efficient in various discrete multi-objective settings including drug discovery.

Published as a conference paper at ICLR 2025
INVERSIONGNN: A DUAL PATH NETWORK FOR MULTI- PROPERTY MOLECULAR OPTIMIZATION
Yifan Niu', Ziqi Gao Tingyang Xu², Yang ? Yatao Bian', Yu Rong", Junzhou Huang?,
Jia Li
'The Hong Kong University of Science and Technology (Guangzhou)
The Hong Kong University of Science and Technology
3 DAMO Academy, Alibaba Group 4 *Tencent Al Lab Uniivessiy of Texas, Arlington
ABSTRACT
Exploring chemical space to find novel molecules that simultaneously satisfy multiple properties crucial in drug discovery. However, existing methods often struggle with trading off multiple properties due to the conflicting or correlated nature chemical properties. To tackle this issue, we introduce InversionGNN framework, an effective yet sample-efficient dual-path graph neural network (GNN) for multi-objective drug discovery. In the direct prediction path of InversionGNN, we train the model for multi-propent prediction to acquire knowledge of the optimal combination of functional groups Then the learned chemical knowledge helps the inversion generation path to generate molecules with required properties. In order to decode the complex knowledge of multiple properties in the inversion path, we propose gradient-based Pareto search method balance conflicting properties and generate Pareto optimal molecules. Additionally, InversionGNN is able to search the full Pareto front approximately in discrete chemical space. Comprehensive experimental evaluations show that InversionGNN is both effective and sample-efficient in various discrete multi-objective settings including drug discovery. The code is available at https//gtthhh.com//vvnmm///essss
INTRODUCTION
Molecular optimization refers to the process of systematically modifying the structure of given molecule to enhance its properties in practical drug discovery. It combines known chemical knowledge with innovative exploration discover and develop unknown high -erffrrrrromecc molecules. Molecular optimization is challenging as it usually involves reasoning about multiple, often conflicting or correlated, objectives (Fromer & Coley] 2023). For example, for a new drug to be successful, must simultaneously be potent, bioavailable, safe, and synthesizable (Dara al. 2022). Generally, these objectives often exhibit implicit relationships, which can be either conflicting or correlated, rather than being independent (Jain al.) (023). For example, molecules that effective against target may also have detrimental effects on humans.
Although several drug discovery models have been proposed tackle Multi-Objective Molecular Optimization (MOMO), most of them do not make full use of acquired chemical knowledge, such as how the combination of substructures affects chemical properties. More specifically, they merely employ a pretrained chemical property predictor as the discriminator, filtering high-performance molecules based the predicted scores (Nigam et al. 2020 Xie al. 2021 Brown al. 2019).
Additionally, most studies neglect the conflicting or correlated relationships among chemical proper- ties and simply use a predefined Linear Scalarization function (c.g.. mean) perform a weighted
summation of the losses (Jin et al. 2018) Wang et al. 2023) De Cao & Kipl 2018 Shi et al. 2020 Liu et al. 2021 Fu al.) 2022; Zhou al.) 2019 Jin et al.] 2020b) Jain et al.) 2023] Xie al. 2021). However, Linear Scalarization often results in biased solutions and leaves certain areas of objective space unexplored, which has been mathematically analyzed by Boyd et al. (2004). Abbasi
al. (2022) employs multi-objective Genetic Algorithms tackle MOMO, but it is computationally expensive and requires a large number Oracle calls. Recently, Jain et al. (2023) and Zhu et al.
"Correspondence Jia Li (jial aeauset.

---

Published as a conference paper at ICLR 2025
(204)) adopt the multi-objectivn Bayesian Optimization to address MOMO. Nevertheless, both of them suffer from high computational costs and struggle with nigh-dimensional optimization.
To design an effective yet sample-efficient model for MOMO, we identify two key challenges:
1. How to make full use of acquired chemical knowledge to facilitate molecular optimization? To
tackle this, we introduce dtul-ppth graph neural network (GNN) (Kipf & Welling 2016 Xu et al.
2019| Velickovic et al.] 2018 Gilmer et al., 2017] Gao et al., 2024b] Liu et al.) [2024] Gao et al. [023]
Li al. 2024 2025) incorporate complicated chemical knowledge and perform molecular property prediction in the direct prediction path. Unlike existing methods that regard it as a discriminator,
in the inversion generation path, we leverage the gradient w.r.t the molecule structure to optimize the molecular graph. The mapping of chemical structure to properties, learned through the direct path, can effectively guide the editing of chemical structure to a base molecule in the molecular optimization process.
2. How to dealing with the conflicting or correlated properties in the inversion generation path? In our dual-path GNN model, the direct path easily learns each property distribution using different classification heads. However, in the inversion generation process, is challenging to apply multiple complicated property constraints one single molecule. For example, given two conflicting proper- ties, enhancing one property will weaken the other property. To capture all possible trade-offs among conflicting correlated properties, we introduce the gradient-baseds Pareto optimization technique, which is designed to balance multiple conflicting or correlated objectives (Zhou et al. 2023 Ju et al.. 2022, Liu al. 2022). Instead of directly deploying existing Pareto methods, we adopt the relaxation technique to adapt the continuous Pareto optimization to the discrete chemical space. We provide a convergence analysis demonstrating that our approach, with our relaxation, still converges to the Pareto optimal solutions approximately within only a few iterations. This means our method can effectively increase the likelihood of successfully generating high-quality molecules with sample efficiency. Our key contributions are summarized below:
We propose a novel dual-path InversionGNN for multi oujti-objectiie molecular optimization, which leverages the acquired property prediction knowledge to facilitate molecular opti- mization. It is effective and sample-efficient
To capture trade-offs among conflicting correlated properties, we adpot relaxation tech- nique that adapts gradient-based Pareto optimization to the discrete chemical space.
We empirically verify that InversionGNN is both effective and sample-efficient in various real-world discrete multi-objective settings including drug discovery.
2 RELATED WORK
Molecular Optimization. Recent years have witnessed the success of applying deep generative models and molecular graph representation learning in drug discovery. Most of the existing works can be categorized into two classes: Constrained Generative Model (CGM) and Combinatorial Optimization (CO) algorithm. CGMs model the molecular distribution with deep generative networks such as VAE Gtómez-Bomband
et al.] 2018 Liu et al. 2018)) Jin et al.)
2018 2019; [Sklic et al. 2023), GAN (Guimaraes 2020). Energy (Liu et al.
2019 Fu al. 2020: Grriffttss & Hernández-Loobate 2017, Cao & Kipf] 2018]
2020 Wang et al.
et al.
et al. 2022). ), Flow (Shi et al.
1021) and Diffusion-based model (Lee et al. (203)) projecting input molecules into latent space. However, obtaining the ideal smooth and discriminative latent space has proven to be a challenge in practice (Brown et al. 2019 Huang et al.) 202T Gao al. 2025). Another research line based on CO directly searches for desired molecules in the explicit discrete space. e.g., Reinforcement
Learning You et al.) 2018, Stähl et al. 2019 Zhou al.
2019 Jin et al., 2020b Gotipati et al.
2020: Gao et al. [2024a] Jain et al. 2023 Popova et al.
2018 Jin et al. (220a)). Exolutionary
Algorithms (Jensen [019 1 Nigam et al. 2020 Chen et al. 2021). Markov Chain Monte Carlo (Xie)
ct al. 2021 Fu et 2021). Tree Search (Fu al. 2022) and Bayesian Optimization (Korovina
ct al., 2020 Moss et al. 2020). CO algorithms require massive numbers of Oracle calls, which is computationally inefficient during the inference time. However, they are still challenged in dealing with confficting or correlated properties.
Gradient-Based Pareto Optimization. The Pareto optimal solution is highly valuable for multi- objective optimization since identifying solutions that simultaneously maximize all objectives is often impractical. In order efficiently find Pareto optimal solutions, MGDA (Désidéri) 2012) is proposed
2

---

Published as a conference paper at ICLR 2025
to identify Pareto optimal solutions for low- www-dimensional data. Sener & Koltun 2018 extend MGDA
to high-dimensional culli--bjeective scenarios. Subsequently, several efficient Pareto optimization
methods (Lin al. 2019; Zhang & Golovin 2020 Ma et al. 2020; Mahapatra & Rajan 20200)
have been proposed to explore the Pareto set, due to the fact that MGDA cannot find Pareto optimal solutions specified by exact objective preference. However, most efforts of Pareto optimization focus on continuous parameter space, ignoring the complex discrete chemical space.
3 PRELIMINARIES
3.1 PARETO OPTIMALITY
In this work, we consider m tasks described by (()) == []i(z)], where each fi(æ), E [m]] represents the performance of the i-th task to be maximized. Given an desired target y E IR" set non-negative objective function (x)())) y) 11...., Im]' where I; for i [m]] is the objective function of the i-th task. Hence, maximizing the performance f(x) is equivalent to minimizing the
objective function. For any two points æ, x RR. x dominates x', denoted by L : L", implies dominated by any other points in R". The set of all Pareto optimal solutions is denoted by P. The set of multi-objective values of the Pareto optimal solutions is called Pareto front, denoted by F. In Multi Multi-Objece Objective Optimization, the ideal goal is to identify set of Pareto solutions that cover all the possible trade-ofi offs among objectives. For a formal definition of Pareto concept, please refer to Appendix A
1 17 2 Vi E [m]. A point x" said be Pareto optimal if æ
3.2 DIFFERENTIABLE SCAFFOLDING TREES
scaffolding tree (Tin al., 2018), Te. a spanning tree whose nodes are substructures. is a high- level representation of molecular graph a E X. For scaffolding tree with K nodes and substructure set S, is represented by T = {N, A, J: (i) node indicator matrix defined by N {0, 13 1 xx|s| and each row of N is a one-hot vector, indicating the substructure of the node; (ii) adjacency matrix denoted by A E [0,1] K [0.1] while indicates unconnected; and (iii) node weight vector w [l.....]] 1]T R indicates the K nodes are equally weighted. Fu ct al. (2022) add virtual expansion node set Vexpond = {u, v E VT. [Vexpand = K Kexpand =
where A, indicates the i-th node and the j-th node are connected
the scaffolding tree for structure modification.
can be converted to a K + K Keapand nodes differentiable scaffolding tree T. = (Ň, A, w). The differentiable scaffolding tree facilitates the substructure addition, deletion, and replacement.
METHOD
In this study, we explore an InversionGNN framework to address the problem of (1) finding Pareto optimal molecules conditioned on desired weight and (2) finding diverse Pareto optimal molecules with all possible trade-offs. We illustrate the pipeline of InversionGNN in Figure
Dual-Path Network: InversionGNN. We introduce a dual-path graph neural network (GNN)
incorporate complicated chemical knowledge in the direct prediction path (Section 4.2) In the inversion generation path, we leverage the gradient w.r.t the molecule structure to guide the molecular optimization process.
Baadient-Based Pareto Inversion. In order to inverse the complicated multi- aulti-property knowledge, we relax the discrete molecule optimization into a locally differentiable Pareto optimization problem. We reorganize the gradients into non-dominating gradient (Section 4.3)
4.1 PROBLEM FORMULATION
Definition 4.1 (Multi-Ohjective Molecular Optimization (MOMO)). Given the Chemical Space X, Oracle function fxx) objective function L, the target property score y E R of m properties, the goal of MOMO find candidate molecules x' E X that minimize all objectives:
x²
arg min me ex C(f(x)) y).
(1)
3

---

Published as a conference paper at ICLR 2025
InNersionGNN
Pareto Front Search
Molecular Graph
Edit
objective space with various
Milocule
W Weight Vector
Search
2 dnd GB
Craph Core
Diject
Updato Soatholding Tree with ded
Pureoo-dreesstty 1
Innvessoo
Mape
15 Caleulate
Non- Dominating Gradiet
Al Ray
Difforntabll Scaffolding Tree
22
Pretrderd Strrogdi Orace (Fied))
fron
(Leomable)
Figure 1: (1) InversionGNN. surrogate Oracle GNN is trained to incorporate complicated chemical knowledge. In the direct prediction path, a molecule x' is fed to the GNN obtain the objective function at the t-th iteration. In the inversion path, we calculate the non-dominating gradient to find local Pareto-optimal molecules P' conditioned on given weight vector X. (2) Pareto Front Search. Exploring the full Pareto front approximately with various weight vectors, improving the Pareto diversity of generated molecules.
When objectives [I,...., Im]] are conflicting, there no single x" which simultaneously maximizes all objectives. Consequently, multi-objective optimization adopts the concept of Pareto optimality, which describes set of solutions P that provide optimal trade-offs among the objectives.
4.2 A DUAL-PATI NETWORK: INVERSIONGNN
In this section, introduce dual-patt network, InversionGNN, consists of direct prediction path and a inversion generation path for multi-objective molecular optimization.
Direct Prediction Path. We aim to develop pretrained GNN [(x; 0) to capture knowledge from the ground truth Oracle O. We imitate the Oracle function O using multi-head architecture to individually predict m property scores ÿ R":
y = f(x; 0) Re
where is the learnable parameters. We adopt the Graph Convolutional Network (GCN) (Kipf & Welling, 2016) to extract the representations of molecules. The updating rule for the I-th layer is H RELU (B + A (H(-)))() where ( E IR kxd Rxxdd/UU /U(() dadd are bias/weight parameters and A is adjacency matrix. We leverage the weighted average as the readout function of the last layer's 's node embeddings, followed by neal MLP to yield the prediction of m properties
9 MLP Σ wk E H (L)) ). We train the model by minimizing the discrepancy between
the prediction y and the ground truth y/:
0" arg min C (y, 9)
where C is the loss function, e.g. binary cross entropy. The parameters of the surrogate Oracle model pretrained at once and freezed in the inversion generation path.
Inversion Generation Path. Instead of using the GNN model as a simple predictor, we decode the stored chemical knowledge in the GNN model and use its gradient w.r.t the input molecule guide molecular optimization. Let f; = VI, represent the gradient of the i-th property objective function. Consequently, we obtain target properties. In order to ensure the molecular learnable, InversionGNN requires a differentiable molecular representation in chemical space. Compared the widely used non-differentiable discrete scaffolding tree Ta, the currently only available differentiable representation is the differentiable scaffolding tree proposed by Fu al. (2022). Hence, we employ the differentiable scaffolding tree, denoted as T as the representation of molccular æ in InversionGNN.
V I91...... 9m.] by back-propagating the derivatives from the
Compared to InversionGNN, vanilla GCN (Kipf & Welling 2016 Liu et al. 2023) is often used for prediction tasks, and its inference prediction process is the direct prediction path in InversionGNN, InversionGNN has the same structure as vanilla GCN but has different computation process. In contrast, InversionGNN contains an additional inversion path for generation tasks, which allows the inverse of the gradients the input molecules.
4

---

Published as a conference paper at ICLR 2025
Algorithm I: InversionGNN
Input:
Input molecule æ. E X, weight vector À E IR"", and step size rj > Output: Generated Molecule x²". Initialization.
Train surrogate Oracle according to Eq.
for f = 1, ,..... . do
Convert molccule x' to differentiable scaffolding tree Tat: for 1: = 1,... do
Compute gradients of target objectives w.r.t. T V= = [91...... 9m]
Determine 3 by solving QP problem Eq.
5
Calculate non-dominating gradient dnd G
Update the differentiable scaffolding tree: Tk+1 x
Th rdndas
10
end
Sample discrete Tat+1 from continuous T and assemble it to molecule x*+1
11
12 end
4.3
GRADIENT-BASED PARETO INVERSION
In this section, capture possible trade-offss among conflicting correlated properties, adopt the relaxation technique and reformulate the discrete molecule Pareto optimization into a locally differentiable problem. At the 1-th iteration, given one molecule æt, aim find local Pareto optimal molecules set pi from the neighborhood set set (xx)):
x ++ pp ((=')
where (Nx')) is the set the possible molecules obtained by (1) imposing one local editing operation (expand, remove replace one substructure) to scaffolding tree and (2) assembling the edited trees into molecules.
Identifying the Non-Dominating Gradient. To approach the Pareto front, Désidéri 2012) demon- strated that the descent direction d can be found within the convex hull of the gradients, i.c., d E CH2 == {GB), where 3 StM belongs the m-dimeeniioall simplex. In order identify the Non-Dominating Descent Direction dndd = G3 inspired by continuous Pareto optimization (Mahap- atra Rajan 2020), we solve the following Quadratic Programming (QP) problem:
0 arg min |G
GB all
1
J [m]
KL A1) & KT. A(1 >E
s.t. B G
93 20 Yj YjEJ=
(5)
--=jjE j E arg max Li A j'E(m]
where J
a is the anchoring direction (Mahapatra Rajan 2020), and R a predefined weight vector that indicates the importance of each property. Then, calculate the non-dominating direction dnd G and update the differentiable scaffolding tree with To T -dddd Therefore, yield solution T- that is not dominated by T in neighborhood (Ta:))
Molecule Search in Discrete Chemical Space. At the 1-th iteration, we begin with molecule o and convert it to differentiable scaffolding tree T... Subsequently, identify the local Pareto optimal solution T: within the neighborhood set AV by performing rounds of gradient descent against the non-dominating direction d.a-- From T we can sample the discrete scaffolding tree T and assemble it to molecules, denoted as f+1 in the following iteration. Our proposed InversionGNN is summarized in Algorithm
5

---

Published as a conference paper at ICLR 2025
Omarle 100M5
-9
Oradie Cate 102-5
Dracle 12003
-5
Orecke Calls 1001
1.0
0.5
36-
0.8
0.5
Ray
Roy
Ray
Ray
30
0.4
0.4
0.4
0.2
0.2
616
Peretovront
Parelo Paretoroon
Fx280
Peretorront
Poretovront
4
1.00
1.00
(a) InversionGNN
(b) I-LS
(c) GAN-GA
(d) HN-GFN
Figure 2: Pareto front (black solid curve) for two loss functions 11,2 and solutions (circles) and Oracle calls (computational cost) for different weights a = A (dashed rays). The weight À conditioned Pareto optimal solution is the intersection points between the Pareto front and X- rays.
4.4 CONVERGENCE ANALYSIS.
In this section, we provide the theoretical analysis of InversionGNN, discussing its convergence properties in the discrete chemical space.
Theorem 4.2 (Approximation Guarantee). Under the assumptions in Sec. B given initial molecule
x" and a weight vector X, InversionGNN guarantees the following approximation when performing T optimization rounds:
L ===: (1-)))))))) <" --
where 7
T [1- A- = (1/Ai,...,
1/2...,,,,/mm/ 1/2m), X and X is the maximum relative objective value
X = max (1yyy 1j E [m]} of x and x
Remark. This theorem tells that InversionGNN can generate desired Pareto optimal molecules approximately within few steps, and it is thus sample-efficient. Moreover, it implies that the Pareto optimal molecules conditioned on the given weight vector, enabling chemical experts to design molecules that meet specific practical drug design requirements. Consequently, we can obtain an approximate set of Pareto optimal molecules that encompasses all possible trade-offs with various weight vectors. For more details about the proof. please refer to Appendix B
Time Complexity. We did computational analysis in terms of inversion calls and computational complexity per iteration. (1) Inversion Calls. InversionGNN requires O(TM) Oracle calls, where T is the number of iterations. M is the number of generated molecules, we have M < N. N is the number of nodes in the scaffolding tree, for small molecule, N is very small. J is the number of enumerated candidates in each node, (2) Computational Complexity per Iteration. The computation of GH G has runtime Om,²n, where n is the dimension of the gradients. With the current best QP solver (Zhang et al. 2021). we have a runtime of (m3)). Thus, the per-iteration time complexity is O + m²² Since in deep networks, usually n >> m., InversionGNN does not significantly increase the computational cost in computing non- con-dominating gradient. The comparison of computational complexity between different methods is included in Appendix C
5 EXPERIMENTS
In this section, we present our empirical findings which aim to answer the following questions: Q1: Can InversionGNN identify the Pareto optimal solution conditioned on desired weight?
Q2: Can InversionGNN explore the full Pareto front approximately?
5.1 SYNTHETIC TASK
This section evaluates the InversionGNN via commonly used synthetic objective in multi-objective optimization from Pareto MTL (Lin et al. 2019). Different from the previous works in continuous space, optimize the problem in discrete space. We aim to minimize two non-convex objective

---

Published as a conference paper at ICLR 2025
functions, denoted as:
(x) =1 -
va (x))) 1
vm
where æ represents a point in discrete Euclidean space, with its dimension set to n 20. For these two objective functions, we are able to obtain the ground truth of the Pareto front.
Metrics. We use the standard metrics in multi-objective optimization: Hypervolume (HV) Zitzler & Thiele 1999) measures the volume in the objective space spanned by a set of non-dominated
solutions and also represents Pareto diversity. We set the reference point (1, 1) in this task.
Pareto Optimization Conditioned on Desired W1ight(Q1). To address this question, adopt 5 weight vectors Ray)- The goal is to find the intersection points between the Pareto front and the weight X-1 ray. We provide the details for generating weight vectors in Appendix D.3 For fair comparison, we incorporate Linear Scalarization into our framework as baseline and term it as I-LS, refer Appendix [.22 Due to the incompatibility of Genetic Algorithms and Bayesian Optimization with our framework, we adopted optimization techniques from two state-of-the-art drug discovery approaches,
Table 1: Hypervolume in Synthetic Task.
Method
HV (t)
I-LS
0.071 +0.003
GAN-GA HN-GFN
0.202
0.187
InversionGNN
0.328
GAN-GA (Abbasi et al., 2022) and MOGFN-AL (Jain et al. 2023), as baselines. InversionGNN and I-LS are optimized from random initialization for each weight vector. For GAN-GA, we evolve 300 iterations with population size of 40 and 10 offsprings. For MOGEN-AL, we start with 1000 points and running 100 optimization loops. We show the whole optimization process and Oracle calls in Figure 2 It illustrates that our InversionGNN framework not only captures the trade-offs among objective based on given weight vectors but also achieves the highest efficiency with 500 Oracle calls. In contrast, other baselines result in biased solutions.
Exploring Full Pareto Front (Q2). To address this question, we utilized 50 weight vectors scan the entire Pareto front for InversionGNN and I- LS. As shown in Figure 3 our pro- posed InversionGNN can explore al- most the full Pareto front. In contrast, the solutions of I-LS tend to cluster at the ends of the Pareto front, leav- ing certain regions unexplored. The results of GAN-GA and MOGEN-AL shown in Figure 2 they merely find a small subset of the Pareto opti- mal solution. We also report HV and Oracle calls in Table[]] which shows that InversionGNN is able to cover the full Pareto front compared with baseline methods. GAN-GA and MOGEN-AL cost 12000 and 3000 Oracle calls respectively. but still achieve suboptimal performance.
1
1.0
0.5
0.6
0.6
0.2
0.2
goond buti posditerr
ground S--h)
prediction
0.00
0.25
0.50
0.75
1.06
0.00
0.25
0.50
0.75
1.00
(a) InversionGNN
(b)
I-LS
Figure 3: Pareto Front Search.
5.2 MULTI- MOLECULAR OPTIMIZATION
In this section, we address the two questions by evaluating InversionGNN in discrete chemical space.
Dataset. We train the model on ZINC 250K dataset (Sterling & Irwin 2015), which consists of 250K drug-like molecules extracted from the ZINC database. We select the substructures that appear more than 1000 times as the vocabulary set S, which consists of 82 frequent substructures.
Implementation Details. We implemented InversionGNN using Pytorch (Paszke et al. 2019). Both the size of substructure embedding and hidden size of GCN are d = 100. The depth of GNN L is 3. In each generation, we keep C' = 10 molecules for the next iteration. The learning rate is 1e - 3 in training and inference procedure. We set the iteration T to large enough number and tracked the result. When Oracle calls budget is used up, we stop it. All results in the tables are from experiments up to T = 50 iterations.
7

---

Published as a conference paper at ICLR 2025
Table 2: Weight Conditioned Molecular Optimization.
Method
Nov(1)
Div(1)
APS(1)
NU(L)
MOEA/D
100%
n/a
0.27910..00
0.088 10..000000
NSGA-III
100%
n/a
0.351 1:0. 5.0.24
0.10220.0117
MOGFN-PC
100%
0.507
0.393
0.088
HN-GFN
100%
0.571
0.418
0.072+00015
I-LS
100% %
0.541 2:0000 007
0.529 2:0 008
0.049
InversionGNN
100%
0.435 000
10.012
0.026.0.001
Table Multi-Objective Drug Discovery.
GSK33 + JNK3
(SK35+NK3++FF445
Method
Nov
Div(T)
(PS(C)
(racle(L)
Nov(1)
Div(1)
APS(T)
Oracle(
LigGPT
100%
0.845
0.271
100K+0
100%
0.902
0.378
100K+0
GCPN
100
0.578
0.293
0+200K
100%
0.596
0.450
0+200K
MolDQN
100%
0.605
0.348
0+200K
100%
0.597
0.365
(4200K
GA+D
100 %
0.657
0.608
0+50K
97%
0.681
0.632
0+50K
RationaleRL
100 %
0.700
0.795
25K+67K
99%
0.720
0.675
25K+67K
MARS
100 %
0.711
0.789
0+50KK
100%
0.714
0.662
0+50K
ChemBO
98%
0.702
0.747
0+50K
99%
0.701
0.648
0+50K
BOSS
99%
0.564
0.504
0+50K
98%
0.561
0.504
0+50K
LSTM
100
0.712
0.680
0+50K
100%
0.706
0.672
0+50K
Graph-GA
100%
0.634
0.825
0+25K
100%
0.723
0.714
0+25K
DST
100 %
0.750
0.827
10K+5K
100%
0.755
0.752
20K+5K
MOGEN-AL
100%
0.673
0.742
50K+20K
100%
0.711
0.621
50K+20K
RetMol
100%
0.688
0.769
50K+5K
100%
0.691
0.642
50K+5K
HN-GFN
100%
0.784
0.725
50K+20K
100%
0.733
0.638
50K+20K
I-LS
100%
0.693
0.823
10K+5K
100%
0.704
0.734
20K+5K
InversionGNN
100%
0.768
0.841
10K+5K
100%
0.769
0.773
20K+5K
Properties and Oracles. (1) QED ranging in [0, 1] that provides quantitative assessment of
molecule's drug-likeness. (2) SA evaluates the ease of synthesizing a molecule, and is normalized to
[0, (Gao & Coley 2020). (3) JNK3 is member of the mitogen-activated protein kinase family,
with scores ranging in [0, 1]. (4) GSK333 is an enzyme encoded by the GSK33 3 gene in humans,
and also has a range of (0, 1). We utilize the RDKit package evaluate QED and SA and evaluate GSK333 and JNK3 following Li et al. (2018) and Jin et al. (020b).
Metrics. We use standard metrics in molecular optimization. (1) Novelty (Nov) represents the proportion of generated molecules not in the training set. (2) Top-K Diversity (Div) (Bengio et al. 2021 Fu et al.) (022) of generated molecules defined as the average pairwise Tanimoto distance between the Morgan fingerprints.
(3) Top-K Average Property Score (APS) (Bengio al. 2021 Fu et al. 2022) refers to the average score of the top-100 molecules. (4) Oracle Calls is represented D" which means allocate A Oracle call budget for pretraining and B for optimization.
A
Molecular Optimization Conditioned on Desired Weight (Q1). In this task, goal is to evaluate that InversionGiNN generate molecules conditioned on desired weight. Here we select two properties, GSK39 and JNK3 and the 5 different weight vectors to serve as in- dependent trials. MOEA/D (Zhang & Lij 2007) and NSGA-III (Deb & Jain 2013) are two multi-objective Genetic Al- gorithms that also incorporate weight. We perform Genetic Algorithms over the tent space learned by JTVAE (Jin et al. 2018). In addition, we also report the Non- Uniformity (NU) that evaluates the dis- tance between propertics and weights vec- tor. For MOEA/D and NSGA-III, report
MOGRN PC
HN-GFN InversianGGN
top-100 PMC1 scones
0.6
0.4
0.16
0.51
1.00
1.96
6.31
Weight Aaars/toryy
Figure 4: The distribution of Top-100 JNK3 scores.
8

---

Published as a conference paper at ICLR 2025
the performance of the molecule with the lowest Non-Uniformity MOGFN-PC (Jin et al. 2023 is multi-objective molecular optimization method that scalarizes reward functions. HN-GFN Zhu et al. 2024) is multi-objective drug discovery method based on Bayesian optimization. For HN-GFN, I-LS, MOGFN-PC, and InversionGNN, we calculate the performance of the top-20 molecules in terms of Non-Uniformity per weight. For each weight, we compute the results separately and report the average results across all 5 trials. The results are shown in Table 2 InversionGNN outperforms most baselines by a significant margin. Our proposed InversionGNN achieves better uniformity and higher APS. We find that the diversity of InversionGNN is lower than that of HN-GFN and MOGEN-AL. A reasonable explanation that the molecules produced by InversionGNN concentrate more around the desired weight with lower diversity. It is attributed to InversionGNN's s ability to identify solutions that are more specifically related to the weight vector. We follow HN-GFN (Zhu et al. 2024) and visualize the top-100 00 JNK3 scores of the molecules generated by weight-based methods (InversionGNN, HN-GFN, and MOGFN-PC) conditioned on the 5 weight vectors in Figure
4 Even though all methods increase the weight vector increases, InversionGNN outperforms the existing methods by a large margin.
Multi-OOjective Drug Discov- ery (Q2). In this task, we use a set of weights scan the Pareto front and evaluate the effective- ness of InversionGNN in synthe- sizing diverse molecules. We compare our InversionGNN with following baselines: (1) LigGPT -string-based distribution learn- ing model) (Bagal et al. 2021);
Table 4: Hypervolume in Mujti-Oe Drug Discovery.
HV (t)
Method
GSK33 JNK3
MOGEN-AL
0.567
0.377
HN-GFN
0.59220..042
0.37420.080
DST
0.497
0.353
I-I
0.475_0.029
0.308
InversionGNN
0.763
0.519 =0.088
(2) GCPN (Graph Convolutional
Policy Network) (You et al.) 2018); (3) MoIDQN (Molecule Deep Q-Network) (Zhou et al. 2019);
(4) GA+D (Genetic Algorithm with Discriminator network) (Nigam et al. 2020): (5) MARS
(Markov Molecular Sampling) (Xie et al. 2021); (6) RationaleRI. (Jin al. (020b); (7) ChemBO
(Chemical Bayesian Optimization) (Korovina et al. 2020); (8) BOSS (Bayesian Optimization over
String Space) (Moss et al.) 2020); LSTM (Long short term memory) (Brown et al.)
2019); (10)
Graph-GA (graph level genetic algorithm) (Brown et al. 2019); (1I) DST (Differential Scaffolding
Tree) (Fu al. x022);(12) MOGFN-AL (weight-condittonall GFlowNett) (Jain et al.
2023); (13)
RetMol (Retrieval Based Generation) (Wang et al. [023]; (14) HN-GFN (Zhu et al.
2024). For
I-LS and InversionGNN, we collect all the solutions of all weights and report the final results. The
results are detailed in Table InversionGNN exhibits superior performance compared to the majority of baselines. Diversity and APS common trade-off. Some methods encounter difficulties in
simultancously achieving high diversity scores and APS, due to their limited capacity to explore the chemical space. Despite LigGPT's achievement of high diversity, the notably low APS indicates its inability to effectively handle this trade-off. In contrast, InversionGNN shows superior performance on both metrics. In addition, we follow the recent multi-objective method HN-GFN ZZuu et al., 2024) and report the Hypervolume (HV) of the molecules generated by recent advanced MOMO approaches in Table including HN-GFN, MOGFN-AL and DST. Higher HV indicates broader coverage of the objective space and higher Pareto diversity. The results show InversionGNNN' ability to capture trade-offs among different properties.
5.3 ABLATION STUDY
Optimization Process. We demonstrate the optimization process of InversionGNN in Figure 5 We begin with an initial molecule 2o and a weight vector [1,3]. For clarity, we focus on two properties, JNK3 and GSK38. At each step, we greedily add one substructure and display the corresponding molecular graphs, property scores, and the loss ratio: ratio = LINKa. As substructures are added, the property scores obtained by InversionGNN gradually increase and become more aligned with the weight vector. However, the I-L.S significantly deviates from the weight vector. It demonstrates that InversionGNN is capable of finding Pareto optimal molecules conditioned weight vector.
tøska5
Search Efficiency. To understand the search efficiency of InversionGNN, we search the Pareto front with the weight vectors number of (2, 5, 10, 15, 20). For GNK38+JNK3, we allocate 10K Oracle

---

Published as a conference paper at ICLR 2025
Objective Space
InversionGNN
X
X1
I-LS
Weight Ray
INKE
GSK3
ratio
1.2
16333
Xu
0.1
0.17
1.30
0.4
X
0.49
0.82
3.59
0.84
3.33
0.
[1,1/3] Ray
X2
fuaca
(a) Objective Space
(b) Molecular Graphs
Figure 5: Optimization process of InversionGNN JNK3 and GSK33 with the weight vector [1, (a) As substructures are added, the property scores obtained by InversionGNN increase and become more aligned with the weight vector. (b) Visualization of corresponding molecular graph.
1a
30
Se. 0x7
1em secee EAEl 1MET DET GHOLL
36
24 86
S.
dfPidoence VOcOorS
(a): Conflicting Objectives
(b): Correlated Objectives
(c): Search Efficiency
Figure 6: Pareto front obtained by Inversion GNN on (a) Conflicting Objectives, and (b) Correlated Objectives. (c) Search Efficiency in Multi-Objective Drug Discovery. The number of weights represents the search scope, and the number of Oracle calls grows with the number of weights.
call budget for surrogate Oracle pretraining, and N Nweiihtt 1K Oracle call budget for optimization. For the optimization involving GNK38-NNK+OEEDSSA the pretraining budget is fixed at 20K, and Nweight 1K Oracle call budget for optimization, As illustrated Figure 6 (c), APS increases with growing weight vectors in multi-objective molecular optimization task.
Conflicting and Correlated Objectives. For fair comparison, we follow the synthetic sequence design task in MOGFN (Jain et al. 2023). The task consists of generating strings with the objectives given by occurrences ['C', 'V'," A'] and special token indicate the end of the sequence. The objectives defined by the number of occurrences of a given set of n-grams in the sequence. Therefore, for conflicting objectives setting, we use 3 the Unigrams task ['C'. V", A']. InversionGNN adequately models the trade-off between conflicting objectives as illustrated by the generated Pareto front in Figure 6 (a). For the Bigrams task with correlated objectives ['C'', VA', 'AC'J, Figure 6 (b) demonstrates InversionGNN can simultancously maximize multiple correlated objectives.
set of d n-grams. We consider a vocabulary size 4, with characters
6 CONCLUSION
In this work, we propose novel dual-path InversionGNN for multi-objective molecular optimization, which is effective and sample-efficient the direct prediction path, we incorporate complicated chemical knowledge. In the inversion generation path, decode the acquired knowledge with the gradient w.r.t the molecule structure. To tackle the complicated multi-property chemical knowledge, relax the discrete molecule optimization into locally differentiable Pareto optimization problem. Through extensive experimental evaluations, we have demonstrated the effectiveness and sample efficiency of InversionGNN in multi-objective drug discovery.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work was supported by HKUST(GZ) Cross-campus Collaborative Research Scheme under the Guangdong "I+1+1" Joint Funding Program.
REFERENCES
Maryam Abbasi, Beatriz P Santos, Tiago C Pereira, Raul Sofia, Nelson RC Monteiro, Carlos JV Simöes, Rui MM Brito, Bernardete Ribeiro, José L Oliveira, and Joel Arrais. Designing optimized drug candidates with generative adversarial network. Journal of cheminformatics, 14(1): 40, 2022.
Viraj Bagal, Rishal Aggarwal, PK Vinod, and U Deva Priyakumar. Liggpt: Molecular generation using transformer-decoder model. 2021.
Emmanuel Bengio, Moksh Jain, Maksym Korablyov, Doina Precup, and Yoshua Bengio. Flow network based generative models for non-iterative diverse candidate generation. Advances in Neural Information Processing Systems, 34:27381-27394 2021.
G Richard Bickerton, Gaia V Paolini, Jérémy Besnard, Sorel Muresan, and Andrew L Hopkins.
Quantifying the chemical beauty of drugs. Nature chemistry, 4(2):90-98, 2012.
Stephen Boyd, Stephen P Boyd, and Lieven Vandenberghe. Convex optimization. Cambridge university press, 2004.
Nathan Brown, Marco Fiseato, Marwin HS Segler, and Alain C Vaucher. Guacamol: benchmarking models for de novo molecular design. Journal of chemical information and modeling, 59(3): 1096-1108, 2019.
Binghong Chen, Tianzhe Wang, Chengtao Li, Hanjun Dai, and Le Song. Molecule optimization by explainable evolution. In International Conference on Learning Representation (ICLR), 2021.
Suresh Dara, Swetha Dhamercherla, Surender Singh Jadav, CH Madhu Babu, and Mohamed Jawed Ahsan. Machine learning in drug discovery: review. Artificial Intelligence Review, 55(3): 1947-1999, 2022.
Nicola De Cao and Thomas Kipf. Molgan: An implicit generative model for small molecular graphs arXiv preprint arXiv: 1805. 1973, 2018
Kalyanmoy Deb and Himanshu Jain. An evolutionary many-objective optimization algorithm using reference-point point-based nondominated sorting approach, part i: solving problems with box constraints. IEEE Iransactions evolutionary computation, 18(4):577-601, 2013.
Jean-Antoine Désidéri. Multiple-gradien descent algorithm (mgda) for multiobjective optimization. Comptes Rendus Mathematique, 350(5):313-316 2012.
Jenna C Fromer and Connor W Coley. Computer-aided discovery. Patterns, 4(2), 2023.
optimization in small molecule
Tianfan Fu, Cao Xiao, and Jimeng Sun. Core: Automatic molecule optimization using copy & refine strategy. Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 638-645, 2020.
Tianfan Fu, Cao Xiao, Xinhao Li, Lucas M Glass, and Jimeng Sun. Mimosa: Multi-constraint molecule sampling for molecule optimization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 133,133, 2021.
Tianfan Fu, Wenhao Gao, Cao Xiao, Jacob Yasonik, Connor W Coley, and Jimeng Sun. Differ- entiable scaffolding tree for molecule optimization. In International Conference on Learning Representations, 2022.
Wenhao Gao and Connor W Coley. The synthesizability molecules proposed by generative models. Journal of chemical information and modeling, 60(12):5714-5733, 2020.
11

---

Published as a conference paper at ICLR 2025
Ziqi Gao, Chenran Jiang. Jiawen Zhang. Xiaosen Jiang, Lanqing Li, Peilin Zhao, Huanming Yang, Yong Huang. and Jia Li. Hierarchical graph learning for protein interaction. Nature Communications, 14(1):1093, 2023.
Ziqi Gao, Tao Feng, Jiaxuan You, Chenyi Zi, Yan Zhou, Chen Zhang, and Jia Li. Deep rein- forcement learning for modelling protein complexes. In International Conference Learning Representations, 2024a.
Ziqi Gao, Xiangguo Sun, Zijing Liu, Yu Li, Hong Cheng, and Jia Li. Protein multimer structure prediction via prompt learning. In International Conference on Learning Representations, 2024b.
Ziqi Gao, Zijing Liu, Yu Li, and Jia Li. Towards stable representations for protein interface prediction. Advances in Neural Information Processing Systems, 37:73079-73097, 2025.
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In ICML, pp. 1263- 1272, 2017.
Rafael Gómez-Bombarelli, Jennifer N Wei, David Duvenaud, José Miguel Hernández-Lobatoo Benjamín Sánchez- Lengellng Dennis Sheberla, Jorge Aguilera-Iparragun
Timothy D Hirzel, Ryan P Adams, and Alán Aspuru-Guzik. Guzik. Automatic chemical design using data-driven continuous representation of molecules. ACS central science, 4(2):268-22 276, 2018.
Sai Krishna Gottipati, Boris Sattarov, Sufeng Niu, Yashaswi Pathak, Haoran Wei, Shengchao Liu, Simon Blackburn, Karam Thomas, Connor Coley, Jian Tang, et al. Learning to navigate the synthetically accessible chemical space using reinforcement learning. In International Conference on Machine Learning, pp. 3668-3679. PMLR, 2020.
Ryan-Rhye Griffiths and José Miguel Hernández-Lobatt Constrained bayesian optimization for automatic chemical design using variational autoencoders. Chemical science, 11(2):577-586. 2020.
Gabriel Lima Guimaraes, Benjamin Sanchez-Lengelingg Carlos Outeiral, Pedro Luis Cunha Farias, and Alán Aspuru-Guzik Objective-reinfooreed generative adversarial networks (organ) for sequence generation models. arXiv preprint arXiv: 705. 2017.
Kexin Huang, Tianfan Fu, Wenhao Gao, Yue Zhao, Yusuf Roohani, Jure Leskovec, Connor W Coley, Cao Xiao, Jimeng Sun, and Marinka Zitnik. Therapeutics data commons: Machine learning datasets and tasks for drug discovery and development. Advances in Neural Information Processing Systems, 2021.
Moksh Jain, Sharath Chandra Raparthy, Alex Hernandez-Gareia, Jarrid Rector-Brooks Yoshua Bengio, Santiago Miret, and Emmanuel Bengio. Multi-objective gflownets. In International Conference Machine Learning, pp. 14631-14653 PMLR, 2023.
Jan H Jensen. A graph- genetic algorithm and generative model/monte carlo tree search for the exploration of chemical space. Chemical science, 10(12):3567-3572) 2019.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. In International conference machine learning, pp. 2323-2332. PMLR, 2018.
Wengong Jin, Kevin Yang, Regina Barzilay, and Tommi Jaakkola. Learning multimodal graph-to- graph translation for molecule optimization. In International Conference on Learning Representa- fiony, 2019.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Hierarchical generation of molecular graphs using structural motifs. In International conference machine learning, pp. 4839 4848. PMLR, 2020a.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Multi-objective molecule generation using interpretable substructures. In International conference on machine learning, pp. 1849-4859 PMLR, 2020b.
12

---

Published as a conference paper at ICLR 2025
Mingxuan Ju, Tong Zhao, Qianlong Wen, Wenhao Yu, Neil Shah, Yanfang Ye, and Chuxu Zhang. Multi-task self-supervised graph neural networks enable stronger task generalization. In The Eleventh International Conference Learning Representations, 2022.
Thomas Kipf and Max Welling. Semi- cemi-supervised classification with graph convolutional networks.
arXiv preprinl arXiv: 1609.02907, 2016.
Ksenia Korovina, Sailun Xu, Kirthevasan Kandasamy, Willie Neiswanger, Barnabas Poczos, Jeff Schneider, and Eric Xing. Chembo: Bayesian optimization of small organic molecules with syn- thesizable recommendations. In International Conference on Artificial Intelligence and Statistics, pp. 1993-3403.. PMLR, 2020.
Seul Lee. Jaehyeong Jo, and Sung Ju Hwang. Exploring chemical space with score-based out-of- distribution generation. In International Conference on Machine Learning, pp. 18872-18892. PMLR, 2023.
Yibo Li, Liangren Zhang, and Zhenming Liu. Multi-objective de novo drug design with conditional graph generative model. Journal of cheminformatics, 10:1-22 142, 2018.
Yuhan Li, Peisong Wang, Zhixun Li, Jeffrey Xu Yu, and Jia Li. Zerog: Investigating cross cross-dataset zero-shot transferability in graphs. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1725-1735, 2024.
Yuhan Li, Xinni Zhang, Linhao Luo, Heng Chang, Yuxiang Ren, Irwin King, and Jia Li. G-refer: Graph vetrieval-augmentee large language model for explainable recommendation. arXiv preprint 1rX2::2502.12586, 2025.
Xi Lin, Hui-Ling Zhen, Zhenhua Li, Qing-Fu Zhang, and Sam Kwong. Pareto multi-task learning. Advances in neural information processing systems, 32, 2019.
Meng Liu, Keqiang Yan, Bora Oztekin, and Shuiwang Ji. Graphebm: Molecular graph generation with energy-basee models. In Energy Based Models Workshop-ICLL 2021, 2021.
Qi Liu, Miltiadis Allamanis, Mare Brockschmidt, and Alexander Gaunt. Constrained graph variational autoencoders for molecule design. Advances in neural information processing systems, 31, 2018.
Xiangyang Liu, Tianxiang Sun, Junliang He, Jiawen Wu, Lingling Wu, Xinyu Zhang, Hao Jiang, Zhao Cao, Xuan-Jing Huang, and Xipeng Qiu. Towards efficient nlp: A standard evaluation and strong baseline. In Proceedings of the 2022 Conference of the North American Chapter the Association for Computational Linguistics: Human Language Technologies, pp. 3288-3303, 2022.
Yang Liu, Yu Rong. Zhuoning Guo, Nuo Chen, Tingyang Xu, Fugee Tsung, and Jia Li. Human mobility modeling during the COVID-19 pandemic via deep graph diffusion infomax. In AAAI, pp. 14347-14355, 2023.
Yang Liu, Jiashun Cheng, Haihong Zhao, Tingyang Xu, Peilin Zhao, Fugee Tsung, Jia Li, and Yu Rong. SEGNO: generalizing equivariant graph neural networks with physical inductive biases. In ICLR, 2024.
Pingchuan Ma, Tao Du, and Wojciech Matusik. Efficient continuous pareto exploration multi-task learning. In International Conference Machine Learning, Proceedings of Machine Learning Research, pp. 6522-6531, 2020.
Debabrata Mahapatra and Vaibhav Rajan. Multi-task learning with user preferences: Gradient descent with controlled ascent in pareto optimization. In International Conference on Machine Learning. pp. 6597-6607. PMLR, 2020.
Henry Moss. David Leslie, Daniel Beck, Javier Gonzalez, and Paul Rayson. Boss: Bayesian optimization over string spaces. Advances in neural information processing systems, 33:15476- 15486, 2020.
AkshatKumar Nigam, Pascal Friederich, Mario Krenn, and Alán Auzuku-Guzik. Augmenting genetic algorithms with deep neural networks for exploring the chemical space. ICL. 2020, 2020.
13

---

Published as a conference paper at ICLR 2025
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
Mariya Popova, Olexandr Isayev, and Alexander Tropsha. Deep reinforcement learning for de novo drug design. Science advances, 4(7)xeaap7885, 2018.
Ozan Sener and Vladlen Koltun. Multi-task learning as multi-objective optimization. In Advances in Neural Information Processing Systems, pp. $25-536, 2018.
Chence Shi, Minkai Xu, Zhaocheng Zhu, Weinan Zhang, Ming Zhang. and Jian Tang. Graphaf: flow-based autoregressive model for molecular graph generation. International Conference on Learning Representations, 2020.
Miha Skalic, José Jiménez, Davide Sabbadin, and Gianni De Fabritiis. Shape-based generative modeling for de novo drug design. Journal of chemical information and modeling, 59(3): 1205- 1214, 2019.
Niclas Stähl, Goran Falkman, Alexander Karlsson, Gunnar Mathiason, and Jonas Bostrom. Deep reinforcement learning for multiparameter optimization in de novo drug design. Journal of chemical information and modeling, 59(7):3166-311 2019.
Teague Sterling and John J Irwin. Zine 15-ligand discovery for everyone. Journal of chemical information and modeling,
2237, 2015.
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In ICLR. OpenReview.net, 2018.
Zichao Wang, Weili Nie, Zhuoran Qiao, Chaowei Xiao, Richard Baraniuk, and Anima Anandkumar. Retrieval-based controllable molecule generation. In The Eleventh International Conference on Learning Representations, 2023.
Yutong Xie, Chence Shi, Hao Zhou, Yuwei Yang, Weinan Zhang, Yong Yu, and Lei Li. Mars: Markov molecular sampling for multi-objective drug discovery. International Conference on Learning Representations, 2021.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanic Jegelka. How powerful are graph neural networks? ICLR. OpenReview.net, 2019.
Jiaxuan You, Bowen Liu, Zhitao Ying. Vijay Pande, and Jure Leskovec. Graph convolutional policy network for goal-directed molecular graph generation. Advances in neural information processing systems, 31, 2018.
Mingwang Zhang, Kun Huang, and Yanli Lv. A wide neighborhood arc-search mneerior-poin algorithm for convex quadratic programming with box constraints and linear constraints. Optimization and Engineering, pp. 1-21, 2021.
Qingfu Zhang and Hui Li. Moea/d: A multiobjective evolutionary algorithm based on decomposition. IEEE Transactions on evolutionary computation, 11(6):712-731, 2007.
Richard Zhang and Daniel Golovin. Random hypervolume scalarizations for provable multi-objective black box optimization. In International Conference on Machine Learning, pp. 11096-11105. 2020.
Zhenpeng Zhou, Steven Kearnes, Li Li, Richard N Zare, and Patrick Riley. Optimization of molecules via deep reinforcement learning. Scientific reports, 9(1):1-10, 2019.
Zhipeng Zhou, Liu Liu, Peilin Zhao, and Wei Gong. Pareto deep long-tailed recognition: conflict- averse solution. In The Twelfth International Conference Learning Representations, 2023.
Yiheng Zhu, Jialu Wu, Chaowen Hu, Jiahuan Yan, Tingjun Hou, Jian Wu, et al. Sample-efficien multi-objective molecular optimization with gflownets. Advances Neural Information Processing Systems, 36, 2024.
14

---

Published as a conference paper at ICLR 2025
Eckart Zitzler and Lothar Thiele. Multiobjective evolutionary algorithms: a comparative case study and the strength pareto approach. IEEE transactions on Evolutionary Computation, 3(4):257-271) 1999.
15

---

Published as a conference paper at ICLR 2025
A PARETO OPTIMAL
In this work, we consider m tasks described by f(x) == [f(()]] R" R for any point æ in Solution Space IR", where each fi(x) IR" R, [m]] represents the performance of the i-th task to be maximized. Given an desired target E R", we set a non-negative objective function ((f(x).)) = [4,.....mm]] Rmm - 0m to be a non-negative objective function mapping the Value Space En to the Objective Space 0", where 4 for i [m] is the objective function of the i-th task. Hence, maximizing the performance ((x is equivalent to minimizing the objective function. Consequently, we have 15" - 0f fi(x') S fi())) for two points x' ER"
For any two points æ; x' E R", dominates æ', denoted by L = if and only L" -C RT, where RT == {C E 0744 2 vi [m]]. The partial ordering L = implies 17 1. Vi ml] When æ strictly dominates x', denoted by L > -L it means there is least one for which 1t - 1.t > O. Geometrically, L > C means that L lies in the positive cone pivoted at LT i.c. L [[T) + _T := {L" + L RT)) A point x" is said to be Pareto optimal if x' is not dominated by any other points in RR. Similarly, x" is locally Pareto optimal if x' is not dominated by any other points in the neighborhood of x" ..i.e. N((x) The set of all Pareto optimal solutions is defined as:
P P==
=
x E R RR
{2") [
(8)
where C LL represents is not dominated by other point æ. The set of multi-objective values of the Pareto optimal solutions is called Pareto front:
F F== ( x EP.
In Multi Objectv Optimization, the ideal goal is to identify set of Pareto solutions that cover all the possible preferences among objectives.
B THEORETICAL ANALYSIS
In this section, we discuss the theoretical properties of InversionGNN Algorithm.
B.1 ASSUMPTIONS AND KEY LEMMAS
Assumption B.1 (Molecule Size Bound), The sizes (i.e., number of substructures) of all the scaffold- ing trees generated are upper bounded by N.
We focus on small molecule optimization; the target molecular properties would decrease significantly when the molecule size is too large (Bickerton et al. 2012). e.g., QED. To perform convergence analysis, we initially establish several assumptions to characterize the geometry of the objective landscape.
Definition B.2 (Non-Unfformity) For any point æ E IR", the Non-Uniformity of its objective values L in relation to given weight vector R as:
pr (C(
Σ log
m
i
KL
A
1
(10)
1/m
m
where 2= (f1,...,I,)) and i is the weighted normalization 1
1
The Kullback-Leibler (KL) divergence of & from the uniform distribution characterizes non- m
uniformity. When the objective value fulfills the weight condition, we have μa, (C) 0: otherwise, (r((C)= O. Consequently, we prefer a lower (E). Definition B.3 (Dominant Set). Given æ' in chemical space X' at the f-th iteration, we define Dominant Set Vzge C Ihth that contains all attainable multi-objective values that dominate the L as:
Vz = ( 1 | 63
(11)
16

---

Published as a conference paper at ICLR 2025
Definition B.4 (Uniform Set). Given molecule x' in the chemical space X' at the t-th iteration and a specified weight vector r E R.", we define a Uniform Set M C R that contains all attainable multi-objective values demonstrating enhanced uniformity compared to L as:
Mr (ur(C) S pr(C")}
(12)
Definition B.5 (Admissible Set), Given a molecule æ' in the chemical space A' at the (-th iteration and specified weight vector Rm" define bounded Admissible Set Arc. C R
A = { E O 3 F where L'= = (L/r1,--. .... 1/rm), and j = max (c]r, 1 [m]].
(13)
Clearly, the admissible set contains all the points in O that dominate the L' i.e. Vice C A Moreover, when 14,(2) > 0, it also contains points exhibiting superior uniformity compared L i.c. A'c Mi j Q. Consequently, the admissible set encompasses the desired solution for the subsequent iteration, fulfilling both uniformity and dominating properties.
As illustrated by Mahapatra & Rajan| (020), a descent direction dnd will be oriented towards the weight-specins Pareto front and within confined admissible set. Since we perform K descents in each iteration, we restructure properties for InversionGNN in the molecular optimization problem as follows:
Lemma B.6 (Bounded Objective Space for the Next Iteration). There exists a step size TJo > O, such that for every E [0, nol. InversionGNN employs Te. = Ta' nuuuu to update differentiable scaffolding tree until convergence. Subsequently, greedily sample molecule x++1 from T by adding substructure, if the solution is nonempty. The multi-objective value L++] of the new solution point x t+1 lies in the t-th admissible set:
L Acc
(14)
Following Theorem in EPO (Mahapatra & Rajan 2020), the empirical loss L of scaffolding tree T lies in the 1-th admissible set Alg. Since greedily sample molecule as a from TuK thus we have [++] C.. Therefore, it follows that C++1 AZ... It demonstrates that for a molecule x' at the 1-th iteration, InversionGNN selects a molecule x+1 from x''s neighborhood set A ((="), moving towards improved uniformity and dominating properties. It provides a theoretical guarantee for the quality of the solution.
Corollary B.7 (Convergence of Admissible Set). The sequence of relative maximum values e obtained by descending against the adjusted gradient dnd is monotonic with gt++1 < ft, which means
A C A'et,,
(15)
and the sequence of bounded sets (Aic44) } converges.
Since L A'c we naturally get pt-1 < f², thus we have Ar. C A It demonstrates the monotonicity of ft. Suppose InversionGNN selects molecule #++1 from st's neighborhood set (xx'), where the lowest şt11 is precisely determined, i.e., finding a solution that maximizes ppt+1
Assumption B.8 (Submodularity and Smoothness). Suppose 1;-1, Fr, .z+1 are generated succes- sively by InversionGNN via growing substructure on the scaffolding tree. We assume that the corresponding objective gain (i.e., Ar") satisfies the diminishing returns property:
pt- pt 2 pt pt+1, (submodularity)
(16)
Submodularity plays the role of concavity/convexity in the discrete regime. On the other hand, we specify the curvature ratio of the objective function L by assuming
- + 2 a(f p*) 0 0<a<l-
N <
(curvature)
(17)
The choice of submodularity as an assumption for our analysis was motivated by experimental observations. For instance, in the optimization process, we noticed that the objective values, such as QED, can rapidly increase to a high point with just a few iterations. However, as we added more atoms with InversionGNN, the growth rate began to decrease, i.e. the return is diminishing when the property scores are reaching the upper bound. This trend is observed in many properties and provides insight into our assumption.
17

---

Published as a conference paper at ICLR 2025
B.2 THEOREM
Theorem B.9 (Approximation Guarantee). Under the assumptions stated in Sec. B Given an initial
molecule x. and weight vector v, InversionGNN guaraniees the following approximation when performing T optimization rounds:
E ( (11))""++(())+="""
(18)
where 1-a7 r- is (1/r....., 8" and 10 is the maximum relative objective value (I-a)N'
z = max (1rr i E [m]} of x² and x
Proof. In the following steps of the proof, simplify mathematical notation, we substitute rf for jt. Starting from x", suppose the path to optimum x: with the weight r is
x
x x²²
x x
(19)
where each step, one substructure is added.
For InversionGNN, we run 7' [K, .N] iterations, and the path produced by InversionGNN is
a <xx"") + a² + x² - ..... x²= where 2 k.
(20)
For the optimum x", based on the submodularity Assumption B.8 we have
to (r r²) 2 (ri-1 r) = r r
(21)
==1
From assumption B.1 it follows that
k (r r²))
N (1:0 - r*).
(22)
For the molecular =T found by InversionGNN, based on curvature ratio in Assumption B.8 we have
TT f3 a (f (1 20 17" (+"-
(23)
Then we have
T
T
1 a-ar aT ((r0 - f1))
0o FT Σ (15- (15-1-
r5)
177 - -1
(24)
1
j=1
Since InversionGNN pick up molecule as 1+1 from x''s neighborhood set Nxx') with lowest p(+1 is exactly solved, i.e. jl < r', and go = p.. Thus we have
+1 1
(25)
From Eq. 22 Eq.
and Eq.
25 it follows that
1 1- 1-a7 αt
0 TT >
[1 a.N - -r*).
(n
(26)
Thus have:
1 T
x+ + (1 1- at
p <
(1- a)r)r
(27)
(1 N)N
Finally, it follows that
c (CEO L (yr* + (1 - )))))) rr-
(28)
18

---

Published as a conference paper at ICLR 2025
C THEORETICAL COMPARISON OF COMPUTATIONAL COMPLEXITY
The current multi-objective molecular optimization methods can be categorized into three classes: (1) gradient-based Pareto search (InversionGNNN) (2) multi-objective genetic algorithm, and (3) multi-objective Bayesian optimization, as illustrated in Section We provide a theoretical analysis of their computational bottlenecks during one iteration:
Gradient-based Pareto search: If we perform K gradient descent steps in each iteration, the time complexity is OK (m3n + m²)), where n is the dimension of the gradients and m is the number of objectives, usually > m.
Genetic Algorithm: The computational bottleneck lies in the non-dominated sorting, which has a time complexity of (mpp²) where p is the population size and is the number of objectives.
Bayesian Optimization: The computational bottleneck lies training the Gaussian process, which has time complexity of O(7"), where is the number of training data points.
D IMPLEMENTATION: DETAILS
D.1 MOLECULAR REPRESENTATION
A molecular graph is representation of molecule, consisting of atoms nodes and chemical bonds as edges. Nonetheless, challenges such as chemical validity constraints, ring integrity, and extensive calculations hinder the explicit reconstruction of potential connectivity. To tackle this, Jin et al. (018) introduced a scaffolding tree, a spanning tree that employs nodes as substructures to model a higher-level representation of a molecule.
scaffolding tree, denoted by Ta = (N, A, w), serves as a high-level representation of a molecule X. Each node is a member of the substructure S (also referred to the vocabulary set). Tan consists of three components: (i) the node indicator matrix defined as E {0, 1) K XS| where each row of N is one-hot hot vector indicating the substructure to which the node belongs; (ii) the adjacency matrix denoted by A E {0, 1] 1 x where Aij = when the i-th and the j-th nodes are connected,
1)T E R* signifies that the K nodes are equally weighted. We convert molecules scaffolding tree for training the surrogate Oracle.
and when they are unconnected; (iii) w = [1,... [.,
To modify the scaffolding
Ta. employ its differentiable version, Ta as proposed by Fu et al. (022). The basic scaffolding tree, Te, can be transformed into a tree containing K + Kezpand nodes, denoted by T (Ñ, A, w), through the addition of an expansion node set Vexpand {u, E Vr, where Venpondl = Kezpond = K. crucial to note that T. contains learnable parameters, which can be interpreted as conditional probability. It can be utilized to sample new tree through processes such as node shrinking, replacement, or expansion. Each scaffolding tree corresponds to multiple molecules, substructures can be combined in various ways.
Assemble the Scaffolding Tree into Molecule. We conduct following steps: (a) Ring-atom connection. When connecting atom and ring in molecule, atom can be connected to any possible atoms in the ring. Ring-ring connection. (b) When connecting ring and ring, there are two general ways, (1) one is another is two rings share two atoms and one bond.
bond (single, double, or triple) connect the atoms in the two rings. (2)
D.2 DETAILS OF I-L.S
The baseline method with Linear Scalarization (LS) is summaried in Algorithm 2 compute the non-dominated gradient dnd but instead linearly weights the gradient using weights, i.e., d = Gr. We keep the remaining parts consistent with InversionGNNN.
LS dose not
D.3 WFIGHT VECTOR
In this section, we describe the detailed process for generating a weight vector uniformly distributed in the objective space. We list the algorithm and 4 for 2 and 4 objectives.
19

---

Published as a conference paper at ICLR 2025
Algorithm 2: Linear Scalarization (I-LS)
Input:
Input molecule æ. E X, weight vector r E RR", and step size > 0.
Output: Generated Molecule x". Initialization.
Train surrogate Oracle according to Eq. for f = ......T do
Convert molccule x' to scaffolding tree : for 1: = ........ do
Compute gradients of each property objective w.r.t. Th: G = VL = [91. 191.....9m.. Iml:
Calculate the direction of descent di Gr;
Update the differentiable scaffolding tree using Th+1 T ndds.
end
Sample discrete Tat+1 from continuous Tx and assemble it molecule x'+1;
10
end
Algorithm 3: Generate Weight for Two Objective
Generate a uniformly distributed variable, u, ranging from 0 to
Compute coordinates' angle: u.
Compute Cartesian coordinates: 71 cos 0, r2 sin 0.
D.4 INVERSIONGNN SETUP
Most of the settings follow the DST (Fu et al. 2022). We implemented InversionGNN using Pytorch 1.7.1, Python 3.7.9. Both the size of substructure embedding and hidden size of GCN are d = 100. The depth of GNN is 3. In each generation, we keep C' 10 molecules for the next iteration. The learning rate is 1e-3 in training and inference procedure. We set the iteration T to large enough number and tracked the result. When Oracle calls budget is used up, we stop it. All results in the tables from experiments up 7 50 iterations. For I-L.S, We only replace the objective function in InversionGNN with Linear Scalarization, and other settings are consistent with InversionGNN,
D.5 BASELINES
In this section, we describe the detailed experimental setting for baseline methods. Most of the settings follow the original papers.
LigGPT is a string-based distribution learning model with a Transformer as decoder (Bagal et al. 20.1). we trained for 10 epochs using the Adam optimizer with a learning rate of 6e 4.
GCPN (Graph Convolutional Policy Network) You et al. 2018) leveraged graph convolutionss network and policy gradient optimize the reward function that incorporates target molecular properties and ad adversarial loss. we trained it using Adam optimizer with 1e-3 initial learning rate, and batch size is 32.
MolDQN (Molecule Deep Q-Networks) (Zhou al. 2019) formulate the molecule generation procedure as a Markov Decision Process (MDP) and use Deep Q-Network to solve it. Adam is trained Adam optimizer with le-4 as the initial learning rate, annealed from to 0.01 in piecewise linear way.
GA+D (Genetic Algorithm with Discriminator network) (Nigam al. 2020) uses a deep neural network as a discriminator to enhance exploration in a genetic algorithm and is trained using the Adam optimizer with a learning rate of le 3, / is set 10.
MARS (Xie al. 2021) leverage Markov chain Monte Carlo sampling (MCMC) molecules with an annealing scheme and an adaptive proposal. It is trained using Adam optimizer with 3e-4 initial learning rate.
20

---

Published as a conference paper at ICLR 2025
Algorithm 4: Generate Weight for Four Objective
Generate two uniformly distributed variables, u, v and 2, ranging from 0 to 1.
Compute spherical coordinates' inclination angle and azimuth angle: 0 2 u, Φb arceos v;
arccos
Compute Cartesian coordinates: 71 = sin @ cos sin a, P2 = sin o sin @ sin rg = sin a cos Ø,
74 C0S
RationaleRL, (Jin et al., 2020b) is deep generative model that grows molecule atom-byatom from an initial rationale (subgraph). It is trained using Adam optimizer on both pre-training and fine-tuning with initial learning rates of le-3, 5e-4, respectively. The annealing rate is 0.9.
ChemBO (chemical Bayesian optimization) (Korovina et al. 2020) leverage Bayesian optimization. It also explores the synthesis graph in sample-efficintt way and produces synthesizable candidates. Following the default setting in the original paper, the number of steps of acquisition optimization is set to 20. The initial pool size is set to 20, while the maximal pool size is 1000.
BOSS (Bayesian Optimization over String Space) (Moss al. 2020) builds a Gaussian process surrogate model based Sub-sequence String Kernel, which naturally supports SMILES strings with variable length, and maximizing acquisition function efficiently for spaces with syntactical constraints. The population size is set to 100, the generation (evolution) number is set 100.
DST (Differentiable Scaffolding Tree) (Fu et al. 2022) utilizes learned knowledge network to convert discrete chemical structures to locally differentiable ones. DST enables a gradient- -ased optimization a chemical graph structure by back-propagating.
MOGEN-PC (weight-conditional GFlowNets) (Jain al.) 2023) is a Reward-conditional GFlowNets based on Linear Scalarization. They introduce the Weighted-log that can help in scenarios where all objectives Weighted-Sum can be dominated by single reward.
be optimized simultaneously, and the scalar reward from
HN-GFN (Zhu et al. 2024 is a multi-objective drug discovery method based on Bayesian opti- mization. It leverages the hypernetwork-basee GFlowNets an acquisition function optimizer.
RetMol (Retrieval-Bitt Based Molecular Generation) (Wang et al. 2023) retrieves and fuses the exemplar molecules with the input molecule, which is trained by a new selfsupervised objective that predicts the nearest neighbor of the input molecule.
E ADDITIONAL EXPERIMENTAL RESULTS
E. MOLECULES GENERATED BY INVERSIONGNN
We provide several molecules synthesized via the InversionGNN approach.
(I) Molecules with JNK3 and GSK33 scores. Each score independently represents the respective values for JNK3 and GSK33, see Figure 8
(2) Molecules with highest average QED, normalized-SA. JNK3 and GSK33 scores. These four scores symbolize the values for QED, normalized SA, JNK3, and GSK388 respectively, see Figure 9
E.2 MOLECULES GENERATED LINEAR SCALARIZATION
We exhibit the molecular graphs produced through Linear Scalarization. Alongside these visual representations, their corresponding property scores are included, and the loss ratio is calculated using the formula ratio = LINka This supplementary information further elaborates on the experimental Taskas
results outlined in Section 6.3 of the paper, as illustrated in Figure TO
E.3 MOLECULES GENERATED BY I INVERSIONGNN
The Oracle requires realistic optimization tasks, which can often be time-consuming. To further verify the Oracle efficiency, we explore special setting of molecule optimization where the budget
21

---

Published as a conference paper at ICLR 2025
of Oracle calls is limited to fixed number (2K, 5K, 10K, 20K, 50K) and compare the optimization performance. For GCPN, MolDQN, GA+D, and MARS, the number of learning iterations is determined by the Oracle call budget. To ensure fair comparison with DST, InversionGNN, and I-LS utilize approximately 80% of the budget to label the dataset (i.e., for training the GNN), reserving the remaining budget for de novo design. Specifically, for each budget (2K, 5K, 10K, 20K, and 50K), we allocate 1.5K, 4K, 8K, 16K, and 40K Oracle calls, respectively, for labeling the data used GNN training. Figure illustrates the APS of the top 100 molecules across different Oracle budgets. Notably. InversionGNN shows significant advantage compared to all the baseline methods manited-budger settings.
J
MARS
8085
0.8
GAIO
Chendo
NiSDON
0.4
0.0
2K
5K
10K
20K
50K
The number Oracle calls
Figure 7: Oracle Efficiency Test.
[0.91, 0.84]
[0.92, 0.84]
[0.87, 0.87]
[0.85, 0.94]
Figure 8: Generated molecules by InversionGNN, These four scores symbolize the values for JNK3 and GSK3,3, respectively.
22

---

Published as a conference paper at ICLR 2025
[0.76, 0.7, 0.87]
[0.75, 1, 0.74, 0.84]
[0.75, 1, 0.69, 0.87]
[0.77, 1, 0.68, 0.85]
[0.77, 1, 0.71, 0.82]
[0.71, 1, 0.72, 0.87]
Figure 9: Generated molecules by InversionGNN. These four scores symbolize the values for QED, normalized SA, JNK3, and GSK33, respectively.
X0
X1
JNK3
GSK3Ø
ratio
X
0.1
0.17
1.30
X1
0.69
0.77
1.42
X2
0.8
0.82
1.12
Xa
0.83
0.85
1.15
X3
X2
Figure 10: Generated molecules by I-LS, property scores and loss ratio.
23


Reviews:
[{"review_id": "eYmB2LGU4u", "reviewer": "Reviewer_ToHW", "summary": "The article introduces InversionGNN, a dual-path graph neural network designed for multi-objective molecular optimization in drug discovery, addressing the challenge of balancing conflicting molecular properties. It  combines a direct prediction path, which learns chemical relationships between substructure and properties, with an inversion generation path that uses gradient-based Pareto optimization to generate molecules achieving trade-offs. The authors demonstrate the effectiveness of InversionGNN by empirical analysis on synthetic tasks and multi-objective drug discovery.", "questions": "Please refer to the questions mentioned in the Weaknesses.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- The paper addresses an important problem, and the overall presentation is good, including writing and visualizations.\n\n- The proposed gradient-based Pareto inversion module aligns with the intuition of multi-objective optimization (MOO) and offers strong interpretability.\n\n- The experimental analysis is thorough, validating the model’s extensive exploration of the Pareto front in MOO from multiple perspectives, not limited to the molecular domain. Additionally, the ablation study provides interesting insights, such as showing that InversionGNN's incremental addition of substructures aligns better with weighted vectors.", "weaknesses": "- My main concern is the claimed superiority of InversionGNN in molecular design results. On close examination of Table 1, it appears that DST, the most competitive baseline, performs nearly on par with InversionGNN across various metrics. Since the primary enhancement in InversionGNN is the gradient-based Pareto inversion module, this raises the question of whether the impact of this module is limited and whether the tokenization approach of the scaffolding tree is actually more critical for molecular design.\n\n- The time complexity analysis is insufficient. In practical applications, QP-solving complexity can be significant. While the paper suggests that computing the non-dominating gradient is negligible in theory, I recommend adding a comparison of computation times across methods to provide clearer insights.\n\n- The motivation behind oracle budget selection is not well-explained. What was the rationale for the oracle call budgets chosen for each baseline? The authors should consider controlling for a consistent budget across models to ensure fair supervision signals, which would also facilitate comparative analysis.\n\n- The definition of the neighborhood set is somewhat ambiguous. How is the neighborhood set specifically defined in the context of molecular design?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "o9BHocUrNH", "reviewer": "Reviewer_QPc3", "summary": "The paper presents the InversionGNN framework, which aims to optimize molecular properties for drug discovery using a dual-path GNN approach. It consists of a Direct Prediction Path, where a GNN is trained to predict multiple chemical properties, leveraging learned chemical knowledge to enhance understanding of optimal functional group combinations and an Inversion Generation Path that employs a gradient-based Pareto optimization method to generate molecules that meet specified properties, addressing challenges posed by conflicting or correlated objectives. Key contributions include the integration of property prediction knowledge into molecular optimization processes and a novel relaxation technique that adapts gradient-based Pareto optimization for discrete chemical spaces, facilitating the exploration of diverse Pareto-optimal solutions. Empirical evaluations show that InversionGNN is both effective and sample-efficient, outperforming existing methods in multi-objective drug discovery scenarios by successfully balancing trade-offs among various chemical properties.", "questions": "Why doesn't InversionGNN make an all-encompassing and detailed comparison with HN-GFN, but rather such a detailed comparison with I-LS.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "①The introduction of a dual-path GNN effectively combines property prediction with molecular generation. This approach enhances the integration of learned chemical knowledge into the optimization process, allowing for more informed and effective molecular design.\n\n②The method demonstrates significant sample efficiency in exploring the chemical space. By leveraging gradient-based Pareto optimization, InversionGNN can identify high-quality molecules with fewer oracle calls compared to traditional methods, making it a practical choice for real-world applications where computational resources may be limited.\n\n③The paper includes extensive experimental evaluations across various real-world datasets and multi-objective settings. These experiments provide robust evidence of the method's effectiveness and its ability to generate diverse Pareto-optimal solutions, validating its utility in drug discovery scenarios.\n\n④The theoretical analysis, including Theorem 4.2, offers insights into the convergence properties of InversionGNN. This foundation not only strengthens the claims made about the method's effectiveness but also provides a framework for future research in multi-objective molecular optimization.", "weaknesses": "①The assumptions underlying Theorem 4.2 are not clearly articulated or justified within the text. Possible assumptions that could be relevant include the continuity and differentiability of the objective functions, the boundedness of molecular sizes, and the stability of the optimization process under varying conditions. The absence of a discussion on these assumptions raises concerns about the robustness of the theoretical claims presented. Without a thorough examination, it becomes challenging to assess the practical applicability of the results in real-world scenarios. Furthermore, the theoretical framework predominantly focuses on the specific architecture and objectives of InversionGNN. This narrow focus limits the generalizability of the findings, as it does not consider how the proposed method may perform under different conditions or with various types of optimization problems. Expanding the theoretical discussion to include a broader range of contexts could enhance the credibility and utility of the results.\n\n②The majority of experiments in this paper compare InversionGNN only with I-LS, which is a very simplistic linear scalarization model. Demonstrating that InversionGNN outperforms I-LS does not provide significant insights into its effectiveness or practical advantages. Additionally, many experiments only contrast InversionGNN with older methods, while comparisons with more recent advancements, such as HN-GFN, are limited to just two lightweight experiments. The primary experiments lack comprehensive comparisons with contemporary approaches, which could better contextualize the performance of InversionGNN and underscore its contributions to the field.\n\n③The paper lacks thorough experimental evaluations against SOTA methods in multi-objective optimization. A detailed comparative analysis with current leading techniques would provide a clearer understanding of InversionGNN's strengths and weaknesses. By not including such comparisons, the paper misses an opportunity to establish a more compelling case for the novelty and superiority of InversionGNN in the context of existing literature. Including SOTA comparisons could significantly enhance the paper's contributions and relevance to researchers in the field.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uWr4IzSn9Q", "reviewer": "Reviewer_QRSa", "summary": "This paper introduces InversionGNN, a method for multi-objective molecular optimization. InversionGNN comprises two paths: a direct prediction path and an inversion generation path. In the direct prediction path, a graph neural network (GNN) is trained to incorporate chemical knowledge. In the inversion generation path, the discrete molecular optimization problem is reformulated as a locally differentiable Pareto optimization problem, allowing the model to capture complex multi-property relationships. Experimental results demonstrate the effectiveness of InversionGNN on multi-objective molecular optimization benchmarks.", "questions": "See Weaknesses Section", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- Comprehensive Literature Review: The authors present a thorough review of the relevant literature, situating their work well within the existing field.\n- The paper is well-organized.\n- Extensive Experiments: The authors conduct a wide range of experiments, supporting the robustness of their results.", "weaknesses": "- Clarity of Main Concepts: The core ideas are difficult to grasp, especially for those without a background in multi-objective optimization. I recommend adding more intuitive explanations at the start of Section 4 to clarify the fundamental concepts.\nMissing Preliminaries on Differentiable Scaffolding Trees: The paper would benefit from an additional preliminary section on differentiable scaffolding trees to provide context for these methods.\n- Unclear Terminology and Contribution: Key terms like “inversion generation path” and “dual-path” lack sufficient explanation. What exactly does \"inversion\" mean in this context, and how does it contribute to the model's novelty? Please clarify the role and purpose of the \"dual-path\" structure.\n- Differentiation from Vanilla GCN: The distinction between the \"Direct Prediction Path\" and a standard Graph Convolutional Network (GCN) is unclear. Section 4.2 could be revised to explicitly highlight how this path diverges from a traditional GCN and its specific contributions to the model.\n- Role of Relaxation Techniques: It is not clear how the relaxation techniques, which reformulate the discrete molecule Pareto optimization problem into a locally differentiable one, aid in capturing trade-offs among conflicting or correlated properties. Additional explanation on how this helps to balance multiple properties would improve understanding.\n\nWhile this paper shows promise, I am not an expert in multi-objective optimization, and some core concepts remain unclear to me. I will finalize my score after reviewing feedback from another reviewer with more expertise in this area.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PxQb0b7PkQ", "reviewer": "Reviewer_GVk8", "summary": "The authors propose a molecular multi-objective optimization algorithm, inversionGNN, which proposes direct prediction paths and inversion generation paths to solve conflicting attribute optimization problems in chemical space. However, some issues need to be further addressed.", "questions": "1. In the introduction, the authors suggest that substructures have an effect on molecular properties, so does inversionGNN take into account the effect of substructures on molecules?\n2. in a multi-objective optimization task, how are the weights for multiple objectives balanced, or is the direct prediction path just viewed as a multi-classification task to bootstrap the Pareto front search?\n3. what is the specific significance of the tasks designed by the authors in the ablation experiments, while the reader should be more interested in seeing how the direct prediction paths and inversion generation paths proposed by the authors affect the method.\n4. Could the authors elaborate on how the surrogate oracle (used in Eq. 3) is trained and validated? Specifically, what types of data and target objectives are used for this model, and how is its accuracy assessed across diverse molecules?\n5. The motivation for the thesis needs to be clear and is currently presented as too generalized. The flowchart of the methodology needs to be further improved so that the reader can understand the whole process.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The method proposed by the authors, inversionGNN, a multi-objective molecular optimization method, is performed in a discrete space, which is different from the traditional latent space-based optimization methods and is methodologically innovative.", "weaknesses": "The authors' motivation for proposing the algorithm in their article by suggesting a conflict in the molecular properties is not specific enough. Also, in the article, it is suggested that the viewpoint of existing methods does not take into account the effect of substructure on molecular properties. Still, in the method introduction and experimental setup of inversionGNN, the authors do not address the effect of substructure on molecular properties either. At the same time, I don't quite understand what the significance of these tasks above the ablation experimental setup is. The authors should highlight the ablation experimental setup, what would be the impact if the current direct prediction path is not used, and what would be the extent of the impact on the current method if another proxy model is used.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)