Paper: Trading-off Multiple Properties for Molecular Optimization (ICLR-2024)

Reviewer comment (Novelty):
The manuscript's primary shortcoming lies in its novelty. As mentioned earlier, the foundational concept was presented in 1987 [1], and an enabling step for translating the molecular design task to gradient-based optimization was introduced in [2].

Author response:
**1. [W1] The multi-objective optimization problem is not trivial.**

**Firstly, we believe it is not appropriate to dismiss the research field of multi-objective optimization as trivial.** General multi-objective optimization is currently an important research direction in machine learning, with numerous efforts dedicated to solving Pareto optimization problems. And Linear Scalarization is considered a weak baseline in multi-objective optimization. Moreover, the Pareto optimization problem remains unresolved in molecular optimization.

**Secondly, from SOMO to MOMO is the research trend in drug design.** Previous works usually focused on single properties. However, it is until this year that we have witnessed the emergence of tasks involving multi-objective molecular optimization in machine learning conferences (ICML 2023 [1] and NeurIPS 2023 [2]). Futhermore, they adopt Linear Scalarization or Bayesian optimization to tackel this problem, which is the baseline of our method.

**Thirdly, Thirdly, thanks for reviewer xqpc and reviewer dF44 to recognize the contribution of our work.** Reviewer xqpc finds our work novel and interesting, and Reviewer dF44 considers it intriguing and a practically valuable contribution.

Reviewer comment (Novelty):
Despite references to many SOMO methods favoring linear scalarization for multi-objectivity, I think the reason they chose linear scalarization is that their emphasis is on the algorithm development while an extension to MOMO is relatively trivial. It might be more apt to redirect this focus towards application.

Author response:
**2. [W2 & Q1] About Linear Scalarization Critique**

It has been mathematically proved by [3]  in Chapter 4.7.4 and explaining why linear scalarization fails to capture preference-conditioned solutions.	Our experiments also confirm this phenomenon. We provide the code for you to reproduce our Figure 1 and Figure 3 within few minutes. To avoid any other factors, we did not employ any tricks or set random seeds. The drawbacks of Linear Scalarization are quite evident as we mentioned in our paper. Please refer to the provided code for further details: https://github.com/ICLR2024/PCI

**Our code can also compute the hypervolume:**

**PCI: 0.33        I-LS: 0.065**

You stated ‘’Contrarily, based on my experience, linear scalarization can navigate along the Pareto front toward both extremes, and with weight tuning and keeping all points visited during the processes, results comparable to PCI's can be achieved.‘’ Could you please provide some evidence to support your viewpoint? Or could you please point out how to improve our code to achieve 'Linear scalarization's results are comparable to PCI'.

Reviewer comment (Theory):
The authors critique linear scalarization methods, arguing that they can't comprehensively traverse the Pareto solution merely by weight adjustments, as depicted in Figure 3(b). Contrarily, based on my experience, linear scalarization can navigate along the Pareto front toward both extremes, and with weight tuning and keeping all points visited during the processes, results comparable to PCI's can be achieved.

Author response:
**2. [W2 & Q1] About Linear Scalarization Critique**

It has been mathematically proved by [3]  in Chapter 4.7.4 and explaining why linear scalarization fails to capture preference-conditioned solutions.	Our experiments also confirm this phenomenon. We provide the code for you to reproduce our Figure 1 and Figure 3 within few minutes. To avoid any other factors, we did not employ any tricks or set random seeds. The drawbacks of Linear Scalarization are quite evident as we mentioned in our paper. Please refer to the provided code for further details: https://github.com/ICLR2024/PCI

**Our code can also compute the hypervolume:**

**PCI: 0.33        I-LS: 0.065**

You stated ‘’Contrarily, based on my experience, linear scalarization can navigate along the Pareto front toward both extremes, and with weight tuning and keeping all points visited during the processes, results comparable to PCI's can be achieved.‘’ Could you please provide some evidence to support your viewpoint? Or could you please point out how to improve our code to achieve 'Linear scalarization's results are comparable to PCI'.

Reviewer comment (Evaluation):
It would be beneficial for the authors to record all points traversed during optimization and employ hypervolume as a performance metric.

Author response:
**4. [W4] About Top-K molecules.**

Top-k is a commonly used metric in molecular optimization, and we are simply following the existing works[1][4][5], as we have already mentioned in our paper. Generally, in molecular optimization, people usually evaluate the top points, which is more normal in optimization (since one generally cares about the optimum reached, not how one gets there). For instance, if we need to add 5 substructures to achieve a high property score, why do we need to consider the molecules (3 substructures) generated during the intermediate steps as part of the final result and include them in the statistics? In addition, we set K to 100 instead of the commonly used values of 3 or 10 in existing works, which makes our result more convincing.

Reviewer comment (Novelty):
DST represents only a handful of cases that render the molecular design problem differentiable, enabling the application of gradient-based methods. The value of PCI is limited if it can not be adapted for broader molecular design techniques.

Author response:
**3. [W3] Generalizability of the Method.**

If the problem is non-differentiable, we can train a differentiable surrogate model and apply PCI.

Reviewer comment (Evaluation):
The authors recurrently state that MOMO's aim is to scrutinize the Pareto front, yet they predominantly highlight a single value representing an average of the top-100 molecules. A comparison of the Pareto fronts analyzed by PCI and LS would bolster their claims.

Author response:
**4. [W4] About Top-K molecules.**

Top-k is a commonly used metric in molecular optimization, and we are simply following the existing works[1][4][5], as we have already mentioned in our paper. Generally, in molecular optimization, people usually evaluate the top points, which is more normal in optimization (since one generally cares about the optimum reached, not how one gets there). For instance, if we need to add 5 substructures to achieve a high property score, why do we need to consider the molecules (3 substructures) generated during the intermediate steps as part of the final result and include them in the statistics? In addition, we set K to 100 instead of the commonly used values of 3 or 10 in existing works, which makes our result more convincing.

Reviewer comment (Experiments):
In Table 2, what is the rationale behind allocating 5 oracle calls for Graph GA to train surrogates?

Author response:
**5. [Q2] Oracle calls for Graph GA.**

We apology for the typo. We have already corrected it to 0+25k.

Reviewer comment (Evaluation):
The procedure for the non-dominating descent direction by linear programming, which is the core of the proposed algorithm, Eq.(9), is designed based on the non-uniformity criterion, eq.(3). But this metric would not be assumed in Linear Scalarization. While it seems natural that the score for eq.(3) is better than LS if we assumed this scoring, the practical significance of this is unclear.

Author response:
**1. [Q1 & Q2] About Non-Uniformity.**

Before we discuss the Non-Uniformity, I would like to share an example I provided to Reviewer xqpc, which effectively demonstrates the importance of Non-Uniformity in MOMO and the advantage of PCI.

For instance, let's consider two properties (A, B) with a solution space of {(1, 1), (1, 0.5), (1,0.33), (0, 0)}. We aim to optimize these attributes using SOMO+scalarization, starting from the initial point (0,0). **Assuming that a very high value of property B has a negative impact, we need to maintain it at a moderate level.** Thus we assign a preference ratio of 2:1 and the ideal optimal solution is (1,0.5), and define the unified attribute as P = A + 0.5B. However, for the solution (1,1), P = 1 + 0.5 = 1.5. For the solution (1,0.5), P = 1 + 0.5 * 0.5 = 1.25. Clearly, 1.5 > 1.25, and we end up with the solution (1,1) instead of the desired (1,0.5). Even if we change the weight to 3:1, we still obtain the same solution (1,1) instead of desired (1,0.33). However, PCI can find the desired solution (1, 0.5) and (1,0.33) since we take preference condition and decent direction in to consideration by solving Eq. 9. We have demonstrated this through both theoretical analysis and experimental validation. **Due to the limitations of the response's length, we strongly recommend reviewer to refer to a blog [1] posted by DeepMind in 2021 which illustrates why linear scalarization often results in biased solutions and fails to adjust scalar weights in an easy-to-understand manner.**

**1.1 [Q2] Why is Non-Uniformity?**

Non-uniformity is a metric used to measure the similarity between the ratio of properties and  preference. Therefore, any metric used to measure the similarity between two vectors can be employed, such as cosine similarity. We use non-uniformity because we follow the general multi-objective optimization. 

**1.2 [Q1] Non-Uniformity is appropriate for MOMO.**

In the aforementioned example, if the preference is 2:1, scalarization fails to find the desired solution (1,0.5) , while PCI can. It is evident that (1,0.5) outperforms (1,1) in terms of Non-Uniformity. **In practical drug design, it is not always desirable for certain properties to be maximized or minimized. Instead, we may need to control them within a certain range with preference. However, scalarization fails to achieve this level of control.** Such requirements are quite common in drug design and remain unresolved.

Reviewer comment (Novelty):
Although this method is proposed as a framework, it heavily relies on the "differentiable scaffolding tree" method (Fu et al, 2022). It's unclear if this specific method is essential, or if other surrogate methods could also be suitable. Moreover, especially the procedure in the inversion step, "sample the discrete scaffolding tree", is not clearly described. A more detailed comparison with the experimental results of I-LS and PCI would be appreciated for readers to interpret the comparisons between I-LS and PCI.

Author response:
**2. [Q3] About Discretization.**

Our sampling process is consistent with the differentiable scaffolding tree  (DST), as detailed in [1]. After discretization, like other molecular optimization methods, we employ oracle to test the discretized molecules and select those that meet the conditions (maximum relative objective value $\check{\lambda}^t$ decreased) while discarding the ones that do not. As a result, the final set of molecules will also satisfy the conditions.  **It is worth noting that in our theoretical analysis, we directly analyze the discretized molecules instead of the continuous form.**

**3. [Q4] Can the surrogate part (DST) be replaced with other methods?**

**The surrogate part can be replaced, but the surrogate part should satisfy some conditions.**  PCI requires the surrogate part to be differentiable, which is why current Combinatorial Optimization (CO) algorithms are not suitable. Another category, Constrained Generative Models (CGMs), conducts optimization in a continuous latent space that is differentiable. However, achieving an ideal smooth and discriminative latent space, typically required by CGMs, has proven to be challenging in practice and CGMs do not perform well in molecular optimization [3][4], which is also shown in our Table. 2. **Therefore, currently the most suitable surrogate part is DST, as it allows us to directly modify the molecular structure and is differentiable.** The current backbones are all developed based on SOMO, but developing a backbone that is more suitable for MOMO could be an interesting future work.

Reviewer comment (Novelty):
This study seems to specialize insights from established multi-objective optimization research, specifically De ́side ́ri et al (2012) and Mahapatra & Rajan (2020), to multi-objective optimization in the discrete space of molecular optimization. If we assume that this work utilizes the existing "differential scaffolding tree" (Fu et al, 2022) and its sampling function as a differentiable surrogate for this purpose, its technical contribution would appears somewhat incremental.

Author response:
**3. [Q4] Can the surrogate part (DST) be replaced with other methods?**

**The surrogate part can be replaced, but the surrogate part should satisfy some conditions.**  PCI requires the surrogate part to be differentiable, which is why current Combinatorial Optimization (CO) algorithms are not suitable. Another category, Constrained Generative Models (CGMs), conducts optimization in a continuous latent space that is differentiable. However, achieving an ideal smooth and discriminative latent space, typically required by CGMs, has proven to be challenging in practice and CGMs do not perform well in molecular optimization [3][4], which is also shown in our Table. 2. **Therefore, currently the most suitable surrogate part is DST, as it allows us to directly modify the molecular structure and is differentiable.** The current backbones are all developed based on SOMO, but developing a backbone that is more suitable for MOMO could be an interesting future work.

**4. [Q5] Is the standpoint of this study to specialize insights from general multi-objective optimization? Or does it also provide new insights into general multi-objective optimization?**

Yes, the disadvantage of linear scalarization was first argued in general multi-objective optimization and this problem is still unsolved in molecular optimization task.  And PCI also provide new insights into general multi-objective optimization. Currently, general multi-objective optimization focuses on continuous spaces, especially the continuous parameters of deep models. It is still unclear how to perform multi-objective optimization in discrete spaces instead of using  black-box search methods such as Genetic Algorithms and Bayesian Optimization. Technically, we have transformed the discrete problem into a locally differentiable problem and provided convergence guarantees. Furthermore, we believe that drawing insights from general multi-objective optimization and addressing an still unsolved and important problem in molecular optimization is both meaningful and novel.

Reviewer comment (Evaluation):
If the alignment to the given reference is evaluated using methods other than eq.(3), can we say that PCI is better than I-LS? Is this criteria of eq.(3) appropriate for this comparison?

Author response:
**1. [Q1 & Q2] About Non-Uniformity.**

Before we discuss the Non-Uniformity, I would like to share an example I provided to Reviewer xqpc, which effectively demonstrates the importance of Non-Uniformity in MOMO and the advantage of PCI.

For instance, let's consider two properties (A, B) with a solution space of {(1, 1), (1, 0.5), (1,0.33), (0, 0)}. We aim to optimize these attributes using SOMO+scalarization, starting from the initial point (0,0). **Assuming that a very high value of property B has a negative impact, we need to maintain it at a moderate level.** Thus we assign a preference ratio of 2:1 and the ideal optimal solution is (1,0.5), and define the unified attribute as P = A + 0.5B. However, for the solution (1,1), P = 1 + 0.5 = 1.5. For the solution (1,0.5), P = 1 + 0.5 * 0.5 = 1.25. Clearly, 1.5 > 1.25, and we end up with the solution (1,1) instead of the desired (1,0.5). Even if we change the weight to 3:1, we still obtain the same solution (1,1) instead of desired (1,0.33). However, PCI can find the desired solution (1, 0.5) and (1,0.33) since we take preference condition and decent direction in to consideration by solving Eq. 9. We have demonstrated this through both theoretical analysis and experimental validation. **Due to the limitations of the response's length, we strongly recommend reviewer to refer to a blog [1] posted by DeepMind in 2021 which illustrates why linear scalarization often results in biased solutions and fails to adjust scalar weights in an easy-to-understand manner.**

**1.1 [Q2] Why is Non-Uniformity?**

Non-uniformity is a metric used to measure the similarity between the ratio of properties and  preference. Therefore, any metric used to measure the similarity between two vectors can be employed, such as cosine similarity. We use non-uniformity because we follow the general multi-objective optimization. 

**1.2 [Q1] Non-Uniformity is appropriate for MOMO.**

In the aforementioned example, if the preference is 2:1, scalarization fails to find the desired solution (1,0.5) , while PCI can. It is evident that (1,0.5) outperforms (1,1) in terms of Non-Uniformity. **In practical drug design, it is not always desirable for certain properties to be maximized or minimized. Instead, we may need to control them within a certain range with preference. However, scalarization fails to achieve this level of control.** Such requirements are quite common in drug design and remain unresolved.

Reviewer comment (Evaluation):
Given that the paper's focus is on molecular optimization with a given preference, it seems natural to have multiple metrics. Are there no metrics other than the non-uniformity in eq.(3)?

Author response:
**1.1 [Q2] Why is Non-Uniformity?**

Non-uniformity is a metric used to measure the similarity between the ratio of properties and  preference. Therefore, any metric used to measure the similarity between two vectors can be employed, such as cosine similarity. We use non-uniformity because we follow the general multi-objective optimization.

Reviewer comment (Reproducibility):
What is the procedure for the "sample the discrete scaffolding tree" part? This process involves discretization, but can its effects be ignored?

Author response:
**2. [Q3] About Discretization.**

Our sampling process is consistent with the differentiable scaffolding tree  (DST), as detailed in [1]. After discretization, like other molecular optimization methods, we employ oracle to test the discretized molecules and select those that meet the conditions (maximum relative objective value $\check{\lambda}^t$ decreased) while discarding the ones that do not. As a result, the final set of molecules will also satisfy the conditions.  **It is worth noting that in our theoretical analysis, we directly analyze the discretized molecules instead of the continuous form.**

Reviewer comment (Experiments):
This study heavily relies on the existing method of the differentiable scaffolding tree (Fu et al, 2022). Is it essential to use this method, or can the surrogate part be replaced with other methods? If it can be replaced, why was only the differentiable scaffolding tree used?

Author response:
**3. [Q4] Can the surrogate part (DST) be replaced with other methods?**

**The surrogate part can be replaced, but the surrogate part should satisfy some conditions.**  PCI requires the surrogate part to be differentiable, which is why current Combinatorial Optimization (CO) algorithms are not suitable. Another category, Constrained Generative Models (CGMs), conducts optimization in a continuous latent space that is differentiable. However, achieving an ideal smooth and discriminative latent space, typically required by CGMs, has proven to be challenging in practice and CGMs do not perform well in molecular optimization [3][4], which is also shown in our Table. 2. **Therefore, currently the most suitable surrogate part is DST, as it allows us to directly modify the molecular structure and is differentiable.** The current backbones are all developed based on SOMO, but developing a backbone that is more suitable for MOMO could be an interesting future work.

Reviewer comment (Novelty):
Is the standpoint of this study to specialize insights from general multi-objective optimization, namely De ́side ́ri et al (2012) and Mahapatra & Rajan (2020), for multi-objective optimization in the discrete space of molecular optimization? Or does it also provide new insights into general multi-objective optimization?

Author response:
**4. [Q5] Is the standpoint of this study to specialize insights from general multi-objective optimization? Or does it also provide new insights into general multi-objective optimization?**

Yes, the disadvantage of linear scalarization was first argued in general multi-objective optimization and this problem is still unsolved in molecular optimization task.  And PCI also provide new insights into general multi-objective optimization. Currently, general multi-objective optimization focuses on continuous spaces, especially the continuous parameters of deep models. It is still unclear how to perform multi-objective optimization in discrete spaces instead of using  black-box search methods such as Genetic Algorithms and Bayesian Optimization. Technically, we have transformed the discrete problem into a locally differentiable problem and provided convergence guarantees. Furthermore, we believe that drawing insights from general multi-objective optimization and addressing an still unsolved and important problem in molecular optimization is both meaningful and novel.

Reviewer comment (Novelty):
It is unclear whether PCI will provide any clear advantage over other SOMO schemes applied to MOMO tasks via linear scalarization - *IF* one has a specific preference condition to impose on the multiple properties to be optimized. In such a case, would there be any advantage for PCI compared to MOMO method via SOMO+scalarization?

Author response:
&emsp;**(1) [W1] A specific preference:** If the preference is 2:1, SOMO+scalarization often result in biased solutions (1,1). On the other hand, PCI can identify (1,0.5) since we take preference condition and decent direction in to consideration by solving Eq. 9. We have demonstrated this through both theoretical analysis and experimental validation.

Reviewer comment (Novelty):
Similarly, if one has not a single but still a "small" number of preference conditions the optimized molecules need to meet, what would be the advantage of using PCI compared to repeating MOMO via SOMO+scalarization for each of the preferences?

Author response:
&emsp;**(2) [W2] A "small" number of preference:** For example, we use preference 2:1 and 3:1. PCI can obtain corresponding Pareto optimal solutions (1,0.5) and (1,0.33) based on different preferences. However, SOMO+scalarization tends to generate biased solutions, and using different preferences may still result in the same solution (1,1).

Reviewer comment (Novelty):
However, when one desires to explore the overall Pareto front for a variety of preferences, it seems that the proposed PCI scheme may begin to provide distinct advantages over MOMO via SOMO+scalarization, since the pre trained surrogate oracle can be put to good use for a large number of different preferences - despite the initial cost (e.g., in terms of oracle calls) of training the surrogate. It would be very helpful if the authors could discuss when PCI would have clear advantages over other simple extensions of SOMO methods.

Author response:
&emsp;**(3) [W3] A variety of preferences:** In the example above, despite using a variety of preferences, SOMO+scalarization consistently misses the points (1,0.5) and (1,0.33). This corresponds to Fig. 3(b), where there is still a certain unexplored space on the Pareto front. However, PCI, being capable of generating preference-conditioned solutions, can approximately cover the overall Pareto front.

Reviewer comment (Theory):
In Fig. 1(b), it is not very intuitive why linear scalarization would result in the optimization trajectories shown in the figure. Please provide a detailed explanation of the LS optimization scheme that is illustrated in Fig. 1(b) and provide some insights as to why the scheme would prefer Pareto optimal solutions located near the end of the allowable preference regions. This also applies to the results shown in Fig. 3(b).

Author response:
&emsp;**1.1 [W4] An intuitive understanding of the disadvantage of Linear Scalarization.**

We provide an example to illustrate the limitations of Linear Scalarization, which also serves as the inspiration behind our work. For instance, let's consider two properties (A, B) with a solution space of {(1, 1), (1, 0.5), (1,0.33), (0, 0)}. We aim to optimize these attributes using SOMO+scalarization, starting from the initial point (0,0). **Assuming that a very high value of property B has a negative impact, we need to maintain it at a moderate level.** Thus we assign a preference ratio of 2:1 and the ideal optimal solution is (1,0.5), and define the unified attribute as P = A + 0.5B. However, for the solution (1,1), P = 1 + 0.5 = 1.5. For the solution (1,0.5), P = 1 + 0.5 * 0.5 = 1.25. Clearly, 1.5 > 1.25, and we end up with the solution (1,1) instead of the desired (1,0.5). Even if we change the weight to 3:1, we still obtain the same solution (1,1) instead of the desired (1,0.33). 

The example intuitively illustrates why SOMO+scalarization fails to achieve the desired preference-based solutions, as shown in Fig. 1(b). Additionally, even when we change the preference, the obtained solutions remain unchanged and located near the end of the preference regions, as shown in Fig. 3(b). **Such requirements are quite common in drug design and remain unresolved in current scalarization-based molecular optimization methods. Due to the limitations of the response's length, we strongly recommend reviewer to refer to a blog [1] posted by DeepMind in 2021 which  illustrates why linear scalarization often results in biased solutions and fails to adjust scalar weights in an easy-to-understand manner.**

Mathematically, A well-known literature [2] on optimization offers a formal mathematical analysis in Chapter 4.7.4, explaining why linear scalarization fails to capture preference-conditioned solutions.

Reviewer comment (Writing):
I-LS is defined as a scheme that uses "the same inversion framework as PCI but adopts linear scalarization". This is somewhat ambiguous, and it would be helpful if the authors could refer to the PCI diagram in Fig. 2 and clearly explain which part is changed and how.

Author response:
**2. [W5] The Details About I-LS.**
 In Fig. 2, I-LS dose not compute the non-dominated gradient $d_{nd}$ but instead linearly weights the gradient using preferences, i.e.,  $d=G*\lambda$. We keep the remaining parts consistent with PCI. We have included the algorithm for I-LS in Section C.3 of the appendix, highlighted in blue.

Reviewer comment (Novelty):
I-LS seems to perform surprisingly well in the evaluations (e.g., Table 1 and Table 2), where PCI doesn't necessarily outperform I-LS by a significant margin for all performance metrics. This again raises questions about the fundamental advantages of PCI over simpler scalarization-based approaches (e.g., comments #1-#3 above).

Author response:
**3. [W6] About experimental results.**
To ensure fairness of comparison, both PCI and I-LS employ DST as the backbone network. Since DST itself performs exceptionally well, I-LS also outperforms many baselines. However, we should focus on the relative improvement of PCI and I-LS over DST, which indicates that PCI outperforms I-LS by a significant margin across all performance metrics. The relative improvement is shown in the table below.

&emsp;&emsp;&emsp;GSK3$\beta$+JNK3  &emsp;GSK3$\beta$+JNK3+QED+SA
| | Nov | Div | APS | Nov | Div | APS | 
| :----: | :----: | :----: | :----: | :----: | :----: | :----: |
| I-LS | 0% | -5.7% | -0.4% | 0%| -5.1% |-1.8%|
| PCI | 0%| 1.8% | 1.4% | 0% | 1.4% | 2.1%|

Reviewer comment (Writing):
Some part of the algorithm is not described clearly. See **Questions**.

Author response:
**1.  [W1 & Q1] How to assemble the scaffolding tree into a molecule?**

 We follow JTVAE[1], a classic work that transforms molecular graphs into substructure-based tree structures, which has been widely used in drug design. The process is as follows: (a) Ring-atom connection. When connecting atom and ring in a molecule, an atom can be connected to any possible atoms in the ring. Ring-ring connection. (b) When connecting ring and ring, there are two general ways, (1) one is to use a bond (single, double, or triple) to connect the atoms in the two rings. (2) another is two rings share two atoms and one bond.

Reviewer comment (Theory):
The theoretical guarantee seems weak. It only shows after $T$ optimization rounds, the maximum distance between the molecule obtained and the optimal molecule is bounded.

Author response:
**3. [W2] About convergence analysis.** 

There may be some misunderstandings here. When we talk about convergence, the goal is to discuss the number of iterations required to ensure that the distance between the current point $x_{T}$ and the minimum value point $x^*$ is guaranteed to be smaller enough. In our work, we provide this guarantee as you state ‘
After T optimization rounds, the maximum distance between the molecule obtained and the optimal molecule is bounded’. This definition of convergence is widely used, including many works about optimization in machine learning. For more detailed information, please refer to https://en.wikipedia.org/wiki/Rate_of_convergence.

Reviewer comment (Reproducibility):
In Algorithm 1, how to assemble the scaffolding tree into a molecule? Can you provide some description or references?

Author response:
**1.  [W1 & Q1] How to assemble the scaffolding tree into a molecule?**

 We follow JTVAE[1], a classic work that transforms molecular graphs into substructure-based tree structures, which has been widely used in drug design. The process is as follows: (a) Ring-atom connection. When connecting atom and ring in a molecule, an atom can be connected to any possible atoms in the ring. Ring-ring connection. (b) When connecting ring and ring, there are two general ways, (1) one is to use a bond (single, double, or triple) to connect the atoms in the two rings. (2) another is two rings share two atoms and one bond.

Reviewer comment (Theory):
What is the complexity of solving the linear programming problem Eq. (9)? And what is the total complexity of the algorithm? How does it compare with the complexity of the baseline algorithms?

Author response:
**2. [Q2] Time Complexity.** 

We have provided an analysis of the time complexity in Section B of  appendix. The computation of $C = G^T G$ has runtime $O(m^2 n)$, where $n$ is the dimension  of the gradients. With the current best LP solver [2], our LP (9), that has $m$ variables and at most $2m+1$ constraints, has a runtime of $O^*(m^{2.38})$. Since in deep networks, usually $n \gg  m$, PCI does not significantly increase the computational cost of backpropagation gradient calculation. IIn molecular optimization, the time spent on oracle calls typically dominates the algorithm's complexity.  PCI has the comparable oracle calls to that of DST, it requires $O(TM)$ oracle calls, where $T$ is the number of iterations (Alg 1). $M$ is the number of generated molecules, we have $M \leq KJ$, $K$ is the number of nodes in the scaffolding tree, for small molecule, K is very small. $J$ is the number of enumerated candidates in each node. Furthermore, the change in wall clock time when Pareto optimization is incorporated to baseline method to a initial molecule: $28s \to 63s$.