Paper: DisCo-DSO: Coupling Discrete and Continuous Optimization for Efficient Generative Design in Hybrid Spaces (ICLR-2024)

Reviewer comment (Theory):
I'm having difficulty understanding why an autoregressive policy structure is necessary for generating solutions to optimization problems involving a mix of continuous and discrete variables. While sequential structures are commonly used in reinforcement learning for combinatorial optimization problems to effectively learn policies for arbitrary problems, this study appears to be focusing on optimizing a specific given problem. In such a case, wouldn't it be more efficient to explore the entire solution space rather than constructing solutions sequentially?

Author response:
We would like to emphasize that DisCo-DSO is explicitly tailored for generative optimization scenarios where the solution is constructed sequentially. 
In this context, the sequential construction of the solution arises due to the following factors: 
1. The length of the sequence is variable.
2. The order of the sequence is not predetermined but is determined by the model.
3. There exist prefix-dependent positional constraints (or priors) dictated by the specific problem at hand. For instance, in symbolic regression, the model should not be allowed to add a trigonometric operator if it has already done so in the past to avoid nested trigonometric operators.

The reviewer is right in that we are solving specific problem instances, but even then, the above factors are present. An autoregressive policy is well-suited to address these factors. We will clarify this in the camera-ready version of the paper.

Reviewer comment (Experiments):
How is the order for selecting optimization variables determined, and does this order have the potential to affect the performance?

Author response:
In this work, we are doing *de novo* design: we start from an empty set and let the model decide which and how many tokens
(or building blocks) to add to the solution. The length of the solution is variable and the order in which the tokens are added is determined by the model. The ordering is a decision variable of the optimization problem (we can not fix it a priori). The reviewer is right that, in the tasks of symbolic regression and decision tree search, the order in which the tokens are added will affect the final design and thus the performance. The model should learn to add the tokens in the order that maximizes the reward. We will clarify this in the camera-ready version of the paper.

Reviewer comment (Novelty):
Mixed Integer Programming (MIP) is a well-known class of optimization problems that involve optimizing both discrete and continuous variables, and many studies attempt to optimize MIPs using deep learning. How does this research differ from those studies involving MIPs?

Author response:
The main differences between DisCo-DSO and approaches that use deep learning to solve MIP problems are the following:
1. The search space in DisCo-DSO is less structured than in MIP problems. The solutions in DisCo-DSO are variable length and the order in which the tokens are added is determined by the model. In MIP problems, the solutions are fixed length and the order is fixed.
2. In DisCo-DSO we have positional constraints that depend on the prefix of the solution. For instance, in symbolic regression, the model should not be allowed to add a trigonometric operator if it has already done so in the past to avoid nested trigonometric operators. In general MIP problems, the constraints are fixed and do not depend on the prefix of the solution.
3. In DisCo-DSO, discrete and continuous variables are linked. For instance, in decision tree search, we have the discrete variable $x_i < \beta$ and the continuous variable $\beta =5.3$, that jointly form the decision tree node $x_i < 5.3$. The search mechanism is designed to optimize both variables jointly. It is unclear if the approaches that use deep learning to solve MIP problems can handle this type of coupling. We will clarify this in the camera-ready version of the paper.

We will add a discussion about the differences between DisCo-DSO and approaches that use deep learning to solve MIP problems in the camera-ready version of the paper.

Reviewer comment (Experiments):
The experimental content seems limited. While three types of problems are presented, the first problem only compares the number of function evaluations, and the second problem presents curves indicating the improvement in solutions for a specific problem. It would be beneficial to solve a more diverse set of problems and provide statistical evidence to validate the performance.

Author response:
In this paper, we wanted to investigate the benefits of DisCo-DSO over the standard decoupled approach in generative optimization scenarios where the solution is constructed sequentially. 
For generality, we wanted to cover deterministic and stochastic optimization problems and also, different search spaces and semantics. We chose 
symbolic regression (deterministic optimization over the space of mathematical expressions) and decision tree policies in RL (stochastic optimization over the space of decision trees) as representative examples of these scenarios.
The implementation of each of these tasks in non-trivial and required a significant amount of work. We believe that the results are conclusive and statistically significant, and show that DisCo-DSO outperforms the standard decoupled approach in terms of generalization over the test data and quality of solution per number of function evaluations.

Reviewer comment (Novelty):
The concatenation of discrete and continuous variables into a single vector and the higher-level approach are very straightforward and their novelty seems limited.

Author response:
**"The concatenation of discrete and continuous variables into a single vector and the higher-level approach are very straightforward and their novelty seems limited."**
We agree with the observation that the concatenation of discrete and continuous variables into a single vector, along with the higher-level approach, may seem straightforward. However, we would like to highlight that the novelty of our approach lies in its application to the specific problem of optimization with generative models for hybrid spaces. 
The integration involves several challenges that we address in the paper, such as the need for prefix-dependent positional constraints, and the extension of the gradient update to handle the new search space. To the best of our knowledge, this is the first instance where such a concept has been systematically employed in this context. We believe this application of the idea adds a significant contribution to the field and opens up new avenues for research.

Reviewer comment (Experiments):
Why are you comparing your method against the baselines you define as opposed to using baselines from the literature in the case of symbolic regression (Figure 3)?

Author response:
**"Why are you comparing your method against the baselines you define as opposed to using baselines from the literature in the case of symbolic regression (Figure 3)?"**
The paper already contains well-established baselines for symbolic regression. In particular:

1. "Decoupled-GP-BFGS": This baseline corresponds to a standard implementation of Genetic Programming for symbolic regression à la Koza, 1994. This represents the current most common approach to symbolic regression in the literature.
2. "Decoupled-RL-BFGS": This baseline corresponds exactly to the method "Deep Symbolic Regression" from Petersen et al., 2021. We use the same codebase and the same hyperparameters. We will clarify this in the camera-ready version of the paper.

We want to emphasize that the main contribution of the paper is not the application of DisCo-DSO to symbolic regression, but rather the investigation of the coupled discrete-continuous optimization approach in generative optimization scenarios where the solution is constructed sequentially. Nevertheless, following the reviewer's suggestion, we have computed results for two recent deep learning approaches to symbolic regression: Kamienny et al., 2022 and Biggio et al., 2021. These are the results that we obtained for a similar number of function evaluations (note that their code does not natively report the number of function evaluations) for Kamienny et al., 2022:

| Model | Mean R test | Std R test |
| --- | --- | --- |
| DisCo-DSO | 0.7045 | 0.3007 |
| Decoupled-RL-BFGS | 0.6400 | 0.3684 |
| Decoupled-RL-evo | 0.0969 | 0.2223 |
| Decoupled-RL-anneal | 0.1436 | 0.3015 |
| Decoupled-GP-BFGS | 0.4953 | 0.4344 |
| Decoupled-GP-evo | 0.0747 | 0.1763 |
| Decoupled-GP-anneal | 0.1364 | 0.2608 |
| Kamienny et al., 2022 | 0.5699 | 0.1065 |

and for Biggio et al., 2021 (note that this system only supports $\leq 3$ dimensions):

| Model | Mean R test | Std R test |
| --- | --- | --- |
| DisCo-DSO | 0.6632 | 0.3194 |
| Decoupled-RL-BFGS | 0.6020 | 0.4169 |
| Decoupled-RL-evo | 0.0324 | 0.1095 |
| Decoupled-RL-anneal | 0.1173 | 0.2745 |
| Decoupled-GP-BFGS | 0.5372 | 0.4386 |
| Decoupled-GP-evo | 0.0988 | 0.1975 |
| Decoupled-GP-anneal | 0.1615 | 0.2765 |
| Biggio et al., 2021 | 0.6858 | 0.1995 |


We can see that the coupled approach of DisCo-DSO still outperforms or gives comparable results to these baselines. We will add these results together with a discussion about the differences between DisCo-DSO and these baselines in the camera-ready version of the paper.


References:

- John R Koza. Genetic programming as a means for programming computers by natural selection. Statistics and computing, 4:87–112, 1994.\\
- Brenden K. Petersen et al. Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients.  ICLR 2021, Virtual Event, Austria, May 3-7, 2021.\\
- Pierre-Alexandre Kamienny, St´ephane d’Ascoli, Guillaume Lample, and Franc¸ois Charton. End-to- end symbolic regression with transformers. arXiv preprint arXiv:2204.10532, 2022.\\
- Luca Biggio, Tommaso Bendinelli, Alexander Neitz, Aurelien Lucchi, and Giambattista Parascan- dolo. Neural symbolic regression that scales. In International Conference on Machine Learning, pp. 936–945. PMLR, 2021.

Reviewer comment (Experiments):
Given recent work on neuro-symbolic regression, the authors should compare against methods in the literature.

Author response:
**"Why are you comparing your method against the baselines you define as opposed to using baselines from the literature in the case of symbolic regression (Figure 3)?"**
The paper already contains well-established baselines for symbolic regression. In particular:

1. "Decoupled-GP-BFGS": This baseline corresponds to a standard implementation of Genetic Programming for symbolic regression à la Koza, 1994. This represents the current most common approach to symbolic regression in the literature.
2. "Decoupled-RL-BFGS": This baseline corresponds exactly to the method "Deep Symbolic Regression" from Petersen et al., 2021. We use the same codebase and the same hyperparameters. We will clarify this in the camera-ready version of the paper.

We want to emphasize that the main contribution of the paper is not the application of DisCo-DSO to symbolic regression, but rather the investigation of the coupled discrete-continuous optimization approach in generative optimization scenarios where the solution is constructed sequentially. Nevertheless, following the reviewer's suggestion, we have computed results for two recent deep learning approaches to symbolic regression: Kamienny et al., 2022 and Biggio et al., 2021. These are the results that we obtained for a similar number of function evaluations (note that their code does not natively report the number of function evaluations) for Kamienny et al., 2022:

| Model | Mean R test | Std R test |
| --- | --- | --- |
| DisCo-DSO | 0.7045 | 0.3007 |
| Decoupled-RL-BFGS | 0.6400 | 0.3684 |
| Decoupled-RL-evo | 0.0969 | 0.2223 |
| Decoupled-RL-anneal | 0.1436 | 0.3015 |
| Decoupled-GP-BFGS | 0.4953 | 0.4344 |
| Decoupled-GP-evo | 0.0747 | 0.1763 |
| Decoupled-GP-anneal | 0.1364 | 0.2608 |
| Kamienny et al., 2022 | 0.5699 | 0.1065 |

and for Biggio et al., 2021 (note that this system only supports $\leq 3$ dimensions):

| Model | Mean R test | Std R test |
| --- | --- | --- |
| DisCo-DSO | 0.6632 | 0.3194 |
| Decoupled-RL-BFGS | 0.6020 | 0.4169 |
| Decoupled-RL-evo | 0.0324 | 0.1095 |
| Decoupled-RL-anneal | 0.1173 | 0.2745 |
| Decoupled-GP-BFGS | 0.5372 | 0.4386 |
| Decoupled-GP-evo | 0.0988 | 0.1975 |
| Decoupled-GP-anneal | 0.1615 | 0.2765 |
| Biggio et al., 2021 | 0.6858 | 0.1995 |


We can see that the coupled approach of DisCo-DSO still outperforms or gives comparable results to these baselines. We will add these results together with a discussion about the differences between DisCo-DSO and these baselines in the camera-ready version of the paper.


References:

- John R Koza. Genetic programming as a means for programming computers by natural selection. Statistics and computing, 4:87–112, 1994.\\
- Brenden K. Petersen et al. Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients.  ICLR 2021, Virtual Event, Austria, May 3-7, 2021.\\
- Pierre-Alexandre Kamienny, St´ephane d’Ascoli, Guillaume Lample, and Franc¸ois Charton. End-to- end symbolic regression with transformers. arXiv preprint arXiv:2204.10532, 2022.\\
- Luca Biggio, Tommaso Bendinelli, Alexander Neitz, Aurelien Lucchi, and Giambattista Parascan- dolo. Neural symbolic regression that scales. In International Conference on Machine Learning, pp. 936–945. PMLR, 2021.