Paper: GenCO: Generating Diverse Solutions to Design Problems with Combinatorial Nature (ICLR-2024)

Reviewer comment (Theory):
How does this research address the mechanism behind generating diverse solutions when assigning combinatorial properties to various generative models, and does it simply rely on the use of generative models to claim diversity?

Author response:
Our research indeed leverages generative models to achieve diversity in solutions by approximating the underlying data distribution. The mechanism involves training the generative model to capture and represent the diverse characteristics inherent in the given combinatorial problem.

Reviewer comment (Theory):
Combining GAN and VAE within a unified framework may seem unusual, given their distinct purposes. Is there a thoughtful consideration of the different objectives of these two generative models, or does it feel mechanically extended without such reflection?

Author response:
Thank you for raising this important point. While GANs and VAEs traditionally serve distinct purposes, our approach aims to highlight the flexibility of combinatorial optimizers in accommodating both GAN and VAE frameworks. The integration is not merely a mechanical extension; rather, it's a thoughtful exploration of the adaptability of combinatorial optimizers to different generative models. We acknowledge the distinct objectives of GANs and VAEs and demonstrate that our approach GenCO is versatile enough to be applicable to both, as demonstrated in Appendix C. Furthermore, we evaluate a new setting of generating inverse photonic designs in which we employ a VQVAE as the generative backbone and ensure that decoded objects satisfy the combinatorial constraints during training.

Reviewer comment (Theory):
In section 3.2, regarding the Constrained Generator, are there any limitations in terms of the expressive power of linear projection? How does it differ from techniques like projected gradient ascent commonly used in constrained optimization?

Author response:
We would like to note that it is a projection in the sense that we are identifying a combinatorially constrained discrete solution that is near the original continuous solution (i.e. projection into combinatorial space). Because our settings have only binary decisions, we can guarantee that each solution is reachable by some continuous solution (meaning that all discrete solutions are possible to generate). However, we would note that the projection is not linear (i.e., not a simple linear transformation). On the contrary, the projection is the solution to a discrete linear optimization problem, which is highly discrete and piecewise constant in that a large area in the original continuous space may map to the same solution in discrete space. Additionally, while projected gradient ascent is a popular approach for solving some constrained optimization problems, it is not generally used in combinatorial optimization where the standard methods would be a variant of tree search for MILP and constraint satisfaction problems or variants of simplex for linear programs.

Reviewer comment (Novelty):
In the penalized generator discussed in section 3.3, the idea of directly using the cost obtained through the combinatorial optimization solver h for model training has been present in prior research. Are there any novel aspects from a learning perspective in this new approach?

Author response:
While we acknowledge prior works that incorporate costs as penalties, such as Donti et al. 2020, it's important to highlight the novelty of our approach from a learning perspective. Existing works typically focus on the continuous case or rely on continuous relaxation, whereas our method directly handles combinatorial penalties or losses without such relaxation and trains the whole pipeline end-to-end. This distinction represents a novel aspect of our approach, as we are not aware of any previous work that addresses combinatorial penalties in the same direct manner.

Reviewer comment (Experiments):
There is limited analysis of how the approach scales with problem size and constraint complexity. More extensive experiments on large and complex problems would be useful.

Author response:
It is indeed important to add more validation. To support our claims, we included one additional benchmark: inverse photonic design, which is a real-world, practical, and large-scale problem. Our results and discussion can be found in general response (2). For now, we have included it in the appendix, and we will include it in the main text of our camera-ready revision.

Reviewer comment (Experiments):
The tradeoff between generative loss and combinatorial loss is not carefully analyzed. There is not enough details on how the parameter $\gamma$ is set and the impact it has on training.

Author response:
Please review the additional results presented below. As anticipated, the SP loss decreases with an increase in gamma. However, the image quality concerning other metrics, such as density/coverage, starts to deteriorate rapidly. Eventually, at a certain gamma value, our approach generates blank or empty images, as such images result in the shortest path with zero cost. This intriguing behavior could be attributed to the combinatorial nature of the problem.  
We use cross-validation to set the gamma (and other hyperparameters). We will make this clear in the next revision of the paper.  
| Gamma        | SP Loss | Density | Coverage |
|--------------|---------|---------|----------|
| gamma = 0    | 36.45   | 0.81    | 0.98     |
| gamma = 1e-4 | 27.66   | 0.93    | 0.93     |
| gamma = 1e-3 | 23.99   | 0.94    | 0.93     |
| gamma = 3e-3 | 23.76   | 0.75    | 0.86     |
| gamma = 5e-3 | 18.02   | 0.49    | 0.61     |
| gamma = 1e-2 | 0.00    | 0.00    | 0.00     |

Note: gamma = 1e-2 generates blank/empty images

Reviewer comment (Experiments):
While backpropagation through combinatorial solvers is possible and works well in a lot of settings, it can be computationally expensive. Alternate approaches to incorporate solvers might be important for scalability.

Author response:
There are indeed alternative methods that can be used depending on problem type. We've employed (Sahoo et al, 2022) for Warcraft experiments, which is much faster computation-wise (and arguably more stable). Additionally, "solver-free" methods could be involved (Shah et al., 2022; Zharmagambetov et al., 2023), which will be important to use as a replacement for the differentiable optimizer as the optimization problem at hand becomes less computationally tractable.

Reviewer comment (Theory):
Theoretical analysis of GenCO's properties is limited. For instance, more analysis of convergence guarantees, sample complexity could strengthen the approach.

Author response:
We only have guarantees on the feasibility of the generated solutions, it is unclear how to obtain meaningful convergence guarantees or sample complexity without resorting to unreasonable and simple assumptions.

Reviewer comment (Theory):
It is not very clear to me how the method would generalize to general logical constraints, which can be important in the design scenarios considered.

Author response:
General logical constraints are possible as long as they can be expressed as the feasible region to an integer linear program which has been flexibly used to encode constraints in scheduling, routing, resource allocation, matching, time windowed delivery, coloring, packing, first-order logic, and more.

Reviewer comment (Reproducibility):
Reproducibility: The authors do not include code to reproduce the results and experimental details are discussed but details about hyperparameter selection are missing.

Author response:
We have clarified training details in the Supplementary files and in the Experiments section. We will add more details in the next revision. Furthermore, we are committed to enhancing openness by open-sourcing our implementation before the camera-ready, enabling our methodology to be extended to relevant domains.

Reviewer comment (Experiments):
Could you elaborate a bit on the scalability of the approach to complex constraints and problems?

Author response:
It is applicable as long as the constraints can be optimized over. That is, as long as the underlying optimization problem can be solved either exactly or with reasonable approximation. Modern solvers such as Gurobi often solve industrial-scale problems in seconds or minutes. Furthermore, there is potential to extend recent advances in "solver-free" methods (Shah et al., 2022; Zharmagambetov et al., 2023) in cases where the optimization model is expensive to solve.

Reviewer comment (Theory):
Could you comment on how more general constraints could be incorporated in this framework?

Author response:
Our approach can handle any kind of constraints that can be expressed using optimization modeling tools that have been made differentiable, such as maxSAT, mixed integer linear programs, cone programs, quadratic programs, and many more. We refer to Kotary, James, et al. "End-to-End Constrained Optimization Learning: A Survey." 2021 for an overview of the different methods here, most of which could be readily integrated into our pipeline depending on the task at hand.

Reviewer comment (Evaluation):
My primary concern regarding this paper is the selection of quantitative evaluation metrics. As a solution for generative tasks, the assessment of diversity and generation quality should draw from some classic evaluation metrics in the traditional generative model field, such as FID [1] and density/coverage [2]. While the evaluation method using a discriminator indeed holds some value, it is not an authoritative network, and this evaluation metric is single-dimensional in terms of discriminability, making its reliability less robust.

Author response:
Thanks for pointing this out. Please see our general response (3) above, where we provide additional quantitative metrics (density/coverage). We will definitely include them in our paper revision.

Reviewer comment (Reproducibility):
Some training details can be more specific. For instance, one aspect to consider is the stability and efficiency of the GAN training process. GANs are renowned for their challenging training dynamics, and the utilization of approximated gradients obtained through black-box optimization methods could potentially exacerbate the risk of training instability. Moreover, since the training data is limited, effectively training a GAN becomes a non-trivial endeavor.

Author response:
We have implemented suggested regularization techniques from (Pogancic et al. (2020); Sahoo et al. (2022)) to stabilize the training of differentiable combinatorial solvers. Also, we have clarified training details in the Supplementary files and in the Experiments section. We will add more details in the next revision. Subjectively, we observe that training our pipeline has similar complexity as training regular GANs in terms of stability, hyperparameter tuning, etc. 

Furthermore, we are committed to enhancing openness by open-sourcing our implementation soon.

Understanding the empirical nature of the concerns raised, we have conducted additional empirical evaluations (photonic design in general response (2)) to strengthen the robustness of our claims. Additionally, we've explored the trade-off of the gamma parameter in Section 3.3 (see our response to reviewer @X2M2).

Reviewer comment (Evaluation):
What about the quantitative evaluation results of traditional metrics in the generative model field such as FID and density/coverage?

Author response:
Thanks for pointing this out. Please see our general response (3) above, where we provide additional quantitative metrics (density/coverage). We will definitely include them in our paper revision.

Reviewer comment (Experiments):
"... , given that the levels are trained on only 50 examples, ...": How is the training dynamic of GAN with so limited data? Are there any additional efforts to stabilize training?

Author response:
In game-level design, the discrete nature of GenCO’s output is beneficial in our scenario. Specifically, there are finite discrete objects that the GAN can generate, and thus it is easier to avoid the issues of mode collapse compared to a traditional GAN, which can effectively hide solutions in the continuous space. Indeed, this is shown in practice as many of the continuous solutions map to the same discrete solution after postprocessing.

Reviewer comment (Theory):
In the discussion of uniqueness in Sec. 4.1.2, it makes the adversary's task easier as it only needs to distinguish between valid discrete levels rather than continuous and unconstrained levels. But if the discriminator is too strong, it can easily lead to the phenomenon of gradient vanishing, does this make training even more challenging and unstable?

Author response:
As in the previous level generation paper, we use Wasserstein GAN (WGAN), which was designed to prevent vanishing gradients as there is always some incoming gradient imposed on the generated solution as long as the generated solution is different from the known solutions and the adversary hasn’t become degenerately poor. Overall, we only have access to 50 training levels, and GenCO is able to perform well in this low data regime by still generating many unique solutions while the postprocessing approach fails to do so.

Reviewer comment (Theory):
In fact, many generative tasks naturally exhibit certain discrete characteristics, but due to the strength of generative models, they can inherently learn these features. For example, generative models can learn that dogs have four legs. Is it possible that with a sufficiently powerful generative model, it can automatically recognize and learn these discrete features?

Author response:
While powerful generative models can capture certain discrete features, there are some limitations to the models, even with large capacity. For example, at the time of writing, large models like stable diffusion or DALL E 2 suffer from generating images of people with incorrect numbers of fingers or toes, or where their pose doesn't match that of their reflection in a mirror unless specifically fine-tuned and instructed for such tasks. Furthermore, going beyond image/video generation, for other design tasks with mathematically well-defined constraints (e.g., industrial design), GenCO can make full use of the domain-specific constraints. It can flexibly and explicitly encode such constraints and eliminate the possibility of generating infeasible solutions, regardless of the size of the datasets. This could lead to better performance with the same computational costs and model capacity.

Reviewer comment (Novelty):
The so-called combinatorial loss is unsurprising, as it has been used in many areas, though without a uniform name or formulation. For example, in areas such as molecule design and chip design, it is usual to use the panelty in reward design for hard or soft constraints. It is also used in solving combinatorial problems to guide the feasible and quality of solutions. This kind of methods are like the Lagrangian multiplier method, and the idea of modeling the constraints as a loss term is straightforward. There are also many methods targeting on solving combinatorial problems, such as mixed-integer programmings, by generative models, where the combinatorial constraints are also considered. This paper is more like a gatherer of those methods. If I am wrong, the authors may want to further emphasise the technical contribution.

Author response:
> Combinatorial loss has been used in many areas, e.g. molecule and chip design, where it is usual to use the penalty in reward design for hard or soft constraints

While it's true that penalizing constraint violation has been employed in various domains like molecule and chip design, it is critical to emphasize that they typically do not guarantee feasibility. One possible reason for this is that in molecule or chip design settings, the constraints are often provided as blackbox simulators that are difficult to represent using constraint optimization solvers, thus, it is difficult to guarantee that solutions satisfy constraints such as synthesis accessibility or water lipid solubility. 

Our work stands out as we uniquely combine generative models (GANs/VAEs) and combinatorial solvers with feasibility guarantees and propose an end-to-end training algorithm for the entire pipeline. 

Also, please refer to our general response regarding the novelty and importance of our work, as well as our experiments on the photonic device design setting to demonstrate generalizable performance.

> ​​This kind of methods are like the Lagrangian multiplier method, and the idea of modeling the constraints as a loss term is straightforward.

Thank you for your observation. While the concept of using combinatorial optimization as a loss term may seem reminiscent of the Lagrangian multiplier method, it's important to emphasize a key distinction in our approach. Unlike the Lagrangian multiplier method, which lacks feasibility guarantees, our method provides explicit feasibility guarantees throughout training. Furthermore, in discrete optimization cases, handling Lagrangian functions and multipliers can be non-trivial, and there may not be feasibility guarantees in cases where the continuous relaxation is very loose. In contrast, our approach incorporates hard constraints seamlessly using a differentiable combinatorial layer in the pipeline, allowing for end-to-end training. These key differences set our methodology apart in terms of both feasibility assurances and empirically improved performance. 


> There are also many methods targeting on solving combinatorial problems, such as mixed-integer programmings, by generative models, where the combinatorial constraints are also considered. 

These methods are mainly concerned with solving a single Mixed Integer Program rather than generating multiple solutions that satisfy the combinatorial constraints. Our goal is to satisfy all three conditions on solutions: diversity, solution quality, and feasibility. Furthermore, these problems generally concern solving a single well-defined problem where all objective coefficients and constraints are known. In our settings, the problem is underspecified in that for the game-level design, inverse photonic design, or shortest path map settings, we don’t have a fixed problem formulation that we are trying to solve.

> This paper is more like a gatherer of those methods. If I am wrong, the authors may want to further emphasise the technical contribution.

Hopefully, our answers above and general response regarding our technical contribution clarified your concerns. Furthermore, as we demonstrate in the game level design and inverse photonic design setting, GenCO is the only approach that is capable of providing many unique solutions which are all guaranteed to satisfy combinatorial constraints, whereas previous approaches that treat optimization as a postprocessing step yield degenerate solutions.

Reviewer comment (Experiments):
If the authors claim that they propose a framework for design problems, it is expected to conduct more experiments such as molecule or chip design. Current experiments are not convincing enough.

Author response:
> If the authors claim that they propose a framework for design problems, it is expected to conduct more experiments such as molecule or chip design. Current experiments are not convincing enough.

It is indeed important to add more validation. To support our claims, we included one additional benchmark: inverse photonic design, which is a real-world, practical, and large-scale problem. Our results and discussion can be found in general response (2). For now, we have included it in the appendix, and we will include it in the main text of our camera-ready revision.