Paper: Learning to Solve Bilevel Programs with Binary Tender (ICLR-2024)

Reviewer comment (Evaluation):
The numerical experiments lack sufficient evidence to demonstrate the advantages of the proposed algorithm. It appears that the algorithm can only achieve optimality for test instances with n = 10. The authors solely conduct comparisons with the baseline method MiBS based on computational times, without assessing solution quality, such as the optimality gap.

Author response:
Thanks for your comments. 
We agree that we do not calculate the optimality gap in our numerical experiments, yet as we have emphasized in the general response, it is currently impossible to obtain the optimal solution to large-size bilevel programs. 
In this paper, we use the objective difference to assess the solution quality, and the results are reported in Section 4.

Reviewer comment (Experiments):
The test instances are only from the authors’ synthetic generated ones. Some public datasets like MIBLP-XU and IBLP-FIS in Tahernejad et al., 2020 should be used.

Author response:
Thanks for your suggestion. 
Actually, the instance generation rule provided in Appendix C of this paper follows closely the rules of generating MIBLP-XU in Tahernejad et al., 2020. 
Because we focus on bilevel programs with binary tender, the existing instances in MIBLP-XU are not applicable. 
So we have used the instance generation rule of MIBLP-XU as the basis to generate new instances with binary tender. 
The following sentence is added in Appendix C to state this point.  

*''Following the instance generation rules of (Tahernejad et al., 2020), all instances are generated in the following forms.''*

Reviewer comment (Theory):
The cost to solve a single instance of Problem (5) is very expensive because it is at least as hard as a mixed-integer quadratic program (MIQP).

Author response:
Thanks for your comments. 
As we have emphasized in our general response (ii) before, we do not need to solve (5) to optimality, which significantly reduces the computational burden during sampling.

Reviewer comment (Theory):
What is the stopping criterion for Algorithm 2 so that the generated point $(x,y)$ is guaranteed to be feasible to Problem 1? That is, $x,y$ is satisfied $x \in X(y), y \in Y(x), g(y,x) \ge \max_{z \in Y(x)}  g(z,x)$.

Author response:
Due to the assumption that the leader's feasible region is independent of $y^{\*}$, i.e., $X(y^{\*})\equiv X$, any generated point $(x,y)$ from Algorithm 2 is feasible, for which there is no extra stopping criterion. 
The following gives a detailed explanation.  

According to Step 8 of Algorithm 2, the obtained $x^{\*}$ must satisfy the constraints of (12), which means $x^{\*}\in X$. 
According to Step 9 of Algorithm 2, we fix $x=x^{\*}$ and solve the lower-level problem to obtain $y^{\*}$, which means $y^{\*}\in Y(x^{\*})$ and $g(y^{\*},x^{\*})=\max_{z\in Y(x^{\*})}g(z,x^{\*})$. 
Therefore, any generated point $(x,y)$ from Algorithm 2 must satisfy $x\in X$, $y\in Y(x)$, and $g(y,x)\geq\max_{z\in Y(x)}g(z,x)$. 
Under the assumption, the generated point $(x,y)$ is naturally feasible to Problem (1). 
The following sentence is highlighted to state the point. 

*''In addition, we assume the leader's feasible region is independent of $y^{\*}$, i.e., $X(y^{\*})\equiv X$, which is a special case of (1). In this case, we can obtain an upper bound for the BP (1) from any feasible solution $x$ and the corresponding follower's optimal solution $y^{\*}$ (e.g., by solving the follower's problem using $x$).''*

Reviewer comment (Evaluation):
What is the definition of “relative error” in Figures 4 and 7? How do you define an error when an optimal solution cannot be found, for example, for large-scale problems with n = 60?

Author response:
Thanks for your question. 
As we have emphasized in our general response (i) before, we have replaced the term ''relative error'' with ''objective difference'' to avoid confusion. 
The objective difference is assessed by using the solution of MiBS as the benchmark. 
We also provide the definition of ''objective difference'' in the revised paper as  

*''We compare our method with the state-of-the-art solver for bilevel problems, MiBS (Tahernejad et al., 2020) and use its solution under a time limit of 1 hour as the benchmark to calculate the objective differences (i.e., the gap between the objectives from our method and MiBS).''*

Reviewer comment (Evaluation):
Why relative errors can be negative (as in Fig. 7)?

Author response:
Thanks for your question. According to the definition provided in the response to Question 2, the objective difference (previously relative error) would be negative if our method provides a better solution than the benchmark. 
The following sentence is added in the revised paper for explanation.  

*''The negative objective difference means that our method provides a better solution than the benchmark.''*

Reviewer comment (Theory):
Why we need to care of supermodularity of an ISNN? I notice that ISNN requires more neurons compared to GNN (i.e., the lower bound value for $N_{nr}$ in Eq. (8) vs Eq. (7)).

Author response:
Thanks for your question. 
First, the number of neurons in GNN and ISNN depend on the values of $N_{s}$ and $n$ and so there is no guarantee that one needs more neurons than the other. 
Also, according to Proposition 3, many weight parameters of ISNN end up being zero after training. Hence, it may not be completely fair to compare the complexity of GNN and ISNN only through the number of neurons.  

Second, the supermodularity of an ISNN helps to improve the solution quality of our method. 
For bilevel programs with binary tender, many lower-level problems generate a supermodular $\phi(x)$ [R3, R4]. If we relax the binary limit $x\in${$\{0,1\}$} of $\hat{\phi}(x)$ to $x\in[0,1]$, $\hat{\phi}(x)$ becomes a lower envelope of the discrete $\phi(x)$. 
Consequently, if we adopt an ISNN, the trained $\tilde{\phi}(x)$ tends to be a lower bound of $\phi(x)$ (see Figure 3) so that with high probability, the optimal solution to the original problem would not be cut off by (12b). 
Conversely, if we adopt a GNN, there is no guarantee that $\tilde{\phi}(x)\geq\phi(x)$ or vice versa, for which the optimal solution to the original problem may be cut off by (12b). 
Therefore, the supermodularity of an ISNN helps to improve the solution quality of our method.  

[R3] X. Chen, D.Z. Long, J. Qi. Preservation of supermodularity in parametric optimization: necessary and sufficient conditions on constraint structures. Operations Research. 2021, 69(1): 1-12.  
[R4] D.Z. Long, J. Qi, A. Zhang. Supermodularity in two-stage distributionally robust optimization. Management Science, ahead of print.

Reviewer comment (Theory):
Do we have to solve $\max_{y \in Y(\hat{x})} g(y, \hat{x})$ to optimality in Algorithm 1?

Author response:
Thanks for your question. 
Yes, in step 7 of Algorithm 1, we need to solve $\max_{y\in Y(y,\hat{x})}$ to optimality to obtain correct $(\hat{x}$, $\phi(\hat{x}))$ pairs for training.

Reviewer comment (Theory):
For feasibility purpose, why do we need the second term $h^T x$ in Eq. (5)?

Author response:
Thanks for your question. 
The objective (5a) is a general formula for quadratic programming, which does not affect feasibility. 
In other words, we can set $h = 0$ in (5) without affecting the feasibility.

Reviewer comment (Presentation):
The name of section 3.3 seems to be incomplete.

Author response:
Thanks for your comments. 
The main steps of our method consist of sampling, training, and solving, which are used as the titles of sections 3.1-3.3, respectively.

Reviewer comment (Presentation):
I don’t see any part of Figures to support the claims “achieving relative error less than 5%,”, “the relative error of instances with n = 30, 50, 60 is larger than 15%.”

Author response:
Thanks for your comments. 
For the claim ''the relative error of instances with n = 30, 50, 60 is larger than 15\%,'' please see Figure 4. 
For the claim ''achieving relative error less than 5\%,'' please see Figure 7.

Reviewer comment (Novelty):
This work employs neural networks to learn and approximate the value function $\phi(x)$. However, training the neural networks is more computationally complex than directly approximating the lower-level optimization problem [2]. What is the advantage of the proposed method over the polyhedral approximation in [2]?

Author response:
Thanks for your comments.
First, we have carefully read the recommended paper [2], in which cutting planes based on [R2] are utilized to form polyhedral approximations.
Specifically, [2] construct these cutting planes using gradient information from the lower-level problem.
However, as we have emphasized in the general response, the class of bilevel programs we study in this paper have binary tenders and the lower-level problem therein contain binary decision variables.
For these reasons, gradient methods become inapplicable, and so do the polyhedral approximation in [2].

Second, since the lower-level problem in this paper involves binary decision variables, it is in general inapproximable (see, e.gl, [R3]), that is, there do not exist polynomial-time algorithms that produce solutions to such lower-level problems with a constant competitive ratio.

Third, we clarify that this paper does not intend to prove that neural networks outperform any approximation methods in solving general bilevel programs.
Instead, in view of the inapplicability of the existing approximation methods (see above) and the power of neural networks in other machine learning problems, we seek to demonstrate that NN also help solve this challenging class of bilevel programs. 

[R2] Stephen Boyd and Lieven Vandenberghe. Localization and cutting-plane methods. From Stanford EE 364b lecture notes, 2007.  
[R3] Trevisan, Luca. Inapproximability of combinatorial optimization problems. Paradigms of Combinatorial Optimization: Problems and New Approaches (2014): 381-434.

Reviewer comment (Novelty):
Since one of the key contributions in this work is to employ neural networks to learn and approximate the value function $\phi(x)$, I suggest the authors clearly discuss the existing approximation-based methods (i.e., which approximate the lower-level optimization problems and reformulate the bilevel optimization problems as single-level optimization problems, for instance, polyhedral approximation) in bilevel optimization since I can't find any discussion about the approximation-based methods in bilevel optimization.

Author response:
Thanks for your suggestion.
Actually, we have discussed some approximation-based methods in the part ''Bilevel (mixed-integer) nonconvex programs'' of Section 2.
In this revision, as suggested, we have modified related sentences to highlight such methods.

*''For BPs with nonconvex objectives/constraints, approximation methods were employed to approximate the lower-level problem and reformulate the BP as a single-level formulation, such as polyhedral approximation (Sato et al., 2021) and other gradient-based methods in machine learning (Hong et al., 2020; Chen et al., 2022).
However, when discrete decision variables are incorporated, gradient-based methods are no longer applicable.
To address this issue, integer programming techniques are employed and various cutting plane-based algorithms, such as the MiBS solver, have been developed (Ralphs et al., 2015; Zeng \& An, 2014).''*

Reviewer comment (Experiments):
Instance dimension up to 60 is too small.

Author response:
1. > Instance dimension up to 60 is too small.

    We would like to emphasize that bilevel programs are challenging to solve, and especially for bilevel programs with binary tender (which is considered in this paper), the computational burden increases exponentially with respect to the number $n$ of the linking variables.
    Therefore, $n=60$ is not a small instance for the class of problems we are studying, which are challenging bilevel binary-integer programs. .
    Nevertheless, we have added new instances with $n$ up to 120 in the revised paper.
    Please see Appendix D.4 for details.

Reviewer comment (Experiments):
The selection of the lower level problem is LP and MILP, which both have linear lower level objective function. At least you could have tried some simple nonlinear functions like quadratic function.

Author response:
2. > The selection of the lower level problem is LP and MILP, which both have linear lower level objective function. At least you could have tried some simple nonlinear functions like quadratic function.

    We remark that our approach applies to lower-level problems with an arbitrary objective function. 
    In fact, we agree with you and look forward to testing our approach in, e.g., quadratic instances.
    However, as the state-of-the-art and the benchmark method to compare to, MiBS is applicable to linear lower-level problems only.
    Consequently, we have to restrict ourselves to linear lower-level problems at this point and look forward to testing our approach in more general instances when a proper benchmark solver becomes available.

Reviewer comment (Evaluation):
The experimental results do not support the conclusion well.

Author response:
3. > The experimental results do not support the conclusion well. I list some of the points in the next Questions section.

    We have carefully addressed all the listed points below.
    Please see corresponding answers for details.

Reviewer comment (Experiments):
In Figure 5 and Figure 6, the computational time of MiBS increases very fast with the increase of n. My question is, even though the MiBS solver might take a long time to reach optimality, have you considered to set a time limit, and compare the relative error of the best found solution given by the solver at time limit with your approaches?

Author response:
Thanks for your question.
Actually, we have set the time limit of MiBS to be 1 hour and the relative error in all existing results is compared to the best-found solution of MiBS within this time limit.
The following sentence is added in Section 4.2.2 to state this point.

*''We compare our method with the state-of-the-art solver for bilevel problems, MiBS (Tahernejad et al., 2020) and use its solution within a 1-hour time limit as the benchmark to calculate the objective differences (i.e., the gap between the objectives from our method and MiBS).''*

In this revision, we also vary the length of the time limit of MiBS to 0.5 hours, 1.5 hours, and 2 hours, respectively.
But for each instance used in the paper, the best-found solution of MiBS keeps the same under different time limits.

Reviewer comment (Experiments):
In your Conclusion section you mentioned: "we demonstrated that the enhanced sampling helps reduce average relative error". However, the whole point of your sampling method is simply to get enough data points for training the neural network. In order to claim that your proposed enhanced sampling can bring some extra benefits, you need to at least compare with other non-trivial sampling methods.

Author response:
Thanks for your comments.
The proposed enhanced sampling brings us two aspects of benefits: sampling efficiency and sample quality.

On the one hand, as we have emphasized in the general response, as compared to naive sampling which draws samples of $x$ from {$\{0,1\}$}$^n$ based on a certain probability distribution, the adopted ''sampling via optimization'' approach guarantees finding a feasible $x$ in each sampling, which enhances the sampling efficiency.
In this revision, we compare the sampling efficiency of our method with a naive sampling method (random sampling) and a non-trivial sampling method (Latin hypercube sampling) and provide the results in Appendix D.3.
It is obvious that our sampling method is more stable and applicable than both benchmarks.

On the other hand, by adding and updating the constraint (5b), we can sample more often in the vicinity of the optimal solution, which enhances the quality of the found samples.
Actually, we have validated this point in our numerical experiments (see Figure 4 and Figure 6).
According to Algorithm 2, $N_{iteration}=1$ means no constraint (5b), $N_{iteration}=2$ means adding a constraint (5b), and $N_{iteration}=3$ means updating the constraint (5b).
We observe that a larger $N_{iteration}$ helps to reduce the objective difference, which corresponds to the mentioned sentence ''we demonstrated that the enhanced sampling helps reduce average relative error".

Reviewer comment (Evaluation):
In Conclusion section: "The computational time of using ...... is significantly shorter than that of MiBS ......" I admit that this is true, but your methods also do not reach optimality, for fair comparison you either need to have enough samples for exactly learning the value function (so that your approach can also produce a true optimal solution), or you need to compare the best feasible solution found by MiBS within a given time limit.

Author response:
Thanks for your comments.
We agree with the comment and in this paper, we have chosen the latter suggestion, which compares the best feasible solution found by MiBS within a given time limit.
Please see our response to Major Question 1 for more details.

Reviewer comment (Experiments):
Still about the definition of "binary tender". On Page 2 "we assume that the entries of x appearing in the lower-level formulation are binary-valued". However, in Appendix C, about the instance generation for the experiments, I notice that the binary constraint on x is enforced in the upper-level instead of the lower-level formulation.

Author response:
Thanks for your comments.
We would like to remark that the lower-level problem can be seen as a parametric optimization problem, where the upper-level variables $x$ are treated as given parameters.
Therefore, the binary constraint on $x$ should be enforced in the upper level, but not in the lower level where $x$ becomes parameters.