Paper: NeuralQP: A General Hypergraph-based Optimization Framework for Large-scale Quadratically Constrained Quadratic Programs (ICLR-2024)

Reviewer comment (Experiments):
While the experimental improvements appear substantial and reasonable, it's worth noting that the number of test cases is limited. Tables 1 and 2 indicate that the authors conducted tests on only nine QP instances. This could raise concerns about the reliability of results, especially in the context of deep learning, say, are these models overfit to a limited number of instances? Also in deep learning experiments, larger datasets are often expected.

Author response:
## Clarification about Testing Instances

Regarding Table 1 and Table 2, we would like to clarify that the results are not based on merely nine instances. Each entry represents the average of three problems of equivalent scale. 

The objective values presented in Table 2 are primarily taken from the median in Table 1. However, there are exceptions for large-scale QIS problems and small-scale QVC problems. We have re-executed the experiments for the QVC problems for consistency. As for the QIS problems, the objective value is selected from the log so that Gurobi can reach it within a reasonable timeframe.

## Dataset Size Considerations

Regarding the size of our dataset, we acknowledge the possibility of expanding it, yet we believe such an enlargement is not currently necessary. Here are our reasons:

1. **Rationale for Current Dataset Size:**
    - Our methodology integrates neural network prediction with small-scale solvers for neighborhood search, addressing QCQPs. This framework is inherently scalable, both in terms of problem size and diversity, which is demonstrated by our existing dataset.
    - While increasing the dataset size could, in theory, improve performance, this comes at the cost of extended training times. This is due to the need for solving numerous QCQPs to generate additional training data, a process that is inherently time-intensive.
    - The results from our experiments indicate that our approach already exhibits promising performance with the current dataset. Excessively enlarging the dataset, given our experimental context, could lead to diminishing returns and potentially unnecessary resource expenditure.

2. **Exploring Future Possibilities:**
    - Nevertheless, we recognize the merit in your suggestion to augment the dataset. In line with this, we are considering the development of a substantially larger model trained on a varied and comprehensive dataset. This approach, akin to the methodologies employed in large language models, aims to tackle an extensive array of problems, encompassing a diverse range of scales and types. Embarking on this path represents an innovative and yet-to-be-explored avenue in the realm of machine learning for optimization.

Reviewer comment (Reproducibility):
I couldn't find information about how the model was trained and the nature and size of the training dataset in my reading. If the training data consists of random instances, as shown in Figure 3, it's important to clarify how these instances were generated and the dataset's size. These details are crucial and should not be omitted in the main paper.

Author response:
##  Details about the Training Data

We give the details about the training data in this paper from the following three aspects, data generation, the training and the testing stage.

1. **Data generation:**
    - Our model comprises two primary components: neural network prediction and iterative neighborhood optimization. For each category of problem and its respective scale, we pre-train a neural network model to forecast optimal solutions. Our training and testing data generation process, as well as the neural network training methodology, are as follows:
    - **To verify the effectiveness of our solution framework on general QCQPs**, we utilized three types of problems: QKP, QIS, and QVC. These vary in their objective functions and constraints, with specific details provided in our updated appendix. The diversity of these test problems offers a robust assessment of our algorithm's performance.
    - **To generate large-scale problems for training and testing**, we adopted a method similar to [1](https://proceedings.mlr.press/v202/ye23e.html) for random parameter generation. For each problem type, we created datasets of small, medium, and large scales, containing 100, 50, and 10 problems, respectively. This approach ensures that each training set comprises approximately 100,000 variables paired with optimal or near-optimal values, allowing us to effectively train neural networks to discern underlying patterns in problems of similar scales and types.

2. **The training process**
    - During the training phase, we trained a hypergraph neural network with 6 convolution layers with a batch size of 1. The training was conducted over 100 epochs, with an early stopping criterion set at 20 epochs to prevent overfitting. Our loss function was the binary cross-entropy with logits loss.

3. **The testing stage**
    - For testing, we generated three problems for each problem type and scale. The results presented in Tables 1 and 2 represent the average outcomes of three problems. We conducted these tests on a machine with 36 Intel (R) Core (TM) i9-9980 XE @ 3.00 GHz CPUs. For the Gurobi benchmark and Gurobi as a small-scale solver, we set the `Threads` to 4 and the `NonConvex` parameter to 2, with all other parameters set to their default values. For the SCIP benchmark and SCIP as a small-scale solver, we used all the default parameters.
    - We further conducted tests on real-world problems, QAPLIB and QPLIB, on a machine with 128 Intel Xeon Platinum 8375 C @ 2.90 GHz CPUs. Preliminary results are included in the updated Appendix D.

Reviewer comment (Evaluation):
It would be beneficial to introduce a more comprehensive experimental metric that illustrates the trade-off between running time and solution quality. For instance, calculating the area under the curve of an objective score versus running time plot could provide valuable insights. Visualizing such plots in the main paper would enhance clarity. Additionally, the objective scores in Table 2 appear to have sharp fluctuations, which might warrant clarification – are these values selectively chosen?

Author response:
## Evaluation Metrics

- We have reviewed the literature on solution metrics for quadratic programming problems and the inherent challenges they pose. The objective value versus time and the optimality gap versus time metric are commonly used (see [1](https://proceedings.neurips.cc/paper_files/paper/2021/hash/fc9e62695def29ccdb9eb3fed5b4c8c8-Abstract.html) [2](https://arxiv.org/abs/2107.10201) [3](https://arxiv.org/abs/2012.13349) and [4](https://proceedings.mlr.press/v202/ye23e.html) ). However, in our setting, the optimal objective value is difficult to acquire since our focus is mainly on large-scale QCQPs and solving them to the optimality is notoriously hard, so we adopted the objective value versus time metric.
- We are intrigued by your suggestion of calculating the area under the curve versus time. If you could provide more explanation about this metric and its , we are open to integrating it into our future experiments. We believe it could be a novel approach in evaluating the solvers' performance.

Reviewer comment (Experiments):
Can you increase the size of the testing dataset?

Author response:
Dear Reviewer,

Thank you very much for your response. As mentioned earlier, excessively enlarging the dataset, given our experimental context, could lead to diminishing returns and potentially unnecessary resource expenditure. This is a significant reason why we did not initially expand the dataset size in our experiments. However, our model has the potential to improve the size of the dataset. It is demonstrated by the supplementary experimental results based on QMKP and QIS below. For each type of QCQP, there are three test problems, and all test problems consist of 20,000 decision variables. Additionally, in Appendix D of the latest submitted PDF, we have also included experimental results based on QPLIB and QAPLIB.

|             | QMKP               | QIS               |
|-------------|--------------------|-------------------|
| SCIP        | 20440.73           | 3353.95           |
| Ours-30\%S  | 28107.57↑          | 6399.03↑          |
| Ours-50\%S  | **28435.14↑**      | **6412.35↑**      |
| Gurobi      | 28532.50           | 6151.02           |
| Ours-30\%G  | 28540.98↑          | 6404.99↑          |
| Ours-50\%S  | **28547.50↑**      | **6417.66↑**      |
| Time        | 18000s             | 36000s            |

Moreover, according to your suggestion,  we have plotted the area-under-curve divided by running time, presented in Appendix E.3, and conducted a comparison of objective values and running times against Gurobi and SCIP.

Reviewer comment (Experiments):
Can you explain more details about the training data in the main paper?

Author response:
##  Details about the Training Data

We give the details about the training data in this paper from the following three aspects, data generation, the training and the testing stage.

1. **Data generation:**
    - Our model comprises two primary components: neural network prediction and iterative neighborhood optimization. For each category of problem and its respective scale, we pre-train a neural network model to forecast optimal solutions. Our training and testing data generation process, as well as the neural network training methodology, are as follows:
    - **To verify the effectiveness of our solution framework on general QCQPs**, we utilized three types of problems: QKP, QIS, and QVC. These vary in their objective functions and constraints, with specific details provided in our updated appendix. The diversity of these test problems offers a robust assessment of our algorithm's performance.
    - **To generate large-scale problems for training and testing**, we adopted a method similar to [1](https://proceedings.mlr.press/v202/ye23e.html) for random parameter generation. For each problem type, we created datasets of small, medium, and large scales, containing 100, 50, and 10 problems, respectively. This approach ensures that each training set comprises approximately 100,000 variables paired with optimal or near-optimal values, allowing us to effectively train neural networks to discern underlying patterns in problems of similar scales and types.

2. **The training process**
    - During the training phase, we trained a hypergraph neural network with 6 convolution layers with a batch size of 1. The training was conducted over 100 epochs, with an early stopping criterion set at 20 epochs to prevent overfitting. Our loss function was the binary cross-entropy with logits loss.

3. **The testing stage**
    - For testing, we generated three problems for each problem type and scale. The results presented in Tables 1 and 2 represent the average outcomes of three problems. We conducted these tests on a machine with 36 Intel (R) Core (TM) i9-9980 XE @ 3.00 GHz CPUs. For the Gurobi benchmark and Gurobi as a small-scale solver, we set the `Threads` to 4 and the `NonConvex` parameter to 2, with all other parameters set to their default values. For the SCIP benchmark and SCIP as a small-scale solver, we used all the default parameters.
    - We further conducted tests on real-world problems, QAPLIB and QPLIB, on a machine with 128 Intel Xeon Platinum 8375 C @ 2.90 GHz CPUs. Preliminary results are included in the updated Appendix D.

Reviewer comment (Evaluation):
Can you improve the experiments by involving the area under curve metric?

Author response:
Dear Reviewer,

Thank you very much for your response. As mentioned earlier, excessively enlarging the dataset, given our experimental context, could lead to diminishing returns and potentially unnecessary resource expenditure. This is a significant reason why we did not initially expand the dataset size in our experiments. However, our model has the potential to improve the size of the dataset. It is demonstrated by the supplementary experimental results based on QMKP and QIS below. For each type of QCQP, there are three test problems, and all test problems consist of 20,000 decision variables. Additionally, in Appendix D of the latest submitted PDF, we have also included experimental results based on QPLIB and QAPLIB.

|             | QMKP               | QIS               |
|-------------|--------------------|-------------------|
| SCIP        | 20440.73           | 3353.95           |
| Ours-30\%S  | 28107.57↑          | 6399.03↑          |
| Ours-50\%S  | **28435.14↑**      | **6412.35↑**      |
| Gurobi      | 28532.50           | 6151.02           |
| Ours-30\%G  | 28540.98↑          | 6404.99↑          |
| Ours-50\%S  | **28547.50↑**      | **6417.66↑**      |
| Time        | 18000s             | 36000s            |

Moreover, according to your suggestion,  we have plotted the area-under-curve divided by running time, presented in Appendix E.3, and conducted a comparison of objective values and running times against Gurobi and SCIP.

Reviewer comment (Theory):
What is the complexity of the approach? Say, given a QP with n variables, how many nodes and (hyper)edges will be created in the corresponding hypergraph? I have a general impression that the proposed approach cannot scale up smoothly due to the complexity and please correct me if I am wrong.

Author response:
## Complexity of our Approach

- For a QP problem containing $n$ variables and $m$ constraints, a total of $m + n + 3$ vertices will be generated. This includes n variable vertices, m constraint vertices, one objective function vertex, one vertex representing the constant 1, and one vertex representing square terms. 
- The number of hyperedges depends on the number of monomials in the original problem. For instance, the expression $x_1 + x_2 + x_1 x_2$ would generate 3 hyperedges. Each constraint contains at most $\frac{1}{2}n(n+3)$ monomials, so a maximum of $\frac{1}{2}n(n+3)(m+1)$ hyperedges could be produced. As the reviewer pointed out, such a hypergraph is very dense and can be challenging for GNNs to process, leading to high computational complexity. 
- However, in reality, it's almost impossible to encounter a quadratic problem of this density. In the test problems we constructed with $n$ variables and $m$ constraints:
    - The QIS problem has $n+m\times \operatorname{avg}{(e_{deg}^2 + e_{deg})}$ hyperedges where $e_{deg}$ is the hyperedge degree. In our test instances, $e_{deg}$ is 5 so the number of hyperedges is $n+30 m$ ;
    - The QKP problem has $\frac{1}{2}n(n+1) + mn$ hyperedges;
    - The QVC problem has $\frac{1}{2}n(n+1) + 3m$ hyperedges.

- Our experimental results show that our hypergraph neural network did not exhibit significant degradation in performance on QVC problems.

We hope this response addresses your concerns appropriately. Thank you again for your valuable feedback.

Reviewer comment (Writing):
The text is unclear and hard to understand. The English is poor. The notation and mathematical formalism are clumsy and not well thought-over. The text is not well structured: sometimes a simple thing is given too much space but some substantial things are only in the supplement. There are a number of mistakes.

Author response:
Dear Reviewer,

Thank you very much for your detailed comments and constructive criticisms. In response to your valuable feedback, we have meticulously reviewed the entire manuscript, making comprehensive revisions to enhance the clarity of the text, structure, and overall expression. We have also carefully refined and standardized the notation and mathematical definitions throughout the paper to ensure accuracy and coherence. Following these extensive revisions, we have resubmitted the paper. Additionally, we are supplementing our experiments with results from the QAPLIB dataset, which are now included in Appendix D. Our response is organized into three main sections: our motivation and experiment results, a thorough description of the problem formulations we have developed, and other detailed modifications and responses to your points.

Reviewer comment (Experiments):
In experimental evaluation, only random QCQP instances (generated by the authors) are used. No real instances available online (e.g., QAPLIB) are used. It is well-known that random instances of optimization problems tend to be easier than real ones. Moreover, the QCQP problems that are used are described (in the supplement) in an insufficient and unclear way and appear to be artificial and sometimes meaningless.

Author response:
1.1. **Motivation:**

   - We opted for randomly generated problems, considering the current state of QCQP research and our primary objectives:
     1. Machine learning methods have been used to optimize problems on a larger scale or at a faster speed. However, **the use of machine learning methods in solving QCQP is scarce, and existing datasets like QPLIB [1](https://link.springer.com/article/10.1007/s12532-018-0147-4) are limited.** QPLIB contains only 453 problems, each distinct and sourced from different distributions, unsuitable for machine learning applications. Meanwhile, other related learning methods in literature have also focused on **smaller scales** [2](http://arxiv.org/abs/2301.00306) or **quadratic problems (quadratic objective with linear constraints)** [3](https://link.springer.com/chapter/10.1007/978-3-319-93031-2_43) [4](https://pubsonline.informs.org/doi/abs/10.1287/ijoc.2022.1181?casa_token=Xb9xbDNG7BYAAAAA:aGPrsKwU_HVVq-Mu9Uv3iBoZD2Id4VNu_2ZfwlGENFg8TPz6YJbeI6W-i8Y1KIlkic0Z_0nqFL5F), which is unsuitable for benchmarking with our method. 
     2. **To validate our solution framework on different problems**, we introduced three problem forms: QKP, QIS, and QVC. Each form represents a different type of constraint and objective function, which we detail in the updated appendix A.4. The variety in test problems is aimed to provide a comprehensive evaluation of our algorithm's performance.
     3. **To obtain large-scale problems for training and testing**, we used random methods to generate QCQP instances. This ensures problems of the same type and scale can be perceived as coming from the same distribution, making them suitable for neural network learning.
     4. Despite the potentially lower complexity of randomly generated problems compared to real-world instances, the computation time taken by solvers like Gurobi and SCIP in our experiments attests to **the complexity of our generated problems**.

   We thank you for suggesting the addition of QAP problems and have included them in our experiments in Appendix D.2.

1.2. **Experiment Results on more real-work problems:**

   - Our experiments demonstrate that our method can address a variety of problems, showing its universality, and performs particularly well on large-scale problems. 
   - Tests on real-world problems, including QAPLIB and QPLIB, have been conducted. We trained neural network models on our generated problems and then tested them on real-world problems. Experiment settings and results are presented in Appendix D in the updated version, which further validates the effectiveness and efficiency of our framework.

Reviewer comment (Theory):
In A.4.1, there is a mismatch between the stated objective of QIS and formulation (7); it is also unclear whether in $\sum_{i,j\in e}$ in (8) one allows $i=j$. This can be expressed by simple linear constraints $\sum_{i\in e}x_i\le 1$. Additionally, (randomly generated?) weights $a_i$ and $q_{ij}$ are introduced in the constraints without explanation of their meaning. It is not mentioned how the hypergraph $\cal H$ is generated when generating the instances.

Author response:
**QIS:**

We developed the QIS (Quadratic Independent Set) problem by building on the foundation of the IS (Independent Set) problem, introducing quadratic non-convex constraints to give it practical significance. The classic IS problem is a typical combinatorial optimization challenge, assuming uniformity among all points and edges. However, in realistic network models, each node can have distinct attributes. To represent this, random weights $c_i$ are often introduced into the objective function to denote the attributes of node $i$. 

Considering that real-world network nodes might exhibit high-order relationships, we employed a hypergraph model where each hyperedge can include an arbitrary number of nodes. Additionally, these hyperedges may have non-linear relationships, so we introduced a function $f$ to represent these non-linear interactions. The coefficients $a_i$ and $q_{ij}$ are randomly generated from $U(0,1)$, and the inclusion of $|e|$ (the degree of the hyperedge) in our constraints is designed to ensure that these constraints are effective yet achievable, while also correlating them with the hyperedges.

The QIS model has potential applications in scenarios involving hypergraph network models with nonlinear inter-node relationships. For instance, in wireless communication networks, particularly in frequency assignment, the challenge often lies in allocating frequencies to transmitters or channels in a manner that minimizes interference and maximizes network capacity or efficiency. This scenario can be conceptually similar to our QIS model, where it's essential to consider the (possibly nonlinear) interactions between various network elements. Other relevant applications include social network analysis and similar problems where complex relationships and attributes play a critical role.

Reviewer comment (Writing):
In A.4.3, the meaning of QVC formulation in (10) is not explained and it appears to make no sense. The asterisk symbol in (10) is undefined. It is unclear why the variables are denoted $u,v$ rather than $i,j$ as before. The choice of coefficients $c_u,q_{u,v}$ is not explained.

Author response:
**QVC:**

We derived the QVC problem from the standard vertex covering problem (appendix A.4.3), introducing quadratic terms in both the objective function and constraints. This represents one of the most challenging classes of QCQP problems and serves to evaluate the effectiveness of our solution framework. Based on our experiment results, QVC problems are the hardest problems among the three.

Reviewer comment (Theory):
The Quadratic Knapsack Problem definition in Section A.4.2 differs from the usual one (see, e.g., https://www.sciencedirect.com/science/article/pii/S0166218X06003878): in (8) there are multiple linear constraints on each individual binary variable $x_i$. This is not justified and seems nonsensical.

Author response:
**Quadratic Multiple Knapsack Problem (QMKP):**

The QMKP problem [5](https://dl.acm.org/doi/abs/10.1145/1143997.1144096?casa_token=SYh1mdTyStsAAAAA:vImGiDvs4SgDbLysHgjth6_zKkURNwRKkhe1DHgItbyoZfTgl4c_G3qpJwwUT4BlJPjxueshUUCexwk) we study is a variant of the QKP problem. We initially used the abbreviation QKP, but following your suggestion, we have changed this to QMKP for clarity.

The real-world significance of the QMKP problem involves optimizing the value of items with multiple attributes under certain constraints. An example would be maximizing the nutritional value of a day's diet, where some food combinations offer greater nutritional benefits (similar to a standard QKP), but each food item also has attributes like calories and sodium content that should not exceed certain limits.

Reviewer comment (Writing):
Section 2.1: The set $\cal X$ is mentioned only in problem formulation (1). In contrast, the set $\cal N$ does not appear in (1) (as as written below (1)). It is not clear if the QCQP has continuous or integer variables.

Author response:
3.1. **Revisions Based on Your Suggestions:**

 We have updated formulation (1) to more clearly reflect integer variables and included both maximization and minimization problems. We also added explanations for $b_i$ in sections 2.2 and 4.1.1 and revised the definition in Def 5. We have made language adjustments and streamlined the paper, integrating algorithm 1 into algorithm 2 for clearer relationships between algorithms. Further specific details of these improvements include:

Reviewer comment (Reproducibility):
Section 2.2: It is not described how the right-hand sides $b_i$ of MILP constraints are represented in the graph (despite the authors writing that the MILP instance encoding by the graph is lossless). The same omission occurs for the QCQP encoding in Section 4.1.1.

Author response:
3.1. **Revisions Based on Your Suggestions:**

 We have updated formulation (1) to more clearly reflect integer variables and included both maximization and minimization problems. We also added explanations for $b_i$ in sections 2.2 and 4.1.1 and revised the definition in Def 5. We have made language adjustments and streamlined the paper, integrating algorithm 1 into algorithm 2 for clearer relationships between algorithms. Further specific details of these improvements include:

Reviewer comment (Writing):
Section 3.1: The definition of "bipartite hypergraph" is unclear. In the literature, the term can mean several different things.

Author response:
3.2. **Clarification on Bipartite Hypergraph Definition:**

   - We have refined the definition of bipartite hypergraph to avoid ambiguity, following standard definitions in literature ([8](https://epubs.siam.org/doi/abs/10.1137/1.9781611974331.ch126), [9](https://link.springer.com/article/10.1007/s10479-022-05073-9)).

A hyper graph $\mathcal{H}$ is defined as
$$
\begin{aligned}
&\mathcal{H} = (\mathcal{V}_1, \mathcal{V}_2, \mathcal{E})\text{ such that}\\
&(1)\quad\mathcal{V}_1 \cap \mathcal{V}_2 = \emptyset,\\
&(2)\quad |e\cap \mathcal{V}_1| = 1 \text{ and } e\cap \mathcal{V}_2 \neq \emptyset, \forall e \in \mathcal{E}.
\end{aligned}
$$

Reviewer comment (Writing):
Definition 1: The incidence matrix $\bf H$ and the "neighborhood relation" $N$ are almost the same and do not deserve separate definitions. Only one of these terms should be used.

Author response:
3.3. **On Definitions 1 and 2:**

   - We initially used definitions consistent with the literature [10](https://ieeexplore.ieee.org/abstract/document/9795251?casa_token=0LL4IvxB06UAAAAA:WyiNXPIHsdqeZEXaD6PWtAMFUZteSAxlvUPj_5fOZrJ8jI_n0BkAtC6SpYM9ebhNuKJt_Fv7iDU) and [11](https://link.springer.com/10.1007/978-981-99-0185-2). To maintain clarity and consistency, we have retained the definitions of incidence matrix $H$ and inter-neighbor relation $N$. Following your advice, we have improved the definitions of $\mathcal{N}_e (v)$ and $\mathcal{N}_v (e)$ :

     $$
     \mathcal{N}_e(v) = \{ e | v\mathrm{N} e, e \in \mathcal{E} \}, \text{ for each } v \in \mathcal{V}.
     $$

     $$
     \mathcal{N}_v(e) = \{ v | v\mathrm{N} e, v \in \mathcal{V} \}, \text{ for each }e \in \mathcal{E}.
     $$

Reviewer comment (Writing):
Definition 5: $v_i$ should be $c_i$. Otherwise, the definitions of ${\cal V}_x$ and ${\cal V}_c$ make no sense if the sets $\cal N$ and $\cal M$ are not disjoint.

Author response:
3.1. **Revisions Based on Your Suggestions:**

 We have updated formulation (1) to more clearly reflect integer variables and included both maximization and minimization problems. We also added explanations for $b_i$ in sections 2.2 and 4.1.1 and revised the definition in Def 5. We have made language adjustments and streamlined the paper, integrating algorithm 1 into algorithm 2 for clearer relationships between algorithms. Further specific details of these improvements include:

Reviewer comment (Presentation):
Algorithm 1 in the supplement should not be called an algorithm; it is just a calculation of formulas. Moreover, these formulas appear already in the main text but with different notation. Algorithm 1 is also not called by any of Algorithms 2, 3, 4, 5.

Author response:
3.1. **Revisions Based on Your Suggestions:**

 We have updated formulation (1) to more clearly reflect integer variables and included both maximization and minimization problems. We also added explanations for $b_i$ in sections 2.2 and 4.1.1 and revised the definition in Def 5. We have made language adjustments and streamlined the paper, integrating algorithm 1 into algorithm 2 for clearer relationships between algorithms. Further specific details of these improvements include:

Reviewer comment (Theory):
The QCQP formulation (1) is a minimization problem, whereas in the experiments (Tables 1 and 2) the problems are maximization ones.

Author response:
3.1. **Revisions Based on Your Suggestions:**

 We have updated formulation (1) to more clearly reflect integer variables and included both maximization and minimization problems. We also added explanations for $b_i$ in sections 2.2 and 4.1.1 and revised the definition in Def 5. We have made language adjustments and streamlined the paper, integrating algorithm 1 into algorithm 2 for clearer relationships between algorithms. Further specific details of these improvements include:

Reviewer comment (Writing):
The initial manuscript was in a very poor shape, which is not good practice to submit. It indicates reliance on reviewers to correct avoidable mistakes.

Author response:
Dear Reviewer,

Thank you very much for your detailed comments and constructive criticisms. In response to your valuable feedback, we have meticulously reviewed the entire manuscript, making comprehensive revisions to enhance the clarity of the text, structure, and overall expression. We have also carefully refined and standardized the notation and mathematical definitions throughout the paper to ensure accuracy and coherence. Following these extensive revisions, we have resubmitted the paper. Additionally, we are supplementing our experiments with results from the QAPLIB dataset, which are now included in Appendix D. Our response is organized into three main sections: our motivation and experiment results, a thorough description of the problem formulations we have developed, and other detailed modifications and responses to your points.