Paper: Equivariant Quantum Graph Neural Network for Mixed-Integer Linear Programming (ICLR-2024)

Reviewer comment (Theory):
I feel that many parts of the construction of the ansatz is not well motivated. However, this seems to be a typical issue for quantum neural networks, perhaps even moreso than classical neural networks.

Author response:
**A10:** Thanks for your comment. In fact, because our network is required to satisfy permutation equivariance, each layer of our model must be designed for encoding or learning while ensuring permutation equivariance, which is much more motivated than other QNNs. We believe we have addressed some of your concerns in our previous answers **A2**, **A3**, and **A9**. Moreover, the new content added in section 3.6 in our current submitted PDF can further address your concerns. 

We hope that the above answers can address your concern. If you have any further questions, please contact me again. I am looking forward to your reply.

Reviewer comment (Novelty):
Unless I am misunderstanding something, I feel that this permutation equivariance is not particularly insightful. For instance, the equivalent circuits in Figure 7 seem obvious, just that the circuit wires are drawn to either have different input order or output order. Perhaps it would be more useful to show a construction that fails to satisfy permutation equivariance.

Author response:
**A9:** First of all, we need to clarify that Figure 7 is just a very simplified example of showing permutation equivariance, and it does not represent our circuit, and designing permutation-equivariant circuits is not easy. To address your concerns, we have rewritten Section 3.5 and 3.6 in the latest uploaded PDF. Section 3.5 formulates the proposed model. In particular, **Section 3.6 presents four definitions and three theorems to prove the permutation equivariance of our proposed model**. Since this part is about two pages long, please refer to the highlighted section of our latest uploaded PDF. In addition, we also provide a shorter version to answer the question.

One of the most intuitive examples of failure to satisfy permutation equivariance is HEA (Hardware efficient ansatz). [Figure R4](https://postimg.cc/qhCCzp3P) shows an example of an HEA with three qubits, which is used to learn the representation of graphs with three nodes. [Figure R4](https://postimg.cc/qhCCzp3P) uses $R_x$ gate to encode the feature of three nodes, and uses $RY(\theta_0), RY(\theta_1)$, $RY(\theta_2)$ and CNOT gate as the learning module. Suppose that the order of node is $(v_0, v_1, v_2)$, we measure each qubit and get the probability of $|0\rangle$ at each qubit as the output $(y_{v_0}, y_{v_1}, y_{v_2})$. Then, the order of node changes to $(v_1, v_0, v_2)$ and the output is 
$(y_{v_1}^{\\prime}, y_{v_0}^{\\prime}, y_{v_2}^{\\prime})$. 
Obviously, $y_{v_0} \neq y'_{v_0}$. The permutation equivariance of the model is vital for graph learning, and the output of the model should be reordered consistently with the permutation on the input nodes, otherwise the model may overfit to the order of nodes in the training data. In the field of classic machine learning, GNN can use the neighborhood-aggregation mechanism to achieve the permutation equivariance. However, in the field of quantum computing, the permutation-equivariant model requires fine designs due to the constraints of quantum systems.

Reviewer comment (Theory):
In the definition of the feasible region $X_{fea}$ in Section 2, the constraint $l \leq x \leq u$ is missing.

Author response:
**A6:** Thank you for pointing this out, and we have corrected the definition of the feasible region $X_{fea}$ in our latest uploaded PDF.

Reviewer comment (Presentation):
In Figure 6, I feel the circuit for $R_{zz}$ gates is somewhat misleading. The cross symbol is typically used for the SWAP gate. Also, the $R_{zz}$ is symmetric with respect to the two qubits it acts on, so there is no difference between choosing which is the target and which is the control qubit.

Author response:
**A7:** Thanks for your comment. The $R_{zz}$ gates with different control qubits indeed have the same unitary matrix. However, [Figure R3](https://postimg.cc/R6QKYKb3) shows an example of using different control qubits. As we can see, although the input quantum state and output quantum state of the whole circuit are kept consistent for Figure (a) and (b), the position where the global phase appears on a single qubit will be different, such as Figure (a) outputs $|\psi_0^a\rangle = |1\rangle$ and Figure (b) outputs $|\psi_0^b\rangle = e^{i\frac{\theta}{2}}|1\rangle$. However, the global phase $e^{i\frac{\theta}{2}}$ has no effect on the observations resulting from a measurement of the state, and it is only the artefact of the mathematical framework and has no physical meaning. We specify the control qubits solely for the certainty of the mathematical formulation. Nevertheless, our specification of control qubits and the use of cross symbols may cause misunderstanding, so we modified Figure 6 in our paper, please see Figure 5 of the latest submitted PDF. Thank you for your suggestion.

Reviewer comment (Writing):
Typo in the sentence "the identical parameteric gates are acted when the order of input nodes."

Author response:
**A8:** Thanks for pointing out the problem. The sentence is incomplete and should be modified to "the identical parametric gates are acted on the circuit no matter how the order of input nodes changes."

Reviewer comment (Writing):
Typo in the sentence "We now study the effect of width of the circuit increased"

Author response:
**A8:** We apologize for not writing this sentence clearly. It should be modified to "We now study the influence on the performance with the increase of circuit width".

Reviewer comment (Theory):
It is not clear to me why there are instances of MILP that cannot be distinguished by GNNs, as in Figure 2. While the vertex degrees are the same, the connectivity is different, so shouldn't GNNs treat them differently? Perhaps I am missing something about how standard GNNs deal with this problem in the context of MILP.

Author response:
**A1:** This inability to distinguish is related to **the essential limitation of the neighbor-aggregation mechanism** of GNNs, and this limitation does not only exist in the context of MILP, which we will elaborate on next. 

To better understand this limitation, let's first take a more fundamental example, which is also not distinguishable by GNNs. The problem of distinguishing two graphs can be abstracted as a graph isomorphism testing problem, i.e., to determine whether two graphs are isomorphic.

*Definition 1: Two graphs G1 and G2 are isomorphic if there exists a matching between their vertices so that two vertices are connected by an edge in G1 if and only if corresponding vertices are connected by an edge in G2.*

That is, two graphs that are isomorphic means that they will have the same topological structure or connectivity. Let's look at a simple example of non-isomorphic graphs, which you can see by clicking on this [Figure R1](https://postimg.cc/56YC7GMt). Both graphs are *2-regular graphs*, and the regular graph is a graph where each vertex has the same number of neighbors, i.e., every vertex has the same degree. The only difference is that Figure R1 (a) has two cycles, and Figure R1 (b) has only one cycle.

The 1-dimensional Weisfeiler-Lehman (1-WL) algorithm is a well-known heuristic for deciding whether two graphs are isomorphic. It is known to work well in general, **with a few exceptions, e.g., regular graphs** [1,2]. Moreover, Xu et al. [3] show that **GNNs are *at most* as powerful as the 1-WL test** in distinguishing graph structures. Therefore, a pair of 1-WL indistinguishable graphs are indistinguishable by GNNs. Next, we will briefly introduce the 1-WL test algorithm and why the [Figure R1](https://postimg.cc/56YC7GMt) cannot be distinguished by 1-WL or by GNNs.

**Algorithm 1:** 1-WL (color refinement)

**Input:** $G = (V, E, X_V)$
1. $c_v^0 = hash(X_v)$ for all $v\in V$
2. **repeat**
3. $\quad c_v^l = hash(c_v^{l-1}, \\{\\{c_w^{l-1}: w \in  \mathcal N_G(v)\\}\\})$, for all $v \in V$
4. **until** $c_v^l= c_v^{l-1}$ for all ${v\\in V}$
5. **return** $\\{\\{c_v^l: v\in V \\}\\}$

In the above algorithm, we consider an undirected graph $G = (V,E,X_V)$, where $V$ is the set of vertices; $E$ is the set of edges; and $X_V$ is the set of node features. The notation $\\{\\{·\\}\\}$ denotes a multiset, which is a generalization of the concept of a set in which elements may appear multiple times: an unordered sequence of elements. $hash(·)$ is a function that maps its input to a color in $\mathcal C$. The algorithm flow can be seen as follows. First, all nodes in $G$ are assigned to an initial color $c_v^0$ according to their node features. Then, $hash$ function maps $v$’s previous color $c_v^{l-1}$ and the multiset of the previous colors of $v$’s neighbors $\\{\\{c_w^{l-1}: w \in  \mathcal N_G(v)\\}\\}$ to a color $c_v^l$ in $\mathcal C$, for each $v \in V$. This process is repeated until the colors of all nodes do not change, i.e., $c_v^l= c_v^{l-1}$ for all ${v\\in V}$. Finally, a histogram $h_G$ of the node colors can be obtained according to $\\{\\{c_v^l: v\in V \\}\\}$, which can be used as a canonical graph representation. That is, the 1-WL test transforms a graph into a canonical representation, and then if the canonical representation of two graphs is equal, 1-WL will consider them isomorphic.

In [Figure R1](https://postimg.cc/56YC7GMt), we assume that all node features are equal, and thereby all nodes are assigned to the same initial color. Then, the hash function aggregates the color of $v$ and the colors of the two neighbors of $v$. It should be noted that Figure R1 (a) and Figure R1 (b) have the same result in this step because their vertex degrees and node features are the same. This also directly leads to the fact that Figure R1 (a) and Figure R1 (b) will certainly have the same canonical graph representation after the 1-WL algorithm. However, they are clearly non-isomorphic graphs, and 1-WL fails to distinguish them. 

(to be continued)

Reviewer comment (Theory):
Why choose $R_{zz}$ over other two-qubit gates that commute with itself such as $R_{xx}$ or $R_{yy}$? This goes back to the weakness of the circuit construction not being well motivated.

Author response:
**A2:** First of all, we need to clarify that the *commute* in this article does not refer to $R_{zz}(\theta_{12})R_{zz}(\theta_{23}) = R_{zz}(\theta_{23})R_{zz}(\theta_{12})$, but to $[R_{zz}(\theta_{12})\otimes I][I\otimes R_{zz}(\theta_{23})] = [I\otimes R_{zz}(\theta_{23})][R_{zz}(\theta_{12})\otimes I]$. More specifically, it refers to $\text{exp}(-\text{i}(\theta_{i_1,j_1} Z_{i_1} Z_{j_1} + \theta_{i_2,j_2} Z_{i_2} Z_{j_2})) = \text{exp}(-\text{i}(\theta_{i_2,j_2} Z_{i_2} Z_{j_2} + \theta_{i_1,j_1} Z_{i_1} Z_{j_1}))$, where $Z_{i_1} = I \otimes ... Z_{i_1} \otimes ... \otimes I$. Since $\text{exp}(-\text{i} A_{i,j} Z_i Z_j)$ is a diagonal matrix, and all diagonal matrices are commute, the equation is proven. However, other two-qubit gates, \emph{e.g.}, $\text{exp}(-\textbf{i} A_{i,j} X_i X_j)$ does not satisfy the equation. In other words, the $\text{R}_\text{zz}(\theta) = \text{exp}(-\textbf{i} \theta Z_i Z_j)$ gate is carefully chosen. In addition, we have added the above to section 3.6 of the updated PDF and shown why this equation is required to hold.

Reviewer comment (Theory):
In the feature encoding layer, is there any reason why $R_x(c_i)$ and $R_x(u_i)$ as well as $R_x(l_i)$ and $R_x(\epsilon_i)$, are applied to different qubits?

Author response:
**A3:** As mentioned in Section 3.2, we set a hyperparameter $\omega$ to control the number of features on each qubit in our method. Figure 5 of our paper only shows an example of $w = 2$. Each variable has four features, and two qubits represent it. Of course, we can also set $\omega$ to 4 so that $(c_i, u_i, l_i, \epsilon_i)$ is encoded on the same qubit. As we can see, the smaller the value of $w$, the larger the number of qubits representing one node. This means that the higher the dimension of the feature representation we can learn for each node, the more expressive the model will be. The choice of $\omega$ is a trade-off between the available quantum resources (the number of qubits) and the expressive power of the model. In addition, since each feature dimension is associated with a learnable parameter, the order in which these four features are used is not important.

Reviewer comment (Experiments):
Out of curiosity, how the MILP instances are generated to be foldable or unfoldable? I suppose this requires something like solving the graph isomorphism problem.

Author response:
**A4:** Thanks for your questions. The generation of foldable or unfoldable MILP instances is consistent with that of the reference (Chen et al., ICLR 2023b) [1], which does not involve solving the graph isomorphism problem. Next, we will introduce the generation process in detail and add this part to our revised paper. 

Note that the foldable and unfoldable datasets are generated separately. *Foldable* dataset is constructed by many pairs of 1-WL *indistinguishable* graphs, and figure 2 in our paper is a foldable example, which is a pair of non-isomorphic graphs that cannot be distinguished by WL-test or by GNNs, as mentioned in the response to Question 1. In contrast, the *unfoldable* dataset refers to other MILP instances that do not have 1-WL indistinguishable graphs. 

In section 4.1, we randomly generate 2000 *foldable* MILPs with 12 variables and $6$ constraints, and there are 1000 feasible MILPs with attachable optimal solutions while the others are infeasible.
We construct the $(2k − 1)$-th and $2k$-th problems via following approach $(1 ≤ k ≤ 500)$.
* Sample $J = \\{j_1, j_2, . . . , j_6\\}$ as a random subset of $\\{1, 2, . . . , 12\\}$ with 6 elements. 
1. For $j \in J$, $x_j \in \\{0, 1\\}$, i.e., $x_j$ is a binary integer variable. 
2. For $j \notin J$, $x_j$ is a continuous variable with bounds $l_j∼ \mathcal{U}(0, \pi), u_j∼ \mathcal{U}(0, \pi)$. If $l_j > u_j$, then switch $l_j$ and $u_j$.
* $c_1 =. . . = c_{12} = 0.01$.
* The constraints for the $(2k−1)$-th problem (feasible) is $x_{j_1}+x_{j_2}=1$, $x_{j_2}+x_{j_3}=1$, $x_{j_3}+x_{j_4}=1$, $x_{j_4} + x_{j_5} = 1$, $x_{j_5} + x_{j_6} = 1$, $x_{j_6} + x_{j_1} = 1$. For example, $x = (0, 1, 0, 1, 0, 1)$ is a feasible solution.

(to be continued)

Reviewer comment (Reproducibility):
When predicting the solution vector, what is the actual representation of the classical information after reading out the qubits? For example, does a 2-qubit state for $v_1$ correspond to integers 0,1,2,3? Also, what happens when reading out a result that does not satisfy the constraints?

Author response:
**A5:** Thanks for your question. As mentioned earlier, the hyperparameter $\omega$ can control the number of qubits representing each node. If the number of qubits is more than 1, like the node $v_1$ in Figure 6, we will add the control gate at the end of the node update layer, such as the CRY gate in Figure 6. Then, at the end of the circuit, we only measure the first qubit representing the node. As mentioned in section 3.5 of our paper, we perform the Pauli-Z measurement on the qubit, which can obtain the $P(|0\rangle)-P(|1\rangle)$, which ranges from -1 to 1. That is, each variable node can obtain one value $F_{sol}^i(G)$ to predict the values of variables. In addition, the maximum range $\lambda$ of variables of the training sample is known. The solution vector of the final output of the model is 
$\\{\lambda F_{sol}^i(G)\\}_{i=0}^{q-1}$.

Assuming the groundtruth of the solution vector is $\\{y_{sol}^i(G)\\}_{i=0}^{q-1}$, we evaluate the performance of the predicted solution vector by:

$$\frac{1}{Mq}\sum_{m=0}^{M-1} \sum_{i=0}^{q-1}(y_{sol}^i(G_m)-\lambda F_{sol}^i(G_m))^2$$

where $G_m$ indicates the $m$-th tested MILP graph, and the number of tested instances is $M$. As we can see, the values of the solution vector are continuous. For integer variables, a post-processing can be applied to make a rounding approximation. In addition, we need to make it clear that the proposed method is similar to the previous methods [1], only providing an approximate solution rather than providing an exact solution. We have added the above explanation to our latest uploaded PDF, and please see our highlights in Section 3.5.

Reviewer comment (Novelty):
In Section 1, the authors claimed that "We propose a so-called (for the first time) Equivariant Quantum Graph Neural Network (EQGNN) capable of representing general MILP instances without the need for dataset partitioning." The authors need to clearly explain what it means by ``without the need for dataset partitioning'' and why it is important.

Author response:
**A1:** Thanks for your comment. This sentence appears after introducing that the classic GNNs need to divide the whole MILP dataset into the *foldable* and *unfoldable* datasets. For unfoldable MILPs, GNNs can solve them directly. However, for foldable MILPs, GNNs require additional processing on the dataset to solve them, such as adding random features. This is because foldable MILPs contain many pairs of graphs that cannot be distinguished by GNNs, such as Figure 2, which has different feasibility for the MILP problem, but their graph representations learned by GNNs are the same due to the fundamental limitations of GNNs. The theoretical explanation of why GNNs cannot distinguish these two graphs can refer to our *response to Weakness 2*, and we have added it to our [lastest submitted PDF](https://openreview.net/pdf?id=KbvKjpqYQR). *More importantly, according to statistics, about 1/4 of the problems in MIPLIB 2017 (Gleixner et al., 2021) [1] involved foldable MILPs*. It means that practitioners using GNNs cannot benefit from that if there are foldable MILPs in their dataset of interest (Chen et al., 2023b) [2]. To this end, Chen et al. [2] proposed a method by appending the vertex features of foldable MILPs with an additional random feature. However, adding random features is likely to change the feasibility or solution of the original problem, thus causing the ground truth of the dataset to change, so this treatment is still controversial. In contrast, as stated by Reviewer pyPd, our proposed EQGNN can "overcome a fundamental limitation of traditional GNNs (i.e., GNNs can not distinguish pairs of foldable MILP instances)". In other words, the meaning of "EQGNN without the need for dataset partitioning" is that our proposed EQGNN can be applied to any MILP instance, and there is no need to make any differential treatment for all MILP instances. It is worth noting that this is one of the core contributions of our paper, which demonstrates the quantum advantage over classical graph neural networks in dealing with MILP problems. We apologize for the clarification of this point not being clear enough in our paper, so we have added more explanation about this sentence. Please refer to the highlighted section in our latest PDF submission. Thanks for your suggestion.

References:

[1] Ambros Gleixner, Gregor Hendel, et al. MIPLIB 2017: Data-Driven Compilation of the 6th Mixed-Integer Programming Library. Mathematical Programming Computation, 2021

[2] Ziang Chen, Jialin Liu, Xinshang Wang, and Wotao Yin. On representing mixed-integer linear programs by graph neural networks. In Proceedings of the International Conference on Learning Representations, 2023b

Reviewer comment (Theory):
Also in Section 1, the authors claimed that "We both theoretically and empirically show the separation power of EQGNN can surpass that of GNNs in terms of representing MILP graphs." However, the current submission does not contain any theoretical analysis of the separation power of EQGNN.

Author response:
**A2:** Thanks for your suggestions. We have added the theoretical analysis to show that the separation power of EQGNN can surpass that of GNNs in terms of representing MILP graphs, as shown in Appendix B in **[our current updated PDF](https://openreview.net/pdf?id=KbvKjpqYQR)**. The separation power of GNN is measured by whether it can distinguish two non-isomorphic graphs [2], and it has been shown that GNN has the same separation power as the WL test [3]. Therefore, we first demonstrate why 1-WL test and GNNs fail to distinguish between some non-isomorphic graphs (i.e., components of the foldable dataset), and then demonstrate how our EQGNN distinguishes between them, and thus show that EGNN can surpass the separation power of GNNs.
 
We present a variant of the WL test specially modified for MILP that follows the same lines as in (Chen et al. 2023b, Algorithm 1) [2].

**Algorithm 2:** WL test for MILP-Graphs

**Input:** A graph instance $(G, H) \in \mathcal{G}_{q,p} \times \mathcal{H}^V_q \times \mathcal{H}^S_p$, and iteration limit $L > 0$.
1. Initialize with $c_v^0 = hash(\boldsymbol f^V)$ for all $v\in V$, $c_s^0 = hash(\boldsymbol f^S)$ for all $s\in S$.
2. **for** $l = 1,2,...L$ **do** 
3. $\quad c_{v_i}^l = hash(c_{v_i}^{l-1}, \sum_{j=0}^{p-1} A_{i,j} hash(c_{s_j}^{l-1}))$, for all $v \in V$
4. $\quad c_{s_j}^l = hash(c_{s_j}^{l-1}, \sum_{i=0}^{q-1} A_{i,j} hash(c_{v_i}^{l-1}))$, for all $s \in S$
4. **end for** 
5. **return** the multisets containing all colors $\\{\\{c_v^l: v\in V, c_s^l: s\in S \\}\\}$.

(to be continued)

Reviewer comment (Experiments):
In Section 4.1, it is unclear why and how the number of parameters are compared between neural networks in classic and quantum computers. The authors need to clarify how the criteria are chosen. In Section 4.2, it is also unclear why neural networks in classical computers with 2,096 and 7,904 parameters are chosen as the baseline. It will be better if the authors can compare the proposed model with a non-data-driven method for solving MILP.

Author response:
**A3:** Thanks for your suggestion. The GNN of (Chen et al., 2023b) has one hyperparameter that controls the number of parameters, i.e., the embedding size $d$. In the tasks of predicting feasibility, the number of their parameters is $30d^2 +30d$. Our model also has a hyperparameter to control the number of parameters, i.e., the number of blocks $T$. Therefore, in our experiments, we vary these two hyperparameters separately to compare their performance results. In section 4.1, we choose $d = 4, 8, 16, 32$ as the embedding size of GNN, and $T= 2, 4, 6, 8$ as the number of blocks of EQGNN. In our experiments, we first gradually increase the embedding_size/blocks to test the performance of the models and then find the embedding_size/blocks $d^*$ or $T^*$ corresponding to the best performance. Then, we select the values near $d^*$ or $T^*$ and show their corresponding results in the experiments of the paper. In Section 4.2,  for the sake of clarity in the curve graph, we only select two hyperparameters of each model for comparison. We trained EQGNN with the number of blocks at $6$, $10$, and the number of parameters is $88$ and $148$. The embedding size $d$ of GNN is chosen as $8$ and $16$ with the number of parameters at $2$,$096$ and $7$,$904$. The reasons for the setting are as follows. We observe that the train performance of GNN increases as the number of parameters increases, but the generalization performance decreases. The train performance of GNN with $d=8$ is worse than that of EQGNN, and the train performance of GNN with $d=16$ is better than that of EQGNN. Therefore, we choose the GNNs with these two hyperparameters for comparison. We have added the above description in the latest PDF. Furthermore, we have added more numerical experiments to show the performance change of the GNN with different parameters in the Appendix.

In addition, current non-data-driven methods for solving MILP, such as Gurobi or SCIP solver, can use brute-force search to find the exact solution of our datasets, because the size of our data set is relatively small. So we didn't compare them in the experiment.

Reviewer comment (Writing):
Throughout the paper, there are many typos/errors/inconsistencies. For example, at the beginning of the third paragraph of Section 2: "Foladable MILP instances"; in equation (4) the authors use $R_x$ gate for encoding $l$ and $\epsilon$ but in Figure 5 $R_z$ is used to encode $l$ and $\epsilon$. It would be better to have consistent annotations for example in Section 2, the same $q$ decision variables and $p$ constraints can be used to index $V$ and $S$ in bipartite graph representation. There are many of these problems.

Author response:
**A4:** Thanks for pointing out our mistakes. "Foladable MILP instances" should be corrected to "Foldable MILP instances", and  Figure 5 should be corrected to [this picture](https://postimg.cc/5X9jtV7f).

We have rewritten the equation (4) in our latest submitted PDF. In addition, we have modified the vertex set of bipartite graph representation as $V \cup S$, where $V=\\{v_0,\cdots,v_i,\cdots, v_{q-1}\\}$ with $v_i$ representing the $i$-th variable and $S=\\{s_0,\cdots, s_j, \cdots, s_{p-1}\\}$ with $s_j$ representing the $j$-th constraint, and $q$ and $p$ represent the number of decision variables and constraints, respectively. We apologize for not checking these errors carefully, and have checked the full text for such problems, and highlighted them in the latest submitted PDF.

Reviewer comment (Theory):
Is that possible for two MILP problems with different feasible/optimal solutions to be encoded into the same quantum state with the same VQC parameters?

Author response:
**A6**: Thanks for your question. First of all, it is known that if the feasible/optimal solutions of two MILP problems are different, their MILP-induced graphs are also not the same. Then, we can answer your question by demonstrating that the proposed EQGNN can encode the two different graphs to different representations. Suppose there are two graphs $\mathcal G_1 = (G_1,H_1) =(V_1 \cup S_1, A_1, H_1)$ and $\mathcal G_2 = (G_2,H_2) = (V_2 \cup S_2, A_2, H_2)$, and the differences between the two graphs may appear in 3 places, i.e., the features of nodes, the weight and connectivity of edges. As we mentioned in Definition 4 in the latest submitted PDF, *there are two types of layers in $U_{\boldsymbol \theta}(G, H)$, one is independent of the input graph, and the other is related to the input graph.* In the proposed EQGNN, the feature encoding layer is $U_x^t(G,H,\alpha_t) = U_{x_1}(c,u,b) \cdot  U_{x_2}(\alpha_t) \cdot U_{x_3}(l,\epsilon,\circ) \cdot U_{x_4}(\alpha_t)$, and the message interaction layer is $U^t_{g}(G,\beta) = U_{g_1}(G,\beta_{t,1}) \cdot U_{g_2}(G,\beta_{t,2}) \cdot U_{g_3} (\beta_{t,3}) \cdot U_{g_1}(G,\beta_{t,4}) \cdot U_{g_2}(G,\beta_{t,5}) \cdot U_{g_4} (\beta_{t,6})$. Among them, only $U_{x_1}(c,u,b)$ and $U_{x_3}(l,\epsilon,\circ)$ are related to the features of nodes, i.e., $H$. And $U_{g_1}(G,\beta_{t})$ and $U_{g_2}(G,\beta_{t})$ are related to the weight and connectivity of edges, i.e., $A$.

* For $A_1 = A_2, H_1 \neq H_2$, the only difference between two quantum circuits is $U_{x_1}(c,u,b)$ and $U_{x_2}(l,\epsilon,\circ)$. Taking $U_{x_1}(c,u,b)$ for example, $U_{x_1}(c,u,b) = \text{exp}(-\textbf{i}(\sum_{i=0}^{q-1}(c_i X_{2i}+u_i X_{2i+1}) + \sum_{j=0}^{p-1} b_j X_{2q+j}))$. If any feature changes, $U_{x_1}$ will change, causing $U_{\theta}(G_1,H_1)$ and $U_{\theta}(G_2,H_2)$ to be different.

* For $A_1 \neq A_2, H_1 = H_2$, there are two cases where the connections of the edges are different or the weights of the edges are different. We have already demonstrated the former in [A2: Response to Weakness 2](https://openreview.net/forum?id=KbvKjpqYQR&noteId=dSaFrTi4MX), and now we focus on the latter. $U_{g_1}(G,\beta) =  \text{exp}(-\textbf{i}(\sum_{(i,j)\in \mathcal E} (A_{i,j}+\beta) Z_{2i}Z_{2q+j}))$. The change of $A_{i,j}$ will lead the change of $U_{g_1}(G,\beta)$, causing $U_{\theta}(G_1,H_1)$ and $U_{\theta}(G_2,H_2)$ to be different.

Reviewer comment (Novelty):
Node permutation equivariance or invariance comes naturally with graph representations. The authors need to clearly state whether the claimed generalizability, expressiveness, and efficiency should be attributed to EQGNN or VQC? What are the main technical contributions? How shall they be positioned in the context of Table 1? Also, why the mentioned reference Schatzki et al. (2022) was not included in Table 1?

Author response:
**A7**: On your first question, I may not have fully understood what you meant. Firstly, it needs to be stated that not every VQC is permutation-equivariant. [Figure R1](https://postimg.cc/qhCCzp3P) is a (Hardware efficient ansatz) HEA, which is an intuitive example of failure to satisfy permutation equivariance. As we can see, the output will not be the same when the order of nodes changes. In contrast, we prove the permutation equivariance of our approach in section 3.6 of the latest uploaded PDF. Moreover, if the model is not equivariant, it will overfit the variable/constraint orders in the training data, directly resulting in a decline in generalization performance and expressiveness. As shown in Table 1 of our previous paper, not every existing quantum graph neural network conforms to equivariance, and we make a comparison with QGCN in experiment section 4.8. The experiment shows that the performance of the model without equivariance will be worse. In Tabel 1, QGNN and EQGC are equivariant, but QGNN does not consider the feature encoding of nodes and edges, and EQGC does not consider the information encoding of edges, so they cannot be directly applied to the MILP task. In other words, the generalization and expressiveness brought by our model on the MILP task is unique. 

On the second and third questions, Table 1 shows that our model (1) conforms to permutation equivariance, (2) can encode both node and edge features, and (3) can handle both node-level and graph-level tasks. More specifically, compared to these existing QGNNs, the main technical contributions of our approach are (1) the extension of multi-qubit encoding node features to address the high-dimensional feature coding problem, (2) the addition of an optional auxiliary layer to control model expressiveness, and (3) the association of a shared learnable parameter for each feature, and the use of repeated coding mechanisms. We have incorporated the above description into our revised paper. Thank you for your suggestions for making our article clearer.

On the last questions, the role of Table 1 is to compare existing quantum **graph** neural networks, but Schatzki et al. (2022) are not specialized for graph tasks. As we discuss in Appendix A, their equivariant QNNs can used to learn various problems with permutation symmetries abound, such as molecular systems, condensed matter systems, and distributed quantum sensors (Peruzzo et al., 2014; Guo et al., 2020). Moreover, they also haven't made any specific design for the high-dimensional features and edges of the graph, and they cannot be directly used for MILP tasks.

Reviewer comment (Reproducibility):
The author should clearly describe how the VQC learning problems are formulated for MILP feasibility/optimal solutions/optimal values based on the proposed EQGNN respectively either in the main text or appendix. The author should also clearly specify how the binary measurements can be used to approximate the optimal value which is often a non-discrete value. If multiple measurements are performed to recover the approximated quantum state of the VQC, the author should also clearly specify how many measurements are needed for the results provided in experiment sections.

Author response:
**A5**: Thank you for your suggestion. Next, we provide a detailed explanation and add it to our revised paper. First of all, it needs to be noted that our final measurement phase uses Pauli-Z measurement on the single qubit. That is, *there is no need to* recover the output quantum state of the entire quantum circuit by techniques like quantum state tomography, which may require an exponentially large number of measurements. 

(to be continued)

Reviewer comment (Theory):
How expressive is the proposed model?

Author response:
**A8**: Thanks for your question. We use a tool presented in [The Power of Quantum Neural Networks](https://www.nature.com/articles/s43588-021-00084-1) to quantify the expressiveness of our model. The authors introduce the **effective dimension** as a useful indicator of how well a particular model will be able to perform on the dataset. In particular, this algorithm follows 4 main steps:
1. Monte Carlo simulation: the forward and backward passes (gradients) of the neural network are computed for each pair of input and weight samples.
2. Fisher Matrix Computation: these outputs and gradients are used to compute the Fisher Information Matrix.
3. Fisher Matrix Normalization: averaging over all input samples and dividing by the matrix trace.
4. Effective Dimension Calculation: according to the formula [in this picture](https://postimg.cc/MM0g2XXb) (it is unable to display in this rebuttal box).

We test the proposed model with two blocks on the MILP dataset and use the random weight to evaluate the normalized effective dimension. As shown in [Figure R2](https://postimg.cc/Wd6CrtPJ), with the increase in the number of data, the normalized effective dimension of the proposed model can converge to near 0.75, which shows that our model can achieve good expressiveness even with two blocks. We have added the above to the Appendix of our latest PDF.

We hope that the above answers can address your concern. If you have any further questions, please contact me again. I am looking forward to your reply.

Reviewer comment (Theory):
In section 3.6, the authors claim that "we can prove that the whole model conforms to permutation equivariance by ensuring that each layer conforms to equivariance". However, I was not able to find a theorem statement in the PDF (including appendices).

Author response:
**A1:** Thanks for your precious feedback. We have rewritten Section 3.5 and 3.6 and added formulas to Section 3.2 and 3.3. We formulate the proposed model and present four definitions and three theorem statements to prove the permutation equivariance of our proposed model. Since this part is about two and a half pages long, we will not show it in this reply box, please refer to the highlighted section of [our latest uploaded PDF](https://openreview.net/pdf?id=KbvKjpqYQR).

Reviewer comment (Evaluation):
Many plots use the "rate of errors" as a performance metric. How is this "rate of errors" defined and evaluated in the experiments? Does it require the ground truth of the tested MILP instances? How to get the ground truth?

Author response:
**A2:** "Rate of errors" is the performance metric of predicting feasibility, and it requires the ground truth of the tested MILP instances. Each MILP instance has a label $y_{fea}$ to represent its feasibility, and $y_{fea}=0$ indicates that the MILP is infeasible, and $y_{fea}=1$ indicates the MILP is feasible. Suppose that the predicted result of our model is $\hat{y}_{fea}$, and we set an indicator function 

$\mathbb I_{\hat{y}_{fea}>1/2}$

to represent the predicted feasibility. Then,  
$\text{Rate of errors} = \frac{1}{M}(\sum_{m=0}^{M-1} y^m_{fea} · \mathbb I^m_{\hat{y}_{fea}>1/2}),$
where $M$ indicates the number of tested MILP instances, and the metric represents the number of correct predictions for feasibility. In addition, the ground-truth is solved by PySCIPOpt, which is a Python interface to the SCIP optimization software. We apologize for not writing this part clearly, and we have added the above description to our latest submitted PDF. Thanks for your suggestion.

Reviewer comment (Experiments):
From the numerical experiments, it appears that an EQGNN deployed on a small- to intermediate-size parametric quantum circuit outperforms a traditional GNN-based model with many more parameters. Can we directly employ this new GNN design on a classical computer to outperform other models requiring similar classical computing resources?

Author response:
**A3**: We can use TorchQuantum (a Pytorch-based library) to simulate the designed EQGNN on classical computers. Take predicting feasibility of unfoldable dataset as an example, we test our model EQGNN and GNN on the same machine (4 physical CPUs with 224 cores Intel(R) Xeon(R) Platinum 8276 CPU @ 2.20GHz, and a GPU (NVIDIA A100)). We adjust the hyperparameters of EQGNN and GNN separately to achieve their best performance, and then compare their computing resource usage.

|                  | EQGNN (blocks=8) | GNN(embeddig size=6) |
| ---------------- | ---------------- | -------------------- |
| GPU Memory Usage | 1219 MiB         | 695 MiB              |
| Average Runtime  | 0.2699 s          | 0.0311 s               |
| Rate of error (test set) |  0.0765      |    0.0888       |

As we can see, EQGNN requires 2x GPU Memory Usage and nearly 9x time to outperform GNN, because the simulating quantum algorithms usually involves more complex matrix operations. It is well known that quantum algorithms still cannot be efficiently simulated by classical computers, which is also seen as a quantum advantage. On the other hand, the GNN using more computing resources will not have better results than GNN (embedding size=6).

Reviewer comment (Novelty):
Is the equivariance feature unique to quantum parametric circuits? If not, I wonder if it's possible to obtain a quantum-inspired equivariance GNN for MILPs that is native to classical computing architecture but still outperforms traditional GNN models.

Author response:
**A4**: Thanks for your question. The equivariance feature is not unique to quantum parametric circuits, and it requires careful design, as shown in section 3.6 of our revised paper. Moreover, we believe it is a promising research direction to design a quantum-inspired equivariance GNN that can outperform traditional GNN models.