Paper: Rethinking the symmetry-preserving circuits for constrained variational quantum algorithms (ICLR-2024)

Reviewer comment (Writing):
Typo: "Hartree-Fork"

Author response:
**Ans:** Thanks for your comments. We will answer the above two questions and weaknesses together.

As suggested by Reviewer ADjN, we have provided an analysis on the trainability of HW preserving ansatz (see the answer for weakness 3 for ADjN for detailed derivation). We initially omit the analysis on the trainability since [1] has provided one derivation. However, after careful review, we believe the results in [1] are incorrect and thus we provide our own derivation. We would like to restate the contributions of this work:
1. We revisit the existing works on HW preserving and point out the limitations of these works.
2. We connect various theories to guide the design of HW preserving ansatz. We first utilize the quantum optimal control theory to analyze the capability and expressivity of HW preserving ansatz to see whether the answer to a problem is within the reachable space of the ansatz. We then provide analysis on the trainability of the HW preserving ansatz to see whether we can find a path to reach the answer if it is guaranteed to be in the reachable space by the previous step. Finally, we utilize the overparameterization theory to indicate the approximate number of layers we need to solve the problem. 
3. We conducted detailed numerical experiments on constrained VQAs, which are better test cases than [1]. Two tasks involve symmetry-preserving ground state energy estimation (well-studied case) and feature selection (well-studied and interested by DWave [2]).

To the best of our knowledge, we are the first to combine so many theories to provide a thorough analysis of a quantum ansatz. Compared to previous HW preserving papers such as [1], we rectify the theoretical analysis and the utility of the HW preserving ansatz beyond their currently limited applications. Compared to previous works on hard constrained QAOAs [3], we provide a tool for constructing and analyzing the whole group of HW preserving ansatz instead of sieving through existing gates to seek HW preserving properties and use numerical results to decide how to design circuits. 

To sum up, HW preserving is not new, but it is never thoroughly analyzed and is often used on unsuitable problems. The theoretical results might not be surprising to you but we never know why certain gates can not solve a given problem and whether the bad results are caused by the fact that the answer is unreachable or poor trainability of the ansatz or lack of layers. We strongly believe that this work can help the QML community by improving the understanding of the HW preserving ansatz. [4] is also a paper that aims to improve the understanding of the community by mostly reviewing the ideas and is accepted as a spotlight paper in NeurIPS24. We hope that this work can also be valued and attract more attention to HW preserving circuits.


[1] Cherrat, El Amine, et al. "Quantum Deep Hedging." arXiv preprint arXiv:2303.16585 (2023).

[2] https://github.com/dwave-examples/feature-selection-notebook/blob/master/01-feature-selection.ipynb

[3] Hadfield, Stuart, et al. "From the quantum approximate optimization algorithm to a quantum alternating operator ansatz." Algorithms 12.2 (2019): 34.

[4] Abbas, Amira, et al. "On quantum backpropagation, information reuse, and cheating measurement collapse." arXiv preprint arXiv:2305.13362 (2023).

Reviewer comment (Experiments):
Based on common gate sets of real machines, I feel the BS gate is quite artificial. It might make more sense to use parameterized $XX$ rotations with single-qubit $Z$ rotations. If the intention is to perform VQAs on near-term machines, then one might want to use more native instructions than the decompositions shown in Fig 5 and Fig 6.

Author response:
**Ans:** Thanks for your valuable suggestion. We do acknowledge that the BS gate is quite 'artificial' since we are trying to construct a single gate that can be universal with both Nearest-Neighbor and Full-Connect connectivity. One of the key factors for HW preserving gate is that the Hamiltonian for the two-qubit parameterized gate is $$H_{HW}=\left(\begin{array}{cccc}
        0 & 0 & 0 & 0\\\        0 & a & b & 0\\\        0 & b^\dagger & c & 0\\\        0 & 0 & 0 & 0
    \end{array}\right).$$ 

It is still quite hard to construct a legit Hamiltonian using XX, YY, and ZZ rotations since only ZZ rotations are HW preserving. 

We could further use the combination of XX, YY, and ZZ rotations to construct HW preserving gates. As mentioned by Reviewer ADjN, XY-mixer is another well-studied HW preserving gate. The hamiltonian of XY-mixer is $H_{XY}=X\otimes X+Y\otimes Y$, and the unitary matrix of XY-mixer is 
$$U_{XY}=\left(\begin{array}{cccc}
        1 & 0 & 0 & 0\\\        0 & \cos (\theta) & -\text{i}\sin (\theta) & 0\\\        0 & -\text{i}\sin (\theta) & \cos (\theta) & 0\\\        0 & 0 & 0 & 1
    \end{array}\right).$$
    
We would like to point out that **the decomposed circuit of those HW preserving gates constructed with XX, YY, and ZZ is not very simple.** Take XY-mixer as an example. Although using XX+YY to construct the Hamiltonian is very elegant, the unitary matrix of the XY-mixer is not very easy to decompose (even if we have iSWAP gate on superconducting hardware). **RBS gate, on the other hand, can be a built-in gate on photonic quantum computers.** Therefore, those that are difficult to decompose on superconducting hardware might be very easy to fulfill in other quantum systems. 

As we have stated right before section 5, despite the fact that RBS and XY-mixer do not have full DLA dimension, if we pick the right problem, they can also be better choices than BS gate. We could also use the combination of RBS and RZZ gates to surpass BS-gate-only circuits. BS gate is only an example to illustrate how to construct a single HW preserving gate with full DLA even under harsh conditions (nearest neighbor connectivity for physical qubits)

Reviewer comment (Theory):
Related to the above point, the decomposition of the BS gate is addressed as a possible limitation. I am pretty sure a better decomposition is possible using $XX$, $YY$ and $ZZ$ rotations.

Author response:
**Ans:** Thanks for your thoughtful advice. We are seeking a better decomposition of the BS gate to make it more hardware-friendly. However, it seems hard to use XX, YY, and ZZ to decompose the Hamiltonian of the BS gate. The Hamiltonians for XX, YY, and ZZ are
$$H_{XX}=\left(\begin{array}{cccc}
        0 & 0 & 0 & 1\\\        0 & 0 & 1 & 0\\\        0 & 1 & 0 & 0\\\        1 & 0 & 0 & 0
    \end{array}\right),\ H_{YY}=\left(\begin{array}{cccc}
        0 & 0 & 0 & -1\\\        0 & 0 & 1 & 0\\\        0 & 1 & 0 & 0\\\        -1 & 0 & 0 & 0
    \end{array}\right),\ H_{ZZ}=\left(\begin{array}{cccc}
        1 & 0 & 0 & 0\\\        0 & -1 & 0 & 0\\\        0 & 0 & -1 & 0\\\        0 & 0 & 0 & 1
    \end{array}\right),\ $$
    
It seems that we can only construct HW preserving gates with $a=c$ and $b=b^\dagger$. These gates such as XY-mixer can not reach full DLA dimension with Nearest-Neighbor connectivity. There is a chance that we misunderstood the weakness brought up by the reviewer so we are eager to hear back from you.

Reviewer comment (Novelty):
I think this idea of preserving the Hamming weight is not very new, and the theoretical results are not very surprising to me. For this reason, I feel the contribution of this work is not so large, although the proposed ansatz appear to perform better than existing ones.

Author response:
**Ans:** Thanks for your comments. We will answer the above two questions and weaknesses together.

As suggested by Reviewer ADjN, we have provided an analysis on the trainability of HW preserving ansatz (see the answer for weakness 3 for ADjN for detailed derivation). We initially omit the analysis on the trainability since [1] has provided one derivation. However, after careful review, we believe the results in [1] are incorrect and thus we provide our own derivation. We would like to restate the contributions of this work:
1. We revisit the existing works on HW preserving and point out the limitations of these works.
2. We connect various theories to guide the design of HW preserving ansatz. We first utilize the quantum optimal control theory to analyze the capability and expressivity of HW preserving ansatz to see whether the answer to a problem is within the reachable space of the ansatz. We then provide analysis on the trainability of the HW preserving ansatz to see whether we can find a path to reach the answer if it is guaranteed to be in the reachable space by the previous step. Finally, we utilize the overparameterization theory to indicate the approximate number of layers we need to solve the problem. 
3. We conducted detailed numerical experiments on constrained VQAs, which are better test cases than [1]. Two tasks involve symmetry-preserving ground state energy estimation (well-studied case) and feature selection (well-studied and interested by DWave [2]).

To the best of our knowledge, we are the first to combine so many theories to provide a thorough analysis of a quantum ansatz. Compared to previous HW preserving papers such as [1], we rectify the theoretical analysis and the utility of the HW preserving ansatz beyond their currently limited applications. Compared to previous works on hard constrained QAOAs [3], we provide a tool for constructing and analyzing the whole group of HW preserving ansatz instead of sieving through existing gates to seek HW preserving properties and use numerical results to decide how to design circuits. 

To sum up, HW preserving is not new, but it is never thoroughly analyzed and is often used on unsuitable problems. The theoretical results might not be surprising to you but we never know why certain gates can not solve a given problem and whether the bad results are caused by the fact that the answer is unreachable or poor trainability of the ansatz or lack of layers. We strongly believe that this work can help the QML community by improving the understanding of the HW preserving ansatz. [4] is also a paper that aims to improve the understanding of the community by mostly reviewing the ideas and is accepted as a spotlight paper in NeurIPS24. We hope that this work can also be valued and attract more attention to HW preserving circuits.


[1] Cherrat, El Amine, et al. "Quantum Deep Hedging." arXiv preprint arXiv:2303.16585 (2023).

[2] https://github.com/dwave-examples/feature-selection-notebook/blob/master/01-feature-selection.ipynb

[3] Hadfield, Stuart, et al. "From the quantum approximate optimization algorithm to a quantum alternating operator ansatz." Algorithms 12.2 (2019): 34.

[4] Abbas, Amira, et al. "On quantum backpropagation, information reuse, and cheating measurement collapse." arXiv preprint arXiv:2305.13362 (2023).

Reviewer comment (Theory):
In Equation 8, should the maximization be over $M$ such that $M\geq M_c$?

Author response:
**Ans:** We apologize for the mistake. We will correct this in the revision.

Reviewer comment (Theory):
Where does the construction for the BS gate come from?

Author response:
**Ans:** Thanks for your comments. The construction for the BS gate comes from the general Hamiltonian form of the HW preserving gate, which is 
$$
H_{HW}=\left(\begin{array}{cccc}
        0 & 0 & 0 & 0\\\        0 & a & b & 0\\\        0 & b^\dagger & c & 0\\\        0 & 0 & 0 & 0
    \end{array}\right). 
$$
As stated in the paper, we can construct an arbitrary HW preserving gate by deciding the values for $a,b,c$. BS gate is only an example to illustrate a gate with full DLA dimension with both NN and FC connectivity. Considering the fact that $H_{HW}$ is a Hermitian matrix, we have the following situations. For $a$ and $c$, we have $a=c$ or $a\neq c$. For $b$, we have $b=b^\dagger$ where $b$ is a real number or $b=-b^\dagger$ where $b$ only has the imaginary part or $b$ is an arbitrary imaginary number with both real and imaginary parts. We have tried different combinations of the $a,b,c$ and come up with the BS gate to be universal for both connectivity (where $a=c\neq 0$ and $b$ has both imaginary and real parts). 


As commented by Reviewer ADjN, we provide "a bottom-up approach to guide quantum circuit design compared to previous top-down approach." Actually, the existing HW preserving gates, such as RBS gate and XY-mixer, are top-down approaches, where they first come up with a gate (maybe with $H=X\otimes X+Y\otimes Y$) and then explore the properties of whether it is HW preserving. It is just like mining in a huge gate mountain to find some golden gates that are HW preserving. Therefore, we provide a way to theoretically construct an HW preserving gate so that we can have enough samples to have a deeper understanding regarding the HW preserving ansatz.

Reviewer comment (Novelty):
Throughout this paper, it is stated that this is a "revisiting" of the Hamming weight preserving ansatz. What is the status of the previous work using these ansatz, and what are the specific improvements made in this paper compared to previous proposals?

Author response:
**Ans:** Thanks for your comments. We will answer the above two questions and weaknesses together.

As suggested by Reviewer ADjN, we have provided an analysis on the trainability of HW preserving ansatz (see the answer for weakness 3 for ADjN for detailed derivation). We initially omit the analysis on the trainability since [1] has provided one derivation. However, after careful review, we believe the results in [1] are incorrect and thus we provide our own derivation. We would like to restate the contributions of this work:
1. We revisit the existing works on HW preserving and point out the limitations of these works.
2. We connect various theories to guide the design of HW preserving ansatz. We first utilize the quantum optimal control theory to analyze the capability and expressivity of HW preserving ansatz to see whether the answer to a problem is within the reachable space of the ansatz. We then provide analysis on the trainability of the HW preserving ansatz to see whether we can find a path to reach the answer if it is guaranteed to be in the reachable space by the previous step. Finally, we utilize the overparameterization theory to indicate the approximate number of layers we need to solve the problem. 
3. We conducted detailed numerical experiments on constrained VQAs, which are better test cases than [1]. Two tasks involve symmetry-preserving ground state energy estimation (well-studied case) and feature selection (well-studied and interested by DWave [2]).

To the best of our knowledge, we are the first to combine so many theories to provide a thorough analysis of a quantum ansatz. Compared to previous HW preserving papers such as [1], we rectify the theoretical analysis and the utility of the HW preserving ansatz beyond their currently limited applications. Compared to previous works on hard constrained QAOAs [3], we provide a tool for constructing and analyzing the whole group of HW preserving ansatz instead of sieving through existing gates to seek HW preserving properties and use numerical results to decide how to design circuits. 

To sum up, HW preserving is not new, but it is never thoroughly analyzed and is often used on unsuitable problems. The theoretical results might not be surprising to you but we never know why certain gates can not solve a given problem and whether the bad results are caused by the fact that the answer is unreachable or poor trainability of the ansatz or lack of layers. We strongly believe that this work can help the QML community by improving the understanding of the HW preserving ansatz. [4] is also a paper that aims to improve the understanding of the community by mostly reviewing the ideas and is accepted as a spotlight paper in NeurIPS24. We hope that this work can also be valued and attract more attention to HW preserving circuits.


[1] Cherrat, El Amine, et al. "Quantum Deep Hedging." arXiv preprint arXiv:2303.16585 (2023).

[2] https://github.com/dwave-examples/feature-selection-notebook/blob/master/01-feature-selection.ipynb

[3] Hadfield, Stuart, et al. "From the quantum approximate optimization algorithm to a quantum alternating operator ansatz." Algorithms 12.2 (2019): 34.

[4] Abbas, Amira, et al. "On quantum backpropagation, information reuse, and cheating measurement collapse." arXiv preprint arXiv:2305.13362 (2023).

Reviewer comment (Novelty):
The premise of the paper is a bit atypical for conference submissions, in that it isn't "selling" a new model or method developed by the authors. Rather, the goal of the paper seems to be improving the community's understanding regarding the expressivity and performance of HW-preserving circuits in general. I think this is a valuable contribution, but I could see other reviewers pointing to a lack of novelty due to this unorthodox aim.

Author response:
**Ans:** We are grateful for your appreciation of our work. We truly hope that this work can be valued and attract more attention to HW preserving ansatz.

Reviewer comment (Presentation):
Figure 4 compares a large number of different circuit ansatzes, but many of these overlap with each other and are barely visible (e.g. the RBS-full points). It would be helpful to revise this figure to permit easier viewing of all of the baseline results, for example by varying the colors, size, and/or ordering of the plot markers.

Author response:
**Ans:** Thanks for your valuable advice and we have adjusted Figure 4 according to your suggestion. The revised figure is shown in https://anonymous.4open.science/r/ICLR3179/SupplementaryMaterial.pdf.

Reviewer comment (Theory):
Regarding the literature review, it is not clear to me why the authors chose to categorize some previous work based on XY-mixer QAOA [1-2] as soft constraints. Their circuit is composed of a problem Hamiltonian layer (ZZ gates) and a mixer layer, both preserving the Hamming weight instead of adding a penalty to the cost function.

Author response:
**Ans:** We apologize for the misleading description of related works. We cite [1] to illustrate that there are quite a lot of constrained QAOA problems and then we claim that soft constraints could be a solution to these constrained optimization problems. The citation is very close to the following claim but we do not intentionally say the models in [1] are soft-constrained. We do confirm that XY-mixer QAOA is a hard-constrained QAOA, and we will make a clear intro on soft-constrained and hard-constrained models in related works in the revision.

[1] Hadfield, Stuart, et al. "From the quantum approximate optimization algorithm to a quantum alternating operator ansatz." Algorithms 12.2 (2019): 34.

Reviewer comment (Experiments):
One of my biggest concerns is when we run this framework on NISQ hardware or noisy simulation. A bit-flip (X) error would suddenly break the Hamming distance. Will this be a big problem or not? It will be better to see more results along this consideration.

Author response:
**Ans:** Thanks for bringing up this problem which is crucial to improving our work. It is quite interesting that **HW preserving circuit is much easier to be bit-flip fault tolerant**. 
We can utilize one ancilla qubit to do the parity check to detect any bit-flip error on the HW preserving circuit. The Hamming weight $k$ implies whether we have an odd number or even number of $1$s in the circuit. One possible solution of the parity check is to link all the working qubits to an ancilla qubit with CNOT gate with the ancilla qubit as the target bit. We initially set the ancilla qubit to $|0\rangle$. If we have an odd number of $1$s in the working qubits, we will have a $|1\rangle$ on the ancilla qubit and a $|0\rangle$ with an even number of $1$s. Thus, it is much easier to detect a bit-flip error on the HW preserving circuit compared to Shor's code with two ancilla qubits required to protect one qubit from bit-flip error. We can either rerun the circuit or locate the qubit to correct the bit-flip error. This is an exciting and primitive conclusion, and we hope this dissolves your concern.

Reviewer comment (Theory):
Need more discussion on the trainability of the proposed ansatz and Barren Plateau (BP) phenomenon. It makes intuitive sense to think of better trainability compared to HEA since it is only exploring a constrained subspace. The optimization problem should be easier. Previous work [3] showed that a compound layer consisting of FBS gates could lead to a gradient decay only polynomially. Will the same conclusion hold for this work?

Author response:
**Ans:** Thanks for your thoughtful advice. We carefully reviewed the results and proof in [1] after seeing your comments and we are surprised to find it incorrect. There are still chances that we misunderstood the calculation for the variance bound in [1], so we decided to give our results here with a sketch proof. Detailed derivation will be included in the revision. 

We first discuss the results shown in [1]. We tried the derivation of theorem 2 and find that $\partial_lC_k=\langle\psi_0|(\sigma_X^i\otimes\sigma_X^j+\sigma_Y^i\otimes\sigma_Y^j)|\psi_0\rangle$ should be $1$ instead of $\frac{1}{4}$. For the initial state in the HW k subspace, the results in [1] are $\partial_lC_k=\langle\psi_k|(\sigma_X^i\otimes\sigma_X^j+\sigma_Y^i\otimes\sigma_Y^j)|\psi_k\rangle=2(\frac{2\tbinom{n-2}{k-1}}{\tbinom{n}{k}})^2=\Omega(\frac{1}{n^6})$, where according to our derivation, the results should be $\partial_lC_k=\langle\psi_k|(\sigma_X^i\otimes\sigma_X^j+\sigma_Y^i\otimes\sigma_Y^j)|\psi_k\rangle=2\frac{2\tbinom{n-2}{k-1}}{\tbinom{n}{k}}$, $\frac{\tbinom{n-2}{k-1}}{\tbinom{n}{k}}=\frac{k(n-k)}{n(n-1)}$. Even if the original result is correct that $\partial_lC_k=8(\frac{k(n-k)}{n(n-1)})^2$, $\partial_lC_k=\mathcal{O}(1)$ if we take $k=n/2$, which seems incorrect. 


Now we provide the results based on our derivation. Without loss of generality, we take RBS gate as an example since [1] also derives the bound based on RBS gate. 

***Theorem 1. Consider a quantum circuit operating in the subspace with Hamming Weight $k$. The variance of the cost function partial derivative is $$Var_\theta[\partial_lC]\approx \frac{16k^2(n-k)^2}{n^4d_k}$$***

*Proof.* Consider the partial derivative of the cost function $C$ with respect to the parameters $\theta$. For some parameter $\theta_l$ in the $l$-th RBS gate, we have:
$$\partial_{l} C(\theta)=\partial_l(Tr[U(\theta)\rho U(\theta)^{\dagger}O])$$
$$=\partial_l(Tr[U_-\rho U_-^{\dagger} O_+]) $$
$$=iTr[U_-\rho U_-^{\dagger} [H_l,O_+]],$$

where $\rho$ is the input state, $O$ is the observable to measure, $U_-$ denotes the unitary matrix of the circuit before the $l$-th gate and $U_+$ denotes the unitary matrix after gate $l$. $O_+=U_+^\dagger OU_+$. The variance of the partial derivative is 
$$Var_{\theta}[\partial_l C]=\int_{U_+}dU_+ \int_{{U_-}}dU_- (\partial_l C(\theta))^2$$
$$=\int_{{U_+}}dU_+ \int_{{U_-}}dU_- (iTr[U_-\rho U_-^{\dagger} [H_l,O_+]])^2,$$
where $[\cdot,\cdot]$ denotes the commutator of two matrices. 
$$Var_\theta[\partial_lC]=-\int_{{U_+}}dU_+ (\frac{Tr[\rho^2]Tr[[H_l,O_+]^2]}{d_k^2-1}-\frac{Tr^2[\rho]Tr[[H_l,O_+]^2]}{d_k(d_k^2-1)})$$
$$=-\int_{U_+}dU_+ (Tr[ [H_l,O_+]^2] \frac{d_k\times Tr[\rho^2]-Tr^2[\rho]}{d_k(d_k^2-1)})$$
$$=-\frac{d_k\times Tr[\rho^2]-Tr^2[\rho]}{d_k(d_k^2-1)}\int_{{U_+}}dU_+ Tr[ [H_l,O_+]^2] .$$
The initial state $|\psi_0\rangle$ is set to be the uniform superposition over all computational basis in the $d_k$ subspace. Thus, we have $Tr[\rho]=1,Tr[\rho^2]=1$.
$$Var_\theta[\partial_lC]=-\frac{1}{d_k(d_k+1)}\int_{\mathcal{U_+}}dU_+ Tr[ [H_l,O_+]^2]$$
$$=-\frac{2}{d_k(d_k+1)}[Tr(H_lO_+H_lO_+)-Tr(H_lH_lO_+O_+)]$$
$$=-\frac{2}{d_k(d_k+1)}(\frac{Tr[H^2_l]Tr^2[O]}{d_k^2-1}-\frac{Tr[H^2_l]Tr[O^2]}{d_k(d_k^2-1)}-\frac{Tr[H^2_l]Tr[O^2]}{d_k})$$
$$=-\frac{2Tr[H_l^2]}{d_k(d_k+1)}(\frac{Tr^2[O]-d_k Tr[O^2]}{d_k^2-1}),$$
where $Tr[H_l^2]=2*\binom{n-2}{k-1}=\frac{2k(n-k)}{n(n-1)}d_k$. We take $Z_0$ as the observable, then $Tr[O]=\frac{d_k(n-2k)}{n}, Tr[O^2]=d_k$ (other observables will also hold with the same magnitude). Thus, we have
$$Var_{\theta}[\partial_l C]=-\frac{2}{d_k(d_k+1)}\times\frac{2k(n-k)d_k}{n(n-1)}\times(\frac{\frac{d_k^2(n-2k)^2}{n^2}-d_k^2}{d_k^2-1})$$
$$=\frac{4k(n-k)}{(d_k+1)n(n-1)}\times\frac{d_k^2(n^2-(n-2k)^2)}{(d_k^2-1)n^2}$$
$$=\frac{4k(n-k)}{(d_k+1)n(n-1)}\times\frac{d_k^2(4nk-4k^2)}{(d_k^2-1)n^2}$$
$$=\frac{16k^2(n-k)^2 d_k^2}{(d_k+1)n^3(n-1)(d_k^2-1)}\approx \frac{16 k^2(n-k)^2}{n^4 d_k}.$$ $\blacksquare$

We can further analyze that if the $k$ is only 1, then $Var_\theta[\partial_lC]\approx \frac{16}{n^3}$. If the $k=\frac{n}{2}$ on the other hand, $Var_\theta[\partial_lC]\approx \binom{n}{n/2}^{-1}$, which is approximate to exponentially small. This result is consistent with the conjecture that the trainability of the circuit is closely related to $d_k$ and smaller $d_k$ will lead to better trainability. We will make sure a detailed version of the analysis on the trainability will be included in the revision.


[1] Cherrat, El Amine, et al. "Quantum Deep Hedging." arXiv preprint arXiv:2303.16585 (2023).

Reviewer comment (Experiments):
Some numerical experiment details don't make sense to me. For example, why did the authors choose the penalty weight in Eq. 16 as \alpha = [0.5, 1, 5, 10]. To my best knowledge, people usually set this empirical parameter to a much larger value like 100.

Author response:
**Ans:** Thanks for pointing out this problem, and we agree with you that more results should be included to provide a comprehensive view of the performance. We conduct the experiments on much larger $\alpha$ and the results are shown in the following table. We will fill up these new results in Table 3 in the revision.



|              | BS   | RBS  | $\alpha=10$ | $\alpha=50$ | $\alpha=100$ | $\alpha=1000$ |
| ------------ | ---- | ---- | ----------- | ------------ | ------------ | ------------ |
| Wine Quality | **1.136342** | 1.135580 | 1.099222 | 1.001571 | 0.983495 | 0.997417 |
| Heart Disease | **3.418328** | 3.418183 | 2.963659 | 2.792978 | 2.697520 | 2.717723 |
| Titanic | **1.197799** | 1.197159 | 0.936198 | 0.929956 | 0.916599 | 0.923464 |
| Dry Bean | 8.616844 | **8.616916** | 7.790612 | 7.928807 | 7.864919 | 7.870827 |

Reviewer comment (Novelty):
It would be nice if the author could discuss connections of their HW preserving VQE with XY-mixer QAOA or other constrained QAOA. And maybe apply the theoretical tools used here to improve those work?

Author response:
**Ans:** Thanks for your thoughtful suggestion. The XY-mixer and other mixers are a subset of the proposed HW preserving gates as RBS gates, which we detailly discussed in the literature. The Hamiltonian of the XY-mixer is 
$$
H_{XY}=\begin{bmatrix}
        0 & 0 & 0 & 0\\\0 & 0 & 1 & 0 \\\        0 & 1 & 0 & 0\\\0 & 0 & 0 & 0 
\end{bmatrix}
,\quad\quad H_{XY}=\begin{bmatrix}
        0 & 0 & 0 & 0 \\\        0 & a & b & 0 \\\        0 & b^\dagger & c & 0 \\\        0 & 0 & 0 & 0
    \end{bmatrix}$$ which is a special case of the general HW preserving gates with $b=1$. The unitary matrix of the XY-mixer is $$U_{XY}=\begin{bmatrix}
        1 & 0 & 0 & 0\\\        0 & \cos (\theta) & -\text{i}\sin (\theta) & 0\\\        0 & -\text{i}\sin (\theta) & \cos (\theta) & 0\\\        0 & 0 & 0 & 1
    \end{bmatrix}.$$
    We can utilize the pipeline in this work to analyze the XY-mixer. For the NN connectivity, the dimension of DLA of the XY-mixer is 
    $$dim(g_{XY})=(n+1)(n-1) \qquad n \text{ is odd}$$
    $$\ \ \quad\qquad \qquad\frac{1}{2}n(n-1) \quad\qquad n \text{ is even.}$$
    
(This is a piecewise function, but it is hard to write in the markdown. Sorry for the inconvenience!) For the FC connectivity, the dimension of DLA of the XY-mixer is
    
$$dim(\mathfrak{g}_{XY})= (d_k+1)(d_k-1) \quad n \neq 2k$$
$$\qquad\quad\qquad\frac{1}{2}(d_k+2)(d_k-2) \quad n = 2k. $$
    
Thus, we can conclude that XY-mixer with full connectivity is quite capable of solving most of the problems except for some rare cases and maybe that is exactly why it is so famous for solving constrained QAOA. However, the XY-mixer on NN-connectivity is not very good even with the RZZ gate as the phase operator (the dimension of DLA is still not full). Therefore, we can conclude that the XY-mixer might not be so capable considering the physical qubit topology. One possible solution to this is to combine other gates to increase the initial generators and gain full controllability under both connectivities.


[1] Hadfield, Stuart, et al. "From the quantum approximate optimization algorithm to a quantum alternating operator ansatz." Algorithms 12.2 (2019): 34.

Reviewer comment (Reproducibility):
What is the basis set used in ground-state energy estimation (such as STO-3G) to discretize the computation space?

Author response:
**Ans:** We do use STO-3G as the computational basis and we will clarify this in the revision, thanks for pointing out this problem.

Reviewer comment (Experiments):
I believe the second test case, feature selection, is not a typical benchmark for VQAs. Is this a classically hard problem? Can you say more about why choose this test case?

Author response:
**Ans:** Thanks for your comment. We choose feature selection as one of our test cases for the following three reasons:
1. Feature selection is a well-studied (2624 citations for [1], 4026 citations for [2], 4926 citations for [3], etc.) problem for the classical machine learning community. 
2. We already have one VQA problem and we are seeking more diversity by choosing feature selection instead of all the problems listed in [4]. 
3. D-Wave has focused on this problem to demonstrate quantum supremacy on classical machine learning problems with their hardware recently [5]. We believe feature selection is a candidate to fulfill quantum supremacy as a QUBO problem utilizing quantum annealing.

We will further explain these issues in the revision to give the readers a better understanding. Thanks again for your valuable questions and suggestions to help us improve the manuscript.