Exploring channel distinguishability in local neighborhoods of the model space in quantum neural networks
Authors: Sabrina Herbst
Sandeep Suresh Cranganore
Vincenzo De Maio
Ivona Brandić
Venue: ICLR 2025 Poster (2025)
Keywords: Quantum Machine Learning
Quantum Neural Network
Primary area: other topics in machine learning (i.e., none of the above)

Abstract: With the increasing interest in Quantum Machine Learning, Quantum Neural Networks (QNNs) have emerged and gained significant attention. These models have, however, been shown to be notoriously difficult to train, which we hypothesize is partially due to the architectures, called ansatzes, that are hardly studied at this point. Therefore, in this paper, we take a step back and analyze ansatzes. We initially consider their expressivity, i.e., the space of operations they are able to express, and show that the closeness to being a 2-design, the primarily used measure, fails at capturing this property. Hence, we look for alternative ways to characterize ansatzes, unrelated to expressivity, by considering the local neighborhood of the model space, in particular, analyzing model distinguishability upon small perturbation of parameters. We derive an upper bound on their distinguishability, showcasing that QNNs using the Hardware Efficient Ansatz with few parameters are hardly discriminable upon update. Our numerical experiments support our bounds and further indicate that there is a significant degree of variability, which stresses the need for warm-starting or clever initialization. Altogether, our work provides an ansatz-centric perspective on training dynamics and difficulties in QNNs, ultimately suggesting that iterative training of small quantum models may not be effective, which contrasts their initial motivation.

Published as a conference paper at ICLR 2025
EXPLORING CHANNEL DISTINGUISHABILITY IN LO- CAL NEIGHBORHOODS OF THE MODEL SPACE IN QUAN- TUM NEURAL NETWORKS
Sabrina Herbst Sandeep
Sandepp Suresh Cranganore Vincenzo De Maio && Ivona Brandić HPC Research Group, Faculty of Informatics, TU Wien, Vienna, Austria
2 Peter Grünberg Institute-8, Forschungszentrum Jülich GmbH, Juelich, Germany
ABSTRACT
With the increasing interest in Quantum Machine Learning, Quantum Neural Net- works (QNNs) have emerged and gained significant attention. These models have, however, been shown to be notoriously difficult to train, which we hypothesize is partially due to the architectures, called ansatzes, that are hardly studied at this point. Therefore, in this paper, we take a step back and analyze ansatzes. We initially consider their expressivity, i.e., the space of operations they are able express, and show that the closeness to being a 2-design, the primarily used mea- sure, fails capturing this property. Hence, acterize ansatzes, unrelated to expressivity, by considering the local neighborhood the model space, in particular, analyzing model distinguishability upon small perturbation of parameters. We derive upper bound on their distinguishability. showcasing that QNNs using the Hardware Efficient Ansatz with few parame- ters are hardly discriminable upon update. Our numerical experiments support our bounds and further indicate that there is a significant degree variability, which stresses the need for warm-starting or clever initialization. Altogether, work provides an ansatz-centric perspective on training dynamics and difficulties in QNNs, ultimately suggesting that iterative training of small quantum models may not be effective, which contrasts their initial motivation.
look for alternative ways char-
INTRODUCTION
With the increasing computational requirements of sthee-of-the-art ML models, the limitations of classical hardware, as theorized by Moore's law (Shali 2020), are becoming increasingly preva- lent (Schulz et al. 1 Reed et al.) 2022). Therefore, alternative computing paradigms are heavily studied and the immense potential of Quantum Computing, combined with recent advances in quan- tum hardware, have made Quantum Machine Learning (QML) promising candidate.
Within that framework, Quantum Neural Networks (QNNs) are heavily studied, as they are suitable for near-term hardware. Besides initial successes, however, the Barren Plateau phenomenon has been proven under various circumstances, meaning that gradients vanish exponentially in the number of qubits, leading to essentially flat loss landscapes that affect trainability (Cerezo al. 202 a))
In light of these issues, we hypothesize that the architectures, called ansatzes, are part of the problem. QNN ansatzes consist of fixed and trainable gates, however, due to hardware limitations, currently mainly so-called Hardware Efficient Ansatzes (HEAs) (Kandala et al. (017) are used, meaning they are built primarily based on hardware constraints. Therefore, we hypothesize that executing the circuit does not lead to meaningful feature representation.
Currently, is unclear how individual operations affect the model space, or, when including data, the data representation. There are no standardized ansatzes for Machine Learning (ML) and, due to the many degrees freedom, ansatz tuning poses an optimization problem with intractable search
maabrina.heraann www.shttttwwwe..aac.o. ac. I vinennzeeecc tuwien. www.eeewwwwt.......ooo.. ac

---

Published as a conference paper at ICLR 2025
space. Further, there is little research the ansatzes themselves, i.e., independently of data and loss, and is non-obvious how feature extraction works in a QNN.
In particular, in classical ML, vanilla CNN is ultimately designed to consider local features in lower layers, before considering global features in deeper layers, independently of the dataset. Transform- (Vaswani et al. 2017) work similarly, however, the trainable one-qubit and fixed two-qubit gates that are used in NN do not allow drawing intuitive conclusions about how features are extracted.
Therefore, the goal of this paper is to analyze and characterize QNN architectures. We aim to take a first step at exploring ansatzes and the model space, initially by considering the expressivity of ansatzes. Using tools from Quantum Information Theory and extensive numerics, later analyze how the quantum circuit, i.c., the applied operation, changes upon parameter update. thus, providing insights into the model space beyond ansatz expressivity. Our contributions are as follows.
Based on literature, we argue that closeness to a 2-design, a widely used measure for ansatz expressivity, does not adequately represent the possible set of unitary operations.
To investigate the model space, study how ansatz changes upon parameter update for different HEAs. In particular, we introduce measure for the distinguishability of an ansatz upon parameter update and provide upper bound. It shows that many parameters required to be able to distinguish the two operations, contrasting with the initial framework.
We provide evidence that HEAs are hardly distinguishable upon parameter update in (I), random perturbations and (2), during training, even in early stages with larger updates.
While not strictly equal, the observed behavior and effects thereof have remarkable simi- larities to Barren Plateaus, insinuating two perspectives on similar trainability issues. The results suggest that the architectures, independently of data and loss, may be flawed and play a fundamental role in the trainability problems observed today.
The paper is structured follows. In Section 2 we provide background information QNNs, existing expressivity measures and related work In Section 3 we discuss the closeness to being 2-design as measure of expressivity and Section 4 introduces measure of distinguishability of quantum channels. We elaborate on our experimental setup in Section 3 and present our results in Section 6 Section discusses implications of our results and Section concludes the paper.
BACKGROUND
2.1
QUANTUM NEURAL NETWORKS
The small systems and the noise in today's Noisy Intermediate-Scaal Quantum (NISQ) technol- ogy (Preskill 2018), prevents execution of large-scale circuits. Consequently, so-called Variational Quantum Circuits (VQC) are applied, which are hybrid models with classical and quantum parts.
Simply put, a QNN consists of (1) an input state |/xx), (2) an ansatz, U()), (3) a loss function, and (4) an optimizer. The features encoded onto the quantum state by means of unitary feature
encoding method V(X,, and the initial state is obtained (xxx) V(X) 10y On with as the number of qubits.
HEAS consists of I. layers of trainable one-qubit rotation gates V,(),) and fixed two-qubit entangling gates W,, with E [1, L). expressed as Equation A measurement is taken in the end, which is done through Hamiltonian observable Ô, and the result is expressed as the expectation value of the ansatz with respect to ((x)) and Ó, as shown in Equation
L U(0) II V.O.)W. = V2 (i 1=1
(1)
f(x, 0) = (Ux(U)())
(2)
We refer Appendix A.1 for brief overview of Quantum Computing.
2

---

Published as a conference paper at ICLR 2025
The loss is calculated with classical loss function (e.g,, the nqua-e error) on the predicted value (the expectation value of the QNN) and the target value, and passed to the classical optimizer. The optimizer computes the parameter updates and the circuit is again executed on the quantum computer, until some stopping criterion is fulfilled. As the optimization is done classically, this approach has the advantage of allowing shallow quantum eircuit, which limits the propagation of errors during execution in light of the significant amount of noise in today's 's hardware.
While HEAs are shallow, they are highly expressive (Holmes et al. 2022), which is main promise of QML stemming from the significantly larger Hilbert space. Recent works, however, uncovered that loss landscapes of today's QNNs are plagued by Barren Plateaus (BPs), meaning exponentially vanishing gradients in problem size, leading to untrainable models (McClean et al. 2018).
2.2 ANSATZ EXPRESSIVITY
We acknowledge a missing clear definition of expressivity in QML for now. It is sensible, therefore, to transfer it from classical MI., where model expressivity is the range of functions a model can compute (Raghu et al. 2017). Hence, it is a function of architecture A, input x and parameters A FA (2; )). On the contrary, our work lies in data-agnostic characterization of models. To this end, will only consider ansatz expressivity, the set of functions generated solely by ansatz itself.
The most commonly used method to evaluate ansatz expressivity is to compare the probability distri- bution of the unitaries of the ansatz to the Haar measure the uniform distribution over all unitaries U(d) (Cerezo et al. 2021a). a). If the probability distribution of the ansatz follows the Haar measure up to the t-th moment, it is considered r-design. Their construction requires exponential time (Harrow) & Low 2009). however, approximate t-designs can be built efficiently (Dankert et al. 2009).
Closeness 2-design state by (Sim et al.] (2019). In Holmes et al.] (2022), it is expanded to a specific input state (i.e., with encoded data) and the measurement operator, as shown in Equation There, an ansatz is viewed as an ensemble of unitary transformations U = {U (0). vO E ) over all possible parameters O.
measure of expressivity for ansatzes was first defined for the |0) input
(
[ du(V)ve
/ aU ()(t)) AV
ju(d)
2.3 RELATED WORK
BPs have been proven for deep (MeClean et al. 2018) and shallow circuits with global measure- ment (Cerezo et al. 2021b), expressivity (Holmes et al., 2022 McClean et al.) 2018), noise (Wang) eet al. 2021), entanglement (Ortiz Marrero et al. (021) and it has been linked to cost concentration and narrow gorges (Arrasmith al. (022). Further, even shallow models without BPs have only
small fraction of local minima close to the global minimum (Anschuetz & Kiani |
The problem has been linked the curse of dimensionality in Ccrezo et al. (2024). therefore, lim- iting the accessible Hilbert space was proposed. However, they showed that such models can be simulated classically, given an initial data acquisition phase on a quantum computer. This was ex- tended to a trade-off between trainability and dequantizability in Gil-Fuster et al. (2024).
Besides that, parameter initialization strategies have been proposed. with some even providing better bounds on gradient magnitudes (Rad et al.) 2022) Wang et al.) 2023 Kulshrestha & Safro 2022] Grant et al.) 2019), however, the results of Herbst et al. (2024) imply that is not the type of statistical distribution, but rather the used parameter ranges that could lead to better starting points.
Further, expressivity is commonly discussed to prove a quantum advantage with respect to classical models. The closeness to being 2-design is the most widely employed measure, however, others have been proposed. In particular, Equation 3 is related to the Frame Potential (Gross et al., 2007] Roberts & Yoshida 2017| Nakaji & Yamamoto 202I). Another approach is to analyze the entan- gling capability as in Sim et al. (2019), i... the entanglement of the produced states. Other measures the Covering Number from Du al. (2022), analyzing Fourier coefficients (Schuld al. 2021
We refer to Appendix A.3 for a short introduction to unitary groups and the Haar measure. For further information, we refer Larocca et al. (2024), who published excellent review the topic.
3

---

Published as a conference paper at ICLR 2025
Caro et al. 2021). the Effective Dimension from Abbas al. (2021). Further, one consider unitary 7- t-designs, i.e., consider the operations without the input state We employ the closeness to being a 2-design in our study, as it is frequently used, due to its intuitive interpretation. Beyond that, due to computational complexity, there is little research on properties of the architectures for now
Moreover, the field of QML has seen a rise in works efficient design of ansatzes. Recent works, such as Leone et al. (024), have actively highlighted that well-defined ansatzes are a crucial step for large-scale deployment. In the field of ML, such approaches particularly include the field of
Geometric Quantum Machine Learning (GQML) (Ragone el al. 2023 Larocca et al. 2022
Tüysüz
al. 2024 Wiersema et al. 2024), or adaptive ansatzes (Bilkis al. 2023).
Further, the practical utility of the HEA has already been questioned and studied by Leone et al. (2024). They find that problems with product input states and data that satisfies a volume law of entanglement should avoid using such ansatzes, whereas area law of entanglement data leads to optimization landscapes without BPs. Moreover, the necessity of understanding and characterizing ansatzes has been discussed in Zhang al. (2024), as well as in Larocca et al. (2023).
Changes quantum channel or loss landscape upon perturbing parameters has not yet been studied for QNNs. However, perturbation analysis and sensitivity analysis have emerged as tools in classical
ML., Examples are perturbing the input for explainability (Ivanovs et al. 2021 Pizarroso et al. 2022]
Fel et al. 2023), checking for stability (Testa et al. (024)) Similarly, y, weights can be perturbed
for trainability (Wen et al. 2018), or to increase robustness and generalization (Wu et al. 2020 He
et al., 2019 Dumford & Scheirer 2020). To the best of our knowledge, there is research
perturbing parameters to check how the underlying function changes, as, due to the stochasticity of quantum computing, this research question seems be much more relevant in the quantum realm.
3 2-DESIGN AS A MEASURE OF EXPRESSIVITY
While 2-designs have turned out to be sufficient for many quantum information processing proto- cols (Holmes et al. 2022 Harrow & Low| 2009). in this section, we summarize and connect results from the literature to argue that closeness to a 2-design is an inadequate measure for model expres- sivity (i.e., the ability to sufficiently approximate the unitary group to a high degree).
For the argument we will use the so-called Welch Bounds which, loosely put, inequalities related to evenly distributing& a finite number of unit vectors in a vector space. This inequality was addressed in Welch 1974), where a lower bound on the inner-product between unit vectors was provided. Intuitively, a smaller lower bound corresponds to more evenly spread vectors in the vector space, i..,, more uniform distribution.
Quantum state t-designs reduce integrals of polynomials over all quantum states to averages over a discrete set (cf. Equation 19). These are probability distributions over pure quantum states that repli- cate properties of the uniform (Haar) measure on the quantum states up to the (-th moment. We mea- sure expressivity of 2-design based architectures by comparing (generalized) Welch bound (Scott) 2008) values for state 2-designs against its t-design counterparts, with t > We state the inequality for a state t-design based architecture.
The choice of the polynomial function for state t-designs is presented in Appendix [A.4] We state the following inequality for a state in a d-dimensional Hilbert space, lsb2) E Cd (cf. Appendix B.2 for the derivation):
State t-design Welch bound:
n²
2
Σ ISS.AD
(bb))
2 (042-1) (d++ l]d++ 1- 2)
Vt E N.
(4)
It is clear that for an increasing t, the term in the denominator (d+t-1)(d=t- 2)....d increases rapidly
(as compared to Equation [4] for 2-designs), yielding a vanishing overlap between the state vec- tor summands. This makes the inequality in Equation 4 more stringent for increasing values.
Uniitary designs and the difference between unitary and state t-designs are discussed in Appendix A.3 As mentioned in Appendix AA.3 unitary t-designs induce state t-designs when acting on fixed input states. Thus, our data-agnostic approach is also valid when considering state t-designs in the following.
4

---

Published as a conference paper at ICLR 2025
This indicates that higher-order designs (t 2) approximating the Haar measure up to the t-th der impose stronger constraints, requiring greater separation between a sequence of state vectors {1b1, .......... ... to maintain equality in the bounds. This means that 2-design models have far fewer degrees of freedom in building complex quantum circuits. On the contrary. t-designs (t > 2) more efficiently construct higher-order representations capturing more complex interactions be- tween quantum states. Thus, evaluating Welch bounds for 2-designs corroborates its inadequacy for constituting an expressive architecture.
An ansatz forming a 2-design is not expressive as following the Haar measure only up to the second moment constrains how spread-out (statistical spread) the unitaries must be, however, does not allow drawing conclusions about how densely it covers U(N). ) If one were to define expressivity as the capability of a model to represent all possible set of unitaries, would be required to go way beyond second moments (for, e.g.. Kurtosis, off-diagonal correlations, etc.) to ensure the necessary weights are captured where there is sufficiently dense population. As the moment operator requires tensor products proportional to the moments, this becomes computationally infeasible very quickly. It is not clear how to overcome the curse of dimensionality while proposing a measure that captures expressivity adequately; in fact, most tools from QIT suffer from this very phenomenon, speaking to the difficulty of designing such a metric. Further, there already exists a line of non-unitare VQA research (e.g., Cong al. (2019); Deshpande al. (2024)), that needs to be considered as well.
As expressivity is currently the most common metric used to describe the ansatzes, our work em- ploys an alternative approach to analyze the model space, unrelated to expressivity, but highly rele- vant for trainability and training dynamics. Therefore, we focus on local neighborhoods of the model space, in particular, we study how the QNN changes upon perturbation of parameters. This allows analyzing training dynamics and tracking the extent to which the QNN changes during training.
4 CHANNEL SENSITIVITY
Comparison between quantum channels is an important topic in Quantum Information Theory. For consistency with respect to, e.g., highly entangling operations. however, operator norms need to be stable under tensor product. Therefore, the diamond norm was proposed in Kitaev (1977) as follows.
Definition The diamond norm of superoperator T is defined as [Harroo Low 2009 Kitaev el al. 2002).
[[[T' 2 ida)
IT = sup [ @ idalloo = sup sup
(5)
|
X
Here, @ denotes the tensor product and id, the identity channel on d dimensions, with d S 2", and n as the number of qubits. The ||X]]] in this context is the Trace norm or Schatten I-norm, which is defined as ||X|| = TrvXtx. The diamond norm is defined for any superoperator T, wherein, superoperators are defined as the set of linear maps acting on vector space of linear operators. For valid quantum channel E₁ (a completely positive and trace preserving (CPTP) matrix), the diamond norm is strictly upper bounded by and lower bounded by O. When using the diamond norm to measure the distance between two CPTP quantum channels & and &g, the channels are subtracted. The resulting matrix not necessarily CPTP, resulting in revised upper bound 2.
Operationally, the diamond norm measures the maximum distinguishability between the output states of the two maps under any input state, i.e., for any input state we will be less or equally able to distinguish the output. A value of O, means that they are indistinguishable, whereas a value of 2 means they are perfectly distinguishable. Computing the diamond norm is non-trivial, however, can be reduced to semi-definite program (Watrous 2009 2013).
We apply the diamond norm to evaluate the distinguishability of a parametrized unitary U(O). rep- resenting a quantum channel, from the unitary U + 8), where 5 is perturbation of 9, as follows.
Definition 2
We define the channel sensitivity as the distinguishability of the quantum channel from itself upon small perturbation of i by 8.
5

---

Published as a conference paper at ICLR 2025
Table 1: Employed ansatzes
COMPONENT
CONFIGURATIONS
Parameterized Entangling
[Rx, Ry, Rz, RyRy, RyRg, RxRz, RxRyRzl (CNOT, CZ, CNOT CZ]
CSU (0,6) (0,6)=IU(()) U( + 8)1l
(6)
We can provide an upper bound on the channel sensitivity through Taylor expansion, which is shown in Equation [ Therefore, we establish direct dependence of the distinguishability of the channels to the sum of changes applied The bound holds if the Hermitian generators of the trainable gates are unitary as well, which is relevant for HEAs, the trainable gates are exponentiations of the X, Y. and Z Pauli gates. The precision of the bound depends on the magnitude of 8, however, considering that the models trained iteratively in small steps, we expect the bound to hold.
dim((5
(
[U(00) - U(0 = (I((0) (U()) +
6
+ 0
dim(5) 5j 00,
(
d
|8,1
11e
5
j=1
2
Unless the feature encoding is trained as well, the upper bound on distinguishability cannot be improved by including data. Intuitively, due to the calculation of the diamond being based on the maximum distinguishability from any input state, includes any possible unitary data encoding as well. Mathematically, this property follows from the unitary invariance of the Schatten 1-norm.
Our bound shows that (assuming small parameters updates), there is direct relationship between the number of parameters and the distinguishability of the operation. In particular, for small models, which are used extensively at the moment, can be expected that subsequent unitaries during train- ing are hardly distinguishable, hindering effective training. This is, to the best of our knowledge, the first result attempting to establish a connection between ansatz and trainability issues. To strengthen results, want to, in the following, support our bounds through numerical experiments
5 EXPERIMENTAL SETUP
Ansatzes We consider layered architectures in our experiments, each consisting of trainable pa- rameterized and fixed entangling components. For the first, we use rotations around the x- (Rx y- (Ry). and z-axis (Rz) of the Bloch sphere, and the controlled-NOT (CNOT) and controlled-Z (CZ) gates for the latter] constituting a widely-used gate set for HEAs. For list of parameterized and entangling components, we refer to Table and we run our experiments using all combinations.
We do not permute the operators, due to considerations on the expressivity per qubit. That is, all rotations are applied, all permutations of Rx (a) RZ) with a, 8 E [0, 2n] span the (2), the set of unitary operations on one qubit. Therefore, permuting the operations does not affect expressivity per layer. Similarly, two rotation operations per layer explore subspace of the (2) owing to the Lie-algebraic closure relations, e.g., Rx (a) Ry (3) spans the upper half of the Bloch sphere, does Ry (8). Any permutation allows the same operations per qubit, which is why we deem all permutations as equal.
Perturbation For the experiments, uniformly sample parameters in range [0, 2T. and choose random 95% of them to perturb their values by +1% (in our case [0.1, 0.5, 1J). These values were in
'For the precise mathematical derivation, we refer to Appendix C
We refer Appendix A. for an overview of the operations.

---

Published as a conference paper at ICLR 2025
no way chosen randomly, rather, by training QNNs and collecting summary statisties on parameter updates, we found that these are the ranges where updates are performed. Then, we evaluate the channel sensitivity with the two obtained unitaries. We take 100 samples per parameter involved in the circuit and consider the distribution of the channel sensitivity we obtain for further analysis.
Training Moreover, we want to compare the channel sensitivity for random perturbations and training runs. Therefore, we train binary classification models on the two largest classes of the wine and breast cancer datasets and use PCA to reduce the features to 2", with n as the number of qubits. Then, the features are normalized and encoded into the amplitudes of the input state. The measurement taken in the z-basis of the first qubit, and we use the mean squared error as the loss.
To account for favorable parameter initializations, we train each architecture 50 times. We 150 iterations, and monitor convergence and parameter changes. We use the Adam optimizer (Kingma & Ba 2013) from Pennylane with default parameters (step size: 0.01, beta I: 0.9, beta 2: 0.99, epsilon: le-08). Further, we calculate the channel sensitivity in every iteration, to get an operationally mean- ingful analysis of how much the model changes. This allows evaluating whether the model changes more when updated based on the loss function, rather than by just randomly permuting parameters. We display aggregated results using confidence intervals with a 95% confidence level.
Qubits Due the computational complexity involved in solving the diamond norm, we run our experiments from one to four qubits and one to five layers. Unfortunately, due to matrix dimensions, it is challenging to scale the diamond norm any further. Nonetheless, the results are relevant, con- sidering that they (1) provide a first step at analyzing small scale architectures, and (2) have direct implications for training dynamics that can be expected in the NISQ era.
Today's ' QNNs use shallow circuits with depth in Olog n) and local cost function, as they can shown to be trainable (Cerezo et al.] 2021b). Even if favorable loss landscapes were found for deeper circuits, adding layers results in noisier loss estimates, prohibiting effective learning. Moreover, scaling in number of qubits is limited due to quantum hardware limitations, or, when the models simulated classically, to the curse of dimensionality. Therefore, many recent works proposing QMI to solve a particular problem use few qubits, e.g., Blance & Spannowsky (202)), with two qubits, Yano et al. (2020) with three and four qubits.
Further, our main goal is to take initial steps in quantifying and characterizing QNN model behavior. As has been mentioned before, due to the power of classical ML, we are unlikely to find an advantage in QML in the near term (Schuld & Killoran 2022). Therefore, it would be important to focus on more fundamental questions. such as "how can we exploit quantum mechanics for ML purposes?", to explore the potential of quantum computing for data processing. In that sense, understanding how quantum channels change upon parameter updates is a foundational topic in ansatz design.
Framework Our experiments are implemented using the Python programming language. We use Pennylane (Bergholm et al. 2022) for obtaining the unitary transformations and QuTIP (Johansson al. 2012 2013) for calculating the diamond norm. Unfortunately, a diamond norm computation between two operators ||A - Bll with a finitely-large overlap A z II, yields numerical instabili- ties with the QuTIP package. To account for these numerical issues, the operator can be multiplied with global phase, which fixes gauge (cf. Gauge-Fixing, in Appendix P
EXPERIMENTAL RESULTS
We run extensive numerical experiments validate our bounds. First, we randomly perturb pa- rameters, then we compare them to the channel sensitivity we observe during actual training runs. Initially, we verify the assumptions about the magnitude of parameter changes. In training runs, the mean parameter change per update in the first 10 iterations is maximum 1.4%, with mean and standard deviation of 0.8% + 0.4%, but quickly decreases. We visualize this in Figure [a] with the iterations on the x-axis, and the mean change in parameters the same number of qubits are aggregated and show very narrow confidence intervals. Further, is also visible that changes in multi-qubit systems tend to be even smaller (less than two orders of magnitude). Moreover, we observe that almost all parameters are updated in every iteration, which is shown in Figure Tbl where the y-axis shows the percentage of parameters changed.
the y-axis. Architectures with
7

---

Published as a conference paper at ICLR 2025
Paramnttrr Update per Step
Moan Numnher of Pararacters Changed
0.02
qu20a
0.100
0.066
13
0.990
0.62
0.00
0
07
75
100
105
150
26
80
15
100
12x
150
Lteration
Itoration
56-
AugC- Ne..alio
(a) Mean absolute change
(b) Amount of parameters changed
Figure 1: Training parameter updates
6.1 RANDOM PERTURBATION
Based on these observations, we run the random perturbation experiments by changing 95% of parameters by 1%, 0.5%, and 0.1%. We visualize the results in Figure showing the depth of the circuit on the x-axis and the obtained channel sensitivity on the y-axis. Per our bounds, the channel sensitivity strongly depends on the number of parameters in the cireuit, hence, we visualize the results based on the number of parameters per layer.
The experiments show the same patterns the bounds predict. In particular, fixing the qubits and parameters layer while increasing the depth of the circuit results in a bigger channel sensitivity. The same can be observed when fixing the number of layers and increasing the qubits. We observe that increasing channel sensitivity is more prevalent for larger systems than for deeper circuits, e.g, architectures with 2 qubits, 5 layers and parameters have a smaller channel sensitivity than archi- tectures with 3 qubits, layers and 2 parameters, despite having the same number of parameters.
The behavior expected for ansatzes covering large parts of the search space, as Hilbert space of larger system is significantly bigger, scaling as 2 This allows more independent search diree- tions, therefore, potentially more changes in channel upon update. Deeper circuits can explore larger parts of the Hilbert space too, however, their expressivity stagnates when achieving overparameter- ization (Larocca et al. 2023). In this regime, more parameters will not result in more independent search directions, hence, the difference channel sensitivity expected to be smaller.
Further, can be seen that there is substantial degree of variability in the channel sensitivity. In the majority of experiments, the channels hardly distinguishable, however, many outliers can be observed. This property could be particularly relevant for warm-starting (Puig et al. 22024), i.c., starting training from regions with high channel sensitivity might lead smoother optimization.
6.2 TRAINING UPDATES
Additionally, we want to consider how different the quantum channels are when updated based on loss and optimizer. particular, it could be possible that changing the parameters during training. changes the underlying channel a lot more than perturbing in random directions. Upon analysis of the results, we can confirm the validity of the Taylor expansion even in early stages of iteration, i.e., in the 45, 500 models we trained, the bound holds for every single parameter update taken.
We plot the channel sensitivity in Figure 3 for different layers, qubits and parameters per layer. We can observe that the confidence intervals are very narrow, i.e., even though we collect data from 50 different runs for every model, the observable channel sensitivity hardly varies. Further, the channel sensitivity is substantially smaller than the bound, speaking to the severity of the issue. The discrepancy grows in particular with the number of qubits, which is visualized Appendix D While cannot observe the magnitude of channel sensitivity that the bound predicts, the pattern of scaling in the number of parameters can be observed. This in particular visible in Figure for early training stages with larger parameter updates, however, it vanishes in later
Figure g in
8

---

Published as a conference paper at ICLR 2025
Chonge Threshold: 0.95, Perturb Threshod: 0.01
Qubite:
Qubits:
Qubite: 3
Qubita: 1
0.8
Pincury/ Layer
0.4
0.2
0.0
31
4
Layer
Layer
Layer
Leyet
(a) Change 95% parameters by 1%
Change Threshold: 0.95, Perturb Threshold: 0.005
Qubits:
Qubits: 2
Qubits:
Qubits:
0.15
Pamm//
0.10
0.05
0.00
3
Layer
Layer
Layer
Layer
(b) Change 95% of parameters by 0.5%
Change Threshold: 0.95, Perturb Threshold: 0.001
Qubits:
Qubits:
Qubits:
Qubita: 1
Pezarza (Lavve
Layer
Layer
Layer
Layer
(c) Change 95% of parameters by 0.01%
Figure Channel sensitivity for random perturbations
stages, when the updates are very small. Further, to showcase robustness with respect to the feature encoding (and different parameter updates that may result thereof), we run our experiments with angle encoding well and verify that the magnitude of channel sensitivity does not change. We refer to Figure 5 in Appendix for more information.
7 DISCUSSION
Our bound establishes direct dependence of the distinguishability of the channels during QNN training the number of parameters. That is, the maximum distinguishabiiityy of two output states of the quantum channels, scales with (1) the magnitude of changes and (2) the number of parameters. As iterative updates with small learning rates are applied, which is independent of the number of parameters in the circuit, the dependence is largely dominated by the number of parameters.
This could significantly contribute the trainability issues of today's QNNs. In particular, for reasonably sized models, the channels may be distinguishable in early training stages, but hardly distinguishable when fine-tuning the ansatz later. Comparison of our bound to the channel sensitivity obtained in training runs reveals that during training, the channels are even less distinguishable. While the channel sensitivity for random permutation is still considerably lower than our bounds would predict, lot more variation can be observed.
This confirms our initial motivation of studying the model space independently of the loss and data, these global properties of ansatzes. In particular, want draw attention to the remarkable similarities to the BP phenomenon. Adapting series of composition of maps viewpoint (Laroccal et al. 2023). our work argues that the unitaries in the unitary space of the considered architectures

---

Published as a conference paper at ICLR 2025
qubits
qubits
qubits
qubits
Distinghishhhiiigg
0.004
0.002
0.000
Didingnishbhin
0.004
0.022
Distngniiiubiitty
0.004
0
100
150
100
150
50
2001
150
50
100
150
Step
Step
Step
Step
Figure 3: Channel sensitivity during training runs
do not change significantly upon parameter update in the local neighborhood of the parameter space. Thus, from small channel distinguishability. follows that the quantum states in the Hilbert space will not differ significantly, will the expectation values (which is consequence of the diamond norm formulation). which are directly related to the loss. Roughly speaking, this behavior is mir- rored in the BP phenomenon. While do not argue for strict equivalence, the similarities hint towards two perspectives on similar trainability issues. Therefore, this phenomenon might not only be tied the exponential Hilbert space, but may be an artifact of the architectures themselves.
Therefore, it seems that the architectures significantly contribute to the untrainable models we ob- serve today, independently of data and loss. Our findings have implications to start a more thorough investigation of the model space of the ansatzes that are currently used, and, ultimately, stress the need for good initialization and warm-starting. is, however, non-obvious how to find such starting points without rrie-onn--error.
CONCLUSION
Altogether, our work provides a first step at exploring the model space of QNNs. In particular, we provide picture of the local neighborhood of the model space of ansatz, which suggest that the channels, at least for small scale circuits, do not differ significantly upon parameter update. Our results for random perturbations reveal that there is substantial degree of variability of the channel sensitivity depending on the neighborhood, hence, stress the need to study the model space more thoroughly. This could reveal valuable information for parameter initialization or warm-starting.
Considering that QNNs were designed for NISQ technology, shallow, small-scale circuits due to hardware limitations, the community will be limited to such small instances in the near-term. There- fore, our findings have direct implications circuits. While true that the models can be scaled to work for larger circuits, we have over- whelming evidence through works BPs, that this is inherently difficult. Together with results classical simulability and dequantizability in the absence of BPs. this work extends the evidence that need paradigm shift in Variational Quantum Computing
the meaningfulness of iteratively optimizing such
QMI altogether.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
The authors would like to thank Adrián Pérez-Salinas for feedback on carlier versions of this work. This work has been partially funded through the Themis project (Trustworthy and Sustain- the Stan- dalone Project Transprecise Edge Computing (Triton), Austrian Science Fund (FWF); P 36870-N, and by the Flagship Project HPQC (High Performance Integrated Quantum Computing) # 897481 Austrian Research Promotion Agency (FFG).
able Code Offloading), Austrian Science Fund (FWF): Grant-DOl 10.55776/PAT16668
REPRODUCIBILITY STATEMENT
The code and results for the experiments are publicly available in a GitHub repository
REFERENCES
Amira Abbas, David Sutter, Christa Zoufal, Aurelien Lucchi, Alessio Figalli, and Stefan Woerner. The power of quantum neural networks. Nature Computational Science, 1(6):403 409, June 2021. ISSN 2662-8457. doi: 10. 0010088843588--01--000---- URL http:://t/// doi org/1 0. 110038/650 0055//435880000 021 00084-1
Eric R. Anschuetz and Bobak T. Kiani. Quantum variational algorithms swamped with traps. Nat Commun 13, 2022. doi: 10.
Andrew Arrasmith, 7oë Holmes, Marco Cerezo, and Patrick Coles. Equivalence of quantum barren plateaus cost concentration and narrow gorges. Quantum Science and Technology, 7(4): 045015, aug 2022. doi: 10. 10010//0088-555///770000 URL
doi org. 10. 10
88/2058 9565/55655 ac7006
Ingemar Bengtsson and Karol Zyezkowski. Geometry of Quantum States: An Introduction to Quan- fum Entanglement. Cambridge University Press, 2006.
Ville Bergholm, Josh Izaac, Maria Schuld, et al. Pennylane: Automatic differentiation of hybrid quantum-classical computations, 2022. URL httpt:://axxiiii///e wttp:///wwrr.aaaaaa.....0
Matias Bilkis, Marco Cerezo, Guillaume Verdon, Patrick J Coles, and Lukasz Cincio. A semi- agnostic ansatz with variable structure for variational quantum algorithms. Quantum Machine Intelligence, 5(2):43, 2023.
Andrew Blance and Michael Spannowsky. Quantum machine learning for particle physics using variational quantum classifier. J. High Energ. Phys, 2021. doi: https://coi.org/10.. http://o....rr111...000//////// (2021)212.
Matthias C. Caro, Elies Gil-Fuster, Johannes Jakob Meyer, Jens Eisert, and Ryan Sweke. Encoding- dependent generalization bounds for parametrized quantum circuits. Quantum, 5:582, November 2021. ISSN 2521-327X. doi: 1/22333//2222212222 11-1--8811--82 0.
URL htt:tp:: doi org/1
Marco Cerezo, Andrew Arrasmith, Ryan Babbush, Simon C. Benjamin, Suguru Endo, Keisuke Fujii, Jarrod R. McClean, Kosuke Mitarai, Xiao Yuan, Lukasz Cincio, and Patrick J. Coles. Variational quantum algorithms. Nature Reviews Physies, 3(9):625 644, 9 2021a. ISSN 2522-5820. doi: 10. 10.10388/42254-00--00333
Marco Cerezo, Akira Sone, Tyler Volkoff, et al. Cost function dependent barren plateaus in shallow parametrized quantum circuits. Nat Commun 12, 1791, 2021b. doi: 10.103%/44147--222211222 -W.
Marco Cerezo, Martin Laroeca, Diego Gartía-Martin, Nahuel I.. Diaz, Paolo Braccia, Enrico Fontana, Manuel S. Rudolph, Pablo Bermejo, Aroosa Ijaz, Supanut Thanasilp, Eric R. Anschuetz, and Zoë Holmes. Does provable absence of barren plateaus imply classical simulability? or, why we need rethink variational quantum computing, 2024.
http::////gitthubb.oooo///////// http://sabiiiaaaattt//mo/// hat http://pnn__oaaaaeeeeenneerhheoo ghborhood

---

Published as a conference paper at ICLR 2025
Iris Cong, Soonwon Choi, and Mikhail D. Lukin. Quantum convolutional neural networks. Nature Physics, 15, 2019. doi: 10. 00.1038/44156--01-066646
Christoph Dankert, Richard Cleve, Joseph Emerson, and Etera Livine. Exact and approximate uni- tary 2-designs and their application to fidelity estimation. Phys. Rev. A, 80:012304, Jul 2009, doi: 10. htt//hyyeeeA...0022000000
htRp:tt:: 1iikiiin..... aps org doi/ http://oo/ll../.1 /t0 103////
evA. 80. 012304
Abhinav Deshpande, Marcel Hinsche, Sona Najafi, Kunal Sharma, Ryan Sweke, and Christa Zoufal.
Dynamic parameterized quantum circuits: expressive and barren-platent free, 2024. URL http
8: s: hts//axxiv.orr/aabs...
05760
Yuxuan Du, Zhuozhuo Tu, Xiao Yuan, and Dacheng Tao. Efficient measure for the expressivity of variational quantum algorithms. Phys. Rev. Lett. 128:080506, Feb 2022. doi: 10.1 103//oReee 128
Utthttps::/ htt./iikk.... aps. htt.t//..aa.....ooo/o///////////
080006
Jacob Dumford and Walter Scheirer. Backdooring convolutional neural networks via targeted weight perturbations. In 2020 IEEE International Joint Conference on Biometrics (IJBB, pp. 1-9, 2020.
doi:
Thomas Fel, Melanie Ducoffe, David Vigouroux, Rémi Cadène, Mikaël Capelle, Claire Nicodème, and Thomas Serre. Don't lie to me! robust and efficient explainability with verified perturbation analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recogni- tion (CVPR), pp. 16153-16163, June 2023.
Elies Gil-Fuster, Casper Gyurik, Adrián Pérez-Salinas, and Vedran Dunjko. On the relation between trainability and dequantization of variational quantum learning models, 2024. URL https:
Edward Grant, Leonard Wossnig, Mateusz Ostaszewski, and Marcello Benedetti. An initialization strategy for addressing barren plateaus in parametrized quantum circuits. Quantum, 3:214, 12 2019. ISSN 2521-327X. doi:
12-09-214.
D. Gross, K. Audenaert, and J. Eisert. Evenly distributed unitaries: On the structure of unitary designs. Journal of Mathematical Physics, 48(5), may 2007. ISSN 1089-7658. doi: 10. 2716992. URL URL http: /xx /ww. org/ 10 11t...11..11..11 2711692
Andrew W. Harrow and Richard A. Low. Random quantum eircuits are approximate 2-designs. Commun. Math. Phys. 291, pp. 257-302, 2009. doi: 00.1007/000220000-00555
Zhezhi He, Adnan Siraj Rakin, and Deliang Fan. Parametric noise injection: Trainable randomness to improve deep neural network robustness against adversarial attack. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
Sabrina Herbst, Vincenzo De Maio, and Ivona Brandic. On optimizing hyperprrmttr for quantum neural networks, 2024. URL
S.G. Hoggar. t-designs in projective spaces. European Journal of Combinatorics, 3(3):233-254 1982. ISSN 0195-6698. doi: http:///oooog/UOluouudooossssss0
URL https:
htt.a//aa.eaaea.oo...69.99923831 / //www. www.ssciiieeeiieec wtt.:reeettttcceoeeeeeeeeeeeeeee http://wwccccccceecooccacceaaaaa
Zoë Holmes, Kunal Sharma, M. Cerezo, and Patrick J. Coles. Connecting ansatz expressibility to gradient magnitudes and barren plateaus. PRX Quantum, 3:010313, 2022. doi: 10.1103/PRXQ uantum.3.01031
Roger A. Horn and Charles R. Johnson. Norms for vectors and matrices, pp. 257-342. Cambridge University Press, 1985.
Maksims Ivanovs, Roberts Kadikis, and Kaspars Ozols. Perturbation- based methods for explaining deep neural networks: A survey. Pattern Recognition Letters, 150:228 234, 2021. ISSN 0167- 8655. doi: ectt htt::co/ccciooeeerrtee/ee//////0
URL (tthttss: :/ www. sciencedir
12

---

Published as a conference paper at ICLR 2025
J. Robert Johansson, Paul D. Nation, and Franco Nori. Qutip: An open-source python framework for the dynamics of open quantum systems. Computer Physics Communications, 183(8):1760- 1772, 2012. ISSN 0010-4655. doi: http:////....eeeeoa.aaaaa/////// //www. sci encedi rect. com
URL https:
J. Robert Johansson, Paul D. Nation, and Franco Nori. Qutip 2: A python framework for the dynamics of open quantum systems. Computer Physics Communications, 183(4):1234-1240, 2013. ISSN 0010-4655. doi: htt.a//a.......aaooooo///aaaaaaa
URL https:
A. Kandala, A. Mezzacapo, K. Temme, et Hardware-efficient variational quantum eigensolver for small molecules and quantum magnets. Nature, pp. 242-246, 2017. doi: 10.10338/aturre33879,
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. 3rd International Conference for Learning Representations, 2015. doi: 00.48550/arXiv.1412..00
Alexei Yu Kitaev. Quantum computations: algorithms and error correction. Russian Mathematical Surveys, 22(6):1191, dec 1997. doi: 10.
Alexei Yu Kitaev, Alexander H Šen', and Mikhail N Vyalyi. Classical and quantum computation. Graduate studies in mathematics. American Math. Soc., Providence, RI, 2002. ISBN 082 83229....
Ankit Kulshrestha and Ilya Safro. Beinit: Avoiding barren plateaus in variational quantum algo- rithms. In 2022 IEEE International Conference on Quantum Computing and Engineering (QCE), pp. 197-203, 2022. doi: 10.1 11110///EE3311522220000000
Martín Larocca, Frédéric Sauvage, Faris M. Sbahi, Guillaume Verdon, Patrick Coles, and Marco Cerezo. Group-invariant quantum machine learning. PRX Quantum, 3:030341, Sep 2022. doi: 10. 10.1100/PRXQuaamum.3.0000 3.00034. URL httpp::/tinne //tp:::///inkk.o// pttttp::/Liin.aps..oo wttpppppprr//o....1111....
3. 030341
Martin Larocca, Nathan Ju, Diego. García-Martin, al. Theory of overparametrrization in quantum neural networks. Nat Comput Sci, pp. 542 551, 2023. doi: 10. 10.1038/s43388-022 0010448888022-0004 023-00467-6.
Martin Larocca, Supanut Thanasilp. Samson Wang. Kunal Sharma, Jacob Biamonte, Patrick J. Coles, Lukasz Cincio, Jarrod R. McClean, Zoë Holmes, and Marco Cerezo. review of bar- plateaus in variational quantum computing, 2024. URL /ttptps:s///ii....ottt wwwp://..rg/a//////////
00780
Lorenzo Leone, Salvatore F.E. Oliviero, Lukasz Cincio, and Marco Cerezo. On the practical useful- ness of the Hardware Efficient Ansatz. Quamtum, 3:1395, July 2024. ISSN 2521 doi: 10.2 1395.. URL fttps://doi.org: 10. 22331/q 2024 07 03 1339
Jarrod R. McClean, Sergio Boixo, Vadim N. Smelyanskiy, Ryan Babbush, and Hartmut Neven. Barren plateaus in quantum neural network training landscapes. Nature Communications, 9(1); 4812, 11 2018. ISSN 2041-1723. doi:
Antonio Anna Mele. Introduction to Haar Measure Tools in Quantum Information: A Beginner's Tutorial. Quantum, 8: 1340 May 2024. ISSN 2521-327X. doi: 10. 10.22331/4-2024-05 1340. URL httppp///oo..oor/.02222332233333
22332
Kouhei Nakaji and Naoki Yamamoto. Expressibility of the alternating layered ansatz for quantum
computation. Quantum, 5:434, April 2021. ISSN 2521-327X. doi: 10.22331/q-2021 202310//2021---111111 10----04-99--
htthhtts::/dol. org 10. 22 2233
Carlos Ortiz. Marrero, Mária Kieferová, and Nathan Wiebe. Entanglement-nduueed barren plateaus. PRX Quantum, 2:040316, 10 2021. doi: 00.1033PRXQuauum.2..0000
Jaime Pizarroso, José Portela, and Antonio Muñoz. Neuralsens: Sensitivity analysis of neural net-
works. Journal of Statistical Software 102(7):1-36, 2022. doi: 10. 10.186637/ss.v1022.077 URL.
https: www. tettetttoff..rriiddeee http:///oooo.caa////////////////
John Preskill. Quantum computing in the nisq era and beyond. Quantum, 2:79, 2018.
13

---

Published as a conference paper at ICLR 2025
Ricard Puig, Mare Drudis, Supanut Thanasilp, and Zoë Holmes. Variational quantum simulation: case study for understanding warm starts, 2024. 0044
Ali Rad, Alireza Seif, and Norbert M. Linke. Surviving the barren plateau in variational quantum circuits with bayesian learning initialization, 2022.
Maithra Raghu, Ben Poole, Jon Kleinberg, Surya Ganguli, and Jascha Sohl Dickstein. On the expressive power of deep neural networks. In Proceedings of the 34th International Conference on Machine Learning Volume 70, ICML' 17, pp. 2847-2854. JoLR.org, 2017.
Michael Ragone, Paolo Braccia, Quynh T. Nguyen, Louis Schatzki, Patrick J. Coles, Frederic Sauvage, Martin Larocca, and M. Cerezo. Representation theory for geometric quantum machine learning, 2023. Uttpttss: //tp://aavvvv///////aa wrg. abs 2210. 22000.0222100
Daniel Reed, Dennis Gannon, and Jack Dongarra. Reinventing high performance computing: Chal- lenges and opportunities, 2022.
Daniel A. Roberts and Beni Yoshida. Chaos and complexity by design. Journal of High Energy Physics. 2017(4). April 2017. ISSN 1029-8479. doi: 10.1007/jhe0042001)12.. URL http:
//xx.oo doi. org/ 10. 1007 JHEPO4 (2017)
Maria Schuld and Nathan Killoran. Is quantum advantage the right goal for quantum machine learning? PRX Quantum, 3(3), July 2022. ISSN 2691-3399. doi: 10. /033/rrqquntuum.30000000
URL http:// /ttp::/dxx doi. org/ 10. 1103/ 2ww033//X/anttt.t um. 3. 030101
Maria Schuld, Ryan Sweke, and Johannes Jakob Meyer. Effect of data encoding the expressive power of variational muantun-machinn-llamnin models. Physical Review A, 103(3), March 2021. ISSN 2469-9934. doi: 10.1 hww.://hhysroa..033..22222 URL http: doi wwwp0............oo
03/PhysReva. 103. 032430
Martin Schulz, Dieter Kranzlmüller, Laura Brandon Schulz, Carsten Trinitis, and Josef Weiden- dorfer. On the inevitability of integrated hpe systems and how they will change hpc system operations. In Proceedings of the 11th International Symposium on Highly Efficient Accelera- tors and Reconfigurable Technologies, HEART '21, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 978 9781450385497.
A J Scott. Opümizing quantum process tomography with unitary 2-designs. Journal of Physics A: Mathematical and Theoretical, 41(5):055308,, jan 2008 doi: 10. 1010088/7771-811134//17055088 URL
dx doi http:///o.......oo///01000/////0 0111////111111111111111555111110 8113/1115/051113
John Shalf. The future of computing beyond moore's law. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 378(2166):20100061) 2020. doi:
10.1098/rsta.2119.000
Sukin Sim, Peter D. Johnson, and Alán Aspuru-Guzik. Expressibility and entangling capability of parameterized quantum circuits for hybrid quantum-classical algorithms. Advanced Quantum Technologies, 2(12), October 2019. ISSN 251 1-9044. doi: 10. (002//ute.2019000)) URL
http://dx doi. www.oo..o......... 100222..000000000 qute. wtt.utee.....00 201900070
Lucia Testa, Claudio Battiloro, Stefania Sardellitti, and Sergio Barbarossa. Stability of graph con- volutional neural networks through the lens of small perturbation analysis. In ICASSP 2024 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp- 6865 2024. doi: 10.1 10.1009/AASS444455..... 2004. 4444444444..002444444333 10447343.
Cenk Tüysüz, Su Yeon Chang, Maria Demidik, Karl Jansen, Sofia Vallecorsa, and Michele Grossi. Symmetry breaking in geometric quantum machine learning in the presence of noise. PRX Quantum, 5:030314, Jul 2024. doi: 10. 133/PXX www.tuu.nnnnnnnn0mmmm
URL https:
//link. . aps. org.orgggooo hogg/doi/10 11011113//10 11133/RXQQantt um. 030314
Vaswani et al. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
14

---

Published as a conference paper at ICLR 2025
S. Wang, E. Fontana, M. Cerezo, et al. Noise-induced barren plateaus in variational quantum algo- rithms. Nat Commun 12, 6961, 2021. doi: 10.1038/441467-0012770444,
Yabo Wang, Bo Qi, Chris Ferrie, and Daoyi Dong. Trainability enhancement of parameterized quantum circuits via reduced-domain parameter initialization, 2023.
John Watrous. Semidefinite programs for completely bounded norms. Theory of Computing, 5(11): 217-238, 2009. doi: 10.4086/toc.2000 2009. v0000.1.
John Watrous. Simpler semidefinite programs for completely bounded nomms. Chicago Journal of Theoretical Computer Science, 2013(8), July 2013.
John Watrous. The Theory of Quantumm Information. Cambridge University Press, 2018.
L. Welch. Lower bounds on the maximum cross correlation signals (corresp.). IEEE Transactions Information Theory, 20(3):397 1974. doi:
Yeming Wen, Paul Vicol, Jimmy Ba, Dustin Tran, and Roger Grosse Flipout: Efficient pseudo- independent weight perturbations on mini-batches. In International Conference on Learning
Representations, 2018. URL https: htttt:::://penrevieeee net wttp:/irwummroeeeeeeeeeeeeeoeeme rrNiiiii--JNiEAAA
Roeland Wiersema, Alexander F. Kemper, Bojko N. Bakalov, and Nathan Killoran. Geometric quantum machine learning with horizontal quantum gates, 2024. URL httLhttt/:/// /ttp::/axxivvvv..
g/abs/ 2406. 04418
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust gener- alization. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.). Advances Curran Associates, Inc., in Neural Information Processing Systems, volume 33, pp. files/ wwlee//paer//02/////
2020. httptp/sss/ppppeeiiinngeeooeeeee wwwp::ewdwiiinn..ee...e..ee..... wwwrips.rrcccoo paper htt////eeccceeeeeeo1662222222222
Hiroshi Yano, Yudai Suzuki, Rudy Raymond, and Naoki Yamamoto. Efficient discrete feature en- coding for variational quantum classifier. In 2020 IEEE International Conference Quantum Computing and Engineering (QCE), pp. 11-21, 2020. doi: 10. 101110900CE//233.3000000111
Shikun Zhang, Zheng Qin, Yang Zhou, Rui Li, Chunxiao Du, and Zhisong Xiao. Single entan- glement connection architecture between multi-layer bipartite hardware efficient ansatz. New Journal of Physics, 26(7):073042, jul 2024. doi: 10. 1 108/ 11////1367--2 - 263 6/30/d https: doi www.10.gr00ooo http:://o...o...oog11000..00000 367 01111/33677222-0666
URL
A ADDITIONAL BACKGROUND
A.1 QUANTUM COMPUTING
Quantum Computing (QC) works with quantum bits (qubits). While classical bits take either the value O qubits in so-called superposition of the two states, meaning they in both states at once, which shown mathematically Equation 8 where a and B are called probability amplitudes. We can do calculations with the superposed qubit, however, once we read out results, we have to take a measurement, where the qubit will collapse into state ()) with probability |a|², and into state [1) with probability | Due to this inherent denermmermiiim in QC, executions are always done multiple times.
( A qubit characterized by four numbers (the real and imaginary parts of a and 3), which, when translated into polar coordinates, gives a convenient geometrical representation the so-called Bloch sphere. This plays a significant part in today's QNNs, as the trainable gates are rotations of the individual qubits around the x-, y- and z-axis of this sphere.
(+)) a (0) 11) s.t. |a|2 1332 a,
Multiple qubits form a quantum register, and its associated quantum state represents 2n states at once, where is the number of qubits, which can make QC very powerful. A quantum state can be
15

---

Published as a conference paper at ICLR 2025
entangled, meaning that individual qubits can be correlated, such that an action qubit affects the ones it entangled with as well.
Quantum states are manipulated through unitary transformations quantum gates (U+U 1, where U is the adjoint of U') Unitary transformations are linear and preserve vector lengths, i.e, applying unitary
quantum state ensures that the squared magnitudes of the probability amplitudes still sum up to one. Every unitary is generated by a Hermitian generator (H = (HH,,, H") initial quantum state so-called unitary evolution,
i.e., U eiH We describe the action which may involve one or more unitary operations (unitary matrices form a group, hence, are closed under multiplication). We also refer this as quantum channel in the following. While the term quantum channel encompasses unitary evolutions, it is a broader term, meaning it can also describe more general dynamics of quantum systems, such as decoherence or noise. It is defined as linear map that is completely positive and trace preserving (Watrous 2018
Among the most fundamental operations in QC, are the so-called Pauli matrices, which are Hermi- tian matrices shown in Equation [ on applies NOT (bit-flip) operation (00) [1), [1] + (0))
By changes the phase and bit ((0) + [1), [1) + -i (0)), and o₂ applies a phase-flip ((0) - (0).
[1] + -1 (1)).
0
0
Ox =
4g =
02 0
Taking the Pauli matrices as generators, we can rotate a qubit around the y- and 7-axis around the Bloch sphere, arriving at the definition of the Rx, Ry and Rz rotation Equation T0
(0) = -in (0)
Rz =
- -890
(10)
Further, we define the two entangling operations we use, CNOT (or CX) and Cz, in Equation 11 The operators entangle two qubits and j, where i referred to as the control qubit and j is the targer qubit. In general terms, the entangling gates apply og to the target qubit if the control is in state [1).
1
0
0
0
L
0
0
0
1
CNOT
0
0
0
CZ.
=
0
0
1
0
(11)
0
0
0
-1
A.2 INNER-PRODUCT & OVERLAP BETWEEN VECTORS
An inner-product space is defined as vector space V over field F (typically IR or C) endowed with an inner-prodee anner-product map (.1.):
(11)) V x X +
(12)
The inner-prodee operation is a generalization the dot product between vectors and measures the overlap or coherence between vectors sampled from vector space.
A.3 UNITARY GROUPS AND THE HAAR MEASURE
We will provide key information on unitary groups and the Haar measure in this section. For an excellent tutorial the topic, we refer Mele 2024), which also base this section on. We begin by defining the unitary group (dd) and the special unitary group SU(d), which are key concepts when studying quantum computing, as follows.
Definition 3 For d N, the unitary group U(d) is the group of isometries of d-dimensional complex Hilbert space Cd. These are canonically identified with the d x d unitary matrices (Cdxd);
u(d) (V c vi =vt- -= = La}
(13)
The unitary group can be decomposed as U(d) = SU((d x (1). Here, the subgroup SU(d) is called the special unitary group, while e0 Vz E (11) is phase restricted on a unit-circle.
16

---

Published as a conference paper at ICLR 2025
Definition 4 For N, the special unitary group SU(d) U(d) group of d x d unitary
matrices (Cdxd) that have a unit determinant:
su(d) {9 Eu(d) der(G) = ]
(14)
The Haar measure forms a uniform probability distribution over sets of unitary matrices, in fact, unique for compact groups, such the U(d).
We define the Haar Measure on the (ddd,, as the left and right invariant probability measure μud over the group. That is, for all integrable functions and VV E U(d):
Definition Mele 2024)
Ju (U))du(()) J ((UV))pa(U)) - -rao f(VU)dμH(U)
(15)
Thus, the Haar measure assigns an invariant volume measure to subsets of locally compact topologi- cal groups. Due to properties of probability measure, it holds that ʃs du (U) 2 0, VS _ ((d)) and Ju(e) (U) 1. It is therefore possible to set the expectation value with respect to the proba- bility measure equal to the integral over the Haar measure as Eu-pn (f(T) = Suta f(U)dpa(U), This leads to the definition of the moment operator of the Haar measure.
Definition (Mele 2024) For all operators O, the 1-th moment operator based on probability measure HH is defined as follows.
M (1) (O) EU-wa u Outer)
(16)
Definition 7 (Mele 2024) A distribution u set of unitaries S S ((d) is a unitary t-design if and only if the following holds for all operators O.
S = Ev (U tut
(17)
State t-designs weaker versions of unitary t-designs in the that unitary t-designs induce state t-designs when they act on a fixed input state. These are defined as follows.
Definition 8 (Mele 2024) A state t-design is probability distribution n over a set of states if and only if the following holds.
E b (15) = B (|bb) (1 (
(18)
Further, we define an equivalent definition of state t-designs as follows, which will be used as well.
Definition 9 (Hoggar 1982) Let Cd denote a d-dimensional Hilbert space with orthonormal ba- sis set { Owing to their redundancy under global phase transformations and normal- ization, the quantum states in this space correspond to points in the complex-projective space CP2-11 (Bengtsson & Zyczkowski 2006). Thus, define the nontrivial complex-projective t-design of states S C CP. sampled according to probability measure μ., satisfying,
f()esf(w)) = =S f(l))
(19)
Where, f(()) is a polynomial of at most degree in its amplitudes and complex-ampliudes of vb) respectively. The canonical measure dey defined on the set of such quantumm states, the unique unit-normaiied volume measure invariant under unitary group action U(d).
Often is not strictly required to have an exact t-design, but rather a distribution close to a t-design, which is termed 6-approximate t-design.
17

---

Published as a conference paper at ICLR 2025
A.4 CHOICE POLYNOMIAL FUNCTION FOR STATE T-DESIGNS:
The function f((x)) is a polynomial of at most degree 1. For example, 2-designs, correspond to the average value of quadratic (second-degree polynomial) function. Hence, common choice of state 2-design polynomial functions is the overlap (ef. Appendix [A.2 for definition) between a collection
of states fısid the Hilbert space (bby/the) Similarly, for an even >> 2, a canonical choice of the polynomial function f is (1b ///))
B
WELCH BOUNDS AND T-DESIGNS
Theorem 1:
(Welch bounds) Let 2 d. If {ut) is sequence of unit vectors in Cd, then,
(0y (k) 12t IgjASn.jjk
n
1
14 E N,
(20a)
((+t-1
implics,
n²
Σ
K0ylow)
Vt E N.
(20b)
(d+1-1)
The combinatorial (binomial) factor featuring in the denominator of Equation 20a 20a and Equation can be expanded in terms of the factorials:
20b
(d + 1)! (att-1)=
(21)
t (d 1)! t!
Here! ! corresponds to the factorial symbol, i.e. n! = n(n 1) 2)...I
B. SIMPLIFICATION OF EQUATION 20A
Substituting Equation 2T into Equation 20a yields the following.
1
max KSR
(u>lvs> 21 2
Id+t-I1 1d-1)
(d+r- 1)dd+ 2)....d
1]
(22)
Here in the denominator, we have used the factorial property. (d+t-1)l//0) 1)1 = 1
t 2). - () - = 1 2)...d
B.2 SIMPLIFICATION OF EQUATION 20
Using Equation 21 it follows that:
2
Σ
Ko,lua))
(23)
(d+t l)(++ 2)...d
ISAASn
For the special case, /. = 2, we get the inequality for 2-designs.
n²
n
Σ
(eyloa)1
(24)
(d+1)
(d + 1)d/2
1SjASB
C CHANNEL SENSITIVITY BOUND
C.I PRELIMINARIES
Before deriving the bound, we want to state some preliminaries for partial derivatives parame- terized unitaries. Equation 25 shows the partial derivative of a circuit (Holmes et al.) 2022 which utilizes the convenient representation essentially splits the original unitary in two parts, and we define the fractions in Equation 26
unitary in terms its generator. The partial derivative
18

---

Published as a conference paper at ICLR 2025
0,U(0) 2
U HJU1
(25)
Un (
j=1
(26)
112
W;
j=1
C.2 DERIVATION OF THE BOUND
We can now analyze the difference in diamond norm upon slight perturbation of parameters in Equa- tion 27 We use Taylor expansion and truncate after the first order. We arrive at Equation 28 after applying the partial derivative from Equation 25 We use the triangle inequality and homogeneity, which are both necessary conditions for matrix norms (Hom & Johnson 1985). to arrive Equa- tion 29 and Equation 30 respectively. Arriving at Equation 31 is non-trivial, as, in general, it is not guaranteed that multiplying hermitian and unitary matrices results in a unitary. However, HEAs use Rx. Ry and Rz rotations with generators X, Y and Z respectively, which are known to be hermitian and unitary. Since unitaries form a group that is closed under multiplication, forms a valid quantum channel, which evaluates to I (Watrous 2018 Proposition 3.44).
dim(()
= (U(0) (())) + Σ
8
O
+ -0(83) 11
(27)
00j
==1
dim((8
OU(1)
~ (U(()) (T(()) +
8;)10
(28)
08,
dim(8)
OU(0)
=
8ا110
(29)
08,
dim(()
i
(30)
2
f=1
dim(E)
$ (1
(31)
1=1
dim
5,11lU541-ddmm H1U1++1lo
(32)
1=1
dimm()) Σ j=1
|5ا|
(33)
2
D COMPARISON OF EMPIRICAL CHANNEL SENSITIVITY AND BOUND
We compare the empirical channel sensitivity with the predicted bound Figure We add this figure to visualize that experimentally we can observe a large discrepancy, although the magnitude of this discrepancy makes it hard to read much more from the plot. We also would like to remark that one should be careful in drawing the conclusion that the bound is thus loose, as there might still be a point in the model space where taking such a step could achieve the bound. Our extensive experiments, however, reveal that this will likely not be the case in large areas of the model space.
19

---

Published as a conference paper at ICLR 2025
qubits
qubits
quiitt
qubits
Meseum
Σ, [6]/2
Disingusbabiliy
0.15
0.30
0.00
AetTuna Comment Son steee Jayur
Disingubbbbhiitt
0.15
0.10
0.06
0.00
Disingukbabilit
0.15
0.30
0.06
0.00
50
100
150
50
100
50
150
50
150
Step
Step
Steg
Step
Σ, |را 2
Figure Comparison
and channel sensitivity
E ANGLE ENCODING
We visualize the channel sensitivity when encoding the data with angle embedding. We run these ex- periments to demonstrate robustness of the channel sensitivity with respect the changed parameter updates that may occur due the different input states. While the channel sensitivity slightly varies, we can observe that its magnitude does not differ when comparing the results using amplitude embedding (we observe maximum distinguishability of approximately 0.5% in both cases).
F GAUGE FIXING
Considering the numerical instabilities whenever A 1. we apply Gauge-Fixing algorithm. In particular, we adjust the global phases the two unitaries, which ensures numerical stability without affecting the diamond norm.
Algorithm 1 Gauge- fixed diamond norm between difference of quantum channels
1: function DIAMOND NORM(
Bloo)
2:
if A & then
E B)
8r E R. + arctan Im(=)
4:
Re
5: B < exp (60')
6:
end
7:
return A B Blb
8: end function
20

---

Published as a conference paper at ICLR 2025
Angle Embediing: Distnguisbahiliit the Quaztuum State Step for Differest Numbers of Layers and Qubits
qubits
qubiit
qubits
qubita
0.005
Disingnishnaaiity
0.00
0.002
0.000
0,005
Detingnsssbittt
0.004
0.003
0.001
0.000
Distingusshabiittg
00004
0.003
0.002
0.001
0.000
C
100
Step
Step
Step
Stegp
Figure: Channel sensitivity for angle embedding
21


Reviews:
[{"review_id": "ltcLyXSoOM", "reviewer": "Reviewer_BvBg", "summary": "The study presents the authors' work on evaluating the expressivity of different ansatzes, while trying to propose novel measures and metrics to capture the behaviour of QNNs during training.", "questions": "Do the authors envision that researchers in the field would be able to leverage their findings to build better application-specific models? Could standard architecture search approaches be coupled with suitable distinguishability metrics to build better ansatzes?\n\nWhat proportion of the expressivity issues observed with QML models are related to the ansatz vs the data encoding?\n\nAs all experiments seem to be classical simulations, do the authors have an idea how much will noise when executing the model on a device contribute to expressivity issues?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper looks from a different angle to the well-known problem of training QNNs and presents new approaches to assessment of quantum model expressivity while discussing the need to develop architectures suitable for QML. This seems like an important work for the QML community.", "weaknesses": "The paper did not explain what particular elements or spatial organisation of gates of ansatzes may be the most contributing to expressivity or trainability issues observed in QNNs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EPZGobiJWw", "reviewer": "Reviewer_U5bz", "summary": "The paper considers methods of characterising the variational ansätze represented within quantum neural networks. The paper argues against the use of so-called 2-designs as a measure for expressivity that have been used in previous works. The work goes on to introduce a notion of channel sensitivity, which considers how much (in terms of a so-called diamond norm, a norm used more in Quantum Information theory) a unitary changes under perturbation of variational parameters. Bounds for the channel sensitivity are derived by consideration of Taylor series expansions and used to argue against the distinguishability of ansätze with few variational parameters on parameter perturbation. Numerical studies then investigate the notions introduced above in terms of both random perturbations of variatonal parameters, and also perturbations introduced during a training loop.", "questions": "Question:\n\n1. My understanding is that the diamond norm is derived to provide a channel distinguishability based on a single measurement of the state. Does this translate to distinguishability of a QNN model, where i) we might make use of several measurements and ii) one might make use of a specific measureable (e.g. single qubit magnetisation) rather than the whole state?\n2. What can be done with the proposed channel distinguishability metric? The fact that it can be bounded from above for small perturbations does not seem useful. All neural networks (classical or quantum) typically won't change too much in a single training step---why is upper bounding this change in terms of some norm useful to a practitioner?\n\nSuggestions:\n\n2. I think section 3 and 4 can be elaborated on more. In particular, the criticism of 2-designs as a measure of expressivity is hard to read without going back and forth between the main text and the appendix several times.\n3. Elaborating on section 3 and 4 would probably need more space. Some of the figures can be moved to the appendix, e.g. the three rows of figure 2 are conveying a similar message, so perhaps two of the rows can be placed in the appendix? Same with figure 3. The conclusions and discussion could be shortened as well.\n4.  Ensure that all figures are included in a vector format or lossless format. Maybe I'm not seeing things right, but the quality of the figures seems a little bit off.\n5. A minor point unrelated to the core contributions of the paper. But amplitude encoding of features into a QNN feels like a non-standard way of conducting the numerical experiments. Apart from amplitude encoding not being near-term friendly, the resulting QNN will not be particularly expressive. I.e. one will end up with something like `<PCA|Hermitian|PCA>`, which, since the PCA is a linear combination of features, will result in a sum of pairwise interactions with no zeroth, first or cubic and higher order terms. Encoding PCA components into single-qubit rotations would be a more standard approach, to my knowledge.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. The introduction and background material are particularly well-written and offer a broader ICLR readership a succinct way of learning about quantum neural networks. However, it is possible that too much space has been allocated within this area at the expense of a fuller exposition of novel aspects of the work.\n2. The work takes aim at foundational issues in variational quantum circuits and the points made are widely applicable to QNNs as a whole.\n3. The proposed channel sensitivity metric based on diamond norms seems to be aiming at something principled, in that it seeks to characterise the action of a unitary on any state. Its application to variational quantum circuits, to my knowledge, is novel.", "weaknesses": "1. Sections 3 and 4 feel like they should be the main contributions of the paper, but they squeezed into about a single page. For example, precise mathematical definitions of the how the quantum states in Eq (4) are missing from the main text, which makes it hard to follow.\n2. The work as a whole considers expressivity of a variational quantum circuit as a problem of the ansatz and does not give any weight to the importance of feature encoding into the quantum circuit. It is known that the spectral representation of a QNN is directly given by the feature encodings ([Schuld, 2021](https://arxiv.org/abs/2008.08605)).\n3. The diamond norm under parameter change seems like an interesting metric, but it remains unclear what can be done with it.\n4. Similarly, the bound on the channel distinguishability under small perturbations seems not to be useful. It seems natural that the QNNs will only change gradually under small parameter perturbations such as those encountered during training, so what use is an upper bound?\n5. The main points of the work seem somewhat incongruous: the criticism of the 2-design as an expressivity metric seems like it should give way to an alternative expressivity metric. However, the channel distinguishability metric proposed does not provide an expressivity metric.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XE7Semkbfd", "reviewer": "Reviewer_kPLW", "summary": "The paper studies quantum neural network expressivity and model distinguishability. The aim is to better understand, at least for small architectures, the model space of so-called Hardware Efficient Ansatzes (HEAs) for quantum neural networks. To do this, the authors first argue why 2-design is an inadequate measure for model expressivity.  Then they define channel sensitivity to measure the effect of parameter changes on quantum neural network output using the diamond-norm from the literature. A Taylor-expansion-based argument allows to provide a natural upper bound on channel sensitivity. Moreover, the paper conducts an extensive numerical study to experimentally measure channel sensitivity for standard HAE quantum neural networks. As its main message, the paper states that its results ultimately suggest that iterative training of small quantum models may not be effective.", "questions": "1.\tWhich optimizer (and optimizer hyperparameters) did you use in the experimental results? \n2.\tHow do the empirical results depend on the choice of optimizer and its hyperparameters?\n3.\tWhere does the factor $\\frac{1}{2}$ in (30) come from? Inserting (25) into (29) would give (30), but without additional factor $\\frac{1}{2}$.\n4.\tWhy do you use mean-squared loss, even though you are solving a classification problem?\n5.\tFrom its definition, the diamond norm seems to depend on number of qubits. How does an increase in number of qubits affect the diamond norm? \n6.\tCan you provide an additional figure, which is the same as Figure 3 but without the upper bounds? \n7.\tThe reported results are averaged over architectures with the same number of parameters. Can you be more specific about what type of \"confidence intervals\" you show in the plots?\n8.\tFor comparison reasons, it would be very insightful to analyse channel sensitivity of a non-entangled circuit. For example, for a circuit applying an $R_X$ gate to each qubit independently, what channel sensitivity do you get for $1,2,3,4$ qubits?\n9. How do the results depend on your choice of input-encoding?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "Overall, the proposed research question appears relevant, as a theoretical understanding of quantum neural networks is, to a large extent, still missing. The paper is mostly clearly structured and well-presented. Although limited to small architectures, the considered setting is still of interest, given the hardware limitations. Numerical study considers several important combinations of architectures.", "weaknesses": "Some of the key conclusions drawn from the numerical results are only partially justified by what is presented in the paper, weakening the overall significance of the reported results. More specifically:\n\n-\tSome important details regarding the setup of numerical experiments (employed optimizer, optimizer hyperparameters) and results (confidence level for confidence intervals) are currently missing in the paper. Also, it appears that robustness with respect to choice of optimizer / hyperparameters has not been studied. \n-\tThe upper bound on channel sensitivity is an upper bound, not an equality. Some of the conclusions drawn in the discussion/abstract/introduction would only be valid in case of equality. For example, in the Discussion section, the paper states that channel sensitivity \"scales with (1) the magnitude of changes and (2) the number of parameters.\" This statement is true for the upper bound, but not necessarily for the channel sensitivity, as the upper bound is very loose (as the experiments show) and no lower bound seems to be available. Similarly, the introduction the authors write as one of their four main contributions: \"we introduce a measure for the distinguishability of an ansatz upon parameter update and provide an upper bound. It shows that many parameters are required to be able to distinguish the two operations, contrasting with the initial framework.\" This would be true in case of an equality, but an upper bound (which is loose, as the experiments show) does not seem sufficient to support such a claim.\n-\tAs one of their four main contributions, the authors state \"We provide numerical evidence that each ansatz is hardly distinguishable from itself\". This seems to be based on Figure 3. Since Figure 3 also shows the (loose) upper bounds, the different scale of bounds and channel sensitivities in Figure 3 makes it impossible to see this solely from the figure.\n-\tTerminology is slightly confusing:  in Holmes et al (2021) the term expressibility was used. For classical neural networks, expressivity refers to their capability of approximating arbitrary given unknown functions. This is a fundamental property for choosing which neural network architecture to use, but does not seem to be discussed in the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6sa4KBVny5", "reviewer": "Reviewer_E1y9", "summary": "The authors introduce an \"expressivity\" measure of quantum neural network (QNN) architectures that they argue is more natural than those previously explored in the literature. One example of the latter is a difference measure between the given QNN architecture when randomly initialized and a $t$-design (for some small $t$); the authors show that having a large expressivity according to such a measure does not necessarily indicate that the model explores a large subspace of Hilbert space. This motivates their introduced definition, which is a sensitivity measure in diamond norm of the quantum network upon perturbing the network parameters. The authors then perform some numerical experiments indeed demonstrating that this measure tracks with how one would intuitively expect \"expressivity\" to behave (it grows with the number of layers and trainable parameters per layer, and so on).", "questions": "Is there a natural reduction of certain, previously-explored measures of expressivity to the authors' introduced measure of expressivity? For instance, I believe demonstrating that having large expressivity based on a measure of closeness to the Haar measure, and showing this implies a large expressivity according to the authors' definition (averaged over parameter space), would strengthen this work.\n\nFurthermore, what relative advantages do the authors' introduced expressivity measure have over expressivity measures based on closeness to the full (i.e., not $t$-design) Haar measure? This impacts the broader reach of this work, as to me it is currently unclear what the relative merits are of the introduced measure to this previously-studied measure.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "Understanding measures of expressivity for QNNs is important for understanding the to-date loosely-defined \"expressivity-trainability trade-off\" known to exist in quantum networks (see, e.g., arXiv:2312.09121 and arXiv:2406.07072). The authors, in a very clear fashion, describe shortcomings of certain previously-introduced measures of quantum network expressivity; given these shortcomings, the authors then introduce a novel, natural notion of expressivity that circumvents these shortcomings.", "weaknesses": "Though not the fault of the authors, the introduced measure of expressivity has the unfortunate side-effect of being difficult to estimate; this limits the scope of their numerical experiments. It also makes the relative advantage of the authors' introduced expressivity measure over an expressivity measure based on closeness to the Haar measure (rather than a small-$t$ $t$-design) unclear, given both are difficult to estimate.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)