Quantum Deep Equilibrium Models
Authors: Philipp Schleich
Marta Skreta
Lasse Bjørn Kristensen
Rodrigo Vargas-Hernandez
Alan Aspuru-Guzik
Venue: NeurIPS 2024 poster (2024)
Keywords: quantum
deep equilibrium models
quantum machine learning
Primary area: machine_learning_for_physical_sciences

Abstract: The feasibility of variational quantum algorithms, the most popular correspondent of neural networks on noisy, near-term quantum hardware, is highly impacted by the circuit depth of the involved parametrized quantum circuits (PQCs). Higher depth increases expressivity, but also results in a detrimental accumulation of errors.  Furthermore, the number of parameters involved in the PQC significantly influences the performance through the necessary number of measurements to evaluate gradients, which scales linearly with the number of parameters.
   Motivated by this, we look at deep equilibrium models (DEQs), which mimic an infinite-depth, weight-tied network using a fraction of the memory by employing a root solver to find the fixed points of the network. In this work, we present Quantum Deep Equilibrium Models (QDEQs): a training paradigm that learns parameters of a quantum machine learning model given by a PQC using DEQs. To our knowledge, no work has yet explored the application of DEQs to QML models. We apply QDEQs to find the parameters of a quantum circuit in two settings: the first involves classifying MNIST-4 digits with 4 qubits; the second extends it to 10 classes of MNIST, FashionMNIST and CIFAR.  We find that QDEQ is not only  competitive with comparable existing baseline models, but also achieves higher performance than a network with 5 times more layers. This demonstrates that the QDEQ paradigm can be used to develop significantly more shallow quantum circuits for a given task, something which is essential for the utility of near-term quantum computers. 
   Our code is available at \url{https://github.com/martaskrt/qdeq}.

Quantum Deep Equilibrium Models
Philipp Schleich"
Marta Skreta"
Department of Computer Science
Department of Computer Science
University of Toronto Vector Institute
University of Toronto
Vector Institute
philipps@cs. toronto. edu
martaskrretascs toronto. edu
Lasse B. Kristensen Department Computer Science University of Copenhagen
Rodrigo A. Vargas-Hernánder
Department of Chemistry & Chemical Biology McMaster University, ON
Alán Aspuru-Guzik
Department of Computer Science
Department of Chemistry University of Toronto Vector Institute
Abstract
The feasibility of variational quantum algorithms, the most popular correspondent of neural networks on noisy, near-term quantum hardware, is highly impacted by the circuit depth of the involved parametrized quantum circuits (PQCs). Higher depth increases expressivity, but also results in detrimental accumulation errors. Furthermore, the number of parameters involved in the PQC significantly influences performance through the necessary number of measurements evaluate gradients, which scales linearly with the number of parameters. Motivated by this, we look at deep equilibrium models (DEQs), which mimic an infinite-depth, weight-tied network using fraction of the memory by employing root solver to find the fixed points of the network. In this work, we present Quantum Deep Equilibrium Models (QDEQs): training paradigm that learns parameters of quantum machine learning model given by a PQC using DEQs. To our knowledge, work has yet explored the application of DEQs to QML models. We apply QDEQs to find the parameters of quantum circuit two settings: the first involves classifying MNIST-4 digits with qubits; the second extends it 10 classes of MNIST, FashionMNIST and CIFAR. We find that QDEQ is not only competitive with comparable existing baseline models, but also achieves higher performance than a network with times more layers. This demonstrates that the QDEQ paradigm can be used develop significantly more shallow quantum eircuits for a given task, something which is essential for the utility of near-term quantum computers. Our code is available stlhttps:/ /tt::///ttttie //github... wwt.//aaaatttttgggeeee...
"Equal contribution. Author order was sampled from quantum computer.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Introduction
Quantum computing holds a lot of theoretical promise for transforming the computational landscape of variety of applications. Machine learning is one of these applications (Biamonte al. (0017). enabled by the capability of quantum computing to handle large amounts of data with significant quantum speedups and by the fact that dimensionality of the output is typically much smaller than the one of the input and during processing, a requirement to recover linear-algebra relevant speedups (Aaronson 2015). Although there have been tremendous advances in hardware development in recent years, the time of full-stack, eπтоr corrected quantum computers which would enable such unear-algebbg speedups is still not yet within tangible reach. Therefore, advances in near-term quantum algorithms for noisy devices with short coherence times remain one of the most promising avenues toward effectively harnessing quantum computing for real-world applications.
The arguably most studied class of algorithms in this line of research is the class of variational quantum algorithms (VQAs). While they originated as a means of physics-inspired learning without data (Peruzzo et al. 2014). they provide the most natural way to extend classical learning models to hybrid quantum-c lassical setting. In VQAs, a parametrized quantum circuit (PQC) together with measurement of suitable observables on the quantum state after the circuit gives rise to function class. Through the choice of a quantum circuit, observables, and loss function, this setup allows different problems to be tackled, with classical optimization of the loss function as the training step. Within this work, we use a VQA for classification tasks. Challenges within VQA training continue be limited circuit depth, which reduces the the expressibility and trainability of the circuit, and disadvantageous loss landscapes, influenced by induced entanglement and the specific loss function (McClean et al.) 2018; Fontana et al. [024]] Ragone et al.] (0224) Further, while the parameter-shin rule allows us to evaluate gradients exactly using a simple finite-difference formula, there a significant measurement overhead to determine the gradients. Thus, striving for training methods that enable the use of shallower circuits with less independent parameters at similar performance is paramount.
The main contribution of our work is the proposition of a class of quantum deep equilibrium models (QDEQ). To our knowledge, this is the first application of deep equilibrium models (DEQ) as first introduced in Bai et al.] (201)) in the context of quantum computing; so far, previous work has explored implicit differentiation techniques for PQCs (Ahmed et al. 2022). Implicit and adjoint methods such as DEQs in machine learning stand out by their memory eficiency compared to their explicit counterparts. While this is still true for QDEQ, we see the main benefit in the reasons outlined in the paragraph above, i.e., the ability to use shallower circuits with fewer independent parameters to tackle given problem. We will give some theoretical intuition why DEQ can be expected to perform well on a specific family of quantum model functions in Section 2.3 design a set of numerical experiments in Section 3 and obtain numerical results that confirm this hypothesis in Section 4
2 Background and related works
2.1 Deep Equilibrium Models
Based on the fact that a neural network comprised of L layers is equivalent to an input-injected, weight-tied network of the same depth, Bai et al. (2019) introduced the concept of Deep Equilibrium Models. Assuming that the respective layer fo has a fixed point (Winston and Kolter, 2020).
instead of explicitly repeating the weight-tied layer I. times, we solve
fo(2": x) - zz = (x*:xx)
using a black-box root finder such as Newton's 's or Broyden's 's method. The latter is comprised by Newton iterations where the Jacobian is approximated by a finite difference formula (Broyden 1965). Finding this fixed point corresponds to evaluating an infinitely deep network, in the sense that lim (5) = lim (fe o......o (z) =z"
1. times
When training this model with respect to a loss function f, even though we use a root-finder to evaluate the model, we still need to be able to differentiate the procedure for training. Instead of explicitly differentiating through the L. repetitions of the same layer (we call this later the DIRECT solver), Bai et al. (2019) make use of the implicit function theorem (Krantz and Parks 2013) to
2

---

perform implicit differentiation in the following sense. During backpropagation, the gradient of the loss function can be expressed as:
0r de
[z((z*;x)
00 Oz (J lx.
00
with (J 12. JI- lx-) )x-)__ While the last factor in Eq. 2 can be computed by standard
automatic differentiation through the model function, determining 2h
o6 (J lx
either requires
assembling the full Jacobian and inverting it (which is quite expensive) or solving an additional
root-finding problem during the backward pass in form of solving for in the equation
de
(J 12 )9"
0,
Oz
with gradients computed efficiently using the vector-Jacobian product.
The overall procedure of a DEQ model thus is to minimize a loss function & given input x, target y. and hypothesis class in the following way: In the backward pass, we use the gradient expression in Eq. (2) and determine the first factor using black-box root finder on Eq. ( Using these gradients, the model is trained. The model is then evaluated on new input by using a root-finder to solve Eq. ( brief overview of this procedure in Fig. [
It was found that pre-training the DEQ model using a shallow (2-layer), weight-tied DIRECT approach that is, explicit application and training of fo f considerably aid the overall convergence behaviour (Bai et al. 2019 (020). This pretraining step a cheap way to give the model reasonable weights as starting point (called warm-up), but the model is too shallow to achieve the best performance, In this work, we will utilize this warm-up strategy. then use implicit differentiation to optimize the loss and obtain better accuracy. We call this the IMPLICIT-WARMUP solver, and refer to training using only implicit differentiation as the IMPLICIT solver.
An important step to ensure stability of the root-finding algorithm is regularizing the Jacobian, as introduced in Bai et al. (202I). Further advances include a convenient inclusion to pytorch through Geng and Kolter (2023) and certifiable (Li et al. 2022) and robust (Chu al. 2023) DEQ models as well as DEQ for diffusion models (Pokle et al. 2022).
2.2 Quantum Models and Variational Algorithms for Classification
Variational quantum algorithms rely on PQC over Q qubits, which span the finite-dimensional Hilbert space C22. Then, {1i)) (1a1) & (i2) @ ... (
an orthonormal basis for this space and (il is the conjugate transpose to a vector (t). We denote the PQC, parametrized by P parameters, by the Q-qubit unitary operation U(6) E C2 Then, one can define an objective (Ø) (0) (O) (d) that is to be variationally minimized, induced by a Hermitian matrix M E C29x29 and an initial state (e) E C2 (Bharti et al. 2022 Cerezo et al. 2021p. Quantum machine learning models have already been the target a fair amount study, yielding insights into several properties, including generalization bounds (Caro et al., 2022), their training in general (Beer
ctt al. 2020), and interpretability (Pira and Ferrie (004)).
So far, we have not discussed how this framework is able to incorporate data into the training; here, this role will be filled by the initial state (o). Given classical data x E R", there exists unitary data encoding circuit Sx that maps the data in some fashion onto a quantum computer and stores it in state ix); there is a variety of techniques to do so, which e.g. stores the data in the amplitudes (coefficients) of the state, applies rotations parametrized by the data, etc. (Schuld et al. 2020 Schuld 2021.. Then, we can define a class of quantum model functions
Definition 1 (Family of Quantum Model Functions). Let (0) be a unitary circuit over Q qubits
parametrized by E RR, x E R features and Sx : (0) ~> (x) a data encoding. Then, we define a quantum model function as
(XU)))
(4)
for a Hermitian observable M. In order to distinguish K labels for a classification problem, we need ensemble of size K. To that end, we define map R' : C2 x2² - R that describes measurement of expectation values with respect to an ensemble of K observables {M} and storage of the respective outcomes in a K -dddiensional vector. Typically, string this together with
3

---

ming {(x, ys fo)
root-finder _
input
output
fo(z) z g0(z)
fo(x) = (olU' x) MU x)
10)
: (0)
U(0,x)
Figure Instance of a Deep Equilibrium Model using a quantum model family. A black-box root-finding method is used to determine the model function's equilibrium state.
upsampling isometry I, so that R = Iy R' maps to R", which allows repeated calls to reach multiple layers. This gives rise to a family of model functions c
fo (x) R (U XXX (0){M))
(5)
We note that the model given in Definition I combined with activation functions and bias vector was shown be universal in Hou et al. (2023). whilst[Schuld al. (2020) provides evidence that for classification problems, as we will consider below, the architecture in Definition is sufficient.
Training of such model proceed equivalently to the training of variational models (Bharti et al. 2022 Cerezo et al.) (021). where one defines loss functions based upon the outputs of the
quantum model family fo While backpropagation is possible in QML framework, it is rather uncommonly done. As was shown in Abbas et al. (202T). achieving the computational efficiency of backpropagation in classical learning models also in quantum models turns out be very challenging and resource demanding. Most approaches to gradient evaluation thus rely on the so-called parameter- be computed exactly using approach
shift rule (Schuld and Killoran| 2019), where gradients similar to central finite differences. The requirement for this to work is that the generator of each unitary gate (0)) e - i has eigenvalues ir. Then, Jo r[Jo+s fo-s] for = 4F where we note that for p parameters 0 E RR, this gradient computation needs to be carried out for each parameter individually. Generalizations beyond two symmetric eigenvalues have been done and are further development (Kottmann al.) 202T: Hubregtsen et al., 2022; Wierichs et al. 2022] Anselmetti et al. 2021 Kyriienko and Elfving (021).. The gradient computation here needs to measure two expectation values per parameter up to sampling accuracy via the parameter-shiff rule. For considerations on an implicit model, we expect similar measurement overhead for the extraction of the Jacobian through finite difference approximation in Broyden's 's method. When parameter shift rules applicable, the finite differences in Broyden's 'ss method can be made exact up sampling accuracy as well.
2.3 A Quantum Deep Equilibrium Model
We next introduce Quantum Deep Equilibrium Models, based on the notion of DEQs with the quantum model families from Definition]] ! Fig. [ depicts the overall process: the course of minimizing a loss function, consider the solution of root finder of the quantum model family as the hidden layer.
While QDEQ is not restricted classification tasks, this what we will use as the context in our work. To perform classification, we use measurement ensembles of dimension K, as necessary for K different classes. Additionally, since the quantum basis scales exponentially the number of qubits, we will often keep K significantly below this exponential threshold. Note that one option for

---

estimating these observables would be to use shadow lomography to obtain a set of classical shadows that can be used for observable estimation (Huang et al. (020).
Given the considerations above, one choice for a measurement ensemble is a set of basis state projections [ This choice allows one to extract more data than the number of qubits; in fact, up to as many as the number of basis states, however this comes with a similar cost as state tomography, thus ideally we choose K well below 29, Another choice of ensemble, which fits the framework of DEQs with input-injection better are Pauli measurements. This is because every Pauli measurement result falls within |-1: 1] per qubit and thus directly gives notion of perturbing the injected input towards smaller and larger. Basis state projections yield strictly positive measurement results, allowing only positive perturbations of the input injection. and tend be smaller in magnitude as they represent probabilities. In the sense of Lloyd et al. (2020), the former corresponds to the fidelity classifier and the latter has similarities with what they call Helstrøm classifier.
We will use quantum model functions for classification into K classes and build our arguments on the following observations, which we discuss in more detail in the appendix:
(i) Such classes of models admit fixed-points, necessary condition for DEQ frameworks to be successful. See Observation 2 The argument is based on the property how quantum models encode and extract information through measurements.
(ii) Weight-tied QDEQ with input injection is equivalent to a sequence of independently parametrized layers. See Theorem 3 extending Theorem 3 in Bai et al. (2019).
In combination, this gives evidence that the quantum model in Definition | with a measurement ensemble of length K can be successfully applied classification using DEQ strategies. We are able to confirm this through our numerical experiments in the next section.
The layer depicted in Fig. [ also contains measurement operations. As such, QDEQ does not correspond to evaluating an infinitely deep unitary quantum circuit; a layer here means quantum model including encoding and measurement. Direct application of the ideas of this paper in the context of infinitely deep circuits is hampered by both the conceptual difficulty of reasoning about fixed points for unitary isometries and the exponentially scaling effort required to extract and compare descriptions of quantum states, beyond classical shadows Huang et al. (2020).
3 Experiments
3.1 Datasets
We consider three datasets in this study. First, we consider MNIST-4, which consists of classes
of MNIST digits (0, 3, 6, 9) (Deng 2012). We then extend our model all 10 classes of MNIST (MNIST-10), as well as FashionMNIST (FashionMNIST-I00 (Xiao et al. 2017). Finally, we tested our setup on natural images with 10 train/test split& and randomly split the training set into 80% train, 20% validation.
al. (2009). For all datasets, we used default
3.2 Architecture
We replicate the circuit from Wang al. (2022a) as architecture, shown in Fig. 2 The circuit consists of four qubits. We downsample the MNIST-4 images from 28x28 to 4x4 using average pooling. For encoding, we look at the options of angle encoding through rotation gates, in Wang et al. (2022a), or an amplitude encoding, as implemented in torchquantum (Wang et al. 2022a We pass the information through parameterized quantum gates and, finally, we measure the qubits to get a readout value from the circuit. Thus far the definition of our quantum model. Then, we transform this readout value using small classifier head, which consists of a linear layer, then pass the result into a cross-entropy loss. As in Bai et al. (2019). we also incorporate variational dropout in the classifier head. For simulations with 10 classes, we extend this circuit by repeating the four-qubit circuit with a stride of 2 qubits in stair-case manner, as shown in Fig. 3
The measurement ensemble choose for simulations is also in alignment with Wang al. (2022a), namely a Pauli-Z 7 matrix per qubit, i.e. {M} = {Z @ IQ-1, I 8 Z & TQ-2..... To Ø Z).
htte:://gyoooc...g.g.//ssss/ssss html
5

---

Note that the baselines Dilip et al. (2022); ; Shen et al. (2244) use linear maps based on measurements of basis state projections. This introduces another layer that needs training and. most importantly. there is a risk that more general observables induce barren plateaus due to non-local support (Fontana (et al. 2024, Ragone et al., 2024). Furthermore, their measurement will be increasingly costly as overlap measurement through Hadamard tests is more involved that single-qubit observables.
When comes to data-encoding, we tested both angle and amplitude encodings for the 4 and 10 qubit experiments and generally found that the performance of the amplitude encoding is superior, both for DIRECT and IMPLICIT scenarios. Thus, we present the results for the datasets with 10 classes in the subsequent section only for amplitude encodings and show both for MNIST-4.
3.3 Models and baselines
For each dataset, we test our setup on six model variations. The first is our IMPLICIT framework. The next four are DIRECT solvers, which weight-tied networks consisting of L repeated layers of the quantum model, where L E {1,2, 2. 5, 103. We explicitly differentiate through these layers to compare with implicit differentiation and understand whether increasing network depth results in better performance. The intuition is that if DIRECT with L repetitions shows a positive trend in performance on task with increasing L, applying the IMPLICIT solver should be expected to be beneficial. Finally, it was found in (Bai et al. 2019. 2020) that pre-training the DEQ model using a shallow (2-layer), weight-tied DIRECT approach can considerably aid the overall convergence behaviour. This pretraining step is cheap way to give the model reasonable weights as starting point (called warm- up). but the model is too shallow achieve the best performance. We then use implicit differentiation to optimize the loss and obtain better accuracy. We call this the IMPLICIT+WARMET
solver.
In the IMPLICIT approaches, we pass the measurements from the quantum circuit into the root finder to get z", which we then pass to the classifier head and update the parameters using implicit differentiation. We also inject the original image into every iteration of the Broyden solver, in alignment with the arguments in Bai et al. (2019) and our universality theorem in Theorem [3] as input-injected weight-tied models can be phrased equivalently We train the implicit models using a Broyden solver for at most 10 steps. For optimization, we use Adam (Kingma and Ba 2014) and cross-entropy loss. We trained each model for 100 total epochs (i.e. if we first pre-trained using warm-up epochs, we then trained using the implicit framework for 100 x epochs) (for CIFAR-IO, we only trained for 25 total epochs since we found converge faster). We selected hyperparameters using the validation set, see Appendix
models that are not weight-lied.
Finally, for each dataset, report the results of baselines from literature. In relation to choice of baseline, the field of quantum computational image classification is rich, with several architectures having been developed for this specific purpose, often inspired by classical convolutional neural networks (Liu et al., 2021: Henderson et al.) 2020). While such models can display impressive performance the tasks investigated here, they often employ significant classical and quantum resources in the form of classical neural networks or multiple different quantum circuit evaluations. More importantly, they represent highly specialized architectures. Since the goal of the paper is to investigate the applicability of DEQ training on general quantum models, the models chosen for this study were not highly specialized for image classification. The merit of DEQ should therefore be evaluated by comparison to results for similarly general models rather than the state-of tatte-of-thhaatt the-art accuracies of larger specialized models of 97% on MNIST (Henderson et al. 2020 and 83% on MNSSI (Jing et al. 2022)
Baseline for MNIST-4 For MNIST-4, we can directly compare our results Wang et al. (002b) as we chose the same circuit. The difference compared to a DIRECT realization of our model with one layer is in the ultimate classification layer. The architecture can be found in Fig. for more details we refer to Section 3.2
An alternative baseline is available in West et al. (004)). For the MNIST-4 case, where they use different classes (digits 0-3) than we did, they use five qubits, compared to four in our case. Data is encoded in amplitudes. The variational circuit they use for training is built from general unitaries and they mention that they employ 20 layers.
Baseline for MNIST-10 As the topic under evaluation is the concept of QDEQ training, the most applicable comparison with the same circuit trained using DIRECT solver. However, we also include results by Alam et al. (202I) using a model of similar complexity. Their approach similarly

---

uses 10 qubits, as well as similar measurement ensemble. However, beyond the training strategy. the two approaches differ in encoding strategy. While we perform a simple image scaling
N = 10 x 10 100 pixel image and then do -depth amplitude encoding in (log N)) qubits, they do simple depth-1 angle encoding in O(N) qubits, facilitated by using more complex image compression by either PCA or trained variational autoencoder. Since they demonstrate that compression scheme has a large impact on performance (see Table this makes direct comparison somewhat difficult. As for MNIST-4, West et al. (2024) is another baseline here; for MNIST with all ten classes, they used 200 layers in training, making the circuit significantly more complex than ours.
Baseline for FashionMNISTT-10 For FashionMNISTT, we look at the classifier circuit from Dilip al. (2022) as a baseline because their smallest setup comes closest to our architecture, consisting of circuits of comparable scale and number of qubits. However, the comparison is again difficult, since the results vary considerably depending on the quality of input state encoding they choose. For their encoding, quantum circuits are trained to generate approximations of amplitude-encoding- like states, called an FRQI states (Le et al., T0 corresponding to each image. Nevertheless, high-quality FRQI states may be comparable to the amplitude encoded states used in our work. Similar work in Shen et al. (2024) additionally provide results either using amplitude encoding or using approximate FQRII and similar classifier heads to our setup. While these results use circuits with a larger number of trainable parameters (thus, likely higher expressivity). they are included for context as well. Finally,
West et al. (224) is again another baseline with equivalent setup to MNIST with all ten classes.
4 Results
In this section, we report on the result of our model on the datasets MNIST-4, MNIST- 10, FashionMNIST-10,, and CIFAR-10. As mentioned, all results were generated using the torchquant um framework (Wang et al., (022a). Note that in the sections below, while the memory observed is in terms of classical memory that has been used for simulation of the QML model, this is a rough measure for the number of parameters in the optimization and thus also quantifies the measurement overhead for QML models. Thus, estimate, comparing different approaches relative to one another, for the necessary resources for evaluation on quantum hardware.
expect this quantity to be reasonable
data encoding
parametrized gates
flxed gates
catta encoding
parumetrized gates
gates
R5
Ry
Rx
10:
Rx
Ry
Ryy
N2
83
Rx
Ry
Ry
Sandor Layaz
50 gazz from
Ry
Ra
[0]
Amplitude
encoding
5q. Randdetaaay
My.
30 gaioa iion
Ry
Ra
S
Rv
Rx
Rx
Ry
Rs
[0:
Rs
(a) 4x4 YZXY Angle encoding.
(b) Amplitude encoding instead of angle en- coding from (a). using the implementation in torchquantum.
Figure 2: Circuit used for classification with up to 4 classes, following Wang et al. 2022a)). Blue circuits correspond input, purple and red shades parametrized and trainable gates and grey to fixed gates. The RandomLayer has on average 12.5 = 50/4 two-qubit gates (CNOTs).
4.1 MNIST-4
In Table I we show the performance of our proposed QDEQ framework on the MNIST-4 test set. We observed that for angle encoding, deeper DIRECT models seems perform better than shallow ones, as expected. However, the IMPLICIT models outperforms all of the DIRECT models, including the baseline. In contrast, amplitude encoding has the shallow DIRECT models perform best, and better performance in general. This may indicate both that deeper DIRECT models can suffer trainability issues compared to shallower DIRECT models, and that QDEQ does not always provide a benefit. However, we will for MNIST-10 see below, these two effects are not linked, and need not co-occur. In addition to this, we provide a result from West et al. (0024) with circuit of 20 layers, thus expectedly much deeper and more variational parameters than our circuit. They achieve an accuracy
7

---

10)
Rx
Rv
(0)
(9)
R2
Ry
(0)
encoding
50 from
Rx
(0)
Rg
Rl
10)
Rx
Rx
(0)
Rs
II
R2
(0)
Figure Extension of the circuit presented in Wang al. (2022b) to scenario of 10 qubits. Similarly to the tensor-nette tensor-network network inspired circuits in Dilip et al. (2022). we repeat the four-qubit stencils in staircase manner.
of approximately 90-92%. This supports our hypothesis that the QDEQ model is able achieve high accuracies (up to 93,4%) at lower cost.
Table Model performance on MNIST-4. Memory refers to the maximum GPU memory occupied by tensors. Runtime was calculated over 100 epochs on NVIDIA RTX 2070 GPU. We present results both for an amplitude and angle encoding as shown in Fig.
Test accuracy
Memory (MB)
Runtime (scc)
Model
Residual
(%)
Amplitude Encoding
QML Circuit [Wang et al. (
85.3
24.49
4257.17
torchquantum example
IMPLICIT solver [ours]
92.9
22.0
3206.43
2.395e-4
IMPLICIT-WARMUS solver [ours]
93.4
22.0
5276.61
6.982c-4
DIRECT solver- 10 layers [ours]
91.8
33.3
7820.00
9.877e-4
DIRECT solver layers [ours]
91.0
26.1
6013.00
0.189
DIRECT solver layers [ours]
92.1
21.9
3126.66
1.003
DIRECT solver layers [ours]
93.5
20.4
2566.00
2.867
Angle Encoding
QML Circuit Wang et al. (0022b)
77.3
24.49
4762.98
IMPLICIT solver [ours]
85.8
21.23
3206.43
5.115e-4
IMPLICITWARMUP solver [ours]
86.7
22.55
2306.72
1.037e-3
DIRECT solver- 10 layers [ours]
84.8
39.38
3581.83
1.047
DIRECT solver layers [ours]
85.3
28.86
1926.27
1,818
DIRECT solver layers [ours]
82.7
22.55
1012.66
3.625
DIRECT solver layers [ours]
83.9
20.45
704.38
4.135
4.2 MNIST-10 and FashionMNIST--U
We then extend our circuit from classes to 10 and show the performance on MNIST-1O (Table and FashionMNIIST-10 (Table B.. In both cases, the IMPLICIT framework performs better than DIRECT circuits that are 3 or more times larger terms of memory required.
For MNIST-10, 10, we observe that the deeper models using DIRECT training perform comparably or slightly worse than the shallowest DIRECT models. Similarly for MNIST-4, this could indicate suboptimal training for deeper circuits. In contrast, the IMPLICIT models seem to circumvent

---

this problem, and slightly outperform both the DIRECT models and the baseline using encoding. The VAE-based baseline on the other hand shows significantly higher performance. showing that such approaches to data compression (and their combination with QDEQ) may be worthy target for further study. The results in West et al. (2024) for the all MNIST classes reach up to 79.3% (Figure 2 in their paper); while this is more accurate than ours (up to 73.68%), this can be ascribed to the vastly larger model.
For FashionMNIST- we observe performance differences between DIRECT and IMPLICIT models very similar the MNIST-10 case. Furthermore, performance is competitive with the baseline results Dilip et al. (2022), possibly reflecting larger similarity between the encoding methods (amplitude encoding and approximate FRQL, respectively). The case for West et al. (2024) the same as for MNIST above: with an accuracy of 74.5% (Figure 2 in their paper). they surpass our best test accuracy of 72.1 %2 while again, their model far larger. Similarly, we ascribe the performance delta with Shen et al. (024) to similar difference in the number of trainable parameters.
Table 2: Model performance on MNIST-10. Memory refers to the maximum GPU memory occupied by tensors. Runtime was calculated over 100 epochs on NVIDIA RTX 2070 GPU.
Test accuracy (%)
Memory (MB)
Runtime (sec)
Model
Residual
QML Circuit (VAE) Alam al.
2021
89.80
QML Circuit (PCA) Alam et al.
2021
71.75
IMPLICIT solver [ours]
73.68
273.1
7301.74
5.598e-3
IMPLICIT-WARMUT solver [ours]
73.33
273.1
7017.38
1.583e-3
DIRECT solver- 10 layers [ours]
71.18
1233.3
2856.94
1.905e-4
DIRECT solver layers [ours]
72.46
628.8
1737.82
2.093e-3
DIRECT solver layers [ours]
72.07
266.1
4304.54
0.596
DIRECT solver layers [ours]
72.78
145.2
3343.40
4.026
Table Model performance on FashionMNIST-10. Memory refers the maximum GPU memory occupied by tensors. Runtime was calculated over 100 epochs on NVIDIA RTX 2070 GPU. For Dilip et al. (2022), performance depends on encoding quality, and thus a range is given, while two comparable baselines are included for Shen et al. (2024) (see Section 3.3 for details).
Test accuracy (%)
Memory (MB)
Runtime (sec)
Model
Residual
QML Circuit Dilip et al.
2022
63-75
QML Circuit Shen et 2024
77-80
IMPLICIT' solver [ours]
72.11
273.1
6424.67
6.892e-3
IMPLICIT-WARMUS solver [ours]
71.17
273.1
7240.35
1.934e-3
DIRECT solver 10 layers [ours]
71.83
1233.3
11651.96
1.41e-5
DIRECT solver 5 layers [ours]
70.87
628.8
6894.50
4.803e-3
DIRECT solver layers [ours]
70.81
266.1
4135.23
0.210
DIRECT solver- layers [ours]
71.05
145.2
3382,09
1.431
4.3 CIFAR-10
Finally, we applied our method CIFAR-10, a dataset of natural images; results can be found in Table 4
We find that in general, near-term quantum ML models that are amenable to numerical experiments not perform well on CIFAR-10, 10, as observed in recent prior work (Baek al.) [0023] Monbroussou et al. (024).. While their performance is higher than ours, we note that our models are not directly comparable. We chose not to boost model performance by adding classical NN components to focus on studying the quantum model. Still, we find that our QDEQ framework (IMPLICTTWWARMUP has higher accuracy on the test set than the direct solver baseline. This motivates the utility of method more realistic datasets.

---

Table Model performance on CIFAR-10.
Test accuracy
Model
(%)
IMPLICIT solver [ours]
24.38
IMPLICIT-WARMUP solver [ours]
25.45
DIRECT solver- 10 layers [ours]
23.71
DIRECT solver layers [ours]
24.19
DIRECT solver layers [ours]
24.90
DIRECT solver layers [ours]
24.70
5 Conclusion and open problems
In this work, we introduce a novel approach to training QML models under the framework of deep equilibrium models. We theoretically demonstrate that under certain conditions, QML models for classification tasks are amenable to DEQ-based optimization. This finding aligns with our numerical observations, suggesting that DEQs can significantly reduce circuit complexity while maintaining or improving performance compared to explicit training methods; this benefit is expected to hold in particular compared to using deeper parametrized circuits in quantum neural networks to achieve similar expressibility to the QDEQ model. This is particularly advantageous for near-term quantum algorithms, where minimizing circuit depth is crucial.
Our numerical experiments indicate that, for the case of quantum model families with theoretical intuition about their fixed-poin properties, the application of an implicit QDEQ model training superior compared to set of explicitly repeated layers in almost all cases, with the exception of amplitude-encoder data on our experiment with only four classes. Generally, our experiments indicate that the accuracy of our model is competitive with baseline models that typically require more classical and quantum resources.
One of the key benefits of the DEQ approach that it avoids the computationally expensive differen- tiation through the root finder. However, measurements are still required during the forward pass to guide the root finder's s Broyden- -based updates. The cost of these measurements likely comparable to the parameter-shift rule commonly used in gradient- needed quantify the measurement overhead and impact on the optimization process. Additionally, the influence of shot noise and other quantum noise sources on DEQ-based training remains to be explored, the expectation being that the impact of noise on DEQ training will behave similarly to general QNN training. Understanding the training of quantum models that are often amenable to vanishing gradients, called barren plateaus, has been greatly advanced by some recent theory works (Fontana et al.) (024]] Ragone al.) 2024). We anticipate QDEQ to behave similarly with respect to vanishing gradients as general VQAs. More in-depth analysis is an open problem. The potential advantage of DEQ here is the option of using shallower circuits for given performance, which are known to be less prone to barren plateaus.
optimization. Further investigation is
Another avenue of further research could be considering the outcomes of a preceding quantum experiment as quantum data. This could be a physical experiment or a preceding quantum algorithm. Either can be seen as advanced encoding map. Further investigations towards this approach, and into whether any advantages transfer, is left for further research. Finally, would be interesting to extend the applicability of DEQs in combination with our deliberations in Observation 2 to broader range of QML models.
Acknowledgements
We thank Zachary Cetinic for his insights and rooting for the ultimate side-quest. AAG thanks Anders G. Frøseth for his generous support and acknowledges the generous support of Natural Resources Canada and the Canada 150 Research Chairs program. Parts of the resources used in preparing this research were provided by the Province of Ontario, the Government of Canada through CIFAR, and companies sponsoring the Vector Institute) LBK acknowledges support from the Carlsherg Foundation, grant CF-21-0669. RAVH acknowledges NSERC Discovery Grant No. RGPIN-2024-06559
10

---

References
Scott Aaronson. 2015. Read the fine print. Nature Physics 11, 4 (2015), 291-293.
https
Amira Abbas, David Sutter, Christa Zoufal, Aurelien Lucchi, Alessio Figalli, and Stefan Woerner.
2021. The power of quantum neural networks. Nature Computational Science 1, (2021), 403-409.
https://doi... org/ 10.
Shahnawaz Ahmed, Nathan Killoran, and Juan Felipe Carrasquilla Álvarez. 2022. Implicit dif- ferentiation of variational quantum algorithms. arXiv:2211. [quant-ph] (2022). https:
7/arxiv. otrriiv.orggrioo abs 13765
Mahabubul Alam, Satwik Kundu, Rasit Onur Topaloglu, and Swaroop Ghosh. 2021. Quantum- classical hybrid machine learning for image classification (icead special session paper). In 2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD). IEEE, 1-7. https
//ieeexplore. ieee. orggabbstracttdoccmeeetto
Gian-Luca R. Anselmetti, David Wierichs, Christian Gogolin, and Robert M. Parrish. 2021. Local, expressive, caettuu--uumbb---eeee Physics 23, 11 (2021), 133010. https://doi org/ 1088/ 00088/1367, 0t////1177726660 2630/ac2cb3
VQE ansätze for fermionic systems. New Journal of
Hankyul Baek, Soohyun Park, and Joongheon Kim. 2023. Logarithmic dimension reduction for quan- tum neural networks. Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. 3738-3742.
https://dl. acm. 4acm..og//oo3/110111//333333
3615240
Shaojie Bai, J Zico Kolter, and Vladlen Koltun. 2019. Deep Equilibrium Models. Advances
in Neural Information Processing Systems 32 (2019). https:// neurips cc/
.ttl
Shaojie Bai, Vladlen Koltun, and J Zico Kolter. 2020.
Multiscale Deep Equi-
librium Models. Advances in Neural Information Processing Systems 33 (2020),
5238-5250. https://proceeddinga hew.eeiiinnnnnnennnneeeeeeeeriee neurips. wwtrripssccpppppeee wttp:eccpppplpeeeeeeeeeeeeeeeeee haappeerfille//pppe///////////
Shaojie Bai, Vladlen Koltun, and J. Zico Kolter. 2021. Stabilizing Equilibrium Models by Jacobian
Regularization. arXiv:2106.1434. 14342 [cs, stat] (2021). http://arxiv. www.rorg/ggoo prg/abs/2106 2106. 14342 arXiv: 2106. 14342.
Kerstin Beer, Dmytro Bondarenko, Terry Farrelly, Tobias J Osborn,, Robert Salzmann, Daniel Scheier- mann, Ramona Wolf. 2020. Training deep quantum neural networks. Nature Communications 11, 1 (2020), 808. httts::/ hhpps://www... nature... com/ setticcesss4444/7
14454-2
Kishor Bharti, Alba Cervera-Lierta, Thi Ha Kyaw, Tobias Haug, Sumner Alperin-Leea Abhinav Anand, Matthias Degroote, Hermanni Heimonen, Jakob Kottmann, Tim Menke, Wai-Keong Mok, Sukin Sim, Leong-Chuan Kwek, and Alán Aspuru-Guzik. 2022. Noisy intermediate-scale quantum algorithms. Reviews of Modern Physics 94, (2022), 015004. https://doi.org:: 1103/RevModPhys .94. .e...140.0000000 015004
Jacob Biamonte, Peter Wittek, Nicola Pancotti, Patrick Rebentrost, Nathan Wiebe, and Seth Lloyd. 2017. Quantum machine learning. Nature 549, 7671 (2017), 195-202. https://doi.org: org/10 1038/nature23474
Charles G. Broyden. 1965. A class of methods for solving nonlinear simultaneous equations. Math. 100.00000000000--0000 http:///...000000000--5///111-10
Comp. 19, 92 (1965), 577-593.
Matthias C Caro, Hsin-Yuan Huang, Marco Cerezo, Kunal Sharma, Andrew Sorborrger, Lukasz Cincio, and Patrick J Coles. 2022. Generalization in quantum machine learning from few training data. Nature communications 13, (2022), 4919. https: :etps://wwat //www. nature. com/ www.artiaaees////////ooo..o.ottt 84 041467-022-2 3225--00-3333

---

M. Cerezo, Andrew Arrasmith, Ryan Babbush, Simon C. Benjamin, Suguru Endo. Keisuke Fujii. Jarrod McClean, Kosuke Mitarai, Xiao Yuan, Lukasz Cincio, and Patrick J. Coles. 2021. Variational quantum algorithms. Nature Reviews Physics 3, 9 (2021), 625 644. https:
0rg/10.10390 -22-2244--20-0-------
Haoyu Chu, Shikui Wei, and Ting Liu. 2023.
Lyapunov-Stable Deep Equilibrium Models. arXiv:2304. 12707 [es] (2023). https: btttps://rxxine org. http//abs23344.27000
Li Deng. 2012. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine 29, 6 (2012), 141-142. http://yann. http///aannnnnnn....mmmmmm www.lneeunn. cww/exdb/meecccccctcococwwwwoeba htrdb//mnst///
Rohit Dilip, Yu-Jie Liu, Adam Smith, and Frank Pollmann. 2022. Data compression for quantum machine learning. Physical Review Research 4, 4 (2022), 043007. hntpps//jjournall. aps
httg///researcc/aaattrttttt////e 1tt.:11..11111111 ht03//PhysReeReeearcchhh 4. 043007
Enrico Fontana, Dylan Herman, Shouvanik Chakrabarti, Niraj Kumar, Romina Yalovetzky, Jamie Heredge, Shree Hari Sureshbabu, and Marco Pistoia. 2024. Characterizing barren plateaus in quantum ansätze with the adjoint representation. Nature Communications 15, 1 (2024), 7171.
https://doi. org/10. 10. 1038/ 041467-024 024-49910-
Zhengyang Geng and Zico Kolter. 2023. Torchdeq: A library for deep equilibrium models. arXiv:2310. 18605 [cs] (2023). atttps//arxiv.orr httpg//bb///31/// 18862510.18805
Maxwell Henderson, Samriddhi Shakya, Shashindra Pradhan, and Tristan Cook. 2020. Quan- volutional neural networks: powering image recognition with quantum circuits. Quantum Machine Intelligence 2, (2020), 2. https:///ink.ssrrigge....
springer
942484-020-00012-y
Xiaokai Hou, Guanyu Zhou, Qingyu Li, Shan Jin, and Xiaoting Wang. 2023. A duplication- free quantum neural network for universal approximation. Science China Physics, Mechanics & Astronomy 66, 7 (2023), 270362. https://link https:///ink.sprinngrr com h/t//articlllll 1007/
111433-023-228 2098-8
Hsin- Yuan Huang, Richard Kueng, and John Preskill. 2020. Predicting many properties of a quantum system from very few measurements. Nature Physics 16, 10 (2020), 1050-1057. https://www
nature.
Thomas Hubregtsen, Frederik Wilde, Shozab Qasim, and Jens Eisert. 2022. Single-component gradient rules for variational quantum algorithms. Quantum Science and Technology 7, (2022), 035008. https://doi. org 10. 1088 0000008/08800888 9565/ 20655//c624
Yu Jing, Xiaogang Li, Yang Yang, Chonghang Wu, Wenbing Fu, Wei Hu, Yuanyuan Li, and Hua Xu. 2022. RGB image classification with quantum convolutional ansatz. Quantum Information Processing 21, (2022), 101. https://link... www.iiwe.....inn....mmm. com www.///ttt/eee.....oooo. 1007/
s11128-022 01178-022--3422-8
Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. arXiv:1412.6980 [cs] (2014). https:///rxxvv wwwariii.rrr..oeeeeo abs 6980
Jakob S. Kottmann, Abhinav Anand, and Alán Aspura-Guzik. 2021.
feasible approach for
automatically differentiable unitary coupled-cluster quantum computers. Chemical Science 12, 10 (2021), 3497-3508.
Steven G. Krantz and Harold R. Parks. 2013. Basic Ideas. In The Implicit Function Theorem: History, Theory, and Applications, Steven G. Krantz and Harold R. Parks (Eds.). New York, NY, 35-59.
Alex Krizhevsky, Geoffrey Hinton, al. 2009. Leaming multiple layers of features from tiny images. (2009). fttps:://www.cs.toooo http:t/t.......ta////aaaainnnnno featurrrrnnneeeueeseeeeeeeee -ow.uurree------e--e TR.
Oleksandr Kyriienko and Vincent E. Elfving. 2021. Generalized quantum circuit differentiation nules. Physical Review 104, 5 (2021), 052417.
11100///////..a.e 104
052417
12

---

Phuc Q. Le, Fangyan Dong, and Kaoru Hirota. 2011. A flexible representation of quantum images for polynomial preparation, image compression, and processsig p operatioon. Quantum Information Processing 10 (2011), 63 84. httpp///ooo..ooo/....00071111220
Mingjie Li, Yisen Wang, and Zhouchen Lin. 2022. Cerdeq: Certifiable deep equilibrium model. In International Conference on Machine Learning. PMLR, 12998 13013. httpss://proceedings mlr. 0press/v62//12255 html
Junhua Liu, Kwan Hui Lim, Kristin L Wood, Wei Huang, Chu Guo, and He-Liang Huang. 2021. Hybrid quantum- -laassccal convolutional neural networks. Science China Physics, Mechanics springer & Astronomy 64, (2021), 290311. https://link. htaps://Aink.sppingger com/ aataaaeeet..11000 1007/
s11433 020-002111-00 1734-3
Seth Lloyd, Maria Schuld, Aroosa Ijaz, Josh Izaac, and Nathan Killoran. 2020. Quantum embeddings for machine learning. arXiv:2001.03622 [quant-ph] (2020). http://arxx org has///b//2000 03622 arXiv: 2001.03622.
Jarrod McClean, Sergio Boixo, Vadim N. Smelyanskiy, Ryan Babbush, and Hartmut Neven. 2018. Barren plateaus in quantum neural network training landscapes. Nature Communications 9, (2018), 4812. hetps://doi.org// 10w.:1133111555 0401//11-11100000000-----
Léo Monbroussou, Jonas Landman, Letao Wang, Alex B. Grilo, and Elham Kashefi. 2024, Subspace Preserving Quantum Convolutional Neural Network Architectures. https://arxiv www.aawwww....raaaaaaaaa /bs/
18918
Alberto Peruzzo, Jarrod McClean, Peter Shadbolt, Man-Hong Yung, Xiao-Qi Zhou, Peter J. Love, Alán Aspuru-Guzik, and Jeremy L. O'brien. 2014. A variational eigenvalue solver photonic quantum processor. Nature Communications 5, 1 (2014), 4213. https://wie hwtte///////.naueeeeemeeaaaaaee com/
nature.
artiicles/nccomss5211
Lirandë Pira and Chris Ferrie. 2024. On the interpretability of quantum neural networks. Quantum Machine Intelligence 6, 2 (2024), 52. https://link.sprian wwwccooo/tta//e http//rrcccccc1100 1007
-42484-024-001917
Ashwini Pokle, Zhengyang Geng, and Zico Kolter. 2022. Deep equilibrium approaches to diffusion models. Advances in Neural Information Processing Systems 35 (2022),
37975-37990. https::/proceeddnni htwp://cwwdddinnn.nne.e...o....o cc/ ctsprrpppsccppppe_ffffe1. pttpp/pppp////aaaaaa/
.tml
Michael Ragone, Bojko Bakalov, Frédéric Sauvage, Alexander Kemper, Carlos Ortiz Mar- rero, Martín Larocca, and M Cerezo. 2024. A Lie algebraic theory of barren plateaus for deep parameterized quantum circuits. Nature Communications 15, (2024), 7172. https
//doi.org/10
02414467-04-4933
Maria Schuld. 2021.
Supervised quantum machine learning models are kernel methods. 110200
arXiv:2101. 1020 [quant-ph, stat] (2021).
Maria Schuld, Alex Bocharov, Krysta M Svore, and Nathan Wiebe. 2020. Circuit-centric quantum classifiers. Physical Review 101, (2020). 032308. https://journals.. wwt.enalls.pp....m...mm org /ra/ abstract/10 110.//yyeee//.............. 201.032308
Maria Schuld and Nathan Killoran. 2019, Quantum machine learning in feature Hilbert spaces.
Physical review letters 122, 4 (2019), 040504. https://doo.org//1 1108/hhysseweett
122. 040504
Kevin Shen, Bernhard Jobst, Elvira Shishenina, and Frank Pollmann. 2024. Classification of the Fashion-MNIST Dataset on a Quantum Computer. arXiv:2403.02405 [quant-ph, es] (2024).
https://arxiv.org// www.:////////.....o.//// 02405
Hanrui Wang, Yongshan Ding, Jiaqi Gu, Yujun Lin, David Z Pan, Frederic T Chong, and Song Han. 2022a. QuantumNAS: Noise-adaptive search for robust quantum circuits. In 2022 IEFE International Symposium on High- Piih--Performance Computer Architecture (HPCA). IEFE, 692-708.
https::/doi. org/10. 10. www.://A5....2222000022822 00057
13

---

Hanrui Wang, Zirui Li, Jiaqi Gu, Yongshan Ding, David Z Pan, and Song Han. 2022b. QOC: quantum on-chip training with parameter shift and gradient pruning. Proceedings of the 59th ACM/IEEE Design Automation Conference. 655-660.. https: //dl..//dl.. acm. abs/
0t555344955173554499
Maxwell West, Azar C. Nakhl, Jamie Heredge, Floyd M. Creevey. Lloyd C. L. Hollenberg. Martin Sevior, and Muhammad Usman. 2024. Drastic Circuit Depth Reductions with Preserved Adversarial Robustness by Approximate Encoding for Quantum Machine Learning. Intelligent
Computing 3 (2024). https://doi... org/ 3413341133333 htw.1/////ompptinng...... 0100
David Wierichs, Josh Izaac, Cody Wang, and Cedric Yen-Yu Yu Lin. 2022. General parameter- shift rules for quantum gradients. Quantum 6 (2022), 677. https://doi.orr: cecooi.orrg/ii 10. 22331/
g-2022-03-30-677
Ezra Winston and Zico Kolter. 2020. Monotone operator equilibrium networks. Advances in neural information processing systems 33 (2020), 10718-10720 https:://proceeddngg neurips
Abstractt html
Han Xiao, Kashif Rasul, and Roland Vollgraf. 2017. MANSI: Novel Image Dataset for
Benchmarking Machine Learning Algorithms, 1eXiv:1708.0774) [cs] (2017). https://arxiv
org/abs/1708.077
14

---

Appendix
A Existence of fixed points for quantum model families
Observation 2 (Contractivenesss of Quantum Model Families).
Below, we provide discussion regarding the existenee of fixed points for quantum model functions according to Definition II with the additional assumption that measurement ensembles only consist of Pauli operators and projectors computational basis states.
Consider two vectors z, z', which, without loss of generality, we may assume be normalized over [0; 2m]". Further, consider Hermitian observable M used for readout. The value of this readout for quantum models as Definition then takes the form
Tr (MUS₂ [0X0] stut).
(6)
Now, if we want to evaluate contractiveness of the quantum model and consider
(z) f( (z))l
for a family of quantum models, we can apply the triangle inequality to arrive at conditions for each entry corresponding to single observable of the ensemble. Thus, argue whether the model is contractive, we need to bound quantities of the form
Ir (MUS, [0X0] Siut) Tr (MUS² 10X0| ut
(8)
We use notation before and say S₂ [0) |2). Note that due to the cyclic properties of the trace, construct equivalent observable M MU with the same spectrum M, and write this as
Tr M (lz - |z 1x()
We furthermore can express M in its eigenbasis, M = Ex Ma [AXA]. Then evaluating the trace in this basis allows us to simplify the expression as
A = Σ -
(10)
Now we assume that all eigenvalues M are bounded, i.c., that it has a finite spectral norm || < co. This spectral bound and another application of the triangle inequality yields
A
|
Σ
(11)
where the right hand side is the total variation distance with respect to some POVM. As such, it is bounded by the trace norm. Then, we can conclude that
A < M| ||x]|| |x'Xx' Il,,, = 2 2 (2
(12)
As detailed in Appendix C this bound can be further sharpened to not require the factor of two in the case of basis-state measurements, i.c.,
MM ((z)2
(13)
For further discussion of the general case, please see the end of this appendix. Note that in the case of a single observable, this expressions is clearly bounded by |M Assuming that this is smaller than or equal one, which is clearly necessary assumption for contraction map be possible, we therefore have that the distances after application of the model are always less than one, meaning for any pair of vectors such that ||z z'll 1, a map of this always subcontractive. Thus, for the case of a single observable M, we only need to consider the case where ||z z'll < 1.
Assume that the following holds for some 0:
sin ||z z²1²))
Zz) fz|z))
2
(14)
2
15

---

Evidence for this bound with c= 1 in the case of amplitude encoding can be found in Appendix B along with evidence for slightly weaker c 2 bound for the specific angle encoding strategy used in this work. From the inequality, we have the following:
1 1- 1=|zz2')) 1
(1
c
sin
||z 2²1²))
(15)
2
- e² sin (Iz x²1²)²
[|z z'1 <1(²
= csin
(16)
4
< csin
[|z z 11))
1 cl|z z²11²
(17)
with the final result that
If(z) - f(z)) S McllMlll z''l
(18)
for a single output variable so that |M]] < 1. Note that the constant appears as an overall scale in the expression. For simplicity, will focus on the amplitude encoding case of e = 1 below-See Appendix B.2 for discussion of how the angle-encoding case differs.
Having bounded single observable, we next discuss how to deal with multiple observables in a family of model functions. If we have an ensemble for K observables, {Ma then by the properties of the & norm,
((M)))))((()))) & VV 11 /2 z'Iles
(19)
So if all ||M&ll || are bounded by 1, we will have a prefactor of VK for K observables, which suggests that for more than one observable, the model family not (sut)cootrecttiv in the &g norm. However, the case would be different if we choose the maximum-nome as a metric. Then, we do not encounter the usual property of the lg norm that it grows with the square-root of the dimension, and we instead have
[L(M)))) (:000(((1)) ((00
(2')
S max
M l||z z z'Ilmax
(20)
max
This means, that our model (family) f from Definition iss subcontractive, which is not sufficient yet for fixed point to exist. One way to proceed would be to show that
II((((z) - f(z)] < (l](z) =||
(21)
except for f(z) = z. i.c. unless z is a fixed point.
We restrict ourselves to observables that are either Pauli-operations, which are unitary and have eigenvalues +1, or projectors on basis states, whose +1 eigenspace is spanned by a single basis state only, as they are rank-one. Note than the bound we used above Eq. (TT) can be loose, as assumes a worst-case where the difference is aligned in eigenspaces so that (T(MM(())) Iz') (z' (/z))) = M||M|||] [|]] |z') 22'2''lll is attained. Such a
Instead, we make make different deliberation.
scenario highly unlikely. In fact, consider the relationship in Eq. 21 and denote [f(z)) = S(() (0).
Then, we look at
[F(((()) - f(z)l = Ir -
I
(22)
By the arguments of Appendix C a quantum model is then not contractive the difference [f(z))f(z)] |z)[z| is fully aligned with either the +1 and -1 eigenspaces for Pauli operators or the +1 eigenspace of a basis state projection, in the sense outlined in that appendix. Surely, as z comes from batch of data with different values for each element, such a construction would either not correspond to a sensible set of observables or the encoding would not capture any diversity in the data. Thus, we can expect that the quantum models on data will high probability. This also likely explains why the architectures not fully covered by the analysis above (i.e., angle encoding and Pauli measurements) work in practice.
contractions in practice with
For the case of amplitude encoding, a re- normalization of the outputs would be required before re-encoding them. Viewing this necessary rescaling of ((M (z) as part of the map may result in a change in norms, since in general ||x y + || x the main text, we conjecture that the arguments for strict contractivity put forth above most cases more than compensates for any such shifts; we leave further investigation as topic for future work.
X | Given the positive empirical results of
16

---

These discussions identify two key aspects to achieve contractive quantum architectures of the form considered in this work: The encoding should encode similar vectors into similar quantum states; and, the measurement operators should not amplify these differences when converting back into classical data. The interplay of these two aspects is captured well by Eq. (.) which highlights the simple fact that the difference in output depends on the difference in the encoded states and the ability of the measurement to resolve this difference. Our approach here used the bound in Eq. (14) as a way to quantify that the encoding preserves closeness, and bounded the resoling/amplliicaatee the readout using simple operator norms, leaving more detailed analysis of the interplay between encoding and readout as future work.
power of
B Evidence for overlap bounds
In Appendix the following property for the encoding of vectors fulfilling ||z - =z'1 S was used:
(z|z")
sin [|z z' 2²1²).
(23)
In this section, we provide evidence for this bound for the two encoding maps used in the main text, ie., amplitude encoding and angle encoding.
B.1 Amplitude encoding
This is the simplest case. Assuming that the inputs are normalized to length one, and using that we only work with real vectors, the following holds by definition of amplitude encoding:
(2/2²) Iz z'
(24)
Furthermore,
||z z/112 =112 =||z|| ||2211"
2z z
(25)
2 2z z'.
(26)
meaning
[z||z)| = 1 - 1 z 112
(27)
Using the fact that the norm in this expression is bounded by one, therefore get
(z|zz>| 1z z'112
(28)
2 2 sin (11 (11--
x'1)). (111==11²)
(29)
as desired. Note that for this particular encoding, the equality in Eq. (28) allows for the introduction of factor in Eq. (17), corresponding to the the constant 1 in the bound in Eq. (23).
B.2 Angle encoding
In the case the angle encoding, consider the unitaries used in the encoding:
Kz|z')| = (0) S [0)
(30)
For the specific encoding used, these unitaries have the property that they consist of single-qubit encodings operating on separate qubits. other words, denoting these single-qubit maps by s and splitting the vectors into their constituent 4-tuples of entries,
z (z) 44+1 (z) 4k (z) (z)
(31)
we can write the encoding map as the following tensor product
S₂
(32)
26
km
17

---

g0 (0)
0.6
5%.
[0)
0.4
Numerical data
sin( [2.
x
1 2:
0.
0.0
1.0
||z.
z/1l
Figure 4: Plots of the relation between the single-qubit overlaps (0)
s 2 [0) and the norm
|zz 21 for 3000 pairs of random vectors in R... Note that all of the points lie above the line corresponding to sin [|2s 2/11²)
This implies that the overlap takes the form:
((//z') II
(0) S
S s()) 2 10)
(33)
Each factor in this expression in principle simply a complicated trigonometric expression in eight variables. While it is likely possible bound this expression analytically, simple numerical investigation was assumed sufficient evidence for our investigation here; see Fig. 4 Drawing 3000 pairs of vectors 2k, 2 IR random that ||z inequality holds,
z'z ll s we see that all cases the following
(0) SQI (1) x 03
03/21 2 sin (12z= --x1)
(34)
To finish the bound, apply the following trigonometric identity.
sin(a b) sin(a) cos(b) + sin(b) cos(a)
(35)
and use this to derive:
(1 sin(a))(1 - sin(b)) 1 sin(a) sin(b) sin sin
(36)
sin sin(b) sain(a) sin sin(a) cos(b) + sin(b) cos(a) sin(a + b)
(37)
(38)
- sin b)
(39)
(co(s) + sin(t) - 1)
(40)
(co(t)(coo(a) + sin(a) 1)-
(41)
For a, 1, the two final terms are either positive or zero. Thus, under this assumption on the arguments, we obtain
(1 (1-sin - sin(b)) 2 1 sin(a + b).
(42)
18

---

Combining Eq.
33) with the bounds of Eq. (4) and Eq.
now yields
Kz/z²)
[0)
s
5(1) (0)
(43)
2
1 sin (122 - z//1()
(44)
k=1
2 1 sin Σ 1zz z'a
1??
(45)
1 sin (1z z 11*)
(46)
as desired.
Note that, in contrast to the amplitude-encoding case, we do not have a pre-factor of 1/2 on the As can be seen
second term. In other words, our bound corresponds to the one in Eq. (23) with by the derivations in Appendix A|] this introduces an overall factor of 2 that would in principle need to be compensated for, c.g., in the magnitude of the readout operators. However, our experiments indicate no problems finding fixed points also without such adaptations. We ascribe this success to the likely looseness of the bounds derived in Appendix [ since this means contractiveness can be present even when the derived bounds are not sufficient to guarantee it. For a further discussion of some the sources of this looseness, see Appendix A
C Further analysis of bound tightness
In this section, we provide more detailed analysis of the steps leading from Eq. ( to Eq. in Appendix AA, including the derivation tighter bound for the case of observables that are projectors. Consider the object to be bounded,
A - Tr M ( 11'X=1111'X=11))
(47)
Looking more closely at the object [z)(z - |z Xz'). we can note that this a Hermitian, traceless rank- 2 operator. This implies that it can be diagonalized unitarily, with at most two nonzero eigenvalues that necessarily sum to zero due to the tracelessness. In other words, it can be written on the form
12X
1 Xz'l (KooXEol - (E,))
(48)
for some À 2 0. In fact, an explicit calculation shows that this constant can be characterized as |z')z'| Il Thus, Eq. (47) can be rewritten
x |z
A = Tr M (I5oXEol ((XEi1)) |||x|z| - 2'Xz'lll
(49)
Tr AM [SoX
- Tr ji (iS11) |||x||| - 1z' 12'X2'1l
(50)
Comparing this to the bound in Eq.
12
A
M| |||x|||
- |x 12/IITT=
(51)
it becomes clear that the tightness of the bound depends on how effectively the observable M distinguishes between the two eigenstates of zzx²| |z')''|| This, in turn, depends on the alignment of these eigenstates with the eigenspaces of the measurement operator M. Specifically, in the case where the spectrum of M takes the form {-II [|]] [|M]], the bound is saturated when one of the eigenstates is contained in the +iiM//l-eeeesspacc of M and the other one is contained in the
-IIM |l-eiggesppac of M. On the other hand, for operators with spectra of the form (O...., || (e.g., basis-state projectors), the expression in Eq. (49) is maximized when one eigenstate is fully contained in the O-eigenspace and one is fully contained in the +I|M lligeessppace. Note that, in this case, the following tighter bound holds:
A <
|M] |||x|||
|z 12
(52)
used in Appendix A
19

---

D Universality of weight-tied quantum models
Theorem 3 (Universality of weight-tied quantum models, in analogy to Theorem in Bai al. (2019)).
Let E((-) =
be channel corresponding to the PQC depth Additionally,
define an map R² _ x29 + RR that describes performing measurements of expectation values ensemble of K observables and storing the respective outcomes in K K-dimmnnsonaal vector. Typically, we string this together with an upsampling map Iu so that = I, R' maps to R.. Let Sz be unitary encoding that encodes a vector from R" into a quantum state in C S² 10) + zz.. This allows us define an encoding map S that maps z to a density matrix, so that 2 (2)2), where the evolution of hidden layers can then be written as
with respect
z
R(E, Sz ). OsicL, z =
(53)
Then, sequence of such layers can be replicated exactly by input-injected, weight-tied network. Specifically,
((++1) (2(E, + Exx)
(54)
where
0
E. S
R
& o S
0
0
Ry
E,
0
Eg
0
S
Ex
(55)
R
0
0
0
ET.-1 S
0.
yields after L iterations an output
(11)
2(4)
(56)
((()
containing the output (1) of the non weight ried network. This follows by construction, similarly to Theorem 3 in Bai al. (2019).
20

---

E Experiments - Hyperparameter search
We used the validation set to search over hyperparameters listed in Table 5 For each hyperparameter, note the ranges considered, well the optimal value for each dataset.
Table 5: Optimal hyperparameters for each model. The search space we considered was: learn- ing rate (0.005, 0.0075, 0.01, 0.05, 0.1). number of warm-up steps (1875, 2355, 3750), number of warm-up layers {1, 2}, weight the Jacobian loss {0, 0.5, 0.8), frequency of the Jacobian loss {0.0, 0.5, 0.8, 1.0].
IMPLICIT-WARMUP
Hyperparamter
MNIST-4
MNIST-10
FashionMNIST-110
CIFAR-10
Learning rate
0.05
0.05
0.05
0.01
Num. warm-up steps
1875
1875
1875
2350
Warm-up layer depth
1
Jac. loss weight
0.0
0.8
0.8
0.8
Jac. loss freq.
0.0
1.0
0.8
IMPLICIT
Hyperparamter
MNIST-4
MNIST-10
FashionMNIST-100
CIFAR-10
Learning rate
0.05
0.05
0.05
0.05
Jac. loss weight
0.0
0.8
0.5
0.0
Jac. loss freq.
0.0
1.0
0.8
0.0
DIRECT 10 layers
Hyperparamter
MNIST-4
MNIST-10 FashionMNIST-100 CIFAR-10
Learning rate
0.1
0.05
0.05
0.0075
DIRECT 5 layers
Hyperparamter
MNIST-4
MNIST-10 FashionMNIST-10 CIFAR-10
Learning rate
0.05
0.05
0.0075
0.0075
DIRECT 2 layers
Hyperparamter
MNIST44
MNIST-10 PashionMNISTT110 CIFAR-10
Learning rate
0.05
0.05
0.05
0.01
DIRECT layer
Hyperparamter
MNIST4
MNIST-10 PashionMNIST-100
CIFAR-10
Learning rate
0.05
0.05
0.01
0.01
21

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The claims of the abstract reflect the findings of the paper the best of our judgement.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Information regarding that can be found in the Results and Conclusion and Open Problems sections.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model cell-specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
22

---

Answer: [Yes]
Justification: Our main theoretical results are in Theorem 3 and Appendices A And To the best of our judgement, our Theorem provides full set of assumptions in combination with Definition Furthermore, we openly discuss the limitations our arguments in the appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof' provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
contains information regarding all our models and baseline models we are referring to. With a full publication, we will publicly share explicit code as well access to weights and biases our experiments ensure full reproducibility.
Justification: Our Section 3
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibilitty In the case of closed-source models, it may be that access to the model is limited in
23

---

reproducible, they some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
All datasets used in the paper are publicly available. The code is publicly available on GitHub the following link: lnttps://github. httpa//....oomaaaaaarr//////ggo
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://niis.cc.
public/guides/CCne wtw.:iddeeeooeeSmiiiiisiiiisssis wwwpssiinoooyyyyl for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
7 //nips.o //nipscccppul sic/pubbic//guddes httpc//guuuds//ooeeuuubbiiiii iendeesubmsssooPolliie
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
The main hyperparameters and methods relevant to reproducing the proposed method reported. Additional hyperparameters will be supplied with publication of the code.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to a level detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Ys]]
Justification: The paper reports individual runs, and has been phrased so as to not make claims about statistical significance guarantees of broader applicability.
Guidelines:
The answer NA means that the paper does not include experiments.

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
For all experiments, the main computational hardware and the runtime is reported. When choosing the level of detail of this reporting, the independence of the paper's claims to the scaling of these resources was considered.
Justification:
Guidelines:
The answer means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported didn't i make it into the paper).
paper (e.g., preliminary or failed experiments that NeurlPS Code of Ethics pectts://neurr http://pp.....ppbb...ccccccceeed
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the
Answer: [Yes]
Justification:
The authors have read the Code of Ethics, and report that the work here is in compliance.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
25

---

Justification:
The paper proposes change in methodology in the already well-established field of image classification. Furthermore, the limited scale of current quantum computers reduces the chance of immediate impact, also within this field.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g.. pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: The work in the paper does not pose a risk of misuse, and no new datasets are released part of the paper.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
All datasets and code used cited, and the relevant licenses named.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
26

---

The authors should state which version of the asset is used and, if possible, include URL.
The name the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperssi thcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper uses existing datasets, and does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: No research involving crowdsourcing human subjects is included in the paper.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No research involvingerowdssorriing or human subjects included in the paper.
27

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
28


Reviews:
[{"review_id": "C0U7gJZyCu", "reviewer": "Reviewer_BzP4", "summary": "This paper introduces a new paradigm for training quantum machine learning models using Deep Equilibrium Models (DEQ). The authors propose Quantum Deep Equilibrium Models (QDEQ) to enhance the performance of parametrized quantum circuits (PQC) while addressing issues related to circuit depth and parameter scalability. QDEQ uses a root solver to find the fixed points of the network, allowing for the training of shallower circuits with fewer parameters. The paper demonstrates the effectiveness of QDEQ on classification tasks using MNIST-4, MNIST-10, and FashionMNIST-10 datasets, showing competitive performance compared to existing baseline models while requiring fewer resources.", "questions": "How does quantum noise, such as shot noise and other errors, influence the DEQ-based training process on noisy quantum hardware?\n\nIn practical implementations, how often do the assumptions about the existence of fixed points hold, and what are the implications if they do not?\n\nCan the authors comment on how this work is related to the know barrel plateau issue in quantum machine learning?", "limitations": "The limitation and broader impact (which may not exist) are addressed.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper introduces the Deep Equilibrium Models to quantum machine learning, a novel approach that has not been explored previously.\n\nUsing DEQ, the paper shows that it is possible to achieve high performance with significantly shallower quantum circuits, which is crucial for near-term quantum devices where noise is a significant factor.\n\nThe paper provides a detailed and solid theoretical framework.\n\nThe presentation of the paper is very easy to understand, with a good mix of equations/theorems and explanations/illustrations. \n\nThe paper demonstrates experiments on MNIST-4, MNIST-10, and FashionMNIST-10 datasets, showing the practical applicability and competitive performance of QDEQ models, and adding empirical evidence to the claim.\n\nThe proposed approach is resource efficient and reduces the need for extensive quantum and classical resources, making it more feasible for implementation on current quantum hardware.", "weaknesses": "The effectiveness of QDEQ relies on the assumption that the quantum model functions admit fixed points. It is unclear how well this assumption can hold in reality\n\nThe empirical validation is limited to a few datasets. The reviewer understands the complexity of performing experiments for quantum machine learning, so this point can be ignored.\n\nThere is only one or two baselines in each benchmark. Additional baselines could help to further validate the result.\n\nAlthough the authors did a lot of work to make the work reproducible, the code is not publicly available yet. The authors promised to post the code after the review, but in the meantime, it would be helpful to have an anonymous repository to host the code.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s insightful comments and are thankful for the positive feedback. In response to the reviewer’s questions, we have provided the following answers:\n\n**Influence of noise**\n\nWe agree that this is a very interesting and relevant question that we opted to defer to further research -- the emphasis being, noise is a very wide and subtle topic and we would like to spend more time on this question in a thorough manner in a follow-up work. Furthermore, accurately modelling it requires code modifications beyond what was feasible in a week of rebuttal. \n\nThe main noise-sensitive component of QDEQ is that gradients need to be retrieved through sampling, which makes the training potentially sensitive to both shot noise and other noise sources. In this sense, we do not expect QDEQ to be significantly different to standard variational algorithms, with similar error mitigation strategies likely being applicable here. \n\nNote that some noise sources, e.g., amplitude damping noise, might aid the existence of fixed points. However, in the presence of strong noise, these fixed points tend to be trivial (maximally mixed state) and thus likely less useful. As such, a closer investigation is required on whether the impact of intermediate noise levels would be beneficial.\n\n**Fixed points in practice**\n\nWe acknowledge that the question of fixed points is important and not fully closed by our work. We consider the presence of fixed points in the architectures we investigated as promising empirical evidence. Contractions are guaranteed to have fixed points, thus Appendix A of our manuscript aims to provide guidelines regarding which properties are important for ensuring this property. The two most important factors are the encoding and the readout. Encoding should not amplify differences between two data points, i.e., neighbouring data points should encode to similar quantum states. The amplitude encoding we use satisfies this if the classical vector to be encoded is not very small. This is supported by input injection in our approach, so this should be satisfied for nontrivial datasets. Other widely-used encodings, such as angle encoding, have the same property. With respect to readout, similar non-amplification is needed. This implies that the norm of the measured operator should not be too large and that there is an upper bound on how well it distinguishes encoded states, in the sense of how states are aligned with the measurement operator's eigenbasis , cf. Eq. (9) or (12). The norm is easy to calculate and relatively small for single-qubit Paulis and basis-state projections, as common in quantum classifiers. The distinguishing-power is more difficult to bound analytically. Based on this and the empirics, we expect fixed points to be common for quantum classifier architectures. Further work on a formal proof, e.g. using arXiv:2006.08591, is left as future work. \n\nIf the map completely lacks fixed points, we expect training of the model to fail; the essential implicit differentiation formalism breaks down. We briefly investigated this by trying to fit our model to a Fourier Series --  there is no fixed point in this setting and thus QDEQ performed well below the baseline model. \n\n**Benchmarks**\n\nWe agree that baselines are vital to establish a context for the presented results. We expanded our search for admissible baselines and have added a new baseline by arXiv:2309.09424 to the MNIST-10 and FashionMNIST-10 benchmarks. This reference provides evidence of performance of $\\\\sim$80-85\\\\% on MNIST-10 and $\\\\sim$75\\\\% on FashionMNIST-10 using a slightly modified amplitude encoding and a similar number of qubits. Additionally, the baseline we already included in the submission for FashionMNIST-10 contains more experiments in its appendix. These experiments show performance of $77-82\\\\%$, and use non-approximate amplitude encoding, similar to our strategy. It should be noted that in both cases, the high performance is likely attributable to significantly deeper circuits with more trainable parameters (about 10x and 3x the parameters of our architecture, respectively). We see no limitations to scaling QDEQ to similar sizes, and leave this as future work.\n\nBeyond these references, existing work is dominated by binary classification, convolutional architectures, or significant initial compression of the images using deep classical NNs. We are concerned that adding baselines with very different architectures or tasks would not support the main focus of the paper -- comparing QDEQ to other training of similar architectures -- and that extended discussions of architectural differences risks reducing the clarity of the paper. As we see it, comparisons between the direct solver and the implicit solver should be the focal point of the discussion, as this most cleanly reflects the effects of QDEQ.  We hope that the reviewer will find these additions an reasonable trade-off between context and focus.\n\n**Barren plateaus**\n\nThe training (and evaluation) of the models still relies on gradients evaluated by measurement. As such, we would expect the barren plateau issues of the model function to be identical to those of other parametrized quantum circuits, presenting similar challenges and being amenable to several known mitigation strategies. One such  mitigation approach, local observables, is present in our experiments. We expect the need to evaluate gradients with respect to $z$ to behave similarly, especially in the case of angle encoding. As we see it, the main bearing of QDEQ on barren plateau issues would be as a potential way of pushing the performance of shallow trainable models further, to some extend mitigating the need to increase circuit depth to hard-to-train levels.\n\nOnce again, we thank the reviewer for their thoughtful feedback and positive evaluation of our work. We hope these additional comments address any remaining questions and are happy to provide further details if needed.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cuM0kLhYuC", "reviewer": "Reviewer_1Wfk", "summary": "In this paper the author present a quantum version of the Deep Equilibrium models. These networks approximate iterative approach through many layers by approximating it with a single set of parameters that would have converged if there were an infinity of layers. The paper is written well and the method is verified on a set of small benchmarks.", "questions": "It would be interesting to see if and how this approach scales with real datasets such as for instance Cifar.", "limitations": "Scalability and novelty.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The paper presents a quantum generalization of the Deep Equilibrium models for the classification task and implemented through an ensemble methods using the VQA for training.", "weaknesses": "- The main disadvantage of the proposed model is the relatively weak result analysis. When looking at the provided results one can conclude that the proposed approach provides in average over one percent accuracy improvement while requiring the double processing time and double memory size. This is when comparing the Implicit solver vs. Single Layer direct learning. Naturally when more layers would be present this could change but with larger dataset the runtime of the implicit solver could change??? \n\nSimply based on these observations I wonder about the usefulness. This is because the MNIST dataset is by default the simplest one and is used as a principle demonstrator. So if such a large overhead is present how will it scale for both the accuracy as well as for the requirements (memory and time)?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s feedback and constructive critique of our work. In response to their concerns, we have provided the following clarifications and additional information:\n\n**Scalability**\n\nWe agree with the reviewer that scaling is an important question, and one that is very difficult to meaningfully assess using the small-scale experiments available in classical simulations of quantum models. This is also why the MNIST dataset was chosen, since the small size of simulatable circuits makes full 10-class MNIST meaningfully challenging task for quantum models built without significant classical components for assistance, i.e., models we develop in this work and our baselines. In general, MNIST-4 and MNIST-10 are used frequently in quantum ML settings because quantum simulations are expensive. With respect to the observed scaling, it should be noted that the compute-cost of classical simulation reported here is only an imperfect proxy for the effort required to train the models with access to a quantum computer. In the long-term quantum context, the amount of measurements required to evaluate gradients would be a central measure of complexity, since measurement and reset is often slower than logic gates. However, even more important currently is circuit depth, which constitutes a hard limit to the size of implementable models. In this respect, we believe QDEQ to be worthwhile for the model-sizes considered here. As for the measurement requirements, the relative overhead of QDEQ scales with the size of the input space relative to the size of the parameter space. Currently, baselines have these spaces at roughly the same size. However, the impact of scaling models and/or problem sizes in the future is hard to predict -- one could hope for deep-learning-like scaling where parameter-size$\\\\gg$data-size, but could also end up being limited by poor optimization landscapes (e.g., the barren plateaus discussed in the response to **BzP4**) to more similar sizes. In the first scenario, the measurement overhead of QDEQ likely becomes insignificant compared to the general effort of estimating gradients wrt. trainable parameters, while the second situation could see more non-negligible overheads. \n\nIn other words, for current devices, circuits are still shallow enough that readout constitutes a significant portion of the runtime. However, since noise sets a hard upper limit on the size of implementable models, QDEQ can still bring advantages despite possessing some measurment overhead (related to measuring gradients in data-space). On the other hand, fault tolerant quantum computers may bring much deeper architectures, where the circuit depth dominates over measurement-time, and where the number of trainable parameters is much larger than the size of the data-space. In such a scenario, the measurement overhead of QDEQ (related to the gradients of the now relatively small data-space) has the potential to be insignificantly small compared to the runtime-cost of running twice as deep a circuit. While predicting how quantum models will scale in the future is very difficult, and certain obstacles in trainability would have to be overcome, we believe that the applicability of QDEQ to both near-term and such future scenarios makes it a worthy topic of study, even with the overhead in measurement complexity. \n\n**CIFAR dataset**\n\nAt the reviewer's request, we have conducted experiments on CIFAR-10 [1], which consists of real-world images. We show the results in a table below. For CIFAR-10, we first converted the images to greyscale and resized from 32x32 to 28x28 to adapt them to our setup.  \n\n| Model    | Test accuracy (\\%) |\n| -------- | ------- |\n| Implicit solver [ours]  |  $0.2438$  |\n| Implicit solver + Warmup [ours] |  $\\\\mathbf{0.2545}$      |\n|  Direct solver [10 layers]  | $0.2371$    |\n|  Direct solver [5 layers]  | $0.2419$     |\n|  Direct solver [2 layers]  | $0.2490$     |\n|  Direct solver [1 layers]  | $0.2470$     |\n\nWe find that in general, near-term quantum ML models that are amenable to numerical experiments do not perform well on CIFAR-10, which has also been observed in recent prior works [1] (models do better if a lot of classical NN components are added, but we refrained from doing this). Despite this, we find that our QDEQ framework (Implicit + Warmup) still does better on the test set than the direct solver baseline. This can motivate the utility of our method on more realistic datasets.\n\n[1] Baek H. et al., Logarithmic dimension reduction for quantum neural networks. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, 2023.\n\n\n**Novelty** \n\nWe believe our work is indeed novel as we develop a new method enabling the first demonstration of applying deep equilibrium models in a quantum machine learning setting. \n\nWe are grateful for the reviewer’s evaluation of our study. We hope our responses address the raised concerns and provide further clarity. We are available to offer additional details if needed.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "m2mCJ45RlU", "reviewer": "Reviewer_fCtD", "summary": "The technique of deep equilibrium models, which were introduced to efficiently handle classical sequential data, is here applied to networks consisting of quantum circuits. The performance is compared to both direct solvers and baseline algorithms (VAE and PCA) for datasets derived from MNIST-4, MNIST, and FashionMNIST. The proposed scheme consistently beats the direct solvers but only sometimes the baselines.", "questions": "Why were all datasets classical and what advantage might a quantum circuit have in their analysis?\nCould quantum datasets be tried instead, and would you expect an advantage there?", "limitations": "The limitations are clearly stated in the article.", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "Bringing deep equilibrium models to quantum circuits seems promising and this is a significant first step demonstrating a speedup for a concrete example. The article clearly describes the setup and what experiments were done. Also limitations are clearly stated as benchmarking is also performed to reference algorithms and not only more basic architectures.", "weaknesses": "All datasets are classical data. It is not clear why one should use quantum circuits for such data except as a first demonstration of the scheme.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for the reviewer’s positive assessment and thoughtful comments on our work. In response to their questions, we offer the following answers. \n\n**Classical vs Quantum Data**\n\nFirstly, we would like to state that we are not intending this work to make any strong statements about quantum advantages for classical data, and we would agree with the reviewer that utility of quantum models for classical data is still an open question. On one hand, applied hybrid quantum/classical methods are receiving a lot of positive attention, and there are supporting theoretical results showing that provable learning advantages on classical data is possible (under certain complexity-theoretic assumptions) on highly artificial problems related to cryptography or contextuality. On the other hand, some claims of advantage have proven hard to reproduce, and there is well-founded scepticism about whether the structure and bias of quantum models aligns with the needs of real-world datasets. Given the differing viewpoints and conflicting evidence, we believe the question open enough that contributing methods to QML for classical data is still meaningful.\n\n\nThe application of QDEQ to quantum data would be a very interesting avenue of further research. However, generalizing the framework to quantum-state inputs/outputs and contractive (i.e., dissipative) parametrized quantum channels presents several subtle questions. For instance, how can we efficiently measure state-closeness when doing root-finding? What is the best way allow the root finder to move in some (efficiently parametrizable) subset of quantum states? How can we adapt the training algorithm to take advantage of this reduced space rather than objects scaling with the full $2^{2Q}$ size of the input/output space?\n\nAnother promising avenue could be considering the outcomes of a preceding quantum experiment as quantum data. This could be an experiment in the sense of a physical experiment, or also in the sense of a preceding quantum algorithm. Either can be seen as an advanced encoding map. For the case of it coming from an algorithm, the final result aka the ML input data if often delivered in amplitude or phase. While this data in amplitude or phase can be transitioned to and from each other by amplitude amplification and quantum phase estimation, such tools may require fault-tolerance and more advanced quantum hardware. In the case of experimental results, in general, we expect this data to be more ``messy'' and it would require a case-by-case basis how to deal with this specifically. Nevertheless, we expect that in the case of data coming from quantum computations and experiments, QDEQ would fare similarly to e.g. variational quantum circuit alternatives on this analysis task, and also similarly inhibit its strengths and weaknesses compared to standard variational circuit approaches.\n\n\nWhile the open questions prevented a direct application of QDEQ as presented here to fully quantum data, further research into answering them, and into whether any advantages transfer, does seems promising. Moreover, we are also hopeful that the broader idea of fixed-point search of quantum maps could be a promising algorithmic building block also more generally, i.e., beyond classification tasks. However, due to the subtleties, it is hard to speculate if the advantages of better shallow-circuit performance transfer, and the considerations in general had to be postponed to future work. Some of the above considerations are in some form mentioned very briefly on lines 173-178 of our submission, to be clarified in future revisions.\n\nWe sincerely appreciate the reviewer’s constructive feedback and encouraging review. We hope these additional insights clarify any outstanding questions, and we are available to provide more details if necessary.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We  sincerely  thank the reviewers for spending time on our paper and providing valuable feedback. We are glad that the reviewers find our generalization of deep equilibrium networks to quantum circuits novel and interesting for real-world applications.  

We have addressed the following points in response to the reviewers comments:
1. Performed experiments on an additional dataset, CIFAR-10. As requested by Reviewer **1Wfk**, we demonstrated our approach on CIFAR10 -- a more realistic dataset. We note that CIFAR10 is a difficult benchmark for current quantum frameworks in general, but we show that using our deep equilibrium framework we can achieve higher performance than with direct solvers, and comparably with fully-quantum baselines. This brings the number of datasets where QDEQ is demonstrated to improve performance to three.
2.   Discussed the impact of (shot) noise. 
3. Provided additional argumentation with respect to the  barren plateau issue and the applicability of QDEQ to quantum data. 
4.  Increased the number of baselines we compare our work to.
5.  Provided a link to an anonymous repository with our codebase. As per the NeurIPS rebuttal policy, we have shared the link with the area chair to forward to the reviewers.

We would like to bring to the reviewers attention that we found a mistake introducing a factor of two in Eq. (17) in the appendix of our submission. This does not change the contractiveness results for the Amplitude encoding -- the main encoding used in this work -- due to cancelling with a factor of $\\frac{1}{2}$ from Eq. (28). However, it does weaken the result for the specific angle encoding used in the MNIST-4 experiments. Since the appendix is only intended to provide observations to elucidate and support the main evidence -- the empirical tests -- this does not change the conclusions of the paper. Nevertheless, we wanted to bring it to the reviewers' attention for the sake of transparency. For more details, see also the discussion of contractiveness in the response to reviewer **1Wfk** below.

Meta-review: The paper extends the deep equilibrium model (DEQ) formulation to quantum machine learning models, and demonstrates that the proposed QDEQ models can achieve better accuracy given same amount of model parameters. Reviewers generally appreciated the novelty and soundness, while also raised concerns including analysis on the influence of noise, insufficient improvement using shallow models, scalability challenge, and limited evaluation datasets. On these points, in the rebuttal the authors made an explanation on the conditions for the method to work, the restriction and potential to carry out larger-scale simulation, and presented additional results on the CIFAR-10 dataset which is relatively larger, and explained the applicability to quantum data. The response seems to have addressed most of the concerns, while others seem reasonably responded. I therefore recommend an accept to the submission.
Decision: Accept (poster)