Quantum Deformed Neural Networks
Authors: Roberto Bondesan
Max Welling
Venue:  (2021)
Keywords: Quantum machine learning
Binary neural networks
Bayesian deep learning

Abstract: We develop a new quantum neural network layer designed to run efficiently on a quantum computer but that can be simulated on a classical computer when restricted in the way it entangles input states. We first ask how a classical neural network architecture, both fully connected or convolutional, can be executed on a quantum computer using quantum phase estimation. We then deform the classical layer into a quantum design which entangles activations and weights into quantum superpositions. While the full model would need the exponential speedups delivered by a quantum computer, a restricted class of designs represent interesting new classical network layers that still use quantum features. We show that these quantum deformed neural networks can be trained and executed on normal data such as images, and even classically deliver modest improvements over standard architectures.  

Under review as a conference paper at ICLR 2021
QUANTUM DEFORMED NEURAL NETWORKS
Anonymous authors Paper under double-bire blind review
ABSTRACT
We develop a new quantum neural network layer designed to run efficiently on quantum computer but that can be simulated classical computer when re- stricted in the way entangles input states. We first ask how a classical neural network architecture, both fully connected or convolutional, can be executed on a quantum computer using quantum phase estimation. We then deform the classical layer into quantum design which entangles activations and weights into quantum superpositions. While the full model would need the exponential speedups deliv- ered by a quantum computer. a restricted class of designs represent interesting new classical network layers that still use quantum features. We show that these quantum deformed neural networks can trained and executed on normal data such as images, and even classically deliver modest improvements over standard architectures.
INTRODUCTION
Quantum mechanics (QM) the most accurate description for physical phenomena very small scales, such as the behavior of molecules, atoms and subatomic particles. QM has a huge impact on our every day lives through technologies such as lasers, transistors (and thus microchips), supercon- ductors and MRI.
recent view of QM has formulated our subjective view of the (quantum) world, and how we update that view in light of evidence (i.e. measurements) (tt Hooft 2016 Fuchs & Schack 2013). This is in perfect analogy to the classical Bayesian view, statistical paradigm extensively used in artificial intelligence where maintain probabilities to represent our beliefs for events the world.
(Bayesian) statistical methodology that only describes
The philosophy of this paper will be to turn this argument its head. If another consistent statistical theory that happens to describe nature at small scales, then we can also use this theory to describe classical signals by endowing them with Hilbert space structure. In some sense, the "only" difference with Bayesian statistics is that the positive probabilities are replaced with complex ''amplitudes.. This however has the dramatic effect that, unlike in classical statistics, interference between events now becomes a possibility. In this paper we show that this point of view uncovers new architectures and potential speedups for running neural networks on quantum computers.
view QM just
We shall restrict our attention here to binary neural networks. We will introduce new class of quan- tum neural networks and interpret them as generalizations of probabilistic binary neural networks, discussing potential speedups by running the models on quantum computer. Then we will devise classically efficient algorithms to train the networks for a restricted set of quantum circuits. We present results of classical simulations of the quantum neural networks on real world data sizes and related gains in accuracy due to the quantum deformations. Contrary to almost all other works on quantum deep learning, our quantum neural networks can be simulated for practical classical prob- lems, such as images sound. The quantum nature of our models is there to increase the flexibility of the model-class and add new operators to the toolbox of the deep learning researcher, some of which may only reach their full potential when quantum computing becomes ubiquitous.

---

Under review as a conference paper at ICLR 2021
1.1 RELATED WORK
In Farhi & Neven (2018) variational quantum circuits that can be learnt via stochastic gradient descent were introduced. Their performance could be studied only small input tasks such as classifying 4 x 4 images, due to the exponential memory requirement to simulate those circuits. Other works on variational quantum circuits for neural networks are Verdon al. (2018): Beer al. (219). Their focus is similarly on the implementation on near term quantum devices and these models cannot be efficiently run on classical computer. Exceptions are models which use tensor network simulations (Cong et al., 2019 Huggins et al., 2019) where the model can be scaled to 8 x 8 image data with classes, at the price of constraining the geometry of the quantum circuit (Huggins) et al. 2019). The quantum deformed neural networks introduced in this paper are instead a class of variational quantum circuits that can be scaled to the size of data that are used in traditional neural networks
demonstrate in section 4.2
Another line of work directly uses tensor networks as full precision machine learning models that can be scaled to the size of real data (Miles Stoudenmire & Schwab 2016 Liu al. 2017 Levine eet al. 2017; Levine et al., 2019). However the constraints on the network geometry to allow for efficient contractions limit the expressivity and performance of the models. See however Cheng iet al. (2020) for recent promising developments. Further, the tensor networks studied in these works unitary maps and do not directly relate to implementations on quantum computers.
A large body of work in quantum machine learning focuses on using quantum computing to provide speedups classical machine learning tasks (Biamonte al., 2017 Ciliberto al., 2018 Wiebel
iet al., 2014). culminating in the discovery of quantum inspired speedups in classical algorithms
(Tang 2019). In particular, (Allcock et al. 2018 Cao et al.. 2017 Schuld al. 2015 Kerenidis et al. 2019) discuss quantum simulations of classical neural networks with the goal of improving the efficiency of classical models on quantum computer. Our models differ from these works in two ways: i) we use quantum wave-functions to model weight uncertainty, in a way that is reminiscent of Bayesian models; ii) we design our network layers in a way that may only reach its full potential on quantum computer due to exponential speedups, but at the same time can, for restricted class of layer designs, be simulated on a classical computer and provide inspiration for new neural architectures. Finally, quantum methods for accelerating Bayesian inference have been discussed in Zhao et al.] (2019b a) but only for Gaussian processes while in this work we shall discuss relations Bayesian neural networks.
2 GENERALIZED PROBABILISTIC BINARY NEURAL NETWORKS
Binary neural networks are neural networks where both weights and activations are binary. Let B- = f0, 1}. A fully connected binary neural network layer maps the N& activations (() at level E to the Ne+1 activations h (e+1) at level l + using weights ((() E NeN+:
N₂ n((+1)= = (0000))00))) = N₂ Σ
7(x)
0
F
(I)
x 2
We divide by N2 + since the sum can take the N: + values (O...., Ne). We do not explicitly consider biases which can be introduced by fixing some activations to 1. In a classification model ( x is the input and the last activation function is typically replaced by a softmax which pro- duces output probabilities p(y /z. W), where W denotes the collection of weights of the network.
(a"...., M), Y - .u......, yy a frequentist approach would determine the binary weights so that the likelihood x W) = IIM p(y: (z, W) is maxi- mized. Here we consider discrete or quantized weights and take the approach of variational opti- mization Staines & Barber (2012), which introduces a weight distribution qo (W) to devise a sur- rogate differential objective. For an objective O(W), one has the bound maxweBs O(W) > Eyp(W) (WW), and the parameters of go (W) are adjusted to maximize the lower bound. In our case we consider the objective:
Given M input/output pairs
M
max log lYg W) 2 L = (ww W)] = Σ WEBN
ll W)]. (2)
2

---

Under review as a conference paper at ICLR 2021
While the optimal solution to equation is Dirac measure,
add regularization term R(Ø) to keep soft. In appendix [A] we review the connection with Bayesian deep learning, where qe (W) the approximate posterior, R(0) is the KL divergence between qo (W) and the prior over weights, and the objective is derived by maximizing the evidence lower bound.
In both variational Bayes and variational optimization frameworks for binary networks, have a variational distribution ((W and probabilistic layers where activations are random variables. We consider an approximate posterior factorized over the layers: ((W) = I (l If
( p(o) equation
1 leads the following recursive definition of distributions:
(n
Σ E
8(h ((0--)--(w())) h )p (h
(3)
hEB WEB
We use the shorthand p (h()) for (h(()) x)) and the dependence is understood. The average
appearing in equation 2 can be written as average over the network output distribution:
E (W)) (low)/oogpr p(y, [e, W)] = - - (2)) ga: (*)),
(4)
where the function Pi is typically MSE for regression and cross-entropy for classification.
In previous works (Shayer et al.
2017 Peters & Welling 2018). the approximate posterior was taken
to be factorized: (W())
Lis
9r (W!!)) which results in factorized activation distribution as Shaaee et al.
2017 Peters & Welling, 2018 used the local reparameterization trick Kingma et al. (2015) to sample activations at each layer.
well: (h( II, 2
(1
The quantum neural network introduce below will naturally give way sample efficiently from complex distributions and view of that we here generalize the setting: we act with a stochastic matrix (h', W W) which depends on parameters & and correlates the weights and the input activations to a layer as follows:
Th. W') = Σ Σ
Sz(h', W h, W))(h)go(W).
(5)
REBN WEBNA
To avoid redundancy, still take qo (W) to be factorized and let S create correlation among the weights as well. The choice S will be related to the choice of unitary matrix D in the quantum circuit of the quantum neural network. A layer is now made of the two operations, Sg and the layer map f. resulting in the following output distribution:
((+1) (h
= Σ Σ
- 11 (Z (()),
(6)
KEBY WAN
which allows one to compute the network output recursively. Both the parameters & and will be learned to solve the following optimization problem:
min R(0) + (o) L.
0,4
where R(0), (Ø) are regularization terms for the parameters o. We call this model generalized probabilistic binary neural network, with & deformation parameters chosen such that ø = 0 gives back the standard probabilistic binary neural network.
To study this model on a classical computer we need to choose S which leads to an efficient sampling could use Markov Chain Monte Carlo, but there exists situations
algorithm for 0a.0- In general, for which the mixing time of the chain grows exponentially in the size the problem (Levin & Peres 2017). In the next section we will show how quantum mechanics binary neural networks that can be efficiently executed and in the subsequent sections we will show experimental results for restricted class of correlated distributions inspired by quantum circuits that can be simulated classically.
enlarge the set of probabilistic
3 QUANTUM IMPLEMENTATION
Quantum computers can sample from certain correlated distributions more efficiently than classical computers (Aaronson & Chen. 2016 Arute al. 2019). In this section, devise quantum circuit
3

---

Under review as a conference paper at ICLR 2021
that implements the generalized probabilistic binary neural networks introduced above, encoding T in a quantum circuit. This leads to an exponential speedup for running this model on a quantum computer, opening up the study of more complex probabilistic neural networks.
A quantum implementation of binary perceptron was introduced in [Schlld et al. 2015) as an application of the quantum phase estimation algorithm (Nielsen & Chuang 2000). However, quantum advantage of the quantum simulation was shown. Here we will extend that result several ways: will modify the algorithm to represent the generalized probabilistic layer introduced above, showing the quantum advantage present in our setting; ii) we will consider the case of multi layer percetrons as well as convolutional networks.
3.1 INTRODUCTION TO QUANTUM NOTATION AND QUANTUM PHASE ESTIMATION
As preliminary step, we introduce notations for quantum mechanics. We refer the reader to Ap- pendix B for a more thorough review of quantum mechanics. A qubit is the vector space of nor- malized vectors (0) E C². N qubits form the set of unit vectors in (C?)@N e C²* spanned by all N-bit strings, 161...., bu)
(61) ..... @ }by). E Quantum circuits are unitary matrices on this space. The probability of a measurement with outcome Φ; is given by matrix element of the projector (6.) {0s| in state / namely p: (vbl))) (0. = (0x1)) 12 formula known as Born's 's rule.
Next, we describe the quantum phase estimation (QPE), a quantum algorithm to estimate the eigen- phases of a unitary U. Denote the eigenvalues and eigenvectors of U by exp( 2mi Pa. and (va). and assume that the PP. 's can be represented with finite number i of bits: Pa 1 (This is the case of relevance for binary network.) Then introduce ancilla qubits in state 10) 11)8 Given an input state (1b), QPE is the following unitary operation:
pa + 20
10)
QPE
Σ
(valv) IPa)) @ (va)
(8)
Appendix B.I reviews the details of the quantum eircuit implementing this map, whose complexity is linear in l. Now using the notation for the threshold non-linearity introduced in equation [ and recalling the expansion 2 2
2 2-1 + ... + 2pp we note that if the first bit ypt 0 then 2 2-pup 1 and 7(2 (p) = O. while if yp 1, then 2 2- > and 7(2 (z)) 1. In other words, 6/166 = (p).6 and the probability p(b) that after the QPE the first ancilla bit is 6 is given by:
Σ (valv)) (saal @ (va)) b) (b) Φ ][ (vale)) 1os)
Σ (va 18.1
where
[b) (b/ & is an operator that projects the first bit to the state (b) and leaves the other bits
untouched.
3.2 DEFINITION AND ADVANTAGES OF QUANTUM DEFORMED NEURAL NETWORKS
Armed with this background, we now apply quantum phase estimation to compute the output of the probabilistic layer of equation 6 Let N be the number of input neurons and M that of output neurons. We introduce qubits to represent inputs and weights bits:
M
h. W) E VA & Vw, VR
(CP)
VW
(10)
i=1
j=1
Then we introduce a Hamiltonian H, acting non trivially only on the N input activations and the N weights at the j-th row:
H, B B.
(11)
and B (BW) is the matrix B =
[1) (1) acting the i-th activation (ji-th weight) qubit. Note that
H, singles out terms from the state (,, W) where both hy 1 and Wj = 1 and then adds them
4

---

Under review as a conference paper at ICLR 2021
(0)
(0) 6
10)
J0y 10y
10y8r
(0) 10
10ye6--1
(pp) (PE(
QME(U
10y81
(m))
QPE(U
Johw...
(Wa)
QPE(U,)
l
QPE M)
(0) 8
WWw?."
[va)
Layer
Layer
(b)
(c)
Figure 1: (a) Quantum circuit implementing a quantum deformed layer. The thin vertical line indi- cates that the gate acts as identity on the wires crossed by the line. (b) Quantum deformed multilayer perceptron with hidden quantum neurons and 1 output quantum neuron. |æ) an encoding of the input signal, is the prediction. The superscript & in U and W refers to layer &. We split the blocks of tę ancilla qubits into a readout qubit that encodes the layer output amplitude and the rest. (c) Modification of layer for classical simulations.
up, i.e the eigenvalues of H are the preactivations of equation !
H, h, W) = phh, Wi.) (),,
sphh, W
Σ Wyshs
(12)
Now define the unitary operators:
U, De N+1
(13)
where D is another generic unitary, and shall shortly, its eigenvectors will be related the entries of the classical stochastic matrix in section 2 Since U,U,y = De (H D-1
UyU diagonalize all the U, 's simultaneously and since they conjugate ex H, they will have the same eigenvalues. Introducing the eigenbasis (h, W) D h, W), we have:
U, /h, W) = = WG)) h, W)
(14)
Note that yo {0,... {O....,NY N} we represent it with exactly bits, N 2' 1. Then add M ancilla resources, each of qubits, and sequentially perform M quantum phase estimations, one for each Uj. as depicted in figure T (a). We choose the following input state
M
(+b) = 12))
X
W...
v Wn = [0) + Vgi(W,; - =1)11> (15)
j=1
1=1
where we have chosen the weight input state according to the factorized variational distribution qı introduced in section 2 In fact, this state corresponds to the following probability distribution via Bor''s rule:
M Phh, W) (h. W 12 p(h) II 93 (W;), p(h) (hl(), 12
(16)
The state (14) is discussed below. Now we show that a non-trivial choice of D leads to an ef- fective correlated distribution. The j-th QPE in figure (a) corresponds equation B where we identify (v.)) 1h, W) D. lyaa) = (h,,, W,.)) and we make use of the j-th block of f. ancillas. After M steps we compute the outcome probability of measurement of the first qubit each of the M registers of the ancillas. We can extend equation 9 to the situation of measuring multiple qubits, and recalling that the first bit an integer is the most significant bit, determining whether 2 ('p(h, W,.) = (N 1) 1 Wj.) is greater smaller than 1/2, the probability of outcome h' = (K....., .......,h. h/r is
p(h') = Σ
Σ
FWW 1w/h, W) W) 2 D
(17)
BERN WEBNM
5

---

Under review as a conference paper at ICLR 2021
where f is the layer function introduced in equation We refer to appendix C for detailed derivation. Equation i7 the generalized probabilisticc binary layer introduced in equation where D corresponds to non-trivial S and a correlated distribution when D entangles the qubits:
(h, W) Kel D D w
(18)
The variational parameters & of S are now parameters of the quantum circuit D. Sampling from can be done by doing repeated measurements of the first M ancilla qubits of this quantum cir- cuit. On quantum hardware e H4 can be efficiently implemented since is a product of diagonal two-qubits quantum gates. We shall consider unitaries D which have efficient quantum circuit ap- proximations. Then computing the quantum deformed layer output on a quantum computer is going to take time (t)) where u(N) is the time it takes to compute the action of U, on an input state. There exists D such that sampling from equation [8] exponentially harder classically than quantum mechanically, statement forming the basis for quantum supremacy experiments noisy, intermediate scale quantum computers (Aaronson & Chen 2016 Arute et al. 2019). Examples random circuits with two-dimensional entanglement patterns, which from machine learning point of view can be natural when considering image data. Other examples are D implementing time evolution operators of physical systems, whose simulation is exponentially hard classically, resulting in hardness of sampling from the time evolved wave function. Quantum supremacy exper- iments give foundations which architectures can benefit from quantum speedups, but we remark that the proposed quantum architecture, which relies on quantum phase estimation, is designed for error-corrected quantum computers.
Even better, on quantum hardware we can avoid sampling intermediate activations altogether. At the first layer, the input can be prepared by encoding the input bits in the state [z)). For the next layers, we simply use the output state as the input to the next layer. One obtains thus the quantum network of figure (b) and the algorithm for a layer summarized in procedure []] Note that all the qubits associated to the intermediate activations are entangled. Therefore the input state (1%)) would have to be replaced by a state in V4 plus all the other qubits, where the gates at the next layer would act only on V/, in the manner described in this section. (An equivalent and more economical mathematical description is to use the reduced density matrix Ph as input state.) We envision two other possible procedures for what happens after the first layer: i) we sample from equation 17 and initialize (wx)) to the bit string sampled in analogy to the classical quantization of activations: ii) we sample many times to reconstruct the classical distribution and encode it in [ In our classical simulations below we will be able to actually calculate the probabilities and can avoid sampling.
Finally, we remark that at present it not clear whether the computational speedup exhibited by our architecture translates to a learning advantage. This is an outstanding question whose full answer will require an empirical evaluation with a quantum computer. Next, we will try get as close as possible to answer this question by studying a quantum model that we can simulate classically.
3.3 MODIFICATIONS FOR CLASSICAL SIMULATIONS
In this paper we will provide classical simulations of the quantum neural networks introduced above for a restricted class of designs. We do this for two reasons: first to convince the reader that the quantum layers hold promise (even though glory due to the lack of access to a quantum computer) and second, to show that these ideas can be interesting as new designs, even "classically" (by which we mean architectures that can be executed on a classical computer).
not simulate the proposed architecture in its full
To parallelize the computations for different output neurons, we do the modifications the setup just explained which are depicted in figure [ (c). We clone the input activation register M times, an operation that quantum mechanically is only approximate (Nielsen & Chuang 2000) but exact classically. Then we associate the j-th copy to the j-th row of the weight matrix, thus forming pairs for each j = I,..., M:
N
h. Wg.) E Vn @ Vw3.
Vw
(C2)
(19)
i=1

---

Under review as a conference paper at ICLR 2021
Fixing j, introduce the unitary e e Hj diagonal in the basis hh, Waa) in equation define the new unitary:
and
U, - Dj Dj H DJ
(20)
where w.r.t equation 13 we now let Dj depend on j. We denote the eigenvectors of Uj by hh, W Woi) Dj h, Wa asd and the eigenvalue (ph,, W5 introduced in equation 12 Supposing that we know p(h) = II. p. (h:), we apply the quantum phase estimation U, with input:
N
1xby)) lvn)) @ (x w...
(bb))
pv (0) + VitVV/llle pi(h, = 1) 11)1)
(21)
and (<) w, defined in equation 15 Going through similar calculations as those done above shows that measurements of the first qubit will be governed by the probability distribution of equa- tion factorized over output channels since the procedure does not couple them: xh, W) T Key Dj lh, W So far, we have focused on fully connected layers. We can extend the derivation of this section to the convolution case, by applying the quantum phase estimation on images patches of the size equal to the kernel size as explained in appendix
4 CLASSICAL SIMULATIONS FOR LOW ENTANGLEMENT
4.1 THEORY
It has been remarked in (Shayer et al. 2017 Peters & Welling 2018) that when the weight and tivation distributions at a given layer are factorized, p(h) = (hx) and ((W) = II, 96 (Way). the output distribution in equation 3 can be efficiently approximated using the central limit theorem
(CLT). The argument goes as follows: for each the preactivations (h, WG.) Σ Wy.hh, are sums of independent binary random variables Wy,th; with mean and variance:
Maa Eur [w))Enpp (h), oj: = -Ewragy (nrop. (h²) - p3, M5:(1 Mgja).
(22)
We used b² = b for a variable b E 1). The CLT implies that for large N we can approximate (h, Wy:) with normal distribution with mean Hí = > Pig and variance of E. The distribution of the activation after the non-linearity of equation caan thus be computed as:
p( P (h, Wj.)) = = p(2p(h, Wj.) - N 0) (-20-
-_n--N 2e;
(23)
T being the CDF of the standard normal distribution. Below fix j and omit it for notation clarity.
As reviewed in appendix E. commuting observables in quantum mechanics behave like classical
random variables. The observable of interest for us, DHD of equation 20 is sum of commut- ing terms K; = DB B D-1 and if their joint probability distribution is such that these random variables weakly correlated, i.e.
(b) K,K₁ (v) (x)) (; (v) (4b) K2 (1+) 0, 1'li-'1-
(24)
then the CLT for weakly correlated random variables applies, stating that measurements of DHD-1 in state (16) are governed by Gaussian distribution N (H, with
# (1b DHD- (v)), o²= DH" 1p) 1²2
(25)
Finally, we can plug these values into equation 23 to get the layer output probability.
We have cast the problem of simulating the quantum neural network to the problem of computing the expectation values in equation 25 physical terms, these are related to correlation functions of H and H2 after evolving state (b) with the operator D. These can be efficiently computed classically for one dimensional and lowly entangled quantum circuits D (Vidal 2003).. In view of that here we consider Id arrangement of activation and weight qubits, labeled by i O,... 2N - 1, where the even qubits are associated with activations and the odd are associated with weights. We then choose:
N-1
N-1
D = II
Q
26
II
P241,2242
(26)
==U
T=U
7

---

Under review as a conference paper at ICLR 2021
where Q2.2/11 acts non-trivially qubits 2i 1, i.e. onto the i-th activation and i-th weight qubits, while 22- on the i-th weight and i + 1-th activation qubits. We depict this quantum circuit in figure (a). As explained in detail in appendix the computation of μi involves the matrix element of K; in the product state (&) while o² involves that of K; K Due to the structure of D, these operators act locally 4 and 6 sites respectively depicted in figure 2 (b)-(c). This
P
P
P
P
P
Q
K;
BB
K,4
BB
BB
hg
Wo
b1
Wi
bg
W2
Q
P
P
Q
11
Q
11
26-1 26:12811
2:1 26:12 24112/226:224
(n)
(b)
(6)
Figure 2: (a) The entangling circuit D for N (b) K; entering the computation μt. (c) K,K,+I entering the computation of o². Indices on P.Q. B are omitted for clarity. Time flows downwards.
implies that the computation of equation 25 and of the full layer, can be done in O(N) and easily parallelized. Appendix [ contains more details on the complexity, while Procedure 2 describes the algorithm for the classical simulation discussed here.
Procedure Quantum deformed layer. QUE, I) is quantum phase estimation for unitary U acting on the set I of activation qubits and the j-th weights/ancilla qubits. H, is in equation
Input: {4g | I/= M-I
N-... (6), I. D,
Output: [+b)
for j = to M
(v) w,.
i=l TH 00) + VI- Zj [1)]
(v) - 10) x
(+) X (V) W,,
U - De 2x1 H. D-1
This requires to approximate the unitary with quantum gates]
(1)
QPE, QUE,(U,UU (U, bb)
end for
Procedure 2 Classical simulation of quantum deformed layer with N (M) inputs (outputs).
Input: {4g
(0,))_.....,N
M-1 M-
(p:) P P 1 M- M-1
Q
(Q
21 15 o.... M-1
i=
Output: [p] (p....
for j = 0 to M 1 do
for = 0 to
do
u2 v Pi, vI pi
U2 Vgj
49
end for for
do t computeMuui, t) P. Q) This implements equation 45 of appendix E
0 to
7,.+1 + computeGamma(i, v. P. P)Q) (This implements equation 49 of appendix E
end for
= 14 a² -- 2
11
Σ
-i=0 (21141 (4/441) + EN' (#4 - 12,)
p/ + +b
2
2/
end for
8

---

Under review as a conference paper at ICLR 2021
4.2 EXPERIMENTS
We present experiments for the model of the previous section. At each layer, Vi and Dj are learnable. They are optimized to minimize the loss of equation 7 where following (Peters) & Welling, 2018: Shayer et al.
Table I: Test accuracies for MNIST and Fashion MNIST. With the notation cKs5 to indicate conv2d layer with C' filters of size K] and stride S, and dN for a dense layer with N output neurons, the architectures (Arch.) A: d10; B: c3s2-8, c3s2-16, d10; C: c3s2-32, c3s2-64, d10.
2017) we take R - 3- Vij (1 (1-2; ' and R' is the L2 regularization loss of the parameters of Dj- L coincides with equation 2 We implemented and trained several architectures with different deformations. Table 0 contains results for two standard image datasets, MNIST and Fashion MNIST. Details of the experiments are in ap- based on (Peters & Welling, 2018), but we use fewer lay- ers make the simulation of the deformation cheaper and use no batch norm, and max pooling.
The deformations are: [/]: P.
Q
(baseline (Peters & Welling 2018)) [PQ]:
P Pi Q generie; [Q]: P. p1 = 1, Q. 0.441 generic.
pendix IF The classical baseline
Arch. Deformation MNIST Fashion MNIST
A
91.1
84.2
[PQ]
94.3
86.8
[Q]
91.6
85.1
B
V/1,/1
96.6
87.5
The general deformation ((PQ)) performs best in all cases. In the simplest case of single dense layer (A), the gain is 3.22% for MNIST and 2.6% for Fashion MNIST on test accu- racy. For convnets, single deformed layer due to computational is- sues and the gain is around or less than 1%. We expect that deforming all layers will give a greater boost as the improvements diminish with decreasing the ratio of deformation parameters over classical parameters (41z). The increase in accuracy comes at the expense of more parameters. In appendix
1,1
97.6
88.1
[Q.1./]
96.8
87.8
C
V.I./]
98.1
89.3
P.////1
98.3
89.6
could only simulate
[Q.4//
98.3
89.5
F we present additional results showing that quantum models can still deliver modest accuracy improvement w.r.t. convolutional networks with the same number of parameters.
5 CONCLUSIONS
In this work we made the following main contributions: 1) we introduced quantum deformed neural networks and identified potential speedups by running these models on quantum computer; 2) we devised classically efficient algorithms train the networks for low entanglement designs of the quantum circuits; 3) for the first time the literature, we simulated the quantum neural networks real world data sizes obtaining good accuracy, and showed modest gains due the quantum deformations. Running these models on a quantum computer will allow one to explore efficiently more general deformations, in partieular those that cannot be approximated by the central limit theorem when the Hamiltonians will be sums of non-commuting operators. Another interesting future direction is to incorporate batch normalization and pooling layers in quantum neural networks.
An outstanding question in quantum machine learning is to find quantum advantages for classical machine learning tasks. The class of known problems for which quantum learner can have provably exponential advantage over a classical learner is small at the moment Liu et al. (2020), and some problems that are classically hard to compute can be predicted easily with classical machine learning Huang et al.] (2020). The approach presented here is the next step in a series of papers that tries to benchmark quantum neural networks empirically, e.g. Farhi & Neven (2018); 8); [Huggins et al. (2019); Grant et al.) (2019) 2018): [Bausch] (2020). We are the first to show that towards the limit of entangling circuits the quantum inspired architecture does improve relative to the classical one for real world data sizes.
9

---

Under review as a conference paper at ICLR 2021
REFERENCES
Scott Aaronson and Lijie Chen. Complexity foundations of quantum supremacy experi- ments, 2016.
Jonathan Allcock, Chang-Yo Yu Hsich, Iordanis Kerenidis, and Shengyu Zhang. Quantum algorithms for feedforward neural networks. arXiv e-prints, art. arXiv:1812.03089 December 2018
Frank Arute, Kunal Arya, Ryan Babbush, Dave Bacon, Joseph C Bardin, Rami Barends, Rupak Biswas, Sergio Boixo, Fernando GSL. Brandao, David A Buell, al. Quantum supremacy using programmable superconducting processor. Nature, 574(7779):005-510 2019.
Johannes Bausch. Recurrent quantum neural networks, 2020.
Kerstin Beer, Dmytro Bondarenko, Terry Farrelly, Tobias Osborne, Robert Salzmann, and Ra- mona Wolf. Efficient Learning for Deep Quantum Neural Networks. arXiv e-prints, art. 10Xiv:1902.10445, February 2019.
Jacob Biamonte, Peter Wittek, Nicola Pancotti, Patrick Rebentrost, Nathan Wiebe, and Seth Lloyd. Quantum machine learning. Nature, 549(7671) 2017.
Yudong Cao, Gian Giacomo Guerreschi, and Alán Aspuru-Guzik, Quantum Neuron: an el- ementary building block for machine learning on quantum computers. arXiv e-prints, art. arXiv:1711. 11240, November 2017.
Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural Ordinary Dif- ferential Equations. arXiv e-prints, art.
Jun 2018.
Song Cheng, Lei Wang, and Pan Zhang. Supervised learning with projected entangled pair states, 2020.
Carlo Ciliberto, Mark Herbster, Alessand ro Davide lalongo, Massimiliano Pontil, Andrea Roc- chetto, Simone Severini, and Leonard Wossnig. Quantum machine learning: a classical perspec- tive. Proceedings of the Royal Society of London Series A, 474(2209):2017055 January 2018. doi: 10.1098/rspa.2017..555
Iris Cong, Soonwon Choi, and Mikhail D. Lukin. Quantum convolutional neural networks. Nature Physics, 17712711 August 2019. doi: 10.1088/441337--199666444
Edward Farhi and Hartmut Neven. Classification with Quantum Neural Networks on Near Term February 2018.
Processors. arXiv e-prints, art.
Christopher A. Fuchs and Rüdiger Schack. Quantum-Bayesian coherence. Reviews of Modern Physics, 85(4):1693-1715 Oct 2013. doi: 10.1105/RewoodPhy...5
Edward Grant, Marcello Benedetti, Shuxiang Cao, Andrew Hallam, Joshua Lockhart, Vid Stojevic, Andrew G Green, and Simone Severini. Hierarchical quantum classifiers. npj Quantum Informa- lion, 4(1):1-8, 2018.
Edward Grant, Leonard Wossnig, Mateusz Ostaszewski, and Marcello Benedetti. An initialization strategy for addressing barren plateaus in parametrized quantum circuits. Quantum, 3:214, Dec 2019. ISSN 2-21-327X doi: 10.223331q-201111 12-09-112 URL http://da. doi /0. 201 9- 1
Hsin-Yuan Huang, Michael Broughton, Masoud Mohseni, Ryan Babbush, Sergio Boixo, Hartmut Neven, and Jarrod R. McClean. Power of data in quantum machine learning, 2020.
William Huggins, Piyush Patil, Bradley Mitchell, K. Birgitta Whaley, and E. Miles Stoudenmire. Towards quantum machine learning with tensor networks. Quantum Science and Technology. (2):024001, Apr 2019. doi: 11108822058-9565/aaae
Iordanis Kerenidis, Jonas Landman, and Anupam Prakash. Quantum algorithms for deep convolu- tional neural networks, 2019.
10

---

Under review as a conference paper at ICLR 2021
Diederik P. Kingma, Tim Salimans, and Max Welling. Variational dropout and the local reparame- terization trick, 2015.
David A Levin and Yuval Peres. Markov chains and mixing times, volume 107. American Mathe- matical Soc., 2017.
Yoav Levine, David Yakira, Nadav Cohen, and Amnon Shashua. Deep Learning and Quantum Entanglement: Fundamental Connections with Implications to Network Design. arXiv e-prints, art. MrXivv1104.015555 April 2017.
Yoav Levine, Or Sharir, Nadav Cohen, and Amnon Shashua. Quantum entanglement in deep learn- ing architectures. Physical review letters, 122(6))065301, 2019.
Ding Liu, Shi-Ju Ran, Peter Wittek, Cheng Peng, Raul Blázquez García, Gang Su, and Maciej Lewenstein. Machine Learning by Unitary Tensor Network of Hierarchical Tree Structure. arXiv e-prints, art, arXiv:1710.0483.. October 2017.
Yunchao Liu, Srinivasan Arunachalam, and Kristan Temme. A rigorous and robust quantum speed- up in supervised machine learning. arXiv e-prints, art. arXiv:2010.02174, October 2020.
E. Miles Stoudenmire and David J. Schwab. Supervised Learning with Quantum-Inspiree Tensor Networks. arXiv e-prints, art. arXiv: 100..000575 May 2016.
M.E. Nielsen and I.L. Chuang. Quantum Computation and Quantum Information. Cambridge
Series Information and the Natural Sciences. Cambridge University Press, 2000. ISBN 9780521635035. URL httpss/::
sattps://books //books......//.oo.......o.o.. google. co www.kaokksiiaaaa..... PV99-14C
Jorn W. T. Peters and Max Welling. Probabilistic binary neural networks, 2018.
Maria Schuld, Ilya Sinayskiy, and Francesco Petruccione. Simulating a perceptron on quantum
computer. Physics Letters A, 379(7))660--66))) 663, Mar 2015. ISSN 0375-9601. doi: 10.101 doi.
physleta 2014.cta.0114.11.016.... URL http::p://a..ddo..g............. ptya:p/l.tt.t/....setttttttttaaa 2014. 061
Oran Shayer, Dan Levi, and Ethan Fetaya. Learning discrete weights using the local reparameteri- zation trick. arXiv preprint arXiv:1710.07739) 2017..
Joe Staines and David Barber. Variational optimization, 2012.
G. :t Hooft. The Cellular Automaton Interpretation of Quantum Mechanics. Fundamental Theories of Physics. Springer International Publishing, 2016. ISBN 9783319412856. URL https:/7 books. google. co. uk/booksiidiicooooooo ItT
Ewin Tang. A quantum-inspired classical algorithm for recommendation systems. In Proceedings of the 51st Annual ACM SIGACT Symposium Theory of Computing, STOC 2019, pp. 217-228, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 9781450367059, doi: 10.1145/313327.333133100
Guillaume Verdon, Jason Pye, and Michael Broughton. A Universal Training Algorithm for Quan- tum Deep Learning. arXiv e-prints, art. arXiv: 09799, June 2018.
Guifré Vidal. Efficient classical simulation of slightly entangled quantum computations. Physical review letters, 91(14):147902, 2003.
Nathan Wiebe, Ashish Kapoor, and Krysta M. Svore. Quantum Deep Learning. arXiv e-prints, art arXiv:1412.348) December 2014.
Zhikuan Zhao, Jack K. Fitzsimons, and Joseph F. Fitzsimons. Quantum assisted gaussian process regression. Phys. Rev. A, 99:052331. May 2019a. doi: 1.L10/PhyyReeA..9.05553 URL https: aps ww.. dot/10 1at1//1100e//aa.... 99. 052331
Zhikuan Zhao, Jack K. Fitzsimons, Michael A. Osborne, Stephen Roberts, and Joseph F. Fitzsi- mons. Quantum algorithms for training gaussian processes. Phys. Rev. A, 100:012304, Jul 2019b. doi: 10.1 10.1103/hyyRerAA.100000 100.00113340 URL https: aps ootpp..oo/o//.c....//
11 1103/Phys 100. 012344
11


Reviews:
[{"review_id": "ze4TIjyau9r", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors introduced the idea of quantum deformed neural network (DQNN) which allows substitution of the positive probabilities in Baysian statistics with complex amplitude inspired by Born representation of quantum wavefunction, thus allowing interference phenomena leading to possible speed up for running neural networks on quantum computers. Combining the complex amplitude and binary neural network the author introduces a new class of quantum neural networks which are the generalized probabilistic neural network. Then, they argue that this class of DQNN can efficiently run on quantum computers using phase estimation algorithms which scale linearly with the number of qubits. By creating entanglement between the activation and weights they show that this new form of DQNN can be simulated on classical computers for certain classes of problems that hold low entanglement. As a numerical experiment they show modest gain in accuracy on real world data both MNIST and fashion MNITS data.\n\nPro: \nNovel encoding: in this work the two main properties of quantum mechanics namely the interference and entanglement is being encoded in the deformed neural network to investigate potential quantum speed up within this platform \n\nCons:\n1- The author loosely discuss the expressive power of the DQNN compared to NN.\n\n2. The implementation of phase estimation algorithm requires large depth (linear in number of qubits) and considering the fact that in this algorithm the number of required qubits increases by increasing the connectivity of the neural network, the quantum implementation becomes quite challenging and beyond NISQ device. It probably would be disadvantages compare to the QNN in Farhi's paper.\n\n3. The author also compare their work with tensor network, however, they seem not to be updated on what has been done on this part and they make a few inaccurate claims: \n     3A. On page 2, section 1.1 on second paragraph the author provides a brief study on the tensor network, claiming that \"However the constraints on the network geometry to allow for efficient contractions limit the expressivity and performance of the models. Further these works do not directly relate to implementations on quantum computers\".\nthis is not a correct statement as there are direct implementation of tensor network on quantum circuit, see ref [Huggins], also in recent paper [https://arxiv.org/abs/1907.03741] led by Cirac, they provide proof that quantum circuits are Born machines or locally purified states. \n     3B. The author also claims that their study is one of the first studied in QML where they are able to work on the real size MNIST data, which again is not true. Indeed, Lei Wang and his collaborators in Ref[https://arxiv.org/abs/2009.09932] were able to use supervised learning on PEPS for MNIST and MNIST fashion data.\n\n4. Also, while the tensor network enjoys a concrete parametrization of entanglement with the bond dimension, it's less clear for me how they can tune the entanglement within their formalism.\n\n5. The authors have multiple referral to exponential advantage (or quantum supremacy) in quantum computers which are either incorrect or at least misleading in this context. Those claims are relevant for either random quantum circuits (which have no clear applications) or quantum simulations of certain physically sparse Hamiltonians. In general, the learning complexity could be  independent of simulation complexity of quantum computers once the data is provided, for example see the recent work on the Power of data in quantum machine learning: arXiv:2011.01938\n\n6. The authors conclusion about limited advantage of quantum-inspired deformed neural networks is not correct as it could be simply explained by the over parametrization of deformed model", "extra_scores": {}, "extra_text": {}}, {"review_id": "PzB74SHx7eK", "reviewer": "AnonReviewer6", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors define a quantum version of a binary neural network (all weights and activations are 0 or 1) by first defining a classical stochastic generalization and then upgrading the stochastic part to a quantum process. The calculations all appear to be correct, and to my knowledge this type of quantum neural network is novel. The quantum network is simulated on MNIST and Fashion MNIST. The quantum network shows improvements in test accuracy, but those improvements are attributed to increased parameterization. Thus the jury is still out on whether this is actually an interesting use-case for quantum computers, and it's not clear that this work will have any real impact on the field. For that reason, I cannot recommend this paper for acceptance.\n\nPros:\n1) Clearly explains the quantum theory in a way that should be understandable to a novice.\n2) A novel design of quantum neural networks, to the best of my knowledge.\n3) Makes an attempt to test the performance in simulation.\n\nCons:\n1) The authors make a number of claims about possible quantum advantage, but there is no reason to suspect quantum advantage based on the evidence presented. (Generally, one should expect that there is no quantum advantage for classical data processing unless clear evidence is given to the contrary.) It would be good to attempt to distinguish between improvements in performance due to overparameterization from improvements due to quantum properties, e.g., entanglement.\n2) By the same token, is there any benefit from using the quantum deformed model over the classical stochastic model? More information about the stochastic model would be helpful.\n3) The paper would benefit from a discussion of near-term vs long-term uses of a quantum computer (i.e., noisy vs error-corrected). The authors use phase estimation in their construction, which is typically an operation that is reserved for error-corrected quantum computers. The authors mention noisy intermediate-scale computers at one point, but it's not clear at all that this algorithm is applicable in that case. If it is, the authors should discuss the effects of noise on their results.\n\n\nMiscellaneous comments:\n1) The authors cite Hooft 2016, but this should be 't Hooft. The 't part is included with the surname.\n2) In the 4th paragraph of the introduction, \"restrict\" should be \"restricted\".\n3) The authors state that Farhi and Neven 2018 restricted themselves to 4x4 images because of exponential time of simulations. I believe it's more likely that it was memory constraints of doing full quantum simulations that was important there.\n4) in the second paragraph of section 2 there is a malformed citation that just says \"staines2012variational\".\n5) In section 3.1, the authors state that a qubit is a normalized vector. A qubit more properly refers to the vector space, or to the set of unit vectors in that space, rather than an individual vector. The wording is a bit sloppy, since N qubits are properly associated with a vector space in the following line.\n6) In the last paragraph of 3.2, the authors refer to the V_h subspace. V_h is not a subspace, but a factor (or subfactor). The distinction is important as it often confuses people.\n7) In the first paragraph of 3.3, \"it's' should be \"its\".", "extra_scores": {}, "extra_text": {}}, {"review_id": "BTT6M8j1bKP", "reviewer": "AnonReviewer7", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "__Summary__\nA \"quantum deformed\" generalization of a probabilistic binary neural network is introduced, which can be either run on a quantum computer or (for certain simplified cases) simulated with a classical computer. The performance of the model under classical simulation is assessed with experiments on the MNIST and Fashion MNIST datasets.\n\n__Strengths__\nThe fusion of primitives from quantum computing with a variational Bayesian framework is interesting, and seems like a promising general means of leveraging the complementary strengths of ML and quantum computing.\n\nThe proposed model can be implemented in two different \"modes of operation\", either as a fully quantum model running on a quantum computer or as a classical simulation procedure running on a classical (i.e. standard modern-day) computer. Hybrid methods with this capability for dual use seem promising for quantum ML, since they allow the same framework to be gracefully transitioned from classical computers to quantum computers as the capabilities of the latter improve with time.\n\n__Weaknesses__\nThe practical benefits of the proposed model aren't clearly explained, and the merits of the architecture appear rather weak to me, both from the standpoints of ML and from that of quantum computing. I will elaborate on this in the following paragraphs, but a short summary is that the work (a) Doesn't give any evidence that these methods help in learning real-world ML datasets relative to standard neural networks, and (b) Doesn't list any advantages of the proposed model for the implementation of quantum computing protocols. I'm open to the possibility of error in this assessment, and would love if the authors could provide clear evidence refuting either of these two points.\n\nFrom the perspective of (classical) ML, I can't see any notable advantages of the proposed methods. The authors report small gains relative to the previous proposal of [1] (although see the last point below), but the primary motivation of [1] was to design a lightweight binarized neural network with reduced compute and memory costs. The present work doesn't do this, with the classical mode of operation requiring significant resources, and also delivering performance below a CNN of the same size.\n\nThe authors frequently reference the \"quantum supremacy\" work of [2] and [3] as evidence of the advantages of quantum computers, and mention that the proposed model could be used for such tasks. However, quantum supremacy experiments are not designed to be useful for real-world problems (see [4] for a detailed discussion regarding this point). Rather, the primary role of quantum supremacy experiments is to unambiguously certify that a resource limited quantum computer/simulator is doing *something* that can't be efficiently reproduced with a classical computer, even if that something has no practical use. As a result, any proposed methods for quantum supremacy should be judged by their theoretical guarantees of classical insimulatability, along with their ease of implementation in noisy present-day quantum computing hardware. With that in mind...\n\nThe present work doesn't give any new theoretical guarantees that the proposed architecture demonstrates quantum supremacy, instead briefly mentioning that the quantum circuits proposed in [2] and [3] could be embedded inside the current model (making the current proposal strictly more difficult to implement than these prior works). The use of quantum phase estimation throughout the quantum deformed neural network means that the proposed algorithm would be very challenging to implement in practice, likely only being possible when quantum hardware has advanced to the point that such supremacy demonstrations are no longer a subject of interest.\n\nThe MNIST and Fashion MNIST experiments are rather weak, with the only baseline being the closely related model of [1]. Relative to standard neural networks, these results are far from state-of-the-art, and the classically simulable version of the protocol is also significantly more expensive to run than a basic MLP or CNN.\n\nOne point of confusion I ran into is the experimental results reported for the baseline model of [1]. The values reported in the present paper for this baseline are significantly lower than the MNIST results originally reported in [1], with the latter giving better results than anything exhibited by the present architecture. Can the authors comment on this discrepancy?\n\nOverall assessment:\nAlthough the present work provides some interesting ideas, the theoretical and practical benefits of the model seem marginal. Consequently, I can't recommend acceptance.\n\n[1] Jorn W.T. Peters and Max Welling. Probabilistic binary neural networks, 2018.\n[2] Scott Aaronson and Lijie Chen. Complexity-theoretic foundations of quantum supremacy experiments, 2016.\n[3] Frank Arute et al. Quantum supremacy using a programmable superconducting processor. Nature 574, 2019.\n[4] Scott Aaronson. Scott’s Supreme Quantum Supremacy FAQ, 2019. (https://www.scottaaronson.com/blog/?p=4317)\n\n### UPDATE AFTER THE REBUTTAL\n\nMany thanks to the authors for their reply, and my sincere apologies for forgetting to include my bibliography in the earlier response (you can find it above)! I also appreciate the information about the comparison to [1], as well as the promise of a clarifying statement about the appropriate experimental context for the use of quantum phase estimation.\n\nWhile I agree that the comparison to previous QNN models is favorable, I still think that the use of quantum phase estimation in the proposed method is problematic, as it would significantly delay any potential deployment of the proposed method on a quantum computer. I don't disagree with the authors that their statement regarding the ability to demonstrate quantum supremacy is technically correct, but they haven't addressed my question about _why_ there would be any advantage to carrying out a quantum supremacy experiment on such a model. Studying QNNs for the sake of QNNs isn't a compelling justification in my eyes, and without any strong practical advantages (either definite or potential) provided for the model, my rating unfortunately remains the same.", "extra_scores": {}, "extra_text": {}}, {"review_id": "AvY1h9NHpD", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The fundamental idea in this paper is to endow classical signals with a complex Hilberspace structure so as to harness the probabilistic nature of quantum mechaniscs for pattern recognition based on quantum computing principles. Following this idea, the authors consider (probabiistic) binary neural networks and develop a corresponding kind of quantum neural networks building on the concept of quantum phase estimation. They (convincingly) argue that this idea would make good use of the exponential speedup offered by quantum computers but can still be implemented on classical, digital devices. Experiments with image data corroborate this claim and demonstrate that simulations of the proposed quantum deformation showed improved accuracies when compared to  classical probabilistic binary neural networks.\n\nThe Ideas brought forth in this paper appear to be novel and (to those with a solid background in quantum computing) technically sound. Overall, the paper presents an interesting approach towards quantum computational intelligence which, though likely not yet realizable on real quantum hardware (due to technical limitations such as measurement noise or decoherence times) can also be simulated digitally (in an arguably elegant manner).\n\nHowever, there also are concerns regarding the  quality of this manuscript. In additions to several broken LaTeX links (e.g. a reference to Table 4.2 which appears to mean to refer to Table 1) which unnecessarily hamper readability, the overall presentation is lackingcan. Most critically, the paper can hardly be considered self-contained. Numerous important details are deferred to the supplementary material where the authors cover algorithmic details as well as details as to their experimental procedures. In other words, the supplementary material is not just supplementary but crucial for understanding / assessing the content of this work. While this would not justfy an outright rejection, it still diminishes the overall quality of this paper. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "-KSiC81GCII", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "After rebuttal\nThanks for responding to the concerns in the review.\nThe main point in the author's rebuttal is that the major contribution of the paper is a novel QNN that can be efficiently simulated. However, the proposed method of simulation is only an approximation of the real run through CLT. The author provided some conceptual justification of this approximation but there is no numerical verification. The approximation can be inaccurate when the variables in the summation are correlated or the number of variables is small. It's unclear how the accuracy of simulation can be guaranteed.\n\nOverall the paper proposes an interesting method of implementing BNN using quantum device, which has potential advantage over classical BNN and may be simulated efficiently with some approximation. More solid justifications of the advantages would improve the quality of the paper. \n\n---------------------------------------------\n\nOriginality:\nThe paper presents a method that uses quantum circuits to realize layers of a stochastic binary neural network, where the activations and weights take binary values, and the weights can be stochastic. It also presents a training method that can be run on a classical computer, based on efficient simulation of specific quantum circuits and approximated computation of the distribution of the activations.\nCompared with the classical stochastic binary neural network, the quantum realization may perform efficient \"deformation\", which creates correlations among the input activations and the weights.\nThe proposed model is interesting, especially in that it supports stochastic weights and deformation. This model also looks to be new.\n\n\nClarity:\nThe presentation of the model and its quantum realization is clear. The description on the classical simulation is less clear but OK.\nThe main concerns are\n1. There is no justification or discussion on the advantage of deformation. \nThe advantage of deformation looks to be a way to increase the expressiveness of the model. This is true if the model has only one layer. How much gain can it provide for multi-layer models?\nAlso, there should be a trade-off between the expressiveness and the realization complexity of the deformation, but this is not discussed.\n2. In the classical simulation, there is no discussion on why the chosen form of P and Q can lead to better performance. Also, the details of P and Q are not revealed.\n3. It is not clear why a modification as in Section 3.3 is needed for classical simulation.\n4. In Section 2, the formulation of the training objective (2) is also unclear. It claims that the weights can be represented by a Bayesian posterior and trained by ELBO, but (2) is different from ELBO.\n\n\nSignificance:\nIn the experiment session, it is reported that the performance on MNIST is better with deformation than without it.\nBut this comparison may be trivial, as deformation adds more parameters into the plain model.\nAlso, this may not mean that the proposed model when used as a classical model can perform better than the classical binary neural network, as the proposed model can have much higher computational complexity for performing the deformation.\nIt would be more convincing if some comparison can be made between the proposed model and the other quantum models or classical models with similar complexity.\n\n\n\n\n\n\n\n\n\n\n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review:  A "quantum deformed" generalization of a probabilistic binary neural network is introduced, which can be either run on a quantum computer or simulated with a classical computer. Reviewers agreed that the paper is well written, introduces some new ideas merging quantum computing with a variational Bayesian framework, and the reported numbers on MNIST and Fashion MNIST outperform prior QNN approachers. However, reviewers questioned how useful the proposed ideas are, noting that: the reported gains could be attributed to increased parameterization (this was not carefully ablated with baseline approaches). Additionally, while the quantum supremacy experiments seem technically correct, there was no clear motivation for empirically demonstrating quantum supremacy when no theoretical guarantees are provided. Taken together, there was no clear path to practical improvements of real systems from the proposed ideas.
Decision: Reject