Title: Exponential Quantum Communication Advantage in Distributed Inference and Learning
Venue: NeurIPS 2024
Accepted: yes
Keywords: Quantum computing

Communication complexity

distributed computation

graph neural networks
Abstract: Training and inference with large machine learning models that far exceed the memory capacity of individual devices necessitates the design of distributed architectures, forcing one to contend with communication constraints.  We present a framework for distributed computation over a quantum network in which data is encoded into specialized quantum states. We prove that for models within this framework, inference and training using gradient descent can be performed with exponentially less communication compared to their classical analogs, and with relatively modest overhead relative to standard gradient-based methods. We show that certain graph neural networks are particularly amenable to implementation within this framework, and moreover present empirical evidence that they perform well on standard benchmarks.
To our knowledge, this is the first example of exponential quantum advantage for a generic class of machine learning problems that hold regardless of the data encoding cost. 
Moreover, we show that models in this class can encode highly nonlinear features of their inputs, and their expressivity increases exponentially with model depth.
We also delineate the space of models for which exponential communication advantages hold by showing that they cannot hold for linear classification. 
Communication of quantum states that potentially limit the amount of information that can be extracted from them about the data and model parameters may also lead to improved privacy guarantees for distributed computation. Taken as a whole, these findings form a promising foundation for distributed machine learning over quantum networks.

Exponential Quantum Communication Advantage in Distributed Inference and Learning

Abstract

Training and inference with large machine learning models that far exceed the memory capacity of individual devices necessitates the design of distributed architectures, forcing one to contend with communication constraints. We present a framework for distributed computation over a quantum network in which data is encoded into specialized quantum states. We prove that for models within this framework, inference and training using gradient descent can be performed with exponentially less communication compared to their classical analogs, and with relatively modest overhead relative to standard gradient-based methods. We show that certain graph neural networks are particularly amenable to implementation within this framework, and moreover present empirical evidence that they perform well on standard benchmarks. To our knowledge, this is the first example of exponential quantum advantage for a generic class of machine learning problems that hold regardless of the data encoding cost. Moreover, we show that models in this class can encode highly nonlinear features of their inputs, and their expressivity increases exponentially with model depth. We also delineate the space of models for which exponential communication advantages hold by showing that they cannot hold for linear classification. Communication of quantum states that potentially limit the amount of information that can be extracted from them about the data and model parameters may also lead to improved privacy guarantees for distributed computation. Taken as a whole, these findings form a promising foundation for distributed machine learning over quantum networks.

1 Introduction

As the scale of the datasets and parameterized models used to perform computation over data continues to grow [62, 51], distributing workloads across multiple devices becomes essential for enabling progress. The choice of architecture for large-scale training and inference must not only make the best use of computational and memory resources, but also contend with the fact that communication may become a bottleneck [97]. This is particularly pertinent as models grow so large that they cannot rely on high-bandwidth interconnects within datacenters [17], but are instead trained across multiple datacenters [108]. When using modern optical interconnects, classical computers exchange bits represented by light. This however does not fully utilize the potential of the physical substrate; given suitable computational capabilities and algorithms, the quantum nature of light can be harnessed as a powerful communication resource. Here we show that for a broad class of parameterized models, if quantum bits ( qubits ) are communicated instead of classical bits, an exponential reduction in the communication required to perform inference and gradient-based training can be achieved. This protocol additionally guarantees improved privacy of both the user data and

Jarrod R. McClean Google Quantum AI Venice, CA, United States

model parameters through natural features of quantum mechanics, without the need for additional cryptographic or privacy protocols. To our knowledge, this is the first example of generic, exponential quantum advantage on problems that occur naturally in the training and deployment of large machine learning models. These types of communication advantages help scope the future roles and interplay between quantum and classical communication for distributed machine learning.

Quantum computers promise dramatic speedups across a number of computational tasks, with perhaps the most prominent example being the ability revolutionize our understanding of nature by enabling the simulation of quantum systems, owing to the inherently quantum nature of many many physical phenomena [39, 72]. However, much of the data that one would like to compute with in practice seems to come from an emergent classical world rather than directly exhibiting quantum properties. While there are some well-known examples of exponential quantum speedups for classical problems, most famously factoring [106] and related hidden subgroup problems [31], these tend to be isolated and at times difficult to relate to practical applications that involve learning from data. In addition, even though significant speedups are known for certain ubiquitous problems in machine learning such as matrix inversion [48] and principal component analysis [73], the advantage is often lost when including the cost of loading classical data into the quantum computer or of reading out the result into classical memory. This is because the complexity of loading dense classical data into the amplitudes of a quantum state (which is typically the encoding needed to obtain an exponential runtime advantage) and of reading out the amplitudes from a quantum state into classical memory, are both polynomial in the number of amplitudes [1]. In applications where an efficient data access model avoids the above pitfalls, the complexity of quantum algorithms tends to depend on condition numbers of matrices which scale with system size in a way that reduces or even eliminates any quantum advantage [82]. It is worth noting that much of the discussion about the impact of quantum technology on machine learning has focused on computational advantage. However quantum resources are not only useful in reducing computational complexity - they can also provide an advantage in communication complexity, enabling exponential reductions in communication for some problems [101, 15]. Inspired by these results, we study a setting where quantum advantage in communication is possible across a wide class of machine learning models. This advantage holds without requiring any sparsity assumptions or elaborate data access models such as QRAM [42].

We focus on compositional distributed learning, known as pipelining [56, 16]. While there are a number of strategies for distributing machine learning workloads that are influenced by the requirements of different applications and hardware constraints [115, 61], splitting up a computational graph in a compositional fashion (Figure 1) is a common approach. We describe distributed, parameterized quantum circuits that can be used to perform inference over data when distributed in this way, and can be trained using gradient methods. The ideas we present can also be used to optimize models that use certain forms of data parallelism (Appendix C). In principle, such circuits could be implemented on quantum computers that are able to communicate quantum states.

We show the following:

Even for simple distributed quantum circuits, there is an exponential quantum advantage in communication for the problem of estimating the loss and the gradients of the loss with respect to the parameters (Section 3). This additionally implies a privacy advantage from Holevo's bound (Appendix H). We also show that this is advantage is not a trivial consequence of the data encoding used, since it does not hold for certain problems like linear classification (Appendix E).

We study a class of models that can efficiently approximate certain graph neural networks (Section 4), and show that they both maintain the exponential communication advantage and achieve performance comparable to standard classical models on common node and graph classification benchmarks (Section 5).

For certain distributed circuits, there is an exponential advantage in communication for the entire training process, and not just for a single round of gradient estimation. This includes circuits for fine-tuning using pre-trained features. The proof is based on convergence rates for stochastic gradient descent under convexity assumptions (Appendix D).

The ability to interleave multiple unitaries encoding nonlinear features of data enables expressivity to grow exponentially with depth, and universal function approximation in some settings. This implies that these models are highly expressive in contrast to popular belief about linear restrictions in quantum neural networks (Appendix F).

2 Preliminaries

2.1 Large-scale learning problems and distributed computation

Pipelining is a commonly used method of distributing a machine learning workload, in which different layers of a deep model are allocated distinct hardware resources [56, 87]. Training and inference then require communication of features between nodes. Pipelining enables flexible changes to the model architecture in a task-dependent manner, since subsets of a large model can be combined in an adaptive fashion to solve many downstream tasks. Additionally, pipelining allows sparse activation of a subset of a model required to solve a task, and facilitates better use of heterogeneous compute resources since it does not require storing identical copies of a large model. The potential for large models to be easily fine-tuned to solve multiple tasks is well-known [25, 20], and pipelined architectures which facilitate this are the norm in the latest generation of large language models [99, 16]. Data parallelism, in contrast, involves storing multiple copies of the model on different nodes, training each on a subsets of the data and exchanging information to synchronize parameter updates. In practice, different parallelization strategies are combined in order to exploit trade-offs between latency and throughput in a task-dependent fashion [115, 61, 97]. Distributed quantum models were considered recently in [94], but the potential for quantum advantage in communication in these settings was not discussed.

2.2 Communication complexity

Communication complexity [117, 65, 98] is the study of distributed computational problems using a cost model that focuses on the communication required between players rather than the time or computational complexity. The key object of study in this area is the tree induced by a communication protocol whose nodes enumerate all possible communication histories and whose leaves correspond to the outputs of the protocol. The product structure induced on the leaves of this tree as a function of the inputs allows one to bound the depth of the tree from below, which gives an unconditional lower bound on the communication complexity. The power of replacing classical bits of communication with qubits has been the subject of extensive study [30, 23, 27]. For certain problems such as Hidden Matching [15] and a variant of classification with deep linear models [101] an exponential quantum communication advantage holds, while for other canonical problems such as Disjointness only a polynomial advantage is possible [102]. Exponential advantage was also recently shown for the problem of sampling from a distribution defined by the solution to a linear regression problem

[83]. While there are many models of both quantum and classical communication, our results apply to randomized classical communication complexity, wherein the players are allowed to exchange random bits independent of their problem inputs, and are allowed to output an incorrect answer with some probability (bounded away from 1 / 2 for a problem with binary output). It is also worth noting that communication advantages of the type we demonstrate can be naturally related to space advantages in streaming algorithms that may be of interest even in settings that do not involve distributed training [103].

At a glance, the development of networked quantum computers may seem much more challenging than the already herculean task of building a fault tolerant quantum computer. However, for some quantum network architectures, the existence of a long-lasting fault tolerant quantum memory as a quantum repeater, may be the enabling component that lifts low rate shared entanglement to a fully functional quantum network [86], and hence the timelines for small fault tolerant quantum computers and quantum networks may be more coincident than it might seem at first. As such, it is well motivated to consider potential communication advantages alongside computational advantages when talking about the applications of fault tolerant quantum computers. In Appendix G we briefly survey approaches to implementing quantum communication in practice, and the associated challenges.

In addition, while we largely restrict ourselves here to discussions of communication advantages, and most other studies focus on purely computational advantages, there may be interesting advantages at their intersection. For example, it is known that no quantum state built from a simple (or polynomial complexity) circuit can confer an exponential communication advantage, however states made from simple circuits can be made computationally difficult to distinguish [59]. Hence the use of quantum pre-computation [57] and communication may confer advantages even when traditional computational and communication cost models do not admit such advantages due to their restriction in scope.

3 Distributed learning with quantum resources

In this work we focus on parameterized models that are representative of the most common models used and studied today in quantum machine learning, sometimes referred to as quantum neural networks [79, 38, 28, 104]. We will use the standard Dirac notation of quantum mechanics throughout. A summary of relevant notation and the fundamentals of quantum mechanics is provided in Appendix A. We define a class models with parameters Θ , taking an input x which is a tensor of size N . The models take the following general form:

Definition 3.1. { A ℓ ( θ A ℓ , x ) } , { B ℓ ( θ B ℓ , x ) } for ℓ ∈ { 1 , . . . , L } are each a set of unitary matrices of size N ′ × N ′ for some N ′ such that log N ′ = O (log N ) 3 . The θ A ℓ , θ B ℓ are vectors of P parameters each. For every ℓ, i , we assume that ∂A ℓ ∂θ A ℓi is anti-hermitian and has two eigenvalues, and similarly for B ℓ 4 .

The model we consider is defined by

where ψ ( x ) is a fixed state of log N ′ qubits.

The loss function is given by

where P 0 is a Pauli matrix that acts on the first qubit.

In standard linear algebra notation, the output of the model is a unit norm N ′ -dimensional complex vector φ L , defined recursively by

where the entries of φ L are represented by the amplitudes of a quantum state. The loss takes the form L (Θ , x ) = ( φ ∗ L ) T P 0 φ L where ∗ indicates the entrywise complex conjugate, and this definition includes the standard L 2 loss as a special case.

Subsequently we omit the dependence on x and Θ (or subsets of it) to lighten notation, and consider special cases where only subsets of the unitaries depend on x , or where the unitaries take a particular form and may not be parameterized. Denote by ∇ A ( B ) L the entries of the gradient vector that correspond to the parameters of { A ℓ } ( { B ℓ } ) .

In the special case where x in a unit norm N -dimensional vector, a simple choice of | ψ ( x ) ⟩ is the amplitude encoding of x , given by

However, despite its exponential compactness in representing the data, a naive implementation of the simplest choice is restricted to representing quadratic features of the data that can offer no substantial quantum advantage in a learning task [55], so the choice of data encoding is critical to the power of a model. The interesting parameter regime for classical data and models is one where N,P are large, while L is relatively modest. For general unitaries P = O ( N 2 ) , which matches the scaling of the number of parameters in fully-connected networks. When the input tensor x is a batch of datapoints, N is equivalent to the product of batch size and input dimension.

The model in Definition 3.1 can be used to define distributed inference and learning problems by dividing the input x and the parameterized unitaries between two players, Alice and Bob. We define their respective inputs as follows:

The problems of interest require that Alice and Bob compute certain joint functions of their inputs. As a trivial base case, it is clear that in a communication cost model, all problems can be solved with communication cost at most the size of the inputs times the number of parties, by a protocol in which each party sends its inputs to all others. We will be interested in cases where one can do much better by taking advantage of quantum communication.

|

ψ

(

x

)

⟩

.

B

1

(

θ

B

1

)

A

1

(

θ

A

1

)

B

2

(

θ

B

2

)

A

2

(

θ

A

2

)

Given the inputs eq. (3.5), we will be interested chiefly in the two problems specified below.

Problem 1 (Distributed Inference) . Alice and Bob each compute an estimate of ⟨ φ | P 0 | φ ⟩ up to additive error ε .

The straightforward algorithm for this problem, illustrated in fig. 2, requires L rounds of communication. The other problem we consider is the following:

Problem 2 (Distributed Gradient Estimation) . Alice computes an estimate of ∇ A ⟨ φ | P 0 | φ ⟩ , while Bob computes an estimate of ∇ B ⟨ φ | Z 0 | φ ⟩ , up to additive error ε in L ∞ .

3.1 Communication complexity of inference and gradient estimation

We show that inference and gradient estimation are achievable with a logarithmic amount of quantum communication, which will represent an exponential improvement over the classical cost for some cases:

Lemma 1. Problem 1 can be solved by communicating O (log N ) qubits over O ( L/ε 2 ) rounds.

Lemma 2. Problem 2 can be solved with probability greater than 1 -δ by communicating ˜ O (log N (log P ) 2 log( L/δ ) /ε 4 ) qubits over O ( L 2 ) rounds. The time and space complexity of the algorithm is √ P L poly( N, log P, ε -1 , log(1 /δ )) .

This upper bound is obtained by simply noting that the problem of gradient estimation at every layer can be reduced to a shadow tomography problem [7]:

Theorem 1 (Shadow Tomography [3] solved with Threshold Search [13]) . For an unknown state | ψ ⟩ of log N qubits, given K known two-outcome measurements E i , there is an explicit algorithm that takes | ψ ⟩ ⊗ k as input, where k = ˜ O (log 2 K log N log(1 /δ ) /ε 4 ) , and produces estimates of ⟨ ψ | E i | ψ ⟩ for all i up to additive error ε with probability greater than 1 -δ . ˜ O hides subdominant polylog factors.

Using reductions from known problems in communication complexity, we can show that the amount of classical communication required to solve this problem is polynomial in the size of the input, and additionally give a lower bound on the number of rounds of communication required by any quantum or classical algorithm:

Lemma 3. i) The classical randomized communication complexity of Problem 1 and Problem 2 with ε < 1 / 2 is Ω(max( √ N,L )) . 5

ii) Any algorithm (quantum or classical) for Problem 1 or Problem 2 requires either Ω( L ) rounds of communication or Ω( N/L 4 ) qubits (or bits) of communication.

Proof: Appendix B

The implication of the second result in Lemma 3 is that Ω( L ) rounds of communication are necessary in order to obtain an exponential communication advantage for small L , since otherwise the number of qubits of communication required can scale linearly with N .

The combination of Lemma 1, Lemma 2 and Lemma 3 immediately implies exponential savings in communication for gradient estimation and inference:

Theorem 2. If L = O (polylog( N )) , P = O (poly( N )) and sufficiently large N , solving Problem 1 or Problem 2 with nontrivial success probability requires Ω( √ N ) bits of classical communication, while O (polylog( N, 1 /δ )poly(1 /ε )) qubits of communication suffice to solve these problems with probability at least 1 -δ .

The regime where L = O (polylog( N )) is relevant for many classes of machine learning models. The required overhead in terms of time and space is only polynomial when compared to the straightforward classical algorithms for these problems.

The distribution of the model as in eq. (3.5) is an example of pipelining. Data parallelism is another common approach to distributed machine learning in which subsets of the data are distributed to identical copies of the model. In Appendix C we show that it can also be implemented using quantum circuits, which can then trained using gradient descent requiring quantum communication that is logarithmic in the number of parameters and input size.

Quantum advantage is possible in these problems because there is a bound on the complexity of the final output, whether it be correlated elements of the gradient up to some finite error or the low-dimensional output of a model. This might lead one to believe that whenever the output takes such a form, encoding the data in the amplitudes of a quantum state will trivially give an exponential advantage in communication complexity. We show however that the situation is slightly more nuanced, by considering the problem of inference with a linear model:

Lemma 4. For the problem of distributed linear classification, there can be no exponential advantage in using quantum communication in place of classical communication.

The precise statement and proof of this result are presented in Appendix E. This result also highlights that the worst case lower bounds such as Lemma 3 may not hold for circuits with certain lowdimensional or other simplifying structure.

4 Graph neural networks

The communication advantages in the previous section apply to relatively unstructured data and quantum circuits (essentially the only structure in the problem is related to the promise of the vector-in-subspace problem [101]), and it is a priori unclear how relevant they are to circuits that approximate useful neural networks, or act on structured data. Here we consider a class of shallow graph neural networks that achieve good performance on node classification tasks on large graphs [40]. We prove that an exponential quantum communication advantage still holds for this class of models.

Consider a graph with N nodes. Define a local message passing operator on the graph A which may be the normalized Laplacian or some other operator. Given some N × D 0 matrix of graph features X , we consider models of the following form:

where W 1 ∈ R D 0 × D 1 , W 2 ∈ R D 1 × D 2 are parameter matrices, σ is a non-linearity and P is a sum pooling operator acting on the first index of its input, which can be represented as multiplication by an N/t × N matrix ˜ P . Since we would like a scalar output and a nonlinearity that can be implemented on a quantum computer, we instead compute

and W 2 ∈ R D 1 × N/t where t is the size of the pooling window.

This allows us to define the following problem:

Problem 3 (Quadratic graph network inference ( QGNI N,t )) . Alice is given X , Bob is given A,W 1 , W 2 . Only Alice is allowed to send messages. Their goal is to estimate φ ( X/ || X || F ) to additive error ε .

This models a scenario where only Bob has access to the connectivity of the graph, while Alice has access to the graph features. The normalization ensures that the choice of X does not introduce a dependence of the final output on N .

In the following, we denote by R → ε and Q → ε the classical (public key randomized) communication complexity and quantum communication complexity respectively. We show:

Proof: Appendix B

Proof: Appendix B

If this upper bound was a polynomial function of N , it would imply that an exponential communication advantage is impossible. For the parameter choices that realize classical communication lower bound, this is not the case, implying the following:

Lemma 7. An exponential quantum advantage in communication holds for solving the inference problem QGNI N,t up to error ε ≤ 1 4 ( t + 1 2 )( t + 3 2 ) , for any t such that t = polylog( N ) .

with

Proof: Appendix B. Note that this exponential advantage does not hold only for a single setting of the model weights, but rather for the entire family of models that can be used to solve f -BHP N,t for functions f that satisfy eq. (B.24).

Note that generically, one would not expect the numerator in the upper bound of Lemma 6 to scale polynomially with N . If A is for example a normalized graph Laplacian then || A || ≤ 2 . If we use a standard initialization scheme for the weights (say Gaussians with variance 1 / ( n in + n out ) ), the upper bound scales like O (( | a | + | b | ) log( ND 0 )poly( t, D 0 , D 1 ) /ε ) in expectation. Note that if the model output decays polynomially with N , the upper bound will not be useful since one would need to choose ε to be inverse polynomial in N . This could happen for example in a classification task considered in Section 5.2.1 when the classes are exactly balanced, or when the network is untrained and not sensitive to the structure in the data. While it is difficult to argue analytically about the scaling out the network output or the norms of the weight matrices after training due to the nonlinearity of the dynamics, we empirically compute these and find that they remain controlled for the datasets we study (see Appendix J.3).

In Section 5, we show that models of the form studied here achieve good performance on standard benchmarks, commensurate with state of the art models. Of particular relevance are the graph classification problems considered in Section 5.2.1, where the output takes the form eq. (4.2).

5 Experimental results

5.1 Model

We evaluate our model 6 (as defined in Equation (4.1)) on several graph tasks using common benchmarks and the DGL library [113]. We use the SIGN model proposed by [40] as a baseline. The SIGN model can be seen as an instance of our model where the message passing operator A represents a column stack of R hops, the original features of X are duplicated R times and W 1 is a block diagonal matrix. In Section 5.2 we simply replace the PReLU activation function with a second-degree polynomial with trainable coefficients [80] and compare the models on three node classification tasks. In Section 5.3, we implement a more general form of SIGN by relaxing W 1 to be a dense matrix and evaluate our model over several graph-classification datasets.

5.2 Node classification

We evaluate our model on three public node classification datasets: OGBN-Products [54], Reddit [47], and Cora [78]. For both the baseline and polynomial model, we use SIGN with 5 hops of the neighbor averaging operator. We train on each dataset for 1000 epochs using Adam optimizer and report the test accuracy averaged on 10 runs (full details in Appendix J). Our results in Table 3 show that replacing the PReLU activation function with a second-degree polynomial causes a reduction of less than 1% on all of the tested datasets.

5.2.1 Decision problems

We reduce the node classification task into a binary graph classification task by proposing the following decision problem: for a pair of classes ( c 1 , c 2 ) , return 1 if c 1 has more nodes; otherwise, return 0. We solve this task for each pair of classes by summing the node classification model output across all nodes and choosing the class with the higher score. We use the node classification training,

choose the model with the highest validation accuracy on the graph classification task, and report its accuracy on the test sets. The model output in this form is given by eq. (4.1).

5.3 Graph classification

We evaluate our model on several graph classification benchmarks: bioinformatics datasets (MUTAG, PTC, NCI1, PROTEINS) [105, 50, 35, 21] and social networks (COLLAB, IMDB-BINARY, IMDBMULTI, REDDIT-BINARY, REDDIT-MULTI) [116]. For the bioinformatics datasets, we use the standard categorical node features. As proposed in [114], we use one-hot encoding of the node degree as node features for the COLLAB and IMDB datasets, and for REDDIT datasets all nodes are set with an identical scalar feature of 1. We convert the polynomial SIGN model in Section 5.2 into a graph classification model by inserting a SumPool operator as described in Equation (4.1). We use the sign diffusion operator [113] and stack R i instances of each of its four message passing operators, where { R i } 4 i =1 are selected during a hyperparameter tuning, as well as the hidden dimension size and optimization setting (see Appendix J for more details). We follow the validation regime proposed by [114]; we perform 10-fold cross-validation, train each fold for 350 epochs using Adam optimizer, and report in Table 2 the maximal value and standard-deviation of the averaged validation accuracy curve. For all datasets, except for REDDIT, our model achieves comparable to or better than other commonly used models, despite most of them using multiple layers. While the results show that on most datasets our shallow architecture suffices given sufficient width in the message passing and hidden layer, we hypothesize that datasets without any node features (such as REDDIT) require at least two layers of message passing.

6 Discussion

This work constitutes a preliminary investigation into a generic class of quantum circuits that has the potential for enabling an exponential communication advantage in problems of classical data processing including training and inference with large parameterized models over large datasets, with inherent privacy advantages. Communication constraints may become even more relevant if such models are trained on data that is obtained by inherently distributed interaction with the physical world [37]. The ability to compute using data with privacy guarantees can be potentially applied to proprietary data. This could become highly desirable even in the near future as the rate of publicly-available data production appears to be outstripped by the growth rate of training sets of large language models [110].

A limitation of the current results is that it's unclear to what extent powerful neural networks can be approximated using quantum circuits, even though we provide positive evidence in the form of the results on graph networks in Section 4. Additionally, the advantages we study require deep ( poly( N ) ), fault-tolerant quantum circuits. While this is a common feature of problems for which quantum communication advantages hold, the overhead of quantum error-correction in such circuits may be considerable. Detailed resource estimates would be necessary to understand better the practicality of this approach for achieving useful quantum advantage. Our results naturally raise further questions regarding the expressive power and trainability of these types of circuits, which may be of independent interest. We collect some of these in Appendix I.

References

Scott Aaronson. Read the fine print. Nat. Phys. , 11(4):291-293, 2015.

Scott Aaronson. Introduction to quantum information science. https://www. scottaaronson.com/qclec.pdf , 2017.

Scott Aaronson. Shadow tomography of quantum states. In Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing , pages 325-338, New York, NY, USA, 2018. ACM.

Scott Aaronson, Andris Ambainis, J¯ anis Iraids, Martins Kokainis, and Juris Smotrovs. Polynomials, quantum query complexity, and grothendieck's inequality. In Proceedings of the 31st Conference on Computational Complexity , number Article 25 in CCC '16, pages 1-19, Dagstuhl, DEU, 2016. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik.

Scott Aaronson, Xinyi Chen, Elad Hazan, Satyen Kale, and Ashwin Nayak. Online learning of quantum states. J. Stat. Mech. , 2019(12):124019, 2019.

Scott Aaronson and Guy N Rothblum. Gentle measurement of quantum states and differential privacy. In STOC'19-Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing , pages 322-333. ACM, New York, 2019.

Amira Abbas, Robbie King, Hsin-Yuan Huang, W Huggins, R Movassagh, D Gilboa, and J McClean. On quantum backpropagation, information reuse, and cheating measurement collapse. Neural Inf Process Syst , abs/2305.13362:44792-44819, 2023.

Dimitris Achlioptas. Database-friendly random projections: Johnson-lindenstrauss with binary coins. J. Comput. System Sci. , 66(4):671-687, 2003.

Alekh Agarwal, Peter L Bartlett, Pradeep Ravikumar, and Martin J Wainwright. Informationtheoretic lower bounds on the oracle complexity of stochastic convex optimization. arXiv [stat.ML] , 2010.

Srinivasan Arunachalam, Uma Girish, and Noam Lifshitz. One clean qubit suffices for quantum communication advantage. arXiv [quant-ph] , 2023.

Frank Arute, Kunal Arya, Ryan Babbush, Dave Bacon, Joseph C Bardin, Rami Barends, Rupak Biswas, Sergio Boixo, Fernando G S L Brandao, David A Buell, Brian Burkett, Yu Chen, Zijun Chen, Ben Chiaro, Roberto Collins, William Courtney, Andrew Dunsworth, Edward Farhi, Brooks Foxen, Austin Fowler, Craig Gidney, Marissa Giustina, Rob Graff, Keith Guerin, Steve Habegger, Matthew P Harrigan, Michael J Hartmann, Alan Ho, Markus Hoffmann, Trent Huang, Travis S Humble, Sergei V Isakov, Evan Jeffrey, Zhang Jiang, Dvir Kafri, Kostyantyn Kechedzhi, Julian Kelly, Paul V Klimov, Sergey Knysh, Alexander Korotkov, Fedor Kostritsa, David Landhuis, Mike Lindmark, Erik Lucero, Dmitry Lyakh, Salvatore Mandrà, Jarrod R McClean, Matthew McEwen, Anthony Megrant, Xiao Mi, Kristel Michielsen, Masoud Mohseni, Josh Mutus, Ofer Naaman, Matthew Neeley, Charles Neill, Murphy Yuezhen Niu, Eric Ostby, Andre Petukhov, John C Platt, Chris Quintana, Eleanor G Rieffel, Pedram Roushan, Nicholas C Rubin, Daniel Sank, Kevin J Satzinger, Vadim Smelyanskiy, Kevin J Sung, Matthew D Trevithick, Amit Vainsencher, Benjamin Villalonga, Theodore White, Z Jamie Yao, Ping Yeh, Adam Zalcman, Hartmut Neven, and John M Martinis. Quantum supremacy using a programmable superconducting processor. Nature , 574(7779):505-510, 2019.

Koji Azuma, Sophia E Economou, David Elkouss, Paul Hilaire, Liang Jiang, Hoi-Kwong Lo, and Ilan Tzitrin. Quantum repeaters: From quantum networks to the quantum internet. arXiv [quant-ph] , 2022.

Costin B˘ adescu and Ryan O'Donnell. Improved quantum data analysis. In Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing , pages 1398-1411, 2021.

Krishna C Balram and Kartik Srinivasan. Piezoelectric optomechanical approaches for efficient quantum microwave-to-optical signal transduction: the need for co-design. arXiv [physics.optics] , 2021.

Ziv Bar-Yossef, T S Jayram, and Iordanis Kerenidis. Exponential separation of quantum and classical one-way communication complexity. SIAM J. Comput. , 38(1):366-384, 2008.

Paul Barham, Aakanksha Chowdhery, Jeff Dean, Sanjay Ghemawat, Steven Hand, Dan Hurt, Michael Isard, Hyeontaek Lim, Ruoming Pang, Sudip Roy, Brennan Saeta, Parker Schuh, Ryan Sepassi, Laurent El Shafey, Chandramohan A Thekkath, and Yonghui Wu. Pathways: Asynchronous distributed dataflow for ML. arXiv [cs.DC] , 2022.

Luiz André Barroso, Jimmy Clidaras, and Urs Hölzle. The datacenter as a computer: An introduction to the design of warehouse-scale machines, second edition. Synth. Lect. Comput. Archit. , 8(3):1-154, 2013.

C H Bennett, G Brassard, C Crépeau, R Jozsa, A Peres, and W K Wootters. Teleporting an unknown quantum state via dual classical and einstein-podolsky-rosen channels. Phys. Rev. Lett. , 70(13):1895-1899, 1993.

Charles H Bennett, Gilles Brassard, Sandu Popescu, Benjamin Schumacher, John A Smolin, and William K Wootters. Purification of noisy entanglement and faithful teleportation via noisy channels. arXiv [quant-ph] , 1995.

Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri Chatterji, Annie Chen, Kathleen Creel, Jared Quincy Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren Gillespie, Karan Goel, Noah Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, Omar Khattab, Pang Wei Koh, Mark Krass, Ranjay Krishna, Rohith Kuditipudi, Ananya Kumar, Faisal Ladhak, Mina Lee, Tony Lee, Jure Leskovec, Isabelle Levent, Xiang Lisa Li, Xuechen Li, Tengyu Ma, Ali Malik, Christopher D Manning, Suvir Mirchandani, Eric Mitchell, Zanele Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayanan, Ben Newman, Allen Nie, Juan Carlos Niebles, Hamed Nilforoshan, Julian Nyarko, Giray Ogut, Laurel Orr, Isabel Papadimitriou, Joon Sung Park, Chris Piech, Eva Portelance, Christopher Potts, Aditi Raghunathan, Rob Reich, Hongyu Ren, Frieda Rong, Yusuf Roohani, Camilo Ruiz, Jack Ryan, Christopher Ré, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam, Andy Shih, Krishnan Srinivasan, Alex Tamkin, Rohan Taori, Armin W Thomas, Florian Tramèr, Rose E Wang, William Wang, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michihiro Yasunaga, Jiaxuan You, Matei Zaharia, Michael Zhang, Tianyi Zhang, Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kaitlyn Zhou, and Percy Liang. On the opportunities and risks of foundation models. arXiv [cs.LG] , 2021.

Karsten M Borgwardt, Cheng Soon Ong, Stefan Schönauer, SVN Vishwanathan, Alex J Smola, and Hans-Peter Kriegel. Protein function prediction via graph kernels. Bioinformatics , 21(suppl_1):i47-i56, 2005.

Fernando G S Brandão, Amir Kalev, Tongyang Li, Cedric Yen-Yu Lin, Krysta M Svore, and Xiaodi Wu. Quantum SDP solvers: Large speed-ups, optimality, and applications to quantum learning. arXiv [quant-ph] , 2017.

Gilles Brassard. Quantum communication complexity (a survey). arXiv [quant-ph] , 2001.

Adam R Brown and Leonard Susskind. The second law of quantum complexity. arXiv [hep-th] , 2017.

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. Advances in Neural Information Processing Systems , 33:1877-1901, 2020.

Sébastien Bubeck. Convex optimization: Algorithms and complexity. arXiv [math.OC] , 2014.

Harry Buhrman, Richard Cleve, Serge Massar, and Ronald de Wolf. Non-locality and communication complexity. arXiv [quant-ph] , 2009.

MCerezo, Andrew Arrasmith, Ryan Babbush, Simon C Benjamin, Suguru Endo, Keisuke Fujii, Jarrod R McClean, Kosuke Mitarai, Xiao Yuan, Lukasz Cincio, and Patrick J Coles. Variational quantum algorithms. arXiv [quant-ph] , 2020.

Shantanav Chakraborty, András Gilyén, and Stacey Jeffery. The power of block-encoded matrix powers: improved regression techniques via faster hamiltonian simulation. arXiv [quant-ph] , 2018.

A Chi-Chih Yao. Quantum circuit complexity. In Proceedings of 1993 IEEE 34th Annual Foundations of Computer Science , pages 352-361, 1993.

Andrew M Childs and Wim van Dam. Quantum algorithms for algebraic problems. arXiv [quant-ph] , 2008.

Nadav Cohen, Or Sharir, and Amnon Shashua. On the expressive power of deep learning: A tensor analysis. arXiv [cs.NE] , 2015.

Nadav Cohen and Amnon Shashua. Inductive bias of deep convolutional networks through pooling geometry. arXiv [cs.NE] , 2016.

Gavin E Crooks. Gradients of parameterized quantum gates using the parameter-shift rule and gate decomposition. arXiv [quant-ph] , 2019.

Asim Kumar Debnath, Rosa L Lopez de Compadre, Gargi Debnath, Alan J Shusterman, and Corwin Hansch. Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. correlation with molecular orbital energies and hydrophobicity. Journal of medicinal chemistry , 34(2):786-797, 1991.

João F Doriguello and Ashley Montanaro. Exponential quantum communication reductions from generalizations of the boolean hidden matching problem. arXiv [quant-ph] , 2020.

Danny Driess, Fei Xia, Mehdi S M Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, Andy Zeng, Igor Mordatch, and Pete Florence. PaLME: An embodied multimodal language model. arXiv [cs.LG] , 2023.

Edward Farhi and Hartmut Neven. Classification with quantum neural networks on near term processors. arXiv [quant-ph] , 2018.

Richard P Feynman. Simulating physics with computers. Int. J. Theor. Phys. , 21(6):467-488, 1982.

Fabrizio Frasca, Emanuele Rossi, Davide Eynard, Ben Chamberlain, Michael Bronstein, and Federico Monti. SIGN: Scalable inception graph neural networks. arXiv [cs.LG] , 2020.

András Gilyén, Yuan Su, Guang Hao Low, and Nathan Wiebe. Quantum singular value transformation and beyond: exponential improvements for quantum matrix arithmetics. arXiv [quant-ph] , 2018.

Vittorio Giovannetti, Seth Lloyd, and Lorenzo Maccone. Quantum random access memory. Phys. Rev. Lett. , 100(16):160501, 2008.

Lukas Gonon and Antoine Jacquier. Universal approximation theorem and error bounds for quantum neural networks and quantum reservoirs. arXiv [quant-ph] , 2023.

Google Quantum AI. Suppressing quantum errors by scaling a surface code logical qubit. Nature , 614(7949):676-681, 2023.

Goren Gordon and Gustavo Rigolin. Generalized teleportation protocol. arXiv [quant-ph] , 2005.

Robert Gower, Donald Goldfarb, and Peter Richtarik. Stochastic block BFGS: Squeezing more curvature out of data. In Maria Florina Balcan and Kilian Q Weinberger, editors, Proceedings of The 33rd International Conference on Machine Learning , volume 48 of Proceedings of Machine Learning Research , pages 1869-1878, New York, New York, USA, 2016. PMLR.

William L. Hamilton, Rex Ying, and Jure Leskovec. Inductive representation learning on large graphs, 2018.

Aram W Harrow, Avinatan Hassidim, and Seth Lloyd. Quantum algorithm for linear systems of equations. Phys. Rev. Lett. , 103(15):150502, 2009.

Aram W Harrow and John C Napp. Low-depth gradient measurements can improve convergence in variational hybrid quantum-classical algorithms. Phys. Rev. Lett. , 126(14):140502, 2021.

Christoph Helma, Ross D. King, Stefan Kramer, and Ashwin Srinivasan. The predictive toxicology challenge 2000-2001. Bioinformatics , 17(1):107-108, 2001.

Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W Rae, Oriol Vinyals, and Laurent Sifre. Training compute-optimal large language models. arXiv [cs.CL] , 2022.

Alexander Semenovich Holevo. Bounds for the quantity of information transmitted by a quantum communication channel. Problemy Peredachi Informatsii , 1973.

Jeremy Howard and Sebastian Ruder. Universal language model fine-tuning for text classification. arXiv [cs.CL] , 2018.

Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs, 2021.

Hsin-Yuan Huang, Michael Broughton, Masoud Mohseni, Ryan Babbush, Sergio Boixo, Hartmut Neven, and Jarrod R McClean. Power of data in quantum machine learning. Nat. Commun. , 12(1):2631, 2021.

Yanping Huang, Youlong Cheng, Ankur Bapna, Orhan Firat, Mia Xu Chen, Dehao Chen, Hyoukjoong Lee, Jiquan Ngiam, Quoc V Le, Yonghui Wu, and Zhifeng Chen. GPipe: Efficient training of giant neural networks using pipeline parallelism. arXiv [cs.CV] , 2018.

William J Huggins and Jarrod R McClean. Accelerating quantum algorithms with precomputation. arXiv [quant-ph] , 2023.

Rahul Jain, Jaikumar Radhakrishnan, and Pranab Sen. The quantum communication complexity of the pointer chasing problem: The bit version. In FST TCS 2002: Foundations of Software Technology and Theoretical Computer Science , Lecture notes in computer science, pages 218-229. Springer Berlin Heidelberg, Berlin, Heidelberg, 2002.

Zhengfeng Ji, Yi-Kai Liu, and Fang Song. Pseudorandom quantum states. In Lecture Notes in Computer Science , Lecture notes in computer science, pages 126-152. Springer International Publishing, Cham, 2018.

Rie Johnson and Tong Zhang. Accelerating stochastic gradient descent using predictive variance reduction. In NeurIPS , 2013.

Norman P Jouppi, George Kurian, Sheng Li, Peter Ma, Rahul Nagarajan, Lifeng Nai, Nishant Patil, Suvinay Subramanian, Andy Swing, Brian Towles, Cliff Young, Xiang Zhou, Zongwei Zhou, and David Patterson. TPU v4: An optically reconfigurable supercomputer for machine learning with hardware support for embeddings. arXiv [cs.AR] , 2023.

J Kaplan, S McCandlish, T Henighan, T B Brown, and others. Scaling laws for neural language models. arXiv preprint arXiv , 2020.

Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and François Fleuret. Transformers are RNNs: Fast autoregressive transformers with linear attention. arXiv [cs.LG] , 2020.

V Krutyanskiy, M Galli, V Krcmarsky, S Baier, D A Fioretto, Y Pu, A Mazloom, P Sekatski, M Canteri, M Teller, J Schupp, J Bate, M Meraner, N Sangouard, B P Lanyon, and T E Northup. Entanglement of trapped-ion qubits separated by 230 meters. arXiv [quant-ph] , 2022.

Eyal Kushilevitz and Noam Nisan. Communication Complexity . Cambridge University Press, Cambridge, England, 2011.

Nikolai Lauk, Neil Sinclair, Shabir Barzanjeh, Jacob P Covey, Mark Saffman, Maria Spiropulu, and Christoph Simon. Perspectives on quantum transduction. Quantum Sci. Technol. , 5(2):020501, 2020.

Nicolas Le Roux, Mark Schmidt, and Francis Bach. A stochastic gradient method with an exponential convergence rate for finite training sets. arXiv [math.OC] , 2012.

James Lee-Thorp, Joshua Ainslie, Ilya Eckstein, and Santiago Ontanon. FNet: Mixing tokens with fourier transforms. arXiv [cs.CL] , 2021.

Yoav Levine, Or Sharir, Alon Ziv, and Amnon Shashua. On the long-term memory of deep recurrent networks. arXiv [cs.LG] , 2017.

Yoav Levine, Noam Wies, Or Sharir, Hofit Bata, and Amnon Shashua. The depth-to-width interplay in self-attention. arXiv [cs.LG] , 2020.

Bo Li, Yuan Cao, Yu-Huai Li, Wen-Qi Cai, Wei-Yue Liu, Ji-Gang Ren, Sheng-Kai Liao, Hui-Nan Wu, Shuang-Lin Li, Li Li, Nai-Le Liu, Chao-Yang Lu, Juan Yin, Yu-Ao Chen, Cheng-Zhi Peng, and Jian-Wei Pan. Quantum state transfer over 1200 km assisted by prior distributed entanglement. Phys. Rev. Lett. , 128(17):170501, 2022.

S Lloyd. Universal quantum simulators. Science , 273(5278):1073-1078, 1996.

Seth Lloyd, Masoud Mohseni, and Patrick Rebentrost. Quantum principal component analysis. Nat. Phys. , 10(9):631-633, 2014.

Guang Hao Low and Isaac L Chuang. Optimal hamiltonian simulation by quantum signal processing. Phys. Rev. Lett. , 118(1):010501, 2017.

P Magnard, S Storz, P Kurpiers, J Schär, F Marxer, J Lütolf, T Walter, J-C Besse, M Gabureac, K Reuer, A Akin, B Royer, A Blais, and A Wallraff. Microwave quantum link between superconducting circuits housed in spatially separated cryogenic systems. Phys. Rev. Lett. , 125(26):260502, 2020.

Vitaly Maiorov and Allan Pinkus. Lower bounds for approximation by MLP neural networks. Neurocomputing , 25(1):81-91, 1999.

John M Martyn, Zane M Rossi, Andrew K Tan, and Isaac L Chuang. A grand unification of quantum algorithms. arXiv [quant-ph] , 2021.

Andrew McCallum, Kamal Nigam, Jason Rennie, and Kristie Seymore. Automating the construction of internet portals with machine learning. Inf. Retr. , 3(2):127-163, 2000.

Jarrod R McClean, Jonathan Romero, Ryan Babbush, and Alán Aspuru-Guzik. The theory of variational hybrid quantum-classical algorithms. arXiv [quant-ph] , 2015.

Hagay Michaeli, Tomer Michaeli, and Daniel Soudry. Alias-free convnets: Fractional shift invariance via polynomial activations. arXiv [cs.CV] , 2023.

Boris Mityagin. The zero set of a real analytic function. arXiv [math.CA] , 2015.

Ashley Montanaro and Sam Pallister. Quantum algorithms and the finite element method. arXiv [quant-ph] , 2015.

Ashley Montanaro and Changpeng Shao. Quantum communication complexity of linear regression. arXiv preprint arXiv:2210.01601 , 2022.

Philipp Moritz, Robert Nishihara, and Michael Jordan. A linearly-convergent stochastic L-BFGS algorithm. In Arthur Gretton and Christian C Robert, editors, Proceedings of the 19th International Conference on Artificial Intelligence and Statistics , volume 51 of Proceedings of Machine Learning Research , pages 249-258, Cadiz, Spain, 2016. PMLR.

Danial Motlagh and Nathan Wiebe. Generalized quantum signal processing. arXiv [quant-ph] , 2023.

William J Munro, Koji Azuma, Kiyoshi Tamaki, and Kae Nemoto. Inside quantum repeaters. IEEE J. Sel. Top. Quantum Electron. , 21(3):78-90, 2015.

Deepak Narayanan, Aaron Harlap, Amar Phanishayee, Vivek Seshadri, Nikhil R Devanur, Gregory R Ganger, Phillip B Gibbons, and Matei Zaharia. PipeDream: generalized pipeline parallelism for DNN training. In Proceedings of the 27th ACM Symposium on Operating Systems Principles , SOSP '19, pages 1-15, New York, NY, USA, 2019. Association for Computing Machinery.

Ashwin Nayak and Felix Wu. The quantum query complexity of approximating the median and related statistics. arXiv [quant-ph] , 1998.

Dai Quoc Nguyen, Tu Dinh Nguyen, and Dinh Phung. Universal graph transformer selfattention networks, 2022.

Michael A Nielsen and Isaac L Chuang. Quantum Computation and Quantum Information: 10th Anniversary Edition . Cambridge University Press, 2010.

Brad G Osgood. Lectures on the Fourier Transform and Its Applications (Pure and Applied Undergraduate Texts) (Pure and Applied Undergraduate Texts, 33) . American Mathematical Society, 2019.

Pál András Papp, Karolis Martinkus, Lukas Faber, and Roger Wattenhofer. Dropgnn: Random dropouts increase the expressiveness of graph neural networks, 2021.

Adrián Pérez-Salinas, Alba Cervera-Lierta, Elies Gil-Fuster, and José I Latorre. Data reuploading for a universal quantum classifier. arXiv [quant-ph] , 2019.

Lirandë Pira and Chris Ferrie. An invitation to distributed quantum neural networks. Quantum Machine Intelligence , 5(2):1-24, 2023.

MPompili, S L N Hermans, S Baier, H K C Beukers, P C Humphreys, R N Schouten, R F L Vermeulen, M J Tiggelman, L Dos Santos Martins, B Dirkse, S Wehner, and R Hanson. Realization of a multinode quantum network of remote solid-state qubits. Science , 372(6539):259-264, 2021.

Stephen J Ponzio, Jaikumar Radhakrishnan, and S Venkatesh. The communication complexity of pointer chasing. J. Comput. System Sci. , 62(2):323-355, 2001.

Reiner Pope, Sholto Douglas, Aakanksha Chowdhery, Jacob Devlin, James Bradbury, Anselm Levskaya, Jonathan Heek, Kefan Xiao, Shivani Agrawal, and Jeff Dean. Efficiently scaling transformer inference. arXiv [cs.LG] , 2022.

Anup Rao and Amir Yehudayoff. Communication Complexity and Applications . Cambridge University Press, 2020.

Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. DeepSpeed: System optimizations enable training deep learning models with over 100 billion parameters. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , KDD '20, pages 3505-3506, New York, NY, USA, 2020. Association for Computing Machinery.

Arthur G Rattew and Patrick Rebentrost. Non-linear transformations of quantum amplitudes: Exponential improvement, generalization, and applications. arXiv [quant-ph] , 2023.

Ran Raz. Exponential separation of quantum and classical communication complexity. In Proceedings of the thirty-first annual ACM symposium on Theory of Computing , STOC '99, pages 358-367, New York, NY, USA, 1999. Association for Computing Machinery.

Alexander Razborov. Quantum communication complexity of symmetric predicates. arXiv [quant-ph] , 2002.

Tim Roughgarden. Communication complexity (for algorithm designers). arXiv [cs.CC] , 2015.

Maria Schuld, Ryan Sweke, and Johannes Jakob Meyer. The effect of data encoding on the expressive power of variational quantum machine learning models. arXiv [quant-ph] , 2020.

Nino Shervashidze, Pascal Schweitzer, Erik Jan van Leeuwen, Kurt Mehlhorn, and Karsten M. Borgwardt. Weisfeiler-lehman graph kernels. J. Mach. Learn. Res. , 12:2539-2561, 2011.

P W Shor. Algorithms for quantum computation: discrete logarithms and factoring. In Proceedings 35th Annual Symposium on Foundations of Computer Science . IEEE Comput. Soc. Press, 1994.

Yutao Sun, Li Dong, Shaohan Huang, Shuming Ma, Yuqing Xia, Jilong Xue, Jianyong Wang, and Furu Wei. Retentive network: A successor to transformer for large language models. arXiv [cs.CL] , 2023.

Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, and Others. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312. 11805 , 2023.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, L Ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I Guyon, U V Luxburg, S Bengio, H Wallach, R Fergus, S Vishwanathan, and R Garnett, editors, Advances in Neural Information Processing Systems , volume 30, pages 5998-6008. Curran Associates, Inc., 2017.

Pablo Villalobos, Jaime Sevilla, Lennart Heim, Tamay Besiroglu, Marius Hobbhahn, and Anson Ho. Will we run out of data? an analysis of the limits of scaling datasets in machine learning. arXiv [cs.LG] , 2022.

A J Walker. New fast method for generating discrete random numbers with arbitrary frequency distributions. Electron. Lett. , 10(8):127-128, 1974.

Changqing Wang, Ivan Gonin, Anna Grassellino, Sergey Kazakov, Alexander Romanenko, Vyacheslav P Yakovlev, and Silvia Zorzetti. High-efficiency microwave-optical quantum transduction based on a cavity electro-optic superconducting system with long coherence time. npj Quantum Information , 8(1):1-10, 2022.

Minjie Wang, Da Zheng, Zihao Ye, Quan Gan, Mufei Li, Xiang Song, Jinjing Zhou, Chao Ma, Lingfan Yu, Yu Gai, Tianjun Xiao, Tong He, George Karypis, Jinyang Li, and Zheng Zhang. Deep graph library: A graph-centric, highly-performant package for graph neural networks. arXiv preprint arXiv:1909.01315 , 2019.

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks?, 2019.

Yuanzhong Xu, Hyoukjoong Lee, Dehao Chen, Blake Hechtman, Yanping Huang, Rahul Joshi, Maxim Krikun, Dmitry Lepikhin, Andy Ly, Marcello Maggioni, Ruoming Pang, Noam Shazeer, Shibo Wang, Tao Wang, Yonghui Wu, and Zhifeng Chen. GSPMD: General and scalable parallelization for ML computation graphs. arXiv [cs.DC] , 2021.

Pinar Yanardag and SVN Vishwanathan. Deep graph kernels. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining , pages 1365-1374, 2015.

Andrew Chi-Chih Yao. Some complexity questions related to distributive computing(preliminary report). In Proceedings of the eleventh annual ACM symposium on Theory of computing , STOC '79, pages 209-213, New York, NY, USA, 1979. Association for Computing Machinery.

Muhan Zhang, Zhicheng Cui, Marion Neumann, and Yixin Chen. An end-to-end deep learning architecture for graph classification. In Proceedings of the AAAI conference on artificial intelligence , volume 32, 2018.

Zhen Zhang, Jiajun Bu, Martin Ester, Jianfeng Zhang, Chengwei Yao, Zhi Yu, and Can Wang. Hierarchical graph pooling with structure learning, 2019.

Qi Zhao and Yusu Wang. Learning metrics for persistence-based summaries and applications for graph classification, 2019.

A Notation and a very brief review of quantum mechanics

We denote by { a i } a set of elements indexed by i , with 1 -based indexing unless otherwise specified, with the maximal value of i explicitly specified when it is not clear from context. [ N ] denotes the set { 0 , . . . , N -1 } . The complex conjugate of a number c is denoted by c ∗ , and the conjugate transpose of a complex-valued matrix A by A † .

We denote by | ψ ⟩ a vector of complex numbers { ψ i } representing the state of a quantum system when properly normalized, and by ⟨ ψ | its dual (assuming it exists). The inner product between two such vectors of length N is denoted by

Denoting by | i ⟩ for i ∈ [ N ] a basis vector in an orthonormal basis with respect to the above inner product, we can also write

Matrices will be denoted by capital letters, and when acting on quantum states will always be unitary. These can be specified in terms of their matrix elements using the Dirac notation defined above, as in

Matrix-vector product are specified naturally in this notation by

Quantum mechanics is, in the simplest possible terms, a theory of probability based on conservation of the L 2 norm rather than the standard probability theory based on the L 1 norm [2, 90]. The state of a pure quantum system is described fully by a complex vector of N numbers known as amplitudes which we denote by { ψ i } where i ∈ { 0 , . . . , N -1 } , and is written using Dirac notation as | ψ ⟩ . The state is normalized so that

which is the L 2 equivalent of the standard normalization condition of classical probability theory. It is a curious fact that the choice of L 2 requires the use of complex rather than real amplitudes, and that no consistent theory can be written in this way for any other L p norm [2]. The most general state of a quantum system is a probabilistic mixture of pure states, in the sense of the standard L 1 -based rules of probability. We will not be concerned with these types of states, and so omit their description here, and subsequently whenever quantum states are discussed, the assumption is that they are pure.

Since any closed quantum system conserves probability, the L 2 norm of a quantum state is conserved during the evolution of a quantum state. Consequently, when representing and manipulating quantum states on a quantum computer, the fundamental operation is the application of a unitary matrix to a quantum state.

Given a quantum system with some discrete degrees of freedom, the number of amplitudes corresponds to the number of possible states of the system, and is thus exponential in the number of degrees of freedom. The simplest such degree of freedom is a binary one, called a qubit, which is analogous to a bit. Thus a state of log N qubits is described by N complex amplitudes.

A fundamental property of quantum mechanics is that the amplitudes of a quantum state are not directly measurable. Given a Hermitian operator

with real eigenvalues { λ i } , a measurement of O with respect to a state | ψ ⟩ gives the result λ i with probability |⟨ v i | ψ ⟩| 2 . The real-valued quantity

is the expectation value of O with respect to | ψ ⟩ , and its value can be estimated by measurements. After a measurement with outcome λ i , the original state is destroyed, collapsing to the state | v i ⟩ . A consequence of the fundamentally destructive nature of quantum measurement is that simply encoding information in the amplitudes of a quantum state dues not necessarily render it useful for downstream computation. It also implies that operations using amplitude-encoded data such as evaluating a simple loss function incur measurement error, unlike their classical counterparts that are typically limited only by machine precision. The design of quantum algorithms essentially amounts to a careful and intricate design of amplitude manipulations and measurements in order to extract useful information from the amplitudes of a quantum state. For a more complete treatment of these topics see [90].

B Proofs

Proof of Lemma 1. ⟨ φ | P 0 | φ ⟩ can be estimated by preparing | φ ⟩ and measuring it O (1 /ε 2 ) times. Preparing each copy of | φ ⟩ requires O ( L ) rounds of communication, with each round involving the communication of a log N ′ -qubit quantum state. Alice first prepares | ψ ( x ) ⟩ , and this state is passed back and forth with each player applying A ℓ or B ℓ respectively for ℓ ∈ { 1 , . . . , L } .

Proof of Lemma 2. We consider the parameters of the unitaries that Alice possesses first, and an identical argument follows for the parameters of Bob's unitaries.

We have where

correspond to forward and backward features for the i -the parameter of A ℓ respectively. This is illustrated graphically in Figure 1. We also write

Attaching an ancilla qubit denoted by a to the feature states defined above, we define

and a Hermitian measurement operator

we then have where X a acts on the ancilla.

Note that ∣ ∣ ψ A ℓ 0 〉 ⊗ k can be prepared by Alice first preparing ( | + ⟩ | ψ ( x ) ⟩ ) ⊗ k and sending this state back and forth at most 2 L times, with each player applying the appropriate unitaries conditioned on the value of the ancilla. Additionally, for any choice of ℓ and any i , Alice has full knowledge of the E A ℓi . They can thus be applied to quantum states and classical hypothesis states without requiring any communication.

The gradient can then be estimated using shadow tomography (Theorem 1). Specifically, for each ℓ , Alice prepares ˜ O (log 2 P log N log( L/δ ) /ε 4 ) copies of ∣ ∣ ∣ ψ A ℓ 0 〉 , which requires O ( L ) rounds of communication, each of ˜ O (log 2 P log 2 N log( L/δ ) /ε 4 ) qubits. She then runs shadow tomography to estimate ∇ A ℓ ⟨ φ | Z 0 | φ ⟩ up to error ε with no additional communication. Bob does the same to estimate ∇ B ℓ ⟨ φ | Z 0 | φ ⟩ . In total O ( L 2 ) rounds are needed to estimate the full gradient. The success probability of all L applications of shadow tomography is at least 1 -δ by a union bound.

Based on the results of [22], the space and time complexity of each application of shadow tomography is √ P poly( N, log P, ε -1 , log(1 /δ )) . This is the query complexity of the algorithm to oracles that implement the measurement operators { E Q ℓi } . Instantiating these oracles will incur a cost of at most O ( N 2 ) . In cases where these operators have low rank the query complexity complexity will depend polynomially only on the rank instead of on N .

Proof of Lemma 3. We first prove an Ω( √ N ) lower bound on the amount of classical communication. Consider the following problem:

Problem 4 ([101]) . Alice is given a vector x ∈ S N -1 and two orthogonal linear subspaces of R N each of dimension N/ 2 , denoted M 1 , M 2 . Bob is given an orthogonal matrix O . Under the promise that either ∥ M 1 Ox ∥ 2 ≥ √ 1 -θ 2 or ∥ M 2 Ox ∥ 2 ≥ √ 1 -θ 2 for 0 < θ < 1 / √ 2 , Alice and Bob must determine which of the two cases holds.

Ref. [101] showed that the randomized 7 classical communication complexity of the problem is Ω( √ N ) .

The reduction from Problem 4 to Problem 1 is obtained by choosing θ = 1 / 2 and simply setting L = 1 , B 1 = O, | ψ ( x ) ⟩ = | x ⟩ , P 0 = Z 0 , and

where the first register contains a single qubit and { v k j } form an orthonormal basis of M k , and picking any ε < 1 / 2 . Note that this choice of | ψ ( x ) ⟩ implies N ′ = N . Estimating L to this accuracy now solves the desired problem since L = ⟨ x | O T (Π 1 -Π 2 ) O | x ⟩ where Π k is a projector onto M k , and hence estimating this quantity up to error 1 / 2 allows Alice and Bob to determine which subspace has large overlap with Ox .

The reduction from Problem 4 to Problem 2 is obtained by setting L = 2 , picking | ψ ( x ) ⟩ , A 1 , B 1 as before, and additionally B 2 = I, A 2 = e -iθ A 2 , 1 X 0 / 2 initialized at θ A 2 , 1 = -π/ 2 . By the parameter shift rule [34], we have that if U = e -iθ P / 2 for some Pauli matrix P , and U is part of the parameterized circuit that defines | φ ⟩ , then

It follows that

Estimating ∇ A ⟨ φ | Z 0 | φ ⟩ to accuracy ε < 2 allows one to determine the sign of L (0) , which as before gives the solution to Problem 4.

Next, we show that Ω( L ) rounds are necessary in both the quantum and classical setting by a reduction from the bit version of pointer-chasing, as studied in [58, 96].

Problem 5 (Pointer-chasing, bit version) . Alice receives a function f A : [ N ] → [ N ] and Bob receives a function f B : [ N ] → [ N ] . Alice is also given a starting point x ∈ [ N ] , and both receive an integer L 0 . Their goal is to compute the least significant bit of f ( L 0 ) ( x ) , where f (1) ( x ) = f B ( x ) , f (2) ( x ) = f A ( f B ( x )) , . . . .

Ref. [58] show that the quantum communication complexity of L 0 -round bit pointer-chasing when Bob speaks first is Ω( N/L 4 0 ) (which holds for classical communication as well). This also bounds the ( L 0 -1) -round complexity when Alice speaks first (since such a protocol is strictly less powerful given that there are fewer rounds of communication). On the other hand, there is a trivial L 0 -round protocol when Alice speaks first that requires log N bits of communication per round, in which Alice sends Bob x , he sends back f (1) ( x ) , she replies with f (2) ( f (1) ( x )) , and so forth. This, combined with the lower bound, implies as exponential separation in communication complexity as a function of the number of rounds.

To reduce this problem to Problem 1, we assume f A , f B are invertible. This should not make the problem any easier since it implies that f A , f B have the largest possible image. In this setting, f A , f B can be described by unitary permutation matrices:

The corresponding circuit eq. (3.5) is then given by

in the case where Bob applies the function last, with an analogous circuit in the converse situation (if Bob performed the swap, Alice applies an additional identity map). Estimating Z 0 to accuracy ε < 1 using this state will then reveal the least significant bit of f ( L 0 ) ( x ) . This gives a circuit with L layers, where L 0 ≤ 2 L -1 . Thus any protocol with less than L 0 rounds (meaning less than 2 L -1 rounds) would require communicating Ω( N/L 4 0 ) = Ω( N/L 4 ) qubits, since the converse will contradict the results of [58]. The reduction to Problem 2 is along similar lines to the one described by eq. (B.9), with the state in that circuit replaced by eq. (B.11). This requires at most two additional rounds of communication.

Since quantum communication is at least as powerful than classical communication, these bounds also hold for classical communication. Since each round involves communicating at least a single bit, this gives an Ω( L ) bound on the classical communication complexity.

Proof of Lemma 5. The proof is based on a reduction from the f -Boolean Hidden Partition problem ( f -BHP N,t ) studied in [36]. This is defined as follows:

Problem 6 (Boolean Hidden Partition [36] ( f -BHP N,t )) . Assume t divides N . Alice is given x ∈ {-1 , 1 } N . Bob is given a permutation Π over [ N ] , a boolean function f : {-1 , 1 } t →{-1 , 1 } , and a vector v ∈ {-1 , 1 } N/t . We are guaranteed that for any k ∈ { 1 , . . . , N/t } ,

for some s ∈ {-1 , 1 } . Their goal is to determine the value of s .

Apolynomial p f : {-1 , 1 } t → R is said to sign-represent a boolean function f if sign( p f ( y )) = f ( y ) for all y ∈ { 0 , 1 } t . The sign-degree of f ( sdeg( f ) ) is the minimal degree of a polynomial that signrepresents it. In the special case sdeg( f ) = 2 , f -BHP N,t can be solved with exponential quantum communication advantage [36]. For a vector y ∈ { 0 , 1 } t , define ˜ y = (1 , y 1 , . . . , y t ) . It is also known that if sdeg( f ) = 2 , then there exists a sign-representing polynomial p f that can be written as

for some matrix real R [4]. Moreover, for any f there exists such a p f with max x ∈{-1 , 1 } t | p f ( x ) | ≤ 3 . We denote by β = min x ∈{-1 , 1 } t | p f ( x ) | the bias of p f .

We now describe a reduction from f -BHP N,t with sdeg( f ) = 2 to QGNI CN,t for some constant 1 ≤ C ≤ 3 / 2 . As is typical in communication complexity, the parties are allowed to exchange bits that are independent of the problem input, and these are not counted when measuring the communication complexity of a protocol that depends on the inputs. Before receiving their inputs, Alice thus sends two orthogonal vectors u 0 , u 1 of length D 0 to Bob, with each entry described by K bits 8 .

Assume Alice and Bob are given an instance of BHP N,t . They use it to construct an instance of QGNI ( t +1) N/t,t with D 1 = 1 . Alice constructs X ∈ R ( t +1) N/t by picking the rows X i according to

Note that with this definition || X || F = 1 . Bob defines a permutation π ′ over [( t +1) N/t ] by

denoting the corresponding permutation matrix Π ′ . Define by x the concatenation of Alice's input x with 1 ( t +1) N/t . The purpose of this permutation is that Π ′ Π x ≡ ˜ x will be a concatenation of N/t vectors of length t +1 , with the i -th vector equal to (1 , [Π x ] t ( i -1)+1 , . . . , [Π x ] ti ) ≡ ˜ x ( i ) .

Note that we can assume wlog that R in eq. (B.13) is symmetric since p f is independent of its antisymmetric part. It can thus be diagonalized by an orthogonal matrix U , and denoting the diagonal matrix of its real eigenvalues by D , we define a (complex-valued) matrix S = U √ D that satisfies R = SS T . Bob therefore defines his model by

Additionally, he picks the pooling operator P : R ( t +1) N/t → R N/t to be sum pooling with window size t +1 (i.e. P ( x ) j = ∑ j ( t +1) k =( j -1)( t +1)+1 x k ). Bob also uses a simple quadratic nonlinearity by choosing a = 1 , b = c = 0 in eq. (4.3).

To see that solving QGNI ( t +1) N/t,t to error ε < 1 / 2 indeed provides a solution to BHP N,t , note that

Given the choice of W 2 , one obtains

It follows that sign( φ ( X )) = s and | φ ( X ) | ≥ β . It is thus possible to decide the value of s if φ ( X/ || X || F ) is estimated to some error smaller than β .

From Theorem 4 of [36], we have R → ( f -BHP N,t ) = Ω( √ N/t ) for any f that has sign-degree 2 and satisfies some additional conditions. The reduction then implies

This can be simplified by noting that since t ≥ 2 , t/ ( t + 1) ≥ 2 / 3 . The lower bound in [36] is based on choosing f which belongs to a specific class of symmetric boolean functions (meaning f ( y ) = ˜ f ( | y | ) where | y | = |{ i : y i = -1 }| ). Specifically, ˜ f is defined by the choice of t and two additional integer parameters θ 1 , θ 2 such that 0 ≤ θ 1 < θ 2 < t and

(and an additional technical condition that will not be of relevance to our analysis).

We next construct a sign-representing polynomial p f for any f that takes the above form, and compute its bias β . Since f is symmetric of sign degree 2 , it suffices to construct a polynomial ˜ p f : R → R such that p f ( y ) = ˜ p f ( | y | ) for this purpose. If we can produce some β ′ that bounds β from below for any choice of t, θ 1 , θ 2 , then the lower bound from Theorem 4 of [36] holds for any error smaller than β ′ .

We choose ˜ p f ( z ) = ˜ az 2 + ˜ bz +1 , with the constraints ˜ p f ( θ 1 +1 / 2) = 0 , ˜ p f ( θ 2 +1 / 2) = 0 . These lead to the solution

Since this is a quadratic function with known roots that is only evaluated at integer inputs, if we want to bound the bias of p f it suffices to check the values of ˜ p f at the integers closest to the roots, namely

{ θ 1 , θ 1 +1 , θ 2 , θ 2 +1 } . Plugging in these values gives

where in the third line we used 1 1+ x ≤ 1 -x/ 2 which holds for 0 ≤ x ≤ 1 . Using θ 2 ≤ θ 1 +1 , we also have

˜ p f ( θ 2 ) takes the same value as ˜ p f ( θ 1 +1) . Similarly,

For θ 2 ≤ 1 this is a monotonically increasing function of θ 2 , and is thus lower bounded by picking θ 2 = 1 , giving ˜ p f (2) ≥ 1 . It follows that for any choice of t, θ 1 , θ 2 , the bias is bounded from below by

Note that our bound on the bias allows us to use the reduction from f -BHP N,t to QGNI ( t +1) N/t,t for any valid choice of f (satisfying eq. (B.24)).

Proof of Lemma 6. Alice encodes her input in the quantum state

over log( ND 0 ) + 1 qubits. She sends this state to Bob. Define D = max { D 0 , D 1 } . Bob augments this state by attaching zero qubits and, reordering the first two qubits, obtains the state

over log( ND ) + 2 qubits.

Define by W 1 the D × D matrix obtained by appending zero rows or columns to the rectangular matrix W 1 to obtain a square matrix, and denote α = ∥ ∥ A ⊗ W 1 ∥ ∥ . Bob prepares an ( α, 1 , 0) -blockencoding of A ⊗ W 1 , denoted U A ⊗ W 1 , which acts on log( ND ) + 1 qubits. Bob then applies this unitary conditioned on the value of the first qubit, giving

where | g ⟩ is an unnormalized garbage state. Above, AXW 1 is an N × D matrix obtained by adding zero columns to W 1 as needed.

The sum pooling operator P can be implemented by multiplication by an N/t × N matrix which we denote by ˜ P . Define by W 2 the D × N/t matrix obtained by appending zero rows to W 2 if needed. Given a matrix M of size N 1 × N 2 , denote by V [ M ] the vectorization of M . Bob then constructs the Hermitian matrix

It follows that

where 1 N × D 1 is an all ones matrix. The last term on the RHS is independent of X and can be computed by Bob without requiring Alice's message. Estimating ⟨ ψ | O | ψ ⟩ to accuracy ε requires O ( ∥O∥ /ε ) measurements. Since

Bob requires O (( | a | α 2 + | b | α ) ∥ ∥ ∥ W 2 ˜ P ∥ ∥ ∥ ∞ /ε ) copies of Alice's state in order to do this.

Proof of lemma 7. For the parameter choices used to obtain the classical lower bound (eq. (B.14) and eq. (B.16)), we have || W 1 || = 1 , || W 2 P || ∞ ≤ t . Additionally, for the polynomials constructed in eq. (B.26), we have | p f ( y ) | ≤ Ct 2 from which it follows that the matrix R used in the matrix representation of p f has constant operator norm C , and thus ∥ A ∥ = ∥ S ∥ = √ C . We also have a = 1 , b = c = 0 for the nonlinearity used (eq. (4.3)), and it thus follows from Lemma 6 that Q → ε (QGNI N,t ) = O ( t 3 log( ND 0 )) for ε ≤ 1 4 ( t + 1 2 )( t + 3 2 ) . With this choice of ε , the classical lower bound in Lemma 5 holds, and thus an exponential advantage in communication is obtained by using quantum communication.

Proof of Lemma 11. Consider first a single variable z , with data-dependent unitaries given by eq. (F.4a). If { λ ℓi } are chosen i.i.d. from a uniform distribution over say [0 , 1] , then with probability

1 they are all unique and so are all sums of the form Λ j = L ∑ ℓ =1 λ ℓj ℓ as well as differences Λ j -Λ k for

k < j where the inequality holds element-wise. Set B ℓ to be the Hadamard transform over log N ′ qubits for all ℓ , and pick the measurement operator P 0 = X 0 . We then have

̸

where ˜ j L = j L +( -1) ⌊ j L / ( N ′ / 2+1) ⌋ N ′ / 2 . In the third line, we dropped the diagonal terms in the double sum since they vanish due to the X 0 matrix having 0 on its diagonal. In the fourth line, we collected terms and used the symmetry of ( B † 1 ) 1 j 1 . . . ( B † L ) j L -1 j L ( X 0 ) j L k L . . . ( B 1 ) k 1 1 to the permutation of j and k . In the last line we performed the sum over k L using the structure of X 0 . By our assumption about the { λ ℓi } , each term in the final sum has a unique frequency so no cancellations are possible. The coefficient of each cosine is nonzero (and is equal to 2 N ′-L or -2 N ′-L ). There are a total of ( N ′ ( N ′ -1) 2 ) L -1 N ′ such summands. This completes the first part of the proof for this choice of { B ℓ } .

Considering instead the case of two variables, with unitaries given by eq. (F.4b), an equivalent calculation gives

where

As before, there are ( N ′ ( N ′ -1) 2 ) L -1 N ′ summands in total. Since

we can rewrite eq. (B.38) as a sum over 2 ( N ′ ( N ′ -1) 2 ) L -1 N ′ terms that are pairwise orthogonal w.r.t. the L 2 inner product over R 2 . It follows from the definition of the separation rank that

We next use the assumption that the real and imaginary parts of each element of B ℓ are real analytic function of parameters Θ . This implies that the same property holds for product of entries of the form

for any choice of j, k . This coefficient is equal to 0 iff both the real and imaginary parts are equal to 0 . Since the zero set of a real analytic function has measure 0 [81], the set of values of Θ for which any of the coefficients in eq. (B.38) vanishes also has measure 0 , for all choices of j, k . The result follows.

Proof of Lemma 12. Consider a periodic function f with period 1 . Denote by S M [ f ] the truncated Fourier series of f written in terms of trigonometric functions:

If f is p -times continuously differentiable, it is known that the Fourier series converges uniformly, with rate

for some absolute constant C [91]. For analytic functions the rate is exponential in M .

We now define the following circuit:

where

Choosing | ψ ( x ) ⟩ = | 0 ⟩ as the initial state, this gives

It follows that

(B.49)

This approximation thus converges uniformly according to eq. (B.44), with error decaying exponentially with number of qubits log N ′ as long as f is continuously differentiable at least once.

Proof of Lemma 10. The algorithm in Theorem 5 of [100] takes as input a state-preparation unitary U acting on n = log N qubits such that U | 0 ⟩ ⊗ n = | z ⟩ . Using O (log 1 /ε ) queries to U and U † and

n +4 ancillas, it creates a state | φ ⟩ such that measuring 0 on the first n +4 qubits of | φ ⟩ results in a state | ˆ φ ⟩ that obeys

Additionally, the probability of measuring 0 on the first n +4 qubits is O (1) .

We will be interested in applying this algorithm to the state | U 1 x ⟩ . The state preparation unitary can be instantiated with a single round of communication by Alice starting with the state | 0 ⟩ ⊗ 2 n +4 , applying a unitary that encodes x in the last n qubits of this state, and then sending it to Bob who applies U 1 to the same n qubits. The conjugate of the state-preparation unitary can be applied in a similar fashion by reversing this procedure. This can include any conditioning required on the values of the other qubits.

Based on the query complexity of the algorithm in [100] to the state preparation unitary, O (log(1 /ε )) rounds will suffice to obtain a state

such that

Bob then applies U 2 to the state | ˜ φ σ ⟩ conditioned on the first n +4 qubits being in the state | 0 ⟩ ⊗ n +4 . The state | ϕ ⟩ is unaffected. Unitary of U 2 combined with the above bound guarantees

Additionally, from Theorem 3 of [100] we are guaranteed that α = O (1) .

C Data parallelism

Data parallelism involves storing multiple copies of a model on different devices and training each copy on a subset of the full data. We consider a model of the form

where x is an N 1 × N 2 matrix which we write as x = [ x A , x B ] for two N 1 / 2 × N 2 matrices x A , x B . Assume also that ∥ x ∥ F = 1 . This model can be used to define a distributed problem with dara parallelism by considering the following inputs to both players:

The state | x ⟩ can be prepared in a single round of communication involving log( N 1 N 2 ) qubits. Alice simply prepares the state

using zero-based indexing of the elements of x A . After sending this to Bob, he applies the unitary

The resulting state is | x ⟩ . As before, the gradients with respect to the parameters of the unitaries { U ℓ } can be estimated by preparing copies of this state and using shadow tomography. The number of copies will again be logarithmic in N 1 , N 2 and the number of trainable parameters.

D Exponential advantages in end-to-end training

So far we have discussed the problems of inference and estimating a single gradient vector. It is natural to also consider when these or other gradient estimators can be used to efficiently solve an optimization problem (i.e. when the entire training processes is considered rather than a single iteration). Applying the gradient estimation algorithm detailed in Lemma 2 iteratively gives a distributed stochastic gradient descent algorithm which we detail in Algorithm 2, yet one may be concerned that a choice of ε = O (log N ) which is needed to obtain an advantage in communication complexity will preclude efficient convergence. Here we present a simpler algorithm that requires a single quantum measurement per iteration, and can provably solve certain convex problems efficiently, as well as an application of shadow tomography to fine-tuning where convergence can be guaranteed, again with only logarithmic communication cost. In both cases, there is an exponential advantage in communication even when considering the entire training process.

D.1 'Smooth' circuits

Consider the case where A ℓ are product of rotations for all ℓ , namely

where P A ℓj are Pauli matrices acting on all qubits, and similarly for B ℓ . These can also be interspersed with other non-trainable unitaries. This constitutes a slight generalization of the setting considered in [49], and the algorithm we present is essentially a distributed distributed version of theirs. Denote by β an 2 PL -dimensional vector with elements β Q ℓj where Q ∈ { A,B } 9 . The quantity ∥ β ∥ 1 is the total evolution time if we interpret the state | φ ⟩ as a sequence of Hamiltonians applied to the initial state | x ⟩ .

In Appendix D.3 we describe an algorithm that converges to the neighborhood of a minimum, or achieves E L (Θ) -L (Θ ⋆ ) ≤ ε 0 , for a convex L after

iterations, where Θ ⋆ are the parameter values at the minimum of L . The expectation is with respect to the randomness of quantum measurement and additional internal randomness of the algorithm. The algorithm is based on classically sampling a single coordinate to update at every iteration, and computing an unbiased estimator of the gradient with a single measurement. It can thus be seen as a form of probabilistic coordinate descent.

This implies an exponential advantage in communication for the entire training process as long as ∥ ∥ Θ (0) -Θ ⋆ ∥ ∥ 2 2 ∥ β ∥ 2 1 = polylog( N ) . Such circuits either have a small number of trainable parameters ( P = O (polylog( N )) ), depend weakly on each parameter (e.g. β Q ℓj = O (1 /P ) for arbitrary P ), or have structure that allows initial parameter guesses whose quality diminishes quite slowly with system size. Nevertheless, over a convex region the loss can rapidly change by an O (1) amount. One may also be concerned that in the setting ∥ ∥ Θ (0) -Θ ⋆ ∥ ∥ 2 2 ∥ β ∥ 2 1 = polylog( N ) only a logarithmic number of parameters is updated during the entire training process and so the total effect of the training process may be negligible. It is important to note however that each such sparse update depends on the structure of the entire gradient vector as seen in the sampling step. In this sense the algorithm is a form of probabilistic coordinate descent, since the probability of updating a coordinate | β Q ℓj | / ∥ β ∥ 1 is proportional to the the magnitude of the corresponding element in the gradient (actually serving as an upper bound for it).

Remarkably, the time complexity of a single iteration of this algorithm is proportional to a forward pass, and so matches the scaling of classical backpropagation. This is in contrast to the polynomial overhead of shadow tomography (Theorem 1). Additionally, it requires a single measurement per iteration, without any of the additional factors in the sample complexity of shadow tomography.

D.2 Fine-tuning the last layer of a model

Consider a model given by eq. (3.1) where only the parameters of A L are trained, and the rest are frozen, and denote this model by | φ f ⟩ . The circuit up to that unitary could include multiple data-dependent unitaries that represent complex features in the data. Training only the final layer in this manner is a common method of fine-tuning a pre-trained model [53]. If we now define

the expectation value of ˜ E A Li using the state | + ⟩ ∣ ∣ µ A L 〉 gives ∂ L ∂θ A ℓi . Here ∣ ∣ µ A L 〉 =

parameters of all the other unitaries frozen (hence the dependence on them is dropped). Since the observables in the shadow tomography problem can be chosen in an online fashion [5, 6, 13], and adaptively based on previous measurements, we can simply define a stream of measurement operators by measuring P observables to estimate the gradients w.r.t. an initial set of parameters, updating these parameters using gradient descent with step size η , and defining a new set of observables using the updated parameters. Repeating this for T iterations gives a total of PT observables (a complete description of the algorithm is given in Algorithm 3).

By the scaling in Lemma 2, the total communication needed is ˜ O (log N (log TP ) 2 log(1 /δ ) /ε 4 ) over O ( L ) rounds (since only O ( L ) rounds are needed to create copies of ∣ ∣ µ A L 〉 . This implies an exponential advantage in communication for the entire training process (under the reasonable assumption T = O (poly( N,P )) ), despite the additional stochasticity introduced by the need to perform quantum measurements. For example, assume one has a bound ∥∇L∥ 2 2 ≤ K . If the circuit is comprised of unitaries with Hermitian derivatives, this holds with K = PL . In that case, denoting by g the gradient estimator obtained by shadow tomography, we have

It then follows directly from Lemma 8 that for an appropriately chosen step size, if L is convex one can find parameter values Θ such that L (Θ) -L (Θ ⋆ ) ≤ ε 0 using

iterations of gradient descent. Similarly if L is λ -strongly convex then T = 2( K + ε 2 PL ) 2 /λε 0 + 1 iterations are sufficient. In both cases therefore an exponential advantage is achieved for the optimization process as a whole, since in both cases one can implement the circuit that is used to obtain the lower bounds in Lemma 3.

In the following, we make use of well-known convergence rates for stochastic gradient descent:

Lemma 8 ([26]) . Given an objective function L (Θ) with a minimum at Θ ⋆ and a stochastic gradient oracle that returns a noisy estimate of the gradient g (Θ) such that E g (Θ) = ∇L (Θ) , E ∥ g ∥ 2 2 ≤ G 2 , and denoting by Θ (0) a point in parameter space and R = ∥ ∥ Θ (0) -Θ ⋆ ∥ ∥ 2 , we have:

If L is convex in a Euclidean ball of radius R around Θ ⋆ , then gradient descent with step size η = R G √ 2 T achieves

ii) If L is λ -strongly convex in a Euclidean ball of radius R around Θ ⋆ , then gradient descent with step size η t = 2 λ ( t +1) achieves

Algorithm 1 Distributed Probabilistic Coordinate Descent

D.3 Distributed Probabilistic Coordinate Descent

Given distributed states of the form eq. (D.1), optimization over Θ can be performed using Algorithm 1. We verify the correctness of this algorithm and provide convergence rates following [49]. Define the Hermitian measurement operator

with eigenvalues in {-1 , 1 } . Note that β Q ℓi 〈 ψ Q ℓ 0 ∣ ∣ ∣ ˆ E Q ℓi ∣ ∣ ∣ ψ Q ℓ 0 〉 = ∂ L ∂θ Q ℓi , and this is essentially a compact way of representing a Hadamard test for the relevant expectation value. Now consider a gradient estimator that first samples ( Q,ℓ, i ) with probability | β Q ℓi | / || β || 1 , then returns a one-sparse vector with g Q ℓi = sign( β Q ℓi ) ∥ β ∥ 1 m , where m is the result of a single measurement of ˆ E Q ℓi using the state ∣ ∣ ∣ ψ Q ℓ 0 〉 . For this estimator we have

where the expectation is taken over both the index sampling process and the quantum measurement. The procedure generates a valid gradient estimator.

In order to show convergence, one simply notes that by construction, ∥ g ∥ 2 = ∥ β ∥ 1 . It then follows immediately from Lemma 8 that, with an appropriately chosen step size, Algorithm 1 achieves E L (Θ) -L (Θ ⋆ ) ≤ ε 0 for a convex L using

queries. For a λ -strongly convex L , only

queries are required. The pre-processing in step 1 of Algorithm 1 requires time O ( P log P ) and subsequently enables sampling in time O (1) using e.g. [111] 10 .

Algorithm 2 Shadow Tomographic Distributed Gradient Descent

Input:

Alice: x, { A ℓ } , Θ (1) A , η, T . Bob: { B ℓ } , Θ (1) B , η, T.

Output:

Alice: Updated parameters Θ ( T ) A . Bob: Updated parameters Θ ( T ) B .

1: for t ∈ { 1 , . . . , T } do

2: for ℓ ∈ { 1 , . . . , L } do

3: Alice prepares ˜ O (log 2 P log N ′ log( L/δ ) /ε 4 ) copies of ∣ ∣ ψ A ℓ 0 (Θ ( t ) ) 〉 { O ( L ) rounds of communication}

4: Alice runs Shadow Tomography to estimate { E E A ℓi (Θ ( t ) ) } P i =1 up to error ε , denoting these { g A ℓi (Θ ( t ) ) } P i =1 .

5: Bob prepares ˜ O (log 2 P log N ′ log( L/δ ) /ε 4 ) copies of ∣ ∣ ψ B ℓ 0 (Θ ( t ) ) 〉 { O ( L ) rounds of communication}

6: Bob runs Shadow Tomography to estimate { E E B ℓi (Θ ( t ) ) } P i =1 up to error ε , denoting these { g B ℓi (Θ ( t ) ) } P i =1 .

7: Alice sets θ A ( t +1) ℓ ← θ A ( t ) ℓ -ηg A ℓ (Θ ( t ) ) .

8: Bob sets θ B ( t +1) ℓ ← θ B ( t ) ℓ -ηg B ℓ (Θ ( t ) ) .

9: end for

10: end for

Algorithm 3 Shadow Tomographic Distributed Fine-Tuning

1: Alice prepares ˜ O (log 2 ( PT ) log N ′ log(1 /δ ) /ε 4 ) copies of ∣ ∣ µ A L 〉 { O ( L ) rounds of communication}

2: for t ∈ { 1 , . . . , T } do

3: Alice runs online Shadow Tomography to estimate { E ˜ E A Li ( θ A ( t ) L ) } up to error ε , denoting these { g A Li ( θ A ( t ) L ) } .

4: Alice sets θ A ( t +1) L ← θ A ( t ) L -ηg A L ( θ A ( t ) L ) .

5: end for

D.4 Algorithms based on Shadow Tomography

E Communication Complexity of Linear Classification

While the separation in communication complexity for expressive networks can be quite large, interestingly we will show that for some of the simplest models this advantage can vanish due to the presence of structure. In particular, when a linear classifier is well-suited to a task such that the margin is large, the communication advantage will start to wane, while a lack of structure in linear classification will make the problem difficult for quantum algorithms as well. More specifically, we consider the following classification problem:

Problem 7 (Distributed Linear Classification) . Alice and Bob are given x, y ∈ S N , with the promise that | x · y | ≥ γ for some 0 ≤ γ ≤ 1 . Their goal is to determine the sign of x · y .

This is one of the simplest distributed inference problem in high dimensions that one can formulate. x can be thought of as the input to the model, while y defines a separating hyperplane with some margin. Since with finite margin we are only required to resolve the inner product between the vectors to some finite precision, it might seem that an exponential quantum advantage should be possible for this problem by encoding the inputs in the amplitudes of a quantum state. However, we show that classical algorithms can leverage this structure as well, and consequently that the quantum advantage in communication that can be achieved for this problem is at most polynomial in N . We

prove this with respect to the the randomized classical communication model, in which Alice and Bob are allowed to share random bits that are independent of their inputs 11 .

Lemma 9. The quantum communication complexity of Problem 7 is Ω ( √ N/ max(1 , ⌈ γN ⌉ ) ) . The randomized classical communication complexity of Problem 7 is O (min( N, 1 /γ 2 )) .

Proof. We first describe a protocol that allows Alice and Bob to solve the linear classification problem with margin γ using O (1 /γ 2 ) bits of classical communication and shared randomness, assuming γ > 0 . Note that this bound accords with the notion that the margin rather than the ambient dimension sets the complexity of these types of problems, which is also manifest in the sample complexity of learning with linearly separable data.

Alice and Bob share kN bits sampled i.i.d. from a uniform distribution over { 0 , 1 } , and that these bits are arranged in a k × N matrix R . Alice and Bob then receive x and y respectively, which are valid inputs to the linear classification problem with margin γ . For any N -dimensional vector z , define the random projection

where addition is element-wise. Applying the Johnson-Lindenstrauss lemma for projections with binary variables [8], we obtain that if k = C/ε 2 , for some absolute constant C , then with probability larger than 2 / 3 we have for any z, z ′ ∈ { x, y, 0 } (all of these being vectors in R N ), f is an approximate isometry in the sense

The key feature of this result is that k is completely independent of N . Applying it repeatedly gives

Obtaining an upper bound in a similar fashion using the converse inequalities, we have

Assume now that x, y are valid inputs to the linear classification problem with margin γ , and specifically that x · y ≥ γ . The lower bound above gives

and if we choose ε = γ/ 8 we obtain

where we used γ ≤ 1 . Similarly, if instead x · y ≤ -γ we obtain

It follows that if Alice computes f ( x ) and sends the resulting O ( k ) = O (1 /γ 2 ) bits that describe this vector to Bob (assuming some finite precision that is large enough so as not to affect the margin, which will contribute ), Bob can simply compute f ( x ) · f ( y ) which will reveal the result of the classification problem, which he can then communicate to Alice using a single bit.

If γ = 0 there is a trivial O ( N ) classical algorithm where Alice sends Bob x .

We next describe the quantum lower bound for Problem 7. Denote by d H the Hamming distance between binary vectors. We will use lower bounds for the following problem:

Problem 8 (Gap Hamming with general gap) . Alice and Bob are given ˆ x, ˆ y ∈ { 0 , 1 } N respectively. Given a promise that either d H (ˆ x, ˆ y ) ≥ N/ 2 + g/ 2 or d H (ˆ x, ˆ y ) ≤ N/ 2 -g/ 2 , Alice and Bob must determine which one is the case.

There is a simple reduction from Problem 8 to Problem 7 for certain values of γ , which we will then use to obtain a result for all γ . Assuming Alice is given ˆ x and Bob is given ˆ y , they construct unit norm real vectors by x = (2ˆ x -1) / √ N,y = (2ˆ y -1) / √ N with addition performed element-wise.

If d H ( x, y ) ≥ N/ 2 + g/ 2 then

̸

Similarly, d H (ˆ x, ˆ y ) ≤ N/ 2 -g/ 2 ⇒ x · y ≤ -g/N . It follows that x, y are valid inputs for a linear classification problem over the unit sphere with margin 2 g/N . From the results of [88], any quantum algorithm for the Gap Hamming problem with gap g ∈ { 1 , . . . , N } requires Ω( √ N/g ) qubits of communication. It follows that the linear classification problem requires Ω( √ 1 /γ ) qubits of communication. This bound holds for integer γN . To get a result for general 0 < γ ≤ 1 we simply note that the communication complexity must be a non-decreasing function of 1 /γ , since any inputs which constitute a valid instance with some γ are also a valid instance for any γ ′ < γ . Given some real γ , the resulting communication problem is at least as hard as the one with margin ⌈ γN ⌉ /N ≥ γ . It follows that a Ω( √ N/ ⌈ γN ⌉ ) bound holds for all 0 < γ ≤ 1 .

If γ = 0 , by a similar argument we can apply the lower bound for γ = 1 /N , implying that Ω( √ N ) qubits of communication are necessary. Once again there is only a polynomial advantage at best.

F Expressivity of quantum circuits

F.1 Expressivity of compositional models

It is natural to ask how expressive models of the form of eq. (3.1) can be, given the unitarity constraint of quantum mechanics on the matrices { A ℓ , B ℓ } . This is a nuanced question that can depend on the encoding of the data that is chosen and the method of readout. On the one hand, if we pick | ψ ( x ) ⟩ as in eq. (3.4) and use { A ℓ , B ℓ } that are independent of x , the resulting state | φ ⟩ will be a linear function of x and the observables measured will be at most quadratic functions of those entries. On the other hand, one could map bits to qubits 1-to-1 and encode any reversible classical function of data within the unitary matrices { A ℓ ( x ) } with the use of extra space qubits. However, this negates the possibility of any space or communication advantages (and does not provide any real computational advantage without additional processing). As above, one prefers to work on more generic functions in the amplitude and phase space, allowing for an exponential compression of the data into a quantum state, but one that must be carefully worked with.

We investigate the consequences of picking { A ℓ ( x ) } that are nonlinear functions of x , and { B ℓ } that are data-independent. This is inspired by a common use case in which Alice holds some data or features of the data, while Bob holds a model that can process these features. Given a scalar variable x , define A ℓ ( x ) = diag(( e -2 πiλ ℓ 1 x , . . . , e -2 πiλ ℓN ′ x )) for ℓ ∈ { 1 , . . . , L } . We also consider parameterized unitaries { B ℓ } that are independent of the { λ ℓi } and inputs x, y , and the state obtained by interleaving the two in the manner of eq. (3.1) by | φ ( x ) ⟩ .

We next set λ ℓ 1 = 0 for all ℓ ∈ { 1 , . . . , L } and λ L 2 = 0 . If we are interested in expressing the frequency

where j ℓ ∈ { 2 , . . . , N ′ } , we simply initialize with | ψ ( x ) ⟩ = | + ⟩ 0 | 0 ⟩ and use

with j 1 = j L = 2 . It is easy to check that the resulting state is | φ ( x ) ⟩ = ( | 0 ⟩ + e -2 πi Λ j x | 1 ⟩ ) / √ 2 . Since the basis state | 0 ⟩ does not accumulate any phase, while the B ℓ s swap the | 1 ⟩ state with

the appropriate basis state at every layer in order to accumulate a phase corresponding to a single summand in eq. (F.1). Choosing to measure the operator P 0 = X 0 , it follows that ⟨ φ ( x ) | X 0 | φ ( x ) ⟩ = cos(2 π Λ j x ) .

It is possible to express O (( N ′ ) L -1 ) different frequencies in this way, assuming the Λ j are distinct, which will be the case for example with probability 1 if the { λ ℓi } are drawn i.i.d. from some distribution with continuous support. This further motivates the small L regime where exponential advantage in communication is possible. These types of circuits with interleaved data-dependent unitaries and parameterized unitaries was considered for example in [104], and is also related to the setting of quantum signal processing and related algorithms [74, 77]. We also show that such circuits can express dense function in Fourier space, and for small N we additionally find that these circuits are universal function approximators (Appendix F.2), though in this setting the possible communication advantage is less clear.

The problem of applying nonlinearities to data encoded efficiently in quantum states is non-trivial and is of interest due to the importance of nonlinearities in enabling efficient function approximation [76]. One approach to resolving the constraints of unitarity with the potential irreversibility of nonlinear functions is the introduction of slack variables via additional ancilla qubits, as typified by the techniques of block-encoding [29, 41]. Indeed, these techniques can be used to apply nonlinearities to amplitude encoded data efficiently, as was recently shown in [100]. This approach can be applied to the distributed setting as well. Consider the communication problem where Alice is given x as input and Bob is given unitaries { U 1 , U 2 } over log N qubits. Denote by σ : R → R a nonlinear function such as the sigmoid, exponential or standard trigonometric functions, and n = 2 N . We show the following:

Lemma 10. There exists a model | φ σ ⟩ of the form definition 3.1 with L = O (log 1 /ε ) , N ′ = 2 n ′ where n ′ = 2 n +4 such that | φ σ ⟩ = α | 0 ⟩ ⊗ n +4 | ˆ y ⟩ + | ϕ ⟩ for some α = O (1) , where | ˆ y ⟩ is a state that obeys

| ϕ ⟩ is a state whose first n +4 registers are orthogonal to | 0 ⟩ ⊗ n +4 .

This result implies that with constant probability, after measurement of the first n +4 qubits of | φ σ ⟩ , one obtains a state whose amplitudes encode the output of a single hidden layer neural network. It may also be possible to generalize this algorithm and apply it recursively to obtain a state representing a deep feed-forward network with unitary weight matrices.

It is also worth noting that the general form of the circuits we consider resembles self-attention based models with their nonlinearities removed (motivated for example by [107]), as we explain in Appendix F.3. Finally, in Appendix F.4 we discuss other strategies for increasing the expressivity of these quantum circuits by combining them with classical networks.

F.2 Additional results on oscillatory features

Extending the unitaries considered in Appendix F.1 to more than one variable, for two scalar variables x, y define

for ℓ ∈ { 1 , . . . , L } . Once again { B ℓ } are data-independent unitaries, and we denote by | φ ( x ) ⟩ , | φ ( x, y ) ⟩ the states defined by interleaving these unitaries in the manner of eq. (3.1), and by L 1 , L 2 the corresponding loss functions when measuring X 0 .

While the circuits in Appendix F.1 enable one to represent a small number of frequencies from a set that is exponential in L , one can easily construct circuits that are supported on an exponentially large number of frequencies, as detailed in Lemma 11. We also use measures of expressivity of classical neural networks known as separation rank to show that circuits within the class eq. (3.1) can represent complex correlations between their inputs. For a function f of two variables y, z , its

separation rank is defined by

If for example f cannot represent any correlations between y and z , then sep( f ) = 1 . When computed for certain classes of neural networks, y, z are taken to be subsets of a high-dimensional input. The separation rank can be used for example to quantify the inductive bias of convolutional networks towards learning local correlations [33], the effect of depth in recurrent networks [69], and the ability of transformers to capture correlations across sequences as a function of their depth and width [70].

Wefind that the output of estimating an observable using circuits of the form eq. (F.4) can be supported on an exponential number of frequencies, and consequently has a large separation rank:

Lemma 11. For { λ ℓi } drawn i.i.d. from any continuous distribution and parameterized unitaries { B ℓ } such that the real and imaginary parts of each entry in these matrices is a real analytic function of parameters Θ drawn from a subset of R PL , aside from a set of measure 0 over the choice of { λ ℓi } , { B ℓ } ,

Proof: Appendix B

This almost saturates the upper bound on the number of frequencies that can be expressed by a circuit of this form that is given in [104]. The separation rank implies that complex correlations between different parts of the sequence can in principle be represented by such circuits. The constraint on { B ℓ } is quite mild, and applies to standard choices of parameterize unitaries.

The main shortcoming of a result such as Lemma 11 is that it is not robust to measurement error as it is based on constructing states that are equal weight superpositions of an exponential number of terms. It is straightforward to show that circuits of this form can serve as universal function approximators, at least for a small number of variables. For high-dimensional functions it is unclear when a communication advantage is possible, as we describe below.

Lemma 12. Let f be a p -times continuously differentiable function with period 1 , and denote by ˆ f : M the vector of the first M Fourier components of f . If ∥ ∥ ∥ ˆ f : M ∥ ∥ ∥ 1 = 1 then there exists a circuit of the form eq. (3.1) over O (log M ) qubits such that

for some absolute constant C .

Proof: Appendix B

This result improves upon the result in [93, 104] about universal approximation with similarly structured circuits both because it is non-asymptotic and because it shows uniform convergence rather than convergence in L 2 . Non-asymptotic results universal approximation results were also obtained recently by [43], however their approximation error scales polynomially with the number of qubits, as opposed to exponentially as in Lemma 12.

The result of Lemma 12 applies to an L = 1 circuit. The special hierarchical structure of the Fourier transform implies that the same result can be obtained using even simpler circuits with larger L . Consider instead single-qubit data-dependent unitaries over L +1 qubits that take the form

for ℓ ∈ { 1 , . . . , L } . This is simply a single term in a hierarchical decomposition of the same feature matrix we had in the shallow case, since

which is identical to eq. (B.45). As before, set

with N ′ / 4 = 2 L and B ℓ = I for ℓ > 1 . This again gives an approximation of f up to normalization. The data-dependent unitaries are particularly simple when decomposed in this way. The fact that they act on a single qubit and thus have "small width" is reminiscent of classical depth-separation result such as [32], where it is shown that (roughly speaking) within certain classes of neural network, in order to represent the function implemented by a network of depth L , a shallow network must have width exponential in L . In this setting as well the expressive power as measured by the convergence rate of the approximation error grows exponentially with L by eq. (B.44).

The circuits above can be generalized in a straightforward way to multivariate functions of the form f : [ -1 / 2 , 1 / 2] D → R and combined with multivariate generalization of eq. (B.44). In this case the scalar m is replaced by a D -dimensional vector taking M D possible values, and we can define

Note that using this feature map, the number of neurons is linear in the spatial dimension D . Because of this, such circuits are not strictly of the form eq. (3.1) for general N since it is not the case that log N ′ = O (log N ) where N ′ is the Hilbert space on which the unitaries in the circuit act and N is the size of x . An alternative setting where the features themselves are also learned from data could enable much more efficient approximation of functions that are sparse in Fourier space.

F.3 Unitary Transformers

Transformers based on self-attention [109] form the backbone of large language models [25, 16] and foundation models more generally [20]. A self-attention layer, which is the central component of transformers, is a map between sequences in R S × N ′ (where S is the sequence length) defined in terms of weight matrices W Q , W K , W V ∈ R N ′ × N ′ , given by

where softmax( x ) i = e x i / ∑ i e x i for a vector x , and acts row-wise on matrices. There is an extensive literature on replacing the softmax-based attention matrix A ( X ) with matrices that can be computed more efficiently, which can markedly improve the time complexity of inference and training without a significant effect on performance [63, 70]. In some cases A ( X ) is replaced by a unitary matrix [68]. Remarkably, recent work shows that models without softmax layers can in fact outperform standard transformers on benchmark tasks while enabling faster inference and a reduced memory footprint [107].

Considering a simplified model that does not contain the softmax operation as in [70] and dropping normalization factors, the linear attention map is given by

Iterating this map twice gives

Iterating this map K times (with different weight matrices at each layer) gives a function of the form:

where the { R ℓ } matrices depend only on the trainable parameters. If we now constrain these to be parameterized unitary matrices, and additionally replace X T X with a unitary matrix U X encoding features of the input sequence itself, then the i -th row of the output of this model is encoded in the amplitudes of a state of the form eq. (3.5) with L = (3 K -1) / 2 + 1 , | ψ ( x ) ⟩ = | X i ⟩ , A ℓ ( x ) = U X , B ℓ = R ℓ .

F.4 Ensembling and point-wise nonlinearities

An additional method for increasing expressivity while maintaining an advantage in communication is through ensembling. Given K models of the form Definition 3.1 with P parameters each, one can combine their loss functions L 1 , . . . , L K into any differentiable nonlinear function

that depends on additional parameters ˜ Θ . As long as K and | ˜ Θ | scale subpolynomially with N and P , the gradients for this more expressive model can be computed while maintaining the exponential communication advantage in terms of N,P .

G Realizing quantum communication

Given the formidable engineering challenges in building a large, fault tolerant quantum processor [11, 44], the problem of exchanging coherent quantum states between such processors might seem even more ambitious. We briefly outline the main problems that need to be solved in order to realize quantum communication and the state of progress in this area, suggesting that this may not be the case.

We first note that sending a quantum state between two processors can be achieved by the well-known protocol of quantum state teleportation [18, 45]. Given an n qubit state | ψ ⟩ , Alice can send | ψ ⟩ to Bob by first sharing n Bell pairs of the form

(sharing such a state involves sending a one of the two qubits to Bob) and subsequently performing local processing on the Bell pairs and exchanging n bits of classical communication. Thus quantum communication can be reduced to communicating Bell pairs up to a logarithmic overhead, and does not require say transmitting an arbitrary quantum state in a fault tolerant manner, which appears to be a daunting challenge given the difficulty of realizing quantum memory on a single processor. Bell pairs can be distributed by a third party using one-way communication.

In order to perform quantum teleportation, the Bell pairs must have high fidelity. As long as the fidelity of the communicated Bell pairs is above . 5 , purification can be used produce high fidelity Bell pairs [19], with the fidelity of the purified Bell pair increasing exponentially with the number of pairs used. Thus communicating arbitrary quantum states can be reduced to communicating noisy Bell pairs.

Bell pair distribution has been demonstrated across multiple hardware platforms including superconducting waveguides [75], optical fibers [64], free space optics at distances of over 1 , 200 kilometers [71]. At least in theory, even greater distances can be covered by using quantum repeaters, which span the distance between two network nodes. Distributing a Bell pair between the nodes can then be reduced to sharing Bell pairs only between adjacent repeaters and local processing [12].

A major challenge in implementing a quantum network is converting entangled states realized in terms of photons used for communication to states used for computation and vice versa, known as transduction [66]. Transduction is a difficult problem due to the several orders of magnitude in energy that can separate optical photons from the energy scale of the platform used for computation. Proof of principle experiments have been performed across a number of platforms including trapped ions [64], solid-state systems [95], and superconducting qubits operating at microwave frequencies [14, 112].

H Privacy of Quantum Communication

In addition to an advantage in communication complexity, the quantum algorithms may potentially lead to advantages in terms of privacy. It is well known that the number of bits of information that

can be extracted from an unknown quantum state is proportional to the number of qubits. It follows immediately that since the above algorithm requires exchanging a logarithmic number of copies of states over O (log N ) qubits, even if all the communication between the two players is intercepted, an attacker cannot extract more than a logarithmic number of bits of classical information about the input data or model parameters. Specifically, we have:

Corollary 1. If Alice and Bob are implementing the quantum algorithm for gradient estimation described in Lemma 2, and all the communication between Alice and Bob is intercepted by an attacker, the attacker cannot extract more than ˜ O ( L 2 (log N ) 2 (log P ) 2 log( L/δ ) /ε 4 ) bits of classical information about the inputs to the players.

This follows directly from Holevo's theorem [52], since the multiple copies exchanged in each round of the protocol can be thought of as a quantum state over ˜ O ((log N ) 2 (log P ) 2 log( L/δ ) /ε 4 ) qubits. As noted in [3], this does not contradict the fact that the protocol allows one to estimate all P elements of the gradient, since if one were to place some distribution over the inputs, the induced distribution over the gradient elements will generally exhibit strong correlations. An analogous result holds for the inference problem described in Lemma 1.

It is also interesting to ask how much information either Bob or Alice can extract about the inputs of the other player by running the protocol. If this amount is logarithmic as well, it provides additional privacy to both the model owner and the data owner. It allows two actors who do not necessarily trust each other, or the channel through which they communicate, to cooperate in jointly training a distributed model or using one for inference while only exposing a vanishing fraction of the information they hold.

It is also worth mentioning that data privacy is also guaranteed in a scenario where the user holding the data also specifies the processing done on the data. In this setting, Alice holds both data x and a full description of the unitaries she wishes to apply to her state. She can send Bob a classical description of these unitaries, and as long as the data and features are communicated in the form of quantum states, only a logarithmic amount of information can be extracted about them. In this setting there is of course no advantage in communication complexity, since the classical description of the unitary will scale like poly( N,P ) .

I Some Open Questions

I.1 Expressivity

Circuits that interleave parameterized unitaries with unitaries that encode features of input data are also used in Quantum Signal Processing [74, 77], where the data-dependent unitaries are time evolution operators with respect to some Hamiltonian of interest. The original QSP algorithm involved a single parameterized rotation at each layer, and it is also known that extending the parameter space from U (1) to SU (2) by including an additional rotation improves the complexity of the algorithm and improves its expressivity [85]. In both cases however the expressive power (in terms of the degree of the polynomial of the singular values that can be expressed) grows only linearly with the number of interleaved unitaries. Given the natural connection to the distributed learning problems considered here, it is interesting to understand the expressive power of such circuits with more powerful multi-qubit parameterized unitaries.

We present a method of applying a single nonlinearity to a distributed circuit using the results of [100]. Since this algorithm requires a state-preparation unitary as input and produces a state with a nonlinearity applied to the amplitudes, it is natural to ask whether it can be applied recursively to produce a state with the output of a deep network with nonlinearties encoded in its amplitudes. This will require extending the results of [100] to handle noisy state-preparation unitaries, yet the effect of errors on compositions of block encodings [29, 41], upon which these results are based, is relatively well understood. It is also worth noting that these approaches rely on the approximation of nonlinear functions by polynomials, and so it may also be useful to take inspiration directly from classical neural network polynomial activations, which in some settings are known to outperform other types of nonlinearities [80].

I.2 Optimization

The results of Appendix D rely on sublinear convergence rates for general stochastic optimization of convex functions (Lemma 8). It is known however that using additional structure, stochastic gradients can be used to obtain linear convergence (meaning that the error decays exponentially with the number of iterations). This is achievable when subsampling is the source of stochasticity [67], or with occasional access to noiseless gradients in order to implement a variance reduction scheme [60, 84, 46], neither of which seem applicable to the setting at hand. It is an interesting open question to ascertain whether there is a way to exploit the structure of quantum circuits to obtain linear convergence rates using novel algorithms. Aside from advantages in time complexity, this could imply an exponential advantage in communication for a more general class of circuits.

Conversely, it is also known that given only black-box access to a noisy gradient oracle, an informationtheoretic lower bound of Ω(1 /T ) on the error holds given T oracle queries, precluding linear convergence without additional structure, even for strongly convex objectives [9]. [49] provide a similar lower bound for their algorithm, at least for a restricted class of circuits. Perhaps these results be used to show optimality of algorithms that rely on the standard variational circuit optimization paradigm that involves making quantum measurements at every iteration and using these to update the parameters. This might imply that linear convergence is only possible if the entire optimization process is performed coherently.

In this context, we note that the treatment of gradient estimation at every layer and every iteration as an independent shadow tomography problem is likely highly suboptimal, since no use is made of the correlations across iterations between the states and the observables of interest. While in Appendix D.2 this is not the case, that algorithm applies only to fine-tuning of a single layer. Is there a way to re-use information between iterations to reduce the resource requirements of gradient descent using shadow tomography? One approach could be warm-starting the classical resource states by reusing them between iterations. Improvements along these lines might find applications for other problems as well.

I.3 Exponential advantage under weaker assumptions

The lower bound in Lemma 3 applies to circuits that contain general unitaries, and thus have depth poly( N ) when compiled using a reasonable gate set. One can ask whether the lower bound can be strengthened to apply to more restricted classes of unitaries as well, and in particular log -depth unitaries. While it is known that exponential communication advantages require the circuits to have poly( N ) gate complexity overall [7], this does not rule out the possibility of computational separations resulting from the clever encoding and transmission of states nor does it rule out communication advantages resulting from very short time preparations from log -depth protocols. The rapid growth of complexity of random circuits composed from local gates with depth suggests that this might be possible [24]. This is particularly interesting since Algorithm 1 requires only a single measurement per iteration and may thus be suitable for implementation on near-term devices whose coherence times restrict them to implementing shallow circuits. It has also been recently shown that an exponential quantum advantage in communication holds for a problem which is essentially equivalent to estimating the loss of a circuit of the form Definition 3.1 with L = 2 , under a weaker model of quantum communication than the standard one we consider [10]. This is the one-clean-qubit model, in which the initial state | ψ ( x ) ⟩ consists of a single qubit in a pure state, while all other qubits are in a maximally mixed state.

J Experiments additional details

J.1 Node classification training

We use the same training regime for all datasets using the recommended hyperparameters in DGL [113] examples, reported in Table 3.

We trained each model 10 times for all three datasets using a single NVIDIA RTX A6000, taking approximately 15 minutes per execution.

J.2 Graph classification training

As, to the best of our knowledge, we are the first to use a SIGN variant on graph classification tasks, we conducted a comprehensive hyperparameter tuning for the model structure (including the number of message passing operators, the hidden dimension, and normalization after the hidden layer) and optimization settings. The tuning was performed using Bayesian hyperparameter optimization to identify the optimal values for each dataset. This process involved varying the hidden dimension, the number of SIGN hops per operation, the learning rate, and dropout rates. The values considered for each hyperparameter are detailed in Table 4. The full results of these experiments are in Table 5.

We scan each task for approximately 150 runs, using a single NVIDIA RTX A6000.

J.3 Empirical bounds

We measure ∥ W 1 ∥ and ∥ W 2 ∥ ∞ of the trained graph classification models in Section 5.2.1, corresponding to Equation (4.2) and report the average results over 10 runs in Table 6 (note that we use P = I so that no pooling matrix is present, and in any case the pooling window will typically be a small constant). W 1 is constructed as a block diagonal matrix of the weights of the SIGN hidden layer. ∥ W 2 ∥ ∞ is the infinity norm of the weight matrix of the output layer of SIGN, multiplied by 2 (since we compute differnces between numbers of nodes in two classes).

We measure the score difference of the graph classification task in Section 5.2.1 and compare them to the differences of the class sizes in Figure 3. Most of the differences are significant (larger than 1 / poly( N ) where N is the number of nodes; see fig. 3(c)). Some class pairs have low differences making them indistinguishable, however, fig. 3(a),(b) indicate those are typically classes with similar

0

10

20

30

40

Class

0

10

20

30

40

Class

Class size difference ogbn-products test

0

100000

200000

300000

400000

500000

(a)

0

10

20

30

40

Class

0

10

20

30

40

Class

Class score difference ogbn-products test

0

2

4

6

8

(b)

0.0

0.5

1.0

1.5

2.0

Difference

0

5

10

15

20

25

Class pairs count

Class score difference ogbn-products test

(c)

number of nodes. This provides evidence that when there is a considerable class imbalance (i.e. one that scales with system size), the magnitude of the model output when computing this difference will not decay with the size of the graph.

While evidence of asymptotic scaling will require experiments on graphs of different sizes, our results suggest that the upper bound in Lemma 6 is not large for models trained on standard benchmarks, implying that they can be efficiently simulated on a quantum computer.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: See end of introduction for list of contributions and pointers to additional details and proofs.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: See the Discussion section for the main limitations we have identified.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: Full proofs are given in the appendices (mostly Appendix B). Assumptions are stated in Lemma and Theorem statements.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Experiments are explained in Section 5 and Appendix J.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: All datasets are publicly available in DGL library.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Hyperparameters and hyper parameters tuning ranges are mentioned in Appendix J. Data splits are either fixed or identical to the DGL examples default.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: All experimental results include standard deviation.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The used computation resources are mentioned in Appendix J.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer:[Yes]

Justification: The code of ethics has been conformed to.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: This work applies to models of computation that have not yet been realized. It serves to motivate further study of these and will have no societal impact within the next several years.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: No risk of misuse (see broader impacts answer).

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: All datasets are from DGL library.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Not relevant.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: Not relevant.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
comments: SUMMARY:
This paper studies distributed learning and inference based on graph neural networks but with quantum communications between the distributed agents. The authors show that quantum networks reduce the communication complexity inference and gradient computation of distributed models. This reduction is shown to be exponential in communication complexity. The main idea of the paper is to have the data encoded into qubits using amplitude encoding. With that one gets an exponential reduction in space and hence communication complexity. As for the gradient estimation, the problem is shown to be converted to Aaronson's Shadow tomography.

STRENGTHS:
The paper proposes an exponential advantage in communication complexity for the task of distributed inference and gradient estimation. This is a nice set of advantages that can be employed in practical problems. Also the lower bounds are nice contributions.

WEAKNESSES:
The result of the paper is not surprising given the known results in quantum communication complexity. Moreover, the results seem to be directly derived from known papers on Shadow tomography and the Threshold search.

QUESTIONS:
Can you elaborate on the run-time of the proposed method?

LIMITATIONS:
Nothing significant.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2024-07-11

comments: SUMMARY:
The paper investigates the exponential quantum advantage in communication complexity over the tasks of training or inference in multiple types of compositional distributed quantum circuit model. The authors also study the quantum communication advantage in a specific class of shallow graph neural networks, and then they conduct experiments showing that it can achieve performance comparable to certain existing classical models and benchmarks while maintaining the exponential communication advantage. At the end, the authors explore the expressivity of the compositional model.

STRENGTHS:
I think this paper is one of the good examples of applying results and tools from quantum complexity theory to problems in distributed computing.

WEAKNESSES:
(1) Lemma 6 addresses the quantum communication complexity of the quadratic graph network inference problem, for which I think a lower bound would make more sense. However, (I am not sure if I miss anything or not), instead of giving a lower bound, Lemma 6 provides an upper bound, which I find confusing.
(2) The paper assumes that the input is amplitude encoded. It is well-known that preparing such quantum states is generally not efficient. This is an important factor to consider regarding the feasibility of the proposed systems, and thus, it is my main concern with the paper. However, I don't see much formal treatment of this issue in the paper.

QUESTIONS:
My questions are those 2 points in Weaknesses.

LIMITATIONS:
Yes, the authos give a fair discussion on possible limitations.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-15

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the quantum advantage on communication of distributed learning. It places a common quantum neural network model in the (two-party) communication scenario, where the data x is assigned to Alice, and the parameterized unitary operators of each layer are alternately given to Alice and Bob in order.\n\nThe paper shows exponential quantum advantage on communication for problems of estimating the loss function (some function on data after the action of unitary operators) and estimating the gradients of the loss function with respect to each parameter of the unitary operators. The quantum upper bound for estimating the loss function is by encoding data x (size-N vector) into O(log N)-qubit state. The quantum upper bound for estimating the gradients is by a reduction to a shadow tomography problem. The classical lower bound is by a reduction from a problem proposed by Raz, also a round-based lower bound by a reduction from pointer-chasing.\n\nFor comparison, the paper shows that no exponential quantum advantage on communication of linear classification, where Alice and Bob are given data x and y as vectors to determine the sign of inner product of x and y. The classical upper bound is by applying Johnson-Lindenstrauss and the quantum lower bound is by a reduction from gap-Hamming.\n\nThe paper also shows exponential quantum advantage in communcication for quadratic graph network inference, and provides experimental results for the above scenarios.\n\nSTRENGTHS:\n- A new research direction on quantum advantage of communication complexity for learning problems, providing a series of results with solid theoretical proofs and experimental data to support them.\n\nWEAKNESSES:\n- For technical part, the upper bounds are straightforward, and the lower bound reductions look standard.\n- These learning tasks (estimating the loss function in quantum neural networks, linear classification, and quadratic graph network inference) seem to have weak correlations, making the paper appear somewhat like a mere compilation of results.\n\nQUESTIONS:\n- Minor comment: the upper bound in Lemma 1 should be O(L log N) instead of O(log N)?\n\nLIMITATIONS:\n- The claim of QUANTUM advantage on QUANTUM neural networks seems a bit like cheating, although the model and problem are described using classical information.\n- The two-party version of quantum neural networks seems artificial to me. The data and parameters of each single unitary operator might be stored distributively between parties, just like in problem 7 of linear classification, where data x and y are assigned to Alice and Bob.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-23"}