Lie-Equivariant Quantum Graph Neural Networks

Authors: jogi suda neto
roy t forestano
sergei  gleyzer
kyoungchul  kong
konstantin t matchev
katia  matcheva
Venue: NeurIPS_cc_2024_Workshop_MLNCP
Published: 2024-09-12 04:47:22

Abstract: Discovering new phenomena at the Large Hadron Collider (LHC) involves the identification of rare signals over conventional backgrounds. Thus binary classification tasks are ubiquitous in analyses of the vast amounts of LHC data. We develop a Lie-Equivariant Quantum Graph Neural Network (Lie-EQGNN), a quantum model that is not only data efficient, but also has symmetry-preserving properties. Since Lorentz group equivariance has been shown to be beneficial for jet tagging, we build a Lorentz-equivariant quantum GNN for quark-gluon jet discrimination and show that its performance is on par with its classical state-of-the-art counterpart LorentzNet, making it a viable alternative to the conventional computing paradigm.

Main paper: 
 introduction: 
 The onset of the high luminosity stage of the Large Hadron Collider (LHC) [1] by the end of this decade presents a formidable computational challenge to effectively manage and analyze the resulting datasets [2] . A promising new computing paradigm for tackling this problem is the application of quantum machine learning (QML), which offers the possibility of reducing the time complexity of classical algorithms by running on quantum computers, which can have access to the exponentially large Hilbert space [3] [4] [5] [6] [7] [8] [9] .In particle physics, symmetries underpin the fundamental laws governing the interactions and behaviors of subatomic particles: the Standard Model (SM), for example, is built upon symmetry principles such as gauge and Lorentz invariance, which dictate the interactions between particles and fields [10] . At the same time, symmetries play a crucial role in machine learning as well -if a neural network is invariant under some group, then its output could be expressed as functions of the orbits of the group [11] . In recent years, the field of Geometric Deep Learning [12] was born, and it posits that despite the curse of dimensionality, the success of deep learning can be explained by two main inductive biases: symmetry and scale separation. By leveraging inherent symmetries in the data, the hypothesis space is effectively reduced, models can become more compact, and require less data for training -attributes that are particularly critical for quantum computing, given the current hardware limitations.Recognizing the intertwined nature of symmetries and deep learning, a plethora of models have been developed that are invariant to different groups. Convolutional neural networks (CNNs), for example, which have revolutionized computer vision, are naturally invariant to image translations. Transformer-based language models, on the other hand, exhibit permutation invariance. These architectures, when provided with appropriate data, are capable of learning stable latent representations under the action of their respective groups. An interesting question to be explored in quantum machine learning is the use of symmetries [13] [14] [15] [16] [17] , especially in the context of particle physics [18] [19] [20] [21] [22] .Since the data generated by collision events is usually represented in a graph format, graph neural networks (GNNs) [23] are a natural choice in particle physics [24] . Furthermore, Lorentz symmetry is a fundamental spacetime symmetry for any relativistic model of elementary particles. The state of the art implementation of Lorentz symmetry in a GNN is LorentzNet [25] , which has been used for jet tagging, i.e., identifying the progenitor elementary particle which initiates the jet. In this paper we focus on a specific version of the jet tagging problem, namely, discriminating between light quark and gluon jets [26] [27] [28] . We build a Lorentz-equivariant quantum GNN for quark-gluon jet discrimination and show that its performance is on par with its classical state-of-the-art counterpart, LorentzNet. This is an encouraging result, given that quantum computation is still in its infancy stage. In the NISQ era, our focus is on quantum utility, i.e., effectiveness and practicality of quantum computers in terms of speed, accuracy or energy consumption, over a classical machine of similar size, weight, and cost [29] . 
lie-equivariance: 
 Given the abstract definition of Lie groups as used in particle physics [30, 31] , our primary goal is to understand how these symmetries can be embedded in hybrid quantum-classical ML architectures. Mathematically, we consider our data as a set of vectors x sampled from an input space X = R n . We then describe how a group element g ∈ G acts linearly on x through T X (g), where T X : G → GL(n) is known as the group representation. Essentially, T X maps each element of the group G to a nonsingular matrix T X (g) ∈ R n×n . It is important to note that each representation of a Lie group induces a corresponding representation in its Lie algebra.The idea of symmetry-preserving behavior in machine learning is essential. By reducing the hypothesis space, the learning process becomes more efficient. Additionally, models that integrate geometric information as an inductive bias tend to be naturally data-efficient, eliminating the need for data augmentation during training [13-17, 25, 32-35] . This is particularly important in Quantum Machine Learning (QML), where the limited number of qubits makes data efficiency crucial [18, 19] .For a given function f : x ∈ X → y ∈ Y (for example, f could be a neural network, X the input space, and Y a latent space) and some group G that acts on both X and Y, the f holds the desirable property of equivariance if ∀g ∈ G, (x, y) ∈ D, T y (g)y = f (T x (g)x). For brevity of notation, we write it as:g • y = f (g • x).It becomes clear that the invariance is a special case of equivariance where f (g • x) = f (x), that is, T y (g) = I, so every group element acting in the space of Y gets mapped to the identity. To incorporate invariance into the architectures, it is essential to first identify the group G that accounts for the underlying transformations of the data. In our context, these groups are Lie groups. In particular we will consider the Lorentz group, which is a fundamental symmetry of the standard model of particle physics. Our objective is to build a Quantum Algorithm for Lorentz-Equivariant Graph Neural Network for gluon/quark discrimination. 
dataset: 
 In this work, we consider the task of determining whether a given jet originated from a quark or a gluon (this is known as jet-tagging). For illustration we use the high energy physics dataset Pythia8 Quark and Gluon Jets for Energy Flow [36] , which contains two million jets split equally into one million quark jets and one million gluon jets. These jets resulted from simulated LHC collisions with total center of mass energy √ s = 14 TeV and were selected to have transverse momenta p jet T between 500 to 550 GeV and rapidities |y jet | < 1.7. For our analysis, we randomly picked N = 12500 jets and used the first 10000 for training, the next 1250 for validation, and the last 1250 for testing. These sets happened to contain 4982, 658, and 583 quark jets, respectively.For our purposes, we consider the jet dataset to be constituted of point-clouds, where each jet is represented as a graph G = {V, E}, i.e., a set of nodes and edges, respectively. (This is the natural data structure used by Graph Neural Networks.) In our case, each node has the transverse momentum p T , pseudorapidity η, azimuthal angle ϕ (and other scalar-like quantities like particle ID, particle mass, etc.) of the respective constituent particle in the jet (see Figure 1 ). Generally, the number of features is always constant, but the number of nodes in each jet may vary. For our analysis we used jets with at least 10 particles. 
graph neural networks: 
 Graph Neural Networks are a class of neural networks designed to operate on graph-structured data. Unlike traditional neural networks that work on Euclidean data such as images or sequences, GNNs are capable of capturing the dependencies and relationships between nodes in a graph. The fundamental idea behind GNNs is to iteratively update the representation of each node by aggregating information from its neighbors, thereby enabling the network to learn both local and global graph structures.Mathematically, a GNN can be described as follows: Let G = (V, E) be a graph where V is the set of nodes and E is the set of edges. Each node v ∈ V has an initial feature vector h (0) v . The node features are updated through multiple layers of the GNN. At each layer l, the feature vector of node v is updated by aggregating the feature vectors of its neighbors N (v) and combining them with its own feature vector. This can be expressed as:h (l+1) v = σ W (l) h (l) v + u∈N (v) W (l) e h (l) uwhere W (l) and W (l)e are learnable weight matrices, σ is a non-linear activation function, and N (v) denotes the set of neighbors of node v. This process is repeated for a fixed number of layers, allowing the network to capture higher-order neighborhood information. The final node representations can be used for various downstream tasks such as node classification, link prediction, or graph classification.As a simple example of equivariance in the context of classical GNNs, consider a dataset of graphs in which the nodes have cartesian coordinates as input features, and let SO(2) be their underlying group. Since the Euclidean norm -induced by the euclidean metric -is invariant to rotations, any node-updating function of the form:EQUATIONis naturally equivariant (for the proof, see Appendix A in [19] ). Here ϕ x is some classical neural network and m l ij is the edge message between nodes i and j in layer l. See Ref. [25] for different classical neural networks, ϕ e , ϕ m and ϕ h .Equivariant GNNs (EGNNs) can be applied to other groups of interests, like the Lorentz group, a central piece in high energy physics [25] .Particles observed at the LHC are moving at velocities comparable to the speed of light, and are therefore described by special relativity. Mathematically, the transformations between two inertial frames of reference are described by the Lorentz group O(1, 3).Having established that classical GNNs aim to learn a lower-dimensional, topologically preserving representation of graphs, we now explore quantum graph neural networks (QGNNs). One approach for QGNNs involves encoding the graph's nodes and edges into any Hamiltonian whose topology of interactions is that of the problem graph [38] . Previous works [19, 38] have utilized the transversefield Ising model, initially introduced to study phase transitions in magnetic systems. Once the graph is encoded in the Hamiltonian, it is transformed into a quantum circuit using a Trotter-Suzuki approximation [39] .This method naturally yields a permutation-equivariant QGNN via the Ising Hamiltonian. As discussed, the symmetries from quantum fields, describing the invariance of physical laws under different inertial frames, are represented by the Lorentz group. Prior research outlines efficient strategies for achieving equivariance. For instance, [40] demonstrates that an invariant model can be constructed using an equivariant set of quantum gates. However, this framework fundamentally assumes the presence of either discrete or compact Lie groups.Given that the Lorentz group is noncompact, here we adopt an alternative method based on Invariant Theory [41, 42] , which is different from the strategy to achieve equivariance explained before [40] . In addition, our approach for QGNNs is different from the one used in [19, 38] . It has previously been applied in the classical setting for High-Energy Physics [25] , and is the main motivation for our work.Similarly to LorentzNet, for standard jet tagging approach, our input is made of 4-momentum vectors and any associated particle scalar one may wish to include, like color and charge. We start with the traditional LorentzNet architecture (Fig. 1 in Ref. [25] ), with the following modifications: ϕ e , ϕ x , ϕ h , and ϕ m , i.e., the classical multilayer perceptrons in LorentzNet, are substituted by quantum counterparts, that is, variational circuits, whose architecture is illustrated below in Fig. 2 :ENCODING U 1 (θ) U L (θ) |0⟩ . . . |0⟩ . . . |0⟩ . . . |0⟩ . . . |0⟩ . . . |0⟩ . . . H R Y (x 1 ) R Y (θ 1 ) R Y (θ 7 ) H R Y (x 2 ) R Y (θ 2 ) R Y (θ 8 ) H R Y (x 3 ) R Y (θ 3 ) R Y (θ 9 ) H R Y (x 4 ) R Y (θ 4 ) R Y (θ 10 ) H R Y (x 5 ) R Y (θ 5 ) R Y (θ 11 ) H R Y (x 6 ) R Y (θ 6 ) R Y (θ 12 )Figure 2 : The ansatz used in our work, which consists of a unitary angle encoding followed by L = 2 trainable variational layers, which in turn consist each of entangling and parameterized RY rotations for each qubit. Lorentz Equivariant Quantum Block (LEQB) is the main piece of our model (see Fig. 3 ). We aim to fundamentally learn deeper quantum representations of x l and h l ,EQUATIONEQUATIONwhere x l are the 4-momentum observables and h l are the particle scalars in the input layer l = 0, and m ij is the message between particles i and j, defined byEQUATIONwhere ||x i -x j || 2 = η(x i -x j , x i -x j ) is the squared Minkowski norm, and x l i , x l j is the Minkowski inner product, defined by the bilinear form ⟨x i , x j ⟩ = x T i ηx j , where η = diag(-1, 1, 1, 1) is the Minkowski metric. Finally, ψ is just a normalization constant for large distributions to improve training stability, as defined in [25] . 
experiments & results: 
 After substituting ϕ e , ϕ x , ϕ h , and ϕ m for parameterized circuits one at a time, we find the results (accuracy and losses over training epochs) shown in the first four panels in Figures 4 and 5 . When all four modules ϕ e , ϕ x , ϕ h and ϕ m are represented with quantum circuits, we find the results shown in the lower left panels in Figures 4 and 5 . Finally, the results in the lower right panels correspond to the fully classical model LorentzNet [25] . In all experiments, we used AdamW [43] as our optimizer with a learning rate γ = 10 -3 and weight decay of λ = 10 -2 . Our simulations are idealized in the sense that we ignore any noise and assume an infinite number of shots. For warming up, we used a combination of cosine annealing [44] and gradual warm-up [45] . The total number of trainable parameters in all models can be seen in Table 1 .  
conclusion: 
 Our study demonstrates that the performance of Lie-EQNNs is comparable to or slightly better than that of their classical counterparts, LorentzNet. This highlights the potential of quantum-inspired architectures in resource-constrained settings. The code used for this study is publicly available at https://github.com/ML4SCI/QMLHEP/tree/main/Lie_EQGNN_for_HEP_Jogi_Suda_Neto.  
 
 Appendix:  



Reviews: review: title: title: The authors choose a promising task and choose a well-motivated circuit design with reasonable classical comparisons.  However, more work needs to be done to motivate the significance of this result, as well as clarify some vague points in the paper
main_review: main_review: The authors decide to apply quantum machine learning to a problem from particle physics - a source of physical data for which a quantum-based architecture might well be naturally suited. Further, they take the novel inspiration from geometric deep learning to choose the inductive bias of their quantum circuit to respect the symmetry of the Lorentz group. With commendable due diligence, the authors also compare their results to a classical neural network that is structured to respect the same symmetry.

Major Comments:

One of the authors' key results is that quantum networks (LEQNN's) empirically appear to perform about as well as LorentzNet. An explanation of the significance of this is missing. Is achieving comparable performance to LorentzNet something that is very difficult for most classical architectures? Is the fact that it performs even equally well surprising? If so, the proper context needs to be provided to justify this. 
Further, it is unclear that comparable or marginally better performance is sufficient to motivate this, considering the additional costs associated with training quantum neural networks - hardware noise, shot noise, as well as the efficiency of gradient-based training for the quantum neural network (e.g. only known case of general backprop scaling in gradient computation requires multiple copies of the circuit whereas this can be easily implemented digitally -important considering the number of parameters!). This is to say nothing for the costs of having a quantum computer on hand to begin with. Therefore, at least *some* empirical evidence  is needed (perhaps numerics with deeper circuits) that significantly better performance is possible (say, 80% vs 67% accuracy), without encountering issues such as barren plateaus, for quantum circuits over LorentzNets with similar depths. 

Additionally, given the marginal improvements seen, it's important to clarify: did the training of the quantum system assume a noiseless implementation in the infinite shot limit? Once the shot noise, or even hardware noise, of NISQ devices is approximated, how does this influence results?

Finally, I think it would be reasonable to leave the door open to negative results, which are comparably valuable. In particular, is there a good reason to suppose that a quantum system with this architecture will always perform, at best, only marginally better than something like LorentzNet (perhaps the latter already captures most of the relevant info)?


Minor Comments:

While the authors list the number of parameters used, it is unclear to me how many layers are used in their experiments.  From Figure 2, it seems that there are only two layers - but such a circuit is shallow enough to be easily classically simulable for any number of qubits, given that they seem to measure *local* observables. My instinct would be to divide the number of parameters by the number of qubits...but it seems that a measurement takes place immediately after the two layers are applied? Do the authors not reset between measurements, and that's how additional layers are applied? I would ask for clarification on this, both in the text and with a clearer image for layers. 


The authors go into quite a bit of detail on the virtues on lie-equivariant neural networks, but don't really go into much detail as to why their quantum circuit architecture is equivariant under the Lorentz group. I would request adding a section elaborating on this (perhaps in lieu of the current section on quantum graph neural networks, which seems largely tangential to the circuit architecture presented).
title: title: Good motivation and methodology.  Probably a decent incremental result.
main_review: main_review: This paper is about symmetric NNs, applied to HEP problems involving LHC data.  The motivation here is the use of GNNs for performing inference on this data, and the potential of using quantum GNNs instead of classical ones, with the appropriate symmetries (Lie equivariance).

A few criticisms:
* Is there a reference for the concept of quantum GNNs, or is this paper the first one to do so?  Previous quantum GNN papers were not cited, to my knowledge.
* Why is this related to an Ising model?  I know what an Ising model is, but that does not usually come up in the context of GNNs.  I am wondering whether they are using the Ising model just because that is one of the things that quantum computers can easily simulate.
* What is LEQB, the thing in Fig. 2?  Unclear.
* Where was this experiment run, on a physical quantum computer or a simulator?  If so, which?

Good they make their code public.

Evaluation: probably a publishable incremental result.  30-50th percentile.