Do graph neural networks learn traditional jet
substructure?
Farouk Mokhtar, Raghav Kansal, Javier Duarte
University of California San Diego
La Jolla, CA 92093, USA{fmokhtar,rkansal,jduarte}@ucsd.edu

Abstract
At the CERN LHC, the task of jet tagging, whose goal is to infer the origin of ajet given a set of final-state particles, is dominated by machine learning methods.
Graph neural networks have been used to address this task by treating jets as point
clouds with underlying, learnable, edge connections between the particles inside.
We explore the decision-making process for one such state-of-the-art network,
ParticleNet, by looking for relevant edge connections identified using the layerwiserelevance propagation technique. As the model is trained, we observe changes
in the distribution of relevant edges connecting different intermediate clustersof particles, known as subjets. The resulting distribution of subjet connections
is different for signal jets originating from top quarks, whose subjets typicallycorrespond to its three decay products, and background jets originating from lighter
quarks and gluons. This behavior indicates that the model is using traditional
jet substructure observables, such as the number of prongs—energetic particle
clusters—within a jet, when identifying jets.
Introduction The CERN LHC is an enormous data generation machine: producing about one
petabyte of sensor-level collision data per second [1]. This collision data passes a series of online (andoffline) filtering through complex trigger systems optimized for data reduction to achieve reasonable
storage capacity for further analyses. Machine learning (ML) algorithms are implemented throughout
the entire LHC workflow, from the early steps of data reduction and compression at the level-1 trigger
for online filtering [2–4] to full-scale physics analyses [5]. In particular, graph neural networks(GNNs) are found to be performant for a wide variety of tasks at the LHC [6, 7] since collider data
can be optimally described, in many cases (for instance when it comes to jets), as a point cloud.
Jets are showers of particles initiated by quarks and gluons, and jet substructure [8, 9] studies theradiation pattern of those jets. A particular observable of interest when identifying jets is the numberof prongs—energetic clusters of particles—within a jet. For example, it is common for a hadronicallydecaying top quark jet to exhibit a three-prong substructure.
Examples of jet tasks that can be addressed with GNNs include jet simulation [10], jet clustering [11–13], jet mass regression [14], and jet classification (or jet tagging) [15–18], which is the subject of
this paper. ParticleNet [17] is a state-of-the-art GNN developed to address the problem of jet tagging
by treating jets as point clouds with underlying, learnable, edge connections between the particles.
ML algorithms are often referred to as black boxes because their behavior is not easily interpretable.
A collection of techniques, called explainable artificial intelligence (XAI) [19], attempt to explain thedecision-making of ML models. One such technique is layerwise relevance propagation (LRP) [20,21], which relies on computing and assigning relevancy (R) scores to neurons of an ML model tomeasure their influence on the prediction. Explaining ParticleNet with LRP is interesting as we canassign R scores to the learned edges connecting the particle nodes in the point cloud. These edge R
scores highlight particular connections between particles that were found to be the most influential on
Machine Learning and the Physical Sciences workshop, NeurIPS 2022.

the final output, as the model learns to discriminate jets. This helps us understand the extent to which
the model is utilizing the physics we know, for example the number of prongs, when identifying jets.
Related Work ML models implemented at the LHC tend to lack physics-informed inductive
bias that are built into alternative algorithms, which makes interpretability studies on such models
especially important [22–24]. Ref. [25] attempts to explain the machine-learned particle-flow [26, 27]algorithm, which is a GNN developed for the task of particle-flow reconstruction in CMS. We build
upon this work to extend the LRP technique for ParticleNet, which makes use of specific graphconvolutions.
At its core, the LRP technique relies on simple operations that systematically redistribute the output
prediction of an ML model over neurons in each layer. The result of the redistribution is an R score
per neuron that reflects its importance to the overall prediction. GNNs naturally impose a challenge
to the application of LRP due to their inherent complexity. The attempts to adapt LRP for GNNs
include the GNN-LRP [28] and GLRP methods [29]. Since each GNN model realizes different, andpossibly unique, operations, it is hard to find an implementation that fits all.
LRP applied to ParticleNet We train a ParticleNet model on the benchmark top quark jet tagging
dataset [30] found on the Zenodo platform [31] under the CC BY 4.0 license. Our code is also
publicly available under the MIT license [32]. We choose top quark jets as our signal in particular
because such jets are known to have distinct features. For example, we investigate whether the model
makes use of the three-pronged substructure of top quark jets to distinguish them from lighter quarkor gluons jets (collectively referred to as quantum chromodynamic (QCD) jets), which are typicallysingle-pronged [8, 9, 33].
The dataset is composed of jets from 14 TeV collisions, corresponding either to hadronically decaying
top quarks for signal or QCD dijets for background. The dataset is generated using PYTHIA 8 [34]and simulated using the DELPHES [35] without multiple parton interactions and without pileup.
Particle-flow entries are clustered using the anti-kT algorithm [36] implemented in the FASTJET
package [37] with a radius parameter of R = 0.8. Jets with transverse momentum (pT ) between 550
and 650 GeV are selected. All top quark jets are matched to a parton-level top quark within a distanceof ∆R = 0.8, and to all top quark decay partons within ∆R = 0.8. Jets are required to have |η| < 2.
The 200 pT -leading jet constituent four-momenta are accessible, sorted by decreasing pT , and are fed
as input to our ParticleNet model. We build the model in PyTorch Geometric (PyG) [38] which has
the advantage of handling variable sized graphs. This means that our input jets can have an arbitrary
number of jet constituents. For example, a jet with 60-constituents will correspond to an input withdimensions [60, 4], containing the four-momenta (px , py , pz , E) of each constituent.
The ParticleNet model primarily makes use of the EdgeConv block consisting of dynamic graphconvolutions proposed in Ref. [39] to learn the graph structure. Following the same choice proposedin the ParticleNet paper, we choose to have three EdgeConv blocks in our model. The EdgeConv
block builds a graph using the k-nearest neighbor (kNN) technique. The model then invokes a fullyconnected deep neural network to learn edge weights to be assigned to each edge connection. This isfollowed by a message-passing step where node features are updated using a pooling operation on thelearned edge weights, followed by a pooling operation on all nodes, and lastly, a final fully-connectedneural network layer. The model outputs a single number, followed by a sigmoid activation function.
From the LRP standpoint, the operations involved in the EdgeConv block can all be cast as matrix
operations that imitate a basic multilayer perceptron operation. This makes it straightforward touse the standard LRP rules after some minimal manipulations. In Figure 1 we illustrate how theredistribution of R scores occurs from a single node to the surrounding edges to obtain the edge R
scores. The edge R score Rijk for a given edge eijk connecting node xi with its kth neighboringnode xjk is computed using the formula:eij
Rijk = PK k Ri
(1)′
k′ eijk
where K denotes the number of neighbors of node xi .
Results We train a ParticleNet model for further LRP tests (see Appendix A for details on the
training parameters). We run the LRP evaluation on 10,000 jet samples that were reserved for testing
during the ParticleNet model training. The evaluation takes 2.5 hours on an Apple M1 Pro chip.
2

eij4

xj4

Rij4

xj3

Rij1

Ri xi

Rij3

eij3

xj1

eij1

eij4

xj4

R score redistribution
from Node to Edge

Rij2

eij2

Ri xi
eij3

xj2

xj1

eij1

xj3

eij2

xj2

Figure 1: Redistribution of the R scores of node xi over the edges connecting the node with its K
neighbors labeled by xjk to obtain the edge R scores Rijk .

In addition, we also run the LRP code on 10 randomly initialized untrained models for baselinecomparison with the trained model. For each jet, we propagate the R score of the output node
backwards, and are able to obtain an intermediate relevancy-graph for each graph constructed in an
EdgeConv block. We refer to these as “edge R graphs”.
Edge R graphs The edge R graph represents the jet constituents as nodes in (η, ϕ) space andinterconnections as edges, whose intensities correspond to the respective edge R score. Each node’s
intensity corresponds to the relative pT of the corresponding particle. In principle, we can propagate
R scores backwards until we reach the input layer, but we find this unnecessary because we focus on
the GNN’s learned graph connectivity. Edge R graphs can be constructed for each EdgeConv block.
Note that every block constructs a new graph using the kNN technique, so we expect the adjacency
matrix to change with each block.
Looking at sample edge R graphs, we observe that by the last EdgeConv block, the model is learning
to connect nodes farther apart in (η, ϕ) space. This happens more often for top quark than for QCD
jets, and such connections have high edge relevancy scores. We present sample edge R graphs inAppendix B for both top quark and QCD jets. We aggregate over many jets by exploring how the
distribution of the ∆R of the five most relevant edges in the last EdgeConv block changes as themodel trains. From Figure 2, we see that the model learns to emphasize higher ∆R edges for top
quark jets compared to QCD jets. The results shown for the randomly initialized untrained models
act as a baseline to compare with the trained model [40]. If the LRP method depends on the learned
parameters of the model, we expect its output to differ substantially between the two cases, indicating
what the model has learned. Moreover, since the top quark and QCD distributions for an untrainedmodel are nearly identical, there is nothing intrinsic in the dataset that biases the model toward
connecting differently-separated edges between the two classes.
This raises the question of whether the model is learning jet substructure, and whether it is using
observables such as the number of prongs within a jet, when identifying top quark jets. We attempt toanswer this question by looking at the fraction of relevant edges that connect particles from different
intermediate clusters, or subjets. We use the Cambridge–Aachen (CA) algorithm [41, 42] to decluster
the jets into exactly three subjets. We choose the CA algorithm because the metric it uses to cluster
particles relies purely on their spatial positions and not their momenta. It has also been used inprevious studies of top quark jets [43]. We use the FASTJET Python package to first recluster each jet
inclusively using the CA algorithm with a radius parameter of R = 0.8. Then, we trace backward
through the history of the sequential combinations until we find exactly three subjets, i.e. the final
subjets to be combined by the CA algorithm.
In Figure 3, we observe that with each learned graph at each EdgeConv block, the model learns torely more on edges connecting different subjets when discriminating top quark from QCD. For the
first EdgeConv block in Figure 3 (left), only the nearest neighbors in (η, ϕ) space are connected,
while for the third EdgeConv block in Figure 3 (right) more long-distance connections between the
different CA subjets are present.
To aggregate over many jets and extract a statistically meaningful result, we explore the fractionof edges connecting different subjets Nedges between subjets among the N most relevant edges Nedges .
3

Top
QCD

100
10 1
0.0

0.5

1.0

101

Normalized counts

Normalized counts

Untrained model

101

Trained model
Top
QCD

100
10 1
10 2
10 3
0.0

1.5

R of the 5 most relevant edges

0.5

1.0

1.5

R of the 5 most relevant edges

Figure 2: We present the distribution of the ∆R of the five most relevant edges for top quark jets
(blue) versus QCD jets (orange) for an untrained ParticleNet model (left) and the learned distribution
by a trained ParticleNet model (right). The result for the untrained model is an average over 10
randomly initialized models.

Figure 3: The three edge R graphs for a true top quark jet corresponding to the three graphs learned
with each EdgeConv block. The jet constituents are represented as nodes in (η, ϕ) space withinterconnections as edges, whose intensities correspond to the connection’s edge R score. Eachnode’s intensity corresponds to the relative pT of the corresponding particle. Constituents belonging
to the three different CA subjets are shown in blue, red, and green in descending pT order. We
observe that by the last EdgeConv block the model learns to rely more on edge connections between
the different subjets.

We scan different values of N and we test this for 10 untrained ParticleNet models, and a trained
ParticleNet model. The results are shown in Figure 4.
From Figure 4, we can see that an untrained model treats both top quark and QCD jets the same,
however, as it trains, it learns to connect and rely more on edges connecting different subjets when it
is identifying top quark jets compared to QCD. This indicates that the model is learning the differencein the respective substructure, and is learning to use observables such as the number of prongs, whenidentifying jets.
Summary We attempt to explain the decision-making process of the ParticleNet model for top
quark jet tagging, and find that the model learns different graph connectivity for discriminating top
quark jets from QCD background. We observe that, on average, the most relevant edges learned for
top quark jets have larger ∆R separation than for QCD jets. This motivates us to further explore
the distribution of relevant edges connecting different subjets. We find that as the model trains, itlearns to rely more on edges connecting different subjets when classifying top quark jets compared to
QCD jets, indicating that the model is learning the difference in the respective substructure, and islearning to use observables such as the number of prongs, when identifying jets. This simple result
motivates us to further explore how the relevant edges correspond to known jet substructure variables,
like N -subjettiness [33], energy correlation functions [44, 45], and energy flow polynomials [46].
4Untrained model
Top
QCD

0.8
0.6
0.4
0.2
0.0 0

3

6

9

12

Nedges between subjets / Nedges

Nedges between subjets / Nedges

1.0

Nedges1.0

Trained model
Top
QCD

0.8
0.6
0.4
0.2
0.0 0

3

6

9

12

NedgesFigure 4: The fraction of edges connecting different subjets Nedges between subjets among the N most
relevant edges Nedges , for different values of N . Results are shown averaged over 10 randomly
initialized untrained ParticleNet models (left) and for the trained ParticleNet model (right).

Limitations One potential limitation is to what degree the results shown in this work depend on
the chosen subjet finding algorithm. Another matter of concern is the robustness of the model, since
LRP (and most XAI techniques) rely on sensitive changes to the input and their effect on the output.
Works such as Ref. [47] have previously shown that it is mathematically impossible to have both
attribution-based explanations (that provide recourse) and robustness at the same time. Nevertheless,
it is still valuable to explore the interpretability of widely used models.

Broader Impact Attempts to explain and interpret ML algorithms are useful in general to increase
confidence in the model prediction as well as to gain insights on making models more performant
and robust. Interpretability studies are especially important for ML models implemented at the
LHC because such models tend to lack physics-informed inductive bias that are built into alternativealgorithms. This effort explores the application of an XAI method on a state-of-the-art GNN model
developed for jet tagging at the LHC, and attempts to show that the model is indeed learning usefulphysics that is already being exploited in physics-designed alternatives. This opens the door for
potentially extracting information and relationships exploited by the ML model that may not be fully
utilized by such alternative algorithms.

Acknowledgments and Disclosure of Funding
F. M. is supported by an Halıcıoğlu Data Science Institute (HDSI) fellowship. F. M., R. K., andJ. D. are supported by the US Department of Energy (DOE), Office of Science, Office of HighEnergy Physics Early Career Research program under Award No. DE-SC0021187, the DOE, Officeof Advanced Scientific Computing Research under Award No. DE-SC0021396 (FAIR4HEP), andthe NSF HDR Institute for Accelerating AI Algorithms for Data Driven Discovery (A3D3) under
Cooperative Agreement OAC-2117997. R. K. is also supported by the LHC Physics Center at
Fermi National Accelerator Laboratory, managed and operated by Fermi Research Alliance, LLC
under Contract No. DE-AC02-07CH11359 with the DOE. This work was performed using the
Pacific Research Platform Nautilus HyperCluster supported by NSF awards CNS-1730158, ACI1540112, ACI-1541349, OAC-1826967, the University of California Office of the President, and the
University of California San Diego’s California Institute for Telecommunications and Information
Technology/Qualcomm Institute. Thanks to CENIC for the 100 Gpbs networks. Funding for cloudcredits was supported by NSF Award #1904444 Internet2 Exploring Clouds to Accelerate Science(E-CAS).
5

References[1] M. Butler, R. Mount, and M. Hildreth, “Working Group Report: Storage and DataManagement”, in Community Summer Study 2013: Snowmass on the Mississippi. 2013.
arXiv:1311.4580.
[2] J. Duarte et al., “Fast inference of deep neural networks in FPGAs for particle physics”, J.
Instrum. 13 (2018) P07027, doi:10.1088/1748-0221/13/07/P07027,
arXiv:1804.06913.
[3] CMS Collaboration, “The Phase-2 upgrade of the CMS Level-1 trigger”, CMS Technical
Design Report CERN-LHCC-2020-004. CMS-TDR-021, 2020.
[4] A. M. Deiana et al., “Applications and Techniques for Fast Machine Learning in Science”,
Front. Big Data 5 (2022) 787421, doi:10.3389/fdata.2022.787421,
arXiv:2110.13041.
[5] D. Guest, K. Cranmer, and D. Whiteson, “Deep Learning and its Application to LHC Physics”,
Ann. Rev. Nucl. Part. Sci. 68 (2018) 161–181,
doi:10.1146/annurev-nucl-101917-021019, arXiv:1806.11484.
[6] J. Shlomi, P. Battaglia, and J.-R. Vlimant, “Graph Neural Networks in Particle Physics”, Mach.
Learn.: Sci. Technol. 2 (2021) 021001, doi:10.1088/2632-2153/abbf9a,
arXiv:2007.13681.
[7] J. Duarte and J.-R. Vlimant, “Graph neural networks for particle tracking and reconstruction”,
in Artificial Intelligence for Particle Physics. World Scientific Publishing, 2020.
arXiv:2012.01249. Submitted to Int. J. Mod. Phys. A. doi:10.1142/12200.
[8] A. J. Larkoski, I. Moult, and B. Nachman, “Jet Substructure at the Large Hadron Collider: A
Review of Recent Advances in Theory and Machine Learning”, Phys. Rept. 841 (2020) 1,
doi:10.1016/j.physrep.2019.11.001, arXiv:1709.04464.
[9] R. Kogler et al., “Jet Substructure at the Large Hadron Collider: Experimental Review”, Rev.
Mod. Phys. 91 (2019), no. 4, 045003, doi:10.1103/RevModPhys.91.045003,
arXiv:1803.06991.
[10] R. Kansalet al., “Particle cloud generation with message passing generative adversarial
networks”, in Advances in Neural Information Processing Systems, volume 34. Curran
Associates, Inc., 2021. arXiv:2106.11535.
[11] S. R. Qasim, J. Kieseler, Y. Iiyama, and M. Pierini, “Learning representations of irregularparticle-detector geometry with distance-weighted graph networks”, Eur. Phys. J. C 79 (2019)
608, doi:10.1140/epjc/s10052-019-7113-9, arXiv:1902.07987.
[12] J. Kieseler, “Object condensation: one-stage grid-free multi-object reconstruction in physics
detectors, graph and image data”, Eur. Phys. J. C 80 (2020), no. 9, 886,
doi:10.1140/epjc/s10052-020-08461-2, arXiv:2002.03605.
[13] J. Guo, J. Li, T. Li, and R. Zhang, “Boosted Higgs boson jet reconstruction via a graph neuralnetwork”, Phys. Rev. D 103 (2021), no. 11, 116025, doi:10.1103/PhysRevD.103.116025,
arXiv:2010.05464.
[14] CMS Collaboration, “Mass regression of highly-boosted jets using graph neural networks”,
CMS Detector Performance Note CMS-DP-2021-017, 2021.
[15] E. A. Moreno et al., “JEDI-net: a jet identification algorithm based on interaction networks”,
Eur. Phys. J. C 80 (2020) 58, doi:10.1140/epjc/s10052-020-7608-4,
arXiv:1908.05318.
[16] E. A. Moreno et al., “Interaction networks for the identification of boosted H → bb decays”,
Phys. Rev. D 102 (2020) 012010, doi:10.1103/PhysRevD.102.012010,arXiv:1909.12285.
[17] H. Qu and L. Gouskos, “ParticleNet: Jet Tagging via Particle Clouds”, Phys. Rev. D 101(2020) 056019, doi:10.1103/PhysRevD.101.056019, arXiv:1902.08570.
[18] V. Mikuni and F. Canelli, “ABCNet: An attention-based method for particle tagging”, Eur.
Phys. J. Plus 135 (2020), no. 6, 463, doi:10.1140/epjp/s13360-020-00497-3,
arXiv:2001.05311.
6

[19] W. Samek et al., eds., “Explainable AI: Interpreting, Explaining and Visualizing Deep
Learning”. Springer International Publishing, Cham, Switzerland, (2019).
doi:10.1007/978-3-030-28954-6.
[20] S. Bach et al., “On pixel-wise explanations for non-linear classifier decisions by layer-wise
relevance propagation”, PLOS ONE 10 (2015) e0130140,
doi:10.1371/journal.pone.0130140.
[21] G. Montavon et al., “Layer-wise relevance propagation: An overview”, in Samek et. al. [19],p. 193. doi:10.1007/978-3-030-28954-6_10.
[22] G. Agarwal et al., “Explainable AI for ML jet taggers using expert variables and layerwise
relevance propagation”, JHEP 05 (2021) 208, doi:10.1007/JHEP05(2021)208,
arXiv:2011.13466.
[23] Y. S. Lai, D. Neill, M. Płoskoń, and F. Ringer, “Explainable machine learning of the underlying
physics of high-energy particle collisions”, arXiv:2012.06582.
[24] M. S. Neubauer and A. Roy, “Explainable AI for High Energy Physics”, in 2022 Snowmass
Summer Study. 2022. arXiv:2206.06632.
[25] F. Mokhtaret al., “Explaining machine-learned particle-flow reconstruction”, in 4th Machine
Learning and the Physical Sciences Workshop at the 35th Conference on Neural Information
Processing Systems. 2021. arXiv:2111.12840.
[26] J. Pata et al., “MLPF: Efficient machine-learned particle-flow reconstruction using graph neuralnetworks”, Eur. Phys. J. C 81 (2021), no. 5, 381,
doi:10.1140/epjc/s10052-021-09158-w, arXiv:2101.08578.
[27] J. Pata et al., “Machine Learning for Particle Flow Reconstruction at CMS”, in 20th
International Workshop on Advanced Computing and Analysis Techniques in Physics Research:
AI Decoded - Towards Sustainable, Diverse, Performant and Effective Scientific Computing.
2022. arXiv:2203.00330.
[28] T. Schnake et al., “Higher-order explanations of graph neural networks via relevant walks”,
arXiv:2006.03589.
[29] H. Chereda et al., “Explaining decisions of graph convolutional neural networks:
patient-specific molecular subnetworks responsible for metastasis prediction in breast cancer”,
Genome Medicine 13 (2021), no. 1, 42, doi:10.1186/s13073-021-00845-7.
[30] A. Butter et al., “The Machine Learning landscape of top taggers”, SciPost Phys. 7 (2019) 014,
doi:10.21468/SciPostPhys.7.1.014, arXiv:1902.09914.
[31] G. Kasieczka, T. Plehn, J. Thompson, and M. Russel, “Top quark tagging reference dataset”,
2019. doi:10.5281/zenodo.2603256, https://doi.org/10.5281/zenodo.2603256.
[32] F. Mokhtar, R. Kansal, and J. Duarte, “xai4hep toolbox”, 2022.
doi:10.5281/zenodo.7266537, https://github.com/farakiko/xai4hep.
[33] J. Thaler and K. Van Tilburg, “Identifying Boosted Objects with N-subjettiness”, JHEP 03(2011) 015, doi:10.1007/JHEP03(2011)015, arXiv:1011.2268.
[34] T. Sjöstrand et al., “An introduction to PYTHIA 8.2”, Comput. Phys. Commun. 191 (2015) 159,
doi:10.1016/j.cpc.2015.01.024, arXiv:1410.3012.
[35] DELPHES 3 Collaboration, “DELPHES 3, a modular framework for fast simulation of a genericcollider experiment”, JHEP 02 (2014) 057, doi:10.1007/JHEP02(2014)057,
arXiv:1307.6346.
[36] M. Cacciari, G. P. Salam, and G. Soyez, “The anti-kT jet clustering algorithm”, JHEP 04(2008) 063, doi:10.1088/1126-6708/2008/04/063, arXiv:0802.1189.
[37] M. Cacciari, G. P. Salam, and G. Soyez, “FASTJET user manual”, Eur. Phys. J. C 72 (2012)
1896, doi:10.1140/epjc/s10052-012-1896-2, arXiv:1111.6097.
[38] M. Fey and J. E. Lenssen, “Fast graph representation learning with PyTorch Geometric”, in
ICLR Workshop on Representation Learning on Graphs and Manifolds. 2019.
arXiv:1903.02428.
[39] Y. Wang et al., “Dynamic graph CNN for learning on point clouds”, ACM Trans. Graph. 38(2019) doi:10.1145/3326362, arXiv:1801.07829.
7

[40] J. Adebayoet al., “Sanity checks for saliency maps”, in Advances in Neural Information
Processing Systems, S. Bengio et al., eds., volume 31. Curran Associates, Inc., 2018.
arXiv:1810.03292.
[41] Y. L. Dokshitzer, G. D. Leder, S. Moretti, and B. R. Webber, “Better jet clustering algorithms”,
JHEP 08 (1997) 001, doi:10.1088/1126-6708/1997/08/001, arXiv:hep-ph/9707323.
[42] M. Wobisch and T. Wengler, “Hadronization corrections to jet cross-sections in deep inelastic
scattering”, in Workshop on Monte Carlo Generators for HERA Physics (Plenary Starting
Meeting), p. 270. 1998. arXiv:hep-ph/9907280.
[43] CMS Collaboration, “A Cambridge-Aachen (C-A) based Jet Algorithm for boosted top-jet
tagging”, CMS Physics Analysis Summary CMS-PAS-JME-09-001, 2009.
[44] A. J. Larkoski, G. P. Salam, and J. Thaler, “Energy Correlation Functions for Jet Substructure”,
JHEP 06 (2013) 108, doi:10.1007/JHEP06(2013)108, arXiv:1305.0007.
[45] I. Moult, L. Necib, and J. Thaler, “New Angles on Energy Correlation Functions”, JHEP 12(2016) 153, doi:10.1007/JHEP12(2016)153, arXiv:1609.07483.
[46] P. T. Komiske, E. M. Metodiev, and J. Thaler, “Energy flow polynomials: A complete linear
basis for jet substructure”, JHEP 04 (2018) 013, doi:10.1007/JHEP04(2018)013,
arXiv:1712.07124.
[47] H. Fokkema, R. de Heide, and T. van Erven, “Attribution-based explanations that provide
recourse cannot be robust”, arXiv:2205.15834.

8

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope? [Yes](b) Have you read the ethics review guidelines and ensured that your paper conforms tothem? [Yes](c) Did you discuss any potential negative societal impacts of your work? [No](d) Did you describe the limitations of your work? [Yes]2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A](b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [N/A] , we include
the data. As for the code and instructions to reproduce, we omit them for the sake ofanonymity, but they are ready to be shared upon acceptance.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they
were chosen)? [Yes](c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)? [No] , it would have been too computationally expensive(d) Did you include the amount of compute and the type of resources used (e.g., type ofGPUs, internal cluster, or cloud provider)? [Yes]4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes](b) Did you mention the license of the assets? [Yes](c) Did you include any new assets either in the supplemental material or as a URL? [Yes](d) Did you discuss whether and how consent was obtained from people whose data you’reusing/curating? [Yes](e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content? [Yes]5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable? [N/A](b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable? [N/A](c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation? [N/A]9

AParticleNet training

The ParticleNet model is implemented in PyTorch Geometric. We choose k = 12, instead of k = 16
which was proposed by the ParticleNet paper, as the number of nearest neighbors to perform kNN.
We also choose EdgeConv blocks with two (instead of three) fully-connected deep neural network
linear layers, each followed by a batch normalization layer and a ReLU activation function. We
believe these changes will decrease the computation cost and should not change the conclusions.
We perform the training over 1.2 M events, with an additional 400 k events used for validation, and400 k events used for testing. We run a training for 40 epochs with early stopping on a single Nvidia
GTX 1080Ti graphics processing unit (GPU). The training converges after 27 epochs which takes
about 3 hours. We use the Adam optimizer with a learning rate of 10−4 . The training performance isshown in Figure 5.

Figure 5: Training and performance of the ParticleNet model. A decreasing, convergent loss plot
is presented (left) and the classification accuracy is presented in the form of a receiver operating
characteristic (ROC) curve (right). We can see that the model stops learning after 27 epochs.

10

B

Sample edge R graphs

We present sample edge R graphs for top quark (QCD) jets in Fig. 6 in the upper (lower) panel. We
observe that by the last EdgeConv block, the model is learning to connect nodes farther apart in (η, ϕ)space.

Figure 6: Sample edge R graphs for top quark (upper 9 graphs) and QCD jets (lower 9 graphs).

11