How machine learningchanges particle physics
MIHOKO M. NOJIRI (KEK)
ISCO2023 MARCH 1, 2023

+−Particle physics• Particle Physics has been answering big questions, and it keeps

challenging remaining big questions• Example• Higgs boson (explaining origin of mass)• Dark matter in the Universe• Origin of the matterand they are Science!
• Big Science connects people!

ex CERN LHC ATLAS experiment: 3000
researcher 1200 PhD 42 countries, 182 Institutions
LeafletHOW PARTICLE PHYSICS RESEARCHWORKS ( DATA AND THEORY)

SM lagrangian+A few very
precise dataPhenomena at Early
Universe.............

Prediction of yet
unseen particles" New Physics"

Other data
Field Theory
Quantum Mechanics+ general relatively

Predictions

Interaction of particles
at any scale

LHC Higgs discovery and its nature

heavyand interesting

Higgs order param. v"yukawa coupling " y"particle mass" ~ yv

HIGGS BOSON AND OUR UNIVERSE• Our vacuum is not stable.
• This is problem!

quantum mechanism says that the vacuum tunnel to the true
vacuum sometime.

• We do not know how

we settle in the current vacuum• It is worth to study Higgs boson in detail!

Circumference 27Km
9593 magnets
1billion collision per second

HL-LHC Challenge
Data Acquisition
Challenges
HighestData rate in the Universe
Image credit: Wired 2013

Can AI (Deep Learning ) can help this?

14

40M collisions/s! → 40 TB/s data rate → Can only keep 1% on disk!
1MB/collision.(01).(100 51)

How ML can uncover "new physics" at LHC
new particle production and its decay to known particles(such as top
quark , Higgs boson) or unexpected distribution (deviation from SM)
New gauge bosons, supersymmetric particles
high energy top quark and Higgs boson can be signal
Identify unexpected events (anomaly search)
Select "strange events" among huge event samples
dark matter, hidden sector particles (dark photons)

40M collisions/s! → 40 TB/s data rate → Can only keep 1% on disk!
1MB/collision HOW ML HELP EXPERIMENT ITSELF.(01).(100 51)

High rate + many data (factor 100 increase)••

We have to process event 10 times faster ( reconstruct , recognize importantevents , record it )•Whole system should not exceed computing budget, and naively it will exceed(because the roadmap takes into account improvements by "R&D")







"ML at HEP workshop" in Japan Asia/Tokyo English  Login

Foundation of "Machine Learning Physics" Grant-in-Aid for Transfor

23-24 February 2023
KEK (High Energy Accelerator Research Organization)
Asia/Tokyo timezone

Overview
Timetable
Registration
Participant List
Poster session
Venue and access to KEK
Covid-19 measures atKEK
Accomodations
Information about entries
to Japan
RestaurantsContact ml-at-hep2023-loc@ml.…

We are delighted to announce that the "ML at HEP workshop" will be held at KEK, Tsukuba, Japan,
from February 23rd to 24th, 2023. After a successful ﬁrst domestic workshop on "ML at HEP" in July
2022, this is our ﬁrst time to organize such an international workshop in Japan by inviting worldleading researchers on Machine-Learning topics in related ﬁelds. This workshop also aims to bring
together researchers in high energy physics, data science, and related communities.
The scope of the workshop is to overview the state-of-the-art ML applications to high energy physics
and beyond, and to provide lively discussions about the directions of ML & AI technical developmentsin the ﬁeld.
The topics of the workshop program include:
Advanced ML techniques useful for HEP,
Advanced architectures for ML,
ML applications in HEP and beyond,
Advanced applications at accelerators and systems, andFuture directions in ML and AI x HEP and beyond.

Overview

Most of the subject of my talks
can be found here
https://kds.kek.jp/event/44830/

The workshop format is hybrid (in-person at KEK and online via zoom). Presentations are given byinvited speakers, and no free abstract submission is available in this occasion. An on-site poster
session is also scheduled. Please indicate your intention to present a poster on your ML-related
work via the registration form. As of January 13th, domestic travel and accommodation support canbe provided to a limited number of presenters. If you would like to request it, please contact us via theLOC mailing list by January 31st.
Registration:
Registration has been closed as of February 14th. In case of modiﬁcations, please modify via theregistration form at https://kds.kek.jp/event/44830/registrations/ or contact LOC via email.
No registration fee is necessary.

Organization

How machine Leaning help Analysis
ptφ
Gavin Salam (CERN)

yJets and jet substructure (2)Jet clustering

candidate of four top quark at CMSComparing Models for Jet Tagg
Example boosted top identification

Task: Top-quark jets vs single quarks/gluonseach, and testing with 200k signal and 200k background jets. For proper comparison, allGraph Neural Networks in Particle Physics
17
algorithms are optimized using the training
and validation samples and all results reported
Thereforea decision needsbe made about how
to constructa graphare obtained using the test sample. For eachalgorithm,
the toclassification
resultfor eachjet from the set of

New Physics
search withMLPhysics Performance
Improvements
ep Convolutional Networks

inputs. Di↵erent graph construction methods are illustrated in figure 6. Depending on
the task,
one might
even
want
toConvolutional
avoid creating
any pairwise
relationships
between
Our analysis
showsthat
DeepNetworks
significantly
improve theclassification ofnodes.newIf physics
the objects
have comparedno pairwise
dependence
a DeepSet[53] features,processes
to conditional
state-of-the-art
methods —basedon physics
enhancingtheonly
discovery
potentialof properties
the LHC. mightMore importantly,
the improved
performancearchitecturewithnode andglobal
be more suitable.
Edgeshighinenergy top quark
suggests
the deep convolutional network is capturing features and representations beyond
the graphservethat
3 roles:convolutional networks in particular — currently represent the state of the art ingnition tasks. We apply a deep convolutional architecture to Jet Images, andection. Below, we visualize a simple architecture used to great success.

issimilar
tolightquarkandgluons(i) The edges are communication channels among the nodes.
there is some difference(ii) Input edge features can indicate a relationship between objects, and canbut
encodephysics-motivated variables.

hitectures with large filters captured the physics response with a higher level ofrned filters from the convolutional layers exhibit a two prong and location based
ds light on phenomenological structures within jets.

physics motivated variables about that relationship (such asReLU

Dropout

ReLU

Dropout

LocalResponseNormalization

R between objects).

(iii) Latent edges store relational information computed during message-passing,allowing the network to encode such variables it sees relevant for the task.

FullyConnectedReLU Unit

In cases where the input sets are small (Nv ⇠ O(10) ) the typical and easiest
choice is to form a fully connected graph, allowing the network to learn which object
relationships are important. In larger sets, as the number of edges between all nodes
Crash Course A spectral function of jet substructure Spectral Analysis of Jet Substructure: Higgs Spectral Analysis of Jet Substructure:
SgluonasSpectralAnalysis
A Quick Sketch on
Top Jetsof Jet Substructure:
increases
Ne /(Nvof)2Jet, Substructure:
the computationalloadof Spectral
usingAnalysis
a neural
networkUnderstanding
to createNeural Networks Conclusion
an edge representation or compute attention weights becomes prohibitive. One possible
FlattenConvolution
Max-PoolConvolution
Max-PoolJet
ETmiss
ple
CNN:
Image
Recognition
Techniques
withjetJetimage
Image in theeric withoverview
slideJet
as Left:Image
asrapidity
sets vs azimuthal angle plane
as graphs
transformerFigure1:
typicalsingle
for the top

Boosted Boson Type Tagging

more non-linearity:
GoingL.Deep-32L. Oliveira, M. Kagan,Mackey, B. Nachman, A. Schwartzman,(1511.05190)

geParticles

Observablesignal after
pre-processing.
Boosted Boson
TypeTagging Center and right: signal and background images averaged over
Per-Particle Representation

Convolved
BenjaminNachman
andAriel Schartzman
FeatureLayersConvolutions
JetETmiss

10,000 individual images.

Receiver Operating Characteristic

March 26,
Benjamin Nachman
and2014
Ariel Schartzman

March 26, 2014 Max-Pooling

4...

...

...

Understanding Improvements
F

SLAC, Stanford University. Nachman (SLAC)

Standard physically motivateddiscriminants — mass (top)and n-subjettiness (bottom)

Latent Space

SLAC, Stanford University

W’→ WZ event

Event Representation

Since the selection of physics-driven variables is driven by physical understanding, we want to be
sure that the representations we learn are more than simple recombinations of basic physicalvariables. We introduce a new method to test this — we derive sample weights to apply such that

Energy/Particle Flow Network
Boosted Boson Type Tagging

March 26, 2014Repeat
Repeat1 / 21

uilding unit: 2D convolutional layer
Apply deep learningtechniques on jet images! [3]
XCNN(2014)
B. Nachman (SLAC)
BoostedBoson Type Tagging
March26, 2014
1 / 21(n)
(n)
(n 1)
(n)
hk,xy = '(wk`, x y h`,(x+ x )(y + y ) + bk )convolutional, nets are a standard imagexy
processing technique; also consider maxout
where this appears is learning from point clouds, sets of data points in space. For instance, thehelearned
convolutional
filters(left)
andthedifference
inbetween
theaverage
outputof spatial sensors such as lidar, relevant for self-driving car technologies, is often in thetion
Zformofa point cloud. As point clouds share the variable-length and permutation-symmetricround image after applying the learned
filters (right). This novel
00 convolutional0properties with collider events, it is worthwhile to understand and expand upon point cloud⇤ g (x)= dxwhatf (x the)g (xx )learns.
ation technique fhelps
understandnetworktechniques for particle physics applications.

Visualizing Learning

number of free parameters by weight and bias sharing.

sparse data

permutation invariancemeaning that physical variables have no discrimination power. Then, we apply our learned building key(stable)
discriminant, and check for improvement in our 1902.08570
figure of merit — the ROC curve.
(a)Figure 1: A visualization of the decomposition of an observable via Eq. (1.1). Each particle
in the event is mapped by
to an internal (latent) particle representation, shown here as
three abstract illustrations for a latent space of dimension three. The latent representation isthen summed over all particles to arrive at a latent event representation, which is mapped by
F to the value of the observable. For the IRC-safe case of Eq. (1.2), takes in the angular
information of the particle and the sum is weighted by the particle energies or transverse
momenta.

Easy to implement
QCD "values"

The Deep Sets framework for point clouds, recently developed in Ref. [63], demonstrates
how permutation-invariant functions of variable-length inputs can be parametrized in a fully
general way. In Ref. [63], the method was applied to a wide variety of problems including redshift estimation of galaxy clusters, finding terms associated with a set of words, and detecting(b)and query
Particle
Netout the individual effects ofNotice that removing
the physics-related variables leads to a likelihood 2202.03772
Lorentz net

performance equivalent to a random guess, but
the Deep
Convolutional
Network retains some( Graphrespectspecial relativity)
discriminative power. This indicates that the deep
network learns beyond theory-driven variables —2018 to 2021

More than factor 10 reduction of background→ current cross section limit improve by factor 10

Two questions

rett
Be

⭐if it improve drastically, "why theory cannot explain it?
⭐Theorists start to get complaint from experimentalists17
that performance isn't that impressive.

!

False Positive Rate

Improved Performance

QCD modeling for the future●With a better QCD modeling, we could:●

Train ML algorithms

Top Quark True Positive Rate
→ better data/MCagreement
ure 5: ROC curves for all algorithms evaluated on the same test sample, shown as the"anwondering→experimentalist
minimize
C ensemble median of multiple trainings. More precisenumbers
as well as signal
uncertainty
ds given by the ensemble analysis are given in Tab. 1.
systematics
whyefficiency
they cannot
touch the2201.08187

top of the shelf "Decorrelate taggers

Instead of extracting these performance measures from single models we can use ensembles.
● define 84 ensemble taggers, each time
this purpose we train nine models for each tagger andbining six of them. They allow us to evaluate the spread of the ensemble taggers and define

FROM TALK BY
P. MAKSIMOVIC AT ML4JETMACHINE DOESN'T CARE PHYSICS VALUES
high energy

low energy•IRC (infrared collinear) safe quantities :theoretically predictableparton shower•SOFTPARTICLES

IRC "sensitive" quantity: (soft physics) (exlow energy particles, number of particles)
Difficult to predict theoretically because ofthe divergence of the theory. We needmodelings but they often do not agree withdata.

MESSAGE

Modern Deep learning models take into
account ALL correlation.
HEP analysis need simulation and weneed updated our simulation
to be consistent at all energy.

•Soft phyiscs might be useful because low
energy particle distribution has
information of branching history strong
interaction. just we need careful calibration

About asking "something" to computer
the Ultimate Question of Life,
the Universe and Everything"

750M years" I think the problem, to be quite honest with you, is that you've
never actually know what the question is""So once you do know what the question actually is, you willknow what the answer means"from "The Hitchhiker's Guide to the Galaxy"
by Douglas Adams

Physics ≠Getting good results by using ML and satisfy

SOFT PARTICLE GEOMETRY

Lim, Nojiri
Phys.Rev.D 105 (2022) 1, 014004
Furuichi, Lim, Nojiri (in preparation )Area (A),boundary length(L)Euler character (E) of S (R)jet imagecircle (radias R)
at every hit

soft geometry model
IRC safe

Soft geometry

MLP

MLP
featureextraction

CONCATENATE

MLP

resulting image
S(R) R=R1, R2, R3,,,

1. compress spase data into real
numbers
2. resilient to fluctuation of theinputs
3. permutation invariant

soft geometry model
inputs has clear interpretation
on distance scale→ easy to infer the reasoning
Maybe we can create
better simulation by DL

Particle Transformer(current world best set up )

40M collisions/s! → 40 TB/s data rate → Can only keep 1% on disk!
1MB/collision HOW ML HELP EXPERIMENT ITSELF.(01).(100 51)

High rate + many data (factor 100 increase)••

Experiment should process events 10 times faster ( reconstruct , recognize
important events , record it )•Whole analysis chain should not exceed computing budget, and naively it
exceeds (because the roadmap takes into account improvement by "R&D")

Machine Learning Across Data Analysis

4Experiment"
DataAnalysis
Result!
ATLAS Software and Computing HL-LHC Roadmap, version 2.1!

Simulation

held on disk; and the number of replicas and versions of datasets will be significantly
reduced.
The current model does not attempt to take into account possible further divisions (i.e.
beyond disk and tape) in the storage quality of service provided by the WLCG sites, and it
assumes only CPU resources without accelerators will be available. The uncertainties when
including accelerators in the resource estimate (e.g. speed improvement factors in eachprocessing step, relative costs of CPU vs accelerator, availability of accelerators on sites)
are too large to make such an exercise useful. As R&D projects conclude and these
uncertainties are reduced, and once the WLCG has undertaken a discussion of the pledge
mechanism for these resources, the model will be extended to include the impact ofaccelerators.

Data Acquisition

Simulation

Low-Latency Real-Time
Decision Making

Fast Simulation andSampling

Reconstruction

Particle/Event Pattern Rec.
Anomaly detection

Inference

Parameter Estimation
Unfolding

ATLAS Software and Computing HL-LHC Roadmap, version 2.1

Agressive R& D assume
improvement by ML

Figure 1: projected evolution of compute usage from 2020 until 2036, under the conservative(blue) and aggressive (red) R&D scenarios. The grey hatched shading between the red andblue lines illustrates the range of resources consumption if the aggressive scenario is only
partially achieved. The black lines indicate the impact of sustained year-on-year budget
increases, and improvements in new hardware, that together amount to a capacity increase

Kagan "ML at HEP"

Calorimater

Up to 10min to simulate 1 event
event
generation
by MLGeant4: simulate atgenerating one
microscopic
levelevent takes 3minstochastic interactionof particles with matter

Bang!

ML modelinput particle {x}E-Deposit toDetector:
sumofeachcomponent
energy depositions
within given cell
granularity P( {E} | {x})full simulation
25

Variational AutoEncoder(VAE): bottle
neck structure x→ z → x'x￼￼

real datapϕ(z | x)z̃Latentwith simple distributionpθ(x′| z)x′reconstructedobject

event generation
atest & greatest: latent diffusion
modelsedients:

CalorimaterBang!

ML models

ed in latent space z (GAN/VAE)

input particle {x}sion in latent spacefull simulation

E-Deposit toeach componentP( {E} | {x})ML toolbox
sformer-based attention
mechanism to GAN:
bine input & conditioning τ0 in latent
space between
competition

Normalizing Flows:
map simple distribution to[blog, 2112.10752, blog, blog]
generator and discriminator
nontrivial function
es on more data & more compute
20 by step
Diffusion : Add noise stepOnce we complete this map, we have fast and more precise model foreverybody (even theorists) . DL model is differentiable so that you cando parameter search using MLThe PF reconstruction algorithm
arXiv : 2212.01328

PF lepton, hadron, photon = FPF (track hits + calo cells)
Sanmay Ganguly (ICEPP)

KEK-2023

talk at ML at HEP

5

BREAKING THE WALL
events in pipeline →select interesting one→save (physical constraints）

Graph Neural Networks and HEP
tracking

Good fit for sparsity, irregular geometry, anTracking: Finding Trajectories from space-points
number of tracks at collision

EXA.TrkX (Eur. Phys J. C(2021) 81:87
detemine near by node → GNN to handle edge classification
training time ~36 hour(by A100)、
prediction 2.2 s/event (GPU), 220 s/event (CPU)Graph NNML on hardware
Real-Time Low Latency ML

15

Goal: Move improvements from ML for reconstruction to online setting
FPGA= Field Programmable Gate Array
HLS =its compiler

hls4ml:• HEP Community developed tool do deploy NN as electronic circuit on hardware• FPGA-as-accelerator library → Whole model on chip
• Model Compression essential
Example: Resource Usage inJet Classification with RNNs

https://fastmachinelearning.org/hls4ml/
1804.06913

Kagan "ML at HEP"

Khoda, …, MK, et. al, 2207.00559

summary
HEP people are working to merge DL to most precise, large
data science under extreme environment.
The outcome will certainly shared to outside community, sostay tuned.